After a brief surge in luxury spending in October, demand for pricey clothing and accessories seems to have hit the doldrums again.

Spending by upper-income shoppes plunged 20 percent in November, in keeping with declines among lower- and middle-income consumers, according to Gallup. That’s despite easy comparisons with the financial crisis a year ago.

While the rising stock market and stabilizing home values may have temporarily stirred up demand for luxury goods earlier this fall, renewed fears about the economy appear to be setting in again, the polling firm found.

The main culprit: a “dismal” job market, Gallup says.