The Dismal Science Relies on Dismal Surveys

The Commerce Department released figures today showing that consumer spending was up 0.7% in November, slightly up from 0.6% in September. Though the difference is marginal, at best, many economists and market pundits will want to know what this portend for the future. Unfortunately, many will turn to surveys of consumers sentiment in an attempt to divine the future. The more notable of these are the Consumer Confidence Index from the Conference Board, the Consumer Comfort Index from ABC News/Money Magazine (go the the polling report site on the sidebar), and the Consumer Sentiment Survey from the University of Michigan.

First things first - how are these surveys constructed?

Consumer Confidence Index: Mail survey of 5,000 individuals asking questions focusing on business conditions and available jobs. Proportion of positive responses are divided by the sum of all positive and negative responses and then multiplied by 100. Results are indexed to a 1985 base and then averaged for the final number. Economists agree that the index is more sensitive to employment conditions than the others. The latest figures show 90.5 for November.

Consumer Comfort Index: Phone survey of 1,000 adults measuring current economic conditions with respect to the national economy, personal finances, and purchasing climate. Proportion of negative responses are subtracted from positive responses and then averaged to construct a scale from all positive (+100) to all negative (-100). In November the figure averaged around -9.

Consumer Sentiment Survey: Phone survey of 500 adults (60% new respondents, 40% second time respondents). Negative responses are subtracted from positive ones and then added to 100. Questions are indexed to a 1966 base and then averaged for the final figure. The latest results show an index of 92.8 for November.

Now, all of these carefully constructed and tested indices are meant to be predictive, that is, if consumer sentiment goes down the received knowledge of economics dictates that an economy built largely on consumer spending should go down commensurately. This is at least how the results are propigated through the business and economics media. However, last year Ivana Rupic from the RBC Financial Group in Canada correlated these indices with actual measures of monthly consumer spending. The results may be judiciously characterized as "less than perfect." Look at the following correlations between what was "predicted" in the monthly poll results with levels of actual consumer spending from 1985 to 2003 (normalized to a range from -100 to +100): Consumer Confidence Index (3.0%); Consumer Comfort Index (-3.2%); Consumer Sentiment Survey (8.3%). Without further analysis, they appear to be within the bounds of normal variation, so if you want to get dibs on the underlying pulse of the consumer's mind, try elsewhere.