I agree that the small sample size makes this index of questionable value. But the larger question becomes: How do you build an index to track commercial real estate performance? Moody's/REAL have gone for a repeat-sale methodology (pros: price movement isn't affected by changes in product mix; cons: very small sample sizes). NCREIF's NPI is appraisal based instead (pros: much larger sample size; cons: appraisals are slow to reflect the market, on the upside and the downside).
It's one thing to just throw up your hands and say "There isn't enough data." But institutional investors want to benchmark performance (maybe that's a problem, too, but it's pretty intractable at this point). And the truth is I don't know what the right answer is.
I don’t know the answer either (although I’ll make an attempt below – I too hate to just throw up my hands when the question is hard). However, I’m pretty sure the answer is not repeat sales, both because the sample size is small and because individual CRE property values are subject to large fluctuations which are not attributable to general market movements. For example, a neighborhood shopping center might be worth X 3 years ago, and is now worth 0.5X because the anchor grocery store closed. With a large enough sample these individual property fluctuations might cancel out, but with a small sample size I think the problem is hopeless. And, I agree with Curren that an appraisal based index has severe limitations; Lansner on Real Estate’s article “Were Appraisers Late to the Price Collapse?” documents the problem.
A partial answer is to look at changes in property rents and vacancy rates. There are a number of sources for this data (for example, MPF, REIS, and PPR). If you don’t want to spend the money or need an idea of what’s going on in a market they don’t cover, I can tell you with confidence that the trends these providers report closely parallel the 12 month change in employment levels in a market, and that data is readily available for free for almost any market at this BLS site. However, looking at income performance alone won’t tell you much about value trends; you need to know what’s going on with cap rates to answer that question.
Maybe the best CRE index is the performance of a REIT ETF like SPDR Dow Jones REIT?
My guess is the peak for this ETF roughly coincides with the CRE peak, and the current price is down roughly 65% from the peak. That seems extreme, but maybe that’s where we’re headed. I know I have a number of readers who know much more about REIT values than I do; if this is a dumb idea I’d genuinely welcome your comments.