The Dot-com and Subprime Mortgage Bubbles: An Example of State-Space Forecasting Techniques

In a prior post, I described state space models and why I think they are useful for time series forecasting. I'm using state-space models to generate forecasts for 2010 and plan to look at how well the models did in 2011. However, I can pretend that I was trying this same exercise in 1990 by estimating the model only on historical data up to 1990, using the 1990 model to forecast forward to the present and see how well the model would forecast two notorious crises that standard economic models seem unable to have predicted, the dot-com bubble from 1998-2000 and the subprime mortgage crisis from 2007-2010. For this exercise, I'll just look at the stock market, particularly the S&P 500 index using data from Yahoo! Finance.

First, as a point of comparison, I estimated a state-space model for the entire period. The model uses the state of the US economy as an input variable and the volumes and price data for the S&P 500 as the output variables. The fit of the model for S&P 500 volumes and prices in the figure above (dotted red line) is excellent.Using a model estimated only up to 1990, the price fit is still good but the volume forecast (upper panel above) starts to deteriorate after 2005. Notice for both series that the model tends to miss the turning points--a common feature of all forecasting models (the models don't know that the bubble has burst until after it has happened).

Although models appear unable to capture turning points, they are able to identify developing bubbles using feather forecasting. A feather forecast uses the model to predict a number of periods into the future (thirty-six in the figure above for monthly S&P 500 data), starting from each period in the sample. Up to 1995, the model forecasts basic linear growth for S&P 500 prices. In 1995, however, the feather forecast identifies the developing dot-com bubble as having started in 1995 rather than the conventional dating of 1998. From 1995 to well into 2001, the feather forecast is predicting a collapse in prices. Indeed, from 1995 to 2007 (the last point where a 36 month feather forecast can be made) the feather forecast is warning of a collapse in prices. A conservative investing strategy would have been not to ignore the developing bubble but realize that one would have to be vigilant and liquid with any new investments made after 1995.

This then brings us back to a future forecast made with the model estimated up to 1990. The model is essentially predicting a sell off for 2010 (increasing volumes and decreasing prices). It will be interesting to see what actually happens over the next year. The model cannot predict monthly rallies and sell-offs or the turning points for recovery, but studying the feather forecast suggests that the S&P 500 cannot realistically reach 1000 for a few more years into the future. In any event, the usual disclaimers apply to this prediction.