Double Bubble, Toil and Trouble

The NY Times today posed the question "Fed Missed This Bubble. Will It See a New One?" Both Fed chairmen Alan Greenspan and Ben Bernanke famously missed the development of the dot-com and the Great Recession bubbles (even conservative News.max accepts this analysis). So, how hard is it really to identify bubbles?

In an earlier post I showed how feather forecasting from a state-space model estimated up to 1990, showed that the market was over-priced during both the dot-com and the subprime mortgage bubbles. Implicitly, the feather forecast points to a sustainable level for the market but doesn't really display the level explicitly. To do that we need to remove the cyclical components and the month-to-month shocks from the model and run a counterfactual simulation for the stock market. The simulation is displayed above. Rather than reaching almost 1600 in the peak of the dot-com bubble, the simulation suggests that the market should have been at about 600. And, 1000 would have been a better level when subprime mortgage crisis broke.

Why is it so difficult for the US Federal Reserve to identify bubbles? Part of the problem is that one of the major cyclical state variable is the US economy is driven by Fed policy. Another cyclical state variable involves more fundamental economic factors such as housing, corporate profits, oil prices and unemployment. Whether the Fed is counter-cyclical or pro-cyclical will have to be a topic for another posting but clearly the Fed is in the middle of market cycles and has trouble looking outside the box.

And, compared to my pessimistic forecast for the S&P 500, the forecast eliminating the cyclical state variables for the US economy is optimistic. In the figure above, the S&P 500 is back to pre-crisis levels by 2015. The usual disclaimers apply even more strongly to this counterfactual forecast.

At best, financial reform might reduce but not totally eliminate bubbles in the US economy. Since late 20th Century bubbles seem to last about a half decade, the pattern suggests a bubble investing strategy that would differ from the standard diversified portfolio approach--a topic for a future posting.

The Dot-com and Subprime Mortgage Bubbles: An Example of State-Space Forecasting Techniques

In a prior post, I described state space models and why I think they are useful for time series forecasting. I'm using state-space models to generate forecasts for 2010 and plan to look at how well the models did in 2011. However, I can pretend that I was trying this same exercise in 1990 by estimating the model only on historical data up to 1990, using the 1990 model to forecast forward to the present and see how well the model would forecast two notorious crises that standard economic models seem unable to have predicted, the dot-com bubble from 1998-2000 and the subprime mortgage crisis from 2007-2010. For this exercise, I'll just look at the stock market, particularly the S&P 500 index using data from Yahoo! Finance.

First, as a point of comparison, I estimated a state-space model for the entire period. The model uses the state of the US economy as an input variable and the volumes and price data for the S&P 500 as the output variables. The fit of the model for S&P 500 volumes and prices in the figure above (dotted red line) is excellent.Using a model estimated only up to 1990, the price fit is still good but the volume forecast (upper panel above) starts to deteriorate after 2005. Notice for both series that the model tends to miss the turning points--a common feature of all forecasting models (the models don't know that the bubble has burst until after it has happened).

Although models appear unable to capture turning points, they are able to identify developing bubbles using feather forecasting. A feather forecast uses the model to predict a number of periods into the future (thirty-six in the figure above for monthly S&P 500 data), starting from each period in the sample. Up to 1995, the model forecasts basic linear growth for S&P 500 prices. In 1995, however, the feather forecast identifies the developing dot-com bubble as having started in 1995 rather than the conventional dating of 1998. From 1995 to well into 2001, the feather forecast is predicting a collapse in prices. Indeed, from 1995 to 2007 (the last point where a 36 month feather forecast can be made) the feather forecast is warning of a collapse in prices. A conservative investing strategy would have been not to ignore the developing bubble but realize that one would have to be vigilant and liquid with any new investments made after 1995.

This then brings us back to a future forecast made with the model estimated up to 1990. The model is essentially predicting a sell off for 2010 (increasing volumes and decreasing prices). It will be interesting to see what actually happens over the next year. The model cannot predict monthly rallies and sell-offs or the turning points for recovery, but studying the feather forecast suggests that the S&P 500 cannot realistically reach 1000 for a few more years into the future. In any event, the usual disclaimers apply to this prediction.

Forecasting Disclaimer

At the beginning of 2010, I'm publishing a collection of economic and environmental forecasts based on state-space time series models. Given that economists were unable to forecast the financial crisis of 2007-2010, I should probably explain why I think my approach is better.

First, I'm not sure my approach is better. That's part of the purpose for doing the forecasts in early 2010. At the start of 2011 (or any point in between), I can look back at the forecasts, see what went wrong (if anything) and try to figure out the reasons for any failures.

The state-space approaches are attractive because they address a number of known problems with existing forecasting models. State-space models forecast based on the "unobserved" state of the system being studied. The state variables are the minimum collection of independent, orthogonal variables that connect the model's input variables to the output variables. Typically, economic forecasting models have a rigid set of structural equations that are used to predict each output variable. Real human systems, however, are too flexible and too emergent to be modeled with a rigid set of equations.

A related attractive feature of state-space models is that there are two sources of error variation, one for the state-vectors and another for the output variables. Typically, economic forecasting models have one source of error for each output equation. And, typically the error distributions are modeled using a bell-shaped normal distribution. Nouriel Roubini, for one, has strongly criticized economic forecasting models both for their unrealistic structure, estimation techniques and assumptions about error terms. Rather than assuming that the functional form of the error term is known, state-space models can easily use a non-parametric bootstrapping technique to estimate confidence intervals around forecasts and the Kalman filter to compute the system state while forecasting (after an approximate system state is used to estimate the model).

Another interesting aspect of state-space models is the use of computer simulation to understand the forecasts. For example, one can estimate a state-space model for a based period (say 1950-1990 for the US economy) and then use simulation techniques to calculate the unobserved system state (using the Kalman filter) and the time path of the output variables to any point in the future. I'll present an example of the analysis in a later post.

For all these reasons, I find state-space models attractive for forecasting. For some of the answers to how well the models actually perform, we'll have to wait until 2011. However, we can also analyze two notorious historical episodes (the dot-com bubble and the financial crisis of 2007-2010) using a model estimated on US data up to 1990. Could these two crises have been predicted with state-space models? That will be the subject of my next post.

A Pessimistic S&P 500 Forecast for 2010-2012

It's probably appropriate that my first forecast for 2010 would involve the stock market. The US has gone through two stock market bubbles (the dot-com bubble from 1998--2000 and the subprime mortgage crisis from 2007--2010, hopefully) in addition to a direct terrorist attack on New York City (September 11, 1002) that paralyzed the financial system. Before presenting a forecast for 2010, let's look at how well my time series models dealt with this period of stock market instability.

The figure above displays a feather forecast of volume (the upper panel) and price for the S&P 500 (data from Yahoo! Finance). Although the models track the stock market data very well (displayed here), the feather forecasts (in this case, the feather forecast starts from every month in the sample and predicts forward three years) show that the S&P 500 was over-valued during the dot-com bubble, undervalued during 9-11 and over-valued again during the subprime mortgage crisis. In other words, using the models would have suggested a conservative investing strategy and warned of developing stock market bubbles.

The S&P 500 model is driven by the state of the US economy. Using financial terminology, the market fundamentals are determined by the state of the economy. Using systems terminology, the stock market is a hierarchical subsystem within the US economy. The three forecasts (dotted red, blue and green lines) in the graphic above suggest that the future will depend on the state of the US economy. Volumes will be flat while prices will either rally or remain flat based on the future of the US economy. Not really that surprising a range of forecasts.

When I use an actual forecast for the state of the US economy through 2012 as input for the model, the forecast is pessimistic: declining prices and increasing volumes, in other words, a sell off. You can compare my forecast to the one provided by the Financial Forecast Center here. Their forecast for prices also points downward (out to Jul 2010).

Of course, it's also possible to create an optimistic forecast for the S&P 500 based on a more optimistic forecast for the US economy and the usual disclaimers apply. We're at a point where uncertainty rules.

Forecasts for the New Year

In prior posts, I have made forecasts for the US economy and the World system. For the US economy, I have developed forecasts for nuclear power (optimistic) and for the airline industry (pessimistic). For the World system, I have forecasted relatively poor performance for the European Cap-and-Trade (emission trading) system. The forecasts were based on outputs from my time series models of the US economy and the World system.

My New Year's resolution is to make a series of forecasts, at least for the next few years and possibly for the entire decade. My intention will be to review these forecasts in January of 2011 to see how well the models performed.

A Future for Nuclear Power?

The New York Times is reporting that a US DOE loan program combined with cap-and-trade legislation may give new life to the moribund US nuclear power industry. However, the future of nuclear power is not very bright given the problems, to include: "high relative costs; perceived adverse safety; environmental and health effects; potential security risks stemming from proliferation; and unresolved challenges in long-term management of nuclear wastes." A study in the Bulletin of the Atomic Scientists is equally pessimistic given problems with the existing fleet of nuclear power plants.

My own forecasts (using a three-factor index model of the US economy) are more optimistic. Net generating capacity (the top panel above) peaks after 2040 at about a 30% share (lower panel) of total electricity generation.

My forecast should be contrasted with the very pessimistic forecast from the US EIA (the solid line above) compared to my forecast (the dotted line) for net generation in billion kilowatt hours. The EIA forecast is based on an analysis of plans and goals of the nuclear power industry. Time will tell; no one knows the future. The EIA forecast seems more reasonable.

P.S. The first new uranium mill built in the US in 25 years is about to come on line...

Airline Re-regulation

The process of airline re-regulation is about to start with DOT's decision to fine airlines that keep passengers waiting in planes for more than three hours before takeoff. Airline deregulation started in 1978 and there are two views about how successful it has been. The first view, advanced by neo-liberal and neo-conservative economists is that airline regulation has had "... overwhelmingly positive results." Any small problems (poor service, bankruptcy, safety violations, monopoly practices, overloaded traffic control systems, NIMBY constraints on more airport construction, lack of profitability, luggage fees, TSA shake-downs, intransigent unions, terrorist attacks, etc.) would be solved by more free market fundamentalism.

The other view is that airline deregulation has been an unmitigated disaster--for all the same reasons. Environmentalist would also add that air travel has a very large carbon footprint (even higher than automobile travel). In fact, airlines will be the first industry (even before coal-burning power utilities) to face cap-and-trade requirements in the European Union.

Which side of the argument you favor depends to some extent on your view of the future. If the future holds unrestrained exponential growth in air travel with continually decreasing prices, your projections about the future are probably based on assumptions about how increasingly free markets generate unlimited economic growth. If the future doesn't look that positive, you probably favor some kind of airline re-regulation. You might also favor a broader perspective that looks at a range of transportation alternatives (you probably recall that the Penn Central railroad failed a few years before airlines were deregulated and marked the end of long-haul private-sector passenger service in the US).

So, how well does the airline free market work and what are the predictions for the future? The figure above (data from the US Bureau of Transportation Statistics) shows a forecast for the airline market with airfare prices on the bottom graph and airline operations on the upper graph. Prior to 2009, there was a huge expansion of airline operations peaking in 2008 with very modest price movements (actually, prices have been highly variable, increasing to 2001, decreasing to 2005, increasing again to 2008 and then collapsing during the subprime mortgage crisis--the time plot is just compressed due to the large forecasted price increase).

The forecast suggests that we haven't seen anything yet: prices are going to skyrocket and operations are going to stagnate. If this happens, travelers are going to divert to automobiles (my wife and I are already doing this for long trips in the US). We would use long-haul passenger train service (as we do in Europe) but that was dismantled in the late 1970's. If only the system had not been dismantled in the brave new world of economic deregulation. Do you think we'll all ever have personal jet packs?

P.S. My forecasting model finds that there is very little interaction between quantities and prices in the airline market. That shouldn't be surprising since operations are flow-constrained. Market fundamentalism won't remove the current network and environmental constraints so the neo-liberal dreams of unconstrained growth in air travel are just dreams.