A Random Walk Among the Undead

The efficient market model was pronounced dead over a year ago at the World Economic Forum in Davos. But, Jeff Anderson-Lee, a commentator on Paul Krugman's blog, commented that "the 'efficient-market hypothesis' ... seems harder to kill than the undead."

What's going on here; why won't the theory die? In this case, I have to agree with Robert Shiller, the theory won't die because it's partly true! If you've been following my attempt to forecast the S&P 500, you know that I've produce a number of plausible forecasts for the market future from optimistic to pessimistic. One explanation is that I don't know what I'm doing, another is that the future is unknowable and a third is the efficient market hypothesis (EMH).

What is the efficient market hypothesis? That's a little difficult to present clearly because the concept has become overloaded with multiple meanings. The simplest form of the hypothesis is the random walk hypothesis, that is, stock prices move according to a random walk. The random walk hypothesis requires that the largest characteristic root of a state-space model including S&P 500 prices as the only output variable should be close to unity. For a monthly model I estimate running from January 1950 to January 2010, the largest characteristic root is 0.9986935 or, with rounding, unity.

A forecast with this model (displayed above) shows that our best prediction for the future of S&P 500 prices is the current price, as called for by the random walk hypothesis. Actually, the forecast above was not made with a pure random walk model but rather a random walk with drift model. The pure random walk equation is X(t) = X(t-1) + e(t-1) where e(t-1) is random, uncorrelated error. The random walk with drift is X(t) = a + X(t-1) + e(t-1) where a is the drift term. Supposedly, the drift term invalidates the random walk hypothesis, but even with drift the market is not very predictable.

Interestingly, plotting just X(t) = a + X(t-1) provides a very basic bubble predictor. From 1950 to 1995, a buy-and-hold investment strategy (advocated by EMH proponents) made some sense. No matter when you purchased the stock that tracked the S&P 500, you made money on any sale. After 1995, things became a lot more risky. Stocks purchased in 2000 and sold in 2010 generated huge loses. Buy-and-hold after 1995, in retrospect, wouldn't have made much sense as an investment strategy.

If EMH was restricted to random walk or random walk with drift models, it is a good basic starting point for understanding investment strategies and market bubbles. However, "efficient" should not be equated with "optimal." A casino is essentially a random walk with drift for the house. Where the EMH embraces visions of perfection, it obviously (after the dot-com bubble and the subprime mortgage bubble) goes too far. In future posts I'll struggle with what optimization would mean in the context of the stock market and struggle even harder with the question of whether, whatever the stock market is, it benefits the US economy.