Rebuilding the edifice of modern finance theory
Orthodox finance theory holds that a financial securities market is informationally efficient when prices fully reflect all available information. The most powerful empirical prediction of the efficient market hypothesis is that money managers, traders, and investors cannot obtain higher returns than the rest of the market without increasing the amount of risk they must bear. Risk, while difficult for investors to measure, is simply the possibility that investors will not receive the return that they expected. The most consequential and most hotly debated prediction of this hypothesis is that market prices are the best unbiased estimate of an asset's intrinsic value.
It is in this prediction, that prices are "right," where so much disagreement has arisen. Michael Jensen, one of the hypothesis' proponents, said in 1978, "I believe there is no other proposition in economics which has more solid empirical evidence supporting it than the Efficient Market Hypothesis" (Jensen 95). On the other side of the debate, Robert Shiller has said repeatedly that "the efficient market hypothesis is the most remarkable error in the history of economic theory" (Donnelly). While the market is very informationally efficient, some kinds of behavioral biases can cause the market to become allocatively inefficient and the limits to arbitrage can prevent market prices from reverting to their intrinsic values for prolonged periods of time.
Modern finance theory, of which the efficient market hypothesis is the foundation, relies too heavily on the assumption that investors are rational. From this assumption, predictions have been made which don't stand up to empirical scrutiny, and dangerous tools have been developed for the finance profession. Behavioral economists and finance theorists have attempted to attack the old assumptions and construct new models which make more accurate predictions.
This paper argues that asset bubbles and resultant crashes that periodically appear in the markets are neither the result of occasional random departures of asset prices from intrinsic values, nor are they full reflections of changes in fundamental information. Bubbles and crashes are frequent consequences of systematic inefficiencies in the pricing of assets, and the mispricing of risk. Were this merely a question of creating winners and losers, policymakers could ignore bubbles and leave it to money managers and financial institutions to take whatever risks seemed prudent in light of experience. But the collapse of credit and liquidity that accompanies a sudden crash in asset prices affects the real economy and demands both preventative and curative responses from policymakers. By recognizing that the risk of these cycles is inherent in the market and that they occur for irrational reasons, policymakers can respond by taking regulatory steps to control leverage, and risk taking more generally, in order to limit the interconnectedness that causes a crash to build on its own momentum. Policymakers can also use fiscal and monetary policy to promote economic recovery after the crash. These steps cannot and will not make markets efficient, but they may limit the effects of the wilder excesses of both the up and downside of asset pricing. Doing so will eliminate the demise of liquidity and risk tolerance that was seen in the markets beginning in 2007 as financial institutions lost faith in the ability of their counterparties to meet their obligations.
Unfortunately, investors, financial institutions, and policymakers are still unnecessarily overly reliant on the old assumptions of rational efficient markets. The risk that this reliance posed to the real economy and to investors was borne out in the recent financial crisis and still exists today.
What is the efficient market hypothesis?
In an efficient market, competing investors ought to use all information available to them to make their investment decisions. Rational competing investors are always trying to maximize their returns and minimize the risk they must bear. Because investors cannot see perfectly into the future, they must make decisions based on what they perceive as the probability of different events occurring. If new information changes expected future outcomes, investment decisions must also change.
If some new piece of unexpected positive information comes out, such as a report of especially high earnings for a company, competing investors will have an opportunity to buy that company's stock to receive a higher return on their investment without bearing any extra risk. This buying will bid up the price of that company's stock until that information has been fully incorporated into the price and there is no more motivation to buy. If the opposite occurs, competing investors will see that their expected return has decreased. They will then sell that stock as long as the price remains high in order to avoid excess losses. This selling will drive the price of that company's stock down.
If information is free and easily accessible, competing investors ought to flock to any anomalies and cause inefficiencies to disappear "instantaneously" (Fama, The Behavior 7). Investors will bid a stock's price up or drive it down as long as the inefficiency exists. The only way to take advantage of informational inefficiencies is to be faster than the competition, or come up with some original analysis. With the advent of information technology, computerized high frequency trading appears to make it impossible for the average investor to be faster than the competition (Duhigg). Reading The Wall Street Journal, therefore, is useless for making investment decisions in an efficient market. If the information contained therein were of any value, it would have already been exploited by competing investors.
Markets, however, aren't and don't need to be perfectly informationally efficient. Not all investors are perfectly rational and not all are able to accurately interpret the effects of new information on a stock's price. There only needs to be a "sufficient number" of well-capitalized and smart investors in the market to arbitrage away the inefficiencies fairly quickly (Fama, Efficient 388).
Seeing that some anomaly exists, that some positive information hasn't been fully reflected in a stock's price for instance, a smart investor could capture the potential excess return by buying that stock. As the stock's price incorporates the positive information, its price will rise and the investor will receive his or her return. That investor could simultaneously eliminate his or her risk by selling short, or borrowing and selling, some covarying security, index, or portfolio. The closer the two positions come to perfect negative covariance, or moving in opposite directions in response to new information, the more risk can be eliminated. If some other new information drives the price of that stock down, the investor's short position ought to protect him or her from losses.
Market efficiency is only a model for how securities markets operate. In finance, like any other science, all models are wrong. In order to be of any use, they must simplify more complex phenomenon. There are a myriad of reasons why a market cannot be perfectly efficient. Information can be costly and slow to spread, financial transactions are not free, information could be incorrect or difficult to interpret, and not all investors are rational. The acid-test for whether or not one can accept the model is if it makes testable predictions. So the question then becomes, not "are markets efficient," but "how efficient are markets?" Eugene Fama defined three different forms of market efficiency in 1970 to help answer this question (Efficient 383).
In a strong form efficient market, prices fully reflect all information. In such a perfect market, no information, not even inside information unavailable to the public, could be used to make a profit. Strong form efficiency is useful as an idea or an ideal, but it is not an accurate description of reality. If it were true, then no trading would ever occur. Both securities holders' asked prices and buyers' bid prices would immediately change to reflect all new information. The holder would not be willing to sell for anything but above the value of the asset and the buyer would not be willing to buy for anything but below the value of the asset. If both agreed on the true value of the asset, neither would be willing to trade.
The idea of a perfectly efficient financial market is inherently self-contradictory because it precludes the existence of the financial market itself, but it has been useful for building economic models in the past. Newer economic models attempt to incorporate inefficient financial markets, but one of the reasons these economic models fail to predict economic and financial crises is because this is so difficult (Kocherlakota 3).
This idea of perfect market efficiency might mistakenly lead one to conclude that insider trading is somehow a good thing. The faster information is incorporated into the price, after all, the faster the price of a security reflects the underlying company's intrinsic value. Investors, however, might be afraid to enter the market in what they perceive as an "unfair game." This would reduce the total amount of capital available to companies in financial markets. This would also discourage active traders from attempting to correct true mispricings for fear that opposing traders will possess some information that they don't have access to. In a market limited to trades based on public information, overpricing due to private information not having been incorporated into prices would be as common as underpricing on average. Good regulation, which creates a level playing field and protects investors from fraud, insider trading, and market manipulation, is necessary for an efficient market.
In a weak form efficient market, prices fully reflect all information regarding past price movements. If prices only move in response to new information, then price movements are unpredictable by definition. Chartists or technical analysts who look at past price movements and search for patterns in order to predict future price movements cannot persistently obtain excess returns. Any excess returns are attributable to chance alone. Fama surveyed much of the empirical research regarding weak form efficient markets in 1970 and found no evidence that investors could persistently gain excess returns through technical analysis (Efficient 389-404). Mainstream finance theory has upheld this conclusion very well ever since (Malkiel 128).
Fundamental analysts will try to follow the news, analyze a company's competitive advantage, forecast future profitability, or do anything they can to try to come up with some present value which will fully reflect all of the information available to them. If the market price is below this value, they will buy the stock in hopes that it will pay dividends or go up in price to be sold later for a capital gain.
They will repeat this process, surveying all available information and rebalancing based on new information, in order to build up a whole portfolio of such undervalued securities. Hopefully, their total return will meet or exceed whatever time value their money had, along with whatever compensation they were hoping to receive for bearing the risk involved with investing. Ideally, the investor will have to bear less risk or receive a higher return than if they had not put forth all that effort and simply bought a passively managed index fund, such as the S&P500.
In a semi-strong form efficient market, prices fully reflect all available public information. This is what is meant when people refer to the efficient market without qualification. In such a market, fundamental analysts cannot obtain excess returns. Money managers, who practice fundamental analysis, cannot increase their returns without increasing the amount of risk they must bear. This is a profound statement. Experts exist in nearly every other profession, from dentists to lawyers to mechanics, but mutual fund managers seem to have incredible difficulty justifying the fees they charge to investors.
Considerable evidence exists to show that the average performance of mutual fund managers, among other professional money managers, is not better than their benchmark indices for periods as long as 20 years. Some managers were able to obtain excess returns for periods as long as 10 years, but were unable to do so persistently for the following 10 years (Malkiel 164-170).
There are two compelling explanations for these results. The first is that some of the managers were good enough to spot market anomalies and exploit them, but consistent with market efficiency, competing managers and investors were able to replicate their strategies and the anomalies disappeared. This would explain managers' inability to repeat past good performance. The second explanation is that the results were due to luck alone. This would explain the inability of managers as a whole, as well as the inability of previously successful managers, to obtain excess returns. It requires very long periods of excess returns to reduce the impact of luck and conclude with any certainty that a money manager, short of having access to inside information, displays talent.
It is not totally implausible for an investor to obtain excess returns through fundamental analysis. If that were absolutely true, there would be little reason for the existence of an entire financial industry which has been built upon the idea of trying to obtain excess returns. If one's expertise were narrowly focused on a small enough niche area, one might be able to assess new or existing available information more accurately than the rest of the market and obtain some excess return. Fundamental analysis is not a realistic strategy for the average investor in an efficient market. The best strategy, it would seem, is to buy and hold a passively managed diversified index fund.
Empirical evidence is consistent with an informationally efficient market where rational competing investors cause prices to reflect all available public information. Because prices quickly reflect all available public information, even professional investors have great difficulty obtaining excess returns. In an efficient market, a passive investment strategy is superior to fundamental and technical analysis.
The second prediction of the efficient market hypothesis, a seemingly logical conclusion from the first, is that "in an efficient market… the action of the many competing participants should cause the actual price of a security to wander randomly about its intrinsic value," as Fama pointed out in 1965 (Random 76). This statement is both more profound and more dubious than the first. It would lay the foundation for nearly every aspect of modern finance theory, and the empirical tests of that theory.
According to the theory, the behavior of stock prices lies somewhere below perfect strong form efficiency, where prices are always equal to their intrinsic values, and somewhere around a near perfect semi-strong form efficiency, where market prices are the best unbiased estimate of an assets' intrinsic value. Prices are "unbiased" estimators because any systematic anomalies would provide arbitrageurs an opportunity to recognize undervalued or overvalued stocks, and gain a risk-free profit. If the market is efficient, these anomalies will disappear instantaneously. Prices are the "best" estimators because they are very accurate. Prices will never stray very far or for very long from the asset's intrinsic value. As prices swung back around toward their intrinsic values, arbitrageurs would again be provided with an opportunity to gain a risk-free profit.
Allocative efficiency is very important for both investors and businesses. Fama points out that "the primary role of the capital market is allocation of ownership of the economy's capital stock… The ideal market is a market in which prices provide accurate signals for resource allocation: that is, a market in which firms can make production-investment decisions, and investors can choose among securities that represent ownership of a firms' activities under the assumption that security prices at any time 'fully reflect' all information" (Efficient 383).
For the average investor, portfolio allocation became easy. Buying and holding a passively managed index fund was best because any effort put forth to identify mispriced assets better or faster than the professionals was futile. Robert Shiller summarized the conventional wisdom, that "if millions of researchers and investors are studying stock prices and confirming their apparent value, why waste one's time in trying to figure out reasonable prices? One might as well take the free ride at the expense of these other diligent investors who have investigated stock prices" (xxiii).
For professionals, the toolkit of modern finance became standard methodology. Modern Portfolio Theory told them that they could successfully reduce their risk through diversification. Many small bets are less risky than one large bet because the bad bets are balanced out by the good. The Capital Asset Pricing Model told them that the market would only reward them for any risk that could not be successfully diversified away. Investors could diversify away any risk that was not represented by a stock's beta, the level of covariance with the rest of the stock market. Value at Risk told them precisely how much downside risk they faced under normal conditions. The Black-Scholes formula told investors how to price options.
With these tools, professionals could fine tune portfolios to target a given level of risk or return by altering the allocation of stocks, bonds, and cash. It allowed them to measure their risk adjusted performance against the market and became part of regulations restricting money managers and debtors risk-taking behavior. It even gave professionals an alibi in the event of poor performance. "It was just bad luck this time," they could claim, pointing to their use of the standard methodology of risk management. All of this was blessed by specious quantitative precision.
For businesses, it was management's legal obligation to protect and increase shareholder value. Investors only allocate money to where the return justifies the risk. Businesses, therefore, should only take risks that can be justified by sufficient returns. A new business venture, such as the building of a factory, must reward the company as much as comparable risks available elsewhere in the market. If it didn't, then that venture ought to drive investors away causing the stock's price to fall. If stock prices truly reflected a company's intrinsic value, then they should send very powerful signals to management about how run a company. As powerful aggregators and processors of information, markets could easily second-guess a CEO's decisions. A sharp increase in a stock's price meant approval and a decrease reflected disapproval.
The Random Walk Model
In the first tests of weak form efficient markets, researchers looked to see if prices followed a random walk. In a true random walk, like the flipping of a coin, future price movements are independent from past price movements and are normally distributed. If financial markets followed a random walk, then that would be a sufficient condition to satisfy weak form efficiency.
Prices do not follow a random walk. Research by Benoit Mandelbrot "and then by others, shows that many financial price series have a 'memory' of sorts. Today does, in fact, influence tomorrow. If prices take a big leap up or down now, there is a measurably greater likelihood that they will move just as violently the next day" (12). Volatility of prices tends to cluster in different periods of time. In financial markets, price changes are not independent from one another the way the last flip of a coin cannot influence the next flip. Not only can tomorrow's price changes be influenced by the same factors as today's, today's price changes could influence tomorrow's.
In a normal, or Gaussian, distribution, one would expect many instances of small price changes in either direction with fewer and fewer instances of larger price changes. The normal distribution is defined by the mean, the average change in a stock's price, and the standard deviation, a yardstick used to measure the level of dispersion of actual price changes around the average. The standard deviation, or sigma, is defined such that about 68.2% of price changes fall within one standard deviation, about 95.4% of price changes fall within two standard deviations, about 99.7% of price changes fall within three standard deviations, and then the likelihood of seeing larger price changes rapidly approaches, but never reaches, 0.
Price changes are not normally distributed in financial markets. The random walk model predicts that there should be 58 days from 1916 to 2003 when the Dow Jones Industrial Average "moved more than 3.4%; in fact, there were 366. And index swings of more than 7% should come once every 300,000 years; in fact, the twentieth century saw 48 such days" (Mandelbrot 13). Andrew Haldane observed during the financial crisis of 2007 that there were 25 sigma events occurring several days in a row. Just one such "event would be expected to occur once every 6 x 10124 lives of the universe" (1). Any investment strategy that relied on financial markets following a random walk would surely fail spectacularly during these seemingly improbable events.
Acceptance of the random walk model is not necessary to accept weak form efficiency (Fama, Efficient 391-395). Weak form efficiency only says that past patterns in price movements cannot be used to profitably predict future movements. Early on, researchers discovered that price movements did not follow a random walk over long periods of time (De Bondt 190). Perhaps, some hoped, the random walk model was a close enough approximation to reality that investors could use it to make decisions.
In the short run, the random walk did approximate financial markets very well because transaction costs would wipe out any ability to exploit anomalies (Fama, Efficient 385-396). In the absence of some substitute, many investors hoped to proceed, but with the working knowledge that extreme events occurred historically in financial markets and would probably occur again. Ignorance of this fact could lead to financial ruin over a long enough investment horizon.
The toolkit of modern finance relied very heavily on the random walk model. The principle behind Modern Portfolio Theory and the Capital Asset Pricing Model, that diversification reduces risk, was sound, but in a non-normal distribution of price changes, they could significantly underestimate risk. In times of crisis, covariance tends to rise and the good bets no longer balance out the bad as well as they used to. Value at Risk was continuously tweaked and adjusted to try to obtain an accurate estimation for how frequently investors would face large losses. As long as it relied on some form of the normal distribution, it seemed to be unable to do so. The Black-Scholes formula was also an unreliable estimator of extreme price movements.
The random walk model and the tools of modern finance were very accurate most of the time. It is only during infrequent extreme events where their usefulness disappears and they begin to underestimate risk. Because of these extreme events, there was a great cost to using these tools. The greatest danger was not that money managers might miss out on profit making opportunities, but that they might be lulled into a false sense of security about the risks they were taking. By making bets that they otherwise would not have made, they faced the risk of being forced to prematurely sell off assets in order to maintain capital requirements or even of becoming totally insolvent.
The random walk was a sufficient condition for an efficient market, but it was not necessary. The mere existence of extreme events was not enough to reject efficient markets. Other types of anomalies were necessary to reject the efficient market hypothesis.
How efficient are markets?
Researchers did begin to notice clear inefficiencies in stock markets where price changes were the result of effects that had nothing to do with the fundaments of a company. Stock prices tended to go up between December and January, small firms' stocks tended to have higher returns than large firms, stock markets tended to do poorly on Mondays, stocks with the higher dividend yields tended to have higher returns, among others (Shiller 189; Malkiel 247-263).
Such small anomalies whether among individual stocks or markets as a whole didn't matter all that much to investors and businesses. They were mere exceptions to the rule of efficiency which pervaded financial markets, and arguably the results of data mining. Some of the effects even seemed to disappear as they were documented and investors noticed them, strengthening the case for efficiency (Malkiel 247-265).
Behavioral economists needed to look at larger anomalies in financial markets in order to attack efficient markets. They particularly looked at asset bubbles, and tried to explain them using knowledge from other social sciences, such as psychology. A bubble occurs when some class of asset prices exceed their intrinsic value for a prolonged period of time. They can occur when some real positive economic trend is overestimated by investors. These inaccurate and biased forecasts are caused by measurable psychological factors and can be inflated by leverage. When they end, prices tend to crash quickly, often to irrationally low levels.
Traditional economic theory assumed that all human beings were rational decision makers. Their decisions were nothing more than the result of a calculation of the cost and benefit of each alternative available to them. If the benefit outweighed the cost by the largest amount, it was a rational decision to select that option. In finance, rational investors ought to select only those securities which offer the highest return for the amount of risk they must bear. Bubbles would be impossible in an efficient market.
Behavioral economists developed an entire field of empirical research to show that, in real life, humans suffered from various psychological biases, often relied on heuristics instead of dispassionate calculations, and could be tricked by framing effects. One of the best observed biases is that investors are overconfident in their skill as traders (Shiller 152-155). This leads to overtrading and transaction costs often wipe out the potential profits of impatient investors. Another bias is overoptimism, which leads investors to think that markets can go nowhere but up, or that if they did go down, then they would surely go back up quickly (Shiller 106-125).
Besides other biases, investors also rely on heuristics, or rules of thumb. They might use the availability heuristic, for instance, to generalize conclusions based on small unreliable but familiar samples. Framing effects can also be used to trick people into producing different decisions based on how two identical situations are presented to them. People might suffer from loss aversion, for instance, where they prefer avoiding a loss to acquiring an identical gain.
Behavioral economics documents numerous different ways in which people are definitively not rational. If investors are so irrational, then allocative efficiency must be called into question. Even so, all investors don't need to be perfectly rational in an efficient market. There could be some who overprice and others who underprice, but they ought to cancel each other out on average. This is not the case. They often suffer the same biases, and they often act in herds (Shiller 157-160).
When behavioral biases cause mispricing in the market, rational profit seeking arbitrageurs ought to push prices back to their intrinsic values, but this is not always the case. There are many limits to arbitrage which often make it ineffective for enforcing market efficiency. There are simple transaction costs beyond the risks involved in arbitrage. Security borrowing costs can also be prohibitively expensive if there is a limited supply of lenders for a specific security. Firm-specific risk cannot be effectively hedged if no highly negatively covarying position is available.
Noise trader risk is the most significant limit to arbitrage. A noise trader is someone whose trades are not based on financially meaningful information or analysis. A noise trader could be anyone from an amateur investor who mistakenly trades based on information they read in The Wall Street Journal, to a money manager whose performance based pay structure encourages him or her to make risky bets with short-term pay offs. They could act together to worsen a mispricing in the short run and squeeze the rational arbitrageur. This is certainly not impossible because it was the noise traders who created the significant mispricing in the first place (DeLong, Noise Trader).
Arbitrage requires a certain amount of faith that several factors will work to correct a market anomaly. Irrational investors can realize their errors and stop causing a mispricing. Rational investors ought to persistently make money and overtake irrational investors who are losing money. With more money gained from profitable trades, the more rational investors ought to have more power to control prices than the irrational investors. Other likeminded arbitrageurs should also attempt to exploit the same anomaly. Hopefully, all of these factors will correct the anomaly within a short enough horizon that the arbitrageur doesn't become insolvent waiting for the correction to occur.
If prices do not revert to their intrinsic values quickly, the seemingly rational arbitrageur, whose decisions are based on sound fundamental analysis, might be squeezed, or forced to liquidate early and sustain losses. By liquidating their positions before prices revert to their intrinsic values, the arbitrageur is forced to buy the already overvalued stock and sell the covarying position, worsening the anomaly. Prices can take as long as 3 to 7 years to revert back to their intrinsic values, making arbitrage extremely ineffective in many cases (De Bondt 190).
Even in a market with no transaction costs for arbitrageurs, there is nothing to stop arbitrageurs from being subject to the same fits of irrationality as the rest of the market and causing even worse mispricing (Fama, Disagreement 683).
Completely rational investors can also make decisions independent of the fundamentals of a company. Even if an asset is overpriced, investors can still sell it to some "greater fool" later and profit. From the point of view of an investor, the "right" price, or that price below which it would be prudent to buy, is when that investor believes that the expected return exceeds that of stocks of similar risk. This says nothing about fundamentals or intrinsic values. If an investor can pull out before the bubble bursts, he can make as much or more money than the buy and hold investor or the arbitrageur.
Even with these limitations, financial markets are not simply casinos where irrational speculators set all of the prices. Many real factors can force a security's price to come in contact with reality. All financial securities represent some claim on the issuer's future income. Bonds have fixed coupons and maturity dates. Stock's issuers can pay out dividends, experience mergers, engage in buy backs, have secondary public offerings, liquidate and enter bankruptcy, or even spinoff separate companies. Behavioral economics is important because it helps explain why some investors ignore these factors.
Even with many very real factors effecting securities' prices, however, there is significant evidence that there are other factors at work. Stock market prices have been far too volatile historically to be justified by fundamentals like dividends (Shiller 191). The market, as a whole, must either be overreacting to new information or making decisions completely independent of real fundamentals to some extent.
Robert Shiller famously looked at the 1998-2000 tech bubble in the stock market in his book Irrational Exuberance. Stock prices were rising relative to companies' earnings and other fundamentals well beyond any level that they had historically (5-8). There are three possible explanations for this behavior. Investors might believe that they would receive higher returns than they had historically. Investors might believe that there was exceptionally low risk in the market. Or investors might simply be demanding lower returns than they had historically because their risk tolerance had increased.
Shiller surveyed investors and found that, regardless of unprecedented stock prices, they believed that stocks would give them even higher returns. They also believed that even if there ever were a crash, then surely the market would quickly come back up (57-58). If the market were to be believed, the economy had entered into a new era of unprecedented growth and stability. Investors turned out to be wrong on both counts, and market activity turned out to be little more than a speculative bubble, not some new era of increased productivity.
Shiller also looked at the housing bubble of 2004-2007 which led to a financial and economic crisis during the crash. Much like the tech bubble, despite historically high price levels, people believed that real estate was the best investment (58). Homebuyers seemed to believe that real estate was safe and would continue to rise.
This time, the bubble turned out to be much larger and more highly leveraged. Mortgages were converted into tradable financial assets. As asset prices rose, more credit was made available to homebuyers to meet the growing demand of investors. As long as home buyers could make their payments either with their income or by refinancing a house that was growing in value, then the financial assets would remain valuable. When the real estate bubble burst, the financial asset bubble soon followed. This resulted in a global economic crisis. Shiller argued that both bubbles, like others historically, were the result of psychological biases among investors.
The problem with such arguments is that it gives a very imprecise view of asset prices. As Alan Greenspan, Chairman of the Federal Reserve 1987-2006, pointed out, "you cannot forecast a crisis. You can very readily forecast when firms, or the system as a whole is significantly underpricing risk. All you have to do is look at yield spreads. You look at it every morning and you know that they're underpriced. What you cannot do is to forecast when that underpricing of risk will all of a sudden erupt in a crisis" (@ Brookings). For investors, behavioral explanations are of limited use. They can explain why bubbles occur, but not how big they will get or how long they will last. It is still difficult for investors to gain the risk-free returns that they seek.
The idea that prices reliably reflected an underlying asset's intrinsic value in an efficient market was empirically and intellectually defeated in a debate that culminated in the financial crisis of 2007. The idea that investors could not readily increase their return without increasing their risk required new asset pricing models and new hypotheses to describe the behavior of financial markets.
As knowledge of statistical dependence and non-normal distribution of returns increased, the weak form efficient market theory necessarily replaced the random walk hypothesis. Mandelbrot developed the Multifractal Model of Asset Returns in 1997 to try to adequately explain the anomalies present in random walk and other Gaussian based models. Besides abnormal distributions of price changes and clustering of volatility, Mandelbrot was able to provide a market model which did not exclude endogenous effects, discontinuity of prices, long-term dependence, self-similarity, and asset bubbles (227-252).
Not entirely inconsistent with weak form efficient market theory, the Multifractal Model of Asset Returns allows investors to more accurately predict volatility and measure and avoid risk, not necessarily obtain risk-free profits. Conventional weak form market efficiency wisdom tells investors to buy and hold over long horizons rather than try to time the market (Malkiel 171-172). If volatility clusters and can be forecasted with any reliability, however, then it might very well make sense to pull out during such risky times (Mandelbrot 234-235).
Much of the problem with testing the efficient market hypothesis is that it needs an asset pricing model in order to be tested at all. If the asset pricing model is falsified, it could be an indication that both are wrong, but only the asset pricing model can be rejected with any certainty. This is called the joint-hypothesis problem. The Capital Asset Pricing Model was rejected from mainstream finance theory because it could not adequately explain stock market returns using data after the 1960s (Fama, The Cross-Section 428).
Based on empirical observation, it seemed asset pricing models had to take into account size, value, and momentum in addition to beta. Smaller stocks with lower market capitalizations tended to have higher returns. Value stocks with higher book-to-market ratios, or a firms market capitalization divided by its total equity, also tended to have higher returns. Market prices also exhibit momentum. A rising or falling stock tends to continue to do so in the short run (Fama, Disagreement 679-683). They also tend to do the opposite over the long run, or in response to extreme movements, and reverse direction (De Bondt 191-198). This is called mean reversion.
It is difficult to reconcile these other factors with the efficient market hypothesis. If the hypothesis is true, then each factor represents a risk. This conclusion seems like a strained, ad hoc solution to the joint hypothesis problem. Moreover, the factors are not always dependable (Malkiel 259-265). The behavioral explanation, that these anomalies represent investor tastes or mispricing, is much more convincing.
In 2004, Andrew Lo proposed the Adaptive Markets Hypothesis in an attempt to reconcile the efficient market hypothesis with observed market anomalies and behavioral biases. In adaptive markets, investors are constantly competing to survive. Different types of investors with different tastes, risk tolerances, and heuristics are like different species. Much like evolutionary theory, those investors with the heuristics most adapted to the current state of the market are most likely to grow rich and survive. In a process of natural selection, those that don't will gradually lose money and either pull out or die off.
As the investment environment changes over time, new species of investors with the heuristics most adapted to the new market will survive. The survivors can only be determined through trial and error. Unlike the efficient market, the adaptive market does not tend towards a static equilibrium. Equilibrium is constantly changing in an adaptive market.
Theoretically, heuristics are something to be actively avoided in an efficient market. In an adaptive market, heuristics are just part of how investors deal with uncertainty. They must make mistakes and learn new heuristics if they wish to survive.
Practically, the Adaptive Markets Hypothesis has limited application because it is still so young. In an adaptive market, investors must recognize and control their preferences, professionals must realize that traditional asset allocation tools are not effective in a market of dynamic risk characteristics, and arbitrageurs can take advantage of substantial opportunities (Lo 32-38).
It is likely that market prices and business cycles are heavily influenced by waves of optimism and pessimism, changing tastes, an adaptive market, as well as fundamentals. While it is very easy to anecdotally confirm these ideas, coming up with coherent theories and models is much more difficult.
The Financial Crisis and Efficient Markets
Previously, financial regulators assumed that markets were efficient and that rational self-interest was sufficient to prevent a serious financial crisis. During the financial crisis beginning in 2007, it turned out that these assumptions had serious consequences for financial markets and the real economy. At best, these assumptions allowed regulators to be apathetic and stand by during the building of the bubble. At worst, it has prevented policymakers from reacting aggressively enough to the economic crisis. Recognizing that markets are not efficient presents opportunities both for economic recovery and for preventing future crises from damaging the real economy.
Efficient markets were at the heart of financial regulatory policy under Alan Greenspan. It was the job of regulators to ensure a fair playing field with no insider trading, market manipulation, or fraud. The market, the theory went, would handle the rest. Risky behavior would be self-regulating because firms wouldn't have any motivation to take risks without being rewarded appropriately. Those that did take on unnecessary risk would be outperformed by those that didn't and gradually go out of business. The continuing progress of information technology and financial innovation could only increase investor's ability to measure and allocate risk and the market could only become ever more efficient.
Moreover, as long as inflation stayed low, the Federal Reserve could maintain low interest rates and try to keep employment high without any consequences. In the unlikely event that speculative bubbles ever got out of control, the Federal Reserve could act as a lender of last resort and bail out the market by lowering interest rates as it did after the tech bubble burst in 2000.
There was some logic to Greenspan's view. More restrictive regulatory policy required second-guessing the consensus opinions of millions of investors. It seemed unjust for the government to stand between consenting adults or firms who wished to borrow or lend. It was not the Federal Reserve's place to intervene in asset bubbles, but it would standby to clean up the resulting mess if need be. Central banks only existed because historically, financial crises had told policymakers that the market had generally gotten things right, but that the market could not always be counted on to correctly price liquidity during brief periods of panic.
It turned out that financial firms were taking on unjustifiable amounts of risk. The yield that was being offered and accepted for this level of risk in the market was exceptionally low by historical standards. Market prices had strayed very far from their intrinsic values. Alan Greenspan explained to Congress in October 2008:
Those of us who have looked to the self-interest of lending institutions to protect shareholder's equity (myself especially) are in a state of shocked disbelief. Such counterparty surveillance is a central pillar of our financial markets' state of balance. If it fails, as occurred this year, market stability is undermined. What went wrong with global economic policies that had worked so effectively for nearly four decades?... In recent decades, a vast risk management and pricing system has evolved, combining the best insights of mathematicians and finance experts supported by major advances in computer and communications technology. A Nobel Prize was awarded for the discovery of the pricing model that underpins much of the advance in derivates markets. This modern risk management paradigm held sway for decades. The whole intellectual edifice, however, collapsed in the summer of last year. (2-3)It became clear that rational self-interest was not sufficient to prevent financial firms from committing suicide or to prevent broader financial crises.
It is important to examine what went wrong in financial markets. During the crisis, asset prices unexpectedly began to fall. As many large financial institutions began to face threats of insolvency, investors lost confidence in markets. Their trust that their investments represented real wealth, the most important component of financial markets, began to disappear and asset prices declined rapidly.
When asset bubbles collapse, prices fall for rational reasons. Prices have risen too high and must revert to their intrinsic values. Investors want to receive compensation for the risk for which they were not previously receiving enough. During a crisis, people are afraid of losing their jobs, health insurance, and retirement funds. They become more risk averse, demanding a higher premium for the same amount of risk. Instead, they build up savings both to cushion potential future losses and to replenish depleted retirement funds and home values. They also anticipate deflation during a recession, incentivizing them to reduce consumption and allow their dollars to appreciate in value. As they put their money into bank deposits, banks turn around and purchase Treasuries.
Prices also fall for irrational, endogenous reasons. As a firm's asset values are falling and it has liabilities it needs to cover, it's forced to sell otherwise profitable securities in order to maintain capital requirements and meet margin calls. It is also during these periods of crisis that credit becomes very scarce, giving firms no alternative but to begin fire sales. Investors also become irrationally risk averse, demanding overly inflated premiums for even the safest of profitable investments.
A market wide irrational drop in prices is a negative bubble. It does not occur directly because of some drop in fundamental values. It occurs because a drop in fundamental values leads investors to lose confidence in both their own ability and the ability of other professionals to correctly measure and price risk. When there is no confidence that the market is able to correctly measure risk, that the market is efficient, then investors will rush out of risky securities and into risk-free securities.
As prices fall for rational and irrational reasons, the required return that investors demand, or the rate at which they discount future cash flows greatly increases. For a debtor, the probability of meeting one's obligations may not have changed at all, but the cost of capital might become prohibitively expensive. In that case, the same debtor who was able to borrow and reinvest profitably yesterday, can no longer do so today.
The purpose of financial markets "is allocation of ownership of the economy's capital stock" (Efficient 383). If prices are rapidly falling across the market, this sends a very powerful signal to firms "that it is time for mass unemployment" (DeLong, Slouching). Previously profitable business ventures no longer represent enough reward for previous levels of risk and that risk has probably increased. It makes little sense, however, for businesses to shut down and lay off workers because risk tolerance in financial markets has temporarily collapsed. Adherence to efficient markets theory, however, mandates economic nihilism on the part of policymakers. According to the theory, market wide price changes fully reflect changes in the fundamentals of the entire economy and only liquidation of businesses, real estate, and labor can return the market to equilibrium.
If the fundamentals of the underlying businesses and of the economy have not changed greatly, then intervention is necessary to prevent damage from temporary investor mispricing. A negative asset bubble occurs due to endogenous effects, not because of changes in fundamentals. If this negative bubble causes a credit crunch, then it will have negative effect on the fundamentals, driving prices down further. Under Greenspan, the market could not be trusted to price liquidity correctly during brief periods of financial panic. Under Bernanke, the market cannot even be trusted to price risk correctly.
Because a crash in financial markets can have severe consequences in the real economy, dealing with the crash itself requires aggressive monetary and fiscal policy. In a crisis, while required returns are irrationally high for risky securities, the government also has access to interest-free financing. As a result of a flight to safety among investors, Treasuries are selling at negative real interest rates. This presents huge opportunities for stimulus deficit spending, or even a sovereign wealth fund. It makes sense for the government to try to pull forward in time future expenses and pay for them with money borrowed from savers for free.
The government making a credible commitment to modest inflation would also motivate people to move money out of savings and into productive risky investments, as well as consumption. This would simultaneously decrease unemployment in the short run and increase economic activity. In order for monetary policy to be effective at all and for investors to have confidence in that policy, the government needs to print more Treasuries in the short run and move out of the current liquidity trap.
Quantitative easing, or printing money to buy risky securities in addition to Treasuries, is also one of the tools available to the Federal Reserve. When markets fail, this provides much needed liquidity to financial institutions and helps support prices for asset backed securities where much of the trouble began. If those assets are truly profitable, quantitative easing can also be attractive because the assets can be sold and the money "unprinted" once the economy recovers and fears of inflation rise.
In the current situation, quantitative easing is of limited usefulness. Even if the government were to replace every dollar of fundamental value lost in mortgage backed securities, it still would not replace the lost trust of investors in financial markets. Because riskless Treasuries and cash are virtually interchangeable in a liquidity trap, it is difficult for the Federal Reserve to sufficiently motivate financial institutions to move the newly printed money out of reserves and into risky assets by purchasing Treasuries. As the Federal Reserve increases the monetary base, the velocity of money in the economy falls and there is no guarantee of economic growth.
Another aspect of quantitative easing, printing several trillion dollars to buy long-term Treasuries is also a very attractive option. Adding short-term debt to the market and removing long-term debt would alter the risk composition and duration of the asset pool available to investors. This alteration would motivate investors to move into riskier productive assets. This alteration would also flatten the intertemporal price structure, depressing the duration premium in order to make money tomorrow worth closer to what it is worth today. This flattening would incentivize investors into choosing longer-term investments. As inflation expectations rise, the Federal Reserve might also have to begin buying back Treasury Inflation Protected Securities to prevent investors from being able to choose Treasuries as an inflation hedge over productive investments.
Rejecting efficient markets means accepting that this pattern of bubble, crash, and need for aggressive intervention will repeat itself without serious regulatory reform. If bubbles exist, then loose monetary policy will feed into them through easy credit to financial institutions. Greenspan believed that the Federal Reserve could get away with loose monetary policy as long as inflation remained low. The free market was supposed to keep asset prices in line with their fundamental values, not the government. It turned out that there were severe consequences for this thinking.
Raising interest rates to fight speculative bubbles could have been effective, but at great costs in the form of unemployment and stifled economic growth. This view was represented by former Federal Reserve Chairman William McChesney Martin, whose job it was "to take away the punch bowl just as the party gets going" (Mankiw). Interest rates, however, are far too blunt an instrument, one meant to combat recessions and inflation. Punishing American workers for the speculative excesses of Wall Street would be neither popular, nor good policy. Strict monetary policy would only serve to push capital and economic growth into other countries.
While asset bubbles can't be entirely avoided, regulatory policy is the best tool for reducing the severity, frequency, and impact on the real economy of future speculative bubbles. If markets cannot be made efficient, they can at least be made safe. Even though the crash following the tech bubble was bad, the stock market never suffered a total failure and monetary policy was sufficient to recover the economy relatively quickly from the resulting recession. A world where bubbles and crashes were only as bad as the tech bubble would be much preferable to crashes that threaten to cause another Great Depression.
Capital requirements for financial institutions need to be raised. As financial institutions use leverage to increase their returns, by definition, they also increase their risk. When firms increase their risk, they increase their counterparty's risk, which in turn increases that firm's counterparty's risk. This is what is meant by too interconnected to fail. Leverage increases fragility in financial markets.
Calls for deleveraging, households and firms converting debt into equity in the short run, are necessary, but don't go far enough. New and poorly understood financial instruments must be closely regulated. Increasing complexity in financial products is intended to transfer risk more effectively and increase the amount of capital available in the system. These are admirable goals, but complexity creates opacity. Derivatives can effectively increase the amount of leverage and risk that institutions are taking on and do so outside the view of investors and counterparties. This creates systemic risk.
High risk institutions need to be segregated from depositors and institutions that are open to taxpayer bailouts. Until then, there will always be moral hazard present in the system. It may very well be impossible for regulators to spot bubbles, and even if it were, it might be impossible to do anything about them. This is why it is necessary to insulate the real economy as much as possible from the riskier parts of the financial sector.
Financial firms must also bring their employees' compensation structures in line with the long-term interests of the company. Too often, money managers are allowed to make very in-the-money bets that pay quickly, but have small chances of leaving financial institutions with very large liabilities. Traders are rewarded for the short-term pay off in incentive based compensation structures, but long-term losses are absorbed entirely by the financial institution. This causes institutions to make bets that they would not rationally make otherwise, and creates systematic mispricing in financial markets. This is known as the agency problem.The Financial crisis has made it clear that even the Federal Reserve's position as lender of last resort, printer of money, and controller of interest rates is insufficient at protecting the economy from asset bubbles. While aggressive monetary and fiscal policies are necessary for an economic recovery, good regulatory policy presents the only hope at preventing the next financial crisis from becoming an economic crisis. The self-regulatory policies of an efficient market are dangerous and should never again be trusted.
The first prediction of the efficient market hypothesis, that it is extremely difficult for investors to obtain excess returns, remains one of the most profound statements in economics. The second prediction, that prices reflect the intrinsic value of an underlying asset, has been soundly rejected in favor of explanations by behavioral economists. Investor irrationality and behavioral biases cause prices to stray from their intrinsic values, and the limits to arbitrage can prevent them from reverting back to their intrinsic values for prolonged periods of time. While new models are developed to deal with risk in financial markets, investors, businesses, and regulators are left waiting in the meantime.
It remains to be seen whether the government will correct its lackluster regulatory, monetary, and fiscal policies. If not, the economy faces the risks of both a prolonged recession, and future cycles of bubble and crash which threaten the real economy. When regulators learn what Wall Street has known for a long time, that markets are not perfectly efficient, then good policy can start to be made. Hopefully, the most recent crisis will strengthen the case for informational efficiency and discourage investors from thinking they can easily obtain such high returns without taking on extra risk. More importantly, this crisis must also repudiate the idea of allocative efficiency and discourage regulator apathy. While the efficient market hypothesis does not need to be discarded to the dustbin of economic thought, it needs to be reconciled with empirical observation and interdisciplinary knowledge.