Stocks overvalued longer and more often than previously thought, says study

Published in the journal Mathematics and Financial Economics, the study may be the first to estimate the number of bubbles by analyzing nearly continuous trading of the top 3,500 stocks in the United States stock market. Between 2000 and 2013, the researchers counted up 13,000 bubbles, or an average of four bubbles per stock.

"Our results add further evidence that financial markets are neither efficient nor rational," said Philip Protter, a statistics professor at Columbia University and a member of the Data Science Institute.

A bubble happens when the price of an asset, be it gold, housing or stocks, is more than what a rational person would be willing to pay based on its expected future cash flows. Thus, the key to detecting when a stock enters a bubble is pinpointing when its market price has exceeded its fundamental worth. In a series of papers, Protter and his colleagues developed a statistical test for doing just that.

In the current study, they applied their test to a big data set of second-by-second trades on NASDAQ and the New York and American stock exchanges -- the Trade and Quote (TAQ) database. Stocks were found to be entering a bubble phase when their market price qualified as a strict local martingale, a statistical measure of high volatility based on the assumption that a process can be stopped at any time.

To adjust for normal volatility, the researchers counted the start of a bubble when their test indicated a bubble was present and its market price had risen at least 5 percent; and the end of a bubble, when their test found a bubble was no longer present and its price had fallen at least 5 percent. After several days of number crunching, the supercomputer they borrowed for their analysis came back with 13,060 bubbles.

"I expected to see lots of bubbles in 2009, after the crash, but there were a lot before and a lot after," said study coauthor Shihao Yang, now a graduate student at Harvard.

The researchers also found that the stock bubbles in their analysis were relatively long-lived, with a median lifespan of 206 days, or about six months. They had expected the bubbles to follow an exponential distribution and die relatively quickly. Instead, they found a generalized gamma distribution, typically associated with physical systems. The longer that a bubble live, the more likely it will be to go on, until something catastrophic happens.

The study could be useful to regulators in deciding how much cash and stocks financial institutions need to set aside to cover losses. Many of the banks involved in the 2008 financial crisis, for example, failed to keep enough capital on hand to cover a series of increasingly risky bets. If regulators could determine which stocks in a bank's capital reserves are overvalued, they could presumably ask for more stocks to be set aside.

The study could also help regulators and investors estimate bubble longevity. "If you know the age distribution, you can give probabilities of how much longer the bubble will last, similar to the probability of how much longer you're going to live," said Protter. "If you're in a bubble you want to know how much longer before it bursts, so to speak."

Economists have developed a number of ways to spot bubbles. Comparing inflation or wages against changing home prices is one way to identify housing bubbles. Stock bubbles can be detected by comparing a company's share price to its per-share earnings, or P/E ratio, or evaluating share value based on a company's estimated future cash flows under a discounted cash flow analysis.

But all of these methods rely on assumptions beyond price information. The local martingale method developed by Protter and his colleagues over the last decade looks at change in price over time alone.

"This is unique," said Robert Jarrow, a finance professor at Cornell University who helped develop the method. "It's based on probability theory and the characteristics of a price process."