Daily Speculations

The Web Site of Victor Niederhoffer & Laurel Kenner

Dedicated to the scientific method, free markets, deflating ballyhoo, creating value, and laughter;  a forum for us to use our meager abilities to make the world of specinvestments a better place.



Write to us at: (address is not clickable)

Long-Term Returns

By Victor Niederhoffer and Alex Castaldo (April 2004)

Traders as well as investors need to know the distinct personality of every available market.  The equity markets' chief characteristic has been a relentless long term upward bias in prices: a random selection of U.S. stocks returned a million and a half percent over the twentieth century.  The implications are monumental.  This guarantees that a large short position in equities held over a sufficiently long time will ruin the wealthiest person. When trading equities we recommend shorting only for brief periods and with positions that are one quarter or one eighth of the size of long positions.  For investors a fairly steady accumulation of equities over time is a sound policy.

How academics and investors came to these views about long run returns is an interesting story in and of itself.

Early Studies

The first credible attempt to estimate the long run return on stocks empirically was made by Edgar Lawrence Smith in his 1926 book <Common Stocks as Long Term Investments>.  Collecting data by hand for the most popular (i.e. most actively traded) stocks from 1901 to 1922, he showed that, assuming no market timing or stock selection ability whatsoever but simply holding on to stocks, an investor could have earned a satisfactory rate of return and in fact outperformed a bond investor over this period.  This result was at variance with traditional opinion, which regarded stock investing as a treacherous field where only the well informed, well connected or lucky make money from in and out trading at the expense of those less fortunate.  Serious long term investors preferred bonds. Mr. Smith said stocks would do better.  Three years after publication, the bottom fell out of the market and Mr. Smith lost favor.  None other than Benjamin Graham blamed Smith's book for inspiring an orgy of uncontrolled speculation that led directly to the market crash.  But on the fundamental point of long run stock and bond returns there is no doubt who was right.  We know now that from 1926 to 2001 stocks returned 10.9% a year while bonds returned 5.8% (Ibbotson Associates).  Warren Buffett still holds that Smith's book was a dangerous one; knowledge of investing is best left to the experts and should be hidden from ordinary people.

Mr Smith did a good job for an amateur, but as Will Goetzmann of Yale reminded us in a recent Email, "the most carefully crafted empirical study of the long term performance of the stock market was by Alfred Cowles III, called Common Stock Indices, published in 1938.  Cowles collected individual stock prices (actually monthly highs and lows by stock) and dividends from 1872 to 1937 for the NYSE, allowing the analysis of total return to equity investing."  Total return means the sum of price changes and dividends, and is now the accepted way of reckoning returns. (Mr. Smith's book used price change only, which makes his figures difficult to compare to later studies). Cowles documented a dramatically positive long term equity performance.

The next step forward took place in 1964 when Fisher and Lorie published their article "Rates of Return on Investments in Common Stock" in the Journal of Business.  It was based on the first computerized database of stock prices recently completed at the University of Chicago's Center for Research In Security Prices (CRSP).

The history of science is full of examples of monumental collections of data that sparked revolutions in knowledge.  Tycho Brahe's careful and extensive measurements of planetary positions, based on observations made with his innovative astronomical instruments, led to the development of Johann Kepler's Laws on the motion of the planets.  The data on plants collected by Carl Linnaeus in the early 18th century laid a base for Charles Darwin's discoveries and groupings.  Painstakingly exact and detailed measurements by Henry Cavendish and Antoine Lavoisier on the various chemical elements led to Dimitri Mendeleyev's discovery of the periodic table.  The CRSP (pronounced "crisp") data base brought the field of security analysis into the modern era. Hypotheses on portfolio analysis, market efficiency, behavioral finance or price reactions to events could now be tested empirically.  The resulting discoveries are chronicled in any standard college finance text and have been rewarded with numerous Nobel prizes.

Fisher and Lorie showed that the total return from a random investment in NYSE stocks from 1926 to 1964 was 9.1% a year.  Again there was some surprise the figure could be so high. Stock analysts felt a conspiracy was afoot to downplay the value of their advice; they derided the idea of investing "randomly".  Once again academic findings had a way of coming down the pike at exactly the wrong time for investors.  Four years after publication, when everyone was familiar with Fisher and Lorie's study and the Ford Foundation had used it to urge an increase in endowment funds' exposure to stocks, the market peaked.  Once again disgruntled investors focused their ire on Fisher and Lorie, for promising that stocks would always go up 9.1% each year, although they had never said such a thing.

Prof. Goetzmann again: "It is also worth mentioning the 1976 Ibbotson Brinson studies published in Journal of Business.  They made a very bullish forecast with confidence intervals for the next 20 years.  It was an early application of the simulation approach to forecasting the market - and it turned out to be right on, depsite the nay-saying tone of equity investors in the mid-1970's. While Fisher and Lorie published an important academic book, Ibbotson really popularized the equity premium among investment managers.  By the way the Dimson data is available via Ibbotson software for your readers".  Ibbotson also publish an annually updated study "Stocks, Bonds and Inflation 1926 to Present". "In my view, the Ibbotson annual study has done more to convince money managers that the equity premium is large and economically important than any other publication.  Also by putting the data on the desktops of thousands of investment managers around the world, they have encouraged the use of the equity premium in asset allocation."

To further refine our estimates of stock returns, we would need large amounts (decades) of fresh data.  Unfortunately such data is hard to come by.  For the United States, the Cowles Index goes back to 1872.  Special research efforts have been needed to go back further in time. Goetzmann, Roger Ibbotson and Liang Peng collected U.S. stock market data by hand from 1816 when the official list of the NYSE became available. "This gives us an index that is not based on the chained, previously-known indices. Our study "A New Historical Database for the NYSE 1815 to 1871: Performance and Predictability" in The Journal of Financial Markets (2001) documents a substantial equity premium for the U.S. over history. The analysis is based, for the first time on hand-collected individual security price data and hand-collected dividend data.  This data - even down to the individual prices - is available for your readers at the web site www.icf.yale.edu.  It took ten years to collect."

Using data available to him for the United States Siegel came to the conclusion that the long term rate of return on stocks was 6.75% after inflation or 8.75 to 9.75% if we assume two to three percent inflation (Siegel: Stocks for the Long Run, 1994, 2002).

Consider how much uncertainty is associated with this number.  With 167 years of data, and assuming an annual standard deviation of 20%, the standard error of the mean is 1.5%.  To quote a 95% confidence band around Siegel's estimate we would have to say "the real long term rate of return on US stocks is 3.75% to 9.75%".  That is quite a large range.

To refine our estimate we have to look at markets outside the U.S.  Jorion and Goetzmann pioneered this so called cross-sectional approach in their article "Global Stock Markets in the Twentieth Century" (1999). They studied a large number of markets around the world beginning in 1921.  They came to three conclusions: "First, the U.S. was the leading performer over this period, as measured by real stock price appreciation.  Second, a large number of markets did very poorly over the period -- particularly South American and Central European exchanges.  Third, a GDP-weighted index of the world's markets performed nearly as well as that of the U.S., suggesting that, while, due to survivorship, the U.S. experience gives us a rosy picture of equity market performance, the world equity market portfolio yielded a significant equity premium (as measured over inflation) as well."

Our favorite book on global stock market performance is Triumph of the Optimists by Dimson, Marsh and Staunton, published in 2002.  It studies 16 markets from 1900 to 2000. Once again the experience of the long term stock investor is found to be quite satisfactory, with rates of return similar to those in the United States; the US did perhaps a little better than the others, but three markets did even better than the US.

Elroy Dimson, Paul Marsh and Mike Staunton of the University of London Business School worked together on this massive project.  Within Triumph's pages, an investor may find definitive information on inflation adjusted returns for stocks, bonds and treasury bills, real dividends, correlation between markets worldwide, and the relative performance of value and growth stocks.

Unlike most books written by academics, Triumph avoids hasty generalizations and biased sampling procedures.  The authors rightly fault earlier investment studies for arbitrary selection of starting and stopping points, the tendency to include the good and exclude the bad, and a parochial focusing on a small slice of the global picture.  Their work epitomizes outstanding investment research.

Great works can be created in humble circumstances.  Shakespeare was an actor and entrepreneur who reworked old plots so that his theatre company could make a buck.  Cervantes wrote a parody of the fashionable knight errantry books to repay his debts.  Dimson told us that he and his colleagues thought of Triumph as "a labor of love, just a small contribution that could lead to a paperback meant for light reading on planes".  He added, "Our families would be less kind about our fixation." Staunton, who collected the data, prefers to gather statistics by himself from original sources at specialized libraries instead of delegating the work.

The main conclusion of Triumph is that a random selection of US stocks returned 1,500,000 percent in the twentieth century.  Yes, big losses occurred at times, such as the back-to-back losses of -28 percent and -44 percent in 1930 and 1931, or the 10 years from 1970 to 1979 when stocks hardly budged while the dollar lost 28 percent of its purchasing power.

But overall, adjusted for inflation, the return on US stocks amounted to 6.3% a year, better than any other class of securities.

Current Studies

Unless intelligent life is discovered on another planet and a stock market is found to have been operating there for some centuries, it is unlikely that much new data can be brought to bear on the problem of long run stock returns. Therefore Triumph of the Optimists may well be the last word on the subject for some time to come.  Nevertheless we try to keep in touch with the literature and have recently reviewed 5 published papers.  We begin with the latest from the authors of Triumph of the optimists.

Elroy Dimson, Paul Marsh, and Mike Staunton, Irrational Optimism, Financial Analysts Journal, January/February 2004.

The article is addressed to Pension Plan managers and individual investors.  The authors feel that the estimates of long term return they are using are too high. They claim pension plan sponsors are assuming 12.5% nominal, 10% real return on stocks, and that many individual investors assume even higher returns. "In this article, we show that, historically, annualized long-run equity returns have not been as high as 10 percent in real terms anywhere in the world. Over the past 103 years, a more typical figure has been 4-6 percent. Furthermore, a careful analysis of historical returns indicates that future risk premiums are likely to be lower than in the past."

The authors also argue against the idea sometimes seen in popular investment books that "stocks are safe if held for 20 years".  That is you can never lose money if you hold onto your investment for 20 years.  This idea originated in Siegel's book and has been repeated widely ever since.

The authors feel that the US record is exceptional, that the US has in some sense been a unique country and therefore they prefer to work with a sample of 16 countries, also over the last 103 years.  This is the most countries for which they could find data, although they admit that "Our sample [] omits such countries as Russia, China, and Argentina, whose stock and bond markets have performed poorly and have been highly volatile."

For the World Index, composed of 16 countries weighted by GDP, the geometric mean real return was 5.4%, the arithmetic mean 6.8%, the standard deviation 17.2%.

The authors point out that some countries have experienced bad returns over a 20 year interval.  For example in Japan 22% of all (overlapping) 20 year intervals have negative real returns.  In fact a negative 20 year return exists for 11 of the countries in the sample, contrary to the myth that you can never lose money over 20 years.

Now the authors are ready to make their forecast.

They agree with Arnott and Bernstein (2003) [see below] that exceptional factors pushed the performance up in the 20th century.  Specifically there was "an uplift in equity valuations" measured in terms of dividends or earnings that is unlikely to be repeated.  They state "We project real equity returns of 5% a year".  This is 0.4% below the observed 5.4 mean of the 16 countries. If we add a 2-3% estimate of inflation we come up with 7% to 8% nominal equity returns.

The authors point out that 5% real return is quite good. In fact over 20 years, even "achieving a mean return of 4% would mean that real wealth had increased by a factor of 2.2".

Conclusion: "when investors look back a century from now, equities should prove to have been the best performing asset class in the 21st century.  Nevertheless, the real return on stocks will turn out to be lower than it was in the 20th century."

Also, the authors feel they have shown the idea that <stocks are safe if held for 20 years> is wrong.

Ibbotson, Roger, and Peng Chen,  Long-Run Stock Returns: Participating in the Real Economy.  Financial Analysts Journal, January/February 2003;

Starts with an interesting classification of previous studies:

  1. Historical studies (what have returns been in the past)
  2. Studies that ask what kind of returns corporations can supply.  These use fundamental information such as earnings, dividends and overall economic productivity.  The present study is in this category.
  3. Studies that ask what return investors demand.  This is the approach we favor. Assuming that there will be plenty of opportunities for investment (if nothing else new technology should provide such opportunities), the question is what payoff will people demand for bearing risk.  And this does not change much over time.
  4. Survey studies that use "opinions of investors and financial professionals garnered from broad surveys"

The details of the paper are complex.  The authors consider 6 methods of analyzing fundamental data about US corporations and their growth rates by decomposing it in various ways.  To give a simple example the growth rate of GDP could be decomposed into the growth rate of GDP per capita times the growth rate of population.  One can then plug in historical or subjective growth rates to come up with a forecast.  The authors narrow the choice down to two models they call Model 3F (based on earnings) and model 4F (based on dividends).

Model 3F gives an expected stock return of 9.37%.  Model 4F gives 5.44%. What are the reasons for this discrepancy and which forecast is best?

Using earnings (3F) is preferable because "the growth in corporate earnings has been in line with the growth of overall economic productivity." Whereas "despite the record earnings growth in the 1990s, the dividend yield and the payout ratio declined sharply, which renders dividends alone a poor measure for corporate profitability and future earnings growth."

Also Model 4F is inconsistent with the Modigliani-Miller theorem of modern finance.  If this disrepancy is remedied one ends up with the same results as model 3F.  Another reason for sticking with that model.

In summary the predicted rate of return is 9.37% nominal or 6.09% in real terms, that is after inflation.  Provided one believes in Modigliani-Miller.

The authors point out that this is less than the 10.70% historical figure from Ibbotson, but higher than figures put forth by other authors such as Shiller and Campbell.  The latter are assuming that current high P/E's signal an overvaluation, this paper assumes current valuations are correct and the P/E's reflect a favorable investing environment (i.e. low interest rates and high future growth).

Arnott, Robert, and Peter Bernstein,  What Risk Premium is  Normal'? Financial Analysts Journal, March/April, 2002;

This article is quite pessimistic. The world is not about to end, but "real stock returns will probably be roughly 2-4 percent, similar to bond returns."

Their approach is based on economic growth and dividends.  They make much of the fact that the latter is always less than the former:

"Can't shareholders expect to participate in the growth of the economy?  No. Shareholders can expect to participate only in the growth of the enterprises they are investing in. An important engine for economic growth is the creation of new enterprises. The investor in today's enterprises does not own tomorrow's new enterprises not without making a separate investment in those new enterprises with new investment capital."

"More than half of the capitalization of the Russell 3000 today consists of enterprises that did not exist 30 years ago."

They condider that "The earnings yield is a better estimate of future real stock returns than any extrapolation of the past. And the dividend yield plus a small premium for real dividend growth is even better, because [...] the earnings yield systematically overstates future real stock returns."

"long-term dividend growth should be equal to long-term economic growth minus a haircut for dilution or entrepreneurial capitalism (the share of economic growth that is tied to new enterprises not yet available in the stock market) plus a premium for hidden dividends, such as stock buybacks."

We must caution against this dividend based approach, since dividends are under the control of the corporation and practices in dividend policy have changed a lot over time as Ibbotson and Chen pointed out.

Conclusions. "The observed real stock returns and the excess return for stocks relative to bonds in the past 75 years have been extraordinary, largely as a result of important nonrecurring developments."

"The current risk premium is approximately zero, and a sensible expectation for the future real return for both stocks and bonds is 2 4 percent".  When their expected inflation rate of 1.2% is added in we get an estimate of long run stock returns of 3.2% to 5.2%.

Ivo Welch: The Equity Premium Consensus Forecast Revisited, September 2001,COWLES FOUNDATION DISCUSSION PAPER NO. 1325

This is an example of the survey method. "This paper presents the results of a survey of 510 finance and economics professors." "The consensus 30-year stock market geometric mean forecast is 9.1% with an inter-quartile range of 8 to 10.5%. The forecasts are considerably lower than those taken just 3 years ago."

The rate of return on bonds is estimated to be about 5%. The consensus forecast for the 30-year equity premium (arithmetic) is about 5% to 5.5%

Estimating the Real Rate of Return on Stocks Over the Long Term
Papers by John Y. Campbell, Peter A. Diamond, John B. Shoven
Presented to the Social Security Advisory Board
August 2001

Three prominent academics present their forecasts:

Campbell: "a geometric average equity return of 5% to 5.5% or an arithmetic average return of 6.5% to 7%" in real terms.

Diamond: "projection values around 6.0% or 6.5% seem to me appropriate for projection purposes" in real terms

Shoven: no single forecast given, argues in various ways that a forecast of 7% in real terms would be too high.

Our conclusion

The survey results are interesting only in that they illustrate the error that people make of expecting too much after several years of good results and cutting back sharply after a bad year.  We recommend fighting that tendency and keeping to a very steady estimate of long run returns.

The proposition that stock's premium over bonds could now be zero, as Messrs. Arnott and Bernstein believe, we find hard to swallow.  It would contradict economic rationality as well as all past experience. The best working hypothesis is that stock returns have nothing to do with reported dividend growth and payouts, but have everything to do with the requirements of entrepreneurs for risk capital, the return they can make on investments, and the requirements of investors who invest at risk. The best working hypothesis for the next 100 years is that investors will achieve what they require a priori for their risky investments.

Like a motorcycle rider who still prefers his old and worn leather jacket to the brand new one given to him by his girlfriend, we still retain a great affection for Fisher and Lorie's estimate of 9.1% nominal return on stocks.  It has served us well in many years of stock market riding and has never been refuted by later studies.  Nevertheless, the current academic practice of quoting inflation-adjusted figures is preferable, and Triumph's use of a 100 year window makes it superior to that path breaking study.  Hence our estimate of US stock returns is about 6.3% real.  And it would not surprise us if the actual outcome is one or two percentage points above that.

We appreciate Prof. Goetzmann's inputs for this article.

A short version of this article appears in Active Trader <www.activetradermagazine.com>


Go to more scholarly reviews