Can You Beat the Market? It's a $100 Billion Question

Investors collectively spend around $100 billion a year trying to beat the stock market. That’s the finding of a rigorous effort to measure the total costs of Americans’ efforts to surpass the returns they would have received by simply holding a stock index fund. The huge price tag helps explain why beating a buy-and-hold strategy is so difficult.

The study, “The Cost of Active Investing,” began circulating earlier this year as an academic working paper. Its author is Kenneth R. French, a finance professor at Dartmouth; he is known for his collaboration with Eugene F. Fama, a finance professor at the University of Chicago, in creating the Fama-French model that is widely used to calculate risk-adjusted performance.

In his new study, Professor French tried to make his estimate of investment costs as comprehensive as possible. He took into account the fees and expenses of domestic equity mutual funds (both open- and closed-end, including exchange-traded funds), the investment management costs paid by institutions (both public and private), the fees paid to hedge funds, and the transactions costs paid by all traders (including commissions and bid-asked spreads). If a fund or institution was only partly allocated to the domestic equity market, he counted only that portion in computing its investment costs.

Professor French then deducted what domestic equity investors collectively would have paid if they instead had simply bought and held an index fund benchmarked to the overall stock market, like the Vanguard Total Stock Market Index fund, whose retail version currently has an annual expense ratio of 0.15 percent.

The difference between those amounts, Professor French says, is what investors as a group pay to try to beat the market.

In 2006, the last year for which he has comprehensive data, this total came to $99.2 billion. Assuming that it grew in 2007 at the average rate of the last two decades, the amount for last year was more than $100 billion. Such a total is noteworthy for its sheer size and its growth over the years — in 1980, for example, the comparable total was just $7 billion, according to Professor French.

The growth occurred despite many developments that greatly reduced the cost of trading, like deeply discounted brokerage commissions, a narrowing in bid-asked spreads, and a big reduction in front-end loads, or sales charges, paid to mutual fund companies.

These factors notwithstanding, Professor French found that the portion of stocks’ aggregate market capitalization spent on trying to beat the market has stayed remarkably constant, near 0.67 percent. That means the investment industry has found new revenue sources in direct proportion to the reductions caused by these factors.

What are the investment implications of his findings? One is that a typical investor can increase his annual return by just shifting to an index fund and eliminating the expenses involved in trying to beat the market. Professor French emphasizes that this typical investor is an average of everyone aiming to outperform the market — including the supposedly best and brightest who run hedge funds.

Professor French’s study can also be used to show just how different the investment arena is from a so-called zero-sum game. In such a game, of course, any one individual’s gains must be matched by equal losses by other players, and vice versa. Investing would be a zero-sum game if no costs were associated with trying to beat the market. But with the costs of that effort totaling around $100 billion a year, active investing is a significantly negative-sum game. The very act of playing reduces the size of the pie that is divided among the various players.

Even that, however, underestimates the difficulties of beating an index fund. Professor French notes that while the total cost of trying to beat the market has grown over the years, the percentage of individuals who bear this cost has declined — precisely because of the growing popularity of index funds.

From 1986 to 2006, according to his calculations, the proportion of the aggregate market cap that is invested in index funds more than doubled, to 17.9 percent. As a result, the negative-sum game played by active investors has grown ever more negative.

The bottom line is this: The best course for the average investor is to buy and hold an index fund for the long term. Even if you think you have compelling reasons to believe a particular trade could beat the market, the odds are still probably against you.