Monday, November 28, 2011

The Paradox of Skill

By Brad Steiman

Statistical significance is extremely important when drawing conclusions from noisy data. Noise in the data is problematic when investors overemphasize what might have been a period-specific outcome rather than a robust and repeatable result that is likely to persist going forward. Manager selection is a process that requires analyzing noisy performance data, so it should lean heavily on the notion of statistical significance.

When advisors, consultants, or investors select a manager, their approach often involves a combination of qualitative and quantitative analysis. The latter, by necessity, requires the use of past performance, as it is essentially all we have to evaluate. The focus is typically on managers with a positive "alpha" (i.e., outperformance relative to a benchmark or asset pricing model). Although past performance represents what actually happened, new investors cannot get those returns; therefore, the objective of the analysis is to determine whether outperformance in the past is any indication of skill, or simply good luck. In other words, is the positive alpha likely to persist in the future, after the manager is selected?

This task is more difficult than many investors might think. Several notable studies of manager performance found little persistence in past performance.1 Regardless, investors rarely hire managers with a history of poor relative performance, so the manager selection merry-go-round often becomes:

  1. Hire managers who have outperformed in the past.
  2. Fire managers who underperform in the future.
  3. Repeat.

This activity meets Einstein's definition of insanity, as many investors do the same thing over and over again while expecting different results. There are many reasons why this is an exercise in futility:

  • It generally takes a long track record for a manager's alpha to be statistically significant.
  • Positive alphas, even statistically significant ones, may not be an indication of skill.
  • By the time you are confident there is skill, it's probably too late to benefit.
  • Past performance is (really) no indication of future performance.
  • If you know for sure the manager has skill, you likely won't benefit because the scarce resource captures the rent!

It Takes Time

Finding managers with positive alphas is like looking up the box scores from last night's games, but picking those who will outperform in the future is anything but trivial. To begin with, a quantitative analysis of past performance should incorporate tests of statistical significance to determine the likelihood their true alpha is not zero. In most cases, a long track record is required for the manager's alpha to be statistically significant. Given the average alpha and the standard deviation of the alpha, we can determine the track record required (in years) to obtain a t-stat of 2.

Table 1. Minimum track record for a statistically significant alpha (t-stat > 2)

Average Alpha

of Alpha

A track record of outperforming a benchmark or asset pricing model by an average of 2% per year (net of fees) over the life of the fund would get the attention of many investors, especially when you consider that the equity premium might only be around 5%. A representative standard deviation of alpha in the Morningstar universe of actively managed US equity mutual funds is approximately 6%.2 As illustrated in the table above, a 2% average alpha and a 6% standard deviation of the alpha requires a track record of thirty-six years before you can be 95% sure that the true alpha is not in fact zero (i.e., there was no skill at all). Based on these parameters, by the time you are reasonably confident there is some amount of skill, the manager is likely retired and on her yacht!

The Effects of Chance

Identifying a skillful manager involves more than simply narrowing down the universe to funds with positive alphas and a t-stat of 2 or more. This approach ignores the effects of chance. There is still a 2.5% probability the outperformance was due to good luck, and the true alpha of the manager is zero. Said another way, one out of forty managers is expected to have a positive alpha with a t-stat of 2 by chance. With so many funds in the universe, many will have statistically significant alphas even when there is no skill at all.

For example, in a 5,000-fund universe, 125 managers are expected to have a positive alpha with a t-stat greater than 2, even if their true alpha is zero. Unfortunately for investors, the opportunity is not in sorting through the many managers with statistically significant alphas, but rather in finding these managers because there are too few of them. Fama and French recently studied 3,156 US equity funds and compared their performance to a simulated universe of funds in which the true alpha for every fund was zero. Their results identified fewer funds with statistically significant alphas than you would expect to find by chance.3

By Then, It's Too Late

An investor concluding that a statistically significant alpha is evidence of skill could be guilty of data mining, meaning he is making inferences from what might have been a chance outcome limited to that time period. To counter these claims, academics conduct out-of-sample tests to confirm a statistically significant result. For example, out-of-sample data can be obtained by repeating the experiment using an independent time period (e.g., 1926–1962 rather than 1963–1992) or a different data set from an overlapping time period (e.g., international rather than US market data).

Practitioners should also conduct out-of-sample tests when analyzing manager performance to help rule out that a statistically significant alpha didn't occur by chance. On the surface, conducting out-of-sample tests might seem like an overly cautious approach akin to wearing a belt and suspenders. However, even if this were true, when you consider the consequences and what is at stake, it sure beats getting caught with your pants down.

The only way to test out-of-sample data when doing performance analysis is by using totally independent time periods. Accordingly, the number of years required in Table 1 gets multiplied by the number of independent periods you're comfortable with before you have faith there is some amount of robust and repeatable skill.

Table 2. Minimum track record for two independent periods with statistically significant alpha (t-stat > 2)*

Average Alpha

of Alpha

* Assumes the average alpha and standard deviation is the same in both time periods.

Using the prior example of a 2% average alpha and a 6% standard deviation of the alpha means you need a track record of seventy-two years if you're satisfied with a statistically significant result from only two independent time periods. But, in this instance, by the time you think there is evidence of skill, it's too late—the manager may be dead!


Having said that, let's assume you've found a manager with statistically significant alpha in multiple independent periods and she is not yet retired or deceased. The question still remains: will the positive alpha continue? You cannot rule out luck because of the effects of chance noted above, but more important, many performance studies conclude that winners do not continue to win, and even when there is alpha in the extremes, it does not persist. The only slight indication of persistence is among the extreme losers, and it is mostly explained by high fees and high turnover.4

If we eliminate these funds from consideration, manager selection becomes a random draw, but whether investors know they are picking them at random is another question. Their goal is often to achieve top quartile performance, and pursuit of this goal usually boils down to choosing from managers among the top quartile in the past. However, excluding the persistent losers noted above, yesterday's top quartile performers have the same 25% probability of being in tomorrow's top quartile as every other manager!

Scarce Resources

Ironically, the predicament is not only for the investor trying to identify skill but for managers trying to prove they have it. A fundamental of economics is that the scarce resource captures the rent. If capital is freely floating and perfectly liquid, then the scarce resource is not the investor's money but the manager's skill! There is an enormous economic incentive for managers to indisputably prove they are in possession of this elusive ability.

Let's assume a manager has a twenty-year track record of outperforming by 4% each year. In this extreme example, a t-stat is meaningless since the standard deviation of the alpha is zero. If we rule out the possibility of a Ponzi scheme, the manager surely has undeniable skill. However, as soon as she proves her unique ability, she can capture the rent by increasing her fees to nearly 4%. An increase in fees of this magnitude may draw the ire of her investors, so, alternatively, she could raise more and more assets, thereby distributing her alpha over a larger asset base, which would dilute investor results.5 This latter approach may largely go unnoticed by investors, but either way they lose as their alpha subsequently becomes zero.

Making Progress

So, herein lies the paradox of skill. Many investors are searching for the Holy Grail of fund management. Their goal is to identify a skillful manager with certainty and participate in future returns. But confirming skill takes an investment lifetime, and you can never be fully confident that the alpha is not random. Even if you could identify skill ahead of time, you probably would not benefit. Winning managers hike their fees or attract large volumes of new investment long before their skill is statistically confirmed—and both actions can dilute returns.

But this paradox is not a case of "damned if you do and damned if you don't" for all investors. You can get off the manager selection merry-go-round and start making progress toward a successful investment experience by following these simple principles:

  1. Diversify by asset class rather than by fund manager, broker, or advisor.
  2. Buy into markets, not managers, and let capitalism be your guru.
  3. Focus on what you can control—costs, asset allocation, risks, and discipline. Ignore what you cannot control—the media, prognosticators, market returns, and your gut.
  4. Work with an advisor who understands these principles and can help you apply them.

The comments of Weston Wellington are gratefully acknowledged.

1. Noteworthy studies include: Mark Carhardt, "On Persistence in Mutual Fund Performance," Journal of Finance 52, no. 1( March 1997). Garrett Quigley and Rex A. Sinquefield, "Performance of UK Equity Unit Trusts," Journal of Asset Management 1, 72-92. James L. Davis, "Mutual Fund Performance and Manager Style," Financial Analysts Journal 57, no. 1 (Jan/Feb 2001).

2. Source: Index Funds Advisors.

3. Eugene F. Fama and Kenneth R. French, "Luck Versus Skill in the Cross Section of Mutual Fund Returns," Journal of Finance 65, no. 5 (October 2010): 1965–1947.

4. Carhardt, "On Persistence in Mutual Fund Performance." Fama and French, "Luck Versus Skill in the Cross Section of Mutual Fund Returns."

5. Jonathan Berk and Richard C. Green, "Mutual Fund Flows and Performance in Rational Markets," NBER Working Paper No. W9275, October 2002.

Wednesday, November 16, 2011

Don't Worry Over Largest Ever Muni Bond Default

By Larry Swedroe


Alabama's most populous county declared bankruptcy in hopes of addressing a massive revenue shortfall and its whopping $4.15 billion in debt. Jefferson County's Chapter 9 filing is the largest municipal bankruptcy in U.S. history.

This formal declaration doesn't necessarily mean investors will lose their money. Consider the 1994 case of Orange County, Calif., which was the largest municipal bankruptcy on record prior to Jefferson County, with debts totaling $1.7 billion. In the end, all debts were fully repaid.

Jefferson County's declaration provides us with a good opportunity to review the overall state of the municipal bond market. Last December, Meredith Whitney caused quite a stir when she predicted on CBS' "60 Minutes" "between 50 and 100 'significant' municipal bond defaults in 2011, totaling 'hundreds of billions' of dollars." Her forecast helped trigger investors to withdraw money from municipal bond funds for 24 consecutive weeks. Unfortunately for Whitney and those investors who gave credence to her forecasts, it would be hard for her to have been more off the mark.

Meredith Whitney on 60 Minutes

The massive scale of problems that Whitney anticipated hasn't appeared because governments have taken actions to address the problem -- cutting spending and raising revenues. Unlike the federal government, states are required to balance their budgets. As a result, budget gaps are being closed by layoffs of public employees, greatly reduced services, renegotiation of contracts with union members on wages and especially benefits, and increased taxes and fees. These actions have gotten results. For example:

-- Preliminary data from the Rockefeller Institute reported that state revenues from July and August were 6.8 percent higher than the same period in 2010.

--Standard and Poor's said municipal bond defaults were down 69.0 percent from January through October, versus the same period in 2010.

--Prior to Jefferson county's declaration, outstanding defaulted issues totaled just $6.6 billion.

It's also important to point out that of the now 200 outstanding defaulted issues, 8.0 percent were issued for multifamily residential projects, 10.5 percent for health care, and 43.7 percent for land-backed deals. These are sectors of the municipal bond market I recommend you don't even consider (regardless of the credit rating) because of their relatively poor historical default records.

While the year isn't quite over, the roughly $11 billion in defaults is a long way from hundreds of billions. Keep these results in mind the next time you're tempted to react to some dire forecast. And remember that the academic research on the ability to forecast the future demonstrates that it's not possible. The only good predictor is fame -- the more famous the forecaster, the more likely he or she is to be wrong.;lst;3

October Fest

By Jim Parker

Ever noticed how gamblers always tell you about their big wins, but tend to keep their even bigger losses close to their chests? People who seek to finesse their entry and exit of financial markets are similar.

Going awfully quiet in recent days have been the analysts who a month ago were saying that that was the time to get out of risk assets. It seemed a good call at the time as global stock markets had suffered their worst quarter in nearly three years.

Pummelling confidence were a host of concerns, including the European sovereign debt crisis, signs that global growth was stalling and a general lack of confidence in policymakers to take effective action to avoid another recession.

One chartist quoted by Dow Jones1 said the US market was breaking down in what could be a very nasty prelude to the fourth quarter. The advice from the technical analysts was that investors needed to be extremely wary buying stocks in October.

Adding to the nerves were the now routine reminders2 to investors about October supposedly being the "scariest" month for shares, with two of the biggest crashes in history occurring in the 10th month of the year–in 1929 and 1987.

Now while further volatility may well still lay ahead, those who took that advice and bailed out of risky assets at the end of September might now be ruing their decision.

The US S&P-500 rose by nearly 11 per cent in October, its largest monthly rise since 1991.3 That was the year that dance act 'C&C Music Factory' was topping the pop charts and 'The Silence of the Lambs' won Best Picture at the Academy Awards.

But it wasn't just a US story. The MSCI All-Country World Index rose by 10 per cent in October in US dollar terms, its largest one-month rally since April, 2009. In Australia, the S&P/ASX-200 gained 7.2 per cent in local currency terms, its best one-month performance since July, 2009. What's more, among the biggest gaining sectors in October were financials, energy and materials sectors, which had all lagged in the defensive mood of the prior months.

These are significant upward movements and will have eased some pain for investors after five-to-six months of consecutive decline in equity markets, but not if you had listened to the advice of some of the Jeremiahs in the financial media.

It's not often appreciated by ordinary investors that markets are forward looking. We know the news has been bad, but it's what comes next that counts. Selling out after a bad run in the markets just means you turn paper losses into real ones and leave yourself with the extremely difficult challenge of finessing your re-entry point. The reversal of direction in October highlights this difficulty.

We don't know if these October gains are sustainable — and already in November, sentiment around Europe has turned sour again. But we do know that markets can move quickly and respond to new information instantaneously. That's why market timing is so hard and why the best approach is to maintain your chosen asset allocation–with periodic rebalancing–irrespective of the week-to-week and month-to-month noise.

1.'MARKET TALK: Use Extreme Caution Buying Stocks' — Dow Jones Newswires, Sept 24, 2011

2.'Share Jitters Deny US Rise', Daily Telegraph, Sydney, Sept 26, 2011

3.'US Stocks Decline Amid Concern About European Funding', Bloomberg, Oct 31, 2011

Thursday, November 3, 2011

Increased Market Volatility: Fact or Fiction?

By Larry Swedroe

The financial media has been filled with headlines about market volatility. From one perspective, this may make sense, as the previous quarter saw 19 trading days experience market moves of more than 2 percent, compared with just one such day the previous quarter.

Based on my conversations with both investors and advisors alike, the perception is that markets have certainly become more volatile. The question is: “Does the historical evidence match up as well?” To find the answer, we go to our trusty videotape.

For the period 1927-1999, the annual standard deviation (measure of volatility) of the S&P 500 Index was 20.3 percent. From 2000 through 2010, the annual standard deviation was a virtually identical 20.5 percent.

Perhaps if we look at the data on a quarterly basis, we’ll find more volatility. The quarterly standard deviation of the S&P 500 prior to 2000 was 11.7 percent. Since then, it has been 9.0 percent — volatility prior to 2000 was 30 percent higher.

So let’s try looking at the data on a monthly basis. Prior to 2000, the monthly standard deviation was 5.7 percent. Since then it has been just 4.7 percent — volatility prior to 2000 was more than 20 percent higher.

Small-Cap Stocks
Since we can’t find the explanation in the large-cap stocks of the S&P 500, perhaps we can find it in small-cap stocks. From 1926 through 1999, the annual standard deviation of the CRSP 6-10 Index (which represents small-cap stocks) was 31.2 percent. For the period 2000 through 2010, the annual standard deviation was 28.7 percent. The annual volatility of small caps was 9 percent more prior to 2000 than it was after. On a quarterly basis, the standard deviation was 19.7 percent prior to 2000 and just 12.9 percent from 2000 through June 2011. Prior to 2000, the quarterly volatility was 53 percent higher. On monthly basis the standard deviation prior to 2000 was 8.1 percent. Since then it was 6.7 percent. Prior to 2000 the monthly volatility was more than 20 higher than it has been since.

International Markets
Perhaps if we look at the volatility of international markets, we’ll find the explanation for the perception that the equity markets have been more volatile. From 1970-99, the annual standard deviation of the MSCI EAFE Index was 20.6 percent. From 2000 through 2010, the annualized standard deviation has been a virtually identical 20.7 percent. So that probably isn’t the source of the perception either, though we note that the monthly and quarterly volatility of the MSCI EAFE was 8 percent and 17 percent higher, respectively, since 2000.

Emerging Markets
Stretching the case, we can also look at emerging markets. Here we do find a relatively small increase in volatility, not the much greater increase the media and the “experts” have portrayed. On annual basis, the standard deviation of returns was 36.7 percent from 1988 through 1999, and 38.3 percent from 2000 through 2010, an increase of just 4 percent. On quarterly basis (through September 2011), the standard deviation increased from 13.6 percent to 14.1 percent, an increase of 4 percent. On monthly basis, the standard deviation increased from 6.9 percent to 7.1 percent, an increase of just 3 percent. These small increases almost assuredly wouldn’t have been noticed by most investors, and thus can’t explain why there’s this perception of greatly increased volatility.

Since I don’t have access to the daily data, I can’t rule the possibility that the perception of higher volatility is caused by the markets being more volatile on a daily basis. But investing for the long run while paying attention to the market’s daily noise is (as Alan Abelson noted) like a man walking up a big hill with a yo-yo and keeping his eyes fixed on the yo-yo instead of the hill. The bottom line is that the media needs to create noise to get you to pay attention, even if paying attention is bad for both your stomach and your investment results.

Photo courtesy of Katrina.Tuliao on Flickr.