Performance measurement

By Jaffe Greenwald, CERF/CCFin Research Associate, Cambridge Judge Business School, University of Cambridge

A fundamental part of scoring someone’s performance is the benchmark. If on a test I received 65 points, I immediately have the following questions:

• what was the maximum possible points?
• how did my friends score?
• what was the average score?   

In these questions, I am searching for a benchmark to give context to my score. The number 65 on its own does not necessarily contain much meaning. Questions (2) and (3) are commonly used to form benchmarks both in school and for mutual funds. Question (3) is analogous to comparing a mutual fund’s return to the return on a broad market index – the average performance of all investors. Question (2) is analogous to comparing a mutual fund’s return to a sector or style specific index, which its peers would also use. While this is a step in the right direction, it is not the end of the story.

Let’s say that I figure out that the average is 60. I exceeded the average score by 5 points. That’s good, right? But how good is it? To address this question in standardised tests, we assign percentiles to the students. The percentile reflects how the student ranked among the population of students in the nation. Let’s say that my score turned out to be the 85th percentile. That means that my score exceeded 85% of student that took the exam. That seems a lot more informative of my effort than 5 points above the mean.

However, while we use percentiles for standardised tests, we still use points above the benchmark for mutual fund performance. Specifically, we measure the mutual fund’s alpha – the number of percentage points of return that the fund earned above its benchmark.

Challenges with measuring mutual fund performance

The challenge is in identifying a suitable population of candidates to compare against the mutual fund. For students in a test, the objective of all the students is the same – to score as high as possible. Universities are unaware of how risky the student’s study strategy was and only see one outcome of their study strategy. However, for mutual funds, their performance is observed at a daily basis. This makes the riskiness of their investment strategy much more apparent to investors. Since the investors can see the riskiness, they can give weight to it. As a result, each mutual fund has unique mix of objectives. In addition to the mutual fund’s overall return, they must also consider their strategy’s risk – the nature of their portfolio’s exposures.

The problem is complex because each mutual fund is concerned with different risks to different degrees. This makes it difficult to find even one benchmark for a mutual fund that we are confident has a similar risk profile. To alleviate this problem, the tradition is to use statistics to find the best blend of several indices that most closely represents the risk of the mutual fund. We call these blends factor models. Even then, I only have one benchmark. I would like a population of benchmarks so that I can compute a percentile. 

My proposed solution

Instead of trying to match the risk of a mutual fund by blending together a fixed set of stock indices, I blend together a random set of stocks. Specifically, I pull a random set of stocks and blend them so that the risk of my stock portfolio matches the risk of the mutual fund. Since the chosen stocks were random, I can draw another random set and make the appropriate blend. Since there are many stocks, I can make many portfolios before I have any repeat portfolios. Each of these random portfolios is a benchmark for the mutual fund. Using my randomly generated population of benchmarks, I can see how the mutual fund ranked within this population.

Findings

Prior to my study most return based performance measures had negative findings. These studies predominantly used a factor model to generate a single benchmark. For example, Carhart (1997) finds performance persistence only in the funds that underperformed. Funds that did poorly with respect to their single benchmark, continued to do poorly. The evidence using these single benchmarks was so robust and puzzling that Berk and Green (2004) developed a theory to explain it. However, using my population of benchmarks, I can match risks more wholistically and measure performance more precisely.  

I find that one of every 4 equity mutual funds significantly outperforms 50% of their random risk-matched portfolios. These mutual funds have skill in that their choice of stocks was better than random. In addition, the mutual funds that got into the top bracket of performance continued to outperform the mutual funds in the bottom bracket of performance. Mutual funds that performed tend to continue to perform well.