The Pitfalls of Back Testing

Active investment managers have a pretty dismal long-term record. Depending on which study you look at somewhere between 80% and 90% of managers underperform their benchmarks over the long-term net of fees.

But what if I told you there was a way to turn those results upside down? What if there was a world where managers beat their benchmark 90% of the time? What if there was a world where managers knew how to position their portfolio in advance for any market environment and were able to adjust to anything the market throws at them?

I want to be the first to tell you that this world does exist. This nirvana where investment managers beat their benchmarks, are able to limit losses or avoid bear markets entirely, and effortlessly switch between value and growth when the time is right is a reality.

This world is the world of back testing.

Now you may have sensed a hint (or a lot) of sarcasm in the things I said above, and that is for good reason. Back testing is one of the most misused tools in investment management. With the rise of computerized investing and index ETFs which pursue more active investment strategies, we are seeing more and more back tested results accompanying investment products. In fact, much of the evidence that support things like value and momentum, which are used in countless investment products, is based on historical testing of the factors that impact stock prices.

As an investor, however, it’s very important to understand what back tested results mean and what insights (if any) they can provide into the investment strategy they represent.

I don’t want to give the impression that all back testing is bad. Back testing is a very important tool that all of us who use quantitative strategies can utilize to help develop and optimize investment strategies. The problem is that much of the back testing that is presented publicly is flawed in one way or another. And even the best back testing can’t perfectly simulate what it is like to run an actual portfolio.