One solution: assume a one-day lag. A “sell” signal is issued at Monday’s close, which translates to assuming that security was sold at the following’s day’s close.
How much difference will such a seemingly minor change make in a strategy’s results? A lot. Indeed, many strategies that look wonderful in backtests turn into dogs after correcting for look-ahead bias.
Neutral signals. This is an especially subtle problem because it’s counterintuitive in some respects.
The problem is when there’s a gray area with one or more trading signals. For instance, let’s say you’re using two signals to determine if the current climate for an asset is bullish or bearish. A “buy” is when both signals are bullish; a “sell” is when both are bearish. If there’s a split decision—one is bullish, the other bearish—the signal is neutral, which is to say that the previous signal holds until both signals indicate a decisive change, one way or the other.
As an example, both signals issued a “buy” signal the first trading day of the month. Two weeks later one of the signals turns bearish but there’s no confirmation in the other signal, which continues to align with a bullish reading. The net result: we no longer have a “buy” signal, but there’s no “sell” signal either. In that case, the previous signal—a “buy”—remains in force until a “sell” signal arrives.
Obvious? Well, sure, once we spell it out and are aware of the subtlety. But designing this nuance into the code can trip up a rookie. The solution: generate a historical record of “buy” and “sell” signals and monitor the net result via a “position” signal. A standard system is to generate a “1” for “buy”, “0” for netural, and “-1” for “sell” in the “position” data. By contrast, a common mistake is to only calculate the “buy” signals and assume that the absence of a “buy” is the equivalent of “sell”. Not necessarily, but that won’t be obvious unless you compute a separate set of “sell” and “neutral” signals.
What’s the relevance? Results. A backtest that equates “neutral” with “buy” signals can and usually does dispense substantially different results vs. a test that recognizes the distinction. Ok, maybe you want to blur the lines for tactical reasons. That’s fine. The danger arises when the analyst doesn’t spot the difference in advance.
These are hardly the only pitfalls in backtesting, but they’re relatively common—and easily avoided. The question is whether these quantitative stumbles have skewed results in some of the more influential backtests that have found a wide audience in recent years? The answer: unclear until (if) we can reproduce the research. Unfortunately, most of the backtests that make the rounds these days don’t provide the accompanying code. That’s one more reason why it’s essential to crunch the numbers directly before making substantial monetary commitments to a given strategy.
As President Reagan famously advised, Trust but Verify. That’s a good policy for geopolitical negotiations and for backtesting investment strategies.