The staff of the Financial Industry Regulatory Authority (FINRA) recently issued an opinion letter discussing the use of “pre-inception index performance (PIP)” data in communications about exchange-traded financial instruments. Importantly, the letter permits the use of PIP data (i.e., backtested or simulated results) in presentations to institutions (although not to retail investors).
No good deed goes unpunished, and FINRA’s decision was not greeted with unanimous praise. Some critics have cautioned that backtested data should be taken with “more than a few grains of salt.” Although we’re quite pleased with FINRA’s action, we appreciate the note of skepticism. Backtests are useful analytic tools – and they should definitely be taken with a grain of salt.
We’ve previously discussed this issue at some length in a conceptual context. Another way to assess backtest results, of course, is simply to compare (where data permit) simulated with live results for the same index. As a simple example, consider the S&P 500 Equal Weight Index, which launched in January 2003. We have 12 years of simulated data (starting 1991), and, as of this writing, just over 10 years of live data. Comparing monthly return data for the backtest (PIP) period with the live period shows some remarkable similarities:
If anything, performance improved during the live period, in both absolute and risk-adjusted terms.
Of course not all backtests are alike, and of course equal-weighting is a much less complex portfolio construction rule than many others in current use. We don’t believe that backtests are always fair and unbiased predictors of future results. But sometimes they are. Judicious use of PIP can add to the index user’s insight.