Show me a man who’s never been burned by relying on a backtest and I’ll show you a man who’s never relied on a backtest at all (either that or a fictional character). Skepticism toward backtested results is endemic among investment professionals, and rightly so. And yet… when a new concept comes along, backtested performance may be the only performance available. Refusing to examine backtested results in that circumstance is tantamount to refusing to consider any new investment idea.
We discuss this problem here: http://us.spindices.com/documents/research/research-the-limits-of-history.pdf, suggesting that skepticism should sometimes extend not only to backtested data, but to live history as well. Past performance is never a guarantee of future results, but sometimes it’s more useful than other times – whether its source is real or simulated.