Out-of-sample test
Trying rules on a stretch of data that was not used while they were being tuned.
How to read the number
Splitting the data into a tuning part and a testing part turns a handsome equity curve into a statement that can be refuted.
When the metric lies
The testing stretch wears out with repeated use: if the rules were adjusted in response to its results, it long ago became part of the tuning and checks nothing any more.
Also known as: walk forward test