Overfitting
Tuning rules until they describe one particular stretch of the past with almost no error.
How to read the number
The more parameters are fitted, the more exactly the past is described and the less of what is described remains general enough to hold on another stretch.
When the metric lies
From the inside it never looks like overfitting: each improvement seems reasonable. The only defence is a test on data that was not seen during the tuning.
Also known as: curve fitting