Point-in-Time Data and Honest Backtesting

What this resource does

A backtest asks how a defined rule would have behaved using information available at each historical decision time. Current databases often contain revised values, surviving companies, and corrected identities that were not available then.

Point-in-time construction preserves the information set, universe, costs, delays, and version used for each decision. Results must include failures, inactive securities, and periods of poor performance.

Methodology

1. Define signal, universe, decision time, execution time, rebalance rule, and benchmark before testing.
2. Use first-available filings and releases with their dissemination timestamps.
3. Model delistings, suspensions, corporate actions, liquidity, fees, taxes, and slippage.
4. Separate development, validation, and untouched evaluation periods.

How to interpret it

A strong historical result can arise from a real relationship, data leakage, repeated experimentation, omitted costs, or chance. Robustness checks narrow those possibilities but do not prove future performance.

Report turnover, drawdown, dispersion, capacity, missing observations, and sensitivity—not only annualized return.

Limitations and failure modes

• Historical markets do not contain every future regime.
• Execution and liquidity models are approximations.
• Multiple testing can create false discoveries.
• A backtest is evidence about a method, not a guaranteed outcome.

Research workflow

1. Register the hypothesis.
2. Build point-in-time inputs.
3. Run controlled evaluation.
4. Publish complete results and replication details.

Questions and answers