- After two losing days on live books, we backtested twelve designs for skipping bad trades: trend-slope gates, tick-persistence gates, regime switches, post-event deepening, time bans, and a naive-Bayes classifier. Every one reduced profit — from −3% to −96% of the best month — and none rescued the day that prompted them.
- The reason is measurable, not philosophical. At the moment of entry, trades that will win and trades that will lose are statistically identical on every feature we could compute: effect sizes of 0.00–0.15σ. The Bayesian likelihood ratio is ≈1, so there is nothing to update.
- Information about a trade appears in two places only: after it opens (exit rules) and at the scale of seasons (capital allocation). Deflated for the 225-asset screen it came from, the strategy itself sits at 54–89% probability of being real — and only unselected live months can raise that number.
Two automated books had a bad 48 hours. A stablecoin market-making fleet paid a “relocation tax” when its anchor price migrated; a thin-market mean-reversion strategy on three altcoins ate three full stop-losses in one token’s unlock-driven sell-off. Both losses were small and inside design limits. But both raised the question every strategy owner eventually asks: couldn’t we have seen those trades coming and skipped them? We spent 48 hours and roughly twenty backtests finding out. The answer was no — and the shape of the “no” is the useful part.
Why this matters: the entry filter is the most common “improvement” proposed for any live strategy, by owners and by reviewers. It is also the easiest to overfit, because the bad trades it targets are perfectly visible in hindsight. What follows is a reproducible way to find out — before coding a single filter — whether the information it needs exists at all.
1Set the test up so the filters can win
Three years of tick data, every real structural rule in the simulator, and two exam questions per filter.
Show the working
Filters are usually tested on the day that motivated them. That is how they pass. We tested each one on three periods: the event day that prompted it, the single most profitable month in the history (the period any filter is most likely to damage), and a quiet control month. The simulators carried the full live structure — conservative fill rules, single-position mutex, interest, circuit breakers, the stablecoin fleet’s volatility regime switch — because an earlier review on this site showed how a backtest missing one structural rule can turn a profit centre into a reported loss.
Two exam questions, in this order: what does the filter cost in the best month? and what does it save on the bad day? A filter that fails the first question never gets to argue the second.
2The graveyard
Twelve designs, six families, one pattern: every filter paid in the good month and collected nothing on the bad day.
Show the working
| Filter family | What it assumed | Cost in best month | Saved on the bad day |
|---|---|---|---|
| Trend-slope gate (4 variants) | Pause when the anchor is moving | −7% to −83% | 0 |
| Tick-persistence gate | Pause when price sits one side of the anchor | −83% | 0 |
| Post-event deeper entry (1–4h) | Demand more discount after a shock | −26% to −29% | negative |
| Post-event entry ban (4–12h) | Wait for the dust to settle | −88% | negative |
| Directional ban after a shock | Block only the side that fights the recovery | −28% | negative |
| Regime-dependent trigger (48h / 7d lookback) | Be stricter in a sell-off | −17% to −25% | 0 — detector fired too late |
| Naive-Bayes P(win) ≥ 0.70…0.85 | Let a classifier choose | −9% to −97% | none — win rate unchanged at 77–79% |
Two findings repeat down the table. First, the trades that hurt were opened before any detector could see the event: a position standing at the old price level when the level moves, or a first stop-loss in the opening minute of a sell-off. Every “after the shock” filter therefore arrives after the tax is paid; what it actually intercepts are the recovery trades that were busy repaying it. Second, the most profitable stretches of the best month — the slow, quiet staircases where a mean-reverting book earns most of its year — have the same surface signature as the slow, quiet bleeds that lose. A gate that blocks one blocks the other.
A late detector is fatal for defence (the damage is done at the event’s birth) and harmless for re-entry (you merely miss a few trades). The only gate in this fleet that survived review is built on the second half of that sentence: a sentinel that pauses on realised losses and resumes when a shadow book proves the market pays again. It judges populations of trades after the fact, never single trades before.
3Measure the information directly
Before coding a filter, compute the effect size. Here it was 0.00–0.15σ — noise.
Show the working
Any entry filter is a bet that something observable at the moment of entry differs between the trades that will win and the trades that will lose. That bet can be measured without building the filter: take every trade in the history, record the candidate features at its entry, split by outcome, and compute the standardised difference in means (Cohen’s d). For the altcoin strategy — 3,346 trades, 77% winners — the result:
| Feature at entry | Mean, future winners | Mean, future losers | Separation (d) |
|---|---|---|---|
| Drop speed, last 60 s | 149 bp | 137 bp | 0.15 |
| Drop speed, last 5 min | 256 bp | 255 bp | 0.01 |
| 1-hour return | −189 bp | −214 bp | 0.05 |
| 24-hour return | +251 bp | +120 bp | 0.09 |
| 30-minute range | 538 bp | 547 bp | 0.02 |
| Hour of day | 12.0 | 12.0 | 0.00 |
A separation of 0.3σ is the usual floor for “there is something here”; 0.8σ is what a genuinely selective filter needs. Nothing in the table reaches 0.15. The winners and losers are the same population photographed at the same moment. A walk-forward naive-Bayes classifier trained on these features confirmed it the expensive way: at every probability threshold it deleted trades without moving the win rate.
The stablecoin fleet told a subtler story. There, one feature — recent volatility — separated winners from losers at 0.4–0.8σ. But bucketing the quiet-season trades by that feature showed 144 of 148 sitting in the same bucket: the separation was between seasons, not between trades within a season. That information was already in use, in the fleet’s regime switch and in the capital allocated to each season. At the scale of the individual trade, the signal vanished again.
4Read it as Bayes would
A likelihood ratio of one leaves the prior where it was. The filters could only delete trades from a positive-expectancy stream.
Show the working
A filter earns its keep only if the features shift the odds: if P(features | win) differs from P(features | loss). An effect size near zero means those two densities overlap almost completely, the likelihood ratio is ≈1, and the posterior equals the prior — the strategy’s base win rate. Whatever the filter then does, it is removing trades at random from a stream with positive expectancy, and random removal from a positive stream has negative expected value by construction. The twelve results above are not twelve bad designs; they are one theorem observed twelve times.
Where, then, is the information? In the two places the books already use it. After entry: whether a dislocation retraces is revealed only once the trade is open, which is why every rule that survived review is an exit rule — a tracking take-profit, a hard stop, a time fuse, a loss-count breaker. At the scale of seasons: volatility regime does predict outcomes, and that is a capital-allocation decision made monthly, not an entry decision made per trade.
5Now deflate the strategy itself
Bonferroni passes trivially; the Deflated Sharpe Ratio says 54–89%, depending on one honesty parameter.
Show the working
If entry filters cannot improve the strategy, the next honest question is how much of the strategy’s own backtest is real. It was found by screening 225 assets, tick-testing roughly 30, and keeping three. Its per-trade Sharpe is 0.099 over 2,472 trades (t = 4.9), with a skew of −1.78 and kurtosis of 5.1 — the fat-tailed signature of many small wins and a few full stops. A Bonferroni-corrected significance test clears even 5,000 hypothetical trials; that is the large-sample artefact, not evidence. The Deflated Sharpe Ratio is the right tool, and its verdict moves with the trial count you admit:
| Trials admitted (N) | What N represents | Probability the Sharpe is real |
|---|---|---|
| 6 | Parameter variants on the final design | 99% |
| 30 | Assets tick-tested on P&L | 89% |
| 225 | Assets in the first screen | 54% |
| 1,000 | Everything anyone on the desk ever tried | 27% |
Two notes on the arithmetic. The negative skew is a genuine penalty, not a technicality: it widens the standard error of the Sharpe, which is the statistical form of the thirteen-month drawdown in this strategy’s history. And an independent route — an explicit Bayesian prior of 0.20 for a screened retail strategy, updated on the evidence chain — landed at a posterior of 0.53. Two methods, two sets of assumptions, one answer: a coin somewhere between 60/40 and 90/10.
The twelve entry filters belong in this accounting too. Had one of them “passed”, it would have been the best of twelve tries on the same data, and its own Deflated Sharpe would have started at N = 12. Filters are trials.
6The only way the number goes up
Selected evidence is discounted; unselected live months count at full weight. Capital waits for them.
Show the working
The 54% is low because the backtest was chosen after the fact from many candidates. A live month cannot be chosen after the fact. Each one is a trial with N = 1, admitted at full weight, and the strategy’s first live fortnight matched its model on win rate (67% vs 67%) and per-trade economics to the cent. Holding that alignment, the arithmetic crosses the 95% line after roughly three live months — the same “graduation date” logic applied to the fleet’s own record earlier this month. The operational rule that fell out: pilot capital until the deflated number clears 95%; no scaling on the strength of the backtest alone; a scheduled re-grading, not a feeling, decides when.
What changed in production: nothing was added. Twelve filters were declined with numbers attached. One existing pause mechanism was demoted to an alert after its own measured cost (−5 units over 330 days) was weighed against two false alarms. The review’s output was a table of effect sizes, a deflated confidence band, and a date.
The reusable checklist
Before building any entry filter — yours or a vendor’s:
- Compute the effect size first. Split historical trades by outcome, measure each candidate feature at entry, and report Cohen’s d. Below 0.3σ, stop; the filter has nothing to read.
- Test every filter on the best month before the bad day. A filter that cannot pay for itself in the harvest never earns the right to defend.
- Check the detector’s latency against the trade that hurt. If the loss opened before the signal could fire, the filter is protecting a corpse.
- Ask whether the information is trade-level or season-level. Season-level information belongs in capital allocation, not in the entry rule.
- Count the filters you tried. Each is a trial; the survivor’s significance must be deflated at that N.
- Deflate the strategy at the honest N of its own screen — and let unselected live months, not further backtests, raise the number.
Where this fits
This is the same discipline MOA applies to enterprise vendor claims under Independent Verification & Validation — extended to quantitative strategies and the improvements proposed for them. We review methodology and evidence; we don’t sell or manage the strategies under review, and nothing here is personalised financial advice. No stake in the answer, no incentive to grade generously.