Why an “80% Accurate” Signal Can Have Less Than 30% Precision
“This signal is 80% accurate” may appear on a backtest screen, yet following it can lose money. The problem can be what the 80% is a percentage of. Understanding that denominator changes how a signal is evaluated.
Two different meanings hidden in one word
Two commonly confused measures use different denominators.
Fraction of actual upward events detected.
80 of 100 rises produced a signal → 80%.
② Precision: Denominator = emitted signals
Fraction of signals followed by the defined event.
30 of 100 signals were followed by a rise → 30%.
What a signal follower needs
② describes event frequency after a signal.
Promotional summaries may instead quote ①.
Neither is a verified trade win rate until entries and exits are replayed.
High recall alone does not establish profitable trading. Emitting “long” every candle detects every rise, giving 100% recall, while precision remains the market's base event frequency. The relevant signal-level conditional measure is ②; execution outcomes require further validation.
Base rate: How common is the event beforehand?
The base rate is the event's frequency without conditioning on a signal. A low rate lets false positives dominate even a detector with apparently reasonable sensitivity.
Actual qualifying rises: 50, a 5% base rate.
Other observations: 950.
Recall 80%: 50 × 0.80 = 40 true positives.
False-positive rate 10%: 950 × 0.10 = 95 false positives.
Total signals: 40 + 95 = 135.
Actual qualifying rises among them: 40.
Precision = 40 ÷ 135 = 29.6%.
The “80%” detector produces the defined event
after only about three in ten signals.
Even a 10% false-positive rate generates 95 errors because there are 950 nonevent observations, overwhelming 40 true positives. Rare-event detection is especially vulnerable to this imbalance.
A different base rate changes the same detector's precision
Hold recall at 80% and the false-positive rate at 10%, changing only the base rate.
5% base rate: 40 true / 95 false → 29.6% precision.
10%: 80 / 90 → 47.1%.
20%: 160 / 80 → 66.7%.
30%: 240 / 70 → 77.4%.
50%: 400 / 50 → 88.9%.
Same stipulated detector rates; precision ranges from 29.6% to 88.9%.
A +5% rise is generally rarer than a +0.5% rise. Redefining the event can raise its base rate and apparent precision, though detector rates may also change. A smaller target also changes risk/reward, so one percentage cannot establish trading quality.
Precision alone does not determine profit
A 29.6% event precision is not sufficient to declare a strategy good or bad. The following payoff arithmetic additionally assumes that the event classifications correspond to completed target-or-stop trade outcomes.
Payoff 1:1: +1R or −1R
0.296 × 1 + 0.704 × (−1)
= −0.408R per trade.
Payoff 1:3: +3R or −1R
0.296 × 3 + 0.704 × (−1)
= +0.184R.
R is the planned loss amount.
With R = $50:
135 × 0.184 × $50 = +$1,242 before costs.
These are hypothetical payoff calculations, not measured trade performance.
The sign reverses under different payoff assumptions. See expectancy and R multiples and win rate for why the fraction alone is insufficient.
False-positive entries still incur fees
If each signal is traded, more false positives mean more unsuccessful entries, and fees apply regardless of outcome.
Each trade costs 5,000 × 0.001 = $5.
First hypothetical case: 135 trades, 29.6%
Fees: 135 × $5 = $675.
Assumed gross +$1,242 − $675 = +$567.
Second hypothetical case: 240 trades, 66.7%
Assumed expectancy: 0.667 × 3 − 0.333 = +1.668R.
240 × 1.668 × $50 = $20,016.
Fees: 240 × $5 = $1,200.
Assumed net: +$18,816.
The guide uses rounded probabilities and fixed payoff assumptions.
In the first case, fees consume 54% of assumed gross profit.
Low-precision signals can generate many entries against a thin hypothetical edge, letting fees consume much of it. Evaluate actual net outcomes after executing the defined trade rules, rather than relying on gross event statistics.
Does adding filters solve it?
Adding another AND condition can remove false positives, but also removes true signals. Improved precision is not automatic, and sample size shrinks.
1 condition: 135 signals, 40 true → 29.6%.
2 conditions: 52 signals, 24 true → 46.2%.
3 conditions: 18 signals, 11 true → 61.1%.
4 conditions: 6 signals, 5 true → 83.3%.
The trap: The final sample contains only six observations.
Five successes in six trials do not by themselves establish an advantage over chance.
The 83.3% figure is too uncertain to support the advertised conclusion by itself. See sample size and overfitting. More restrictive conditions leave fewer observations on which to judge the rule.
Four steps to measure your signal
Use records instead of trusting someone else's headline. Fix the definitions before examining outcomes.
Example: “A candle closes with RSI crossing upward through 30.”
② Fix the success definition first.
Example: “After entry, +3% target occurs before the −1% stop.”
③ Replay the entire period chronologically.
Do not select trades after seeing results.
④ Count all four cells using one consistent event definition:
Signal/event; signal/no event;
no signal/event; no signal/no event.
Precision = signal-and-event ÷ all signals.
Base rate = all event observations ÷ all observations.
Illustrative reporting format
“Precision 31.4%, 43 of 137 signals; base rate 5.2%;
net expectancy +0.12R only if confirmed by complete trade replay.”
Step ③ often fails when known bad outcomes are excluded because “the market was unusual.” That turns a systematic sample into selective memory. See lookahead bias and backtesting. Signal observations become trade evidence only after defined entry-to-exit replay with costs and execution constraints.
Why people overlook base rates
The difficulty often comes from memory filtering, rather than arithmetic.
A signal followed by a rise → strongly remembered.
A signal without a rise → explained away as “news that day.”
A rise without a signal → never counted.
Result: Perceived accuracy is built from one cell
and can exceed measured precision.
Count all four cells instead of relying on impressions. See confirmation bias and paper trading for examining signals without committing a live account.
Recap
② Precision describes the event fraction after a signal; trade win rate requires full replay.
③ Low base rates let false positives outnumber true positives.
④ The example combines 80% recall and 10% false positives into 29.6% precision.
⑤ Changing the base rate yields 29.6–88.9% under fixed detector assumptions.
⑥ Profit requires payoff and cost information as well.
⑦ Every executed false-positive entry still incurs fees.
⑧ More filters shrink the sample.
⑨ Count all four signal/event cells.
⑩ Report precision, base rate, sample size and separately validated net trade expectancy.
A signal's conditional event frequency depends on how common that event is in the market. On seeing an advertised percentage, ask what the denominator is and how often the event occurred without a signal. Without those counts, the headline does not establish the claimed quality.
NOONOO TRADING invites you to follow live trading in our free chat.
Start in the bot📈 OKX trading fee discount for new registrations
Register for the OKX Fee Discount →