The Rankings Bug We Found, Measured, and Then Refused to Fix

We tested it, it did not survive the evidence, and the projections did not change. Here is the whole result, including what it does not prove.

A reasonable belief: if a projection system nudges players toward a fixed reference point, then any rule that decides who gets spared that nudge ought to use the same reference point. Ours does not. The regression step pulls every player toward a career-plate-appearance anchor of 160, meaning it drags players above 160 down and pushes players below 160 up. The rule that exempts certain young hitters from that pull, however, tests a threshold of 100. So a player sitting between 100 and 160 is being "spared" a correction that would have moved him up the board. That is backwards, and nobody on our side disputes it. On the live board it affects 10 rows out of 1,322.

The coherence argument is airtight. The evidence is not.

We wrote the test down before we ran it, including the minimum sample we would accept: at least 50 comparison pairs, since anything smaller cannot separate a real effect from noise. Over 2021 to 2025 the replay produced 1,705 player-season pairs, of which 60 cleared the pedigree, age and career-plate-appearance conditions, and only 8 landed in the band where a boundary of 100 and a boundary of 160 actually disagree. The average size of a projection miss, regardless of direction, came in at 87.04 with the boundary at 100 and 87.68 with it at 160.

Eight pairs. The pre-registered rule at n=8 is "ship nothing," and the current boundary was nominally the better of the two anyway.

Then we widened the window and the answer flipped

Stretching back to 2016 through 2025, with 2020 excluded, was not the test we registered, so treat it as exploratory. It gave 3,223 pairs, 140 clearing the gate conditions, and 17 in the disagreement band. Now the error runs 81.87 at a boundary of 100 against 79.73 at 160, with 6 of 9 seasons favoring 160. Doubling the window moved us from 8 pairs to 17 and reversed the conclusion outright.

That is the strongest case against shipping, and it is also the strongest case for the fix, depending on which table you trust. I read two underpowered tests pointing in opposite directions as the signature of noise, not of a real effect waiting for more data. If one more season of history can flip the sign, the sign is not information.

Why this particular rule may never be testable

The gate is structurally rare. Only 140 of 3,223 historical pairs, 4.3%, clear pedigree plus age plus career plate appearances, and only 17 sit where the two boundaries disagree. The live board shows the same scarcity at 10 rows of 1,322, so this is not a quirk of the years we chose. Any boundary change here is evidence-free by construction.

Two further limits surfaced from reading the logic rather than from the numbers. The gate drives two separate lifts, one tied to a 900 career-plate-appearance figure and one to 800, so moving the boundary moves both. And the second of those lifts is wired specifically to 2026, while the replay only ever feeds it seasons earlier than the one being projected. It can never fire in a historical test. That is a permanent blind spot, not a bug in the run.

One trap for anyone repeating this: the pedigree lookup keyed on player name returns today's grades with no season attached, which would apply 2026 pedigree to a 2021 decision. We used the year-aware lookup instead.

What we did, and what we did not prove

Nothing shipped. The rule still tests 100. We recorded the measurement, the two limits and the trap alongside it, with an instruction not to re-run this expecting a cleaner answer without a better instrument.

What the evidence does not establish: that 100 is correct, and it almost certainly is not, since the coherence argument stands unrebutted. That 160 is better, because the wider window says so at n=17 and the registered window says the opposite at n=8. And anything at all about the proven-debut lift, which the replay cannot reach.

Shipping on coherence alone was available. We declined it. A change that cannot be shown to help is not obviously worth 10 rows of movement.

All articles · Redraft rankings · Dynasty rankings