What Doesn't Work

Six signals that sound like edges, tested on seasons the model had never seen, and failed. Publishing the nulls is the only way the wins mean anything.

Almost every public analytics piece reports something that worked. That is a selection effect, and it quietly makes every model look smarter than it is. So here are the signals we tested, believed in, and threw away. Each one was a real hypothesis with a real mechanism, and each one failed a test we wrote down before we ran it, on seasons the model had never seen.

Our rule: a ranking input has to earn its way in on held out seasons, not on a plausible story or a single before and after screenshot. Six candidates below did not earn in. They are not in the model.

Pitch shape tells you almost nothing once you know the role

This is the one we most expected to work. Velocity, movement, spin axis, release point: the whole Statcast arsenal, folded into a pitcher's fantasy projection. The theory is obvious. Better stuff should mean better results.

Tested with the pitcher's role held constant, using leave one fold out validation, it helped in 0 of 4 folds. The confidence interval was tight enough to exclude even a half percent gain in our average miss. Not "we could not detect it." Closer to "if an effect is there, it is smaller than half a percent."

The reason is that role already carries the information. Managers do not hand high leverage innings to pitchers with bad stuff. By the time you know a pitcher is an ace, a mid rotation starter or a middle reliever, the shape data has told you what it has to tell. We ran the same family five more ways, including within season and per pitch versions. All rejected.

Spin rate predicts strikeouts, not fantasy points

Spin is real. It is one of the better understood physical inputs in baseball, and it genuinely predicts whiffs.

It still failed as a ranking input, and the distinction is the entire point. Fantasy scoring is an accumulation of innings, runs, wins and strikeouts. A spin driven strikeout edge that comes with fewer innings is close to a wash in points formats. Spin is rejected as a standalone ranking input, not as a baseball fact, and one narrow subgroup effect did survive. We kept the finding at the size the evidence actually supports.

Height, weight and body type: nothing, to five decimal places

We joined body measurements to dynasty value on a fully ID keyed match, so this was not a name matching failure or a coverage problem. It changed how much of dynasty value the model can explain by −0.00000.

Bloodline for hitters, sprint speed and raw exit velocity were rejected in the same study. That last one surprises people most, so it is worth being precise: exit velocity is not useless, it is already inside the projection through the median outcomes it produces. Adding it again as a separate term is double counting, and double counting shows up as noise.

Breakouts are detectable, but not early

We built five independent attempts at spotting a breakout before it appeared in results, using different data every time, including a Statcast arm and a pitcher specific arm.

The cleanest of them was negative in 8 of 8 seasons at both checkpoints. That is not a noisy null that a bigger sample would rescue. What our detectors actually do is identify breakouts as they happen, which feels like foresight and is not. There is a real gap between a model that recognizes a change quickly and one that sees it coming, and we could not cross it.

Youth is already priced

Dynasty formats reward young players, so a thin sample youth premium sounds like free value: a 23 year old with 400 major league plate appearances should carry more forward value than his production alone suggests.

Tested on seasons the model had never seen, the premium did not get earned more reliably by the players it was supposed to help. It is refuted and gated off. The related idea, that we should warm our dynasty ranks toward market consensus, failed the same way: every lever that moved us toward the market made the forward looking backtest worse, and the further we moved, the worse it got.

Editorial prospect ranks beat scouting grades

Blending scouting future value grades into prospect rankings failed on classes the model had never seen. The editorial top 100 rank alone out predicted the grades in every class we tested.

Why we publish these

Two reasons, and neither is modesty.

The first is that a null result is a real finding. Knowing that pitch shape adds nothing once role is known is worth as much as any positive result, because it tells you where not to spend your attention.

The second is calibration. A model that only reports its hits cannot be evaluated by anyone, including the people building it. We keep an audit record of every one of these, including the ones where our own first answer was wrong and we had to withdraw it. If we tell you a signal works, this page is the reason that claim should carry weight.

All articles · Redraft rankings · Dynasty rankings