How to Read a Ranking Disagreement

A reader compared our dynasty starters to another public board and concluded we back proven production harder than the market. Under the full join, the conclusion flipped.

A reader sent over a careful comparison of our dynasty starting pitcher board against another public board. It listed thirteen starters we rank far above them, drew a reasonable conclusion from the pattern, and asked us to pressure test it. The conclusion was that our model believes more than the market does in established major league production becoming durable skill.

Every one of those thirteen observations was correct. The conclusion was still backwards, and the reason is worth more than the answer.

A "biggest disagreements" list is a tail by construction. If you read a philosophy off one, you will get the right answer only when the tail and the middle happen to agree. For a model that differs from the market in degree rather than in direction, they usually do not.

The full join says the opposite

Instead of comparing hand picked names, we joined our whole dynasty board against a published consensus ranking and looked at the typical gap rather than the extremes. As of the August 2 board, 259 players matched.

We are colder than consensus on starting pitchers. Not by a little: on the median starter we sit forty places below where the market has him, and we rank almost twice as many starters lower than higher.

The thirteen names in the original comparison are real and they reproduce against this feed too. They are simply the far end of a spread whose middle points the other way. The same board that has us aggressive on a handful of arms also has a former Cy Young winner at our 352nd dynasty asset while consensus keeps him at 133.

Truncation quietly helps the wrong answer

One detail worth stealing for your own work. We ran this two ways: once against the top 500 players our board renders, and once against the full board.

The rendered top 500 gives a milder answer. That is not noise, it is structural. Cutting at 500 removes the deepest disagreements, and those are disproportionately players we rank far lower than the market, because a player we have at 480 and the market has at 120 is exactly the kind of row a top 500 cut can drop. Truncating the data moved the result toward the comparison's original conclusion.

Any time you compare two ranked lists, check whether your cut is symmetric. Usually it is not.

What actually orders our starters

If the board is not rewarding proven workload, what is it doing? We checked, across every starter eligible arm on the board.

The dominant term is the rate a pitcher produces at when healthy. Innings matter, and matter clearly, but they are second.

The board hands us a natural experiment on that point. Three starters currently project for effectively the same workload, within one inning of each other, and we rank them nowhere near each other.

Same volume. Forty eight places between the top and the bottom. The entire spread is rate.

That is also the answer to a disagreement the original comparison flagged. Kirby is a command first starter the market likes considerably more than we do, and the write up explaining their side argued that command ages well. We are not arguing with that. We are pricing what he produces when he pitches, and on this board that number is less than half of Cease's at the same innings.

You can see the same thing at the very top from the other direction: our number one dynasty starter projects for 140 innings, while the arm we have seventh projects for 205. If this were a workload machine that ordering would be impossible. It is a quality machine that respects volume, which is a different thing.

Age is a kink, not a slope

The comparison also concluded that we punish age harder than the market. That is true at the old end and misleading as a general statement.

From 28 to 32 the penalty is about two and a half points a year. After 32 it roughly doubles. Across every starter in the sample, age correlates with dynasty value at only −.17, because most starters sit between 24 and 31 where that band is narrow. One 38 year old carrying a 0.57 multiplier is not evidence of a general anti veteran tilt. It is evidence of a curve that gets steep late.

Why we have not warmed the board up

Being colder than the market on pitchers is a choice we have already tested rather than a position we defend on principle. We built the levers that would warm our dynasty ranks toward consensus and measured them against realized forward production. Every one of them made the forward looking backtest worse, and the further we warmed, the worse it got. A related idea, a premium for young players on thin major league samples, was refuted the same way and is switched off.

So the coldness stays. Not because we like it, but because every attempt to remove it cost accuracy on the only test that matters.

The honest caveat

This measurement moves. Three days before this piece, the same join put the starter median at 62 places rather than 40, and showed starters running colder than everyone else. Today starters and non starters are almost identical.

The direction has been stable across every look: we are colder, and we rank far more starters below consensus than above it. The size of the gap is not fixed and should not be quoted as though it were. If you ever see that median hit zero or the split reverse, the claim in this article is dead, and that is the number to watch.

That is the same discipline the article is about. One snapshot supports a direction. It does not support a decimal place.

All articles · Redraft rankings · Dynasty rankings