Accuracy policy
We will publish how often our projections are right, and how often they are wrong, using rules fixed in advance. This page sets out those rules. There is no live track record yet.
Current status: no live projections have been graded yet. The only results we have are a backtest of a simple baseline on the 2025 season, published on the methodology page and labelled as a backtest. Live grading is planned to start with the first regular-season game of 2027.
Principles
- Rules first, results second. The measures on this page are fixed before the season starts. If we change one, we say so here, with the date, and we do not rewrite past results.
- Locked snapshots. We grade what we actually showed, not what we would have shown. Our projections will be saved at fixed times before each game, about 24 hours and about 3 hours before the scheduled first pitch. Each snapshot will get a time stamp and a fingerprint (hash) when the site is built, so it cannot be edited quietly afterwards.
- Misses count as much as hits. We show the full distribution of results, not a curated selection.
- Compared with naive baselines. A hit rate means little on its own. We compare our numbers with two simple rules anyone could use: the team's lineup from its previous game, and its most common lineup over the previous seven games. These are deliberately different from the baseline on our methodology page: that baseline is the first version of our own method, while these two are the simplest rules a reader could apply without any model.
- Backtests are labelled. Results calculated after the fact on past seasons are always marked as backtests and kept separate from live results.
What we will measure
| Projection | Measure |
|---|---|
| Start probability (each player) | Brier score, and a calibration chart: of all the times we said 70%, how often did the player actually start? |
| Batting order | Share of projected starters placed in exactly the right slot, and the average number of slots we were off. |
| Full lineup | Average number of the nine starters we got right, and the share of lineups with all nine right. |
| Probable starting pitcher | Share of games where our most likely starter actually started. |
The answer key is the lineup that was actually used at the start of the game. Games that are postponed before they start are not graded for that date. If the published starting lineup changes before the game starts, the projection is graded against the lineup that actually started, and the change is recorded in the lineup changes log.
Where results will appear
This page will carry the running record: by day, by month and by team, next to the baselines. Each day's final lineups and our earlier projections will be kept in a dated archive so that anyone can check individual games.
Questions about a result
If you think we have graded something wrongly, contact us. Corrections are noted on the page with the date.