Times-Through-the-Order Penalty Methodology

Methods and sources for "The Penalty Arrives When the Lineup Turns Over." It asks how much better hitters do the second, third and fourth time they face the same MLB starting pitcher in a game, whether that change behaves like a step at each new trip through the lineup or a gradual rise with pitch count, and whether it changed from season to season. The data window is 2023-2026 regular seasons, April 15 onward.

1. The question and the sample
2. Where each number comes from
3. How the penalty is estimated
4. Step or gradual
5. Seasons and agreement with earlier work
6. Units
7. Limits
8. Reproducing the results

The question and the sample

This study (internal ID MLB-05) was chartered on Oct. 24, 2026. Its analysis plan, the preregistration, was frozen at 2026-10-24T19:10:29Z, before any outcome in the data was read; three amendments, also written before any outcome was read, fixed a control target, the selection estimator and the primary step model. Every gate in the plan passed, one of them narrowly (section 4).

A start is a team's first pitcher in a regular-season game; openers are excluded, so the primary sample is starters who were expected to work deep. A trip (time through the order) for a plate appearance is one plus the number of completed plate appearances that batter has already had against this pitcher in the game. The sample is 374,503 classified plate appearances by primary starters in 16,909 starts, 2023-2026 regular seasons, April 15 onward. About 86% of these starts reached a third trip and about 5% a fourth (measured). Those shares describe what happened, not what managers intended or what they should have done.

Where each number comes from

Every number is produced by code in research/mlb05/ of the Pascal research repository and recorded in the run manifest output/run1/manifest.json (SHA-256 4b30f877…). A second, isolated run produced a byte-identical manifest, and on Oct. 29, 2026 a fresh rebuild from the stored data reproduced it again.

How the penalty is estimated

Run value. Each plate appearance is scored by its run value: the change in expected runs for the rest of the inning, given bases and outs before and after (RE24 linear weights), computed separately for each season's run environment. A positive number means the hitter did better.

Talent adjustment. Each plate appearance is compared with what that hitter and that pitcher would be expected to produce, using season-to-date matchup rates built only from games before that day and shrunk toward league averages (the method of "When to Pull the Starter"). This keeps a trip from looking better simply because it reached a stronger part of the lineup.

Which starters go deeper. Starters who do badly early are more likely to be removed, so the pitchers still in the game for a third trip are, on average, having better days. A plain comparison of trips therefore understates the penalty. Our primary estimator, C, corrects for this by estimating, from the data, how much of a pitcher's first-trip result carries over within the same start, and adjusting later trips for it. It assumes a pitcher's day quality is constant within a start and that selection depends linearly on the first trip; selection on second-trip results is not corrected, which pushes the estimate toward zero. We always report C beside two simpler estimators that lean in opposite directions: a within-start comparison (W), biased upward, and a pooled comparison (P), biased toward zero. For the second and third trips, C fell between them.

Intervals. 95% intervals come from 1,000 resamples of whole pitchers (seed 20261024). Talent rates and run weights are held fixed in the resamples, so the intervals are somewhat narrow. A fourth-trip estimate exists, but it rests on 1,644 plate appearances, its interval spans zero, and we make no claim about it.

Step or gradual

Trip number and pitch count rise together, so their effects are hard to separate. The preregistered approach holds pitch count fixed and asks whether hitters' results change at the moment a new trip begins (a step) or rise with pitch count within a trip (gradual). The primary model, F_C, is a pooled model anchored to estimator C; the preregistered model F uses start fixed effects. In both, and under every alternative pitch-count measure we tried, the change at the start of the second and third trips is positive, and it is larger than every "placebo" step placed at boundaries where no new trip begins. Neither model finds hitters doing better as pitch count rises within a trip. Holding pitch count fixed, hitters' gain shows up as a jump when the lineup turns over for the second and third time, not as a steady rise with the pitch count.

This split is fragile, and we say so. Its stability gate required the size of the third-trip step to agree within a set tolerance across model variants, and it passed by a very small margin. The size of each step depends on the model, so we report none. A velocity term changes sign between the two models, so we make no claim about velocity. "Step" describes where the gain appears, not why: the data cannot tell whether hitters learn the pitcher, the pitcher tires, or something else is at work.

Disagreement with prior work. Brill, Deshpande and Wyner ("A Bayesian analysis of the time through the order penalty in baseball," Journal of Quantitative Analysis in Sports, 2023) adjust for batter and pitcher quality, handedness and home field and find little evidence of a discontinuity between trips; in their analysis, performance declines gradually. Our result disagrees. Our identification differs (a selection-anchored pooled model and start fixed effects, tested against placebo boundaries), and with the narrow stability pass we present this as a disagreement, not a refutation.

Seasons and agreement with earlier work

The third-time-through penalty was similar in each season from 2023 to 2026 (Cochran's Q test of equal season estimates, p = 0.39), and we see no change in 2026, the first season of the ABS challenge system: 2026 minus 2023-2025 was -0.005 runs per plate appearance (95% interval -0.014 to +0.005). That does not show that the challenge system had no effect on pitchers. The 2023 pitch clock and shift limits cannot be tested here, because every season in the window has them.

With three more seasons and a correction for which starters get to go deeper, the 2026 estimate is unchanged from "When to Pull the Starter": about 0.013 runs per plate appearance, third time versus first. As a gate in the plan, this study's code also reproduced that article's 2026 figure within a set tolerance.

Units

The article reports runs per 100 plate appearances, which is runs per plate appearance times 100. To size the third-time-through figure, we also multiply it by the roughly 38 batters a starter faced per nine innings in these seasons, which gives about six-tenths of a run per nine innings (95% interval about 0.47 to 0.78). That scaling is an assumption: it treats every batter in a nine-inning stretch as if he were facing the pitcher a third time. It is a way to picture the size of the gain, not a forecast for any game or any pitcher.

Limits

• Observational. No claim is causal or about what hitters or pitchers think or feel. A "step" means only that results jump at a trip boundary with pitch count held fixed.
• Selection. Estimator C rests on the assumptions in section 3; the W and P bracket shows the range of simpler answers.
• Fragile split. The step-versus-gradual stability gate passed narrowly, and step sizes depend on the model.
• Scope. Starters only; openers, relievers and bullpen games are excluded. No individual pitcher is rated, and nothing here says when a manager should make a change.
• Window. Four seasons, all with the pitch clock and shift limits. Data before April 15 of each season are excluded because those games are used only to build the talent estimates; including them changed the estimates very little.

Reproducing the results

Preregistration SHA-256 fc6961ceff8073648fa2defd3fd72a65bc2e27b83844c5c51575c570a2e051df; run manifest SHA-256 4b30f8771a1c3c9bc46a5a8ef4103f1c57809c9b83d06d1582a8f7a07585141e. Data were acquired once (772 public requests to Baseball Savant and the MLB Stats API, about 1.8 GB) and stored unchanged. The build and analysis commands, environment and every gate result are in research/mlb05/RESULTS.md.

Return to "The Penalty Arrives When the Lineup Turns Over"