Foul Fly Decision Threshold Methodology
Methods, validation and limits for "The Out You Shouldn't Take?" It asks whether Tampa Bay was better off catching or letting drop a foul fly with runners on second and third and one out in Game 5 of the 2026 World Series, and how the answer changes with the inning, the score, the outs and the count.
- The question and the case
- Where each number comes from
- The inning and win-probability model
- The tag-up model
- Break-even points and verdict labels
- Uncertainty
- The count problem and how we checked it
- Sensitivity
- Beyond Game 5: inning, score and outs
- Limits
- Reproducing the results
The question and the case
This study (internal ID MLB-06) was chartered on Oct. 31, 2026. Its analysis plan, the preregistration, was frozen at 2026-11-01T05:58:53Z, before any confirmatory result was computed. The outcome of the play was known before the plan was written, and the plan says so; nothing in the analysis uses it. The data window is the 2023 through 2026 regular seasons for league rates, transitions and the tag-up model, plus the official Game 5 game feed and the 2026 sprint-speed and arm-strength leaderboards for the case.
The case, from the official feed: top of the sixth, Milwaukee up 3-0, Contreras on third and Turang on second, one out, Pratt batting against Wells with a 1-0 count. Pratt's fly (90.9 mph, launch angle 53 degrees, 249 feet) was caught by Vilade in foul territory, about 7.9 degrees beyond the right-field line, which is about 34 feet from the line at the catch. The feed records a sacrifice fly with Contreras scoring and Turang taking third; it records nothing about the throw. A drop would have left the count at 1-1 with the runners in place.
Where each number comes from
Every modeled number is produced by code in research/mlb06/ of the Pascal research repository and recorded in the build manifest (SHA-256 35afbd5833cee3f17b4f3007c8424517f643cd6509a784ae901e4c832f62c797). An independent second build produced byte-identical outputs, and a third, rebuilt from backups on Nov. 4, matched again. The validation checks requested before publication are in research/mlb06/review/.
| Numbers in the article | Source | Script or output |
|---|---|---|
| The play (count, distance, distance from the line, result) | MLB Stats API Game 5 live feed | research/c21_c22/case_facts.py, CASE_FACTS.md |
| 6,114 caught fly balls; 80% scored, 18% held, 2% out at home | Baseball Savant Statcast, 2023 through 2026 regular seasons | research/mlb06/build_data.py, output/p3_tagup_summary.json |
| Win probability 9.0% and 9.2%, difference 0.17 points, interval -0.54 to +0.77 | Inning, win-probability and tag-up models | research/mlb06/build_model.py, output/case_values.json, output/bootstrap.json |
| Break-even about 77% (range 70% to 84%); estimate about 80% (72% to 88%) | Same | Same |
| Expected runs 1.15 and 1.32; expected-runs break-even about 98% | Same | output/case_values.json; accounting check in review/METHODS_REVIEW.md |
| Scoreless 16% and 35%; two or more runs 22% and 40% | Same | review/run_distributions.py |
| Walk-off range 40% to 65% | Walk-off check across count models | output/validation.json, review/results.json |
| Late-inning and three-run-lead values (3.6, 7.5; 3.5 to 6.3) | League-average grid | output/grid.json |
| League-average pitcher; 3.5% fair-ball flip point | Sensitivity runs | output/sensitivity.json, review/results.json |
| Figures 1-3 | The build manifest above | research/mlb06/build_figures.py, figures/manifest.json |
The inning and win-probability model
The inning. Each plate appearance ends in one of eight outcomes (strikeout, walk, single, double, triple, home run, ground-ball out, fly-ball out). How the bases, outs and runs change after each outcome comes from every plate appearance in the 2023 through 2026 regular seasons. A Markov model, the same one used in "What the Turang Walk Traded," then gives the full distribution of runs for the rest of the inning from any base-out state. Its results are anchored to the observed run distributions, so the model adds player information without drifting from what actually happens.
Players. Pratt, the Milwaukee hitters behind him and Wells enter through their 2026 outcome rates, pulled toward the league average (by the equivalent of 250 plate appearances for hitters and 400 for the pitcher) and combined with an odds-ratio matchup. Wells is assumed to finish the inning, with no pinch hitters.
The two branches. After a catch, there are two outs: the runner on third scores, holds or is thrown out at home with the probabilities from the tag-up model, and the runner on second advances when the lead runner scores at the league rate. After a drop, Pratt's plate appearance continues from a 1-1 count, using league outcome rates for plate appearances that reached 1-1, adjusted for Pratt and Wells. Both branches count every Milwaukee run from the moment of the decision, including the run scored on a catch. Expected runs is the average of that run distribution.
Win probability is the chance that Tampa Bay goes on to win, computed from the run distribution for the rest of the inning and the league distribution of runs per half-inning for the remaining innings, applied to both teams. A tie after nine is valued without the automatic runner, because the game was in the postseason.
The tag-up model
The chance that the runner scores, holds or is thrown out on a catch comes from a three-outcome logistic model fitted to all 6,114 caught fly balls with a runner on third and fewer than two outs, 2023 through 2026. Its predictors are distance (a smooth curve), the angle from center field, whether the catch was in foul territory, the outs and the runner's sprint speed. Arm strength and launch angle were tested as alternatives. For Game 5 it gives about 80% that Contreras scores (72% to 88%) and about 7% that he is thrown out.
The probability includes the third-base coach's decision. Coaches sent 82% of runners overall and 91% on plays like this one, and 97.7% of sent runners scored, but the fielder decides before the coach does, so the right input is the chance of scoring after a catch, holds included, not the chance of scoring when sent.
Validation. Fitted on 2023-2025 and tested on 2026 (4,581 and 1,524 plays), it beat distance alone (log loss 0.248 against 0.257) and was well calibrated overall (calibration slope 0.95). Distance does nearly all the work: removing it raises the 2026 log loss by 0.283, while spray angle, sprint speed and the foul indicator each add under 0.006. Near the Game 5 estimate the picture is mixed:
| Predicted chance of scoring | 2026 plays | Average predicted | Observed | 95% interval |
|---|---|---|---|---|
| 0.70 to 0.75 | 24 | 0.73 | 0.83 | 0.64 to 0.93 |
| 0.75 to 0.80 | 24 | 0.77 | 0.83 | 0.64 to 0.93 |
| 0.80 to 0.85 | 40 | 0.83 | 0.95 | 0.84 to 0.99 |
| 0.85 to 0.90 | 35 | 0.88 | 1.00 | 0.90 to 1.00 |
In 2026 the model underpredicted scoring in this band. Two checks closer to the case agree with it: among 2026 foul-territory catches it predicted 0.76 and 0.76 scored (45 plays), and among every foul-territory catch from 230 to 270 feet with one out in 2023-2026, 36 of 44 runners scored (0.82, 95% interval 0.68 to 0.90). The Game 5 decision changes if the true chance is about 3 points lower than the estimate, and no band has enough plays to calibrate that precisely. That is one reason the verdict is a toss-up.
Break-even points and verdict labels
For each objective, the break-even point is the chance of scoring at which catching and dropping are worth the same, holding the chance of an out at home in proportion. On win probability it is about 77%; on expected runs, about 98%. Expected catch runs rise in a straight line with the chance of scoring (0.382 + 0.956 times that chance), and setting that equal to the 1.32 runs after a drop gives the 98% point exactly.
The labels were set in the preregistration before any result: a difference under 0.5 points of win probability is a toss-up, 0.5 to 1.0 a lean, and 1.0 or more clear. A result that changes sign under any listed sensitivity is never described more strongly than a toss-up.
Uncertainty
Intervals are 95% percentile bootstrap confidence intervals from 500 resamples of whole games (seed 20261030). Each resample re-estimates everything the data supply: league outcome rates, base-out transitions, count tables, run anchors, the half-inning run distribution, the player rates, the tag-up model and the runner-advance rate. They do not include the shrinkage strengths, the tie value, the choice of count model, the fair-ball and catch assumptions, or factors outside the data; the win-probability calculation itself is exact.
The win-probability difference favored the drop in 72% of resamples (interval -0.54 to +0.77 points), so it is not statistically distinguishable from zero. The expected-runs gap favored the catch in 99.4% (0.04 to 0.32 runs), so that difference is supported. The win-probability break-even ranged from 0.70 to 0.84.
The count problem and how we checked it
After a drop the at-bat continues from a new count, so the model needs to know how plate appearances end from each count. A check written before the analysis compared each count's outcome rates with what the counts after it imply. It failed at 7 of 12 counts, because hitters and pitchers who reach a count by different routes end their plate appearances differently. The errors are small (at most 0.85 points, in the walk rate at 3-1) and systematic: walks are underpredicted at 2-0, 2-1 and 3-1 and overpredicted with two strikes. At 1-1, the count a Game 5 drop would reach, the largest error is 0.11 points. Following a dated amendment, every number that uses the count table is labeled post-hoc, and no claim is made at the failing counts.
Before publication we compared three versions of the drop branch: the pooled count table above; a history-aware table using only plate appearances that went from 1-0 to 1-1, as Pratt's would have; and a count-free version that treats the drop as a fresh plate appearance. Fitted on 2023-2025 and tested on 2026, the history-aware table predicted outcomes only slightly better than the pooled one, while the count-free version was clearly worse at every path tested. The pooled table is the simplest adequate model and remains primary.
| Pooled (primary) | History-aware | Count-free | |
|---|---|---|---|
| Game 5, drop minus catch | +0.17 points | +0.22 | -0.16 |
| Win-probability break-even | 0.77 | 0.76 | 0.83 |
| Tied, 7th / 8th / 9th | +1.5 / +3.6 / +7.5 | +1.7 / +3.8 / +7.7 | +0.2 / +2.0 / +5.5 |
| Up three, 6th to 9th | -3.5 to -6.3 | -3.4 to -6.2 | -4.1 to -6.5 |
| Walk-off break-even | 0.42 to 0.56 | 0.39 to 0.64 | 0.51 to 0.53 |
Game 5 stays a toss-up under all three, the three-run lead and the 8th and 9th innings hold under all three, and the 7th inning does not, which is why the article says the drop is only favored there under our primary model.
Sensitivity
| Change | Game 5 difference (points) | Break-even |
|---|---|---|
| None (primary) | +0.17 | 0.77 |
| League-average pitcher instead of Wells | -0.02 | 0.81 |
| League-average hitters | +0.34 | 0.74 |
| Shrinkage halved or doubled | +0.19, +0.16 | 0.77, 0.77 |
| Tie value 0.45 or 0.58 | +0.17, +0.17 | 0.77, 0.77 |
| Tag-up model with arm strength; with launch angle | +0.13, +0.21 | 0.78, 0.77 |
| 2026-only rates and transitions | +0.16 | 0.78 |
| Attempted catch made 95% of the time | +0.16 | 0.77 |
| Chance a dropped ball lands fair: 1%, 2%, 5%, 10%, 20% | +0.12, +0.07, -0.07, -0.31, -0.80 | 0.78, 0.79, 0.82, 0.86, 0.95 |
The fair-ball chance is an illustrative assumption, not an estimate: uncaught foul flies have no landing point in the data. The result flips at about 3.5%. The Game 5 ball was caught about 34 feet into foul territory. The fair-ball branch also assumes both runners score and the batter reaches first.
How the runs are spread. After a catch, Milwaukee is held scoreless in the rest of the inning about 16% of the time, scores exactly one run 62% and two or more 22%. After a drop the figures are 35%, 25% and 40%. Expected runs averages these outcomes; win probability weighs each by its effect on Tampa Bay's chance of coming back from three down.
Beyond Game 5: inning, score and outs
The grid uses league-average players, the home team in the field, a 1-0 count and a 250-foot foul fly, for which the tag-up model gives about 78% with nobody out and 85% with one out. The table shows the break-even chance of scoring on win probability with runners on second and third and one out: letting the ball drop is favored when the runner's chance of scoring on a catch is above the number shown.
| Inning | Down 3 | Down 2 | Down 1 | Tied | Up 1 | Up 2 or 3 |
|---|---|---|---|---|---|---|
| 6th | 0.78 | 0.79 | 0.81 | 0.82 | 0.94 | Catch |
| 7th | 0.76 | 0.77 | 0.78 | 0.78 | 0.92 | Catch |
| 8th | 0.74 | 0.74 | 0.75 | 0.72 | 0.90 | Catch |
| 9th | 0.71 | 0.72 | 0.72 | 0.65 | 0.87 | Catch |
"Catch" means catching is better at any chance of scoring. With nobody out the thresholds are higher (0.90 to 0.98 in the 6th and 0.68 to 0.79 in the 9th, tied or trailing), so the catch is usually right. With two outs a catch ends the inning and no run can score on it, so there is no trade to make. The count moves the threshold only modestly at the counts where the count table passes (0.67 to 0.72 in a tied 8th). These thresholds come from the primary model; under the count-free model they are a few points higher.
Limits
- Modeled, not observed. No one dropped this ball, so every comparison is between modeled outcomes. A deliberately dropped foul looks like any other foul in the data, so we cannot count how often teams choose to drop one.
- Post-hoc count tables. Every number that uses the count table is labeled post-hoc (above).
- Missing inputs. Fielder momentum, hang time, the runner's jump and the throw are not in the data; a few points of error in the chance of scoring would change the Game 5 direction.
- Assumptions. Wells finishes the inning, no pinch hitters, regular-season rates applied to a postseason game, and an illustrative fair-ball chance.
- Scope. Major League Baseball only. No claim about intent, and no claim that the catch or the drop was the wrong play.
Reproducing the results
Preregistration SHA-256 f11abb299da123c9007433a1bfdae588c60e99fb485af40629583bf9bba563c1; build manifest SHA-256 35afbd5833cee3f17b4f3007c8424517f643cd6509a784ae901e4c832f62c797. The steps are in research/mlb06/README.md: build_data.py, build_model.py and build_figures.py, then review/methods_review.py and review/run_distributions.py for the pre-publication checks. Raw Statcast and MLB Stats API data are MLB Advanced Media content used under research terms and are not redistributed.