Challenge Threshold Methodology

Full methods and validation appendix for “Pricing an ABS Challenge.” CT-S1 was developed on games from March 25 through September 9, 2026 and evaluated on September 11 through September 27, 2026.

1. Reference definition
2. Periods and support
3. Temporal stability
4. Sampling uncertainty
5. Convention sensitivity
6. The matched comparison
7. Decision Opportunity Score
8. Representative situations
9. Figures
10. Interpretive limits

Reference definition

Challenge Threshold Standard 1 (CT-S1, version ct-s1.0) is a named expected-run reference. For each decision state it compares the immediate correction value V, the change in expected runs if an incorrect call is corrected, with C, the modeled future expected-run value of the challenge unit lost on a failed challenge. With passing as the zero baseline, the break-even threshold is C / (V + C).

The full decision equation can also contain a continuation term for differences outside the inventory effect. CT-S1 fixes that term at zero by holding the observed post-decision opportunity path constant. This is a retrospective valuation convention, not information available to a player.

The reference convention uses an expected-runs objective, a constrained partially pooled count-aware run-expectancy estimator, a fixed future-opportunity probability p = 0.60, future use of a challenge when p × correction value is at least 0.05 expected runs, and a reviewed extra-inning restoration rule. The future p is a standardization input used to price future inventory. It is not an estimate of any player’s belief.

Periods and support

The development period ran from March 25 through September 9, 2026. The confirmation period ran from September 11 through September 27, 2026, and was untouched while CT-S1 was built. September 10 belongs to neither period.

A row is one distinct decision state and challenge-inventory combination. The confirmation period produced 24,880 legal state/inventory rows. Of those, 22,584 (90.8%) had support in both periods under the frozen common-support hierarchy. Unsupported states were not filled in, imputed or described as validated.

Temporal stability

Across the 22,584 finite matched rows, weighted equally rather than by frequency, the median absolute change in CT-S1 from development to confirmation was 0.007628 (0.76 percentage points) and the 90th percentile was 0.026143 (2.61 percentage points). The rank correlation between periods was 0.995354. This is temporal stability for one frozen convention. It is not agreement among alternative conventions or proof that the convention is correct.

Sampling uncertainty

Sampling intervals come from 2,000 game-clustered bootstrap replicates with the CT-S1 definition held fixed. Whole games were resampled, and state-level continuation costs were recomputed for each replicate.

These are per-state intervals. No paired sampling interval was calculated for the difference between the two counts, and the intervals do not include uncertainty in the modeling conventions.

Convention sensitivity

The eight-convention grid changes one declared component of the reference at a time. The range across the grid is not a confidence interval, not a probability distribution, and not a claim that every defensible model lies inside it.

In each period, 22,428 supported rows had a complete eight-convention grid. The range was wider than 10 percentage points for 21,942 development rows and 21,929 confirmation rows, about 98% in each period. Another 156 supported rows per period lacked a complete grid; they are included in the temporal-drift population but are not reported as point estimates. Two diagnostic variants, public_tracking and selected_transport, sit outside the grid and do not set its endpoints.

The matched comparison

The 1-1 versus 1-2 example is the one article-eligible matched comparison. Three ordinary representative comparisons were selected mechanically from the development period before confirmation results were viewed. The frozen rule required complete finite cells, the same direction in both periods and under all eight conventions, and reference separation of at least 0.05 on the 0-to-1 scale in each period. The count comparison met the rule. An early-versus-late comparison and an inventory comparison did not, and they were not replaced.

The reference CT-S1 was 0.668030 at 1-1 and 0.595135 at 1-2 in development, and 0.672804 and 0.600331 in confirmation, a separation of about 7.2–7.3 percentage points. All eight conventions preserved the direction of the difference but not its magnitude.

Inside the reference calculation, both states share a modeled continuation cost C of 0.292 expected runs. The immediate correction value V is 0.142 at 1-1 and 0.194 at 1-2, and C / (V + C) reproduces both thresholds. This is the arithmetic of the named reference, not a general rule about two-strike counts.

Decision Opportunity Score

Decision Opportunity Score is a presentation transform: DOS = 100 × (1 − CT-S1), applied to the full-precision threshold before rounding. Higher values mean the immediate correction opportunity is larger relative to the modeled future value of the challenge at risk under CT-S1. DOS is not a probability, does not measure player confidence and is not a recommendation. It inherits every CT-S1 limitation, including convention sensitivity. It is not intended as in-game guidance, as a player grade, or as a score comparable across domains without a supporting model.

Across the 22,428 qualified confirmation rows (supported, with a complete eight-convention grid and a complete sampling interval), CT-S1 ranged from 1.4% to 94.1% with a median of 63.2%, or DOS from 98.6 to 5.9 with a median of 36.8. These rows are unweighted and do not reflect how often each situation occurs.

Representative situations

The ten-row table in the article uses a deterministic display rule. It was specified after confirmation results existed, so it is an illustrative selection, not a preregistered test or a statistically representative sample.

1. Start from the 22,428 qualified confirmation rows.
2. Keep rows matched at the exact-state level, so the displayed score state is part of the estimate (14,414 rows).
3. Keep rows whose 95% sampling interval is no wider than 10 percentage points (13,172 rows).
4. Include the two matched-comparison anchors.
5. Divide the pool into eight equal-count bands of CT-S1. Working from the lowest band to the highest, choose one row per band that adds the most game-state attributes not already shown: side, count, outs, bases empty or occupied, base state, inning stage, score, challenges left, home or away, and whether the call ends the plate appearance. Ties go to the row closest to the band median, then to a fixed hash of the state key. Rows differing only in challenges left from a chosen row are skipped.

Only the two anchors form a qualified matched comparison. Differences between other rows should not be read as the effect of a count, inning, runner state, score or inventory level. Counts are before the pitch, and score is from the challenging team’s view.

Figures

Figure 1 is conceptual. Figures 2 and 4 report the anchor thresholds, sampling intervals and convention values listed above. The Figure 3 scatter plots all 22,584 supported matched rows from the accepted matched-reference table on equal 0-to-100 axes with a no-change line.

Reproducibility record

Two isolated complete CT-S1 builds reproduced 25 generated files byte-for-byte with zero differences or exclusions, left 447 protected inputs unchanged, and passed 99 focused validation tests. The web-publication values (the representative table, the population summary, the anchor decomposition, the DOS conversions and the Figure 3 scatter) are generated by a separate deterministic build (CTS12) from the accepted outputs and validated against a content-addressed claim registry. Computational reproducibility does not establish that the chosen convention is uniquely correct.

Interpretive limits

CT-S1 tells us how sure a challenger would need to be under a named value model. It does not tell us how sure the challenger was. Public tracking cannot reproduce a hitter’s or catcher’s view, preparation or split-second confidence, and the decision-time evidence remains unidentified. A successful challenge can follow a weak decision and a failed challenge can follow a sound one.

CT-S1 values expected runs, not win probability or game leverage. The score bucket is part of the state key, not a leverage measure. The reference does not support a challenge playbook, an optimal policy, action-colored challenge bands, universal rules for a count or grades of individual player decisions.

Return to “Pricing an ABS Challenge” · Read the Article 4 methodology