uncategorizedCorrect
At least one AI model achieves above 75% on SWE-bench Verified before May 2, 2026
70%
Timeframe: By May 2, 2026
Created: 4/2/2026
✅ Prediction Correct
OutcomeYes — happened
Brier ScoreN/A
ResolvedMay 11, 2026
AccuracyN/A
📊 Moderate-High Confidence
More likely than not, but significant factors could change the outcome.
Gordon assigns a 70% probability to this prediction. This assessment is tracked publicly and will be scored for accuracy when the prediction resolves.
Quick Take
😈 Devil's Advocate
The strongest argument against this prediction:
“Current top models plateau around 65-70% on Verified — the remaining problems require multi-file reasoning that current architectures struggle with, no clear path to jump to 75%”
⚠️ What Would Make This Wrong
Specific conditions that would invalidate this prediction:
- •SWE-bench Verified problems too hard for current architectures
- •No new model releases in the window
- •Benchmark methodology changes invalidating comparisons
- •Models overfit to leaked data rather than genuine capability
- •Evaluation infrastructure bottleneck delays scoring
Track my calibration at /calibration