Research Notes

Top Coding Agent Benchmarks To Track

6 min read

Snapshot: 2026-06-12.

Purpose: track coding-agent benchmarks that still separate frontier models on real software engineering work.

Answer First

For coding, I would reorder the same five model families like this:

  1. GPT-5.5 / GPT-5.4 / Codex line
  2. Claude Fable 5 / Claude Opus line
  3. Gemini 3 / Gemini 3.1 line
  4. GLM / Kimi / Qwen / MiniMax Chinese frontier
  5. DeepSeek V3.2 / V4 line

Why this ordering:

  • GPT-5.5 leads DeepSWE and Terminal-Bench 2.1, and GPT-5.4 leads the standardized SWE-bench Pro public leaderboard.
  • Claude Opus remains the best evidence for code quality and mergeability; Opus 4.8 leads FrontierCode Diamond, and Opus 4.8/4.6 are strong across Terminal-Bench and SWE-bench Pro.
  • Fable 5 likely belongs near or at the top for coding, but several primary-source non-saturated coding benchmarks checked here do not yet have direct Fable 5 rows.
  • Gemini is strong on Terminal-Bench and SWE-bench Pro, but its DeepSWE score is weak in the checked public snapshot.
  • Chinese frontier models are now real contenders, but the best row varies by benchmark: Kimi on DeepSWE, GLM on Terminal-Bench 2.1, Qwen on SWE-bench Pro, DeepSeek on SWE-rebench.

Top Benchmarks

RankBenchmarkBest use-caseCurrent top signalSaturation read
1DeepSWELong-horizon repo engineeringgpt-5.5[xhigh]: 70% +/- 3Very useful differentiator
2FrontierCode DiamondMergeable production-quality codeClaude Opus 4.8: 13.4 scoreVery unsaturated
3SWE-bench ProHard real repo issue resolutiongpt-5.4 (xHigh): 59.10 +/- 3.56 on Scale publicUnsaturated, but scaffold-sensitive
4Terminal-Bench 2.1Terminal autonomy, debugging, buildsCodex CLI + GPT-5.5: 83.4% +/- 2.2Still useful, but moving fast
5SWE-rebenchRolling decontaminated SWE tasksClaude Code: 62.1% in the checked public viewUseful rolling signal

Do not use SWE-bench Verified as the main frontier signal anymore. Keep it as a compatibility/reference benchmark, because it is popular and easy to compare, but it is much closer to saturation.

Model Ranking

This is an opinionated ranking based on the checked benchmarks. It is not a clean average: each benchmark tests a different thing and often mixes model quality with agent scaffold quality.

Overall Coding Agent Ranking

  1. GPT-5.5 / GPT-5.4 / Codex line

    Best current evidence on hard long-horizon coding. GPT-5.5 leads DeepSWE and Terminal-Bench 2.1; GPT-5.4 leads the Scale SWE-bench Pro public leaderboard.

  2. Claude Fable 5 / Claude Opus line

    Best bet for maintainable code and agentic code review quality. Opus 4.8 leads FrontierCode Diamond, and Opus 4.8/4.6 are strong on Terminal-Bench and SWE-bench Pro. Fable 5 has strong reported coding scores, but direct primary rows are still missing from several of the hardest benchmark pages checked here.

  3. Gemini 3 / Gemini 3.1 line

    Strong terminal and SWE-bench Pro contender. The weakness is DeepSWE: Gemini 3.1 Pro is only 10% in the checked DeepSWE snapshot.

  4. GLM / Kimi / Qwen / MiniMax Chinese frontier

    Best broad Chinese frontier bucket for coding right now. Kimi K2.6 is the best Chinese-family DeepSWE row checked, GLM 5.1 appears on Terminal-Bench 2.1, Qwen3 Coder is strong on SWE-bench Pro, and GLM/Kimi/Qwen show up across rolling SWE-style evals.

  5. DeepSeek V3.2 / V4 line

    Worth tracking, but less convincing on the hardest checked coding-agent benchmarks. DeepSeek V3.2 is good on SWE-rebench, while DeepSeek V4 Pro is weak on DeepSWE and DeepSeek V3.2 is low on the Scale SWE-bench Pro public leaderboard.

By Job

Long-horizon repo engineering:

  1. GPT-5.5
  2. Claude Opus 4.8 / 4.7
  3. GPT-5.4
  4. Kimi K2.6
  5. GLM-5.1 / DeepSeek V4 Pro

Mergeable production code:

  1. Claude Opus 4.8
  2. GPT-5.5
  3. Gemini 3.1 Pro
  4. Kimi K2.6
  5. No confident DeepSeek/GLM row from the checked FrontierCode source

Terminal coding work:

  1. GPT-5.5
  2. Claude Opus 4.8
  3. Gemini 3 / Gemini 3.1
  4. GLM 5.1
  5. DeepSeek needs a current Terminal-Bench 2.1 row

Rolling GitHub issue repair:

  1. Claude Code / Claude Opus line
  2. GPT-5.2 / GPT-5 / Codex line
  3. Gemini 3 Pro Preview
  4. DeepSeek V3.2
  5. GLM / Kimi / Qwen

Score Cards

1. DeepSWE

Best use-case: long-horizon software engineering on original tasks.

Current top rows checked:

  • gpt-5.5[xhigh]: 70% +/- 3
  • claude-opus-4.8[max]: 58% +/- 2
  • gpt-5.4[xhigh]: 56% +/- 2
  • claude-opus-4.7[max]: 54% +/- 5
  • kimi-k2.6: 24% +/- 2
  • glm-5.1: 18% +/- 1
  • gemini-3.1-pro: 10% +/- 3
  • deepseek-v4-pro: 8% +/- 3

Why track it: it is the cleanest current public stress test for long-horizon coding agents. The tasks are original, contamination-resistant, and run under a consistent mini-SWE-agent harness.

2. FrontierCode Diamond

Best use-case: maintainable, mergeable production code quality.

Current top rows checked:

  • Claude Opus 4.8: 13.4 on Diamond
  • GPT-5.5: 6.3 on Diamond
  • Gemini 3.1 Pro: 4.7 on Diamond
  • Kimi K2.6: 3.8 on Diamond

Why track it: it asks whether code would actually be merged by maintainers, not just whether a hidden unit test passes. This is probably the best “quality, scope, style, tests, and maintainability” signal.

3. SWE-bench Pro

Best use-case: hard real-world repo issue resolution.

Current Scale public leaderboard rows checked:

  • gpt-5.4 (xHigh): 59.10 +/- 3.56
  • claude-opus-4-6 (thinking): 51.90 +/- 3.61
  • gemini-3.1-pro (thinking): 46.10 +/- 3.60
  • gemini-3-pro-preview: 43.30 +/- 3.60
  • gpt-5-2025-08-07 (High): 41.78 +/- 3.49
  • qwen3-coder-480b-a35b: 38.70 +/- 3.55
  • minimax-2.1: 36.81 +/- 3.55
  • deepseek-v3p2: 15.56 +/- 2.63
  • glm-4.6: 9.67 +/- 2.15

Why track it: it is a harder successor-style signal to SWE-bench Verified. Use the Scale public leaderboard for standardized comparison; keep vendor-reported Fable/Mythos rows separate because scaffolds and splits differ.

4. Terminal-Bench 2.1

Best use-case: coding through a terminal: builds, installs, debugging, CLI workflows.

Current official rows checked:

  • Codex CLI + GPT-5.5: 83.4% +/- 2.2
  • Claude Code + Claude Opus 4.8: 78.9% +/- 2.5
  • Terminus 2 + GPT-5.5: 78.2% +/- 2.4
  • Terminus 2 + Claude Opus 4.8: 74.6% +/- 2.4
  • Terminus 2 + Gemini 3 Pro: 74.4% +/- 2.6
  • Gemini CLI + Gemini 3.1 Pro: 70.7% +/- 2.9
  • Claude Code + GLM 5.1: 58.7% +/- 2.4

Why track it: it measures whether an agent can actually work in a shell, not just produce patches.

5. SWE-rebench

Best use-case: continuously refreshed and decontaminated SWE-style tasks.

Current public-view rows checked:

  • Claude Code: 62.1%
  • gpt-5.2-2025-12-11-medium: 61.3%
  • Claude Sonnet 4.5: 60.9%
  • Claude Opus 4.5: 60.4%
  • gpt-5-2025-08-07-medium: 58.7%
  • Gemini 3 Pro Preview: 56.6%
  • gpt-5-codex: 51.1%
  • DeepSeek-V3.2: 46.4%
  • GLM-4.6: 46.0%
  • Kimi K2 Instruct 0905: 41.7%
  • Qwen3-Coder-480B-A35B-Instruct: 39.1%

Why track it: it keeps moving, which makes it harder to overfit than a fixed leaderboard.

Watchlist

Use these as secondary signals:

  • CodeClash - goal-oriented software engineering; good idea, but the public leaderboard is older and does not yet cover the newest five model families cleanly.
  • SWE-bench Verified - still important for comparability, but too saturated for frontier ranking.
  • SWE-bench Multilingual - useful when non-Python/product-stack breadth matters.
  • SWE-Bench Mobile - useful for industrial mobile work, but not as broad a frontier model tracker.
  • SWE-WebDevBench - useful for app-builder/product-readiness claims.

Refresh Protocol

  • Keep model and scaffold together. Codex CLI + GPT-5.5 is not the same claim as Terminus 2 + GPT-5.5.
  • Keep benchmark splits separate. SWE-bench Pro public, private, vendor-reported, and Scale standardized rows are not interchangeable.
  • Prefer DeepSWE, FrontierCode Diamond, SWE-bench Pro, Terminal-Bench 2.1, and SWE-rebench over SWE-bench Verified for frontier coding-agent tracking.
  • Treat missing Fable 5 rows as missing rows, not as weak scores.
  • Re-rank the model families only after checking current primary leaderboards.

Sources