LiveBench 2.0
Objective long-horizon tasks. We use Overall, Coding, Reasoning, Research and Data Analysis.
METHOD NOTE · v0.1 · 2026-10-20
Model Evidence Index v0.1
How 3qk7 turns direct third-party model evidence into its own transparent ranking—without copying another aggregate leaderboard.
Core rule: 3qk7 no longer imports another site's aggregate score or ranking. v0.1 reads raw metrics from independent evaluation sources and applies a public, reproducible 3qk7 Evidence Index. NewLab owns method/evidence audit; 3qk7 owns the reader-facing board.
Why aggregate independently?
Copying an existing aggregate creates no reason to use 3qk7. The product value is combining objective long-horizon task ability, an independent intelligence bundle, and real agent-work evidence while keeping price/API data outside the capability score.
Direct sources in v0.1
Artificial Analysis
Independent Intelligence Index. We use the highest comparable public reasoning-effort variant for each model family.
Agent Arena · Work
Real agent-session work signal. We use Net Improvement as a separate practical-work dimension.
Raw evidence snapshot
Machine-readable values, source dates, model canonicalization and formula inputs used by the current 3qk7 board.
Calculation
Each raw metric is converted to a midrank percentile within the current candidate cohort, then combined by board weights. The result is a relative evidence index for this snapshot, not an absolute percentage correct.
percentile(metric) = midrank(model) / (N - 1) × 100
board_score = Σ(weight × percentile)
missing metric = neutral 50th percentile, but coverage is reduced
ranking gate = weighted coverage ≥ 70%
Board weights
| Board | Inputs |
|---|---|
| Overall | LiveBench Overall 50% · Artificial Analysis 30% · Agent Arena Work 20% |
| Coding | LiveBench Coding 65% · Agent Arena Work 35% |
| Reasoning | LiveBench Reasoning 70% · Artificial Analysis 30% |
| Professional | Agent Arena Work 55% · LiveBench Research 25% · LiveBench Data Analysis 20% |
| Research | LiveBench Research 70% · Artificial Analysis 30% |
Missing data and model variants
Missing evidence is not scored as zero: it receives a neutral 50th-percentile placeholder and lowers coverage. Below 70% weighted coverage a model stays unranked. Price, API availability, context length, launch date and vendor claims never affect capability rank.
Arena Text preference is intentionally excluded from v0.1 until model-variant canonicalization is stable enough. It will be added as a separate preference signal rather than forced into the score.
Not a NewLab benchmark result
The raw measurements are from the direct third-party sources above. NewLab's role here is methodology, provenance, version mapping and reproducible aggregation. Any future NewLab hands-on model tests will be shown separately.