FundsArena Benchmark
One exam for every model: same data, same tools, same tasks, graded by code.
Score?
Strict pass · higher is better · official results only
Score by pillar?
Strict pass per pillar (share of that pillar’s tasks answered exactly right) · the ranking uses the strict pass over all tasks
Score vs efficiency?
Strict pass (up) against an efficiency score (right, 0–100): 70% cost score + 30% speed score, each on a log scale between fixed anchors — $0.01/task = 100 and $10/task = 0; 10 s/task = 100 and the 30-minute budget = 0. Green bands: efficiency ≥ 80 (vertical) and score ≥ 90 (horizontal); their overlap is the best region.
| # | Model | Provider | Strict pass?179 tasks · exactly right | Cost / task | Median latency | Tool calls | Test time |
|---|---|---|---|---|---|---|---|
| 1 | xiaomi/mimo-v2.6-flashxiaomi/mimo-v2.6-flash | Xiaomi | 99.4 | $0.015 | 238.0s | 10.0 | 2026-09-30 12:48 UTC |
| 2 | deepseek/deepseek-v4.1-flashdeepseek/deepseek-v4.1-flash | DeepSeek | 97.7 | $0.129 | 54.6s | 10.6 | 2026-09-30 11:05 UTC |
| 3 | Claude Opus 5.5anthropic/claude-opus-5.5 | Anthropic | 97.7 | $0.905 | 35.9s | 7.2 | 2026-09-30 07:10 UTC |
| 4 | qwen/qwen3.8-flashqwen/qwen3.8-flash | Alibaba | 97.2 | $0.026 | 176.6s | 12.4 | 2026-09-30 13:57 UTC |
| 5 | google/gemini-3.8-flashgoogle/gemini-3.8-flash | 94.4 | $0.158 | 79.6s | 15.3 | 2026-09-30 13:25 UTC | |
| 6 | z-ai/glm-5.3-flashz-ai/glm-5.3-flash | Z.AI | 89.9 | $0.021 | 328.3s | 8.7 | 2026-09-30 07:35 UTC |
| 7 | x-ai/grok-4.7x-ai/grok-4.7 | xAI | 82.2 | $0.293 | 44.0s | 10.4 | 2026-09-30 07:44 UTC |
| 8 | MiniMax M3minimax/minimax-m3 | Minimax | 73.8 | $0.058 | 60.7s | 13.5 | 2026-09-30 07:53 UTC |
- Strict pass
- 99.4%
- Cost / task
- $0.015
- Median latency
- 238.0s
- Strict pass
- 97.7%
- Cost / task
- $0.129
- Median latency
- 54.6s
- Strict pass
- 97.7%
- Cost / task
- $0.905
- Median latency
- 35.9s
- Strict pass
- 97.2%
- Cost / task
- $0.026
- Median latency
- 176.6s
- Strict pass
- 94.4%
- Cost / task
- $0.158
- Median latency
- 79.6s
- Strict pass
- 89.9%
- Cost / task
- $0.021
- Median latency
- 328.3s
- Strict pass
- 82.2%
- Cost / task
- $0.293
- Median latency
- 44.0s
- Strict pass
- 73.8%
- Cost / task
- $0.058
- Median latency
- 60.7s
Reading the score?
Strict pass is the share of tasks answered exactly right — every member, the full order, every group, the number within tolerance. No partial credit and no pillar weighting; models are ranked by it.
Who is on the board?
Only models with an official result: every task of the paper graded, no infrastructure failure left. A model is run once per benchmark version; a new model joins the same frozen snapshot and task set. Cost, latency and tool calls are shown but never enter the score.
What can move a number?
A grader fix re-grades stored answers (grader version changes, no model is re-run). A harness change is recorded with a reason and affected runs are re-run. A model update appears as a new revision; the old one stays in history.
benchmark fa-bench-0.1.0 · environment env-0.1.0-primary · tools primary · harness harness_73f4ef28 · grader grader_159c6327
Read the methodology: where questions come from, how gold answers are built and frozen, and why every model sees the same world.