← Leaderboard · task_2ec3a094db90787d349a0352
F04-R · Revenue Growth — six companies rankedFinancial AnalysisCalculationFinancial filingsRANKED_SETHARDpublic example

Question

For each of these six companies — MSFT, META, JPM, GOOGL, PANW, NVDA — answer the same question: "By what percentage did the company's full-year revenue grow from fiscal year 2023 to fiscal year 2025? Answer as a decimal ratio (e.g. 0.082 for 8.2%)." Then rank the six tickers by that value, highest first (all six have the required filings). Return the six tickers in order; order is graded. Filing-version policy: every financial value is taken from the LATEST filing accepted by the SEC at or before the data cutoff that reports the requested period. If a later filing (before the cutoff) restates an earlier period, the restated value is the answer; the original filing's value is not. A fiscal-year value is taken only from an annual report (10-K / 20-F / 40-F, including amendments) — a quarterly report never supplies an annual value.
Data as of 2026-09-10 · grader ranking_match

Expected / gold answer

ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 25.7s · cost $0.0016
tool calls 3
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (3)
get_equity_financials → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 11.6s · cost $0.0139
tool calls 5
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (5)
list_metrics → get_equity_financials → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 18.2s · cost $0.1386
tool calls 3
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (3)
list_metrics → get_equity_financials → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 41.2s · cost $0.0025
tool calls 5
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (5)
list_metrics → get_equity_financials → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 24.3s · cost $0.0325
tool calls 5
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (5)
get_equity_financials → run_calculation → run_calculation✗ → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
100.0strict pass
latency 113.5s · cost $0.0034
tool calls 3
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (3)
get_equity_financials → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 17.4s · cost $0.0420
tool calls 5
Model answer
ranked · 6 items
1. NVDA2. META3. PANW4. MSFT5. GOOGL6. JPM
Why this score
Order graded by pairwise over the top 6: 1.00.
method pairwise · k 6 · value 1.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order: NVDA › META › PANW › MSFT › GOOGL › JPM
Tool calls (5)
list_metrics → get_equity_financials → run_calculation → get_equity_financials → submit_answer
Minimax
MiniMax M3
0.0not a strict pass
budget exhausted → forced answer
latency 95.4s · cost $0.0847
tool calls 41
Model answer
ranked · 0 items
Why this score
Budget ran out and the answer was forced. Order graded by pairwise over the top 6: 0.00.
method pairwise · k 6 · value 0.000
gold order: NVDA › META › PANW › MSFT › GOOGL › JPM
got order:
Tool calls (41)
list_metrics → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_metrics → list_metrics → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_metrics → get_equity_financials → get_equity_financials → get_equity_financials → search_equities → get_equity_financials → get_equity_financials → get_equity_financials → search_equities → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → search_equities → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → submit_answer
Status
FeedbackTwitterTelegram