← Leaderboard · task_7b56f342fc284f65984e96f5
S-EQ34-N · Multi-period Consistency — count of matchesMarket & SignalScreeningEquityNUMBERHARDpublic example

Question

Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. The condition must hold in EACH of the last three reported quarters (i.e. the minimum YoY growth across them exceeds the threshold). Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $2B, 20-day average dollar volume greater than $100M, minimum revenue growth (YoY) across the last 3 quarters greater than 15%. Scope: only entities with a known value for 20-day average dollar volume, minimum revenue growth (YoY) across the last 3 quarters in the snapshot; entities lacking any of them are out of scope and must not be returned. How many entities match ALL of the conditions above? Answer with the integer count (0 if none). Filing-version policy: every financial value is taken from the LATEST filing accepted by the SEC at or before the data cutoff that reports the requested period. If a later filing (before the cutoff) restates an earlier period, the restated value is the answer; the original filing's value is not. A fiscal-year value is taken only from an annual report (10-K / 20-F / 40-F, including amendments) — a quarterly report never supplies an annual value. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.
Data as of 2026-09-10 · grader numeric_tolerance

Expected / gold answer

178 (178 count)

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 165.7s · cost $0.3246
tool calls 18
Model answer
178 (178 count)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (18)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 447.1s · cost $2.9387
tool calls 15
Model answer
178 (178 shares)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (15)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 806.1s · cost $0.1082
tool calls 30
Model answer
178 (178 ratio)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (30)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials✗ → run_calculation → run_calculation → get_equity_financials → run_calculation✗ → run_calculation → get_equity_financials → run_calculation → get_equity_financials → run_calculation → get_equity_financials → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 214.4s · cost $0.4012
tool calls 23
Model answer
178 (178)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (23)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation✗ → run_calculation → run_calculation → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 148.4s · cost $1.0202
tool calls 18
Model answer
178 (178 shares)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (18)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → get_equity_financials → run_calculation → submit_answer
Minimax
MiniMax M3
100.0strict pass
latency 325.8s · cost $0.2987
tool calls 38
Model answer
178 (178 ratio)
Why this score
Expected 178, got 178 (0% off) — within tolerance.
expected 178 · got 178 · error 0.0% · within tolerance
Tool calls (38)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials → run_calculation → run_calculation✗ → run_calculation✗ → run_calculation → run_calculation → run_calculation → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
XiaomiMiMo
xiaomi/mimo-v2.6-flash
0.0not a strict pass
budget exhausted → forced answer
latency 1139.2s · cost $0.0579
tool calls 20
Model answer
180 (180 entities)
Why this score
Budget ran out and the answer was forced. Expected 178, got 180 (1% off) — outside tolerance.
expected 178 · got 180 · error 1.1% · outside tolerance
Tool calls (20)
list_metrics → search_equities → search_equities → search_equities → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
0.0not a strict pass
latency 1607.5s · cost $0.0550
tool calls 19
Model answer
177 (177 USD)
Why this score
Expected 178, got 177 (1% off) — outside tolerance.
expected 178 · got 177 · error 0.6% · outside tolerance
Tool calls (19)
list_metrics → search_equities → get_equity_financials → search_equities → search_equities → search_equities → search_equities → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → submit_answer