← Leaderboard · task_0da109232850041fbf96fdf8
S-EQ10-XX · Pullback in Uptrend — plus two more hidden conditionsMarket & SignalScreeningEquitySETHARDpublic example

Question

Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. Uptrend means EMA50 > EMA200; the pullback is 1 − close / (highest high of the last 5 sessions, today included), split-adjusted. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $10B, 20-day average dollar volume greater than $300M, sector equal to Consumer Discretionary, EMA50 above EMA200, pullback from the 5-session high between 5% and 12%, P/E (market cap / trailing-twelve-month net income from the filings) less than 40, turnover ratio greater than 0.002%. Scope: only entities with a known value for 20-day average dollar volume, sector, EMA50 above EMA200, pullback from the 5-session high, P/E (market cap / trailing-twelve-month net income from the filings), turnover ratio in the snapshot; entities lacking any of them are out of scope and must not be returned. Return every matching ticker; order does not matter. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.
Data as of 2026-09-10 · grader set_f1

Expected / gold answer

set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 531.8s · cost $0.0212
tool calls 8
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (8)
list_metrics → search_equities → get_market_history → get_equity_financials → run_calculation → run_calculation → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 99.8s · cost $0.1450
tool calls 11
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (11)
list_metrics → search_equities → get_market_history → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 44.3s · cost $0.9546
tool calls 8
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (8)
list_metrics → search_equities → run_calculation → get_market_history → run_calculation → get_equity_financials → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 240.5s · cost $0.0234
tool calls 12
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (12)
list_metrics → search_equities → get_market_history → run_calculation✗ → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 86.0s · cost $0.1504
tool calls 18
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (18)
list_metrics → search_equities → run_calculation → get_market_history → run_calculation → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation✗ → run_calculation✗ → run_calculation → run_calculation → get_equity_financials → run_calculation → get_equity_metrics → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 38.7s · cost $0.2371
tool calls 8
Model answer
set · 12 items
BKRBURLDGDLTREXPEFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 12: 12 correct → F1 100%. Strict pass.
gold 12 · returned 12 · correct 12 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (8)
list_metrics → search_equities → get_market_history → run_calculation → get_equity_financials → run_calculation → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
95.7not a strict pass
latency 201.4s · cost $0.0124
tool calls 7
Model answer
set · 11 items
BKRBURLDGDLTRFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 11: 11 correct, 1 missed → F1 96%. Not a strict pass.
gold 12 · returned 11 · correct 11 · false positives 0 · missed 1
precision 100.0% · recall 91.7% · F1 95.7% · not a strict pass
−EXPE
Tool calls (7)
list_metrics → search_equities → get_market_history → run_calculation → get_equity_financials → run_calculation → submit_answer
Minimax
MiniMax M3
95.7not a strict pass
latency 152.4s · cost $0.0298
tool calls 11
Model answer
set · 11 items
BKRBURLDGDLTRFIVEKMBKVUESHWTGTTPRURI
Why this score
Gold has 12, the model returned 11: 11 correct, 1 missed → F1 96%. Not a strict pass.
gold 12 · returned 11 · correct 11 · false positives 0 · missed 1
precision 100.0% · recall 91.7% · F1 95.7% · not a strict pass
−EXPE
Tool calls (11)
list_metrics → search_equities✗ → search_equities → search_equities → get_market_history → get_market_history → run_calculation → get_equity_financials → run_calculation → run_calculation → submit_answer