← Leaderboard · task_3671d8b16d62749a50822e9b
S-EQ16-XX · Relative Strength vs Index — plus two more hidden conditionsMarket & SignalDerived metricEquitySETHARDpublic example

Question

Relative strength is the ticker's split-adjusted return over the last N sessions minus the S&P 500 (SPX) return over the same sessions, as pre-computed in the snapshot. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $50B, 20-day average dollar volume greater than $300M, sector equal to Consumer Staples, return 60d vs spx greater than 10%, EMA50 above EMA200, turnover ratio greater than 0.002%. Scope: only entities with a known value for 20-day average dollar volume, sector, return 60d vs spx, EMA50 above EMA200, turnover ratio in the snapshot; entities lacking any of them are out of scope and must not be returned. Return every matching ticker; order does not matter. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.
Data as of 2026-09-10 · grader set_f1

Expected / gold answer

set · 1 items
CTVA

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 90.5s · cost $0.0039
tool calls 7
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (7)
list_metrics → search_equities → search_equities → search_equities → get_market_history → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 18.2s · cost $0.0239
tool calls 5
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (5)
list_metrics → search_equities → get_market_history → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 23.6s · cost $0.3816
tool calls 5
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (5)
list_metrics → search_equities → get_market_history → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 81.3s · cost $0.0060
tool calls 7
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (7)
list_metrics → search_equities → search_equities → get_market_history → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 36.9s · cost $0.0927
tool calls 10
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (10)
list_metrics → search_equities → search_equities → get_equity_metrics → get_market_history → run_calculation → get_market_history → run_calculation → search_equities → submit_answer
Z.ai
z-ai/glm-5.3-flash
100.0strict pass
latency 58.6s · cost $0.0043
tool calls 5
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (5)
list_metrics → search_equities → get_market_history → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 51.8s · cost $0.1474
tool calls 8
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (8)
list_metrics → search_equities → get_market_history → search_equities → search_equities → run_calculation → search_equities → submit_answer
Minimax
MiniMax M3
100.0strict pass
latency 17.0s · cost $0.0122
tool calls 7
Model answer
set · 1 items
CTVA
Why this score
Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
gold 1 · returned 1 · correct 1 · false positives 0 · missed 0
precision 100.0% · recall 100.0% · F1 100.0% · strict pass
Tool calls (7)
list_metrics → search_equities → search_equities → get_market_history → run_calculation → search_equities → submit_answer