← Leaderboard · task_2aba5115565292418a235dd8
S-CR19-N · Perp Volatility Expansion (ATR% threshold) — count of matchesMarket & SignalDerived metricCrypto / PerpNUMBERHARDpublic example

Question

Perpetual technicals use COMPLETED UTC daily bars only (Hyperliquid 1d candles). "The session" below means the last closed UTC session — its date is the row attribute last_closed_bar — never the bar still forming at snapshot time. ATR% = 14-day Wilder ATR divided by the session's close. Open interest (USD) is taken at snapshot time, not at the session close. Find Hyperliquid perpetual markets with open interest greater than $20M, perp atr pct 14 greater than 5%. Scope: only entities with a known value for perp atr pct 14 in the snapshot; entities lacking any of them are out of scope and must not be returned. How many entities match ALL of the conditions above? Answer with the integer count (0 if none). Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.
Data as of 2026-09-10 · grader numeric_tolerance

Expected / gold answer

30 (30 count)

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 476.5s · cost $0.0128
tool calls 11
Model answer
30 (30 ratio)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (11)
list_metrics → search_perps → get_perp_history → run_calculation → run_calculation → search_perps → search_perps → get_perp_history → get_perp_history → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 22.0s · cost $0.0264
tool calls 6
Model answer
30 (30 count)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (6)
list_metrics → search_perps → get_perp_history → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 20.7s · cost $0.3157
tool calls 5
Model answer
30 (30 count)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (5)
list_metrics → search_perps → get_perp_history → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 68.9s · cost $0.0057
tool calls 6
Model answer
30 (30 ratio)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (6)
list_metrics → search_perps → get_perp_history → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 195.8s · cost $0.1473
tool calls 18
Model answer
30 (30 ratio)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (18)
list_metrics → search_perps → run_calculation → get_perp_history → run_calculation → list_metrics → run_calculation → run_calculation → run_calculation → get_perp → get_perp_history → run_calculation → run_calculation → get_equity_metrics → run_calculation → get_perp_history → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
100.0strict pass
latency 85.8s · cost $0.0054
tool calls 5
Model answer
30 (30 USD)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (5)
list_metrics → search_perps → get_perp_history → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 23.6s · cost $0.0663
tool calls 5
Model answer
30 (30 shares)
Why this score
Expected 30, got 30 (0% off) — within tolerance.
expected 30 · got 30 · error 0.0% · within tolerance
Tool calls (5)
list_metrics → search_perps → get_perp_history → run_calculation → submit_answer
Minimax
MiniMax M3
0.0not a strict pass
latency 72.0s · cost $0.0192
tool calls 8
Model answer
29 (29 ratio)
Why this score
Expected 30, got 29 (3% off) — outside tolerance.
expected 30 · got 29 · error 3.3% · outside tolerance
Tool calls (8)
list_metrics → search_perps → get_perp_history → run_calculation → get_perp → run_calculation → run_calculation → submit_answer