← Leaderboard · task_6136e2057f1135a7fcd3189e
S-EQ24-A · Valuation Screen (trailing P/E) — count, mean and median of a rebuilt metric over the matchesMarket & SignalScreeningEquitySTRUCTURED_FIELDSHARDpublic example

Question

Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. P/E here is market cap divided by trailing-twelve-month net income — the sum of the four latest single fiscal quarters of net_income from the filings (latest filing copy per quarter, consecutive quarters); names whose TTM net income is not positive have no P/E and are excluded. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $10B, 20-day average dollar volume greater than $300M, P/E (market cap / trailing-twelve-month net income from the filings) between 10 and 20. Scope: only entities with a known value for 20-day average dollar volume, P/E (market cap / trailing-twelve-month net income from the filings) in the snapshot; entities lacking any of them are out of scope and must not be returned. For the set of ALL entities that match the conditions above, report three numbers: count = how many match; mean = the arithmetic mean of P/E (TTM, from filings) (market_cap_usd divided by trailing-twelve-month net income = the sum of the four latest single-quarter net_income facts (consecutive quarters by period_end, latest filing copy per period); null when fewer than four quarters exist or TTM net income ≤ 0.) across them; median = its median (middle value, or the mean of the two middle values for an even count). Give mean and median as a decimal ratio (0.25 for 25%). Every matching entity's value must be rebuilt exactly — one wrong value moves the mean. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions. Clarification: a fiscal Q4 that the filings do not state as a single quarter is derived as the fiscal-year value minus Q1, Q2 and Q3 of the same fiscal year, and counts as a reported single quarter for TTM sums, latest-quarter growth and quarter sequences.
Data as of 2026-09-10 · grader structured_fields

Expected / gold answer

count68 (68 count)
mean15.32 (15.319459694911647 ratio)
median15.60 (15.596190131025754 ratio)

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 818.8s · cost $0.0305
tool calls 13
Model answer
count68 (68)
mean15.32 (15.319459694911647)
median15.60 (15.596190131025754)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 68 · got 68 · error 0.0% · within tolerance
mean1✓
expected 15.319459694911647 · got 15.319459694911647 · error 0.0% · within tolerance
median1✓
expected 15.596190131025754 · got 15.596190131025754 · error 0.0% · within tolerance
Tool calls (13)
list_metrics → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 117.2s · cost $2.2717
tool calls 10
Model answer
count68 (68)
mean15.32 (15.31946)
median15.60 (15.59619)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 68 · got 68 · error 0.0% · within tolerance
mean1✓
expected 15.319459694911647 · got 15.31946 · error 0.0% · within tolerance
median1✓
expected 15.596190131025754 · got 15.59619 · error 0.0% · within tolerance
Tool calls (10)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
budget exhausted → forced answer
latency 938.8s · cost $0.0904
tool calls 20
Model answer
count68 (68)
mean15.32 (15.31945975)
median15.60 (15.5961905)
Why this score
Budget ran out and the answer was forced. All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 68 · got 68 · error 0.0% · within tolerance
mean1✓
expected 15.319459694911647 · got 15.31945975 · error 0.0% · within tolerance
median1✓
expected 15.596190131025754 · got 15.5961905 · error 0.0% · within tolerance
Tool calls (20)
list_metrics → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials✗ → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 165.7s · cost $0.2582
tool calls 23
Model answer
median15.60 (15.5961901310258)
mean15.32 (15.3194596949116)
count68 (68)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 68 · got 68 · error 0.0% · within tolerance
mean1✓
expected 15.319459694911647 · got 15.3194596949116 · error 0.0% · within tolerance
median1✓
expected 15.596190131025754 · got 15.5961901310258 · error 0.0% · within tolerance
Tool calls (23)
list_metrics → search_equities → search_equities → run_calculation → run_calculation✗ → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation✗ → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
100.0strict pass
latency 825.7s · cost $0.0447
tool calls 10
Model answer
count68 (68)
mean15.32 (15.319459694911647)
median15.60 (15.596190131025754)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 68 · got 68 · error 0.0% · within tolerance
mean1✓
expected 15.319459694911647 · got 15.319459694911647 · error 0.0% · within tolerance
median1✓
expected 15.596190131025754 · got 15.596190131025754 · error 0.0% · within tolerance
Tool calls (10)
list_metrics → search_equities → search_equities → get_equity_financials → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
66.7not a strict pass
latency 93.0s · cost $0.1947
tool calls 12
Model answer
count67 (67)
mean15.26 (15.26478749860563)
median15.57 (15.56719372218318)
Why this score
2 of 3 fields correct; wrong: count.
passed weight 2 / 3
fieldweightmust passresultdetail
count1✗
expected 68 · got 67 · error 1.5% · outside tolerance
mean1✓
expected 15.319459694911647 · got 15.26478749860563 · error 0.4% · within tolerance
median1✓
expected 15.596190131025754 · got 15.56719372218318 · error 0.2% · within tolerance
Tool calls (12)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Grok
x-ai/grok-4.7
66.7not a strict pass
latency 229.2s · cost $1.2054
tool calls 15
Model answer
count67 (67)
mean15.26 (15.26478749860563)
median15.57 (15.56719372218318)
Why this score
2 of 3 fields correct; wrong: count.
passed weight 2 / 3
fieldweightmust passresultdetail
count1✗
expected 68 · got 67 · error 1.5% · outside tolerance
mean1✓
expected 15.319459694911647 · got 15.26478749860563 · error 0.4% · within tolerance
median1✓
expected 15.596190131025754 · got 15.56719372218318 · error 0.2% · within tolerance
Tool calls (15)
list_metrics → search_equities → search_equities → get_equity_financials → get_equity_financials → run_calculation → run_calculation✗ → run_calculation → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → submit_answer
Minimax
MiniMax M3
66.7not a strict pass
latency 420.5s · cost $0.2269
tool calls 30
Model answer
count67 (67)
mean15.26 (15.26478749860563)
median15.57 (15.56719372218318)
Why this score
2 of 3 fields correct; wrong: count.
passed weight 2 / 3
fieldweightmust passresultdetail
count1✗
expected 68 · got 67 · error 1.5% · outside tolerance
mean1✓
expected 15.319459694911647 · got 15.26478749860563 · error 0.4% · within tolerance
median1✓
expected 15.596190131025754 · got 15.56719372218318 · error 0.2% · within tolerance
Tool calls (30)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials✗ → get_equity_financials → get_equity_financials → run_calculation✗ → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation✗ → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer