← Leaderboard · task_24f130574f0575ffb9c74dc7
S-EQ22-A · Profitability Screen (gross + operating margin) — count, mean and median of a rebuilt metric over the matchesMarket & SignalScreeningEquitySTRUCTURED_FIELDSHARDpublic example

Question

Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. Gross margin and operating margin are for the latest reported fiscal year (SEC filings). Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $10B, 20-day average dollar volume greater than $300M, gross margin (latest fiscal year) greater than 60%, operating margin (latest fiscal year) greater than 30%. Scope: only entities with a known value for 20-day average dollar volume, gross margin (latest fiscal year), operating margin (latest fiscal year) in the snapshot; entities lacking any of them are out of scope and must not be returned. For the set of ALL entities that match the conditions above, report three numbers: count = how many match; mean = the arithmetic mean of Gross margin (latest FY) (gross_profit / revenue for the latest fiscal year that reports revenue (fiscal_period FY; latest filing copy per period).) across them; median = its median (middle value, or the mean of the two middle values for an even count). Give mean and median as a decimal ratio (0.25 for 25%). Every matching entity's value must be rebuilt exactly — one wrong value moves the mean. Filing-version policy: every financial value is taken from the LATEST filing accepted by the SEC at or before the data cutoff that reports the requested period. If a later filing (before the cutoff) restates an earlier period, the restated value is the answer; the original filing's value is not. A fiscal-year value is taken only from an annual report (10-K / 20-F / 40-F, including amendments) — a quarterly report never supplies an annual value. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.
Data as of 2026-09-10 · grader structured_fields

Expected / gold answer

count11 (11 count)
mean0.7097 (0.709715175900079 ratio)
median0.6794 (0.6794409337058032 ratio)

Computed by code from the frozen snapshot and verified by an independent recomputation before the batch was frozen. No model ever saw it.

Model answers (8)

XiaomiMiMo
xiaomi/mimo-v2.6-flash
100.0strict pass
latency 465.2s · cost $0.0245
tool calls 15
Model answer
count11 (11)
mean0.7097 (0.709715175900079)
median0.6794 (0.6794409337058032)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337058032 · error 0.0% · within tolerance
Tool calls (15)
list_metrics → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → run_calculation → submit_answer
DeepSeek
deepseek/deepseek-v4.1-flash
100.0strict pass
latency 103.5s · cost $0.1837
tool calls 12
Model answer
count11 (11)
mean0.7097 (0.709715175900079)
median0.6794 (0.6794409337058032)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337058032 · error 0.0% · within tolerance
Tool calls (12)
list_metrics → search_equities → search_equities → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → submit_answer
Anthropic
Claude Opus 5.5
100.0strict pass
latency 80.1s · cost $1.7095
tool calls 9
Model answer
count11 (11)
mean0.7097 (0.709715175900079)
median0.6794 (0.6794409337058032)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337058032 · error 0.0% · within tolerance
Tool calls (9)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → submit_answer
Qwen
qwen/qwen3.8-flash
100.0strict pass
latency 746.3s · cost $0.1208
tool calls 20
Model answer
count11 (11)
mean0.7097 (0.7097151759)
median0.6794 (0.6794409337)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.7097151759 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337 · error 0.0% · within tolerance
Tool calls (20)
list_metrics → search_equities → search_equities → get_equity_financials → get_equity_financials → get_equity_financials✗ → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → get_equity_financials → run_calculation✗ → run_calculation → run_calculation → run_calculation → submit_answer
Gemini
google/gemini-3.8-flash
100.0strict pass
latency 113.8s · cost $0.1495
tool calls 15
Model answer
mean0.7097 (0.709715175900079)
count11 (11)
median0.6794 (0.679440933705803)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.679440933705803 · error 0.0% · within tolerance
Tool calls (15)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials → get_equity_financials → run_calculation → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → submit_answer
Z.ai
z-ai/glm-5.3-flash
100.0strict pass
latency 1606.4s · cost $0.0329
tool calls 9
Model answer
count11 (11)
mean0.7097 (0.7097151759)
median0.6794 (0.6794409337)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.7097151759 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337 · error 0.0% · within tolerance
Tool calls (9)
list_metrics → search_equities → search_equities → get_equity_financials → get_equity_financials → run_calculation → get_equity_financials → run_calculation → submit_answer
Grok
x-ai/grok-4.7
100.0strict pass
latency 153.3s · cost $0.7402
tool calls 16
Model answer
count11 (11)
mean0.7097 (0.709715175900079)
median0.6794 (0.6794409337058032)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337058032 · error 0.0% · within tolerance
Tool calls (16)
list_metrics → search_equities → search_equities → get_equity_financials → run_calculation → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → get_equity_financials → run_calculation → run_calculation → get_equity_financials → run_calculation → run_calculation → submit_answer
Minimax
MiniMax M3
100.0strict pass
latency 1834.3s · cost $0.0806
tool calls 18
Model answer
count11 (11)
mean0.7097 (0.709715175900079)
median0.6794 (0.6794409337058032)
Why this score
All 3 fields correct.
passed weight 3 / 3
fieldweightmust passresultdetail
count1✓
expected 11 · got 11 · error 0.0% · within tolerance
mean1✓
expected 0.709715175900079 · got 0.709715175900079 · error 0.0% · within tolerance
median1✓
expected 0.6794409337058032 · got 0.6794409337058032 · error 0.0% · within tolerance
Tool calls (18)
list_metrics → search_equities → search_equities → run_calculation → get_equity_financials → run_calculation → get_equity_financials → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation → run_calculation✗ → run_calculation → run_calculation → submit_answer