deepseek/deepseek-v4.1-flash

Correctness?

Efficiency?

Robustness?

Correctness says whether it can do the work; efficiency whether it finishes inside the same budget everyone gets; robustness whether it invents or omits. A low score should be readable as one of these, not a mystery.

Task type × domain?

Strict pass over this model’s tasks (share answered exactly right · task count). Rows are task types, columns are the data the answer lives in.

Where the points went?

Every task not answered exactly right, bucketed by what went wrong. 4 tasks.

Task families?

Lowest-scoring tasks?

Public example tasks link to the full question and every model’s answer; hidden tasks show only the family, the score and why.

• 66.7
  S-EQ24-AValuation Screen (trailing P/E) — count, mean and median of a rebuilt metric over the matchesview task

  Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. P/E here is market cap divided by trailing-twelve-month net income — the sum of the four latest single fiscal quarters of net_income from the filings (latest filing copy per quarter, consecutive quarters); names whose TTM net income is not positive have no P/E and are excluded. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $10B, 20-day average dollar volume greater than $300M, P/E (market cap / trailing-twelve-month net income from the filings) between 10 and 20. Scope: only entities with a known value for 20-day average dollar volume, P/E (market cap / trailing-twelve-month net income from the filings) in the snapshot; entities lacking any of them are out of scope and must not be returned. For the set of ALL entities that match the conditions above, report three numbers: count = how many match; mean = the arithmetic mean of P/E (TTM, from filings) (market_cap_usd divided by trailing-twelve-month net income = the sum of the four latest single-quarter net_income facts (consecutive quarters by period_end, latest filing copy per period); null when fewer than four quarters exist or TTM net income ≤ 0.) across them; median = its median (middle value, or the mean of the two middle values for an even count). Give mean and median as a decimal ratio (0.25 for 25%). Every matching entity's value must be rebuilt exactly — one wrong value moves the mean. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions. Clarification: a fiscal Q4 that the filings do not state as a single quarter is derived as the fiscal-year value minus Q1, Q2 and Q3 of the same fiscal year, and counts as a reported single quarter for TTM sums, latest-quarter growth and quarter sequences.

  2 of 3 fields correct; wrong: count.
• 88.9
  J01Statement Audit — Equityhidden

  Gold has 5, the model returned 4: 4 correct, 1 missed → F1 89%. Not a strict pass.
• 97.0
  S-EQ02-XXLiquidity Screen — plus two more hidden conditionshidden

  Gold has 17, the model returned 16: 16 correct, 1 missed → F1 97%. Not a strict pass.
• 98.9
  S-EQ21-XXGrowth Acceleration — plus two more hidden conditionshidden

  Gold has 45, the model returned 46: 45 correct, 1 false positive → F1 99%. Not a strict pass.
• 100.0
  J02Statement Audit — Perpetualshidden

  Gold has 2, the model returned 2: 2 correct → F1 100%. Strict pass.
• 100.0
  J05Statement Audit — Equity Technicalshidden

  Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
• 100.0
  J05Statement Audit — Equity Technicalshidden

  Gold has 1, the model returned 1: 1 correct → F1 100%. Strict pass.
• 100.0
  S-EQ21-XXGrowth Acceleration — plus two more hidden conditionsview task

  Fundamental ratios are not served as fields: rebuild each one from the raw SEC facts (get_equity_financials) exactly as defined in list_metrics.compute_yourself, using the latest filing at or before the data cutoff; a company whose filings lack the needed facts is out of scope. Acceleration means the latest quarter's revenue YoY growth is higher than the previous quarter's. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $10B, 20-day average dollar volume greater than $1B, sector equal to Basic Materials, revenue growth accelerating (latest-quarter YoY above the prior quarter's), P/E (market cap / trailing-twelve-month net income from the filings) less than 40, EMA50 above EMA200. Scope: only entities with a known value for 20-day average dollar volume, sector, revenue growth accelerating (latest-quarter YoY above the prior quarter's), P/E (market cap / trailing-twelve-month net income from the filings), EMA50 above EMA200 in the snapshot; entities lacking any of them are out of scope and must not be returned. Return every matching ticker; order does not matter. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.

  Gold has 2, the model returned 2: 2 correct → F1 100%. Strict pass.