← Leaderboard · batch_8a1d7974251e933dfc1f89a2
MiniMax M3.1 Flash (free)
minimax/minimax-m3.1-flash:free · revision r1 · provider Minimax · rank #7 · tested 2026-10-20 10:19 UTC · data as of 2026-09-10
94.9%
Strict pass · 179 tasks answered exactly right
179
Tasks in the paper
$0.000
Cost / task (not scored)
215.0s
Median latency (not scored) · 15.4 tool calls / task
Correctness?
Strict pass, all tasks94.9%
Strict pass, data-processing tasks96% (128)
Strict pass, judgment tasks (statement audits, restatements, calendar quarters)92% (51)
Strict pass, tasks that rebuild hidden metrics96% (119)
Strict pass, tasks answerable from served fields93% (60)
Efficiency?
Budget exhausted (forced answer)18%
Output tokens exhausted (reasoning cut off)1%
No answer submitted0%
Tool calls / task · median latency · cost / task15.4 · 215.0s · $0.000
Robustness?
Empty-set tasks answered "none" correctly50% (2)
False positives among returned entities0%
Gold entities missed0%
Correctness says whether it can do the work; efficiency whether it finishes inside the same budget everyone gets; robustness whether it invents or omits. A low score should be readable as one of these, not a mystery.
Pillars?
Strict pass per pillar (share of that pillar’s tasks answered exactly right); the ranking uses the strict pass over all tasks.
Financial Analysis94.035%
Market & Signal95.350%
Failures: NONE 174 · MODEL_BUDGET_EXCEEDED 5
Financial Analysis breakdown
Retrieval—25%
Calculation100.030%
Comparison / Trend92.925%
Risk / Position—20%
Market & Signal breakdown
Screening96.725%
Ranking / Top-K100.020%
Derived metric100.015%
Signal detection90.225%
Multi-step100.015%
Task type × domain?
Strict pass over this model’s tasks (share answered exactly right · task count). Rows are task types, columns are the data the answer lives in.
| type | Crypto / Perp | DEX pools | Equity | Financial filings |
|---|---|---|---|---|
| Calculation | · | · | · | 100.0 8 tasks |
| Comparison / Trend | · | · | 93.0 43 tasks | 100.0 1 tasks |
| Screening | · | 100.0 4 tasks | 96.4 56 tasks | · |
| Ranking / Top-K | · | · | 100.0 4 tasks | · |
| Derived metric | 100.0 6 tasks | · | 100.0 12 tasks | · |
| Signal detection | 75.0 4 tasks | 100.0 4 tasks | 90.9 33 tasks | · |
| Multi-step | · | · | 100.0 4 tasks | · |
Where the points went?
Every task not answered exactly right, bucketed by what went wrong. 9 tasks.
| cause | tasks | share |
|---|---|---|
| Budget exhausted (forced answer) | 6 | 67% |
| Missed entities (misses only) | 2 | 22% |
| Returned extra entities (false positives only) | 1 | 11% |
Task families?
| family | type | tasks | mean | strict |
|---|---|---|---|---|
| J05 Statement Audit — Equity Technicals | Signal detection | 4 | 100.0 | 100% |
| J01 Statement Audit — Equity | Comparison / Trend | 39 | 92.3 | 92% |
| S-CR12-A Pool Buyer/Seller Breadth — count, mean and median of a rebuilt metric over the matches | Signal detection | 3 | 100.0 | 100% |
| S-EQ12-XX Oversold / Overbought (RSI) — plus two more hidden conditions | Signal detection | 4 | 100.0 | 100% |
| S-EQ25-A Growth + Valuation (trailing P/E as a proxy for EV/Sales) — count, mean and median of a rebuilt metric over the matches | Screening | 4 | 75.0 | 75% |
| S-EQ21-XX Growth Acceleration — plus two more hidden conditions | Signal detection | 4 | 100.0 | 100% |
| J02 Statement Audit — Perpetuals | Signal detection | 4 | 75.0 | 75% |
| S-CR18-A Perp RSI Range + Positive Trend — count, mean and median of a rebuilt metric over the matches | Derived metric | 4 | 100.0 | 100% |
| S-EQ05-X First Breakout 20D — plus one more hidden condition | Signal detection | 4 | 100.0 | 100% |
| S-EQ18-A Low-Drawdown Leader — count, mean and median of a rebuilt metric over the matches | Screening | 4 | 75.0 | 75% |
| S-EQ10-XX Pullback in Uptrend — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| S-EQ01-XX Size Screen — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| S-CR08-N Pool Buy/Sell Transaction-Count Imbalance — count of matches | Signal detection | 1 | 100.0 | 100% |
| J03 Restatement Screen | Comparison / Trend | 4 | 100.0 | 100% |
| S-EQ09-X Moving Average Cross (EMA20 above EMA50 today) — plus one more hidden condition | Signal detection | 3 | 100.0 | 100% |
| S-CR06-M Token DEX Liquidity Threshold (pools) — median of a rebuilt metric over the matches | Screening | 4 | 100.0 | 100% |
| S-EQ22-A Profitability Screen (gross + operating margin) — count, mean and median of a rebuilt metric over the matches | Screening | 4 | 100.0 | 100% |
| S-EQ13-XX ATR Volatility — plus two more hidden conditions | Derived metric | 4 | 100.0 | 100% |
| F09 Free Cash Flow | Calculation | 1 | 100.0 | 100% |
| S-CR19-N Perp Volatility Expansion (ATR% threshold) — count of matches | Derived metric | 1 | 100.0 | 100% |
| F08-R EPS Growth (diluted) — six companies ranked | Calculation | 1 | 100.0 | 100% |
| S-EQ20-XX EPS Growth Screen — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| S-EQ26 Cash Flow Quality (FCF margin) | Screening | 4 | 100.0 | 100% |
| F04-R Revenue Growth — six companies ranked | Calculation | 1 | 100.0 | 100% |
| S-EQ19-XX-N Revenue Growth Screen — plus two more hidden conditions — count of matches | Screening | 4 | 100.0 | 100% |
| S-EQ16-XX Relative Strength vs Index — plus two more hidden conditions | Derived metric | 4 | 100.0 | 100% |
| S-EQ06-XX-N Breakout (N-day high) — plus two more hidden conditions — count of matches | Signal detection | 3 | 100.0 | 100% |
| S-EQ33 Market-cap Bucket Ranking | Ranking / Top-K | 4 | 100.0 | 100% |
| S-EQ29-XX Breakout with Exclusion — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| F29 Trailing-twelve-month revenue | Calculation | 1 | 100.0 | 100% |
| S-EQ04-XX Unusual Volume — plus two more hidden conditions | Signal detection | 4 | 100.0 | 100% |
| S-EQ23-XX-N Quality Screen (ROE + operating margin as proxies) — plus two more hidden conditions — count of matches | Screening | 4 | 100.0 | 100% |
| F05-R10 CAGR — ten companies ranked | Calculation | 4 | 100.0 | 100% |
| S-EQ07-XX-N Breakdown 60D — plus two more hidden conditions — count of matches | Signal detection | 3 | 100.0 | 100% |
| F15 Peer Comparison (growth ranking) | Comparison / Trend | 1 | 100.0 | 100% |
| S-EQ15-M Gap + Volume — median of a rebuilt metric over the matches | Signal detection | 1 | 0.0 | 0% |
| S-EQ11-A RSI Range — count, mean and median of a rebuilt metric over the matches | Derived metric | 4 | 100.0 | 100% |
| S-EQ25-XX Growth + Valuation (trailing P/E as a proxy for EV/Sales) — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| S-EQ02-XX Liquidity Screen — plus two more hidden conditions | Screening | 4 | 100.0 | 100% |
| S-EQ24-A Valuation Screen (trailing P/E) — count, mean and median of a rebuilt metric over the matches | Screening | 4 | 100.0 | 100% |
| S-EQ34-N Multi-period Consistency — count of matches | Screening | 4 | 100.0 | 100% |
| S-X06 Compound Screen with Staged Narrative (liquidity → growth → momentum → rank) | Multi-step | 4 | 100.0 | 100% |
| S-EQ14 Gap Up | Signal detection | 3 | 33.3 | 33% |
| S-CR17-X Perp Trend Alignment (close > EMA20 > EMA50) — plus one more hidden condition | Derived metric | 1 | 100.0 | 100% |
Lowest-scoring tasks?
Public example tasks link to the full question and every model’s answer; hidden tasks show only the family, the score and why.
- 0.0S-EQ25-AGrowth + Valuation (trailing P/E as a proxy for EV/Sales) — count, mean and median of a rebuilt metric over the matcheshiddenBudget ran out and the answer was forced. Invalid answer: MODEL_BUDGET_EXCEEDED.
- 0.0Gap = today's open / yesterday's close − 1; relative volume = today's volume / 20-day average volume. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $1B, opening gap greater than 3%, relative volume (today / 20-day average) greater than 2. Scope: only entities with a known value for opening gap, relative volume (today / 20-day average) in the snapshot; entities lacking any of them are out of scope and must not be returned. Across ALL entities that match the conditions above, what is the median Opening gap (Current session open relative to the previous close, minus one.)? Median = the middle value, or the mean of the two middle values for an even count. Answer as a decimal ratio (0.25 for 25%). Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.Budget ran out and the answer was forced. Invalid answer: MODEL_BUDGET_EXCEEDED.
- 0.0Gap = today's open relative to yesterday's close, as a fraction. Find US-listed common equities (instruments without a reported market cap — warrants, units, rights, preferred series, notes — are not in the universe) with market cap greater than $1B, opening gap greater than 3%. Scope: only entities with a known value for opening gap in the snapshot; entities lacking any of them are out of scope and must not be returned. Return every matching ticker; order does not matter. Rebuild policy: metrics that are not served as fields (fundamental ratios, growth rates, valuation, technical signals) must be recomputed by you from the raw facts and daily bars the tools serve, following the definitions in list_metrics.compute_yourself to the letter (period selection, latest-filing copy, quarter arithmetic); do the arithmetic with run_calculation over the fetched data. The reference answer was computed from the same facts by the same definitions.Budget ran out and the answer was forced. Invalid answer: MODEL_BUDGET_EXCEEDED.
- 0.0S-EQ14Gap UphiddenBudget ran out and the answer was forced. Invalid answer: MODEL_BUDGET_EXCEEDED.
- 0.0S-EQ18-ALow-Drawdown Leader — count, mean and median of a rebuilt metric over the matcheshiddenBudget ran out and the answer was forced. Invalid answer: MODEL_BUDGET_EXCEEDED.
- 66.7J01Statement Audit — EquityhiddenGold has 1, the model returned 2: 1 correct, 1 false positive → F1 67%. Not a strict pass.
- 66.7J01Statement Audit — EquityhiddenGold has 4, the model returned 2: 2 correct, 2 missed → F1 67%. Not a strict pass.
- 88.9J02Statement Audit — PerpetualshiddenGold has 5, the model returned 4: 4 correct, 1 missed → F1 89%. Not a strict pass.
Historical runs?
Every batch and every revision, latest first. Old runs are never overwritten: a score belongs to a batch, not to the model forever.
| batch | kind | revision | rank | strict pass | test time | data as of | environment |
|---|---|---|---|---|---|---|---|
| batch_8a1d7974251e933dfc1f89a2 | RELEASE | r1 | #7 | 94.9 | 2026-10-20 10:19 UTC | 2026-09-10 | env-0.1.0-primary |