How Tetris Bench works

In 30 seconds

A brain is anything that can answer one question. The question is the board, the piece in play, the hold slot, the next five pieces, and every legal placement already simulated one step ahead. The answer is the placement it wants. The engine plays it and asks again. Every brain, from a seeded random choice to a hosted language model, plays the same 30 public seeds through the same engine.

The headline number is the mean final score with gravity paused (IQ). Beside it sits a 95% interval across seeds, a median latency and a cost. Blitz, where the piece keeps falling, is reported on its own. None of this is a test of general intelligence. It measures how well a decision policy plays one deterministic game when given the same evidence as every other entrant.

Words the tables use

Brain

Anything that answers the question: a local heuristic, a search, a hosted language model, a classifier.

Adapter

The thin code that turns the question into the brain's input and the brain's output back into an answer. One per brain.

Seed

A string that fixes the piece sequence. seed-01 to seed-30 are public and every brain plays all of them.

Tick

One question. In IQ a tick is one piece decision. In Blitz a piece may take several ticks when an answer is unreachable or the piece locks before the answer arrives.

Candidate

A legal placement (column and rotation) already simulated one step ahead: the board after it locks, lines cleared, points.

IQ

The mode where gravity pauses while the brain thinks (up to 20 s a move). The ranking comes from IQ only.

Blitz

The real-time mode. The piece keeps falling and the deadline is the next gravity step. Reported separately.

Choice

The required part of an answer: a probability over the listed placements. The highest probability is played; ties go to the lower column, then the lower rotation.

Hold (noul)

Optional: swap the piece in play with the hold slot instead of placing it. Boolean, or a number above 0.5.

Risk (score)

Optional: the brain's probability that the game tops out within the next ten pieces. Scored for calibration only.

Stale

An answer carrying the hash of an earlier question. Discarded.

Late

A Blitz answer that arrived after gravity moved the piece. Still played when the placement is still reachable.

Locked by gravity

A Blitz question the piece answered for itself: it hit the stack before the brain replied.

Adapter failure

Three unusable answers in a row (invalid, stale, error or timeout) end the game.

Piece cap

Games stop after 500 locked pieces or 10,000 ticks. A game that gets there ends 'piece cap' rather than 'top-out'.

How a game works

1. The engine creates a game from the seed: a 10 × 20 board, a seven-bag piece sequence, a hold slot and a five-piece preview.
2. Each tick the harness lists every reachable placement of the piece in play, simulates each one (drop, lock, clear) and shuffles the list deterministically from the state hash so the first option carries no information.
3. The brain receives the state, the candidates and the hash, and returns a probability over the listed placements, optionally with a hold request and a risk estimate.
4. The harness validates the answer against the question it asked. The highest-probability placement is hard-dropped. A hold swaps the piece. Anything malformed, stale or unreachable is recorded and the game moves on.
5. The game ends on top-out, after 500 locked pieces, after 10,000 ticks, or after three unusable answers in a row. Every decision is recorded as a replayable event log.

The contract

This is a real question from a recorded game (Jev 1.13, seed-01, tick 14) with the answer that was recorded for it, shown with v3 field names. The full input has 34 candidates. Three are shown.

AgentInput

{
  "ruleset": "tetris-bench@3",
  "mode": "IQ", "tick": 14,
  "stateHash": "7e417cf6",            // copy this back verbatim
  "active": { "type": "T", "x": 3, "y": -1, "rotation": 0 },
  "hold": null, "canHold": true,
  "next": ["Z", "O", "J", "I", "L"],
  "level": 1, "score": 676, "lines": 2, "pieces": 14, "combo": -1, "backToBack": false,
  "grid": [                            // 20 rows x 10 columns, row 0 = top; null or a piece letter
    /* rows 0-14: all null */
    ["I","J",null,null,null,null,"S","S",null,null],   // row 15   ##....##..
    ["I","J","J","J","Z","S","S",null,"L",null],       // row 16   #######.#.
    ["I","O","O","Z","Z","T","L","L","L",null],        // row 17   #########.
    ["L",null,"S","S","O","O",null,"T","Z","Z"],       // row 18   #.####.###
    [null,"S","S",null,"O","O","T","T","T",null]       // row 19   .##.#####.
  ],
  "features": { "heights": [5,5,4,4,4,4,5,5,4,2], "maxHeight": 5, "aggregateHeight": 42,
                "holes": 6, "bumpiness": 5, "wells": 2, "rowTransitions": 50, "colTransitions": 22 },
  "legal": [ { "x": 3, "rotation": 0 }, { "x": 8, "rotation": 3 }, { "x": 2, "rotation": 0 } /* ...31 more */ ],
  "candidates": [                      // legal[i] is candidates[i].placement; ids are opaque and reshuffled every tick
    { "id": "p14", "placement": { "x": 3, "rotation": 0 },
      "linesCleared": 0, "scoreDelta": 30, "dropPoints": 30, "clearPoints": 0, "topOut": false,
      "features": { "heights": [5,5,4,5,6,5,5,5,4,2], "maxHeight": 6, "holes": 6, "bumpiness": 7, "wells": 3, ... },
      "grid": [ /* rows 0-13 empty */ "....#.....", "##.#####..", "#######.#.", "#########.", "#.####.###", ".##.#####." ] },
    { "id": "p28", "placement": { "x": 8, "rotation": 3 },
      "linesCleared": 0, "scoreDelta": 30, "dropPoints": 30, "clearPoints": 0, "topOut": false,
      "features": { "heights": [5,5,4,4,4,4,5,5,5,6], "maxHeight": 6, "holes": 7, "bumpiness": 3, "wells": 0, ... },
      "grid": [ /* ... */ ".........#", "##....####", "#######.##", "#########.", "#.####.###", ".##.#####." ] },
    { "id": "p23", "placement": { "x": 2, "rotation": 0 },
      "linesCleared": 0, "scoreDelta": 30, "dropPoints": 30, "clearPoints": 0, "topOut": false,
      "features": { "heights": [5,5,5,6,5,4,5,5,4,2], "maxHeight": 6, "holes": 6, "bumpiness": 7, "wells": 3, ... },
      "grid": [ /* ... */ "...#......", "#####.##..", "#######.#.", "#########.", "#.####.###", ".##.#####." ] }
    /* ...31 more */
  ]
}

Candidate grids are real (Piece | null)[][] arrays like grid. They are drawn as # and . here for space. scoreDelta is dropPoints (two per row hard-dropped) plus clearPoints (line clears and spins). features is the same board summary the local heuristics use: column heights, covered holes, bumpiness (sum of neighbouring height differences), wells and transitions.

What a hosted model sees

Hosted models get this same evidence rendered as text with a fixed system prompt: the board as rows of # and ., then one line per candidate. Ids are two random letters so the order carries nothing, and the reply is constrained to a JSON schema over those ids.

Active piece: T   Next: Z O J I L   Hold slot: empty (canHold=true)   Level 1, score 676, lines 2, pieces 14, combo -1, back-to-back false

Current board (rows 14-19 shown; rows above 14 are empty):
14 ..........
15 ##....##..
16 #######.#.
17 #########.
18 #.####.###
19 .##.#####.
   0123456789
Column heights now: [5,5,4,4,4,4,5,5,4,2]  holes now: 6  maxH now: 5

Candidates (34):
RM: x=3 rot=0 | lines=0 clear=+0 drop=+30 maxH=6 holes=6 bump=7 wells=3 topOut=no
EG: x=8 rot=3 | lines=0 clear=+0 drop=+30 maxH=6 holes=7 bump=3 wells=0 topOut=no
HV: x=2 rot=0 | lines=0 clear=+0 drop=+30 maxH=6 holes=6 bump=7 wells=3 topOut=no
... 31 more

Reply with JSON only: {"candidate":"<ID>","hold":false,"risk":0.1}

AgentAnswer

The answer Jev recorded for this position under v2, re-keyed to the v3 hash (34 entries, abbreviated); the risk score and the meta block are what the v3 adapter adds. A language model's {"candidate":"RM","hold":false,"risk":0.1} becomes the same shape with a single entry at p: 1.

{
  "stateHash": "7e417cf6",
  "choice": [
    { "x": 3, "rotation": 0, "p": 0.27 },   // p14, the highest p, is played
    { "x": 8, "rotation": 3, "p": 0.24 },   // p28
    { "x": 2, "rotation": 0, "p": 0.09 }    // p23
    /* ...31 more, one per listed placement, summing to 1 within 0.001 */
  ],
  "noul": { "hold": 0.3 },                  // optional: boolean or a number, hold iff > 0.5
  "score": { "risk": 0.1 },                 // optional: P(top-out within ten pieces)
  "costUsd": 0.00019068,
  "meta": { "retries": 0 }
}

The smallest equivalent answer is { "stateHash": "7e417cf6", "choice": [ { "x": 3, "rotation": 0, "p": 1 } ] }.

Effect. The answer validates and placement (3, 0) is played. The T hard-drops 15 rows (30 drop points) and locks with cells at (4, 14), (3, 15), (4, 15), (5, 15). No line clears. Score 676 → 706, pieces 14 → 15. The next question is tick 15 with a new hash.

Validation

• stateHash must equal the question's hash, otherwise the answer is stale.
• choice must be non-empty. Every entry must be a listed placement, with finite p in [0, 1], no duplicates, and the probabilities must sum to 1 within 0.001. Otherwise the answer is invalid.
• The played placement is the highest p. Ties go to the lower x, then the lower rotation.
• noul, score, costUsd and meta are optional. A malformed optional field is dropped and noted. It no longer invalidates the answer. A hold requested when holding is not available is ignored and the choice is played.
• In Blitz the placement must also still be reachable from the piece's current height and produce the disclosed candidate grid. Otherwise it is unreachable and the piece keeps falling.

Reading the leaderboard

#

Rank by mean IQ score. A ≈ after a rank means the head-to-head interval with the row above includes zero: the two are not separated by this data.

Brain

Name, kind (baseline, heuristic, search, LLM, classifier) and provider for hosted models.

Score

Mean final score over the 30 seeds with gravity paused. The headline number.

±95%

Half-width of a 95% t-interval across seeds. Shown only for official 30-seed results. A quick run lists its per-seed scores instead.

Lines

Mean lines cleared per game.

Latency

Median time of completed calls, from the question leaving the harness to the answer arriving. Excludes the harness's own preparation and calls that were aborted.

$ / 100 decisions

What the provider reported charging, per hundred calls, over both modes. Local brains cost nothing and show a dash. Calls the provider did not price are counted on the brain page.

Blitz · Level reached

The highest level any Blitz game reached. Gravity speeds up with level.

Blitz · Answered in time

Share of Blitz questions whose answer was accepted before gravity moved the piece. Late answers, unusable answers and pieces that locked first all count against it.

not run

The brain has no Blitz games in this index. Blitz is optional per brain. IQ is not.

Rating

The rating is the mean score. Each brain plays every one of the 30 public seeds once in IQ. Its rating is the mean of those 30 final scores, and the leaderboard is sorted by it (ties broken by mean lines, then slug).

The interval is mean ± t(0.975, S−1) · sd / √S over the S seed scores. It describes how much the mean would move on a different draw of 30 seeds. It says nothing about provider variation, prompt choice or adaptation to public seeds. No interval is shown for fewer than ten seeds.

Head-to-head compares two brains seed by seed: the mean of the paired score differences, with a 2,000-replicate seed-block bootstrap percentile interval, and the count of seeds each brain won. Two adjacent rows whose interval includes zero are marked as a tie on the leaderboard.

The Elo-style number on brain pages is a penalised Bradley-Terry fit over the same IQ seed points (prior precision 0.25, each pair weighted 1/(n−1)), on an Elo scale centred at 1500, with a bootstrap interval that is withheld when a brain's record is separated (it won or lost every comparison). It is a footnote: it moves whenever a brain joins the field, which a mean score does not.

What it does not claim. Seeds are public, so a brain could in principle be tuned to them. Only IQ is rated, so speed is invisible in the rank. Blitz results depend on hardware and network as much as on the brain and are reported, not rated.

Rules of the game

The ruleset id is tetris-bench@3 and any behavioural change bumps it. In brief:

• 10 × 20 board, SRS rotation and kicks, seeded seven-bag, one hold per piece, five-piece preview.
• A placement is a column and rotation reachable from the piece's current height by moving and rotating, then hard-dropped. Physically identical outcomes are listed once.
• Scoring: two points per row dropped, 100 / 300 / 500 / 800 × level for one to four lines, T-spin and mini T-spin bonuses, back-to-back ×1.5, combos and perfect clears. A T-spin is credited only when the piece's last move was a rotation and it dropped zero rows.
• Gravity at level L moves one row every max(16, round(1000 · 0.8^(L−1))) ms, and level is 1 + floor(lines / 10). A piece locks on the gravity step that finds it on the stack.
• Top-out when a piece cannot spawn or locks with any cell above the board. Games also end at 500 pieces, 10,000 ticks or three unusable answers in a row.

The full rules, coordinate conventions and piece tables ↗

Add a brain

Fork the public repository ↗ and implement the Brain interface. A hosted model that speaks the OpenRouter chat API needs no code at all: add an entry to the model registry and it gets the shared adapter, prompt and JSON schema.

export interface Brain {
  slug: string;
  name: string;
  description: string;
  kind: 'baseline' | 'heuristic' | 'search' | 'llm' | 'classifier';
  provider?: string;
  model?: string;
  via?: string;
  adapterVersion: string;
  decide(
    input: AgentInput,
    ctx: { signal: AbortSignal; budgetMs: number | null },
  ): Promise<unknown> | unknown;
}

decide returns an AgentAnswer (or a promise of one). Pass ctx.signal to any network call so a timed-out or gravity-locked question can be aborted. ctx.budgetMs is the IQ timeout, or null in Blitz where the falling piece is the only clock. Report costUsd when the provider tells you.

Registering a hosted model

// lib/tetris-bench/models.ts
{
  slug: 'my-model', name: 'My Model', provider: 'Vendor',
  model: 'vendor/my-model', kind: 'llm',
  temperature: 0, maxTokens: 80, jsonSchema: true,
  pricing: { in: 0.15, out: 0.6 },   // USD per million tokens, optional
}

Running it

node --test scripts/tetris-bench/*.test.ts               # unit tests, no network
node scripts/tetris-bench/probe.ts --brain=my-model       # ten fixed positions: does it avoid holes?
node scripts/tetris-bench/run.ts --brains my-model        # quick: 3 seeds, both modes
node scripts/tetris-bench/run.ts --official --brains my-model --merge   # 30 seeds, merged into the index

Hosted brains need OPENROUTER_API_KEY in the environment. The runner makes one real pre-flight call before a batch and stops with a clear message if it fails.

Pull request checklist

• The brain has a slug, a plain-language description and an adapterVersion.
• Unit tests stub fetch. Nothing in the test suite touches the network.
• The probe passes: no candidate that adds a hole is chosen on the fixed positions, under both candidate orders.
• Model settings (temperature, reasoning, max tokens, schema support) are in the registry, not in prose.
• A quick run completes without an adapter failure and its recordings verify by re-simulation.

Changelog

Results published before 2026-09-20 under tetris-bench@1 and @2 are withdrawn. The reasons and every protocol change are listed on the changelog. The withdrawn indices are kept in the archive.