Skip to content

Protocol history

Changelog

The ruleset id changes whenever the engine, the question, the clocks, the caps, the seeds or the rating method change. Results are only comparable within one id.

Versions

tetris-bench@3 · 2026-09-20 · current

Blitz becomes real time, recordings become event logs, the rating becomes a mean

Engine. A T-spin is credited only when the piece's last move was a rotation and it dropped zero rows. v2 credited spins on hard drops. Candidate twins (placements that produce the same board, line count and top-out) are listed once, keeping the higher score. A lock-out writes the visible cells, ends the game and does not count the piece.

Blitz. The 100 ms cap is gone. The deadline is the falling piece: the harness races the call against the next gravity step, applies due steps as they come, and accepts a late answer when the placement is still reachable from the new height. Under v2 every hosted model scored exactly zero in Blitz because no network call fits in 100 ms.

IQ. The timeout is 20 s and a single timeout no longer ends the game. Three unusable answers in a row do, in both modes. A malformed optional field (hold, risk, cost) is dropped rather than invalidating the answer.

Evidence. Every hosted model sees the same rendered board and candidate table, with points split into clear and drop so the deepest drop no longer looks like the best move, two-letter ids that carry no order, and a strict JSON schema over those ids.

Recordings and rating. Games are stored as compact event logs that the browser re-simulates and verifies by hash. Brains are ranked by mean IQ score over 30 public seeds with a t-interval. Head-to-head bootstrap intervals mark ties. The Bradley-Terry number is a footnote.

tetris-bench@2 · 2026-09-20 · withdrawn

Shared candidate outcomes, IQ-only rating, 30 seeds

Gave every adapter the same simulated candidate outcomes, separated the IQ rating from Blitz, moved to 30 official seeds and an order-independent Bradley-Terry fit, and ended IQ games after repeated failures.

Withdrawn because the engine credited T-spins on hard drops, exposed duplicate I/S/Z candidates, ended an IQ game on a single timeout, asked slow brains only every other Blitz tick, and capped Blitz at 100 ms so every hosted model scored zero. Its prompt led every language model to pick the largest score delta, which is the deepest drop. The quick-suite results it published are kept in the archive.

tetris-bench@1 · 2026-09-20 · withdrawn

First arena: five seeds, Elo over IQ and Blitz

The original field of six brains on five seeds, with IQ and 100 ms Blitz averaged into one Elo-style rating updated in pairing order.

Withdrawn because the rating mixed network latency with decision quality, depended on pairing order, and gave local adapters board information that remote models did not receive. Jev chose the first listed option in all 57 of its accepted IQ decisions, which was always the spawn placement. Its index is kept in the archive. The recordings were removed from the site and remain in git history.