# ChessBench

> The 24/7 LLM Chess Benchmark. Made by MadeByLupu (@bylupu). LLMs complete placement games and then play a continuous ladder. The public site requires no login.

## Primary pages

- [Benchmark and leaderboard](https://chessbench.org/): Current games, Glicko-2 ratings, rejected-move rates, API costs, and match records. HTML includes the data before JavaScript runs.
- [Methodology](https://chessbench.org/methodology): Design, limits, interpretation, and exact shared system prompts.
- [Previous matches](https://chessbench.org/previous-matches): Paginated match records. Use ?page=2 for the next page.
- [Human challenges](https://chessbench.org/play): Start a human-vs-LLM game.
- [Cost Breakdown](https://chessbench.org/cost-breakdown): Recorded operating costs across rating eras, with one LLM-and-thinking graph split by game purpose.
- [Human leaderboard](https://chessbench.org/human-leaderboard): X players ranked by human Elo, with opponent history and public game replays.
- [Methodology in Markdown](https://chessbench.org/methodology.md): The same method in plain text.
- [Sitemap](https://chessbench.org/sitemap.xml): Public pages and stable match URLs.
- [Creator](https://x.com/bylupu): MadeByLupu, @bylupu.

## Read-only data

- [Current results](https://chessbench.org/api/public): JSON with standings, the rating phase, live and latest game IDs, the next-game deadline, and match totals. Placement and editable queue details are admin-only.
- [Cost data](https://chessbench.org/api/costs): Known costs by LLM setup, purpose, and era, plus dated expenses. Optional query fields: from, to (YYYY-MM-DD), purpose, modelKey, eraId.
- Previous matches: GET https://chessbench.org/api/previous-matches?page=1. Returns 12 completed matches per page.
- Match page: https://chessbench.org/matches/{id}
- Match detail: GET https://chessbench.org/api/matches/{id}. Returns match settings, accepted moves, viewer evaluations, call metrics, and legal moves for an active game.
- Match JSON export: GET https://chessbench.org/api/matches/{id}/export
- Match PGN: GET https://chessbench.org/api/matches/{id}/export?format=pgn
- LLM-battle MP4 files require the signed-in administrator. Unshared human-game MP4 files require the owning player session or administrator. A player can explicitly share a human video, which enables its public player and video-card metadata.
- Human standings: GET https://chessbench.org/api/human-leaderboard
- Player history: GET https://chessbench.org/api/human-players/{id}?page=1. Returns 20 completed games per page with LLM opponents, saved human Elo changes, and page metadata. Player totals cover the full history. HTML: https://chessbench.org/players/{id}?page=2. Use Review game to reach video controls.
- Human game replay: https://chessbench.org/human-games/{id}. JSON: GET https://chessbench.org/api/human-games/{id}. Only completed games are public.

Use actual IDs from current results or the sitemap. A missing match returns HTTP 404. Cache responses locally and avoid rapid polling. All times are UTC. Prices and charges are USD. Match snapshots preserve historical settings. Raw model conversations and credentials are not public data.

## Interpretation

The first placement games start every selected identity at 1500, with deviation 350 and volatility 0.06. Every pair plays both colours. All required results form one Glicko-2 batch. Later games update both players once per completed result. New identities receive priority placement games against all saved opponents with both colours and may lead when placement completes. Retirement preserves historical results and established ratings.

The normal ladder selects the LLM and reasoning setup with the fewest completed ladder and manual games in 48 hours. Initial placement, new-model placement, and human games are excluded. Both players count. Ties and opponents within ±5 configured rank positions are selected uniformly. The player with the lower White share in the same window gets White; no games means 50%, and equal shares use random colours.

The Elo display uses Glicko-2 and compares LLM setups in this pool. It is not equivalent to Chess.com, FIDE, or other human ratings. For exactly equal Elo, the setup with the lower known average game cost ranks higher. Rejected % counts rejected moves divided by answered move attempts. Only completed rated games change ratings.

The leaderboard shows ratings, rejected moves, time per turn, cost per game, output tokens per game, and results. Its LLM-only rating notice distinguishes benchmark Elo from human ratings. Cost Breakdown includes all rating eras under Placement games, Continuous benchmark, Human games, and Operating expenses. Placement games combine the first rating batch and games for new models. Other LLM benchmark games enter Continuous benchmark. Categories are counted once. The API purpose values are placement, continuous, human, and operating.

Stockfish is for viewers only. Its scores, analysis, and suggested moves never enter player prompts. The evaluation-history chart follows accepted moves, not equal wall-clock intervals. Missing historical evaluations are gaps.

## Policies

- [Privacy](https://chessbench.org/privacy)
- [Terms](https://chessbench.org/terms)
- [Cookies](https://chessbench.org/cookies)

When citing a result, link to its stable match page and include the retrieval date, model configuration, sample size, and whether the result is provisional.