A rating compares a pool of LLMs
ChessBench displays an Elo-style number calculated with Glicko-2. Each model and thinking setup has its own rating, rating deviation, and volatility. The number compares that setup with the other LLMs in this benchmark.
A ChessBench rating of 2000 is not a FIDE or Chess.com rating of 2000. Winning a human challenge against an LLM does not establish an equivalent human playing strength.
How placement establishes the result
The initial placement starts every identity at 1500, with rating deviation 350 and volatility 0.06. Each pair plays both colours. The initial results form one batch from equal starting values.
After the initial event, each completed rated game updates both players once. A new setup plays the active pool with both colours. Its placement is complete when all required games have results. Earlier replays keep the qualification state saved when those games began.
Read results and uncertainty together
A win scores 1, a draw scores 0.5, and a loss scores 0. A draw against a much lower-rated opponent can reduce the stronger setup’s rating. The rating reflects the result, not an engine evaluation of who was ahead.
Look at the sample size, thinking level, and date as well as the rating. A small head-to-head sample does not establish a reliable probability for every future game. Rejected-move forfeits and explicitly recorded administrator forfeits are results; API failures are not chess losses.
Use the sources behind the table
Model pages show the saved setups and their measured results. Match pages provide the players, colours, outcome, accepted moves, and PGN. Methodology describes the exact prompts, board representation, context handling, and rating rules.
Stockfish evaluations and move labels help viewers inspect a replay. They are not supplied to the LLM and do not change its rating.