Chess Analytics Engine
Upload a batch of games and get two independent model scores per game for signs of engine assistance, streamed to the browser one game at a time as each finishes.
Overview
Each finished game becomes one feature vector. A PyTorch network and an XGBoost model score that vector in separate processes, and the two scores are joined into one result. Showing both side by side makes disagreement visible: a game where the models disagree deserves a closer look, not a verdict.
The scores run from zero to one, but they are not calibrated probabilities, and the app says so. They flag games for review; they do not prove cheating.
How it works
- Flask accepts a PGN upload or paste, spools it to private disk storage rather than holding it in memory, and records a job in PostgreSQL.
- An ingestion worker publishes every move to a chess-moves topic on Kafka.
- A Go service extracts per-move features, and an aggregation stage turns them into one row per game: the mean, standard deviation, minimum, maximum and last value of every feature, plus the move count.
- The PyTorch and XGBoost workers each score the game. Neither imports the other, and Flask imports neither.
- A result worker joins the two scores, stores them, and writes an ordered event that the browser receives over Server-Sent Events.
Decisions worth explaining
The same maths in training and in production
The live workers originally did not normalise features the way training did, so the models saw numbers on a different scale from the ones they learned on. This is called training/serving skew. Now training saves the exact XGBoost min/max values (from training rows only) to a file the live worker checks and reuses, and the PyTorch worker repeats the same per-game L2 normalisation.
Split by game, not by move
Moves from one game are strongly correlated. If some moves of a game land in training and others in testing, the test score looks better than the model really is. Grouping by game before splitting keeps every held-out game fully unseen.
PostgreSQL instead of SQLite
Several workers claim jobs at once. PostgreSQL's FOR UPDATE SKIP LOCKED lets each worker grab a different job without blocking the others, and the score updates and browser events commit in the same transaction. SQLite allows only one writer at a time and is awkward to share between machines.
Streams that survive a refresh
Every event the browser sees is a numbered row in the database. After a reload, the page fetches the current snapshot and then resumes the stream after the last event it saw, so a big job neither replays its whole history nor loses updates.