Hex Tic-Tac-Toe AI

Reinforcement Learning · AlphaZero

Scrape two million human moves, bootstrap a network, then let it teach itself by playing itself.

hex illustration

A from-scratch AlphaZero implementation for Infinity Hex Tic-Tac-Toe. I didn't build the game — I built the brain that learns to win it, from scraped human games all the way to self-play reinforcement learning.

2M+training positions
6residual blocks
35×35board encoding
>55%arena gate to promote

The game (not mine)

Infinity Hex Tic-Tac-Toe is a two-player connection game on an infinite hexagonal grid: you win by forming a straight line of exactly six of your own hexes along any of the three axes. Player one opens with a single hex; after that each turn places two. It's a clean, deep game — and it's not mine. It lives at an existing, still-playable site; I have no involvement with it beyond being fascinated by it.

What I built is the AI: an agent that learns this game well enough to play it strongly, using the same recipe that cracked Go.

The AlphaZero pipeline

Scrape2M human moves
Bootstrapsupervised ResNet
Self-playMCTS games
Arenapromote if >55%

The network is a dual-headed PyTorch ResNet — a 2-channel 35×35 board goes into a 6-block residual tower, splitting into a policy head (a probability over all 1,225 board cells) and a value head (a single win/loss forecast). Training runs in two phases:

  • Supervised bootstrap — before any self-play, the network is taught human intuition on 2M+ scraped pro positions, so generation zero already plays sensibly.
  • Self-play RL loop — a persistent generation loop then improves it: self-play generates 100 MCTS-guided games per iteration, a candidate network trains on the freshest data with early stopping, and an arena plays 40 games candidate-vs-champion. The candidate only becomes the new champion if it wins over 55%.

Notable touches

  • Curriculum schedule — MCTS depth ramps from 100 up to 1,000 simulations as the agent matures, and early-game exploration shrinks over generations.
  • Blunder detection & punishment — if the agent has a winning threat and ignores it (or fails to block a clear loss), the game short-circuits with a forced result instead of burning simulation budget.
  • Auto VRAM tuner — probes the GPU for the largest safe batch size before training and scales the learning rate to match, so it runs on whatever card it lands on.

The data

The supervised phase needs human games, so there's a whole harvesting pipeline in front of it — parallel scrapers that collect played games, plus analytics notebooks that study human blunders and Elo-vs-mistake patterns to shape the training set into the 2M-position dataset.

Blunder-vs-Elo analysis from the scraped game database
Self-play arena — candidate vs. champion win rate over generations

Status

Honest version: this one is a finished idea, unfinished project. The full pipeline runs and produces stronger generations, but I stopped before pushing it to its ceiling — and the game itself has gone quiet. I'm keeping it here because the engineering (scrape → bootstrap → self-play → gate) is some of the most involved ML work I've done, even if the scoreboard never got its final number.

Built with

PyTorch (dual-headed residual network), a custom MCTS, ProcessPoolExecutor self-play across CPU cores, CUDA with an automatic batch-size tuner, and a Python scraping + analytics stack (parallel harvesters, Jupyter dashboards) for the human-game dataset.