Training a Bot Model

An AI bot needs a model: an .onnx network that, given a position, rates the moves and the position. DiceyTable trains one the way AlphaZero did — the bot plays thousands of games against itself with the shared search, a network learns from the results, and the next generation of games is played with the better network.

Two tools, one set of files:

• The training page (/editor/train) runs in your browser on your own machine. Good for small networks and an overnight run; nothing to install. See The training page for every control.
• The trainer CLI (diceytable-bot-trainer) runs on Node, with many CPU cores or a GPU, and trains with PyTorch. It is for anything bigger.

Both read and write the same game-record (.jsonl), weights (.weights.json) and model (.onnx) formats, so games played in the browser can be trained on the command line and a model from either imports through the editor's Manifest ▸ Bots tab.

Before you train: do you need a model?#

Probably not yet. Write a rules kernel with a good heuristic and search it with ai.mcts without a model. Measured on Reversi, that beats a greedy player 90% of the time at 64 simulations and plays level with a depth-4 minimax at 256. A model is only worth shipping if it beats that search at the same number of simulations, and getting there takes real compute:

• A 6-minute browser run on the smallest network (Reversi, 30 generations, measured on a fast desktop) beat random 70% of the time, roughly matched greedy (47%), and won 18% against the heuristic search.
• No overnight browser run has been measured. The design estimate for a 12-hour run on a small network is a few thousand games — "beats greedy, competitive with shallow minimax" is a target, not a measurement.
• A published Reversi-zero run took 8 h 20 min on a datacentre GPU, 72,000 games to reach strong play.

Step 1 — a rules kernel#

Training needs your game as a RulesKernel: pure functions for the starting position, legal moves, applying a move, the end of the game, an encoding of the position for the network (encode and encoding) and a numbering of every possible move (policySize, moveIndex, moveAt).

Declare it in a scene script, in the outermost scope, as const kernel — with no export keyword (a scene script is not a module, and export would stop it loading at the table). The training page finds it by that name in the script's compiled body and copies out that declaration and the helpers it mentions, nothing else. So the kernel and everything it uses must be pure:

• No table API: world, globalEvents, refObject, bots, ai, declareVariables.
• No way out: globalThis, self, window, document, Function, eval, require, process, importScripts, import, and this outside a function.
• No reaching into what objects share — not as a property, and not even as a string: constructor, __proto__, prototype, getPrototypeOf, setPrototypeOf, defineProperty, defineProperties, getOwnPropertyDescriptor(s), getOwnPropertySymbols, __defineGetter__ / __defineSetter__ / __lookupGetter__ / __lookupSetter__, and the stack-trace hooks (prepareStackTrace, captureStackTrace, stackTraceLimit). Rules code builds plain objects and arrays and never needs them.
• No Math.random and no other global outside a short list of language built-ins (Math, Number, Array, Object, JSON, String, Boolean, the typed arrays, Map, Set, the Error types, isFinite, isNaN, parseInt, parseFloat, Infinity, NaN, undefined).
• Names are matched, not resolved: a helper whose parameter is called board pulls in an outer const board = world… and is refused. Rename one of them.

A kernel that breaks a rule is refused as kernel-impure (or kernel-missing / kernel-unparsable) with the name and the compiled line. The extracted code also passes through the publish scanner, which is whole-word and reads comments: a comment mentioning window, parent or fetch is refused too.

Before any self-play, both tools run a contract check on the kernel and stop on any violation: apply changing its input, a non-deterministic apply, a state or move that does not survive JSON, a moveIndex out of range or shared by two legal moves, moveAt(moveIndex(m)) not giving m back, an encode of the wrong length or with a NaN, a heuristic outside -1..1, a terminal result that does not name every seat, a non-terminal position with no legal move, or a game that never ends.

Step 2a — train in the browser#

1. Open your game in the editor, save it (the page reads the scripts as last saved on this device), and on Manifest ▸ Bots choose Train a model…. That opens /editor/train?project=<your project> in the same tab. Without a project, /editor/train opens in demo mode on the built-in Reversi and tic-tac-toe rules.
2. The first time, agree to "Training runs your project's rules code on this device". The kernel runs in background workers outside the table's sandbox. It is checked first; the workers are created inside an isolated frame on a separate site (training.diceytable-sandbox.com) with no access to this one (your other projects, your account, the network); and inside them the browser's built-ins are frozen and every global but a few language built-ins is hidden. On WebGPU a separate worker in that frame drives the graphics card and the rules code can only hand it positions. The code can still use your processor and memory while the run lasts, so train on scripts you wrote, have read, or trust.
3. Pick the kernel, the network size (tiny 4×32, small 6×64 — the default — or medium 6×128), simulations per move (default 64), games per generation (default 96), and how long to run (default 12 hours, at most 24).
4. Press ▶ Start and keep the window open and in the foreground. The page holds a screen wake lock while it runs; closing the tab, a sleeping laptop, or the browser discarding a background tab to save memory (Chrome's Memory Saver) stops it. Exclude the site from Memory Saver for a long run.
5. Each generation — self-play, then training, then evaluation — ends with a checkpoint saved in your browser. A run stopped for any reason resumes from the last checkpoint: open the page again and choose Resume under Unfinished runs. A run is trained by one tab at a time.
6. Watch the strength gauge: every checkpoint plays random, greedy and shallow minimax (for kernels that have them), the heuristic search at the same simulations, and the previous checkpoint. The heuristic-search row is the one that decides whether the model is worth shipping.
7. When a checkpoint beats the heuristic search, enter a model id and label and choose Export checkpoint to project as model…, then Open the editor to add it. The editor's Bots tab offers the exported model and imports it through the normal model import — the same validation as a picked file, the import credit gate, and the file written to ai/<id>.onnx — so Save and Publish include it. Exporting works whether or not an editor tab is open.

Self-play runs on the processor (WebAssembly) by default; WebGPU self-play is offered where the browser has it and is much faster with many games batched together (8–14× on a small network, measured), with every checkpoint cross-checked against the processor before use. For your project's kernel it runs in the training sandbox site's frame: the kernel in one locked-down worker with no access to the GPU, the network on WebGPU in another. If the sandbox site cannot be reached, the page falls back to an isolated frame of its own on WebAssembly only, and says so. The network trains with TensorFlow.js on WebGL inside a worker, falling back to the processor — about 400 times slower, and not practical beyond tiny — where WebGL is missing or fails its self-check.

Nothing is uploaded. Runs live in your browser's private storage until you delete them; Export records (.jsonl) downloads a run's games for the CLI.

Step 2b — train with the CLI#

packages/bot-trainer in the DiceyTable repository builds a command-line tool:

npm run build --workspace @diceytable/bot-trainer
node packages/bot-trainer/dist/cli.js --help

Every command has --help, and the ones a script would read have --json. Exit codes: 0 success, 1 "no" (an invalid file, a refused model, a failed step), 2 a bad command line.

--kernel is reversi, tictactoe, or the path of a compiled kernel module that exports the kernel as kernel or as its default export. The training page writes one for you: in project mode, Export kernel for the CLI… downloads <kernel id>.kernel.mjs, cut from your scene script exactly as the browser trainer uses it. A kernel module is imported and run with your full permissions — there is no sandbox on the command line. Never point it at a file you did not write or read.

Players for eval: random, heuristic (the shared search with no model — exactly a host's fallback; --sims sets its playouts), model:<file.onnx>, and for the built-in Reversi only greedy and minimax:<depth>.

Measured self-play throughput (20-core desktop, Reversi, 64 simulations, 8 CPU threads): tiny 21 games/s, small 3.6, medium 0.94; with DirectML on a GPU, small reaches 18 games/s. 72,000 games of small is about 5 hours on 19 CPU threads or about 70 minutes with DirectML — before training time.

PyTorch caveat. train shells out to python/train.py. Its standard-library half and the file formats are tested, but the PyTorch training loop itself has not yet been executed on a machine with PyTorch. The first run checks itself before training (a layout check against the shared reference forward pass) and stops rather than produce a scrambled model. Publish models written by model emit (or train --emit): they always pass the scanner. PyTorch's own ONNX export may produce a file the scanner refuses.

The browser's run directory uses the same file names as loop (gen-<n>.weights.json, gen-<n>.onnx, records-<n>.jsonl, eval-<n>.json; the browser adds a run.json), and Export records (.jsonl) gives you the browser's games as one file for records validate and train. That works one way: the CLI can continue a browser run, but the browser cannot resume one the CLI extended — run.json is its commit record, and on Resume it deletes every generation above it as the remains of an interrupted step.

Step 3 — measure honestly#

Play the model against the heuristic search at the same simulations:

diceytable-bot-trainer eval --kernel reversi --player model:club.onnx --opponent heuristic --sims 64 --games 200

Beating greedy is not enough — the model-less search already does that 90% of the time. A model that does not beat heuristic at equal simulations makes your bot slower and heavier for nothing.

Step 4 — declare and publish#

Import the model on Manifest ▸ Bots (or let the training page's export hand it over). The import checks it with the scanner's own code, suggests a minTier from its size and your device, and fills in the aiModels entry. Then point an AI bot's modelId at it (or, for a separate network per difficulty, a difficulty's own modelId), pass ctx.modelId to ai.mcts in your bot's handler, give the bot a scripted fallbackBotId, and publish. A model over 45 MiB has to be pushed to your repository with git first.

See also#

• The training page — every control on /editor/train.
• Game bots — tiers, host migration, and what a model is worth.
• RulesKernel — writing the kernel.
• Manifest ▸ Bots — importing the model.