deepgrain-foundry/docs/TESTING.md

Testing

There are three layers. Use all three.

1. Headless suites (test/t_*.js) run the simulation in Node with no browser. They are fast and deterministic, and they are where most checks live.
2. Browser tests (test/play.py, test/t_smooth.py) drive the real page in headless Chromium with real key, mouse and gamepad input. They check the UI, saving, the smooth look, and that the page throws no errors.
3. Screenshots (tools/shot.py) show what something looks like. Open the PNG and look at it.

Run everything with python3 test/run_all.py --browser. It exits non-zero if anything failed and prints the failing checks.

Headless suites

Each suite is a plain Node script:

// What this suite covers, in a sentence or two.
const { load, suite, flatWorld, rect, count, ticks } = require('./harness');
const { run } = load();                 // loads src/sim/ (the files in src/manifest.json)
const { check, done } = suite();

flatWorld(run, 240, 160, 100);          // 240x160 grains, sky above row 100, stone below
run('money = 5000; bot.x = 60; bot.y = 40;');
run('for (let bx = 20; bx < 30; bx++) placeBlock(BP_BELT, 1, bx, 20, true);'); // true: free, no reach check
rect(run, 84, 60, 95, 79, run('SAND'));
ticks(run, 300);
check('sand rides the belt off its end', count(run, run('SAND'), 120, 80, 239, 99) > 100, count(run, run('SAND')));
done();                                 // prints ALL PASS or N FAILED, sets the exit code

• run(code) evaluates JavaScript inside the simulation and returns the result. Every top-level name in src/sim/ is available. Return plain values (numbers, strings, JSON.stringify(...)).
• Harness helpers:
  • flatWorld(run, w, h, ground);
  • put(run, x, y, m);
  • rect(run, x0, y0, x1, y1, m);
  • count(run, m, x0, y0, x1, y1);
  • maxTemp(run, ...);
  • ticks(run, n);
  • dump(run, x0, y0, x1, y1, withTemp) for an ASCII picture of an area.
• check(name, ok, detail): name the behaviour the player would notice ("a pump moves water over a wall"). Put the measured numbers in detail, so a failure shows how far off it was.
• Coordinates are in grains. Block coordinates (for placeBlock, placeMachineAt) are grains divided by 4.
• Errors name the real file and line (src/sim/pipes.js:42).
• A new suite named test/t_<topic>.js is picked up by run_all.py automatically.

Determinism. The simulation's randomness comes from seeded generators. The same code and the same test give the same output every run, apart from lines with timings in ms. To prove a refactor changed nothing, save every suite's output before and after, and diff them:

for t in test/t_*.js; do node $t > /tmp/before/$(basename $t).txt; done   # before the change
for t in test/t_*.js; do node $t > /tmp/after/$(basename $t).txt; done    # after
diff -r /tmp/before /tmp/after

Browser tests

• test/play.py (about 2.5 minutes) builds a world through the title screen and plays it with real input. It checks the tools, building, copy/cut/paste, undo, the build menu, filters, pipes, the notebook and market, saving and continuing, exporting and importing a world file, the gamepad and world sizes. Screenshots go to shots/play/. Pixel look by default; python3 test/play.py shots/play smooth uses the smooth look.
• test/t_smooth.py checks that the WebGL2 smooth look compiles and draws, that it falls back to pixels cleanly, and that the thermal lens works.
• Run them one at a time. Two headless browsers at once can starve each other, and t_smooth.py's half-resolution check then fails. run_all.py runs them one after the other.
• Rebuild first (python3 build.py). They load dist/index.html. run_all.py --browser rebuilds for you.

Screenshots: tools/shot.py

python3 tools/shot.py shots/silo.png --setup tools/scenes/clear-sky.js \
    --js "$(cat tools/scenes/stone-uses.js)" --ticks 240 --zoom 7 --no-light

• It starts a new world, pauses it at once, runs your setup code in the page, runs --ticks simulation steps, then saves the picture. Because the world is paused otherwise, the same options always give the same picture.

• Setup code can use every game global, plus SX, SY (the bot's starting position). tools/scenes/clear-sky.js clears a flat work area and sets FLOOR, BX, BY and CAM. Set CAM = [x, y] to frame the shot.

• The HUD and the cursor previews are hidden unless you pass --hud or --previews.

• --look smooth for the smooth look. Test in the pixel look unless you are checking graphics.

• --freeze draws the last frame at a fixed time and frame count, so animations are caught at the same moment. Two shots of the same scene then match pixel for pixel, and a before / after pair differs only where something really changed. A quick comparison with Pillow:

  from PIL import Image, ImageChops
  a, b = (Image.open(p).convert('RGB') for p in ('before.png', 'after.png'))
  print(ImageChops.difference(a, b).getbbox())   # None: identical
  

  To compare against an older version, build it somewhere else (for example from a git worktree) and pass that page with --page.

• Page errors are printed with their source file and line, and the exit status is 1.

• Look at the picture. A shot that "ran without errors" can still show the wrong thing.

Pitfalls

• WebGL in headless Chromium. There is no GPU, so the smooth look needs SwiftShader: launch with --use-gl=angle --use-angle=swiftshader --enable-unsafe-swiftshader. Also set window.__allowSoftwareGL = true before the page loads (add_init_script); otherwise the game sees software rendering and switches to the pixel look. tools/shot.py does both.
• Top-level const and let are not window properties. In the page, window.SM is undefined even though SM exists. Write typeof SM !== 'undefined', or use the bare name. Functions declared with function are on window.
• Playwright calls a string that evaluates to a function. page.evaluate("x = function () {}") assigns the function and then calls it. Wrap statements in an arrow function: page.evaluate("() => { ... }").
• Starting a world takes a moment. startNewWorld(seed, w, h) generates on a timer. Wait for GAME.running === true. Then remove the welcome card if it is in the way: document.querySelectorAll('.card, .toast').forEach(e => e.remove()).
• Real-time play is not deterministic. In the browser the simulation runs as fast as frames arrive. For repeatable results, pause (GAME.paused = true) and call simTick() yourself, as tools/shot.py does.
• World sizes offered by the game: 2048×1024, 3072×1280, 4096×1536 grains.
• Rebuild after every source change before any browser test or screenshot. They all load dist/index.html.

Measuring

• tools/perf.py: renderer and tick times in the browser.
• tools/experiments/factory_cost.js: what running factories cost the simulation.
• tools/experiments/flow.js: belt and tube throughput in grains a second.
• tools/experiments/worldgen.js: material counts and timings of world generation.