# VM Build System

## Overview

VM provisioning uses a modular driver + profile pattern. One driver script handles
the full build pipeline; per-VM profile files declare what makes each machine
distinct. Adding a new VM means writing one profile file — no changes to the driver.

## Structure

```
tools/vm/
  build-vm.sh              # Driver — sources a profile and runs the build pipeline
  build-workstation.sh     # Wrapper → build-vm.sh profiles/workstation.sh
  build-web-server.sh      # Wrapper → build-vm.sh profiles/web-server.sh
  build-build-machine.sh   # Wrapper → build-vm.sh profiles/build-machine.sh
  profiles/
    workstation.sh         # sc-workstation / ares    — XFCE desktop (Debian)
    web-server.sh          # sc-web-server  / hermes  — nginx app server (Debian)
    build-machine.sh       # sc-build-machine / vulcan — build toolchain (Arch)
  lib/
    common.sh              # Shared libvirt helpers (pool, domain, seed ISO, wait-for-IP)
```

## Invocation

```bash
# By wrapper (backwards-compatible)
./build-workstation.sh [--dry-run] [--force]
./build-web-server.sh  [--dry-run] [--force]
./build-build-machine.sh [--dry-run] [--force]

# By driver directly — profile name (no extension) or explicit path
./build-vm.sh workstation [--dry-run] [--force]
./build-vm.sh profiles/web-server.sh --force
```

`--dry-run` skips all libvirt/qemu-img calls and prints what would run.
`--force` destroys and recreates a domain that already exists.

## Profile Contract

A profile is a bash file sourced by `build-vm.sh`. It must set these variables:

| Variable | Example | Description |
|----------|---------|-------------|
| `DOMAIN` | `sc-web-server` | libvirt domain name |
| `HOSTNAME` | `hermes` | Guest hostname |
| `RAM_MB` | `512` | Memory in MB |
| `VCPUS` | `1` | vCPU count |
| `DISK_SIZE` | `8G` | qcow2 overlay size |
| `GRAPHICS` | `vnc` | `vnc`, `spice`, `spice-qxl`, or `none` |
| `BASE_URL` | `https://...` | URL to download base cloud image from |
| `BASE_IMAGE` | `$SC_BASE_DIR/...` | Local path to cache the base image |

It must also define `generate_user_data()` — a function that prints the complete
cloud-init `#cloud-config` YAML to stdout. The driver calls this function and writes
the output to the seed ISO. The following variables are available when the function
runs (set by the driver after sourcing the profile):

| Variable | Value |
|----------|-------|
| `PUBKEY` | Contents of `${SC_SSH_KEY}.pub` |
| `GAME_HOST_IP` | `${SC_GAME_HOST_IP:-10.42.0.1}` |
| `POOL_DIR` | Resolved libvirt pool path |
| `DISK_PATH` | `$POOL_DIR/${DOMAIN}.qcow2` |
| `SEED_ISO` | `$SC_SEED_DIR/${DOMAIN}-seed.iso` |

Profile-specific variables (e.g. `HUD_URL`, `SAGE_URL`, `PRIVKEY_INDENT`) are set
in the profile before `generate_user_data` is defined and are available inside it.

## Writing a New Profile

1. Copy `profiles/web-server.sh` as a starting point.
2. Set the 8 required variables.
3. Write `generate_user_data()` with the cloud-init YAML for the new machine.
4. Run `./build-vm.sh profiles/my-new-vm.sh --dry-run` to validate.
5. Run without `--dry-run` to build.

No changes to the driver or any other file are needed.

## Build Pipeline (driver)

1. Parse `--dry-run` / `--force` flags
2. Resolve and source the profile file
3. Validate required variables and `generate_user_data` function exist
4. Source `lib/common.sh` (sets `SC_*` env, exposes helpers)
5. Run `ensure_vm_tooling` (checks virsh, qemu-img, virt-install, SSH keys, pool/network)
6. If domain exists and `--force` not set: exit cleanly
7. `download_if_missing` — fetch base image if not cached
8. Call `generate_user_data` → write to tmpdir, build NoCloud seed ISO
9. `destroy_domain` — remove existing domain if present
10. `create_backing_disk` — qcow2 overlay over the base image
11. `build_import_domain` — `virt-install --import`, enable autostart
12. `wait_for_agent_ip` — poll QEMU guest agent for IP (up to 300 s)
13. Cleanup tmpdir on exit (trap)

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `SC_GAME_HOST_IP` | `10.42.0.1` | Host machine IP on the game network |
| `SC_SSH_KEY` | `~/.ssh/sc_host_key` | SSH key pair used for all host→guest connections |
| `SC_BASE_DIR` | See `common.sh` | Where base cloud images are cached |
| `SC_SEED_DIR` | See `common.sh` | Where cloud-init seed ISOs are written |
| `SC_POOL_NAME` | `sc-images` | libvirt storage pool |
| `SC_NETWORK_NAME` | `sc-internal` | libvirt network |
| `LIBVIRT_DEFAULT_URI` | `qemu:///system` | Override to `qemu:///session` for user-mode libvirt |
| `SC_WORKSTATION_GRAPHICS` | `spice` | Override workstation graphics backend |

## Current VMs

| Profile | Domain | Hostname | OS | RAM | vCPUs | Disk | Graphics |
|---------|--------|----------|----|-----|-------|------|----------|
| `workstation.sh` | `sc-workstation` | `ares` | Debian 12 | 2048 MB | 2 | 20 G | SPICE |
| `web-server.sh` | `sc-web-server` | `hermes` | Debian 12 | 512 MB | 1 | 8 G | VNC |
| `build-machine.sh` | `sc-build-machine` | `vulcan` | Arch Linux | 768 MB | 2 | 10 G | VNC |

## Hostname Resolution

All VMs resolve internal hostnames via static `/etc/hosts`. There is no DNS server
on the game network — this matches how small company networks often work before a
proper internal DNS is set up.

Each VM only has entries for the hosts it needs to reach:

- **ares** (workstation): knows `hermes`, `vulcan`, `portal.axiomworks.internal`, `sage.axiomworks.internal`
- **hermes**: knows `portal.axiomworks.internal`
- **vulcan**: knows `hermes` (deploy target), `portal.axiomworks.internal`

The `.axiomworks.internal` domain is fictional but realistic — real companies use
private suffixes like `.internal` or `.corp` for their infrastructure.

## Networking Notes

- All VMs attach to the `sc-internal` libvirt network
- The host machine (10.42.0.1) serves the game portal (`:3000`) and Sage KB (`/sage/`)
- Fixed IPs used in `/etc/hosts` across VMs: hermes=10.42.0.40, vulcan=10.42.0.24
- These must match the DHCP reservations configured in `network-sc-internal.xml`
- IPv6 disabled on all VMs (sysctl) — not needed, reduces noise

## Performance Tuning

All VMs share a common sysctl baseline applied via `/etc/sysctl.d/`:

| Setting | Value | Rationale |
|---------|-------|-----------|
| `vm.swappiness` | 10 | Prefer RAM; swap only under real pressure |
| `vm.vfs_cache_pressure` | 50 | Keep inode cache warm longer |
| `vm.dirty_ratio` | 15–25 | Batch writes; vulcan higher for build workloads |
| IPv6 disabled | — | Removes unnecessary network overhead |

All VMs have a swap file (512 MB – 1 GB depending on role) created at first boot.

## DHCP Reservations and MAC Addresses

Fixed IPs are set via DHCP reservations in `network-sc-internal.xml` and the live
libvirt network. The reservations reference MAC addresses, which virt-install
**generates fresh on every `--force` rebuild**. After any rebuild, the old
reservation is stale and the VM will get a random IP from the pool.

After a `--force` rebuild, update the reservations:

```bash
# 1. Get the new MAC
virsh domiflist sc-web-server   # (or sc-workstation, sc-build-machine)

# 2. Remove the old reservation (use the old MAC from network-sc-internal.xml)
sudo virsh net-update sc-internal delete ip-dhcp-host \
  "<host mac='OLD_MAC' name='hermes' ip='10.42.0.40'/>" --live --config

# 3. Add the new one
sudo virsh net-update sc-internal add ip-dhcp-host \
  "<host mac='NEW_MAC' name='hermes' ip='10.42.0.40'/>" --live --config

# 4. Update network-sc-internal.xml to match
```

The VM will pick up the reserved IP on its next DHCP renewal (or reboot).

### Current reservations

| VM | Domain | Hostname | MAC | IP |
|----|--------|----------|-----|----|
| Workstation | sc-workstation | ares | `52:54:00:bd:aa:29` | 10.42.0.36 |
| Web server | sc-web-server | hermes | `52:54:00:49:9b:64` | 10.42.0.40 |
| Build machine | sc-build-machine | vulcan | `52:54:00:5e:9f:b9` | 10.42.0.24 |