# Operate the house bots

Open `/admin#house` to enable, disable or restart a house policy without restarting the ladder.
Use the search field to find a policy or ladder. **Disable all house bots** drains the whole fleet
for maintenance or recovery from several unavailable policies. M-Bot defaults **off** and is assigned only to
Public. Off creates no identity on a fresh ladder and shows no current leaderboard entry.

## Everyday controls

| action                 | result                                                                                                                                                    |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Enable                 | Starts the policy's worker if needed and connects its assigned ladders. The same bot identity and version history return.                                 |
| Disable                | Removes the current leaderboard entry, stops new games, lets active games finish, then closes those connections. The last connection releases the worker. |
| Restart                | Drains that policy's games and replaces its worker. Use this after fixing code, replacing weights, or a stopped retry circuit.                            |
| Use deployment default | Removes the saved override for this policy.                                                                                                               |

Saved admin choices take precedence over the deployment's defaults and survive restarts,
deployments and unrelated operational-settings edits. Choices use the same revision checks and
audit trail as Settings: a stale edit is rejected and refreshed, never silently retried.

> [!NOTE]
> Disabling keeps bot pages, versions, ratings, matches and historical season standings. Changing
> the roster does not edit an event's four approved achievement opponents. Disabling one of those
> opponents makes its event challenges unavailable until it returns.

## One runner, isolated policies

The ladder launches **one house-runner process**. Each enabled policy gets one worker thread and
loads its code and model once. The worker serves all that policy's ladder connections; every
connection has its own token, match context and seeded random generator. A slow policy blocks
its own worker. A policy exception or worker heap exhaustion restarts that worker; a native
process failure affects the whole house runner, while the ladder remains a separate process.

Status shows the runner's total RSS, worker/connection counts, each connection's worker and heap,
retries, last failure and next retry time. Heap figures describe an isolate, so connections sharing
a worker report the same heap. Do not add them together. RSS includes native and WebAssembly
allocations and is measured for the entire runner.

## Configure capacity and add policies

`house/default.json` is the shipped roster. Set `--house-roster <file>` or
`ARENA_HOUSE_ROSTER_FILE` to select another server-side file. Paths inside it are relative to
the repository root; policy code and weights must already be in the server image.

| setting                | shipped default | effect                                                                       |
| ---------------------- | --------------- | ---------------------------------------------------------------------------- |
| `max_workers`          | 8               | Maximum simultaneously loaded policies, across all ladders.                  |
| `max_connections`      | 256             | Maximum policy/ladder connections.                                           |
| `memory_budget_mb`     | 2048            | Admission budget and runner RSS restart threshold, in MiB.                   |
| `drain_timeout_ms`     | 600000          | Maximum wait for a policy's games to finish before forcing it to stop.       |
| `heap_mb` per policy   | 128             | Worker JavaScript heap limit.                                                |
| `memory_mb` per policy | 256             | Reserved model/worker memory for admission. Counted once across its ladders. |

The runner reserves another 128 MiB. Enabling a roster that exceeds its worker, connection or
memory reservations is rejected before choices change. JavaScript heap limits do not cap native
or WebAssembly allocations. The RSS watchdog samples the runner and restarts it if it exceeds
the budget; this is a recovery threshold, not an OS memory limit. Leave headroom for model loading
and monitor the actual RSS before increasing capacity. Scale CPU capacity as well as memory.

A policy module exports either a `Policy` object (`version`, synchronous `decide`) or a factory
returning one. Its factory runs once per worker. Keep per-match state in `ctx.memory`; module
state is shared by that policy's connections. For example, add this item to `bots`:

```json
{
  "id": "aegis",
  "label": "Aegis",
  "module": "house/aegis.ts",
  "export": "createPolicy",
  "enabled": false,
  "ladders": ["public"],
  "required_files": ["house/weights/aegis.bin"],
  "heap_mb": 128,
  "memory_mb": 384,
  "environment": { "ARENA_POLICY_TEMPERATURE": "0.25" }
}
```

IDs are stable lowercase slugs up to 20 characters; do not reuse an ID for a different opponent.
Use `["*"]` for all enabled, non-closed ladders, or an explicit list. The public name is
`house-<id>`; event names are scoped by ladder. Only Starter, Brawler and Kiter anchor the 1500
rating scale. Additional bound policies float above/below it without changing its anchor.

`optional: true` skips a missing deployment-default policy (as Champion does when its weights are
absent). Explicitly enabling an unavailable policy is rejected. The admin UI cannot upload or
execute arbitrary code: it operates on this validated roster only.

Policy workers receive connection credentials as connection options. Their environment contains
OS variables and explicitly declared `environment` / `inherit_env` settings, limited to
`ARENA_POLICY_*`, `ARENA_CHAMPION_*` and `ARENA_MBOT_*`. Operator tokens, event flags, encryption
keys and global connection/version variables are not inherited. A policy can set `version` in the
roster when an exported model needs an explicit version label.

## Reload and recover

Send `SIGHUP` to the ladder process to validate and reload its roster without restarting it. Invalid
reloads keep the previous configuration. Saved admin choices still win over file defaults. Removed
or disabled policies drain; changed policy definitions restart after draining. When replacing
weights at the same path, use **Restart** so the factory reads the new file.

Restarts use exponential backoff with jitter: roughly 1 second initially, capped near 60 seconds.
A worker crash consumes one shared retry budget across its ladders. After 8 consecutive worker
failures, automatic retries stop; a token/connection failure has its own connection retry budget. A healthy, connected minute resets the
failure count. The roster's `restart` object can override `initial_ms`, `max_ms`, `stable_ms`
and `max_failures`. Startup launches are staggered, and shutdown cancels pending retries before
matches drain. A hung worker is terminated after its drain deadline; unexpected stops remain
visible in status and structured operational logs.

If a saved choice becomes invalid after a deployment, the ladder and admin console still start.
The operations panel reports the rejected plan so staff can disable the unavailable policy.

## M-Bot deployment defaults

Prefer the live admin controls. `--mbot` / `--no-mbot` or `ARENA_MBOT_ENABLED=1` / `0` select
M-Bot's deployment default; a saved admin choice takes precedence. Unset/empty uses the roster
(default off). Changing a container environment requires recreating it; a plain Docker restart
keeps the previous environment. `ARENA_MBOT_TEMPERATURE` defaults to 0.25 and
`ARENA_MBOT_PRECISION` to int8 (or `float32`). Champion has independent settings.

## Admin API

All operations require an admin bearer token and are served with private, non-cacheable responses.

| method and path                      | request                                                                                                  |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| `GET /api/admin/house`               | Current settings revision, overrides and runner status; `runtime: null` when `--house` is absent.        |
| `PUT /api/admin/house`               | `{ "revision": 3, "bots": { "mbot": true } }`; `false` disables, `null` restores the deployment default. |
| `POST /api/admin/house/mbot/restart` | `{ "revision": 4 }`; drains and restarts the policy.                                                     |

The regular settings export/preset format includes optional `house` overrides. Older edits that
omit this field preserve the current choices. Explicit `"house": {}` resets every saved override.
