nest/USAGE.md

# UVM Usage Guide

This guide explains how to install, operate, and troubleshoot UVM as an
operator. It covers the command-line tool, the optional FastAPI server, common
scenarios, and expected limitations.

For the project architecture and contributor guidance, see
[DEVELOPER.md](DEVELOPER.md). For a concise overview, see [README.md](README.md).

## 1. What UVM Does

UVM creates small Linux microVMs through Firecracker and KVM on one Linux host.
Each VM receives:

- A unique VM ID and, optionally, a unique local name.
- A private writable root disk copied from a shared template.
- A private IP address and MAC address.
- A TAP interface connected to a host bridge.
- A dedicated Firecracker process, API socket, log, and config file.
- A managed private-IPv4 TCP allowlist, initially allowing port 22 unless
  default ports are explicitly disabled.

By default, guest networking uses:

```text
Guest network: 10.42.0.0/24
Gateway:       10.42.0.1
Bridge:        uvm0
Guest pool:    10.42.0.2 through 10.42.0.254
```

The default locations on the host are under `/var/lib/uvm`. Stop/start and
restart reuse the private disk; delete removes it. UVM is a local CLI, not an
autostart service or a daemon that continuously repairs VM or firewall state.

## 2. Before You Start

### Host Requirements

You need:

- Linux, preferably Ubuntu or a compatible distribution.
- Python 3.12 or newer.
- KVM enabled and readable/writable at `/dev/kvm`.
- Linux >= 5.3 with native bridge conntrack and safe pidfd support.
- Root access for installation and VM lifecycle operations.
- An x86_64 host for the default guest kernel/rootfs downloads.
- Internet access if downloading Firecracker and guest assets.
- `iproute2`, `iptables`, `nftables`, `conntrack`, `e2fsprogs`, `util-linux`,
  `openssl`, and `openssh-client`; `install` installs the host packages.
- An STP-disabled software bridge without traffic offload around its hooks.

Check virtualization before installing:

```sh
ls -l /dev/kvm
lscpu | grep -i virtualization
```

### Source Checkout Setup

From this project root:

```sh
./uvm.py --help
python3 -B -m uvm --help
```

To install the optional HTTP API dependencies and the console command:

```sh
python3 -m pip install .
uvm --help
```

The rest of this guide uses `./uvm.py` so it works from a source checkout. If
you installed the package, replace `./uvm.py` with `uvm`. For the API, use the
Python environment containing the installed dependencies; under `sudo`, use
the full path to that environment's console script when necessary.

### Privilege Rules

| Operation | Root required? |
|---|---|
| `install`, `create`, `start`, `stop`, `restart`, `delete`/`destroy` | Yes |
| `ports open/close/set/sync` | Yes |
| `list`, `inspect`, `ports list`, dry-run target lookup | Access to the default root-owned registry requires root |
| `ssh`, `logs` | Access to the protected registry/runtime files requires root |
| `doctor` | Run as root for meaningful privileged checks |
| API server that manages VMs | Yes |

The registry contains plaintext guest credentials and is mode `0600`. Direct
SSH to a known guest IP does not require root.

## 3. Command Overview

The following is syntax notation, not a shell script. `VM` means an exact VM
ID, local name, or guest IPv4 address. Flags are command-specific; in
particular, create has no `--dry-run`, and start has no `--timeout`.

```text
uvm install [--force]
uvm create [--name NAME] [--cpu CPU] [--ram RAM] [--guest-ip GUEST_IP]
           [--username USER] [--password PASSWORD | --password-stdin]
           [--ports PORTS]... [--no-default-ports] [--json]
uvm list [--status STATUS]... [--json | -q | --quiet]
uvm inspect VM [--json]
uvm start (VM [VM ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm stop (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm restart (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm delete (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm destroy (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm ssh VM [--user USER] [--key PATH] [--port PORT] [--insecure-host-key] [-- COMMAND ...]
uvm logs VM [--tail N] [--follow | -f]
uvm doctor [--json]
uvm ports list (VM | --all) [--json]
uvm ports open VM --ports PORTS [--ports PORTS ...] [--dry-run] [--json]
uvm ports close VM (--ports PORTS [--ports PORTS ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm ports set VM (--ports PORTS [--ports PORTS ...] | --none) [--yes | -y] [--dry-run] [--json]
uvm ports sync (VM [VM ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm --serve [--host HOST] [--port PORT]
```

`--host-ip` is an alias for `--guest-ip`; neither refers to the host's public
or uplink address. `--ports` and `list --status` are repeatable flags, not
space-separated positional lists. `destroy` is an alias for `delete`.

JSON is written to stdout; warnings, errors, prompts, and host-command traces
go to stderr. VM JSON omits passwords. Lifecycle and sync results are arrays of
`id`, `action`, `outcome`, and `error`; dry runs return an object with
`dry_run`, `action`, and `targets` (port edits also include before/after
`changes`). `ports list --json` returns an array even for one VM. JSON does
not disable confirmations, and errors may still be plain text on stderr.

## 4. Install UVM Assets

UVM installs host packages, Firecracker, Jailer, a guest kernel, and a guest
rootfs template. It requires SHA-256 checksums by default because it downloads
artifacts that will be used by a privileged process.

### Scenario: Strict First Installation

Obtain trusted checksums independently, then run:

```sh
sudo env \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
  ./uvm.py install
```

Installation prints the Firecracker, kernel, and rootfs paths under
`/var/lib/uvm`. Checksums and verification provenance are recorded in
`/var/lib/uvm/integrity.json`.

### Scenario: Refresh Firecracker or Guest Assets

Use `--force` after changing a URL, replacing an image, or rotating checksums:

```sh
sudo env \
  UVM_FIRECRACKER_SHA256='<new-firecracker-archive-sha256>' \
  UVM_KERNEL_SHA256='<new-kernel-sha256>' \
  UVM_ROOTFS_SHA256='<new-rootfs-sha256>' \
  ./uvm.py install --force
```

`--force` downloads to temporary files and only replaces the existing artifact
after checksum verification succeeds.

### Scenario: Local Experimentation Without Checksums

This mode is unsafe for production; use it only when deliberately accepting
unverified development artifacts:

```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
```

An unverified manifest is not publisher trust. Reinstall with trusted digests
to establish a verified installation. Until then, retain the explicit opt-out
on later create/start/restart commands and on an API server that creates VMs:

```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py create
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py start VM
```

### Installation Failures

| Message | Meaning and action |
|---|---|
| `/dev/kvm does not exist` | Enable hardware/nested virtualization first. |
| `checksum is required` | Provide the requested `UVM_*_SHA256` variable. |
| `SHA-256 mismatch` | Stop and obtain the correct digest or artifact URL. |
| `unsupported host architecture` | Use x86_64 defaults or provide compatible assets. |
| `default guest assets support x86_64 only` | Configure both custom kernel and rootfs URLs for ARM. |

## 5. Create VMs

### Scenario: Create a Default VM

The default is one CPU, 512 MiB RAM, username `root`, password `root`, and an
allowlist containing private TCP/22:

```sh
sudo ./uvm.py create
```

Output includes the ID, optional name, guest IP, resources, TAP, username, and
private port rules. The default password produces a warning on stderr. Use
`--json` for a password-free VM object instead of the human-readable summary.

### Names and Credentials

```sh
sudo ./uvm.py create --name web --username root --password '<guest-password>'
```

Names are optional, unique within the registry, and case-sensitive. They must
be 1-63 lowercase letters, digits, or hyphens, starting with a letter. Names
that impersonate generated IDs (`vm-` plus 32 hexadecimal characters) are
rejected. Names and IPs stay reserved while a VM is stopped. Names do not
create DNS records, and identifiers are not matched by ID prefix.

`--username` must name an account already present in the guest image. A
`--password` value is visible in process arguments and may enter shell
history. Prefer `--password-stdin`, which reads one line, strips the trailing
line ending, and rejects empty passwords, control characters, or passwords
longer than 128 characters. It cannot be combined with `--password`.

This Bash example prompts without echoing the password and avoids putting a
literal secret in the command line or history. Do not enable shell tracing:

```bash
sudo -v
read -r -s -p 'Guest password: ' guest_password
printf '\n'
printf '%s\n' "$guest_password" | sudo ./uvm.py create --name app \
  --cpu 1 --ram 1G --guest-ip 10.42.0.10 \
  --ports 80:HTTP,443:HTTPS --password-stdin
unset guest_password
```

UVM still stores the password in plaintext in the root-only registry. Never
reuse a host/service password; replace `root` / `root` outside throwaway tests.

### Initial Ports

```sh
# Adds 80 and 443 to the default private TCP/22 rule.
sudo ./uvm.py create --name frontend --ports 80:HTTP --ports 443:HTTPS

# Exactly TCP/443, with no SSH allowance.
sudo ./uvm.py create --name tls-app --ports 443:HTTPS --no-default-ports

# Managed empty service allowlist, not an unmanaged VM.
sudo ./uvm.py create --name worker --no-default-ports
```

`--ports` uses colon syntax `PORT[:TYPE]`, not slash notation or host:guest
port mappings. See Private Port Policies below for live edits and network
scope. UVM does not install the services named by these labels.

### Scenario: Choose CPU and RAM

```sh
sudo ./uvm.py create --cpu 2 --ram 2G
sudo ./uvm.py create --cpu 0.5 --ram 512
```

RAM is in MiB by default. Supported suffixes include `B`, `K`, `M`, `G`,
`KiB`, `MiB`, and `GiB`.

UVM enforces a minimum of 128 MiB. Firecracker uses whole vCPUs. A fractional
request such as `--cpu 0.5` is accepted but rounds the VM up to one Firecracker
vCPU; it does not currently apply a host CPU quota.

### Scenario: Request a Specific Guest IP

```sh
sudo ./uvm.py create --guest-ip 10.42.0.20
# The legacy spelling --host-ip has the same meaning.
```

The address must be a private RFC1918 IPv4 address inside the configured guest
network. It cannot be the network address, broadcast address, or gateway, or
already belong to another persisted VM (including a stopped one).

### What Happens During Create

UVM performs these operations in order:

1. Checks root access, KVM, artifact integrity, and nftables/conntrack support.
2. Reserves identity, name/IP, and initial port policy in the registry.
3. Copies the template into a private disk and provisions credentials and SSH host keys.
4. Ensures bridge/egress networking and creates an owned TAP attached but down.
5. Installs and verifies the nftables bridge policy, then activates the TAP.
6. Starts Firecracker, persists process identity, and configures/boots the guest.
7. Confirms policy enforcement and records `running`.

On failure, UVM attempts containment and cleanup. It does not release the
disk or IP reservation if safe process exit or cleanup cannot be confirmed;
inspect the retained `failed` record and `last_error` before retrying.

## 6. Inspect VMs

```sh
sudo ./uvm.py list
sudo ./uvm.py list --status running --status dead --json
sudo ./uvm.py list --status stopped -q
sudo ./uvm.py inspect web
sudo ./uvm.py inspect 10.42.0.10 --json
```

List includes names and policy status. `--status` filters by **observed**
status and may be repeated; `-q`/`--quiet` prints IDs only and is mutually
exclusive with `--json`. Inspect includes retained disk/log/config paths,
`disk_ready`, pending `operation`, and `last_error`, without the password.

Status meanings:

| Status | Meaning |
|---|---|
| `starting` | Create/start is in progress or was interrupted. |
| `running` | Firecracker started; this is not an SSH/application readiness probe. |
| `stopping` | Stop is in progress or was interrupted. |
| `stopped` | VMM and TAP are gone; disk remains. |
| `terminating` | Deletion is pending. |
| `failed` | A lifecycle operation or cleanup failed. |
| `dead` | Observed-only status: stored state says running but the VMM is proven gone. |
| `unknown` | Process identity cannot be established safely; do not guess a PID to kill. |

Stored `status` and `observed_status` can differ. Inspection does not restart,
adopt a legacy process, or reconcile a VM. Policy status is a separate snapshot:

| Policy Status | Meaning |
|---|---|
| `enforced` | The actual owned rules and required bridge attachment settings match. |
| `inactive` | Managed VM is cleanly stopped; saved rules apply on next start. |
| `unmanaged` | Legacy record has no managed policy. |
| `missing` | Expected managed nftables table is absent. |
| `drift` | Actual rules, ownership, or bridge attachment settings differ. |
| `quarantined` | The inspected quarantine policy blocks the VM's TAP traffic. |
| `unknown` | Inspection failed or metadata could not establish policy status. |

`enforced` is not a service-connectivity test or a lasting guarantee against
external rule changes. Mutation JSON may report policy `unknown` without a
fresh observation; use inspect or ports list for a current readback.

### Logs and Doctor

```sh
sudo ./uvm.py logs web
sudo ./uvm.py logs web --tail 50
sudo ./uvm.py logs web --tail 0 -f
sudo ./uvm.py doctor
sudo ./uvm.py doctor --json
```

Logs reads the local Firecracker stdout/stderr log, defaulting to the last
100 lines. `--tail` takes a non-negative integer; `--follow`/`-f` watches for
new output until interrupted. It is not an in-guest log API.

Doctor checks root/KVM access, guest templates and integrity, the non-committing
nftables capability probe, and required command availability. It returns a
nonzero status if any check fails. It does not repair VMs or rules, boot a
guest, or prove SSH/egress connectivity. Use inspect for per-VM policy health.

## 7. Connect with SSH

UVM provisions a password for an existing account. The default credentials are
`root` / `root`. Your guest image must include:

- An SSH server.
- The requested login user plus standard passwd and shadow files.

UVM replaces conventional RSA, ECDSA, and Ed25519 SSH host keys in the private
disk during creation and removes Firecracker's publicly known demo login key.

This is applied only while creating a new VM. Start/restart preserves the
existing disk, credentials, and host keys; legacy disks are not changed
retroactively.

```sh
sudo ./uvm.py ssh web
ssh root@10.42.0.2
sudo ./uvm.py ssh web --user ubuntu --key ~/.ssh/id_ed25519 --port 2222
sudo ./uvm.py ssh web -- uname -a
```

By default, UVM uses SSH `StrictHostKeyChecking=accept-new` and a stable
host-key alias derived from the VM ID. This lets a guest IP be reused later
without confusing it with the prior VM's known-hosts entry.

The key must already be authorized by the guest; UVM does not inject it.
`--port` selects the guest SSH port (default 22), not a host mapping. The
guest's sshd and private allowlist must allow that port. SSH does not feed the
stored password to the client. A remote command must follow a literal `--`;
UVM SSH options can appear before or after the VM identifier.

Only for troubleshooting a guest you trust, disable host-key verification
for one connection:

```sh
sudo ./uvm.py ssh web --insecure-host-key
```

## 8. Start, Stop, Restart, and Delete

### Scenario: Stop a VM and Keep Its Disk

```sh
sudo ./uvm.py stop web --timeout 5
```

Stop quarantines/deactivates the TAP, terminates Firecracker, removes the TAP,
cleans up VM-scoped connection tracking, and removes the owned policy table.
The private disk, config, log, credentials, name, IP, and saved ports remain.

**Stop is not a guest-coordinated shutdown.** UVM sends SIGTERM to the VMM,
waits, and sends SIGKILL if needed. `--timeout` accepts finite non-negative
seconds, including fractional values, and defaults to `5`. It controls the
SIGTERM wait, not the whole command: confirmed exit after SIGKILL and resource
cleanup can take additional time. `--timeout 0` skips the TERM wait, not exit
confirmation. Flush/quiesce important guest workloads separately before stop,
restart, or delete; retaining a disk does not guarantee application consistency.

### Boot or Restart the Retained Disk

```sh
sudo ./uvm.py start web
sudo ./uvm.py restart web --timeout 10
```

Start boots the same complete private disk, without recopying the template,
resetting passwords, or regenerating SSH host keys. Guest changes survive;
guest RAM/process state does not. The verified kernel and VMM, KVM, firewall
support, and compatible network settings are still required. Removing the
original rootfs template does not prevent retained-disk start, although doctor
checks templates and can report it missing.

Start on an already running, identified VM synchronizes its managed port
policy without rebooting. Restart is stop followed by start under the same
operation lock. Unknown process identity, an incomplete/unsafe disk, or pending
deletion blocks a new boot; inspect the error rather than editing registry
flags. An interrupted boot with a live VMM must be stopped before start.

### Scenario: Permanently Remove a VM

```sh
sudo ./uvm.py delete web --dry-run
sudo ./uvm.py delete web -y
# Equivalent destructive command: sudo ./uvm.py destroy web -y
```

Delete performs the stop cleanup, then removes the runtime directory/private
disk and registry record, releasing the name and IP. `destroy` is the same
command with the same flags. Back up data first. A failed deletion remains
pending; retry delete rather than trying to resurrect it with start.

### Multiple VMs and Confirmation

```sh
sudo ./uvm.py stop web app --timeout 10 --json
sudo ./uvm.py start web app
sudo ./uvm.py restart --all --dry-run --json
sudo ./uvm.py restart --all -y
sudo ./uvm.py stop --all -y
sudo ./uvm.py start --all -y
sudo ./uvm.py delete --all --dry-run
sudo ./uvm.py delete --all --timeout 5 -y --json
```

Start/stop/restart/delete accept explicit identifiers **or** `--all`, never both.
Targets are resolved and deduplicated to a fixed ID snapshot before any prompt.
Later-created VMs are not added, and a reused name/IP is not retargeted. If
initial lookup fails, nothing is attempted. Once running, the batch reports
each result and continues after ordinary per-VM failures; it is not all-or-nothing.
An interrupt stops the batch and marks remaining targets skipped.

Every delete and every mutating `--all` selection requires confirmation when
there are targets. Use `--yes`/`-y` for scripts; piping `yes` is not sufficient
because the prompt requires a TTY. `--dry-run` previews without prompting or
performing lifecycle mutations, but is not a boot/firewall preflight. Empty
`--all` selections succeed without prompting. Exit codes are `0` for success
or cancellation, `1` for operational/batch failure, `2` for invalid CLI input,
and `130` for interruption; inspect per-target JSON outcomes as well.

## 9. Private Port Policies

### Syntax and Scope

Use `PORT[:TYPE]`, such as `22`, `80:HTTP`, `443:HTTPS`, and `8080:TCP`.
Ports must be integers from 1 to 65535. CLI labels are case-insensitive and
normalized to `TCP`, `HTTP`, or `HTTPS`; all three use TCP transport. Repeat
`--ports` or separate entries with commas. Identical entries are deduplicated;
conflicting labels for the same port in one request are rejected. Slash
syntax, ranges, UDP labels, and host:guest mappings are not accepted.

Opening a port permits service traffic to the **guest private IPv4 address**
through the UVM bridge/TAP policy, including host-to-guest and VM-to-VM paths.
It does not start a listener, bypass the guest firewall, or configure HTTP or
TLS. Public connectivity, host-port mappings/DNAT, DNS publication, reverse
proxies, and guest-service TLS management are out of scope. External routing
or NAT can change reachability; a private IP is not by itself an exposure
guarantee, and UVM does not integrate with external firewall managers yet.

UDP and IPv6 service policies are not yet supported. Unsolicited inbound UDP
is blocked, as is IPv6 in both directions on managed TAPs. Guest-initiated IPv4 TCP/UDP (including
DNS queries), replies to that egress, ARP, ICMP echo, and required related ICMP
errors remain allowed. An empty TCP allowlist is not a complete network
disconnect. Other VMs' policies still apply to guest-to-guest egress.

### List, Open, Close, and Set

```sh
sudo ./uvm.py ports list web --json
sudo ./uvm.py ports list --all
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
sudo ./uvm.py ports close web --ports 80
sudo ./uvm.py ports close web --ports 22 --dry-run --json
sudo ./uvm.py ports close web --ports 22 -y
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
sudo ./uvm.py ports close web --all -y
sudo ./uvm.py ports set web --none -y
```

- `list` inspects one VM or all VMs; it does not repair rules.
- `open` adds missing ports, preserving other ports and existing labels.
- `close` removes by port number, regardless of label; closing an absent port
  is harmless. `close VM --all` means all ports of **one VM**, not all VMs.
- `set` replaces the entire policy, including labels. `--none` is an explicit
  empty managed allowlist. It is not legacy/unmanaged mode.

Only create adds TCP/22 by default. Set/open/close never add SSH implicitly;
include `22` yourself when replacing a policy if you want to keep SSH.
Open/close/set support `--json` and `--dry-run`. Close/set accept `--yes`/`-y`:
set, close-all, and removing TCP/22 prompt unless approved (an already empty
managed policy can be a no-op). Ordinary open has no confirmation flag.

For a running VM, edits persist the desired policy and apply it atomically.
For a cleanly stopped VM, they update saved metadata for the next start.
After a successful close, subsequent packets on already established service
connections are dropped in **both directions**, as are new connections.
There is no guaranteed TCP RST, immediate socket close, or application-level
disconnect. Live edits retain conntrack state and preserve unrelated egress.

### Explicit Sync and Recovery

```sh
sudo ./uvm.py ports sync web app --dry-run --json
sudo ./uvm.py ports sync web
sudo ./uvm.py ports sync --all -y --json
```

Sync reapplies the saved managed policy to running VMs; it does not start
stopped VMs. It accepts multiple identifiers or `--all`, plus `--dry-run`,
`--json`, and `--yes`/`-y`. `--all` uses the same confirmation and fixed-target
batch behavior as lifecycle commands.

UVM has no background firewall reconciler. If another administrator/tool
removes or changes UVM rules, inspect status and explicitly run `ports sync`
after resolving the external conflict. A failed policy update retains
the new intent with a pending `operation` and attempts TAP deactivation and
quarantine; it does not restore older, more permissive rules. Inspect
`last_error`, fix the underlying failure, then retry sync. Read-only status
commands and starting the API server do not repair lost rules.

### Legacy VM Migration

The default registry is `/var/lib/uvm/state.json`, schema version `2`. Older
registries are accepted and migrated on the next write. Missing/null `ports`
stays `None`/`unmanaged`, not an empty policy and not default SSH enforcement.
Open/close/sync/start do not silently adopt a policy for such a VM:

```sh
sudo ./uvm.py inspect VM --json
sudo ./uvm.py stop VM
sudo ./uvm.py ports set VM --ports 22,80:HTTP -y
sudo ./uvm.py start VM
```

Choose the actual required ports, or use `ports set VM --none -y`. Policy must
first be set while the legacy VM is stopped. Start also requires a safe,
completely provisioned private disk; very old records without one cannot be
resumed by copying a new template over their data.

A live legacy PID missing only the host boot ID can be adopted during a
locked mutation only if its saved start time, running Firecracker executable,
exact API-socket command line, and protected binary/socket paths match. A
missing start token, inaccessible evidence, or any ambiguity fails closed;
UVM will not blindly signal that PID or delete a possibly in-use disk. Listing
does not perform adoption. Inspect such failures rather than manually
inventing identity fields in `state.json`.

## 10. Use the FastAPI Server

The API uses the same local lifecycle services, but exposes only the existing
health/install/create/list/inspect/stop/delete endpoints. There are no new
start/restart, port-management, or bulk HTTP endpoints; SSH, logs, and doctor
also remain CLI-only. The server does not autostart VMs or reconcile rules,
and is not safe to expose directly to the public internet.

### Start a Local Server

After installing the project dependencies, start the server as root:

```sh
export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  ./uvm.py --serve --host 127.0.0.1 --port 8000
```

Liveness is at `http://127.0.0.1:8000/health`; OpenAPI docs are at
`http://127.0.0.1:8000/docs`. The default bind is `127.0.0.1:8000`.

The API token is required even for a loopback server. All VM and installation
routes require it. FastAPI's informational `/docs`, `/redoc`, and
`/openapi.json` endpoints are also available without a token; they do not
perform host mutations. Tokens must be visible ASCII with no whitespace.
Use a protected environment file or secret manager rather than recording a
real token in shell history; do not share tokens or guest passwords in reports.

### Start a TLS Server

Non-loopback binds require a valid TLS certificate and key. The files must be
readable by the server process:

```sh
export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
  UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
  ./uvm.py --serve --host 0.0.0.0 --port 8443
```

For real remote use, use a certificate trusted by clients and restrict network
access with a firewall or reverse proxy. Do not use a self-signed development
certificate for an untrusted network.

### API Authentication

Set variables for the request examples:

```sh
export UVM_API_TOKEN='replace-with-the-server-token'
export UVM_API_URL='http://127.0.0.1:8000'
```

Every management request needs the `X-UVM-Token` header, as below.

For a TLS server, set `UVM_API_URL` to an `https://` URL and use your CA:

```sh
export UVM_API_URL='https://host.example:8443'
curl --cacert /path/to/ca.pem \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms"
```

## 11. API Endpoint Examples

| Method | Path | Operation |
|---|---|---|
| `GET` | `/health` | Unauthenticated liveness |
| `POST` | `/install` | Install/refresh with a `force` boolean |
| `GET` | `/vms` | List VM responses |
| `POST` | `/vms` | Create and boot, HTTP 201 on success |
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/stop` | Stop and retain disk, using the default TERM wait |
| `DELETE` | `/vms/{identifier}` | Delete VM and private disk |

These stop/delete endpoints do not expose the CLI timeout, dry-run, or bulk
flags and do not prompt for confirmation.

### Health, List, and Inspect

```sh
# Health is unauthenticated and returns {"status":"ok"}.
curl "$UVM_API_URL/health"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms/api-web"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms/10.42.0.30"
# An exact VM ID from create/list can replace the name or IP.
```

Each response contains `id`, `name`, `cpu`, `ram_mib`, `guest_ip`, `gateway`,
`mac`, `username`, `status`, `observed_status`, `ports`, `policy_status`,
`operation`, `pid`, timestamps, and `last_error`. The response schema excludes
passwords and disk/socket/config/log path fields. Operational error messages
can still contain host paths; review them before sharing.

### Create a VM

Create a named VM with a chosen guest address and private service ports:

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"name":"api-web","cpu":1,"ram":"1G","guest_ip":"10.42.0.30","username":"root","password":"replace-me","ports":[{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}],"no_default_ports":false}' \
  "$UVM_API_URL/vms"
```

Create an explicit empty service policy, with no default SSH allowance:

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"name":"api-worker","password":"replace-me","ports":[],"no_default_ports":true}' \
  "$UVM_API_URL/vms"
```

Request fields are `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`,
`ports`, and `no_default_ports`. The name has the same validation as the CLI.
`ports` is an array of objects (or null/omitted), not a CLI colon string.
Each object has a strict integer `port` from 1 to 65535 and an optional,
case-sensitive `protocol` of `TCP` (default), `HTTP`, or `HTTPS`.
`no_default_ports` is a strict boolean, default `false`; port 22 is added
unless that flag is true, even if `ports` is null, empty, or contains other
ports. Null create input does not create a legacy/unmanaged VM.

The response never includes the password. Omitting credentials still uses
`root` / `root`. Extra request fields, invalid ports/names, and conflicting
labels return HTTP `422`. Example passwords above are placeholders; real
secrets in curl arguments can enter shell history and process listings.

### Stop and Delete

```sh
curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web/stop"

# Destructive: no interactive confirmation over HTTP.
curl -X DELETE \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web"
```

Stop returns the VM response with `status: "stopped"`; delete returns its
exact `id` and `status: "terminated"`. Start the retained disk through the
local CLI, not a new HTTP route.

### Install or Refresh Through the API

The server process must itself have checksum variables configured for strict
installation. Start the server with those environment variables before using
this endpoint.

For a deliberately unverified local-development server, the server process
must instead include `UVM_ALLOW_UNVERIFIED_DOWNLOADS=1`. Do not use that mode
on a host that runs untrusted workloads.

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"force": false}' \
  "$UVM_API_URL/install"
```

Use `{"force": true}` to refresh all downloaded artifacts.

## 12. API Errors

| Status | Typical meaning |
|---|---|
| `401` | Missing or incorrect `X-UVM-Token`. |
| `403` | UVM process lacks root privileges for a host mutation. |
| `404` | Requested VM was not found. |
| `409` | Name/identifier conflict, incompatible retained state, or an unsafe/pending operation. |
| `422` | Invalid JSON body, extra request fields, or invalid UVM input. |
| `500` | Host command, Firecracker, state, or unexpected operational failure. |

UVM operation errors use this envelope:

```json
{"error": {"message": "human-readable explanation"}}
```

FastAPI request-schema errors use FastAPI's normal `detail` response format.

## 13. Configuration Scenarios

### Custom Storage and Network

For example, install under `/srv/uvm` with a different private subnet. All
binaries, images, VM runtime directories, locks, and state use the base path:

```sh
sudo env \
  UVM_BASE='/srv/uvm' \
  UVM_NETWORK='10.50.0.0/24' \
  UVM_GATEWAY='10.50.0.1' \
  UVM_BRIDGE='uvm50' \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
  ./uvm.py install
```

Use the same base/network/gateway/bridge values for all later lifecycle,
inspection, port, and server commands. Start rejects conflicting retained
network settings. Use a non-overlapping private subnet; do not assume UVM
automatically detects every conflict with host routes or other managers.

### Environment Variables

| Variable | Use |
|---|---|
| `UVM_BASE` | Base data directory; default `/var/lib/uvm` |
| `UVM_NETWORK` | Guest subnet; default `10.42.0.0/24` |
| `UVM_GATEWAY` | Guest gateway / bridge IP; default `10.42.0.1` |
| `UVM_BRIDGE` | Linux bridge name; default `uvm0` |
| `UVM_KERNEL_URL` | Custom kernel download URL |
| `UVM_ROOTFS_URL` | Custom rootfs download URL |
| `UVM_FIRECRACKER_SHA256` | Strict Firecracker archive checksum |
| `UVM_KERNEL_SHA256` | Strict kernel checksum |
| `UVM_ROOTFS_SHA256` | Strict rootfs checksum |
| `UVM_FIRECRACKER_BINARY_SHA256` | Optional local binary checksum |
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | Set to `1` only for local development |
| `UVM_API_TOKEN` | Required API token |
| `UVM_API_TLS_CERT` | TLS certificate for remote server binds |
| `UVM_API_TLS_KEY` | TLS key for remote server binds |

## 14. Files, Logs, and State

The default host layout is:

```text
/var/lib/uvm/
├── bin/                 # Firecracker and Jailer binaries
├── images/              # Shared kernel and rootfs template
├── vms/<vm-id>/         # Private rootfs, config, socket, and log
├── integrity.json       # Artifact checksums and verification status
└── state.json           # Mode-0600 VM inventory and plaintext guest credentials
```

Registry writes are atomic, with a short `state.lock`; lifecycle, policy, and
installation mutations share `operations.lock`. Avoid editing `state.json`,
runtime files, TAPs, or UVM-owned nftables/iptables rules by hand. Use supported
commands so ownership, locking, and cleanup remain correct. Back up important
disks while stopped and keep the associated registry backup protected.

`state.json` contains plaintext guest passwords. Keep it root-only, never copy
it into logs or bug reports, and use unique non-default passwords outside local
throwaway environments. Prefer inspect/list to dumping the registry. Raw
Firecracker logs may include guest/application secrets; redact them before sharing.

## 15. Networking Troubleshooting

The existing `iptables` egress MASQUERADE and forwarding rules are unchanged.
Private port filtering is a separate native nftables `bridge` policy, not
DNAT. It must be installed and read back before the TAP is active. The bridge
must have STP disabled and each TAP must have bridge-slave `group_fwd_mask`
`0x4000` (16384), so LLDP takes the early identity-filter path rather than
bypassing it before MAC learning. These are software-bridge requirements;
do not offload traffic around the hooks. Native bridge conntrack needs Linux
>= 5.3; this layer does not configure `br_netfilter` sysctls.

Check the bridge and TAP state:

```sh
ip -d link show uvm0
ip addr show dev uvm0
ip route show default
```

Check port policy and the separate egress NAT/forwarding rules:

```sh
sudo ./uvm.py doctor
sudo ./uvm.py inspect web --json
sudo ./uvm.py ports list --all
sudo nft --json list tables bridge
sudo iptables -t nat -S POSTROUTING
sudo iptables -S FORWARD
```

Common issues:

| Symptom | First checks |
|---|---|
| Guest cannot reach the internet | Bridge is up, TAP is up, host has default route, NAT and FORWARD rules exist. |
| Requested guest IP is rejected | Confirm it is inside the configured subnet and not used, gateway, network, or broadcast. |
| Create says TAP already exists | Inspect the interface; do not delete it unless you know it is stale UVM state. |
| Existing bridge is rejected | Verify it is a software Linux bridge with STP disabled. |
| Firewall unavailable/unknown | Check privileges, `nft`, `conntrack`, kernel support, and bridge/TAP settings. |
| Policy missing/drift after external reload | Resolve the other manager's conflict, then explicitly run `ports sync VM`; there is no automatic repair. |
| SSH cannot connect | Confirm boot/sshd, guest user and password/key, chosen SSH port, allowlist, and guest firewall. |
| Closed SSH session hangs rather than disconnects | Close drops subsequent traffic in both directions; a TCP RST is not guaranteed. |

External firewall integration (including UFW, firewalld, Docker, and libvirt)
is deferred. A UVM policy readback does not prove that external rules permit
traffic, or that they will not later remove UVM's rules.

## 16. Important Limitations

Keep these constraints in mind when deciding whether UVM fits a scenario:

- Firecracker is launched directly; the downloaded Jailer is not yet used.
- No quotas or cgroup CPU, memory, I/O, or process limits are applied.
- Fractional CPU requests are advisory only.
- No autostart, background lifecycle/firewall reconciliation, or guest-coordinated stop.
- No UDP/IPv6 service policies, public connectivity setup, host-port mapping,
  DNS publication, guest reverse proxy/TLS setup, DHCP, load balancing,
  snapshots, multi-network tenancy, or multi-host scheduling.
- The API uses one shared bearer token, not individual users, roles, or tenants.
- There is no guest readiness probe beyond Firecracker process state.
- UVM is not a hardened multi-tenant platform.

## 17. Safe Operating Checklist

Before relying on a VM for useful work:

1. Use strict checksums for installed artifacts.
2. Confirm `/dev/kvm` and nested virtualization are available.
3. Confirm UVM generated unique guest SSH host keys and set a non-default password.
4. Test guest egress and SSH on the actual host network.
5. Quiesce guest workloads before stopping/restarting and back up data before deletion.
6. Keep the API loopback-only unless you have a strong remote-access need.
7. For remote API access, use a long random token, valid TLS, and firewall
   restrictions.
8. Review `/var/lib/uvm/vms/<vm-id>/firecracker.log` after failed starts.

## 18. Verification

From the project root, normal unit tests need no root, KVM, or host network
changes. Install the test extra in your Python environment for API tests:

```sh
python3 -m pip install '.[test]'
python3 -B -m unittest discover -s tests -v
```

The opt-in packet test runs **only in disposable isolated user/network
namespaces**, which may require privileges. It needs `unshare`, `ip`,
`bridge`, `nft`, `conntrack`, and `sysctl`:

```sh
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
```

Namespace creation was denied in this workspace (`Operation not permitted`),
so the packet test skipped. That is a permission limitation, not real-packet
or real-host validation. The harness uses veths as TAP stand-ins and routed
egress without NAT; it does not boot Firecracker or validate the host's real
TAP/NAT setup. Run actual VM/SSH/egress checks on an isolated Linux host before
relying on the deployment.

## 19. Get Help

```sh
./uvm.py --help
./uvm.py create --help
./uvm.py ports set --help
./uvm.py ssh --help
```

When reporting a problem, collect:

- The command or HTTP request, with passwords and tokens redacted.
- The full error message.
- Password-free output from `list --json`, `inspect VM --json`, and `doctor`.
- The VM Firecracker log, reviewed for guest/application secrets.
- The relevant `UVM_*` variables, with secrets such as `UVM_API_TOKEN` removed.

Never include `/var/lib/uvm/state.json` because it contains guest passwords.