nest/README.md

# uvm

`uvm` is a small, local Firecracker microVM command-line manager for Ubuntu.
It creates one Firecracker process per VM, attaches it to a host TAP device,
and keeps a private disk and local registry for named VMs. The CLI supports
create, start, stop, restart, delete, inspection, SSH, logs, diagnostics, and
enforced private-IPv4 TCP port allowlists.

This directory is a self-contained project. All UVM code, tests, packaging
metadata, launcher scripts, and documentation live here.

For architecture, lifecycle invariants, extension guidance, and contributor
workflows, see [DEVELOPER.md](DEVELOPER.md).

For end-to-end operator scenarios, CLI examples, and HTTP API calls, see
[USAGE.md](USAGE.md).

For the original firewall proposal and backend evaluation, see
[FIREWALL.md](FIREWALL.md). That proposal is design background, not the current
feature contract; these guides describe the implementation in the workspace.

## Quick Start

Run commands from this project root:

```sh
./uvm.py --help
python3 -B -m uvm --help
```

The `pyproject.toml` file also defines an installable `uvm` console command.
The project is an application, not a reusable Python SDK.

## Requirements

- Ubuntu or another compatible Linux host
- Python 3.12 or newer
- Root access for UVM commands that read or mutate the protected VM registry
- KVM enabled and accessible at `/dev/kvm`
- Linux 5.3 or newer with native bridge conntrack and safe pidfd support
- An x86_64 host for the default guest assets
- Working host networking, `iproute2`, `iptables`, `nftables`, `conntrack`,
  `e2fsprogs`, `util-linux`, `openssl`, and `openssh-client`
- An STP-disabled software bridge; no offload path that bypasses bridge hooks

`uvm install` installs the required host packages. It validates `/dev/kvm`
before downloading VM artifacts. Boot operations additionally check that the
native nftables bridge policy is supported; installing packages alone is not
a connectivity or isolation test.

## Trusted Installation

Installation requires trusted SHA-256 digests by default. Supply digests from
an independently trusted source before running the privileged installer:

```sh
sudo env \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
  ./uvm.py install
```

Use `--force` to redownload Firecracker and guest assets when replacing an
image or refreshing artifacts:

```sh
sudo env \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
  ./uvm.py install --force
```

For local experimentation only, the old unverified quick-start behavior is
available through an explicit opt-out:

```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
```

An unverified manifest is not accepted as publisher trust in a later strict
run. Reinstall with trusted digests to establish a verified local manifest.
To boot VMs from an unverified local-development install, retain
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create`, `start`, and `restart`
commands and on the API server process.

## Commands

These are separate operation examples, not a script to run from top to bottom.

```sh
# Install after supplying the trusted checksum variables shown above.
# A bare `install` command fails under the default strict integrity policy.

# Create a named VM: 1 CPU, 512 MiB RAM, root/root, and private TCP/22.
# Use a unique non-default password for anything beyond a throwaway VM.
sudo ./uvm.py create --name web

# Choose resources, a private guest IP, and additional service ports.
sudo ./uvm.py create --name app --cpu 1 --ram 1G \
  --guest-ip 10.42.0.10 --ports 80:HTTP,443:HTTPS

# Read a password from one line of stdin, rather than a --password argument.
# USAGE.md includes a non-echoing Bash prompt example.
sudo ./uvm.py create --name worker --password-stdin

# Explicitly allow no inbound service ports, including SSH.
sudo ./uvm.py create --name closed --no-default-ports

# Inspect persisted state and observed process/policy status.
sudo ./uvm.py list
sudo ./uvm.py list --status running --json
sudo ./uvm.py list -q
sudo ./uvm.py inspect web --json
sudo ./uvm.py logs web --tail 50 --follow
sudo ./uvm.py doctor --json

# Resolve VMs by exact ID, name, or guest IP.
sudo ./uvm.py ssh web
sudo ./uvm.py ssh app --port 22 -- uname -a
ssh root@10.42.0.10

# Stop and boot the same private disk, or restart selected VMs.
sudo ./uvm.py stop web --timeout 5
sudo ./uvm.py start web
sudo ./uvm.py restart web app --timeout 10

# Preview bulk operations before confirming them.
sudo ./uvm.py stop --all --dry-run --json
sudo ./uvm.py stop --all -y
sudo ./uvm.py start --all -y
sudo ./uvm.py restart --all -y
sudo ./uvm.py delete --all --dry-run
sudo ./uvm.py delete web -y
# destroy is an alias for delete, with the same flags and destructive effect.
```

`start`, `stop`, `restart`, and `delete` accept one or more VM identifiers or
`--all`, plus `--json`, `--dry-run`, and `--yes`/`-y`. Deletion and mutating
`--all` selections require confirmation; non-interactive callers must use
`--yes`. `--timeout` is available on stop/restart/delete (and `destroy`), not
start. It is a finite non-negative SIGTERM wait in seconds, default `5`, not
a total command deadline or a guest shutdown timeout.

Stop sends SIGTERM and, if needed, SIGKILL to the Firecracker VMM. It is **not
a guest-coordinated shutdown**. Start boots the retained disk without resetting
guest data, SSH host keys, or credentials; it does not restore guest memory.
Delete permanently removes the disk and releases the VM's name and IP after
cleanup. See [USAGE.md](USAGE.md) for the complete flag and recovery reference.

Names are unique local identifiers, not DNS names. `--host-ip` remains an
alias for `--guest-ip`; both select the guest's private IPv4 address. JSON
output omits passwords, with warnings and host-command traces on stderr.

SSH uses `StrictHostKeyChecking=accept-new` and a VM-ID-specific host-key
alias by default. `--user`, `--key`, and `--port` customize the SSH client;
append a remote command after a literal `--`. `--insecure-host-key` disables
verification for one connection. SSH does not automatically supply the stored
password or open a closed port.

### Private Ports

```sh
sudo ./uvm.py ports list web --json
sudo ./uvm.py ports list --all
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
sudo ./uvm.py ports close web --ports 80
sudo ./uvm.py ports set web --ports 22,443:HTTPS --dry-run --json
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
sudo ./uvm.py ports close web --all -y
sudo ./uvm.py ports set web --none -y
sudo ./uvm.py ports sync web
sudo ./uvm.py ports sync --all --dry-run
sudo ./uvm.py ports sync --all -y
```

Rules use `PORT[:TYPE]`, for example `22`, `80:HTTP`, `443:HTTPS`, or
`8080:TCP`, repeated or comma-separated. HTTP and HTTPS are **TCP labels**,
not proxies or TLS configuration. Create adds `22:TCP` even when other ports
are supplied, unless `--no-default-ports` is set. Later port edits do not
automatically add SSH back. `ports close VM --all` closes ports on that one
VM; only `ports list --all` and `ports sync --all` select all VMs.

Port mutations support `--dry-run` and `--json`; close/set/sync also support
`--yes`/`-y`. Set, close-all, and removal of TCP/22 require confirmation
(an already empty managed policy can be a no-op). Closing a service blocks
subsequent packets on existing service connections in **both directions**, as
well as new connections. This is packet dropping, not a guaranteed TCP RST or
immediate application disconnect.

These rules govern access to the **guest private IP**. They do not start a
guest service or provide public connectivity, host-port mappings, DNS,
reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not
yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required
ARP/ICMP traffic remain allowed; managed TAPs block IPv6.

## Management API

Install the project dependencies before serving the API:

```sh
python3 -m pip install .
```

Launch the FastAPI management server using the environment where the project
dependencies were installed (use the full console-script path if needed):

```sh
export UVM_API_TOKEN='replace-with-a-long-random-secret'
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
  uvm --serve --host 127.0.0.1 --port 8000

# Equivalent when running the source checkout directly.
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
  ./uvm.py --serve --host 127.0.0.1 --port 8000
```

The default bind address is `127.0.0.1` and the default port is `8000`.
OpenAPI documentation is available at `/docs` once the server is running.

The server exposes these routes:

| Method | Route | Purpose |
|---|---|---|
| `GET` | `/health` | Unauthenticated liveness response |
| `POST` | `/install` | Install or refresh host assets; body: `{ "force": false }` |
| `GET` | `/vms` | List VM records and observed status |
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/stop` | Stop a VM while retaining its private disk |
| `DELETE` | `/vms/{identifier}` | Delete a VM and release its resources |

API port rules are objects such as `{"port": 80, "protocol": "HTTP"}`;
`no_default_ports` is a boolean, default `false`. Responses include `name`,
`ports`, `policy_status`, and `operation`, but never passwords. See
[USAGE.md](USAGE.md) for request examples and strict schema details.

The endpoint set is unchanged: there are **no start/restart, port-management,
or bulk HTTP endpoints**. SSH, logs, and doctor are also CLI-only. Use the
local CLI for those operations; the API is not a background reconciler.

### API Security

The server controls privileged host operations. `UVM_API_TOKEN` is required
for every API server, including loopback-only deployments. All management
routes require it in the `X-UVM-Token` header. FastAPI's informational `/docs`,
`/redoc`, and `/openapi.json` endpoints do not require a token.

A non-loopback bind additionally requires TLS certificate and key paths:

```sh
export UVM_API_TOKEN='replace-with-a-long-random-secret'
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'
sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
  UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
  ./uvm.py --serve --host 0.0.0.0 --port 8000
```

Send the token on management requests:

```sh
curl \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  http://127.0.0.1:8000/vms
```

Do not expose this server directly to an untrusted network. For remote access,
keep TLS enabled and consider placing it behind an additional authenticated
reverse proxy.

## Guest Image Contract

The installed kernel and rootfs are templates. `create` copies the rootfs to
the VM runtime directory before Firecracker opens it read-write, so VMs do not
share a mutable root disk.

During creation, UVM updates the private ext4 disk with a password for an
existing guest account and enables OpenSSH password authentication. The
defaults are username `root` and password `root`. A custom image must provide:

- `sshd`
- `/etc/passwd`, `/etc/shadow`, and `/etc/ssh/sshd_config`
- The requested user account; UVM does not create missing users

The known public demo key bundled in Firecracker's default bionic image is
removed from each private disk. UVM also replaces the conventional RSA, ECDSA,
and Ed25519 SSH host keys in every private disk so cloned VMs have distinct
host identities. Never rely on SSH private keys baked into a shared template.

## Host Networking

By default, UVM manages:

```text
Network:  10.42.0.0/24
Gateway:  10.42.0.1
Bridge:   uvm0
TAPs:     uvm-<unique suffix>
```

It creates the bridge when absent, validates an existing bridge before using
it, enables IPv4 forwarding, adds an egress masquerade rule, and adds matching
forward rules for return traffic. Guest IPs, gateway, bridge name, and guest
boot netmask derive from the configured CIDR.

The existing `iptables` egress NAT/forwarding behavior is unchanged. Private
port enforcement uses separate per-VM native nftables `bridge` tables, not
DNAT or host listeners. Policy is installed and read back before an owned TAP
is activated and before the guest starts. The software bridge must have STP
disabled; each TAP's bridge-slave `group_fwd_mask` is set to `0x4000` so LLDP
cannot bypass the early identity filter. Native bridge conntrack requires
Linux >= 5.3; no `br_netfilter` sysctl setup is required by this policy layer.

Host-to-VM and VM-to-VM traffic crossing these hooks is filtered. An allowed
port is not a reachability guarantee, and a private IP alone is not protection
against administrator-supplied external routing or NAT. Integration with UFW,
firewalld, Docker, or other external firewall managers remains deferred.
Do not offload traffic around the software bridge hooks.

There is no autostart service or daemon reconciliation. After externally lost
or changed UVM rules, inspect policy status and explicitly run `ports sync`.
Read-only commands do not repair rules; sync does not boot stopped VMs.

## Persistent State

The default data directory is `/var/lib/uvm`:

```text
/var/lib/uvm/
├── bin/                 # Firecracker and Jailer binaries
├── images/              # Kernel and rootfs templates
├── vms/<vm-id>/         # Per-VM disk, config, log, and API socket
├── integrity.json       # Download/build checksums and verification provenance
└── state.json           # Root-only VM inventory and guest credentials
```

The default registry is `/var/lib/uvm/state.json`, schema version `2`. Writes
are locked and atomic. Lifecycle, port-policy, and installation mutations
share an operation lock; bulk operations use a fixed snapshot of VM IDs.

The registry stores guest usernames and passwords in plaintext, so `state.json`
is mode `0600`. Treat it as a secret, do not include it in bug reports, and
replace the default `root` password immediately.

Credential provisioning applies only to VMs created by this version. Existing
VM disks are not modified automatically.

Older registries are read and migrated on the next write. A missing/null
`ports` policy remains `None`/`unmanaged`, not an empty allowlist or an
implicitly enforced default. Migrate a legacy VM explicitly with `stop`,
`ports set VM --ports 22,80:HTTP -y` (choose the actual required ports), then
`start`. Resume requires a complete, safe private disk, a verified kernel and
VMM, and compatible network settings. Removing the original rootfs template
does not block start of a retained private disk.

Process identity includes PID, Linux process start time, and host boot ID.
A running legacy PID missing the boot ID is adopted only when its saved start
time, running binary, exact API-socket command line, and protected paths all
match. Missing or ambiguous evidence fails closed: UVM will not guess which
process to signal or release a disk while process exit is unconfirmed.

## Configuration

Host and API settings can be overridden through environment variables:

| Variable | Purpose |
|---|---|
| `UVM_BASE` | Data directory; defaults to `/var/lib/uvm` |
| `UVM_NETWORK` | Guest private IPv4 CIDR; defaults to `10.42.0.0/24` |
| `UVM_GATEWAY` | Bridge/guest gateway; defaults to `10.42.0.1` |
| `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` |
| `UVM_KERNEL_URL` | Guest kernel download URL |
| `UVM_ROOTFS_URL` | Guest rootfs download URL |
| `UVM_FIRECRACKER_SHA256` | Trusted Firecracker release archive SHA-256 |
| `UVM_KERNEL_SHA256` | Trusted guest kernel SHA-256 |
| `UVM_ROOTFS_SHA256` | Trusted guest rootfs SHA-256 |
| `UVM_FIRECRACKER_BINARY_SHA256` | Optional trusted local Firecracker binary SHA-256 |
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | Set to `1` only for unverified local development |
| `UVM_API_TOKEN` | Required for every API server; sent as `X-UVM-Token` |
| `UVM_API_TLS_CERT` | TLS certificate path required for a non-loopback API bind |
| `UVM_API_TLS_KEY` | TLS private-key path required for a non-loopback API bind |

## Project Layout

```text
./
├── README.md, USAGE.md, DEVELOPER.md
├── pyproject.toml
├── uvm.py               # Executable compatibility launcher
├── uvm/                 # Private application implementation
│   ├── cli.py           # argparse, terminal output, SSH execution
│   ├── lifecycle.py     # Lifecycle, bulk operations, recovery, port-policy intent
│   ├── network.py       # IP/MAC/TAP/bridge/NAT management
│   ├── firewall.py      # Owned nftables bridge policies and offline conntrack cleanup
│   └── firecracker/     # Firecracker config, API, and process adapters
└── tests/               # Unit tests and opt-in isolated namespace packet tests
```

See [DEVELOPER.md](DEVELOPER.md) for the complete module map and ownership rules.

## Testing

Run the unit suite from this project root:

```sh
python3 -m pip install '.[test]'
python3 -B -m unittest discover -s tests -v
```

The unit tests do not need root, KVM, Firecracker, or host network changes.
They cover validation, state migration and locking, integrity manifests,
network/firewall rendering, Firecracker request ordering, process safety,
retained-disk resume, bulk operations, and lifecycle rollback. API route tests
run when the FastAPI and `httpx` test dependencies are installed.

Opt-in packet checks run **only in disposable isolated user/network
namespaces**, which may require privileges and the installed network tools:

```sh
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
```

The packet test was skipped in this workspace because namespace creation was
denied (`Operation not permitted`). A skip is not packet-path or real-host
validation. The harness uses veths in place of TAPs and does not boot a real VM;
validate KVM, TAP, NAT, SSH, and guest connectivity on an isolated Linux host.

## Current Boundaries

- Fractional CPU requests still round Firecracker vCPUs up and are advisory.
- No quotas or cgroup CPU, memory, I/O, or process limits are applied.
- Firecracker is launched directly by this local CLI. Jailer/cgroup hardening
  is not implemented by the current command set.
- No VM autostart, background reconciliation, guest readiness probing,
  snapshots, DHCP, public-port publishing, or multi-host scheduling.
- The API uses one shared token, not users, roles, or tenants. UVM is not a
  hardened multi-tenant platform.
- Run real KVM, TAP, and guest-connectivity checks on an isolated Linux host
  before relying on UVM for workloads.