Files

T

kstyagi dca3cbde2f feat: add host-backed entropy device support and update kernel configuration

2026-09-23 07:00:22 +05:30

57 KiB

Raw Blame History

UVM Usage Guide

This guide explains how to install, operate, and troubleshoot UVM as an operator. It covers the command-line tool, the optional FastAPI server, common scenarios, and expected limitations.

For the project architecture and contributor guidance, see DEVELOPER.md. For a concise overview, see README.md.

1. What UVM Does

UVM creates small Linux microVMs through Firecracker and KVM on one Linux host. Each VM receives:

A unique VM ID and, optionally, a unique local name.
A private writable root disk copied from a shared template.
A private IP address and MAC address.
A TAP interface connected to a host bridge.
A dedicated Firecracker process, API socket, log, and config file.
A managed private-IPv4 TCP allowlist, initially allowing port 22 unless default ports are explicitly disabled.

By default, guest networking uses:

Guest network: 10.42.0.0/24
Gateway:       10.42.0.1
Bridge:        uvm0
Guest pool:    10.42.0.2 through 10.42.0.254

Guest IPv4 internet access shares the host's connection automatically. Host offline means guest internet is offline; reconnecting the host restores guest egress. UVM installs sharing rules even when booting offline, and follows the host's current routes when uplinks change. The private network remains usable without internet. Applications may need to reconnect after a network interruption.

The default locations on the host are under /var/lib/uvm. Stop/start and restart reuse the private disk; delete removes it. UVM is a local CLI, not an autostart service or a daemon that continuously repairs VM or firewall state.

2. Before You Start

Host Requirements

You need:

Linux, preferably Ubuntu or a compatible distribution.
Python 3.12 or newer.
KVM enabled and readable/writable at /dev/kvm.
Linux >= 5.3 with native bridge conntrack and safe pidfd support.
Root access for installation and VM lifecycle operations.
An x86_64 or ARM64 (aarch64) host, including NVIDIA DGX Spark.
Internet access if downloading Firecracker and guest assets.
iproute2, iptables, nftables, conntrack, coreutils, e2fsprogs, util-linux, openssl, and openssh-client; install installs the host packages.
An STP-disabled software bridge without traffic offload around its hooks.

Check virtualization before installing:

ls -l /dev/kvm
lscpu | grep -i virtualization

Source Checkout Setup

From this project root:

./uvm.py --help
python3 -B -m uvm --help

To install the optional HTTP API dependencies and the console command:

python3 -m pip install .
uvm --help

The rest of this guide uses ./uvm.py so it works from a source checkout. If you installed the package, replace ./uvm.py with uvm. For the API, use the Python environment containing the installed dependencies; under sudo, use the full path to that environment's console script when necessary.

Privilege Rules

Operation	Root required?
`install`, `create`, `start`, `stop`, `restart`, `resize`, `delete`/`destroy`	Yes
`ports open/close/set/sync`	Yes
`list`, `inspect`, `ports list`, dry-run target lookup	Access to the default root-owned registry requires root
`ssh`, `logs`	Access to the protected registry/runtime files requires root
`doctor`	Run as root for meaningful privileged checks
API server that manages VMs	Yes

The registry contains plaintext guest credentials and is mode 0600. Direct SSH to a known guest IP does not require root.

3. Command Overview

The following is syntax notation, not a shell script. VM means an exact VM ID, local name, or guest IPv4 address. Flags are command-specific; in particular, create has no --dry-run, and start has no --timeout. Examples elsewhere are separate scenarios; substitute your VM identifiers and do not run every destructive example as a single script.

uvm install [--force]
uvm create [--name NAME] [--cpu CPU] [--ram RAM] [--storage STORAGE] [--guest-ip GUEST_IP]
           [--username USER] [--password PASSWORD | --password-stdin]
           [--ports PORTS]... [--no-default-ports] [--json]
uvm list [--status STATUS]... [--json | -q | --quiet]
uvm inspect VM [--json]
uvm start (VM [VM ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm stop (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm restart (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm resize VM [--cpu CPU] [--ram RAM] [--storage STORAGE] [--timeout SECONDS] [--json]
uvm delete (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm destroy (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm ssh VM [--user USER] [--key PATH] [--port PORT] [--insecure-host-key] [-- COMMAND ...]
uvm logs VM [--tail N] [--follow | -f]
uvm doctor [--json]
uvm ports list (VM | --all) [--json]
uvm ports open VM --ports PORTS [--ports PORTS ...] [--dry-run] [--json]
uvm ports close VM (--ports PORTS [--ports PORTS ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm ports set VM (--ports PORTS [--ports PORTS ...] | --none) [--yes | -y] [--dry-run] [--json]
uvm ports sync (VM [VM ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm --serve [--host HOST] [--port PORT]

--host-ip is an alias for --guest-ip; neither refers to the host's public or uplink address. --ports and list --status are repeatable flags, not space-separated positional lists. destroy is an alias for delete.

JSON is written to stdout; warnings, errors, prompts, and host-command traces go to stderr. VM JSON omits passwords. Lifecycle and sync results are arrays of id, action, outcome, and error; dry runs return an object with dry_run, action, and targets (port edits also include before/after changes). ports list --json returns an array even for one VM. JSON does not disable confirmations, and errors may still be plain text on stderr.

Stop the Management API

The API can be stopped remotely through its authenticated shutdown endpoint. This stops only the UVM API process; it does not stop or delete any VM:

curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  http://127.0.0.1:8000/exit

The server first returns 202 {"status":"stopping"}, then performs Uvicorn's graceful shutdown. Use the actual API host and port when they differ from the defaults. GET /exit is intentionally not supported.

4. Install UVM Assets

UVM installs host packages, Firecracker, Jailer, a guest kernel, and a guest rootfs template. The default rootfs is built locally from authenticated Ubuntu 22.04 packages, including OpenSSH, sudo, and complete apt/dpkg metadata. The downloaded Firecracker archive and kernel require trusted SHA-256 checksums.

ARM64 Hosts (NVIDIA DGX Spark)

UVM detects the host architecture and selects matching Firecracker and kernel downloads and native Ubuntu packages. NVIDIA DGX Spark uses aarch64 artifacts and arm64 packages; the arm64 architecture alias is also accepted:

Kernel: https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/aarch64/vmlinux-6.1.155
Rootfs: Ubuntu 22.04 built with debootstrap from ports.ubuntu.com/ubuntu-ports.

Use the installation commands below with checksums for the ARM64 kernel and Firecracker release archive. UVM_KERNEL_URL overrides the kernel download; UVM_ROOTFS_URL selects a prebuilt ext4 image instead of the local build. Custom images must match the host architecture and the guest image contract. On ARM64, the kernel must be an uncompressed ARM64 Linux Image (PE format), and the rootfs must be an ARM64 ext4 filesystem. UVM adds keep_bootcon to the guest boot arguments for ARM64 console output.

Both architectures default to the pinned Linux 6.1.155 kernel from the published v1.15 CI artifact series, independently of the Firecracker binary version. These kernels have CONFIG_HW_RANDOM=y and CONFIG_HW_RANDOM_VIRTIO=y built in. UVM configures Firecracker's entropy device before boot; custom kernels also need these drivers to receive host-backed randomness.

Scenario: Strict First Installation

Obtain trusted checksums independently, then run:

sudo env \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  ./uvm.py install

Installation prints the Firecracker, kernel, and rootfs paths under /var/lib/uvm. Checksums and verification provenance are recorded in /var/lib/uvm/integrity.json.

The default build installs debootstrap and ubuntu-keyring on the host, checks Ubuntu archive signatures, bootstraps the base release, then uses apt to apply updates and install the guest packages. It builds inside private mount/PID namespaces and requires native root execution with namespace support. Leave UVM_ROOTFS_URL and UVM_ROOTFS_SHA256 unset for this mode. To use a prebuilt ext4 image, supply both its URL and SHA-256 instead.

Scenario: Refresh Firecracker or Guest Assets

Use --force after changing a URL, replacing an image, or rotating checksums:

sudo env \
  UVM_FIRECRACKER_SHA256='<new-firecracker-archive-sha256>' \
  UVM_KERNEL_SHA256='<new-kernel-sha256>' \
  ./uvm.py install --force

--force stages downloads and builds in temporary paths. Downloaded artifacts are checksum-verified; the built rootfs is checked for installed packages, sudo permissions, and filesystem consistency before replacement.

To upgrade from the old stripped Bionic demo rootfs, run this refresh with UVM_ROOTFS_URL and UVM_ROOTFS_SHA256 unset, then create a new VM. Restart a running API server to load the new provisioner. Existing VM disks retain their installed OS, files, and credentials.

Scenario: Local Experimentation Without Checksums

This mode is unsafe for production; use it only when deliberately accepting unverified development artifacts:

sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install

An unverified manifest is not publisher trust. Reinstall with trusted digests to establish a verified installation. Until then, retain the explicit opt-out on later create/start/restart/resize commands and on an API server that creates VMs:

sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py create
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py start VM

Installation Failures

Message	Meaning and action
`/dev/kvm does not exist`	Enable hardware/nested virtualization first.
`checksum is required`	Provide the requested `UVM_*_SHA256` variable.
`SHA-256 mismatch`	Stop and obtain the correct digest or artifact URL.
`unsupported host architecture`	Use an x86_64 or ARM64 Linux host.
`unshare: Operation not permitted`	Run the builder as root on a host permitting private mount/PID namespaces.
`Ubuntu rootfs build failed`	Read `images/ubuntu-build.log` under `UVM_BASE`, fix the reported bootstrap/apt error, and rerun installation.

The latest image-build log is retained after temporary build directories are cleaned up, including detailed debootstrap and apt output:

sudo tail -n 100 /var/lib/uvm/images/ubuntu-build.log

A failed build leaves the previous rootfs template installed. If creation then reports missing sudo, complete the image installation successfully before retrying creation. The username itself does not cause that error.

5. Create VMs

Scenario: Create a Default VM

The default is one CPU, 512 MiB RAM, 30 GiB storage, username root, password root, and an allowlist containing private TCP/22:

sudo ./uvm.py create

Output includes the ID, optional name, guest IP, resources, TAP, username, and private port rules. The default password produces a warning on stderr. Use --json for a password-free VM object instead of the human-readable summary.

Names and Credentials

sudo ./uvm.py create --name web --username codemate-user --password '<guest-password>'

Names are optional, unique within the registry, and case-sensitive. They must be 1-63 lowercase letters, digits, or hyphens, starting with a letter. Names that impersonate generated IDs (vm- plus 32 hexadecimal characters) are rejected. Names and IPs stay reserved while a VM is stopped. Names do not create DNS records, and identifiers are not matched by ID prefix.

--username selects the guest login account. If a custom username is missing, UVM creates it on the private disk with a primary group, /home/USERNAME, and the /bin/sh login shell. Custom logins receive sudo access using their guest password. Existing accounts keep their identity and home.

A --password value is visible in process arguments and may enter shell history. Prefer --password-stdin, which reads one line, strips the trailing line ending, and rejects empty passwords, control characters, or passwords longer than 128 characters. It cannot be combined with --password.

This Bash example prompts without echoing the password and avoids putting a literal secret in the command line or history. Do not enable shell tracing:

sudo -v
IFS= read -r -s -p 'Guest password: ' guest_password
printf '\n'
printf '%s\n' "$guest_password" | sudo ./uvm.py create --name app \
  --cpu 1 --ram 1G --guest-ip 10.42.0.10 \
  --ports 80:HTTP,443:HTTPS --password-stdin
unset guest_password

UVM still stores the password in plaintext in the root-only registry. Never reuse a host/service password; replace root / root outside throwaway tests.

Initial Ports

# Adds 80 and 443 to the default private TCP/22 rule.
sudo ./uvm.py create --name frontend --ports 80:HTTP --ports 443:HTTPS

# Exactly TCP/443, with no SSH allowance.
sudo ./uvm.py create --name tls-app --ports 443:HTTPS --no-default-ports

# Managed empty service allowlist, not an unmanaged VM.
sudo ./uvm.py create --name worker --no-default-ports

--ports uses colon syntax PORT[:TYPE], not slash notation or host:guest port mappings. See Private Port Policies below for live edits and network scope. UVM does not install the services named by these labels.

Scenario: Choose CPU, RAM, and Storage

sudo ./uvm.py create --cpu 2 --ram 2G --storage 50G
sudo ./uvm.py create --cpu 0.5 --ram 512

RAM is in MiB by default. Supported suffixes include B, K, M, G, KiB, MiB, and GiB.

UVM enforces a minimum of 128 MiB. Firecracker uses whole vCPUs. A fractional request such as --cpu 0.5 is accepted but rounds the VM up to one Firecracker vCPU; it does not currently apply a host CPU quota.

Storage defaults to 30 GiB, with a 30 GiB minimum. --storage 50G gives the private ext4 root disk 50 GiB of capacity; bare numbers use GiB. Accepted suffixes include M, G, T, MiB, GiB, and TiB (also B and K). GB and TB use binary units, consistently with RAM. Fractional amounts round up to whole MiB. The API and JSON output report capacity as storage_mib. Sparse images consume host space as data is written, rather than reserving the whole configured capacity up front.

Scenario: Request a Specific Guest IP

sudo ./uvm.py create --guest-ip 10.42.0.20
# The legacy spelling --host-ip has the same meaning.

The address must be a private RFC1918 IPv4 address inside the configured guest network. It cannot be the network address, broadcast address, or gateway, or already belong to another persisted VM (including a stopped one).

What Happens During Create

UVM performs these operations in order:

Checks root access, KVM, artifact integrity, and nftables/conntrack support.
Reserves identity, name/IP, and initial port policy in the registry.
Copies the template into a private disk, resizes ext4 to the requested capacity, and provisions credentials and SSH host keys.
Ensures bridge/egress networking and creates an owned TAP attached but down.
Installs and verifies the nftables bridge policy, then activates the TAP.
Starts Firecracker, persists process identity, and configures/boots the guest.
Confirms policy enforcement and records running.

On failure, UVM attempts containment and cleanup. It does not release the disk or IP reservation if safe process exit or cleanup cannot be confirmed; inspect the retained failed record and last_error before retrying.

After updating UVM to fix a create/cleanup error, restart the API server to load the updated code. To discard a failed creation and release its reserved IP, run sudo ./uvm.py delete VM -y with that failed VM's exact ID, or send an authenticated DELETE /vms/VM. Deletion retries the outstanding cleanup and removes the failed VM's private disk. Then retry the create request.

6. Inspect VMs

sudo ./uvm.py list
sudo ./uvm.py list --status running --status dead --json
sudo ./uvm.py list --status stopped -q
sudo ./uvm.py inspect web
sudo ./uvm.py inspect 10.42.0.10 --json

List includes names and policy status. --status filters by observed status and may be repeated; -q/--quiet prints IDs only and is mutually exclusive with --json. Inspect includes retained disk/log/config paths, disk_ready, pending operation, and last_error, without the password.

Status meanings:

Status	Meaning
`starting`	Create/start is in progress or was interrupted.
`running`	Firecracker started; this is not an SSH/application readiness probe.
`stopping`	Stop is in progress or was interrupted.
`stopped`	VMM and TAP are gone; disk remains.
`terminating`	Deletion is pending.
`failed`	A lifecycle operation or cleanup failed.
`dead`	Observed-only status: stored state says running but the VMM is proven gone.
`unknown`	Process identity cannot be established safely; do not guess a PID to kill.

Stored status and observed_status can differ. Inspection does not restart, adopt a legacy process, or reconcile a VM. Policy status is a separate snapshot:

Policy Status	Meaning
`enforced`	The actual owned rules and required bridge attachment settings match.
`inactive`	Managed VM is cleanly stopped; saved rules apply on next start.
`unmanaged`	Legacy record has no managed policy.
`missing`	Expected managed nftables table is absent.
`drift`	Actual rules, ownership, or bridge attachment settings differ.
`quarantined`	The inspected quarantine policy blocks the VM's TAP traffic.
`unknown`	Inspection failed or metadata could not establish policy status.

enforced is not a service-connectivity test or a lasting guarantee against external rule changes. Dry-run target snapshots may show policy unknown because a preview does not verify enforcement. Use inspect or ports list for a current readback.

Logs and Doctor

sudo ./uvm.py logs web
sudo ./uvm.py logs web --tail 50
sudo ./uvm.py logs web --tail 0 -f
sudo ./uvm.py doctor
sudo ./uvm.py doctor --json

Logs reads the local Firecracker stdout/stderr log, defaulting to the last 100 lines. --tail takes a non-negative integer; --follow/-f watches for new output until interrupted. It is not an in-guest log API.

Doctor checks root/KVM access, guest templates and integrity, the non-committing nftables capability probe, and required command availability. It returns a nonzero status if any check fails. It does not repair VMs or rules, boot a guest, or prove SSH/egress connectivity. Use inspect for per-VM policy health.

7. Connect with SSH

UVM provisions a password for the requested account, creating missing custom users during VM creation. The default credentials are root / root. Your guest image must include:

An SSH server.
Standard passwd and shadow files, plus /etc/ssh/sshd_config.
/etc/group and /bin/sh for new users; the default root account must exist.
Root-owned setuid /usr/bin/sudo and /etc/sudoers for a custom login user.

For a custom login, run these commands inside the VM:

sudo -i                 # Authenticate with the custom user's SSH password.
sudo apt update
sudo apt install htop

su - requests root's password, which is separate from the custom user's password. Use sudo -i for administrative access. The default Ubuntu image includes an intact /var/lib/dpkg database; an old Bionic demo image missing that database needs a replacement template and a new VM for package management.

UVM replaces conventional RSA, ECDSA, and Ed25519 SSH host keys in the private disk during creation and removes Firecracker's publicly known demo login key.

This is applied only while creating a new VM. Start/restart preserves the existing disk, credentials, and host keys; legacy disks are not changed retroactively.

sudo ./uvm.py ssh web
ssh root@10.42.0.2
sudo ./uvm.py ssh web --user ubuntu --key ~/.ssh/id_ed25519 --port 2222
sudo ./uvm.py ssh web -- uname -a

By default, UVM uses SSH StrictHostKeyChecking=accept-new and a stable host-key alias derived from the VM ID. This lets a guest IP be reused later without confusing it with the prior VM's known-hosts entry.

When a VM is deleted, UVM removes its explicit IP and uvm-<VM-ID> entries from local SSH caches. This also covers direct ssh user@GUEST_IP connections and hashed hostnames. Plain [host]:port entries are removed for the VM; hashed port entries are checked for TCP/22 and the ports in the VM's allowlist. Other hosts (including other aliases on a shared line), wildcard trust rules, comments, and file ownership/mode are preserved. Missing caches are harmless.

The default files are ~/.ssh/known_hosts and ~/.ssh/known_hosts2 for the effective user and, when running under sudo, its invoking user. Homes are resolved from the local account database. The API uses the user context that launched its server. To select specific local files for a service or custom SSH configuration, set this variable when launching UVM:

export UVM_KNOWN_HOSTS_FILES='/root/.ssh/known_hosts:/home/dgxdev/.ssh/known_hosts'
# Pass it explicitly through sudo when starting the API or running deletion.
sudo env UVM_KNOWN_HOSTS_FILES="$UVM_KNOWN_HOSTS_FILES" ./uvm.py delete VM -y

These paths are local to the UVM host. An SSH client on another computer manages its own cache there. For a stale entry left by a VM deleted with an older UVM version, remove that entry once from the SSH client's account:

ssh-keygen -f "$HOME/.ssh/known_hosts" -R '10.42.0.2'

The key must already be authorized by the guest; UVM does not inject it. --port selects the guest SSH port (default 22), not a host mapping. The guest's sshd and private allowlist must allow that port. SSH does not feed the stored password to the client. A remote command must follow a literal --; UVM SSH options can appear before or after the VM identifier.

Only for troubleshooting a guest you trust, disable host-key verification for one connection:

sudo ./uvm.py ssh web --insecure-host-key

8. Start, Stop, Restart, Resize, and Delete

Scenario: Stop a VM and Keep Its Disk

sudo ./uvm.py stop web --timeout 5

Stop quarantines/deactivates the TAP, terminates Firecracker, removes the TAP, cleans up VM-scoped connection tracking, and removes the owned policy table. The private disk, config, log, credentials, name, IP, and saved ports remain.

Stop is not a guest-coordinated shutdown. UVM sends SIGTERM to the VMM, waits, and sends SIGKILL if needed. --timeout accepts finite non-negative seconds, including fractional values, and defaults to 5. It controls the SIGTERM wait, not the whole command: confirmed exit after SIGKILL and resource cleanup can take additional time. --timeout 0 skips the TERM wait, not exit confirmation. Flush/quiesce important guest workloads separately before stop, restart, or delete; retaining a disk does not guarantee application consistency.

Boot or Restart the Retained Disk

sudo ./uvm.py start web
sudo ./uvm.py restart web --timeout 10

Start boots the same complete private disk, without recopying the template, resetting passwords, or regenerating SSH host keys. Guest changes survive; guest RAM/process state does not. The verified kernel and VMM, KVM, firewall support, and compatible network settings are still required. Removing the original rootfs template does not prevent retained-disk start, although doctor checks templates and can report it missing.

Start on a VM recorded as running with a verified process identity synchronizes its managed port policy without rebooting. Restart is stop followed by start under the same operation lock. Unknown process identity, an incomplete/unsafe disk, or pending deletion blocks a new boot; inspect the error rather than editing registry flags. An interrupted boot with a live VMM must be stopped before start.

Scenario: Resize an Existing VM

sudo ./uvm.py resize web --cpu 2 --ram 4G --storage 80G
sudo ./uvm.py resize web --ram 2G --timeout 10 --json
sudo ./uvm.py resize web --storage 30G

Supply at least one resource; omitted fields retain their current values (or the pending target when retrying an interrupted resize). UVM validates the request and boot prerequisites, records the target, stops the VM, applies the changes, and boots it under one operation lock. A stopped VM also boots after resize. The operation retains the VM ID, IP/MAC, name, username/password, port rules, guest data, and SSH host keys. The dashboard provides these controls under Resize resources on a machine's detail page.

CPU and RAM can increase or decrease. Storage can grow or shrink to at least 30 GiB, subject to the existing data fitting in ext4. Disk resizing uses a temporary sparse copy; the checked and resized filesystem replaces the original only after success. This may require additional physical host space for the copy.

A failed stop prevents resource changes. A failed disk resize leaves the VM stopped with its retained disk and a pending_resize target; correct the cause and retry resize (use a larger storage target if needed). start on the stopped VM also retries the pending target. A boot failure after the new resources were applied retains those values for the next start. stop preserves a pending resize; delete removes the VM and its disk normally.

Scenario: Permanently Remove a VM

sudo ./uvm.py delete web --dry-run
sudo ./uvm.py delete web -y
# Equivalent destructive command: sudo ./uvm.py destroy web -y

Delete performs the stop cleanup, removes the runtime directory/private disk, cleans matching local SSH host-key entries, then removes the registry record, releasing the name and IP. A cache cleanup error retains the pending deletion and IP reservation so cleanup can be retried before the IP is reused. destroy is the same command with the same flags. Back up data first. A failed deletion remains pending; retry delete rather than trying to resurrect it with start.

Multiple VMs and Confirmation

sudo ./uvm.py stop web app --timeout 10 --json
sudo ./uvm.py start web app
sudo ./uvm.py restart --all --dry-run --json
sudo ./uvm.py restart --all -y
sudo ./uvm.py stop --all -y
sudo ./uvm.py start --all -y
sudo ./uvm.py delete --all --dry-run
sudo ./uvm.py delete --all --timeout 5 -y --json

Start/stop/restart/delete accept explicit identifiers or --all, never both. Targets are resolved and deduplicated to a fixed ID snapshot before any prompt. Later-created VMs are not added, and a reused name/IP is not retargeted. If initial lookup fails, nothing is attempted. Once running, the batch reports each result and continues after ordinary per-VM failures; it is not all-or-nothing. An interrupt stops the batch and marks remaining targets skipped.

Every delete and every mutating --all selection requires confirmation when there are targets. Use --yes/-y for scripts; piping yes is not sufficient because the prompt requires a TTY. --dry-run previews without prompting or performing lifecycle mutations, but is not a boot/firewall preflight. Empty --all selections succeed without prompting. Exit codes are 0 for success or cancellation, 1 for operational/batch failure, 2 for invalid CLI input, and 130 for interruption; inspect per-target JSON outcomes as well.

9. Private Port Policies

Syntax and Scope

Use PORT[:TYPE], such as 22, 80:HTTP, 443:HTTPS, and 8080:TCP. Ports must be integers from 1 to 65535. CLI labels are case-insensitive and normalized to TCP, HTTP, or HTTPS; all three use TCP transport. Repeat --ports or separate entries with commas. Identical entries are deduplicated; conflicting labels for the same port in one request are rejected. Slash syntax, ranges, UDP labels, and host:guest mappings are not accepted.

Opening a port permits service traffic to the guest private IPv4 address through the UVM bridge/TAP policy, including host-to-guest and VM-to-VM paths. It does not start a listener, bypass the guest firewall, or configure HTTP or TLS. Public connectivity, host-port mappings/DNAT, DNS publication, reverse proxies, and guest-service TLS management are out of scope. External routing or NAT can change reachability; a private IP is not by itself an exposure guarantee, and UVM does not integrate with external firewall managers yet.

UDP and IPv6 service policies are not yet supported. Unsolicited inbound UDP is blocked, as is IPv6 in both directions on managed TAPs. Guest-initiated IPv4 TCP/UDP (including DNS queries), replies to that egress, ARP, ICMP echo, and required related ICMP errors remain allowed. An empty TCP allowlist is not a complete network disconnect. Other VMs' policies still apply to guest-to-guest egress.

List, Open, Close, and Set

sudo ./uvm.py ports list web --json
sudo ./uvm.py ports list --all
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
sudo ./uvm.py ports close web --ports 80
sudo ./uvm.py ports close web --ports 22 --dry-run --json
sudo ./uvm.py ports close web --ports 22 -y
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
sudo ./uvm.py ports close web --all -y
sudo ./uvm.py ports set web --none -y

list inspects one VM or all VMs; it does not repair rules.
open adds missing ports, preserving other ports and existing labels.
close removes by port number, regardless of label; closing an absent port is harmless. close VM --all means all ports of one VM, not all VMs.
set replaces the entire policy, including labels. --none is an explicit empty managed allowlist. It is not legacy/unmanaged mode.

Only create adds TCP/22 by default. Set/open/close never add SSH implicitly; include 22 yourself when replacing a policy if you want to keep SSH. Open/close/set support --json and --dry-run. Close/set accept --yes/-y: set, close-all, and removing TCP/22 prompt unless approved (an already empty managed policy can be a no-op). Ordinary open has no confirmation flag.

For a running VM, edits persist the desired policy and apply it atomically. For a cleanly stopped VM, they update saved metadata for the next start. After a successful close, subsequent packets on already established service connections are dropped in both directions, as are new connections. There is no guaranteed TCP RST, immediate socket close, or application-level disconnect. Live edits retain conntrack state and preserve unrelated egress.

Explicit Sync and Recovery

sudo ./uvm.py ports sync web app --dry-run --json
sudo ./uvm.py ports sync web
sudo ./uvm.py ports sync --all -y --json

Sync reapplies the saved managed policy to running VMs; it does not start stopped VMs. It accepts multiple identifiers or --all, plus --dry-run, --json, and --yes/-y. --all uses the same confirmation and fixed-target batch behavior as lifecycle commands.

UVM has no background firewall reconciler. If another administrator/tool removes or changes UVM rules, inspect status and explicitly run ports sync after resolving the external conflict. A failed policy update retains the new intent with a pending operation and attempts TAP deactivation and quarantine; it does not restore older, more permissive rules. Inspect last_error, fix the underlying failure, then retry sync. Read-only status commands and starting the API server do not repair lost rules.

Legacy VM Migration

The default registry is /var/lib/uvm/state.json, schema version 3. Older registries are accepted and migrated on the next write. Missing/null ports stays None/unmanaged, not an empty policy and not default SSH enforcement. Open/close/sync/start do not silently adopt a policy for such a VM:

sudo ./uvm.py inspect VM --json
sudo ./uvm.py stop VM
sudo ./uvm.py ports set VM --ports 22,80:HTTP -y
sudo ./uvm.py start VM

Choose the actual required ports, or use ports set VM --none -y. Policy must first be set while the legacy VM is stopped. Start also requires a safe, completely provisioned private disk; very old records without one cannot be resumed by copying a new template over their data.

VMs created before configurable storage retain storage_mib: null until their first resize. That resize uses the existing disk capacity, rounded up to MiB and at least 30 GiB, unless an explicit storage target is supplied. Merely reading or migrating the registry does not resize disks.

A live legacy PID missing only the host boot ID can be adopted during a locked mutation only if its saved start time, running Firecracker executable, exact API-socket command line, and protected binary/socket paths match. A missing start token, inaccessible evidence, or any ambiguity fails closed; UVM will not blindly signal that PID or delete a possibly in-use disk. Listing does not perform adoption. Inspect such failures rather than manually inventing identity fields in state.json.

10. Use the FastAPI Server

The API uses the same local lifecycle services and exposes health, installation, VM lifecycle, diagnostics, bounded VM-log tails, and managed private-port controls. The local dashboard at /dashboard uses those same endpoints. There API also supports fixed-target bulk actions; interactive SSH remains CLI-only. The server does not autostart VMs or reconcile rules, and is not safe to expose directly to the public internet.

Start a Local Server

After installing the project dependencies, start the server as root:

export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  ./uvm.py --serve --host 127.0.0.1 --port 8000

Liveness is at http://127.0.0.1:8000/health; the dashboard is at http://127.0.0.1:8000/dashboard; OpenAPI docs are at http://127.0.0.1:8000/docs. The default bind is 127.0.0.1:8000.

The API token is required even for a loopback server. All VM, diagnostics, and installation routes require it. The dashboard shell, liveness route, and FastAPI's informational /docs, /redoc, and /openapi.json endpoints are available without a token; they do not perform host mutations. The dashboard keeps an entered token only for the active browser session. Tokens must be visible ASCII with no whitespace. Use a protected environment file or secret manager rather than recording a real token in shell history; do not share tokens or guest passwords in reports.

Start a TLS Server

Non-loopback binds require a valid TLS certificate and key. The files must be readable by the server process:

export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
  UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
  ./uvm.py --serve --host 0.0.0.0 --port 8443

For real remote use, use a certificate trusted by clients and restrict network access with a firewall or reverse proxy. Do not use a self-signed development certificate for an untrusted network.

API Authentication

Set variables for the request examples:

export UVM_API_TOKEN='replace-with-the-server-token'
export UVM_API_URL='http://127.0.0.1:8000'

Every management request needs the X-UVM-Token header, as below.

For a TLS server, set UVM_API_URL to an https:// URL and use your CA:

export UVM_API_URL='https://host.example:8443'
curl --cacert /path/to/ca.pem \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms"

11. API Endpoint Examples

Method	Path	Operation
`GET`	`/`	Redirect to `/dashboard`
`GET`	`/dashboard`	Serve the local management dashboard
`GET`	`/health`	Unauthenticated liveness
`GET`	`/diagnostics`	Run host readiness checks
`POST`	`/install`	Install/refresh with a `force` boolean
`GET`	`/vms`	List VM responses
`POST`	`/vms`	Create and boot, HTTP 201 on success
`POST`	`/vms/actions`	Run selected/all VM lifecycle actions from a fixed snapshot
`GET`	`/vms/{identifier}`	Inspect by exact ID, name, or guest IP
`POST`	`/vms/{identifier}/start`	Start a retained VM
`POST`	`/vms/{identifier}/restart`	Restart a VM; accepts `timeout` query parameter
`POST`	`/vms/{identifier}/resize`	Stop, resize CPU/RAM/storage, and boot; accepts `timeout` query parameter
`POST`	`/vms/{identifier}/stop`	Stop and retain disk; accepts `timeout` query parameter
`DELETE`	`/vms/{identifier}`	Delete VM and private disk; accepts `timeout` query parameter
`POST`	`/vms/{vm_id}/ports`	Set, open, or close private ports for an exact VM ID
`POST`	`/vms/{vm_id}/ports/sync`	Reapply the policy for an exact VM ID
`GET`	`/vms/{identifier}/logs`	Read a bounded, non-following log tail

Individual lifecycle requests do not expose a CLI dry-run flag and do not prompt for confirmation. timeout is an optional finite, non-negative query parameter on restart, stop, resize, and delete; it has the same SIGTERM wait semantics as the CLI.

Health, List, and Inspect

# Health is unauthenticated and returns {"status":"ok"}.
curl "$UVM_API_URL/health"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms/api-web"
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms/10.42.0.30"
# An exact VM ID from create/list can replace the name or IP.

Each response contains id, name, cpu, ram_mib, storage_mib, pending_resize, guest_ip, gateway, mac, username, status, observed_status, ports, policy_status, operation, pid, timestamps, and last_error. The response schema excludes passwords and disk/socket/config/log path fields. Operational error messages can still contain host paths; review them before sharing.

Create a VM

Create a named VM with a chosen guest address and private service ports:

curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"name":"api-web","cpu":1,"ram":"1G","storage":"50G","guest_ip":"10.42.0.30","username":"root","password":"replace-me","ports":[{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}],"no_default_ports":false}' \
  "$UVM_API_URL/vms"

Create an explicit empty service policy, with no default SSH allowance:

curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"name":"api-worker","password":"replace-me","ports":[],"no_default_ports":true}' \
  "$UVM_API_URL/vms"

Request fields are cpu, ram, storage, guest_ip, username, password, name, ports, and no_default_ports. The name has the same validation as the CLI. ports is an array of objects (or null/omitted), not a CLI colon string. Each object has a strict integer port from 1 to 65535 and an optional, case-sensitive protocol of TCP (default), HTTP, or HTTPS. no_default_ports is a strict boolean, default false; port 22 is added unless that flag is true, even if ports is null, empty, or contains other ports. Null create input does not create a legacy/unmanaged VM. storage defaults to "30G"; numeric storage values use GiB, with the same 30 GiB minimum and suffixes as the CLI.

The response never includes the password. Omitting credentials still uses root / root. Extra request fields, invalid ports/names, and conflicting labels return HTTP 422. Example passwords above are placeholders; real secrets in curl arguments can enter shell history and process listings.

Start, Restart, Stop, and Delete

# Start a retained private disk.
curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web/start"

# Restart with an explicit SIGTERM wait.
curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web/restart?timeout=10"

curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web/stop"

# Destructive: no interactive confirmation over HTTP.
curl -X DELETE \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web"

Stop returns the VM response with status: "stopped"; delete returns its exact id and status: "terminated". Start and restart return the updated VM response. Start preserves the retained private disk and does not restore guest memory.

Resize CPU, RAM, or Storage

curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"cpu":2,"ram":"4G","storage":"80G"}' \
  "$UVM_API_URL/vms/api-web/resize?timeout=10"

The body accepts optional cpu, ram, and storage fields; at least one must be non-null. For example, {"ram":"2G"} changes only memory. IP and credentials are retained and cannot be edited through this endpoint. A successful response is the updated running VM, including storage_mib and pending_resize: null. During a failed/interrupted resize, pending_resize contains the full target as { "cpu": 2, "ram_mib": 4096, "storage_mib": 81920 } until it is applied.

Bulk Lifecycle Actions

The dashboard batch toolbar uses this endpoint for selected VMs. The server resolves the selected identifiers into a fixed ID snapshot before it invokes the existing serialized bulk lifecycle service.

# Restart two named VMs with a shared TERM wait.
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"action":"restart","identifiers":["api-web","worker"],"timeout":10}' \
  "$UVM_API_URL/vms/actions"

# Sync managed policies for every VM currently in the registry.
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"action":"ports-sync","all_vms":true}' \
  "$UVM_API_URL/vms/actions"

Valid actions are start, stop, restart, delete, and ports-sync. Provide identifiers or all_vms: true, but not both. The response contains one {id, action, outcome, error} object per selected VM. HTTP bulk requests do not prompt for confirmation or provide a dry-run; require an explicit operator confirmation in a caller before destructive use.

Manage Private Ports

The port mutation and sync routes require an exact VM ID returned by create or list. They deliberately do not resolve a name or guest IP, preventing a later name/IP reuse from changing the wrong policy. Port rules are strict JSON objects, not CLI colon strings.

# Replace the full managed TCP allowlist.
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"mode":"set","ports":[{"port":22,"protocol":"TCP"},{"port":443,"protocol":"HTTPS"}]}' \
  "$UVM_API_URL/vms/<vm-id>/ports"

# Add or remove one or more rules without replacing the other rules.
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"mode":"open","ports":[{"port":80,"protocol":"HTTP"}]}' \
  "$UVM_API_URL/vms/<vm-id>/ports"
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"mode":"close","ports":[{"port":80,"protocol":"HTTP"}]}' \
  "$UVM_API_URL/vms/<vm-id>/ports"

# Reapply a policy after an external firewall change.
curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/<vm-id>/ports/sync"

set can use an empty ports array to close every managed service. A legacy VM with ports: null must be stopped and use set before it can start or use the other policy operations. open, close, and sync reject a legacy policy. Mutating a live policy is subject to the same containment and confirmation rules as the CLI.

Diagnostics and Log Tails

The dashboard's System view calls the protected diagnostics endpoint. It runs the same host-readiness checks as uvm doctor, but does not repair a host or change VM state.

curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/diagnostics"

# Read the final 200 lines without opening a streaming follow request.
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/api-web/logs?tail=200"

Log responses have content and truncated fields. tail is an integer from 0 through 1000, defaulting to 100. Reads are limited to the final 256 KiB of a log; truncated: true means an oversized line or the read bound prevented a complete requested tail. Use uvm logs VM --follow for continuous output.

Install or Refresh Through the API

The server process must itself have checksum variables configured for strict installation. Start the server with those environment variables before using this endpoint.

For a deliberately unverified local-development server, the server process must instead include UVM_ALLOW_UNVERIFIED_DOWNLOADS=1. Do not use that mode on a host that runs untrusted workloads.

curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"force": false}' \
  "$UVM_API_URL/install"

Use {"force": true} to refresh all downloaded artifacts.

12. API Errors

Status	Typical meaning
`401`	Missing or incorrect `X-UVM-Token`.
`403`	UVM process lacks root privileges for a host mutation.
`404`	Requested VM was not found.
`409`	Name/identifier conflict, incompatible retained state, or an unsafe/pending operation.
`422`	Invalid JSON body, extra request fields, or invalid UVM input.
`500`	Host command, Firecracker, state, or unexpected operational failure.

UVM operation errors use this envelope:

{"error": {"message": "human-readable explanation"}}

FastAPI request-schema errors use FastAPI's normal detail response format.

13. Configuration Scenarios

Custom Storage and Network

For example, install under /srv/uvm with a different private subnet. All binaries, images, VM runtime directories, locks, and state use the base path:

sudo env \
  UVM_BASE='/srv/uvm' \
  UVM_NETWORK='10.50.0.0/24' \
  UVM_GATEWAY='10.50.0.1' \
  UVM_BRIDGE='uvm50' \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  ./uvm.py install

Use the same base/network/gateway/bridge values for all later lifecycle, inspection, port, and server commands. Start rejects conflicting retained network settings. Use a non-overlapping private subnet; do not assume UVM automatically detects every conflict with host routes or other managers.

Environment Variables

Variable	Use
`UVM_BASE`	Base data directory; default `/var/lib/uvm`
`UVM_NETWORK`	Guest subnet; default `10.42.0.0/24`
`UVM_GATEWAY`	Guest gateway / bridge IP; default `10.42.0.1`
`UVM_BRIDGE`	Linux bridge name; default `uvm0`
`UVM_KNOWN_HOSTS_FILES`	Override deletion's local SSH cache files; colon-separated absolute paths
`UVM_KERNEL_URL`	Override the host-architecture default kernel URL
`UVM_ROOTFS_URL`	Optional prebuilt ext4 URL; unset to build Ubuntu 22.04 locally
`UVM_FIRECRACKER_SHA256`	Strict Firecracker archive checksum
`UVM_KERNEL_SHA256`	Strict kernel checksum
`UVM_ROOTFS_SHA256`	Strict prebuilt rootfs checksum, used with `UVM_ROOTFS_URL`
`UVM_FIRECRACKER_BINARY_SHA256`	Optional local binary checksum
`UVM_ALLOW_UNVERIFIED_DOWNLOADS`	Set to `1` only for local development
`UVM_API_TOKEN`	Required API token
`UVM_API_TLS_CERT`	TLS certificate for remote server binds
`UVM_API_TLS_KEY`	TLS key for remote server binds

14. Files, Logs, and State

The default host layout is:

/var/lib/uvm/
├── bin/                 # Firecracker and Jailer binaries
├── images/              # Shared kernel and rootfs template
├── vms/<vm-id>/         # Private rootfs, config, socket, and log
├── integrity.json       # Artifact checksums and verification status
└── state.json           # Mode-0600 VM inventory and plaintext guest credentials

Registry writes are atomic, with a short state.lock; lifecycle, policy, and installation mutations share operations.lock. Avoid editing state.json, runtime files, TAPs, or UVM-owned nftables/iptables rules by hand. Use supported commands so ownership, locking, and cleanup remain correct. Back up important disks while stopped and keep the associated registry backup protected.

state.json contains plaintext guest passwords. Keep it root-only, never copy it into logs or bug reports, and use unique non-default passwords outside local throwaway environments. Prefer inspect/list to dumping the registry. Raw Firecracker logs may include guest/application secrets; redact them before sharing.

15. Networking Troubleshooting

UVM enables IPv4 forwarding and installs uvm-egress-commented iptables MASQUERADE and forward/return rules. These match the private subnet and bridge without fixing an uplink, so guest traffic follows the host's current routes. Scoped forwarding exceptions are inserted ahead of existing host-chain rules. Private port filtering is a separate native nftables bridge policy, not DNAT. It must be installed and read back before the TAP is active. The bridge must have STP disabled and each TAP must have bridge-slave group_fwd_mask 0x4000 (16384), so LLDP takes the early identity-filter path rather than bypassing it before MAC learning. These are software-bridge requirements; do not offload traffic around the hooks. Native bridge conntrack needs Linux

= 5.3; this layer does not configure br_netfilter sysctls.

Check the bridge and TAP state:

ip -d link show uvm0
ip addr show dev uvm0
ip route show default

Check port policy and the separate egress NAT/forwarding rules:

sudo ./uvm.py doctor
sudo ./uvm.py inspect web --json
sudo ./uvm.py ports list --all
sudo nft --json list tables bridge
sudo iptables -t nat -S POSTROUTING
sudo iptables -S FORWARD

After upgrading UVM, apply the sharing rules to an already-running VM with:

sudo ./uvm.py start web

start on a running VM refreshes its private policy and shared host networking without a reboot. Use the updated CLI checkout, or restart the API server to load the updated code before using its start endpoint. Subsequent host disconnects, reconnects, and route changes require no UVM command.

Check DNS and HTTPS from the guest:

sudo ./uvm.py ssh web -- getent ahostsv4 example.com
sudo ./uvm.py ssh web -- curl -4 --connect-timeout 5 --max-time 15 -I https://example.com

Common issues:

Symptom	First checks
Guest cannot reach the internet	Confirm the host can reach the same destination, bridge/TAP are up, and `uvm-egress` NAT/FORWARD rules exist; use `start VM` to refresh sharing.
IP connectivity works but DNS fails	Check guest `/etc/resolv.conf` and DNS reachability. The Ubuntu template uses `1.1.1.1` and `8.8.8.8`; host loopback resolvers such as `127.0.0.53` are not reachable as guest loopback addresses.
Requested guest IP is rejected	Confirm it is inside the configured subnet and not used, gateway, network, or broadcast.
Create says TAP already exists	Inspect the interface; do not delete it unless you know it is stale UVM state.
Existing bridge is rejected	Verify it is a software Linux bridge with STP disabled.
Firewall unavailable/unknown	Check privileges, `nft`, `conntrack`, kernel support, and bridge/TAP settings.
Policy missing/drift after external reload	Resolve the other manager's conflict, then explicitly run `ports sync VM`; there is no automatic repair.
SSH cannot connect	Confirm boot/sshd, guest user and password/key, chosen SSH port, allowlist, and guest firewall.
Closed SSH session hangs rather than disconnects	Close drops subsequent traffic in both directions; a TCP RST is not guaranteed.

External firewall integration (including UFW, firewalld, Docker, and libvirt) is deferred. A UVM policy readback does not prove that external rules permit traffic, or that they will not later remove UVM's rules.

SSH Startup and Guest Entropy

running checks the Firecracker process, and enforced checks the private firewall policy. Neither probes the guest's SSH listener. If ping works but TCP/22 returns Connection refused, inspect the guest boot log:

sudo ./uvm.py logs VM --tail 200
sudo ./uvm.py inspect VM --json

An SSH service stuck at Starting OpenBSD Secure Shell server..., together with an unfinished Load/Save Random Seed service and no random: crng init done, strongly suggests entropy starvation. A login prompt can still appear while SSH is waiting for its cryptographic random generator. On the old ARM demo kernel, random: fast init done is not full cryptographic RNG initialization.

Earlier UVM versions downloaded a Linux 4.14.174 ARM quickstart kernel without the built-in virtio RNG driver and did not attach an entropy device. Recovery requires both the updated UVM boot code and an RNG-capable kernel. A restart with the old cached kernel is insufficient.

From the updated checkout, stop the affected VM before refreshing assets. Stop any other VMs using the same installed Firecracker binary before the forced reinstall as well. Use the Firecracker archive checksum for the configured release and a new trusted checksum for the selected architecture's Linux 6.1.155 kernel; the old kernel checksum will not match:

sudo ./uvm.py stop VM
sudo env -u UVM_KERNEL_URL \
  UVM_FIRECRACKER_SHA256='<trusted-firecracker-archive-sha256>' \
  UVM_KERNEL_SHA256='<trusted-new-kernel-sha256>' \
  ./uvm.py install --force
sudo ./uvm.py start VM
sudo ./uvm.py logs VM --tail 100
sudo ./uvm.py ssh VM

This uses the default kernel URL and boots the retained private disk with the updated device configuration. The VM's identity, files, credentials, and SSH host keys are retained. install --force also refreshes the shared rootfs template for future creates. If using a custom kernel, supply its URL and matching checksum with the required drivers instead. For an explicitly unverified development installation, retain the existing UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 setting as described in the installation section.

Restart a running UVM API server to load the updated code before using dashboard start/restart actions. Look for Started OpenBSD Secure Shell server in the new boot log and confirm an actual SSH connection. start on an already-running VM only refreshes networking; use a stopped VM or restart to attach the device.

16. Important Limitations

Keep these constraints in mind when deciding whether UVM fits a scenario:

Firecracker is launched directly; the downloaded Jailer is not yet used.
No quotas or cgroup CPU, memory, I/O, or process limits are applied.
Fractional CPU requests are advisory only.
No autostart, background lifecycle/firewall reconciliation, or guest-coordinated stop.
No UDP/IPv6 service policies, public inbound connectivity setup, host-port mapping, DNS publication, guest reverse proxy/TLS setup, DHCP, load balancing, snapshots, multi-network tenancy, or multi-host scheduling.
The API uses one shared bearer token, not individual users, roles, or tenants.
There is no guest readiness probe beyond Firecracker process state.
UVM is not a hardened multi-tenant platform.

17. Safe Operating Checklist

Before relying on a VM for useful work:

Use strict checksums for installed artifacts.
Confirm /dev/kvm and nested virtualization are available.
Confirm UVM generated unique guest SSH host keys and set a non-default password.
Test guest egress and SSH on the actual host network.
Quiesce guest workloads before stopping/restarting and back up data before deletion.
Keep the API loopback-only unless you have a strong remote-access need.
For remote API access, use a long random token, valid TLS, and firewall restrictions.
Review /var/lib/uvm/vms/<vm-id>/firecracker.log after failed starts.

18. Verification

From the project root, normal unit tests need no root, KVM, or host network changes. Install the test extra in your Python environment for API tests:

python3 -m pip install '.[test]'
python3 -B -m unittest discover -s tests -v

The opt-in packet test runs only in disposable isolated user/network namespaces, which may require privileges. It needs unshare, ip, bridge, nft, iptables, conntrack, and sysctl:

UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v

Namespace creation was denied in this workspace (Operation not permitted), so the packet test skipped. That is a permission limitation, not real-packet or real-host validation. The harness uses veths as TAP stand-ins and simulated upstreams to check NAT, offline startup, reconnects and uplink changes. It does not boot Firecracker or validate the host's real TAP setup. Run actual VM/SSH/egress checks on an isolated Linux host before relying on the deployment.

19. Get Help

./uvm.py --help
./uvm.py create --help
./uvm.py ports set --help
./uvm.py ssh --help

When reporting a problem, collect:

The command or HTTP request, with passwords and tokens redacted.
The full error message.
Password-free output from list --json, inspect VM --json, and doctor.
The VM Firecracker log, reviewed for guest/application secrets.
The relevant UVM_* variables, with secrets such as UVM_API_TOKEN removed.

Never include /var/lib/uvm/state.json because it contains guest passwords.

57 KiB Raw Blame History