633 lines
29 KiB
Markdown
633 lines
29 KiB
Markdown
# uvm
|
|
|
|
`uvm` is a small, local Firecracker microVM command-line manager for Ubuntu.
|
|
It creates one Firecracker process per VM, attaches it to a host TAP device,
|
|
and keeps a private disk and local registry for named VMs. The CLI supports
|
|
create, start, stop, restart, resize, delete, inspection, SSH, logs, diagnostics,
|
|
host-shared IPv4 internet access, and enforced private-IPv4 TCP port allowlists.
|
|
|
|
This directory is a self-contained project. All UVM code, tests, packaging
|
|
metadata, launcher scripts, and documentation live here.
|
|
|
|
For architecture, lifecycle invariants, extension guidance, and contributor
|
|
workflows, see [DEVELOPER.md](DEVELOPER.md).
|
|
|
|
For end-to-end operator scenarios, CLI examples, and HTTP API calls, see
|
|
[USAGE.md](USAGE.md).
|
|
|
|
For the original firewall proposal and backend evaluation, see
|
|
[FIREWALL.md](FIREWALL.md). That proposal is design background, not the current
|
|
feature contract; these guides describe the implementation in the workspace.
|
|
|
|
## Quick Start
|
|
|
|
Run commands from this project root:
|
|
|
|
```sh
|
|
./uvm.py --help
|
|
python3 -B -m uvm --help
|
|
```
|
|
|
|
The `pyproject.toml` file also defines an installable `uvm` console command.
|
|
The project is an application, not a reusable Python SDK.
|
|
|
|
## Requirements
|
|
|
|
- Ubuntu or another compatible Linux host
|
|
- Python 3.12 or newer
|
|
- Root access for UVM commands that read or mutate the protected VM registry
|
|
- KVM enabled and accessible at `/dev/kvm`
|
|
- Linux 5.3 or newer with native bridge conntrack and safe pidfd support
|
|
- An x86_64 or ARM64 (`aarch64`) host, including NVIDIA DGX Spark
|
|
- Working host networking, `iproute2`, `iptables`, `nftables`, `conntrack`,
|
|
`coreutils`, `e2fsprogs`, `util-linux`, `openssl`, and `openssh-client`
|
|
- An STP-disabled software bridge; no offload path that bypasses bridge hooks
|
|
|
|
`uvm install` installs the required host packages. It validates `/dev/kvm`
|
|
before downloading VM artifacts. Boot operations additionally check that the
|
|
native nftables bridge policy is supported; installing packages alone is not
|
|
a connectivity or isolation test.
|
|
|
|
Firecracker and the guest kernel are selected automatically for the host
|
|
architecture. The default rootfs is built locally from Ubuntu 22.04 (Jammy)
|
|
packages with OpenSSH, sudo, and a complete apt/dpkg database. On NVIDIA DGX
|
|
Spark, UVM uses ARM64 assets and packages.
|
|
`arm64` and `amd64` architecture names are normalized to `aarch64` and
|
|
`x86_64`, respectively. Custom `UVM_KERNEL_URL` and `UVM_ROOTFS_URL` overrides
|
|
must point to assets compatible with the host architecture.
|
|
|
|
The default guest kernel is Linux 6.1.155 from Firecracker's published `v1.15`
|
|
CI artifacts, with built-in `virtio-rng` support. UVM attaches a host-backed
|
|
entropy device before each guest boot so services such as SSH can initialize
|
|
their cryptographic random generator. Custom kernels must include
|
|
`CONFIG_HW_RANDOM=y` and `CONFIG_HW_RANDOM_VIRTIO=y` for this device.
|
|
|
|
Older installs used a Linux 4.14 ARM demo kernel and supplied no entropy
|
|
device. If SSH stays at **Starting OpenBSD Secure Shell server** while ping
|
|
works, see [SSH startup and guest entropy](USAGE.md#ssh-startup-and-guest-entropy)
|
|
for the kernel-refresh and retained-VM recovery steps.
|
|
|
|
## Trusted Installation
|
|
|
|
Installation requires trusted SHA-256 digests by default. Supply digests from
|
|
an independently trusted source for the selected architecture before running
|
|
the privileged installer:
|
|
|
|
```sh
|
|
sudo env \
|
|
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
|
|
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
|
|
./uvm.py install
|
|
```
|
|
|
|
The Ubuntu build uses `debootstrap`, authenticated Ubuntu archive signatures,
|
|
and native packages. It bootstraps the base release first, then uses apt to
|
|
apply updates/security fixes and install the guest packages. UVM installs the
|
|
builder prerequisites and records the built ext4 image's SHA-256 in the
|
|
integrity manifest. Building requires root
|
|
with mount/PID namespace support and internet access to Ubuntu mirrors.
|
|
|
|
Use `--force` to refresh downloads and rebuild the Ubuntu template:
|
|
|
|
```sh
|
|
sudo env \
|
|
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
|
|
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
|
|
./uvm.py install --force
|
|
```
|
|
|
|
To use a prebuilt ext4 image instead, set both `UVM_ROOTFS_URL` and its trusted
|
|
`UVM_ROOTFS_SHA256`. Leave both unset for the default Ubuntu build.
|
|
|
|
### Upgrade from the Stripped Bionic Demo Image
|
|
|
|
Run `install --force` with the updated code and with `UVM_ROOTFS_URL` and
|
|
`UVM_ROOTFS_SHA256` unset, then create a new VM. The replacement template
|
|
includes sudo and the package database that the old demo image lacked.
|
|
Existing VM disks retain their original OS and credentials; installation
|
|
updates the template used for future VMs. Restart a running API server to load
|
|
the updated provisioner before creating VMs through the dashboard.
|
|
|
|
If the build fails, its detailed bootstrap/package output is retained in
|
|
`/var/lib/uvm/images/ubuntu-build.log` (mode `0600`). Correct the build failure
|
|
and rerun installation before creating a custom-user VM; a failed build leaves
|
|
the previous template installed.
|
|
|
|
For local experimentation only, the old unverified quick-start behavior is
|
|
available through an explicit opt-out:
|
|
|
|
```sh
|
|
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
|
|
```
|
|
|
|
An unverified manifest is not accepted as publisher trust in a later strict
|
|
run. Reinstall with trusted digests to establish a verified local manifest.
|
|
To boot VMs from an unverified local-development install, retain
|
|
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create`, `start`, `restart`, and `resize`
|
|
commands and on the API server process.
|
|
|
|
## Commands
|
|
|
|
These are separate operation examples, not a script to run from top to bottom.
|
|
|
|
```sh
|
|
# Install after supplying the trusted checksum variables shown above.
|
|
# A bare `install` command fails under the default strict integrity policy.
|
|
|
|
# Create a named VM: 1 CPU, 512 MiB RAM, 30 GiB storage, root/root, and private TCP/22.
|
|
# Use a unique non-default password for anything beyond a throwaway VM.
|
|
sudo ./uvm.py create --name web
|
|
|
|
# Choose resources, a private guest IP, and additional service ports.
|
|
sudo ./uvm.py create --name app --cpu 1 --ram 1G --storage 50G \
|
|
--guest-ip 10.42.0.10 --ports 80:HTTP,443:HTTPS
|
|
|
|
# Read a password from one line of stdin, rather than a --password argument.
|
|
# USAGE.md includes a non-echoing Bash prompt example.
|
|
sudo ./uvm.py create --name worker --password-stdin
|
|
|
|
# Explicitly allow no inbound service ports, including SSH.
|
|
sudo ./uvm.py create --name closed --no-default-ports
|
|
|
|
# Inspect persisted state and observed process/policy status.
|
|
sudo ./uvm.py list
|
|
sudo ./uvm.py list --status running --json
|
|
sudo ./uvm.py list -q
|
|
sudo ./uvm.py inspect web --json
|
|
sudo ./uvm.py logs web --tail 50 --follow
|
|
sudo ./uvm.py doctor --json
|
|
|
|
# Resolve VMs by exact ID, name, or guest IP.
|
|
sudo ./uvm.py ssh web
|
|
sudo ./uvm.py ssh app --port 22 -- uname -a
|
|
ssh root@10.42.0.10
|
|
|
|
# Stop and boot the same private disk, or restart selected VMs.
|
|
sudo ./uvm.py stop web --timeout 5
|
|
sudo ./uvm.py start web
|
|
sudo ./uvm.py restart web app --timeout 10
|
|
|
|
# Stop, change resources, and boot the same VM with its existing identity/data.
|
|
sudo ./uvm.py resize app --cpu 2 --ram 4G --storage 80G
|
|
sudo ./uvm.py resize app --ram 2G
|
|
|
|
# Preview bulk operations before confirming them.
|
|
sudo ./uvm.py stop --all --dry-run --json
|
|
sudo ./uvm.py stop --all -y
|
|
sudo ./uvm.py start --all -y
|
|
sudo ./uvm.py restart --all -y
|
|
sudo ./uvm.py delete --all --dry-run
|
|
sudo ./uvm.py delete web -y
|
|
# destroy is an alias for delete, with the same flags and destructive effect.
|
|
```
|
|
|
|
`start`, `stop`, `restart`, and `delete` accept one or more VM identifiers or
|
|
`--all`, plus `--json`, `--dry-run`, and `--yes`/`-y`. Deletion and mutating
|
|
`--all` selections require confirmation; non-interactive callers must use
|
|
`--yes`. `--timeout` is available on stop/restart/delete (and `destroy`), not
|
|
start. It is a finite non-negative SIGTERM wait in seconds, default `5`, not
|
|
a total command deadline or a guest shutdown timeout.
|
|
|
|
### Storage and Resizing
|
|
|
|
New VMs default to a **30 GiB private disk**, which is also the minimum.
|
|
`--storage` accepts values such as `30G`, `50G`, or `1T`; bare numbers use GiB.
|
|
As with RAM, `G`/`GB`/`GiB` are binary units. Capacities round up to whole MiB
|
|
and are returned as `storage_mib`. Disks are sparse: the configured capacity
|
|
does not immediately reserve that much physical host space.
|
|
|
|
`resize VM` accepts any combination of `--cpu`, `--ram`, and `--storage`, plus
|
|
`--timeout` and `--json`. It stops the VM, resizes the ext4 disk when needed,
|
|
updates the resource settings, and boots it. A previously stopped VM also boots
|
|
at the end. VM ID, IP/MAC, username/password, port rules, guest files, and SSH
|
|
host keys are retained. The dashboard exposes the same operation under
|
|
**Resize resources**; the API uses `POST /vms/{identifier}/resize`.
|
|
|
|
Storage can grow or shrink, provided ext4 can fit its existing data into the
|
|
requested capacity. Disk changes are prepared on a temporary sparse copy and
|
|
published after verification. Failed resizes retain a `pending_resize` target
|
|
for retry; `start` on the stopped VM completes that target before booting.
|
|
Older VMs report `storage_mib: null` until their first resize, which retains
|
|
their current capacity or grows it to the 30 GiB minimum when storage is omitted.
|
|
|
|
Stop sends SIGTERM and, if needed, SIGKILL to the Firecracker VMM. It is **not
|
|
a guest-coordinated shutdown**. Start boots the retained disk without resetting
|
|
guest data, SSH host keys, or credentials; it does not restore guest memory.
|
|
Delete permanently removes the disk and releases the VM's name and IP after
|
|
cleanup. See [USAGE.md](USAGE.md) for the complete flag and recovery reference.
|
|
|
|
Names are unique local identifiers, not DNS names. `--host-ip` remains an
|
|
alias for `--guest-ip`; both select the guest's private IPv4 address. JSON
|
|
output omits passwords, with warnings and host-command traces on stderr.
|
|
|
|
SSH uses `StrictHostKeyChecking=accept-new` and a VM-ID-specific host-key
|
|
alias by default. `--user`, `--key`, and `--port` customize the SSH client;
|
|
append a remote command after a literal `--`. `--insecure-host-key` disables
|
|
verification for one connection. SSH does not automatically supply the stored
|
|
password or open a closed port.
|
|
|
|
Deletion removes matching local SSH `known_hosts` entries for the guest IP and
|
|
UVM's VM-ID host-key alias before releasing the IP for reuse. It checks the
|
|
effective user's files and, under `sudo`, the invoking user's files, including
|
|
hashed entries. Unrelated hosts and file ownership/permissions are preserved.
|
|
For API services or custom SSH cache locations, set `UVM_KNOWN_HOSTS_FILES` to a
|
|
colon-separated list of absolute paths on the UVM host. See the SSH section in
|
|
[USAGE.md](USAGE.md#7-connect-with-ssh) for configuration and one-time cleanup of
|
|
entries left by older versions.
|
|
|
|
### Private Ports
|
|
|
|
```sh
|
|
sudo ./uvm.py ports list web --json
|
|
sudo ./uvm.py ports list --all
|
|
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
|
|
sudo ./uvm.py ports close web --ports 80
|
|
sudo ./uvm.py ports set web --ports 22,443:HTTPS --dry-run --json
|
|
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
|
|
sudo ./uvm.py ports close web --all -y
|
|
sudo ./uvm.py ports set web --none -y
|
|
sudo ./uvm.py ports sync web
|
|
sudo ./uvm.py ports sync --all --dry-run
|
|
sudo ./uvm.py ports sync --all -y
|
|
```
|
|
|
|
Rules use `PORT[:TYPE]`, for example `22`, `80:HTTP`, `443:HTTPS`, or
|
|
`8080:TCP`, repeated or comma-separated. HTTP and HTTPS are **TCP labels**,
|
|
not proxies or TLS configuration. Create adds `22:TCP` even when other ports
|
|
are supplied, unless `--no-default-ports` is set. Later port edits do not
|
|
automatically add SSH back. `ports close VM --all` closes ports on that one
|
|
VM; only `ports list --all` and `ports sync --all` select all VMs.
|
|
|
|
Port mutations support `--dry-run` and `--json`; close/set/sync also support
|
|
`--yes`/`-y`. Set, close-all, and removal of TCP/22 require confirmation
|
|
(an already empty managed policy can be a no-op). Closing a service blocks
|
|
subsequent packets on existing service connections in **both directions**, as
|
|
well as new connections. This is packet dropping, not a guaranteed TCP RST or
|
|
immediate application disconnect.
|
|
|
|
These rules govern access to the **guest private IP**. They do not start a
|
|
guest service or provide public inbound connectivity, host-port mappings,
|
|
DNS publication, reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not
|
|
yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required
|
|
ARP/ICMP traffic remain allowed; managed TAPs block IPv6.
|
|
|
|
## Management API
|
|
|
|
Install the project dependencies before serving the API:
|
|
|
|
```sh
|
|
python3 -m pip install .
|
|
```
|
|
|
|
Launch the FastAPI management server using the environment where the project
|
|
dependencies were installed (use the full console-script path if needed):
|
|
|
|
```sh
|
|
export UVM_API_TOKEN='replace-with-a-long-random-secret'
|
|
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
|
|
uvm --serve --host 127.0.0.1 --port 8000
|
|
|
|
# Equivalent when running the source checkout directly.
|
|
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
|
|
./uvm.py --serve --host 127.0.0.1 --port 8000
|
|
```
|
|
|
|
The default bind address is `127.0.0.1` and the default port is `8000`.
|
|
OpenAPI documentation is available at `/docs` once the server is running. The
|
|
root URL redirects to the local dashboard at `/dashboard`; enter the API token
|
|
there for the current browser session.
|
|
|
|
The dashboard includes a fleet overview, searchable machine collection, private
|
|
network and port-policy view, host diagnostics, and machine provisioning. Open a
|
|
machine to manage its lifecycle, edit or sync its port rules, copy its SSH command,
|
|
or read recent logs. Select machines in the collection for bulk operations.
|
|
Use `Ctrl+K` / `⌘K` (or `/`) to find a machine or page. Fleet data refreshes every
|
|
five seconds while the tab is visible; host checks refresh every thirty seconds
|
|
and can also be run manually. Charts show observed machine states and allocated
|
|
resources, rather than guest CPU or memory utilization. All frontend assets,
|
|
including the font, are served locally; no frontend build step is required.
|
|
|
|
The server exposes these routes:
|
|
|
|
| Method | Route | Purpose |
|
|
|---|---|---|
|
|
| `GET` | `/` | Redirect to the local dashboard |
|
|
| `GET` | `/dashboard` | Serve the local management dashboard |
|
|
| `GET` | `/health` | Unauthenticated liveness response |
|
|
| `POST` | `/exit` | Gracefully stop the UVM API server |
|
|
| `GET` | `/diagnostics` | Run host readiness checks for the dashboard/system view |
|
|
| `POST` | `/install` | Install or refresh host assets; body: `{ "force": false }` |
|
|
| `GET` | `/vms` | List VM records and observed status |
|
|
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
|
|
| `POST` | `/vms/actions` | Run a fixed snapshot of selected or all VM lifecycle actions |
|
|
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
|
|
| `POST` | `/vms/{identifier}/start` | Boot a retained private disk |
|
|
| `POST` | `/vms/{identifier}/restart` | Stop then boot a VM; optional `timeout` query parameter |
|
|
| `POST` | `/vms/{identifier}/resize` | Stop, resize CPU/RAM/storage, and boot; optional `timeout` query parameter |
|
|
| `POST` | `/vms/{identifier}/stop` | Stop a VM while retaining its private disk; optional `timeout` query parameter |
|
|
| `DELETE` | `/vms/{identifier}` | Delete a VM and release its resources; optional `timeout` query parameter |
|
|
| `POST` | `/vms/{vm_id}/ports` | Set, open, or close private port rules for an exact VM ID |
|
|
| `POST` | `/vms/{vm_id}/ports/sync` | Reapply a managed private port policy for an exact VM ID |
|
|
| `GET` | `/vms/{identifier}/logs` | Read a bounded, non-following tail of a VM log |
|
|
|
|
API port rules are objects such as `{"port": 80, "protocol": "HTTP"}`;
|
|
`no_default_ports` is a boolean, default `false`. Create also accepts `storage`,
|
|
defaulting to `"30G"`; resize accepts optional `cpu`, `ram`, and `storage` fields.
|
|
Port changes use
|
|
`{"mode":"set"|"open"|"close", "ports":[...]}`. A legacy VM must be
|
|
stopped and use `set` to establish a managed policy. Responses include `name`,
|
|
`ports`, `policy_status`, and `operation`, but never passwords. See
|
|
[USAGE.md](USAGE.md) for request examples and strict schema details.
|
|
Log reads use an optional `tail` query parameter from `0` to `1000`, default
|
|
`100`, and are capped at the final 256 KiB of the log.
|
|
|
|
Bulk requests use an action (`start`, `stop`, `restart`, `delete`, or
|
|
`ports-sync`) plus either `identifiers` or `all_vms: true`; they return a result
|
|
for each fixed target and do not prompt for confirmation. Interactive SSH
|
|
remains CLI-only. The API and dashboard are not background reconcilers.
|
|
|
|
### API Security
|
|
|
|
The server controls privileged host operations. `UVM_API_TOKEN` is required
|
|
for every API server, including loopback-only deployments. All management
|
|
routes require it in the `X-UVM-Token` header. The dashboard shell, liveness
|
|
route, and FastAPI's informational `/docs`, `/redoc`, and `/openapi.json`
|
|
endpoints do not require a token; the dashboard stores the entered token only
|
|
for its current browser session and applies it to management requests.
|
|
|
|
A non-loopback bind additionally requires TLS certificate and key paths:
|
|
|
|
```sh
|
|
export UVM_API_TOKEN='replace-with-a-long-random-secret'
|
|
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
|
|
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'
|
|
sudo env \
|
|
UVM_API_TOKEN="$UVM_API_TOKEN" \
|
|
UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
|
|
UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
|
|
./uvm.py --serve --host 0.0.0.0 --port 8000
|
|
```
|
|
|
|
Send the token on management requests:
|
|
|
|
```sh
|
|
curl \
|
|
-H "X-UVM-Token: $UVM_API_TOKEN" \
|
|
http://127.0.0.1:8000/vms
|
|
```
|
|
|
|
Stop the API server without stopping its managed VMs:
|
|
|
|
```sh
|
|
curl -X POST \
|
|
-H "X-UVM-Token: $UVM_API_TOKEN" \
|
|
http://127.0.0.1:8000/exit
|
|
```
|
|
|
|
The endpoint returns `202 {"status":"stopping"}` and then gracefully terminates
|
|
the UVM API process. It is a protected management endpoint; it does not stop or
|
|
delete managed VMs.
|
|
|
|
Do not expose this server directly to an untrusted network. For remote access,
|
|
keep TLS enabled and consider placing it behind an additional authenticated
|
|
reverse proxy.
|
|
|
|
## Guest Image Contract
|
|
|
|
The installed kernel and rootfs are templates. `create` copies the rootfs to
|
|
the VM runtime directory before Firecracker opens it read-write, so VMs do not
|
|
share a mutable root disk.
|
|
|
|
During creation, UVM updates the private ext4 disk with a password for the
|
|
requested guest account and enables OpenSSH password authentication. Missing
|
|
custom usernames are created with an unused regular-user UID, a primary group,
|
|
a private `/home/USERNAME` directory, and `/bin/sh` as their login shell. The
|
|
defaults are username `root` and password `root`. A custom image must provide:
|
|
|
|
- `sshd`
|
|
- `/etc/passwd`, `/etc/shadow`, and `/etc/ssh/sshd_config`
|
|
- `/etc/group` and `/bin/sh` when creating a new user
|
|
- A root-owned, setuid `/usr/bin/sudo` and `/etc/sudoers` for custom login users
|
|
|
|
Custom login users receive password-authenticated sudo access. Inside a new VM:
|
|
|
|
```sh
|
|
sudo -i # Enter the same password used for this user's SSH login.
|
|
sudo apt update
|
|
sudo apt install htop
|
|
```
|
|
|
|
`su -` asks for root's separate password; creating a custom login does not set
|
|
root's password. Use `sudo -i` for an administrative shell. The default Ubuntu
|
|
template retains apt's package metadata and enables `main` and `universe`,
|
|
including updates/security repositories. Guest DNS defaults to `1.1.1.1` and
|
|
`8.8.8.8` in `/etc/resolv.conf`.
|
|
|
|
Existing accounts retain their UID, groups, home directory, and shell. A new
|
|
user reuses a same-named group if present; otherwise UVM creates one with an
|
|
unused GID. `/etc/gshadow` is updated when present. The default `root` account
|
|
must already exist in the image.
|
|
|
|
The known public demo key bundled in Firecracker's older Bionic image is
|
|
removed from each private disk. UVM also replaces the conventional RSA, ECDSA,
|
|
and Ed25519 SSH host keys in every private disk so cloned VMs have distinct
|
|
host identities. Never rely on SSH private keys baked into a shared template.
|
|
|
|
## Host Networking
|
|
|
|
By default, UVM manages:
|
|
|
|
```text
|
|
Network: 10.42.0.0/24
|
|
Gateway: 10.42.0.1
|
|
Bridge: uvm0
|
|
TAPs: uvm-<unique suffix>
|
|
```
|
|
|
|
VMs share the host's IPv4 internet connection by default. When the host has
|
|
internet access, guests can initiate outbound connections through it. When
|
|
the host loses connectivity, guest internet access is lost too; the private
|
|
VM network remains usable. Reconnecting the host restores guest egress.
|
|
|
|
UVM creates the bridge when absent, validates an existing bridge before using
|
|
it, enables IPv4 forwarding, and installs subnet-scoped `iptables` masquerade
|
|
and forward/return rules. These rules are installed even while the host is
|
|
offline and follow its current routing table, so switching uplinks does not
|
|
require restarting VMs. Existing connections may need application retries
|
|
after an outage or upstream address change. Guest IPs, gateway, bridge name,
|
|
and guest boot netmask derive from the configured CIDR.
|
|
|
|
Forwarding exceptions are inserted ahead of existing host-chain rules and
|
|
allow guest-originated traffic and its replies. Masquerading excludes traffic
|
|
back onto the UVM bridge, preserving private peer addresses.
|
|
|
|
To apply updated sharing rules to an existing running VM, use:
|
|
|
|
```sh
|
|
sudo ./uvm.py start web
|
|
```
|
|
|
|
On a running VM, `start` refreshes its private policy and shared host networking
|
|
without rebooting it. New VM boots set up the same networking automatically.
|
|
|
|
Private VM creation and start require an RFC 1918 subnet (`10.0.0.0/8`,
|
|
`172.16.0.0/12`, or `192.168.0.0/16`). Retained VMs prevent incompatible
|
|
gateway, subnet, or bridge changes within the same registry. Private
|
|
port enforcement uses separate per-VM native nftables `bridge` tables, not
|
|
DNAT or host listeners. Policy is installed and read back before an owned TAP
|
|
is activated and before the guest starts. The software bridge must have STP
|
|
disabled; each TAP's bridge-slave `group_fwd_mask` is set to `0x4000` so LLDP
|
|
cannot bypass the early identity filter. Native bridge conntrack requires
|
|
Linux >= 5.3; no `br_netfilter` sysctl setup is required by this policy layer.
|
|
|
|
Host-to-VM and VM-to-VM traffic crossing these hooks is filtered. An allowed
|
|
port is not a reachability guarantee, and a private IP alone is not protection
|
|
against administrator-supplied external routing or NAT. Integration with UFW,
|
|
firewalld, Docker, or other external firewall managers remains deferred.
|
|
Do not offload traffic around the software bridge hooks.
|
|
|
|
There is no autostart service or daemon reconciliation. After externally lost
|
|
or changed private policy, inspect its status and explicitly run `ports sync`.
|
|
Use `start` to restore shared bridge/NAT/forwarding rules as well. Read-only
|
|
commands do not repair rules; port sync does not boot stopped VMs.
|
|
|
|
Use one authoritative registry per bridge/subnet. Independent `UVM_BASE`
|
|
directories do not coordinate IP or MAC allocation with each other.
|
|
|
|
## Persistent State
|
|
|
|
The default data directory is `/var/lib/uvm`:
|
|
|
|
```text
|
|
/var/lib/uvm/
|
|
├── bin/ # Firecracker and Jailer binaries
|
|
├── images/ # Kernel and rootfs templates
|
|
├── vms/<vm-id>/ # Per-VM disk, config, log, and API socket
|
|
├── integrity.json # Download/build checksums and verification provenance
|
|
└── state.json # Root-only VM inventory and guest credentials
|
|
```
|
|
|
|
The default registry is `/var/lib/uvm/state.json`, schema version `3`. Writes
|
|
are locked and atomic. Lifecycle, port-policy, and installation mutations
|
|
share an operation lock; bulk operations use a fixed snapshot of VM IDs.
|
|
|
|
The registry stores guest usernames and passwords in plaintext, so `state.json`
|
|
is mode `0600`. Treat it as a secret, do not include it in bug reports, and
|
|
replace the default `root` password immediately.
|
|
|
|
Credential provisioning applies only to VMs created by this version. Existing
|
|
VM disks are not modified automatically.
|
|
|
|
Older registries are read and migrated on the next write. A missing/null
|
|
`ports` policy remains `None`/`unmanaged`, not an empty allowlist or an
|
|
implicitly enforced default. Migrate a legacy VM explicitly with `stop`,
|
|
`ports set VM --ports 22,80:HTTP -y` (choose the actual required ports), then
|
|
`start`. Resume requires a complete, safe private disk, a verified kernel and
|
|
VMM, and compatible network settings. Removing the original rootfs template
|
|
does not block start of a retained private disk.
|
|
|
|
Process identity includes PID, Linux process start time, and host boot ID.
|
|
A running legacy PID missing the boot ID is adopted only when its saved start
|
|
time, running binary, exact API-socket command line, and protected paths all
|
|
match. Missing or ambiguous evidence fails closed: UVM will not guess which
|
|
process to signal or release a disk while process exit is unconfirmed.
|
|
|
|
Schema 3 adds disk capacity and durable resize targets. Older registries migrate
|
|
on the next write; a recorded pending resize is completed before a stopped VM
|
|
can boot. A boot failure after a successful resize retains the new allocation
|
|
for a later start.
|
|
|
|
## Configuration
|
|
|
|
Host and API settings can be overridden through environment variables:
|
|
|
|
| Variable | Purpose |
|
|
|---|---|
|
|
| `UVM_BASE` | Data directory; defaults to `/var/lib/uvm` |
|
|
| `UVM_NETWORK` | Guest private IPv4 CIDR; defaults to `10.42.0.0/24` |
|
|
| `UVM_GATEWAY` | Bridge/guest gateway; defaults to `10.42.0.1` |
|
|
| `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` |
|
|
| `UVM_KNOWN_HOSTS_FILES` | Override local SSH cache files cleaned on deletion; colon-separated absolute paths |
|
|
| `UVM_KERNEL_URL` | Override the host-architecture default guest kernel URL |
|
|
| `UVM_ROOTFS_URL` | Optional prebuilt ext4 image URL; unset to build Ubuntu 22.04 locally |
|
|
| `UVM_FIRECRACKER_SHA256` | Trusted Firecracker release archive SHA-256 |
|
|
| `UVM_KERNEL_SHA256` | Trusted guest kernel SHA-256 |
|
|
| `UVM_ROOTFS_SHA256` | Trusted prebuilt rootfs SHA-256; use with `UVM_ROOTFS_URL` |
|
|
| `UVM_FIRECRACKER_BINARY_SHA256` | Optional trusted local Firecracker binary SHA-256 |
|
|
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | Set to `1` only for unverified local development |
|
|
| `UVM_API_TOKEN` | Required for every API server; sent as `X-UVM-Token` |
|
|
| `UVM_API_TLS_CERT` | TLS certificate path required for a non-loopback API bind |
|
|
| `UVM_API_TLS_KEY` | TLS private-key path required for a non-loopback API bind |
|
|
|
|
## Project Layout
|
|
|
|
Selected files:
|
|
|
|
```text
|
|
./
|
|
├── README.md
|
|
├── USAGE.md
|
|
├── DEVELOPER.md
|
|
├── pyproject.toml
|
|
├── uvm.py # Executable compatibility launcher
|
|
├── uvm/ # Private application implementation
|
|
│ ├── cli.py # argparse, terminal output, SSH execution
|
|
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port-policy intent
|
|
│ ├── network.py # IP/MAC/TAP/bridge/NAT management
|
|
│ ├── firewall.py # Owned nftables bridge policies and offline conntrack cleanup
|
|
│ ├── dashboard/ # Local dashboard HTML, CSS, and JavaScript
|
|
│ ├── routers/ # FastAPI routes, including the dashboard document
|
|
│ └── firecracker/ # Firecracker config, API, and process adapters
|
|
└── tests/ # Unit tests and opt-in isolated namespace packet tests
|
|
```
|
|
|
|
See [DEVELOPER.md](DEVELOPER.md) for the complete module map and ownership rules.
|
|
|
|
## Testing
|
|
|
|
Run the unit suite from this project root:
|
|
|
|
```sh
|
|
python3 -m pip install '.[test]'
|
|
python3 -B -m unittest discover -s tests -v
|
|
```
|
|
|
|
The unit tests do not need root, KVM, Firecracker, or host network changes.
|
|
They cover validation, state migration and locking, integrity manifests,
|
|
network/firewall rendering, Firecracker request ordering, process safety,
|
|
retained-disk resume, bulk operations, lifecycle rollback, dashboard delivery,
|
|
and API routes. API route tests run when the FastAPI and `httpx` test
|
|
dependencies are installed.
|
|
|
|
When `nft` and `conntrack` are installed, the suite also checks their command
|
|
parsers using deliberately invalid final commands/options that prevent mutations.
|
|
With e2fsprogs installed, offline disk tests grow and shrink temporary sparse ext4
|
|
images and check that data and credential/key fixtures survive; no VM is booted.
|
|
|
|
Opt-in packet checks run **only in disposable isolated user/network
|
|
namespaces**, which may require privileges and the installed network tools:
|
|
|
|
```sh
|
|
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
|
|
```
|
|
|
|
The packet test was skipped in this workspace because namespace creation was
|
|
denied (`Operation not permitted`). A skip is not packet-path or real-host
|
|
validation. The harness uses veths in place of TAPs and simulated upstream
|
|
networks to check NAT, offline startup, reconnects, and uplink changes alongside
|
|
private port policies. It does not boot a real VM; validate KVM, TAP, SSH, and
|
|
guest connectivity on an isolated Linux host.
|
|
|
|
## Current Boundaries
|
|
|
|
- Fractional CPU requests still round Firecracker vCPUs up and are advisory.
|
|
- No quotas or cgroup CPU, memory, I/O, or process limits are applied.
|
|
- Firecracker is launched directly by this local CLI. Jailer/cgroup hardening
|
|
is not implemented by the current command set.
|
|
- No VM autostart, background reconciliation, guest readiness probing,
|
|
snapshots, DHCP, public-port publishing, or multi-host scheduling.
|
|
- The API uses one shared token, not users, roles, or tenants. UVM is not a
|
|
hardened multi-tenant platform.
|
|
- Run real KVM, TAP, and guest-connectivity checks on an isolated Linux host
|
|
before relying on UVM for workloads.
|