# UVM Usage Guide

This guide explains how to install, operate, and troubleshoot UVM as an
operator. It covers the command-line tool, the optional FastAPI server, common
scenarios, and expected limitations.

For the project architecture and contributor guidance, see
[DEVELOPER.md](DEVELOPER.md). For a concise overview, see [README.md](README.md).

## 1. What UVM Does

UVM creates small Linux microVMs through Firecracker and KVM on one Linux host.
Each VM receives:

- A unique VM ID.
- A private writable root disk copied from a shared template.
- A private IP address and MAC address.
- A TAP interface connected to a host bridge.
- A dedicated Firecracker process, API socket, log, and config file.

By default, guest networking uses:

```text
Guest network: 10.42.0.0/24
Gateway:       10.42.0.1
Bridge:        uvm0
Guest pool:    10.42.0.2 through 10.42.0.254
```

The default locations on the host are under `/var/lib/uvm`.

## 2. Before You Start

### Host Requirements

You need:

- Linux, preferably Ubuntu or a compatible distribution.
- Python 3.12 or newer.
- KVM enabled and readable/writable at `/dev/kvm`.
- Root access for installation and VM lifecycle operations.
- An x86_64 host for the default guest kernel/rootfs downloads.
- Internet access if downloading Firecracker and guest assets.

Check virtualization before installing:

```sh
ls -l /dev/kvm
lscpu | grep -i virtualization
```

### Source Checkout Setup

From the parent directory of this project:

```sh
cd uvm
./uvm.py --help
python3 -m uvm --help
```

To install the optional HTTP API dependencies and the console command:

```sh
cd uvm
python3 -m pip install .
uvm --help
```

The rest of this guide uses `./uvm.py` so it works from a source checkout. If
you installed the package, replace `./uvm.py` with `uvm`.

### Privilege Rules

| Operation | Root required? |
|---|---|
| `install` | Yes |
| `create` | Yes |
| `stop` | Yes |
| `destroy` | Yes |
| `list` | Yes with the default root-owned registry |
| `ssh` | Yes when resolving a VM through the registry |
| API server that manages VMs | Yes |

The registry contains plaintext guest credentials and is mode `0600`. Direct
SSH to a known guest IP does not require root.

## 3. Command Overview

```sh
./uvm.py --help

./uvm.py install [--force]
./uvm.py create [--cpu CPU] [--ram RAM] [--host-ip GUEST_IP] \
  [--username USER] [--password PASSWORD]
sudo ./uvm.py list
sudo ./uvm.py ssh VM_OR_IP [--user USER] [--key PATH] [--insecure-host-key]
./uvm.py stop VM_ID
./uvm.py destroy VM_ID

./uvm.py --serve [--host HOST] [--port PORT]
```

`--host-ip` is a legacy CLI name. It means the **guest IP address**, not the
host's public or uplink address.

## 4. Install UVM Assets

UVM installs host packages, Firecracker, Jailer, a guest kernel, and a guest
rootfs template. It requires SHA-256 checksums by default because it downloads
artifacts that will be used by a privileged process.

### Scenario: Strict First Installation

Obtain trusted checksums independently, then run:

```sh
cd uvm

sudo env \
  UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
  UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
  UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
  ./uvm.py install
```

On success, UVM prints paths similar to:

```text
Firecracker: /var/lib/uvm/bin/firecracker
Kernel:      /var/lib/uvm/images/vmlinux
Rootfs:      /var/lib/uvm/images/ubuntu.ext4
```

The checksums and whether the installation was verified are recorded in:

```text
/var/lib/uvm/integrity.json
```

### Scenario: Refresh Firecracker or Guest Assets

Use `--force` after changing a URL, replacing an image, or rotating checksums:

```sh
sudo env \
  UVM_FIRECRACKER_SHA256='<new-firecracker-archive-sha256>' \
  UVM_KERNEL_SHA256='<new-kernel-sha256>' \
  UVM_ROOTFS_SHA256='<new-rootfs-sha256>' \
  ./uvm.py install --force
```

`--force` downloads to temporary files and only replaces the existing artifact
after checksum verification succeeds.

### Scenario: Local Experimentation Without Checksums

This mode is unsafe for production. It is useful only when you deliberately
accept the source artifacts without checksum verification:

```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
```

An unverified install is marked as unverified. It cannot later become a strict
trusted installation merely by setting checksum variables. Reinstall with the
strict command above when you are ready to trust the host assets.

For later local-development VM creation, keep the opt-out on the command too:

```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py create
```

The same rule applies to an API server that creates VMs from an unverified
installation: start that server with `UVM_ALLOW_UNVERIFIED_DOWNLOADS=1`, or
perform a strict reinstall first.

### Installation Failures

| Message | Meaning and action |
|---|---|
| `/dev/kvm does not exist` | Enable hardware/nested virtualization first. |
| `checksum is required` | Provide the requested `UVM_*_SHA256` variable. |
| `SHA-256 mismatch` | Stop and obtain the correct digest or artifact URL. |
| `unsupported host architecture` | Use x86_64 defaults or provide compatible assets. |
| `default guest assets support x86_64 only` | Configure both custom kernel and rootfs URLs for ARM. |

## 5. Create VMs

### Scenario: Create a Default VM

The default is one CPU, 512 MiB RAM, username `root`, and password `root`:

```sh
sudo ./uvm.py create
```

Typical output includes the VM ID, guest IP, TAP name, and SSH target:

```text
VM created: vm-...
  IP:       10.42.0.2
  RAM:      512 MiB
  CPU:      1.0
  TAP:      uvm-...
  Username: root
  SSH:      ssh root@10.42.0.2

WARNING: the guest is using the default password 'root'. Change it promptly.
```

Set a different password during creation:

```sh
sudo ./uvm.py create --username root --password '<guest-password>'
```

`--username` must name an account already present in the guest image. The
password is visible in the process arguments and may be recorded in shell
history, so avoid reusing a sensitive host or service password.

### Scenario: Choose CPU and RAM

```sh
sudo ./uvm.py create --cpu 2 --ram 2G
sudo ./uvm.py create --cpu 1 --ram 1024M
sudo ./uvm.py create --cpu 1 --ram 768
```

RAM is in MiB by default. Supported suffixes include `B`, `K`, `M`, `G`,
`KiB`, `MiB`, and `GiB`.

UVM enforces a minimum of 128 MiB. Firecracker uses whole vCPUs. A fractional
request such as `--cpu 0.5` is accepted but rounds the VM up to one Firecracker
vCPU; it does not currently apply a host CPU quota.

```sh
sudo ./uvm.py create --cpu 0.5 --ram 512
```

### Scenario: Request a Specific Guest IP

```sh
sudo ./uvm.py create --host-ip 10.42.0.10
```

The address must be inside the configured guest network, must not be the
network address, broadcast address, or gateway, and must not already belong to
another persisted VM.

### What Happens During Create

UVM performs these operations in order:

1. Validates root access, KVM, assets, and integrity state.
2. Reserves a VM ID, IP, MAC, and state record.
3. Creates `/var/lib/uvm/vms/<vm-id>/`.
4. Copies the rootfs template to a private `rootfs.ext4` disk.
5. Sets the requested account password and enables SSH password login in the copy.
6. Ensures the bridge/NAT/forwarding rules exist.
7. Creates a TAP device.
8. Starts and configures Firecracker.
9. Marks the VM as `running`.

If a step fails, UVM attempts to terminate the new VMM, remove the TAP, remove
the private runtime directory, and release the reservation.

## 6. Inspect VMs

```sh
sudo ./uvm.py list
```

Example:

```text
ID                                    IP               USER             CPU     RAM      STATUS     PID
vm-0123456789abcdef...                10.42.0.2       root             1.0     512      running    12345
```

Status meanings:

| Status | Meaning |
|---|---|
| `starting` | VM creation is in progress. |
| `running` | Firecracker started successfully. |
| `stopping` | Stop is in progress. |
| `stopped` | VMM and TAP are gone; disk remains. |
| `terminating` | Destruction is in progress. |
| `failed` | A lifecycle operation or cleanup failed. |
| `dead` | Stored state claims the VM is active but its VMM process is gone. |

If a VM is `failed` or `dead`, inspect its log before deciding whether to stop
or destroy it:

```sh
sudo ls -la /var/lib/uvm/vms/<vm-id>
sudo cat /var/lib/uvm/vms/<vm-id>/firecracker.log
```

## 7. Connect with SSH

UVM provisions a password for an existing account. The default credentials are
`root` / `root`. Your guest image must include:

- An SSH server.
- The requested login user plus standard passwd and shadow files.
- Unique SSH host keys generated at first boot.

This is applied only while creating a new VM. Existing VM disks and legacy
registry entries are not changed retroactively.

### Scenario: SSH as Root

```sh
sudo ./uvm.py ssh vm-<id>
ssh root@10.42.0.2
```

### Scenario: Use a Different User and Private Key

```sh
sudo ./uvm.py ssh vm-<id> --user ubuntu --key ~/.ssh/id_ed25519
```

By default, UVM uses SSH `StrictHostKeyChecking=accept-new` and a stable
host-key alias derived from the VM ID. This lets a guest IP be reused later
without confusing it with the prior VM's known-hosts entry.

### Scenario: Temporarily Bypass Host-Key Verification

Use only for troubleshooting a guest you trust:

```sh
sudo ./uvm.py ssh vm-<id> --insecure-host-key
```

This disables normal host-key checking for that connection.

## 8. Stop and Destroy VMs

### Scenario: Stop a VM and Keep Its Disk

```sh
sudo ./uvm.py stop vm-<id>
```

Stopping terminates Firecracker and deletes the TAP device. The private root
disk, config, and log remain under the VM runtime directory.

UVM does not currently offer a `start` command, so a stopped disk cannot be
rebooted through the public CLI yet. Keep it only if you need to inspect it or
expect a future start feature.

### Scenario: Permanently Remove a VM

```sh
sudo ./uvm.py destroy vm-<id>
```

Destroying a VM:

1. Terminates the VMM if it is alive.
2. Deletes the TAP device.
3. Deletes the VM runtime directory and private root disk.
4. Removes the VM record from `state.json`.
5. Releases its IP address for later allocation.

This is destructive. Back up guest data before running it.

## 9. Use the FastAPI Server

The API is an alternative management surface for the same local lifecycle
services. It does not make UVM multi-host or safe to expose directly to the
public internet.

### Scenario: Start a Local API Server

Install dependencies first:

```sh
cd uvm
python3 -m pip install .
```

Then start the server as root so lifecycle endpoints can manipulate KVM,
networking, and host files:

```sh
export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  ./uvm.py --serve --host 127.0.0.1 --port 8000
```

Open these local URLs after it starts:

```text
Health:  http://127.0.0.1:8000/health
OpenAPI: http://127.0.0.1:8000/docs
```

The API token is required even for a loopback server. All VM and installation
routes require it. FastAPI's informational `/docs`, `/redoc`, and
`/openapi.json` endpoints are also available without a token; they do not
perform host mutations.

### Scenario: Start a Remote TLS API Server

Non-loopback binds require a valid TLS certificate and key. The files must be
readable by the server process:

```sh
export UVM_API_TOKEN='replace-with-a-long-random-visible-ascii-token'
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'

sudo env \
  UVM_API_TOKEN="$UVM_API_TOKEN" \
  UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
  UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
  ./uvm.py --serve --host 0.0.0.0 --port 8443
```

For real remote use, use a certificate trusted by clients and restrict network
access with a firewall or reverse proxy. Do not use a self-signed development
certificate for an untrusted network.

### API Authentication

Set one reusable shell variable for API examples:

```sh
export UVM_API_TOKEN='replace-with-the-server-token'
export UVM_API_URL='http://127.0.0.1:8000'
```

Every management request needs:

```http
X-UVM-Token: <token>
```

For a TLS server, set `UVM_API_URL` to an `https://` URL and use your CA:

```sh
export UVM_API_URL='https://host.example:8443'
curl --cacert /path/to/ca.pem \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms"
```

## 10. API Endpoint Examples

### Health Check

No token is required:

```sh
curl "$UVM_API_URL/health"
```

Response:

```json
{
  "status": "ok"
}
```

### List VMs

```sh
curl \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms"
```

Example response:

```json
[
  {
    "id": "vm-0123456789abcdef",
    "cpu": 1.0,
    "ram_mib": 512,
    "guest_ip": "10.42.0.2",
    "gateway": "10.42.0.1",
    "mac": "02:fc:00:00:00:01",
    "username": "root",
    "status": "running",
    "observed_status": "running",
    "pid": 12345,
    "created_at": 1700000000,
    "updated_at": 1700000000,
    "last_error": null
  }
]
```

### Create a VM

Create a default-sized VM:

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"cpu": 1, "ram": "512", "username": "root", "password": "root"}' \
  "$UVM_API_URL/vms"
```

Create a VM with a chosen guest address:

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"cpu": 2, "ram": "2G", "guest_ip": "10.42.0.10", "username": "root", "password": "replace-me"}' \
  "$UVM_API_URL/vms"
```

The response is the VM record with its username, but never its password.
Omitting credentials uses `root` / `root`. Request fields are strict:
misspelled or extra fields return HTTP `422` instead of being silently ignored.

### Look Up One VM

By ID:

```sh
curl \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/vm-0123456789abcdef"
```

By guest IP:

```sh
curl \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/10.42.0.2"
```

### Stop a VM

```sh
curl -X POST \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/vm-0123456789abcdef/stop"
```

The response is the VM record with `status: "stopped"`.

### Destroy a VM

```sh
curl -X DELETE \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  "$UVM_API_URL/vms/vm-0123456789abcdef"
```

Response:

```json
{
  "id": "vm-0123456789abcdef",
  "status": "terminated"
}
```

### Install or Refresh Through the API

The server process must itself have checksum variables configured for strict
installation. Start the server with those environment variables before using
this endpoint.

For a deliberately unverified local-development server, the server process
must instead include `UVM_ALLOW_UNVERIFIED_DOWNLOADS=1`. Do not use that mode
on a host that runs untrusted workloads.

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"force": false}' \
  "$UVM_API_URL/install"
```

Refresh all downloaded artifacts:

```sh
curl -X POST \
  -H "Content-Type: application/json" \
  -H "X-UVM-Token: $UVM_API_TOKEN" \
  -d '{"force": true}' \
  "$UVM_API_URL/install"
```

## 11. API Errors

| Status | Typical meaning |
|---|---|
| `401` | Missing or incorrect `X-UVM-Token`. |
| `403` | UVM process lacks root privileges for a host mutation. |
| `404` | Requested VM was not found. |
| `409` | A lifecycle operation is already in progress. |
| `422` | Invalid JSON body, extra request fields, or invalid UVM input. |
| `500` | Host command, Firecracker, state, or unexpected operational failure. |

UVM operation errors use this envelope:

```json
{
  "error": {
    "message": "human-readable explanation"
  }
}
```

FastAPI request-schema errors use FastAPI's normal `detail` response format.

## 12. Configuration Scenarios

### Scenario: Use a Different Storage Directory

```sh
sudo env \
  UVM_BASE='/srv/uvm' \
  UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 \
  ./uvm.py install
```

All UVM binaries, images, VM runtime directories, locks, and state use the
configured base path.

### Scenario: Use a Different Guest Network

```sh
sudo env \
  UVM_NETWORK='10.50.0.0/24' \
  UVM_GATEWAY='10.50.0.1' \
  UVM_BRIDGE='uvm50' \
  UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 \
  ./uvm.py install
```

Use the same environment values for later `create`, `list`, `stop`, `destroy`,
and server commands. Changing the network variables after VMs exist can make
their persisted state inconsistent with host networking.

### Environment Variables

| Variable | Use |
|---|---|
| `UVM_BASE` | Base data directory; default `/var/lib/uvm` |
| `UVM_NETWORK` | Guest subnet; default `10.42.0.0/24` |
| `UVM_GATEWAY` | Guest gateway / bridge IP; default `10.42.0.1` |
| `UVM_BRIDGE` | Linux bridge name; default `uvm0` |
| `UVM_KERNEL_URL` | Custom kernel download URL |
| `UVM_ROOTFS_URL` | Custom rootfs download URL |
| `UVM_FIRECRACKER_SHA256` | Strict Firecracker archive checksum |
| `UVM_KERNEL_SHA256` | Strict kernel checksum |
| `UVM_ROOTFS_SHA256` | Strict rootfs checksum |
| `UVM_FIRECRACKER_BINARY_SHA256` | Optional local binary checksum |
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | Set to `1` only for local development |
| `UVM_API_TOKEN` | Required API token |
| `UVM_API_TLS_CERT` | TLS certificate for remote server binds |
| `UVM_API_TLS_KEY` | TLS key for remote server binds |

## 13. Files, Logs, and State

The default host layout is:

```text
/var/lib/uvm/
├── bin/                 # Firecracker and Jailer binaries
├── images/              # Shared kernel and rootfs template
├── vms/<vm-id>/         # Private rootfs, config, socket, and log
├── integrity.json       # Artifact checksums and verification status
└── state.json           # Mode-0600 VM inventory and plaintext guest credentials
```

Useful inspection commands:

```sh
sudo cat /var/lib/uvm/state.json
sudo ls -la /var/lib/uvm/vms
sudo cat /var/lib/uvm/vms/<vm-id>/firecracker.log
sudo cat /var/lib/uvm/vms/<vm-id>/config.json
```

Avoid editing `state.json`, VM config files, runtime directories, TAPs, or
iptables rules by hand while UVM is running. Use UVM commands or the API so
locks and cleanup rules remain correct.

`state.json` contains plaintext guest passwords. Keep it root-only, never copy
it into logs or bug reports, and use unique non-default passwords outside local
throwaway environments.

## 14. Networking Troubleshooting

Check the bridge and TAP state:

```sh
ip link show uvm0
ip addr show dev uvm0
ip route show default
```

Check NAT and forwarding rules:

```sh
sudo iptables -t nat -S POSTROUTING
sudo iptables -S FORWARD
```

Common issues:

| Symptom | First checks |
|---|---|
| Guest cannot reach the internet | Bridge is up, TAP is up, host has default route, NAT and FORWARD rules exist. |
| Requested guest IP is rejected | Confirm it is inside the configured subnet and not used, gateway, network, or broadcast. |
| Create says TAP already exists | Inspect the interface; do not delete it unless you know it is stale UVM state. |
| Existing bridge is rejected | `UVM_BRIDGE` points to an interface that is not a Linux bridge. |
| SSH cannot connect | Confirm guest booted, the requested user exists, the image runs OpenSSH, and the configured password is being used. |

## 15. Important Limitations

Keep these constraints in mind when deciding whether UVM fits a scenario:

- Firecracker is launched directly; the downloaded Jailer is not yet used.
- No cgroup CPU, memory, I/O, or process limits are applied.
- Fractional CPU requests are advisory only.
- There is no public `start` or `restart` command after a VM is stopped.
- There is no DHCP, inbound port mapping, load balancing, multi-network
  isolation, or multi-host scheduling.
- The API uses one shared bearer token, not individual users, roles, or tenants.
- There is no guest readiness probe beyond Firecracker process state.
- UVM is not a hardened multi-tenant platform.

## 16. Safe Operating Checklist

Before relying on a VM for useful work:

1. Use strict checksums for installed artifacts.
2. Confirm `/dev/kvm` and nested virtualization are available.
3. Use a guest image with unique first-boot SSH host keys and set a non-default password.
4. Test guest egress and SSH on the actual host network.
5. Back up guest data before destroy operations.
6. Keep the API loopback-only unless you have a strong remote-access need.
7. For remote API access, use a long random token, valid TLS, and firewall
   restrictions.
8. Review `/var/lib/uvm/vms/<vm-id>/firecracker.log` after failed starts.

## 17. Get Help

```sh
./uvm.py --help
./uvm.py install --help
./uvm.py --serve --help
```

When reporting a problem, collect:

- The exact command or HTTP request.
- The full error message.
- Output from `sudo ./uvm.py list`.
- The VM Firecracker log, if a VM was created.
- The relevant `UVM_*` variables, with secrets such as `UVM_API_TOKEN` removed.

Never include `/var/lib/uvm/state.json` because it contains guest passwords.