kstyagi/nest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor VM lifecycle management and enhance validation

Browse Source 
- Introduced a new method `installed_kernel` in `ImageStore` to handle kernel verification and checksum validation.
- Updated `Installer` to include additional packages: `nftables` and `conntrack`.
- Refined `LifecycleService` to improve error handling and state management during VM creation, starting, stopping, and deletion.
- Enhanced port management with new methods for setting and syncing ports, including validation for port rules.
- Improved error handling in `StateStore` to raise specific `NotFoundError` and `ConflictError` exceptions.
- Updated network management to ensure proper TAP device handling and firewall integration.
- Enhanced API responses in `vms.py` to include policy status for better client-side handling.
- Added validation for VM names and ports in `validation.py` to enforce stricter rules and prevent conflicts.
This commit is contained in:
kstyagi
2026-09-09 17:27:50 +05:30
parent 4ce14c3378
commit d8a25209b9
31 changed files with 9576 additions and 1331 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+192 -67
View File
@@ -2,7 +2,9 @@
`uvm` is a small, local Firecracker microVM command-line manager for Ubuntu.
It creates one Firecracker process per VM, attaches it to a host TAP device,
and persists enough state to list, stop, and destroy the VM later.
and keeps a private disk and local registry for named VMs. The CLI supports
create, start, stop, restart, delete, inspection, SSH, logs, diagnostics, and
enforced private-IPv4 TCP port allowlists.
This directory is a self-contained project. All UVM code, tests, packaging
metadata, launcher scripts, and documentation live here.
@@ -13,14 +15,17 @@ workflows, see [DEVELOPER.md](DEVELOPER.md).
For end-to-end operator scenarios, CLI examples, and HTTP API calls, see
[USAGE.md](USAGE.md).
For the original firewall proposal and backend evaluation, see
[FIREWALL.md](FIREWALL.md). That proposal is design background, not the current
feature contract; these guides describe the implementation in the workspace.
## Quick Start
From the parent repository directory, run commands from this project root:
Run commands from this project root:
```sh
cd uvm
./uvm.py --help
python3 -m uvm --help
python3 -B -m uvm --help
```
The `pyproject.toml` file also defines an installable `uvm` console command.
@@ -32,12 +37,16 @@ The project is an application, not a reusable Python SDK.
- Python 3.12 or newer
- Root access for UVM commands that read or mutate the protected VM registry
- KVM enabled and accessible at `/dev/kvm`
- Linux 5.3 or newer with native bridge conntrack and safe pidfd support
- An x86_64 host for the default guest assets
- Working host networking, `iproute2`, `iptables`, `e2fsprogs`, `openssl`, and
`openssh-client`
- Working host networking, `iproute2`, `iptables`, `nftables`, `conntrack`,
`e2fsprogs`, `util-linux`, `openssl`, and `openssh-client`
- An STP-disabled software bridge; no offload path that bypasses bridge hooks
`uvm install` installs the required host packages. It validates `/dev/kvm`
before downloading or launching a microVM.
before downloading VM artifacts. Boot operations additionally check that the
native nftables bridge policy is supported; installing packages alone is not
a connectivity or isolation test.
## Trusted Installation
@@ -45,8 +54,6 @@ Installation requires trusted SHA-256 digests by default. Supply digests from
an independently trusted source before running the privileged installer:
```sh
cd uvm
sudo env \
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
@@ -72,59 +79,134 @@ available through an explicit opt-out:
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
```
An unverified install is recorded as such and cannot satisfy a later strict
An unverified manifest is not accepted as publisher trust in a later strict
run. Reinstall with trusted digests to establish a verified local manifest.
To create VMs from an unverified local-development install, retain
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create` commands and on the API
server process.
To boot VMs from an unverified local-development install, retain
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create`, `start`, and `restart`
commands and on the API server process.
## Commands
These are separate operation examples, not a script to run from top to bottom.
```sh
# Install after supplying the trusted checksum variables shown above.
# A bare `install` command fails under the default strict integrity policy.
# Create and boot a VM with the defaults: 1 CPU, 512 MiB RAM, and root/root.
sudo ./uvm.py create
# Create a named VM: 1 CPU, 512 MiB RAM, root/root, and private TCP/22.
# Use a unique non-default password for anything beyond a throwaway VM.
sudo ./uvm.py create --name web
# Request CPU and RAM explicitly. RAM defaults to MiB; B/K/M/G suffixes work.
sudo ./uvm.py create --cpu 1 --ram 1G
# Choose resources, a private guest IP, and additional service ports.
sudo ./uvm.py create --name app --cpu 1 --ram 1G \
--guest-ip 10.42.0.10 --ports 80:HTTP,443:HTTPS
# Set credentials for an account that already exists in the guest image.
sudo ./uvm.py create --username root --password '<guest-password>'
# Read a password from one line of stdin, rather than a --password argument.
# USAGE.md includes a non-echoing Bash prompt example.
sudo ./uvm.py create --name worker --password-stdin
# Select the guest IP. Despite the legacy flag name, this is the guest IP.
sudo ./uvm.py create --host-ip 10.42.0.10
# Explicitly allow no inbound service ports, including SSH.
sudo ./uvm.py create --name closed --no-default-ports
# Show persisted VM state and observed process status.
# Inspect persisted state and observed process/policy status.
sudo ./uvm.py list
sudo ./uvm.py list --status running --json
sudo ./uvm.py list -q
sudo ./uvm.py inspect web --json
sudo ./uvm.py logs web --tail 50 --follow
sudo ./uvm.py doctor --json
# Connect over SSH by VM ID or guest IP.
sudo ./uvm.py ssh vm-<id>
# Resolve VMs by exact ID, name, or guest IP.
sudo ./uvm.py ssh web
sudo ./uvm.py ssh app --port 22 -- uname -a
ssh root@10.42.0.10
# Stop or remove a VM.
sudo ./uvm.py stop vm-<id>
sudo ./uvm.py destroy vm-<id>
# Stop and boot the same private disk, or restart selected VMs.
sudo ./uvm.py stop web --timeout 5
sudo ./uvm.py start web
sudo ./uvm.py restart web app --timeout 10
# Preview bulk operations before confirming them.
sudo ./uvm.py stop --all --dry-run --json
sudo ./uvm.py stop --all -y
sudo ./uvm.py start --all -y
sudo ./uvm.py restart --all -y
sudo ./uvm.py delete --all --dry-run
sudo ./uvm.py delete web -y
# destroy is an alias for delete, with the same flags and destructive effect.
```
`start`, `stop`, `restart`, and `delete` accept one or more VM identifiers or
`--all`, plus `--json`, `--dry-run`, and `--yes`/`-y`. Deletion and mutating
`--all` selections require confirmation; non-interactive callers must use
`--yes`. `--timeout` is available on stop/restart/delete (and `destroy`), not
start. It is a finite non-negative SIGTERM wait in seconds, default `5`, not
a total command deadline or a guest shutdown timeout.
Stop sends SIGTERM and, if needed, SIGKILL to the Firecracker VMM. It is **not
a guest-coordinated shutdown**. Start boots the retained disk without resetting
guest data, SSH host keys, or credentials; it does not restore guest memory.
Delete permanently removes the disk and releases the VM's name and IP after
cleanup. See [USAGE.md](USAGE.md) for the complete flag and recovery reference.
Names are unique local identifiers, not DNS names. `--host-ip` remains an
alias for `--guest-ip`; both select the guest's private IPv4 address. JSON
output omits passwords, with warnings and host-command traces on stderr.
SSH uses `StrictHostKeyChecking=accept-new` and a VM-ID-specific host-key
alias by default. `--insecure-host-key` restores the old no-verification
behavior for a single connection.
alias by default. `--user`, `--key`, and `--port` customize the SSH client;
append a remote command after a literal `--`. `--insecure-host-key` disables
verification for one connection. SSH does not automatically supply the stored
password or open a closed port.
### Private Ports
```sh
sudo ./uvm.py ports list web --json
sudo ./uvm.py ports list --all
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
sudo ./uvm.py ports close web --ports 80
sudo ./uvm.py ports set web --ports 22,443:HTTPS --dry-run --json
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
sudo ./uvm.py ports close web --all -y
sudo ./uvm.py ports set web --none -y
sudo ./uvm.py ports sync web
sudo ./uvm.py ports sync --all --dry-run
sudo ./uvm.py ports sync --all -y
```
Rules use `PORT[:TYPE]`, for example `22`, `80:HTTP`, `443:HTTPS`, or
`8080:TCP`, repeated or comma-separated. HTTP and HTTPS are **TCP labels**,
not proxies or TLS configuration. Create adds `22:TCP` even when other ports
are supplied, unless `--no-default-ports` is set. Later port edits do not
automatically add SSH back. `ports close VM --all` closes ports on that one
VM; only `ports list --all` and `ports sync --all` select all VMs.
Port mutations support `--dry-run` and `--json`; close/set/sync also support
`--yes`/`-y`. Set, close-all, and removal of TCP/22 require confirmation
(an already empty managed policy can be a no-op). Closing a service blocks
subsequent packets on existing service connections in **both directions**, as
well as new connections. This is packet dropping, not a guaranteed TCP RST or
immediate application disconnect.
These rules govern access to the **guest private IP**. They do not start a
guest service or provide public connectivity, host-port mappings, DNS,
reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not
yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required
ARP/ICMP traffic remain allowed; managed TAPs block IPv6.
## Management API
Install the project dependencies before serving the API:
```sh
cd uvm
python3 -m pip install .
```
Launch the FastAPI management server with the requested CLI form:
Launch the FastAPI management server using the environment where the project
dependencies were installed (use the full console-script path if needed):
```sh
cd uvm
export UVM_API_TOKEN='replace-with-a-long-random-secret'
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
uvm --serve --host 127.0.0.1 --port 8000
@@ -144,13 +226,19 @@ The server exposes these routes:
| `GET` | `/health` | Unauthenticated liveness response |
| `POST` | `/install` | Install or refresh host assets; body: `{ "force": false }` |
| `GET` | `/vms` | List VM records and observed status |
| `POST` | `/vms` | Create a VM; body supports `cpu`, `ram`, `guest_ip`, `username`, and `password` |
| `GET` | `/vms/{id-or-ip}` | Retrieve one VM by ID or guest IP |
| `POST` | `/vms/{id}/stop` | Stop a VM while retaining its private disk |
| `DELETE` | `/vms/{id}` | Destroy a VM and release its resources |
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/stop` | Stop a VM while retaining its private disk |
| `DELETE` | `/vms/{identifier}` | Delete a VM and release its resources |
The API intentionally does not proxy interactive SSH. Use the returned guest
IP with `uvm ssh` or a normal SSH client.
API port rules are objects such as `{"port": 80, "protocol": "HTTP"}`;
`no_default_ports` is a boolean, default `false`. Responses include `name`,
`ports`, `policy_status`, and `operation`, but never passwords. See
[USAGE.md](USAGE.md) for request examples and strict schema details.
The endpoint set is unchanged: there are **no start/restart, port-management,
or bulk HTTP endpoints**. SSH, logs, and doctor are also CLI-only. Use the
local CLI for those operations; the API is not a background reconciler.
### API Security
@@ -219,6 +307,24 @@ it, enables IPv4 forwarding, adds an egress masquerade rule, and adds matching
forward rules for return traffic. Guest IPs, gateway, bridge name, and guest
boot netmask derive from the configured CIDR.
The existing `iptables` egress NAT/forwarding behavior is unchanged. Private
port enforcement uses separate per-VM native nftables `bridge` tables, not
DNAT or host listeners. Policy is installed and read back before an owned TAP
is activated and before the guest starts. The software bridge must have STP
disabled; each TAP's bridge-slave `group_fwd_mask` is set to `0x4000` so LLDP
cannot bypass the early identity filter. Native bridge conntrack requires
Linux >= 5.3; no `br_netfilter` sysctl setup is required by this policy layer.
Host-to-VM and VM-to-VM traffic crossing these hooks is filtered. An allowed
port is not a reachability guarantee, and a private IP alone is not protection
against administrator-supplied external routing or NAT. Integration with UFW,
firewalld, Docker, or other external firewall managers remains deferred.
Do not offload traffic around the software bridge hooks.
There is no autostart service or daemon reconciliation. After externally lost
or changed UVM rules, inspect policy status and explicitly run `ports sync`.
Read-only commands do not repair rules; sync does not boot stopped VMs.
## Persistent State
The default data directory is `/var/lib/uvm`:
@@ -232,9 +338,9 @@ The default data directory is `/var/lib/uvm`:
└── state.json # Root-only VM inventory and guest credentials
```
State writes are locked and atomic. Lifecycle operations share an additional
operation lock so concurrent `create`, `stop`, `destroy`, and `install`
commands cannot overwrite each other's state or binaries.
The default registry is `/var/lib/uvm/state.json`, schema version `2`. Writes
are locked and atomic. Lifecycle, port-policy, and installation mutations
share an operation lock; bulk operations use a fixed snapshot of VM IDs.
The registry stores guest usernames and passwords in plaintext, so `state.json`
is mode `0600`. Treat it as a secret, do not include it in bug reports, and
@@ -243,14 +349,28 @@ replace the default `root` password immediately.
Credential provisioning applies only to VMs created by this version. Existing
VM disks are not modified automatically.
Older registries are read and migrated on the next write. A missing/null
`ports` policy remains `None`/`unmanaged`, not an empty allowlist or an
implicitly enforced default. Migrate a legacy VM explicitly with `stop`,
`ports set VM --ports 22,80:HTTP -y` (choose the actual required ports), then
`start`. Resume requires a complete, safe private disk, a verified kernel and
VMM, and compatible network settings. Removing the original rootfs template
does not block start of a retained private disk.
Process identity includes PID, Linux process start time, and host boot ID.
A running legacy PID missing the boot ID is adopted only when its saved start
time, running binary, exact API-socket command line, and protected paths all
match. Missing or ambiguous evidence fails closed: UVM will not guess which
process to signal or release a disk while process exit is unconfirmed.
## Configuration
All optional configuration is supplied through environment variables:
Host and API settings can be overridden through environment variables:
| Variable | Purpose |
|---|---|
| `UVM_BASE` | Data directory; defaults to `/var/lib/uvm` |
| `UVM_NETWORK` | Guest IPv4 CIDR; defaults to `10.42.0.0/24` |
| `UVM_NETWORK` | Guest private IPv4 CIDR; defaults to `10.42.0.0/24` |
| `UVM_GATEWAY` | Bridge/guest gateway; defaults to `10.42.0.1` |
| `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` |
| `UVM_KERNEL_URL` | Guest kernel download URL |
@@ -267,52 +387,57 @@ All optional configuration is supplied through environment variables:
## Project Layout
```text
uvm/
├── README.md
./
├── README.md, USAGE.md, DEVELOPER.md
├── pyproject.toml
├── uvm.py # Executable compatibility launcher
├── uvm/ # Private application implementation
│ ├── cli.py # argparse, terminal output, SSH execution
│ ├── app.py # Application composition root
│ ├── api_models.py # FastAPI request and response models
│ ├── config.py # Settings and filesystem layout
│ ├── domain.py # Typed VM and state models
│ ├── errors.py # Expected application failures
│ ├── integrity.py # Checksums and manifest provenance
│ ├── install.py # Host/artifact installation
│ ├── images.py # Guest templates and per-VM disks
│ ├── lifecycle.py # Create/list/stop/destroy orchestration
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port-policy intent
│ ├── network.py # IP/MAC/TAP/bridge/NAT management
│ ├── routers/ # Health, installation, and VM HTTP routers
│ ├── server.py # FastAPI application factory and Uvicorn launcher
│ ├── state.py # Locked atomic JSON persistence
│ ├── system.py # Host command and KVM adapters
│ ├── firewall.py # Owned nftables bridge policies and offline conntrack cleanup
│ └── firecracker/ # Firecracker config, API, and process adapters
└── tests/ # Unit tests with mocked host boundaries
└── tests/ # Unit tests and opt-in isolated namespace packet tests
```
See [DEVELOPER.md](DEVELOPER.md) for the complete module map and ownership rules.
## Testing
From the parent repository directory, run the unit suite from this project root:
Run the unit suite from this project root:
```sh
cd uvm
python3 -m pip install '.[test]'
PYTHONDONTWRITEBYTECODE=1 python3 -B -m unittest discover -s tests -v
python3 -B -m unittest discover -s tests -v
```
The unit tests do not need root, KVM, Firecracker, or host network changes.
They cover validation, state migration and locking, integrity manifests,
network command rendering, Firecracker request ordering, process safety, and
lifecycle rollback. API route tests run automatically when the FastAPI
and `httpx` test dependencies are installed.
network/firewall rendering, Firecracker request ordering, process safety,
retained-disk resume, bulk operations, and lifecycle rollback. API route tests
run when the FastAPI and `httpx` test dependencies are installed.
Opt-in packet checks run **only in disposable isolated user/network
namespaces**, which may require privileges and the installed network tools:
```sh
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
```
The packet test was skipped in this workspace because namespace creation was
denied (`Operation not permitted`). A skip is not packet-path or real-host
validation. The harness uses veths in place of TAPs and does not boot a real VM;
validate KVM, TAP, NAT, SSH, and guest connectivity on an isolated Linux host.
## Current Boundaries
- Fractional CPU requests still round Firecracker vCPUs up and are advisory.
- `stop` retains the private VM disk; the current command surface does not yet
include a separate `start` command.
- No quotas or cgroup CPU, memory, I/O, or process limits are applied.
- Firecracker is launched directly by this local CLI. Jailer/cgroup hardening
is not implemented by the current command set.
- No VM autostart, background reconciliation, guest readiness probing,
snapshots, DHCP, public-port publishing, or multi-host scheduling.
- The API uses one shared token, not users, roles, or tenants. UVM is not a
hardened multi-tenant platform.
- Run real KVM, TAP, and guest-connectivity checks on an isolated Linux host
before relying on UVM for workloads.