kstyagi/nest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor VM lifecycle management and enhance validation

Browse Source 
- Introduced a new method `installed_kernel` in `ImageStore` to handle kernel verification and checksum validation.
- Updated `Installer` to include additional packages: `nftables` and `conntrack`.
- Refined `LifecycleService` to improve error handling and state management during VM creation, starting, stopping, and deletion.
- Enhanced port management with new methods for setting and syncing ports, including validation for port rules.
- Improved error handling in `StateStore` to raise specific `NotFoundError` and `ConflictError` exceptions.
- Updated network management to ensure proper TAP device handling and firewall integration.
- Enhanced API responses in `vms.py` to include policy status for better client-side handling.
- Added validation for VM names and ports in `validation.py` to enforce stricter rules and prevent conflicts.
This commit is contained in:
kstyagi
2026-09-09 17:27:50 +05:30
parent 4ce14c3378
commit d8a25209b9
31 changed files with 9576 additions and 1331 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DEVELOPER.md
+389 -291
View File
@@ -10,60 +10,53 @@ general-purpose Python library or a multi-host control plane.
## 1. Goals and Boundaries
UVM currently provides a small host-local workflow:
UVM installs host/guest assets, provisions private disks, and manages named
VMs and private IPv4 TCP allowlists on one host. The CLI offers retained-disk
start/stop/restart, deletion, fixed-target bulk operations, inspection, SSH,
logs, doctor, and explicit port edits/sync. The optional API exposes only the
existing health/install/create/list/inspect/stop/delete operations.
1. Install Firecracker, a guest kernel, and a rootfs template.
2. Allocate an ID, private IPv4 address, MAC address, TAP device, and VM
runtime directory.
3. Copy the rootfs template to a private writable VM disk and provision the
requested existing guest account password.
4. Configure and launch one Firecracker process.
5. List, stop, SSH to, and destroy the VM.
6. Optionally expose those operations through an authenticated FastAPI API.
The design deliberately does not provide these capabilities yet:
- Multi-host scheduling or remote state coordination.
- User tenancy, quotas, or a public-facing control plane.
- A `start` or `restart` command for stopped VMs.
- Guest snapshots, extra disks, port forwarding, or DHCP.
- Cgroup CPU enforcement for fractional CPU requests.
- Jailer-based Firecracker execution, namespaces, or seccomp hardening.
- Production-grade image building or guest SSH-key injection.
There is no autostart or background reconciler, multi-host control plane,
tenancy/quotas, or UVM-provided Jailer/cgroup/namespace hardening. Private port
filtering is not public-port publishing or a hardened multi-tenant boundary;
see Current Production Gaps for the remaining scope limits.
Treat UVM as a privileged local systems tool. It manipulates `/dev/kvm`, TAP
interfaces, bridge addresses, iptables rules, host filesystems, and processes.
interfaces, bridge addresses, nftables/conntrack and existing iptables egress
rules, host filesystems, and processes. For the complete operator command and
flag reference, see [USAGE.md](USAGE.md).
## 2. First Run
All UVM-specific files live in this directory. Start here:
```sh
cd uvm
python3 -m pip install -e '.[test]'
PYTHONDONTWRITEBYTECODE=1 python3 -B -m unittest discover -s tests -v
python3 -B -m unittest discover -s tests -v
./uvm.py --help
```
The `uvm` console script is registered by `pyproject.toml`. During source
development, `./uvm.py` and `python3 -m uvm` are equivalent entry points.
development, `./uvm.py` and `python3 -B -m uvm` are equivalent entry points.
Requirements for real host operations:
- Python 3.12+
- Linux with KVM enabled and `/dev/kvm` accessible
- Linux >= 5.3 with native bridge conntrack and safe pidfd support
- Root privileges for installation and lifecycle mutations
- `iproute2`, `iptables`, `e2fsprogs`, `util-linux`, `openssl`, and OpenSSH
client tools
- `iproute2`, `iptables`, `nftables`, `conntrack`, `e2fsprogs`, `util-linux`,
`openssl`, and OpenSSH client tools
- An x86_64 host for the default kernel/rootfs URLs
Do not run real integration operations on a developer workstation unless the
bridge, subnet, iptables changes, and guest workload are acceptable there.
bridge, subnet, firewall changes, and guest workload are acceptable there.
Packet integration tests must remain inside disposable isolated namespaces.
## 3. Project Layout
```text
uvm/
./
├── DEVELOPER.md # This document
├── README.md # Operator-facing usage guide
├── pyproject.toml # Package metadata and dependencies
@@ -73,16 +66,17 @@ uvm/
│ ├── cli.py # argparse, CLI rendering, serve dispatch
│ ├── app.py # Composition root
│ ├── config.py # Settings and environment loading
│ ├── domain.py # VmSpec, VmRecord, State
│ ├── domain.py # PortRule, VmSpec, VmRecord, State
│ ├── errors.py # Expected error hierarchy
│ ├── validation.py # CPU and RAM parsing
│ ├── validation.py # Resources, names, credentials, ports, timeouts
│ ├── system.py # CommandRunner, root/KVM/directory checks
│ ├── state.py # JSON state, atomic writes, file locks
│ ├── integrity.py # SHA-256 and integrity manifest logic
│ ├── install.py # Apt, Firecracker, kernel, rootfs installation
│ ├── images.py # Template lookup, private disks, and guest credentials
│ ├── network.py # IP/MAC/TAP/bridge/NAT/forward rules
│ ├── lifecycle.py # Create/list/stop/destroy orchestration
│ ├── firewall.py # Owned nft bridge tables and offline conntrack cleanup
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port intent
│ ├── api_models.py # Pydantic HTTP request/response models
│ ├── server.py # FastAPI factory and Uvicorn launcher
│ ├── routers/
@@ -94,7 +88,7 @@ uvm/
│ ├── config.py # Firecracker JSON and boot arguments
│ ├── api.py # Unix-socket HTTP client
│ └── process.py # Process launch, liveness, termination
└── tests/ # Unit tests and conditional API tests
└── tests/ # Unit/API tests; integration/ has opt-in namespace packets
```
Keep feature code in the most specific existing module. Do not add a new
@@ -112,9 +106,11 @@ flowchart LR
APP --> STATE[StateStore]
LIFE --> IMAGES[ImageStore]
LIFE --> NET[NetworkManager]
NET --> FW[FirewallManager]
LIFE --> FC[Firecracker adapters]
INSTALL --> RUNNER[CommandRunner]
NET --> RUNNER
FW --> RUNNER
FC --> HOST[Linux host primitives]
RUNNER --> HOST
STATE --> DISK[(state.json)]
@@ -126,29 +122,6 @@ The public entry points are thin adapters. The lifecycle layer is the only
place that coordinates multiple host resources for a VM. Lower-level modules
must not import `cli.py`, `server.py`, or `lifecycle.py`.
### Dependency Direction
```text
cli.py / server.py / routers/
|
v
app.py
|
+-------+--------+
| |
install.py lifecycle.py
|
+------------+------------+
| | |
state.py network.py images.py
|
firecracker/*
| | |
+------ system.py --------+
|
Linux host
```
Leaf modules are `config.py`, `domain.py`, `errors.py`, and `validation.py`.
They must remain free of subprocess, network, CLI, and FastAPI side effects.
@@ -156,45 +129,37 @@ They must remain free of subprocess, network, CLI, and FastAPI side effects.
### CLI Path
```mermaid
sequenceDiagram
participant U as User
participant C as cli.py
participant A as app.py
participant L as LifecycleService
participant H as Linux host
`cli.py` builds the application, validates arguments, and calls lifecycle
services. It owns confirmation, terminal/JSON output, exit handling, SSH
`execvp`, local log streaming from a validated path, and `--serve` dispatch.
It must not coordinate state files, interfaces, or Firecracker sockets itself.
U->>C: uvm create --cpu 1 --ram 512
C->>A: build_application()
A->>L: composed service graph
C->>L: create(VmSpec)
L->>H: allocate files, TAP, process, Firecracker API
L-->>C: VmRecord
C-->>U: formatted VM details
```
The local surface is install/create/list/inspect/start/stop/restart/delete
(`destroy` alias)/ssh/logs/doctor and ports list/open/close/set/sync. Identifiers
resolve by exact ID, name, or guest IP, never an ID prefix. Generated-ID-shaped
inputs resolve only by ID; names cannot impersonate generated IDs. Ambiguous
non-ID aliases raise `ConflictError`.
`cli.py` owns parsing, terminal output, exit handling, SSH `execvp`, and the
`--serve` switch. It must not manipulate state files, interfaces, or
Firecracker sockets directly.
Create alone adds `22:TCP` unless `--no-default-ports` is supplied. Port syntax
is `PORT[:TCP|HTTP|HTTPS]`, comma-separated or repeated via `--ports`; all labels
use TCP. `--password-stdin` reads one line instead of using a password argument.
`_vm_output()` is an explicit output whitelist, not `VmRecord.to_dict()`.
JSON stays on stdout, while warnings, prompts, and host traces use stderr.
Bulk lifecycle actions and ports sync resolve/deduplicate IDs before prompting.
Delete and mutating `--all` selections require a TTY confirmation or `--yes`/`-y`.
Port set/close-all and removal of TCP/22 also prompt, except empty managed
no-ops. `--dry-run` previews without lifecycle/policy mutations or confirmation;
it is not a full preflight. Stop/restart/delete accept `--timeout`; start does
not. Logs follow does not hold the operation lock. Doctor runs readiness
checks, not per-VM repair or guest application probes.
### HTTP Path
```mermaid
sequenceDiagram
participant Client
participant Uvicorn
participant API as server.py / routers
participant L as LifecycleService
participant H as Linux host
Client->>Uvicorn: HTTP(S) request + X-UVM-Token
Uvicorn->>API: route dispatch
API->>API: token dependency and Pydantic validation
API->>L: lifecycle operation
L->>H: host mutation
L-->>API: VmRecord or UvmError
API-->>Client: JSON response or JSON error
```
Routers validate the token and Pydantic request, invoke application services,
and render whitelisted response models. The CLI expansion does not imply HTTP
endpoint parity: start/restart, port edits/sync, bulk operations, logs, doctor,
and interactive SSH have no HTTP endpoints.
Use only `uvm --serve` or `./uvm.py --serve` to launch the API. The
`create_api()` factory requires an explicit host specifically to prevent it
@@ -212,13 +177,15 @@ process:
| StateStore | `StateStore` | Inventory, locks, JSON persistence |
| ImageStore | `ImageStore` | Template verification, VM disks, and offline credentials |
| NetworkManager | `NetworkManager` | Guest allocation and Linux networking |
| FirewallManager | Constructed by `NetworkManager` | Owned native bridge policy and scoped offline conntrack cleanup |
| FirecrackerProcessManager | `FirecrackerProcessManager` | VMM process control |
| LifecycleService | `LifecycleService` | Ordered VM resource changes and rollback |
| Installer | `Installer` | Host packages and artifacts |
The FastAPI factory calls `build_application(emit=None)` so API-driven host
commands do not print shell-style command traces to the server output. The CLI
uses the default emitter and prints commands as `+ ...` for transparency.
Composition can disable command tracing with `emit=None`; `create_api()` does
so when it constructs its own application. The CLI supplies an emitter that
prints `+ ...` traces to stderr, including when passing its application to
the server launcher.
## 7. Persistent Data and Models
@@ -253,42 +220,38 @@ state file.
### State Schema
[`domain.py`](uvm/domain.py) is the schema authority. The current state
version is `1`.
[`domain.py`](uvm/domain.py) is the schema authority. The default registry is
`/var/lib/uvm/state.json`; its current schema version is **2**. An empty registry:
```json
{
"schema_version": 1,
"next_mac_index": 2,
"vms": {
"vm-<uuid>": {
"id": "vm-<uuid>",
"cpu": 1.0,
"ram_mib": 512,
"guest_ip": "10.42.0.2",
"gateway": "10.42.0.1",
"tap": "uvm-...",
"mac": "02:fc:00:00:00:01",
"socket": "/var/lib/uvm/vms/.../firecracker.sock",
"config": "/var/lib/uvm/vms/.../config.json",
"log": "/var/lib/uvm/vms/.../firecracker.log",
"disk": "/var/lib/uvm/vms/.../rootfs.ext4",
"username": "root",
"password": "replace-me",
"status": "running",
"pid": 1234,
"process_start_time": "...",
"created_at": 0,
"updated_at": 0,
"last_error": null
}
}
"schema_version": 2,
"next_mac_index": 1,
"vms": {}
}
```
`VmSpec` is the validated create request. `VmRecord` is the persisted local
record. Do not make API models or CLI namespaces the persistence format.
| Record Fields | Purpose |
|---|---|
| `id`, `name`, `cpu`, `ram_mib` | Stable identity, optional unique name, resource request |
| `guest_ip`, `gateway`, `network`, `bridge`, `tap`, `mac` | Retained network identity/configuration |
| `socket`, `config`, `log`, `disk` | Canonical per-VM runtime paths |
| `username`, `password` | Guest credentials; password stays out of public output |
| `ports` | Null for legacy/unmanaged, or a list of `{ "port": 80, "protocol": "HTTP" }` objects |
| `disk_ready`, `runtime_owned`, `tap_owned` | Completed provisioning and resource ownership evidence |
| `status`, `operation`, `spawn_pending`, `last_error` | Durable state, pending intent, ambiguous-spawn guard, failure detail |
| `pid`, `process_start_time`, `boot_id` | VMM identity, including host reboot detection |
| `created_at`, `updated_at` | Registry timestamps |
`PortRule` accepts TCP/HTTP/HTTPS labels and integer ports 1-65535. A port
number is unique regardless of label. `VmSpec.ports` defaults to TCP/22,
whereas `VmRecord.ports=None` deliberately preserves the legacy distinction.
An explicit empty tuple/list is a managed empty TCP allowlist, not unmanaged
mode. There is no persisted boolean asserting that a policy is enforced.
### State Guarantees
- `state.json` writes use a sibling temporary file, `fsync`, atomic replace,
@@ -296,11 +259,18 @@ record. Do not make API models or CLI namespaces the persistence format.
- `state.json` is mode `0600`; it must be treated as a credential registry and
excluded from logs and bug reports.
- `state.lock` serializes short JSON read-modify-write transactions.
- `operations.lock` serializes long lifecycle and installation operations.
- `operations.lock` serializes long lifecycle, bulk, policy, and installation operations.
- State reads are lock-free because atomic replacement gives readers either a
complete old file or a complete new file.
- Legacy state without a schema version is accepted as version `0`; missing
`disk` and `process_start_time` fields have safe defaults.
- Legacy versions 0 (including unversioned state) and 1 are loaded into schema
2 in memory and written as schema 2 on the next transaction. Newer schemas
are rejected. Reads do not reconcile VMs or adopt process identities.
- Missing `name`, `ports`, and `boot_id` remain null. No policy or default SSH
rule is silently imposed on a legacy VM. Use stop, explicit `ports set`,
then start; first policy adoption requires a stopped, proven-gone VM.
- Missing `disk` and process start time remain absent. Legacy `disk_ready` is
inferred only for running/stopped records with a recorded private disk;
start still validates the path, file, and provisioning state.
- Records created before credential provisioning default to username `root`
and password `null`; UVM does not apply a password retroactively.
- `next_mac_index` is reconstructed from older locally administered MACs when
@@ -324,62 +294,108 @@ The persisted status is a string, not an enum, to keep legacy JSON readable.
| `stopped` | Process/TAP removed; private disk remains |
| `terminating` | Destruction is in progress |
| `failed` | A lifecycle step or cleanup failed; `last_error` explains why |
| `dead` | Observed-only list status when a supposedly active process is gone |
| `dead` | Observed-only status when a stored running process is proven gone |
| `unknown` | Observed process identity cannot be established safely |
`dead` is not persisted. `destroy` removes the record rather than retaining a tombstone.
Transition statuses can survive interruption. `dead`/observed `unknown` do
not rewrite inventory. Delete removes the record rather than keeping a
tombstone. Pending deletion remains authoritative even if a subsequent stop
finishes runtime cleanup; start and policy edits must not resurrect it.
`ListedVm` separates the record, observed process status, and observed policy
status. Policy can be `enforced`, `missing`, `drift`, `quarantined`, `unknown`,
legacy `unmanaged`, or `inactive` for a cleanly stopped managed VM. Readback is
a ruleset/attachment snapshot, not application health or a reachability test.
### Create Order
[`LifecycleService.create`](uvm/lifecycle.py) follows this sequence:
```mermaid
flowchart TD
A[Validate root, KVM, binary, integrity] --> B[Acquire operations.lock]
B --> C[Reserve ID, IP, MAC, and starting state]
C --> D[Create 0700 runtime directory]
D --> E[Copy template to private rootfs.ext4]
E --> F[Provision guest password and SSH policy]
F --> G[Ensure bridge, NAT, and forwarding rules]
G --> H[Create TAP]
H --> I[Write Firecracker config atomically]
I --> J[Spawn Firecracker and wait for API socket]
J --> K[Persist PID and process start token]
K --> L[Configure Firecracker through Unix HTTP]
L --> M[Persist running state]
```
1. Validate the request/root access and acquire `operations.lock`.
2. Check KVM, verified Firecracker, firewall capability, and installed templates.
3. Reserve ID/name/IP/MAC/ports with `status=starting`, `operation=preparing`.
4. Create the owned mode-0700 runtime directory, copy a mode-0600 private disk,
provision credentials/host keys, and persist `disk_ready=True`.
5. Ensure bridge/egress networking and prepare an owned, attached TAP **down**.
6. Apply/read back the nftables bridge policy, then activate the TAP.
7. Write Firecracker config; persist `operation=spawning`, `spawn_pending=True`,
and the host boot ID before `Popen`.
8. Publish PID/start time/boot ID through `on_spawn` before socket readiness
or API configuration; clear the pending-spawn guard.
9. Configure Firecracker and `InstanceStart`, verify policy enforcement again,
then save `running` with no pending operation.
The record is persisted as `starting` before host side effects. The PID and
Linux process-start token are persisted before Firecracker API configuration.
This makes cleanup and later diagnosis possible if API configuration fails.
This records both intent and acquired ownership across crash windows. Never
make a managed TAP usable before enforcement or mark a partial disk ready.
### Rollback Rules
Any `Exception`, `KeyboardInterrupt`, or `SystemExit` during create triggers
compensation:
1. Terminate a started Firecracker process when its identity is known.
2. Remove the TAP when it was created.
3. Remove the VM runtime directory and private disk.
4. Remove the reserved state record if cleanup succeeds.
5. Preserve a `failed` record with `last_error` if cleanup itself fails.
1. Attempt quarantine/TAP deactivation and confirm safe process exit.
2. Remove the owned TAP, clean up scoped conntrack entries, and remove its policy.
3. Remove the owned runtime directory/private disk only after confirmed exit.
4. Remove the reservation only if cleanup succeeds.
5. Otherwise retain a `failed` record with identity/intent and `last_error`;
do not release its disk or IP for reuse.
Do not reorder these operations without considering leaked TAPs, orphaned VMM
processes, and allocation reuse.
### Stop and Destroy
### Start and Restart
Start requires managed ports, a proven-gone process (or an already identified
running VM), a complete safe private disk, and compatible retained gateway,
network, and bridge settings. It verifies the VMM and installed **kernel**;
the original rootfs template is not needed. It never recopies the rootfs,
reprovisions credentials, or regenerates guest SSH host keys. Name/IP/MAC,
ports, and guest disk changes survive stop/start; guest RAM does not.
Start on a healthy running VM synchronizes policy without spawning another
VMM. An ambiguous process or interrupted live boot/stop is not adopted as a
successful start. Failed resume retains the private disk for retry. Restart
performs stop and start under one operation lock, not two independent public
calls with a race between them.
### Stop and Delete
```text
stop:
running/failed/stopped -> stopping -> terminate process -> delete TAP -> stopped
private disk and config remain
persist stop intent -> quarantine/deactivate TAP -> confirm VMM exit
-> delete owned TAP -> scoped conntrack cleanup -> remove owned nft table
-> stopped (retain disk, credentials, name/IP and desired ports)
destroy:
any non-transition state -> terminating -> terminate process -> delete TAP
-> remove VM runtime directory -> remove state record
delete (destroy alias):
persist delete intent -> same runtime cleanup -> remove owned runtime directory
-> remove state record -> release name/IP
```
`stop` is intentionally disk-preserving. There is no public `start` command
yet, so do not add start behavior by modifying stop semantics.
Stop signals the **VMM**, not a guest shutdown agent or ACPI-coordinated
shutdown. SIGTERM is followed by SIGKILL when needed. `--timeout` is the
per-VM TERM wait, default 5 seconds, finite and non-negative; it is not a total
deadline. Even zero still requires confirmed kernel exit before cleanup.
Application consistency requires separately quiescing the guest.
Runtime cleanup retains reservations when process exit, TAP ownership, or
connection cleanup is uncertain. A pending spawn cannot be declared gone
merely because its PID was never persisted or its socket has not appeared.
### Policy and Bulk Operations
`set_ports()` persists the new allowlist with `operation=policy` before
applying it. Live updates apply atomically and retain conntrack; cleanly
stopped edits are metadata-only. On apply/readback failure, preserve the new
intent, attempt link-down/quarantine, and leave a failed pending record. Never
restore an older, more permissive allowlist after a failed close. An explicit
`ports sync` retries the desired policy and can reactivate a contained live VM.
`run_many()` holds one outer operation lock while processing the fixed ID
snapshot produced before CLI confirmation. It reloads by **exact ID**, not a
reusable name/IP. Results are `succeeded`, `failed`, `skipped` (already absent),
or `interrupted`; ordinary failures do not undo other successes or stop later
targets. Interruptions skip the remaining targets. Read-only views and API
startup never reconcile; there is no autostart or background repair loop.
## 9. Firecracker Integration
@@ -406,14 +422,30 @@ comes from `Settings.network`, not a hard-coded `/24` value.
### Process Safety
Firecracker starts detached in a new session, with stdout/stderr redirected to
the per-VM log. The manager records `/proc/<pid>/stat` start time and only
signals a process when its persisted token still matches. This prevents a stale
state record from killing an unrelated process after PID reuse.
Firecracker starts detached in a new session, appending stdout/stderr to the
per-VM log. Identity consists of PID, `/proc/<pid>/stat` start time, and the
host boot ID. `observe()` distinguishes `alive`, `gone`, and `unknown`;
unreadable evidence is not proof of exit. A changed boot ID or start token can
prove the old process gone without signaling a reused PID.
Legacy records with a PID but no start token are only cleaned up automatically
when that PID is already gone. If it remains alive, UVM fails closed and asks
the operator to inspect it.
Start probes safe pidfd support before spawning. Termination opens a pidfd,
rechecks identity, signals through that handle (never a bare PID), and confirms
whole-process exit before unlinking the socket. Python bindings or libc
bindings may supply pidfd support; there is no unsafe PID-signal fallback.
A zombie leader alone does not prove its worker threads have exited.
Legacy adoption is deliberately narrow. During a locked mutation,
`adopt_legacy_identity()` may fill a missing boot ID only when all evidence
matches: a saved numeric start token, a live pidfd-pinned process, the configured
Firecracker executable path **and inode**, the exact `--api-sock` command line,
and root-owned/protected binary, socket, and ancestor directories. It checks
that the process has not exited before returning, and the lifecycle layer
persists the adopted boot ID. Listing/inspection never adopts it.
A live legacy PID without a saved start token, or with mismatched/inaccessible
binary/socket evidence, remains unknown and fails closed. UVM does not guess
an identity from the current occupant of a PID or delete an in-use disk to
make recovery convenient. Proven-gone legacy processes can be cleaned up.
## 10. Guest Images and Integrity
@@ -451,6 +483,10 @@ writes a manifest marked `verified: false`; a later strict run will not treat
that manifest as publisher trust. Use `install --force` with trusted digests to
refresh an existing artifact without manually deleting managed files.
Retain the explicit opt-out on create/start/restart (or the API server) when
using an unverified development installation. It is not appropriate for
untrusted workloads. A bare strict install requires the trusted digests.
The optional `UVM_FIRECRACKER_BINARY_SHA256` validates a local Firecracker
binary when no verified manifest is available.
@@ -474,7 +510,8 @@ host default uplink
internet
```
[`network.py`](uvm/network.py) owns all networking details:
[`network.py`](uvm/network.py) owns address allocation and host attachments;
[`firewall.py`](uvm/firewall.py) owns native bridge filtering:
- IP allocation from `Settings.network`, excluding network, broadcast, and
gateway addresses.
@@ -484,16 +521,89 @@ internet
Linux bridge.
- Gateway address setup and IPv4 forwarding.
- One egress NAT rule and forward/return rules for the default uplink.
- TAP creation, cleanup, and error reporting.
- TAP ownership aliases (`uvm:<id>`), preparation while down, activation,
deactivation, cleanup, and error reporting.
Network actions are idempotent where practical. Do not adopt an existing TAP:
it may belong to another VM. A name collision is an error.
it may belong to another VM. A create-time name collision is an error. Cleanup
checks the TAP kind and ownership alias, with a narrow unaliased-TAP allowance
for legacy records; it does not treat an arbitrary interface name as ownership.
### Native Bridge Policy
The supported path is a **software Linux bridge** with STP disabled. Each
managed TAP's bridge-slave `group_fwd_mask` must include `0x4000` (the network
layer sets 16384). This sends LLDP through the early filter instead of a
pre-hook delivery/learning path. Firewall apply/status checks the bridge and
port settings when the TAP exists. Do not use hardware/flow offloads or other
paths that bypass these hooks. This is not an offload-isolation guarantee.
Native bridge conntrack requires Linux >= 5.3. Capability checks run
`conntrack --version`, inspect the kernel release, and ask `nft --json --check`
to validate a full policy without committing it. The nft CLI must support
the rendered JSON. This layer needs no `br_netfilter` sysctl changes.
Each VM owns one `bridge` table named `uvm_fw_<digest>`, derived from the
resolved `UVM_BASE` and exact VM ID, with a full ownership comment. Apply uses
an atomic nft JSON transaction, delete-by-handle for the old owned table, and
exclusive create for its replacement. Foreign ownership is an error, not an
adoption opportunity. Readback normalizes presentation differences but compares
all effective objects, ordered rules, hooks, identity, and the full port set;
rule counts or an owner comment alone are not proof of enforcement.
The policy has prerouting/input/forward/output hooks. Early prerouting identity
checks run at priority -310, before conntrack and bridge MAC learning; input
paths recheck identity as well. Managed traffic is bound to the recorded MAC
and private IPv4 address. Spoofed identity, IPv6, VLAN encapsulation, and other
unsupported EtherTypes are dropped. Host-to-guest and peer-to-guest paths are
covered, not just routed IP forwarding.
The current TCP allowlist is checked against conntrack's **original service
destination address/port in both directions**, including established service
flows. There is no blanket established/related accept before it. Consequently
close blocks subsequent packets on existing service connections in both
directions after commit, without promising a TCP RST or immediate application
disconnect. Live close does not flush conntrack, which would risk
reclassifying midstream service traffic as guest egress.
Guest-initiated IPv4 TCP/UDP and replies remain allowed, including outbound DNS.
ARP, ICMP echo, and necessary related ICMP errors are supported. Other inbound
UDP services are not; HTTP/HTTPS are only TCP labels. Empty managed ports
means no inbound TCP service allowance, not quarantine or loss of all egress.
Quarantine drops every scoped TAP path, including ARP.
Offline cleanup requires the TAP to be absent. It deletes and verifies absence
of conntrack entries scoped to the guest IP in all four original/reply source
and destination fields, including NAT reply tuples. Only then may teardown
release the address. Rule removal also requires an absent TAP and verified
table ownership; it never flushes global firewall or conntrack state.
### Coexistence and Recovery
Existing `iptables` egress MASQUERADE/forwarding remains unchanged. The new
policy adds no DNAT, public listener, or host-port mapping and does not replace
the host firewall manager. Public connectivity, DNS, guest reverse proxies/TLS,
and integration with UFW/firewalld/Docker/libvirt remain out of scope. An
administrator's external routing/NAT can still affect reachability; do not
infer source-CIDR restrictions or complete public-exposure protection from the
use of private addresses. Operators must choose a non-conflicting topology.
If an external actor loses or changes UVM rules, recovery is explicit
`ports sync`, not a promised daemon repair. An `enforced` snapshot cannot
prevent later root-level changes or prove that other rules allow traffic.
[FIREWALL.md](FIREWALL.md) retains the original proposal/backend evaluation;
do not import its unimplemented topology checks into current documentation.
## 12. Installer and Host Commands
`Installer` is the only owner of apt installation, Firecracker artifact
handling, downloads, and integrity-manifest publication.
The host package list includes `nftables` and `conntrack` as well as the
existing iproute2/iptables, ext4, OpenSSL, OpenSSH, download, and util-linux
tools. UVM adds no reconciliation daemon or tested external firewall-manager
integration profile.
`system.py` centralizes host command execution:
- `CommandRunner.run()` converts subprocess failures into `CommandError`.
@@ -513,27 +623,12 @@ Python's safe tar extraction filter.
### Server Startup
```sh
cd uvm
python3 -m pip install .
export UVM_API_TOKEN='long-random-visible-ascii-token'
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
./uvm.py --serve --host 127.0.0.1 --port 8000
```
For non-loopback binds, both TLS variables are required and loaded into an
`ssl.SSLContext` before Uvicorn starts:
```sh
export UVM_API_TLS_CERT='/etc/uvm/api-cert.pem'
export UVM_API_TLS_KEY='/etc/uvm/api-key.pem'
sudo env \
UVM_API_TOKEN="$UVM_API_TOKEN" \
UVM_API_TLS_CERT="$UVM_API_TLS_CERT" \
UVM_API_TLS_KEY="$UVM_API_TLS_KEY" \
./uvm.py --serve --host 0.0.0.0 --port 8443
```
See [USAGE.md](USAGE.md) for dependency installation, loopback deployment,
TLS deployment, and authenticated curl examples. Launch through `uvm --serve`
using the Python environment with the installed API dependencies. Every server
requires `UVM_API_TOKEN`, even at `127.0.0.1`. Non-loopback binds additionally
require `UVM_API_TLS_CERT` and `UVM_API_TLS_KEY`; paired certificate/key files
are loaded into an `ssl.SSLContext` before Uvicorn starts.
The token must contain visible ASCII characters without whitespace. It is
never accepted as a UVM command-line argument, so it does not appear in UVM
@@ -548,10 +643,10 @@ or secret manager when that matters.
| `GET` | `/health` | `health.py` | Unauthenticated liveness response |
| `POST` | `/install` | `installation.py` | `{ "force": false }` body |
| `GET` | `/vms` | `vms.py` | List persisted VM records and observed status |
| `POST` | `/vms` | `vms.py` | `cpu`, `ram`, optional `guest_ip`, `username`, and `password` |
| `GET` | `/vms/{id-or-ip}` | `vms.py` | Lookup by ID or guest IP |
| `POST` | `/vms/{id}/stop` | `vms.py` | Stop while retaining private disk |
| `DELETE` | `/vms/{id}` | `vms.py` | Remove runtime files and state |
| `POST` | `/vms` | `vms.py` | `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `GET` | `/vms/{identifier}` | `vms.py` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/stop` | `vms.py` | Stop while retaining private disk |
| `DELETE` | `/vms/{identifier}` | `vms.py` | Remove runtime files and state |
All management routes use `get_authorized_application()` and require:
@@ -561,29 +656,31 @@ X-UVM-Token: <UVM_API_TOKEN>
Request schemas reject unexpected fields. `UvmError` instances are rendered as
JSON errors by `server.py`; Pydantic request validation uses FastAPI's normal
422 response format. VM responses include the username but never the password.
422 response format. Create accepts `name: str | null`, `ports: list | null`,
and strict boolean `no_default_ports` (default false). Port objects contain a
strict integer `port` in 1-65535 and uppercase `protocol` TCP/HTTP/HTTPS (TCP
default). The router normalizes/deduplicates ports and adds TCP/22 unless
default ports are disabled, even for null/empty input. It never creates an
unmanaged VM from an omitted policy. Names share CLI validation.
The API intentionally does not expose interactive SSH, raw Firecracker calls,
arbitrary command execution, or host path access.
VM responses include `name`, `ports`, `policy_status`, and `operation` alongside
resource/status fields and username. The schema excludes password and
disk/socket/config/log path fields; error messages may still contain paths.
`ports: null` in a legacy response still means unmanaged. The existing
stop/delete routes use service defaults; no timeout/dry-run/bulk HTTP schemas
were added. Typed not-found/conflict errors map to 404/409.
There are **no new start/restart, ports, or bulk HTTP endpoints**. The API also
does not expose logs, doctor, interactive SSH, raw Firecracker calls, arbitrary
command execution, or host path access. Public `/docs`, `/redoc`, and
`/openapi.json` are informational, not privileged management routes.
## 14. Configuration Reference
| Variable | Default | Used by |
|---|---|---|
| `UVM_BASE` | `/var/lib/uvm` | Storage layout |
| `UVM_NETWORK` | `10.42.0.0/24` | IP allocation and bridge subnet |
| `UVM_GATEWAY` | `10.42.0.1` | Bridge and guest boot args |
| `UVM_BRIDGE` | `uvm0` | Linux bridge name |
| `UVM_KERNEL_URL` | Firecracker quickstart URL | Guest kernel download |
| `UVM_ROOTFS_URL` | Firecracker quickstart URL | Guest rootfs download |
| `UVM_FIRECRACKER_SHA256` | unset | Trusted release archive checksum |
| `UVM_KERNEL_SHA256` | unset | Trusted kernel checksum |
| `UVM_ROOTFS_SHA256` | unset | Trusted rootfs checksum |
| `UVM_FIRECRACKER_BINARY_SHA256` | unset | Optional direct binary checksum |
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | `false` | Development-only integrity bypass |
| `UVM_API_TOKEN` | unset | Required API bearer token |
| `UVM_API_TLS_CERT` | unset | TLS certificate for remote API bind |
| `UVM_API_TLS_KEY` | unset | TLS key for remote API bind |
The environment-variable table is in [README.md](README.md#configuration);
[`config.py`](uvm/config.py) is the implementation authority. Keep the same
base/network/gateway/bridge settings across CLI and server operations. The
network identity is retained per VM and checked before resume.
`Settings` validates the gateway, bridge-name length, API token format, and
environment boolean values at process startup. Add configuration fields there,
@@ -598,11 +695,17 @@ Expected failures inherit from `UvmError`:
| `ConfigurationError` | Invalid local settings or environment |
| `ValidationError` | Invalid CLI/API resource input |
| `StateError` | Unsafe or malformed persisted state |
| `NotFoundError` | Missing registry entry |
| `ConflictError` | Ambiguous identifier, allocation conflict, or unsafe/pending operation |
| `CommandError` | Required host command failure |
| `FirecrackerError` | VMM process or Unix API failure |
| `TapCreationError` | TAP setup failed after interface creation |
CLI code catches `UvmError` and prints `uvm: error: ...` without a traceback.
Invalid CLI input returns 2, ordinary operational failure 1, and interruption
130. Batch JSON retains per-target outcomes rather than pretending a partial
success was atomic.
The FastAPI server maps the same class to an error envelope. New expected
operational failures should use this hierarchy rather than leaking raw
`OSError`, `CalledProcessError`, or implementation-specific exceptions.
@@ -611,22 +714,26 @@ operational failures should use this hierarchy rather than leaking raw
These invariants are critical:
1. Only `LifecycleService` may coordinate VM state, disks, TAPs, and
Firecracker as one operation.
1. Only `LifecycleService` may coordinate VM state, disks, TAPs, firewall,
and Firecracker as one operation.
2. Only `StateStore` may read/write `state.json`.
3. All long host mutations acquire `operations.lock`.
4. All short inventory mutations acquire `state.lock`.
5. The installer shares `operations.lock` with lifecycle actions, preventing a
Firecracker binary replacement between verification and launch.
6. A VM rootfs is private before it becomes writable.
7. A PID is not signaled unless its persisted start token still matches
`/proc/<pid>/stat`.
8. A new VM never adopts a pre-existing TAP.
7. Process signaling requires a safe pidfd and matching persisted start/boot
identity; unknown is not gone. Legacy adoption must prove binary/socket identity.
8. A new VM never adopts a pre-existing TAP or activates it before verified policy.
9. A server must have an API token; a remote server must also have usable TLS.
10. Routers call application services; they do not run shell commands or touch
state files directly.
11. Plaintext guest passwords remain in the mode-`0600` registry and never
enter Firecracker config, helper command arguments/traces, or API responses.
enter Firecracker config, helper command arguments/traces, or CLI/API VM output.
12. Retained-disk start never overwrites guest data, credentials, or SSH host keys.
13. No disk/IP release before confirmed exit and scoped network cleanup; no
old-policy rollback after a failed close, and no resurrection of pending deletion.
14. Policy state is observed from the actual owned rules, not a cached applied flag.
When a change would violate one of these rules, redesign the change rather
than adding a special-case bypass.
@@ -636,80 +743,75 @@ than adding a special-case bypass.
### Standard Unit Suite
```sh
cd uvm
python3 -m pip install -e '.[test]'
PYTHONDONTWRITEBYTECODE=1 python3 -B -m unittest discover -s tests -v
python3 -B -m unittest discover -s tests -v
```
The suite covers:
- CPU/RAM parsing and validation.
- State serialization, legacy migration, locking, and permissions.
- Checksum verification and integrity-manifest provenance.
- Private rootfs behavior.
- Guest credential persistence, validation, and offline rootfs provisioning.
- IP, MAC, bridge, TAP, NAT, and forwarding command construction.
- Firecracker configuration and API request ordering.
- PID safety and startup interrupt cleanup.
- Lifecycle success, rollback, stop, and destroy behavior.
- CLI SSH safety and `--serve` parsing.
- Server token/TLS validation and conditional API route tests.
Ordinary tests require no root, KVM, Firecracker, or host network mutations.
They cover validation, schema migration/locking/permissions, artifact trust,
private disks/credentials, network/firewall rendering and readback, conntrack
semantics, pidfd/legacy identity, lifecycle recovery/resume/bulk targeting,
CLI parsing/output, and server security. Mock/fixture policy checks do not
prove the kernel's actual packet behavior.
FastAPI route tests are conditionally enabled when both `fastapi` and `httpx`
are installed. They should be enabled in CI using the `test` extra.
### Real Integration Tests
### Isolated Packet Tests
No current test automatically boots a real microVM. Before a release, run an
isolated host test that covers:
The opt-in packet harness must run **only in disposable isolated user/network
namespaces**. It verifies namespace separation before creating links or
changing rules, and never falls back to the parent host namespace. It may
require privileges (or enabled unprivileged user namespaces) plus `unshare`,
`ip`, `bridge`, `nft`, `conntrack`, and `sysctl`:
```sh
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
```
The harness uses bridge-connected veths in place of guest TAPs and routed
egress without NAT. It exercises host/peer service traffic, both-direction
close of existing streams while preserving conntrack/egress, IPv6/spoof
blocking, quarantine, and offline cleanup. It does not boot a microVM or
validate real TAP/NAT deployment.
The packet test skipped in this workspace because namespace creation was
denied (`Operation not permitted`). Document that permission limitation, not
an assertion that real packets or a real host were validated. Unit tests and
nft JSON parser checks are not substitutes for kernel packet-path execution.
### Isolated Host Validation
No current test automatically boots a real microVM. Before a release, validate
on an isolated Linux host:
1. Strict checked installation.
2. `create` and successful guest boot.
3. Guest IP reachability and egress.
4. SSH with unique guest host keys.
5. Stop and retained disk inspection.
6. Destroy and IP/TAP release.
7. API launch with token and TLS on a non-loopback test address.
8. Firecracker crash and interrupted create recovery.
5. Stop/start/restart with retained guest data, SSH keys, and credentials.
6. Live private-port close in both directions, empty policy, and explicit sync
after deliberately lost rules inside the isolated setup.
7. Delete and IP/TAP/policy/conntrack cleanup.
8. API launch with token and TLS on a non-loopback test address.
9. Firecracker crash, interrupted create/start/stop/delete, and legacy recovery.
## 18. Common Development Tasks
### Add a CLI Command
1. Put business logic in an existing service or a cohesive new internal
module, not `cli.py`.
2. Add parser arguments in `build_parser()`.
3. Add display/dispatch behavior in `_run_command()`.
4. Use `UvmError` subclasses for expected failures.
5. Add unit tests for parsing and service behavior.
6. Update `README.md` and this document if the architecture changes.
### Add an HTTP Endpoint
1. Add or extend a Pydantic request/response model in `api_models.py`.
2. Put the endpoint in the appropriate `routers/` module.
3. Use `get_authorized_application()` for every privileged endpoint.
4. Keep `/health` as the only intentionally unauthenticated management route.
FastAPI's default documentation endpoints are informational and public.
5. Add the router to `create_api()`.
6. Add `httpx`/FastAPI tests and update the API table in both docs.
### Add a Host Primitive
1. Put subprocess construction in `system.py`, `network.py`, `install.py`, or
a specific Firecracker adapter.
2. Use `CommandRunner` when invoking external commands.
3. Make repeated application safe where possible.
4. Define compensation behavior before calling it from lifecycle code.
5. Add fake-runner tests; do not require root in ordinary unit tests.
### Change State
1. Increment `STATE_SCHEMA_VERSION` only when the serialized interpretation
changes incompatibly.
2. Keep older state readable or provide an explicit migration path.
3. Update `VmRecord`, `State`, state tests, and runtime cleanup behavior.
4. Consider lock ownership and crash/retry behavior before changing fields.
- CLI: keep business logic in services; wire arguments in `build_parser()` and
dispatch/output in `_run_command()`. Test validation, prompts, dry runs,
JSON, and service behavior; use `UvmError` for expected failures.
- HTTP: define strict models in `api_models.py`, use the appropriate router and
`get_authorized_application()`, and register new routers in `create_api()`.
Add API tests and update all route tables. New CLI commands do not implicitly
authorize new HTTP endpoints; `/health` remains the only unauthenticated
operational route.
- Host primitives: use the specific network/firewall/install/process module
and `CommandRunner` for external commands. Define ownership, idempotence,
and compensation before invoking a primitive from lifecycle code. Use fake
runners rather than requiring privileges in unit tests.
- State: update `VmRecord`/`State`, schema tests, cleanup, and documentation.
Version incompatible interpretations and preserve older data or provide an
explicit migration. Consider lock order and crash/retry behavior first.
## 19. Debugging Guide
@@ -718,27 +820,19 @@ isolated host test that covers:
| `guest assets missing` | `/var/lib/uvm/images`, integrity variables, `integrity.json` |
| Firecracker exits early | VM `firecracker.log`, VM config, `/dev/kvm` access |
| VM is `dead` in `list` | PID/start token in state, `/proc/<pid>`, VM log |
| Cannot stop legacy VM | Missing process start token; inspect PID manually before cleanup |
| Cannot stop legacy VM | Missing start/boot identity or failed protected binary/socket match; never guess a PID |
| Guest has no egress | `ip route show default`, bridge address, TAP state, iptables rules |
| Policy missing/drift/unknown | Actual nft JSON and ownership, STP/group mask, permissions; explicit sync after fixing cause |
| Retained-disk start fails | `disk_ready`, safe private path, kernel/VMM integrity, process identity, retained network, pending operation |
| API server refuses to start | `UVM_API_TOKEN`, requested host, TLS cert/key files |
| API returns 401 | `X-UVM-Token` header mismatch |
| API returns 422 | Request schema or UVM input validation failure |
Useful host commands during isolated debugging:
```sh
ip link show uvm0
ip addr show dev uvm0
ip route show default
iptables -t nat -S POSTROUTING
iptables -S FORWARD
ls -la /var/lib/uvm/vms
sudo cat /var/lib/uvm/state.json # Contains plaintext guest passwords; do not share.
```
Do not use these commands to mutate UVM resources manually while a lifecycle
operation is active. Prefer fixing the service behavior and retrying through
the CLI or API.
Use `inspect VM --json`, `logs VM`, `ports list --all`, and `doctor` before
examining host state. [USAGE.md](USAGE.md) lists read-only bridge/nft/iptables
debugging commands. Never share the credential registry or unredacted guest
logs. Do not manually mutate UVM resources during a lifecycle operation;
fix the service/host failure and retry through supported commands.
## 20. Current Production Gaps
@@ -749,9 +843,13 @@ The code is intentionally honest about current limits:
- No cgroup CPU, memory, I/O, or process limits are applied.
- Fractional CPU is represented by rounding vCPUs up and printing an advisory.
- Guest health is inferred from VMM process state, not application readiness.
- There is no DHCP, port mapping, ingress firewall policy, or network tenancy.
- `iptables` rules are incrementally managed rather than rendered from a full
declarative firewall policy.
- No quotas, DHCP, host-port mapping, UDP/IPv6 services, or network tenancy.
- No guest snapshots, extra disks, production image builder, or SSH-key injection.
- Per-VM private TCP ingress policy exists, but public connectivity, DNS,
guest reverse proxy/TLS, and external firewall integration remain out of scope.
- Existing egress `iptables` rules are incremental and unchanged; only the
private bridge policy is atomically rendered per VM.
- No autostart, background reconciliation, or guest-coordinated shutdown.
- The JSON inventory is suitable for one host, not distributed coordination.
- Guest passwords are stored in plaintext in the root-only local registry.
- API authentication is a single bearer token, not a user/tenant model.
FIREWALL.md
+735
View File
@@ -0,0 +1,735 @@
# UVM Local Firewall Design
> Status: proposed design. This document describes a host-local firewall layer
> to be implemented after the VM lifecycle and registry work. It does not
> describe functionality currently available in `uvm`.
## 1. Decision
UVM should use **native nftables**, managed locally through an internal Python
adapter that invokes the `nft` command through the existing `CommandRunner`.
The initial firewall layer will enforce per-VM **private IPv4 TCP allowlists**
at the UVM bridge/TAP boundary. It will not require a cloud firewall, remote
agent, external controller, or long-running UVM firewall daemon.
This choice fits UVM because nftables provides:
- Native Linux bridge filtering, which sees VM-to-VM traffic on `uvm0`.
- Stateful connection tracking in the bridge family on supported kernels.
- Atomic rule transactions through `nft --file -`.
- Named sets and a dedicated table, allowing UVM to update only resources it
owns.
- A local CLI and JSON inspection interface that can be used safely through
UVM's existing subprocess boundary.
The recommended implementation is a focused adapter, not a general firewall
framework or a public Python SDK.
## 2. Scope
### In scope
- Enforce a per-VM TCP allowlist for traffic to the VM's private IPv4 address.
- Cover host-to-VM and VM-to-VM traffic on the existing UVM bridge.
- Preserve ordinary guest-initiated egress and replies to that egress.
- Make `close` block subsequent packets on existing service connections as
well as new connections.
- Install the policy before a VM TAP becomes usable.
- Let explicit lifecycle or policy-sync operations reconcile UVM-owned
firewall state from the local registry after interruption or firewall-rule
loss.
- Work entirely on the Linux host that runs UVM.
### Explicitly out of scope
- Public IP allocation, port forwarding, DNAT, reverse proxies, DNS, TLS
termination, load balancing, or cloud security groups.
- A remote firewall API, external controller, or multi-host coordination.
- Automatic integration with Docker, UFW, firewalld, Kubernetes, libvirt, or
a site-managed firewall. Those need separately tested integration profiles.
- IPv6 service policies, UDP service policies, source-CIDR ACLs, rate limits,
or egress restrictions in the first version.
- A claim that the current Firecracker launch model is a hardened,
multi-tenant isolation boundary.
Opening TCP/80 in this design means that TCP/80 is reachable at the guest's
private UVM address by permitted local paths. It does not create a public
listener, start a guest service, configure TLS, or bypass a guest firewall.
### Supported topology profile
The first version supports a UVM-dedicated private bridge only. Preflight must
verify all of these conditions before activating a managed TAP:
- `UVM_NETWORK` is wholly contained in one RFC1918 range and does not overlap
another local address or non-default route, apart from the expected UVM
bridge address and connected bridge route. The host's generic default route
is expected and is not a conflict.
- `UVM_BRIDGE` has only UVM-owned TAP member ports. It has no physical uplink,
foreign veth, libvirt, container, or otherwise unknown bridge port.
- New inbound service connections originate from the UVM bridge gateway or a
valid address in `UVM_NETWORK`. This is a fixed private-path boundary, not a
user-configurable source-CIDR feature.
- UVM itself installs no DNAT, port-forward, or public-address rule.
The source restriction permits host-to-guest traffic, whose source is normally
the bridge gateway address, and VM-to-VM traffic. It does not affect replies
to guest-initiated external connections, which are identified as reply traffic
through connection tracking.
A private RFC1918 address is not itself a complete public-exposure guarantee.
An administrator can still add host routing, source NAT, DNAT, or another
firewall rule that changes reachability. UVM does not discover or override
such external policy; it only guarantees the scoped bridge policy after it has
successfully applied and verified it.
## 3. Existing UVM Topology
UVM currently creates one TAP per Firecracker VM and attaches every TAP to the
same Linux bridge. The host bridge address is the guest default gateway.
```text
guest eth0
|
Firecracker virtio-net
|
per-VM TAP: uvm-<suffix>
|
Linux bridge: uvm0 (10.42.0.1/24 by default)
|
host IPv4 routing and existing egress NAT
|
host uplink
```
Relevant current code:
- [`uvm/network.py`](uvm/network.py) allocates guest addresses and MACs,
creates the bridge/TAPs, enables IPv4 forwarding, and installs the current
egress `iptables` rules.
- [`uvm/lifecycle.py`](uvm/lifecycle.py) coordinates VM creation, rollback,
stop, and destruction under the shared operation lock.
- [`uvm/state.py`](uvm/state.py) owns the atomic registry and the operation
lock.
- [`uvm/system.py`](uvm/system.py) provides the testable `CommandRunner`
boundary for local host commands.
An IP-family `FORWARD` rule alone is insufficient for the future port policy.
Traffic between two VM TAPs is normally switched by the Linux bridge at layer
2 and does not traverse an `inet`/`ip` forwarding hook. Host-to-guest traffic
also follows bridge output rather than the ordinary routed-forward path.
Native nftables bridge filtering is designed for these paths and does not
require enabling the legacy `br_netfilter` module. The Linux bridge
documentation specifically discourages `br_netfilter` for new filtering
designs in favor of nftables.
## 4. Candidate Evaluation
All options below are open source and can run entirely on the local Linux
host. The comparison is about fit for UVM's intended lifecycle-managed policy,
not whether the tool is generally useful.
| Option | Local operation | Advantages | Limitations for UVM | Decision |
|---|---|---|---|---|
| Native nftables | Kernel Netfilter plus local `nft` CLI; no daemon required | Bridge-family hooks, conntrack, named sets, atomic batches, precise ownership | Requires a supported kernel and careful coexistence rules | Adopt |
| `iptables-nft` compatibility frontend | Local CLI backed by nftables | Already used by existing UVM egress code; familiar syntax | Does not give a clean bridge-policy model and makes native ownership/order harder to reason about | Do not extend for the new layer |
| `iptables-legacy` plus ebtables/ipset | Local CLI tools | Available on older systems | Separate legacy data path, incremental updates, more components, and bridge IP filtering commonly depends on legacy `br_netfilter` behavior | Do not use |
| firewalld | Local daemon with D-Bus API | Zones, policies, IP sets, local programmatic control | Daemon ownership, runtime/permanent configuration, and policy interactions add operational complexity; it is not optimized for UVM's per-TAP hard-close contract | Optional future adapter only |
| UFW | Local Ubuntu-oriented CLI | Simple operator workflow and routed rules | CLI/rule-number model is not a suitable authoritative dynamic policy API; reload/default-policy changes have broad effects | Do not use as UVM's backend |
| eBPF/XDP or tc/eBPF | Local kernel programs and maps | High performance and dynamic maps | Requires a custom data plane, attachment/pinning lifecycle, verifier-compatible programs, and does not naturally provide UVM's conntrack semantics | Future performance option, not v1 |
### Why not reuse the current iptables implementation?
The existing `NetworkManager._ensure_masquerade()` method incrementally adds
egress NAT and forwarding rules. It does not manage a UVM-owned firewall
table, does not filter bridge-forwarded VM-to-VM frames, and does not provide
atomic whole-policy replacement.
On systems where `iptables` points to the nftables compatibility backend,
native nftables and `iptables-nft` share the kernel nftables data plane but
still create separately managed rule structures. Adding the new policy through
native nftables while leaving current egress rules unchanged is the smallest
safe migration. A later, separately tested change may migrate UVM's existing
egress rules into a dedicated nftables table.
UVM must never change a system-wide `iptables` alternative, flush a global
ruleset, or adopt unrelated rules just because they resemble UVM's topology.
### Why not use firewalld or UFW as the primary backend?
firewalld is a valid local-only solution on hosts where it is already the
authoritative firewall manager. Its D-Bus API and policy model make a future
adapter possible. It adds a daemon and a separate permanent/runtime
configuration model, however, and a firewalld reload intentionally preserves
established connections. That does not by itself meet UVM's close-port
contract.
UFW is well suited to static host policy and can express routed rules. Its own
manual states that its command interface is not intended to expose complete
firewall functionality. Dynamic per-VM reconciliation would depend on parsing
or manipulating a higher-level CLI-managed ruleset, and it still would not
solve bridge-path filtering cleanly.
If a host already uses firewalld or UFW, UVM must not write that manager's
generated tables directly. The operator must choose and test a dedicated
integration profile before using UVM's firewall layer on that host.
## 5. Recommended Security Contract
The v1 policy is deliberately narrow:
| Item | Contract |
|---|---|
| Protocol | IPv4 TCP only |
| Inbound access | Default deny for managed VMs; only explicitly allowed guest TCP ports pass |
| Default new-VM policy | Persist TCP/22 as an ordinary initial allowlist entry unless creation explicitly disables default ports |
| Egress | Guest-initiated TCP, UDP, DNS, and required control traffic remain allowed |
| Existing service connections after close | Subsequent packets are blocked in both directions after the policy update commits |
| IPv6 | Drop IPv6 traffic on managed TAPs until an explicit IPv6 policy exists |
| Guest identity | Bind each managed TAP to its recorded MAC and IPv4 address; reject spoofed source identity |
| New inbound source | Only the UVM bridge gateway or an address in `UVM_NETWORK` may initiate a service connection |
| Legacy records | A missing policy is `unmanaged`, not an implicit allow-all or deny-all policy |
The policy must distinguish an explicit empty list from a legacy record:
```text
open_tcp_ports = None -> legacy/unmanaged; do not claim enforcement
open_tcp_ports = [] -> managed deny-all inbound policy
open_tcp_ports = [22] -> managed policy allowing SSH only
```
HTTP and HTTPS are user-facing labels for TCP/80 and TCP/443. They do not
change packet treatment.
An active unmanaged VM may not share `uvm0` with managed VMs. Before managed
enforcement is enabled, all active legacy records must be stopped and migrated
to an explicit policy, or the operation must fail. This avoids treating an
unfiltered legacy TAP as a trustworthy peer for MAC/IP/ARP identity checks.
### What close guarantees
When a managed VM closes TCP/443, the resulting policy must match both:
- Original-direction packets traveling to that VM's TCP/443 service.
- Reply-direction packets leaving that service on an already established flow.
The implementation must not place a broad `ct state established,related
accept` rule before this service-policy check. Instead, it checks the
connection's original destination address and destination port for every
service-flow packet.
This gives the following guarantee:
> Once the atomic firewall update commits, later packets traversing the VM's
> service path are blocked while the port remains closed.
It cannot retract data already delivered, gracefully close a guest process, or
guarantee an immediately visible close for a completely idle TCP socket. The
design may use `reject with tcp reset` after packet-path tests establish that
the behavior is appropriate; a drop rule is the conservative baseline.
Connection tracking must remain available while enforcing a live close. A
targeted `conntrack` cleanup is useful before a deleted VM IP is reused, but
it is not the primary live-close mechanism. Deleting a conntrack entry too
early can discard the original tuple needed to distinguish a service reply
from a new guest-originated connection.
## 6. nftables Architecture
UVM will own one dedicated bridge-family table, conceptually:
```text
table bridge uvm
```
The table contains only UVM-generated rules, sets, and chains. It must never
use `flush ruleset`, modify a built-in host policy, or clear tables owned by a
different tool.
```mermaid
flowchart LR
LIFE[LifecycleService] --> FIREWALL[NftFirewall]
LIFE --> NETWORK[NetworkManager]
FIREWALL --> RUNNER[CommandRunner]
RUNNER --> NFT[nftables bridge table]
NETWORK --> BRIDGE[uvm0 bridge and TAPs]
NFT --> BRIDGE
BRIDGE --> VM1[Firecracker VM A]
BRIDGE --> VM2[Firecracker VM B]
```
### Hook Coverage
| Bridge hook | UVM use | Required policy action |
|---|---|---|
| `prerouting` | Frame arrives from a managed TAP | Validate expected source MAC/IP before bridge learning; allow only necessary ARP; reject unsupported IPv6 |
| `input` | VM frame enters the host stack, including the gateway path | Apply the VM's source-side policy and allow guest-originated traffic/relevant replies |
| `output` | Host stack sends a frame toward a VM | Apply the destination VM's inbound policy or allow a reply to guest-originated traffic |
| `forward` | Frame moves directly between bridge ports | Apply both source-VM and destination-VM policy; never let one VM's acceptance skip the other VM's check |
The base chains have an `accept` policy so that UVM does not impose a global
deny policy on non-UVM bridge ports. The UVM dispatch rules select only known,
registry-backed TAP names. A denial occurs inside a selected VM's policy path.
Once a frame is selected as to or from a managed TAP, an unmatched frame is a
terminal denial. The v1 policy permits only untagged IPv4 and the narrowly
required ARP traffic on that attachment. It drops IPv6, VLAN-tagged frames,
and other unsupported EtherTypes in both directions. This is separate from
the base-chain policy for non-UVM bridge ports.
For VM-to-VM traffic, dispatch into source and destination policy chains must
use `jump` and `return` semantics. A source-side success returns to the
dispatcher so the destination VM's inbound policy still runs. It must not use
an early terminal `accept` that bypasses the destination check.
### Per-VM Rule Model
Each managed VM contributes a typed policy record:
```text
VM ID
TAP name
guest IPv4 address
guest MAC address
open TCP ports
runtime mode: active or quarantined
```
The generated rules maintain two logical checks:
| Chain role | Original-direction traffic | Reply-direction traffic |
|---|---|---|
| Toward VM | Allow a new TCP flow only from the private-path sources and when its destination port is in the VM's current allowlist | Allow a reply only when it belongs to a guest-originated connection |
| From VM | Allow guest-originated traffic | Allow a service reply only when the original destination VM/port is still in the allowlist |
The implementation uses connection tracking's original tuple, rather than the
packet's current source/destination port alone, for service replies. For
example, a web server reply is sent *from* TCP/443 to a client ephemeral port;
the original tuple still identifies TCP/443 as the guest service.
The actual nft syntax is an implementation detail and must be generated from
typed data. It will use `ct direction`, `ct original ip daddr`, and `ct
original proto-dst` only after an isolated integration test verifies the
target kernel's bridge conntrack behavior.
### Identity and Layer-2 Controls
Filtering only TCP ports is not enough if a guest can spoof another guest's
source identity. For every managed TAP, the policy must:
- Accept a frame from the TAP only when its Ethernet source equals the
persisted guest MAC.
- Accept an IPv4 frame from the TAP only when its source equals the persisted
guest IPv4 address.
- Bind ARP sender hardware and protocol addresses from the TAP to that same
MAC/IP pair; allow only the required target/address-probe forms after packet
testing.
- Drop IPv6, tagged Ethernet frames, and unsupported EtherTypes both to and
from the managed TAP while they have no corresponding policy.
The identity check belongs before or at the earliest bridge ingress point. It
must be validated against the actual bridge forwarding database behavior on
the target kernel. This is meaningful network-policy hardening, but it does
not turn UVM into a complete hostile-tenant platform. Stronger tenant
isolation may eventually require per-VM L2/L3 segments, bridge-port
isolation, VLANs, or network namespaces.
### Ownership and Rule Rendering
UVM owns only its dedicated table and resources carrying a fixed versioned
UVM marker. The adapter must:
1. Refuse an unexpected pre-existing UVM table or an ownership-marker
mismatch rather than adopting it.
2. Render the complete desired contents of its own table from current state.
3. Apply the generated table in one nftables transaction.
4. Inspect the owned table after a mutation when a stronger confirmation is
required.
5. Remove a per-VM dispatch entry only after its TAP is confirmed absent or
safely quarantined.
The exact table replacement batch must be tested against the supported nft
version. It may rebuild only `table bridge uvm`; it must never rebuild the
host's whole nftables ruleset.
## 7. Persistence and Reconciliation
The JSON registry remains the source of desired policy. Kernel firewall state
is observed state and is never treated as durable truth.
The future `VmRecord` model needs to persist at least:
```text
name (if VM naming is added)
open_tcp_ports: list[int] | None
policy_intent / pending lifecycle phase
policy generation
```
`None` preserves the legacy/unmanaged distinction. New VMs always have an
explicit policy, including a deliberately empty policy.
The registry also needs a topology fingerprint at the state level containing
the configured network CIDR, gateway, and bridge name. Lifecycle and firewall
preflight compare the current `Settings` against that persisted fingerprint
before touching interfaces or rules. A legacy registry without this fingerprint
requires an explicit migration; it must not infer that a changed `/24` and
`/16`, for example, describe the same UVM topology.
UVM should not persist an authoritative `policy_enforced: true` boolean. It
would become stale after a reboot, manual firewall change, partial failure, or
host firewall reload. Instead, `NftFirewall.reconcile()` renders the desired
state from the registry and inspects the UVM-owned table when needed.
Reconciliation has no continuous trigger in this design. It runs during
create, start, port mutations, and an explicit future policy-sync operation.
Read-only list/inspect commands may report an owned table as missing or
unverified, but must not silently mutate the host. If an active VM loses its
UVM firewall table between operations, its policy is unknown until a successful
explicit reconciliation; UVM must not claim it remains enforced.
### Mutation Ordering
All firewall mutations use the existing `operations.lock`. State and kernel
updates cannot be one cross-resource transaction, so durable intent and
observed enforcement must remain distinct:
| Action | Required order |
|---|---|
| Create | Persist explicit desired policy, create the TAP down, install/verify policy, activate TAP, then launch Firecracker |
| Start | Reconcile the saved policy before activating the retained TAP or launching a new VMM |
| Open port | Persist desired policy first, atomically apply it, then clear the pending state |
| Close port | Persist the more restrictive desired policy first, atomically apply it, then clear the pending state |
| Stop | Persist stop intent, quarantine the TAP, confirm VMM exit, remove/deactivate TAP, retain saved policy |
| Delete | Quarantine, confirm exit, remove TAP, remove UVM policy for the VM, clear stale flow state before IP reuse, then remove registry data |
If an open operation cannot be applied, the desired state may say "open" while
the host remains more restrictive. UVM reports failure and retries through
reconciliation; it must not claim the port is open. If a close operation
cannot be confirmed, UVM retains its restrictive intent and attempts to
quarantine the attachment rather than restoring the old policy.
There is an unavoidable crash window after restrictive intent is persisted and
before the nftables transaction commits. During that window, an already-running
VM can still have its old permissive kernel policy. An interrupted close stays
pending and is not reported as enforced until an explicit reconciliation or
quarantine successfully commits. The operation lock cannot close this window
after the UVM process exits.
### Quarantine
An empty inbound allowlist is not a quarantine because it still permits a VM
to initiate outbound traffic. A quarantine mode drops traffic both to and from
the owned TAP. It is required while a stop, delete, or failed boot leaves
process/TAP ownership unresolved.
Do not remove this guard merely because a cleanup operation was attempted. It
remains until the TAP is gone or a successful start replaces it with the
saved active policy.
## 8. Python Integration
### Internal Adapter Boundary
Add a focused internal module such as `uvm/firewall.py`. It should be
constructed in [`uvm/app.py`](uvm/app.py), similarly to the current
`NetworkManager` and Firecracker process manager.
```python
firewall = NftFirewall(settings, runner)
lifecycle = LifecycleService(settings, state_store, images, network, process, firewall)
```
The exact constructor arrangement can vary, but these boundaries must remain:
| Component | Responsibility |
|---|---|
| `LifecycleService` | Owns ordering, operation locking, persistence intent, rollback, and VM status transitions |
| `NetworkManager` | Owns bridge validation plus TAP preparation, activation, and deletion |
| `NftFirewall` | Owns only UVM's generated nftables policy and its local inspection/reconciliation |
| `CommandRunner` | Runs local commands without a shell and makes command construction testable |
| CLI/API routers | Validate user input and call services; never send raw firewall syntax or commands |
The adapter should expose typed operations, for example:
```python
@dataclass(frozen=True, slots=True)
class VmFirewallPolicy:
vm_id: str
tap: str
guest_ip: IPv4Address
guest_mac: str
open_tcp_ports: tuple[int, ...]
quarantined: bool = False
class NftFirewall:
def preflight(self) -> None: ...
def reconcile(self, policies: Iterable[VmFirewallPolicy]) -> FirewallObservation: ...
def inspect(self) -> FirewallObservation: ...
def purge_vm_flows(self, vm: VmFirewallPolicy) -> None: ...
```
The public surface should accept domain objects, not interface names or raw
rules supplied by a CLI/API caller. There should be no `apply_raw`, `flush`,
or generic arbitrary-command method.
### Command Transport
The first implementation should call the local `nft` executable through
`CommandRunner`, rather than add a Python package dependency:
```python
runner.run(
("nft", "--check", "--file", "-"),
input_text=ruleset,
capture=True,
timeout=settings.firewall_timeout_s,
)
runner.run(
("nft", "--file", "-"),
input_text=ruleset,
capture=True,
timeout=settings.firewall_timeout_s,
)
```
This follows the same engineering model as Firecracker's Unix-socket adapter:
a narrow typed UVM boundary hides an external host interface. For nftables,
the host interface is a local command/Netfilter transaction rather than HTTP.
`nft --check` and `nft --file -` are separate operations. The checked batch is
only a validation aid; UVM may claim a policy change only after the apply
transaction succeeds and the adapter records its observation.
Benefits of the CLI approach:
- No shell interpolation or raw shell scripts.
- Atomic batch application through `nft --file -`.
- Syntax/preflight checking through `nft --check`.
- Easy fake-runner unit tests and visible CLI traces.
- No additional Python runtime dependency.
Inspection should use JSON rather than human-formatted output:
```python
runner.run(
("nft", "--json", "list", "table", "bridge", "uvm"),
capture=True,
timeout=settings.firewall_timeout_s,
)
```
`python3-nftables`/libnftables is a viable future transport if avoiding a
subprocess becomes necessary. It must not be introduced casually: package API
and kernel compatibility need a version-pinned integration suite. Direct
Netlink programming is not justified for the first implementation.
### Errors
Introduce a `FirewallError(UvmError)` for semantic firewall failures such as
unsupported local capabilities, unexpected ownership, or an unsafe observed
table. Preserve `CommandError` for an `nft` or `conntrack` command failure.
Errors must include the failed operation and safe diagnostics, but never
include raw untrusted rule input because the adapter has none. The CLI and API
can retain their existing `UvmError` rendering conventions.
## 9. Installation and Host Requirements
The installer remains the only component that invokes `apt-get`. The future
package list should add:
```text
nftables
conntrack
```
`conntrack` is used for targeted stale-flow cleanup before a deleted VM IP is
reallocated. It is not a substitute for the firewall's live close policy.
On Ubuntu-compatible hosts, the installation step is conceptually:
```sh
apt-get install --yes nftables conntrack
```
Do not automatically enable `nftables.service`, write `/etc/nftables.conf`,
or install a system-wide persistence service. UVM dynamically owns only its
dedicated table and recreates/reconciles it during lifecycle operations.
### Required Host Capabilities
| Requirement | Reason |
|---|---|
| Linux with native nftables support | UVM uses local Netfilter tables |
| Kernel with bridge-family connection tracking support | Required for original-tuple service checks; upstream support starts in Linux 5.3 |
| `nft`, `conntrack`, and `ip` commands | Local policy, selected flow cleanup, and TAP/bridge management |
| `CONFIG_BRIDGE` and `CONFIG_TUN` | Existing UVM bridge/TAP topology |
| Netfilter nftables, conntrack, bridge, reject, and relevant protocol modules | Required to load/apply the generated table |
| Root or `CAP_NET_ADMIN` in the bridge network namespace | Interface and firewall administration |
| No flowtable/hardware offload that bypasses this filter path | Required for reliable close enforcement |
Preflight should verify the actual generated ruleset with `nft --check` on the
target host. Kernel-version checks alone are not enough; distributions can
build or package Netfilter capabilities differently.
UVM must not enable `br_netfilter` or change `net.bridge.bridge-nf-call-*`
sysctls. Native bridge nftables works without that legacy compatibility layer.
## 10. Configuration Approach
The first version should avoid a generic `UVM_FIREWALL_RULES` setting or any
configuration value containing raw nftables syntax.
The generated policy derives from existing UVM configuration and state:
| Source | Firewall use |
|---|---|
| `UVM_NETWORK` | Validates guest IPv4 addresses and UVM subnet membership |
| `UVM_GATEWAY` | Establishes the bridge/guest topology for ARP and gateway traffic |
| `UVM_BRIDGE` | Selects the UVM bridge and its managed TAP attachments |
| `VmRecord.tap`, `VmRecord.mac`, `VmRecord.guest_ip` | Binds policy to one owned VM attachment |
| Persisted `open_tcp_ports` | Creates the per-VM TCP allowlist |
One bounded command timeout, such as `UVM_FIREWALL_TIMEOUT_S`, is reasonable
as a `Settings` value. It should default to a small value and apply only to
UVM's local command invocations.
Changing `UVM_NETWORK`, `UVM_GATEWAY`, or `UVM_BRIDGE` after VMs exist is not
a firewall migration mechanism. UVM must fail rather than infer ownership of
old interfaces or rules from changed environment values.
## 11. Lifecycle Integration
The existing `NetworkManager.create_tap()` activates a TAP immediately. The
firewall layer requires that operation to be split into preparation and
activation:
```text
create TAP down -> attach to bridge -> install policy -> bring TAP up
```
### Create and Start
```mermaid
sequenceDiagram
participant L as LifecycleService
participant S as StateStore
participant N as NetworkManager
participant F as NftFirewall
participant P as Firecracker
L->>S: persist VM and explicit policy intent
L->>N: ensure bridge; prepare TAP down
L->>F: reconcile active policy including new VM
F-->>L: policy confirmed
L->>N: activate TAP
L->>P: spawn, configure, and start VMM
L->>S: persist running state
```
A policy failure prevents TAP activation and Firecracker launch. A startup
failure after policy installation leaves the policy in place until lifecycle
cleanup confirms that the TAP is gone. This is safer than briefly booting an
unfiltered guest.
### Stop, Delete, and Failure Cleanup
```text
stop/delete intent
-> quarantine owned TAP
-> terminate verified Firecracker process and confirm exit
-> deactivate/delete TAP
-> stop: retain desired port policy for a later start
-> delete: remove VM policy, purge stale flows, remove disk/state
```
If process identity or TAP ownership is unresolved, keep the quarantine guard,
disk, and registry entry. Do not delete a possibly live VM disk or remove its
network guard simply because a cleanup command raised an error.
Shared table infrastructure may remain after the last VM is deleted. It is
not a resource leak: it is a UVM-owned baseline that can be reconciled by the
next lifecycle operation. It must contain no stale per-VM dispatch/policy
entries after successful deletion.
## 12. Coexistence and Security Boundaries
The initial supported profile is a host where UVM may own `table bridge uvm`
and no other component modifies that table, using the dedicated private bridge
profile described above.
This does not mean UVM owns the host firewall. In particular:
- A terminal `drop` in UVM's bridge table can protect a UVM TAP.
- An `accept` in one nftables base chain does not promise that later hooks or
another firewall manager will accept the packet.
- UVM must not change global default policies, Docker chains, firewalld zones,
UFW framework files, or a host's `nftables.service` configuration.
- If Docker, Podman, UFW, firewalld, libvirt, or site firewall tooling is
active, allowed traffic may still be blocked by that tooling. UVM should
report its own enforcement state honestly rather than claim reachability.
The present design controls the local private network path only. It does not
protect against a privileged host process, a guest root exploit, raw host
network administration, or future external DNAT rules. Source-IP allowlists
and public exposure, if added, require a separate `inet`-family and ingress
design.
## 13. Testing and Release Gates
### Unit Tests
Add a focused `tests/test_firewall.py` with a stateful fake runner. Test:
- Generated table ownership, no `flush ruleset`, and no built-in chain policy
changes.
- Exact TAP-scoped dispatch and VM policy rendering.
- One atomic batch per policy update.
- Validation failures before host mutation.
- Idempotent reconciliation and recovery from missing UVM-owned rules.
- Refusal to adopt an unexpected table or marker mismatch.
- No broad `established,related` acceptance before service policy checks.
- Close-policy rendering that blocks both request and reply service packets.
- Quarantine and policy retention across stop/start/delete.
- Lifecycle ordering: policy before TAP activation and Firecracker spawn.
- Failure injection at state, nft, process, TAP, and cleanup boundaries.
### Isolated Network-Namespace Proof
Before integrating with real Firecracker, build a disposable network-namespace
test using a bridge and veth peers standing in for TAPs. It must prove:
1. Host-to-VM traffic reaches an allowed TCP port and fails at a closed port.
2. VM-to-VM traffic is governed by the destination VM's policy.
3. Guest-initiated TCP/UDP egress and replies still work.
4. Closing a port during one-way server streaming blocks subsequent reply
packets as well as new requests.
5. A guest outbound connection using a source port equal to a closed service
port is not falsely blocked as inbound service traffic.
6. ARP/gateway behavior remains functional.
7. IPv6 cannot bypass the IPv4-only policy.
8. MAC/IP spoofing from a managed TAP is rejected.
9. Tagged IPv4/IPv6 and unsupported EtherTypes are denied at managed TAPs.
10. An active unmanaged TAP prevents managed-policy activation.
11. Unrelated bridge ports and host firewall tables are unaffected.
Only after that proof should an isolated Firecracker test verify real TAP/MAC
behavior, default SSH access, an allowed guest listener, and live close.
## 14. References
- [Linux bridge Netfilter documentation](https://docs.kernel.org/networking/bridge.html#netfilter)
- [nftables bridge filtering](https://wiki.nftables.org/wiki-nftables/index.php/Bridge_filtering)
- [nftables atomic rule replacement](https://wiki.nftables.org/wiki-nftables/index.php/Atomic_rule_replacement)
- [nftables connection-tracking metadata](https://wiki.nftables.org/wiki-nftables/index.php/Matching_connection_tracking_stateful_metainformation)
- [firewalld concepts and policies](https://firewalld.org/documentation/concepts.html)
- [UFW manual](https://manpages.ubuntu.com/manpages/noble/man8/ufw.8.html)
- [iptables manual](https://man7.org/linux/man-pages/man8/iptables.8.html)
- [Linux `bpf(2)` manual](https://man7.org/linux/man-pages/man2/bpf.2.html)
README.md
+192 -67
View File
@@ -2,7 +2,9 @@
`uvm` is a small, local Firecracker microVM command-line manager for Ubuntu.
It creates one Firecracker process per VM, attaches it to a host TAP device,
and persists enough state to list, stop, and destroy the VM later.
and keeps a private disk and local registry for named VMs. The CLI supports
create, start, stop, restart, delete, inspection, SSH, logs, diagnostics, and
enforced private-IPv4 TCP port allowlists.
This directory is a self-contained project. All UVM code, tests, packaging
metadata, launcher scripts, and documentation live here.
@@ -13,14 +15,17 @@ workflows, see [DEVELOPER.md](DEVELOPER.md).
For end-to-end operator scenarios, CLI examples, and HTTP API calls, see
[USAGE.md](USAGE.md).
For the original firewall proposal and backend evaluation, see
[FIREWALL.md](FIREWALL.md). That proposal is design background, not the current
feature contract; these guides describe the implementation in the workspace.
## Quick Start
From the parent repository directory, run commands from this project root:
Run commands from this project root:
```sh
cd uvm
./uvm.py --help
python3 -m uvm --help
python3 -B -m uvm --help
```
The `pyproject.toml` file also defines an installable `uvm` console command.
@@ -32,12 +37,16 @@ The project is an application, not a reusable Python SDK.
- Python 3.12 or newer
- Root access for UVM commands that read or mutate the protected VM registry
- KVM enabled and accessible at `/dev/kvm`
- Linux 5.3 or newer with native bridge conntrack and safe pidfd support
- An x86_64 host for the default guest assets
- Working host networking, `iproute2`, `iptables`, `e2fsprogs`, `openssl`, and
`openssh-client`
- Working host networking, `iproute2`, `iptables`, `nftables`, `conntrack`,
`e2fsprogs`, `util-linux`, `openssl`, and `openssh-client`
- An STP-disabled software bridge; no offload path that bypasses bridge hooks
`uvm install` installs the required host packages. It validates `/dev/kvm`
before downloading or launching a microVM.
before downloading VM artifacts. Boot operations additionally check that the
native nftables bridge policy is supported; installing packages alone is not
a connectivity or isolation test.
## Trusted Installation
@@ -45,8 +54,6 @@ Installation requires trusted SHA-256 digests by default. Supply digests from
an independently trusted source before running the privileged installer:
```sh
cd uvm
sudo env \
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
@@ -72,59 +79,134 @@ available through an explicit opt-out:
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
```
An unverified install is recorded as such and cannot satisfy a later strict
An unverified manifest is not accepted as publisher trust in a later strict
run. Reinstall with trusted digests to establish a verified local manifest.
To create VMs from an unverified local-development install, retain
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create` commands and on the API
server process.
To boot VMs from an unverified local-development install, retain
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` on later `create`, `start`, and `restart`
commands and on the API server process.
## Commands
These are separate operation examples, not a script to run from top to bottom.
```sh
# Install after supplying the trusted checksum variables shown above.
# A bare `install` command fails under the default strict integrity policy.
# Create and boot a VM with the defaults: 1 CPU, 512 MiB RAM, and root/root.
sudo ./uvm.py create
# Create a named VM: 1 CPU, 512 MiB RAM, root/root, and private TCP/22.
# Use a unique non-default password for anything beyond a throwaway VM.
sudo ./uvm.py create --name web
# Request CPU and RAM explicitly. RAM defaults to MiB; B/K/M/G suffixes work.
sudo ./uvm.py create --cpu 1 --ram 1G
# Choose resources, a private guest IP, and additional service ports.
sudo ./uvm.py create --name app --cpu 1 --ram 1G \
--guest-ip 10.42.0.10 --ports 80:HTTP,443:HTTPS
# Set credentials for an account that already exists in the guest image.
sudo ./uvm.py create --username root --password '<guest-password>'
# Read a password from one line of stdin, rather than a --password argument.
# USAGE.md includes a non-echoing Bash prompt example.
sudo ./uvm.py create --name worker --password-stdin
# Select the guest IP. Despite the legacy flag name, this is the guest IP.
sudo ./uvm.py create --host-ip 10.42.0.10
# Explicitly allow no inbound service ports, including SSH.
sudo ./uvm.py create --name closed --no-default-ports
# Show persisted VM state and observed process status.
# Inspect persisted state and observed process/policy status.
sudo ./uvm.py list
sudo ./uvm.py list --status running --json
sudo ./uvm.py list -q
sudo ./uvm.py inspect web --json
sudo ./uvm.py logs web --tail 50 --follow
sudo ./uvm.py doctor --json
# Connect over SSH by VM ID or guest IP.
sudo ./uvm.py ssh vm-<id>
# Resolve VMs by exact ID, name, or guest IP.
sudo ./uvm.py ssh web
sudo ./uvm.py ssh app --port 22 -- uname -a
ssh root@10.42.0.10
# Stop or remove a VM.
sudo ./uvm.py stop vm-<id>
sudo ./uvm.py destroy vm-<id>
# Stop and boot the same private disk, or restart selected VMs.
sudo ./uvm.py stop web --timeout 5
sudo ./uvm.py start web
sudo ./uvm.py restart web app --timeout 10
# Preview bulk operations before confirming them.
sudo ./uvm.py stop --all --dry-run --json
sudo ./uvm.py stop --all -y
sudo ./uvm.py start --all -y
sudo ./uvm.py restart --all -y
sudo ./uvm.py delete --all --dry-run
sudo ./uvm.py delete web -y
# destroy is an alias for delete, with the same flags and destructive effect.
```
`start`, `stop`, `restart`, and `delete` accept one or more VM identifiers or
`--all`, plus `--json`, `--dry-run`, and `--yes`/`-y`. Deletion and mutating
`--all` selections require confirmation; non-interactive callers must use
`--yes`. `--timeout` is available on stop/restart/delete (and `destroy`), not
start. It is a finite non-negative SIGTERM wait in seconds, default `5`, not
a total command deadline or a guest shutdown timeout.
Stop sends SIGTERM and, if needed, SIGKILL to the Firecracker VMM. It is **not
a guest-coordinated shutdown**. Start boots the retained disk without resetting
guest data, SSH host keys, or credentials; it does not restore guest memory.
Delete permanently removes the disk and releases the VM's name and IP after
cleanup. See [USAGE.md](USAGE.md) for the complete flag and recovery reference.
Names are unique local identifiers, not DNS names. `--host-ip` remains an
alias for `--guest-ip`; both select the guest's private IPv4 address. JSON
output omits passwords, with warnings and host-command traces on stderr.
SSH uses `StrictHostKeyChecking=accept-new` and a VM-ID-specific host-key
alias by default. `--insecure-host-key` restores the old no-verification
behavior for a single connection.
alias by default. `--user`, `--key`, and `--port` customize the SSH client;
append a remote command after a literal `--`. `--insecure-host-key` disables
verification for one connection. SSH does not automatically supply the stored
password or open a closed port.
### Private Ports
```sh
sudo ./uvm.py ports list web --json
sudo ./uvm.py ports list --all
sudo ./uvm.py ports open web --ports 80:HTTP --ports 443:HTTPS,8080:TCP
sudo ./uvm.py ports close web --ports 80
sudo ./uvm.py ports set web --ports 22,443:HTTPS --dry-run --json
sudo ./uvm.py ports set web --ports 22,443:HTTPS -y
sudo ./uvm.py ports close web --all -y
sudo ./uvm.py ports set web --none -y
sudo ./uvm.py ports sync web
sudo ./uvm.py ports sync --all --dry-run
sudo ./uvm.py ports sync --all -y
```
Rules use `PORT[:TYPE]`, for example `22`, `80:HTTP`, `443:HTTPS`, or
`8080:TCP`, repeated or comma-separated. HTTP and HTTPS are **TCP labels**,
not proxies or TLS configuration. Create adds `22:TCP` even when other ports
are supplied, unless `--no-default-ports` is set. Later port edits do not
automatically add SSH back. `ports close VM --all` closes ports on that one
VM; only `ports list --all` and `ports sync --all` select all VMs.
Port mutations support `--dry-run` and `--json`; close/set/sync also support
`--yes`/`-y`. Set, close-all, and removal of TCP/22 require confirmation
(an already empty managed policy can be a no-op). Closing a service blocks
subsequent packets on existing service connections in **both directions**, as
well as new connections. This is packet dropping, not a guaranteed TCP RST or
immediate application disconnect.
These rules govern access to the **guest private IP**. They do not start a
guest service or provide public connectivity, host-port mappings, DNS,
reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not
yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required
ARP/ICMP traffic remain allowed; managed TAPs block IPv6.
## Management API
Install the project dependencies before serving the API:
```sh
cd uvm
python3 -m pip install .
```
Launch the FastAPI management server with the requested CLI form:
Launch the FastAPI management server using the environment where the project
dependencies were installed (use the full console-script path if needed):
```sh
cd uvm
export UVM_API_TOKEN='replace-with-a-long-random-secret'
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
uvm --serve --host 127.0.0.1 --port 8000
@@ -144,13 +226,19 @@ The server exposes these routes:
| `GET` | `/health` | Unauthenticated liveness response |
| `POST` | `/install` | Install or refresh host assets; body: `{ "force": false }` |
| `GET` | `/vms` | List VM records and observed status |
| `POST` | `/vms` | Create a VM; body supports `cpu`, `ram`, `guest_ip`, `username`, and `password` |
| `GET` | `/vms/{id-or-ip}` | Retrieve one VM by ID or guest IP |
| `POST` | `/vms/{id}/stop` | Stop a VM while retaining its private disk |
| `DELETE` | `/vms/{id}` | Destroy a VM and release its resources |
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/stop` | Stop a VM while retaining its private disk |
| `DELETE` | `/vms/{identifier}` | Delete a VM and release its resources |
The API intentionally does not proxy interactive SSH. Use the returned guest
IP with `uvm ssh` or a normal SSH client.
API port rules are objects such as `{"port": 80, "protocol": "HTTP"}`;
`no_default_ports` is a boolean, default `false`. Responses include `name`,
`ports`, `policy_status`, and `operation`, but never passwords. See
[USAGE.md](USAGE.md) for request examples and strict schema details.
The endpoint set is unchanged: there are **no start/restart, port-management,
or bulk HTTP endpoints**. SSH, logs, and doctor are also CLI-only. Use the
local CLI for those operations; the API is not a background reconciler.
### API Security
@@ -219,6 +307,24 @@ it, enables IPv4 forwarding, adds an egress masquerade rule, and adds matching
forward rules for return traffic. Guest IPs, gateway, bridge name, and guest
boot netmask derive from the configured CIDR.
The existing `iptables` egress NAT/forwarding behavior is unchanged. Private
port enforcement uses separate per-VM native nftables `bridge` tables, not
DNAT or host listeners. Policy is installed and read back before an owned TAP
is activated and before the guest starts. The software bridge must have STP
disabled; each TAP's bridge-slave `group_fwd_mask` is set to `0x4000` so LLDP
cannot bypass the early identity filter. Native bridge conntrack requires
Linux >= 5.3; no `br_netfilter` sysctl setup is required by this policy layer.
Host-to-VM and VM-to-VM traffic crossing these hooks is filtered. An allowed
port is not a reachability guarantee, and a private IP alone is not protection
against administrator-supplied external routing or NAT. Integration with UFW,
firewalld, Docker, or other external firewall managers remains deferred.
Do not offload traffic around the software bridge hooks.
There is no autostart service or daemon reconciliation. After externally lost
or changed UVM rules, inspect policy status and explicitly run `ports sync`.
Read-only commands do not repair rules; sync does not boot stopped VMs.
## Persistent State
The default data directory is `/var/lib/uvm`:
@@ -232,9 +338,9 @@ The default data directory is `/var/lib/uvm`:
└── state.json # Root-only VM inventory and guest credentials
```
State writes are locked and atomic. Lifecycle operations share an additional
operation lock so concurrent `create`, `stop`, `destroy`, and `install`
commands cannot overwrite each other's state or binaries.
The default registry is `/var/lib/uvm/state.json`, schema version `2`. Writes
are locked and atomic. Lifecycle, port-policy, and installation mutations
share an operation lock; bulk operations use a fixed snapshot of VM IDs.
The registry stores guest usernames and passwords in plaintext, so `state.json`
is mode `0600`. Treat it as a secret, do not include it in bug reports, and
@@ -243,14 +349,28 @@ replace the default `root` password immediately.
Credential provisioning applies only to VMs created by this version. Existing
VM disks are not modified automatically.
Older registries are read and migrated on the next write. A missing/null
`ports` policy remains `None`/`unmanaged`, not an empty allowlist or an
implicitly enforced default. Migrate a legacy VM explicitly with `stop`,
`ports set VM --ports 22,80:HTTP -y` (choose the actual required ports), then
`start`. Resume requires a complete, safe private disk, a verified kernel and
VMM, and compatible network settings. Removing the original rootfs template
does not block start of a retained private disk.
Process identity includes PID, Linux process start time, and host boot ID.
A running legacy PID missing the boot ID is adopted only when its saved start
time, running binary, exact API-socket command line, and protected paths all
match. Missing or ambiguous evidence fails closed: UVM will not guess which
process to signal or release a disk while process exit is unconfirmed.
## Configuration
All optional configuration is supplied through environment variables:
Host and API settings can be overridden through environment variables:
| Variable | Purpose |
|---|---|
| `UVM_BASE` | Data directory; defaults to `/var/lib/uvm` |
| `UVM_NETWORK` | Guest IPv4 CIDR; defaults to `10.42.0.0/24` |
| `UVM_NETWORK` | Guest private IPv4 CIDR; defaults to `10.42.0.0/24` |
| `UVM_GATEWAY` | Bridge/guest gateway; defaults to `10.42.0.1` |
| `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` |
| `UVM_KERNEL_URL` | Guest kernel download URL |
@@ -267,52 +387,57 @@ All optional configuration is supplied through environment variables:
## Project Layout
```text
uvm/
├── README.md
./
├── README.md, USAGE.md, DEVELOPER.md
├── pyproject.toml
├── uvm.py # Executable compatibility launcher
├── uvm/ # Private application implementation
│ ├── cli.py # argparse, terminal output, SSH execution
│ ├── app.py # Application composition root
│ ├── api_models.py # FastAPI request and response models
│ ├── config.py # Settings and filesystem layout
│ ├── domain.py # Typed VM and state models
│ ├── errors.py # Expected application failures
│ ├── integrity.py # Checksums and manifest provenance
│ ├── install.py # Host/artifact installation
│ ├── images.py # Guest templates and per-VM disks
│ ├── lifecycle.py # Create/list/stop/destroy orchestration
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port-policy intent
│ ├── network.py # IP/MAC/TAP/bridge/NAT management
│ ├── routers/ # Health, installation, and VM HTTP routers
│ ├── server.py # FastAPI application factory and Uvicorn launcher
│ ├── state.py # Locked atomic JSON persistence
│ ├── system.py # Host command and KVM adapters
│ ├── firewall.py # Owned nftables bridge policies and offline conntrack cleanup
│ └── firecracker/ # Firecracker config, API, and process adapters
└── tests/ # Unit tests with mocked host boundaries
└── tests/ # Unit tests and opt-in isolated namespace packet tests
```
See [DEVELOPER.md](DEVELOPER.md) for the complete module map and ownership rules.
## Testing
From the parent repository directory, run the unit suite from this project root:
Run the unit suite from this project root:
```sh
cd uvm
python3 -m pip install '.[test]'
PYTHONDONTWRITEBYTECODE=1 python3 -B -m unittest discover -s tests -v
python3 -B -m unittest discover -s tests -v
```
The unit tests do not need root, KVM, Firecracker, or host network changes.
They cover validation, state migration and locking, integrity manifests,
network command rendering, Firecracker request ordering, process safety, and
lifecycle rollback. API route tests run automatically when the FastAPI
and `httpx` test dependencies are installed.
network/firewall rendering, Firecracker request ordering, process safety,
retained-disk resume, bulk operations, and lifecycle rollback. API route tests
run when the FastAPI and `httpx` test dependencies are installed.
Opt-in packet checks run **only in disposable isolated user/network
namespaces**, which may require privileges and the installed network tools:
```sh
UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
```
The packet test was skipped in this workspace because namespace creation was
denied (`Operation not permitted`). A skip is not packet-path or real-host
validation. The harness uses veths in place of TAPs and does not boot a real VM;
validate KVM, TAP, NAT, SSH, and guest connectivity on an isolated Linux host.
## Current Boundaries
- Fractional CPU requests still round Firecracker vCPUs up and are advisory.
- `stop` retains the private VM disk; the current command surface does not yet
include a separate `start` command.
- No quotas or cgroup CPU, memory, I/O, or process limits are applied.
- Firecracker is launched directly by this local CLI. Jailer/cgroup hardening
is not implemented by the current command set.
- No VM autostart, background reconciliation, guest readiness probing,
snapshots, DHCP, public-port publishing, or multi-host scheduling.
- The API uses one shared token, not users, roles, or tenants. UVM is not a
hardened multi-tenant platform.
- Run real KVM, TAP, and guest-connectivity checks on an isolated Linux host
before relying on UVM for workloads.
USAGE.md
+515 -313
View File
File diff suppressed because it is too large Load Diff
tests/integration/test_private_ports.py
+421
View File
@@ -0,0 +1,421 @@
"""Opt-in real packet tests, using only disposable user/network namespaces.
Run: UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
Needs Linux, nft, ip, bridge, conntrack, sysctl and permission to create network
namespaces (root, or unprivileged user namespaces). No sudo, installation,
/run/netns mounts, host firewall changes or br_netfilter sysctls are used.
Bridge-connected veths stand in for guest TAPs. A third, routed namespace tests
guest egress without NAT. Ordinary test runs skip without executing commands.
"""
from __future__ import annotations
import contextlib
import json
import os
import re
import select
import shutil
import signal
import socket
import struct
import subprocess
import sys
import time
import unittest
from ipaddress import IPv4Address, IPv4Network
from pathlib import Path
class SocketEndpoint:
"""Synchronous socket operations; the kernel completes queued handshakes."""
def __init__(self):
self.sockets = {}
def close(self):
for sock in self.sockets.values():
sock.close()
def request(self, operation, **args):
if operation == "run":
result = subprocess.run(args["command"], check=True, text=True,
capture_output=True, timeout=5)
return result.stdout
if operation in {"listen", "connect", "udp_bind"}:
family = socket.AF_INET6 if ":" in args["ip"] else socket.AF_INET
kind = socket.SOCK_DGRAM if operation == "udp_bind" else socket.SOCK_STREAM
sock = socket.socket(family, kind)
sock.settimeout(1)
if kind == socket.SOCK_STREAM:
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
if family == socket.AF_INET6:
sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
try:
if operation == "connect":
sock.connect((args["ip"], args["port"]))
else:
sock.bind((args["ip"], args["port"]))
if operation == "listen":
sock.listen(16)
except TimeoutError:
sock.close()
return False
except BaseException:
sock.close()
raise
self.sockets[args["name"]] = sock
return True
if operation == "accept":
sock, _ = self.sockets[args["listener"]].accept()
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
sock.settimeout(1)
self.sockets[args["name"]] = sock
return True
if operation in {"send", "udp_send"}:
sock = self.sockets[args["name"]]
data = args["data"].encode("ascii")
if operation == "send":
sock.sendall(data)
else:
sock.sendto(data, (args["ip"], args["port"]))
return True
if operation in {"recv", "udp_recv"}:
sock = self.sockets[args["name"]]
sock.settimeout(0.4 if args.get("blocked") else 2)
data = b""
try:
if operation == "udp_recv":
data, address = sock.recvfrom(4096)
return [data.decode("ascii"), list(address)]
while len(data) < args["size"]:
chunk = sock.recv(args["size"] - len(data))
if not chunk:
break
data += chunk
except TimeoutError:
return data.decode("ascii") if data else None
return data.decode("ascii")
if operation == "ping":
ident = os.getpid() & 0xFFFF
frame = struct.pack("!BBHHH", 8, 0, 0, ident, 1) + b"uvm-private-ports!"
padded = frame + b"\0" * (len(frame) % 2)
checksum = sum(struct.unpack(f"!{len(padded) // 2}H", padded))
checksum = (checksum & 65535) + (checksum >> 16)
checksum = (~((checksum & 65535) + (checksum >> 16))) & 65535
frame = frame[:2] + struct.pack("!H", checksum) + frame[4:]
with socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP) as sock:
sock.settimeout(1)
sock.sendto(frame, (args["ip"], 0))
deadline = time.monotonic() + 1
while time.monotonic() < deadline:
try:
data, address = sock.recvfrom(65535)
except TimeoutError:
return False
offset = (data[0] & 15) * 4
if (address[0] == args["ip"] and data[offset] == 0
and data[offset + 4:offset + 8] == frame[4:8]):
return True
return False
if operation == "raw":
with socket.socket(socket.AF_PACKET, socket.SOCK_RAW) as sock:
sock.bind(("eth0", 0))
sock.send(bytes.fromhex(args["frame"]))
return True
raise AssertionError(f"unknown endpoint operation {operation}")
def _require_isolated(parent_fd):
parent, current = os.fstat(parent_fd), os.stat("/proc/self/ns/net")
if (parent.st_dev, parent.st_ino) == (current.st_dev, current.st_ino):
raise RuntimeError("refusing packet/firewall operations in the parent network namespace")
os.close(parent_fd)
class NamespaceEndpoint:
def __enter__(self):
with open("/proc/self/ns/net", "rb") as parent:
self.process = subprocess.Popen(
("unshare", "--net", sys.executable, "-B", str(Path(__file__).resolve()),
"--agent", str(parent.fileno())),
pass_fds=(parent.fileno(),), stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, text=True,
)
try:
assert self._read() == "ready"
except BaseException:
self.__exit__(None, None, None)
raise
return self
def __exit__(self, *_):
if self.process.poll() is None:
self.process.terminate()
try:
self.process.wait(timeout=3)
except subprocess.TimeoutExpired:
self.process.kill()
self.process.wait(timeout=3)
for pipe in (self.process.stdin, self.process.stdout, self.process.stderr):
pipe.close()
def _read(self):
if not select.select([self.process.stdout], [], [], 8)[0]:
raise AssertionError("namespace endpoint timed out")
line = self.process.stdout.readline()
if not line:
detail = self.process.stderr.read() if self.process.poll() is not None else ""
raise AssertionError(f"namespace endpoint exited: {detail}")
response = json.loads(line)
if "error" in response:
raise AssertionError(response["error"])
return response["result"]
def request(self, operation, **args):
self.process.stdin.write(json.dumps({"operation": operation, **args}) + "\n")
self.process.stdin.flush()
return self._read()
def _agent(parent_fd):
_require_isolated(parent_fd)
endpoint = SocketEndpoint()
try:
print(json.dumps({"result": "ready"}), flush=True)
for line in sys.stdin:
request = json.loads(line)
try:
result = endpoint.request(**request)
response = {"result": result}
except Exception as error:
response = {"error": f"{type(error).__name__}: {error}"}
print(json.dumps(response), flush=True)
finally:
endpoint.close()
def _sandbox(parent_fd):
_require_isolated(parent_fd)
if not __debug__:
raise RuntimeError("packet verification requires assertions; disable PYTHONOPTIMIZE/-O")
# Direct script execution must import the workspace, not an installed uvm.
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from uvm.config import Settings
from uvm.domain import PortRule, VmRecord
from uvm.errors import UvmError
from uvm.firewall import FirewallManager
from uvm.system import CommandRunner
runner = CommandRunner(emit=None)
def run(*args):
return runner.run(args, capture=True, timeout=5).stdout
assert [link["ifname"] for link in json.loads(run("ip", "-j", "link", "show"))] == ["lo"]
assert not any("table" in item for item in json.loads(run("nft", "--json", "list", "tables"))["nftables"])
settings = Settings(base=Path("/tmp/uvm-firewall-namespace-test"), bridge="fw-br0",
network=IPv4Network("10.231.73.0/24"), gateway=IPv4Address("10.231.73.1"))
manager = FirewallManager(settings, runner)
manager.check_available()
run("ip", "link", "set", "lo", "up")
run("ip", "link", "add", settings.bridge, "type", "bridge", "stp_state", "0")
run("ip", "addr", "add", f"{settings.gateway}/24", "dev", settings.bridge)
run("ip", "link", "set", settings.bridge, "up")
# This IPv4 sysctl belongs only to the freshly verified network namespace.
run("sysctl", "-w", "net.ipv4.ip_forward=1")
metadata = dict(cpu=1, ram_mib=128, gateway=str(settings.gateway),
socket="", config="", log="", network=str(settings.network), bridge=settings.bridge)
vm = VmRecord(id="integration-vm", tap="fw-tap1", guest_ip="10.231.73.2",
mac="02:fc:00:00:00:01", ports=(PortRule(8080, "HTTP"),), **metadata)
peer_vm = VmRecord(id="integration-peer", tap="fw-tap2", guest_ip="10.231.73.3",
mac="02:fc:00:00:00:02", ports=(PortRule(9090),), **metadata)
with contextlib.ExitStack() as stack:
guest = stack.enter_context(NamespaceEndpoint())
peer = stack.enter_context(NamespaceEndpoint())
outside = stack.enter_context(NamespaceEndpoint())
host = SocketEndpoint()
stack.callback(host.close)
for endpoint, tap, address, mac in (
(guest, vm.tap, vm.guest_ip, vm.mac),
(peer, peer_vm.tap, peer_vm.guest_ip, peer_vm.mac),
(outside, "fw-up", "172.30.73.2", "02:fc:00:00:00:03"),
):
run("ip", "link", "add", tap, "type", "veth", "peer", "name", "fw-end")
run("ip", "link", "set", "fw-end", "netns", str(endpoint.process.pid))
endpoint.request("run", command=["ip", "link", "set", "fw-end", "name", "eth0"])
endpoint.request("run", command=["ip", "link", "set", "lo", "up"])
endpoint.request("run", command=["ip", "link", "set", "eth0", "address", mac])
endpoint.request("run", command=["ip", "addr", "add", f"{address}/24", "dev", "eth0"])
endpoint.request("run", command=["ip", "link", "set", "eth0", "up"])
if endpoint is outside:
run("ip", "addr", "add", "172.30.73.1/24", "dev", tap)
endpoint.request("run", command=["ip", "route", "add", str(settings.network), "via", "172.30.73.1"])
run("ip", "link", "set", tap, "up")
else:
record = vm if endpoint is guest else peer_vm
run("ip", "link", "set", "dev", tap, "alias", f"uvm:{record.id}")
run("ip", "link", "set", tap, "master", settings.bridge)
run("ip", "link", "set", tap, "type", "bridge_slave", "group_fwd_mask", "0x4000")
endpoint.request("run", command=["ip", "route", "add", "default", "via", str(settings.gateway)])
manager.apply(vm)
manager.apply(peer_vm)
for tap in (vm.tap, peer_vm.tap):
run("ip", "link", "set", tap, "up")
assert manager.status(vm) == manager.status(peer_vm) == "enforced"
def stream(client, server, target, port, name):
assert client.request("connect", name=name, ip=target, port=port)
assert server.request("accept", listener="listener", name=name)
def transfer(sender, receiver, name, text, *, blocked=False):
assert sender.request("send", name=name, data=text)
received = receiver.request("recv", name=name, size=len(text), blocked=blocked)
assert received == (None if blocked else text), (name, text, received, blocked)
guest.request("listen", name="listener", ip=vm.guest_ip, port=8080)
peer.request("listen", name="listener", ip=peer_vm.guest_ip, port=9090)
host.request("listen", name="listener", ip=str(settings.gateway), port=9000)
outside.request("listen", name="listener", ip="172.30.73.2", port=9000)
stream(host, guest, vm.guest_ip, 8080, "host-service")
stream(peer, guest, vm.guest_ip, 8080, "peer-service")
stream(guest, host, str(settings.gateway), 9000, "host-egress")
stream(guest, outside, "172.30.73.2", 9000, "routed-egress")
stream(guest, peer, peer_vm.guest_ip, 9090, "peer-egress")
pairs = ((host, guest, "host-service"), (peer, guest, "peer-service"),
(guest, host, "host-egress"), (guest, outside, "routed-egress"),
(guest, peer, "peer-egress"))
for client, server, name in pairs:
transfer(client, server, name, "client-before")
transfer(server, client, name, "server-before")
print("PASS: host/peer inbound services and host/peer/routed guest TCP egress", flush=True)
def service_ids():
listing = run("conntrack", "--list", "--family", "ipv4", "--proto", "tcp",
"--orig-dst", vm.guest_ip, "--dport", "8080", "-o", "id")
return set(re.findall(r"\bid=(\d+)", listing))
before = service_ids()
assert len(before) == 2, before
vm.ports = ()
manager.apply(vm)
assert manager.status(vm) == "enforced"
assert service_ids() == before, "closing a port must preserve connection tracking"
for client, server, name in pairs[:2]:
transfer(client, server, name, "client-after-close", blocked=True)
transfer(server, client, name, "server-after-close", blocked=True)
for client, server, name in pairs[2:]:
transfer(client, server, name, "egress-after-close")
transfer(server, client, name, "reply-after-close")
assert not host.request("connect", name="closed-host", ip=vm.guest_ip, port=8080)
assert not peer.request("connect", name="closed-peer", ip=vm.guest_ip, port=8080)
print("PASS: existing service streams drop BOTH directions, CT IDs and guest egress survive", flush=True)
peer_vm.ports = ()
manager.apply(peer_vm)
transfer(guest, peer, "peer-egress", "peer-client-closed", blocked=True)
transfer(peer, guest, "peer-egress", "peer-server-closed", blocked=True)
transfer(guest, outside, "routed-egress", "other-table-survives")
transfer(outside, guest, "routed-egress", "other-table-reply")
guest.request("udp_bind", name="dns", ip=vm.guest_ip, port=40053)
outside.request("udp_bind", name="dns", ip="172.30.73.2", port=53)
guest.request("udp_send", name="dns", ip="172.30.73.2", port=53, data="dns-query")
query = outside.request("udp_recv", name="dns")
assert query[0] == "dns-query"
outside.request("udp_send", name="dns", ip=query[1][0], port=query[1][1], data="dns-reply")
assert guest.request("udp_recv", name="dns")[0] == "dns-reply"
guest.request("udp_bind", name="unsolicited", ip=vm.guest_ip, port=40054)
outside.request("udp_send", name="dns", ip=vm.guest_ip, port=40054, data="unsolicited")
assert guest.request("udp_recv", name="unsolicited", blocked=True) is None
for endpoint, target in ((host, vm.guest_ip), (peer, vm.guest_ip),
(guest, str(settings.gateway)), (guest, "172.30.73.2")):
assert endpoint.request("ping", ip=target), (endpoint, target)
print("PASS: independent peer policy, outbound UDP/DNS and ICMP; unsolicited UDP blocked", flush=True)
# Spoofed source MACs must not reach FDB learning, including the local
# link-local path which can skip bridge prerouting entirely.
spoof = bytes.fromhex("02aabbccddee")
arp = struct.pack("!HHBBH", 1, 0x0800, 6, 4, 1) + spoof + socket.inet_aton(vm.guest_ip)
arp += b"\0" * 6 + socket.inet_aton(str(settings.gateway))
for destination in (b"\xff" * 6, bytes.fromhex("0180c200000e"), bytes.fromhex("0180c2000003")):
guest.request("raw", frame=(destination + spoof + b"\x08\x06" + arp).hex())
time.sleep(0.1)
fdb = json.loads(run("bridge", "-j", "fdb", "show", "br", settings.bridge))
assert not any(entry.get("mac") == "02:aa:bb:cc:dd:ee" for entry in fdb)
run("ip", "-6", "addr", "add", "fd42:73::1/64", "dev", settings.bridge, "nodad")
guest.request("run", command=["ip", "-6", "addr", "add", "fd42:73::2/64", "dev", "eth0", "nodad"])
host_mac = json.loads(run("ip", "-j", "link", "show", "dev", settings.bridge))[0]["address"]
run("ip", "-6", "neigh", "replace", "fd42:73::2", "lladdr", vm.mac, "nud", "permanent", "dev", settings.bridge)
guest.request("run", command=["ip", "-6", "neigh", "replace", "fd42:73::1", "lladdr", host_mac,
"nud", "permanent", "dev", "eth0"])
guest.request("listen", name="v6-listener", ip="fd42:73::2", port=8080)
host.request("listen", name="v6-listener", ip="fd42:73::1", port=9000)
assert not host.request("connect", name="v6-inbound", ip="fd42:73::2", port=8080)
assert not guest.request("connect", name="v6-outbound", ip="fd42:73::1", port=9000)
print("PASS: pre-learning MAC identity and IPv6 blocking in both directions", flush=True)
manager.apply(vm, quarantine=True)
assert manager.status(vm) == "quarantined"
transfer(guest, host, "host-egress", "quarantined-out", blocked=True)
transfer(host, guest, "host-egress", "quarantined-in", blocked=True)
try:
manager.remove(vm)
except UvmError:
pass
else:
raise AssertionError("removal must refuse a live port")
run("ip", "neigh", "flush", "dev", settings.bridge)
assert not host.request("ping", ip=vm.guest_ip), "quarantine must also stop ARP"
run("ip", "link", "del", vm.tap)
manager.cleanup_connections(vm)
manager.remove(vm)
assert manager.status(vm) == "missing"
assert manager.status(peer_vm) == "enforced"
print("PASS: quarantine, offline CT cleanup and sibling-safe removal", flush=True)
@unittest.skipUnless(os.environ.get("UVM_FIREWALL_INTEGRATION") == "1",
"opt in with UVM_FIREWALL_INTEGRATION=1; needs isolated network namespaces")
class PrivatePortNamespaceTests(unittest.TestCase):
def test_private_port_packet_paths_and_live_close(self):
if sys.platform != "linux" or shutil.which("unshare") is None:
self.skipTest("Linux and unshare are required")
prefix = ("unshare", "--net") if os.geteuid() == 0 else (
"unshare", "--user", "--map-root-user", "--net",
)
probe = subprocess.run((*prefix, "true"), text=True, capture_output=True, timeout=5)
if probe.returncode:
self.skipTest(f"isolated namespace permission unavailable: {probe.stderr.strip()}")
missing = [name for name in ("ip", "bridge", "nft", "conntrack", "sysctl") if shutil.which(name) is None]
if missing:
self.skipTest(f"integration tools unavailable: {', '.join(missing)}")
with open("/proc/self/ns/net", "rb") as parent:
process = subprocess.Popen(
(*prefix, sys.executable, "-B", str(Path(__file__).resolve()), "--sandbox", str(parent.fileno())),
pass_fds=(parent.fileno(),), stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, start_new_session=True,
)
try:
stdout, stderr = process.communicate(timeout=120)
except BaseException:
os.killpg(process.pid, signal.SIGKILL)
process.communicate(timeout=5)
raise
self.assertEqual(process.returncode, 0, f"namespace scenario failed:\n{stdout}\n{stderr}")
print(stdout, end="")
if __name__ == "__main__":
if len(sys.argv) == 3 and sys.argv[1] == "--sandbox":
_sandbox(int(sys.argv[2]))
elif len(sys.argv) == 3 and sys.argv[1] == "--agent":
_agent(int(sys.argv[2]))
else:
unittest.main()
tests/test_cli.py
+11
View File
@@ -44,6 +44,8 @@ class CliSshTests(unittest.TestCase):
key=None,
user="root",
insecure_host_key=False,
port=22,
remote_command=[],
)
with patch("uvm.cli.os.execvp") as execvp:
@@ -61,6 +63,8 @@ class CliSshTests(unittest.TestCase):
key=None,
user="root",
insecure_host_key=True,
port=22,
remote_command=[],
)
with patch("uvm.cli.os.execvp") as execvp:
@@ -77,6 +81,8 @@ class CliSshTests(unittest.TestCase):
key=None,
user=None,
insecure_host_key=False,
port=22,
remote_command=[],
)
with patch("uvm.cli.os.execvp") as execvp:
@@ -105,6 +111,11 @@ class CliCreateTests(unittest.TestCase):
host_ip=None,
username="admin",
password="custom-secret",
name=None,
ports=[],
no_default_ports=False,
password_stdin=False,
json=False,
)
output = io.StringIO()
tests/test_cli_commands.py
+886
View File
@@ -0,0 +1,886 @@
from __future__ import annotations
import io
import json
import sys
import tempfile
import unittest
from contextlib import redirect_stderr, redirect_stdout
from dataclasses import replace
from ipaddress import IPv4Address
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock, create_autospec, patch
from tests.test_cli import make_vm
from uvm.cli import build_parser, main
from uvm.config import Settings
from uvm.domain import PortRule, VmRecord
from uvm.errors import UvmError
from uvm.lifecycle import LifecycleService, ListedVm, OperationResult
from uvm.system import CommandRunner
class TtyInput(io.StringIO):
def isatty(self) -> bool:
return True
class CliTestCase(unittest.TestCase):
def setUp(self) -> None:
self.vm = replace(
make_vm(), name="web", ports=(PortRule(22), PortRule(80, "HTTP")),
status="running", disk_ready=True,
)
self.lifecycle = create_autospec(LifecycleService, instance=True, spec_set=True)
self.listed = ListedVm(vm=self.vm, observed_status="running", policy_status="enforced")
self.lifecycle.list_vms.return_value = [self.listed]
self.lifecycle.inspect.return_value = self.listed
self.lifecycle.find_for_ssh.return_value = self.vm
self.lifecycle.resolve_targets.return_value = [self.vm]
self.lifecycle.set_ports.return_value = self.vm
self.lifecycle.run_many.side_effect = lambda action, ids, **_kwargs: [
OperationResult(id=vm_id, action=action, outcome="succeeded", error=None)
for vm_id in ids
]
self.lifecycle.create.side_effect = lambda spec: replace(
self.vm, cpu=spec.cpu, ram_mib=spec.ram_mib, username=spec.username,
password=spec.password, name=spec.name, ports=spec.ports,
)
self.application = SimpleNamespace(
settings=Settings(), lifecycle=self.lifecycle,
state_store=Mock(spec_set=("operation_lock",)),
)
self.application.state_store.operation_lock.side_effect = AssertionError(
"CLI must not take an operation lock"
)
def invoke(self, arguments, *, stdin=None):
stdout, stderr = io.StringIO(), io.StringIO()
with (
patch("uvm.cli.build_application", return_value=self.application) as build,
patch("uvm.cli.sys.stdin", stdin if stdin is not None else io.StringIO()),
redirect_stdout(stdout), redirect_stderr(stderr),
):
try:
result = main(arguments)
except SystemExit as error:
result = error.code
self.build_call = build.call_args
return result, stdout.getvalue(), stderr.getvalue()
class CliParsingTests(CliTestCase):
def test_guest_ip_alias_keeps_the_original_destination(self) -> None:
for option in ("--host-ip", "--guest-ip"):
with self.subTest(option=option):
args = build_parser(self.application).parse_args(["create", option, "10.42.0.8"])
self.assertEqual(args.host_ip, "10.42.0.8")
self.assertEqual(args.password, "root")
self.assertFalse(args.password_stdin)
def test_bulk_selectors_are_exclusive_and_required(self) -> None:
for command in (("start",), ("stop",), ("restart",), ("delete",), ("destroy",), ("ports", "sync")):
with self.subTest(command=command):
parser = build_parser(self.application)
named = parser.parse_args([*command, "web", "10.42.0.3", "--yes", "--json"])
self.assertEqual(named.vms, ["web", "10.42.0.3"])
self.assertFalse(named.all_vms)
all_vms = parser.parse_args([*command, "--all", "--dry-run"])
self.assertTrue(all_vms.all_vms)
self.assertEqual(all_vms.vms, [])
for selector in ([], ["web", "--all"], ["--all", "web"]):
code, _, _ = self.invoke([*command, *selector])
self.assertEqual(code, 2)
self.lifecycle.run_many.assert_not_called()
def test_ports_subcommand_selector_semantics(self) -> None:
parser = build_parser(self.application)
close = parser.parse_args(["ports", "close", "web", "--all"])
self.assertEqual(close.vm, "web")
self.assertTrue(close.all_ports)
listing = parser.parse_args(["ports", "list", "--all"])
self.assertTrue(listing.all_vms)
self.assertIsNone(listing.vm)
for arguments in (
["ports"], ["ports", "list"], ["ports", "list", "web", "--all"],
["ports", "open", "web"], ["ports", "open", "web", "--all"],
["ports", "close", "--all"], ["ports", "close", "web"],
["ports", "close", "web", "--ports", "80", "--all"],
["ports", "set", "web"], ["ports", "set", "web", "--all"],
["ports", "set", "web", "--ports", "80", "--none"],
):
with self.subTest(arguments=arguments):
self.assertEqual(self.invoke(arguments)[0], 2)
self.lifecycle.set_ports.assert_not_called()
def test_mutually_exclusive_output_and_password_options(self) -> None:
for arguments in (
["list", "--json", "-q"],
["create", "--password", "root", "--password-stdin"],
):
with self.subTest(arguments=arguments):
self.assertEqual(self.invoke(arguments)[0], 2)
self.lifecycle.create.assert_not_called()
def test_no_bare_identifier_guessing_or_public_port_flags(self) -> None:
for arguments in (
["web"], ["vm-test"], ["10.42.0.2"],
["create", "--public-port", "8080"],
["create", "--ports", "127.0.0.1:8080:80"],
["create", "--ports", "8080:80"],
):
with self.subTest(arguments=arguments):
self.assertEqual(self.invoke(arguments)[0], 2)
self.lifecycle.create.assert_not_called()
def test_explicit_invalid_ports_fail_before_any_lifecycle_calls(self) -> None:
self.vm.ports = ()
for command in (["create"], ["ports", "open", "web"],
["ports", "close", "web"], ["ports", "set", "web"]):
for invalid in ("", "80/http", "443/HTTPS", "22/TCP", "80:UDP", "80:",
"80:HTTP:TCP", "22,", "80:HTTP,65536:TCP"):
with self.subTest(command=command, invalid=invalid):
code, stdout, _ = self.invoke([
*command, "--ports", "8080:TCP", "--ports", invalid, "--json",
])
self.assertEqual(code, 2)
self.assertEqual(stdout, "")
self.assertEqual(self.lifecycle.mock_calls, [])
def test_port_help_documents_only_colon_syntax(self) -> None:
for command in (["create"], ["ports", "open"], ["ports", "close"], ["ports", "set"]):
with self.subTest(command=command):
code, stdout, stderr = self.invoke([*command, "--help"])
self.assertEqual(code, 0, stderr)
self.assertIn("PORT[:TYPE]", stdout)
self.assertIn("80:HTTP", stdout)
self.assertNotIn("80/HTTP", stdout)
def test_timeout_help_describes_process_signals_not_graceful_guest_shutdown(self) -> None:
for command in ("stop", "restart", "delete", "destroy"):
with self.subTest(command=command):
code, stdout, stderr = self.invoke([command, "--help"])
self.assertEqual(code, 0, stderr)
self.assertIn("SIGTERM", stdout)
self.assertIn("SIGKILL", stdout)
self.assertIn("does not coordinate guest shutdown", " ".join(stdout.split()))
self.assertNotIn("graceful", stdout.lower())
def test_server_and_ssh_ports_have_independent_defaults(self) -> None:
parser = build_parser(self.application)
self.assertEqual(parser.parse_args(["--serve"]).port, 8000)
self.assertEqual(parser.parse_args(["ssh", "web"]).port, 22)
self.assertEqual(parser.parse_args(["--port", "8123", "ssh", "web"]).port, 22)
for value in ("0", "65536", "tcp", "1.5"):
for command in (["--serve", "--port", value], ["ssh", "web", "--port", value]):
with self.subTest(command=command):
self.assertEqual(self.invoke(command)[0], 2)
def test_server_mode_is_preserved_and_cannot_run_a_command(self) -> None:
with patch("uvm.cli._run_server", return_value=0) as server:
code, _, _ = self.invoke(["--serve", "--host", "127.0.0.1", "--port", "8123"])
self.assertEqual(code, 0)
self.assertEqual(server.call_args.args[1].port, 8123)
self.assertEqual(self.invoke(["--serve", "list"])[0], 2)
self.assertEqual(self.invoke([])[0], 2)
class CliCreateCommandTests(CliTestCase):
def test_create_combines_default_ports_and_stdin_credentials(self) -> None:
code, stdout, stderr = self.invoke([
"create", "--name", "web", "--cpu", "0.5", "--ram", "1G",
"--guest-ip", "10.42.0.8", "--username", "admin", "--password-stdin",
"--ports", "80:http,443:https", "--ports", "8080:tcp", "--json",
], stdin=io.StringIO(" stdin-secret \r\n"))
self.assertEqual(code, 0, stderr)
spec = self.lifecycle.create.call_args.args[0]
self.assertEqual(spec.name, "web")
self.assertEqual(spec.cpu, 0.5)
self.assertEqual(spec.ram_mib, 1024)
self.assertEqual(spec.guest_ip, IPv4Address("10.42.0.8"))
self.assertEqual(spec.username, "admin")
self.assertEqual(spec.password, " stdin-secret ")
self.assertEqual(spec.ports, (
PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS"), PortRule(8080),
))
self.assertEqual(json.loads(stdout)["ports"], [rule.to_dict() for rule in spec.ports])
self.assertNotIn("stdin-secret", stdout + stderr)
self.assertNotIn("password", json.loads(stdout))
self.assertIn("fractional", stderr)
self.lifecycle.run_many.assert_not_called()
def test_create_defaults_to_ssh_but_can_create_an_explicit_empty_policy(self) -> None:
for options, expected in (
([], (PortRule(22),)),
(["--ports", "22,80"], (PortRule(22), PortRule(80))),
(["--no-default-ports"], ()),
(["--no-default-ports", "--ports", "443:https"], (PortRule(443, "HTTPS"),)),
):
with self.subTest(options=options):
code, stdout, stderr = self.invoke(["create", *options, "--json"])
self.assertEqual(code, 0, stderr)
self.assertEqual(self.lifecycle.create.call_args.args[0].ports, expected)
self.assertNotIn("WARNING", stdout)
self.assertIn("default password", stderr)
self.assertEqual(json.loads(stdout)["policy_status"], "enforced")
def test_create_does_not_claim_enforcement_for_a_legacy_returned_record(self) -> None:
self.lifecycle.create.side_effect = None
self.lifecycle.create.return_value = replace(self.vm, ports=None)
code, stdout, stderr = self.invoke(["create", "--password", "stored-secret", "--json"])
self.assertEqual(code, 0, stderr)
self.assertEqual(json.loads(stdout)["policy_status"], "unmanaged")
self.assertIsNone(json.loads(stdout)["ports"])
self.assertNotIn("stored-secret", stdout + stderr)
def test_invalid_create_inputs_do_not_call_the_service(self) -> None:
for options in (
["--name", "../bad"], ["--cpu", "nan"], ["--ram", "12"],
["--guest-ip", "invalid"], ["--username", "Invalid"],
["--ports", "80,70000"], ["--ports", "80:http", "--ports", "53:udp"],
["--ports", ""],
):
with self.subTest(options=options):
code, stdout, _ = self.invoke(["create", *options, "--json"])
self.assertEqual(code, 2)
self.assertEqual(stdout, "")
self.lifecycle.create.assert_not_called()
def test_invalid_stdin_password_is_not_echoed(self) -> None:
for password in ("", "\n", "secret\tcontrol\n", "long-secret-" * 20):
with self.subTest(password=password):
code, stdout, stderr = self.invoke(
["create", "--password-stdin", "--json"], stdin=io.StringIO(password)
)
self.assertEqual(code, 2)
self.assertEqual(stdout, "")
self.assertNotIn("secret", stderr)
self.lifecycle.create.assert_not_called()
def test_json_uses_a_whitelist_not_persisted_vm_serialization(self) -> None:
with patch.object(VmRecord, "to_dict", side_effect=AssertionError("unsafe serialization")):
for arguments in (
["create", "--password", "stored-secret", "--json"],
["list", "--json"], ["inspect", "web", "--json"],
["ports", "list", "--all", "--json"],
["ports", "open", "web", "--ports", "443", "--json"],
["delete", "--all", "--dry-run", "--json"],
["ports", "set", "web", "--none", "--dry-run", "--json"],
):
with self.subTest(arguments=arguments):
code, stdout, stderr = self.invoke(arguments)
self.assertEqual(code, 0, stderr)
json.loads(stdout)
self.assertNotIn("password", stdout)
self.assertNotIn("stored-secret", stdout + stderr)
def test_application_and_subprocess_progress_use_stderr_for_json(self) -> None:
self.invoke(["list", "--json"])
runner = CommandRunner(emit=self.build_call.kwargs["emit"])
create_vm = self.lifecycle.create.side_effect
def create(spec):
runner.run(["cli-output-probe"])
return create_vm(spec)
def subprocess_run(_command, **kwargs):
self.assertIs(kwargs["stdout"], sys.stderr)
print("child process progress", file=kwargs["stdout"])
return SimpleNamespace(returncode=0, stdout="", stderr="")
self.lifecycle.create.side_effect = create
with patch("uvm.system.subprocess.run", side_effect=subprocess_run) as run:
code, stdout, stderr = self.invoke(["create", "--password", "stored-secret", "--json"])
self.assertEqual(code, 0, stderr)
self.assertEqual(json.loads(stdout)["id"], self.vm.id)
self.assertIn("+ cli-output-probe", stderr)
self.assertIn("child process progress", stderr)
self.assertNotIn("stored-secret", stdout + stderr)
run.assert_called_once()
class CliListingTests(CliTestCase):
def test_status_filters_use_observed_status_and_quiet_prints_only_ids(self) -> None:
self.lifecycle.list_vms.return_value = [
ListedVm(vm=self.vm, observed_status="dead", policy_status="missing"),
ListedVm(
vm=replace(self.vm, id="vm-two", status="stopped"),
observed_status="stopped", policy_status="inactive",
),
ListedVm(
vm=replace(self.vm, id="vm-three"), observed_status="running", policy_status="enforced",
),
]
code, stdout, stderr = self.invoke(["list", "--status", "dead", "--status", "stopped", "-q"])
self.assertEqual(code, 0, stderr)
self.assertEqual(stdout, "vm-test\nvm-two\n")
code, stdout, _ = self.invoke(["list", "--status", "dead", "--json"])
self.assertEqual(code, 0)
self.assertEqual(json.loads(stdout)[0]["observed_status"], "dead")
self.assertEqual(json.loads(stdout)[0]["status"], "running")
self.assertEqual(json.loads(stdout)[0]["policy_status"], "missing")
def test_empty_list_formats(self) -> None:
self.lifecycle.list_vms.return_value = []
for options, expected in (([], "No VMs.\n"), (["-q"], ""), (["--json"], "[]\n")):
with self.subTest(options=options):
code, stdout, _ = self.invoke(["list", *options])
self.assertEqual(code, 0)
self.assertEqual(stdout, expected)
def test_inspect_delegates_identifier_and_exposes_observed_state(self) -> None:
self.lifecycle.inspect.return_value = ListedVm(
vm=self.vm, observed_status="dead", policy_status="drift",
)
for options in ([], ["--json"]):
code, stdout, stderr = self.invoke(["inspect", "10.42.0.2", *options])
self.assertEqual(code, 0, stderr)
self.assertIn("dead", stdout)
self.assertIn("drift", stdout)
self.assertNotIn("stored-secret", stdout)
self.lifecycle.inspect.assert_called_with("10.42.0.2")
class CliBulkTests(CliTestCase):
def test_named_lifecycle_actions_use_resolved_ids_and_configured_timeouts(self) -> None:
self.application.settings = replace(Settings(), terminate_timeout_s=7.5)
self.lifecycle.resolve_targets.return_value = [self.vm, replace(self.vm, id="vm-two")]
for action in ("start", "stop", "restart", "delete", "destroy"):
with self.subTest(action=action):
code, _, stderr = self.invoke([action, "web", "10.42.0.3", "-y"])
self.assertEqual(code, 0, stderr)
self.lifecycle.resolve_targets.assert_called_with(["web", "10.42.0.3"], all_vms=False)
self.lifecycle.run_many.assert_called_with(
"delete" if action == "destroy" else action,
("vm-test", "vm-two"), timeout=None if action == "start" else 7.5,
)
def test_explicit_timeouts_are_validated_before_target_resolution(self) -> None:
for action in ("stop", "restart", "delete", "destroy"):
for timeout in ("-1", "nan", "inf", "bad"):
with self.subTest(action=action, timeout=timeout):
code, _, _ = self.invoke([action, "web", "--timeout", timeout, "-y"])
self.assertEqual(code, 2)
self.lifecycle.resolve_targets.assert_not_called()
self.lifecycle.run_many.assert_not_called()
self.assertEqual(self.invoke(["stop", "web", "--timeout", "2.25"])[0], 0)
self.lifecycle.run_many.assert_called_once_with("stop", ("vm-test",), timeout=2.25)
def test_every_delete_and_all_selector_requires_tty_or_yes(self) -> None:
for arguments in (
["delete", "web"], ["destroy", "web"],
["start", "--all"], ["stop", "--all"], ["restart", "--all"],
["delete", "--all"], ["ports", "sync", "--all"],
):
with self.subTest(arguments=arguments):
code, stdout, stderr = self.invoke([*arguments, "--json"], stdin=io.StringIO("yes\n"))
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("--yes", stderr)
self.lifecycle.run_many.assert_not_called()
self.assertEqual(self.invoke(["start", "--all", "-y"])[0], 0)
def test_snapshot_is_taken_before_confirmation_and_never_expanded(self) -> None:
events = []
selected = [self.vm]
def resolve(identifiers, *, all_vms):
self.assertEqual(identifiers, [])
self.assertTrue(all_vms)
events.append("resolve")
return selected
def approve():
events.append("prompt")
selected.append(replace(self.vm, id="vm-new"))
self.vm.id = "vm-renamed-during-prompt"
return "yes\n"
self.lifecycle.resolve_targets.side_effect = resolve
stdin = Mock(isatty=Mock(return_value=True), readline=Mock(side_effect=approve))
code, stdout, stderr = self.invoke(["delete", "--all", "--json"], stdin=stdin)
self.assertEqual(code, 0, stderr)
self.assertEqual(events, ["resolve", "prompt"])
self.lifecycle.resolve_targets.assert_called_once()
self.lifecycle.run_many.assert_called_once_with("delete", ("vm-test",), timeout=5.0)
self.assertIn("vm-test", stderr)
self.assertEqual([result["id"] for result in json.loads(stdout)], ["vm-test"])
def test_declining_confirmation_is_a_noop_and_keeps_stdout_json(self) -> None:
for answer in ("n\n", "\n", "", "not yes\n"):
with self.subTest(answer=answer):
code, stdout, stderr = self.invoke(["delete", "web", "--json"], stdin=TtyInput(answer))
self.assertEqual(code, 0)
self.assertEqual(json.loads(stdout), [])
self.assertIn("Cancelled", stderr)
self.lifecycle.run_many.assert_not_called()
def test_dry_run_neither_mutates_nor_prompts(self) -> None:
for action in ("start", "stop", "restart", "delete", "destroy"):
with self.subTest(action=action):
stdin = Mock(isatty=Mock(side_effect=AssertionError("must not prompt")))
code, stdout, stderr = self.invoke([action, "--all", "--dry-run", "--json"], stdin=stdin)
self.assertEqual(code, 0, stderr)
preview = json.loads(stdout)
self.assertTrue(preview["dry_run"])
self.assertEqual(preview["action"], "delete" if action == "destroy" else action)
self.assertEqual([vm["id"] for vm in preview["targets"]], [self.vm.id])
self.lifecycle.run_many.assert_not_called()
def test_empty_target_set_is_success_without_confirmation_or_mutation(self) -> None:
self.lifecycle.resolve_targets.return_value = []
for command in (["delete"], ["start"], ["ports", "sync"]):
with self.subTest(command=command):
code, stdout, stderr = self.invoke([*command, "--all", "--json"])
self.assertEqual(code, 0, stderr)
self.assertEqual(json.loads(stdout), [])
self.lifecycle.run_many.assert_not_called()
def test_failed_target_resolution_aborts_the_entire_bulk_request(self) -> None:
self.lifecycle.resolve_targets.side_effect = UvmError("VM not found: missing")
code, stdout, stderr = self.invoke(["stop", "web", "missing", "--json"])
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("missing", stderr)
self.lifecycle.run_many.assert_not_called()
def test_result_outcomes_determine_exit_code_and_preserve_each_result(self) -> None:
for outcomes, expected in (
(("succeeded", "skipped"), 0),
(("succeeded", "failed"), 1),
(("failed", "interrupted", "skipped"), 130),
):
with self.subTest(outcomes=outcomes):
self.lifecycle.run_many.side_effect = None
self.lifecycle.run_many.return_value = [
OperationResult(id=f"vm-{index}", action="stop", outcome=outcome,
error="failure detail" if outcome == "failed" else None)
for index, outcome in enumerate(outcomes)
]
code, stdout, _ = self.invoke(["stop", "web", "--json"])
self.assertEqual(code, expected)
data = json.loads(stdout)
self.assertEqual([result["outcome"] for result in data], list(outcomes))
self.assertTrue(all(set(result) == {"id", "action", "outcome", "error"} for result in data))
def test_keyboard_interrupt_returns_130_without_traceback(self) -> None:
self.lifecycle.run_many.side_effect = KeyboardInterrupt
code, stdout, stderr = self.invoke(["stop", "web"])
self.assertEqual(code, 130)
self.assertEqual(stdout, "")
self.assertIn("interrupted", stderr)
class CliPortsTests(CliTestCase):
def test_list_distinguishes_unmanaged_empty_and_enforced_policies(self) -> None:
self.lifecycle.list_vms.return_value = [
self.listed,
ListedVm(
vm=replace(self.vm, id="vm-legacy", ports=None),
observed_status="running", policy_status="unmanaged",
),
ListedVm(
vm=replace(self.vm, id="vm-empty", ports=()),
observed_status="running", policy_status="enforced",
),
]
code, stdout, stderr = self.invoke(["ports", "list", "--all", "--json"])
self.assertEqual(code, 0, stderr)
data = json.loads(stdout)
self.assertEqual(data[0]["ports"], [{"port": 22, "protocol": "TCP"}, {"port": 80, "protocol": "HTTP"}])
self.assertIsNone(data[1]["ports"])
self.assertEqual(data[1]["policy_status"], "unmanaged")
self.assertEqual(data[2]["ports"], [])
code, stdout, _ = self.invoke(["ports", "list", "web"])
self.assertEqual(code, 0)
self.assertIn("PRIVATE-IP", stdout)
self.assertIn("22:TCP,80:HTTP", stdout)
self.assertNotIn("22/TCP", stdout)
self.lifecycle.inspect.assert_called_with("web")
def test_open_and_close_call_service_modes_using_resolved_id(self) -> None:
for mode, values, expected in (
("open", "443:https,8080:tcp", (PortRule(443, "HTTPS"), PortRule(8080))),
("close", "80:tcp", (PortRule(80),)),
):
with self.subTest(mode=mode):
code, _, stderr = self.invoke(["ports", mode, "web", "--ports", values, "--json"])
self.assertEqual(code, 0, stderr)
self.lifecycle.set_ports.assert_called_with(self.vm.id, expected, mode=mode)
def test_mutation_json_reports_policy_from_the_successful_returned_state(self) -> None:
self.lifecycle.inspect.return_value = replace(self.listed, policy_status="drift")
for status, policy_status in (("running", "enforced"), ("stopped", "inactive")):
self.lifecycle.set_ports.return_value = replace(self.vm, status=status, operation=None)
for options in (
["open", "web", "--ports", "443:HTTPS"],
["close", "web", "--ports", "80:TCP"],
["set", "web", "--ports", "443:HTTPS", "--yes"],
):
with self.subTest(status=status, options=options):
code, stdout, stderr = self.invoke(["ports", *options, "--json"])
self.assertEqual(code, 0, stderr)
data = json.loads(stdout)
self.assertEqual(data["status"], status)
self.assertEqual(data["policy_status"], policy_status)
self.assertIsNone(data["operation"])
def test_invalid_port_in_later_argument_aborts_before_service_calls(self) -> None:
for mode in ("open", "close", "set"):
with self.subTest(mode=mode):
code, stdout, _ = self.invoke([
"ports", mode, "web", "--ports", "80:http", "--ports", "53:udp", "--json",
])
self.assertEqual(code, 2)
self.assertEqual(stdout, "")
self.lifecycle.inspect.assert_not_called()
self.lifecycle.set_ports.assert_not_called()
def test_set_close_all_and_removing_ssh_require_confirmation(self) -> None:
for options in (
["set", "web", "--ports", "22,80"], ["set", "web", "--none"],
["close", "web", "--all"], ["close", "web", "--ports", "22"],
):
with self.subTest(options=options):
code, stdout, stderr = self.invoke(["ports", *options, "--json"])
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("--yes", stderr)
self.lifecycle.set_ports.assert_not_called()
def test_set_and_close_all_replace_policy_without_touching_other_vms(self) -> None:
for options, expected in (
(["close", "web", "--all"], ()),
(["set", "web", "--none"], ()),
(["set", "web", "--ports", "443:https"], (PortRule(443, "HTTPS"),)),
):
with self.subTest(options=options):
code, _, stderr = self.invoke(["ports", *options, "--yes"])
self.assertEqual(code, 0, stderr)
self.lifecycle.set_ports.assert_called_with(self.vm.id, expected, mode="set")
self.lifecycle.resolve_targets.assert_not_called()
self.lifecycle.run_many.assert_not_called()
def test_empty_managed_port_target_is_a_noop_without_confirmation(self) -> None:
for status, policy_status in (("running", "enforced"), ("stopped", "inactive")):
vm = replace(self.vm, ports=(), status=status, operation=None)
self.lifecycle.inspect.return_value = ListedVm(vm, status, policy_status)
for options in (
["close", "web", "--all"], ["close", "web", "--ports", "22:TCP"],
["set", "web", "--none"],
):
for output in ([], ["--json"]):
with self.subTest(status=status, options=options, output=output):
stdin = Mock(isatty=Mock(side_effect=AssertionError("must not prompt")))
code, stdout, stderr = self.invoke(["ports", *options, *output], stdin=stdin)
self.assertEqual(code, 0, stderr)
if output:
self.assertEqual(json.loads(stdout)["ports"], [])
self.assertEqual(json.loads(stdout)["policy_status"], policy_status)
else:
self.assertIn("no port changes", stdout)
self.assertEqual(stderr, "")
self.lifecycle.set_ports.assert_not_called()
self.lifecycle.run_many.assert_not_called()
def test_empty_policy_requires_service_when_unverified_or_pending(self) -> None:
for status, observed, policy, operation in (
("running", "running", "missing", None),
("running", "running", "drift", None),
("running", "running", "failed", None),
("running", "running", "pending", None),
("running", "running", "unknown", None),
("running", "running", "unmanaged", None),
("running", "running", "enforced", "ports-sync"),
("stopped", "stopped", "inactive", "ports-sync"),
("failed", "failed", "enforced", None),
("running", "dead", "enforced", None),
("stopped", "running", "inactive", None),
("stopped", "stopped", "missing", None),
):
vm = replace(self.vm, ports=(), status=status, operation=operation)
self.lifecycle.inspect.return_value = ListedVm(vm, observed, policy)
updated_status = "stopped" if status == observed == "stopped" else "running"
self.lifecycle.set_ports.return_value = replace(vm, status=updated_status, operation=None)
for options, requested, mode in (
(["close", "web", "--all"], (), "set"),
(["close", "web", "--ports", "22:TCP"], (PortRule(22),), "close"),
(["set", "web", "--none"], (), "set"),
):
with self.subTest(state=(status, observed, policy, operation), options=options):
self.lifecycle.set_ports.reset_mock()
code, stdout, stderr = self.invoke(["ports", *options, "--yes", "--json"])
self.assertEqual(code, 0, stderr)
self.lifecycle.set_ports.assert_called_once_with(vm.id, requested, mode=mode)
self.assertEqual(json.loads(stdout)["policy_status"],
"inactive" if updated_status == "stopped" else "enforced")
self.assertIsNone(json.loads(stdout)["operation"])
def test_empty_policy_reconciliation_failure_is_not_reported_as_a_noop(self) -> None:
vm = replace(self.vm, ports=(), status="failed", operation="ports-sync")
self.lifecycle.inspect.return_value = ListedVm(vm, "failed", "drift")
self.lifecycle.set_ports.side_effect = UvmError("policy reconciliation failed")
code, stdout, stderr = self.invoke(["ports", "close", "web", "--all", "--yes", "--json"])
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("policy reconciliation failed", stderr)
self.lifecycle.set_ports.assert_called_once_with(vm.id, (), mode="set")
def test_confirmation_uses_stable_id_and_decline_does_not_mutate(self) -> None:
code, stdout, stderr = self.invoke(
["ports", "close", "web", "--ports", "22", "--json"], stdin=TtyInput("n\n")
)
self.assertEqual(code, 0)
self.assertIn(self.vm.id, stderr)
self.assertEqual(json.loads(stdout), [])
self.lifecycle.set_ports.assert_not_called()
code, _, _ = self.invoke(
["ports", "close", "web", "--ports", "22"], stdin=TtyInput("YES\n")
)
self.assertEqual(code, 0)
self.lifecycle.set_ports.assert_called_once_with(self.vm.id, (PortRule(22),), mode="close")
def test_port_dry_runs_render_effective_changes_without_mutation(self) -> None:
for options, expected in (
(["open", "web", "--ports", "443:https"],
(PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS"))),
(["close", "web", "--ports", "80:tcp"], (PortRule(22),)),
(["close", "web", "--ports", "22"], (PortRule(80, "HTTP"),)),
(["close", "web", "--all"], ()),
(["set", "web", "--none"], ()),
(["set", "web", "--ports", "443:https"], (PortRule(443, "HTTPS"),)),
):
with self.subTest(options=options):
code, stdout, stderr = self.invoke(["ports", *options, "--dry-run", "--json"])
self.assertEqual(code, 0, stderr)
data = json.loads(stdout)
self.assertTrue(data["dry_run"])
self.assertEqual(data["changes"]["ports"], {
"before": [rule.to_dict() for rule in self.vm.ports],
"after": [rule.to_dict() for rule in expected],
})
self.lifecycle.set_ports.assert_not_called()
def test_open_preview_preserves_labels_of_existing_ports(self) -> None:
code, stdout, stderr = self.invoke([
"ports", "open", "web", "--ports", "22:HTTPS,80:TCP", "--ports", "443:HTTPS",
"--dry-run", "--json",
])
self.assertEqual(code, 0, stderr)
self.assertEqual(json.loads(stdout)["changes"]["ports"], {
"before": [PortRule(22).to_dict(), PortRule(80, "HTTP").to_dict()],
"after": [
PortRule(22).to_dict(), PortRule(80, "HTTP").to_dict(), PortRule(443, "HTTPS").to_dict(),
],
})
self.assertEqual(self.vm.ports, (PortRule(22), PortRule(80, "HTTP")))
self.lifecycle.set_ports.assert_not_called()
def test_unmanaged_open_and_close_are_not_silently_treated_as_empty(self) -> None:
self.vm.ports = None
for options in (
["open", "web", "--ports", "80"], ["close", "web", "--ports", "80"],
["close", "web", "--all", "--yes"],
):
for preview in ([], ["--dry-run"]):
with self.subTest(options=options, preview=preview):
code, _, stderr = self.invoke(["ports", *options, *preview])
self.assertEqual(code, 1)
self.assertIn("unmanaged", stderr)
self.lifecycle.set_ports.assert_not_called()
def test_legacy_migration_stopped_requirement_is_owned_by_service(self) -> None:
self.vm.ports = None
self.lifecycle.set_ports.side_effect = UvmError("stop the VM before adopting a port policy")
code, _, stderr = self.invoke(["ports", "set", "web", "--ports", "22", "--yes"])
self.assertEqual(code, 1)
self.assertIn("stop", stderr)
self.lifecycle.set_ports.assert_called_once_with(self.vm.id, (PortRule(22),), mode="set")
def test_setting_an_empty_legacy_policy_is_not_a_noop(self) -> None:
self.vm.ports = None
code, _, stderr = self.invoke(["ports", "set", "web", "--none"])
self.assertEqual(code, 1)
self.assertIn("--yes", stderr)
self.lifecycle.set_ports.assert_not_called()
self.lifecycle.set_ports.return_value = replace(self.vm, ports=(), status="stopped")
code, _, stderr = self.invoke(["ports", "set", "web", "--none", "--yes"])
self.assertEqual(code, 0, stderr)
self.lifecycle.set_ports.assert_called_once_with(self.vm.id, (), mode="set")
def test_sync_uses_bulk_action_and_supports_preview(self) -> None:
code, _, stderr = self.invoke(["ports", "sync", "web", "--json"])
self.assertEqual(code, 0, stderr)
self.lifecycle.resolve_targets.assert_called_with(["web"], all_vms=False)
self.lifecycle.run_many.assert_called_once_with("ports-sync", (self.vm.id,), timeout=None)
self.lifecycle.run_many.reset_mock()
code, stdout, stderr = self.invoke(["ports", "sync", "--all", "--dry-run", "--json"])
self.assertEqual(code, 0, stderr)
self.assertEqual(json.loads(stdout)["action"], "ports-sync")
self.lifecycle.run_many.assert_not_called()
self.lifecycle.sync_ports.assert_not_called()
class CliSshCommandTests(CliTestCase):
def test_options_after_vm_are_not_swallowed_by_remote_command(self) -> None:
with patch("uvm.cli.os.execvp") as execute:
code, _, stderr = self.invoke([
"ssh", "web", "--key", "/tmp/key with spaces", "--port", "2222", "--user", "admin",
"--", "printf", "%s", "--user", "--port", "literal argument",
])
self.assertEqual(code, 0, stderr)
self.lifecycle.find_for_ssh.assert_called_once_with("web")
executable, arguments = execute.call_args.args
self.assertEqual(executable, "ssh")
self.assertEqual(arguments[:5], ["ssh", "-p", "2222", "-i", "/tmp/key with spaces"])
self.assertIn("HostKeyAlias=uvm-vm-test", arguments)
self.assertEqual(arguments[-6:], [
"admin@10.42.0.2", "printf", "%s", "--user", "--port", "literal argument",
])
def test_options_before_vm_and_remote_separator_are_supported(self) -> None:
with patch("uvm.cli.os.execvp") as execute:
code, _, stderr = self.invoke([
"ssh", "--port", "2222", "--insecure-host-key", "web", "--", "uname", "-a",
])
self.assertEqual(code, 0, stderr)
arguments = execute.call_args.args[1]
self.assertIn("StrictHostKeyChecking=no", arguments)
self.assertNotIn("HostKeyAlias=uvm-vm-test", arguments)
self.assertEqual(arguments[-3:], ["admin@10.42.0.2", "uname", "-a"])
def test_remote_commands_require_literal_separator_and_unknown_options_fail(self) -> None:
with patch("uvm.cli.os.execvp") as execute:
for arguments in (
["ssh", "web", "uname", "-a"], ["ssh", "web", "--unknown"],
["ssh", "web", "--bad-option", "--", "uname"],
["ssh", "web", "--user", "-bad"],
):
with self.subTest(arguments=arguments):
self.assertEqual(self.invoke(arguments)[0], 2)
execute.assert_not_called()
def test_ssh_help_and_option_values_after_separator_belong_to_remote_command(self) -> None:
for remote in (["--help"], ["--port", "not-a-port", "--user", "Not-A-Local-User"], ["--", "-x"]):
with self.subTest(remote=remote), patch("uvm.cli.os.execvp") as execute:
code, _, stderr = self.invoke(["ssh", "web", "--port", "2222", "--", *remote])
self.assertEqual(code, 0, stderr)
arguments = execute.call_args.args[1]
self.assertEqual(arguments[:3], ["ssh", "-p", "2222"])
self.assertEqual(arguments[-len(remote):], remote)
with patch("uvm.cli.os.execvp") as execute:
self.assertEqual(self.invoke(["ssh", "web", "--help"])[0], 0)
execute.assert_not_called()
def test_missing_ssh_executable_is_reported_without_traceback(self) -> None:
with patch("uvm.cli.os.execvp", side_effect=FileNotFoundError("ssh not found")):
code, stdout, stderr = self.invoke(["ssh", "web"])
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("ssh not found", stderr)
class CliLogTests(CliTestCase):
def test_tail_handles_empty_partial_and_fewer_lines(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "safe.log"
self.lifecycle.log_path.return_value = path
for content, count, expected in (
(b"", 100, ""), (b"one\ntwo\nthree\n", 2, "two\nthree\n"),
(b"one\ntwo\nthree", 2, "two\nthree"), (b"one\n", 100, "one\n"),
(b"one\ntwo\n", 0, ""), (b"\n\n", 1, "\n"),
(b"bad\xff\n", 1, "bad\ufffd\n"),
):
with self.subTest(content=content, count=count):
path.write_bytes(content)
code, stdout, stderr = self.invoke(["logs", "web", "--tail", str(count)])
self.assertEqual(code, 0, stderr)
self.assertEqual(stdout, expected)
self.lifecycle.log_path.assert_called_with("web")
def test_default_tail_is_100_and_reads_are_bounded_near_eof(self) -> None:
content = b"not in tail\n" * 200000 + b"last line\n" * 100
stream = io.BytesIO(content)
bounded = Mock(wraps=stream)
path = Mock()
path.open.return_value.__enter__ = Mock(return_value=bounded)
path.open.return_value.__exit__ = Mock(return_value=False)
self.lifecycle.log_path.return_value = path
code, stdout, stderr = self.invoke(["logs", "web"])
self.assertEqual(code, 0, stderr)
self.assertEqual(stdout, "last line\n" * 100)
reads = [call.args[0] for call in bounded.read.call_args_list]
self.assertTrue(all(0 < size <= 65536 for size in reads))
self.assertLess(sum(reads), 131072)
path.open.assert_called_once_with("rb")
def test_multibyte_and_very_long_lines_are_streamed_correctly(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "safe.log"
content = "skip\n" + "x" * 65535 + "\u00e9" + "z" * 70000 + "\n"
path.write_text(content, encoding="utf-8")
self.lifecycle.log_path.return_value = path
code, stdout, stderr = self.invoke(["logs", "web", "--tail", "1"])
self.assertEqual(code, 0, stderr)
self.assertEqual(stdout, content.removeprefix("skip\n"))
def test_follow_uses_safe_path_without_holding_an_operation_lock(self) -> None:
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "safe.log"
path.write_text("old\n", encoding="utf-8")
self.lifecycle.log_path.return_value = path
events = []
def append_then_interrupt(_seconds):
if events:
raise KeyboardInterrupt
with path.open("a", encoding="utf-8") as stream:
stream.write("new\n")
events.append("appended")
with patch("uvm.cli.time.sleep", side_effect=append_then_interrupt):
code, stdout, stderr = self.invoke(["logs", "web", "--tail", "0", "-f"])
self.assertEqual(code, 130)
self.assertEqual(stdout, "new\n")
self.assertIn("interrupted", stderr)
self.lifecycle.log_path.assert_called_once_with("web")
self.application.state_store.operation_lock.assert_not_called()
def test_missing_or_unsafe_log_path_fails_without_opening_record_path(self) -> None:
self.lifecycle.log_path.side_effect = UvmError("unsafe VM log path")
with patch("uvm.cli.Path.open") as open_path:
code, stdout, stderr = self.invoke(["logs", "web"])
self.assertEqual(code, 1)
self.assertEqual(stdout, "")
self.assertIn("unsafe", stderr)
open_path.assert_not_called()
self.lifecycle.log_path.side_effect = None
with tempfile.TemporaryDirectory() as directory:
self.lifecycle.log_path.return_value = Path(directory) / "missing.log"
self.assertEqual(self.invoke(["logs", "web"])[0], 1)
def test_invalid_tail_is_an_argument_error(self) -> None:
for value in ("-1", "1.5", "all"):
with self.subTest(value=value):
self.assertEqual(self.invoke(["logs", "web", "--tail", value])[0], 2)
self.lifecycle.log_path.assert_not_called()
class CliDoctorTests(CliTestCase):
def test_doctor_reports_all_checks_and_fails_if_any_check_fails(self) -> None:
for healthy in (True, False):
with self.subTest(healthy=healthy):
self.lifecycle.diagnostics.return_value = [
{"name": "kvm", "ok": True, "message": "available", "password": "not-output"},
{"name": "firewall", "ok": healthy, "message": "checked"},
]
for options in ([], ["--json"]):
code, stdout, stderr = self.invoke(["doctor", *options])
self.assertEqual(code, 0 if healthy else 1, stderr)
self.assertIn("firewall", stdout)
self.assertNotIn("password", stdout)
self.assertNotIn("not-output", stdout)
if options:
self.assertEqual(len(json.loads(stdout)), 2)
tests/test_firewall.py
+926
View File
@@ -0,0 +1,926 @@
"""Unprivileged command, drift and rule-semantics tests, not packet verification."""
from __future__ import annotations
import copy
import json
import os
import re
import shutil
import subprocess
import unittest
from dataclasses import replace
from pathlib import Path
from types import SimpleNamespace
from uvm.config import Settings
from uvm.domain import PortRule, VmRecord
from uvm.errors import CommandError, UvmError, ValidationError
from uvm.firewall import FirewallManager, _ct, _normalize
from uvm.system import CommandResult
from uvm.validation import parse_ports
def vm_record(**changes):
fields = dict(id="vm-one", tap="tap-one", guest_ip="10.42.0.2",
mac="02:fc:00:00:00:01", ports=(PortRule(8080),))
fields.update(changes)
return SimpleNamespace(**fields)
class FakeRunner:
"""Model atomic nft transactions and readback without executing commands."""
def __init__(self):
self.calls = []
self.tables = {}
self.links = []
self.handle = 1
self.release = "6.8.0-test"
self.fail_command = None
self.fail_batch = False
self.before_batch = None
self.after_batch = None
self.listing = None
self.ct_delete_code = 0
self.ct_remaining = ""
def run(self, command, *, check=True, capture=False, input_text=None, timeout=None):
args = tuple(command)
self.calls.append((args, dict(check=check, capture=capture,
input_text=input_text, timeout=timeout)))
if args == self.fail_command:
raise CommandError("simulated permission failure or timeout")
stdout, code = "", 0
if args == ("uname", "-r"):
stdout = self.release
elif args == ("ip", "-j", "-d", "link", "show"):
stdout = json.dumps(self.links)
elif args[:2] == ("nft", "--json") and "list" in args:
if self.listing is not None:
stdout = self.listing
else:
if "tables" in args:
objects = [items[0] for items in self.tables.values()]
else:
if args[-1] not in self.tables:
raise CommandError("table disappeared")
objects = self.tables[args[-1]]
stdout = json.dumps({"nftables": [{"metainfo": {"version": "1.0.9"}}, *objects]})
elif args[:2] == ("nft", "--json") and args[-2:] == ("-f", "-"):
commands = json.loads(input_text)["nftables"]
if self.before_batch:
self.before_batch()
self.before_batch = None
if self.fail_batch:
raise CommandError("atomic nft transaction rejected")
candidate = copy.deepcopy(self.tables)
for command in commands:
operation, item = next(iter(command.items()))
kind, obj = next(iter(item.items()))
# Match nft 1.0.9 json_parse_set_expr(), not an echo-only fake.
if kind == "set" and obj.get("elem") == []:
raise CommandError("Invalid set elem expression")
if operation == "delete":
name = next((name for name, items in candidate.items()
if items[0]["table"].get("handle") == obj["handle"]), None)
if name is None:
raise CommandError("table handle disappeared")
del candidate[name]
else:
obj["handle"] = self.handle
self.handle += 1
if kind == "table":
if obj["name"] in candidate:
raise CommandError("exclusive create found an existing table")
candidate[obj["name"]] = [item]
else:
candidate[obj["table"]].append(item)
if "--check" not in args:
self.tables = candidate
if self.after_batch:
self.after_batch()
elif args == ("conntrack", "--version"):
stdout = "conntrack v1.4.8 (conntrack-tools)"
elif args[:2] == ("conntrack", "--delete"):
code = self.ct_delete_code
elif args[:2] == ("conntrack", "--list"):
stdout = self.ct_remaining
else:
raise AssertionError(f"unexpected command: {args}")
if check and code:
raise CommandError("simulated command failure")
return CommandResult(args=args, returncode=code, stdout=stdout)
@property
def batches(self):
return [json.loads(options["input_text"])["nftables"] for args, options in self.calls
if args[0] == "nft" and options["input_text"] is not None and "--check" not in args]
class FirewallManagerTests(unittest.TestCase):
def setUp(self):
self.runner = FakeRunner()
self.manager = FirewallManager(Settings(), self.runner)
self.vm = vm_record()
def policy(self):
return next(iter(self.runner.tables.values()))
def test_apply_is_atomic_bounded_and_contains_no_nat_or_global_mutations(self):
self.manager.apply(self.vm)
self.assertEqual(self.manager.status(self.vm), "enforced")
self.assertEqual(len(self.runner.batches), 1)
self.assertIn("create", self.runner.batches[0][0])
for args, options in self.runner.calls:
if args[0] != "nft":
self.assertEqual(args, ("ip", "-j", "-d", "link", "show"))
self.assertTrue(options["capture"])
self.assertGreater(options["timeout"], 0)
self.assertLessEqual(options["timeout"], 10)
rendered = json.dumps(self.runner.batches)
for forbidden in ("flush", "dnat", "snat", "masquerade", "redirect", "notrack"):
self.assertNotIn(f'"{forbidden}"', rendered)
for item in self.policy():
self.assertEqual(next(iter(item.values()))["family"], "bridge")
def test_update_and_remove_do_not_rebuild_or_delete_siblings(self):
self.manager.apply(self.vm)
sibling = vm_record(id="vm-two", tap="tap-two", guest_ip="10.42.0.3")
self.manager.apply(sibling)
name = next(reversed(self.runner.tables))
saved = copy.deepcopy(self.runner.tables[name])
old_handle = self.policy()[0]["table"]["handle"]
self.vm.ports = ()
self.manager.apply(self.vm)
self.assertEqual(self.runner.batches[-1][0], {
"delete": {"table": {"family": "bridge", "handle": old_handle}},
})
self.assertEqual(self.runner.tables[name], saved)
self.assertEqual(self.manager.status(self.vm), "enforced")
self.manager.remove(self.vm)
self.manager.remove(self.vm)
self.assertEqual(self.runner.tables, {name: saved})
self.assertEqual(self.manager.status(self.vm), "missing")
def test_table_identity_is_stable_but_scoped_to_base_and_vm_id(self):
self.manager.apply(self.vm)
name = next(iter(self.runner.tables))
self.vm.ports = ()
self.manager.apply(self.vm)
self.assertEqual(list(self.runner.tables), [name])
other = FirewallManager(replace(Settings(), base=Path("/var/lib/another-uvm")), self.runner)
other.apply(self.vm)
self.assertEqual(len(self.runner.tables), 2)
self.assertRegex(name, r"^uvm_fw_[0-9a-f]{24}$")
self.assertLess(len(self.policy()[0]["table"]["comment"]), 128)
def test_legacy_is_unmanaged_and_normal_apply_refuses_without_commands(self):
self.vm.ports = None
with self.assertRaises(ValidationError):
self.manager.apply(self.vm)
self.assertEqual(self.runner.calls, [])
self.assertEqual(self.manager.status(self.vm), "unmanaged")
self.vm.guest_ip, self.vm.mac = "damaged", "damaged"
self.manager.apply(self.vm, quarantine=True)
self.assertEqual(self.manager.status(self.vm), "quarantined")
def test_none_does_not_hide_an_existing_managed_table(self):
self.manager.apply(self.vm)
self.vm.ports = None
self.assertEqual(self.manager.status(self.vm), "drift")
def test_protocol_aliases_deduplicate_tcp_ports_and_allow_empty_policy(self):
self.vm.ports = tuple(PortRule(port, protocol) for port, protocol in (
(443, "HTTPS"), (8080, "HTTP"), (443, "TCP"), (22, "TCP"),
))
self.manager.apply(self.vm)
self.assertEqual(next(item["set"]["elem"] for item in self.policy() if "set" in item),
[22, 443, 8080])
self.vm.ports = ()
self.manager.apply(self.vm)
self.assertNotIn("elem", next(item["set"] for item in self.policy() if "set" in item))
def test_real_port_models_and_cli_syntax_reach_the_same_tcp_set(self):
ports = parse_ports(("1:tcp,80:http", "443:HTTPS,65535"))
vm = VmRecord(id="vm-real", cpu=1, ram_mib=128, guest_ip="10.42.0.2",
gateway="10.42.0.1", tap="tap-real", mac=self.vm.mac,
socket="", config="", log="", ports=ports,
network="10.42.0.0/24", bridge="uvm0")
self.manager.apply(vm)
self.assertEqual(self.manager.status(vm), "enforced")
self.assertEqual(next(item["set"]["elem"] for item in self.policy() if "set" in item),
[1, 80, 443, 65535])
for text in ("0", "65536", "-1", "1.5", "80:UDP", "80:HTTP:TCP", "TCP:80", "80:", ":TCP"):
with self.subTest(text=text), self.assertRaises(ValidationError):
parse_ports((text,))
def test_invalid_ports_are_rejected_before_commands(self):
for port, protocol in ((True, "TCP"), (0, "TCP"), (-1, "TCP"), (65536, "TCP"),
(80.0, "TCP"), ("80", "TCP"), (80, "tcp"), (53, "UDP"),
(80, "HTTP:TCP"), (80, None), (80, "HTTP; flush ruleset")):
with self.subTest(port=port, protocol=protocol):
self.vm.ports = (SimpleNamespace(port=port, protocol=protocol),)
with self.assertRaises(ValidationError):
self.manager.apply(self.vm)
self.assertEqual(self.runner.calls, [])
def test_invalid_identity_is_rejected_before_commands(self):
for field, value in (("tap", "tap*"), ("tap", 'x"; drop'), ("tap", "-bad"),
("tap", "a" * 16), ("tap", "uvm0"), ("id", ""),
("mac", "ff:ff:ff:ff:ff:ff"), ("mac", "00:00:00:00:00:00"),
("guest_ip", "10.42.0.1"), ("guest_ip", "10.42.0.255"),
("guest_ip", "10.43.0.2"), ("guest_ip", "127.0.0.1"),
("guest_ip", "8.8.8.8"), ("guest_ip", "::1")):
with self.subTest(field=field, value=value), self.assertRaises(ValidationError):
self.manager.apply(vm_record(**{field: value}))
self.assertEqual(self.runner.calls, [])
def test_persisted_network_can_differ_from_current_settings(self):
vm = vm_record(guest_ip="172.20.1.2", network="172.20.0.0/16", bridge="old-bridge")
self.manager.apply(vm)
self.assertEqual(self.manager.status(vm), "enforced")
def test_live_tap_must_route_lldp_through_early_filter_and_disable_stp(self):
port = {"info_slave_kind": "bridge", "info_slave_data": {"group_fwd_mask": "0x4000"}}
bridge = {"info_kind": "bridge", "info_data": {"stp_state": 0}}
self.runner.links = [{"ifname": self.vm.tap, "master": "uvm0", "linkinfo": port},
{"ifname": "uvm0", "ifindex": 7, "linkinfo": bridge}]
self.manager.apply(self.vm)
self.assertEqual(self.manager.status(self.vm), "enforced")
for stp, mask in ((1, "0x4000"), (0, 0), (0, "unreadable")):
bridge["info_data"]["stp_state"] = stp
port["info_slave_data"]["group_fwd_mask"] = mask
self.assertEqual(self.manager.status(self.vm), "drift")
for quarantine in (False, True):
with self.assertRaisesRegex(UvmError, "LLDP forwarding"):
self.manager.apply(self.vm, quarantine=quarantine)
bridge["info_data"]["stp_state"] = 0
port["info_slave_data"]["group_fwd_mask"] = 0x4008
self.runner.links[0]["master"] = 7 # iproute may use an index instead of a name.
self.manager.apply(self.vm, quarantine=True)
self.assertEqual(self.manager.status(self.vm), "quarantined")
def test_foreign_ownership_is_never_adopted_even_for_quarantine_or_remove(self):
self.manager.apply(self.vm)
self.policy()[0]["table"]["comment"] = "someone else's firewall"
saved = copy.deepcopy(self.runner.tables)
self.assertEqual(self.manager.status(self.vm), "drift")
for operation in (lambda: self.manager.apply(self.vm),
lambda: self.manager.apply(self.vm, quarantine=True),
lambda: self.manager.remove(self.vm)):
with self.assertRaisesRegex(UvmError, "foreign"):
operation()
self.assertEqual(self.runner.tables, saved)
self.assertEqual(len(self.runner.batches), 1)
def test_table_replacement_race_uses_exclusive_create_and_old_handle(self):
self.manager.apply(self.vm)
def replace_table():
self.policy()[0]["table"].update(handle=90000, comment="foreign replacement")
self.runner.before_batch = replace_table
with self.assertRaises(CommandError):
self.manager.apply(self.vm, quarantine=True)
self.assertEqual(self.policy()[0]["table"]["comment"], "foreign replacement")
def test_initial_create_race_does_not_append_to_a_foreign_table(self):
table = self.manager._policy(self.vm, quarantine=True)[0]["table"]
def create_foreign():
self.runner.tables[table["name"]] = [{"table": {**table, "handle": 1000, "comment": "foreign"}}]
self.runner.before_batch = create_foreign
with self.assertRaises(CommandError):
self.manager.apply(self.vm)
self.assertEqual(len(self.policy()), 1)
def test_atomic_failure_keeps_old_policy_and_readback_failure_is_not_success(self):
self.manager.apply(self.vm)
saved = copy.deepcopy(self.runner.tables)
self.runner.fail_batch = True
self.vm.ports = ()
with self.assertRaises(CommandError):
self.manager.apply(self.vm)
self.assertEqual(self.runner.tables, saved)
self.runner.fail_batch = False
self.runner.after_batch = lambda: self.policy().pop()
with self.assertRaisesRegex(UvmError, "readback"):
self.manager.apply(self.vm)
def test_status_does_not_trust_owner_comment_rule_count_or_saved_flags(self):
self.manager.apply(self.vm)
original = copy.deepcopy(self.runner.tables)
self.vm.firewall_applied = True
mutations = (
lambda items: items.pop(),
lambda items: next(item["set"] for item in items if "set" in item)["elem"].append(22),
lambda items: next(item["chain"] for item in items if "chain" in item).update(hook="postrouting"),
lambda items: next(item["chain"] for item in items if "chain" in item).update(prio=1),
lambda items: next(item["chain"] for item in items if "chain" in item).update(policy="drop"),
lambda items: items[0]["table"].update(flags="dormant"),
lambda items: next(item["rule"] for item in items if "rule" in item)["expr"].pop(0),
lambda items: next(item["rule"] for item in items if "rule" in item).update(expr=[{"accept": None}]),
lambda items: items.append({"flowtable": {"family": "bridge", "name": "bypass"}}),
)
for mutate in mutations:
with self.subTest(mutation=mutate):
self.runner.tables = copy.deepcopy(original)
mutate(self.policy())
self.assertEqual(self.manager.status(self.vm), "drift")
self.runner.tables = original
for item in self.policy():
if "rule" in item and item["rule"]["chain"] == "from_vm":
item["rule"]["expr"] = [{"return": None}]
break
self.assertEqual(self.manager.status(self.vm), "drift")
def test_rule_order_is_not_normalized_away(self):
self.manager.apply(self.vm)
positions = [index for index, item in enumerate(self.policy())
if "rule" in item and item["rule"]["chain"] == "from_vm"]
first, last = positions[0], positions[-1]
self.policy()[first], self.policy()[last] = self.policy()[last], self.policy()[first]
self.assertEqual(self.manager.status(self.vm), "drift")
def test_normalizes_nft_json_presentation_not_effective_policy(self):
self.manager.apply(self.vm)
for item in self.policy():
obj = next(iter(item.values()))
obj["handle"] += 100
if "set" in item:
obj["elem"].reverse()
obj["flags"] = "constant"
if "rule" not in item:
continue
exprs = obj["expr"]
protocols = {expr.get("match", {}).get("left", {}).get("payload", {}).get("protocol")
for expr in exprs}
# nft elides EtherType dependencies of ARP/IP payloads, and
# coalesces/sorts adjacent pure payload matches when listing.
has_tcp_payload = '"protocol": "tcp"' in json.dumps(exprs)
if protocols & {"arp", "ip"} and not has_tcp_payload:
exprs[:] = [expr for expr in exprs if expr.get("match", {}).get("left") != {
"payload": {"protocol": "ether", "field": "type"},
}]
for expr in exprs:
match = expr.get("match", {})
if match.get("right") == "@tcp_ports":
match["op"] = "in"
if match.get("left") == {"ct": {"key": "state"}} and isinstance(match["right"], list):
match["right"].reverse()
if "&" in match.get("left", {}):
match["left"]["&"][1] = {"|": [{"|": ["fin", "syn"]}, {"|": ["rst", "ack"]}]}
match["right"] = "syn"
exprs[:-1] = reversed(exprs[:-1])
self.assertEqual(self.manager.status(self.vm), "enforced")
def test_empty_sets_can_omit_elements_and_missing_conditions_are_drift(self):
self.vm.ports = ()
self.manager.apply(self.vm)
self.assertNotIn("elem", next(item["set"] for item in self.policy() if "set" in item))
self.assertEqual(self.manager.status(self.vm), "enforced")
next(item["set"] for item in self.policy() if "set" in item)["elem"] = []
self.assertEqual(self.manager.status(self.vm), "enforced")
for item in self.policy():
if "rule" in item:
item["rule"]["expr"] = [expr for expr in item["rule"]["expr"]
if expr.get("match", {}).get("left") != {"ct": {"key": "direction"}}]
self.assertEqual(self.manager.status(self.vm), "drift")
def test_missing_tools_permissions_timeouts_and_bad_json_are_unknown_not_missing(self):
self.runner.fail_command = ("nft", "--json", "list", "tables", "bridge")
self.assertEqual(self.manager.status(self.vm), "unknown")
with self.assertRaises(CommandError):
self.manager.apply(self.vm)
self.runner.fail_command = None
for listing in ("", "not json", "{}", '{"nftables": null}',
'{"nftables": [null]}', '{"nftables": [{"table": {}}]}'):
with self.subTest(listing=listing):
self.runner.listing = listing
self.assertEqual(self.manager.status(self.vm), "unknown")
with self.assertRaises(UvmError):
self.manager.apply(self.vm)
self.assertEqual(self.runner.batches, [])
def test_remove_and_cleanup_refuse_a_live_tap_even_when_quarantined(self):
self.manager.apply(self.vm, quarantine=True)
self.runner.links = [{"ifname": self.vm.tap, "master": "uvm0", "linkinfo": {
"info_slave_kind": "bridge", "info_slave_data": {"group_fwd_mask": 0x4000},
}}, {"ifname": "uvm0", "linkinfo": {"info_kind": "bridge", "info_data": {"stp_state": 0}}}]
for operation in (self.manager.remove, self.manager.cleanup_connections):
with self.assertRaisesRegex(UvmError, "still exists"):
operation(self.vm)
self.assertEqual(self.manager.status(self.vm), "quarantined")
self.assertFalse(any(args[0] == "conntrack" for args, _ in self.runner.calls))
def test_offline_check_errors_do_not_release_policy(self):
self.manager.apply(self.vm, quarantine=True)
self.runner.fail_command = ("ip", "-j", "-d", "link", "show")
with self.assertRaises(CommandError):
self.manager.remove(self.vm)
self.runner.fail_command = None
self.runner.links = [{}]
with self.assertRaises(UvmError):
self.manager.cleanup_connections(self.vm)
self.assertEqual(self.manager.status(self.vm), "unknown")
def test_offline_conntrack_cleanup_is_scoped_to_all_four_tuple_addresses(self):
self.runner.ct_delete_code = 1 # conntrack uses 1 for no matching entries too.
self.manager.cleanup_connections(self.vm)
for selector in ("--orig-src", "--orig-dst", "--reply-src", "--reply-dst"):
for command in ("--delete", "--list"):
self.assertIn(("conntrack", command, "--family", "ipv4", selector, self.vm.guest_ip),
[args for args, _ in self.runner.calls])
self.assertFalse(any("--flush" in args or "-F" in args for args, _ in self.runner.calls))
self.runner.ct_remaining = "tcp 6 100 ESTABLISHED ..."
with self.assertRaisesRegex(UvmError, "do not reuse"):
self.manager.cleanup_connections(self.vm)
self.runner.ct_remaining = ""
self.runner.fail_command = ("conntrack", "--list", "--family", "ipv4", "--orig-src", self.vm.guest_ip)
with self.assertRaises(CommandError):
self.manager.cleanup_connections(self.vm)
def test_availability_checks_kernel_and_full_policy_without_committing(self):
self.manager.check_available()
self.assertEqual(self.runner.tables, {})
self.assertEqual(self.runner.calls[0][0], ("conntrack", "--version"))
self.assertTrue(self.runner.calls[0][1]["check"])
self.assertEqual(self.runner.calls[0][1]["timeout"], 5)
batch = next((args, options) for args, options in self.runner.calls if "--check" in args)
self.assertIn('"ct"', batch[1]["input_text"])
self.assertIn('"hook": "prerouting"', batch[1]["input_text"])
for release in ("5.2.99", "4.19.0", "not-linux"):
self.runner.release = release
with self.assertRaisesRegex(UvmError, ">= 5.3"):
self.manager.check_available()
self.runner.release = "5.3.0"
self.runner.fail_batch = True
with self.assertRaisesRegex(UvmError, "unavailable"):
self.manager.check_available()
def test_missing_cleanup_utility_fails_preflight_before_nft(self):
self.runner.fail_command = ("conntrack", "--version")
with self.assertRaises(CommandError):
self.manager.check_available()
self.assertEqual([args for args, _ in self.runner.calls], [("conntrack", "--version")])
def test_status_rejects_changed_ct_keys_masks_verdicts_and_quarantine_hooks(self):
self.manager.apply(self.vm)
original = copy.deepcopy(self.runner.tables)
for alteration in ("tuple-key", "ct-mask", "tcp-mask", "return", "extra-set-property"):
self.runner.tables = copy.deepcopy(original)
rules = [item["rule"] for item in self.policy() if "rule" in item]
expressions = [expr for rule in rules for expr in rule["expr"]]
if alteration == "tuple-key":
match = next(expr["match"] for expr in expressions
if expr.get("match", {}).get("left") == {"ct": {"key": "ip saddr", "dir": "original"}})
match["left"] = {"ct": {"key": "saddr", "family": "ip", "dir": "original"}}
elif alteration == "ct-mask":
next(expr["match"] for expr in expressions
if expr.get("match", {}).get("left") == {"ct": {"key": "state"}})["right"] = "established"
elif alteration == "tcp-mask":
next(expr["match"] for expr in expressions
if "&" in expr.get("match", {}).get("left", {}))["left"]["&"][1] = 2
elif alteration == "return":
verdict = next(expr for expr in expressions if "return" in expr)
verdict.clear()
verdict["accept"] = None
else:
next(item["set"] for item in self.policy() if "set" in item)["timeout"] = 30
with self.subTest(alteration=alteration):
self.assertEqual(self.manager.status(self.vm), "drift")
self.runner.tables = original
self.manager.apply(self.vm, quarantine=True)
self.policy().pop()
self.assertEqual(self.manager.status(self.vm), "drift")
@unittest.skipUnless(os.path.exists("/proc/self/ns/net"), "Linux network namespace handles required")
def test_integration_harness_refuses_to_run_in_the_parent_namespace(self):
from tests.integration.test_private_ports import _sandbox
with open("/proc/self/ns/net", "rb") as parent:
with self.assertRaisesRegex(RuntimeError, "parent network namespace"):
_sandbox(parent.fileno())
class NftDumpFixtureTests(unittest.TestCase):
"""Literal upstream JSON fixtures, not output synthesized by FakeRunner.
Sources below are from nftables tag v1.0.9:
https://git.netfilter.org/nftables/tree/tests?h=v1.0.9
https://github.com/1g4-mirror/nftables/tree/v1.0.9/tests
tests/py/nft-test.py lists with numeric-protocol enabled. Files ending in
.json.output are its recorded expected nft dumps, not packet tests run here.
"""
@staticmethod
def normalize_rule(text):
return _normalize([{"rule": {"family": "bridge", "table": "fixture", "chain": "input",
"expr": json.loads(text)}}])
def test_explicit_ct_address_keys_match_upstream_wire_format(self):
# tests/py/inet/ct.t.json, "ct original ip daddr 1.2.3.4 accept".
# There is no .json.output override for this example: dump == input.
dumped = json.loads('''[
{"match": {"left": {"ct": {"dir": "original", "key": "ip daddr"}},
"op": "==", "right": "1.2.3.4"}},
{"accept": null}
]''')
self.assertEqual(_ct("ip daddr", "original"), dumped[0]["match"]["left"])
# ct_templates[NFT_CT_SRC_IP] / [NFT_CT_DST_IP] in src/ct.c; the
# v1.0.9 json_parse_ct_expr() looks up these keys and ignores family.
self.assertEqual(_ct("ip saddr", "original"), {"ct": {"key": "ip saddr", "dir": "original"}})
policy = FirewallManager(Settings(), FakeRunner())._policy(vm_record(), quarantine=False)
addresses = [expr["match"]["left"]["ct"] for item in policy if "rule" in item
for expr in item["rule"]["expr"] if "ct" in expr.get("match", {}).get("left", {})
and "addr" in expr["match"]["left"]["ct"]["key"]]
self.assertEqual({ct["key"] for ct in addresses}, {"ip saddr", "ip daddr"})
self.assertTrue(all(ct["dir"] == "original" and "family" not in ct for ct in addresses))
def test_upstream_ct_list_and_set_dumps_preserve_distinct_operators(self):
# tests/py/any/ct.t.json and ct.t.json.output, ct state list vs set.
pairs = (
('''[{"match": {"left": {"ct": {"key": "state"}}, "op": "in",
"right": ["new", "established", "related", "untracked"]}}]''',
'''[{"match": {"left": {"ct": {"key": "state"}}, "op": "in",
"right": ["established", "related", "new", "untracked"]}}]'''),
('''[{"match": {"left": {"ct": {"key": "state"}}, "op": "in",
"right": {"set": ["new", "established", "related", "untracked"]}}}]''',
'''[{"match": {"left": {"ct": {"key": "state"}}, "op": "==",
"right": {"set": ["established", "related", "new", "untracked"]}}}]'''),
('''[{"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": 8}}]''',
'''[{"match": {"left": {"ct": {"key": "state"}}, "op": "in", "right": "new"}}]'''),
)
for source, dumped in pairs:
with self.subTest(source=source):
self.assertEqual(self.normalize_rule(source), self.normalize_rule(dumped))
# Equality against the ORed mask is NOT the list's any-bit test.
self.assertNotEqual(self.normalize_rule(pairs[0][1]),
self.normalize_rule(pairs[0][1].replace('"op": "in"', '"op": "=="')))
def test_upstream_bridge_icmp_numeric_protocol_dump(self):
# tests/py/bridge/icmpX.t.json and icmpX.t.json.output.
source = '''[
{"match": {"left": {"payload": {"field": "protocol", "protocol": "ip"}},
"op": "==", "right": "icmp"}},
{"match": {"left": {"payload": {"field": "type", "protocol": "icmp"}},
"op": "==", "right": "echo-request"}}
]'''
dumped = '''[
{"match": {"left": {"payload": {"field": "protocol", "protocol": "ip"}},
"op": "==", "right": 1}},
{"match": {"left": {"payload": {"field": "type", "protocol": "icmp"}},
"op": "==", "right": "echo-request"}}
]'''
self.assertEqual(self.normalize_rule(source), self.normalize_rule(dumped))
self.assertNotEqual(self.normalize_rule(source), self.normalize_rule(dumped.replace('"right": 1', '"right": 6')))
def test_upstream_arp_payload_field_names_and_reordering(self):
# tests/py/arp/arp.t.json and arp.t.json.output:
# arp daddr ether fe:ed:00:c0:ff:ee arp saddr ip 192.168.1.1
source = '''[
{"match": {"left": {"payload": {"field": "daddr ether", "protocol": "arp"}},
"op": "==", "right": "fe:ed:00:c0:ff:ee"}},
{"match": {"left": {"payload": {"field": "saddr ip", "protocol": "arp"}},
"op": "==", "right": "192.168.1.1"}}
]'''
dumped = '''[
{"match": {"left": {"payload": {"field": "saddr ip", "protocol": "arp"}},
"op": "==", "right": "192.168.1.1"}},
{"match": {"left": {"payload": {"field": "daddr ether", "protocol": "arp"}},
"op": "==", "right": "fe:ed:00:c0:ff:ee"}}
]'''
self.assertEqual(self.normalize_rule(source), self.normalize_rule(dumped))
self.assertNotEqual(self.normalize_rule(source), self.normalize_rule(dumped.replace("saddr ip", "daddr ip")))
policy = FirewallManager(Settings(), FakeRunner())._policy(vm_record(), quarantine=False)
fields = {expr["match"]["left"]["payload"]["field"] for item in policy if "rule" in item
for expr in item["rule"]["expr"]
if expr.get("match", {}).get("left", {}).get("payload", {}).get("protocol") == "arp"}
self.assertEqual(fields, {"htype", "ptype", "hlen", "plen", "operation",
"saddr ip", "saddr ether", "daddr ip", "daddr ether"})
def test_upstream_tcp_flag_dump_does_not_hide_mask_changes(self):
# tests/py/inet/tcp.t.json.output, tcp flags & (syn|fin) == (syn|fin).
dumped = '''[{"match": {
"left": {"&": [{"payload": {"field": "flags", "protocol": "tcp"}}, {"|": ["fin", "syn"]}]},
"op": "==", "right": {"|": ["fin", "syn"]}
}}]'''
numeric = '''[{"match": {
"left": {"&": [{"payload": {"field": "flags", "protocol": "tcp"}}, 3]},
"op": "==", "right": 3
}}]'''
self.assertEqual(self.normalize_rule(dumped), self.normalize_rule(numeric))
self.assertNotEqual(self.normalize_rule(dumped), self.normalize_rule(numeric.replace('}}, 3]', '}}, 2]')))
def test_upstream_full_dump_ignores_only_handles_and_metainfo(self):
# Literal RULESET from tests/shell/testcases/json/0006obj_comment_0.
dumped = json.loads('''{"nftables": [
{"metainfo": {"version": "1.0.5", "release_name": "Lester Gooch #4", "json_schema_version": 1}},
{"table": {"family": "inet", "name": "t", "handle": 9}},
{"counter": {"family": "inet", "name": "mycounter", "table": "t", "handle": 1,
"comment": "my comment in counter", "packets": 0, "bytes": 0}}
]}''')["nftables"]
expected = [
{"table": {"family": "inet", "name": "t"}},
{"counter": {"family": "inet", "name": "mycounter", "table": "t",
"comment": "my comment in counter", "packets": 0, "bytes": 0}},
]
self.assertEqual(_normalize(dumped), _normalize(expected))
dumped[1]["table"]["flags"] = "dormant"
self.assertNotEqual(_normalize(dumped), _normalize(expected))
@unittest.skipUnless(shutil.which("nft"), "nft unavailable for noncommitting JSON parser checks")
class NftJsonParserTests(unittest.TestCase):
"""Exercise the actual JSON parser, NOT semantic/kernel/packet validation.
A deliberately invalid final command must be the first command rejected.
This proves all earlier commands parsed even when cache initialization lacks
CAP_NET_ADMIN. --check and the invalid command both prevent any commit.
"""
@staticmethod
def parse(commands):
commands = [*commands, {"add": {"uvm_schema_sentinel": None}}]
result = subprocess.run(
("nft", "--check", "--json", "-f", "-"),
input=json.dumps({"nftables": commands}), text=True, capture_output=True,
timeout=5, env={**os.environ, "LC_ALL": "C"},
)
indices = [int(index) for index in re.findall(r"Parsing command array at index (\d+) failed", result.stderr)]
return result, indices
@classmethod
def setUpClass(cls):
result, indices = cls.parse([])
if indices != [0]:
raise unittest.SkipTest(f"nft parser diagnostics unavailable before netlink: {result.stderr.strip()}")
def test_all_generated_transactions_reach_only_the_final_invalid_command(self):
runner = FakeRunner()
manager = FirewallManager(Settings(), runner)
vm = vm_record(ports=(PortRule(1), PortRule(443, "HTTPS"), PortRule(65535)))
manager.apply(vm)
vm.ports = ()
manager.apply(vm)
vm.ports = None
manager.apply(vm, quarantine=True)
manager.remove(vm)
manager.check_available()
transactions = [json.loads(options["input_text"])["nftables"]
for args, options in runner.calls if args[0] == "nft" and options["input_text"]]
for index, commands in enumerate(transactions):
with self.subTest(transaction=index):
result, rejected = self.parse(commands)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(rejected, [len(commands)], result.stderr)
def test_parser_negative_controls_fail_before_the_sentinel(self):
bad_commands = (
{"add": {"set": {"family": "bridge", "table": "probe", "name": "ports",
"type": "inet_service", "elem": []}}},
{"delete": {"table": {"family": "bridge", "handle": "not-an-integer"}}},
{"add": {"chain": {"family": "bridge", "table": "probe", "name": "input",
"type": "filter", "hook": "not-a-hook", "prio": 0}}},
{"add": {"rule": {"family": "bridge", "table": "probe", "chain": "input", "expr": [
{"match": {"left": {"ct": {"key": "state", "dir": "original"}}, "op": "in", "right": "new"}},
]}}},
{"add": {"rule": {"family": "bridge", "table": "probe", "chain": "input", "expr": [
{"match": {"left": {"payload": {"protocol": "arp", "field": "saddr"}}, "op": "==", "right": "10.42.0.2"}},
]}}},
)
for command in bad_commands:
with self.subTest(command=command):
result, rejected = self.parse([command])
self.assertNotEqual(result.returncode, 0)
self.assertEqual(rejected, [0], result.stderr)
class RulesetModel:
"""Evaluate the emitted pure match/jump/return/drop subset, not a kernel.
Conntrack metadata is supplied by each test. This catches policy/direction
mistakes but deliberately does not pretend to test native bridge tracking.
"""
def __init__(self, tables):
self.tables = tables
def permits(self, packet, *hooks):
for hook in hooks:
for objects in self.tables:
chains = {item["chain"]["name"]: item["chain"] for item in objects if "chain" in item}
rules = {}
sets = {item["set"]["name"]: item["set"].get("elem", []) for item in objects if "set" in item}
for item in objects:
if "rule" in item:
rules.setdefault(item["rule"]["chain"], []).append(item["rule"]["expr"])
def value(expr):
if isinstance(expr, str) and expr.startswith("@"):
return sets[expr[1:]]
if not isinstance(expr, dict):
return expr
if "meta" in expr:
return packet.get(expr["meta"]["key"])
if "payload" in expr:
payload = expr["payload"]
return packet.get((payload["protocol"], payload["field"]))
if "ct" in expr:
ct = expr["ct"]
return packet.get(("ct", ct["key"], ct.get("dir")))
if "set" in expr:
return expr["set"]
if "&" in expr:
left, right = map(value, expr["&"])
return None if left is None else left & right
raise AssertionError(f"unmodeled expression {expr}")
def evaluate(name):
for expressions in rules.get(name, []):
for expr in expressions:
if "match" in expr:
match = expr["match"]
left, right = value(match["left"]), value(match["right"])
if left is None:
break
same = left in right if isinstance(right, list) else left == right
if not (not same if match["op"] == "!=" else same):
break
elif "jump" in expr:
verdict = evaluate(expr["jump"]["target"])
if verdict != "return":
return verdict
elif "return" in expr:
return "return"
elif "drop" in expr:
return "drop"
elif "accept" in expr:
return "accept"
else:
raise AssertionError(f"unmodeled statement {expr}")
return "return"
for name, chain in chains.items():
if chain.get("hook") == hook and evaluate(name) == "drop":
return False
return True
def packet(src, dst, *, protocol="tcp", reply=False, state="established", dport=8080, flags=16):
original_src, original_dst = (dst, src) if reply else (src, dst)
return {
"iifname": src.tap, "oifname": dst.tap,
("ether", "type"): "ip", ("ether", "saddr"): src.mac, ("ether", "daddr"): dst.mac,
("ip", "saddr"): src.guest_ip, ("ip", "daddr"): dst.guest_ip, ("ip", "protocol"): protocol,
("tcp", "flags"): flags, ("ct", "direction", None): "reply" if reply else "original",
("ct", "ip saddr", "original"): original_src.guest_ip,
("ct", "ip daddr", "original"): original_dst.guest_ip,
("ct", "proto-dst", "original"): dport, ("ct", "protocol", None): protocol,
("ct", "state", None): state, ("icmp", "type"): "echo-reply" if reply else "echo-request",
}
class FirewallPolicyTests(unittest.TestCase):
def setUp(self):
self.manager = FirewallManager(Settings(), FakeRunner())
self.guest = vm_record()
self.peer = vm_record(id="vm-two", tap="tap-two", guest_ip="10.42.0.3",
mac="02:fc:00:00:00:02", ports=())
self.host = vm_record(tap="", guest_ip="10.42.0.1", mac="02:00:00:00:00:01")
def model(self, *, quarantine=False, peer=True, reverse=False):
tables = [self.manager._policy(self.guest, quarantine=quarantine)]
if peer:
tables.append(self.manager._policy(self.peer, quarantine=False))
return RulesetModel(list(reversed(tables)) if reverse else tables)
def test_all_hooks_and_return_verdicts_are_present_with_early_identity(self):
policy = self.manager._policy(self.guest, quarantine=False)
chains = [item["chain"] for item in policy if "chain" in item and "hook" in item["chain"]]
self.assertEqual({chain["hook"] for chain in chains}, {"prerouting", "input", "forward", "output"})
self.assertLess(next(chain["prio"] for chain in chains if chain["hook"] == "prerouting"), -200)
verdicts = [expr for item in policy if "rule" in item for expr in item["rule"]["expr"]]
self.assertIn({"return": None}, verdicts)
self.assertNotIn({"accept": None}, verdicts)
def test_host_and_peer_service_connections_close_in_both_directions(self):
for client, incoming, outgoing in ((self.host, ("output",), ("prerouting", "input")),
(self.peer, ("prerouting", "forward"), ("prerouting", "forward"))):
for reverse in (False, True):
with self.subTest(client=client.guest_ip, reverse=reverse):
self.guest.ports = (SimpleNamespace(port=8080, protocol="TCP"),)
traffic = packet(client, self.guest)
response = packet(self.guest, client, reply=True)
self.assertTrue(self.model(reverse=reverse).permits(traffic, *incoming))
self.assertTrue(self.model(reverse=reverse).permits(response, *outgoing))
self.guest.ports = ()
self.assertFalse(self.model(reverse=reverse).permits(traffic, *incoming))
self.assertFalse(self.model(reverse=reverse).permits(response, *outgoing))
def test_closed_policy_preserves_guest_egress_tcp_udp_and_icmp_replies(self):
self.guest.ports = ()
for protocol in ("tcp", "udp", "icmp"):
with self.subTest(protocol=protocol):
request = packet(self.guest, self.host, protocol=protocol, state="new", flags=2)
response = packet(self.host, self.guest, protocol=protocol, reply=True)
self.assertTrue(self.model().permits(request, "prerouting", "input"))
self.assertTrue(self.model().permits(response, "output"))
response[("ct", "direction", None)] = "original"
self.assertFalse(self.model().permits(response, "output"))
def test_peer_egress_never_skips_the_destination_policy(self):
for reverse in (False, True):
traffic = packet(self.guest, self.peer, state="new", flags=2)
self.assertFalse(self.model(reverse=reverse).permits(traffic, "prerouting", "forward"))
self.peer.ports = (SimpleNamespace(port=8080, protocol="TCP"),)
self.assertTrue(self.model(reverse=reverse).permits(traffic, "prerouting", "forward"))
self.peer.ports = ()
def test_new_tcp_requires_syn_for_inbound_and_egress_tracking_loss(self):
for flags in (0, 16, 18, 24, 1, 4, 3):
with self.subTest(flags=flags):
ingress = packet(self.host, self.guest, state="new", flags=flags)
lost_server_flow = packet(self.guest, self.host, state="new", flags=flags)
self.assertFalse(self.model().permits(ingress, "output"))
self.assertFalse(self.model().permits(lost_server_flow, "prerouting", "input"))
for flags in (2, 2 | 64 | 128):
self.assertTrue(self.model().permits(packet(self.host, self.guest, state="new", flags=flags), "output"))
for state in ("invalid", "untracked", "related"):
self.assertFalse(self.model().permits(packet(self.host, self.guest, state=state), "output"))
def test_current_original_destination_port_and_address_are_required(self):
for key, value in ((("ct", "proto-dst", "original"), 22),
(("ct", "ip daddr", "original"), "10.42.0.99"),
(("ct", "protocol", None), "udp")):
for reply in (False, True):
traffic = packet(self.guest, self.host, reply=True) if reply else packet(self.host, self.guest)
traffic[key] = value
self.assertFalse(self.model().permits(traffic, *(('prerouting', 'input') if reply else ('output',))))
def test_inbound_udp_and_non_tcp_transports_are_not_opened(self):
for protocol in ("udp", "sctp", "gre"):
for state in ("new", "established", "related"):
self.assertFalse(self.model().permits(packet(self.host, self.guest, protocol=protocol, state=state), "output"))
def test_required_icmp_is_authorized_by_parent_tuple_not_blanket_related(self):
self.guest.ports = ()
self.assertTrue(self.model().permits(packet(self.host, self.guest, protocol="icmp", state="new"), "output"))
for protocol in ("tcp", "udp", "icmp"):
traffic = packet(self.host, self.guest, protocol=protocol, state="related", reply=True)
traffic[("ip", "protocol")] = "icmp"
for icmp in ("destination-unreachable", "time-exceeded", "parameter-problem"):
traffic[("icmp", "type")] = icmp
self.assertTrue(self.model().permits(traffic, "output"))
traffic[("icmp", "type")] = "redirect"
self.assertFalse(self.model().permits(traffic, "output"))
service_error = packet(self.host, self.guest, state="related")
service_error.update({("ip", "protocol"): "icmp", ("icmp", "type"): "destination-unreachable"})
self.assertFalse(self.model().permits(service_error, "output"))
def test_identity_is_enforced_before_learning_and_ipv6_and_vlan_are_dropped(self):
for key, value in ((("ether", "saddr"), self.peer.mac),
(("ip", "saddr"), self.peer.guest_ip),
(("ether", "type"), "ip6"), (("ether", "type"), "vlan"),
(("ether", "type"), "8021ad")):
traffic = packet(self.guest, self.host)
traffic[key] = value
self.assertFalse(self.model().permits(traffic, "prerouting"))
for ethertype in ("ip6", "vlan", "8021ad"):
traffic = packet(self.host, self.guest)
traffic[("ether", "type")] = ethertype
self.assertFalse(self.model().permits(traffic, "output"))
traffic = packet(self.host, self.guest)
traffic[("ip", "daddr")] = self.peer.guest_ip
self.assertFalse(self.model().permits(traffic, "output"))
def test_arp_identity_and_address_probes_are_narrow(self):
traffic = packet(self.guest, self.host)
traffic.update({("ether", "type"): "arp", ("arp", "htype"): 1,
("arp", "ptype"): "ip", ("arp", "hlen"): 6, ("arp", "plen"): 4,
("arp", "operation"): "request", ("arp", "saddr ether"): self.guest.mac,
("arp", "saddr ip"): self.guest.guest_ip, ("arp", "daddr ip"): self.host.guest_ip})
self.assertTrue(self.model().permits(traffic, "prerouting", "input"))
for key, bad in ((("arp", "saddr ip"), self.peer.guest_ip),
(("arp", "saddr ether"), self.peer.mac), (("arp", "hlen"), 5),
(("arp", "operation"), "rreply")):
self.assertFalse(self.model().permits({**traffic, key: bad}, "prerouting"))
self.assertFalse(self.model().permits({**traffic, key: bad}, "input"))
traffic[("arp", "saddr ip")] = "0.0.0.0"
self.assertFalse(self.model().permits(traffic, "prerouting"))
traffic[("arp", "daddr ip")] = self.guest.guest_ip
self.assertTrue(self.model().permits(traffic, "prerouting"))
def test_quarantine_drops_every_path_but_unrelated_traffic_is_untouched(self):
for protocol in ("tcp", "udp", "icmp"):
for endpoint, hook in ((self.host, "input"), (self.peer, "forward")):
self.assertFalse(self.model(quarantine=True).permits(packet(self.guest, endpoint, protocol=protocol), "prerouting", hook))
self.assertFalse(self.model(quarantine=True).permits(packet(endpoint, self.guest, protocol=protocol), "output" if hook == "input" else "forward"))
other = vm_record(tap="unmanaged", guest_ip="10.42.0.99")
self.assertTrue(self.model(quarantine=True, peer=False).permits(packet(other, self.host), "prerouting", "input"))
if __name__ == "__main__":
unittest.main()
tests/test_lifecycle.py
+570 -65
View File
@@ -2,109 +2,325 @@ from __future__ import annotations
import tempfile
import unittest
from collections.abc import Callable, Iterable, Iterator
from contextlib import contextmanager
from dataclasses import replace
from ipaddress import IPv4Address
from pathlib import Path
from unittest.mock import Mock, patch
from unittest.mock import patch
from uvm.config import Settings
from uvm.domain import VmSpec
from uvm.errors import FirecrackerError, UvmError
from uvm.domain import State, VmRecord, VmSpec
from uvm.errors import ConflictError, FirecrackerError, UvmError, ValidationError
from uvm.firecracker.process import ProcessInfo
from uvm.images import ImageStore
from uvm.lifecycle import LifecycleService
from uvm.network import NetworkManager
from uvm.state import StateStore
class TrackedStateStore(StateStore):
def __init__(self, settings: Settings) -> None:
super().__init__(settings)
self.operation_active = False
self.transaction_active = False
self.operation_locks = 0
@contextmanager
def operation_lock(self) -> Iterator[None]:
if self.operation_active or self.transaction_active:
raise AssertionError("nested or inverted operation lock")
self.operation_locks += 1
with super().operation_lock():
self.operation_active = True
try:
yield
finally:
self.operation_active = False
@contextmanager
def transaction(self) -> Iterator[State]:
if self.transaction_active:
raise AssertionError("nested state transaction")
self.transaction_active = True
try:
with super().transaction() as state:
yield state
finally:
self.transaction_active = False
class FakeHost:
"""Record boundary calls and inject one-shot failures before their effects."""
def __init__(self, store: TrackedStateStore) -> None:
self.store = store
self.events: list[tuple[str, str | None]] = []
self.records: dict[tuple[str, str], VmRecord] = {}
self.failures: dict[tuple[str, str | None], BaseException] = {}
def call(self, action: str, vm_id: str | None = None, *, readonly: bool = False) -> None:
if self.store.transaction_active:
raise AssertionError(f"state transaction held during {action}")
if not readonly and not self.store.operation_active:
raise AssertionError(f"operation lock missing during {action}")
self.events.append((action, vm_id))
if vm_id is not None:
vm = self.store.load().vms.get(vm_id)
if vm is not None:
self.records[action, vm_id] = vm
error = self.failures.pop((action, vm_id), None)
if error is None:
error = self.failures.pop((action, None), None)
if error is not None:
raise error
def actions(self, vm_id: str) -> list[str]:
return [action for action, target in self.events if target == vm_id]
class FakeFirewall:
def __init__(self, host: FakeHost, network: FakeNetwork) -> None:
self.host = host
self.network = network
self.tables: dict[str, tuple[VmRecord, bool]] = {}
def check_available(self) -> None:
self.host.call("firewall.check_available", readonly=True)
def apply(self, vm: VmRecord, *, quarantine: bool = False) -> None:
self.host.call("firewall.quarantine" if quarantine else "firewall.apply", vm.id)
if vm.ports is None and not quarantine:
raise ValidationError("legacy VM has no managed port policy")
self.tables[vm.id] = (replace(vm), quarantine)
self.host.call("firewall.readback", vm.id)
def status(self, vm: VmRecord) -> str:
self.host.call("firewall.status", vm.id, readonly=True)
if vm.id not in self.tables:
return "unmanaged" if vm.ports is None else "missing"
installed, quarantine = self.tables[vm.id]
if installed.tap != vm.tap:
return "drift"
if quarantine:
return "quarantined"
if vm.ports is None:
return "drift"
same_identity = installed.guest_ip == vm.guest_ip and installed.mac == vm.mac
same_ports = {rule.port for rule in installed.ports} == {rule.port for rule in vm.ports}
return "enforced" if same_identity and same_ports else "drift"
def cleanup_connections(self, vm: VmRecord) -> None:
self.host.call("firewall.cleanup_connections", vm.id)
if vm.tap in self.network.taps:
raise UvmError("cannot clean connections while TAP exists")
def remove(self, vm: VmRecord) -> None:
self.host.call("firewall.remove", vm.id)
if vm.tap in self.network.taps:
raise UvmError("cannot remove guard while TAP exists")
self.tables.pop(vm.id, None)
class FakeNetwork:
def __init__(self) -> None:
def __init__(self, settings: Settings, host: FakeHost) -> None:
self._settings = settings
self.host = host
self.taps: dict[str, str] = {}
self.active_taps: set[str] = set()
self.created_taps: list[str] = []
self.deleted_taps: list[str] = []
self.firewall = FakeFirewall(host, self)
def allocate_ip(self, _vms, requested: IPv4Address | None) -> IPv4Address:
return requested or IPv4Address("10.42.0.2")
def allocate_ip(self, vms: Iterable[VmRecord], requested: IPv4Address | None) -> IPv4Address:
return NetworkManager.allocate_ip(self, vms, requested) # type: ignore[arg-type]
@staticmethod
def mac_for(index: int) -> str:
return f"02:fc:00:00:00:{index:02x}"
@staticmethod
def tap_name(vm_id: str) -> str:
return f"uvm-{vm_id[-8:]}"[:15]
mac_for = staticmethod(NetworkManager.mac_for)
tap_name = staticmethod(NetworkManager.tap_name)
def ensure_bridge(self) -> None:
return None
self.host.call("network.ensure_bridge")
def create_tap(self, name: str) -> None:
self.created_taps.append(name)
def prepare_tap(self, vm: VmRecord) -> None:
self.host.call("network.prepare_tap", vm.id)
if vm.tap in self.taps:
raise ConflictError(f"TAP device already exists: {vm.tap}")
self.taps[vm.tap] = vm.id
self.created_taps.append(vm.tap)
vm.tap_owned = True
def delete_tap(self, name: str) -> None:
self.deleted_taps.append(name)
def has_tap(self, vm: VmRecord) -> bool:
self.host.call("network.has_tap", vm.id, readonly=True)
owner = self.taps.get(vm.tap)
if owner is not None and owner != vm.id:
raise ConflictError(f"refusing to modify unowned TAP {vm.tap}")
return owner is not None
def activate_tap(self, vm: VmRecord) -> None:
self.host.call("network.activate_tap", vm.id)
if not self.has_tap(vm):
raise UvmError(f"TAP device missing: {vm.tap}")
self.active_taps.add(vm.tap)
def deactivate_tap(self, vm: VmRecord) -> None:
self.host.call("network.deactivate_tap", vm.id)
if self.has_tap(vm):
self.active_taps.discard(vm.tap)
def delete_vm_tap(self, vm: VmRecord) -> None:
self.host.call("network.delete_vm_tap", vm.id)
if self.has_tap(vm):
del self.taps[vm.tap]
self.active_taps.discard(vm.tap)
self.deleted_taps.append(vm.tap)
vm.tap_owned = False
class FakeProcess:
def __init__(self) -> None:
self.started = False
self.terminated: list[int | None] = []
self.alive = True
def __init__(self, host: FakeHost) -> None:
self.host = host
self.boot_id = "test-host-boot"
self.next_pid = 12345
self.running: dict[str, ProcessInfo] = {}
self.unknown: set[str] = set()
self.started: list[ProcessInfo] = []
self.terminated: list[tuple[str, int | None, float | None]] = []
def start(self, _vm) -> ProcessInfo:
self.started = True
return ProcessInfo(pid=12345, start_time="42")
def host_boot_id(self) -> str:
return self.boot_id
def is_alive(self, _vm) -> bool:
return self.alive
def observe(self, vm: VmRecord) -> str:
self.host.call("process.observe", vm.id, readonly=True)
if vm.pid is not None and vm.pid <= 0:
return "unknown"
if vm.boot_id and vm.boot_id != self.boot_id:
return "gone"
if vm.id in self.unknown:
return "unknown"
if vm.pid is None:
if (vm.spawn_pending or vm.operation == "spawning"
or vm.process_start_time is not None or Path(vm.socket).exists()):
return "unknown"
return "gone"
info = self.running.get(vm.id)
if info is None or info.pid != vm.pid:
return "gone"
if vm.process_start_time is not None and info.start_time != vm.process_start_time:
return "gone"
if vm.process_start_time is None or vm.boot_id is None:
return "unknown"
return "alive"
def terminate(self, vm) -> None:
self.terminated.append(vm.pid)
self.alive = False
def adopt_legacy_identity(self, vm: VmRecord) -> bool:
self.host.call("process.adopt_legacy_identity", vm.id)
info = self.running.get(vm.id)
if (vm.boot_id is None and info is not None and vm.id not in self.unknown
and (vm.pid, vm.process_start_time) == (info.pid, info.start_time)):
vm.boot_id = info.boot_id
return True
return False
def start(self, vm: VmRecord, *, on_spawn: Callable[[ProcessInfo], None] | None = None) -> ProcessInfo:
self.host.call("process.start", vm.id)
if self.observe(vm) != "gone":
raise FirecrackerError("refusing to replace an existing or ambiguous process")
vm.operation = "spawning"
vm.spawn_pending = True
vm.boot_id = self.host_boot_id()
# A failure here models an interrupt inside Popen, before identity is known.
self.host.call("process.spawn", vm.id)
info = ProcessInfo(self.next_pid, str(self.next_pid * 10), self.boot_id)
self.next_pid += 1
self.running[vm.id] = info
self.started.append(info)
vm.pid, vm.process_start_time = info.pid, info.start_time
if on_spawn is not None:
on_spawn(info)
else:
vm.spawn_pending = False
Path(vm.socket).touch()
with Path(vm.log).open("ab") as log:
log.write(f"boot {info.pid}\n".encode("ascii"))
self.host.call("process.ready", vm.id)
return info
def terminate(self, vm: VmRecord, *, timeout: float | None = None) -> None:
self.terminated.append((vm.id, vm.pid, timeout))
self.host.call("process.terminate", vm.id)
observation = self.observe(vm)
if observation == "unknown":
raise FirecrackerError("process identity is ambiguous")
if observation == "alive":
del self.running[vm.id]
Path(vm.socket).unlink(missing_ok=True)
self.host.call("process.gone", vm.id)
class FakeClient:
def __init__(self, should_fail: bool = False) -> None:
self.should_fail = should_fail
self.config = None
def __init__(self, host: FakeHost, vm_id: str) -> None:
self.host = host
self.vm_id = vm_id
self.config: dict | None = None
def configure_and_start(self, config) -> None:
def configure_and_start(self, config: dict) -> None:
self.host.call("client.configure", self.vm_id)
self.config = config
if self.should_fail:
raise FirecrackerError("simulated API failure")
class LifecycleTests(unittest.TestCase):
class LifecycleTestCase(unittest.TestCase):
def setUp(self) -> None:
self._temporary_directory = tempfile.TemporaryDirectory()
base = Path(self._temporary_directory.name) / "uvm"
temporary = tempfile.TemporaryDirectory()
self.addCleanup(temporary.cleanup)
base = Path(temporary.name) / "uvm"
self.settings = replace(Settings(), base=base, allow_unverified_downloads=True)
self.settings.bin_dir.mkdir(parents=True)
self.settings.firecracker_binary.touch()
self.settings.images_dir.mkdir(parents=True)
self.settings.kernel_image.write_bytes(b"kernel")
self.settings.rootfs_image.write_bytes(b"rootfs")
self.store = StateStore(self.settings)
self.store = TrackedStateStore(self.settings)
self.host = FakeHost(self.store)
self.images = ImageStore(self.settings)
self.images.provision_credentials = Mock() # type: ignore[method-assign]
self.network = FakeNetwork()
self.process = FakeProcess()
self.client = FakeClient()
self.provision = self.enterContext(patch.object(self.images, "provision_credentials"))
self.copy_disk = self.enterContext(patch.object(
self.images, "create_vm_disk", wraps=self.images.create_vm_disk,
))
self.network = FakeNetwork(self.settings, self.host)
self.process = FakeProcess(self.host)
self.clients: dict[str, FakeClient] = {}
self.service = LifecycleService(
self.settings,
self.store,
self.images,
self.network, # type: ignore[arg-type]
self.process, # type: ignore[arg-type]
client_factory=lambda _socket, _timeout: self.client, # type: ignore[arg-type]
client_factory=self.make_client, # type: ignore[arg-type]
)
self._root_patch = patch("uvm.lifecycle.require_root")
self._kvm_patch = patch("uvm.lifecycle.check_kvm")
self._root_patch.start()
self._kvm_patch.start()
self.require_root = self.enterContext(patch("uvm.lifecycle.require_root"))
self.check_kvm = self.enterContext(patch("uvm.lifecycle.check_kvm"))
for target in ("uvm.system.CommandRunner.run", "subprocess.run", "subprocess.Popen", "os.kill"):
self.enterContext(patch(target, side_effect=AssertionError(f"unexpected host operation: {target}")))
def tearDown(self) -> None:
self._kvm_patch.stop()
self._root_patch.stop()
self._temporary_directory.cleanup()
def make_client(self, socket: Path, _timeout: float) -> FakeClient:
client = FakeClient(self.host, socket.parent.name)
self.clients[client.vm_id] = client
return client
def create_vm(self, **kwargs) -> VmRecord:
return self.service.create(VmSpec(cpu=1, ram_mib=512, **kwargs))
def persist(self, vm: VmRecord, **changes) -> VmRecord:
with self.store.transaction() as state:
updated = replace(state.vms[vm.id], **changes)
state.vms[vm.id] = updated
return updated
def assert_call_order(self, vm: VmRecord, *expected: str) -> None:
actual = [action for action in self.host.actions(vm.id) if action in expected]
self.assertEqual(actual, list(expected))
class LifecycleTests(LifecycleTestCase):
def test_create_uses_a_private_disk_and_persists_running_state(self) -> None:
vm = self.service.create(VmSpec(cpu=1, ram_mib=512))
@@ -115,8 +331,14 @@ class LifecycleTests(unittest.TestCase):
self.assertEqual(vm.password, "root")
self.assertEqual(self.store.load().vms[vm.id].pid, 12345)
self.assertEqual(self.network.created_taps, [vm.tap])
self.assertEqual(self.client.config["drives"][0]["path_on_host"], vm.disk)
self.images.provision_credentials.assert_called_once_with(
self.assertEqual(self.clients[vm.id].config["drives"][0]["path_on_host"], vm.disk)
self.assertTrue(vm.runtime_owned)
self.assertTrue(vm.tap_owned)
self.assertTrue(vm.disk_ready)
self.assertFalse(vm.spawn_pending)
self.assertEqual(self.process.observe(vm), "alive")
self.assertEqual(self.network.firewall.status(vm), "enforced")
self.provision.assert_called_once_with(
Path(vm.disk),
"root",
"root",
@@ -130,27 +352,30 @@ class LifecycleTests(unittest.TestCase):
persisted = self.store.load().vms[vm.id]
self.assertEqual(persisted.username, "admin")
self.assertEqual(persisted.password, "secret-value")
self.images.provision_credentials.assert_called_once_with(
self.provision.assert_called_once_with(
Path(vm.disk),
"admin",
"secret-value",
)
def test_create_rolls_back_when_firecracker_configuration_fails(self) -> None:
self.client.should_fail = True
self.host.failures["client.configure", None] = FirecrackerError("simulated API failure")
with self.assertRaises(FirecrackerError):
with self.assertRaisesRegex(UvmError, r"failed to create vm-.*: simulated API failure") as raised:
self.service.create(VmSpec(cpu=1, ram_mib=512))
self.assertIsInstance(raised.exception.__cause__, FirecrackerError)
self.assertEqual(self.store.load().vms, {})
self.assertEqual(len(self.network.deleted_taps), 1)
self.assertEqual(self.process.terminated, [12345])
self.assertEqual([pid for _, pid, _ in self.process.terminated], [12345])
self.assertEqual(self.network.firewall.tables, {})
self.assertEqual(self.process.running, {})
self.assertEqual(list(self.settings.vms_dir.iterdir()), [])
def test_create_rolls_back_before_network_setup_when_provisioning_fails(self) -> None:
self.images.provision_credentials.side_effect = UvmError("invalid guest image")
self.provision.side_effect = UvmError("invalid guest image")
with self.assertRaises(UvmError):
with self.assertRaisesRegex(UvmError, r"failed to create vm-.*: invalid guest image"):
self.service.create(VmSpec(cpu=1, ram_mib=512))
self.assertEqual(self.store.load().vms, {})
@@ -159,10 +384,7 @@ class LifecycleTests(unittest.TestCase):
self.assertEqual(list(self.settings.vms_dir.iterdir()), [])
def test_create_rolls_back_when_interrupted(self) -> None:
def interrupting_create_tap(_name: str) -> None:
raise KeyboardInterrupt
self.network.create_tap = interrupting_create_tap
self.host.failures["network.prepare_tap", None] = KeyboardInterrupt()
with self.assertRaises(KeyboardInterrupt):
self.service.create(VmSpec(cpu=1, ram_mib=512))
@@ -183,3 +405,286 @@ class LifecycleTests(unittest.TestCase):
self.assertEqual(destroyed.id, vm.id)
self.assertFalse(self.settings.vm_dir(vm.id).exists())
self.assertEqual(self.store.load().vms, {})
def test_stop_edit_disk_start_preserves_guest_data_credentials_and_identity(self) -> None:
vm = self.create_vm(name="database", username="admin", password="original-secret")
original = replace(vm)
disk = Path(vm.disk)
inode = disk.stat().st_ino
stopped = self.service.stop(vm.name, timeout=0.25)
self.assertEqual(self.process.terminated[-1], (vm.id, vm.pid, 0.25))
self.assertEqual(stopped.status, "stopped")
self.assertNotIn(vm.tap, self.network.taps)
self.assertNotIn(vm.id, self.network.firewall.tables)
disk.write_bytes(b"guest-edited-data\npassword-hash\noriginal-ssh-host-keys\n")
edited = disk.read_bytes()
started = self.service.start(vm.guest_ip)
for field in ("id", "name", "guest_ip", "mac", "tap", "disk", "username", "password", "ports", "created_at"):
self.assertEqual(getattr(started, field), getattr(original, field), field)
self.assertNotEqual(started.pid, original.pid)
self.assertNotEqual(started.process_start_time, original.process_start_time)
self.assertEqual(started.boot_id, self.process.host_boot_id())
self.assertEqual(disk.stat().st_ino, inode)
self.assertEqual(disk.read_bytes(), edited)
self.assertEqual(self.settings.rootfs_image.read_bytes(), b"rootfs")
self.provision.assert_called_once_with(disk, "admin", "original-secret")
self.copy_disk.assert_called_once()
self.assertEqual(self.store.get(vm.id).to_dict(), started.to_dict())
def test_start_uses_private_disk_when_rootfs_template_has_been_removed(self) -> None:
vm = self.create_vm()
self.service.stop(vm.id)
self.settings.rootfs_image.unlink()
with patch.object(self.images, "installed_assets", side_effect=AssertionError("start needs only the kernel")):
started = self.service.start(vm.id)
self.assertEqual(started.status, "running")
self.assertEqual(Path(started.disk).read_bytes(), b"rootfs")
self.assertEqual(self.provision.call_count, 1)
self.assertEqual(self.copy_disk.call_count, 1)
def test_boot_persists_intent_and_identity_with_policy_before_tap_and_vmm(self) -> None:
vm = self.create_vm()
self.assert_call_order(
vm, "network.prepare_tap", "firewall.apply", "firewall.readback",
"network.activate_tap", "process.start", "process.spawn", "process.ready",
"client.configure", "firewall.status",
)
policy_record = self.host.records["firewall.apply", vm.id]
self.assertTrue(policy_record.disk_ready)
self.assertTrue(policy_record.tap_owned)
pending = self.host.records["process.start", vm.id]
self.assertTrue(pending.spawn_pending)
self.assertEqual(pending.operation, "spawning")
self.assertIsNone(pending.pid)
spawned = self.host.records["process.ready", vm.id]
self.assertEqual((spawned.pid, spawned.process_start_time, spawned.boot_id),
(vm.pid, vm.process_start_time, vm.boot_id))
self.assertFalse(spawned.spawn_pending)
self.assertEqual(spawned.operation, "configuring")
self.assertEqual(self.host.records["client.configure", vm.id].pid, vm.pid)
def test_failed_initial_policy_never_activates_tap_or_starts_vmm(self) -> None:
self.host.failures["firewall.apply", None] = UvmError("policy unavailable")
with self.assertRaisesRegex(UvmError, "policy unavailable"):
self.create_vm()
self.assertFalse(any(action in {"network.activate_tap", "process.start", "client.configure"}
for action, _ in self.host.events))
self.assertEqual(self.store.load().vms, {})
self.assertEqual(self.network.taps, {})
self.assertEqual(self.network.firewall.tables, {})
def test_unavailable_firewall_fails_preflight_before_reserving_or_copying(self) -> None:
self.host.failures["firewall.check_available", None] = UvmError("bridge filtering unavailable")
with self.assertRaisesRegex(UvmError, "bridge filtering unavailable"):
self.create_vm()
self.assertEqual(self.store.load().vms, {})
self.assertEqual(self.host.events, [("firewall.check_available", None)])
self.assertEqual(list(self.settings.vms_dir.iterdir()), [])
self.provision.assert_not_called()
self.copy_disk.assert_not_called()
def test_create_cleans_up_if_policy_readback_is_not_enforced_after_boot(self) -> None:
with patch.object(self.network.firewall, "status", return_value="drift"):
with self.assertRaisesRegex(UvmError, "enforcement could not be confirmed after boot"):
self.create_vm()
self.assertEqual(self.store.load().vms, {})
self.assertEqual(self.process.running, {})
self.assertEqual(self.network.taps, {})
self.assertEqual(self.network.firewall.tables, {})
self.assertEqual(list(self.settings.vms_dir.iterdir()), [])
def test_start_rejects_missing_or_unprovisioned_disk_without_network_mutation(self) -> None:
for problem in ("missing", "unprovisioned"):
with self.subTest(problem=problem):
vm = self.create_vm()
self.service.stop(vm.id)
if problem == "missing":
Path(vm.disk).unlink()
else:
self.persist(vm, disk_ready=False)
before = self.settings.state_path.read_bytes()
self.host.events.clear()
with self.assertRaisesRegex(UvmError, "disk"):
self.service.start(vm.id)
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertFalse(set(self.host.actions(vm.id)) - {"process.observe", "process.adopt_legacy_identity"})
def test_stop_quarantines_and_confirms_exit_before_releasing_network(self) -> None:
vm = self.create_vm()
self.host.events.clear()
self.service.stop(vm.id, timeout=0)
self.assert_call_order(
vm, "firewall.quarantine", "network.deactivate_tap", "process.terminate",
"process.gone", "network.delete_vm_tap", "firewall.cleanup_connections", "firewall.remove",
)
pending = self.host.records["process.terminate", vm.id]
self.assertEqual((pending.status, pending.operation), ("stopping", "stop"))
self.assertEqual(self.process.terminated[-1], (vm.id, vm.pid, 0))
def test_failed_process_cleanup_keeps_disk_tap_guard_identity_and_reservation(self) -> None:
for action in ("stop", "destroy", "restart"):
with self.subTest(action=action):
vm = self.create_vm(name=f"failed-{action}")
disk = Path(vm.disk)
self.host.events.clear()
self.host.failures["process.terminate", vm.id] = FirecrackerError("SIGKILL exit unconfirmed")
with self.assertRaisesRegex(UvmError, "process exit unconfirmed.*SIGKILL"):
getattr(self.service, action)(vm.id, timeout=0.1)
retained = self.store.get(vm.id)
self.assertEqual(retained.status, "failed")
self.assertEqual((retained.pid, retained.process_start_time), (vm.pid, vm.process_start_time))
self.assertEqual(retained.guest_ip, vm.guest_ip)
self.assertEqual(disk.read_bytes(), b"rootfs")
self.assertTrue(retained.runtime_owned)
self.assertTrue(retained.tap_owned)
self.assertEqual(self.network.taps[vm.tap], vm.id)
self.assertNotIn(vm.tap, self.network.active_taps)
self.assertEqual(self.network.firewall.status(retained), "quarantined")
self.assertEqual(self.process.observe(retained), "alive")
self.assertFalse(set(self.host.actions(vm.id)) & {
"network.delete_vm_tap", "firewall.cleanup_connections", "firewall.remove", "process.start",
})
def test_create_retains_resources_when_rollback_cannot_confirm_exit(self) -> None:
self.host.failures["client.configure", None] = FirecrackerError("API failed")
self.host.failures["process.terminate", None] = FirecrackerError("still alive")
with self.assertRaisesRegex(UvmError, "failed to create.*API failed; cleanup incomplete.*still alive"):
self.create_vm()
vm, = self.store.load().vms.values()
self.assertEqual(vm.status, "failed")
self.assertEqual(self.process.observe(vm), "alive")
self.assertTrue(Path(vm.disk).exists())
self.assertTrue(vm.runtime_owned)
self.assertTrue(vm.tap_owned)
self.assertEqual(self.network.taps[vm.tap], vm.id)
self.assertEqual(self.network.firewall.status(vm), "quarantined")
self.assertNotIn("network.delete_vm_tap", self.host.actions(vm.id))
self.assertNotIn("firewall.remove", self.host.actions(vm.id))
def test_uncertain_spawn_interrupt_retains_resources_and_survives_stop_retry(self) -> None:
for interrupt in (KeyboardInterrupt(), SystemExit(143)):
with self.subTest(interrupt=type(interrupt).__name__):
self.host.failures["process.spawn", None] = interrupt
previous = set(self.store.load().vms)
with self.assertRaises(type(interrupt)):
self.create_vm()
vm, = [vm for vm in self.store.load().vms.values() if vm.id not in previous]
self.assertEqual(self.process.observe(vm), "unknown")
self.assertTrue(vm.spawn_pending or vm.operation == "spawning")
self.assertIsNone(vm.pid)
self.assertEqual(vm.status, "failed")
self.assertTrue(Path(vm.disk).exists())
self.assertEqual(self.network.taps[vm.tap], vm.id)
self.assertEqual(self.network.firewall.status(vm), "quarantined")
self.assertNotIn(vm.tap, self.network.active_taps)
for action in ("stop", "destroy"):
with self.assertRaisesRegex(UvmError, "process exit unconfirmed"):
getattr(self.service, action)(vm.id)
retained = self.store.get(vm.id)
self.assertEqual(self.process.observe(retained), "unknown")
self.assertTrue(Path(vm.disk).exists())
self.assertIn(vm.tap, self.network.taps)
self.assertNotIn("network.delete_vm_tap", self.host.actions(vm.id))
self.assertNotIn("firewall.remove", self.host.actions(vm.id))
def test_directory_reservation_collision_never_deletes_existing_contents(self) -> None:
directory = self.settings.vm_dir("vm-existing")
directory.mkdir(parents=True)
disk = directory / "rootfs.ext4"
disk.write_bytes(b"not-owned-by-this-create")
marker = directory / "operator-data"
marker.write_bytes(b"keep")
before = {path.name: (path.stat().st_ino, path.read_bytes()) for path in directory.iterdir()}
with patch("uvm.lifecycle.new_vm_id", return_value="vm-existing"):
with self.assertRaisesRegex(UvmError, "failed to create vm-existing"):
self.create_vm()
self.assertEqual({path.name: (path.stat().st_ino, path.read_bytes()) for path in directory.iterdir()}, before)
self.provision.assert_not_called()
self.copy_disk.assert_not_called()
self.assertEqual(self.network.created_taps, [])
self.assertEqual(self.process.started, [])
retained = self.store.load().vms.get("vm-existing")
if retained is not None:
self.assertFalse(retained.runtime_owned)
self.assertFalse(retained.disk_ready)
def test_stale_stopping_and_deleting_records_can_finish_cleanup(self) -> None:
for action, status, operation in (("stop", "stopping", "stop"), ("destroy", "terminating", "delete")):
with self.subTest(action=action):
vm = self.create_vm()
self.persist(vm, status=status, operation=operation)
del self.process.running[vm.id]
self.host.events.clear()
result = getattr(self.service, action)(vm.id)
self.assertEqual(result.id, vm.id)
self.assertNotIn(vm.tap, self.network.taps)
self.assertNotIn(vm.id, self.network.firewall.tables)
self.assertFalse(Path(vm.socket).exists())
self.assertNotIn("process.start", self.host.actions(vm.id))
if action == "stop":
retained = self.store.get(vm.id)
self.assertEqual((retained.status, retained.operation), ("stopped", None))
self.assertTrue(Path(vm.disk).exists())
else:
self.assertNotIn(vm.id, self.store.load().vms)
self.assertFalse(self.settings.vm_dir(vm.id).exists())
def test_failed_delete_intent_cannot_be_resurrected_by_stop_or_start(self) -> None:
vm = self.create_vm()
self.host.failures["firewall.cleanup_connections", vm.id] = UvmError("conntrack remains")
with self.assertRaisesRegex(UvmError, "failed to delete.*conntrack remains"):
self.service.destroy(vm.id)
retained = self.store.get(vm.id)
self.assertEqual((retained.status, retained.operation), ("failed", "delete"))
self.assertTrue(Path(vm.disk).exists())
self.assertIn(vm.id, self.network.firewall.tables)
for action in ("start", "restart", "sync_ports"):
with self.subTest(action=action), self.assertRaisesRegex(ConflictError, "deletion is pending"):
getattr(self.service, action)(vm.id)
with self.assertRaisesRegex(ConflictError, "deletion is pending"):
self.service.set_ports(vm.id, ())
stopped = self.service.stop(vm.id)
self.assertEqual((stopped.status, stopped.operation), ("stopped", "delete"))
with self.assertRaisesRegex(ConflictError, "deletion is pending"):
self.service.start(vm.id)
self.service.destroy(vm.id)
self.assertNotIn(vm.id, self.store.load().vms)
self.assertFalse(self.settings.vm_dir(vm.id).exists())
self.assertEqual(len(self.process.started), 1)
def test_failed_start_retains_edited_disk_and_retries_without_provisioning(self) -> None:
vm = self.create_vm()
self.service.stop(vm.id)
Path(vm.disk).write_bytes(b"retained edits and host keys")
self.host.failures["client.configure", vm.id] = FirecrackerError("API retry needed")
with self.assertRaisesRegex(UvmError, "failed to start.*API retry needed"):
self.service.start(vm.id)
failed = self.store.get(vm.id)
self.assertEqual(failed.status, "failed")
self.assertIsNone(failed.pid)
self.assertTrue(failed.disk_ready)
self.assertTrue(failed.runtime_owned)
self.assertNotIn(vm.tap, self.network.taps)
started = self.service.start(vm.id)
self.assertEqual(started.status, "running")
self.assertEqual(Path(vm.disk).read_bytes(), b"retained edits and host keys")
self.assertEqual(len({info.pid for info in self.process.started}), 3)
self.assertEqual(self.provision.call_count, 1)
self.assertEqual(self.copy_disk.call_count, 1)
def test_restart_uses_one_lock_and_replaces_only_target_process(self) -> None:
vm = self.create_vm()
other = self.create_vm()
other_record = self.store.get(other.id).to_dict()
before_locks = self.store.operation_locks
self.host.events.clear()
restarted = self.service.restart(vm.id, timeout=0.75)
self.assertEqual(self.store.operation_locks - before_locks, 1)
self.assertNotEqual(restarted.pid, vm.pid)
self.assertEqual(self.process.terminated[0], (vm.id, vm.pid, 0.75))
self.assertEqual(self.store.get(other.id).to_dict(), other_record)
self.assertEqual(self.process.observe(other), "alive")
self.assertEqual(self.network.firewall.status(other), "enforced")
self.assertEqual(self.provision.call_count, 2)
tests/test_lifecycle_commands.py
+493
View File
@@ -0,0 +1,493 @@
from __future__ import annotations
from dataclasses import replace
from ipaddress import IPv4Address
from pathlib import Path
from unittest.mock import patch
from tests.test_lifecycle import LifecycleTestCase
from uvm.domain import PortRule
from uvm.errors import ConflictError, FirecrackerError, NotFoundError, UvmError, ValidationError
from uvm.lifecycle import LifecycleService, OperationResult
from uvm.state import StateStore
class LifecycleLookupTests(LifecycleTestCase):
def test_vm_resources_are_distinct_and_aliases_resolve_to_the_same_record(self) -> None:
first = self.create_vm(name="web")
second = self.create_vm(name="database", guest_ip=IPv4Address("10.42.0.9"))
third = self.create_vm()
for field in ("id", "guest_ip", "mac", "tap", "pid", "disk"):
self.assertEqual(len({getattr(vm, field) for vm in (first, second, third)}), 3, field)
self.assertEqual((first.guest_ip, second.guest_ip, third.guest_ip),
("10.42.0.2", "10.42.0.9", "10.42.0.3"))
for identifier in (second.id, second.name, second.guest_ip):
with self.subTest(identifier=identifier):
self.assertEqual(self.service.inspect(identifier).vm.id, second.id)
self.assertEqual(self.service.find_for_ssh(identifier).id, second.id)
self.assertEqual(self.service.log_path(identifier), Path(second.log))
selected = self.service.resolve_targets([
second.name, first.guest_ip, second.id, first.name, third.id, second.guest_ip,
])
self.assertEqual([vm.id for vm in selected], [second.id, first.id, third.id])
self.service.stop(second.name)
self.assertEqual(self.process.observe(self.store.get(second.id)), "gone")
self.assertEqual(self.process.observe(first), "alive")
self.assertEqual(self.process.observe(third), "alive")
self.assertEqual(self.network.firewall.status(first), "enforced")
self.assertEqual(self.network.firewall.status(third), "enforced")
def test_name_and_stopped_ip_collisions_do_not_reserve_or_provision_again(self) -> None:
vm = self.create_vm(name="reserved")
self.service.stop(vm.id)
before = self.settings.state_path.read_bytes()
for kwargs, message in (
({"name": vm.name}, "VM name already exists"),
({"name": vm.id}, "VM name already exists"),
({"guest_ip": IPv4Address(vm.guest_ip)}, "IP already allocated"),
):
with self.subTest(kwargs=kwargs), self.assertRaisesRegex(UvmError, message):
self.create_vm(**kwargs)
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertEqual(self.provision.call_count, 1)
self.assertEqual(self.copy_disk.call_count, 1)
self.assertEqual(len(self.process.started), 1)
def test_generated_id_retries_id_name_and_tap_collisions(self) -> None:
with patch("uvm.lifecycle.new_vm_id", return_value="vm-first-same1234567"):
first = self.create_vm(name="reserved-name")
collision = "vm-other-same1234567"
self.assertEqual(self.network.tap_name(collision), first.tap)
with patch("uvm.lifecycle.new_vm_id", side_effect=[
first.id, first.name, collision, "vm-new-unique12345",
]) as generate:
second = self.create_vm()
self.assertEqual(generate.call_count, 4)
self.assertEqual(second.id, "vm-new-unique12345")
self.assertNotEqual(second.tap, first.tap)
self.assertNotEqual(second.guest_ip, first.guest_ip)
self.assertEqual(self.network.taps, {first.tap: first.id, second.tap: second.id})
def test_ambiguous_persisted_name_ip_or_id_never_selects_an_arbitrary_vm(self) -> None:
first = self.create_vm(name="first")
second = self.create_vm(name="second")
for identifier in (first.guest_ip, first.id, first.name):
with self.subTest(identifier=identifier):
self.persist(second, name=identifier)
before = self.settings.state_path.read_bytes()
self.host.events.clear()
for method in (self.service.inspect, self.service.find_for_ssh,
self.service.log_path, self.service.stop):
with self.assertRaisesRegex(ConflictError, "ambiguous VM identifier"):
method(identifier)
with self.assertRaisesRegex(ConflictError, "ambiguous VM identifier"):
self.service.resolve_targets([identifier])
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertEqual(self.host.events, [])
def test_target_resolution_rejects_invalid_or_missing_targets_without_mutation(self) -> None:
vm = self.create_vm(name="web")
before = self.settings.state_path.read_bytes()
before_locks = self.store.operation_locks
self.host.events.clear()
with self.assertRaises(ValidationError):
self.service.resolve_targets([])
with self.assertRaises(ValidationError):
self.service.resolve_targets([vm.id], all_vms=True)
with self.assertRaises(NotFoundError):
self.service.resolve_targets([vm.name, "missing"])
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertEqual(self.store.operation_locks, before_locks)
self.assertEqual(self.host.events, [])
class LifecyclePortTests(LifecycleTestCase):
def test_stopped_port_edits_and_sync_are_metadata_only(self) -> None:
vm = self.create_vm()
stopped = self.service.stop(vm.id)
self.assertFalse(stopped.tap_owned)
for mode, ports, expected in (
("open", (PortRule(443, "HTTPS"), PortRule(80, "HTTP")),
(PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS"))),
("close", (PortRule(80),), (PortRule(22), PortRule(443, "HTTPS"))),
("set", (), ()),
):
with self.subTest(mode=mode):
self.host.events.clear()
updated = self.service.set_ports(vm.id, ports, mode=mode)
self.assertEqual(updated.ports, expected)
self.assertEqual((updated.status, updated.operation, updated.pid), ("stopped", None, None))
self.assertFalse(updated.tap_owned)
self.assertEqual(self.store.get(vm.id).ports, expected)
self.assertFalse(set(self.host.actions(vm.id)) - {"process.observe", "process.adopt_legacy_identity"})
self.host.events.clear()
self.service.sync_ports(vm.id)
self.assertFalse(set(self.host.actions(vm.id)) - {"process.observe", "process.adopt_legacy_identity"})
self.assertEqual(self.network.firewall.tables, {})
self.assertEqual(len(self.process.started), 1)
self.assertEqual(self.provision.call_count, 1)
def test_live_port_modes_persist_sorted_rules_without_flushing_connections(self) -> None:
vm = self.create_vm(ports=(PortRule(22), PortRule(80, "HTTP")))
other = self.create_vm(ports=(PortRule(443, "HTTPS"),))
before_other = self.store.get(other.id).to_dict()
for mode, ports, expected in (
("open", (PortRule(443, "HTTPS"), PortRule(80)),
(PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS"))),
("close", (PortRule(80), PortRule(9000)), (PortRule(22), PortRule(443, "HTTPS"))),
("set", (), ()),
):
with self.subTest(mode=mode):
self.host.events.clear()
updated = self.service.set_ports(vm.id, ports, mode=mode)
self.assertEqual(updated.ports, expected)
self.assertEqual(updated.pid, vm.pid)
self.assertEqual(updated.status, "running")
self.assertIsNone(updated.operation)
pending = self.host.records["firewall.apply", vm.id]
self.assertEqual(pending.ports, expected)
self.assertEqual(pending.operation, "policy")
self.assert_call_order(vm, "firewall.apply", "network.activate_tap")
self.assertFalse(set(self.host.actions(vm.id)) & {
"firewall.cleanup_connections", "firewall.remove", "network.delete_vm_tap",
"process.terminate", "process.start", "client.configure",
})
self.assertEqual(self.network.firewall.status(updated), "enforced")
self.assertEqual(self.store.get(other.id).to_dict(), before_other)
self.assertEqual(self.network.firewall.status(other), "enforced")
self.assertIn(other.tap, self.network.active_taps)
def test_failed_live_close_persists_pending_policy_quarantines_and_never_reopens_old_ports(self) -> None:
for failure in ("firewall.apply", "firewall.readback", "network.activate_tap"):
with self.subTest(failure=failure):
vm = self.create_vm(ports=(PortRule(22), PortRule(80, "HTTP")))
self.host.events.clear()
self.host.failures[failure, vm.id] = UvmError("policy update failed")
with self.assertRaisesRegex(UvmError, "policy pending.*policy update failed"):
self.service.set_ports(vm.id, (PortRule(80),), mode="close")
pending = self.store.get(vm.id)
self.assertEqual((pending.status, pending.operation), ("failed", "policy"))
self.assertEqual(pending.ports, (PortRule(22),))
self.assertEqual(pending.pid, vm.pid)
self.assertIn("policy update failed", pending.last_error)
self.assertEqual(self.host.records["firewall.apply", vm.id].ports, pending.ports)
self.assertEqual(self.host.records["firewall.quarantine", vm.id].ports, pending.ports)
self.assertEqual(self.network.firewall.status(pending), "quarantined")
self.assertNotIn(vm.tap, self.network.active_taps)
self.assertFalse(set(self.host.actions(vm.id)) & {
"process.terminate", "firewall.cleanup_connections", "firewall.remove",
})
self.host.events.clear()
synced = self.service.sync_ports(vm.id)
self.assertEqual(synced.ports, (PortRule(22),))
self.assertEqual((synced.status, synced.operation, synced.last_error), ("running", None, None))
self.assertEqual(self.network.firewall.status(synced), "enforced")
self.assertIn(vm.tap, self.network.active_taps)
self.assert_call_order(vm, "firewall.apply", "network.activate_tap")
def test_failed_close_still_quarantines_when_tap_deactivation_fails(self) -> None:
vm = self.create_vm(ports=(PortRule(22), PortRule(80, "HTTP")))
self.host.events.clear()
self.host.failures["firewall.apply", vm.id] = UvmError("policy rejected")
self.host.failures["network.deactivate_tap", vm.id] = UvmError("link-down failed")
with self.assertRaisesRegex(UvmError, "policy pending"):
self.service.set_ports(vm.id, (PortRule(80),), mode="close")
pending = self.store.get(vm.id)
self.assertEqual(pending.ports, (PortRule(22),))
self.assertEqual(pending.operation, "policy")
self.assertEqual(self.network.firewall.status(pending), "quarantined")
self.assertIn("firewall.quarantine", self.host.actions(vm.id))
def test_failed_close_keeps_tap_down_if_quarantine_also_fails(self) -> None:
vm = self.create_vm(ports=(PortRule(22), PortRule(80)))
self.host.events.clear()
self.host.failures["firewall.apply", vm.id] = UvmError("policy rejected")
self.host.failures["firewall.quarantine", vm.id] = UvmError("quarantine rejected")
with self.assertRaisesRegex(UvmError, "policy pending"):
self.service.set_ports(vm.id, (PortRule(80),), mode="close")
pending = self.store.get(vm.id)
self.assertEqual(pending.ports, (PortRule(22),))
self.assertEqual(pending.operation, "policy")
self.assertNotIn(vm.tap, self.network.active_taps)
self.assertIn(vm.id, self.network.firewall.tables)
self.assertNotIn("network.activate_tap", self.host.actions(vm.id))
self.assertNotIn("firewall.remove", self.host.actions(vm.id))
def test_interrupted_close_retains_new_intent_and_quarantines(self) -> None:
for interrupt in (KeyboardInterrupt(), SystemExit(143)):
with self.subTest(interrupt=type(interrupt).__name__):
vm = self.create_vm(ports=(PortRule(22), PortRule(80)))
self.host.failures["firewall.apply", vm.id] = interrupt
with self.assertRaises(type(interrupt)):
self.service.set_ports(vm.id, (PortRule(80),), mode="close")
pending = self.store.get(vm.id)
self.assertEqual((pending.ports, pending.operation), ((PortRule(22),), "policy"))
self.assertNotIn(vm.tap, self.network.active_taps)
self.assertEqual(self.network.firewall.status(pending), "quarantined")
def test_legacy_policy_requires_explicit_set_while_stopped(self) -> None:
vm = self.create_vm()
self.persist(vm, ports=None)
before = self.settings.state_path.read_bytes()
for mode in ("open", "close", "set"):
with self.subTest(mode=mode), self.assertRaisesRegex(ConflictError, "legacy VM policy"):
self.service.set_ports(vm.id, (PortRule(22),), mode=mode)
with self.assertRaisesRegex(ConflictError, "legacy VM has no port policy"):
self.service.start(vm.id)
with self.assertRaisesRegex(ConflictError, "legacy VM has no managed policy"):
self.service.sync_ports(vm.id)
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.service.stop(vm.id)
self.assertEqual(self.service.inspect(vm.id).policy_status, "unmanaged")
for mode in ("open", "close"):
with self.subTest(mode=mode), self.assertRaises(ConflictError):
self.service.set_ports(vm.id, (), mode=mode)
migrated = self.service.set_ports(vm.id, (), mode="set")
self.assertEqual(migrated.ports, ())
self.assertEqual(self.service.inspect(vm.id).policy_status, "inactive")
self.assertEqual(self.service.start(vm.id).status, "running")
def test_invalid_port_requests_fail_before_lock_state_or_host_mutation(self) -> None:
vm = self.create_vm()
before = self.settings.state_path.read_bytes()
before_locks = self.store.operation_locks
self.host.events.clear()
for ports, mode in (
([PortRule(22)], "set"), (("22",), "set"),
((PortRule(80, "HTTP"), PortRule(80, "TCP")), "set"),
((PortRule(22),), "replace"),
):
with self.subTest(ports=ports, mode=mode), self.assertRaises(ValidationError):
self.service.set_ports(vm.id, ports, mode=mode)
self.assertEqual(self.store.operation_locks, before_locks)
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertEqual(self.host.events, [])
def test_start_of_running_vm_repairs_policy_without_reboot_or_reprovision(self) -> None:
vm = self.create_vm()
del self.network.firewall.tables[vm.id]
self.host.events.clear()
started = self.service.start(vm.id)
self.assertEqual(started.pid, vm.pid)
self.assertEqual(self.network.firewall.status(started), "enforced")
self.assertNotIn("process.start", self.host.actions(vm.id))
self.assertNotIn("process.terminate", self.host.actions(vm.id))
self.assertNotIn("client.configure", self.host.actions(vm.id))
self.assertEqual(self.provision.call_count, 1)
self.assertEqual(self.copy_disk.call_count, 1)
class LifecycleBulkTests(LifecycleTestCase):
def test_all_bulk_actions_take_one_outer_lock_and_use_only_selected_ids(self) -> None:
untouched = self.create_vm()
untouched_record = self.store.get(untouched.id).to_dict()
for action in ("start", "stop", "restart", "delete", "ports-sync"):
with self.subTest(action=action):
first, second = self.create_vm(), self.create_vm()
ids = (first.id, second.id)
if action == "start":
for vm_id in ids:
self.service.stop(vm_id)
before_locks = self.store.operation_locks
self.host.events.clear()
self.process.terminated.clear()
results = self.service.run_many(action, ids, timeout=0.5)
self.assertEqual(results, [OperationResult(vm_id, action, "succeeded") for vm_id in ids])
self.assertEqual(self.store.operation_locks - before_locks, 1)
self.assertFalse(self.store.operation_active)
self.assertFalse(self.store.transaction_active)
self.assertEqual(self.store.get(untouched.id).to_dict(), untouched_record)
self.assertEqual(self.host.actions(untouched.id), [])
if action in {"stop", "restart", "delete"}:
self.assertEqual([(vm_id, timeout) for vm_id, _, timeout in self.process.terminated
if timeout is not None], [(vm_id, 0.5) for vm_id in ids])
for vm in (first, second):
if action == "delete":
self.assertNotIn(vm.id, self.store.load().vms)
self.assertFalse(self.settings.vm_dir(vm.id).exists())
else:
current = self.store.get(vm.id)
self.assertEqual(current.status, "stopped" if action == "stop" else "running")
if action in {"start", "restart"}:
self.assertNotEqual(current.pid, vm.pid)
def test_partial_failure_reports_each_target_and_continues(self) -> None:
first, failed, last = self.create_vm(), self.create_vm(), self.create_vm()
self.host.failures["process.terminate", failed.id] = FirecrackerError("exit unconfirmed")
results = self.service.run_many("stop", (first.id, failed.id, last.id), timeout=0)
self.assertEqual([result.id for result in results], [first.id, failed.id, last.id])
self.assertEqual([result.outcome for result in results], ["succeeded", "failed", "succeeded"])
self.assertIn(failed.id, results[1].error)
self.assertIn("exit unconfirmed", results[1].error)
self.assertIsNone(results[0].error)
self.assertIsNone(results[2].error)
self.assertEqual(self.store.get(first.id).status, "stopped")
self.assertEqual(self.store.get(last.id).status, "stopped")
retained = self.store.get(failed.id)
self.assertEqual(retained.pid, failed.pid)
self.assertTrue(Path(failed.disk).exists())
self.assertIn(failed.tap, self.network.taps)
self.assertEqual(self.network.firewall.status(retained), "quarantined")
retry = self.service.run_many("stop", (failed.id,))
self.assertEqual(retry, [OperationResult(failed.id, "stop", "succeeded")])
def test_fixed_snapshot_excludes_later_vms_and_reused_name_or_ip(self) -> None:
removed = self.create_vm(name="web")
retained = self.create_vm(name="database")
selected = tuple(vm.id for vm in self.service.resolve_targets([], all_vms=True))
self.service.destroy(removed.id)
replacement = self.create_vm(name=removed.name, guest_ip=IPv4Address(removed.guest_ip))
later = self.create_vm(name="later")
before = {vm.id: self.store.get(vm.id).to_dict() for vm in (replacement, later)}
results = self.service.run_many("stop", selected)
self.assertEqual({result.id: result.outcome for result in results},
{removed.id: "skipped", retained.id: "succeeded"})
for vm in (replacement, later):
self.assertEqual(self.store.get(vm.id).to_dict(), before[vm.id])
self.assertEqual(self.process.observe(vm), "alive")
self.assertEqual(self.store.get(retained.id).status, "stopped")
def test_bulk_exact_ids_do_not_retarget_a_new_vm_named_after_a_deleted_id(self) -> None:
original = self.create_vm(name="original")
selected = tuple(vm.id for vm in self.service.resolve_targets([original.name]))
self.service.destroy(original.id)
replacement = self.create_vm(name=original.id)
before = self.store.get(replacement.id).to_dict()
self.host.events.clear()
results = self.service.run_many("delete", selected)
self.assertEqual(results, [OperationResult(original.id, "delete", "skipped", "already absent")])
self.assertEqual(self.store.get(replacement.id).to_dict(), before)
self.assertEqual(self.host.actions(replacement.id), [])
self.assertTrue(Path(replacement.disk).exists())
def test_interrupt_preserves_current_resources_and_skips_remaining_targets(self) -> None:
first, interrupted, last = self.create_vm(), self.create_vm(), self.create_vm()
last_record = self.store.get(last.id).to_dict()
self.host.events.clear()
self.host.failures["process.terminate", interrupted.id] = KeyboardInterrupt()
results = self.service.run_many("stop", (first.id, interrupted.id, last.id))
self.assertEqual(results, [
OperationResult(first.id, "stop", "succeeded"),
OperationResult(interrupted.id, "stop", "interrupted", "interrupted"),
OperationResult(last.id, "stop", "skipped", "not attempted after interruption"),
])
self.assertEqual(self.store.get(first.id).status, "stopped")
pending = self.store.get(interrupted.id)
self.assertEqual((pending.status, pending.operation), ("failed", "stop"))
self.assertEqual(pending.pid, interrupted.pid)
self.assertTrue(Path(interrupted.disk).exists())
self.assertIn(interrupted.tap, self.network.taps)
self.assertEqual(self.network.firewall.status(pending), "quarantined")
self.assertEqual(self.store.get(last.id).to_dict(), last_record)
self.assertEqual(self.host.actions(last.id), [])
self.assertFalse(self.store.operation_active)
self.assertEqual(self.service.stop(last.id).status, "stopped")
def test_empty_bulk_requests_have_no_vm_or_host_effects(self) -> None:
self.assertEqual(self.service.resolve_targets([], all_vms=True), [])
for action in ("start", "stop", "restart", "delete", "ports-sync"):
with self.subTest(action=action):
self.assertEqual(self.service.run_many(action, ()), [])
self.assertEqual(self.store.load().vms, {})
self.assertFalse(self.settings.state_path.exists())
self.assertEqual(self.host.events, [])
self.provision.assert_not_called()
self.copy_disk.assert_not_called()
def test_invalid_bulk_action_fails_before_lock_or_mutation(self) -> None:
with self.assertRaisesRegex(ValidationError, "invalid lifecycle action"):
self.service.run_many("create", ("vm-missing",))
self.assertEqual(self.store.operation_locks, 0)
self.assertEqual(self.host.events, [])
class LifecycleInspectionTests(LifecycleTestCase):
def test_unknown_pid_inspection_is_read_only_and_never_adopts_or_cleans_up(self) -> None:
vm = self.create_vm(name="legacy-identity")
self.persist(vm, boot_id=None)
before = self.settings.state_path.read_bytes()
before_files = {path: (path.stat().st_ino, path.read_bytes())
for path in self.settings.vm_dir(vm.id).iterdir()}
before_locks = self.store.operation_locks
self.host.events.clear()
self.require_root.reset_mock()
self.check_kvm.reset_mock()
with patch.object(self.store, "transaction", side_effect=AssertionError("read must not persist")):
inspected = self.service.inspect(vm.name)
listed, = self.service.list_vms()
self.assertEqual(self.service.find_for_ssh(vm.guest_ip).id, vm.id)
self.assertEqual(self.service.log_path(vm.id), Path(vm.log))
self.assertEqual([item.id for item in self.service.resolve_targets([vm.name])], [vm.id])
self.assertEqual(inspected.observed_status, "unknown")
self.assertEqual(listed.observed_status, "unknown")
self.assertEqual(inspected.vm.status, "running")
self.assertIsNone(inspected.vm.boot_id)
self.assertEqual(self.store.operation_locks, before_locks)
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertEqual({path: (path.stat().st_ino, path.read_bytes())
for path in self.settings.vm_dir(vm.id).iterdir()}, before_files)
self.assertFalse(set(self.host.actions(vm.id)) - {"process.observe", "firewall.status"})
self.require_root.assert_not_called()
self.check_kvm.assert_not_called()
def test_mutation_adopts_legacy_identity_under_lock_before_termination(self) -> None:
vm = self.create_vm()
self.persist(vm, boot_id=None)
self.host.events.clear()
stopped = self.service.stop(vm.id)
self.assertEqual(stopped.status, "stopped")
self.assertEqual(self.host.records["process.terminate", vm.id].boot_id, self.process.host_boot_id())
self.assert_call_order(vm, "process.adopt_legacy_identity", "process.terminate")
def test_unknown_identity_blocks_start_and_port_edits_without_mutation(self) -> None:
vm = self.create_vm()
self.process.unknown.add(vm.id)
before = self.settings.state_path.read_bytes()
self.host.events.clear()
for method in (self.service.start, self.service.sync_ports):
with self.assertRaisesRegex(ConflictError, "unknown"):
method(vm.id)
with self.assertRaisesRegex(ConflictError, "known process identity"):
self.service.set_ports(vm.id, ())
self.assertEqual(self.settings.state_path.read_bytes(), before)
self.assertFalse(set(self.host.actions(vm.id)) - {"process.observe", "process.adopt_legacy_identity"})
def test_observed_dead_or_running_status_never_rewrites_desired_state(self) -> None:
dead = self.create_vm()
alive = self.create_vm()
del self.process.running[dead.id]
self.persist(alive, status="stopped")
before = self.settings.state_path.read_bytes()
observations = {item.vm.id: item for item in self.service.list_vms()}
self.assertEqual(observations[dead.id].observed_status, "dead")
self.assertEqual(observations[dead.id].vm.status, "running")
self.assertEqual(observations[alive.id].observed_status, "running")
self.assertEqual(observations[alive.id].vm.status, "stopped")
self.assertEqual(self.settings.state_path.read_bytes(), before)
def test_empty_inspection_does_not_create_an_uninstalled_registry(self) -> None:
settings = replace(self.settings, base=self.settings.base / "not-installed")
service = LifecycleService(settings, StateStore(settings), self.images, self.network, self.process)
self.assertEqual(service.list_vms(), [])
self.assertEqual(service.resolve_targets([], all_vms=True), [])
for method in (service.inspect, service.find_for_ssh, service.log_path):
with self.assertRaises(NotFoundError):
method("missing")
self.assertFalse(settings.base.exists())
self.assertEqual(self.host.events, [])
def test_diagnostics_collects_failures_without_host_commands_or_lifecycle_effects(self) -> None:
self.check_kvm.side_effect = UvmError("KVM unavailable")
self.host.failures["firewall.check_available", None] = UvmError("nft unavailable")
with patch("uvm.lifecycle.shutil.which", return_value=None):
checks = {item["name"]: item for item in self.service.diagnostics()}
self.assertTrue(checks["root"]["ok"])
self.assertTrue(checks["guest assets"]["ok"])
self.assertFalse(checks["kvm"]["ok"])
self.assertEqual(checks["kvm"]["message"], "KVM unavailable")
self.assertFalse(checks["private firewall"]["ok"])
self.assertEqual(checks["private firewall"]["message"], "nft unavailable")
self.assertFalse(checks["nft"]["ok"])
self.assertEqual(self.store.operation_locks, 0)
self.assertFalse(self.settings.state_path.exists())
self.assertEqual(self.host.events, [("firewall.check_available", None)])
tests/test_network.py
+355 -33
View File
@@ -1,35 +1,78 @@
from __future__ import annotations
import json
import unittest
from copy import deepcopy
from dataclasses import replace
from ipaddress import IPv4Address, IPv4Network
from uvm.config import Settings
from uvm.domain import VmRecord
from uvm.errors import UvmError, ValidationError
from uvm.domain import PortRule, VmRecord
from uvm.errors import CommandError, ConflictError, UvmError, ValidationError
from uvm.network import NetworkManager
from uvm.system import CommandResult
LINK_INVENTORY = ("ip", "-j", "-d", "link", "show")
class FakeRunner:
def __init__(self) -> None:
self.calls: list[tuple[str, ...]] = []
self.existing_links: set[str] = set()
self.link_details: dict[str, str] = {}
self.links: dict[str, dict] = {}
self.inventory: str | None = None
self.failures: dict[tuple[str, ...], BaseException] = {}
def run(self, command, *, check=True, capture=False, timeout=None) -> CommandResult:
del check, capture, timeout
del capture, timeout
args = tuple(str(part) for part in command)
self.calls.append(args)
if args[:3] == ("ip", "link", "show") and args[3] in self.existing_links:
return CommandResult(args=args, returncode=0)
if args[:6] == ("ip", "-j", "-d", "link", "show", "dev"):
return CommandResult(args=args, returncode=0, stdout=self.link_details[args[6]])
if args[:4] == ("ip", "link", "show", "uvm0"):
return CommandResult(args=args, returncode=1)
if failure := self.failures.get(args):
if check or not isinstance(failure, CommandError):
raise failure
return CommandResult(args=args, returncode=1, stderr=str(failure))
if args[:5] == LINK_INVENTORY:
links = list(self.links.values()) if args == LINK_INVENTORY else [self.links[args[6]]]
stdout = self.inventory if self.inventory is not None else json.dumps(links)
return CommandResult(args=args, returncode=0, stdout=stdout)
if args[:3] == ("ip", "link", "show"):
return CommandResult(args=args, returncode=0 if args[3] in self.links else 1)
if args[:4] == ("ip", "tuntap", "add", "dev"):
if args[4] in self.links:
raise CommandError("interface already exists")
self.links[args[4]] = {
"ifname": args[4], "flags": [],
"linkinfo": {"info_kind": "tun", "info_data": {"type": "tap"}},
}
elif args[:3] == ("ip", "link", "add"):
self.links[args[3]] = {
"ifname": args[3], "flags": [],
"linkinfo": {"info_kind": "bridge", "info_data": {"stp_state": 0}},
}
elif args[:3] == ("ip", "link", "set"):
name, *options = args[4:] if args[3] == "dev" else args[3:]
link = self.links[name]
if options[0] == "alias":
link["ifalias"] = options[1]
elif options[0] == "master":
link["master"] = options[1]
elif options in (["up"], ["down"]):
link["flags"] = ["UP"] if options == ["up"] else []
elif options[:3] == ["type", "bridge_slave", "group_fwd_mask"]:
link["linkinfo"]["info_slave_kind"] = "bridge"
link["linkinfo"]["info_slave_data"] = {"group_fwd_mask": int(options[3])}
else:
raise AssertionError(f"unexpected link command: {args}")
elif args[:3] == ("ip", "link", "del"):
self.links.pop(args[3], None)
elif args[:3] == ("ip", "addr", "replace") or args[:2] == ("sysctl", "-w"):
pass
elif args[0] == "iptables":
return CommandResult(args=args, returncode=1 if "-C" in args else 0)
elif args != ("ip", "route", "show", "default"):
raise AssertionError(f"unexpected host command: {args}")
if args == ("ip", "route", "show", "default"):
return CommandResult(args=args, returncode=0, stdout="default via 192.0.2.1 dev eth0\n")
if len(args) > 3 and args[0] == "iptables" and "-C" in args:
return CommandResult(args=args, returncode=1)
return CommandResult(args=args, returncode=0)
@@ -37,13 +80,8 @@ class NetworkManagerTests(unittest.TestCase):
def setUp(self) -> None:
self.runner = FakeRunner()
self.network = NetworkManager(Settings(), runner=self.runner) # type: ignore[arg-type]
def test_allocates_first_available_guest_address(self) -> None:
self.assertEqual(self.network.allocate_ip([], None), IPv4Address("10.42.0.2"))
def test_rejects_gateway_and_used_requested_addresses(self) -> None:
vm = VmRecord(
id="vm-test",
self.vm = VmRecord(
id="vm-0123456789abcdef0123456789abcdef",
cpu=1,
ram_mib=512,
guest_ip="10.42.0.2",
@@ -53,11 +91,23 @@ class NetworkManagerTests(unittest.TestCase):
socket="/tmp/firecracker.sock",
config="/tmp/config.json",
log="/tmp/firecracker.log",
ports=(PortRule(22),),
bridge="uvm0",
)
self.tap_link = {
"ifname": self.vm.tap, "ifalias": f"uvm:{self.vm.id}", "flags": [],
"master": "uvm0",
"linkinfo": {"info_kind": "tun", "info_data": {"type": "tap"}},
}
def test_allocates_first_available_guest_address(self) -> None:
self.assertEqual(self.network.allocate_ip([], None), IPv4Address("10.42.0.2"))
def test_rejects_gateway_and_used_requested_addresses(self) -> None:
with self.assertRaises(ValidationError):
self.network.allocate_ip([vm], IPv4Address("10.42.0.1"))
self.network.allocate_ip([self.vm], IPv4Address("10.42.0.1"))
with self.assertRaises(ValidationError):
self.network.allocate_ip([vm], IPv4Address("10.42.0.2"))
self.network.allocate_ip([self.vm], IPv4Address("10.42.0.2"))
with self.assertRaises(ValidationError):
self.network.allocate_ip([], IPv4Address("10.42.0.0"))
with self.assertRaises(ValidationError):
@@ -130,19 +180,21 @@ class NetworkManagerTests(unittest.TestCase):
)
def test_refuses_to_adopt_an_existing_tap_device(self) -> None:
self.runner.existing_links.add("uvm-existing")
with self.assertRaises(UvmError):
self.network.create_tap("uvm-existing")
self.assertNotIn(
("ip", "tuntap", "add", "dev", "uvm-existing", "mode", "tap"),
self.runner.calls,
)
for alias in (None, "foreign-owner", f"uvm:{self.vm.id}"):
for owned in (False, True):
with self.subTest(alias=alias, owned=owned):
link = {**deepcopy(self.tap_link), "ifalias": alias}
self.runner.links = {self.vm.tap: link}
self.runner.calls.clear()
self.vm.tap_owned = owned
with self.assertRaisesRegex(ConflictError, "already exists"):
self.network.prepare_tap(self.vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertEqual(self.runner.links, {self.vm.tap: link})
self.assertIs(self.vm.tap_owned, owned)
def test_refuses_to_reconfigure_an_existing_non_bridge_interface(self) -> None:
self.runner.existing_links.add("uvm0")
self.runner.link_details["uvm0"] = '[{"linkinfo": {"info_kind": "dummy"}}]'
self.runner.links["uvm0"] = {"ifname": "uvm0", "linkinfo": {"info_kind": "dummy"}}
with self.assertRaises(UvmError):
self.network.ensure_bridge()
@@ -151,3 +203,273 @@ class NetworkManagerTests(unittest.TestCase):
("ip", "addr", "replace", "10.42.0.1/24", "dev", "uvm0"),
self.runner.calls,
)
def test_existing_stp_disabled_bridge_is_reused(self) -> None:
self.runner.links["uvm0"] = {
"ifname": "uvm0", "linkinfo": {"info_kind": "bridge", "info_data": {"stp_state": 0}},
}
self.network.ensure_bridge()
self.assertNotIn(("ip", "link", "add", "uvm0", "type", "bridge"), self.runner.calls)
self.assertIn(("ip", "addr", "replace", "10.42.0.1/24", "dev", "uvm0"), self.runner.calls)
def test_stp_enabled_bridge_is_rejected_before_any_mutation(self) -> None:
for stp_state in (1, 2):
with self.subTest(stp_state=stp_state):
self.runner.links["uvm0"] = {
"ifname": "uvm0",
"linkinfo": {"info_kind": "bridge", "info_data": {"stp_state": stp_state}},
}
self.runner.calls.clear()
with self.assertRaisesRegex(UvmError, "STP-disabled"):
self.network.ensure_bridge()
self.assertEqual(self.runner.calls, [
("ip", "link", "show", "uvm0"), (*LINK_INVENTORY, "dev", "uvm0"),
])
def test_invalid_bridge_inventory_is_rejected_before_any_mutation(self) -> None:
self.runner.links["uvm0"] = {"ifname": "uvm0"}
for inventory in ("invalid-json", "null", "{}", "[]", "[{}]"):
with self.subTest(inventory=inventory):
self.runner.inventory = inventory
self.runner.calls.clear()
with self.assertRaises(UvmError):
self.network.ensure_bridge()
self.assertEqual(self.runner.calls, [
("ip", "link", "show", "uvm0"), (*LINK_INVENTORY, "dev", "uvm0"),
])
def test_prepare_tap_sets_owner_bridge_and_lldp_mask_without_activating_it(self) -> None:
unrelated = {"ifname": f"{self.vm.tap}-other", "linkinfo": {"info_kind": "dummy"}}
for bridge in (None, "uvm-retained"):
with self.subTest(bridge=bridge):
self.runner.links = {unrelated["ifname"]: deepcopy(unrelated)}
self.runner.calls.clear()
vm = replace(self.vm, bridge=bridge, tap_owned=False)
self.network.prepare_tap(vm)
link = self.runner.links[vm.tap]
self.assertIs(vm.tap_owned, True)
self.assertEqual(link["ifalias"], f"uvm:{vm.id}")
self.assertEqual(link["master"], bridge or "uvm0")
self.assertEqual(link["linkinfo"]["info_kind"], "tun")
self.assertEqual(link["linkinfo"]["info_data"]["type"], "tap")
self.assertEqual(link["linkinfo"]["info_slave_data"]["group_fwd_mask"], 16384)
self.assertNotIn("UP", link["flags"])
self.assertFalse(any(call[-1] == "up" for call in self.runner.calls))
self.assertEqual(self.runner.links[unrelated["ifname"]], unrelated)
self.assertEqual(self.runner.calls[0], LINK_INVENTORY)
commands = (
("ip", "tuntap", "add", "dev", vm.tap, "mode", "tap"),
("ip", "link", "set", "dev", vm.tap, "alias", f"uvm:{vm.id}"),
("ip", "link", "set", vm.tap, "master", bridge or "uvm0"),
("ip", "link", "set", "dev", vm.tap, "type", "bridge_slave", "group_fwd_mask", "16384"),
)
positions = [self.runner.calls.index(command) for command in commands]
self.assertEqual(positions, sorted(positions))
def test_owned_tap_can_be_activated_deactivated_and_deleted(self) -> None:
unrelated = {"ifname": "eth0", "flags": ["UP"], "linkinfo": {"info_kind": "veth"}}
for owned in (False, True):
with self.subTest(owned=owned):
vm = replace(self.vm, tap_owned=owned)
self.runner.links = {"eth0": deepcopy(unrelated), vm.tap: deepcopy(self.tap_link)}
self.runner.calls.clear()
self.assertTrue(self.network.has_tap(vm))
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.network.activate_tap(vm)
self.assertIn("UP", self.runner.links[vm.tap]["flags"])
self.network.deactivate_tap(vm)
self.assertNotIn("UP", self.runner.links[vm.tap]["flags"])
self.assertIs(vm.tap_owned, owned)
self.network.delete_vm_tap(vm)
self.assertNotIn(vm.tap, self.runner.links)
self.assertIs(vm.tap_owned, False)
self.assertFalse(self.network.has_tap(vm))
self.assertEqual(self.runner.links, {"eth0": unrelated})
def test_missing_tap_cannot_be_activated_and_cleanup_is_idempotent(self) -> None:
self.vm.tap_owned = True
with self.assertRaisesRegex(UvmError, "TAP device missing"):
self.network.activate_tap(self.vm)
self.assertIs(self.vm.tap_owned, True)
self.network.deactivate_tap(self.vm)
self.assertFalse(self.network.has_tap(self.vm))
self.network.delete_vm_tap(self.vm)
self.network.delete_vm_tap(self.vm)
self.assertIs(self.vm.tap_owned, False)
self.assertTrue(all(call == LINK_INVENTORY for call in self.runner.calls))
def test_named_foreign_taps_are_never_activated_deactivated_deleted_or_claimed(self) -> None:
cases = (
((PortRule(22),), True, None), ((), True, ""), ((), False, None),
(None, False, None), (None, True, "uvm:another-vm"),
((), True, "uvm:another-vm"), ((), True, self.vm.id),
((), True, f"uvm:{self.vm.id}-suffix"),
)
for ports, owned, alias in cases:
for operation in ("activate_tap", "deactivate_tap", "delete_vm_tap", "has_tap"):
with self.subTest(ports=ports, owned=owned, alias=alias, operation=operation):
vm = replace(self.vm, ports=ports, tap_owned=owned)
link = {**deepcopy(self.tap_link), "ifalias": alias}
self.runner.links = {vm.tap: link}
before = deepcopy(self.runner.links)
self.runner.calls.clear()
with self.assertRaisesRegex(ConflictError, "unowned TAP"):
getattr(self.network, operation)(vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertEqual(self.runner.links, before)
self.assertIs(vm.tap_owned, owned)
def test_owned_legacy_unaliased_records_can_only_manage_real_tun_taps(self) -> None:
for alias in (None, ""):
with self.subTest(alias=alias):
vm = replace(self.vm, ports=None, tap_owned=True)
link = deepcopy(self.tap_link)
if alias is None:
link.pop("ifalias")
else:
link["ifalias"] = alias
self.runner.links = {vm.tap: link}
self.assertTrue(self.network.has_tap(vm))
self.network.activate_tap(vm)
self.assertIn("UP", link["flags"])
self.network.deactivate_tap(vm)
self.assertNotIn("UP", link["flags"])
self.network.delete_vm_tap(vm)
self.assertIs(vm.tap_owned, False)
self.assertNotIn(vm.tap, self.runner.links)
def test_ownership_markers_do_not_authorize_non_tap_interfaces(self) -> None:
for info in (
{}, {"info_kind": "dummy"}, {"info_kind": "bridge"}, {"info_kind": "veth"},
{"info_kind": "tap"}, {"info_kind": "tun"},
{"info_kind": "tun", "info_data": {"type": "tun"}},
):
for ports, alias in ((None, None), ((), f"uvm:{self.vm.id}")):
for operation in ("activate_tap", "deactivate_tap", "delete_vm_tap", "has_tap"):
with self.subTest(info=info, ports=ports, operation=operation):
vm = replace(self.vm, ports=ports, tap_owned=True)
link = {"ifname": vm.tap, "ifalias": alias, "linkinfo": info}
self.runner.links = {vm.tap: link}
self.runner.calls.clear()
with self.assertRaises(ConflictError):
getattr(self.network, operation)(vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertIs(vm.tap_owned, True)
def test_invalid_or_ambiguous_inventory_fails_closed(self) -> None:
for inventory in (
"invalid-json", "null", "{}", "[null]", "[[]]", "[1]",
json.dumps([self.tap_link, self.tap_link]),
):
for operation in ("prepare_tap", "activate_tap", "deactivate_tap", "delete_vm_tap", "has_tap"):
with self.subTest(inventory=inventory, operation=operation):
self.runner.inventory = inventory
self.runner.calls.clear()
self.vm.tap_owned = True
with self.assertRaisesRegex(UvmError, "cannot inspect network interfaces"):
getattr(self.network, operation)(self.vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertIs(self.vm.tap_owned, True)
def test_invalid_interface_names_cannot_authorize_creation_or_cleanup(self) -> None:
for link in ({}, {"ifname": None}, {"ifname": 123}, {"ifname": ""}):
for operation, owned in (("prepare_tap", False), ("delete_vm_tap", True)):
with self.subTest(link=link, operation=operation):
self.runner.inventory = json.dumps([link])
self.runner.links.clear()
self.runner.calls.clear()
self.vm.tap_owned = owned
with self.assertRaises(UvmError):
getattr(self.network, operation)(self.vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertIs(self.vm.tap_owned, owned)
def test_malformed_tap_metadata_is_a_controlled_error_without_mutation(self) -> None:
for info in (
None, [], {"info_kind": "tun", "info_data": None},
{"info_kind": "tun", "info_data": []},
):
for operation in ("activate_tap", "delete_vm_tap"):
with self.subTest(info=info, operation=operation):
self.runner.links = {self.vm.tap: {**self.tap_link, "linkinfo": info}}
self.runner.calls.clear()
self.vm.tap_owned = True
with self.assertRaises(UvmError):
getattr(self.network, operation)(self.vm)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertIs(self.vm.tap_owned, True)
def test_inventory_command_failure_never_mutates_a_link_or_ownership(self) -> None:
failure = CommandError("inventory unavailable")
self.runner.failures[LINK_INVENTORY] = failure
for operation in ("prepare_tap", "activate_tap", "deactivate_tap", "delete_vm_tap", "has_tap"):
with self.subTest(operation=operation):
self.runner.calls.clear()
self.vm.tap_owned = True
with self.assertRaises(CommandError) as raised:
getattr(self.network, operation)(self.vm)
self.assertIs(raised.exception, failure)
self.assertEqual(self.runner.calls, [LINK_INVENTORY])
self.assertIs(self.vm.tap_owned, True)
def test_prepare_failure_cleans_created_taps_or_retains_the_persistable_owned_flag(self) -> None:
stages = (
("ip", "tuntap", "add", "dev", self.vm.tap, "mode", "tap"),
("ip", "link", "set", "dev", self.vm.tap, "alias", f"uvm:{self.vm.id}"),
("ip", "link", "set", self.vm.tap, "master", "uvm0"),
("ip", "link", "set", "dev", self.vm.tap, "type", "bridge_slave", "group_fwd_mask", "16384"),
)
delete = ("ip", "link", "del", self.vm.tap)
for index, stage in enumerate(stages):
for cleanup_fails in (False, True):
with self.subTest(stage=stage, cleanup_fails=cleanup_fails):
vm = replace(self.vm, tap_owned=False)
self.runner.links.clear()
self.runner.calls.clear()
failure = CommandError("preparation failed")
self.runner.failures = {stage: failure}
if cleanup_fails:
self.runner.failures[delete] = CommandError("deletion failed")
with self.assertRaises(CommandError) as raised:
self.network.prepare_tap(vm)
retained = index > 0 and cleanup_fails
self.assertIs(raised.exception, failure)
self.assertIs(vm.tap_owned, retained)
restored = VmRecord.from_dict(json.loads(json.dumps(vm.to_dict())))
self.assertIs(restored.tap_owned, retained)
self.assertEqual(vm.tap in self.runner.links, retained)
self.assertEqual(delete in self.runner.calls, index > 0)
self.assertFalse(any(call[-1] == "up" for call in self.runner.calls))
def test_interrupted_preparation_cleans_up_and_reraises_the_interrupt(self) -> None:
stage = ("ip", "link", "set", self.vm.tap, "master", "uvm0")
for failure in (KeyboardInterrupt(), SystemExit(2)):
with self.subTest(failure=type(failure).__name__):
self.runner.calls.clear()
self.runner.failures = {stage: failure}
with self.assertRaises(type(failure)) as raised:
self.network.prepare_tap(self.vm)
self.assertIs(raised.exception, failure)
self.assertNotIn(self.vm.tap, self.runner.links)
self.assertIs(self.vm.tap_owned, False)
self.assertIn(("ip", "link", "del", self.vm.tap), self.runner.calls)
def test_failed_deletion_keeps_ownership_until_a_successful_retry(self) -> None:
self.vm.tap_owned = True
self.runner.links[self.vm.tap] = deepcopy(self.tap_link)
delete = ("ip", "link", "del", self.vm.tap)
self.runner.failures[delete] = CommandError("device busy")
with self.assertRaisesRegex(UvmError, "could not delete TAP.*device busy"):
self.network.delete_vm_tap(self.vm)
self.assertIn(self.vm.tap, self.runner.links)
self.assertIs(self.vm.tap_owned, True)
self.assertIs(VmRecord.from_dict(self.vm.to_dict()).tap_owned, True)
self.runner.failures.clear()
self.network.delete_vm_tap(self.vm)
self.assertNotIn(self.vm.tap, self.runner.links)
self.assertIs(self.vm.tap_owned, False)
tests/test_process.py
+1251 -38
View File
File diff suppressed because it is too large Load Diff
tests/test_server.py
+298 -32
View File
@@ -11,8 +11,10 @@ from unittest.mock import patch
from uvm.cli import build_parser, main
from uvm.config import Settings
from uvm.errors import ConfigurationError, UvmError
from uvm.server import create_api, run_server
from uvm.domain import PortRule, VmRecord, VmSpec
from uvm.errors import ConfigurationError, ConflictError, NotFoundError, UvmError, ValidationError
from uvm.lifecycle import ListedVm
from uvm.server import _http_status_for, create_api, run_server
def fake_application(*, api_token: str | None = None):
@@ -117,6 +119,20 @@ class ServerCliTests(unittest.TestCase):
Settings(api_token=token)
class ServerErrorTests(unittest.TestCase):
def test_domain_errors_map_to_http_status_codes(self) -> None:
for error, expected in (
(NotFoundError("unknown guest"), 404),
(ConflictError("ambiguous name"), 409),
(ValidationError("invalid name"), 422),
(ConfigurationError("invalid settings"), 422),
(UvmError("this operation needs root"), 403),
(UvmError("host operation failed"), 500),
):
with self.subTest(error=error):
self.assertEqual(_http_status_for(error), expected)
FASTAPI_AVAILABLE = (
importlib.util.find_spec("fastapi") is not None
and importlib.util.find_spec("httpx") is not None
@@ -125,15 +141,15 @@ FASTAPI_AVAILABLE = (
if FASTAPI_AVAILABLE:
from fastapi.testclient import TestClient
from uvm.domain import VmRecord
from uvm.lifecycle import ListedVm
from uvm.server import create_api
class FakeLifecycle:
class FakeLifecycle:
def __init__(self) -> None:
self.last_spec = None
self.last_spec: VmSpec | None = None
self.observed_status = "running"
self.policy_status = "enforced"
self.vm = VmRecord(
id="vm-test",
name="test-vm",
cpu=1,
ram_mib=512,
guest_ip="10.42.0.2",
@@ -143,28 +159,62 @@ if FASTAPI_AVAILABLE:
socket="/tmp/firecracker.sock",
config="/tmp/config.json",
log="/tmp/firecracker.log",
password="guest-secret",
status="running",
ports=(PortRule(22),),
operation=None,
disk_ready=True,
boot_id="test-boot",
)
def list_vms(self) -> list[ListedVm]:
return [ListedVm(vm=self.vm, observed_status=self.vm.status)]
return [ListedVm(
vm=self.vm,
observed_status=self.observed_status,
policy_status=self.policy_status,
)]
def create(self, spec):
def inspect(self, identifier: str) -> ListedVm:
if identifier not in {self.vm.id, self.vm.guest_ip, self.vm.name}:
raise NotFoundError(f"VM not found: {identifier}")
return ListedVm(
vm=self.vm,
observed_status=self.observed_status,
policy_status=self.policy_status,
)
def create(self, spec: VmSpec) -> VmRecord:
self.last_spec = spec
self.vm.cpu = spec.cpu
self.vm.ram_mib = spec.ram_mib
if spec.guest_ip is not None:
self.vm.guest_ip = str(spec.guest_ip)
self.vm.username = spec.username
self.vm.password = spec.password
self.vm.name = spec.name
self.vm.ports = spec.ports
self.vm.status = "running"
self.vm.operation = None
self.observed_status = "running"
self.policy_status = "enforced"
return self.vm
def stop(self, _vm_id: str):
def stop(self, identifier: str) -> VmRecord:
self.inspect(identifier)
self.vm.status = "stopped"
self.vm.pid = None
self.vm.operation = None
self.observed_status = "stopped"
self.policy_status = "inactive"
return self.vm
def destroy(self, _vm_id: str):
def destroy(self, identifier: str) -> VmRecord:
self.inspect(identifier)
return self.vm
@unittest.skipUnless(FASTAPI_AVAILABLE, "FastAPI is not installed")
class ApiRouterTests(unittest.TestCase):
@unittest.skipUnless(FASTAPI_AVAILABLE, "FastAPI and httpx are required for API tests")
class ApiRouterTests(unittest.TestCase):
def setUp(self) -> None:
application = fake_application(api_token="test-token")
self.lifecycle = FakeLifecycle()
@@ -176,7 +226,9 @@ if FASTAPI_AVAILABLE:
)
)
application.state_store = SimpleNamespace(initialize=lambda: None)
self.headers = {"X-UVM-Token": "test-token"}
self.client = TestClient(create_api(application, host="127.0.0.1"))
self.addCleanup(self.client.close)
def test_health_is_available_without_credentials(self) -> None:
response = self.client.get("/health")
@@ -185,19 +237,30 @@ if FASTAPI_AVAILABLE:
self.assertEqual(response.json(), {"status": "ok"})
def test_vm_routes_require_and_accept_the_api_token(self) -> None:
unauthorized = self.client.get("/vms")
authorized = self.client.get("/vms", headers={"X-UVM-Token": "test-token"})
for method, path in (
("GET", "/vms"),
("POST", "/vms"),
("GET", "/vms/test-vm"),
("POST", "/vms/test-vm/stop"),
("DELETE", "/vms/test-vm"),
("POST", "/install"),
):
for headers in ({}, {"X-UVM-Token": "wrong-token"}):
with self.subTest(method=method, path=path, headers=headers):
response = self.client.request(method, path, headers=headers, json={})
self.assertEqual(response.status_code, 401)
self.assertIsNone(self.lifecycle.last_spec)
self.assertEqual(self.lifecycle.vm.status, "running")
self.assertEqual(unauthorized.status_code, 401)
authorized = self.client.get("/vms", headers=self.headers)
self.assertEqual(authorized.status_code, 200)
self.assertEqual(authorized.json()[0]["id"], "vm-test")
self.assertNotIn("password", authorized.json()[0])
def test_vm_lifecycle_routes_and_extra_field_validation(self) -> None:
headers = {"X-UVM-Token": "test-token"}
created = self.client.post(
"/vms",
headers=headers,
headers=self.headers,
json={
"cpu": 1,
"ram": "512",
@@ -205,37 +268,240 @@ if FASTAPI_AVAILABLE:
"password": "api-secret",
},
)
detail = self.client.get("/vms/vm-test", headers=headers)
stopped = self.client.post("/vms/vm-test/stop", headers=headers)
destroyed = self.client.delete("/vms/vm-test", headers=headers)
invalid = self.client.post("/vms", headers=headers, json={"forse": True})
detail = self.client.get("/vms/vm-test", headers=self.headers)
stopped = self.client.post("/vms/vm-test/stop", headers=self.headers)
destroyed = self.client.delete("/vms/vm-test", headers=self.headers)
invalid = self.client.post("/vms", headers=self.headers, json={"forse": True})
self.assertEqual(created.status_code, 201)
self.assertEqual(created.json()["username"], "root")
self.assertNotIn("password", created.json())
self.assertEqual(self.lifecycle.last_spec.password, "api-secret")
for response in (created, detail, stopped, destroyed):
self.assertNotIn("password", response.json())
self.assertNotIn("api-secret", response.text)
self.assertEqual(detail.status_code, 200)
self.assertEqual(stopped.json()["status"], "stopped")
self.assertEqual(destroyed.json()["status"], "terminated")
self.assertEqual(stopped.json()["policy_status"], "inactive")
self.assertEqual(destroyed.json(), {"id": "vm-test", "status": "terminated"})
self.assertEqual(invalid.status_code, 422)
def test_vm_create_defaults_guest_credentials_to_root(self) -> None:
response = self.client.post(
"/vms",
headers={"X-UVM-Token": "test-token"},
json={"cpu": 1, "ram": "512"},
"/vms", headers=self.headers, json={"cpu": 1, "ram": "512"},
)
self.assertEqual(response.status_code, 201)
self.assertEqual(self.lifecycle.last_spec.username, "root")
self.assertEqual(self.lifecycle.last_spec.password, "root")
self.assertIsNone(self.lifecycle.last_spec.name)
self.assertEqual(self.lifecycle.last_spec.ports, (PortRule(22),))
def test_create_accepts_valid_names(self) -> None:
for name in (None, "a", "web-1", "a" * 63):
with self.subTest(name=name):
response = self.client.post("/vms", headers=self.headers, json={"name": name})
self.assertEqual(response.status_code, 201)
self.assertEqual(self.lifecycle.last_spec.name, name)
self.assertEqual(response.json()["name"], name)
def test_create_rejects_invalid_names_before_lifecycle_calls(self) -> None:
for name in ("", "-web", "9web", "Web", "web_name", "web.name", "two words", "a" * 64, 1, True):
with self.subTest(name=name):
response = self.client.post("/vms", headers=self.headers, json={"name": name})
self.assertEqual(response.status_code, 422)
self.assertIsNone(self.lifecycle.last_spec)
def test_create_normalizes_private_ports_and_default_ssh(self) -> None:
cases = (
({}, (PortRule(22),)),
({"ports": None}, (PortRule(22),)),
({"ports": []}, (PortRule(22),)),
(
{"ports": [{"port": 443, "protocol": "HTTPS"}, {"port": 80, "protocol": "HTTP"}]},
(PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS")),
),
(
{"ports": [{"port": 80}, {"port": 22}, {"port": 80, "protocol": "TCP"}]},
(PortRule(22), PortRule(80)),
),
({"ports": [{"port": 22, "protocol": "HTTP"}]}, (PortRule(22, "HTTP"),)),
({"ports": [{"port": 65535}, {"port": 1}]}, (PortRule(1), PortRule(22), PortRule(65535))),
({"no_default_ports": True}, ()),
({"no_default_ports": True, "ports": None}, ()),
({"no_default_ports": True, "ports": []}, ()),
(
{"no_default_ports": True, "ports": [{"port": 443, "protocol": "HTTPS"}]},
(PortRule(443, "HTTPS"),),
),
({"no_default_ports": True, "ports": [{"port": 22}]}, (PortRule(22),)),
)
for payload, expected in cases:
with self.subTest(payload=payload):
response = self.client.post("/vms", headers=self.headers, json=payload)
self.assertEqual(response.status_code, 201, response.text)
self.assertEqual(self.lifecycle.last_spec.ports, expected)
self.assertEqual(response.json()["ports"], [rule.to_dict() for rule in expected])
self.assertEqual(response.json()["policy_status"], "enforced")
def test_create_rejects_invalid_ports_before_lifecycle_calls(self) -> None:
invalid_rules = [
{"port": value}
for value in (0, -1, 65536, True, False, 22.0, 22.5, "22", None, {}, [])
] + [
{"port": 80, "protocol": value}
for value in ("UDP", "http", "HTTP ", "", None, True, 1)
] + [
{},
{"protocol": "TCP"},
{"port": 80, "public_port": 8080},
{"port": 80, "host_port": 8080},
]
payloads = [{"ports": [rule]} for rule in invalid_rules] + [
{"ports": value} for value in ("22:TCP", {}, [22], ["22:TCP"], [None])
] + [
{"ports": [{"port": 80}, {"port": 80, "protocol": "HTTP"}]},
{"ports": [{"port": 443, "protocol": "HTTP"}, {"port": 443, "protocol": "HTTPS"}]},
{"public_ports": [80]},
] + [
{"no_default_ports": value} for value in (None, 0, 1, "true", "false")
]
for payload in payloads:
with self.subTest(payload=payload):
response = self.client.post("/vms", headers=self.headers, json=payload)
self.assertEqual(response.status_code, 422, response.text)
self.assertIsNone(self.lifecycle.last_spec)
def test_list_and_get_whitelist_fields_and_preserve_observed_policy(self) -> None:
expected_fields = {
"id", "name", "cpu", "ram_mib", "guest_ip", "gateway", "mac", "username",
"status", "observed_status", "ports", "policy_status", "operation", "pid",
"created_at", "updated_at", "last_error",
}
self.lifecycle.vm.operation = "ports"
self.lifecycle.observed_status = "dead"
for ports, policy_status in (
(None, "unmanaged"),
((), "enforced"),
((PortRule(443, "HTTPS"),), "missing"),
((PortRule(22),), "unknown"),
((PortRule(22),), "drift"),
):
with self.subTest(ports=ports, policy_status=policy_status):
self.lifecycle.vm.ports = ports
self.lifecycle.policy_status = policy_status
listed = self.client.get("/vms", headers=self.headers)
detail = self.client.get("/vms/test-vm", headers=self.headers)
self.assertEqual(listed.status_code, 200)
self.assertEqual(detail.status_code, 200)
self.assertIsInstance(listed.json(), list)
self.assertEqual(listed.json(), [detail.json()])
body = detail.json()
self.assertEqual(set(body), expected_fields)
self.assertEqual(body["name"], "test-vm")
self.assertEqual(body["status"], "running")
self.assertEqual(body["observed_status"], "dead")
self.assertEqual(body["policy_status"], policy_status)
self.assertEqual(body["operation"], "ports")
self.assertEqual(body["ports"], None if ports is None else [p.to_dict() for p in ports])
self.assertNotIn("guest-secret", detail.text)
def test_get_delegates_id_ip_and_name_resolution_to_inspect(self) -> None:
with (
patch.object(self.lifecycle, "inspect", wraps=self.lifecycle.inspect) as inspect,
patch.object(self.lifecycle, "list_vms", side_effect=AssertionError("must use inspect")),
):
for identifier in ("vm-test", "10.42.0.2", "test-vm"):
with self.subTest(identifier=identifier):
response = self.client.get(f"/vms/{identifier}", headers=self.headers)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json()["id"], "vm-test")
inspect.assert_called_with(identifier)
def test_stop_and_destroy_pass_identifiers_to_lifecycle(self) -> None:
with (
patch.object(self.lifecycle, "stop", wraps=self.lifecycle.stop) as stop,
patch.object(self.lifecycle, "destroy", wraps=self.lifecycle.destroy) as destroy,
):
for identifier in ("vm-test", "10.42.0.2", "test-vm"):
with self.subTest(identifier=identifier):
stopped = self.client.post(f"/vms/{identifier}/stop", headers=self.headers)
self.assertEqual(stopped.status_code, 200)
self.assertEqual(stopped.json()["id"], "vm-test")
self.assertEqual(stopped.json()["policy_status"], "inactive")
self.assertEqual(stopped.json()["ports"], [{"port": 22, "protocol": "TCP"}])
stop.assert_called_with(identifier)
destroyed = self.client.delete(f"/vms/{identifier}", headers=self.headers)
self.assertEqual(destroyed.status_code, 200)
self.assertEqual(destroyed.json()["id"], "vm-test")
destroy.assert_called_with(identifier)
def test_create_claims_enforcement_only_for_successful_managed_vms(self) -> None:
for ports, status, operation, policy_status in (
(None, "running", None, "unmanaged"),
((), "running", None, "enforced"),
((PortRule(22),), "failed", None, "unknown"),
((PortRule(22),), "starting", "create", "unknown"),
((PortRule(22),), "running", "ports", "unknown"),
):
with self.subTest(ports=ports, status=status, operation=operation):
self.lifecycle.vm.ports = ports
self.lifecycle.vm.status = status
self.lifecycle.vm.operation = operation
with patch.object(self.lifecycle, "create", return_value=self.lifecycle.vm):
response = self.client.post("/vms", headers=self.headers, json={})
self.assertEqual(response.status_code, 201)
self.assertEqual(response.json()["policy_status"], policy_status)
def test_lifecycle_errors_use_typed_status_codes(self) -> None:
for method, path, operation in (
("GET", "/vms", "list_vms"),
("GET", "/vms/test-vm", "inspect"),
("POST", "/vms", "create"),
("POST", "/vms/test-vm/stop", "stop"),
("DELETE", "/vms/test-vm", "destroy"),
):
for error, expected in (
(NotFoundError("unknown guest"), 404),
(ConflictError("ambiguous name"), 409),
(ValidationError("invalid request"), 422),
(ConfigurationError("invalid settings"), 422),
(UvmError("this operation needs root"), 403),
(UvmError("host operation failed"), 500),
):
with self.subTest(operation=operation, error=error):
with patch.object(self.lifecycle, operation, side_effect=error):
response = self.client.request(method, path, headers=self.headers, json={})
self.assertEqual(response.status_code, expected)
self.assertEqual(response.json(), {"error": {"message": str(error)}})
def test_missing_vm_returns_not_found(self) -> None:
response = self.client.get("/vms/missing", headers=self.headers)
self.assertEqual(response.status_code, 404)
self.assertEqual(response.json(), {"error": {"message": "VM not found: missing"}})
def test_api_schema_keeps_only_existing_operations(self) -> None:
schema = self.client.get("/openapi.json").json()
self.assertEqual(
{(path, method) for path, methods in schema["paths"].items() for method in methods},
{
("/health", "get"),
("/install", "post"),
("/vms", "get"),
("/vms", "post"),
("/vms/{identifier}", "get"),
("/vms/{vm_id}/stop", "post"),
("/vms/{vm_id}", "delete"),
},
)
self.assertEqual(
set(schema["components"]["schemas"]["PortRuleModel"]["properties"]),
{"port", "protocol"},
)
def test_install_route_uses_the_existing_installer(self) -> None:
response = self.client.post(
"/install",
headers={"X-UVM-Token": "test-token"},
json={"force": True},
)
response = self.client.post("/install", headers=self.headers, json={"force": True})
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json()["firecracker"], "/tmp/firecracker")
tests/test_state.py
+220 -11
View File
@@ -9,8 +9,8 @@ from pathlib import Path
from unittest.mock import patch
from uvm.config import Settings
from uvm.domain import VmRecord
from uvm.errors import StateError
from uvm.domain import STATE_SCHEMA_VERSION, PortRule, State, VmRecord, VmSpec, new_vm_id
from uvm.errors import ConflictError, NotFoundError, StateError
from uvm.state import StateStore
@@ -61,26 +61,103 @@ class StateStoreTests(unittest.TestCase):
def test_legacy_state_is_loaded_and_migrated_on_next_write(self) -> None:
self.base.mkdir(parents=True)
legacy = {"vms": {"vm-test": make_vm().to_dict()}}
legacy["vms"]["vm-test"].pop("disk")
legacy["vms"]["vm-test"].pop("username")
legacy["vms"]["vm-test"].pop("password")
legacy["vms"]["vm-test"].pop("updated_at")
for version in (None, 1):
with self.subTest(version=version):
raw_vm = make_vm().to_dict()
for field in (
"disk", "username", "password", "updated_at", "name", "ports",
"disk_ready", "runtime_owned", "tap_owned", "operation", "boot_id",
"spawn_pending", "network", "bridge",
):
raw_vm.pop(field)
legacy = {"vms": {"vm-test": raw_vm}}
if version is not None:
legacy["schema_version"] = version
self.settings.state_path.write_text(json.dumps(legacy), encoding="utf-8")
loaded = self.store.load()
self.assertEqual(loaded.vms["vm-test"].disk, "")
self.assertEqual(loaded.vms["vm-test"].username, "root")
self.assertIsNone(loaded.vms["vm-test"].password)
vm = loaded.vms["vm-test"]
self.assertEqual(loaded.schema_version, 2)
self.assertEqual(vm.disk, "")
self.assertEqual(vm.username, "root")
self.assertIsNone(vm.password)
self.assertIsNone(vm.name)
self.assertIsNone(vm.ports)
self.assertIsNone(vm.network)
self.assertIsNone(vm.bridge)
self.assertFalse(vm.disk_ready)
self.assertFalse(vm.runtime_owned)
self.assertFalse(vm.tap_owned)
self.assertFalse(vm.spawn_pending)
self.assertEqual(vm.updated_at, vm.created_at)
self.assertEqual(loaded.next_mac_index, 2)
self.assertEqual(
json.loads(self.settings.state_path.read_text(encoding="utf-8")), legacy
)
with self.store.transaction() as state:
state.vms["vm-test"].status = "stopped"
persisted = json.loads(self.settings.state_path.read_text(encoding="utf-8"))
self.assertEqual(persisted["schema_version"], 1)
self.assertEqual(persisted["schema_version"], 2)
self.assertEqual(persisted["next_mac_index"], 2)
self.assertEqual(persisted["vms"]["vm-test"]["status"], "stopped")
self.assertIsNone(persisted["vms"]["vm-test"]["ports"])
def test_round_trip_preserves_policy_ownership_and_recovery_fields(self) -> None:
for ports in (None, (), (PortRule(22), PortRule(8080, "http"))):
with self.subTest(ports=ports):
vm = replace(
make_vm(), name="app-server", ports=ports, disk_ready=True,
runtime_owned=True, tap_owned=True, operation="configuring",
boot_id="host-boot-id", spawn_pending=True, pid=123,
process_start_time="456789", network="10.42.0.0/24", bridge="uvm0",
last_error="interrupted boot",
)
with self.store.transaction() as state:
state.vms[vm.id] = vm
loaded = self.store.load()
self.assertEqual(loaded.schema_version, 2)
self.assertEqual(loaded.vms[vm.id], vm)
raw = json.loads(self.settings.state_path.read_text(encoding="utf-8"))
self.assertEqual(raw["schema_version"], 2)
self.assertEqual(
raw["vms"][vm.id]["ports"],
None if ports is None else [port.to_dict() for port in ports],
)
for field in ("disk_ready", "runtime_owned", "tap_owned", "spawn_pending"):
self.assertIs(raw["vms"][vm.id][field], True)
def test_lookup_requires_full_case_sensitive_ids_names_or_ips(self) -> None:
vm = replace(make_vm("vm-0123456789abcdef0123456789abcdef"), name="app-server")
with self.store.transaction() as state:
state.vms[vm.id] = vm
for identifier in (vm.id, vm.name, vm.guest_ip):
with self.subTest(identifier=identifier):
self.assertEqual(self.store.find_by_id_or_ip(identifier), vm)
self.assertEqual(self.store.get(vm.id), vm)
for identifier in (
"", vm.id[:15], vm.id.upper(), "app", "APP-SERVER", " app-server ", "10.42.0",
):
with self.subTest(identifier=identifier), self.assertRaises(NotFoundError):
self.store.find_by_id_or_ip(identifier)
for identifier in (vm.name, vm.guest_ip):
with self.subTest(get_identifier=identifier), self.assertRaises(NotFoundError):
self.store.get(identifier)
def test_load_rejects_a_record_whose_id_does_not_match_its_key(self) -> None:
self.base.mkdir(parents=True)
for version in (0, 1, 2):
with self.subTest(version=version):
raw = {
"schema_version": version,
"vms": {"vm-wrong-key": make_vm("vm-record-id").to_dict()},
}
self.settings.state_path.write_text(json.dumps(raw), encoding="utf-8")
with self.assertRaisesRegex(StateError, "ID.*registry key"):
self.store.load()
def test_initialize_protects_registry_credentials(self) -> None:
self.base.mkdir(parents=True)
@@ -110,3 +187,135 @@ class StateStoreTests(unittest.TestCase):
self.assertRaisesRegex(StateError, "Run this command with sudo"),
):
self.store.load()
class DomainStateTests(unittest.TestCase):
def test_new_specs_default_to_ssh_without_overriding_an_explicit_empty_policy(self) -> None:
self.assertEqual(VmSpec(cpu=1, ram_mib=512).ports, (PortRule(22),))
self.assertEqual(VmSpec(cpu=1, ram_mib=512, ports=()).ports, ())
self.assertIsNone(make_vm().ports)
def test_port_rule_serialization_normalizes_labels_and_defaults_to_tcp(self) -> None:
for port, protocol in ((1, "tcp"), (8080, "hTtP"), (65535, "https")):
with self.subTest(port=port, protocol=protocol):
rule = PortRule.from_dict({"port": port, "protocol": protocol})
self.assertEqual(rule.to_dict(), {"port": port, "protocol": protocol.upper()})
self.assertEqual(PortRule.from_dict(rule.to_dict()), rule)
self.assertEqual(PortRule.from_dict({"port": 22}), PortRule(22, "TCP"))
def test_port_rule_deserialization_rejects_invalid_types_fields_and_values(self) -> None:
invalid = (
None, True, 22, "22", [], (), {}, {"protocol": "TCP"},
{"port": True}, {"port": False}, {"port": None}, {"port": "22"},
{"port": 22.0}, {"port": 0}, {"port": 65536},
{"port": 22, "protocol": None}, {"port": 22, "protocol": True},
{"port": 22, "protocol": "UDP"}, {"port": 22, "protocol": " TCP"},
{"port": 22, "type": "TCP"}, {"port": 22, "protocol": "TCP", "extra": 1},
)
for value in invalid:
with self.subTest(value=value), self.assertRaises(StateError):
PortRule.from_dict(value)
def test_persisted_ports_must_be_a_json_list_or_legacy_null(self) -> None:
for ports in (True, False, 22, "22", {}, (), (PortRule(22),)):
with self.subTest(ports=ports), self.assertRaisesRegex(StateError, "ports"):
VmRecord.from_dict({**make_vm().to_dict(), "ports": ports})
for ports in ([22], [None], ["22:TCP"], [{"port": True}]):
with self.subTest(ports=ports), self.assertRaises(StateError):
VmRecord.from_dict({**make_vm().to_dict(), "ports": ports})
def test_persisted_duplicate_and_conflicting_tcp_rules_are_rejected(self) -> None:
for labels in (("TCP", "TCP"), ("tcp", "TCP"), ("TCP", "HTTP"), ("HTTP", "HTTPS")):
with self.subTest(labels=labels):
raw = make_vm().to_dict()
raw["ports"] = [{"port": 80, "protocol": label} for label in labels]
with self.assertRaisesRegex(StateError, "duplicate TCP port"):
VmRecord.from_dict(raw)
def test_ownership_and_spawn_pending_fields_require_real_booleans(self) -> None:
for field in ("disk_ready", "runtime_owned", "tap_owned", "spawn_pending"):
for value in (None, 0, 1, 0.0, 1.0, "false", "true", [], {}):
with self.subTest(field=field, value=value):
raw = {**make_vm().to_dict(), field: value}
with self.assertRaisesRegex(StateError, f"{field} must be a boolean"):
VmRecord.from_dict(raw)
def test_explicit_recovery_flags_override_legacy_inference(self) -> None:
fields = ("disk_ready", "runtime_owned", "tap_owned", "spawn_pending")
for flag in (False, True):
for operation in (None, "spawning", "configuring"):
with self.subTest(flag=flag, operation=operation):
raw = replace(make_vm(), status="running", pid=123, operation=operation).to_dict()
raw.update(dict.fromkeys(fields, flag))
vm = VmRecord.from_dict(raw)
for field in fields:
self.assertIs(getattr(vm, field), flag)
self.assertIs(vm.to_dict()[field], flag)
def test_legacy_ownership_is_inferred_from_disk_status_and_pid(self) -> None:
cases = (
("running", "/tmp/rootfs.ext4", 123, (True, True, True)),
("stopped", "/tmp/rootfs.ext4", None, (True, True, False)),
("error", "/tmp/rootfs.ext4", 123, (False, True, True)),
("starting", "/tmp/rootfs.ext4", None, (False, True, False)),
("running", "", 123, (False, False, True)),
("unknown", "", None, (False, False, False)),
)
for status, disk, pid, expected in cases:
with self.subTest(status=status, disk=disk, pid=pid):
raw = replace(make_vm(), status=status, disk=disk, pid=pid).to_dict()
for field in ("disk_ready", "runtime_owned", "tap_owned", "ports"):
raw.pop(field)
vm = VmRecord.from_dict(raw)
self.assertEqual((vm.disk_ready, vm.runtime_owned, vm.tap_owned), expected)
self.assertIsNone(vm.ports)
def test_missing_spawn_pending_recovers_only_a_legacy_spawning_operation(self) -> None:
for operation in (None, "spawning", "configuring", "stopping", "policy"):
with self.subTest(operation=operation):
raw = replace(make_vm(), operation=operation).to_dict()
raw.pop("spawn_pending")
vm = VmRecord.from_dict(raw)
self.assertIs(vm.spawn_pending, operation == "spawning")
self.assertEqual(VmRecord.from_dict(vm.to_dict()), vm)
def test_new_state_uses_schema_two_and_rejects_future_versions(self) -> None:
self.assertEqual(State().schema_version, 2)
with self.assertRaisesRegex(StateError, "newer than"):
State.from_dict({"schema_version": STATE_SCHEMA_VERSION + 1, "vms": {}})
def test_generated_ids_have_the_reserved_uuid_format(self) -> None:
first, second = new_vm_id(), new_vm_id()
self.assertNotEqual(first, second)
for vm_id in (first, second):
self.assertRegex(vm_id, r"\Avm-[0-9a-f]{32}\Z")
def test_exact_generated_id_wins_over_a_legacy_name_collision(self) -> None:
vm = make_vm("vm-" + "a" * 32)
other = replace(make_vm("vm-" + "b" * 32), name=vm.id, guest_ip="10.42.0.3")
state = State(vms={other.id: other, vm.id: vm})
self.assertIs(StateStore.resolve(state, vm.id), vm)
del state.vms[vm.id]
with self.assertRaises(NotFoundError):
StateStore.resolve(state, vm.id)
def test_ambiguous_names_ips_and_legacy_ids_require_an_exact_id(self) -> None:
cases = (
("vm-" + "a" * 32, "app", "app", "10.42.0.3", "app"),
("vm-" + "a" * 32, "app", "other", "10.42.0.2", "10.42.0.2"),
("vm-" + "a" * 32, "app", "10.42.0.2", "10.42.0.3", "10.42.0.2"),
("vm-legacy", "app", "vm-legacy", "10.42.0.3", "vm-legacy"),
)
for vm_id, name, other_name, other_ip, identifier in cases:
with self.subTest(identifier=identifier, other_name=other_name):
vm = replace(make_vm(vm_id), name=name)
other = replace(make_vm("vm-" + "b" * 32), name=other_name, guest_ip=other_ip)
for records in ((vm, other), (other, vm)):
state = State(vms={record.id: record for record in records})
with self.assertRaisesRegex(ConflictError, "ambiguous.*exact VM ID"):
StateStore.resolve(state, identifier)
self.assertIs(StateStore.resolve(state, other.id), other)
def test_multiple_matching_fields_on_one_vm_are_not_ambiguous(self) -> None:
vm = replace(make_vm("vm-legacy"), name="vm-legacy")
self.assertIs(StateStore.resolve(State(vms={vm.id: vm}), "vm-legacy"), vm)
tests/test_system.py
+67 -14
View File
@@ -1,5 +1,7 @@
from __future__ import annotations
import io
import subprocess
import unittest
from types import SimpleNamespace
from unittest.mock import patch
@@ -9,9 +11,41 @@ from uvm.system import CommandRunner
class CommandRunnerTests(unittest.TestCase):
def test_uncaptured_child_stdout_is_redirected_to_current_stderr(self) -> None:
stderr = io.StringIO()
completed = SimpleNamespace(returncode=0, stdout=None, stderr=None)
with (
patch("uvm.system.sys.stderr", stderr),
patch("uvm.system.subprocess.run", return_value=completed) as run,
):
result = CommandRunner(emit=None).run(("host-tool", "--check"), timeout=2.5)
run.assert_called_once_with(
("host-tool", "--check"), check=False, text=True, input=None,
stdout=stderr, stderr=None, timeout=2.5,
)
self.assertEqual(result.args, ("host-tool", "--check"))
self.assertEqual(result.returncode, 0)
self.assertEqual(result.stdout, "")
self.assertEqual(result.stderr, "")
def test_captured_stdout_and_stderr_still_use_separate_pipes(self) -> None:
completed = SimpleNamespace(returncode=0, stdout="result\n", stderr="diagnostic\n")
with patch("uvm.system.subprocess.run", return_value=completed) as run:
result = CommandRunner(emit=None).run(("host-tool",), capture=True)
self.assertEqual(run.call_args.kwargs["stdout"], subprocess.PIPE)
self.assertEqual(run.call_args.kwargs["stderr"], subprocess.PIPE)
self.assertEqual(result.stdout, "result\n")
self.assertEqual(result.stderr, "diagnostic\n")
def test_passes_sensitive_input_over_stdin_without_logging_it(self) -> None:
emitted: list[str] = []
completed = SimpleNamespace(returncode=0, stdout="result\n", stderr="")
completed = SimpleNamespace(
returncode=0, stdout="result secret-value\n", stderr="warning secret-value\n",
)
with patch("uvm.system.subprocess.run", return_value=completed) as run:
result = CommandRunner(emit=emitted.append).run(
@@ -21,26 +55,45 @@ class CommandRunnerTests(unittest.TestCase):
sensitive=True,
)
self.assertEqual(result.stdout, "result\n")
self.assertEqual(result.stdout, "result secret-value\n")
self.assertEqual(result.stderr, "warning secret-value\n")
self.assertEqual(run.call_args.kwargs["stdout"], subprocess.PIPE)
self.assertEqual(run.call_args.kwargs["stderr"], subprocess.PIPE)
self.assertEqual(run.call_args.kwargs["input"], "secret-value\n")
self.assertNotIn("secret-value", emitted[0])
self.assertEqual(emitted, ["+ credential-tool --stdin"])
def test_redacts_sensitive_command_output_from_errors(self) -> None:
completed = SimpleNamespace(
returncode=1,
stdout="",
stderr="failure mentioning secret-value",
)
for stdout, stderr in (
("failure mentioning secret-value", ""),
("", "failure mentioning secret-value"),
("secret-value", "failure mentioning secret-value"),
):
with self.subTest(stdout=stdout, stderr=stderr):
completed = SimpleNamespace(returncode=1, stdout=stdout, stderr=stderr)
emitted: list[str] = []
with (
patch("uvm.system.subprocess.run", return_value=completed),
self.assertRaises(CommandError) as raised,
):
CommandRunner(emit=None).run(
("credential-tool", "--stdin"),
capture=True,
input_text="secret-value\n",
sensitive=True,
CommandRunner(emit=emitted.append).run(
("credential-tool", "--stdin"), capture=True,
input_text="secret-value\n", sensitive=True,
)
self.assertNotIn("secret-value", str(raised.exception))
self.assertEqual(emitted, ["+ credential-tool --stdin"])
def test_unchecked_sensitive_failure_preserves_captured_output_for_the_caller(self) -> None:
completed = SimpleNamespace(
returncode=1, stdout="secret-value\n", stderr="failure mentioning secret-value",
)
with patch("uvm.system.subprocess.run", return_value=completed):
result = CommandRunner(emit=None).run(
("credential-tool", "--stdin"), check=False, capture=True,
input_text="secret-value\n", sensitive=True,
)
self.assertEqual(result.returncode, 1)
self.assertEqual(result.stdout, completed.stdout)
self.assertEqual(result.stderr, completed.stderr)
tests/test_validation.py
+105 -1
View File
@@ -2,8 +2,11 @@ from __future__ import annotations
import unittest
from uvm.domain import PortRule
from uvm.errors import ValidationError
from uvm.validation import parse_cpu, parse_password, parse_ram, parse_username
from uvm.validation import (
parse_cpu, parse_name, parse_password, parse_ports, parse_ram, parse_timeout, parse_username,
)
class ValidationTests(unittest.TestCase):
@@ -43,3 +46,104 @@ class ValidationTests(unittest.TestCase):
with self.subTest(value=value):
with self.assertRaises(ValidationError):
parse_password(value)
def test_names_accept_full_lowercase_names_up_to_63_characters(self) -> None:
for name in ("a", "app-server", "worker2", "a" * 63, "vm-test", "vm-" + "g" * 32):
with self.subTest(name=name):
self.assertEqual(parse_name(name), name)
def test_names_reject_invalid_format_case_and_length(self) -> None:
for name in (
"", "App", "APP", "9app", "-app", "app_name", "app.name", "app/name",
" app", "app ", "app\n", "a" * 64, "caf\u00e9", "app;id",
):
with self.subTest(name=name), self.assertRaises(ValidationError):
parse_name(name)
def test_names_cannot_impersonate_a_generated_vm_id(self) -> None:
for suffix in ("0" * 32, "0123456789abcdef" * 2, "f" * 32):
with self.subTest(suffix=suffix):
with self.assertRaisesRegex(ValidationError, "impersonate generated VM IDs"):
parse_name("vm-" + suffix)
for suffix in ("f" * 31, "f" * 33):
self.assertEqual(parse_name("vm-" + suffix), "vm-" + suffix)
def test_ports_accept_repeated_comma_lists_whitespace_and_case_insensitive_labels(self) -> None:
values = ("443:https,22", " 80:HtTp ,\t8080:tcp ", "65535, 1")
expected = (
PortRule(1), PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS"),
PortRule(8080), PortRule(65535),
)
self.assertEqual(parse_ports(values), expected)
self.assertEqual(parse_ports(list(values)), expected)
def test_bare_ports_are_tcp_and_empty_sequences_do_not_add_defaults(self) -> None:
self.assertEqual(parse_ports(("80,443",)), (PortRule(80, "TCP"), PortRule(443, "TCP")))
self.assertEqual(parse_ports([]), ())
self.assertEqual(parse_ports(()), ())
def test_identical_port_labels_are_deduplicated_across_options_and_comma_lists(self) -> None:
self.assertEqual(
parse_ports(("22", "22:tcp", "80:HTTP,80:hTtP", "443:HTTPS", "443:https")),
(PortRule(22), PortRule(80, "HTTP"), PortRule(443, "HTTPS")),
)
def test_conflicting_tcp_service_labels_are_rejected(self) -> None:
for values in (
("80", "80:HTTP"), ("80:HTTP,80:TCP",),
("80:HTTP", "80:HTTPS"), ("80:https,80:http",),
):
with self.subTest(values=values):
with self.assertRaisesRegex(ValidationError, "conflicting labels for TCP port 80"):
parse_ports(values)
def test_ports_require_strict_port_colon_label_syntax(self) -> None:
for value in (
"", ",", "22,", ",22", "22,,80", "22, ,80", "TCP", "HTTP", "HTTPS",
"TCP:80", "80/tcp", ":HTTP", "80:", "80::HTTP", "80 :HTTP", "80: HTTP",
"80:HTTP S", "80:UDP", "80-81", "80.0", "+80", "0x50", "22 80", "80;id",
):
with self.subTest(value=value), self.assertRaises(ValidationError):
parse_ports((value,))
def test_parsed_ports_reject_out_of_range_numbers_and_non_integer_text(self) -> None:
for value in ("-1", "0", "65536", "999999999999", "True", "False", "None", "1.5"):
with self.subTest(value=value), self.assertRaises(ValidationError):
parse_ports((value,))
def test_ports_reject_scalar_strings_instead_of_splitting_them_into_digits(self) -> None:
for values in ("22", "", b"22"):
with self.subTest(values=values), self.assertRaises(ValidationError):
parse_ports(values)
def test_ports_require_a_sequence_not_a_scalar_mapping_or_set(self) -> None:
for values in (None, True, 22, {"22": "HTTP"}, {"22"}):
with self.subTest(values=values), self.assertRaises(ValidationError):
parse_ports(values)
def test_ports_require_string_sequence_members(self) -> None:
for value in (None, True, False, 22, 22.0, b"22", [], {}, PortRule(22)):
with self.subTest(value=value), self.assertRaises(ValidationError):
parse_ports(("80:HTTP", value))
def test_port_rules_require_real_integers_within_tcp_port_bounds(self) -> None:
self.assertEqual(PortRule(1).protocol, "TCP")
self.assertEqual(PortRule(65535, "https").protocol, "HTTPS")
for port in (None, True, False, 0, -1, 65536, 22.0, "22", b"22", [], {}):
with self.subTest(port=port), self.assertRaises(ValidationError):
PortRule(port)
def test_port_rule_labels_are_normalized_but_not_coerced_or_trimmed(self) -> None:
self.assertEqual(PortRule(8080, "hTtP"), PortRule(8080, "HTTP"))
for protocol in (None, True, 1, "", "UDP", "SCTP", " TCP", "TCP ", "tcp\n", [], {}):
with self.subTest(protocol=protocol), self.assertRaises(ValidationError):
PortRule(22, protocol)
def test_timeout_accepts_zero_and_fractional_seconds(self) -> None:
self.assertEqual(parse_timeout("0"), 0)
self.assertEqual(parse_timeout("0.5"), 0.5)
def test_timeout_rejects_negative_non_finite_or_invalid_values(self) -> None:
for value in ("-1", "nan", "inf", "-inf", "", "seconds", None):
with self.subTest(value=value), self.assertRaises(ValidationError):
parse_timeout(value)
uvm/api_models.py
+37 -9
View File
@@ -3,8 +3,9 @@
from __future__ import annotations
from ipaddress import IPv4Address
from typing import Literal
from pydantic import BaseModel, Field, SecretStr
from pydantic import BaseModel, ConfigDict, Field, SecretStr
from .domain import VmRecord
@@ -14,10 +15,9 @@ class HealthResponse(BaseModel):
class InstallRequest(BaseModel):
force: bool = False
model_config = ConfigDict(extra="forbid")
class Config:
extra = "forbid"
force: bool = False
class InstallResponse(BaseModel):
@@ -26,19 +26,31 @@ class InstallResponse(BaseModel):
rootfs: str
class PortRuleModel(BaseModel):
"""A private guest TCP port; HTTP and HTTPS are descriptive labels only."""
model_config = ConfigDict(extra="forbid")
port: int = Field(strict=True, ge=1, le=65535)
protocol: Literal["TCP", "HTTP", "HTTPS"] = "TCP"
class VmCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
cpu: float = Field(default=1, gt=0)
ram: str | int | float = "512"
guest_ip: IPv4Address | None = None
username: str = "root"
password: SecretStr = SecretStr("root")
class Config:
extra = "forbid"
name: str | None = None
ports: list[PortRuleModel] | None = None
no_default_ports: bool = Field(default=False, strict=True)
class VmResponse(BaseModel):
id: str
name: str | None
cpu: float
ram_mib: int
guest_ip: str
@@ -47,6 +59,9 @@ class VmResponse(BaseModel):
username: str
status: str
observed_status: str
ports: list[PortRuleModel] | None
policy_status: str
operation: str | None
pid: int | None
created_at: int
updated_at: int
@@ -58,9 +73,15 @@ class DestroyResponse(BaseModel):
status: str = "terminated"
def vm_response(vm: VmRecord, *, observed_status: str | None = None) -> VmResponse:
def vm_response(
vm: VmRecord,
*,
observed_status: str | None = None,
policy_status: str = "unknown",
) -> VmResponse:
return VmResponse(
id=vm.id,
name=vm.name,
cpu=vm.cpu,
ram_mib=vm.ram_mib,
guest_ip=vm.guest_ip,
@@ -68,7 +89,14 @@ def vm_response(vm: VmRecord, *, observed_status: str | None = None) -> VmRespon
mac=vm.mac,
username=vm.username,
status=vm.status,
observed_status=observed_status or vm.status,
observed_status=observed_status if observed_status is not None else vm.status,
ports=(
[PortRuleModel(port=rule.port, protocol=rule.protocol) for rule in vm.ports]
if vm.ports is not None
else None
),
policy_status=policy_status,
operation=vm.operation,
pid=vm.pid,
created_at=vm.created_at,
updated_at=vm.updated_at,
uvm/cli.py
+484 -33
View File
@@ -3,22 +3,54 @@
from __future__ import annotations
import argparse
import codecs
import json
import os
import sys
import time
from collections.abc import Sequence
from ipaddress import IPv4Address
from pathlib import Path
from .app import Application, build_application
from .domain import VmSpec
from .domain import PortRule, VmRecord, VmSpec
from .errors import UvmError, ValidationError
from .firecracker.config import vcpu_count
from .validation import parse_cpu, parse_password, parse_ram, parse_username
from .validation import (
parse_cpu,
parse_name,
parse_password,
parse_ports,
parse_ram,
parse_timeout,
parse_username,
)
class _CommandParser(argparse.ArgumentParser):
def __init__(self, *args, ssh_command: bool = False, **kwargs) -> None:
super().__init__(*args, allow_abbrev=False, **kwargs)
self._ssh_command = ssh_command
def parse_known_args(self, args=None, namespace=None):
if not self._ssh_command:
return super().parse_known_args(args, namespace)
values = list(sys.argv[1:] if args is None else args)
remote_command = []
if "--" in values:
separator = values.index("--")
values, remote_command = values[:separator], values[separator + 1 :]
# REMAINDER would swallow SSH options placed after the VM identifier.
parsed, unknown = super().parse_known_args(values, namespace)
parsed.remote_command = remote_command
return parsed, unknown
def build_parser(application: Application) -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog=application.settings.app_name,
description="Tiny Firecracker microVM CLI for Ubuntu.",
allow_abbrev=False,
)
parser.add_argument(
"--serve",
@@ -36,7 +68,7 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
default=8000,
help="server bind port used with --serve; defaults to 8000",
)
subcommands = parser.add_subparsers(dest="command")
subcommands = parser.add_subparsers(dest="command", parser_class=_CommandParser)
install = subcommands.add_parser(
"install",
@@ -53,7 +85,8 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
help="redownload Firecracker, kernel, and rootfs even when local assets already exist",
)
create = subcommands.add_parser("create", help="create and boot a microVM")
create = subcommands.add_parser("create", help="create and boot a microVM detached")
create.add_argument("--name", help="unique local VM name")
create.add_argument(
"--cpu",
default=str(application.settings.default_vcpu),
@@ -65,6 +98,7 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
help="RAM, e.g. 512, 1G, 512M",
)
create.add_argument(
"--guest-ip",
"--host-ip",
dest="host_ip",
help="guest IP to assign; otherwise uvm allocates one",
@@ -74,36 +108,133 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
default=application.settings.default_ssh_user,
help="existing guest account to configure; defaults to root",
)
create.add_argument(
credentials = create.add_mutually_exclusive_group()
credentials.add_argument(
"--password",
default="root",
help="guest login password; defaults to root",
)
credentials.add_argument(
"--password-stdin",
action="store_true",
help="read the guest password from one line of stdin",
)
port_help = (
"private guest PORT[:TYPE], e.g. 22,80:HTTP,443:HTTPS; "
"repeat or separate with commas (TYPE: HTTP, HTTPS, TCP; default: TCP); "
"no host-public ports, forwarding, or proxy"
)
create.add_argument("--ports", action="append", default=[], help=port_help)
create.add_argument(
"--no-default-ports", action="store_true", help="do not add the default 22:TCP rule"
)
create.add_argument("--json", action="store_true")
subcommands.add_parser("list", help="list VMs")
listing = subcommands.add_parser("list", help="list VMs")
listing.add_argument("--status", action="append", default=[], help="observed status (repeatable)")
output = listing.add_mutually_exclusive_group()
output.add_argument("--json", action="store_true")
output.add_argument("-q", "--quiet", action="store_true", help="print only VM IDs")
ssh = subcommands.add_parser("ssh", help="SSH into a VM")
ssh.add_argument("vm", help="VM ID or guest IP")
inspect = subcommands.add_parser("inspect", help="inspect one VM")
inspect.add_argument("vm", help="VM ID, name, or guest IP")
inspect.add_argument("--json", action="store_true")
for action in ("start", "stop", "restart", "delete"):
command = subcommands.add_parser(
action,
aliases=["destroy"] if action == "delete" else [],
help=f"{action} selected VMs",
)
targets = command.add_mutually_exclusive_group(required=True)
targets.add_argument(
"vms", nargs="*", default=[], metavar="VM", help="VM IDs, names, or guest IPs"
)
targets.add_argument("--all", dest="all_vms", action="store_true", help="select all VMs")
command.add_argument("--yes", "-y", action="store_true", help="skip confirmation")
command.add_argument("--dry-run", action="store_true", help="show targets without changes")
command.add_argument("--json", action="store_true")
if action != "start":
command.add_argument(
"--timeout",
help=(
"seconds to wait after SIGTERM before SIGKILL "
"(defaults to the configured timeout, normally 5s); "
"does not coordinate guest shutdown"
),
)
ssh = subcommands.add_parser(
"ssh",
ssh_command=True,
help="SSH into a VM",
usage="%(prog)s [options] VM [-- COMMAND ...]",
epilog="Append a remote command after a literal --, e.g. ssh VM -- uname -a.",
)
ssh.add_argument("vm", help="VM ID, name, or guest IP")
ssh.add_argument("--user", help="override the username stored for the VM")
ssh.add_argument("--key")
ssh.add_argument("--port", type=_port_number, default=22, help="guest SSH port (default: 22)")
ssh.add_argument(
"--insecure-host-key",
action="store_true",
help="disable SSH host-key verification for this connection",
)
stop = subcommands.add_parser("stop", help="stop a VM")
stop.add_argument("vm")
logs = subcommands.add_parser("logs", help="read a VM's local Firecracker log")
logs.add_argument("vm", help="VM ID, name, or guest IP")
logs.add_argument("--tail", type=_line_count, default=100, metavar="N")
logs.add_argument("-f", "--follow", action="store_true")
destroy = subcommands.add_parser("destroy", help="stop and remove a VM")
destroy.add_argument("vm")
ports = subcommands.add_parser(
"ports",
help="manage enforced private-IP guest ports (not public ports)",
description=(
"Manage private-IP TCP allowlists. HTTP/HTTPS are TCP labels, not proxies "
"or TLS configuration. No host-public ports or NAT rules are created."
),
)
port_commands = ports.add_subparsers(dest="ports_command", required=True)
for action in ("list", "open", "close", "set", "sync"):
command = port_commands.add_parser(action, help=f"{action} private-IP port rules")
command.add_argument("--json", action="store_true")
if action in {"list", "sync"}:
targets = command.add_mutually_exclusive_group(required=True)
targets.add_argument(
"vms" if action == "sync" else "vm",
nargs="*" if action == "sync" else "?",
default=[] if action == "sync" else None,
metavar="VM",
help="VM ID, name, or guest IP",
)
targets.add_argument("--all", dest="all_vms", action="store_true")
else:
command.add_argument("vm", help="VM ID, name, or guest IP")
if action == "open":
command.add_argument("--ports", action="append", required=True, help=port_help)
else:
selection = command.add_mutually_exclusive_group(required=True)
selection.add_argument("--ports", action="append", default=[], help=port_help)
if action == "close":
selection.add_argument(
"--all", dest="all_ports", action="store_true", help="close all ports of this VM"
)
else:
selection.add_argument("--none", action="store_true", help="deny all inbound ports")
if action != "list":
command.add_argument("--dry-run", action="store_true", help="show changes without applying")
if action in {"close", "set", "sync"}:
command.add_argument("--yes", "-y", action="store_true", help="skip confirmation")
doctor = subcommands.add_parser("doctor", help="check local host readiness and policy health")
doctor.add_argument("--json", action="store_true")
return parser
def main(argv: Sequence[str] | None = None) -> int:
try:
application = build_application()
application = build_application(emit=lambda message: print(message, file=sys.stderr))
parser = build_parser(application)
args = parser.parse_args(argv)
if args.serve:
@@ -113,9 +244,15 @@ def main(argv: Sequence[str] | None = None) -> int:
if args.command is None:
parser.error("a command is required unless --serve is used")
return _run_command(application, args)
except ValidationError as error:
print(f"uvm: error: {error}", file=sys.stderr)
return 2
except UvmError as error:
print(f"uvm: error: {error}", file=sys.stderr)
return error.exit_code
except OSError as error:
print(f"uvm: error: {error}", file=sys.stderr)
return 1
except KeyboardInterrupt:
print("uvm: interrupted", file=sys.stderr)
return 130
@@ -146,54 +283,125 @@ def _run_command(application: Application, args: argparse.Namespace) -> int:
if args.command == "create":
guest_ip = _parse_guest_ip(args.host_ip)
ports = parse_ports(args.ports)
if not args.no_default_ports and not any(rule.port == 22 for rule in ports):
ports = tuple(sorted((*ports, PortRule(22)), key=lambda rule: rule.port))
password = args.password
if args.password_stdin:
password = sys.stdin.readline(131).removesuffix("\n").removesuffix("\r")
vm = application.lifecycle.create(
VmSpec(
cpu=parse_cpu(args.cpu),
ram_mib=parse_ram(args.ram),
guest_ip=guest_ip,
username=parse_username(args.username),
password=parse_password(args.password),
password=parse_password(password),
name=parse_name(args.name) if args.name is not None else None,
ports=ports,
)
)
if vm.cpu < 1.0:
print(
f"NOTE: requested {vm.cpu} CPU. Firecracker uses {vcpu_count(vm.cpu)} vCPU;"
f"NOTE: requested {vm.cpu} CPU. Firecracker uses {vcpu_count(vm.cpu)} vCPU; "
"fractional CPU enforcement is not yet applied.",
file=sys.stderr,
)
print(" fractional CPU enforcement is not yet applied by this MVP.")
print()
if password == "root":
print(
"WARNING: the guest is using the default password 'root'. Change it promptly.",
file=sys.stderr,
)
if args.json:
_print_json(_vm_output(
vm, policy_status="unmanaged" if vm.ports is None else "enforced"
))
else:
print(f"VM created: {vm.id}")
if vm.name is not None:
print(f" Name: {vm.name}")
print(f" IP: {vm.guest_ip}")
print(f" RAM: {vm.ram_mib} MiB")
print(f" CPU: {vm.cpu}")
print(f" TAP: {vm.tap}")
print(f" Username: {vm.username}")
print(f" Private ports: {_format_ports(vm.ports)}")
if any(rule.port == 22 for rule in vm.ports or ()):
print(f" SSH: ssh {vm.username}@{vm.guest_ip}")
print()
if vm.password == "root":
print("WARNING: the guest is using the default password 'root'. Change it promptly.")
return 0
if args.command == "list":
vms = application.lifecycle.list_vms()
vms = [
listed for listed in application.lifecycle.list_vms()
if not args.status or listed.observed_status in args.status
]
if args.json:
_print_json([
_vm_output(listed.vm, listed.observed_status, listed.policy_status)
for listed in vms
])
return 0
if args.quiet:
for listed in vms:
print(listed.vm.id)
return 0
if not vms:
print("No VMs.")
return 0
print(
f"{'ID':<37} {'IP':<16} {'USER':<16} {'CPU':<7} "
f"{'RAM':<8} {'STATUS':<10} PID"
f"{'ID':<37} {'NAME':<20} {'IP':<16} {'USER':<16} {'CPU':<7} "
f"{'RAM':<8} {'STATUS':<12} {'POLICY':<12} PID"
)
for listed in vms:
vm = listed.vm
pid = vm.pid if vm.pid is not None else ""
print(
f"{vm.id:<37} {vm.guest_ip:<16} {vm.username:<16} {vm.cpu:<7} "
f"{vm.ram_mib:<8} {listed.observed_status:<10} {pid}"
f"{vm.id:<37} {vm.name or '-':<20} {vm.guest_ip:<16} {vm.username:<16} {vm.cpu:<7} "
f"{vm.ram_mib:<8} {listed.observed_status:<12} {listed.policy_status:<12} {pid}"
)
return 0
if args.command == "inspect":
listed = application.lifecycle.inspect(args.vm)
data = _vm_output(listed.vm, listed.observed_status, listed.policy_status)
if args.json:
_print_json(data)
else:
for key, value in data.items():
if key == "ports":
value = _format_ports(listed.vm.ports)
print(f"{key}: {value if value is not None else '-'}")
return 0
if args.command in {"start", "stop", "restart", "delete", "destroy"} or (
args.command == "ports" and args.ports_command == "sync"
):
action = "ports-sync" if args.command == "ports" else args.command
if action == "destroy":
action = "delete"
timeout = None
if action in {"stop", "restart", "delete"}:
timeout = parse_timeout(
args.timeout if args.timeout is not None else str(application.settings.terminate_timeout_s)
)
targets = application.lifecycle.resolve_targets(args.vms, all_vms=args.all_vms)
vm_ids = tuple(vm.id for vm in targets)
if args.dry_run:
_print_preview(action, targets, as_json=args.json)
return 0
if not targets:
return _print_results([], as_json=args.json)
if action == "delete" or args.all_vms:
if not _confirm(f"{action} VMs: {', '.join(vm_ids)}?", yes=args.yes):
if args.json:
_print_json([])
return 0
results = application.lifecycle.run_many(action, vm_ids, timeout=timeout)
return _print_results(results, as_json=args.json)
if args.command == "ssh":
user = parse_username(args.user) if args.user is not None else None
vm = application.lifecycle.find_for_ssh(args.vm)
command = ["ssh"]
command = ["ssh", "-p", str(args.port)]
if args.key:
command.extend(("-i", args.key))
if args.insecure_host_key:
@@ -214,23 +422,266 @@ def _run_command(application: Application, args: argparse.Namespace) -> int:
"StrictHostKeyChecking=accept-new",
)
)
command.append(f"{args.user or vm.username}@{vm.guest_ip}")
command.append(f"{user or vm.username}@{vm.guest_ip}")
command.extend(args.remote_command)
os.execvp(command[0], command)
return 0
if args.command == "stop":
vm = application.lifecycle.stop(args.vm)
print(f"Stopped {vm.id}")
if args.command == "logs":
_show_logs(application.lifecycle.log_path(args.vm), args.tail, follow=args.follow)
return 0
if args.command == "destroy":
vm = application.lifecycle.destroy(args.vm)
print(f"Destroyed {vm.id}")
if args.command == "ports":
mode = args.ports_command
if mode == "list":
listed_vms = (
application.lifecycle.list_vms() if args.all_vms
else [application.lifecycle.inspect(args.vm)]
)
if args.json:
_print_json([
_vm_output(listed.vm, listed.observed_status, listed.policy_status)
for listed in listed_vms
])
elif not listed_vms:
print("No VMs.")
else:
print(f"{'ID':<37} {'NAME':<20} {'IP':<16} {'POLICY':<12} PRIVATE-IP PORTS")
for listed in listed_vms:
vm = listed.vm
print(
f"{vm.id:<37} {vm.name or '-':<20} {vm.guest_ip:<16} "
f"{listed.policy_status:<12} {_format_ports(vm.ports)}"
)
return 0
requested = parse_ports(args.ports)
listed = application.lifecycle.inspect(args.vm)
vm = listed.vm
vm_id = vm.id
before = vm.ports
if mode in {"open", "close"} and before is None:
raise UvmError(
f"{vm_id} has an unmanaged port policy; use ports set on a stopped VM first"
)
service_mode = mode
if mode == "close" and args.all_ports:
service_mode = "set"
requested = ()
if service_mode == "set":
after = requested
elif service_mode == "open":
merged = {rule.port: rule for rule in before}
for rule in requested:
merged.setdefault(rule.port, rule)
after = tuple(merged[port] for port in sorted(merged))
else:
removed = {rule.port for rule in requested}
after = tuple(rule for rule in before if rule.port not in removed)
if args.dry_run:
_print_preview(
f"ports-{mode}", [vm], as_json=args.json,
changes={"ports": {"before": _ports_output(before), "after": _ports_output(after)}},
)
return 0
if (
before == after == ()
and vm.operation is None
and (vm.status, listed.observed_status, listed.policy_status) in {
("running", "running", "enforced"),
("stopped", "stopped", "inactive"),
}
):
if args.json:
_print_json(_vm_output(vm, listed.observed_status, listed.policy_status))
else:
print(f"{vm_id}: no port changes.")
return 0
removes_ssh = (
any(rule.port == 22 for rule in before or ())
and not any(rule.port == 22 for rule in after)
)
if service_mode == "set" or removes_ssh:
if not _confirm(
f"{mode} private-IP ports for {vm_id}: "
f"{_format_ports(before)} -> {_format_ports(after)}?",
yes=args.yes,
):
if args.json:
_print_json([])
return 0
updated = application.lifecycle.set_ports(vm_id, requested, mode=service_mode)
if args.json:
_print_json(_vm_output(
updated, policy_status="enforced" if updated.status == "running" else "inactive"
))
else:
print(f"{updated.id} private-IP ports: {_format_ports(updated.ports)}")
return 0
if args.command == "doctor":
checks = [
{"name": check["name"], "ok": check["ok"], "message": check["message"]}
for check in application.lifecycle.diagnostics()
]
if args.json:
_print_json(checks)
else:
for check in checks:
print(f"{'OK' if check['ok'] else 'FAIL'} {check['name']}: {check['message']}")
return 0 if all(check["ok"] for check in checks) else 1
raise UvmError(f"unsupported command: {args.command}")
def _vm_output(
vm: VmRecord, observed_status: str | None = None, policy_status: str = "unknown"
) -> dict:
# This is an output whitelist, not the persisted record (which contains credentials).
return {
"id": vm.id,
"name": vm.name,
"cpu": vm.cpu,
"ram_mib": vm.ram_mib,
"guest_ip": vm.guest_ip,
"gateway": vm.gateway,
"tap": vm.tap,
"mac": vm.mac,
"username": vm.username,
"status": vm.status,
"observed_status": observed_status if observed_status is not None else vm.status,
"policy_status": policy_status,
"ports": _ports_output(vm.ports),
"pid": vm.pid,
"disk": vm.disk,
"disk_ready": vm.disk_ready,
"socket": vm.socket,
"config": vm.config,
"log": vm.log,
"operation": vm.operation,
"boot_id": vm.boot_id,
"created_at": vm.created_at,
"updated_at": vm.updated_at,
"last_error": vm.last_error,
}
def _ports_output(ports: tuple[PortRule, ...] | None) -> list[dict] | None:
return None if ports is None else [rule.to_dict() for rule in ports]
def _format_ports(ports: tuple[PortRule, ...] | None) -> str:
if ports is None:
return "unmanaged"
return ",".join(f"{rule.port}:{rule.protocol}" for rule in ports) or "(none)"
def _print_json(value: object) -> None:
print(json.dumps(value, indent=2))
def _print_preview(
action: str, targets: Sequence[VmRecord], *, as_json: bool, changes: dict | None = None
) -> None:
if as_json:
data = {"dry_run": True, "action": action, "targets": [_vm_output(vm) for vm in targets]}
if changes is not None:
data["changes"] = changes
_print_json(data)
else:
print(f"Dry run: {action}")
if not targets:
print("No VMs.")
for vm in targets:
print(f" {vm.id} {vm.name or '-'} {vm.guest_ip}")
if changes is not None:
for field, change in changes.items():
print(f" {field}: {json.dumps(change['before'])} -> {json.dumps(change['after'])}")
def _confirm(message: str, *, yes: bool) -> bool:
if yes:
return True
if not sys.stdin.isatty():
raise UvmError("confirmation requires a TTY; use --yes or preview with --dry-run")
print(f"{message} [y/N] ", end="", file=sys.stderr, flush=True)
if sys.stdin.readline().strip().lower() in {"y", "yes"}:
return True
print("Cancelled.", file=sys.stderr)
return False
def _print_results(results: Sequence, *, as_json: bool) -> int:
if as_json:
_print_json([
{"id": result.id, "action": result.action, "outcome": result.outcome, "error": result.error}
for result in results
])
elif not results:
print("No VMs.")
else:
for result in results:
detail = f": {result.error}" if result.error else ""
print(f"{result.id} {result.action}: {result.outcome}{detail}")
if any(result.outcome == "interrupted" for result in results):
return 130
return 1 if any(result.outcome == "failed" for result in results) else 0
def _show_logs(path: Path, tail: int, *, follow: bool) -> None:
with path.open("rb") as stream:
end = stream.seek(0, os.SEEK_END)
position = end
remaining = tail
# Locate the starting line backwards without loading or retaining the whole log.
while position > 0 and remaining:
size = min(position, 65536)
position -= size
stream.seek(position)
block = stream.read(size)
if position + size == end and block.endswith(b"\n"):
block = block[:-1]
newlines = block.count(b"\n")
if newlines >= remaining:
offset = len(block)
for _ in range(remaining):
offset = block.rfind(b"\n", 0, offset)
position += offset + 1
break
remaining -= newlines
stream.seek(position)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
while stream.tell() < end:
block = stream.read(min(65536, end - stream.tell()))
if not block:
break
sys.stdout.write(decoder.decode(block))
if not follow:
sys.stdout.write(decoder.decode(b"", final=True))
return
sys.stdout.flush()
while True:
block = stream.read(65536)
if block:
sys.stdout.write(decoder.decode(block))
sys.stdout.flush()
else:
if os.fstat(stream.fileno()).st_size < stream.tell():
stream.seek(0)
decoder.reset()
time.sleep(0.2)
def _line_count(value: str) -> int:
try:
count = int(value)
except ValueError as error:
raise argparse.ArgumentTypeError("tail must be a nonnegative integer") from error
if count < 0:
raise argparse.ArgumentTypeError("tail must be a nonnegative integer")
return count
def _parse_guest_ip(value: str | None) -> IPv4Address | None:
if value is None:
return None
uvm/domain.py
+73 -2
View File
@@ -8,10 +8,37 @@ from dataclasses import dataclass, field
from ipaddress import IPv4Address
from typing import Any
from .errors import StateError
from .errors import StateError, ValidationError
STATE_SCHEMA_VERSION = 1
STATE_SCHEMA_VERSION = 2
@dataclass(frozen=True, slots=True)
class PortRule:
"""One private guest TCP port; HTTP/HTTPS are descriptive service labels."""
port: int
protocol: str = "TCP"
def __post_init__(self) -> None:
if type(self.port) is not int or not 1 <= self.port <= 65535:
raise ValidationError("port must be an integer between 1 and 65535")
if not isinstance(self.protocol, str) or self.protocol.upper() not in {"TCP", "HTTP", "HTTPS"}:
raise ValidationError("port type must be TCP, HTTP, or HTTPS")
object.__setattr__(self, "protocol", self.protocol.upper())
def to_dict(self) -> dict[str, Any]:
return {"port": self.port, "protocol": self.protocol}
@classmethod
def from_dict(cls, value: object) -> "PortRule":
if not isinstance(value, dict) or set(value) - {"port", "protocol"}:
raise StateError("invalid port rule")
try:
return cls(port=value["port"], protocol=value.get("protocol", "TCP"))
except (KeyError, ValidationError) as error:
raise StateError(f"invalid port rule: {error}") from error
@dataclass(frozen=True, slots=True)
@@ -23,6 +50,8 @@ class VmSpec:
guest_ip: IPv4Address | None = None
username: str = "root"
password: str = field(default="root", repr=False)
name: str | None = None
ports: tuple[PortRule, ...] = (PortRule(22),)
@dataclass(slots=True)
@@ -48,6 +77,16 @@ class VmRecord:
created_at: int = field(default_factory=lambda: int(time.time()))
updated_at: int = field(default_factory=lambda: int(time.time()))
last_error: str | None = None
name: str | None = None
ports: tuple[PortRule, ...] | None = None
disk_ready: bool = False
runtime_owned: bool = False
tap_owned: bool = False
operation: str | None = None
boot_id: str | None = None
spawn_pending: bool = False
network: str | None = None
bridge: str | None = None
def to_dict(self) -> dict[str, Any]:
data: dict[str, Any] = {
@@ -67,6 +106,16 @@ class VmRecord:
"status": self.status,
"created_at": self.created_at,
"updated_at": self.updated_at,
"name": self.name,
"ports": [port.to_dict() for port in self.ports] if self.ports is not None else None,
"disk_ready": self.disk_ready,
"runtime_owned": self.runtime_owned,
"tap_owned": self.tap_owned,
"operation": self.operation,
"boot_id": self.boot_id,
"spawn_pending": self.spawn_pending,
"network": self.network,
"bridge": self.bridge,
}
if self.pid is not None:
data["pid"] = self.pid
@@ -99,6 +148,16 @@ class VmRecord:
try:
pid_value = value.get("pid")
raw_ports = value.get("ports")
if raw_ports is not None and not isinstance(raw_ports, list):
raise StateError("VM ports must be a list or null for a legacy VM")
ports = tuple(PortRule.from_dict(port) for port in raw_ports) if raw_ports is not None else None
if ports is not None and len({port.port for port in ports}) != len(ports):
raise StateError("VM has duplicate TCP port rules")
legacy_ready = bool(value.get("disk")) and value.get("status") in {"running", "stopped"}
for name in ("disk_ready", "runtime_owned", "tap_owned", "spawn_pending"):
if name in value and type(value[name]) is not bool:
raise StateError(f"VM {name} must be a boolean")
return cls(
id=str(value["id"]),
cpu=float(value["cpu"]),
@@ -131,6 +190,16 @@ class VmRecord:
if value.get("last_error") is not None
else None
),
name=str(value["name"]) if value.get("name") is not None else None,
ports=ports,
disk_ready=value.get("disk_ready", legacy_ready),
runtime_owned=value.get("runtime_owned", bool(value.get("disk"))),
tap_owned=value.get("tap_owned", bool(value.get("pid"))),
operation=str(value["operation"]) if value.get("operation") is not None else None,
boot_id=str(value["boot_id"]) if value.get("boot_id") is not None else None,
spawn_pending=value.get("spawn_pending", value.get("operation") == "spawning"),
network=str(value["network"]) if value.get("network") is not None else None,
bridge=str(value["bridge"]) if value.get("bridge") is not None else None,
)
except (TypeError, ValueError) as error:
raise StateError(f"invalid VM record: {error}") from error
@@ -170,6 +239,8 @@ class State:
raise StateError("state file has an invalid vms collection")
vms = {str(vm_id): VmRecord.from_dict(vm) for vm_id, vm in raw_vms.items()}
if any(vm_id != vm.id for vm_id, vm in vms.items()):
raise StateError("VM record ID does not match its registry key")
next_mac_index = value.get("next_mac_index")
if next_mac_index is None:
next_mac_index = _next_mac_index(vms.values())
uvm/errors.py
+8
View File
@@ -23,6 +23,14 @@ class StateError(UvmError):
"""Raised when persisted VM state cannot be safely used."""
class NotFoundError(UvmError):
"""A requested registry entry does not exist."""
class ConflictError(UvmError):
"""An allocation or pending operation prevents the requested action."""
class CommandError(UvmError):
"""Raised when a required host command cannot be executed successfully."""
uvm/firecracker/process.py
+358 -86
View File
@@ -2,8 +2,13 @@
from __future__ import annotations
import ctypes
import errno
import math
import os
import select
import signal
import stat
import subprocess
import time
from collections.abc import Callable
@@ -26,6 +31,50 @@ class _PopenLike(Protocol):
class ProcessInfo:
pid: int
start_time: str
boot_id: str | None = None
def _libc_pidfd(name: str, *arguments: int | None) -> int:
# Some Linux Python builds omit pidfd bindings even when libc supports them.
try:
function = getattr(ctypes.CDLL(None, use_errno=True), name)
except (AttributeError, OSError) as error:
raise OSError(errno.ENOSYS, f"{name} is unavailable") from error
function.argtypes = (
[ctypes.c_int, ctypes.c_uint]
if name == "pidfd_open"
else [ctypes.c_int, ctypes.c_int, ctypes.c_void_p, ctypes.c_uint]
)
function.restype = ctypes.c_int
result = function(*arguments)
if result == -1:
error_number = ctypes.get_errno()
raise OSError(error_number, os.strerror(error_number))
return result
def _open_pidfd(pid: int) -> int:
native = getattr(os, "pidfd_open", None)
if native is not None:
return native(pid, 0)
return _libc_pidfd("pidfd_open", pid, 0)
def _signal_pidfd(pidfd: int, signal_number: int) -> None:
native = getattr(signal, "pidfd_send_signal", None)
if native is not None:
native(pidfd, signal_number)
else:
_libc_pidfd("pidfd_send_signal", pidfd, signal_number, None, 0)
def _pidfd_exited(pidfd: int) -> bool:
poller = select.poll()
poller.register(pidfd, select.POLLIN)
events = poller.poll(0)
if any(event & (select.POLLERR | select.POLLNVAL) for _, event in events):
raise OSError(errno.EIO, "could not observe the Firecracker pidfd")
return any(event & (select.POLLIN | select.POLLHUP) for _, event in events)
class FirecrackerProcessManager:
@@ -43,103 +92,334 @@ class FirecrackerProcessManager:
self._popen = popen
self._sleep = sleep
self._monotonic = monotonic
self._children: dict[int, _PopenLike] = {}
def start(self, vm: VmRecord) -> ProcessInfo:
socket_path = Path(vm.socket)
if socket_path.exists():
try:
socket_path.unlink()
except OSError as error:
def start(
self,
vm: VmRecord,
*,
on_spawn: Callable[[ProcessInfo], None] | None = None,
) -> ProcessInfo:
"""Publish identity before readiness; never replace an ambiguous process."""
observation = self.observe(vm)
if observation != "gone":
raise FirecrackerError(
f"could not remove stale Firecracker socket {socket_path}: {error}"
f"refusing to start Firecracker: existing process is {observation}"
)
boot_id = self._boot_id()
if boot_id is None:
raise FirecrackerError("could not establish the current host boot identity")
# Check support before creating a child that we might be unable to stop safely.
try:
pidfd = _open_pidfd(os.getpid())
try:
_signal_pidfd(pidfd, 0)
finally:
os.close(pidfd)
except (OSError, NotImplementedError) as error:
raise FirecrackerError(
f"safe pidfd support is required to start Firecracker: {error}"
) from error
socket_path = Path(vm.socket)
self._remove_socket(socket_path)
log_path = Path(vm.log)
log_path.parent.mkdir(parents=True, exist_ok=True)
process: _PopenLike | None = None
try:
log_path.parent.mkdir(parents=True, exist_ok=True)
with log_path.open("ab", buffering=0) as log_file:
previous_identity = (
vm.pid, vm.process_start_time, vm.boot_id, vm.operation, vm.spawn_pending,
)
vm.pid = None
vm.process_start_time = None
vm.boot_id = boot_id
vm.operation = "spawning"
vm.spawn_pending = True
try:
process = self._popen(
[str(self._settings.firecracker_binary), "--api-sock", str(socket_path)],
stdout=log_file,
stderr=log_file,
start_new_session=True,
)
self._wait_for_socket(process, socket_path, log_path)
start_time = self.process_start_time(process.pid)
if start_time is None:
except OSError as error:
# Popen cleans up an exec/fork failure, but an interrupt is uncertain.
if not isinstance(error, InterruptedError):
(
vm.pid, vm.process_start_time, vm.boot_id, vm.operation, vm.spawn_pending,
) = previous_identity
raise
vm.pid = process.pid
self._children[process.pid] = process
vm.process_start_time = self.process_start_time(process.pid)
if vm.process_start_time is None:
raise FirecrackerError(
f"could not establish a safe process identity for Firecracker process {process.pid}"
)
return ProcessInfo(pid=process.pid, start_time=start_time)
except OSError as error:
info = ProcessInfo(process.pid, vm.process_start_time, boot_id)
if on_spawn is not None:
on_spawn(info)
else:
vm.spawn_pending = False
self._wait_for_socket(process, socket_path, log_path)
return info
except BaseException as error:
if process is not None:
self._cleanup_started_process(process, socket_path)
try:
self.terminate(vm)
except BaseException as cleanup_error:
raise FirecrackerError(
f"Firecracker startup failed ({type(error).__name__}: {error}); "
f"cleanup incomplete for PID {vm.pid}: {cleanup_error}"
) from error
if isinstance(error, OSError):
raise FirecrackerError(f"could not start Firecracker: {error}") from error
except BaseException:
if process is not None:
self._cleanup_started_process(process, socket_path)
raise
def is_alive(self, vm: VmRecord) -> bool:
if vm.pid is None or vm.pid <= 0 or vm.process_start_time is None:
def adopt_legacy_identity(self, vm: VmRecord) -> bool:
"""Validate a v1 process; the caller must lock and persist the adopted boot ID."""
pid, start_time = vm.pid, vm.process_start_time
if (
vm.boot_id is not None
or type(pid) is not int
or not 0 < pid < 2**31
or not isinstance(start_time, str)
or not start_time.isascii()
or not start_time.isdigit()
):
return False
try:
os.kill(vm.pid, 0)
except ProcessLookupError:
pidfd = _open_pidfd(pid)
try:
process_stat = self._process_stat(pid)
if (
process_stat is None
or process_stat[1] != start_time
or process_stat[0] in {"Z", "X", "x"}
):
return False
except PermissionError:
binary_path = self._settings.firecracker_binary
socket_path = self._settings.vm_dir(vm.id) / "firecracker.sock"
if (
not binary_path.is_absolute()
or ".." in binary_path.parts
or not vm.id
or vm.id in {".", ".."}
or Path(vm.id).name != vm.id
or vm.socket != str(socket_path)
):
return False
# Sticky ancestors are safe only because every existing entry is root-owned.
for directory in set(binary_path.parents) | set(socket_path.parents):
metadata = directory.lstat()
if (
not stat.S_ISDIR(metadata.st_mode)
or metadata.st_uid != 0
or (directory == socket_path.parent and metadata.st_mode & 0o077)
or (metadata.st_mode & 0o022 and not metadata.st_mode & stat.S_ISVTX)
):
return False
binary_stat = binary_path.lstat()
socket_stat = socket_path.lstat()
if (
not stat.S_ISREG(binary_stat.st_mode)
or binary_stat.st_uid != 0
or binary_stat.st_mode & 0o022
or not stat.S_ISSOCK(socket_stat.st_mode)
or socket_stat.st_uid != 0
):
return False
proc_path = Path(f"/proc/{pid}")
executable = proc_path / "exe"
if executable.readlink() != binary_path or not executable.samefile(binary_path):
return False
expected_cmdline = b"\0".join(
(os.fsencode(binary_path), b"--api-sock", os.fsencode(socket_path), b"")
)
if (proc_path / "cmdline").read_bytes() != expected_cmdline:
return False
boot_id = self._boot_id()
if boot_id is None or _pidfd_exited(pidfd):
return False
finally:
os.close(pidfd)
except (OSError, ValueError, NotImplementedError):
return False
if (vm.pid, vm.process_start_time, vm.boot_id) != (pid, start_time, None):
return False
vm.boot_id = boot_id
return True
return self.process_start_time(vm.pid) == vm.process_start_time
def observe(self, vm: VmRecord) -> str:
"""Distinguish proven exit from missing or unreadable process identity."""
def terminate(self, vm: VmRecord) -> None:
if vm.pid is not None and vm.pid <= 0:
return "unknown"
try:
self._reap_child(vm.pid)
except OSError:
return "unknown"
boot_id = self._boot_id()
if vm.boot_id and boot_id and vm.boot_id != boot_id:
return "gone"
if vm.pid is None:
self._remove_socket(Path(vm.socket))
return
if vm.pid <= 0:
raise FirecrackerError(f"invalid persisted Firecracker PID: {vm.pid}")
if vm.process_start_time is None:
if not self._pid_exists(vm.pid):
self._remove_socket(Path(vm.socket))
return
raise FirecrackerError(
f"refusing to signal Firecracker PID {vm.pid} without a process identity token"
if vm.spawn_pending or vm.operation == "spawning" or vm.process_start_time is not None:
return "unknown"
try:
Path(vm.socket).lstat()
except FileNotFoundError:
return "gone"
except OSError:
return "unknown"
# An unowned socket may belong to a child whose identity was never saved.
return "unknown"
stat = self._process_stat(vm.pid)
if stat is None:
return "unknown" if self._pid_exists(vm.pid) else "gone"
state, start_time = stat
has_start_time = (
isinstance(vm.process_start_time, str)
and vm.process_start_time.isascii()
and vm.process_start_time.isdigit()
)
if not self.is_alive(vm):
if has_start_time and vm.process_start_time != start_time:
return "gone"
if state in {"Z", "X", "x"}:
# A zombie thread-group leader can still have live worker threads.
try:
pidfd = _open_pidfd(vm.pid)
try:
return "gone" if _pidfd_exited(pidfd) else "unknown"
finally:
os.close(pidfd)
except ProcessLookupError:
return "gone"
except (OSError, NotImplementedError):
return "unknown"
if not has_start_time or not vm.boot_id or not boot_id:
return "unknown"
return "alive"
def is_alive(self, vm: VmRecord) -> bool:
"""Conservative compatibility query: unknown processes may still be alive."""
return self.observe(vm) != "gone"
def terminate(self, vm: VmRecord, *, timeout: float | None = None) -> None:
"""Wait for exit before unlinking; timeout controls the SIGTERM grace period."""
if vm.pid is not None and vm.pid <= 0:
raise FirecrackerError(f"invalid persisted Firecracker PID: {vm.pid}")
grace_timeout = self._settings.terminate_timeout_s if timeout is None else timeout
if not math.isfinite(grace_timeout) or grace_timeout < 0:
raise FirecrackerError("Firecracker termination timeout must be finite and non-negative")
observation = self.observe(vm)
if observation == "unknown":
raise FirecrackerError(
f"refusing to stop Firecracker PID {vm.pid}: process identity is ambiguous"
)
if observation == "gone":
self._remove_socket(Path(vm.socket))
return
try:
os.kill(vm.pid, signal.SIGTERM)
pidfd = _open_pidfd(vm.pid)
except ProcessLookupError:
self._reap_child(vm.pid)
self._remove_socket(Path(vm.socket))
return
except OSError as error:
raise FirecrackerError(f"could not stop Firecracker process {vm.pid}: {error}") from error
except (OSError, NotImplementedError) as error:
raise FirecrackerError(
f"could not safely open a pidfd for Firecracker process {vm.pid}: {error}"
) from error
deadline = self._monotonic() + self._settings.terminate_timeout_s
while self.is_alive(vm) and self._monotonic() < deadline:
self._sleep(0.05)
if self.is_alive(vm):
self._terminate_process_id(vm.pid, force=True)
try:
if not _pidfd_exited(pidfd):
# Open the handle BEFORE rechecking identity; never signal by bare PID.
observation = self.observe(vm)
if observation == "unknown":
raise FirecrackerError(
f"refusing to stop Firecracker PID {vm.pid}: process identity became ambiguous"
)
if observation == "alive":
try:
_signal_pidfd(pidfd, signal.SIGTERM)
except ProcessLookupError:
pass
if not self._wait_for_exit(pidfd, grace_timeout):
try:
_signal_pidfd(pidfd, signal.SIGKILL)
except ProcessLookupError:
pass
# Even an immediate forced stop needs time to confirm kernel exit.
if not self._wait_for_exit(pidfd, max(1.0, grace_timeout)):
raise FirecrackerError(
f"could not confirm exit of Firecracker process {vm.pid} after SIGKILL; "
"process identity and socket retained"
)
self._reap_child(vm.pid)
self._remove_socket(Path(vm.socket))
except (OSError, NotImplementedError) as error:
raise FirecrackerError(
f"could not safely stop Firecracker process {vm.pid}: {error}"
) from error
finally:
os.close(pidfd)
def process_start_time(self, pid: int) -> str | None:
"""Read Linux proc start time so a reused PID is not mistaken for a VM."""
stat = self._process_stat(pid)
return stat[1] if stat is not None else None
def host_boot_id(self) -> str | None:
"""Return the current host boot identity, or None when it is unreadable."""
return self._boot_id()
@staticmethod
def _boot_id() -> str | None:
try:
stat = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8")
except OSError:
return Path("/proc/sys/kernel/random/boot_id").read_text(encoding="ascii").strip() or None
except (OSError, UnicodeError):
return None
@staticmethod
def _process_stat(pid: int) -> tuple[str, str] | None:
if pid <= 0:
return None
try:
stat = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8", errors="replace")
except (OSError, UnicodeError):
return None
opening_parenthesis = stat.find("(")
closing_parenthesis = stat.rfind(")")
if closing_parenthesis < 0:
if (
opening_parenthesis < 0
or closing_parenthesis < opening_parenthesis
or stat[:opening_parenthesis].strip() != str(pid)
):
return None
fields = stat[closing_parenthesis + 2 :].split()
try:
return fields[19]
except IndexError:
fields = stat[closing_parenthesis + 1 :].split()
if (
len(fields) < 20
or fields[0] not in {"R", "S", "D", "Z", "T", "t", "X", "x", "K", "W", "P", "I"}
or not fields[19].isascii()
or not fields[19].isdigit()
):
return None
return fields[0], fields[19]
def _wait_for_socket(
self,
@@ -148,13 +428,32 @@ class FirecrackerProcessManager:
log_path: Path,
) -> None:
deadline = self._monotonic() + self._settings.api_socket_timeout_s
while self._monotonic() < deadline:
if socket_path.exists():
return
while True:
if process.poll() is not None:
raise FirecrackerError(f"Firecracker exited early; see {log_path}")
self._sleep(0.05)
if socket_path.is_socket():
return
remaining = deadline - self._monotonic()
if remaining <= 0:
raise FirecrackerError("timed out waiting for Firecracker API socket")
self._sleep(min(0.05, remaining))
def _wait_for_exit(self, pidfd: int, timeout: float) -> bool:
deadline = self._monotonic() + timeout
while True:
if _pidfd_exited(pidfd):
return True
remaining = deadline - self._monotonic()
if remaining <= 0:
return False
self._sleep(min(0.05, remaining))
def _reap_child(self, pid: int | None) -> None:
if pid is None:
return
process = self._children.get(pid)
if process is not None and process.poll() is not None:
self._children.pop(pid, None)
@staticmethod
def _remove_socket(socket_path: Path) -> None:
@@ -162,9 +461,10 @@ class FirecrackerProcessManager:
socket_path.unlink()
except FileNotFoundError:
pass
except OSError:
# A stale socket is cleaned on the next launch; never hide a successful stop.
pass
except OSError as error:
raise FirecrackerError(
f"could not remove Firecracker socket {socket_path}: {error}"
) from error
@staticmethod
def _pid_exists(pid: int) -> bool:
@@ -172,34 +472,6 @@ class FirecrackerProcessManager:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
except (OSError, OverflowError):
return True
return True
def _cleanup_started_process(self, process: _PopenLike, socket_path: Path) -> None:
try:
self._terminate_process_id(process.pid)
except FirecrackerError:
pass
deadline = self._monotonic() + self._settings.terminate_timeout_s
while process.poll() is None and self._monotonic() < deadline:
self._sleep(0.05)
if process.poll() is None:
try:
self._terminate_process_id(process.pid, force=True)
except FirecrackerError:
pass
self._remove_socket(socket_path)
@staticmethod
def _terminate_process_id(pid: int, *, force: bool = False) -> None:
if pid <= 0:
raise FirecrackerError(f"invalid Firecracker PID: {pid}")
signal_to_send = signal.SIGKILL if force else signal.SIGTERM
try:
os.kill(pid, signal_to_send)
except ProcessLookupError:
pass
except OSError as error:
action = "force-stop" if force else "stop"
raise FirecrackerError(f"could not {action} Firecracker process {pid}: {error}") from error
uvm/firewall.py
+496
View File
@@ -0,0 +1,496 @@
"""Private IPv4 service filtering, not host-port forwarding or publishing.
The lifecycle owner must serialize mutations, keep a live VM's identity immutable,
install the policy before starting its guest, and quarantine before teardown.
Only bridge/TAP traffic is covered; do not offload it around the bridge hooks.
Before making a TAP usable, attach it to an STP-disabled software bridge and set
the TAP's bridge_slave group_fwd_mask bit 0x4000. Otherwise Linux can deliver LLDP
(and STP) before *any* bridge hook and learn spoofed MACs. Network/lifecycle owns
these link settings; this manager only verifies them when the TAP exists.
No br_netfilter sysctls, NAT, persistent applied flags, or live conntrack flushes
are used. Native bridge conntrack (Linux >= 5.3) and nft JSON support are required.
"""
from __future__ import annotations
import hashlib
import json
import re
import uuid
from ipaddress import IPv4Address, IPv4Network
from types import SimpleNamespace
from typing import Any
from .config import Settings
from .domain import PortRule, VmRecord
from .errors import UvmError, ValidationError
from .system import CommandResult, CommandRunner
_COMMAND_TIMEOUT_S = 5.0
_PRIVATE_NETWORKS = tuple(IPv4Network(value) for value in (
"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16",
))
def _match(left: dict[str, Any], right: Any, op: str = "==") -> dict[str, Any]:
return {"match": {"op": op, "left": left, "right": right}}
def _payload(protocol: str, field: str) -> dict[str, Any]:
return {"payload": {"protocol": protocol, "field": field}}
def _ct(key: str, direction: str | None = None) -> dict[str, Any]:
# nft 1.0.9 parser_json.c/ct.c use the literal keys "ip saddr"/"ip daddr".
# Its CT parser ignores the "family" property advertised by the JSON manual.
return {"ct": {"key": key, **({"dir": direction} if direction else {})}}
class FirewallManager:
"""Own one table per (resolved settings.base, VM id) in the current netns.
``ports=None`` is legacy/unmanaged, not an empty allowlist. Normal apply
refuses it; quarantine works even without a port policy or valid IP/MAC.
Errors leave existing rules in place (or the committed replacement on a
readback/timeout failure). Callers must not start/release a VM on an error.
"""
def __init__(self, settings: Settings, runner: CommandRunner) -> None:
self._settings = settings
self._runner = runner
def apply(self, vm: VmRecord, *, quarantine: bool = False) -> None:
"""Atomically install/replace only this VM's owned table; retain CT state."""
expected = self._policy(vm, quarantine=quarantine)
if not self._bridge_paths_safe(vm):
raise UvmError(
"firewall needs an STP-disabled software bridge and TAP bridge_slave "
"group_fwd_mask bit 0x4000 (LLDP forwarding) before enforcement"
)
table = expected[0]["table"]
current = self._read_table(table["name"])
commands = []
if current is not None:
commands.append(self._delete_owned(current, table))
# Exclusive create plus deletion by handle also protects against a table
# being replaced by a foreign owner between inspection and commit.
commands.append({"create": expected[0]})
commands.extend({"add": item} for item in expected[1:])
self._transaction(commands)
installed = self._read_table(table["name"])
if installed is None or _normalize(installed) != _normalize(expected):
raise UvmError("firewall readback differs from the requested policy; keep VM isolated")
def remove(self, vm: VmRecord) -> None:
"""Remove an owned table only after the TAP is gone; absence is harmless."""
self._require_offline(vm)
table = self._table_identity(vm)
current = self._read_table(table["name"])
if current is not None:
self._transaction([self._delete_owned(current, table)])
def status(self, vm: VmRecord) -> str:
"""Inspect actual hooks, ordered rules, identity and complete port set.
Missing legacy tables are unmanaged. Foreign or altered tables are
drift; failed inspection/invalid metadata is unknown. A legacy VM can
report quarantined. This is a ruleset snapshot, not a reachability test.
"""
try:
table = self._table_identity(vm)
current = self._read_table(table["name"])
if current is None:
return "unmanaged" if getattr(vm, "ports", None) is None else "missing"
if _normalize(current) == _normalize(self._policy(vm, quarantine=True)):
return "quarantined" if self._bridge_paths_safe(vm) else "drift"
if getattr(vm, "ports", None) is not None:
if _normalize(current) == _normalize(self._policy(vm, quarantine=False)):
return "enforced" if self._bridge_paths_safe(vm) else "drift"
return "drift"
except (UvmError, ValueError, TypeError, KeyError):
return "unknown"
def check_available(self) -> None:
"""Check cleanup tooling, kernel and a full non-committing nft policy."""
self._run(("conntrack", "--version"))
release = self._run(("uname", "-r")).stdout.strip()
version = re.match(r"^(\d+)\.(\d+)", release)
if version is None or tuple(map(int, version.groups())) < (5, 3):
raise UvmError("private ports require Linux >= 5.3 with native bridge conntrack")
probe = SimpleNamespace(
id=f"probe-{uuid.uuid4().hex}", tap=f"uvm{uuid.uuid4().hex[:11]}",
guest_ip="10.0.0.2", gateway="10.0.0.1",
network="10.0.0.0/24", mac="02:00:00:00:00:02",
ports=(PortRule(443),),
)
policy = self._policy(probe, quarantine=False) # type: ignore[arg-type]
commands = [{"create": policy[0]}, *({"add": item} for item in policy[1:])]
try:
self._transaction(commands, check_only=True)
except UvmError as error:
raise UvmError(f"nft bridge/conntrack policy is unavailable: {error}") from error
def cleanup_connections(self, vm: VmRecord) -> None:
"""Offline, IP-scoped CT deletion before address reuse, never for close.
All four tuple addresses are checked, including NAT reply tuples. A
failed deletion is tolerated only if a successful list proves absence.
The caller must retain the address reservation until this succeeds.
"""
self._require_offline(vm)
address = self._guest_ip(vm)
for field in ("--orig-src", "--orig-dst", "--reply-src", "--reply-dst"):
selector = ("--family", "ipv4", field, address)
self._run(("conntrack", "--delete", *selector), check=False)
remaining = self._run(("conntrack", "--list", *selector))
if remaining.stdout.strip():
raise UvmError(f"conntrack entries remain for {address}; do not reuse its IP")
def _run(self, command: tuple[str, ...], **kwargs: Any) -> CommandResult:
return self._runner.run(
command, capture=True, timeout=_COMMAND_TIMEOUT_S, **kwargs,
)
def _transaction(self, commands: list[dict[str, Any]], *, check_only: bool = False) -> None:
args = ("nft", "--json", *(("--check",) if check_only else ()), "-f", "-")
self._run(args, input_text=json.dumps({"nftables": commands}))
def _table_identity(self, vm: VmRecord) -> dict[str, Any]:
if not isinstance(vm.id, str) or not vm.id:
raise ValidationError("firewall requires a nonempty VM ID")
identity = json.dumps([str(self._settings.base.resolve()), vm.id]).encode("utf-8")
digest = hashlib.sha256(identity).hexdigest()
return {"family": "bridge", "name": f"uvm_fw_{digest[:24]}",
"comment": f"uvm-private-ports:v1:{digest}"}
def _tap(self, vm: VmRecord) -> str:
if not isinstance(vm.tap, str) or re.fullmatch(r"[A-Za-z0-9_][A-Za-z0-9_.-]{0,14}", vm.tap) is None:
raise ValidationError("firewall requires an exact, safe TAP name (1-15 characters)")
if vm.tap in {self._settings.bridge, getattr(vm, "bridge", None)}:
raise ValidationError("firewall TAP must not be the bridge itself")
return vm.tap
def _guest_ip(self, vm: VmRecord) -> str:
try:
address = IPv4Address(vm.guest_ip)
network = IPv4Network(getattr(vm, "network", None) or self._settings.network)
except (ValueError, TypeError) as error:
raise ValidationError("firewall requires a valid guest IPv4 address/network") from error
gateway = str(getattr(vm, "gateway", None) or self._settings.gateway)
if (not any(address in private for private in _PRIVATE_NETWORKS)
or address not in network
or address in {network.network_address, network.broadcast_address}
or str(address) == gateway):
raise ValidationError("firewall requires a usable private guest IP inside its network")
return str(address)
def _links(self) -> list[dict[str, Any]]:
try:
links = json.loads(self._run(("ip", "-j", "-d", "link", "show")).stdout)
if not isinstance(links, list) or any(
not isinstance(link, dict) or not isinstance(link.get("ifname"), str)
for link in links
):
raise ValueError("invalid link listing")
except (ValueError, TypeError) as error:
raise UvmError("cannot inspect network interfaces; retaining firewall/conntrack state") from error
return links
def _bridge_paths_safe(self, vm: VmRecord) -> bool:
links = self._links()
name = self._tap(vm)
taps = [link for link in links if link["ifname"] == name]
if not taps:
return True # Policy may precede TAP creation; main must prepare it before bringing it up.
if len(taps) != 1:
return False
tap = taps[0]
bridges = [link for link in links if tap.get("master") in (link["ifname"], link.get("ifindex"))]
if len(bridges) != 1:
return False
try:
port = tap["linkinfo"]
bridge = bridges[0]["linkinfo"]
mask = int(str(port["info_slave_data"]["group_fwd_mask"]), 0)
return (port["info_slave_kind"] == "bridge" and bridge["info_kind"] == "bridge"
and bridge["info_data"]["stp_state"] == 0
and 0 <= mask <= 65535 and bool(mask & 0x4000))
except (KeyError, TypeError, ValueError):
return False
def _require_offline(self, vm: VmRecord) -> None:
tap = self._tap(vm)
if any(link["ifname"] == tap for link in self._links()):
raise UvmError(f"TAP {tap} still exists; quarantine it and remove the TAP first")
def _read_table(self, name: str) -> list[dict[str, Any]] | None:
# A successful table inventory, not a localized 'not found' error,
# distinguishes missing policy from missing privileges/tools.
def objects(command: tuple[str, ...]) -> list[dict[str, Any]]:
try:
data = json.loads(self._run(command).stdout)["nftables"]
if not isinstance(data, list) or any(
not isinstance(item, dict) or len(item) != 1
or not isinstance(next(iter(item.values())), dict) for item in data
):
raise ValueError("invalid nft object list")
return [item for item in data if "metainfo" not in item]
except (ValueError, KeyError, TypeError) as error:
raise UvmError("could not inspect nft bridge tables") from error
tables = objects(("nft", "--json", "list", "tables", "bridge"))
if any("table" not in item or item["table"].get("family") != "bridge"
or not isinstance(item["table"].get("name"), str) for item in tables):
raise UvmError("invalid nft bridge table inventory")
matches = [item for item in tables if item["table"].get("name") == name]
if not matches:
return None
if len(matches) != 1:
raise UvmError("ambiguous nft bridge table inventory")
data = objects(("nft", "--json", "--numeric-protocol", "list", "table", "bridge", name))
headers = [item["table"] for item in data if "table" in item]
if len(headers) != 1 or headers[0].get("name") != name or headers[0].get("family") != "bridge":
raise UvmError("invalid nft bridge table snapshot")
return data
@staticmethod
def _delete_owned(current: list[dict[str, Any]], table: dict[str, Any]) -> dict[str, Any]:
header = next(item["table"] for item in current if "table" in item)
if any(header.get(key) != value for key, value in table.items()):
raise UvmError(f"refusing to touch foreign nft table {table['name']}")
handle = header.get("handle")
if type(handle) is not int or handle <= 0:
raise UvmError("owned nft table has no valid handle; refusing unsafe deletion")
return {"delete": {"table": {"family": "bridge", "handle": handle}}}
def _policy(self, vm: VmRecord, *, quarantine: bool) -> list[dict[str, Any]]:
ports = getattr(vm, "ports", None)
if ports is None and not quarantine:
raise ValidationError("legacy VM has no managed port policy; only quarantine is allowed")
table = self._table_identity(vm)
tap = self._tap(vm)
scope = {"family": "bridge", "table": table["name"]}
policy: list[dict[str, Any]] = [{"table": table}]
def chain(name: str, hook: str | None = None, priority: int = 0) -> None:
definition = {**scope, "name": name}
if hook is not None:
definition.update(type="filter", hook=hook, prio=priority, policy="accept")
policy.append({"chain": definition})
def rule(name: str, *expressions: dict[str, Any]) -> None:
policy.append({"rule": {**scope, "chain": name, "expr": list(expressions)}})
for hook in ("prerouting", "input", "forward", "output"):
chain(hook, hook, -310 if hook == "prerouting" else 0)
if not quarantine:
for name in ("identity", "from_vm", "to_vm"):
chain(name)
for hook, interface, target in (
("prerouting", "iifname", "identity"),
("input", "iifname", "from_vm"),
("forward", "iifname", "from_vm"),
("forward", "oifname", "to_vm"),
("output", "oifname", "to_vm"),
):
rule(hook, _match({"meta": {"key": interface}}, tap),
{"drop": None} if quarantine else {"jump": {"target": target}})
if quarantine:
return policy
address = self._guest_ip(vm)
mac = vm.mac.lower() if isinstance(vm.mac, str) else ""
if (re.fullmatch(r"(?:[0-9a-f]{2}:){5}[0-9a-f]{2}", mac) is None
or int(mac[:2], 16) & 1 or mac == "00:00:00:00:00:00"):
raise ValidationError("firewall requires a nonzero unicast guest MAC")
if not isinstance(ports, tuple):
raise ValidationError("managed ports must be a tuple of TCP/HTTP/HTTPS port rules")
allowed = set()
for port_rule in ports:
port = getattr(port_rule, "port", None)
if (type(port) is not int or not 1 <= port <= 65535
or getattr(port_rule, "protocol", None) not in ("TCP", "HTTP", "HTTPS")):
raise ValidationError("private ports must be TCP/HTTP/HTTPS integers from 1 to 65535")
allowed.add(port)
# json_parse_set_expr() in nft 1.0.9 rejects [], even for an empty named
# set. Omit elem, as set_print_json() does when listing an empty set.
policy.insert(1, {"set": {**scope, "name": "tcp_ports", "type": "inet_service",
"flags": ["constant"],
**({"elem": sorted(allowed)} if allowed else {})}})
drop, success = {"drop": None}, {"return": None}
ipv4 = _match(_payload("ether", "type"), "ip")
arp = _match(_payload("ether", "type"), "arp")
arp_header = [arp, _match(_payload("arp", "htype"), 1),
_match(_payload("arp", "ptype"), "ip"),
_match(_payload("arp", "hlen"), 6), _match(_payload("arp", "plen"), 4)]
# Before conntrack (-200) and FDB learning: no MAC/IP/ARP spoofing,
# IPv6, VLAN encapsulation or other EtherTypes from a managed port.
rule("identity", _match(_payload("ether", "saddr"), mac, "!="), drop)
rule("identity", ipv4, _match(_payload("ip", "saddr"), address), success)
rule("identity", *arp_header,
_match(_payload("arp", "operation"), {"set": ["request", "reply"]}),
_match(_payload("arp", "saddr ether"), mac),
_match(_payload("arp", "saddr ip"), address), success)
# RFC 5227 address probes may use zero, but only probe this VM's IP.
rule("identity", *arp_header, _match(_payload("arp", "operation"), "request"),
_match(_payload("arp", "saddr ether"), mac),
_match(_payload("arp", "saddr ip"), "0.0.0.0"),
_match(_payload("arp", "daddr ip"), address), success)
rule("identity", drop)
# Some link-local bridge frames take input without prerouting. Recheck
# identity there too, before its local-finish callback can learn a MAC.
rule("from_vm", {"jump": {"target": "identity"}})
rule("from_vm", arp, success)
rule("to_vm", *arp_header, _match(_payload("arp", "daddr ip"), address),
_match(_payload("arp", "operation"), "request"), success)
rule("to_vm", *arp_header, _match(_payload("arp", "daddr ip"), address),
_match(_payload("arp", "daddr ether"), mac),
_match(_payload("arp", "operation"), "reply"), success)
for name, outbound in (("from_vm", True), ("to_vm", False)):
rule(name, _match(_payload("ether", "type"), "ip", "!="), drop)
if not outbound:
rule(name, ipv4, _match(_payload("ip", "daddr"), address, "!="), drop)
rule(name, _match(_payload("ether", "daddr"), mac, "!="), drop)
# Loose TCP tracking must not reclassify a server's midstream data
# as a newly guest-initiated connection after tracking loss.
rule(name, ipv4, _match(_payload("ip", "protocol"), "tcp"),
_match(_ct("state"), "new", "in"),
_match({"&": [_payload("tcp", "flags"), 23]}, 2, "!="), drop)
for client in (True, False):
original = outbound == client
flow = [_match(_ct("direction"), "original" if original else "reply"),
_match(_ct("ip saddr" if client else "ip daddr", "original"), address)]
states = ["new", "established"] if original else ["established"]
state = _match(_ct("state"), states[0] if len(states) == 1 else states, "in")
for protocol in (("tcp", "udp", "icmp") if client else ("tcp", "icmp")):
parent = [*flow, _match(_ct("protocol"), protocol)]
if not client and protocol == "tcp":
parent.append(_match(_ct("proto-dst", "original"), "@tcp_ports"))
packet = [ipv4, _match(_payload("ip", "protocol"), protocol)]
if protocol == "icmp":
packet.append(_match(_payload("icmp", "type"),
"echo-request" if original else "echo-reply"))
rule(name, *packet, *parent, state, success)
# Related is restricted to essential ICMP errors and the
# same authorized parent tuple, never TCP/UDP helper flows.
rule(name, ipv4, _match(_payload("ip", "protocol"), "icmp"),
_match(_payload("icmp", "type"), {"set": [
"destination-unreachable", "time-exceeded", "parameter-problem",
]}), *parent, _match(_ct("state"), "related", "in"), success)
rule(name, drop)
return policy
def _normalize(objects: list[dict[str, Any]]) -> str:
"""Canonical nft JSON, retaining every effective object, rule and verdict.
Ignore only handles, empty set presentation and nft's redundant protocol
dependencies. Rule order is significant; pure conjunctions and sets are not.
Unknown expressions/properties remain different rather than being ignored.
"""
def canonical(value: Any) -> Any:
if isinstance(value, list):
return [canonical(item) for item in value]
if not isinstance(value, dict):
return value
value = {key: canonical(item) for key, item in value.items()}
if (set(value) == {"|"} and isinstance(value["|"], list)
and all(type(item) is int for item in value["|"])):
result = 0
for item in value["|"]:
result |= item
return result
if set(value) == {"set"} and isinstance(value["set"], list):
value["set"] = sorted(value["set"], key=lambda item: json.dumps(item, sort_keys=True))
return value
def bitmask(value: Any, names: dict[str, int]) -> Any:
if isinstance(value, str):
return names.get(value, value)
if isinstance(value, dict) and set(value) == {"|"}:
value = value["|"]
if isinstance(value, list):
bits = [bitmask(item, names) for item in value]
if all(type(item) is int for item in bits):
result = 0
for item in bits:
result |= item
return result
return value
definitions, rules = [], {}
for item in objects:
kind, raw = next(iter(item.items()))
if kind == "metainfo":
continue
obj = {key: canonical(value) for key, value in raw.items() if key != "handle"}
if kind == "set":
elements = obj.get("elem", [])
if not isinstance(elements, list):
elements = [elements]
obj["elem"] = sorted(elements, key=lambda value: json.dumps(value, sort_keys=True))
if obj.get("flags") == []:
del obj["flags"]
elif isinstance(obj.get("flags"), str):
obj["flags"] = [obj["flags"]]
if kind != "rule":
definitions.append({kind: obj})
continue
expressions = obj.get("expr", [])
if not isinstance(expressions, list) or not isinstance(obj.get("chain"), str):
raise UvmError("invalid nft rule snapshot")
tcp_flags = {"fin": 1, "syn": 2, "rst": 4, "psh": 8, "ack": 16,
"urg": 32, "ecn": 64, "cwr": 128}
for expression in expressions:
if not isinstance(expression, dict) or not isinstance(expression.get("match", {}), dict):
continue
match = expression.get("match", {})
left = match.get("left")
if match and not {"left", "right", "op"}.issubset(match):
raise UvmError("invalid nft match snapshot")
right = match.get("right")
if left in (_payload("ip", "protocol"), _ct("protocol")) and type(right) is int:
match["right"] = {1: "icmp", 6: "tcp", 17: "udp"}.get(right, right)
if match.get("op") == "in" and (
(isinstance(right, str) and right.startswith("@"))
or (isinstance(right, dict) and set(right) == {"set"})
):
match["op"] = "=="
if left == _ct("state"):
match["right"] = bitmask(match["right"], {
"invalid": 1, "established": 2, "related": 4, "new": 8, "untracked": 64,
})
if (isinstance(left, dict) and isinstance(left.get("&"), list)
and len(left["&"]) == 2 and left["&"][0] == _payload("tcp", "flags")):
left["&"][1] = bitmask(left["&"][1], tcp_flags)
match["right"] = bitmask(match["right"], tcp_flags)
# nft's replayable JSON omits dependencies implied by named payloads.
encoded = json.dumps(expressions, sort_keys=True)
has_tcp = '"protocol": "tcp"' in encoded
has_icmp = '"protocol": "icmp"' in encoded
expressions = [expr for expr in expressions if not (
(has_tcp and expr == _match(_payload("ip", "protocol"), "tcp"))
or (has_icmp and expr == _match(_payload("ip", "protocol"), "icmp"))
)]
encoded = json.dumps(expressions, sort_keys=True)
expressions = [expr for expr in expressions if not (
('"protocol": "arp"' in encoded and expr == _match(_payload("ether", "type"), "arp"))
or (('"protocol": "ip"' in encoded or has_icmp)
and expr == _match(_payload("ether", "type"), "ip"))
)]
count = len(expressions)
if expressions and (not isinstance(expressions[-1], dict) or set(expressions[-1]) != {"match"}):
count -= 1
if all(isinstance(expr, dict) and set(expr) == {"match"} for expr in expressions[:count]):
expressions = sorted(expressions[:count], key=lambda expr: json.dumps(expr, sort_keys=True)) + expressions[count:]
obj["expr"] = expressions
rules.setdefault(obj.get("chain", ""), []).append(obj)
definitions.sort(key=lambda item: json.dumps(item, sort_keys=True))
return json.dumps([definitions, rules], sort_keys=True, separators=(",", ":"))
uvm/images.py
+14 -10
View File
@@ -58,23 +58,14 @@ class ImageStore:
self._runner = runner or CommandRunner()
def installed_assets(self) -> GuestAssets:
kernel = self._settings.kernel_image
kernel = self.installed_kernel()
rootfs = self._settings.rootfs_image
if not kernel.exists() or not rootfs.exists():
raise UvmError("guest assets missing. Run: sudo uvm install")
manifest = load_manifest(self._settings.integrity_manifest_path)
kernel_checksum = self._settings.kernel_sha256
rootfs_checksum = self._settings.rootfs_sha256
if manifest.verified or self._settings.allow_unverified_downloads:
kernel_checksum = kernel_checksum or manifest.checksums.get("kernel")
rootfs_checksum = rootfs_checksum or manifest.checksums.get("rootfs")
verify_file(
kernel,
kernel_checksum,
"guest kernel",
"UVM_KERNEL_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
verify_file(
rootfs,
rootfs_checksum,
@@ -84,6 +75,19 @@ class ImageStore:
)
return GuestAssets(kernel=kernel, rootfs=rootfs)
def installed_kernel(self) -> Path:
"""A retained private disk does not depend on the original rootfs template."""
kernel = self._settings.kernel_image
if not kernel.is_file():
raise UvmError("guest kernel missing. Run: sudo uvm install")
manifest = load_manifest(self._settings.integrity_manifest_path)
checksum = self._settings.kernel_sha256
if manifest.verified or self._settings.allow_unverified_downloads:
checksum = checksum or manifest.checksums.get("kernel")
verify_file(kernel, checksum, "guest kernel", "UVM_KERNEL_SHA256",
allow_unverified=self._settings.allow_unverified_downloads)
return kernel
def create_vm_disk(self, source: Path, destination: Path) -> Path:
"""Copy the template before Firecracker opens it read-write for a VM."""
uvm/install.py
+2
View File
@@ -110,6 +110,8 @@ class Installer:
"jq",
"iproute2",
"iptables",
"nftables",
"conntrack",
"e2fsprogs",
"util-linux",
"ca-certificates",
uvm/lifecycle.py
+421 -198
View File
@@ -1,43 +1,49 @@
"""Ordered VM lifecycle operations and compensation for partial failures."""
"""Serialized lifecycle and private-port policy changes with durable intent."""
from __future__ import annotations
import re
import shutil
import time
from collections.abc import Callable
from dataclasses import dataclass
from collections.abc import Callable, Sequence
from dataclasses import dataclass, replace
from pathlib import Path
from .config import Settings
from .domain import VmRecord, VmSpec, new_vm_id
from .errors import TapCreationError, UvmError
from .domain import PortRule, VmRecord, VmSpec, new_vm_id
from .errors import ConflictError, NotFoundError, UvmError, ValidationError
from .firecracker.api import FirecrackerClient
from .firecracker.config import build_config, write_config
from .firecracker.process import FirecrackerProcessManager
from .firecracker.process import FirecrackerProcessManager, ProcessInfo
from .images import ImageStore
from .integrity import load_manifest, verify_file
from .network import NetworkManager
from .state import StateStore
from .system import check_kvm, ensure_data_directories, require_root
from .validation import parse_cpu, parse_name, parse_password, parse_ports, parse_username
@dataclass(frozen=True, slots=True)
class ListedVm:
vm: VmRecord
observed_status: str
policy_status: str = "unknown"
@dataclass(frozen=True, slots=True)
class OperationResult:
id: str
action: str
outcome: str
error: str | None = None
class LifecycleService:
"""The only module allowed to coordinate multiple VM host resources."""
"""All mutating entry points use operation lock -> short state transaction."""
def __init__(
self,
settings: Settings,
state_store: StateStore,
images: ImageStore,
network: NetworkManager,
process: FirecrackerProcessManager,
*,
self, settings: Settings, state_store: StateStore, images: ImageStore,
network: NetworkManager, process: FirecrackerProcessManager, *,
client_factory: Callable[[Path, float], FirecrackerClient] = FirecrackerClient,
clock: Callable[[], float] = time.time,
) -> None:
@@ -50,235 +56,452 @@ class LifecycleService:
self._clock = clock
def create(self, spec: VmSpec) -> VmRecord:
"""Reserve identity first, then create all VM resources in dependency order."""
require_root()
parse_cpu(str(spec.cpu))
if type(spec.ram_mib) is not int or spec.ram_mib < 128:
raise ValidationError("RAM must be an integer of at least 128 MiB")
parse_username(spec.username)
parse_password(spec.password)
if spec.name is not None:
parse_name(spec.name)
ports = self._validated_ports(spec.ports)
ensure_data_directories(self._settings)
check_kvm()
if not self._settings.firecracker_binary.exists():
raise UvmError("Firecracker is not installed. Run: sudo uvm install")
with self._state_store.operation_lock():
manifest = load_manifest(self._settings.integrity_manifest_path)
firecracker_checksum = self._settings.firecracker_binary_sha256
if manifest.verified or self._settings.allow_unverified_downloads:
firecracker_checksum = firecracker_checksum or manifest.checksums.get("firecracker")
verify_file(
self._settings.firecracker_binary,
firecracker_checksum,
"Firecracker binary",
"UVM_FIRECRACKER_BINARY_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
return self._create_locked(spec)
def _create_locked(self, spec: VmSpec) -> VmRecord:
self._check_boot()
assets = self._images.installed_assets()
vm = self._reserve_vm(spec)
tap_created = False
vm = self._reserve_vm(replace(spec, ports=ports))
try:
runtime_dir = self._settings.vm_dir(vm.id)
runtime_dir.mkdir(parents=True, exist_ok=False)
runtime_dir.chmod(0o700)
runtime = self._settings.vm_dir(vm.id)
runtime.mkdir(mode=0o700, exist_ok=False)
vm.runtime_owned = True
self._save(vm)
disk = self._images.create_vm_disk(assets.rootfs, Path(vm.disk))
self._images.provision_credentials(disk, vm.username, vm.password)
self._network.ensure_bridge()
try:
self._network.create_tap(vm.tap)
except TapCreationError as error:
tap_created = error.tap_created
raise
else:
tap_created = True
config = build_config(self._settings, vm, assets.kernel, disk)
write_config(Path(vm.config), config)
process_info = self._process.start(vm)
vm.pid = process_info.pid
vm.process_start_time = process_info.start_time
vm.updated_at = self._now()
self._replace_vm(vm)
client = self._client_factory(Path(vm.socket), self._settings.api_timeout_s)
client.configure_and_start(config)
vm.disk_ready = True
self._save(vm)
self._boot(vm, assets.kernel)
return vm
except BaseException as error:
cleanup_errors = self._rollback_create(vm, tap_created)
cleanup_error = None
try:
if vm.operation != "preparing":
self._cleanup_runtime(vm)
self._remove_runtime(vm)
self._remove_record(vm.id)
except BaseException as cleanup:
cleanup_error = cleanup
message = f"failed to create {vm.id}: {error}"
if cleanup_errors:
message = f"{message}; cleanup failed: {'; '.join(cleanup_errors)}"
vm.status = "failed"
vm.last_error = message
vm.updated_at = self._now()
self._replace_vm_if_present(vm)
else:
self._remove_vm_if_present(vm.id)
if cleanup_error is not None:
message += f"; cleanup incomplete: {cleanup_error}"
self._record_failure(vm, message)
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
if isinstance(error, UvmError):
raise error
raise UvmError(message) from error
vm.status = "running"
vm.last_error = None
vm.updated_at = self._now()
self._replace_vm(vm)
def _check_boot(self) -> None:
check_kvm()
manifest = load_manifest(self._settings.integrity_manifest_path)
checksum = self._settings.firecracker_binary_sha256
if manifest.verified or self._settings.allow_unverified_downloads:
checksum = checksum or manifest.checksums.get("firecracker")
verify_file(self._settings.firecracker_binary, checksum, "Firecracker binary",
"UVM_FIRECRACKER_BINARY_SHA256",
allow_unverified=self._settings.allow_unverified_downloads)
self._network.firewall.check_available()
def _boot(self, vm: VmRecord, kernel: Path) -> None:
vm.status, vm.operation = "starting", "network"
vm.pid = vm.process_start_time = None
vm.spawn_pending = False
self._save(vm)
self._network.ensure_bridge()
self._network.prepare_tap(vm)
self._save(vm)
self._network.firewall.apply(vm)
self._network.activate_tap(vm)
config = build_config(self._settings, vm, kernel, Path(vm.disk))
write_config(Path(vm.config), config)
# Persist the crash window before spawning, but give start() the already
# verified process-free record. A restored pending spawn is never guessed gone.
vm.operation = "boot"
vm.boot_id = self._process.host_boot_id()
self._save(replace(vm, operation="spawning", spawn_pending=True))
def spawned(info: ProcessInfo) -> None:
vm.pid, vm.process_start_time, vm.boot_id = info.pid, info.start_time, info.boot_id
vm.spawn_pending = False
vm.operation = "configuring"
self._save(vm)
self._process.start(vm, on_spawn=spawned)
client = self._client_factory(Path(vm.socket), self._settings.api_timeout_s)
client.configure_and_start(config)
if self._network.firewall.status(vm) != "enforced":
raise UvmError("private-port enforcement could not be confirmed after boot")
vm.status, vm.operation, vm.last_error = "running", None, None
self._save(vm)
def start(self, identifier: str) -> VmRecord:
require_root()
with self._state_store.operation_lock():
return self._start_locked(self._load_for_operation(identifier))
def _start_locked(self, vm: VmRecord) -> VmRecord:
self._not_deleting(vm)
if vm.ports is None:
raise ConflictError("legacy VM has no port policy; stop it, use ports set, then start")
observation = self._process.observe(vm)
if observation == "unknown":
raise ConflictError(f"cannot start {vm.id}: process identity is unknown; inspect before retrying")
if observation == "alive":
if vm.status != "running" or vm.operation not in {None, "policy"}:
raise ConflictError(f"{vm.id} has an interrupted boot/stop; stop it before starting")
return self._sync_ports_locked(vm)
if not vm.disk_ready:
raise ConflictError(f"{vm.id} has no completely provisioned private disk")
disk = Path(vm.disk)
if not disk.is_file() or disk.is_symlink() or disk.stat().st_nlink != 1:
raise UvmError(f"private VM disk is missing or unsafe: {disk}")
if (vm.gateway != str(self._settings.gateway)
or vm.network not in {None, str(self._settings.network)}
or vm.bridge not in {None, self._settings.bridge}):
raise ConflictError("configured network differs from the VM's retained network")
self._check_boot()
kernel = self._images.installed_kernel()
try:
self._cleanup_runtime(vm)
vm.network, vm.bridge = str(self._settings.network), self._settings.bridge
self._boot(vm, kernel)
return vm
except BaseException as error:
message = f"failed to start {vm.id}: {error}"
try:
self._cleanup_runtime(vm)
except BaseException as cleanup:
message += f"; cleanup incomplete: {cleanup}"
self._record_failure(vm, message)
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
raise UvmError(message) from error
def stop(self, identifier: str, *, timeout: float | None = None) -> VmRecord:
require_root()
with self._state_store.operation_lock():
return self._stop_locked(self._load_for_operation(identifier), timeout=timeout)
def _stop_locked(self, vm: VmRecord, *, timeout: float | None = None) -> VmRecord:
deleting = vm.operation == "delete" or vm.status == "terminating"
vm.status, vm.operation = "stopping", "delete" if deleting else "stop"
self._save(vm)
try:
self._cleanup_runtime(vm, timeout=timeout)
vm.status = "stopped"
vm.operation = "delete" if deleting else None
vm.last_error = None
self._save(vm)
return vm
except BaseException as error:
self._record_failure(vm, f"failed to stop {vm.id}: {error}")
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
raise UvmError(vm.last_error) from error
def restart(self, identifier: str, *, timeout: float | None = None) -> VmRecord:
require_root()
with self._state_store.operation_lock():
vm = self._load_for_operation(identifier)
self._not_deleting(vm)
self._stop_locked(vm, timeout=timeout)
return self._start_locked(vm)
def destroy(self, identifier: str, *, timeout: float | None = None) -> VmRecord:
require_root()
with self._state_store.operation_lock():
return self._delete_locked(self._load_for_operation(identifier), timeout=timeout)
def _delete_locked(self, vm: VmRecord, *, timeout: float | None = None) -> VmRecord:
vm.status, vm.operation = "terminating", "delete"
self._save(vm)
try:
self._cleanup_runtime(vm, timeout=timeout)
self._remove_runtime(vm)
self._remove_record(vm.id)
return vm
except BaseException as error:
self._record_failure(vm, f"failed to delete {vm.id}: {error}")
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
raise UvmError(vm.last_error) from error
def _cleanup_runtime(self, vm: VmRecord, *, timeout: float | None = None) -> None:
containment_errors = []
try:
if self._network.has_tap(vm):
try:
self._network.firewall.apply(vm, quarantine=True)
finally:
self._network.deactivate_tap(vm)
except UvmError as error:
containment_errors.append(str(error))
try:
self._process.terminate(vm, timeout=timeout)
except UvmError as error:
detail = f"; containment: {'; '.join(containment_errors)}" if containment_errors else ""
raise UvmError(f"process exit unconfirmed: {error}{detail}") from error
vm.pid = vm.process_start_time = None
vm.spawn_pending = False
if vm.operation == "spawning":
vm.operation = "boot"
self._network.delete_vm_tap(vm)
self._network.firewall.cleanup_connections(vm)
self._network.firewall.remove(vm)
def set_ports(self, identifier: str, ports: tuple[PortRule, ...], *, mode: str = "set") -> VmRecord:
require_root()
ports = self._validated_ports(ports)
if mode not in {"open", "close", "set"}:
raise ValidationError(f"invalid port operation: {mode}")
with self._state_store.operation_lock():
vm = self._load_for_operation(identifier)
self._not_deleting(vm)
observation = self._process.observe(vm)
if observation == "unknown":
raise ConflictError("port changes require a known process identity")
if vm.ports is None and (mode != "set" or observation != "gone" or vm.status != "stopped"):
raise ConflictError("legacy VM policy must be explicitly set while stopped")
if vm.operation not in {None, "policy"}:
raise ConflictError(f"finish the pending {vm.operation} operation before editing ports")
current = {port.port: port for port in vm.ports or ()}
if mode == "set":
current = {port.port: port for port in ports}
elif mode == "open":
for port in ports:
current.setdefault(port.port, port)
else:
for port in ports:
current.pop(port.port, None)
vm.ports = tuple(current[number] for number in sorted(current))
vm.operation = "policy"
self._save(vm)
return self._sync_ports_locked(vm)
def sync_ports(self, identifier: str) -> VmRecord:
require_root()
with self._state_store.operation_lock():
return self._sync_ports_locked(self._load_for_operation(identifier))
def _sync_ports_locked(self, vm: VmRecord) -> VmRecord:
self._not_deleting(vm)
if vm.ports is None:
raise ConflictError("legacy VM has no managed policy; stop it and use ports set")
if vm.operation not in {None, "policy"}:
raise ConflictError(f"finish the pending {vm.operation} operation before syncing ports")
observation = self._process.observe(vm)
if observation == "unknown":
raise ConflictError("cannot enforce policy for a VM with unknown process identity")
vm.operation = "policy"
self._save(vm)
try:
if observation == "alive":
if not self._network.has_tap(vm):
raise UvmError("running VM has no owned network attachment")
self._network.firewall.apply(vm)
self._network.activate_tap(vm)
vm.status = "running"
else:
if vm.status != "stopped" or vm.tap_owned:
self._cleanup_runtime(vm)
vm.status = "stopped"
vm.operation, vm.last_error = None, None
self._save(vm)
return vm
except BaseException as error:
# Never restore an older, more permissive policy after a failed close.
containment_errors = []
for contain in (
lambda: self._network.deactivate_tap(vm),
lambda: self._network.firewall.apply(vm, quarantine=True),
):
try:
contain()
except UvmError as containment_error:
containment_errors.append(str(containment_error))
detail = f"; containment errors: {'; '.join(containment_errors)}" if containment_errors else ""
self._record_failure(vm, f"policy pending for {vm.id}: {error}{detail}")
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
raise UvmError(vm.last_error) from error
def resolve_targets(self, identifiers: Sequence[str], *, all_vms: bool = False) -> list[VmRecord]:
if all_vms and identifiers:
raise ValidationError("--all cannot be combined with explicit VM targets")
state = self._state_store.load()
if all_vms:
return list(state.vms.values())
if not identifiers:
raise ValidationError("specify a VM or --all")
resolved = [self._state_store.resolve(state, identifier) for identifier in identifiers]
return list({vm.id: vm for vm in resolved}.values())
def run_many(self, action: str, vm_ids: Sequence[str], *, timeout: float | None = None) -> list[OperationResult]:
require_root()
if action not in {"start", "stop", "restart", "delete", "ports-sync"}:
raise ValidationError(f"invalid lifecycle action: {action}")
results = []
with self._state_store.operation_lock():
for index, vm_id in enumerate(vm_ids):
try:
vm = self._load_for_operation(vm_id, exact_id=True)
if action == "start":
self._start_locked(vm)
elif action == "stop":
self._stop_locked(vm, timeout=timeout)
elif action == "delete":
self._delete_locked(vm, timeout=timeout)
elif action == "ports-sync":
self._sync_ports_locked(vm)
else:
self._not_deleting(vm)
self._stop_locked(vm, timeout=timeout)
self._start_locked(vm)
results.append(OperationResult(vm_id, action, "succeeded"))
except NotFoundError:
results.append(OperationResult(vm_id, action, "skipped", "already absent"))
except KeyboardInterrupt:
results.append(OperationResult(vm_id, action, "interrupted", "interrupted"))
results.extend(OperationResult(remaining, action, "skipped", "not attempted after interruption")
for remaining in vm_ids[index + 1:])
break
except (UvmError, OSError) as error:
results.append(OperationResult(vm_id, action, "failed", str(error)))
return results
def list_vms(self) -> list[ListedVm]:
state = self._state_store.load()
listed: list[ListedVm] = []
for vm in state.vms.values():
observed_status = vm.status
if vm.status in {"starting", "running", "stopping", "terminating"}:
if not self._process.is_alive(vm):
observed_status = "dead"
listed.append(ListedVm(vm=vm, observed_status=observed_status))
return listed
return [self._listed(vm) for vm in self._state_store.load().vms.values()]
def inspect(self, identifier: str) -> ListedVm:
return self._listed(self._state_store.find_by_id_or_ip(identifier))
def _listed(self, vm: VmRecord) -> ListedVm:
observation = self._process.observe(vm)
status = vm.status
if observation == "unknown":
status = "unknown"
elif observation == "alive" and status == "stopped":
status = "running"
elif observation == "gone" and status == "running":
status = "dead"
policy = ("inactive" if observation == "gone" and vm.status == "stopped" and vm.operation is None
else self._network.firewall.status(vm))
if vm.ports is None and policy == "inactive":
policy = "unmanaged"
return ListedVm(vm, status, policy)
def find_for_ssh(self, identifier: str) -> VmRecord:
return self._state_store.find_by_id_or_ip(identifier)
def stop(self, vm_id: str) -> VmRecord:
"""Stop one VM while retaining its private writable disk for future use."""
def log_path(self, identifier: str) -> Path:
vm = self._state_store.find_by_id_or_ip(identifier)
self._validate_paths(vm)
return Path(vm.log)
require_root()
with self._state_store.operation_lock():
return self._stop_locked(vm_id)
def _stop_locked(self, vm_id: str) -> VmRecord:
vm = self._begin_transition(vm_id, "stopping")
def diagnostics(self) -> list[dict]:
checks = [("root", require_root), ("kvm", check_kvm),
("guest assets", self._images.installed_assets),
("private firewall", self._network.firewall.check_available)]
results = []
for name, check in checks:
try:
self._process.terminate(vm)
self._network.delete_tap(vm.tap)
except BaseException as error:
vm.status = "failed"
vm.last_error = f"failed to stop VM: {error}"
vm.updated_at = self._now()
self._replace_vm_if_present(vm)
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
if isinstance(error, UvmError):
raise error
raise UvmError(vm.last_error) from error
check()
results.append({"name": name, "ok": True, "message": "available"})
except (UvmError, OSError) as error:
results.append({"name": name, "ok": False, "message": str(error)})
for command in ("ip", "iptables", "nft", "conntrack", "debugfs", "openssl", "ssh-keygen", "ssh"):
path = shutil.which(command)
results.append({"name": command, "ok": path is not None, "message": path or "not installed"})
return results
vm.status = "stopped"
vm.pid = None
vm.process_start_time = None
vm.last_error = None
vm.updated_at = self._now()
self._replace_vm(vm)
def _load_for_operation(self, identifier: str, *, exact_id: bool = False) -> VmRecord:
vm = (self._state_store.get(identifier) if exact_id
else self._state_store.find_by_id_or_ip(identifier))
self._validate_paths(vm)
if self._process.adopt_legacy_identity(vm):
self._save(vm)
return vm
def destroy(self, vm_id: str) -> VmRecord:
"""Stop a VM, remove its private resources, then release its allocation."""
def _validate_paths(self, vm: VmRecord) -> None:
if re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,80}", vm.id) is None:
raise ConflictError("invalid persisted VM ID; refusing resource access")
directory = self._settings.vm_dir(vm.id).absolute()
if directory.is_symlink():
raise ConflictError("VM directory must not be a symbolic link")
for field, filename in (("socket", "firecracker.sock"), ("config", "config.json"),
("log", "firecracker.log"), ("disk", "rootfs.ext4")):
path = getattr(vm, field)
if field == "disk" and not path:
continue # Legacy records may have used a shared template, never delete it.
if Path(path).absolute() != directory / filename or Path(path).is_symlink():
raise ConflictError(f"unsafe persisted VM {field} path")
require_root()
with self._state_store.operation_lock():
return self._destroy_locked(vm_id)
@staticmethod
def _not_deleting(vm: VmRecord) -> None:
if vm.operation == "delete" or vm.status == "terminating":
raise ConflictError(f"deletion is pending for {vm.id}; retry delete instead")
def _destroy_locked(self, vm_id: str) -> VmRecord:
vm = self._begin_transition(vm_id, "terminating")
try:
self._process.terminate(vm)
self._network.delete_tap(vm.tap)
try:
shutil.rmtree(self._settings.vm_dir(vm.id))
except FileNotFoundError:
pass
except BaseException as error:
vm.status = "failed"
vm.last_error = f"failed to destroy VM: {error}"
vm.updated_at = self._now()
self._replace_vm_if_present(vm)
if isinstance(error, (KeyboardInterrupt, SystemExit)):
raise
if isinstance(error, UvmError):
raise error
raise UvmError(vm.last_error) from error
self._remove_vm_if_present(vm.id)
return vm
@staticmethod
def _validated_ports(ports: tuple[PortRule, ...]) -> tuple[PortRule, ...]:
if not isinstance(ports, tuple) or any(not isinstance(port, PortRule) for port in ports):
raise ValidationError("ports must be a tuple of private TCP port rules")
return parse_ports([f"{port.port}:{port.protocol}" for port in ports])
def _reserve_vm(self, spec: VmSpec) -> VmRecord:
with self._state_store.transaction() as state:
if spec.name is not None and any(spec.name in {vm.name, vm.id} for vm in state.vms.values()):
raise ConflictError(f"VM name already exists: {spec.name}")
vm_id = new_vm_id()
used_taps = {existing_vm.tap for existing_vm in state.vms.values()}
while vm_id in state.vms or self._network.tap_name(vm_id) in used_taps:
used_taps = {vm.tap for vm in state.vms.values()}
while vm_id in state.vms or any(vm.name == vm_id for vm in state.vms.values()) or self._network.tap_name(vm_id) in used_taps:
vm_id = new_vm_id()
guest_ip = self._network.allocate_ip(state.vms.values(), spec.guest_ip)
mac = self._network.mac_for(state.next_mac_index)
state.next_mac_index += 1
now = self._now()
runtime_dir = self._settings.vm_dir(vm_id)
directory = self._settings.vm_dir(vm_id)
vm = VmRecord(
id=vm_id,
cpu=spec.cpu,
ram_mib=spec.ram_mib,
guest_ip=str(guest_ip),
gateway=str(self._settings.gateway),
tap=self._network.tap_name(vm_id),
mac=mac,
socket=str(runtime_dir / "firecracker.sock"),
config=str(runtime_dir / "config.json"),
log=str(runtime_dir / "firecracker.log"),
disk=str(runtime_dir / "rootfs.ext4"),
username=spec.username,
password=spec.password,
status="starting",
created_at=now,
updated_at=now,
id=vm_id, name=spec.name, cpu=spec.cpu, ram_mib=spec.ram_mib,
guest_ip=str(guest_ip), gateway=str(self._settings.gateway),
network=str(self._settings.network), bridge=self._settings.bridge,
tap=self._network.tap_name(vm_id), mac=mac,
socket=str(directory / "firecracker.sock"), config=str(directory / "config.json"),
log=str(directory / "firecracker.log"), disk=str(directory / "rootfs.ext4"),
username=spec.username, password=spec.password, ports=spec.ports,
status="starting", operation="preparing", created_at=int(self._clock()), updated_at=int(self._clock()),
)
state.vms[vm.id] = vm
return vm
def _begin_transition(self, vm_id: str, target_status: str) -> VmRecord:
with self._state_store.transaction() as state:
try:
vm = state.vms[vm_id]
except KeyError as error:
raise UvmError(f"VM not found: {vm_id}") from error
if vm.status in {"stopping", "terminating"}:
raise UvmError(f"VM operation is already in progress: {vm_id}")
vm.status = target_status
vm.updated_at = self._now()
state.vms[vm_id] = vm
return vm
def _remove_runtime(self, vm: VmRecord) -> None:
self._validate_paths(vm)
directory = self._settings.vm_dir(vm.id)
if directory.exists():
if not vm.runtime_owned:
raise ConflictError(f"refusing to remove unowned VM directory {directory}")
if self._process.observe(vm) != "gone":
raise ConflictError("refusing to remove disk without confirmed process exit")
shutil.rmtree(directory)
vm.runtime_owned = False
def _replace_vm(self, vm: VmRecord) -> None:
def _save(self, vm: VmRecord) -> None:
vm.updated_at = int(self._clock())
with self._state_store.transaction() as state:
if vm.id not in state.vms:
raise UvmError(f"VM not found: {vm.id}")
raise NotFoundError(f"VM not found: {vm.id}")
state.vms[vm.id] = vm
def _replace_vm_if_present(self, vm: VmRecord) -> None:
with self._state_store.transaction() as state:
if vm.id in state.vms:
state.vms[vm.id] = vm
def _record_failure(self, vm: VmRecord, message: str) -> None:
vm.status, vm.last_error = "failed", message
self._save(vm)
def _remove_vm_if_present(self, vm_id: str) -> None:
def _remove_record(self, vm_id: str) -> None:
with self._state_store.transaction() as state:
state.vms.pop(vm_id, None)
def _rollback_create(self, vm: VmRecord, tap_created: bool) -> list[str]:
errors: list[str] = []
if vm.pid is not None:
try:
self._process.terminate(vm)
except Exception as error:
errors.append(f"process: {error}")
if tap_created:
try:
self._network.delete_tap(vm.tap)
except Exception as error:
errors.append(f"network: {error}")
try:
shutil.rmtree(self._settings.vm_dir(vm.id))
except FileNotFoundError:
pass
except OSError as error:
errors.append(f"files: {error}")
return errors
def _now(self) -> int:
return int(self._clock())
uvm/network.py
+66 -1
View File
@@ -9,7 +9,8 @@ from ipaddress import IPv4Address
from .config import Settings
from .domain import VmRecord
from .errors import TapCreationError, UvmError, ValidationError
from .errors import ConflictError, TapCreationError, UvmError, ValidationError
from .firewall import FirewallManager
from .system import CommandRunner
@@ -19,6 +20,7 @@ class NetworkManager:
def __init__(self, settings: Settings, runner: CommandRunner) -> None:
self._settings = settings
self._runner = runner
self.firewall = FirewallManager(settings, runner)
def allocate_ip(
self,
@@ -95,6 +97,69 @@ class NetworkManager:
raise UvmError(f"could not determine whether existing interface {bridge} is a bridge") from error
if kind != "bridge":
raise UvmError(f"configured bridge {bridge} exists but is not a Linux bridge")
if links[0]["linkinfo"].get("info_data", {}).get("stp_state", 0) != 0:
raise UvmError("private ports require an STP-disabled UVM bridge")
def prepare_tap(self, vm: VmRecord) -> None:
"""Create an owned, attached TAP down; policy must precede activation."""
if self._link(vm.tap) is not None:
raise ConflictError(f"TAP device already exists: {vm.tap}")
try:
self._runner.run(("ip", "tuntap", "add", "dev", vm.tap, "mode", "tap"), capture=True)
vm.tap_owned = True
self._runner.run(("ip", "link", "set", "dev", vm.tap, "alias", f"uvm:{vm.id}"), capture=True)
self._runner.run(("ip", "link", "set", vm.tap, "master", vm.bridge or self._settings.bridge), capture=True)
self._runner.run(("ip", "link", "set", "dev", vm.tap, "type", "bridge_slave",
"group_fwd_mask", "16384"), capture=True)
except BaseException:
if vm.tap_owned:
try:
self.delete_tap(vm.tap)
vm.tap_owned = False
except UvmError:
pass
raise
def activate_tap(self, vm: VmRecord) -> None:
if self._owned_link(vm) is None:
raise UvmError(f"TAP device missing: {vm.tap}")
self._runner.run(("ip", "link", "set", vm.tap, "up"), capture=True)
def deactivate_tap(self, vm: VmRecord) -> None:
if self._owned_link(vm) is not None:
self._runner.run(("ip", "link", "set", vm.tap, "down"), capture=True)
def delete_vm_tap(self, vm: VmRecord) -> None:
if self._owned_link(vm) is not None:
self.delete_tap(vm.tap)
vm.tap_owned = False
def has_tap(self, vm: VmRecord) -> bool:
return self._owned_link(vm) is not None
def _link(self, name: str) -> dict | None:
result = self._runner.run(("ip", "-j", "-d", "link", "show"), capture=True)
try:
links = json.loads(result.stdout)
if not isinstance(links, list) or any(not isinstance(link, dict) for link in links):
raise ValueError("invalid interface inventory")
matches = [link for link in links if link.get("ifname") == name]
if len(matches) > 1:
raise ValueError("ambiguous interface inventory")
return matches[0] if matches else None
except (ValueError, TypeError) as error:
raise UvmError(f"cannot inspect network interfaces: {error}") from error
def _owned_link(self, vm: VmRecord) -> dict | None:
link = self._link(vm.tap)
if link is None:
return None
info = link.get("linkinfo", {})
is_tap = info.get("info_kind") == "tun" and info.get("info_data", {}).get("type") == "tap"
legacy = vm.ports is None and vm.tap_owned and not link.get("ifalias")
if not is_tap or (link.get("ifalias") != f"uvm:{vm.id}" and not legacy):
raise ConflictError(f"refusing to modify unowned TAP {vm.tap}")
return link
def create_tap(self, name: str) -> None:
existing = self._runner.run(("ip", "link", "show", name), check=False, capture=True)
uvm/routers/vms.py
+24 -11
View File
@@ -2,12 +2,12 @@
from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi import APIRouter, Depends, status
from ..api_models import DestroyResponse, VmCreateRequest, VmResponse, vm_response
from ..app import Application
from ..domain import VmSpec
from ..validation import parse_cpu, parse_password, parse_ram, parse_username
from ..domain import PortRule, VmSpec
from ..validation import parse_cpu, parse_name, parse_password, parse_ports, parse_ram, parse_username
from .dependencies import get_authorized_application
@@ -19,7 +19,11 @@ def list_vms(
application: Application = Depends(get_authorized_application),
) -> list[VmResponse]:
return [
vm_response(listed.vm, observed_status=listed.observed_status)
vm_response(
listed.vm,
observed_status=listed.observed_status,
policy_status=listed.policy_status,
)
for listed in application.lifecycle.list_vms()
]
@@ -29,6 +33,9 @@ def create_vm(
request: VmCreateRequest,
application: Application = Depends(get_authorized_application),
) -> VmResponse:
ports = parse_ports([f"{rule.port}:{rule.protocol}" for rule in request.ports or ()])
if not request.no_default_ports and not any(rule.port == 22 for rule in ports):
ports = tuple(sorted((*ports, PortRule(22)), key=lambda rule: rule.port))
vm = application.lifecycle.create(
VmSpec(
cpu=parse_cpu(str(request.cpu)),
@@ -36,9 +43,14 @@ def create_vm(
guest_ip=request.guest_ip,
username=parse_username(request.username),
password=parse_password(request.password.get_secret_value()),
name=parse_name(request.name) if request.name is not None else None,
ports=ports,
)
)
return vm_response(vm)
policy_status = "unmanaged" if vm.ports is None else "unknown"
if vm.ports is not None and vm.status == "running" and vm.operation is None:
policy_status = "enforced"
return vm_response(vm, policy_status=policy_status)
@router.post("/{vm_id}/stop", response_model=VmResponse)
@@ -46,7 +58,7 @@ def stop_vm(
vm_id: str,
application: Application = Depends(get_authorized_application),
) -> VmResponse:
return vm_response(application.lifecycle.stop(vm_id))
return vm_response(application.lifecycle.stop(vm_id), policy_status="inactive")
@router.delete("/{vm_id}", response_model=DestroyResponse)
@@ -63,8 +75,9 @@ def get_vm(
identifier: str,
application: Application = Depends(get_authorized_application),
) -> VmResponse:
for listed in application.lifecycle.list_vms():
vm = listed.vm
if identifier in {vm.id, vm.guest_ip}:
return vm_response(vm, observed_status=listed.observed_status)
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="VM not found")
listed = application.lifecycle.inspect(identifier)
return vm_response(
listed.vm,
observed_status=listed.observed_status,
policy_status=listed.policy_status,
)
uvm/server.py
+6 -7
View File
@@ -8,7 +8,7 @@ import ssl
from typing import Any
from .app import Application, build_application
from .errors import ConfigurationError, UvmError, ValidationError
from .errors import ConfigurationError, ConflictError, NotFoundError, UvmError, ValidationError
def create_api(application: Application | None = None, *, host: str | None = None) -> Any:
@@ -115,13 +115,12 @@ def _validate_server_settings(
def _http_status_for(error: UvmError) -> int:
if isinstance(error, NotFoundError):
return 404
if isinstance(error, ConflictError):
return 409
if isinstance(error, (ConfigurationError, ValidationError)):
return 422
message = str(error)
if message.startswith("VM not found:"):
return 404
if "operation is already in progress" in message:
return 409
if "needs root" in message:
if "needs root" in str(error):
return 403
return 500
uvm/state.py
+17 -8
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
import fcntl
import json
import os
import re
import tempfile
from collections.abc import Iterator
from contextlib import contextmanager
@@ -12,7 +13,7 @@ from pathlib import Path
from .config import Settings
from .domain import State, VmRecord
from .errors import StateError, UvmError
from .errors import ConflictError, NotFoundError, StateError
from .system import ensure_data_directories
@@ -63,16 +64,24 @@ class StateStore:
try:
return state.vms[vm_id]
except KeyError as error:
raise UvmError(f"VM not found: {vm_id}") from error
raise NotFoundError(f"VM not found: {vm_id}") from error
def find_by_id_or_ip(self, identifier: str) -> VmRecord:
state = self.load()
if identifier in state.vms:
return self.resolve(self.load(), identifier)
@staticmethod
def resolve(state: State, identifier: str) -> VmRecord:
if re.fullmatch(r"vm-[0-9a-f]{32}", identifier):
try:
return state.vms[identifier]
for vm in state.vms.values():
if vm.guest_ip == identifier:
return vm
raise UvmError(f"VM not found: {identifier}")
except KeyError as error:
raise NotFoundError(f"VM not found: {identifier}") from error
matches = [vm for vm in state.vms.values() if identifier in {vm.id, vm.name, vm.guest_ip}]
if len(matches) > 1:
raise ConflictError(f"ambiguous VM identifier: {identifier}; use an exact VM ID")
if not matches:
raise NotFoundError(f"VM not found: {identifier}")
return matches[0]
@contextmanager
def _locked(self) -> Iterator[None]:
uvm/system.py
+2 -1
View File
@@ -5,6 +5,7 @@ from __future__ import annotations
import os
import shlex
import subprocess
import sys
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from pathlib import Path
@@ -47,7 +48,7 @@ class CommandRunner:
check=False,
text=True,
input=input_text,
stdout=subprocess.PIPE if capture else None,
stdout=subprocess.PIPE if capture else sys.stderr,
stderr=subprocess.PIPE if capture else None,
timeout=timeout,
)
uvm/validation.py
+34
View File
@@ -4,9 +4,11 @@ from __future__ import annotations
import math
import re
from collections.abc import Sequence
from decimal import Decimal, InvalidOperation
from .errors import ValidationError
from .domain import PortRule
_RAM_PATTERN = re.compile(r"\s*(\d+(?:\.\d+)?)\s*([BKMG]i?B?)?\s*", re.IGNORECASE)
@@ -81,3 +83,35 @@ def parse_password(value: str) -> str:
if any(ord(character) < 32 or ord(character) == 127 for character in value):
raise ValidationError("password must not contain control characters")
return value
def parse_name(value: str) -> str:
if re.fullmatch(r"[a-z][a-z0-9-]{0,62}", value) is None:
raise ValidationError("VM name must be 1-63 lowercase letters, digits, or hyphens, starting with a letter")
if re.fullmatch(r"vm-[0-9a-f]{32}", value):
raise ValidationError("VM names must not impersonate generated VM IDs")
return value
def parse_ports(values: Sequence[str]) -> tuple[PortRule, ...]:
ports: dict[int, PortRule] = {}
for value in values:
for item in value.split(","):
match = re.fullmatch(r"\s*(\d+)(?::(TCP|HTTP|HTTPS))?\s*", item, re.IGNORECASE)
if match is None:
raise ValidationError(f"invalid private port: {item!r}; use PORT[:TCP|HTTP|HTTPS]")
rule = PortRule(int(match.group(1)), match.group(2) or "TCP")
if rule.port in ports and ports[rule.port] != rule:
raise ValidationError(f"conflicting labels for TCP port {rule.port}")
ports[rule.port] = rule
return tuple(ports[number] for number in sorted(ports))
def parse_timeout(value: str) -> float:
try:
timeout = float(value)
except (ValueError, TypeError) as error:
raise ValidationError("timeout must be a finite non-negative number of seconds") from error
if not math.isfinite(timeout) or timeout < 0:
raise ValidationError("timeout must be a finite non-negative number of seconds")
return timeout