feat: introduce Nest workspace for local Firecracker management
- Added a new server module for the Nest workspace, integrating with UVM's API. - Updated project description to reflect the new Nest functionality. - Created tests for the Nest server and CLI, ensuring proper functionality and error handling. - Refactored existing UVM server code to separate Nest and UVM functionalities. - Removed the old UVM dashboard assets package as it is now part of the Nest workspace. - Enhanced error messages for clarity regarding server shutdown commands.
This commit is contained in:
+61
-33
@@ -1,11 +1,12 @@
|
||||
# UVM Developer Guide
|
||||
# Nest Developer Guide
|
||||
|
||||
This document is the developer handoff for the entire UVM codebase. Read it
|
||||
This document is the developer handoff for Nest and its UVM backend. Read it
|
||||
before changing lifecycle behavior, host networking, Firecracker launch
|
||||
semantics, persistent state, or the HTTP API.
|
||||
|
||||
`uvm` is a local, privileged Firecracker microVM manager. It is an
|
||||
application with a CLI and optional FastAPI management server. It is not a
|
||||
Nest owns the workspace dashboard and its server entry point. Its `uvm`
|
||||
component is a local, privileged Firecracker microVM manager with a CLI and
|
||||
standalone FastAPI management API. These are application components, not a
|
||||
general-purpose Python library or a multi-host control plane.
|
||||
|
||||
## 1. Goals and Boundaries
|
||||
@@ -13,7 +14,7 @@ general-purpose Python library or a multi-host control plane.
|
||||
UVM installs host/guest assets, provisions private disks, and manages named
|
||||
VMs and private IPv4 TCP allowlists on one host. The CLI offers retained-disk
|
||||
start/stop/restart, CPU/RAM/storage resizing, deletion, fixed-target bulk operations, inspection, SSH,
|
||||
logs, doctor, and explicit port edits/sync. The optional API and local dashboard
|
||||
logs, doctor, and explicit port edits/sync. The UVM API and Nest dashboard
|
||||
expose health, installation, VM lifecycle controls, and managed port-policy
|
||||
controls, host diagnostics, bounded VM-log tails, and fixed-target bulk
|
||||
operations. Interactive SSH remains CLI-only.
|
||||
@@ -30,16 +31,18 @@ flag reference, see [USAGE.md](USAGE.md).
|
||||
|
||||
## 2. First Run
|
||||
|
||||
All UVM-specific files live in this directory. Start here:
|
||||
All Nest and UVM files live in this project. Start here:
|
||||
|
||||
```sh
|
||||
python3 -m pip install -e '.[test]'
|
||||
python3 -B -m unittest discover -s tests -v
|
||||
./nest.py --help
|
||||
./uvm.py --help
|
||||
```
|
||||
|
||||
The `uvm` console script is registered by `pyproject.toml`. During source
|
||||
development, `./uvm.py` and `python3 -B -m uvm` are equivalent entry points.
|
||||
The `nest` and `uvm` console scripts are registered by `pyproject.toml`.
|
||||
During source development, `./nest.py` and `python3 -B -m nest` are equivalent
|
||||
Nest entry points; `./uvm.py` and `python3 -B -m uvm` launch UVM.
|
||||
|
||||
Requirements for real host operations:
|
||||
|
||||
@@ -63,7 +66,15 @@ Packet integration tests must remain inside disposable isolated namespaces.
|
||||
├── README.md # Project overview
|
||||
├── USAGE.md # Operator command/deployment guide
|
||||
├── pyproject.toml # Package metadata and dependencies
|
||||
├── uvm.py # Executable source-tree launcher
|
||||
├── nest.py # Executable Nest source-tree launcher
|
||||
├── nest/
|
||||
│ ├── __main__.py # `python -m nest`
|
||||
│ ├── cli.py # Nest workspace serve command
|
||||
│ ├── server.py # Nest composition of the dashboard and UVM API
|
||||
│ └── dashboard/
|
||||
│ ├── router.py # Nest dashboard document route
|
||||
│ └── static/ # Packaged HTML, CSS, JavaScript, and fonts
|
||||
├── uvm.py # Executable UVM source-tree launcher
|
||||
├── uvm/
|
||||
│ ├── __main__.py # `python -m uvm`
|
||||
│ ├── cli.py # argparse, CLI rendering, serve dispatch
|
||||
@@ -83,11 +94,10 @@ Packet integration tests must remain inside disposable isolated namespaces.
|
||||
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port intent
|
||||
│ ├── api_models.py # Pydantic HTTP request/response models
|
||||
│ ├── server.py # FastAPI factory and Uvicorn launcher
|
||||
│ ├── dashboard/ # Packaged local dashboard HTML, CSS, and JS
|
||||
│ ├── routers/
|
||||
│ │ ├── dashboard.py # Dashboard document route
|
||||
│ │ ├── diagnostics.py # Host diagnostic endpoint
|
||||
│ │ ├── dependencies.py # API application/token dependencies
|
||||
│ │ ├── exit.py # Authenticated server shutdown
|
||||
│ │ ├── health.py # Liveness endpoint
|
||||
│ │ ├── installation.py # Installation endpoint
|
||||
│ │ └── vms.py # VM lifecycle endpoints
|
||||
@@ -106,8 +116,10 @@ are intentionally small and operationally focused.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
CLI[CLI: uvm] --> APP[app.py composition root]
|
||||
HTTP[FastAPI server] --> APP
|
||||
DASH[Nest dashboard] --> NEST[nest/server.py workspace]
|
||||
NEST --> HTTP[UVM management API]
|
||||
CLI[CLI: uvm] --> APP[uvm/app.py composition root]
|
||||
HTTP --> APP
|
||||
APP --> INSTALL[Installer]
|
||||
APP --> LIFE[LifecycleService]
|
||||
APP --> STATE[StateStore]
|
||||
@@ -125,8 +137,9 @@ flowchart LR
|
||||
FC --> KVM[/dev/kvm]
|
||||
```
|
||||
|
||||
The public entry points are thin adapters. The lifecycle layer is the only
|
||||
place that coordinates multiple host resources for a VM. Lower-level modules
|
||||
The public entry points are thin adapters. Nest composes the UVM API and adds
|
||||
the workspace dashboard; UVM does not import Nest. The lifecycle layer is the
|
||||
only place that coordinates multiple host resources for a VM. Lower-level modules
|
||||
must not import `cli.py`, `server.py`, or `lifecycle.py`.
|
||||
|
||||
Leaf modules are `config.py`, `domain.py`, `errors.py`, and `validation.py`.
|
||||
@@ -163,6 +176,11 @@ checks, not per-VM repair or guest application probes.
|
||||
|
||||
### HTTP Path
|
||||
|
||||
`nest/server.py` owns the workspace application. Its `create_app()` composes
|
||||
`uvm.server.create_api()` with the dashboard document and static assets from
|
||||
`nest/dashboard/`. The same-origin API paths and `X-UVM-Token` authentication
|
||||
are shared with the standalone UVM API.
|
||||
|
||||
Routers validate the token and Pydantic request, invoke application services,
|
||||
and render whitelisted response models. Lifecycle, managed port-policy,
|
||||
diagnostics, and bounded log-tail operations have explicit HTTP routes for the
|
||||
@@ -171,9 +189,11 @@ SSH has no HTTP endpoint. The public dashboard shell does not bypass token
|
||||
validation: it stores a supplied token only for the active browser session and
|
||||
sends it with management requests.
|
||||
|
||||
Use only `uvm --serve` or `./uvm.py --serve` to launch the API. The
|
||||
`create_api()` factory requires an explicit host specifically to prevent it
|
||||
from being used as an insecure generic Uvicorn factory.
|
||||
Launch the workspace with `nest --serve` or `./nest.py --serve`. The standalone
|
||||
backend uses `uvm --serve` or `./uvm.py --serve`. Both `create_app()` and
|
||||
`create_api()` require an explicit host so validation covers the intended bind.
|
||||
Nest passes its factory to the shared UVM server runner, which enforces token
|
||||
and TLS requirements and wires graceful server shutdown.
|
||||
|
||||
## 6. Application Composition
|
||||
|
||||
@@ -837,8 +857,9 @@ Python's safe tar extraction filter.
|
||||
### Server Startup
|
||||
|
||||
See [USAGE.md](USAGE.md) for dependency installation, loopback deployment,
|
||||
TLS deployment, and authenticated curl examples. Launch through `uvm --serve`
|
||||
using the Python environment with the installed API dependencies. Every server
|
||||
TLS deployment, and authenticated curl examples. Launch the Nest workspace
|
||||
through `nest --serve`, or the standalone backend through `uvm --serve`, using
|
||||
the Python environment with the installed API dependencies. Every server
|
||||
requires `UVM_API_TOKEN`, even at `127.0.0.1`. Non-loopback binds additionally
|
||||
require `UVM_API_TLS_CERT` and `UVM_API_TLS_KEY`; paired certificate/key files
|
||||
are loaded into an `ssl.SSLContext` before Uvicorn starts.
|
||||
@@ -851,10 +872,13 @@ or secret manager when that matters.
|
||||
|
||||
### API Contract
|
||||
|
||||
The root redirect and dashboard document are Nest routes. Both server entry
|
||||
points expose the remaining management routes below.
|
||||
|
||||
| Method | Route | Router | Notes |
|
||||
|---|---|---|---|
|
||||
| `GET` | `/` | `server.py` | Redirect to `/dashboard` |
|
||||
| `GET` | `/dashboard` | `dashboard.py` | Public static dashboard shell |
|
||||
| `GET` | `/` | `nest/server.py` | Redirect to `/dashboard` (Nest only) |
|
||||
| `GET` | `/dashboard` | `nest/dashboard/router.py` | Public Nest dashboard shell (Nest only) |
|
||||
| `GET` | `/health` | `health.py` | Unauthenticated liveness response |
|
||||
| `POST` | `/exit` | `exit.py` | Request graceful Uvicorn shutdown |
|
||||
| `GET` | `/diagnostics` | `diagnostics.py` | Host readiness checks |
|
||||
@@ -878,14 +902,14 @@ All management routes use `get_authorized_application()` and require:
|
||||
X-UVM-Token: <UVM_API_TOKEN>
|
||||
```
|
||||
|
||||
`POST /exit` is a management route. `run_server()` creates the Uvicorn server,
|
||||
stores it on the FastAPI application state, and runs it in-process. The route
|
||||
returns `202` before a background task sets `should_exit`, allowing Uvicorn to
|
||||
deliver the response and perform its normal graceful shutdown. It does not
|
||||
stop or delete any managed VM.
|
||||
`POST /exit` is a management route. The shared `uvm.server.run_server()` creates
|
||||
the Uvicorn server, stores it on the FastAPI application state, and runs it
|
||||
in-process. The route returns `202` before a background task sets `should_exit`,
|
||||
allowing Uvicorn to deliver the response and perform its normal graceful
|
||||
shutdown for Nest or UVM. It does not stop or delete any managed VM.
|
||||
|
||||
Request schemas reject unexpected fields. `UvmError` instances are rendered as
|
||||
JSON errors by `server.py`; Pydantic request validation uses FastAPI's normal
|
||||
JSON errors by `uvm/server.py`; Pydantic request validation uses FastAPI's normal
|
||||
422 response format. Create accepts `name: str | null`, `ports: list | null`,
|
||||
and strict boolean `no_default_ports` (default false). Port objects contain a
|
||||
strict integer `port` in 1-65535 and uppercase `protocol` TCP/HTTP/HTTPS (TCP
|
||||
@@ -996,11 +1020,13 @@ Ordinary tests require no root, KVM, Firecracker, or host network mutations.
|
||||
They cover validation, schema migration/locking/permissions, artifact trust,
|
||||
private disks/credentials, network/firewall rendering and readback, conntrack
|
||||
semantics, pidfd/legacy identity, lifecycle recovery/resume/bulk targeting,
|
||||
CLI parsing/output, and server security. Mock/fixture policy checks do not
|
||||
prove the kernel's actual packet behavior.
|
||||
CLI parsing/output, Nest dashboard delivery, and server security. Mock/fixture
|
||||
policy checks do not prove the kernel's actual packet behavior.
|
||||
|
||||
FastAPI route tests are conditionally enabled when both `fastapi` and `httpx`
|
||||
are installed. They should be enabled in CI using the `test` extra.
|
||||
`test_nest_server.py` checks the workspace launcher, relocated frontend assets,
|
||||
authenticated machine controls, API documentation, and graceful shutdown.
|
||||
When `nft` and `conntrack` are installed, command-parser tests append an invalid
|
||||
final command/option so parsing is checked without executing any mutation.
|
||||
These checks supplement fixture-based readback and cleanup tests; they do not
|
||||
@@ -1075,8 +1101,10 @@ on an isolated Linux host:
|
||||
`get_authorized_application()`, and register new routers in `create_api()`.
|
||||
Add API tests and update all route tables. New CLI commands do not implicitly
|
||||
authorize new HTTP endpoints; `/health` remains the only unauthenticated
|
||||
operational API route, while `/` and `/dashboard` only serve the public
|
||||
dashboard shell.
|
||||
operational API route. Nest owns the public `/` and `/dashboard` routes.
|
||||
- Dashboard: keep the document route and frontend assets in `nest/dashboard/`.
|
||||
Compose workspace-level routes in `nest/server.py`; machine-management
|
||||
operations continue through the UVM API and its application services.
|
||||
- Host primitives: use the specific network/firewall/install/process module
|
||||
and `CommandRunner` for external commands. Define ownership, idempotence,
|
||||
and compensation before invoking a primitive from lifecycle code. Use fake
|
||||
@@ -1137,7 +1165,7 @@ Before submitting a change:
|
||||
|
||||
1. Run the complete unit suite.
|
||||
2. Run `git diff --check`.
|
||||
3. Confirm a new feature stays inside the `uvm/` project directory.
|
||||
3. Keep workspace/dashboard features in `nest/` and machine-management features in `uvm/`.
|
||||
4. Verify command/API errors use `UvmError` rather than raw tracebacks.
|
||||
5. Verify host mutations have a lock owner and cleanup path.
|
||||
6. Verify persistent state remains readable across upgrades.
|
||||
|
||||
Reference in New Issue
Block a user