kstyagi/nest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: kstyagi/nest:dev
Branches Tags
kstyagi/nest:main kstyagi/nest:dev
..
compare: kstyagi/nest:4663fd19235bb015b732ab7b9a741da70ea0a2f7
Branches Tags
kstyagi/nest:dev kstyagi/nest:main
5 Commits
dev .. 4663fd1923
Author SHA1 Message Date
kstyagi 4663fd1923 Merge pull request 'Dev' (#5) from dev into main 
Reviewed-on: #5
 2026-09-25 14:20:33 +00:00
kstyagi 283eb4ecf7 Merge pull request 'Dev' (#4) from dev into main 
Reviewed-on: #4
 2026-09-23 02:53:57 +00:00
kstyagi 10a58daa56 Merge pull request 'feat: enhance user account management during VM creation with automatic user creation and metadata preservation' (#3) from dev into main 
Reviewed-on: #3
 2026-09-21 18:46:55 +00:00
kstyagi 309f28018e Merge pull request 'Dev' (#2) from dev into main 
Reviewed-on: #2
 2026-09-21 15:50:24 +00:00
kstyagi 99f9ffa57d Merge pull request 'Dev' (#1) from dev into main 
Reviewed-on: #1
 2026-09-20 03:41:01 +00:00
54 changed files with 81 additions and 6125 deletions
Expand all files Collapse all files
DEVELOPER.md
+13 -225
View File
@@ -19,7 +19,7 @@ expose health, installation, VM lifecycle controls, and managed port-policy
controls, host diagnostics, bounded VM-log tails, and fixed-target bulk
operations, and bounded guest command execution over vsock. Interactive SSH remains CLI-only.
There is no general VM autostart or background reconciler, multi-host control plane,
There is no autostart or background reconciler, multi-host control plane,
tenancy/quotas, or UVM-provided Jailer/cgroup/namespace hardening. Private port
filtering is not public-port publishing or a hardened multi-tenant boundary;
see Current Production Gaps for the remaining scope limits.
@@ -71,9 +71,6 @@ Packet integration tests must remain inside disposable isolated namespaces.
│ ├── __main__.py # `python -m nest`
│ ├── cli.py # Nest workspace serve command
│ ├── server.py # Nest composition of the dashboard and UVM API
│ ├── sessions.py # Live guest terminals and browser-independent session lifecycle
│ ├── session_store.py # Private SQLite session history and bounded transcripts
│ ├── session_routes.py # VM-scoped session HTTP and WebSocket endpoints
│ └── dashboard/
│ ├── router.py # Nest dashboard document route
│ └── static/ # Packaged HTML, CSS, JavaScript, and fonts
@@ -94,11 +91,6 @@ Packet integration tests must remain inside disposable isolated namespaces.
│ ├── known_hosts.py # Deletion-only cleanup of local SSH host-key caches
│ ├── guest_agent.py # Standalone guest daemon, job scopes, and bounded wire protocol
│ ├── guest_exec.py # Host-side guest sessions and execution results
│ ├── guest_terminal.py # Async PTY transport over a process-pinned guest connection
│ ├── tool_catalog.py # Allowlisted tools, Python selections, and pinned release recipes
│ ├── guest_tools.py # Standalone guest-only package/download installer and probes
│ ├── tool_store.py # Atomic per-VM job history and inventory observations
│ ├── tools.py # Bounded background tool installation workers
│ ├── network.py # IP/MAC/TAP/bridge/NAT/forward rules
│ ├── firewall.py # Owned nft bridge tables and offline conntrack cleanup
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port intent
@@ -110,7 +102,6 @@ Packet integration tests must remain inside disposable isolated namespaces.
│ │ ├── exit.py # Authenticated server shutdown
│ │ ├── health.py # Liveness endpoint
│ │ ├── installation.py # Installation endpoint
│ │ ├── tools.py # Catalog, inventory, and installation job endpoints
│ │ └── vms.py # VM lifecycle endpoints
│ └── firecracker/
│ ├── config.py # Firecracker JSON and boot arguments
@@ -223,8 +214,6 @@ process:
| FirecrackerProcessManager | `FirecrackerProcessManager` | VMM process control |
| LifecycleService | `LifecycleService` | Ordered VM resource changes and rollback |
| Installer | `Installer` | Host packages and artifacts |
| ToolService | `ToolService` | Browser-independent guest installation queue and workers |
| ToolJobStore | Constructed by lifecycle/tool services | Per-VM installation intent, results, and cached inventory |
Composition can disable command tracing with `emit=None`; `create_api()` does
so when it constructs its own application. The CLI supplies an emitter that
@@ -252,12 +241,10 @@ Default runtime storage is `/var/lib/uvm`:
│ ├── config.json
│ ├── firecracker.log
│ ├── firecracker.sock
│ ├── tools.json
│ └── vsock.sock
├── state.json
├── state.lock
├── operations.lock
├── tool-worker.lock
└── integrity.json
```
@@ -281,8 +268,6 @@ state file.
`VmSpec` is the validated create request. `VmRecord` is the persisted local
record. Do not make API models or CLI namespaces the persistence format.
`VmSpec.tools` is a tuple of validated `ToolSelection` values. Tool job state
uses its own schema-1 `vms/<id>/tools.json`; the VM registry remains schema 3.
| Record Fields | Purpose |
|---|---|
@@ -374,8 +359,7 @@ a ruleset/attachment snapshot, not application health or a reachability test.
3. Reserve ID/name/IP/MAC/ports with `status=starting`, `operation=preparing`.
4. Create the owned mode-0700 runtime directory, sparse-copy a mode-0600 private
disk, resize ext4 to the requested storage capacity, provision credentials/host
keys, the managed guest network profile, and the agent, then persist `disk_ready=True`.
If tools were selected, persist their queued job in the owned runtime directory.
keys and the managed guest network profile, then persist `disk_ready=True`.
5. Ensure bridge/egress networking and prepare an owned, attached TAP **down**.
6. Apply/read back the nftables bridge policy, then activate the TAP.
7. Write Firecracker config; persist `operation=spawning`, `spawn_pending=True`,
@@ -383,13 +367,10 @@ a ruleset/attachment snapshot, not application health or a reachability test.
8. Publish PID/start time/boot ID through `on_spawn` before socket readiness
or API configuration; clear the pending-spawn guard.
9. Configure Firecracker and `InstanceStart`, verify policy enforcement again,
then save `running` with no pending operation.
then save `running` with no pending operation.
This records both intent and acquired ownership across crash windows. Never
make a managed TAP usable before enforcement or mark a partial disk ready.
Tool workers claim the queued intent after this operation releases its lock.
Tool installation errors are job results on the successfully created VM and
never enter create's resource rollback path.
### Rollback Rules
@@ -520,8 +501,7 @@ snapshot produced before CLI confirmation. It reloads by **exact ID**, not a
reusable name/IP. Results are `succeeded`, `failed`, `skipped` (already absent),
or `interrupted`; ordinary failures do not undo other successes or stop later
targets. Interruptions skip the remaining targets. Read-only views and API
startup never reconcile lifecycle or firewall intent. The tool worker resumes
explicitly queued installation requests, as described under Guest Tools below.
startup never reconcile; there is no autostart or background repair loop.
## 9. Firecracker Integration
@@ -808,7 +788,7 @@ do not import its unimplemented topology checks into current documentation.
## 12. Installer and Host Commands
`Installer` is the owner of host apt installation, Firecracker artifact
`Installer` is the only owner of apt installation, Firecracker artifact
handling, downloads, and integrity-manifest publication.
`Settings` detects and normalizes the host architecture at construction time
@@ -897,8 +877,8 @@ or secret manager when that matters.
### API Contract
The root redirect, dashboard document, and terminal-session routes belong to
Nest. Both server entry points expose the UVM management routes below.
The root redirect and dashboard document are Nest routes. Both server entry
points expose the remaining management routes below.
| Method | Route | Router | Notes |
|---|---|---|---|
@@ -909,7 +889,7 @@ Nest. Both server entry points expose the UVM management routes below.
| `GET` | `/diagnostics` | `diagnostics.py` | Host readiness checks |
| `POST` | `/install` | `installation.py` | `{ "force": false }` body |
| `GET` | `/vms` | `vms.py` | List persisted VM records and observed status |
| `POST` | `/vms` | `vms.py` | `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports`, `tools` |
| `POST` | `/vms` | `vms.py` | `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `POST` | `/vms/actions` | `vms.py` | Fixed-ID snapshot bulk lifecycle action |
| `GET` | `/vms/{identifier}` | `vms.py` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/start` | `vms.py` | Boot a retained private disk |
@@ -921,18 +901,7 @@ Nest. Both server entry points expose the UVM management routes below.
| `POST` | `/vms/{vm_id}/ports/sync` | `vms.py` | Exact ID only; reapply managed policy |
| `POST` | `/vms/{vm_id}/exec` | `vms.py` | Exact ID; `command`, strict `sudo`, bounded `timeout_seconds` |
| `POST` | `/vms/{vm_id}/agent/install` | `vms.py` | Exact ID; install/upgrade the agent on a stopped disk |
| `GET` | `/tools` | `tools.py` | Authenticated catalog and supported Python versions |
| `GET` | `/vms/{vm_id}/tools` | `tools.py` | Cached inventory/history; `refresh=true` probes the running guest |
| `POST` | `/vms/{vm_id}/tools/install` | `tools.py` | Exact ID; `tools`, `start_if_stopped`, `restart`, optional `request_id`; HTTP 202 |
| `GET` | `/vms/{vm_id}/tools/jobs/{job_id}` | `tools.py` | Exact VM-scoped installation job and bounded output |
| `GET` | `/vms/{identifier}/logs` | `vms.py` | Bounded non-following log tail |
| `GET` | `/vms/{vm_id}/sessions` | `nest/session_routes.py` | Saved session list (Nest only) |
| `POST` | `/vms/{vm_id}/sessions` | `nest/session_routes.py` | Launch a guest PTY; optional rows/cols in a JSON object (Nest only) |
| `GET` | `/vms/{vm_id}/sessions/{session_id}` | `nest/session_routes.py` | VM-scoped session metadata (Nest only) |
| `PATCH` | `/vms/{vm_id}/sessions/{session_id}` | `nest/session_routes.py` | Rename via a strict `name` body; identity retained (Nest only) |
| `DELETE` | `/vms/{vm_id}/sessions/{session_id}` | `nest/session_routes.py` | Close channel, join output reader, delete history; 204 (Nest only) |
| `POST` | `/vms/{vm_id}/sessions/{session_id}/close` | `nest/session_routes.py` | End a shell; retain history (Nest only) |
| `WS` | `/vms/{vm_id}/sessions/{session_id}/terminal` | `nest/session_routes.py` | Authenticated replay/input/resize (Nest only) |
All management routes use `get_authorized_application()` and require:
@@ -1000,8 +969,7 @@ The synchronous HTTP response exposes only exit code, stdout/stderr, timeout,
and truncation. Captured text uses UTF-8 replacement. Nonzero process exits are
HTTP 200 results; agent failures are 502, and failures before submission are 503.
Lost/malformed responses after submission carry `outcome_unknown: true`. There
is no command retry, durable job registry, streaming response, or reconnection
for these one-shot exec requests.
is no command retry, durable job registry, streaming response, or reconnection.
The root systemd agent accepts only host-CID connections and limits concurrent
connections/jobs to eight. Each request creates a delegated cgroup v2 job scope.
@@ -1009,8 +977,7 @@ A fresh child interpreter joins the scope before initializing the chosen user's
supplementary groups, GID, UID, home, and environment, then execs `/bin/sh -c`.
`sudo` selects guest root directly; otherwise the recorded username is used.
Password authentication is not involved, and a recorded `root` login remains
root for both flag values. One-shot commands have closed stdin; Nest terminal
sessions use the separate PTY channel described below.
root for both flag values. Input is closed and there is no PTY.
Both output pipes are drained concurrently. The guest enforces the deadline,
signals the process group, escalates via `cgroup.kill`, and confirms cleanup.
@@ -1019,29 +986,15 @@ PGID reuse before reap. Normal exits clean up remaining background descendants
too. An unconfirmed cleanup is an unknown outcome. Stopping the agent service
also kills its job scopes; dropping an HTTP connection does not cancel a job.
The v5 Ubuntu builder installs Python, the terminal-capable service, and an
ownership manifest. Agent setup is mandatory on every compatible image:
`create()` provisions the VM ID/username in root-only `/etc/uvm-agent.json` before
marking the disk ready, and `_start_locked()` ensures the agent after confirmed
runtime cleanup and pending resize, before booting. This covers stopped starts,
restarts, and resizing. There is no image-marker allowlist for agent setup.
An observed live or ambiguous VMM never permits offline agent writes.
`_guest_agent_current()` checks the expected payloads, identity, file metadata,
ownership manifest, and enabled service without mutating the disk. Current
installations return immediately. The explicit `agent/install` operation forces
a staged recheck for manual repair. The original manifest-less v4 agent is
recognized by its exact published source digest.
The v4 Ubuntu builder installs Python and the service. Creation provisions the
VM ID/username in root-only `/etc/uvm-agent.json` before marking the disk ready.
Older templates are left for an explicit stopped-disk `agent/install` request.
Provisioning and upgrades stage a sparse copy, replay/check ext4, validate guest
path ownership and recorded agent-file hashes, enable the service, verify writes,
and publish atomically. A root-only guest manifest records managed file hashes
so later upgrades preserve guest edits. An older guest missing Python installs
it through apt inside the guest on its next boot; no guest binaries execute on
the host. This bootstrap requires working guest egress until the runtime is ready.
Incompatible custom images or agent setup failures prevent boot with an actionable
error; they are not silently skipped. New-VM failures roll back creation, while
retained-VM failures preserve the private disk for retry.
The guest needs systemd, Python 3.10+, built-in virtio-vsock, and cgroup v2 with
`cgroup.kill` (Linux 5.14+). Host socket paths are derived from Settings, so the
@@ -1050,142 +1003,6 @@ device and uses CID 3. The process adapter retains both sockets while identity
is uncertain and removes them only after confirmed VMM exit. IPv4 port policies
do not govern this host-only management channel.
### Guest Tools
`tool_catalog.py` is the source of truth for catalog metadata, normalized
`ToolSelection` values, and installer recipes. The API accepts catalog IDs and
explicit Python minors; executable text, download URLs, paths, and other runtime
versions are not request fields. There are ten tools and twelve selectable
entries when Python 3.11/3.12/3.13 are expanded. Update publisher versions, URLs,
and SHA-256 values together when changing a downloaded default.
`ToolService` owns two bounded worker threads by default. The shared UVM server
lifespan starts/stops them, so both Nest and the standalone API process the same
queue. An exclusive, mode-0600 `$UVM_BASE/tool-worker.lock` permits one management
server to own these workers per registry. Construction alone starts no threads.
Queued requests are dispatched on server startup; prior in-flight jobs are
marked interrupted with unknown outcomes after ownership has been acquired.
`LifecycleService.queue_tools()` validates an exact VM ID and persists a request
under the operation lock. Only one queued/active batch is allowed per VM.
Identical active requests are deduplicated. Optional request IDs also identify
completed jobs until they age out of the ten-job history; conflicting reuse is
an error. `prepare_tool_install()` performs an explicitly requested start or
restart under the normal lifecycle lock and returns the selected process identity.
Creation writes its tool job after disk provisioning and before boot.
Workers wait up to `Settings.tool_ready_timeout_s` (120 seconds) for the agent.
Each tool has `Settings.tool_install_timeout_s` (1,200 seconds), bounded by the
existing guest-exec protocol maximum. Readiness connection failures can be
retried before submission; an uncertain installation result is never replayed.
Each step passes `expected_identity` to `LifecycleService.execute()`, so a restart
between steps cannot retarget the batch to a new VMM. The guest boot ID also
detects in-guest reboots. Guest downloads and package waits hold no host operation
lock. Shutdown closes tracked waiting sockets, preserves interruption results,
and keeps worker ownership until outstanding host work has finished.
`guest_tools.py` is a standalone stdlib-only Python 3.10+ payload. Its source and
server-generated JSON recipes are shell-quoted into the bounded guest exec
request. Importing it on the host performs no installation. In the guest it:
- Checks Ubuntu 22.04 and the expected x86_64/ARM64 architecture before mutation.
- Serializes installation through `/run/uvm-tools.lock`, including inherited
lock ownership during package commands. A previous disconnected job can finish
in the guest, and this lock prevents overlapping retries.
- Uses non-interactive apt with bounded dpkg-lock waits and authenticated sources.
A pending marker permits a retry to finish its interrupted dpkg configuration.
PHP's repository key is checked against a pinned fingerprint and scoped with
`signed-by`.
- Checks pinned hashes before executing/extracting downloads. Archive extraction
validates paths/links, file types, and size bounds. Versioned payloads live in
`/opt/uvm-tools`; executable links and login-shell profiles have an ownership
manifest that detects guest edits.
- Checks command versions and companions as the configured guest account, including
npm/npx, Python pip/venv imports, and both Java tools. Python has versioned
executables; Conda uses per-user environments/caches. Get-started commands use
explicit POSIX shell hooks for activation in Nest's non-login shells.
- Captures a bounded 16 KiB output tail and returns structured JSON when each
step finishes. An ordinary recipe failure allows later tools to be attempted.
`ToolJobStore` keeps cached inventory and the last ten jobs in a mode-0600,
schema-1 `tools.json` in the existing private VM runtime directory. Atomic
replacement/fsync and short operation-lock transactions prevent races with
deletion or stale lifecycle record saves. Store writes require a still-registered,
owned VM directory and never recreate a deleted VM. VM deletion removes this
history with the runtime. Full inventory refreshes update `checked_at`; step
results also update individual tool observations. GET without `refresh=true`
returns cached observations, including the `stale` flag.
The dashboard's `tools.js` owns create selections, the VM Tools tab, job polling,
history, retries, and output rendering. Active jobs poll every 1.5 seconds while
visible; idle views poll cached status every five seconds. New selections use the
shared catalog. Navigation/logout invalidate in-flight UI responses, and tool
output is escaped before rendering.
### Nest Terminal Sessions
Nest owns session HTTP/WebSocket routes and history. UVM owns the process-pinned
connection and guest PTY. `LifecycleService.open_terminal()` validates an exact
running VM ID and connects under the operation lock, then opens the PTY outside
the lock. It never reconnects that socket to a replacement VMM.
The existing version-1 hello and exec request stay compatible. A terminal request
uses `{"version":1,"terminal":{"rows":24,"cols":80}}`. The guest replies with
`{"type":"ready"}` and exchanges length-prefixed JSON output/input/resize/close
messages. Bytes are base64 encoded; raw chunks are at most 8192 bytes, frames
16384 bytes, and dimensions 2–200 rows / 2–400 columns. Partial frames have finite
deadlines while idle shells can remain connected indefinitely. An old agent
rejects the new request; the host reports the stopped-disk upgrade procedure.
The guest opens a real PTY and spawns a fresh interpreter in a new OS session.
The child joins its delegated job cgroup, establishes the controlling terminal,
assumes the recorded account, and execs that account's interactive shell with
`TERM=xterm-256color`. Input, terminal echo, Ctrl+C, foreground job control, and
window-size changes are handled by the guest terminal driver. Normal exit,
explicit close, or host connection loss cleans up the job cgroup and processes.
`SessionManager` owns background output readers independently of WebSocket
attachments. Accepted creation, rename, close, and delete operations are tracked
and shielded from HTTP disconnects. Creation and termination share a session
lock, so deleting a starting session waits for its channel to be tracked and
then closes it. Deletion joins the output reader and its pending writes before
atomically removing the metadata and transcript rows. Browser navigation,
reload, and logout detach only
the browser. Nest shutdown closes guest connections and marks live sessions
interrupted. In-flight transcript writes finish before releasing store ownership.
`SessionStore` keeps metadata and raw output in
`$UVM_BASE/nest/sessions.sqlite3`, separate from the VM registry. A private
directory, mode-0600 files, an exclusive owner lock, and SQLite transactions
protect history. On reopening after a server restart, unfinished records become
interrupted. Session lookups always include the exact VM ID. Output is capped at
the newest 8 MiB per session with byte offsets and an explicit truncation marker;
session metadata is retained until explicit deletion. Session names are trimmed,
1–80 printable characters, and are display labels rather than identifiers.
Renaming updates only the name and modification time; generated default names
avoid labels still present after renames or deletions. Only PTY output is
recorded, not a separate input
or password log. Nest permits 32 active sessions; guest terminals share the
agent's eight connection slots with one-shot jobs.
HTTP routes use the existing `X-UVM-Token` dependency. Browser WebSockets check
the Origin and require `{"type":"auth","token":"...","offset":0}` as their
first frame within ten seconds. Tokens never enter query strings. Each output
message has its byte offset and base64 data; the browser acknowledges the next
offset after xterm renders the chunk. Only one chunk is outstanding, providing
backpressure. Reattachment resumes from an acknowledged offset; a truncated
prefix resets the viewer. Closed sessions use the same channel for read-only
replay. WebSocket transport support comes from the `websockets` dependency.
Renames notify attached viewers through updated session metadata. A deleted
session sends a `deleted` event to authenticated attachments; polling also clears
stale selections for completed sessions whose replay connection already closed.
The dashboard's Sessions tab and nested hash routes belong to the selected VM.
`sessions.js` owns attachment teardown, replay, reconnects, and the terminal
widget. Vendored xterm 5.5.0 and fit addon 0.10.0 (with MIT licenses) are served
locally under `static/vendor/`. Terminal keyboard input bypasses dashboard search
shortcuts. The widget has a 10,000-line scrollback and a 64 KiB per-paste limit.
## 14. Configuration Reference
The environment-variable table is in [README.md](README.md#configuration);
@@ -1271,35 +1088,6 @@ FastAPI route tests are conditionally enabled when both `fastapi` and `httpx`
are installed. They should be enabled in CI using the `test` extra.
`test_nest_server.py` checks the workspace launcher, relocated frontend assets,
authenticated machine controls, API documentation, and graceful shutdown.
`test_sessions.py` covers scoped persistence, interrupted-session recovery,
authentication, reconnects, output acknowledgements, and cancellation cleanup.
`test_guest_terminal.py` exercises actual PTYs, persistent shell state, resizing,
Ctrl+C, and child cleanup using temporary account homes and fake cgroup scopes.
Optional browser regression checks use Playwright against an isolated fixture
server with fake VMs and guest channels. With the project's test extra installed:
```sh
npm install --prefix /tmp/nest-browser playwright
/tmp/nest-browser/node_modules/.bin/playwright install chromium
NODE_PATH=/tmp/nest-browser/node_modules \
NEST_TEST_PYTHON="$(command -v python)" node tests/browser_sessions.cjs
```
This checks launch/input, reload/reconnect, session switching, per-VM isolation,
closed-session replay, rename persistence, active/ended deletion, cancellation,
multi-tab updates, and the mobile layout. It does not boot a real microVM.
With the same browser dependencies, `node tests/browser_tools.cjs` checks creation
selections, persisted background jobs across reload, installed-tool detection,
partial failure/retry, escaped output, history, stopped-VM startup, per-VM isolation,
multi-select installation, and mobile layout. Its fixture uses the real API,
worker threads, and atomic job store with simulated guest execution.
`tests/test_tool_*.py` cover validation, architecture-specific payloads, checksum
failure, archive traversal, exact-ID targeting, idempotency, recovery, and shutdown.
The opt-in real-VM catalog/runtime smoke test is documented in
[USAGE.md](USAGE.md#18-verification); run it on disposable x86_64 and ARM64 VMs.
When `nft` and `conntrack` are installed, command-parser tests append an invalid
final command/option so parsing is checked without executing any mutation.
These checks supplement fixture-based readback and cleanup tests; they do not
README.md
+11 -113
View File
@@ -310,9 +310,8 @@ and `/dashboard` routes belong to the Nest server.
The Nest dashboard includes a fleet overview, searchable machine collection,
private network and port-policy view, host diagnostics, and machine provisioning.
Open a machine to manage its lifecycle, install developer tools, launch terminal sessions, edit or sync
its port rules, copy its SSH command, or read recent logs. Select machines in the
collection for bulk operations.
Open a machine to manage its lifecycle, edit or sync its port rules, copy its SSH
command, or read recent logs. Select machines in the collection for bulk operations.
Use `Ctrl+K` / `⌘K` (or `/`) to find a machine or page. Fleet data refreshes every
five seconds while the tab is visible; host checks refresh every thirty seconds
and can also be run manually. Charts show observed machine states and allocated
@@ -330,7 +329,7 @@ The server exposes these routes:
| `GET` | `/diagnostics` | Run host readiness checks for the dashboard/system view |
| `POST` | `/install` | Install or refresh host assets; body: `{ "force": false }` |
| `GET` | `/vms` | List VM records and observed status |
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports`, `tools` |
| `POST` | `/vms` | Create; supports `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`, `ports`, `no_default_ports` |
| `POST` | `/vms/actions` | Run a fixed snapshot of selected or all VM lifecycle actions |
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/start` | Boot a retained private disk |
@@ -342,18 +341,7 @@ The server exposes these routes:
| `POST` | `/vms/{vm_id}/ports/sync` | Reapply a managed private port policy for an exact VM ID |
| `POST` | `/vms/{vm_id}/exec` | Execute a command/script inside a running VM over vsock, with optional root privileges |
| `POST` | `/vms/{vm_id}/agent/install` | Install/upgrade the execution agent on a stopped VM's private disk |
| `GET` | `/tools` | Guest tool catalog and supported Python versions |
| `GET` | `/vms/{vm_id}/tools` | Tool inventory and installation history; `refresh=true` probes the running guest |
| `POST` | `/vms/{vm_id}/tools/install` | Queue selected guest tools; HTTP 202 with a job ID |
| `GET` | `/vms/{vm_id}/tools/jobs/{job_id}` | Per-tool progress, results, and bounded installation output |
| `GET` | `/vms/{identifier}/logs` | Read a bounded, non-following tail of a VM log |
| `GET` | `/vms/{vm_id}/sessions` | List this VM's saved sessions (Nest) |
| `POST` | `/vms/{vm_id}/sessions` | Launch a guest terminal; body: `{}` or `{"rows":24,"cols":80}` (Nest) |
| `GET` | `/vms/{vm_id}/sessions/{session_id}` | Session metadata and status (Nest) |
| `PATCH` | `/vms/{vm_id}/sessions/{session_id}` | Rename a session; body: `{"name":"Deployment"}` (Nest) |
| `DELETE` | `/vms/{vm_id}/sessions/{session_id}` | Close the session and delete its saved history; returns 204 (Nest) |
| `POST` | `/vms/{vm_id}/sessions/{session_id}/close` | End the shell and retain its history (Nest) |
| `WS` | `/vms/{vm_id}/sessions/{session_id}/terminal` | Authenticated terminal input, output, resize, and replay (Nest) |
API port rules are objects such as `{"port": 80, "protocol": "HTTP"}`;
`no_default_ports` is a boolean, default `false`. Create also accepts `storage`,
@@ -369,95 +357,7 @@ Log reads use an optional `tail` query parameter from `0` to `1000`, default
Bulk requests use an action (`start`, `stop`, `restart`, `delete`, or
`ports-sync`) plus either `identifiers` or `all_vms: true`; they return a result
for each fixed target and do not prompt for confirmation. Interactive SSH
remains CLI-only. VM lifecycle changes are explicit; accepted tool installation
jobs are processed in the background.
### One-click Guest Tools
Choose tools under **New machine → Pick your tools** to install them automatically
after boot. For an existing VM, open **Machines → select a VM → Tools** and click
**Install**, or select several tools and click **Install selected**. A stopped VM
offers **Start & install**. Installation works over vsock with SSH disabled.
The initial catalog targets Ubuntu 22.04 guests on x86_64 and ARM64:
| Tool | Default |
|---|---|
| Git, wget, curl, htop | Ubuntu packages |
| Node.js | 24 LTS (24.21.0), including npm and npx |
| Python | 3.11, 3.12, and 3.13 with pip and venv; versions can coexist |
| JDK | OpenJDK 21 LTS, including `java` and `javac` |
| PHP | 8.5 CLI from the signed Ondřej Surý PHP PPA |
| Conda | Miniforge 26.5.3-0, using conda-forge |
| micromamba | 2.9.0 |
Node, Python, Miniforge, and micromamba downloads use pinned release checksums.
Python uses Astral's standalone distributions, exposed as `python3.11`,
`python3.12`, and `python3.13`. The guest's OS Python continues serving the agent.
Downloaded tools live under `/opt/uvm-tools`, with executable links in
`/usr/local/bin`. Conda environments and caches are writable by the configured
guest user. **Get started** shows commands, including shell initialization for
Conda/micromamba activation in Nest's non-login terminals.
Jobs run independently of the browser and show progress and captured output per
tool. Already available tools are verified and skipped. A failed tool can be
retried on the created VM. Guest internet access is required for downloads.
Restart the management server to load the feature; older running VMs may need
**Restart & install** to apply the current guest-agent setup.
The most recent ten jobs and cached inventory are stored in the VM's private
`tools.json` file. Queued requests resume when the server starts. An interrupted
in-flight installation has an uncertain outcome and may finish inside the guest;
use **Check installed tools** before retrying. See
[API examples and recovery](USAGE.md#install-guest-tools) for details.
### MicroVM Terminal Sessions
Open **Machines → select a microVM → Sessions**, then click **Launch session**.
Each session opens an interactive guest shell in the dashboard, with terminal
colors, keyboard input, Ctrl+C, resizing, and persistent shell state such as the
working directory. Sessions run as the VM's configured guest user; normal guest
`sudo` commands can request that user's password in the terminal.
The history list belongs to the selected VM. Reopen an active session to
continue using the same shell, or select an ended session to view its saved
output. Navigation, browser reloads, and signing out leave the shell running.
**End session** closes the shell and its processes. Stopping/restarting the VM
or the Nest server ends live sessions; their saved history remains available.
Use **Rename** in a history row or the terminal toolbar to give any session a
custom name. Names are trimmed and accept 1–80 printable characters; they persist
across reloads and server restarts. The session keeps its identity and shell.
**Delete** removes a session and its saved terminal output after confirmation,
closing its shell first if it is active. Both actions are scoped to the session's
microVM.
Nest stores session metadata and the latest **8 MiB of output per session** in
`$UVM_BASE/nest/sessions.sqlite3` (default `/var/lib/uvm/nest/sessions.sqlite3`).
The viewer indicates truncated output. Raw keyboard input is not logged
separately; the transcript contains what the guest terminal outputs. History
files are private to the host account running Nest. One Nest server owns
sessions for a given VM registry at a time, with up to 32 active sessions; the
guest agent shares its eight connection slots between terminals and exec calls.
**Vsock and the guest agent are configured by default for every microVM.**
Creation installs and enables the agent on every compatible image, including
cached older templates and custom images. Each stopped-disk boot also installs
or updates it automatically; an already-current installation is a read-only
check. Existing running VMs receive the setup on their next restart.
The **Sessions → Guest terminal support → Repair / update terminal support**
control is available for manual repair on a stopped VM. The guest-agent/kernel
requirements described below apply to terminals.
Terminal support adds the Python `websockets` dependency. In your micromamba
environment, install the current project and launch with its interpreter:
```sh
micromamba activate nest
python -m pip install -e '.[test]'
sudo env UVM_API_TOKEN="$UVM_API_TOKEN" \
"$CONDA_PREFIX/bin/python" -m nest --serve
```
remains CLI-only. The API and dashboard are not background reconcilers.
### Execute Commands Without SSH
@@ -478,15 +378,14 @@ uses guest root without a password prompt. A VM configured with username `root`
still runs as root when `sudo` is false.
Execution uses a guest systemd agent over Firecracker vsock. It works with SSH
disabled and with an empty TCP allowlist. Every Firecracker boot attaches the
vsock device, and creation/start/restart/resize ensure the guest service is
installed and enabled before boot. Restart the management server to load this
behavior, then restart older VMs once to apply it automatically.
disabled and with an empty TCP allowlist. Rebuild the default Ubuntu template
with `uvm install --force` using your configured checksums to include the agent
and Python in new VMs. Restart the management server to load these routes.
Installation stages and verifies a private disk copy when changes are needed,
preserving guest data and credentials. If Python is missing, the installed
bootstrap service acquires it inside the guest on boot; this one-time step needs
guest internet access. The default Ubuntu template already includes Python.
For an existing VM, call **stop → `POST /vms/{vm_id}/agent/install` → start**.
Installation stages and verifies a private disk copy, preserving guest data and
credentials. If Python is missing, the installed bootstrap service acquires it
inside the guest on the next boot; this one-time step needs guest internet access.
The guest needs Python 3.10+, systemd with cgroup v2, and a vsock-capable Linux
5.14+ kernel; UVM's default Linux 6.1 kernel provides the required drivers.
@@ -555,7 +454,6 @@ defaults are username `root` and password `root`. A custom image must provide:
- `/etc/passwd`, `/etc/shadow`, and `/etc/ssh/sshd_config`
- `/etc/group` and `/bin/sh` when creating a new user
- A root-owned, setuid `/usr/bin/sudo` and `/etc/sudoers` for custom login users
- systemd and apt for the automatically installed vsock guest agent (Ubuntu 22.04+)
Custom login users receive password-authenticated sudo access. Inside a new VM:
USAGE.md
+17 -188
View File
@@ -801,8 +801,7 @@ The installed command is `nest --serve`; `python3 -B -m nest --serve` also
works from the source checkout. Nest uses `UVM_API_TOKEN` and the `UVM_*` host
settings for its machine-management backend. To launch the standalone UVM API,
use `./uvm.py --serve` with the same host/port options and configuration. That
server exposes the UVM management endpoints and `/docs`; Nest serves the
dashboard and terminal-session endpoints.
server exposes the management endpoints and `/docs`; Nest serves the dashboard.
The API token is required even for a loopback server. All VM, diagnostics, and
installation routes require it. The dashboard shell, liveness route, and FastAPI's
@@ -813,50 +812,6 @@ with no whitespace. Use a protected environment file or secret manager rather
than recording a real token in shell history; do not share tokens or guest
passwords in reports.
### Use Sessions Inside a MicroVM
1. Open **Machines** and select the microVM.
2. Start it if it is stopped.
3. Click **Launch session**, or open its **Sessions** tab and launch there.
4. Type directly in the embedded terminal. Each session keeps its own shell,
current directory, environment, and foreground program.
5. Use **Session history** to switch between that VM's sessions. Active sessions
reconnect to their existing shell; ended sessions replay saved output.
6. Use **End session** when finished, or run `exit` inside the shell.
7. Use **Rename** in the history row or terminal toolbar to save a custom label.
Names accept 1–80 printable characters, with surrounding whitespace trimmed.
8. Use **Delete** to permanently remove a session and its saved output. Confirming
deletion closes an active shell before removing its history. **End session**
retains history for later viewing.
Sessions continue while you navigate elsewhere, reload, or sign out of the
dashboard. VM shutdown/restart or Nest shutdown ends the shell. Session metadata,
start/end times, exit status, and output survive Nest restarts in
`$UVM_BASE/nest/sessions.sqlite3`; files are mode `0600` in a mode-`0700` directory.
Only sessions belonging to the selected exact VM ID are shown. Reusing a VM name
does not attach another VM's history.
Renaming works for active and ended sessions and keeps their IDs and URLs stable.
Deleting the open session clears the terminal selection; other sessions continue
to appear under their own history entries.
The retained transcript is capped at its latest 8 MiB, and the terminal viewer
has a 10,000-line scrollback. A truncation note appears when older output is
discarded. Terminal input supports pastes up to 64 KiB at a time. Guest output
is saved even while no browser is attached.
Vsock and the guest agent are set up automatically for every microVM during
creation and on each stopped-disk boot. This includes compatible older and custom
images. Restart an older VM once to apply the setup; **Sessions → Guest terminal
support → Repair / update terminal support** remains available for manual repair
while stopped. If the agent is still starting or installing Python, wait for it
to become ready before launching a session.
The terminal transport uses the guest agent over vsock. It logs in as the VM's
configured username; `sudo` inside the terminal uses normal guest authentication.
Nest supports 32 active sessions per server; each guest agent shares eight
connection slots with its command-execution endpoint. Run one Nest session
manager per `UVM_BASE` registry. Reverse proxies must forward WebSocket upgrades.
### Start a TLS Server
Non-loopback binds require a valid TLS certificate and key. The files must be
@@ -920,18 +875,7 @@ curl --cacert /path/to/ca.pem \
| `POST` | `/vms/{vm_id}/ports/sync` | Reapply the policy for an exact VM ID |
| `POST` | `/vms/{vm_id}/exec` | Execute a guest command/script over vsock, with optional root privileges |
| `POST` | `/vms/{vm_id}/agent/install` | Install/upgrade the agent on a stopped VM's disk |
| `GET` | `/tools` | Authenticated tool catalog and Python version options |
| `GET` | `/vms/{vm_id}/tools` | Cached tool inventory/history; `refresh=true` checks the running guest |
| `POST` | `/vms/{vm_id}/tools/install` | Queue tool installation; HTTP 202 with a job ID |
| `GET` | `/vms/{vm_id}/tools/jobs/{job_id}` | Installation progress, results, and captured output |
| `GET` | `/vms/{identifier}/logs` | Read a bounded, non-following log tail |
| `GET` | `/vms/{vm_id}/sessions` | List saved sessions for an exact VM ID (Nest) |
| `POST` | `/vms/{vm_id}/sessions` | Launch a terminal, with `{}` or optional `rows`/`cols` (Nest) |
| `GET` | `/vms/{vm_id}/sessions/{session_id}` | Inspect a session (Nest) |
| `PATCH` | `/vms/{vm_id}/sessions/{session_id}` | Rename with `{"name":"Deployment"}`; returns updated metadata (Nest) |
| `DELETE` | `/vms/{vm_id}/sessions/{session_id}` | End the shell and remove its record/output; HTTP 204 (Nest) |
| `POST` | `/vms/{vm_id}/sessions/{session_id}/close` | End a session and retain its output (Nest) |
| `WS` | `/vms/{vm_id}/sessions/{session_id}/terminal` | Authenticated live terminal/replay (Nest) |
Individual lifecycle requests do not expose a CLI dry-run flag and do not prompt
for confirmation. `timeout` is an optional finite, non-negative query parameter
@@ -977,7 +921,7 @@ curl -X POST \
```
Request fields are `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`,
`ports`, `no_default_ports`, and `tools`. The name has the same validation as the CLI.
`ports`, and `no_default_ports`. The name has the same validation as the CLI.
`ports` is an array of objects (or null/omitted), not a CLI colon string.
Each object has a strict integer `port` from 1 to 65535 and an optional,
case-sensitive `protocol` of `TCP` (default), `HTTP`, or `HTTPS`.
@@ -992,106 +936,6 @@ The response never includes the password. Omitting credentials still uses
labels return HTTP `422`. Example passwords above are placeholders; real
secrets in curl arguments can enter shell history and process listings.
### Install Guest Tools
The dashboard provides tool checkboxes during creation and a **Tools** tab on
each VM. Choose **Install** for a single tool or **Install selected** for a batch.
**Start & install** also boots a stopped VM. The catalog covers Ubuntu 22.04 on
x86_64 and ARM64, with guest internet access for package downloads.
Use the same API selections during creation:
```sh
curl -X POST \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"dev-tools","username":"developer","password":"replace-me","tools":[{"id":"git"},{"id":"nodejs"},{"id":"python","version":"3.12"}]}' \
"$UVM_API_URL/vms"
```
The VM response is HTTP 201 after creation. Selected tools install asynchronously;
read the returned VM's `/tools` endpoint for its job. Omitted `tools` defaults to
an empty list. An explicit null is rejected.
For an existing VM, use its exact ID from create/list:
```sh
export VM_ID='<exact-vm-id>'
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/tools"
curl -X POST \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"tools":[{"id":"jdk"},{"id":"conda"},{"id":"micromamba"}],"start_if_stopped":true,"request_id":"dev-tools-1"}' \
"$UVM_API_URL/vms/$VM_ID/tools/install"
# Use the job ID from the HTTP 202 response.
export JOB_ID='<tools-job-id>'
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/tools/jobs/$JOB_ID"
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/tools?refresh=true"
```
Selections are objects with `id` and an optional `version`. IDs are `git`,
`nodejs`, `python`, `jdk`, `wget`, `curl`, `htop`, `php`, `conda`, and
`micromamba`. Python requires `version: "3.11"`, `"3.12"`, or `"3.13"`;
the other tools use the curated catalog default and accept no version override.
Up to twelve selections are accepted, with identical selections deduplicated.
Multiple Python versions can be installed together.
Install requests require a nonempty `tools` list. `start_if_stopped` and `restart`
are strict booleans, both default false. Set `restart: true` to restart first and
refresh guest-agent setup. In the dashboard, the **Restart & install** recovery
button appears when agent readiness fails. Unknown tools/options return 422;
unavailable VM states or a different active install return 409.
`request_id` is an optional idempotency key of 1–80 letters, digits, underscores,
or hyphens. Reusing it with the same request returns the retained job; different
selections/options return 409. Use a new key for an intentional retry. Idempotency
keys last as long as their job in the ten-job history. One batch per VM is active
at a time, with two VM workers by default and sequential tools within a batch.
Jobs move through `queued`, `starting`, `waiting`, and `installing`, then
`succeeded`, `failed`, or `interrupted`. Each item reports `installed`,
`already_installed`, `failed`, `interrupted`, or `skipped`, plus its detected
version, error, bounded output, and truncation flag. Ordinary recipe failures
allow subsequent tools to run; uncertain execution outcomes end that batch.
Output is captured as a 16 KiB tail per tool and returned when the step finishes.
Default limits are 120 seconds for agent readiness and 1,200 seconds per tool.
`GET /vms/{vm_id}/tools` returns cached inventory, the latest job, and summaries
of the last ten jobs. `refresh=true` probes all tools in a running VM and updates
`checked_at`; it does not install software or start a stopped VM. `stale` marks
old or stopped-VM observations. Installed binaries and their version checks run
as the configured guest user. Matching installed runtimes are accepted, including
tools installed outside Nest. The install operation is not a general updater.
Tool jobs survive browser navigation, reload, and logout. The active management
server owns the queue through `$UVM_BASE/tool-worker.lock`. After server restart,
queued work resumes; in-flight jobs become interrupted with
`outcome_unknown: true`. The last command may still finish inside the guest.
Refresh inventory before retrying; a guest-side lock blocks overlapping installs.
Stopping/restarting the VM also interrupts a running batch. Successful tools and
the VM's private disk remain available. Deleting the VM removes its tool history.
Use versioned Python commands, for example `python3.12 -m venv .venv`. For Conda
or micromamba activation in a Nest terminal, initialize the current POSIX shell:
```sh
eval "$(conda shell.posix hook)"
conda create -n myenv python=3.13
conda activate myenv
# In another shell, for micromamba:
eval "$(micromamba shell hook --shell posix)"
micromamba create -n myenv -c conda-forge python=3.13
micromamba activate myenv
```
The installer also adds login-shell profiles under `/etc/profile.d`. See
[the catalog](README.md#one-click-guest-tools) for default versions and sources.
### Start, Restart, Stop, and Delete
```sh
@@ -1294,26 +1138,26 @@ print(result["exit_code"])
#### Prepare New or Existing VMs
The default Ubuntu template includes Python 3 and `uvm-agent.service`. UVM also
installs the service automatically on compatible older/custom images during
creation and before booting a retained private disk. Every VM gets a Firecracker
vsock device by default. Agent files, VM identity configuration, the ownership
manifest, and the enabled service are checked; an already-current installation
does not rewrite the disk.
The default Ubuntu template recipe v4 includes Python 3 and `uvm-agent.service`.
Rebuild it with `uvm install --force` using the checksum settings from the
installation section, then create new VMs. Cached older templates and custom
images require the explicit agent installation below. Restart a running Nest or
UVM API server to load the new code before making these requests.
Restart a running Nest or UVM API server to load the new code. Existing VMs pick
up the setup on their next start or restart. For a currently running VM:
For an existing VM, run these requests in order, checking each succeeds:
```sh
curl --fail-with-body -X POST -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/restart"
"$UVM_API_URL/vms/$VM_ID/stop"
curl --fail-with-body -X POST -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/agent/install"
curl --fail-with-body -X POST -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/start"
```
The explicit `POST /vms/{vm_id}/agent/install` endpoint is available for manual
repair while stopped. It returns
`{"id":"...","status":"installed","restart_required":true}` and does not start
the VM itself. All agent installation requires a proven-offline private disk;
updates are checked on a sparse copy before publication. Guest files,
Installation returns `{"id":"...","status":"installed","restart_required":true}`.
It requires a cleanly stopped, proven-offline private disk and does not start the
VM itself. Updates are checked on a sparse copy before publication. Guest files,
credentials, and SSH keys are retained; customized agent files are preserved
rather than overwritten. Repeating installation also upgrades unedited managed
agent files using their recorded checksums.
@@ -1322,7 +1166,7 @@ The guest image must provide systemd and apt (Ubuntu 22.04 or newer is the
supported baseline). If Python 3.10+ is missing, a bootstrap service installs
`python3` inside the VM on its next boot. This one-time install needs guest DNS,
internet access, and working apt repositories. The service retries failed startup;
execution returns 503 until it can connect. Current default templates already include
execution returns 503 until it can connect. New v4 templates already include
the runtime and do not need this download.
The guest kernel needs built-in `CONFIG_VSOCKETS=y`, `CONFIG_VIRTIO_VSOCKETS=y`,
@@ -1701,21 +1545,6 @@ exercise real child processes with fake cgroup scopes, local Unix-socket protoco
fixtures, and temporary ext4 upgrades; they do not prove the real vsock/cgroup
path inside a VM.
The tools smoke test installs the full catalog in a prepared **disposable Ubuntu
22.04 VM with a non-root login and no open TCP ports**. It downloads packages,
checks all runtimes, compiles Java, creates Python venvs, and creates/activates
temporary Conda and micromamba environments as the guest user. Run it separately
on x86_64 and ARM64 hosts:
```sh
UVM_TOOLS_INTEGRATION=1 UVM_TOOLS_VM_ID='<disposable-vm-id>' \
python3 -B -m unittest discover -s tests/integration -p test_guest_tools.py -v
```
Set `UVM_API_URL` and `UVM_API_TOKEN` as above. The test is opt-in and was not run
against a real microVM during this implementation. Automated API and browser
checks use simulated guest execution and real job persistence/workers.
## 19. Get Help
```sh
nest/dashboard/static/css/sessions.css
-60
View File
@@ -1,60 +0,0 @@
.machine-tabs { display: flex; gap: 8px; margin: 0 0 24px; padding-bottom: 12px; border-bottom: 1px solid var(--line); }
.machine-tab { display: inline-flex; align-items: center; gap: 8px; padding: 8px 16px; border-radius: 22px; color: var(--muted); font-size: 11px; }
.machine-tab.active { background: var(--white); color: var(--ink); box-shadow: var(--shadow-card); }
.machine-tab .icon { width: 14px; height: 14px; }
.session-section-heading { margin-bottom: 14px; }
.session-workspace { display: grid; grid-template-columns: 205px minmax(0, 1fr); gap: 16px; align-items: stretch; }
.session-history { padding: 14px 10px; min-width: 0; display: flex; flex-direction: column; }
.session-history > header { display: flex; align-items: center; justify-content: space-between; padding: 0 6px 10px; }
.session-history h3 { font-size: 11px; }
.session-history > .panel-footnote { margin-top: auto; padding: 15px 8px 0; }
.session-list { display: grid; gap: 6px; max-height: 510px; overflow-y: auto; }
.session-list-empty { padding: 24px 9px; color: var(--muted); font-size: 10px; line-height: 1.8; }
.session-item { display: grid; gap: 9px; min-width: 0; padding: 12px 10px; border: 1px solid transparent; border-radius: 12px; color: var(--ink-soft); }
.session-link { display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 7px; }
.session-item-actions { display: flex; gap: 15px; padding-top: 8px; border-top: 1px solid var(--line); }
.session-item-actions .quiet-link { min-height: 24px; font-size: 9px; }
.session-item:hover { background: #fff8; }
.session-item.active { background: var(--white); border-color: #d9d0e4; }
.session-item strong { overflow-wrap: anywhere; font-size: 11px; }
.session-item small { grid-column: 1 / -1; color: var(--muted); font-size: 9px; }
.session-state { align-self: center; width: 6px; height: 6px; border-radius: 50%; background: var(--lilac); }
.session-state.active { background: var(--sage); box-shadow: 0 0 0 3px var(--sage-soft); }
.session-state.failed, .session-state.interrupted { background: var(--peach); }
.terminal-panel { min-width: 0; overflow: hidden; border-radius: 17px; background: #25222d; color: #e8e3ef; box-shadow: var(--shadow-card); }
.terminal-toolbar { min-height: 56px; display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: 12px; padding: 12px 16px; border-bottom: 1px solid #ffffff12; background: #302c39; }
.terminal-toolbar > div:first-child { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; min-width: 0; }
.terminal-toolbar strong { font-size: 11px; overflow-wrap: anywhere; }
.terminal-status { color: #b8afc6; font-size: 9px; }
.terminal-toolbar .button { color: #e8e3ef; background: #ffffff0c; border-color: #ffffff17; }
.terminal-toolbar .button:hover { background: #ffffff19; }
#session-delete { color: #e5b4aa; }
.session-rename-dialog .field { margin-top: 22px; }
#session-rename-error { color: var(--red); }
.terminal-lights { display: inline-flex; gap: 4px; margin-right: 3px; }
.terminal-lights i { height: 6px; width: 6px; border-radius: 50%; background: var(--peach); }
.terminal-lights i:nth-child(2) { background: var(--yellow); }
.terminal-lights i:nth-child(3) { background: var(--sage); }
.session-terminal { height: 480px; padding: 14px 12px 10px; overflow: hidden; }
.session-terminal .xterm { height: 100%; }
.session-terminal .xterm-viewport { scrollbar-width: thin; scrollbar-color: #766a88 #25222d; }
.terminal-blank { min-height: 480px; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 17px; text-align: center; padding: 24px; }
.terminal-blank > .icon { width: 37px; height: 37px; color: #b3a5ca; }
.terminal-blank h3 { font-size: 20px; font-weight: 450; }
.terminal-blank p { color: #b2a9bf; font-size: 11px; line-height: 1.8; }
.terminal-footer { min-height: 43px; padding: 11px 16px; color: #b5abc4; font-size: 9px; border-top: 1px solid #ffffff0d; overflow-wrap: anywhere; }
.terminal-support { margin-top: 18px; padding: 15px 19px; font-size: 10px; color: var(--muted); }
.terminal-support summary { cursor: pointer; color: var(--ink-soft); }
.terminal-support p { margin: 12px 0; max-width: 680px; line-height: 1.8; }
.terminal-support .muted { margin-left: 12px; }
@media (max-width: 1100px) {
.session-workspace { grid-template-columns: minmax(0, 1fr); }
.session-list { grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); max-height: 190px; }
.session-history > .panel-footnote { display: none; }
}
@media (max-width: 700px) {
.session-terminal { height: 400px; }
.terminal-blank { min-height: 330px; }
.session-section-heading { flex-wrap: wrap; }
.terminal-support .muted { display: block; margin: 10px 0 0; }
}
nest/dashboard/static/css/tools.css
-60
View File
@@ -1,60 +0,0 @@
.tool-choices { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 9px; }
.tool-choice { display: flex; align-items: center; gap: 11px; padding: 12px; border: 1px solid var(--line); border-radius: 13px; background: #ffffff75; cursor: pointer; }
.tool-choice:hover { background: #fff; }
.tool-choice.is-selected { background: var(--yellow-soft); border-color: #ded09a; }
.tool-choice > span { display: grid; gap: 2px; min-width: 0; }
.tool-choice strong { font-size: 11px; }
.tool-choice small { color: var(--muted); font-size: 9px; }
.tool-choice input, .tool-card input { flex: 0 0 auto; }
.tools-section-heading { margin-block: 7px 15px; }
.tools-section-heading .button-row { justify-content: flex-end; }
.tools-section-heading .button-row > span { font-size: 10px; }
.tools-note { min-height: 22px; margin-bottom: 12px; }
.tools-error { margin-bottom: 12px; }
.tools-error:empty { display: none; }
.tools-grid { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 14px; }
.tool-card { display: flex; flex-direction: column; min-width: 0; padding: 18px; border-radius: 18px; border: 1px solid #ffffffbd; background: #faf9fcbf; box-shadow: var(--shadow-card); }
.tool-card-heading { display: flex; gap: 11px; align-items: center; cursor: pointer; }
.tool-card-heading > span { min-width: 0; display: grid; gap: 4px; }
.tool-card-heading strong { font-size: 14px; letter-spacing: -.02em; }
.tool-card-heading small { font-size: 9px; color: var(--muted); }
.tool-card > p { margin: 12px 0 5px; color: var(--muted); font-size: 10px; }
.tool-source { font-size: 8px; color: var(--faint); margin-bottom: 17px; }
.tool-card-footer { display: flex; align-items: center; justify-content: space-between; flex-wrap: wrap; gap: 8px; margin-top: auto; }
.tool-installed { border-color: #b8c6ae70; }
.tool-installed button:disabled { cursor: default; background: var(--sage-soft); color: #64705c; opacity: 1; }
.tool-state { font-size: 9px; color: var(--muted); }
.tool-state-good { color: #68765c; }
.tool-state-error { color: #a45f57; }
.tool-state-active { color: #827037; }
.tool-state-active::before { content: ''; display: inline-block; width: 5px; height: 5px; margin-right: 5px; border-radius: 50%; background: var(--yellow); animation: tool-pulse 1.5s ease-in-out infinite; }
.tool-usage { margin-top: 13px; padding-top: 10px; border-top: 1px solid var(--line); font-size: 9px; color: var(--muted); }
.tool-usage summary { cursor: pointer; }
.tool-usage code { display: block; margin: 9px 0; font-family: var(--mono); font-size: 9px; overflow-wrap: anywhere; }
.tool-job { margin-top: 22px; padding: 22px; }
.tool-job .section-heading { flex-wrap: wrap; gap: 10px; margin-bottom: 8px; }
.tool-job .select-wrap { max-width: 100%; }
.tool-job select { max-width: 100%; }
.tool-job-message { color: var(--muted); font-size: 11px; overflow-wrap: anywhere; }
.tool-progress { display: block; width: 100%; height: 5px; margin: 15px 0; border: none; border-radius: 10px; background: #e7e2eb; accent-color: #b8ad69; }
.tool-progress::-webkit-progress-bar { background: #e7e2eb; border-radius: 10px; }
.tool-progress::-webkit-progress-value { background: #c3b875; border-radius: 10px; }
.tool-progress::-moz-progress-bar { background: #c3b875; border-radius: 10px; }
.tool-job-items { padding: 0; margin: 0; list-style: none; }
.tool-job-items > li { padding: 10px 0; border-top: 1px solid var(--line); }
.tool-job-items summary { cursor: pointer; font-size: 11px; }
.tool-job-items summary > .tool-state { float: right; margin-left: 12px; }
.tool-job-items .form-error { margin-top: 10px; }
.tool-output { max-height: 240px; overflow: auto; margin: 10px 0 0; padding: 15px; border-radius: 12px; background: #34313c; color: #ede8f1; white-space: pre-wrap; overflow-wrap: anywhere; font: 10px/1.7 var(--mono); }
.tools-recovery { margin-top: 16px; }
@keyframes tool-pulse { 50% { opacity: .35; } }
@media (max-width: 1250px) { .tools-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); } }
@media (max-width: 700px) {
.tools-section-heading { align-items: flex-start; flex-direction: column; }
.tools-section-heading .button-row { justify-content: flex-start; }
.tools-grid { grid-template-columns: 1fr; }
.tool-card { padding: 16px; }
.tool-job { padding: 16px; }
.tool-choice { padding: 10px; gap: 8px; }
}
@media (prefers-reduced-motion: reduce) { .tool-state-active::before { animation: none; } }
nest/dashboard/static/index.html
+2 -62
View File
@@ -9,15 +9,8 @@
<meta name="description" content="Nest — a local workspace for your machines, network, and infrastructure." />
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='10' fill='%23eee7ac'/%3E%3Cpath d='M9 22V10l14 12V10' fill='none' stroke='%234b4854' stroke-width='3' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E" />
<link rel="preload" href="/dashboard/static/fonts/manrope-latin.woff2" as="font" type="font/woff2" crossorigin />
<link rel="stylesheet" href="/dashboard/static/vendor/xterm/css/xterm.css?v=5.5.0" />
<link rel="stylesheet" href="/dashboard/static/css/dashboard.css?v=3" />
<link rel="stylesheet" href="/dashboard/static/css/sessions.css?v=2" />
<link rel="stylesheet" href="/dashboard/static/css/tools.css?v=1" />
<script src="/dashboard/static/vendor/xterm/lib/xterm.js?v=5.5.0" defer></script>
<script src="/dashboard/static/vendor/xterm-fit/lib/addon-fit.js?v=0.10.0" defer></script>
<script src="/dashboard/static/js/sessions.js?v=3" defer></script>
<script src="/dashboard/static/js/tools.js?v=1" defer></script>
<script src="/dashboard/static/js/dashboard.js?v=6" defer></script>
<script src="/dashboard/static/js/dashboard.js?v=3" defer></script>
</head>
<body>
<svg class="icon-sprite" xmlns="http://www.w3.org/2000/svg" aria-hidden="true">
@@ -178,12 +171,6 @@
<div id="detail-content" hidden>
<div class="page-heading detail-heading"><div><span class="subtle-label">A closer look</span><h1 id="detail-heading" tabindex="-1"></h1><span id="detail-id" class="mono muted"></span></div><div id="detail-actions" class="button-row"></div></div>
<div id="detail-error" class="notice notice-error" hidden></div>
<nav class="machine-tabs" aria-label="Selected machine views">
<a id="vm-overview-tab" class="machine-tab" href="#vms">Overview</a>
<a id="vm-tools-tab" class="machine-tab" href="#vms"><svg class="icon"><use href="#i-download"/></svg>Tools</a>
<a id="vm-sessions-tab" class="machine-tab" href="#vms"><svg class="icon"><use href="#i-terminal"/></svg>Sessions <span id="vm-session-count" class="nav-count">—</span></a>
</nav>
<div id="vm-overview-panel">
<div class="detail-grid"><article class="soft-panel detail-card"><header class="section-heading"><h2>The essentials</h2><span id="detail-status"></span></header><dl id="detail-config" class="detail-list"></dl></article><article class="soft-panel detail-card"><header class="section-heading"><h2>A place in the network</h2><svg class="icon caption-icon"><use href="#i-network"/></svg></header><dl id="detail-network" class="detail-list"></dl></article></div>
<article class="soft-panel resources-panel">
<header class="section-heading"><div><h2>Room to grow</h2><span>Adjust CPU, memory, and private storage.</span></div><button id="edit-resources-btn" class="button button-small" type="button" data-action="edit-resources">Resize resources</button></header>
@@ -200,43 +187,6 @@
<article class="soft-panel log-panel"><header class="section-heading"><div><h2>A window into the runtime</h2><span>Recent Firecracker log entries.</span></div><div class="button-row"><label class="select-wrap"><span class="sr-only">Log tail length</span><select id="log-tail"><option value="100">100 lines</option><option value="250">250 lines</option><option value="500">500 lines</option><option value="1000">1,000 lines</option></select></label><button id="log-refresh-btn" class="icon-button" type="button" data-action="refresh-logs" aria-label="Refresh logs"><svg class="icon"><use href="#i-refresh"/></svg></button></div></header><pre id="log-output" class="log-output" tabindex="0"></pre><p id="log-meta" class="panel-footnote"></p></article>
<div class="detail-delete-row"><span>Every machine has its own private disk.</span><button class="quiet-link danger-text" type="button" data-action="delete-current"><svg class="icon"><use href="#i-trash"/></svg>Delete machine</button></div>
</div>
<section id="vm-tools-panel" aria-label="Selected machine tools" hidden>
<header class="section-heading tools-section-heading"><div><h2>Tools for your next idea</h2><span>One click. Ready inside this machine.</span></div><div class="button-row"><span id="tools-selected-count" class="muted">0 selected</span><button id="tools-install-selected" class="button button-primary" type="button" disabled>Install selected</button><button id="tools-refresh" class="icon-button" type="button" aria-label="Check installed tools" title="Check installed tools"><svg class="icon"><use href="#i-refresh"/></svg></button></div></header>
<p id="tools-note" class="panel-footnote tools-note" role="status"></p>
<p id="tools-error" class="form-error tools-error" role="alert"></p>
<div id="tools-list" class="tools-grid"></div>
<article id="tools-job" class="soft-panel tool-job" hidden>
<header class="section-heading"><h3 id="tools-job-title">Installation</h3><label class="select-wrap"><span class="sr-only">Installation history</span><select id="tools-job-select"></select></label></header>
<p id="tools-job-message" class="tool-job-message" role="status"></p>
<progress id="tools-progress" class="tool-progress" max="1" value="0" aria-label="Installation progress"></progress>
<ol id="tools-job-items" class="tool-job-items"></ol>
</article>
<div class="button-row tools-recovery"><button id="tools-retry" class="button button-small" type="button" hidden>Retry unfinished tools</button><button id="tools-restart" class="button button-small" type="button" hidden>Restart &amp; install</button></div>
</section>
<section id="vm-sessions-panel" aria-label="Selected machine sessions" hidden>
<header class="section-heading session-section-heading"><div><h2>Sessions</h2><span>A shell for each task. A history for this machine.</span></div><button class="button button-primary" type="button" data-action="launch-session"><svg class="icon"><use href="#i-plus"/></svg><span>Launch session</span></button></header>
<p id="sessions-error" class="form-error" role="alert"></p>
<div class="session-workspace">
<aside class="session-history soft-panel" aria-label="Session history">
<header><h3>Session history</h3><button id="sessions-refresh" class="icon-button" type="button" aria-label="Refresh sessions"><svg class="icon"><use href="#i-refresh"/></svg></button></header>
<div id="session-list" class="session-list"></div>
<p class="panel-footnote">Saved for this microVM.</p>
</aside>
<article class="terminal-panel">
<header class="terminal-toolbar"><div><span class="terminal-lights" aria-hidden="true"><i></i><i></i><i></i></span><strong id="session-title">Terminal</strong><span id="session-status" class="terminal-status">Select a session</span></div><div class="button-row"><button id="session-reconnect" class="button button-small" type="button" hidden>Reconnect</button><button id="session-rename" class="button button-small" type="button" disabled>Rename</button><button id="session-end" class="button button-small" type="button" disabled>End session</button><button id="session-delete" class="button button-small" type="button" disabled>Delete</button></div></header>
<div id="session-blank" class="terminal-blank"><svg class="icon"><use href="#i-terminal"/></svg><h3>A little room to work.</h3><p>Select a saved session, or launch a fresh terminal<br />inside this microVM.</p><button class="button button-primary" type="button" data-action="launch-session"><svg class="icon"><use href="#i-plus"/></svg><span>Launch session</span></button></div>
<div id="session-terminal" class="session-terminal" aria-label="MicroVM terminal" hidden></div>
<footer id="session-meta" class="terminal-footer">Your sessions and terminal output are saved on the host.</footer>
</article>
</div>
<details id="terminal-support" class="terminal-support soft-panel">
<summary>Guest terminal support</summary>
<p>Vsock and the guest agent are set up automatically when a microVM is created or started. Restart an older VM once to apply the setup. Manual repair is available here while the machine is stopped.</p>
<button id="session-install-agent" class="button button-small" type="button">Repair / update terminal support</button>
<span id="session-install-hint" class="muted"></span>
</details>
</section>
</div>
</section>
<section id="page-create" class="page" aria-labelledby="create-heading" hidden>
@@ -245,8 +195,7 @@
<div class="create-fields"><section class="soft-panel form-section"><header class="section-heading"><div><h2><span class="step-number">01</span>A name and a little space</h2><span>Give your new machine a shape.</span></div></header><label class="field" for="create-name"><span>Machine name <small>Optional</small></span><input id="create-name" type="text" name="name" placeholder="e.g. my-little-server" pattern="[a-z][a-z0-9\-]{0,62}" maxlength="63" autocomplete="off" /></label><fieldset class="preset-fieldset"><legend>Start with a size</legend><div class="size-presets"><button class="size-preset active" type="button" data-preset="small" aria-pressed="true"><svg class="icon"><use href="#i-server"/></svg><strong>Small</strong><span>1 vCPU · 512 MiB</span></button><button class="size-preset" type="button" data-preset="balanced" aria-pressed="false"><svg class="icon"><use href="#i-server"/></svg><strong>Balanced</strong><span>2 vCPU · 1 GiB</span></button><button class="size-preset" type="button" data-preset="roomy" aria-pressed="false"><svg class="icon"><use href="#i-server"/></svg><strong>Roomy</strong><span>4 vCPU · 2 GiB</span></button></div></fieldset><div class="form-grid"><label class="field" for="create-cpu"><span>CPU capacity</span><input id="create-cpu" name="cpu" type="number" value="1" min="0.01" step="any" required /><small>Fractional requests round up to whole vCPUs.</small></label><label class="field" for="create-ram"><span>Memory</span><input id="create-ram" name="ram" type="text" value="512M" required /><small>At least 128 MiB. Try 512M, 1G, or 2G.</small></label></div></section>
<section class="soft-panel form-section"><label class="field" for="create-storage"><span>Private storage</span><input id="create-storage" name="storage" type="text" value="30G" required /><small>Minimum 30 GiB. Try 30G, 50G, or 1T. You can resize this disk later.</small></label></section>
<section class="soft-panel form-section"><header class="section-heading"><div><h2><span class="step-number">02</span>A way to connect</h2><span>Private networking and guest access.</span></div></header><label class="field" for="create-guest-ip"><span>Guest IPv4 address <small>Optional</small></span><input id="create-guest-ip" name="guest_ip" type="text" placeholder="Automatically assigned" autocomplete="off" /></label><div class="form-grid"><label class="field" for="create-username"><span>Guest user</span><input id="create-username" name="username" type="text" value="root" autocomplete="username" required /></label><label class="field" for="create-password"><span>Guest password</span><span class="password-field"><input id="create-password" name="password" type="password" value="root" autocomplete="new-password" maxlength="128" required /><button class="icon-button" type="button" data-action="toggle-password" aria-label="Show password" aria-pressed="false"><svg class="icon"><use href="#i-eye"/></svg></button></span></label></div><p class="panel-footnote">Missing users are created with a home directory and sudo access using their guest password. Credentials are applied to the machine’s private disk.</p></section>
<section class="soft-panel form-section"><header class="section-heading"><div><h2><span class="step-number">03</span>Open the right doors</h2><span>Choose which private TCP services are allowed.</span></div></header><label class="toggle-field"><span><strong>Include the default SSH rule</strong><small>Allow TCP port 22 for terminal access.</small></span><input id="create-default-ssh" name="default_ssh" type="checkbox" checked /><span class="toggle-track" aria-hidden="true"></span></label><label class="field" for="create-ports"><span>Additional port rules <small>Optional</small></span><input id="create-ports" name="ports" type="text" placeholder="80:HTTP, 443:HTTPS" spellcheck="false" /><small>Separate rules with commas. HTTP and HTTPS both use TCP.</small></label></section>
<section class="soft-panel form-section create-tools-section"><header class="section-heading"><div><h2><span class="step-number">04</span>Pick your tools</h2><span>Optional. Install automatically after your machine boots.</span></div><button id="create-tools-refresh" class="icon-button" type="button" aria-label="Reload tool catalog"><svg class="icon"><use href="#i-refresh"/></svg></button></header><div id="create-tools-list" class="tool-choices"><span class="muted">Loading available tools…</span></div><p id="create-tools-error" class="form-error" role="alert"></p><p class="panel-footnote">Python versions can coexist. You can add more tools later from the machine’s Tools tab.</p></section></div>
<section class="soft-panel form-section"><header class="section-heading"><div><h2><span class="step-number">03</span>Open the right doors</h2><span>Choose which private TCP services are allowed.</span></div></header><label class="toggle-field"><span><strong>Include the default SSH rule</strong><small>Allow TCP port 22 for terminal access.</small></span><input id="create-default-ssh" name="default_ssh" type="checkbox" checked /><span class="toggle-track" aria-hidden="true"></span></label><label class="field" for="create-ports"><span>Additional port rules <small>Optional</small></span><input id="create-ports" name="ports" type="text" placeholder="80:HTTP, 443:HTTPS" spellcheck="false" /><small>Separate rules with commas. HTTP and HTTPS both use TCP.</small></label></section></div>
<aside class="create-summary soft-panel"><div class="summary-art" aria-hidden="true"><div></div><div></div><span><svg class="icon"><use href="#i-server"/></svg></span></div><span class="subtle-label">Your next little machine</span><h2 id="create-summary-name">Room for an idea.</h2><p>A private disk, a network identity,<br />and a fresh start.</p><dl id="create-summary-details" class="detail-list"></dl><p id="create-error" class="form-error" role="alert"></p><button id="create-submit" class="button button-primary button-wide" type="submit"><span>Create machine</span><svg class="icon"><use href="#i-arrow"/></svg></button><a href="#vms" class="quiet-link summary-cancel">Maybe later</a><span class="summary-note"><svg class="icon"><use href="#i-spark"/></svg>Built on Firecracker</span></aside>
</form>
</section>
@@ -263,15 +212,6 @@
<div id="action-menu" class="action-menu" role="menu" aria-label="Machine actions" hidden></div>
<div id="toast-container" class="toast-container" aria-live="polite" aria-atomic="false"></div>
<dialog id="confirm-dialog" class="confirm-dialog" aria-labelledby="confirm-title" aria-describedby="confirm-body"><form method="dialog"><span class="round-icon" id="confirm-icon"><svg class="icon"><use href="#i-info"/></svg></span><span class="subtle-label">A moment before we continue</span><h2 id="confirm-title"></h2><p id="confirm-body"></p><div class="dialog-actions"><button class="button button-quiet" value="cancel" autofocus>Cancel</button><button id="confirm-ok" class="button button-primary" value="confirm">Confirm</button></div></form></dialog>
<dialog id="session-rename-dialog" class="confirm-dialog session-rename-dialog" aria-labelledby="session-rename-title">
<form id="session-rename-form">
<span class="subtle-label">A name for this workspace</span>
<h2 id="session-rename-title">Rename session</h2>
<label class="field" for="session-name-input"><span>Session name</span><input id="session-name-input" type="text" maxlength="80" autocomplete="off" spellcheck="false" required aria-describedby="session-rename-error" /></label>
<p id="session-rename-error" class="form-error" role="alert"></p>
<div class="dialog-actions"><button id="session-rename-cancel" class="button button-quiet" type="button">Cancel</button><button id="session-rename-save" class="button button-primary" type="submit">Save name</button></div>
</form>
</dialog>
<noscript><p class="noscript-message">Enable JavaScript to connect to your Nest workspace.</p></noscript>
</body>
</html>
nest/dashboard/static/js/dashboard.js
+10 -67
View File
@@ -26,8 +26,6 @@
page: 'overview',
detailId: null,
detail: null,
detailTab: 'overview',
terminalId: null,
portEditing: false,
resizeEditing: false,
selected: new Set(),
@@ -47,18 +45,6 @@
menuTrigger: null,
};
const sessions = new window.NestSessions({
api, getToken: () => state.token,
isSelectedVm: (id) => state.page === 'vm-detail' && state.detailId === id,
onExpired: () => logout('Your token was not accepted. Connect again to continue.'),
esc, formatDate, toast, confirmAction,
});
const tools = new window.NestTools({
api, esc, toast, formatDate,
onSelectionChange: updateCreateSummary,
onMutation: refreshAfterMutation,
});
function esc(value) {
return String(value ?? '').replace(/[&<>"']/g, (character) => ({
'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;',
@@ -175,7 +161,7 @@
return `The server returned HTTP ${status}.`;
}
async function api(path, { method = 'GET', body, timeout = 20000 } = {}) {
async function api(path, { method = 'GET', body } = {}) {
const session = state.session;
const controller = new AbortController();
state.controllers.add(controller);
@@ -184,7 +170,7 @@
const timer = method === 'GET' ? window.setTimeout(() => {
timedOut = true;
controller.abort();
}, timeout) : null;
}, 20000) : null;
try {
const response = await fetch(path, {
method,
@@ -612,26 +598,9 @@
const busy = state.busy.has(vm.id);
const running = runtimeStatus(vm) === 'running';
const actionButton = (action, caption, symbol, primary = false) => `<button type="button" class="button${primary ? ' button-primary' : ''}" data-action="vm-action" data-operation="${action}" data-id="${esc(vm.id)}" ${busy ? 'disabled' : ''}>${icon(symbol)}${caption}</button>`;
html('#detail-actions', (running
html('#detail-actions', running
? actionButton('restart', 'Restart', 'refresh') + actionButton('stop', 'Stop machine', 'stop')
: actionButton('start', 'Start machine', 'play', true))
+ `<button class="button button-primary" type="button" data-action="launch-session">${icon('terminal')}<span>Launch session</span></button>`);
const showingSessions = state.detailTab === 'sessions';
const showingTools = state.detailTab === 'tools';
$('#vm-overview-panel').hidden = showingSessions || showingTools;
$('#vm-sessions-panel').hidden = !showingSessions;
$('#vm-tools-panel').hidden = !showingTools;
$('#vm-overview-tab').href = vmHref(vm.id);
$('#vm-sessions-tab').href = `${vmHref(vm.id)}/sessions`;
$('#vm-tools-tab').href = `${vmHref(vm.id)}/tools`;
[['#vm-overview-tab', !showingSessions && !showingTools], ['#vm-sessions-tab', showingSessions], ['#vm-tools-tab', showingTools]].forEach(([selector, active]) => {
$(selector).classList.toggle('active', active);
if (active) $(selector).setAttribute('aria-current', 'page');
else $(selector).removeAttribute('aria-current');
});
text('#breadcrumb', `Machines / ${machineName(vm)}${showingSessions ? ' / Sessions' : showingTools ? ' / Tools' : ''}`);
sessions.sync(vm, state.terminalId, showingSessions);
tools.sync(vm, showingTools);
: actionButton('start', 'Start machine', 'play', true));
html('#detail-config', detailRows([
['Recorded state', label(vm.status)], ['Observed state', label(runtimeStatus(vm))],
['vCPUs', vcpu(vm)], ['Memory', formatRam(vm.ram_mib)], ['Storage', formatStorage(vm.storage_mib)], ['Guest user', vm.username],
@@ -686,15 +655,13 @@
state.detail = vm;
state.detailId = vm.id;
renderDetail();
if (state.detailTab === 'overview') await loadLogs();
await loadLogs();
} catch (error) {
if (isAborted(error) || request !== state.requests.detail || session !== state.session || state.page !== 'vm-detail') return;
if (state.detail && error.status !== 404) {
text('#detail-error', error.message);
$('#detail-error').hidden = false;
} else {
sessions.leave();
tools.leave();
state.detail = null;
$('#detail-content').hidden = true;
$('#detail-loading').hidden = false;
@@ -937,7 +904,6 @@
['Compute', Number.isFinite(cpu) && cpu > 0 ? `${Math.ceil(cpu)} vCPU` : '—'],
['Memory', memory], ['Storage', storage], ['Guest IP', $('#create-guest-ip').value.trim() || 'Automatic'],
['Guest user', $('#create-username').value.trim() || '—'], ['Allowed ports', ports],
['Tools', tools.createSummary()],
]));
const presets = { small: [1, 512], balanced: [2, 1024], roomy: [4, 2048] };
let mib = null;
@@ -977,7 +943,6 @@
password: $('#create-password').value,
ports: parsePorts($('#create-ports').value),
no_default_ports: !$('#create-default-ssh').checked,
tools: tools.createSelections(),
};
if (!payload.password) throw new Error('Add a password for the guest account.');
} catch (error) {
@@ -989,25 +954,22 @@
const previous = button.innerHTML;
const unlock = lockForm($('#create-form'));
state.busy.add('create');
tools.setCreateBusy(true);
button.textContent = 'Creating your machine…';
try {
const vm = await api('/vms', { method: 'POST', body: payload });
if (session !== state.session) return;
if (!vm?.id) throw new Error('The machine response was not recognized. Refresh the registry to check its state.');
$('#create-form').reset();
tools.resetCreate();
resetPasswordVisibility();
updateCreateSummary();
toast(payload.tools.length ? 'Machine created. Your selected tools are queued for installation.' : 'A new little machine, ready for you.');
location.hash = vmHref(vm.id) + (payload.tools.length ? '/tools' : '');
toast('A new little machine, ready for you.');
location.hash = vmHref(vm.id);
await loadVms();
} catch (error) {
if (!isAborted(error) && session === state.session) text('#create-error', error.message);
} finally {
unlock();
button.innerHTML = previous;
tools.setCreateBusy(false);
if (session === state.session) state.busy.delete('create');
}
}
@@ -1157,19 +1119,8 @@
if (!state.authenticated) return;
let value = location.hash.slice(1) || 'overview';
let identifier = null;
let detailTab = 'overview';
let terminalId = null;
if (value.startsWith('vm/')) {
const parts = value.split('/');
try {
identifier = decodeURIComponent(parts[1]);
if (parts[2] === 'sessions' && parts.length <= 4) {
detailTab = 'sessions';
terminalId = parts[3] ? decodeURIComponent(parts[3]) : null;
} else if (parts[2] === 'tools' && parts.length === 3) {
detailTab = 'tools';
} else if (parts.length !== 2) identifier = null;
} catch { identifier = null; }
try { identifier = decodeURIComponent(value.slice(3)); } catch { identifier = null; }
value = identifier ? 'vm-detail' : 'overview';
}
if (!Object.prototype.hasOwnProperty.call(pages, value)) value = 'overview';
@@ -1178,10 +1129,6 @@
state.requests.detail += 1;
state.requests.logs += 1;
state.detailId = identifier;
state.detailTab = detailTab;
state.terminalId = terminalId;
sessions.leave();
tools.leave();
state.detail = null;
state.portEditing = false;
state.resizeEditing = false;
@@ -1210,7 +1157,7 @@
loadDetail(identifier);
}
if (value === 'system') loadDiagnostics();
if (value === 'create') { updateCreateSummary(); tools.mountCreate(); }
if (value === 'create') updateCreateSummary();
if (!initial && value !== 'vm-detail') $(`#page-${value} h1`)?.focus({ preventScroll: true });
}
@@ -1270,8 +1217,6 @@
}
function logout(message = '') {
sessions.leave();
tools.reset();
state.session += 1;
state.controllers.forEach((controller) => controller.abort());
state.controllers.clear();
@@ -1340,7 +1285,6 @@
if (action === 'clear-filters') { state.query = ''; state.filter = 'all'; $('#vm-search').value = ''; renderMachines(); }
if (action === 'dismiss-results') $('#bulk-results').hidden = true;
if (action === 'install') installAssets();
if (action === 'launch-session' && state.detail) sessions.launch(state.detail);
if (action === 'refresh-logs') loadLogs();
if (action === 'copy-ssh') copyText($('#ssh-command').textContent);
if (action === 'delete-current' && state.detail) machineAction('delete', state.detail.id);
@@ -1444,10 +1388,9 @@
if (target && !target.disabled) handleAction(target);
});
document.addEventListener('keydown', (event) => {
if (event.target.closest('.xterm')) return;
const editing = event.target.closest('input, textarea, select, [contenteditable="true"]');
if (state.authenticated && ((event.key.toLowerCase() === 'k' && (event.metaKey || event.ctrlKey)) || (event.key === '/' && !editing))) {
if (document.querySelector('dialog[open]')) return;
if ($('#confirm-dialog').open) return;
event.preventDefault();
focusSearch();
}
nest/dashboard/static/js/sessions.js
-521
View File
@@ -1,521 +0,0 @@
(function () {
'use strict';
const $ = (selector) => document.querySelector(selector);
const href = (vmId, sessionId = '') => `#vm/${encodeURIComponent(vmId)}/sessions${sessionId ? `/${encodeURIComponent(sessionId)}` : ''}`;
const path = (vmId, suffix = '') => `/vms/${encodeURIComponent(vmId)}/sessions${suffix}`;
const bytes = (encoded) => Uint8Array.from(atob(encoded), (character) => character.charCodeAt(0));
const encoded = (data) => btoa(String.fromCharCode(...data));
window.NestSessions = class {
constructor({ api, getToken, isSelectedVm, onExpired, esc, formatDate, toast, confirmAction }) {
Object.assign(this, { api, getToken, isSelectedVm, onExpired, esc, formatDate, toast, confirmAction });
this.vm = null;
this.records = [];
this.current = null;
this.selectedId = null;
this.visible = false;
this.sequence = 0;
this.generation = 0;
this.launching = new Set();
this.upgrading = new Set();
this.mutating = new Map();
this.offset = 0;
this.renderPending = Promise.resolve();
$('#sessions-refresh').addEventListener('click', () => this.refresh());
$('#session-end').addEventListener('click', () => this.end());
$('#session-reconnect').addEventListener('click', () => this.connectSocket());
$('#session-install-agent').addEventListener('click', () => this.installAgent());
$('#session-rename').addEventListener('click', () => this.rename(this.current));
$('#session-delete').addEventListener('click', () => this.deleteSession(this.current));
$('#session-list').addEventListener('click', (event) => {
const button = event.target.closest('[data-session-action]');
if (!button || button.disabled) return;
const record = this.records.find((item) => item.id === button.dataset.sessionId);
if (button.dataset.sessionAction === 'rename') this.rename(record);
if (button.dataset.sessionAction === 'delete') this.deleteSession(record);
});
$('#session-rename-form').addEventListener('submit', (event) => { event.preventDefault(); this.saveName(); });
$('#session-rename-cancel').addEventListener('click', () => $('#session-rename-dialog').close());
$('#session-rename-dialog').addEventListener('close', () => {
if (!$('#session-rename-dialog').open) this.renameTarget = null;
});
$('#session-rename-dialog').addEventListener('cancel', (event) => {
if (this.isBusy(this.renameTarget?.id)) event.preventDefault();
});
$('#session-name-input').addEventListener('input', () => { $('#session-rename-error').textContent = ''; });
}
sync(vm, selectedId, visible) {
if (this.vm?.id !== vm.id) {
this.leave();
this.vm = vm;
$('#vm-session-count').textContent = '—';
}
this.vm = vm;
if (this.selectedId !== selectedId || this.visible !== visible) {
this.detachTerminal();
this.showError('');
}
this.selectedId = selectedId;
this.visible = visible;
this.updateControls();
if (this.pendingError?.vmId === vm.id && visible) {
this.showError(this.pendingError.message);
$('#terminal-support').open = this.pendingError.setup;
this.pendingError = null;
}
this.renderList();
this.openSelected();
if (!this.loading && (!this.lastRefresh || Date.now() - this.lastRefresh > 3000)) this.refresh();
}
showError(message) {
$('#sessions-error').textContent = message;
}
updateControls() {
const running = this.vm && (this.vm.observed_status || this.vm.status) === 'running'
&& !this.vm.operation && !this.vm.pending_resize;
document.querySelectorAll('[data-action="launch-session"]').forEach((button) => {
button.disabled = !running || this.launching.has(this.vm?.id);
button.querySelector('span').textContent = this.launching.has(this.vm?.id) ? 'Launching…' : 'Launch session';
button.title = running ? 'Open an interactive shell inside this microVM' : 'Start this machine before launching a session';
});
const stopped = this.vm?.status === 'stopped' && (this.vm.observed_status || this.vm.status) === 'stopped'
&& !this.vm.operation && !this.vm.pending_resize;
$('#session-install-agent').disabled = !stopped || this.upgrading.has(this.vm?.id);
$('#session-install-agent').textContent = this.upgrading.has(this.vm?.id) ? 'Updating terminal support…' : 'Repair / update terminal support';
$('#session-install-hint').textContent = stopped ? 'Setup runs automatically at startup. Manual repair is available now.' : 'Vsock is set up automatically on boot. Restart to apply updates.';
this.renderStatus();
}
renderList() {
const markup = this.records.length ? this.records.map((record) => `
<div class="session-item${record.id === this.selectedId ? ' active' : ''}" data-session-id="${this.esc(record.id)}">
<a class="session-link" href="${href(this.vm.id, record.id)}" ${record.id === this.selectedId ? 'aria-current="page"' : ''}>
<strong>${this.esc(record.name)}</strong><i class="session-state ${this.esc(record.status)}" aria-hidden="true"></i>
<small>${this.esc(this.formatDate(record.created_at))}<br />${this.esc(record.status)}${record.exit_code != null ? ` · exit ${record.exit_code}` : ''}</small>
</a>
<div class="session-item-actions">
<button class="quiet-link" type="button" data-session-action="rename" data-session-id="${this.esc(record.id)}" aria-label="Rename ${this.esc(record.name)}" ${this.isBusy(record.id) ? 'disabled' : ''}>Rename</button>
<button class="quiet-link danger-text" type="button" data-session-action="delete" data-session-id="${this.esc(record.id)}" aria-label="Delete ${this.esc(record.name)}" ${this.isBusy(record.id) ? 'disabled' : ''}>${this.mutating.get(record.id) === 'delete' ? 'Deleting…' : 'Delete'}</button>
</div>
</div>`).join('') : `<p class="session-list-empty">${this.loading ? 'Loading sessions…' : 'No sessions yet.<br />Launch a session to get started.'}</p>`;
if ($('#session-list').innerHTML !== markup) $('#session-list').innerHTML = markup;
}
async refresh() {
if (!this.vm) return;
const vmId = this.vm.id;
const sequence = ++this.sequence;
this.loading = true;
$('#sessions-refresh').disabled = true;
try {
const records = await this.api(path(vmId));
if (sequence !== this.sequence || this.vm?.id !== vmId) return;
if (!Array.isArray(records)) throw new Error('The session history response was not recognized.');
this.records = records;
if (this.current) {
const latest = records.find((record) => record.id === this.current.id);
if (!latest) {
const localDelete = this.mutating.get(this.current.id) === 'delete';
this.removeRecord(vmId, this.current.id);
if (!localDelete) this.toast('This session was deleted.');
return;
}
if (latest && latest.updated_at >= this.current.updated_at) this.current = latest;
else if (latest) this.records[this.records.indexOf(latest)] = this.current;
}
this.lastRefresh = Date.now();
$('#vm-session-count').textContent = records.length;
this.openSelected();
this.renderStatus();
} catch (error) {
if (sequence === this.sequence && error.name !== 'AbortError') this.showError(error.message);
} finally {
if (sequence === this.sequence) {
this.loading = false;
$('#sessions-refresh').disabled = false;
this.renderList();
}
}
}
async launch(vm) {
if (!vm || this.launching.has(vm.id)) return;
const token = this.getToken();
this.launching.add(vm.id);
this.showError('');
this.updateControls();
try {
const record = await this.api(path(vm.id), { method: 'POST', body: { rows: 24, cols: 80 } });
if (token === this.getToken() && this.isSelectedVm(vm.id)) location.hash = href(vm.id, record.id);
} catch (error) {
if (token === this.getToken() && error.name !== 'AbortError') {
this.toast(error.message, true);
if (this.isSelectedVm(vm.id)) {
this.pendingError = { vmId: vm.id, message: error.message, setup: error.status === 503 };
location.hash = href(vm.id);
if (this.visible) this.sync(vm, null, true);
}
}
} finally {
this.launching.delete(vm.id);
this.lastRefresh = 0;
this.updateControls();
}
}
async installAgent() {
const vm = this.vm;
if (!vm || this.upgrading.has(vm.id)) return;
this.upgrading.add(vm.id);
this.showError('');
this.updateControls();
try {
await this.api(`/vms/${encodeURIComponent(vm.id)}/agent/install`, { method: 'POST' });
if (this.vm?.id === vm.id) {
this.toast('Terminal support updated. Start this machine, then launch a session.');
$('#session-install-hint').textContent = 'Updated. Start the machine to use the new agent.';
}
} catch (error) {
if (this.vm?.id === vm.id && error.name !== 'AbortError') this.showError(error.message);
} finally {
this.upgrading.delete(vm.id);
this.updateControls();
}
}
openSelected() {
if (!this.visible || !this.selectedId || this.current?.id === this.selectedId) return;
const record = this.records.find((item) => item.id === this.selectedId);
if (!record) {
if (this.lastRefresh) this.showError('This session was not found for the selected machine.');
return;
}
this.showError('');
this.detachTerminal();
this.current = record;
$('#session-blank').hidden = true;
$('#session-terminal').hidden = false;
this.term = new window.Terminal({
cols: record.cols, rows: record.rows, cursorBlink: true, fontSize: 13,
fontFamily: '"SFMono-Regular", Consolas, "Liberation Mono", monospace',
scrollback: 10000, disableStdin: true,
theme: { background: '#25222d', foreground: '#e8e3ef', cursor: '#ede39f', selectionBackground: '#74628780' },
});
this.fit = new window.FitAddon.FitAddon();
this.term.loadAddon(this.fit);
this.term.open($('#session-terminal'));
this.term.onData((text) => this.input(new TextEncoder().encode(text)));
this.term.onBinary((text) => this.input(Uint8Array.from(text, (character) => character.charCodeAt(0))));
this.term.onResize(({ rows, cols }) => this.sendSize(rows, cols));
this.observer = new ResizeObserver(() => this.fitTerminal());
this.observer.observe($('#session-terminal'));
this.fitTerminal();
this.connectSocket();
}
fitTerminal() {
if (this.term && this.visible && $('#session-terminal').clientWidth > 0) {
this.fit.fit();
}
}
input(data) {
if (!this.canType()) return;
if (data.length > 65536 || this.socket.bufferedAmount > 65536) {
this.showError('Terminal input is busy or the paste exceeds 64 KiB. Please send a smaller chunk.');
return;
}
for (let offset = 0; offset < data.length; offset += 8192) {
this.socket.send(JSON.stringify({ type: 'input', data: encoded(data.subarray(offset, offset + 8192)) }));
}
}
canType() {
return this.socket?.readyState === WebSocket.OPEN && this.current?.status === 'active' && this.caughtUp
&& this.ending !== this.current.id && this.mutating.get(this.current.id) !== 'delete';
}
sendSize(rows, cols) {
if (this.canType()) this.socket.send(JSON.stringify({ type: 'resize', rows: Math.max(2, Math.min(200, rows)), cols: Math.max(2, Math.min(400, cols)) }));
}
async connectSocket() {
if (!this.current || !this.visible || !this.getToken()) return;
const generation = this.generation;
await this.renderPending;
if (generation !== this.generation || !this.current || !this.visible) return;
window.clearTimeout(this.reconnectTimer);
const previous = this.socket;
this.socket = null;
previous?.close();
this.caughtUp = false;
this.replayEnd = null;
this.connection = 'Connecting…';
const url = new URL(path(this.vm.id, `/${encodeURIComponent(this.current.id)}/terminal`), location.origin);
url.protocol = location.protocol === 'https:' ? 'wss:' : 'ws:';
const socket = new WebSocket(url);
this.socket = socket;
let rejected = false;
const current = () => generation === this.generation && this.socket === socket;
this.renderStatus();
socket.onopen = () => {
if (current()) socket.send(JSON.stringify({ type: 'auth', token: this.getToken(), offset: this.offset }));
};
socket.onmessage = (event) => {
if (!current()) return;
try {
const message = JSON.parse(event.data);
if (message.type === 'session') {
const wasReady = this.caughtUp;
if (message.session.updated_at >= this.current.updated_at) this.current = message.session;
if (message.offset > this.offset) {
this.term.reset();
this.offset = message.offset;
}
if (this.replayEnd == null) this.replayEnd = message.session.output_end;
this.caughtUp = this.offset >= this.replayEnd;
this.connection = this.caughtUp ? 'Connected' : 'Loading history…';
const index = this.records.findIndex((record) => record.id === message.session.id);
if (index >= 0) this.records[index] = this.current;
this.renderList();
this.renderStatus();
if (!wasReady && this.canType()) { this.sendSize(this.term.rows, this.term.cols); this.term.focus(); }
} else if (message.type === 'output') {
const data = bytes(message.data);
if (message.offset !== this.offset) throw new Error('Terminal output is out of sequence. Reconnect to load history.');
this.renderPending = new Promise((resolve) => {
this.finishRender = resolve;
this.term.write(data, () => {
if (generation === this.generation) {
this.offset += data.length;
const wasReady = this.caughtUp;
this.caughtUp = this.offset >= this.replayEnd;
this.connection = this.caughtUp ? 'Connected' : 'Loading history…';
if (current()) socket.send(JSON.stringify({ type: 'ack', offset: this.offset }));
this.renderStatus();
if (!wasReady && this.canType()) { this.sendSize(this.term.rows, this.term.cols); this.term.focus(); }
this.finishRender = null;
}
resolve();
});
});
} else if (message.type === 'deleted') {
const vmId = this.vm.id;
const sessionId = this.current.id;
const localDelete = this.mutating.get(sessionId) === 'delete';
this.removeRecord(vmId, sessionId);
if (!localDelete) this.toast('This session was deleted.');
} else if (message.type === 'error') {
rejected = true;
this.showError(message.message);
}
} catch (error) {
rejected = true;
this.showError(error.message);
socket.close();
}
};
socket.onclose = (event) => {
if (!current()) return;
this.connection = 'Disconnected';
this.caughtUp = false;
this.renderStatus();
if (event.code === 4401) { this.onExpired(); return; }
if (!rejected && ['active', 'starting'].includes(this.current?.status)) {
this.reconnectTimer = window.setTimeout(() => this.connectSocket(), 2000);
}
};
}
renderStatus() {
const record = this.current;
const active = record?.status === 'active';
$('#session-title').textContent = record ? `${record.name} · ${this.vm?.name || this.vm?.id || ''}` : 'Terminal';
$('#session-status').textContent = record ? active ? this.connection || 'Connecting…' : `${record.status}${record.exit_code != null ? ` · exit ${record.exit_code}` : ''}` : 'Select a session';
$('#session-end').disabled = !active || this.isBusy(record?.id);
$('#session-rename').disabled = !record || this.isBusy(record.id);
$('#session-delete').disabled = !record || this.isBusy(record.id);
$('#session-delete').textContent = record && this.mutating.get(record.id) === 'delete' ? 'Deleting…' : 'Delete';
$('#session-reconnect').hidden = !active || this.socket?.readyState === WebSocket.OPEN;
if (this.term) this.term.options.disableStdin = !this.canType();
$('#session-meta').textContent = record
? `${record.username || 'Guest'} · Started ${this.formatDate(record.created_at)}${record.ended_at ? ` · Ended ${this.formatDate(record.ended_at)}` : ' · Reopen this session to continue'}${record.truncated ? ' · Showing the latest 8 MiB of output' : ''}${record.error ? ` · ${record.error}` : ''}`
: 'Your sessions and terminal output are saved on the host.';
}
async end() {
const record = this.current;
if (!record || this.isBusy(record.id)) return;
const confirmed = await this.confirmAction(`End ${record.name}?`, 'Close this shell and its running processes. Its saved terminal output will be available in session history.', 'End session');
if (!confirmed) return;
this.ending = record.id;
this.renderList();
this.renderStatus();
try {
const ended = await this.api(path(record.vm_id, `/${encodeURIComponent(record.id)}/close`), { method: 'POST' });
if (this.current?.id === record.id) this.current = ended;
if (this.vm?.id === record.vm_id) await this.refresh();
} catch (error) {
if (error.name !== 'AbortError' && this.vm?.id === record.vm_id) this.showError(error.message);
} finally {
this.ending = null;
this.renderList();
this.renderStatus();
}
}
isBusy(id) {
return Boolean(id && (this.mutating.has(id) || this.ending === id));
}
invalidateRefresh() {
this.sequence += 1;
this.loading = false;
this.lastRefresh = 0;
$('#sessions-refresh').disabled = false;
}
rename(record) {
if (!record || this.isBusy(record.id) || document.querySelector('dialog[open]')) return;
this.renameTarget = record;
$('#session-name-input').value = record.name;
$('#session-name-input').disabled = false;
$('#session-rename-save').disabled = false;
$('#session-rename-save').textContent = 'Save name';
$('#session-rename-cancel').disabled = false;
$('#session-rename-error').textContent = '';
$('#session-rename-dialog').showModal();
$('#session-name-input').focus();
$('#session-name-input').select();
}
async saveName() {
const target = this.renameTarget;
if (!target || this.isBusy(target.id)) return;
const name = $('#session-name-input').value.trim();
if (!name) {
$('#session-rename-error').textContent = 'Enter a session name.';
$('#session-name-input').focus();
return;
}
const token = this.getToken();
this.mutating.set(target.id, 'rename');
$('#session-name-input').disabled = true;
$('#session-rename-save').disabled = true;
$('#session-rename-save').textContent = 'Saving…';
$('#session-rename-cancel').disabled = true;
this.renderList();
this.renderStatus();
try {
const renamed = await this.api(path(target.vm_id, `/${encodeURIComponent(target.id)}`), { method: 'PATCH', body: { name } });
if (this.renameTarget === target) $('#session-rename-dialog').close();
if (token === this.getToken() && this.vm?.id === target.vm_id) {
this.invalidateRefresh();
const index = this.records.findIndex((item) => item.id === target.id);
if (index >= 0 && renamed.updated_at >= this.records[index].updated_at) this.records[index] = renamed;
if (this.current?.id === target.id && renamed.updated_at >= this.current.updated_at) this.current = renamed;
this.toast('Session renamed.');
}
} catch (error) {
if (error.name !== 'AbortError' && this.renameTarget === target) $('#session-rename-error').textContent = error.message;
} finally {
this.mutating.delete(target.id);
if (this.renameTarget === target) {
$('#session-name-input').disabled = false;
$('#session-rename-save').disabled = false;
$('#session-rename-save').textContent = 'Save name';
$('#session-rename-cancel').disabled = false;
}
this.renderList();
this.renderStatus();
}
}
async deleteSession(record) {
if (!record || this.isBusy(record.id) || document.querySelector('dialog[open]')) return;
const live = ['active', 'starting'].includes(record.status);
const confirmed = await this.confirmAction(`Delete ${record.name}?`,
live ? 'End this shell and permanently delete the session and its saved terminal output.'
: 'Permanently delete this session and its saved terminal output.', 'Delete session', true);
if (!confirmed) return;
const token = this.getToken();
this.mutating.set(record.id, 'delete');
this.showError('');
this.renderList();
this.renderStatus();
try {
await this.api(path(record.vm_id, `/${encodeURIComponent(record.id)}`), { method: 'DELETE' });
if (token === this.getToken() && this.vm?.id === record.vm_id) {
this.removeRecord(record.vm_id, record.id);
this.toast('Session deleted.');
}
} catch (error) {
if (error.name !== 'AbortError' && token === this.getToken() && this.vm?.id === record.vm_id) this.showError(error.message);
} finally {
this.mutating.delete(record.id);
this.renderList();
this.renderStatus();
}
}
removeRecord(vmId, sessionId) {
if (this.vm?.id !== vmId) return;
this.invalidateRefresh();
this.records = this.records.filter((record) => record.id !== sessionId);
$('#vm-session-count').textContent = this.records.length;
if (this.renameTarget?.id === sessionId) {
$('#session-rename-dialog').close();
this.renameTarget = null;
}
if (this.selectedId === sessionId || this.current?.id === sessionId) {
this.detachTerminal();
this.selectedId = null;
if (this.visible && this.isSelectedVm(vmId)) location.hash = href(vmId);
}
this.showError('');
this.renderList();
this.renderStatus();
}
detachTerminal() {
this.generation += 1;
window.clearTimeout(this.reconnectTimer);
this.observer?.disconnect();
this.observer = null;
const socket = this.socket;
this.socket = null;
socket?.close();
this.finishRender?.();
this.finishRender = null;
this.renderPending = Promise.resolve();
this.term?.dispose();
this.term = null;
this.current = null;
this.offset = 0;
this.caughtUp = false;
$('#session-terminal').replaceChildren();
$('#session-terminal').hidden = true;
$('#session-blank').hidden = false;
this.renderStatus();
}
leave() {
if ($('#session-rename-dialog').open) $('#session-rename-dialog').close();
this.renameTarget = null;
this.sequence += 1;
this.loading = false;
this.lastRefresh = 0;
this.detachTerminal();
this.visible = false;
this.vm = null;
this.records = [];
this.selectedId = null;
this.showError('');
$('#terminal-support').open = false;
}
};
})();
nest/dashboard/static/js/tools.js
-331
View File
@@ -1,331 +0,0 @@
(function () {
'use strict';
const $ = (selector) => document.querySelector(selector);
const active = (job) => ['queued', 'starting', 'waiting', 'installing'].includes(job?.status);
const retryable = (item) => ['failed', 'interrupted', 'skipped'].includes(item.status);
const labels = { queued: 'Queued', starting: 'Starting machine', waiting: 'Waiting for guest', installing: 'Installing',
installed: 'Installed', already_installed: 'Already installed', succeeded: 'Complete', failed: 'Failed',
interrupted: 'Interrupted', skipped: 'Not attempted' };
window.NestTools = class NestTools {
constructor({ api, esc, toast, formatDate, onSelectionChange, onMutation }) {
Object.assign(this, { api, esc, toast, formatDate, onSelectionChange, onMutation });
this.catalog = null;
this.catalogPromise = null;
this.epoch = 0;
this.generation = 0;
this.createSelected = new Set();
this.selected = new Set();
this.cache = new WeakMap();
this.createBusy = false;
this.vm = null;
this.data = null;
this.timer = null;
this.loading = false;
this.submitting = false;
this.revision = 0;
this.probed = false;
this.lastProbe = 0;
this.viewing = null;
this.historyJob = null;
this.pendingRequest = null;
$('#create-tools-list').addEventListener('change', (event) => {
const input = event.target.closest('[data-create-tool]');
if (!input) return;
if (input.checked) this.createSelected.add(input.dataset.createTool);
else this.createSelected.delete(input.dataset.createTool);
input.closest('label').classList.toggle('is-selected', input.checked);
this.onSelectionChange();
});
$('#create-tools-refresh').addEventListener('click', () => this.mountCreate());
$('#tools-refresh').addEventListener('click', () => this.load(true));
$('#tools-install-selected').addEventListener('click', () => this.submit([...this.selected]));
$('#tools-retry').addEventListener('click', () => this.submit(this.retryKeys()));
$('#tools-restart').addEventListener('click', () => this.submit(this.selected.size ? [...this.selected] : this.retryKeys(), true));
$('#tools-job-select').addEventListener('change', (event) => this.viewJob(event.target.value));
$('#tools-list').addEventListener('change', (event) => {
const input = event.target.closest('[data-tool-select]');
if (!input) return;
if (input.checked) this.selected.add(input.dataset.toolSelect);
else this.selected.delete(input.dataset.toolSelect);
this.render();
});
$('#tools-list').addEventListener('click', (event) => {
const button = event.target.closest('[data-install-tool]');
if (button && !button.disabled) this.submit([button.dataset.installTool]);
});
}
html(selector, value) {
const element = $(selector);
if (this.cache.get(element) !== value) {
element.innerHTML = value;
this.cache.set(element, value);
}
}
variants() {
return (this.catalog || []).flatMap((tool) => (tool.versions.length ? tool.versions : [null]).map((version) => ({
...tool, version, key: version ? `${tool.id}-${version}` : tool.id,
title: version ? `${tool.name} ${version}` : tool.name,
})));
}
async loadCatalog() {
if (this.catalog) return this.catalog;
if (this.catalogPromise) return this.catalogPromise;
const epoch = this.epoch;
const request = this.api('/tools').then((catalog) => {
if (!Array.isArray(catalog) || catalog.some((tool) => !tool.id || !Array.isArray(tool.versions))) {
throw new Error('The tool catalog response was not recognized.');
}
if (epoch === this.epoch) this.catalog = catalog;
return catalog;
}).finally(() => { if (this.catalogPromise === request) this.catalogPromise = null; });
this.catalogPromise = request;
return request;
}
async mountCreate() {
const epoch = this.epoch;
$('#create-tools-error').textContent = '';
try {
await this.loadCatalog();
if (epoch !== this.epoch) return;
this.html('#create-tools-list', this.variants().map((tool) => `<label class="tool-choice${this.createSelected.has(tool.key) ? ' is-selected' : ''}">
<input type="checkbox" data-create-tool="${this.esc(tool.key)}" ${this.createSelected.has(tool.key) ? 'checked' : ''} ${this.createBusy ? 'disabled' : ''} />
<span><strong>${this.esc(tool.title)}</strong><small>${this.esc(tool.version ? 'pip + venv' : tool.version_label)}</small></span></label>`).join(''));
} catch (error) {
if (epoch === this.epoch && error.name !== 'AbortError') $('#create-tools-error').textContent = error.message;
}
}
createSelections() {
return this.variants().filter((tool) => this.createSelected.has(tool.key)).map(({ id, version }) => ({ id, version }));
}
createSummary() {
return this.variants().filter((tool) => this.createSelected.has(tool.key)).map((tool) => tool.title).join(', ') || 'None selected';
}
resetCreate() {
this.createSelected.clear();
document.querySelectorAll('[data-create-tool]').forEach((input) => {
input.checked = false;
input.closest('label').classList.remove('is-selected');
});
}
setCreateBusy(busy) {
this.createBusy = busy;
document.querySelectorAll('[data-create-tool]').forEach((input) => { input.disabled = busy; });
}
sync(vm, visible) {
if (!visible) return;
if (this.vm?.id !== vm.id) {
this.leave();
this.vm = vm;
this.html('#tools-list', '<div class="loading-state">Bringing your tools into view…</div>');
$('#tools-job').hidden = true;
$('#tools-error').textContent = '';
this.load();
} else {
const started = this.vm.observed_status !== 'running' && vm.observed_status === 'running';
this.vm = vm;
this.render();
if (started && !this.loading) this.load(true);
}
}
leave() {
this.generation += 1;
window.clearTimeout(this.timer);
this.timer = null;
this.vm = null;
this.data = null;
this.loading = false;
this.submitting = false;
this.probed = false;
this.selected.clear();
this.viewing = null;
this.historyJob = null;
this.pendingRequest = null;
}
reset() {
this.leave();
this.epoch += 1;
this.catalog = null;
this.catalogPromise = null;
this.resetCreate();
}
schedule() {
window.clearTimeout(this.timer);
if (!this.vm) return;
this.timer = window.setTimeout(() => {
if (document.hidden) { this.schedule(); return; }
this.load(Boolean(this.data?.guest_busy && Date.now() - this.lastProbe > 20000));
}, active(this.data?.latest_job) ? 1500 : 5000);
}
async load(refresh = false) {
if (!this.vm || this.loading) return;
const id = this.vm.id;
const generation = this.generation;
const revision = this.revision;
const wasActive = active(this.data?.latest_job);
this.loading = true;
if (refresh) { this.probed = true; this.lastProbe = Date.now(); }
$('#tools-refresh').disabled = true;
if (refresh) $('#tools-note').textContent = 'Checking installed tools inside this machine…';
let probeNext = false;
try {
const [, data] = await Promise.all([this.loadCatalog(), this.api(`/vms/${encodeURIComponent(id)}/tools${refresh ? '?refresh=true' : ''}`, { timeout: 100000 })]);
if (generation !== this.generation || revision !== this.revision) return;
this.data = data;
if (!data?.vm_id || data.vm_id !== id) throw new Error('The tool status response was not recognized.');
if (this.viewing && !data.jobs.some((job) => job.id === this.viewing)) { this.viewing = null; this.historyJob = null; }
this.selected = new Set([...this.selected].filter((key) => !data.inventory.some((item) => item.key === key && item.installed)));
$('#tools-error').textContent = data.error || '';
probeNext = !active(data.latest_job) && this.vm.observed_status === 'running'
&& ((wasActive && !refresh) || (!this.probed && data.stale));
this.render();
} catch (error) {
if (generation === this.generation && error.name !== 'AbortError') $('#tools-error').textContent = error.message;
} finally {
if (generation === this.generation) {
this.loading = false;
$('#tools-refresh').disabled = false;
if (probeNext) this.load(true);
else this.schedule();
}
}
}
retryKeys() {
return (this.data?.latest_job?.items || []).filter(retryable).map((item) => item.key);
}
canInstall() {
const status = this.vm?.observed_status || this.vm?.status;
return ['running', 'stopped'].includes(status) && !this.vm.operation && !this.vm.pending_resize
&& (status !== 'stopped' || this.vm.ports !== null) && this.data?.supported !== false;
}
render() {
if (!this.vm || !this.catalog) return;
const job = this.data?.latest_job;
const occupied = this.submitting || active(job) || this.data?.guest_busy;
const disabled = occupied || !this.canInstall();
const stopped = (this.vm.observed_status || this.vm.status) === 'stopped';
const focused = document.activeElement?.dataset.toolSelect;
const inventory = new Map((this.data?.inventory || []).map((item) => [item.key, item]));
const items = new Map((job?.items || []).map((item) => [item.key, item]));
this.html('#tools-list', this.variants().map((tool) => {
const found = inventory.get(tool.key);
const item = items.get(tool.key);
const installing = active(job) && item && ['queued', 'installing'].includes(item.status);
const installed = found?.installed && !installing;
const failed = item && retryable(item) && !installed && !installing;
const state = installing ? labels[item.status] : installed ? `Installed · ${found.detected_version}` : failed ? labels[item.status]
: found?.detected_version ? `Found ${found.detected_version}` : found ? 'Available' : 'Not checked yet';
const caption = stopped ? 'Start & install' : failed ? 'Retry' : 'Install';
return `<article class="tool-card${installed ? ' tool-installed' : ''}">
<label class="tool-card-heading"><input type="checkbox" data-tool-select="${this.esc(tool.key)}" aria-label="Select ${this.esc(tool.title)}" ${this.selected.has(tool.key) ? 'checked' : ''} ${disabled || installed ? 'disabled' : ''} />
<span><strong>${this.esc(tool.title)}</strong><small>${this.esc(tool.version ? `Python ${tool.version} · pip + venv` : tool.version_label)}</small></span></label>
<p>${this.esc(tool.description)}</p><small class="tool-source">${this.esc(tool.source)}</small>
<div class="tool-card-footer"><span class="tool-state ${installed ? 'tool-state-good' : failed ? 'tool-state-error' : installing ? 'tool-state-active' : ''}">${this.esc(state)}</span>
<button class="button button-small" type="button" data-install-tool="${this.esc(tool.key)}" ${disabled || installed ? 'disabled' : ''}>${installed ? 'Installed' : this.esc(caption)}</button></div>
${installed && found.usage ? `<details class="tool-usage"><summary>Get started</summary><code>${this.esc(found.usage)}</code><small>Run these commands in this machine’s terminal.</small></details>` : ''}</article>`;
}).join(''));
if (focused) document.querySelector(`[data-tool-select="${CSS.escape(focused)}"]`)?.focus({ preventScroll: true });
$('#tools-selected-count').textContent = `${this.selected.size} selected`;
$('#tools-install-selected').textContent = this.submitting ? 'Queuing…' : stopped ? 'Start & install selected' : 'Install selected';
$('#tools-install-selected').disabled = disabled || !this.selected.size;
$('#tools-retry').hidden = !this.retryKeys().length || active(job);
$('#tools-retry').disabled = disabled;
$('#tools-restart').hidden = stopped || !this.data?.error || this.data.agent_ready || active(job);
$('#tools-restart').disabled = disabled || (!this.selected.size && !this.retryKeys().length);
$('#tools-note').textContent = this.data?.guest_busy
? 'An installation is still running inside the guest. Its status will be checked again shortly.'
: stopped ? 'Start & install boots this machine and installs your selected tools. Versions shown are last observed.'
: this.data?.checked_at ? `Last checked ${this.formatDate(this.data.checked_at)}${this.data.stale ? ' · cached observations' : ''}. Use Get started for commands and shell initialization.`
: 'Tools are installed directly inside this VM. Guest internet access is required for downloads.';
this.renderJob();
}
renderJob() {
const job = this.historyJob || this.data?.latest_job;
$('#tools-job').hidden = !job;
if (!job) return;
$('#tools-job-title').textContent = labels[job.status] || job.status;
$('#tools-job-message').textContent = job.error || (active(job) ? 'You can leave this page. Installation continues in the background.' : 'Tool installation results are saved with this machine.');
const complete = job.items.filter((item) => !['queued', 'installing'].includes(item.status)).length;
$('#tools-progress').max = job.items.length;
$('#tools-progress').value = complete;
$('#tools-progress').setAttribute('aria-label', `${complete} of ${job.items.length} tools processed`);
const names = new Map(this.variants().map((tool) => [tool.key, tool.title]));
const open = new Set([...document.querySelectorAll('#tools-job-items details[open]')].map((item) => item.dataset.output));
this.html('#tools-job-items', job.items.map((item) => `<li><details data-output="${this.esc(item.key)}" ${open.has(item.key) ? 'open' : ''}>
<summary><strong>${this.esc(names.get(item.key) || item.key)}</strong><span class="tool-state ${retryable(item) ? 'tool-state-error' : ''}">${this.esc(labels[item.status] || item.status)}</span></summary>
${item.error ? `<p class="form-error">${this.esc(item.error)}</p>` : ''}<pre class="tool-output" tabindex="0">${this.esc(item.output || 'No output recorded.')}</pre>
${item.truncated ? '<p class="panel-footnote">Showing the bounded tail of installation output.</p>' : ''}</details></li>`).join(''));
this.html('#tools-job-select', (this.data.jobs || []).map((item) => `<option value="${this.esc(item.id)}">${this.esc(this.formatDate(item.created_at))} · ${this.esc(labels[item.status] || item.status)}</option>`).join(''));
$('#tools-job-select').value = job.id;
}
async viewJob(id) {
if (!this.vm) return;
const generation = this.generation;
this.viewing = id === this.data?.latest_job?.id ? null : id;
this.historyJob = null;
if (!this.viewing) { this.renderJob(); return; }
try {
const job = await this.api(`/vms/${encodeURIComponent(this.vm.id)}/tools/jobs/${encodeURIComponent(id)}`);
if (generation !== this.generation || this.viewing !== id) return;
this.historyJob = job;
this.renderJob();
} catch (error) {
if (generation === this.generation && error.name !== 'AbortError') this.toast(error.message, true);
}
}
async submit(keys, restart = false) {
if (!this.vm || this.submitting || active(this.data?.latest_job) || !this.canInstall()) return;
const tools = this.variants().filter((tool) => keys.includes(tool.key)).map(({ id, version }) => ({ id, version }));
if (!tools.length) return;
const id = this.vm.id;
const generation = this.generation;
const payload = { tools, start_if_stopped: true, restart };
const signature = JSON.stringify(payload);
if (this.pendingRequest?.signature !== signature) this.pendingRequest = { signature, id: crypto.randomUUID() };
payload.request_id = this.pendingRequest.id;
this.submitting = true;
this.revision += 1;
$('#tools-error').textContent = '';
this.render();
try {
const job = await this.api(`/vms/${encodeURIComponent(id)}/tools/install`, { method: 'POST', body: payload });
if (generation !== this.generation) return;
this.pendingRequest = null;
this.selected.clear();
this.viewing = null;
this.historyJob = null;
if (this.data) { this.data.latest_job = job; this.data.jobs = [{ id: job.id, status: job.status, created_at: job.created_at }, ...this.data.jobs.filter((item) => item.id !== job.id)]; }
this.toast('Tool installation queued. You can follow its progress here.');
this.onMutation(id);
} catch (error) {
if (generation === this.generation && error.name !== 'AbortError') $('#tools-error').textContent = error.message;
} finally {
if (generation === this.generation) {
this.submitting = false;
this.render();
this.load();
}
}
}
};
})();
nest/dashboard/static/vendor/README.md
Vendored
-9
View File
@@ -1,9 +0,0 @@
# Local terminal assets
Runtime files from the published npm packages:
- `@xterm/xterm` **5.5.0** — `xterm/lib/xterm.js`, `xterm/css/xterm.css`
- `@xterm/addon-fit` **0.10.0** — `xterm-fit/lib/addon-fit.js`
Both use the MIT license; the original `LICENSE` is included in each directory.
Assets are served locally. Running Nest requires no npm install or frontend build.
nest/dashboard/static/vendor/xterm-fit/LICENSE
Vendored
-19
View File
@@ -1,19 +0,0 @@
Copyright (c) 2019, The xterm.js authors (https://github.com/xtermjs/xterm.js)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
nest/dashboard/static/vendor/xterm-fit/lib/addon-fit.js
Vendored
-2
View File
@@ -1,2 +0,0 @@
!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.FitAddon=t():e.FitAddon=t()}(self,(()=>(()=>{"use strict";var e={};return(()=>{var t=e;Object.defineProperty(t,"__esModule",{value:!0}),t.FitAddon=void 0,t.FitAddon=class{activate(e){this._terminal=e}dispose(){}fit(){const e=this.proposeDimensions();if(!e||!this._terminal||isNaN(e.cols)||isNaN(e.rows))return;const t=this._terminal._core;this._terminal.rows===e.rows&&this._terminal.cols===e.cols||(t._renderService.clear(),this._terminal.resize(e.cols,e.rows))}proposeDimensions(){if(!this._terminal)return;if(!this._terminal.element||!this._terminal.element.parentElement)return;const e=this._terminal._core,t=e._renderService.dimensions;if(0===t.css.cell.width||0===t.css.cell.height)return;const r=0===this._terminal.options.scrollback?0:e.viewport.scrollBarWidth,i=window.getComputedStyle(this._terminal.element.parentElement),o=parseInt(i.getPropertyValue("height")),s=Math.max(0,parseInt(i.getPropertyValue("width"))),n=window.getComputedStyle(this._terminal.element),l=o-(parseInt(n.getPropertyValue("padding-top"))+parseInt(n.getPropertyValue("padding-bottom"))),a=s-(parseInt(n.getPropertyValue("padding-right"))+parseInt(n.getPropertyValue("padding-left")))-r;return{cols:Math.max(2,Math.floor(a/t.css.cell.width)),rows:Math.max(1,Math.floor(l/t.css.cell.height))}}}})(),e})()));
//# sourceMappingURL=addon-fit.js.map
nest/dashboard/static/vendor/xterm/LICENSE
Vendored
-21
View File
@@ -1,21 +0,0 @@
Copyright (c) 2017-2019, The xterm.js authors (https://github.com/xtermjs/xterm.js)
Copyright (c) 2014-2016, SourceLair Private Company (https://www.sourcelair.com)
Copyright (c) 2012-2013, Christopher Jeffrey (https://github.com/chjj/)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
nest/dashboard/static/vendor/xterm/css/xterm.css
Vendored
-218
View File
@@ -1,218 +0,0 @@
/**
* Copyright (c) 2014 The xterm.js authors. All rights reserved.
* Copyright (c) 2012-2013, Christopher Jeffrey (MIT License)
* https://github.com/chjj/term.js
* @license MIT
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
* Originally forked from (with the author's permission):
* Fabrice Bellard's javascript vt100 for jslinux:
* http://bellard.org/jslinux/
* Copyright (c) 2011 Fabrice Bellard
* The original design remains. The terminal itself
* has been extended to include xterm CSI codes, among
* other features.
*/
/**
* Default styles for xterm.js
*/
.xterm {
cursor: text;
position: relative;
user-select: none;
-ms-user-select: none;
-webkit-user-select: none;
}
.xterm.focus,
.xterm:focus {
outline: none;
}
.xterm .xterm-helpers {
position: absolute;
top: 0;
/**
* The z-index of the helpers must be higher than the canvases in order for
* IMEs to appear on top.
*/
z-index: 5;
}
.xterm .xterm-helper-textarea {
padding: 0;
border: 0;
margin: 0;
/* Move textarea out of the screen to the far left, so that the cursor is not visible */
position: absolute;
opacity: 0;
left: -9999em;
top: 0;
width: 0;
height: 0;
z-index: -5;
/** Prevent wrapping so the IME appears against the textarea at the correct position */
white-space: nowrap;
overflow: hidden;
resize: none;
}
.xterm .composition-view {
/* TODO: Composition position got messed up somewhere */
background: #000;
color: #FFF;
display: none;
position: absolute;
white-space: nowrap;
z-index: 1;
}
.xterm .composition-view.active {
display: block;
}
.xterm .xterm-viewport {
/* On OS X this is required in order for the scroll bar to appear fully opaque */
background-color: #000;
overflow-y: scroll;
cursor: default;
position: absolute;
right: 0;
left: 0;
top: 0;
bottom: 0;
}
.xterm .xterm-screen {
position: relative;
}
.xterm .xterm-screen canvas {
position: absolute;
left: 0;
top: 0;
}
.xterm .xterm-scroll-area {
visibility: hidden;
}
.xterm-char-measure-element {
display: inline-block;
visibility: hidden;
position: absolute;
top: 0;
left: -9999em;
line-height: normal;
}
.xterm.enable-mouse-events {
/* When mouse events are enabled (eg. tmux), revert to the standard pointer cursor */
cursor: default;
}
.xterm.xterm-cursor-pointer,
.xterm .xterm-cursor-pointer {
cursor: pointer;
}
.xterm.column-select.focus {
/* Column selection mode */
cursor: crosshair;
}
.xterm .xterm-accessibility:not(.debug),
.xterm .xterm-message {
position: absolute;
left: 0;
top: 0;
bottom: 0;
right: 0;
z-index: 10;
color: transparent;
pointer-events: none;
}
.xterm .xterm-accessibility-tree:not(.debug) *::selection {
color: transparent;
}
.xterm .xterm-accessibility-tree {
user-select: text;
white-space: pre;
}
.xterm .live-region {
position: absolute;
left: -9999px;
width: 1px;
height: 1px;
overflow: hidden;
}
.xterm-dim {
/* Dim should not apply to background, so the opacity of the foreground color is applied
* explicitly in the generated class and reset to 1 here */
opacity: 1 !important;
}
.xterm-underline-1 { text-decoration: underline; }
.xterm-underline-2 { text-decoration: double underline; }
.xterm-underline-3 { text-decoration: wavy underline; }
.xterm-underline-4 { text-decoration: dotted underline; }
.xterm-underline-5 { text-decoration: dashed underline; }
.xterm-overline {
text-decoration: overline;
}
.xterm-overline.xterm-underline-1 { text-decoration: overline underline; }
.xterm-overline.xterm-underline-2 { text-decoration: overline double underline; }
.xterm-overline.xterm-underline-3 { text-decoration: overline wavy underline; }
.xterm-overline.xterm-underline-4 { text-decoration: overline dotted underline; }
.xterm-overline.xterm-underline-5 { text-decoration: overline dashed underline; }
.xterm-strikethrough {
text-decoration: line-through;
}
.xterm-screen .xterm-decoration-container .xterm-decoration {
z-index: 6;
position: absolute;
}
.xterm-screen .xterm-decoration-container .xterm-decoration.xterm-decoration-top-layer {
z-index: 7;
}
.xterm-decoration-overview-ruler {
z-index: 8;
position: absolute;
top: 0;
right: 0;
pointer-events: none;
}
.xterm-decoration-top {
z-index: 2;
position: relative;
}
nest/dashboard/static/vendor/xterm/lib/xterm.js
Vendored
-2
View File
File diff suppressed because one or more lines are too long
nest/server.py
-18
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Any
@@ -22,29 +21,12 @@ def create_app(application: Application | None = None, *, host: str | None = Non
from fastapi.staticfiles import StaticFiles
from .dashboard.router import router as dashboard_router
from .session_routes import router as session_router
from .sessions import SessionManager
sessions = SessionManager(api.state.uvm_application)
api.state.nest_sessions = sessions
previous_lifespan = api.router.lifespan_context
@asynccontextmanager
async def lifespan(app):
async with previous_lifespan(app):
try:
yield
finally:
await sessions.shutdown()
api.router.lifespan_context = lifespan
api.title = "Nest"
api.description = "Nest workspace API, with local Firecracker machine management powered by UVM."
static_dir = Path(__file__).parent / "dashboard" / "static"
api.mount("/dashboard/static", StaticFiles(directory=static_dir), name="dashboard_static")
api.include_router(dashboard_router, prefix="/dashboard")
api.include_router(session_router)
@api.get("/")
async def root() -> Any:
nest/session_routes.py
-196
View File
@@ -1,196 +0,0 @@
"""VM-scoped session history, creation, and authenticated browser terminal streams."""
from __future__ import annotations
import asyncio
import base64
import hmac
import json
from contextlib import suppress
from typing import Literal
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, Request, Response, WebSocket, WebSocketDisconnect
from pydantic import BaseModel, ConfigDict, Field, StrictInt, StrictStr
from uvm.app import Application
from uvm.errors import NotFoundError, UvmError
from uvm.guest_agent import MAX_TERMINAL_FRAME, ProtocolError, terminal_data
from uvm.routers.dependencies import get_authorized_application
from .sessions import SessionManager
from .session_store import MAX_SESSION_NAME_LENGTH
router = APIRouter(prefix="/vms/{vm_id}/sessions", tags=["sessions"])
class SessionCreate(BaseModel):
model_config = ConfigDict(extra="forbid")
rows: StrictInt = Field(default=24, ge=2, le=200)
cols: StrictInt = Field(default=80, ge=2, le=400)
class SessionResponse(BaseModel):
id: str
vm_id: str
name: str
username: str
status: Literal["starting", "active", "closed", "interrupted", "failed"]
created_at: float
updated_at: float
ended_at: float | None
rows: int
cols: int
exit_code: int | None
error: str | None
output_start: int
output_end: int
truncated: bool
class SessionRename(BaseModel):
model_config = ConfigDict(extra="forbid")
name: StrictStr = Field(min_length=1, max_length=MAX_SESSION_NAME_LENGTH)
def manager(request: Request, _application: Application = Depends(get_authorized_application)) -> SessionManager:
return request.app.state.nest_sessions
@router.get("", response_model=list[SessionResponse])
async def list_sessions(vm_id: str, sessions: SessionManager = Depends(manager)):
return await sessions.list(vm_id)
@router.post("", response_model=SessionResponse, status_code=201)
async def create_session(vm_id: str, body: SessionCreate, sessions: SessionManager = Depends(manager)):
return await sessions.create(vm_id, body.rows, body.cols)
@router.get("/{session_id}", response_model=SessionResponse)
async def get_session(vm_id: str, session_id: str, sessions: SessionManager = Depends(manager)):
return await sessions.get(vm_id, session_id)
@router.post("/{session_id}/close", response_model=SessionResponse)
async def close_session(vm_id: str, session_id: str, sessions: SessionManager = Depends(manager)):
return await sessions.end(vm_id, session_id)
@router.patch("/{session_id}", response_model=SessionResponse)
async def rename_session(vm_id: str, session_id: str, body: SessionRename, sessions: SessionManager = Depends(manager)):
return await sessions.rename(vm_id, session_id, body.name)
@router.delete("/{session_id}", status_code=204)
async def delete_session(vm_id: str, session_id: str, sessions: SessionManager = Depends(manager)) -> Response:
await sessions.delete(vm_id, session_id)
return Response(status_code=204)
def _same_origin(websocket: WebSocket) -> bool:
origin = websocket.headers.get("origin")
if origin is None:
return True # Non-browser API clients must still authenticate.
try:
parsed = urlsplit(origin)
except ValueError:
return False
scheme = "https" if websocket.url.scheme == "wss" else "http"
return parsed.scheme == scheme and parsed.netloc.lower() == websocket.headers.get("host", "").lower() and not parsed.path
async def _message(websocket: WebSocket, *, limit: int = MAX_TERMINAL_FRAME) -> dict:
text = await websocket.receive_text()
if len(text) > limit:
raise ProtocolError("terminal message exceeds its size limit")
message = json.loads(text)
if not isinstance(message, dict):
raise ProtocolError("terminal message must be an object")
return message
@router.websocket("/{session_id}/terminal")
async def terminal(websocket: WebSocket, vm_id: str, session_id: str):
if not _same_origin(websocket):
await websocket.close(code=1008)
return
await websocket.accept()
tasks = []
attached = False
try:
# Browser WebSockets cannot set X-UVM-Token. Authenticate in the first
# frame so tokens never appear in URLs, access logs, or session history.
auth = await asyncio.wait_for(_message(websocket, limit=4096), timeout=10)
token = auth.get("token")
expected = websocket.app.state.uvm_application.settings.api_token
if (auth.get("type") != "auth" or not isinstance(token, str) or not token.isascii()
or not expected or not hmac.compare_digest(token, expected)):
await websocket.close(code=4401)
return
sessions = websocket.app.state.nest_sessions
await sessions.get(vm_id, session_id)
attached = True
offset = auth.get("offset", 0)
acknowledgements = asyncio.Queue(maxsize=1)
async def output():
cursor = offset
previous = None
while True:
record, start, data = await sessions.read(vm_id, session_id, cursor)
if previous != record or start != cursor:
await websocket.send_json({"type": "session", "session": SessionResponse(**record).model_dump(), "offset": start})
previous = record
cursor = start
if data:
await websocket.send_json({"type": "output", "offset": cursor, "data": base64.b64encode(data).decode("ascii")})
cursor += len(data)
# Only one output chunk may be waiting for the terminal renderer.
acknowledged = await asyncio.wait_for(acknowledgements.get(), timeout=30)
if type(acknowledged) is not int or acknowledged != cursor:
raise ProtocolError("invalid terminal output acknowledgement")
continue
if record["status"] not in {"active", "starting"} and not sessions.is_deleting(session_id):
return
await sessions.wait(session_id)
async def controls():
while True:
message = await _message(websocket)
if set(message) == {"type", "data"} and message["type"] == "input":
await sessions.write(vm_id, session_id, terminal_data(message["data"]))
elif set(message) == {"type", "rows", "cols"} and message["type"] == "resize":
await sessions.resize(vm_id, session_id, message["rows"], message["cols"])
elif set(message) == {"type", "offset"} and message["type"] == "ack":
if acknowledgements.full():
raise ProtocolError("unexpected terminal output acknowledgement")
acknowledgements.put_nowait(message["offset"])
else:
raise ProtocolError("invalid terminal control message")
tasks = [asyncio.create_task(output()), asyncio.create_task(controls())]
done, _pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
for task in done:
task.result()
except WebSocketDisconnect:
pass
except NotFoundError as error:
with suppress(WebSocketDisconnect, RuntimeError):
await websocket.send_json(
{"type": "deleted", "session_id": session_id} if attached
else {"type": "error", "message": str(error)}
)
except (ValueError, KeyError, TypeError, UnicodeError, RecursionError, TimeoutError):
with suppress(WebSocketDisconnect, RuntimeError):
await websocket.send_json({"type": "error", "message": "Invalid terminal message or authentication timeout."})
except UvmError as error:
with suppress(WebSocketDisconnect, RuntimeError):
await websocket.send_json({"type": "error", "message": str(error)})
finally:
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
with suppress(WebSocketDisconnect, RuntimeError):
await websocket.close()
nest/session_store.py
-203
View File
@@ -1,203 +0,0 @@
"""Private, durable session metadata and bounded PTY transcripts, separate from VM state."""
from __future__ import annotations
import fcntl
import os
import sqlite3
import stat
import threading
import time
import uuid
from contextlib import contextmanager
from pathlib import Path
from uvm.errors import ConflictError, NotFoundError, StateError, ValidationError
MAX_TRANSCRIPT_BYTES = 8 * 1024 * 1024
MAX_SESSION_NAME_LENGTH = 80
def validate_session_name(value: object) -> str:
if not isinstance(value, str):
raise ValidationError("Session name must be text.")
name = value.strip()
if not 1 <= len(name) <= MAX_SESSION_NAME_LENGTH or not name.isprintable():
raise ValidationError(f"Session name must contain 1–{MAX_SESSION_NAME_LENGTH} printable characters.")
return name
class SessionStore:
def __init__(self, path: Path, *, max_output: int = MAX_TRANSCRIPT_BYTES) -> None:
self.path = path
self.max_output = max_output
self._mutex = threading.RLock()
self._owner = None
try:
path.parent.mkdir(parents=True, mode=0o700, exist_ok=True)
if path.parent.is_symlink():
raise StateError("session history directory must not be a symlink")
path.parent.chmod(0o700)
self._owner = self._private_file(path.with_suffix(".lock"))
try:
fcntl.flock(self._owner, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError as error:
raise ConflictError("Another Nest server is already managing sessions for this VM registry.") from error
descriptor = self._private_file(path)
os.close(descriptor)
with self._db() as db:
db.executescript("""
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, vm_id TEXT NOT NULL, name TEXT NOT NULL,
username TEXT NOT NULL, status TEXT NOT NULL,
created_at REAL NOT NULL, updated_at REAL NOT NULL, ended_at REAL,
rows INTEGER NOT NULL, cols INTEGER NOT NULL, exit_code INTEGER, error TEXT,
output_start INTEGER NOT NULL DEFAULT 0, output_end INTEGER NOT NULL DEFAULT 0,
truncated INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX IF NOT EXISTS sessions_vm ON sessions(vm_id, created_at);
CREATE TABLE IF NOT EXISTS output (
session_id TEXT NOT NULL REFERENCES sessions(id), offset INTEGER NOT NULL,
data BLOB NOT NULL, PRIMARY KEY (session_id, offset)
);
""")
now = time.time()
db.execute("""UPDATE sessions SET status='interrupted', ended_at=?, updated_at=?,
error='Nest restarted; this shell is no longer connected.'
WHERE status IN ('starting', 'active')""", (now, now))
except BaseException:
self.close()
raise
@staticmethod
def _private_file(path: Path) -> int:
descriptor = os.open(path, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600)
try:
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode) or metadata.st_nlink != 1:
raise StateError("session history must use private regular files")
os.fchmod(descriptor, 0o600)
return descriptor
except BaseException:
os.close(descriptor)
raise
@contextmanager
def _db(self):
with self._mutex:
db = None
try:
db = sqlite3.connect(self.path, timeout=5)
db.row_factory = sqlite3.Row
with db:
yield db
except sqlite3.Error as error:
raise StateError("Could not read or save terminal session history.") from error
finally:
if db is not None:
db.close()
@staticmethod
def _record(row) -> dict:
if row is None:
raise NotFoundError("Session not found for this VM.")
result = dict(row)
result["truncated"] = bool(result["truncated"])
return result
def create(self, vm_id: str, rows: int, cols: int) -> dict:
session_id = "session-" + uuid.uuid4().hex
now = time.time()
with self._db() as db:
names = [row[0] for row in db.execute("SELECT name FROM sessions WHERE vm_id=?", (vm_id,))]
number = len(names) + 1
while f"Session {number}" in names:
number += 1
db.execute("""INSERT INTO sessions
(id, vm_id, name, username, status, created_at, updated_at, rows, cols)
VALUES (?, ?, ?, '', 'starting', ?, ?, ?, ?)""",
(session_id, vm_id, f"Session {number}", now, now, rows, cols))
return self.get(vm_id, session_id)
def list(self, vm_id: str) -> list[dict]:
with self._db() as db:
return [self._record(row) for row in db.execute(
"SELECT * FROM sessions WHERE vm_id=? ORDER BY created_at DESC, id DESC", (vm_id,),
)]
def get(self, vm_id: str, session_id: str) -> dict:
with self._db() as db:
return self._record(db.execute("SELECT * FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id)).fetchone())
def rename(self, vm_id: str, session_id: str, name: str) -> dict:
name = validate_session_name(name)
with self._db() as db:
self._record(db.execute("SELECT * FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id)).fetchone())
db.execute("UPDATE sessions SET name=?, updated_at=? WHERE vm_id=? AND id=?",
(name, time.time(), vm_id, session_id))
return self._record(db.execute("SELECT * FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id)).fetchone())
def delete(self, vm_id: str, session_id: str) -> None:
with self._db() as db:
record = self._record(db.execute("SELECT * FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id)).fetchone())
if record["status"] in {"starting", "active"}:
raise ConflictError("End the session before deleting its history.")
# Both tables are removed in one transaction, after the output reader
# has finished. Explicit cleanup also supports existing databases.
db.execute("DELETE FROM output WHERE session_id=?", (session_id,))
db.execute("DELETE FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id))
def activate(self, session_id: str, username: str) -> None:
with self._db() as db:
db.execute("UPDATE sessions SET status='active', username=?, updated_at=? WHERE id=?",
(username, time.time(), session_id))
def resize(self, session_id: str, rows: int, cols: int) -> None:
with self._db() as db:
db.execute("UPDATE sessions SET rows=?, cols=? WHERE id=?", (rows, cols, session_id))
def finish(self, session_id: str, status: str, exit_code: int | None = None, error: str | None = None) -> None:
now = time.time()
with self._db() as db:
db.execute("""UPDATE sessions SET status=?, ended_at=?, updated_at=?, exit_code=?, error=?
WHERE id=? AND status IN ('starting', 'active')""", (status, now, now, exit_code, error, session_id))
def append(self, session_id: str, data: bytes) -> None:
if not data:
return
with self._db() as db:
row = db.execute("SELECT output_end FROM sessions WHERE id=?", (session_id,)).fetchone()
if row is None:
raise NotFoundError("Session not found.")
offset = row[0]
end = offset + len(data)
start = max(0, end - self.max_output)
db.execute("INSERT INTO output VALUES (?, ?, ?)", (session_id, offset, data))
db.execute("DELETE FROM output WHERE session_id=? AND offset + length(data)<=?", (session_id, start))
first = db.execute("SELECT offset, data FROM output WHERE session_id=? AND offset<?", (session_id, start)).fetchone()
if first is not None:
db.execute("UPDATE output SET offset=?, data=? WHERE session_id=? AND offset=?",
(start, first["data"][start - first["offset"]:], session_id, first["offset"]))
db.execute("UPDATE sessions SET output_start=?, output_end=?, truncated=?, updated_at=? WHERE id=?",
(start, end, int(start > 0), time.time(), session_id))
def read(self, vm_id: str, session_id: str, offset: int, limit: int = 65536) -> tuple[dict, int, bytes]:
with self._db() as db:
session = self._record(db.execute("SELECT * FROM sessions WHERE vm_id=? AND id=?", (vm_id, session_id)).fetchone())
if type(offset) is not int or not 0 <= offset <= session["output_end"]:
raise ValidationError("invalid terminal output offset")
start = max(offset, session["output_start"])
data = bytearray()
for row in db.execute("""SELECT offset, data FROM output
WHERE session_id=? AND offset + length(data)>? ORDER BY offset""", (session_id, start)):
part = row["data"][max(0, start - row["offset"]):]
data.extend(part[:limit - len(data)])
if len(data) >= limit:
break
return session, start, bytes(data)
def close(self) -> None:
if self._owner is not None:
os.close(self._owner)
self._owner = None
nest/sessions.py
-234
View File
@@ -1,234 +0,0 @@
"""Nest-owned live terminal sessions; browser attachments do not own guest shells."""
from __future__ import annotations
import asyncio
from dataclasses import dataclass, field
from pathlib import Path
from uvm.app import Application
from uvm.errors import ConflictError, GuestExecutionError, NotFoundError, StateError, UvmError
from uvm.guest_terminal import GuestTerminal, validate_size
from .session_store import SessionStore, validate_session_name
async def _persist(function, *arguments):
# Cancellation cannot release the history-owner lock while a worker still
# has an outstanding write to that database.
task = asyncio.create_task(asyncio.to_thread(function, *arguments))
try:
return await asyncio.shield(task)
except asyncio.CancelledError:
await task
raise
@dataclass
class _LiveSession:
vm_id: str
terminal: GuestTerminal
changed: asyncio.Event = field(default_factory=asyncio.Event)
task: asyncio.Task | None = None
class SessionManager:
def __init__(self, application: Application) -> None:
self.application = application
self._store: SessionStore | None = None
self._live: dict[str, _LiveSession] = {}
self._pending: set[asyncio.Task] = set()
self._deleting: set[str] = set()
self._session_lock = asyncio.Lock()
self._closing = False
def _history(self) -> SessionStore:
if self._closing:
raise ConflictError("Nest is shutting down.")
if self._store is None:
try:
self._store = SessionStore(Path(self.application.settings.base) / "nest" / "sessions.sqlite3")
except OSError as error:
raise StateError("Could not open private terminal history; run Nest with access to the VM registry.") from error
return self._store
async def _vm(self, vm_id: str) -> None:
# Exact IDs prevent a session from following a reused VM name or IP.
await asyncio.to_thread(self.application.state_store.get, vm_id)
async def list(self, vm_id: str) -> list[dict]:
await self._vm(vm_id)
return await asyncio.to_thread(self._history().list, vm_id)
async def get(self, vm_id: str, session_id: str) -> dict:
await self._vm(vm_id)
return await asyncio.to_thread(self._history().get, vm_id, session_id)
async def create(self, vm_id: str, rows: int, cols: int) -> dict:
validate_size(rows, cols)
await self._vm(vm_id)
history = self._history()
# A dropped HTTP request must not abandon a just-created guest connection.
return await self._run_pending(self._create(history, vm_id, rows, cols))
async def _run_pending(self, operation):
"""Finish accepted mutations even if their HTTP request disconnects."""
task = asyncio.create_task(operation)
self._pending.add(task)
def completed(future):
self._pending.discard(future)
if not future.cancelled():
future.exception()
task.add_done_callback(completed)
return await asyncio.shield(task)
async def _create(self, history: SessionStore, vm_id: str, rows: int, cols: int) -> dict:
async with self._session_lock:
if self._closing:
raise ConflictError("Nest is shutting down.")
if len(self._live) >= 32:
raise ConflictError("There are already 32 active terminal sessions. End a session before launching another.")
record = await asyncio.to_thread(history.create, vm_id, rows, cols)
terminal = None
try:
terminal, username = await asyncio.to_thread(self.application.lifecycle.open_terminal, vm_id, rows=rows, cols=cols)
await asyncio.to_thread(history.activate, record["id"], username)
live = _LiveSession(vm_id, terminal)
self._live[record["id"]] = live
live.task = asyncio.create_task(self._read(history, record["id"], live))
except BaseException as error:
if terminal is not None:
terminal.close()
message = str(error) if isinstance(error, UvmError) else "Could not launch the guest terminal."
await asyncio.to_thread(history.finish, record["id"], "failed", None, message)
raise
return await asyncio.to_thread(history.get, vm_id, record["id"])
async def _read(self, history: SessionStore, session_id: str, live: _LiveSession) -> None:
status, exit_code, error = "interrupted", None, None
try:
while True:
message = await live.terminal.read()
if message["type"] == "exit":
status, exit_code = "closed", message["exit_code"]
break
await _persist(history.append, session_id, message["data"])
live.changed.set()
except asyncio.CancelledError:
error = "Nest stopped; this shell is no longer connected." if self._closing else "The guest terminal connection was interrupted."
except (UvmError, OSError) as exception:
error = str(exception) if isinstance(exception, UvmError) else "The guest terminal connection was interrupted."
finally:
live.terminal.close()
try:
await _persist(history.finish, session_id, status, exit_code, error)
finally:
live.changed.set()
self._live.pop(session_id, None)
async def read(self, vm_id: str, session_id: str, offset: int) -> tuple[dict, int, bytes]:
return await asyncio.to_thread(self._history().read, vm_id, session_id, offset)
async def wait(self, session_id: str) -> None:
live = self._live.get(session_id)
if live is None:
await asyncio.sleep(0.1)
return
try:
await asyncio.wait_for(live.changed.wait(), timeout=1)
except TimeoutError:
pass
live.changed.clear()
async def _active(self, vm_id: str, session_id: str) -> _LiveSession:
live = self._live.get(session_id)
if live is None:
await self.get(vm_id, session_id)
raise ConflictError("This session has ended. Launch a new session to run commands.")
if live.vm_id != vm_id:
raise NotFoundError("Session not found for this VM.")
return live
async def write(self, vm_id: str, session_id: str, data: bytes) -> None:
live = await self._active(vm_id, session_id)
try:
await live.terminal.write(data)
except OSError as error:
live.task.cancel()
raise GuestExecutionError("The terminal connection was interrupted.", outcome_unknown=True) from error
async def resize(self, vm_id: str, session_id: str, rows: int, cols: int) -> None:
validate_size(rows, cols)
live = await self._active(vm_id, session_id)
try:
await live.terminal.resize(rows, cols)
await asyncio.to_thread(self._history().resize, session_id, rows, cols)
except OSError as error:
live.task.cancel()
raise GuestExecutionError("The terminal connection was interrupted.", outcome_unknown=True) from error
async def end(self, vm_id: str, session_id: str) -> dict:
await self.get(vm_id, session_id)
return await self._run_pending(self._end(self._history(), vm_id, session_id))
async def _end(self, history: SessionStore, vm_id: str, session_id: str) -> dict:
async with self._session_lock:
return await self._end_locked(history, vm_id, session_id)
async def _end_locked(self, history: SessionStore, vm_id: str, session_id: str) -> dict:
record = await asyncio.to_thread(history.get, vm_id, session_id)
live = self._live.get(session_id)
if live is not None:
try:
await live.terminal.end()
await asyncio.wait_for(asyncio.shield(live.task), timeout=8)
except (OSError, UvmError):
live.task.cancel()
await asyncio.gather(live.task, return_exceptions=True)
elif record["status"] in {"starting", "active"}:
# The session lock excludes creation here. A missing channel can
# only be an interrupted session whose final metadata write failed.
await _persist(history.finish, session_id, "interrupted", None, "This shell is no longer connected.")
return await asyncio.to_thread(history.get, vm_id, session_id)
async def rename(self, vm_id: str, session_id: str, name: str) -> dict:
name = validate_session_name(name)
await self.get(vm_id, session_id)
return await self._run_pending(self._rename(self._history(), vm_id, session_id, name))
async def _rename(self, history: SessionStore, vm_id: str, session_id: str, name: str) -> dict:
record = await _persist(history.rename, vm_id, session_id, name)
live = self._live.get(session_id)
if live is not None:
live.changed.set()
return record
async def delete(self, vm_id: str, session_id: str) -> None:
await self.get(vm_id, session_id)
await self._run_pending(self._delete(self._history(), vm_id, session_id))
async def _delete(self, history: SessionStore, vm_id: str, session_id: str) -> None:
# Creation must finish before deletion can close its channel. Join the
# output reader (including pending disk writes) before removing history.
async with self._session_lock:
self._deleting.add(session_id)
try:
await self._end_locked(history, vm_id, session_id)
await _persist(history.delete, vm_id, session_id)
finally:
self._deleting.discard(session_id)
def is_deleting(self, session_id: str) -> bool:
return session_id in self._deleting
async def shutdown(self) -> None:
self._closing = True
await asyncio.gather(*self._pending, return_exceptions=True)
tasks = [live.task for live in self._live.values()]
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
if self._store is not None:
self._store.close()
pyproject.toml
+1 -2
View File
@@ -11,7 +11,6 @@ dependencies = [
"fastapi>=0.115,<1",
"pydantic>=2,<3",
"uvicorn>=0.30,<1",
"websockets>=13,<17",
]
[project.optional-dependencies]
@@ -26,4 +25,4 @@ packages = ["nest", "nest.dashboard", "uvm", "uvm.firecracker", "uvm.routers"]
include-package-data = false
[tool.setuptools.package-data]
"nest.dashboard" = ["static/*.html", "static/css/*.css", "static/js/*.js", "static/fonts/*.woff2", "static/fonts/*.txt", "static/vendor/*.md", "static/vendor/*/LICENSE", "static/vendor/*/lib/*.js", "static/vendor/*/css/*.css"]
"nest.dashboard" = ["static/*.html", "static/css/*.css", "static/js/*.js", "static/fonts/*.woff2", "static/fonts/*.txt"]
tests/browser_session_server.py
-82
View File
@@ -1,82 +0,0 @@
"""Isolated browser-test server: all VM and terminal operations are fakes."""
from __future__ import annotations
import argparse
from dataclasses import replace
from pathlib import Path
from types import SimpleNamespace
import uvicorn
from nest.server import create_app
from tests.test_server import FakeLifecycle, fake_application
from tests.test_sessions import FakeTerminal
from uvm.errors import ConflictError, NotFoundError
from uvm.lifecycle import ListedVm
class BrowserTerminal(FakeTerminal):
def __init__(self):
super().__init__()
self.command = bytearray()
async def write(self, data):
await self.events.put({"type": "output", "data": data.replace(b"\r", b"\r\n")})
for byte in data:
if byte == 13:
await self.events.put({"type": "output", "data": b"result:" + bytes(self.command) + b"\r\nguest$ "})
self.command.clear()
elif byte == 3:
self.command.clear()
else:
self.command.append(byte)
class BrowserLifecycle(FakeLifecycle):
def __init__(self):
super().__init__()
self.vm.username = "worker"
self.records = {self.vm.id: self.vm, "vm-other": replace(self.vm, id="vm-other", name="second-vm", status="stopped")}
def get(self, vm_id):
try:
return self.records[vm_id]
except KeyError as error:
raise NotFoundError("VM not found") from error
def inspect(self, identifier):
for vm in self.records.values():
if identifier in {vm.id, vm.name, vm.guest_ip}:
return ListedVm(vm, vm.status, "enforced" if vm.status == "running" else "inactive")
raise NotFoundError("VM not found")
def list_vms(self):
return [self.inspect(vm_id) for vm_id in self.records]
def open_terminal(self, vm_id, **_size):
if self.get(vm_id).status != "running":
raise ConflictError("Start the machine first")
return BrowserTerminal(), "worker"
def install_guest_agent(self, vm_id):
return self.get(vm_id)
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--base", type=Path, required=True)
parser.add_argument("--port", type=int, required=True)
args = parser.parse_args()
application = fake_application(api_token="browser-test-token")
application.settings.base = args.base
application.lifecycle = BrowserLifecycle()
application.state_store = SimpleNamespace(get=application.lifecycle.get)
api = create_app(application, host="127.0.0.1")
server = uvicorn.Server(uvicorn.Config(api, host="127.0.0.1", port=args.port, log_level="warning"))
api.state.uvicorn_server = server
server.run()
if __name__ == "__main__":
main()
tests/browser_sessions.cjs
-181
View File
@@ -1,181 +0,0 @@
// Optional integration test: requires Playwright + Chromium and the project's test extra.
const assert = require('node:assert/strict');
const { spawn } = require('node:child_process');
const { mkdtemp, rm } = require('node:fs/promises');
const os = require('node:os');
const path = require('node:path');
const { chromium } = require('playwright');
(async () => {
const temporary = await mkdtemp(path.join(os.tmpdir(), 'nest-browser-'));
const port = process.env.NEST_BROWSER_TEST_PORT || '18765';
const origin = `http://127.0.0.1:${port}`;
const server = spawn(process.env.NEST_TEST_PYTHON || 'python3',
['-B', '-m', 'tests.browser_session_server', '--base', temporary, '--port', port],
{ cwd: path.resolve(__dirname, '..'), stdio: ['ignore', 'pipe', 'pipe'] });
let logs = '';
server.stdout.on('data', (data) => { logs = (logs + data).slice(-16000); });
server.stderr.on('data', (data) => { logs = (logs + data).slice(-16000); });
let browser;
let page;
try {
let ready = false;
for (let attempt = 0; attempt < 100; attempt++) {
if (server.exitCode !== null) throw new Error(`Fixture server exited: ${logs}`);
try {
const response = await fetch(`${origin}/health`);
if (response.ok) { ready = true; break; }
} catch {}
await new Promise((resolve) => setTimeout(resolve, 100));
}
assert.ok(ready, `Fixture server did not start: ${logs}`);
browser = await chromium.launch({ headless: true });
const context = await browser.newContext({ viewport: { width: 1440, height: 1100 } });
page = await context.newPage();
const errors = [];
page.on('pageerror', (error) => errors.push(error.message));
const connected = () => page.waitForFunction(() => document.querySelector('#session-status').textContent === 'Connected');
const output = (value) => page.waitForFunction((text) => document.querySelector('#session-terminal').textContent.includes(text), value);
await page.goto(`${origin}/dashboard#vm/vm-test`);
await page.locator('#token-input').fill('browser-test-token');
await page.locator('#auth-btn').click();
await page.locator('#detail-actions [data-action="launch-session"]').click();
await connected();
const firstUrl = page.url();
assert.match(firstUrl, /#vm\/vm-test\/sessions\/session-/);
await page.keyboard.type('echo alpha');
await page.keyboard.press('Enter');
await output('result:echo alpha');
await page.reload();
await connected();
await output('result:echo alpha');
assert.equal(await page.locator('.session-item').count(), 1);
// Rename is cancellable, validated, persisted, and preserves the live widget.
const firstId = new URL(firstUrl).hash.split('/').at(-1);
const firstRow = () => page.locator(`.session-item[data-session-id="${firstId}"]`);
await page.locator('#session-rename').click();
await page.locator('#session-name-input').fill('Discarded name');
await page.locator('#session-rename-cancel').click();
assert.equal(await firstRow().locator('strong').textContent(), 'Session 1');
await page.evaluate(() => { window.originalTerminal = document.querySelector('#session-terminal .xterm'); });
await page.locator('#session-rename').click();
await page.locator('#session-name-input').fill(' ');
await page.locator('#session-rename-save').click();
assert.ok(await page.locator('#session-rename-dialog').isVisible());
assert.match(await page.locator('#session-rename-error').textContent(), /Enter a session name/);
await page.locator('#session-name-input').fill(' Deployment ');
await page.locator('#session-rename-save').click();
await page.locator('#session-rename-dialog').waitFor({ state: 'hidden' });
await page.waitForFunction(() => document.querySelector('#session-title').textContent.startsWith('Deployment'));
assert.equal(page.url(), firstUrl);
assert.ok(await page.evaluate(() => window.originalTerminal === document.querySelector('#session-terminal .xterm')));
await page.reload();
await connected();
await output('result:echo alpha');
assert.equal(await firstRow().locator('strong').textContent(), 'Deployment');
await page.locator('#detail-actions [data-action="launch-session"]').click();
await page.waitForURL((url) => url.href !== firstUrl);
await connected();
const secondUrl = page.url();
assert.equal(await page.locator('.session-item').count(), 2);
assert.ok(!(await page.locator('#session-terminal').textContent()).includes('result:echo alpha'));
await page.keyboard.type('echo beta');
await page.keyboard.press('Enter');
await output('result:echo beta');
// History actions also work without selecting the affected session.
await firstRow().locator('[data-session-action="rename"]').click();
await page.locator('#session-name-input').fill('Operations <alpha>');
await page.locator('#session-rename-save').click();
await page.locator('#session-rename-dialog').waitFor({ state: 'hidden' });
assert.equal(page.url(), secondUrl);
assert.equal(await firstRow().locator('strong').textContent(), 'Operations <alpha>');
assert.equal(await firstRow().locator('alpha').count(), 0);
await output('result:echo beta');
await firstRow().locator('.session-link').click();
await connected();
await output('result:echo alpha');
assert.ok(!(await page.locator('#session-terminal').textContent()).includes('result:echo beta'));
await page.goto(`${origin}/dashboard#vm/vm-other/sessions`);
await page.locator('#vm-sessions-panel').waitFor({ state: 'visible' });
await page.waitForFunction(() => document.querySelector('#vm-session-count').textContent === '0');
assert.equal(await page.locator('.session-item').count(), 0);
assert.ok(await page.locator('#detail-actions [data-action="launch-session"]').isDisabled());
assert.ok(!(await page.locator('#session-terminal').textContent()).includes('alpha'));
await page.goto(firstUrl);
await connected();
await output('result:echo alpha');
await page.locator('#session-end').click();
await page.locator('#confirm-ok').click();
await page.waitForFunction(() => document.querySelector('#session-status').textContent.startsWith('closed'));
await page.reload();
await output('result:echo alpha');
assert.ok(await page.locator('#session-end').isDisabled());
await page.locator('#session-rename').click();
await page.locator('#session-name-input').fill('Archived run');
await page.locator('#session-rename-save').click();
await page.locator('#session-rename-dialog').waitFor({ state: 'hidden' });
await page.reload();
await output('result:echo alpha');
assert.equal(await firstRow().locator('strong').textContent(), 'Archived run');
await page.setViewportSize({ width: 390, height: 844 });
await page.waitForTimeout(300);
const fits = await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth);
assert.ok(fits, 'mobile layout overflows the viewport');
if (process.env.NEST_BROWSER_SCREENSHOT) await page.screenshot({ path: process.env.NEST_BROWSER_SCREENSHOT, fullPage: true });
// Cancel preserves history; confirmation removes an ended session and clears selection.
await page.locator('#session-delete').click();
await page.locator('#confirm-dialog button[value="cancel"]').click();
assert.equal(await page.locator('.session-item').count(), 2);
await firstRow().locator('[data-session-action="delete"]').click();
await page.locator('#confirm-ok').click();
await page.waitForURL((url) => url.hash === '#vm/vm-test/sessions');
await page.waitForFunction(() => document.querySelector('#vm-session-count').textContent === '1');
assert.ok(await page.locator('#session-terminal').isHidden());
await page.reload();
await page.waitForFunction(() => document.querySelector('#vm-session-count').textContent === '1');
assert.equal(await firstRow().count(), 0);
// Deleting an active session clears other attached browser views as well.
await page.goto(secondUrl);
await connected();
await output('result:echo beta');
const observer = await page.context().newPage();
observer.on('pageerror', (error) => errors.push(error.message));
await observer.goto(secondUrl);
await observer.locator('#token-input').fill('browser-test-token');
await observer.locator('#auth-btn').click();
await observer.waitForFunction(() => document.querySelector('#session-status').textContent === 'Connected');
await page.locator('#session-rename').click();
await page.locator('#session-name-input').fill('Second workspace');
await page.locator('#session-rename-save').click();
await page.locator('#session-rename-dialog').waitFor({ state: 'hidden' });
await observer.waitForFunction(() => document.querySelector('#session-title').textContent.startsWith('Second workspace'));
await page.locator('#session-delete').click();
await page.locator('#confirm-ok').click();
await page.waitForFunction(() => document.querySelector('#vm-session-count').textContent === '0');
await observer.waitForURL((url) => url.hash === '#vm/vm-test/sessions');
assert.ok(await observer.locator('#session-terminal').isHidden());
await page.reload();
await page.waitForFunction(() => document.querySelector('#vm-session-count').textContent === '0');
assert.equal(await page.locator('.session-item').count(), 0);
assert.deepEqual(errors, []);
await observer.close();
console.log('Browser sessions passed: terminal/reconnect, VM isolation, persistent rename, active/ended deletion, cancellation, live browser updates, and mobile layout.');
} catch (error) {
if (page) console.error('Browser state:', await page.locator('#sessions-error').textContent(), await page.locator('#session-status').textContent());
console.error(logs);
throw error;
} finally {
await browser?.close();
server.kill('SIGTERM');
if (server.exitCode === null) await new Promise((resolve) => server.once('exit', resolve));
await rm(temporary, { recursive: true, force: true });
}
})().catch((error) => { console.error(error); process.exitCode = 1; });
tests/browser_tool_server.py
-130
View File
@@ -1,130 +0,0 @@
"""Browser fixture with the real API/job workers and simulated guest execution."""
from __future__ import annotations
import argparse
import json
import shlex
import threading
from dataclasses import replace
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import uvicorn
from nest.server import create_app
from tests.browser_session_server import BrowserLifecycle
from uvm.config import Settings
from uvm.domain import new_vm_id
from uvm.errors import ConflictError, GuestExecutionError
from uvm.guest_exec import ExecResult
from uvm.state import StateStore
from uvm.tool_store import ToolJobStore
from uvm.tools import ToolService
class FixtureConnection:
def __init__(self):
self.closed = threading.Event()
def close(self):
self.closed.set()
class ToolBrowserLifecycle(BrowserLifecycle):
def __init__(self, settings, state, *, delay=1.2):
super().__init__()
self.settings, self.state, self.delay = settings, state, delay
self.tools = ToolJobStore(settings, state)
self.installed = {}
self.attempts = {}
for index, vm in enumerate(self.records.values()):
vm.runtime_owned = True
vm.pid, vm.process_start_time, vm.boot_id = index + 100, "1", "fixture-boot"
settings.vm_dir(vm.id).mkdir(parents=True, mode=0o700, exist_ok=True)
with state.transaction() as document:
document.vms.update(self.records)
def create(self, spec):
vm = replace(self.vm, id=new_vm_id(), name=spec.name, username=spec.username, cpu=spec.cpu,
ram_mib=spec.ram_mib, storage_mib=spec.storage_mib, ports=spec.ports)
with self.state.operation_lock():
self.settings.vm_dir(vm.id).mkdir(mode=0o700)
self.records[vm.id] = vm
with self.state.transaction() as document:
document.vms[vm.id] = vm
if spec.tools:
self.tools.enqueue_locked(vm.id, spec.tools, source="create")
return vm
def queue_tools(self, vm_id, selections, **options):
self.state.get(vm_id)
with self.state.operation_lock():
return self.tools.enqueue_locked(vm_id, selections, **options)
def prepare_tool_install(self, vm_id, *, start_if_stopped=False, restart=False):
with self.state.operation_lock():
vm = self.records[vm_id]
if vm.status == "stopped" and not (start_if_stopped or restart):
raise ConflictError("start the VM first")
vm.status = "running"
with self.state.transaction() as document:
document.vms[vm.id] = vm
return vm
def execute(self, vm_id, command, *, expected_identity=None, on_connect=None, **_options):
self.state.get(vm_id)
payload = json.loads(shlex.split(command)[-1])
connection = FixtureConnection()
if on_connect:
on_connect(connection)
installed = self.installed.setdefault(vm_id, {"curl"})
def observation(tool):
versions = {"nodejs": "24.21.0", "jdk": "21.0.12", "php": "8.5.11", "conda": "26.5.3", "micromamba": "2.9.0"}
version = tool["version"] + ".15" if tool["id"] == "python" else versions.get(tool["id"], "2.0.0")
return {"key": tool["key"], "id": tool["id"], "version": tool["version"], "installed": tool["key"] in installed,
"detected_version": version if tool["key"] in installed else None, "error": None, "usage": tool["usage"]}
if payload["action"] == "probe":
result = {"ok": True, "busy": False, "boot_id": "guest-fixture", "supported": True,
"tools": [observation(tool) for tool in payload["tools"]]}
else:
if connection.closed.wait(self.delay):
raise GuestExecutionError("fixture connection closed", outcome_unknown=True)
tool = payload["tools"][0]
key = (vm_id, tool["key"])
self.attempts[key] = self.attempts.get(key, 0) + 1
if tool["id"] == "php" and self.attempts[key] == 1:
result = {"ok": False, "error": "Simulated mirror failure", "output": "<script>bad()</script>\nTry again later."}
else:
installed.add(tool["key"])
result = {"ok": True, "status": "installed", "tool": observation(tool),
"output": f"Verified and installed {tool['key']} inside the guest."}
return ExecResult(0, json.dumps(result), "", False, False)
def fixture_application(base, *, delay=1.2):
settings = replace(Settings(), base=base, api_token="browser-test-token", tool_workers=2)
state = StateStore(settings)
lifecycle = ToolBrowserLifecycle(settings, state, delay=delay)
return SimpleNamespace(settings=settings, state_store=state, lifecycle=lifecycle,
tools=ToolService(settings, state, lifecycle))
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--base", type=Path, required=True)
parser.add_argument("--port", type=int, required=True)
args = parser.parse_args()
application = fixture_application(args.base)
with patch("uvm.tools.require_root"):
api = create_app(application, host="127.0.0.1")
server = uvicorn.Server(uvicorn.Config(api, host="127.0.0.1", port=args.port, log_level="warning"))
api.state.uvicorn_server = server
server.run()
if __name__ == "__main__":
main()
tests/browser_tools.cjs
-106
View File
@@ -1,106 +0,0 @@
// Optional browser integration: real API/job persistence, simulated guest commands.
const assert = require('node:assert/strict');
const { spawn } = require('node:child_process');
const { mkdtemp, rm } = require('node:fs/promises');
const os = require('node:os');
const path = require('node:path');
const { chromium } = require('playwright');
(async () => {
const temporary = await mkdtemp(path.join(os.tmpdir(), 'nest-tool-browser-'));
const port = process.env.NEST_BROWSER_TEST_PORT || '18766';
const origin = `http://127.0.0.1:${port}`;
const server = spawn(process.env.NEST_TEST_PYTHON || 'python3',
['-B', '-m', 'tests.browser_tool_server', '--base', temporary, '--port', port],
{ cwd: path.resolve(__dirname, '..'), stdio: ['ignore', 'pipe', 'pipe'] });
let logs = '';
server.stdout.on('data', (data) => { logs = (logs + data).slice(-16000); });
server.stderr.on('data', (data) => { logs = (logs + data).slice(-16000); });
let browser;
let page;
try {
let ready = false;
for (let attempt = 0; attempt < 100; attempt++) {
if (server.exitCode !== null) throw new Error(`Fixture server exited: ${logs}`);
try { if ((await fetch(`${origin}/health`)).ok) { ready = true; break; } } catch {}
await new Promise((resolve) => setTimeout(resolve, 100));
}
assert.ok(ready, logs);
browser = await chromium.launch({ headless: true });
page = await browser.newPage({ viewport: { width: 1440, height: 1100 } });
const errors = [];
page.on('pageerror', (error) => errors.push(error.message));
const installed = (key) => page.waitForFunction((value) =>
document.querySelector(`[data-install-tool="${value}"]`)?.textContent === 'Installed', key);
await page.goto(`${origin}/dashboard#create`);
await page.locator('#token-input').fill('browser-test-token');
await page.locator('#auth-btn').click();
await page.locator('[data-create-tool="python-3.12"]').check();
await page.locator('[data-create-tool="git"]').check();
await page.locator('[data-create-tool="nodejs"]').check();
assert.equal(await page.locator('[data-create-tool]').count(), 12);
assert.match(await page.locator('#create-summary-details').textContent(), /Git, Node.js, Python 3.12/);
const layout = await page.evaluate(() => ({
parent: document.querySelector('.create-tools-section').parentElement.className,
children: document.querySelector('#create-form').children.length,
}));
assert.equal(layout.parent, 'create-fields');
assert.equal(layout.children, 2);
await page.locator('#create-name').fill('tool-workspace');
await page.locator('#create-submit').click();
await page.waitForURL((url) => /#vm\/vm-[a-f0-9]+\/tools$/.test(url.hash));
const createdUrl = page.url();
await page.locator('#tools-job').waitFor({ state: 'visible' });
// A reload can occur while the server-owned worker is still installing.
await page.reload();
await page.waitForFunction(() => document.querySelector('#tools-job-title').textContent === 'Complete');
await installed('curl');
assert.equal(await page.locator('.tool-installed').count(), 4); // Includes image-provided curl.
assert.ok(await page.locator('[data-install-tool="python-3.12"]').isDisabled());
await page.locator('#tools-job-items details').first().locator('summary').click();
assert.match(await page.locator('#tools-job-items').textContent(), /Verified and installed git/);
await page.locator('[data-install-tool="php"]').click();
await page.waitForFunction(() => document.querySelector('#tools-job-title').textContent === 'Failed');
await page.locator('#tools-job-items details[data-output="php"] summary').click();
assert.match(await page.locator('.tool-output').textContent(), /<script>bad/);
assert.equal(await page.locator('#tools-job-items script').count(), 0);
await page.locator('#tools-retry').click();
await page.waitForFunction(() => document.querySelector('#tools-job-title').textContent === 'Complete');
await installed('php');
assert.ok(await page.locator('[data-install-tool="php"]').isDisabled());
assert.equal(await page.locator('#tools-job-select option').count(), 3);
// The same click boots a stopped VM before installing its selected tool.
await page.goto(`${origin}/dashboard#vm/vm-other/tools`);
await page.locator('[data-install-tool="htop"]').waitFor();
assert.equal(await page.locator('[data-install-tool="htop"]').textContent(), 'Start & install');
await page.locator('[data-install-tool="htop"]').click();
await installed('htop');
await page.waitForFunction(() => document.querySelector('#tools-job-title').textContent === 'Complete');
assert.ok(await page.locator('[data-install-tool="htop"]').isDisabled());
assert.ok(!(await page.locator('#tools-job-items').textContent()).includes('python-3.12'));
await page.goto(createdUrl);
await page.locator('[data-tool-select="wget"]').check();
await page.locator('[data-tool-select="micromamba"]').check();
await page.locator('#tools-install-selected').click();
await installed('wget');
await installed('micromamba');
await page.waitForFunction(() => document.querySelector('#tools-job-title').textContent === 'Complete');
await page.setViewportSize({ width: 390, height: 844 });
await page.waitForTimeout(250);
assert.ok(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth), 'mobile layout overflows');
if (process.env.NEST_BROWSER_SCREENSHOT) await page.screenshot({ path: process.env.NEST_BROWSER_SCREENSHOT, fullPage: true });
assert.deepEqual(errors, []);
console.log('Browser tools passed: creation, background/reload, detection, failure/retry, safe output, history, Start & install, VM isolation, multi-select, and mobile layout.');
} catch (error) {
if (page) console.error('Tool UI:', await page.locator('#tools-error').textContent(), await page.locator('#tools-job-message').textContent());
console.error(logs);
throw error;
} finally {
await browser?.close();
server.kill('SIGTERM');
if (server.exitCode === null) await new Promise((resolve) => server.once('exit', resolve));
await rm(temporary, { recursive: true, force: true });
}
})().catch((error) => { console.error(error); process.exitCode = 1; });
tests/integration/test_guest_tools.py
-96
View File
@@ -1,96 +0,0 @@
"""Opt-in tool installation smoke test against a prepared, disposable microVM."""
from __future__ import annotations
import json
import os
import time
import unittest
import uuid
from urllib.parse import quote
from urllib.request import Request, urlopen
@unittest.skipUnless(os.environ.get("UVM_TOOLS_INTEGRATION") == "1",
"set UVM_TOOLS_INTEGRATION=1 on an isolated disposable VM host")
class GuestToolsIntegrationTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.api_url = os.environ.get("UVM_API_URL", "http://127.0.0.1:8000").rstrip("/")
cls.token = os.environ["UVM_API_TOKEN"]
cls.vm_id = os.environ["UVM_TOOLS_VM_ID"]
cls.path = "/vms/" + quote(cls.vm_id, safe="")
cls.vm = cls.request("GET", cls.path)
if (cls.vm["id"] != cls.vm_id or cls.vm["observed_status"] != "running"
or cls.vm["username"] == "root" or cls.vm["ports"] != []):
raise RuntimeError("provide a disposable running Ubuntu 22.04 VM with a non-root user and no open TCP ports")
@classmethod
def request(cls, method, path, body=None, *, timeout=120):
request = Request(cls.api_url + path, method=method,
data=json.dumps(body).encode() if body is not None else None,
headers={"X-UVM-Token": cls.token, "Content-Type": "application/json"})
with urlopen(request, timeout=timeout) as response:
return json.load(response)
def test_install_catalog_and_use_runtimes_as_the_guest_user(self):
catalog = self.request("GET", "/tools")
selections = [{"id": tool["id"], "version": version} for tool in catalog for version in tool["versions"] or [None]]
body = {"tools": selections, "request_id": str(uuid.uuid4())}
job = self.request("POST", self.path + "/tools/install", body)
deadline = time.monotonic() + 3600
while job["status"] in {"queued", "starting", "waiting", "installing"} and time.monotonic() < deadline:
time.sleep(2)
job = self.request("GET", self.path + "/tools/jobs/" + job["id"])
self.assertEqual(job["status"], "succeeded", json.dumps(job, indent=2))
self.assertEqual(self.request("POST", self.path + "/tools/install", body)["id"], job["id"])
inventory = self.request("GET", self.path + "/tools?refresh=true")
self.assertEqual(len(inventory["inventory"]), 12)
self.assertTrue(all(item["installed"] for item in inventory["inventory"]), inventory)
script = r'''set -eu
test "$(id -u)" != 0
git --version
node -e 'if (1 + 1 !== 2) process.exit(1)'
npm --version
wget --version >/dev/null
curl --version >/dev/null
htop --version
php -r 'if (2 + 2 !== 4) exit(1);'
work=$(mktemp -d "$HOME/nest-tools-smoke.XXXXXX")
name=$(basename "$work")
export MAMBA_ROOT_PREFIX="$HOME/micromamba"
cleanup() {
conda env remove -n "$name" -y >/dev/null 2>&1 || true
micromamba env remove -n "$name" -y >/dev/null 2>&1 || true
rm -rf -- "$work"
}
trap cleanup EXIT
printf 'class Hello { public static void main(String[] args) { System.out.println("ok"); } }\n' > "$work/Hello.java"
javac "$work/Hello.java"
test "$(java -cp "$work" Hello)" = ok
for version in 3.11 3.12 3.13; do
"python$version" -m venv "$work/venv-$version"
"$work/venv-$version/bin/python" -c 'import ssl, sqlite3, sys; assert sys.prefix != sys.base_prefix'
"$work/venv-$version/bin/python" -m pip --version
done
conda create -y -n "$name" python=3.13
conda run -n "$name" python -c 'import sys; assert sys.version_info[:2] == (3, 13)'
eval "$(conda shell.posix hook)"
conda activate "$name"
python -c 'import sys; assert sys.version_info[:2] == (3, 13)'
conda deactivate
micromamba create -y -n "$name" -c conda-forge python=3.12
micromamba run -n "$name" python -c 'import sys; assert sys.version_info[:2] == (3, 12)'
eval "$(micromamba shell hook --shell posix)"
micromamba activate "$name"
python -c 'import sys; assert sys.version_info[:2] == (3, 12)'
micromamba deactivate
'''
result = self.request("POST", self.path + "/exec",
{"command": script, "sudo": False, "timeout_seconds": 1200}, timeout=1220)
self.assertFalse(result["timed_out"], result)
self.assertEqual(result["exit_code"], 0, result)
after = self.request("GET", self.path)
for key in ("id", "guest_ip", "mac", "username", "ports", "cpu", "ram_mib", "storage_mib"):
self.assertEqual(after[key], self.vm[key])
tests/test_firecracker_config.py
-1
View File
@@ -46,7 +46,6 @@ class FirecrackerConfigTests(unittest.TestCase):
arguments = config["boot-source"]["boot_args"].split()
self.assertEqual("keep_bootcon" in arguments, architecture != "x86_64")
self.assertIn("console=ttyS0", arguments)
self.assertEqual(config["vsock"], {"guest_cid": 3, "uds_path": "/var/lib/uvm/vms/vm-test/vsock.sock"})
self.assertIn("ip=10.42.0.2::10.42.0.1:255.255.255.0::eth0:off", arguments)
client = RecordingFirecrackerClient()
client.configure_and_start(config)
tests/test_guest_agent_images.py
+5 -59
View File
@@ -74,14 +74,10 @@ class GuestAgentImageTests(unittest.TestCase):
def make_disk(self, marker="uvm-ubuntu-jammy-v3\n") -> None:
root = self.base / "tree"
if root.exists():
shutil.rmtree(root)
payloads = {
**self.retained,
**self.retained, "/etc/uvm-image": marker,
"/usr/lib/systemd/systemd": "systemd fixture", "/usr/bin/apt-get": "apt fixture",
}
if marker is not None:
payloads["/etc/uvm-image"] = marker
for path, content in payloads.items():
target = root / path.lstrip("/")
target.parent.mkdir(parents=True, exist_ok=True)
@@ -138,63 +134,13 @@ class GuestAgentImageTests(unittest.TestCase):
self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True)
self.assertEqual(self.guest("cat /home/worker/data"), self.retained["/home/worker/data"])
def test_all_compatible_images_get_vsock_agent_setup_without_opt_in(self) -> None:
for marker in ("uvm-ubuntu-jammy-v1\n", "uvm-ubuntu-jammy-v3\n", "uvm-ubuntu-jammy-v4\n",
"uvm-ubuntu-jammy-v5\n", "custom-image\n", None):
with self.subTest(marker=marker):
self.make_disk(marker)
template = hashlib.sha256(self.settings.rootfs_image.read_bytes()).digest()
self.assertTrue(self.images.provision_guest_agent(self.disk, "vm-test", "worker"))
self.assertEqual(json.loads(self.guest("cat /etc/uvm-agent.json")), {"vm_id": "vm-test", "username": "worker"})
self.assertIn('Fast link dest: "/etc/systemd/system/uvm-agent.service"',
self.guest("stat /etc/systemd/system/multi-user.target.wants/uvm-agent.service"))
self.assertEqual(self.guest("cat /usr/lib/uvm/agent.py"), guest_agent_files()["/usr/lib/uvm/agent.py"].decode())
for path, content in self.retained.items():
self.assertEqual(self.guest(f"cat {path}"), content)
self.assertEqual(hashlib.sha256(self.settings.rootfs_image.read_bytes()).digest(), template)
def test_current_agent_setup_is_read_only_on_later_boots_and_tracks_vm_identity(self) -> None:
def test_current_templates_are_configured_and_older_templates_require_explicit_install(self) -> None:
self.make_disk()
self.images.provision_guest_agent(self.disk, "vm-test", "worker")
before = self.snapshot()
with patch.object(self.images, "_copy_sparse") as copy:
self.assertFalse(self.images.provision_guest_agent(self.disk, "vm-test", "worker"))
copy.assert_not_called()
self.assertEqual(self.snapshot(), before)
self.assertTrue(self.images.provision_guest_agent(self.disk, "vm-recreated", "worker"))
self.assertEqual(json.loads(self.guest("cat /etc/uvm-agent.json")), {"vm_id": "vm-recreated", "username": "worker"})
def test_default_setup_reenables_an_existing_agent_service(self) -> None:
self.make_disk()
self.images.provision_guest_agent(self.disk, "vm-test", "worker")
self.runner.run(("debugfs", "-w", "-R", "rm /etc/systemd/system/multi-user.target.wants/uvm-agent.service", self.disk), capture=True)
self.replace_guest_file("/etc/uvm-image", "uvm-ubuntu-jammy-v4\n")
self.assertTrue(self.images.provision_guest_agent(self.disk, "vm-test", "worker"))
self.assertIn('Fast link dest: "/etc/systemd/system/uvm-agent.service"',
self.guest("stat /etc/systemd/system/multi-user.target.wants/uvm-agent.service"))
def test_incompatible_images_fail_default_setup_without_changing_the_disk(self) -> None:
for binary in ("/usr/lib/systemd/systemd", "/usr/bin/apt-get"):
with self.subTest(binary=binary):
self.make_disk(marker=None)
self.runner.run(("debugfs", "-w", "-R", f"rm {binary}", self.disk), capture=True)
before = self.snapshot()
with self.assertRaisesRegex(UvmError, "automatic vsock guest-agent setup requires a systemd/apt image"):
self.images.provision_guest_agent(self.disk, "vm-test", "worker")
self.assertEqual(self.snapshot(), before)
def test_cached_template_without_a_manifest_only_adopts_the_known_v4_payload(self) -> None:
self.make_disk()
self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True)
self.runner.run(("debugfs", "-w", "-R", "rm /etc/uvm-agent-files.json", self.disk), capture=True)
old_payload = "previous packaged agent\n"
self.replace_guest_file("/usr/lib/uvm/agent.py", old_payload)
before = self.snapshot()
with self.assertRaisesRegex(UvmError, "customized guest agent"):
self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True)
self.assertEqual(self.snapshot(), before)
with patch("uvm.images._V4_TEMPLATE_AGENT_SHA256", hashlib.sha256(old_payload.encode()).hexdigest()):
self.assertTrue(self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True))
self.assertEqual(self.guest("cat /usr/lib/uvm/agent.py"), guest_agent_files()["/usr/lib/uvm/agent.py"].decode())
def test_agent_updates_use_recorded_hashes_but_preserve_guest_customizations(self) -> None:
self.make_disk()
@@ -202,12 +148,12 @@ class GuestAgentImageTests(unittest.TestCase):
updated = guest_agent_files()
updated["/usr/lib/uvm/agent.py"] += b"\n# next packaged revision\n"
with patch("uvm.images.guest_agent_files", return_value=updated):
self.images.provision_guest_agent(self.disk, "vm-test", "worker")
self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True)
self.assertTrue(self.guest("cat /usr/lib/uvm/agent.py").endswith("# next packaged revision\n"))
self.replace_guest_file("/usr/lib/uvm/agent.py", "guest customization\n")
before = self.snapshot()
with self.assertRaisesRegex(UvmError, "customized guest agent file"):
self.images.provision_guest_agent(self.disk, "vm-test", "worker")
self.images.provision_guest_agent(self.disk, "vm-test", "worker", install=True)
self.assertEqual(self.snapshot(), before)
self.assertEqual(list(self.disk.parent.iterdir()), [self.disk])
tests/test_guest_exec_lifecycle.py
+1 -60
View File
@@ -4,7 +4,7 @@ from pathlib import Path
from unittest.mock import Mock
from tests.test_lifecycle import LifecycleTestCase
from uvm.errors import ConflictError, GuestAgentUnavailableError, GuestExecutionError, NotFoundError, UvmError
from uvm.errors import ConflictError, GuestAgentUnavailableError, GuestExecutionError, NotFoundError
from uvm.guest_exec import ExecResult
@@ -22,65 +22,6 @@ class GuestExecLifecycleTests(LifecycleTestCase):
vm = self.create_vm(username="worker")
self.assert_call_order(vm, "images.provision_agent", "process.start")
def test_stopped_boots_automatically_provision_agent_before_starting_the_vmm(self) -> None:
vm = self.create_vm(username="worker", password="retained-password")
identity = (vm.id, vm.guest_ip, vm.mac, vm.username, vm.password)
def provision(disk, vm_id, username):
self.host.call("images.provision_agent", vm_id)
record = self.store.get(vm_id)
self.assertTrue(self.store.operation_active)
self.assertEqual(self.process.observe(record), "gone")
self.assertEqual((record.id, record.guest_ip, record.mac, record.username, record.password), identity)
self.assertEqual((disk, username), (Path(vm.disk), "worker"))
self.provision_agent.side_effect = provision
for action in ("start", "restart", "resize"):
with self.subTest(action=action):
if action == "start":
self.service.stop(vm.id)
self.host.events.clear()
self.provision_agent.reset_mock()
self.provision.reset_mock()
if action == "resize":
started = self.service.resize(vm.id, cpu=2)
else:
started = getattr(self.service, action)(vm.id)
self.assertEqual(started.status, "running")
self.provision_agent.assert_called_once_with(Path(vm.disk), vm.id, "worker")
self.provision.assert_not_called()
self.assert_call_order(vm, "images.provision_agent", "process.start")
def test_live_or_ambiguous_vms_never_run_offline_agent_provisioning(self) -> None:
vm = self.create_vm()
self.provision_agent.reset_mock()
self.service.start(vm.id)
self.process.unknown.add(vm.id)
with self.assertRaises(ConflictError):
self.service.start(vm.id)
self.provision_agent.assert_not_called()
def test_default_agent_setup_failure_blocks_boot_and_keeps_the_retained_disk(self) -> None:
vm = self.create_vm()
self.service.stop(vm.id)
before = Path(vm.disk).read_bytes()
count = len(self.process.started)
self.provision_agent.side_effect = UvmError("agent setup failed")
with self.assertRaisesRegex(UvmError, "agent setup failed"):
self.service.start(vm.id)
self.assertEqual(len(self.process.started), count)
self.assertEqual(Path(vm.disk).read_bytes(), before)
self.assertEqual(self.store.get(vm.id).status, "failed")
self.provision_agent.side_effect = None
self.assertEqual(self.service.start(vm.id).status, "running")
def test_default_agent_setup_failure_rolls_back_a_new_vm_before_boot(self) -> None:
self.provision_agent.side_effect = UvmError("unsupported guest image")
with self.assertRaisesRegex(UvmError, "unsupported guest image"):
self.create_vm()
self.assertEqual(self.store.load().vms, {})
self.assertEqual(self.process.started, [])
def test_execution_pins_the_exact_vm_under_lock_and_waits_without_holding_it(self) -> None:
vm = self.create_vm(name="worker", ports=())
result = ExecResult(7, "output", "problem", False, False)
tests/test_guest_network.py
+1 -3
View File
@@ -108,9 +108,7 @@ class OfflineGuestNetworkTests(unittest.TestCase):
def test_templates_gain_an_explicit_address_gateway_and_boot_route_preservation(self) -> None:
for marker, profile in (("uvm-ubuntu-jammy-v1\n", LEGACY_PROFILE),
("uvm-ubuntu-jammy-v2\n", LEGACY_PROFILE),
("uvm-ubuntu-jammy-v3\n", GUEST_NETWORK_TEMPLATE),
("uvm-ubuntu-jammy-v4\n", GUEST_NETWORK_TEMPLATE),
("uvm-ubuntu-jammy-v5\n", GUEST_NETWORK_TEMPLATE)):
("uvm-ubuntu-jammy-v3\n", GUEST_NETWORK_TEMPLATE)):
with self.subTest(marker=marker):
self.make_disk(marker=marker, profile=profile)
original_template = hashlib.sha256(self.settings.rootfs_image.read_bytes()).digest()
tests/test_guest_terminal.py
-165
View File
@@ -1,165 +0,0 @@
from __future__ import annotations
import asyncio
import os
import pwd
import socket
import subprocess
import sys
import tempfile
import threading
import time
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
from tests.test_guest_exec import FakeScope
from tests.test_lifecycle import LifecycleTestCase
from uvm import guest_agent as agent
from uvm.errors import ConflictError, GuestAgentUnavailableError, NotFoundError
from uvm.guest_terminal import GuestTerminal
class GuestTerminalTests(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.scope = FakeScope()
self.addCleanup(self.scope.temporary.cleanup)
self.home = self.enterContext(tempfile.TemporaryDirectory())
username = pwd.getpwuid(os.geteuid()).pw_name
original_popen = subprocess.Popen
# Exercise the actual guest child entry point with a temporary account
# home, so interactive shells never read/write the developer's dotfiles.
script = f"""
import os, runpy
from pathlib import Path
from types import SimpleNamespace
agent = runpy.run_path({agent.__file__!r})
agent['pwd'].getpwnam = lambda _: SimpleNamespace(pw_name={username!r}, pw_uid=os.geteuid(),
pw_gid=os.getegid(), pw_dir={self.home!r}, pw_shell='/bin/sh')
agent['child_main'](Path({str(self.scope.path)!r}), {username!r}, None)
"""
def launch(_args, **kwargs):
return original_popen([sys.executable, "-I", "-c", script], **kwargs)
self.enterContext(patch.object(agent.subprocess, "Popen", side_effect=launch))
left, right = socket.socketpair()
self.terminal = GuestTerminal(left)
self.addCleanup(self.terminal.close)
def runner(connection, account, rows, cols):
return agent.run_terminal(connection, account, rows, cols, scope_factory=lambda: self.scope)
self.thread = threading.Thread(target=agent.serve_connection,
args=(right, {"vm_id": "vm-test", "username": username}),
kwargs={"terminal_runner": runner}, daemon=True)
self.thread.start()
self.addCleanup(self.thread.join, 3)
agent.receive_frame(left, 4096, time.monotonic() + 2)
await asyncio.to_thread(self.terminal.start, 24, 80)
self.prompt = b"# " if os.geteuid() == 0 else b"$ "
await self.until(self.prompt)
async def until(self, wanted):
data = bytearray()
try:
async with asyncio.timeout(5):
while wanted not in data:
message = await self.terminal.read()
self.assertEqual(message["type"], "output", message)
data.extend(message["data"])
except TimeoutError:
self.fail(f"Missing {wanted!r} in terminal output: {bytes(data)!r}")
return bytes(data)
async def asyncTearDown(self):
self.terminal.close()
await asyncio.to_thread(self.thread.join, 3)
self.assertFalse(self.thread.is_alive())
async def test_terminal_has_tty_persistent_shell_state_and_exit_code(self):
await self.terminal.write(b"test -t 0 && printf '\\nREAL-TTY\\n'\r")
await self.until(b"\r\nREAL-TTY\r\n")
await self.terminal.write(b"export NEST_TEST=hello; cd /\r")
await self.terminal.write(b"printf 'STATE:%s:%s\\n' \"$NEST_TEST\" \"$PWD\"\r")
await self.until(b"STATE:hello:/\r\n")
await self.terminal.write(b"exit 7\r")
async with asyncio.timeout(5):
while True:
message = await self.terminal.read()
if message["type"] == "exit":
self.assertEqual(message["exit_code"], 7)
break
async def test_resize_and_control_c_reach_the_guest_foreground_job(self):
await self.terminal.resize(40, 120)
await self.terminal.write(b"stty size\r")
await self.until(b"40 120\r\n")
await self.terminal.write(b"sh -c 'printf \"\\nJOB-STARTED\\n\"; exec sleep 20'\r")
await self.until(b"\r\nJOB-STARTED\r\n")
await self.terminal.write(b"\x03")
await self.until(self.prompt)
await self.terminal.write(b"printf '\\nINTERRUPTED\\n'\r")
await self.until(b"\r\nINTERRUPTED\r\n")
async def test_explicit_end_terminates_the_shell(self):
await self.terminal.write(b"printf '\\nREADY\\n'\r")
await self.until(b"\r\nREADY\r\n")
await self.terminal.end()
async with asyncio.timeout(5):
while (message := await self.terminal.read())["type"] != "exit":
pass
self.assertIsInstance(message["exit_code"], int)
self.assertGreater(self.scope.kills, 0)
class TerminalCompatibilityTests(unittest.TestCase):
def test_old_agent_rejection_explains_how_to_upgrade(self):
left, right = socket.socketpair()
terminal = GuestTerminal(left)
self.addCleanup(terminal.close)
def old_agent():
with right:
agent.receive_frame(right, 4096, time.monotonic() + 2)
agent.send_frame(right, {"error": "invalid execution request fields", "outcome_unknown": False},
4096, time.monotonic() + 2)
thread = threading.Thread(target=old_agent)
thread.start()
try:
with self.assertRaisesRegex(GuestAgentUnavailableError, "Restart this VM to apply automatic guest-agent setup"):
terminal.start(24, 80)
finally:
thread.join(3)
class TerminalLifecycleTests(LifecycleTestCase):
def test_terminal_connects_under_lock_and_opens_without_holding_the_lock(self):
vm = self.create_vm(name="worker", ports=())
terminal = Mock()
def connect(path, vm_id, username, pid):
self.assertTrue(self.store.operation_active)
self.assertEqual((path, vm_id, username, pid), (self.settings.guest_socket_path(vm.id), vm.id, vm.username, vm.pid))
return terminal
def start(rows, cols):
self.assertFalse(self.store.operation_active)
self.assertEqual((rows, cols), (40, 120))
self.service._terminal_connector = Mock(side_effect=connect)
terminal.start.side_effect = start
self.assertEqual(self.service.open_terminal(vm.id, rows=40, cols=120), (terminal, vm.username))
terminal.close.assert_not_called()
def test_aliases_and_stopped_vms_cannot_create_a_terminal(self):
vm = self.create_vm(name="worker")
self.service._terminal_connector = Mock()
for alias in (vm.name, vm.guest_ip):
with self.assertRaises(NotFoundError):
self.service.open_terminal(alias)
self.service.stop(vm.id)
with self.assertRaises(ConflictError):
self.service.open_terminal(vm.id)
self.service._terminal_connector.assert_not_called()
tests/test_server.py
-4
View File
@@ -878,10 +878,6 @@ class ApiRouterTests(unittest.TestCase):
("/vms/{vm_id}/ports/sync", "post"),
("/vms/{vm_id}/exec", "post"),
("/vms/{vm_id}/agent/install", "post"),
("/tools", "get"),
("/vms/{vm_id}/tools", "get"),
("/vms/{vm_id}/tools/install", "post"),
("/vms/{vm_id}/tools/jobs/{job_id}", "get"),
("/diagnostics", "get"),
},
)
tests/test_sessions.py
-502
View File
@@ -1,502 +0,0 @@
from __future__ import annotations
import asyncio
import base64
import os
import sqlite3
import tempfile
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from nest.server import create_app
from nest.session_store import SessionStore
from nest.sessions import SessionManager
from tests.test_server import FASTAPI_AVAILABLE, fake_application
from uvm.errors import ConflictError, GuestAgentUnavailableError, GuestExecutionError, NotFoundError, StateError, ValidationError
if FASTAPI_AVAILABLE:
from fastapi.testclient import TestClient
from starlette.websockets import WebSocketDisconnect
class FakeTerminal:
def __init__(self) -> None:
self.events = asyncio.Queue()
self.events.put_nowait({"type": "output", "data": b"guest$ "})
self.input = []
self.sizes = []
self.closed = False
self.end_calls = 0
async def read(self):
event = await self.events.get()
if isinstance(event, Exception):
raise event
return event
async def write(self, data):
self.input.append(data)
await self.events.put({"type": "output", "data": data + b"\r\nguest$ "})
async def resize(self, rows, cols):
self.sizes.append((rows, cols))
async def end(self):
self.end_calls += 1
await self.events.put({"type": "exit", "exit_code": 0})
def close(self):
self.closed = True
def session_application(base):
application = fake_application(api_token="test-token")
application.settings.base = Path(base)
application.terminals = []
application.vm_ids = {"vm-test", "vm-other"}
def get(vm_id):
if vm_id not in application.vm_ids:
raise NotFoundError("VM not found")
return SimpleNamespace(id=vm_id)
def open_terminal(vm_id, **_size):
get(vm_id)
terminal = FakeTerminal()
application.terminals.append(terminal)
return terminal, "worker"
application.state_store = SimpleNamespace(get=get)
application.lifecycle = SimpleNamespace(open_terminal=open_terminal)
return application
class SessionStoreTests(unittest.TestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory()
self.addCleanup(temporary.cleanup)
self.path = Path(temporary.name) / "nest" / "sessions.sqlite3"
self.store = SessionStore(self.path, max_output=8)
self.addCleanup(self.store.close)
def test_history_is_scoped_by_exact_vm_id_and_survives_reopening(self):
record = self.store.create("vm-a", 24, 80)
self.store.activate(record["id"], "worker")
self.store.append(record["id"], b"hello")
self.store.finish(record["id"], "closed", 7)
self.assertEqual(self.store.list("vm-b"), [])
with self.assertRaises(NotFoundError):
self.store.get("vm-b", record["id"])
self.store.close()
reopened = SessionStore(self.path)
self.addCleanup(reopened.close)
saved, start, output = reopened.read("vm-a", record["id"], 0)
self.assertEqual((saved["status"], saved["exit_code"], saved["username"], start, output), ("closed", 7, "worker", 0, b"hello"))
self.assertEqual(self.path.stat().st_mode & 0o777, 0o600)
self.assertEqual(self.path.parent.stat().st_mode & 0o777, 0o700)
def test_restarted_server_marks_only_live_sessions_interrupted(self):
live = self.store.create("vm-a", 24, 80)
ended = self.store.create("vm-a", 24, 80)
self.store.activate(live["id"], "worker")
self.store.finish(ended["id"], "closed", 0)
self.store.close()
reopened = SessionStore(self.path)
self.addCleanup(reopened.close)
self.assertEqual(reopened.get("vm-a", live["id"])["status"], "interrupted")
self.assertIsNotNone(reopened.get("vm-a", live["id"])["ended_at"])
self.assertEqual(reopened.get("vm-a", ended["id"])["status"], "closed")
def test_transcript_is_bounded_and_offsets_support_exact_replay(self):
record = self.store.create("vm-a", 24, 80)
self.store.append(record["id"], b"abcdefghij")
self.store.append(record["id"], b"kl")
saved, start, output = self.store.read("vm-a", record["id"], 0)
self.assertEqual((start, output, saved["output_end"], saved["truncated"]), (4, b"efghijkl", 12, True))
self.assertEqual(self.store.read("vm-a", record["id"], 9)[1:], (9, b"jkl"))
self.assertEqual(self.store.read("vm-a", record["id"], 12)[2], b"")
def test_second_server_cannot_reconcile_another_servers_live_sessions(self):
record = self.store.create("vm-a", 24, 80)
self.store.activate(record["id"], "worker")
with self.assertRaises(ConflictError):
SessionStore(self.path)
self.assertEqual(self.store.get("vm-a", record["id"])["status"], "active")
def test_symlinked_history_file_is_not_opened_or_rewritten(self):
self.store.close()
self.path.unlink()
target = self.path.parent / "unrelated"
target.write_bytes(b"keep this")
self.path.symlink_to(target)
with self.assertRaises(OSError):
SessionStore(self.path)
self.assertEqual(target.read_bytes(), b"keep this")
def test_rename_preserves_session_identity_state_and_output_across_reopening(self):
record = self.store.create("vm-a", 24, 80)
self.store.activate(record["id"], "worker")
self.store.append(record["id"], b"history")
before = self.store.get("vm-a", record["id"])
renamed = self.store.rename("vm-a", record["id"], " Déploiement 'alpha' ")
self.assertEqual(renamed["name"], "Déploiement 'alpha'")
for key in before.keys() - {"name", "updated_at"}:
self.assertEqual(renamed[key], before[key])
self.store.close()
reopened = SessionStore(self.path)
self.addCleanup(reopened.close)
saved, _start, output = reopened.read("vm-a", record["id"], 0)
self.assertEqual(saved["name"], renamed["name"])
self.assertEqual(output, b"history")
def test_invalid_session_names_do_not_change_the_record(self):
record = self.store.create("vm-a", 24, 80)
for name in ("", " ", "a" * 81, "two\nlines", "bad\x00name", "\ud800", 123, None):
with self.subTest(name=repr(name)), self.assertRaises(ValidationError):
self.store.rename("vm-a", record["id"], name)
self.assertEqual(self.store.get("vm-a", record["id"]), record)
def test_rename_and_delete_cannot_cross_vm_boundaries(self):
record = self.store.create("vm-a", 24, 80)
self.store.finish(record["id"], "closed", 0)
for operation in (lambda: self.store.rename("vm-b", record["id"], "other"),
lambda: self.store.delete("vm-b", record["id"])):
with self.assertRaises(NotFoundError):
operation()
self.assertEqual(self.store.get("vm-a", record["id"])["name"], record["name"])
def test_delete_removes_metadata_and_transcript_and_keeps_other_sessions(self):
first = self.store.create("vm-a", 24, 80)
second = self.store.create("vm-a", 24, 80)
self.store.append(first["id"], b"remove")
self.store.append(second["id"], b"keep")
self.store.finish(first["id"], "closed", 0)
self.store.delete("vm-a", first["id"])
with self.assertRaises(NotFoundError):
self.store.read("vm-a", first["id"], 0)
with sqlite3.connect(self.path) as db:
self.assertEqual(db.execute("SELECT COUNT(*) FROM output WHERE session_id=?", (first["id"],)).fetchone()[0], 0)
self.assertEqual(self.store.read("vm-a", second["id"], 0)[2], b"keep")
def test_failed_delete_rolls_back_metadata_and_output_together(self):
record = self.store.create("vm-a", 24, 80)
self.store.append(record["id"], b"keep")
self.store.finish(record["id"], "closed", 0)
with sqlite3.connect(self.path) as db:
db.execute("CREATE TRIGGER reject_delete BEFORE DELETE ON sessions BEGIN SELECT RAISE(ABORT, 'test failure'); END")
with self.assertRaises(StateError):
self.store.delete("vm-a", record["id"])
self.assertEqual(self.store.read("vm-a", record["id"], 0)[2], b"keep")
def test_running_history_requires_closure_and_default_names_avoid_existing_labels(self):
records = [self.store.create("vm-a", 24, 80) for _ in range(3)]
with self.assertRaises(ConflictError):
self.store.delete("vm-a", records[1]["id"])
self.store.finish(records[1]["id"], "closed", 0)
self.store.delete("vm-a", records[1]["id"])
created = self.store.create("vm-a", 24, 80)
self.assertNotIn(created["name"], {records[0]["name"], records[2]["name"]})
class SessionManagerTests(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.temporary = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary.cleanup)
self.application = session_application(self.temporary.name)
self.manager = SessionManager(self.application)
self.addAsyncCleanup(self.manager.shutdown)
async def test_failed_launch_is_saved_in_history(self):
def unavailable(*_args, **_kwargs):
raise GuestAgentUnavailableError("update the guest agent")
self.application.lifecycle.open_terminal = unavailable
with self.assertRaises(GuestAgentUnavailableError):
await self.manager.create("vm-test", 24, 80)
history = await self.manager.list("vm-test")
self.assertEqual(history[0]["status"], "failed")
self.assertEqual(history[0]["error"], "update the guest agent")
async def test_shutdown_closes_live_channels_and_saves_interrupted_history(self):
record = await self.manager.create("vm-test", 24, 80)
await self.manager.shutdown()
self.assertTrue(self.application.terminals[0].closed)
store = SessionStore(Path(self.temporary.name) / "nest" / "sessions.sqlite3")
self.addCleanup(store.close)
self.assertEqual(store.get("vm-test", record["id"])["status"], "interrupted")
async def test_cancelling_the_create_request_does_not_orphan_the_shell(self):
import threading
entered, release = threading.Event(), threading.Event()
connect = self.application.lifecycle.open_terminal
def slow_connect(*args, **kwargs):
entered.set()
release.wait(2)
return connect(*args, **kwargs)
self.application.lifecycle.open_terminal = slow_connect
request = asyncio.create_task(self.manager.create("vm-test", 24, 80))
await asyncio.to_thread(entered.wait, 2)
request.cancel()
release.set()
await asyncio.gather(request, return_exceptions=True)
await self.manager.shutdown()
self.assertTrue(self.application.terminals[0].closed)
async def test_guest_loss_ends_the_session_and_retains_output(self):
record = await self.manager.create("vm-test", 24, 80)
terminal = self.application.terminals[0]
await terminal.events.put(GuestExecutionError("VM connection lost", outcome_unknown=True))
async with asyncio.timeout(2):
while (await self.manager.get("vm-test", record["id"]))["status"] == "active":
await asyncio.sleep(0.01)
saved, _start, output = await self.manager.read("vm-test", record["id"], 0)
self.assertEqual(saved["status"], "interrupted")
self.assertEqual(output, b"guest$ ")
self.assertTrue(terminal.closed)
async def test_shutdown_finishes_inflight_output_before_releasing_history_ownership(self):
import threading
entered, release = threading.Event(), threading.Event()
history = self.manager._history()
append = history.append
def slow_append(*args):
entered.set()
release.wait(2)
append(*args)
with patch.object(history, "append", side_effect=slow_append):
record = await self.manager.create("vm-test", 24, 80)
self.assertTrue(await asyncio.to_thread(entered.wait, 2))
closing = asyncio.create_task(self.manager.shutdown())
try:
await asyncio.sleep(0.03)
self.assertFalse(closing.done())
with self.assertRaises(ConflictError):
SessionStore(history.path)
finally:
release.set()
await closing
reopened = SessionStore(history.path)
self.addCleanup(reopened.close)
self.assertEqual(reopened.read("vm-test", record["id"], 0)[2], b"guest$ ")
async def test_rename_keeps_the_same_live_terminal(self):
record = await self.manager.create("vm-test", 24, 80)
terminal = self.application.terminals[0]
renamed = await self.manager.rename("vm-test", record["id"], "Deployment")
self.assertEqual((renamed["id"], renamed["status"], renamed["name"]), (record["id"], "active", "Deployment"))
await self.manager.write("vm-test", record["id"], b"pwd\r")
self.assertEqual(terminal.input, [b"pwd\r"])
self.assertFalse(terminal.closed)
self.assertEqual(len(self.application.terminals), 1)
async def test_delete_closes_only_its_live_terminal_and_removes_all_history(self):
record = await self.manager.create("vm-test", 24, 80)
other = await self.manager.create("vm-test", 24, 80)
await self.manager.write("vm-test", record["id"], b"last output\r")
await self.manager.delete("vm-test", record["id"])
self.assertTrue(self.application.terminals[0].closed)
self.assertFalse(self.application.terminals[1].closed)
self.assertEqual([item["id"] for item in await self.manager.list("vm-test")], [other["id"]])
with sqlite3.connect(self.manager._history().path) as db:
self.assertEqual(db.execute("SELECT COUNT(*) FROM output WHERE session_id=?", (record["id"],)).fetchone()[0], 0)
async def test_delete_waits_for_an_inflight_launch_before_closing_its_channel(self):
import threading
entered, release = threading.Event(), threading.Event()
connect = self.application.lifecycle.open_terminal
def slow_connect(*args, **kwargs):
entered.set()
release.wait(2)
return connect(*args, **kwargs)
self.application.lifecycle.open_terminal = slow_connect
creating = asyncio.create_task(self.manager.create("vm-test", 24, 80))
self.assertTrue(await asyncio.to_thread(entered.wait, 2))
record = (await self.manager.list("vm-test"))[0]
deleting = asyncio.create_task(self.manager.delete("vm-test", record["id"]))
try:
await asyncio.sleep(0.03)
self.assertFalse(deleting.done())
finally:
release.set()
await asyncio.gather(creating, deleting)
self.assertTrue(self.application.terminals[0].closed)
self.assertEqual(await self.manager.list("vm-test"), [])
async def test_cancelled_delete_request_finishes_cleanup_even_during_shutdown(self):
record = await self.manager.create("vm-test", 24, 80)
terminal = self.application.terminals[0]
entered, release = asyncio.Event(), asyncio.Event()
end = terminal.end
async def slow_end():
entered.set()
await release.wait()
await end()
terminal.end = slow_end
request = asyncio.create_task(self.manager.delete("vm-test", record["id"]))
await asyncio.wait_for(entered.wait(), timeout=2)
request.cancel()
await asyncio.gather(request, return_exceptions=True)
release.set()
await self.manager.shutdown()
reopened = SessionStore(Path(self.temporary.name) / "nest" / "sessions.sqlite3")
self.addCleanup(reopened.close)
self.assertTrue(terminal.closed)
self.assertEqual(reopened.list("vm-test"), [])
async def test_close_and_delete_are_serialized_for_the_same_session(self):
record = await self.manager.create("vm-test", 24, 80)
results = await asyncio.gather(self.manager.end("vm-test", record["id"]),
self.manager.delete("vm-test", record["id"]), return_exceptions=True)
self.assertTrue(all(not isinstance(result, Exception) or isinstance(result, NotFoundError) for result in results))
self.assertTrue(self.application.terminals[0].closed)
self.assertEqual(self.application.terminals[0].end_calls, 1)
self.assertEqual(await self.manager.list("vm-test"), [])
@unittest.skipUnless(FASTAPI_AVAILABLE, "FastAPI and httpx are required")
class SessionApiTests(unittest.TestCase):
def setUp(self):
self.temporary = self.enterContext(tempfile.TemporaryDirectory())
self.application = session_application(self.temporary)
self.api = create_app(self.application, host="127.0.0.1")
self.client = self.enterContext(TestClient(self.api))
self.headers = {"X-UVM-Token": "test-token"}
self.base = "/vms/vm-test/sessions"
def create(self):
response = self.client.post(self.base, headers=self.headers, json={})
self.assertEqual(response.status_code, 201, response.text)
return response.json()
def connect(self, record, *, vm_id="vm-test", offset=0):
ws = self.enterContext(self.client.websocket_connect(f"/vms/{vm_id}/sessions/{record['id']}/terminal"))
ws.send_json({"type": "auth", "token": "test-token", "offset": offset})
return ws
def output(self, ws):
while True:
message = ws.receive_json()
if message["type"] == "output":
data = base64.b64decode(message["data"])
end = message["offset"] + len(data)
ws.send_json({"type": "ack", "offset": end})
return data, end
self.assertEqual(message["type"], "session", message)
def test_all_http_operations_and_websocket_output_require_authentication(self):
for method, path in (("GET", self.base), ("POST", self.base),
("GET", self.base + "/session-unknown"), ("POST", self.base + "/session-unknown/close"),
("PATCH", self.base + "/session-unknown"), ("DELETE", self.base + "/session-unknown")):
self.assertEqual(self.client.request(method, path, json={}).status_code, 401)
for token in ("wrong", "", "\u00e9"):
with self.client.websocket_connect(self.base + "/session-unknown/terminal") as ws:
ws.send_json({"type": "auth", "token": token})
with self.assertRaises(WebSocketDisconnect) as raised:
ws.receive_json()
self.assertEqual(raised.exception.code, 4401)
self.assertEqual(self.application.terminals, [])
self.assertFalse((Path(self.temporary) / "nest").exists())
def test_foreign_browser_origin_is_rejected(self):
with self.assertRaises(WebSocketDisconnect):
with self.client.websocket_connect(self.base + "/unknown/terminal", headers={"Origin": "https://foreign.example"}):
self.fail("foreign origin accepted")
def test_create_rejects_unknown_vm_aliases_and_invalid_sizes(self):
for body in ({"rows": 0}, {"cols": True}, {"cols": 401}, {"command": "id"}):
self.assertEqual(self.client.post(self.base, headers=self.headers, json=body).status_code, 422)
self.assertEqual(self.client.post("/vms/name-alias/sessions", headers=self.headers, json={}).status_code, 404)
self.assertEqual(self.application.terminals, [])
def test_sessions_are_nested_under_the_selected_vm(self):
record = self.create()
self.assertEqual(record["username"], "worker")
other = "/vms/vm-other/sessions"
self.assertEqual(self.client.get(other, headers=self.headers).json(), [])
self.assertEqual(self.client.get(other + "/" + record["id"], headers=self.headers).status_code, 404)
self.assertEqual(self.client.post(other + "/" + record["id"] + "/close", headers=self.headers).status_code, 404)
self.assertEqual(self.client.patch(other + "/" + record["id"], headers=self.headers, json={"name": "Other"}).status_code, 404)
self.assertEqual(self.client.delete(other + "/" + record["id"], headers=self.headers).status_code, 404)
ws = self.connect(record, vm_id="vm-other")
self.assertEqual(ws.receive_json()["type"], "error")
self.assertFalse(self.application.terminals[0].closed)
def test_disconnect_reconnect_input_resize_and_closed_history(self):
record = self.create()
ws = self.connect(record)
output, offset = self.output(ws)
self.assertEqual(output, b"guest$ ")
ws.send_json({"type": "resize", "rows": 35, "cols": 120})
ws.send_json({"type": "input", "data": base64.b64encode(b"pwd\r").decode()})
output, offset = self.output(ws)
self.assertEqual(output, b"pwd\r\r\nguest$ ")
self.assertEqual(self.application.terminals[0].sizes, [(35, 120)])
ws.close()
self.assertFalse(self.application.terminals[0].closed)
reopened = self.connect(record, offset=offset)
self.assertEqual(reopened.receive_json()["session"]["status"], "active")
self.assertEqual(len(self.application.terminals), 1)
closed = self.client.post(self.base + "/" + record["id"] + "/close", headers=self.headers)
self.assertEqual(closed.json()["status"], "closed")
self.assertEqual(closed.json()["exit_code"], 0)
replay = self.connect(record)
transcript, _end = self.output(replay)
self.assertEqual(transcript, b"guest$ pwd\r\r\nguest$ ")
self.assertEqual(self.client.get(self.base, headers=self.headers).json()[0]["status"], "closed")
self.assertTrue(self.application.terminals[0].closed)
def test_malformed_input_ends_only_the_browser_attachment(self):
record = self.create()
ws = self.connect(record)
self.output(ws)
ws.send_json({"type": "input", "data": "not base64!"})
while (message := ws.receive_json())["type"] == "session":
pass
self.assertEqual(message["type"], "error")
self.assertEqual(self.application.terminals[0].input, [])
self.assertFalse(self.application.terminals[0].closed)
self.assertEqual(self.client.get(self.base + "/" + record["id"], headers=self.headers).json()["status"], "active")
def test_rename_is_validated_persisted_and_sent_to_attached_terminals(self):
record = self.create()
url = self.base + "/" + record["id"]
for body in ({}, {"name": " "}, {"name": 123}, {"name": "x" * 81},
{"name": "two\nlines"}, {"name": "bad\x00name"}, {"name": "New", "vm_id": "vm-other"}):
with self.subTest(body=body):
self.assertEqual(self.client.patch(url, headers=self.headers, json=body).status_code, 422)
ws = self.connect(record)
self.output(ws)
response = self.client.patch(url, headers=self.headers, json={"name": " Déploiement "})
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(response.json()["name"], "Déploiement")
while (message := ws.receive_json()).get("session", {}).get("name") != "Déploiement":
self.assertEqual(message["type"], "session")
self.assertEqual(message["session"]["status"], "active")
self.assertEqual(self.client.get(url, headers=self.headers).json()["name"], "Déploiement")
self.assertFalse(self.application.terminals[0].closed)
def test_delete_live_session_returns_no_content_and_notifies_attached_browser(self):
record = self.create()
ws = self.connect(record)
self.output(ws)
url = self.base + "/" + record["id"]
response = self.client.delete(url, headers=self.headers)
self.assertEqual(response.status_code, 204, response.text)
self.assertEqual(response.content, b"")
while (message := ws.receive_json())["type"] == "session":
pass
self.assertEqual(message, {"type": "deleted", "session_id": record["id"]})
self.assertTrue(self.application.terminals[0].closed)
self.assertEqual(self.client.get(url, headers=self.headers).status_code, 404)
self.assertEqual(self.client.get(self.base, headers=self.headers).json(), [])
tests/test_tool_api.py
-113
View File
@@ -1,113 +0,0 @@
from __future__ import annotations
import tempfile
import unittest
from dataclasses import replace
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock
from tests.test_server import FASTAPI_AVAILABLE, FakeLifecycle
from uvm.config import Settings
from uvm.server import create_api
from uvm.state import StateStore
from uvm.tools import ToolService
if FASTAPI_AVAILABLE:
from fastapi.testclient import TestClient
@unittest.skipUnless(FASTAPI_AVAILABLE, "FastAPI and httpx are required")
class ToolApiTests(unittest.TestCase):
def setUp(self):
temporary = self.enterContext(tempfile.TemporaryDirectory())
settings = replace(Settings(), base=Path(temporary), api_token="test-token")
state = StateStore(settings)
self.lifecycle = FakeLifecycle()
self.lifecycle.vm.runtime_owned = True
settings.vm_dir(self.lifecycle.vm.id).mkdir(parents=True, mode=0o700)
with state.transaction() as document:
document.vms[self.lifecycle.vm.id] = self.lifecycle.vm
self.tools = ToolService(settings, state, self.lifecycle)
self.tools.start = Mock()
self.tools.shutdown = Mock()
self.tools.notify = Mock()
self.application = SimpleNamespace(settings=settings, state_store=state, lifecycle=self.lifecycle, tools=self.tools)
self.api = create_api(self.application, host="127.0.0.1")
self.client = TestClient(self.api)
self.addCleanup(self.client.close)
self.headers = {"X-UVM-Token": "test-token"}
def job(self):
with self.application.state_store.operation_lock():
return self.tools.store.enqueue_locked("vm-test", [{"id": "git"}], request_id="request-1")
def test_catalog_and_job_routes_require_the_management_token(self):
for method, path in (("GET", "/tools"), ("GET", "/vms/vm-test/tools"),
("POST", "/vms/vm-test/tools/install"),
("GET", "/vms/vm-test/tools/jobs/tools-missing")):
with self.subTest(path=path):
response = self.client.request(method, path, json={"tools": [{"id": "git"}]})
self.assertEqual(response.status_code, 401)
self.tools.start.assert_not_called()
def test_catalog_has_python_options_and_create_passes_normalized_selections(self):
result = self.client.get("/tools", headers=self.headers)
self.assertEqual(result.status_code, 200)
python = next(item for item in result.json() if item["id"] == "python")
self.assertEqual(python["versions"], ["3.11", "3.12", "3.13"])
created = self.client.post("/vms", headers=self.headers, json={"tools": [
{"id": "python", "version": "3.12"}, {"id": "git"}, {"id": "git"},
]})
self.assertEqual(created.status_code, 201, created.text)
self.assertEqual([item.key for item in self.lifecycle.last_spec.tools], ["git", "python-3.12"])
self.tools.start.assert_called_once()
self.tools.notify.assert_called_once()
self.assertNotIn("password", created.json())
def test_invalid_selections_and_options_are_rejected_before_submission(self):
self.tools.submit = Mock()
invalid = [
{"tools": []}, {"tools": None}, {"tools": ["git"]}, {"tools": [{"id": "unknown"}]},
{"tools": [{"id": "python", "version": "3.14"}]}, {"tools": [{"id": "python"}]},
{"tools": [{"id": "git", "script": "id"}]}, {"tools": [{"id": "git"}], "start_if_stopped": "true"},
{"tools": [{"id": "git"}], "restart": 1}, {"tools": [{"id": "git"}], "request_id": "../../x"},
{"tools": [{"id": "git"}], "sudo": False},
]
for body in invalid:
with self.subTest(body=body):
response = self.client.post("/vms/vm-test/tools/install", headers=self.headers, json=body)
self.assertEqual(response.status_code, 422, response.text)
self.tools.submit.assert_not_called()
for body in ({"tools": None}, {"tools": [{"id": "not-in-catalog"}]}):
response = self.client.post("/vms", headers=self.headers, json=body)
self.assertEqual(response.status_code, 422)
self.assertIsNone(self.lifecycle.last_spec)
def test_install_returns_accepted_job_and_whitelists_its_response(self):
job = self.job()
job.update(password="secret", host_path="/host/private")
self.tools.submit = Mock(return_value=job)
result = self.client.post("/vms/vm-test/tools/install", headers=self.headers,
json={"tools": [{"id": "git"}], "start_if_stopped": True, "request_id": "request-1"})
self.assertEqual(result.status_code, 202, result.text)
self.assertEqual(result.json()["id"], job["id"])
self.assertNotIn("password", result.json())
self.assertNotIn("host_path", result.json())
self.assertTrue(self.tools.submit.call_args.kwargs["start_if_stopped"])
self.assertEqual(self.tools.submit.call_args.args[0], "vm-test")
def test_job_history_remains_exact_vm_scoped(self):
job = self.job()
result = self.client.get(f"/vms/vm-test/tools/jobs/{job['id']}", headers=self.headers)
self.assertEqual(result.status_code, 200, result.text)
self.assertEqual(result.json()["items"][0]["status"], "queued")
for identifier in ("test-vm", "10.42.0.2", "vm-other"):
response = self.client.get(f"/vms/{identifier}/tools/jobs/{job['id']}", headers=self.headers)
self.assertEqual(response.status_code, 404)
def test_server_lifespan_starts_and_stops_workers(self):
with TestClient(self.api) as client:
self.tools.start.assert_called_once()
self.assertEqual(client.get("/tools", headers=self.headers).status_code, 200)
self.tools.shutdown.assert_called_once()
tests/test_tool_catalog.py
-162
View File
@@ -1,162 +0,0 @@
from __future__ import annotations
import ast
import hashlib
import io
import shlex
import tarfile
import tempfile
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from uvm import guest_tools
from uvm.errors import ValidationError
from uvm.tool_catalog import CATALOG, ToolSelection, all_selections, normalize_tools, recipe
from uvm.tools import guest_command
class ToolCatalogTests(unittest.TestCase):
def test_requested_catalog_and_every_guest_payload_fit_the_agent_protocol(self):
self.assertEqual({tool["id"] for tool in CATALOG},
{"git", "nodejs", "python", "jdk", "wget", "curl", "htop", "php", "conda", "micromamba"})
for arch in ("x86_64", "aarch64"):
recipes = [recipe(item, arch) for item in all_selections()]
command = guest_command({"action": "probe", "tools": recipes, "username": "worker", "architecture": arch})
self.assertLess(len(command.encode()), 65536)
argv = shlex.split(command)
self.assertEqual(argv[:3], ["/usr/bin/python3", "-I", "-c"])
compile(argv[3], "guest-tools", "exec")
ast.parse(argv[3], feature_version=(3, 10))
for item in recipes:
if "url" in item:
self.assertTrue(item["url"].startswith("https://"))
self.assertNotIn("/latest", item["url"])
self.assertRegex(item["sha256"], r"^[0-9a-f]{64}$")
self.assertEqual(len(all_selections()), 12)
def test_selection_validation_is_allowlisted_strict_and_deduplicated(self):
expected = (ToolSelection("git"), ToolSelection("python", "3.12"))
self.assertEqual(normalize_tools([{"id": "python", "version": "3.12"}, {"id": "git"}, {"id": "git"}]), expected)
for value in (None, "git", [], ["git"], [{"id": "git; id"}], [{"id": "python"}],
[{"id": "python", "version": "3.14"}], [{"id": "git", "version": "latest"}],
[{"id": "git", "command": "id"}], [{"id": ["git"]}], [{"id": "git"}] * 13):
with self.subTest(value=value), self.assertRaises(ValidationError):
normalize_tools(value)
self.assertEqual(normalize_tools([], allow_empty=True), ())
def test_versioned_python_commands_and_jdk_compiler_are_verified(self):
for version in ("3.11", "3.12", "3.13"):
item = recipe(ToolSelection("python", version), "aarch64")
self.assertEqual(set(item["links"]), {"python" + version})
self.assertIn("venv", item["checks"][0][-1])
self.assertEqual(item["checks"][1][:3], ["python" + version, "-m", "pip"])
jdk = recipe(ToolSelection("jdk"), "aarch64")
self.assertEqual(jdk["checks"], [["javac", "-version"], ["java", "-version"]])
self.assertIn("arm64", jdk["java_home"])
class GuestToolRecipeTests(unittest.TestCase):
def setUp(self):
self.temporary = self.enterContext(tempfile.TemporaryDirectory())
self.root = Path(self.temporary)
def test_download_mismatch_removes_artifact_and_https_redirect_is_checked(self):
target = self.root / "download"
response = io.BytesIO(b"publisher payload")
response.geturl = lambda: "https://publisher.test/tool"
with patch.object(guest_tools.urllib.request, "urlopen", return_value=response):
with self.assertRaisesRegex(guest_tools.ToolError, "checksum"):
guest_tools.download("https://publisher.test/tool", target, "0" * 64)
self.assertFalse(target.exists())
response = io.BytesIO(b"publisher payload")
response.geturl = lambda: "https://publisher.test/tool"
with patch.object(guest_tools.urllib.request, "urlopen", return_value=response):
guest_tools.download("https://publisher.test/tool", target, hashlib.sha256(b"publisher payload").hexdigest())
self.assertEqual(target.read_bytes(), b"publisher payload")
with self.assertRaisesRegex(guest_tools.ToolError, "HTTPS"):
guest_tools.download("http://publisher.test/tool", target)
response = io.BytesIO(b"publisher payload")
response.geturl = lambda: "http://publisher.test/tool"
with patch.object(guest_tools.urllib.request, "urlopen", return_value=response):
with self.assertRaisesRegex(guest_tools.ToolError, "HTTPS"):
guest_tools.download("https://publisher.test/tool", target)
def test_archive_traversal_links_and_devices_cannot_escape_staging(self):
cases = [("../outside", tarfile.REGTYPE, ""), ("/outside", tarfile.REGTYPE, ""),
("link", tarfile.SYMTYPE, "../../outside"), ("link", tarfile.LNKTYPE, "../outside"),
("device", tarfile.CHRTYPE, "")]
for name, kind, link in cases:
with self.subTest(name=name, kind=kind):
archive = self.root / "archive.tar"
with tarfile.open(archive, "w") as stream:
item = tarfile.TarInfo(name)
item.type = kind
item.linkname = link
stream.addfile(item)
output = self.root / "output"
output.mkdir(exist_ok=True)
with self.assertRaises(guest_tools.ToolError):
guest_tools.extract_archive(archive, output)
self.assertFalse((self.root / "outside").exists())
def test_internal_archive_symlinks_are_supported(self):
archive = self.root / "archive.tar"
with tarfile.open(archive, "w") as stream:
item = tarfile.TarInfo("python/bin/python3.12")
item.size = 4
item.mode = 0o755
stream.addfile(item, io.BytesIO(b"test"))
link = tarfile.TarInfo("python/bin/python3")
link.type = tarfile.SYMTYPE
link.linkname = "python3.12"
stream.addfile(link)
output = self.root / "output"
output.mkdir()
guest_tools.extract_archive(archive, output)
self.assertEqual((output / "python/bin/python3").read_bytes(), b"test")
def test_probe_uses_guest_account_and_rejects_wrong_runtime_or_missing_companion(self):
node = recipe(ToolSelection("nodejs"), "x86_64")
account = SimpleNamespace(pw_dir="/home/worker", pw_uid=1001, pw_gid=1002)
with (patch.object(guest_tools.pwd, "getpwnam", return_value=account),
patch.object(guest_tools.os, "getgrouplist", return_value=[1002, 1003]),
patch.object(guest_tools.shutil, "which", return_value="/usr/local/bin/node"),
patch.object(guest_tools.subprocess, "run") as run):
run.return_value = SimpleNamespace(returncode=0, stdout=b"v22.10.0\n")
result = guest_tools.probe(node, "worker")
self.assertFalse(result["installed"])
self.assertEqual(result["detected_version"], "22.10.0")
self.assertEqual(run.call_args.kwargs["user"], 1001)
self.assertEqual(run.call_args.kwargs["env"]["HOME"], "/home/worker")
run.side_effect = [SimpleNamespace(returncode=0, stdout=b"v24.21.0\n"),
SimpleNamespace(returncode=1, stdout=b"broken npm")]
self.assertFalse(guest_tools.probe(node, "worker")["installed"])
def test_managed_files_preserve_edits_and_can_finish_interrupted_publication(self):
installer = guest_tools.Installer("root")
path = self.root / "profile"
# Isolated directory fixtures stand in for the root-owned guest hierarchy.
with (patch.object(guest_tools, "ROOT", self.root),
patch.object(guest_tools, "safe_directory"),
patch.object(guest_tools, "regular_file")):
installer.write_managed(path, b"initial")
installer.write_managed(path, b"updated")
self.assertEqual(path.read_bytes(), b"updated")
path.write_bytes(b"guest edit")
with self.assertRaisesRegex(guest_tools.ToolError, "customized"):
installer.write_managed(path, b"next")
self.assertEqual(path.read_bytes(), b"guest edit")
path.write_bytes(b"updated")
installer.remember(path, hashlib.sha256(b"next").hexdigest())
installer.write_managed(path, b"next")
self.assertEqual(path.read_bytes(), b"next")
def test_log_capture_retains_a_bounded_tail(self):
installer = guest_tools.Installer("root")
installer.append(b"a" * 20000)
installer.append(b"last failure")
self.assertEqual(len(installer.log), guest_tools.LOG_LIMIT)
self.assertTrue(installer.truncated)
self.assertTrue(installer.log.endswith(b"last failure"))
tests/test_tool_flow.py
-73
View File
@@ -1,73 +0,0 @@
from __future__ import annotations
import tempfile
import time
import unittest
from pathlib import Path
from unittest.mock import patch
from tests.test_server import FASTAPI_AVAILABLE
if FASTAPI_AVAILABLE:
from fastapi.testclient import TestClient
from tests.browser_tool_server import fixture_application
from uvm.server import create_api
@unittest.skipUnless(FASTAPI_AVAILABLE, "FastAPI and httpx are required")
class ToolFlowTests(unittest.TestCase):
def setUp(self):
temporary = self.enterContext(tempfile.TemporaryDirectory())
self.application = fixture_application(Path(temporary), delay=0.01)
self.enterContext(patch("uvm.tools.require_root"))
self.headers = {"X-UVM-Token": "browser-test-token"}
def wait_for(self, condition):
deadline = time.monotonic() + 5
while time.monotonic() < deadline:
if value := condition():
return value
time.sleep(0.01)
self.fail("background tool worker did not reach the expected state")
def test_create_and_later_install_use_real_workers_and_durable_job_responses(self):
with TestClient(create_api(self.application, host="127.0.0.1")) as client:
response = client.post("/vms", headers=self.headers, json={"name": "tools-vm", "tools": [
{"id": "git"}, {"id": "python", "version": "3.12"},
]})
self.assertEqual(response.status_code, 201, response.text)
vm_id = response.json()["id"]
self.wait_for(lambda: self.application.tools.store.read(vm_id)["jobs"][-1]["status"] == "succeeded")
snapshot = client.get(f"/vms/{vm_id}/tools?refresh=true", headers=self.headers)
self.assertEqual(snapshot.status_code, 200, snapshot.text)
installed = {item["key"] for item in snapshot.json()["inventory"] if item["installed"]}
self.assertEqual(installed, {"git", "python-3.12", "curl"})
self.assertEqual(snapshot.json()["latest_job"]["source"], "create")
later = client.post("/vms/vm-other/tools/install", headers=self.headers,
json={"tools": [{"id": "htop"}], "start_if_stopped": True})
self.assertEqual(later.status_code, 202, later.text)
job_id = later.json()["id"]
self.wait_for(lambda: self.application.tools.store.get_job("vm-other", job_id)["status"] == "succeeded")
self.assertEqual(self.application.state_store.get("vm-other").status, "running")
self.assertEqual(client.get(f"/vms/{vm_id}/tools/jobs/{job_id}", headers=self.headers).status_code, 404)
def test_queued_jobs_are_dispatched_on_server_start(self):
with self.application.state_store.operation_lock():
job = self.application.tools.store.enqueue_locked("vm-test", [{"id": "git"}])
with TestClient(create_api(self.application, host="127.0.0.1")):
self.wait_for(lambda: self.application.tools.store.get_job("vm-test", job["id"])["status"] == "succeeded")
self.assertEqual(self.application.lifecycle.attempts[("vm-test", "git")], 1)
def test_shutdown_wakes_command_waiter_and_keeps_an_interrupted_job(self):
self.application.lifecycle.delay = 30
with TestClient(create_api(self.application, host="127.0.0.1")) as client:
response = client.post("/vms/vm-test/tools/install", headers=self.headers, json={"tools": [{"id": "git"}]})
job_id = response.json()["id"]
self.wait_for(lambda: self.application.tools.store.get_job("vm-test", job_id)["status"] == "installing"
and bool(self.application.tools._sessions))
started = time.monotonic()
self.assertLess(time.monotonic() - started, 3)
job = self.application.tools.store.get_job("vm-test", job_id)
self.assertEqual(job["status"], "interrupted")
self.assertTrue(job["outcome_unknown"])
self.assertTrue(all(not thread.is_alive() for thread in self.application.tools._threads))
tests/test_tool_jobs.py
-149
View File
@@ -1,149 +0,0 @@
from __future__ import annotations
import json
import tempfile
import unittest
from concurrent.futures import ThreadPoolExecutor
from dataclasses import replace
from pathlib import Path
from unittest.mock import Mock, patch
from tests.test_server import FakeLifecycle
from uvm.config import Settings
from uvm.errors import ConflictError, GuestExecutionError, NotFoundError, StateError
from uvm.state import StateStore
from uvm.tool_catalog import ToolSelection
from uvm.tool_store import ToolJobStore
from uvm.tools import ToolService
class ToolJobTests(unittest.TestCase):
def setUp(self):
temporary = self.enterContext(tempfile.TemporaryDirectory())
self.settings = replace(Settings(), base=Path(temporary), tool_workers=1)
self.state = StateStore(self.settings)
self.vm = replace(FakeLifecycle().vm, runtime_owned=True, pid=123, process_start_time="1", boot_id="host-boot")
self.settings.vm_dir(self.vm.id).mkdir(parents=True, mode=0o700)
with self.state.transaction() as state:
state.vms[self.vm.id] = self.vm
self.store = ToolJobStore(self.settings, self.state)
def enqueue(self, selections=None, **options):
with self.state.operation_lock():
return self.store.enqueue_locked(self.vm.id, selections or [ToolSelection("git")], **options)
def test_duplicate_concurrent_submissions_have_one_durable_job(self):
with ThreadPoolExecutor(max_workers=4) as executor:
jobs = list(executor.map(lambda _: self.enqueue(request_id="request-1"), range(4)))
self.assertEqual(len({job["id"] for job in jobs}), 1)
self.assertEqual(len(self.store.read(self.vm.id)["jobs"]), 1)
path = self.settings.vm_dir(self.vm.id) / "tools.json"
self.assertEqual(path.stat().st_mode & 0o777, 0o600)
self.assertNotIn("guest-secret", path.read_text())
with self.assertRaises(ConflictError):
self.enqueue([ToolSelection("htop")])
def test_history_is_vm_scoped_and_deletion_is_never_undone(self):
job = self.enqueue()
with self.assertRaises(NotFoundError):
self.store.get_job(self.vm.name, job["id"])
with self.state.transaction() as state:
del state.vms[self.vm.id]
with self.assertRaises(NotFoundError):
self.store.update_job(self.vm.id, job["id"], lambda current, data: current.update(status="succeeded"))
self.assertNotIn(self.vm.id, self.state.load().vms)
def test_recovery_preserves_queued_work_and_marks_inflight_outcomes_unknown(self):
job = self.enqueue()
self.store.recover()
self.assertEqual(self.store.get_job(self.vm.id, job["id"])["status"], "queued")
self.store.claim()
self.store.update_job(self.vm.id, job["id"], lambda current, data: current["items"][0].update(status="installing"))
self.store.recover()
recovered = self.store.get_job(self.vm.id, job["id"])
self.assertEqual(recovered["status"], "interrupted")
self.assertTrue(recovered["outcome_unknown"])
self.assertEqual(recovered["items"][0]["status"], "interrupted")
self.assertIsNone(self.store.claim())
def test_symlinks_and_foreign_history_are_rejected(self):
outside = self.settings.base / "outside"
outside.write_text("private data")
path = self.settings.vm_dir(self.vm.id) / "tools.json"
path.symlink_to(outside)
with self.assertRaises((StateError, OSError)):
self.store.read(self.vm.id)
self.assertEqual(outside.read_text(), "private data")
path.unlink()
self.enqueue()
data = json.loads(path.read_text())
data["jobs"][0]["vm_id"] = "vm-other"
path.write_text(json.dumps(data))
with self.assertRaises(StateError):
self.store.read(self.vm.id)
def test_only_one_worker_can_own_this_registry_and_shutdown_releases_it(self):
first = ToolService(self.settings, self.state, Mock())
second = ToolService(self.settings, self.state, Mock())
self.addCleanup(first.shutdown)
self.addCleanup(second.shutdown)
first.start()
with self.assertRaises(ConflictError):
second.start()
first.shutdown()
second.start()
def make_service(self):
lifecycle = Mock()
lifecycle.prepare_tool_install.return_value = self.vm
service = ToolService(self.settings, self.state, lifecycle)
return service
@staticmethod
def observation(selection, installed=False):
return {**selection.to_dict(), "key": selection.key, "installed": installed,
"detected_version": "2.0.0" if installed else None, "usage": "tool --version", "error": None}
def test_worker_reports_partial_failure_and_retains_successful_tools(self):
selections = [ToolSelection("git"), ToolSelection("htop")]
job = self.enqueue(selections)
claimed = self.store.claim()
service = self.make_service()
results = [
{"ok": True, "busy": False, "boot_id": "guest-boot", "tools": [self.observation(item) for item in selections]},
{"ok": True, "status": "installed", "tool": self.observation(selections[0], True), "output": "git installed"},
{"ok": False, "error": "mirror unavailable", "output": "apt download failed"},
]
with patch.object(service, "_execute", side_effect=results) as execute:
service._run_job(claimed)
result = self.store.get_job(self.vm.id, job["id"])
self.assertEqual(result["status"], "failed")
self.assertEqual([item["status"] for item in result["items"]], ["installed", "failed"])
self.assertEqual(result["items"][1]["output"], "apt download failed")
self.assertEqual(self.state.get(self.vm.id).status, "running")
self.assertEqual(execute.call_args.args[1]["boot_id"], "guest-boot")
def test_lost_result_is_never_replayed_and_remaining_tools_are_skipped(self):
selections = [ToolSelection("git"), ToolSelection("htop")]
job = self.enqueue(selections)
service = self.make_service()
with patch.object(service, "_execute", side_effect=[
{"ok": True, "busy": False, "boot_id": "boot", "tools": [self.observation(item) for item in selections]},
GuestExecutionError("connection lost", outcome_unknown=True),
]) as execute:
service._run_job(self.store.claim())
result = self.store.get_job(self.vm.id, job["id"])
self.assertEqual(execute.call_count, 2)
self.assertEqual(result["status"], "interrupted")
self.assertTrue(result["outcome_unknown"])
self.assertEqual([item["status"] for item in result["items"]], ["interrupted", "skipped"])
def test_prior_guest_installation_blocks_a_retry_before_submission(self):
job = self.enqueue()
service = self.make_service()
with patch.object(service, "_execute", return_value={"ok": True, "busy": True}) as execute:
service._run_job(self.store.claim())
self.assertEqual(execute.call_count, 1)
result = self.store.get_job(self.vm.id, job["id"])
self.assertEqual(result["status"], "interrupted")
self.assertFalse(result["outcome_unknown"])
tests/test_tool_lifecycle.py
-57
View File
@@ -1,57 +0,0 @@
from __future__ import annotations
from unittest.mock import Mock
from tests.test_lifecycle import LifecycleTestCase
from uvm.errors import ConflictError, NotFoundError, UvmError, ValidationError
from uvm.tool_catalog import ToolSelection
class ToolLifecycleTests(LifecycleTestCase):
def test_create_persists_tool_intent_before_boot(self):
def configure(config):
vm_id = next(iter(self.store.load().vms))
jobs = self.service._tool_store._read(vm_id)["jobs"]
self.assertEqual(jobs[0]["selections"], [{"id": "python", "version": "3.12"}])
self.assertEqual(jobs[0]["status"], "queued")
original = self.service._client_factory
def client(socket, timeout):
result = original(socket, timeout)
result.configure_and_start = configure
return result
self.service._client_factory = client
vm = self.create_vm(tools=(ToolSelection("python", "3.12"),))
self.assertEqual(vm.status, "running")
self.assertEqual(self.service._tool_store.read(vm.id)["jobs"][0]["source"], "create")
def test_invalid_tools_fail_before_reserving_a_vm_and_failed_boot_removes_intent(self):
with self.assertRaises(ValidationError):
self.create_vm(tools=({"id": "invalid"},))
self.assertEqual(self.store.load().vms, {})
self.host.failures["client.configure", None] = UvmError("failed boot")
with self.assertRaises(UvmError):
self.create_vm(tools=(ToolSelection("git"),))
self.assertEqual(list(self.settings.vms_dir.iterdir()), [])
def test_stopped_installation_starts_only_when_requested_and_aliases_are_rejected(self):
vm = self.create_vm(name="worker")
self.service.stop(vm.id)
for alias in (vm.name, vm.guest_ip):
with self.assertRaises(NotFoundError):
self.service.queue_tools(alias, [ToolSelection("git")], start_if_stopped=True)
with self.assertRaises(ConflictError):
self.service.queue_tools(vm.id, [ToolSelection("git")])
job = self.service.queue_tools(vm.id, [ToolSelection("git")], start_if_stopped=True)
self.assertEqual(self.store.get(vm.id).status, "stopped")
ready = self.service.prepare_tool_install(vm.id, start_if_stopped=job["start_if_stopped"])
self.assertEqual(ready.status, "running")
self.assertEqual(ready.guest_ip, vm.guest_ip)
def test_a_tool_step_cannot_reconnect_to_a_restarted_vm(self):
vm = self.create_vm()
identity = vm.pid, vm.process_start_time, vm.boot_id
self.service.restart(vm.id)
self.service._guest_connector = Mock()
with self.assertRaisesRegex(ConflictError, "restarted"):
self.service.execute(vm.id, "id", expected_identity=identity)
self.service._guest_connector.assert_not_called()
uvm/api_models.py
-88
View File
@@ -11,7 +11,6 @@ from .domain import VmRecord
from .errors import ValidationError
from .guest_agent import DEFAULT_TIMEOUT_SECONDS, MAX_COMMAND_BYTES, MAX_TIMEOUT_SECONDS
from .guest_exec import validate_exec
from .tool_catalog import MAX_SELECTIONS, ToolSelection
class HealthResponse(BaseModel):
@@ -43,19 +42,6 @@ class PortRuleModel(BaseModel):
protocol: Literal["TCP", "HTTP", "HTTPS"] = "TCP"
class ToolSelectionModel(BaseModel):
model_config = ConfigDict(extra="forbid")
id: str = Field(strict=True, min_length=1, max_length=32)
version: str | None = Field(default=None, strict=True, max_length=16,
description="Required for Python: 3.11, 3.12, or 3.13. Omit for other tools.")
@model_validator(mode="after")
def validate_selection(self):
ToolSelection(self.id, self.version)
return self
class VmCreateRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
@@ -68,8 +54,6 @@ class VmCreateRequest(BaseModel):
name: str | None = None
ports: list[PortRuleModel] | None = None
no_default_ports: bool = Field(default=False, strict=True)
tools: list[ToolSelectionModel] = Field(default_factory=list, max_length=MAX_SELECTIONS,
description="Optional guest tools to install asynchronously after boot")
class VmResizeRequest(BaseModel):
@@ -165,78 +149,6 @@ class VmAgentInstallResponse(BaseModel):
restart_required: bool = True
class ToolCatalogResponse(BaseModel):
id: str
name: str
description: str
version_label: str
versions: list[str]
source: str
class ToolInstallRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
tools: list[ToolSelectionModel] = Field(min_length=1, max_length=MAX_SELECTIONS)
start_if_stopped: bool = Field(default=False, strict=True)
restart: bool = Field(default=False, strict=True, description="Restart first to refresh the guest agent")
request_id: str | None = Field(default=None, strict=True, max_length=80, pattern=r"^[A-Za-z0-9_-]{1,80}$",
description="Optional idempotency key, retained with the most recent 10 jobs")
class ToolInventoryResponse(BaseModel):
key: str
id: str
version: str | None
installed: bool
detected_version: str | None
error: str | None
usage: str
class ToolJobItemResponse(BaseModel):
key: str
id: str
version: str | None
status: Literal["queued", "installing", "installed", "already_installed", "failed", "interrupted", "skipped"]
detected_version: str | None
error: str | None
output: str
truncated: bool
class ToolJobSummaryResponse(BaseModel):
id: str
status: Literal["queued", "starting", "waiting", "installing", "succeeded", "failed", "interrupted"]
created_at: float
completed_at: float | None
class ToolJobResponse(ToolJobSummaryResponse):
vm_id: str
source: Literal["create", "user"]
updated_at: float
start_if_stopped: bool
restart: bool
request_id: str | None
error: str | None
outcome_unknown: bool
items: list[ToolJobItemResponse]
class VmToolsResponse(BaseModel):
vm_id: str
inventory: list[ToolInventoryResponse]
checked_at: float | None
agent_ready: bool
guest_busy: bool
supported: bool | None
error: str | None
stale: bool
latest_job: ToolJobResponse | None
jobs: list[ToolJobSummaryResponse]
class DiagnosticResponse(BaseModel):
name: str
ok: bool
uvm/app.py
-3
View File
@@ -14,7 +14,6 @@ from .lifecycle import LifecycleService
from .network import NetworkManager
from .state import StateStore
from .system import CommandRunner
from .tools import ToolService
@dataclass(slots=True)
@@ -25,7 +24,6 @@ class Application:
state_store: StateStore
installer: Installer
lifecycle: LifecycleService
tools: ToolService | None = None
def build_application(emit: Callable[[str], None] | None = print) -> Application:
@@ -43,5 +41,4 @@ def build_application(emit: Callable[[str], None] | None = print) -> Application
state_store=state_store,
installer=installer,
lifecycle=lifecycle,
tools=ToolService(settings, state_store, lifecycle),
)
uvm/config.py
-11
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
import math
import os
from dataclasses import dataclass, field
from ipaddress import IPv4Address, IPv4Network
@@ -47,9 +46,6 @@ class Settings:
api_timeout_s: float = 2.0
api_socket_timeout_s: float = 5.0
terminate_timeout_s: float = 5.0
tool_workers: int = 2
tool_ready_timeout_s: float = 120.0
tool_install_timeout_s: float = 1200.0
def __post_init__(self) -> None:
architecture = self.architecture.lower()
@@ -58,13 +54,6 @@ class Settings:
raise ConfigurationError(f"unsupported host architecture: {self.architecture}")
object.__setattr__(self, "architecture", architecture)
if type(self.tool_workers) is not int or not 1 <= self.tool_workers <= 8:
raise ConfigurationError("tool_workers must be an integer between 1 and 8")
for name in ("tool_ready_timeout_s", "tool_install_timeout_s"):
value = getattr(self, name)
if type(value) not in (int, float) or not 0 < value <= 3600 or not math.isfinite(value):
raise ConfigurationError(f"{name} must be finite, greater than zero, and at most 3600 seconds")
# Resolve defaults at construction time, while preserving individual URL overrides.
asset_base = f"{DEFAULT_GUEST_ASSETS_URL}/{architecture}"
if not self.kernel_url:
uvm/domain.py
-2
View File
@@ -11,7 +11,6 @@ from typing import Any
from .config import MAX_STORAGE_MIB, MIN_STORAGE_MIB
from .errors import StateError, ValidationError
from .tool_catalog import ToolSelection
STATE_SCHEMA_VERSION = 3
@@ -56,7 +55,6 @@ class VmSpec:
name: str | None = None
ports: tuple[PortRule, ...] = (PortRule(22),)
storage_mib: int = MIN_STORAGE_MIB
tools: tuple[ToolSelection, ...] = ()
@dataclass(frozen=True, slots=True)
uvm/guest_agent.py
+3 -134
View File
@@ -6,9 +6,6 @@ The host also imports its wire-format helpers; importing it never starts a serve
from __future__ import annotations
import base64
import errno
import fcntl
import json
import math
import os
@@ -21,7 +18,6 @@ import stat
import struct
import subprocess
import sys
import termios
import threading
import time
import uuid
@@ -39,8 +35,6 @@ MAX_RESPONSE_BYTES = 6 * MAX_OUTPUT_BYTES + 4096
CONNECT_TIMEOUT_SECONDS = 5.0
CLEANUP_SECONDS = 5.0
MAX_CONCURRENT_COMMANDS = 8
MAX_TERMINAL_BYTES = 8192
MAX_TERMINAL_FRAME = 16384
CONFIG_PATH = Path("/etc/uvm-agent.json")
@@ -54,23 +48,6 @@ class ExecutionFailure(Exception):
self.outcome_unknown = outcome_unknown
def validate_terminal_size(rows: object, cols: object) -> None:
if type(rows) is not int or type(cols) is not int or not (2 <= rows <= 200 and 2 <= cols <= 400):
raise ProtocolError("terminal dimensions must be integers: 2–200 rows and 2–400 columns")
def terminal_data(value: object) -> bytes:
if not isinstance(value, str) or len(value) > 4 * ((MAX_TERMINAL_BYTES + 2) // 3):
raise ProtocolError("terminal data exceeds its size limit")
try:
data = base64.b64decode(value, validate=True)
except (ValueError, UnicodeError) as error:
raise ProtocolError("invalid terminal data") from error
if len(data) > MAX_TERMINAL_BYTES:
raise ProtocolError("terminal data exceeds its size limit")
return data
def validate_command(command: object, sudo: object, timeout_seconds: object) -> None:
if not isinstance(command, str) or not command.strip() or "\0" in command:
raise ProtocolError("command must be non-empty text without NUL characters")
@@ -190,9 +167,7 @@ def _cleanup_job(process: subprocess.Popen | None, scope: JobScope) -> None:
failed = True
finally:
for name in ("stdout", "stderr"):
stream = getattr(process, name)
if stream is not None:
stream.close()
getattr(process, name).close()
try:
scope.close()
except (OSError, ExecutionFailure):
@@ -264,12 +239,9 @@ def run_command(command: str, username: str, timeout_seconds: float, *, scope_fa
}
def child_main(scope: Path, username: str, command: str | None) -> None:
def child_main(scope: Path, username: str, command: str) -> None:
"""Enter the job cgroup and account in a fresh interpreter, then replace it."""
(scope / "cgroup.procs").write_text(str(os.getpid()), encoding="ascii")
if command is None:
# Popen's start_new_session already performed setsid in this fresh child.
fcntl.ioctl(0, termios.TIOCSCTTY, 0)
account = pwd.getpwnam(username)
if os.geteuid() == 0:
os.initgroups(account.pw_name, account.pw_gid)
@@ -284,111 +256,15 @@ def child_main(scope: Path, username: str, command: str | None) -> None:
"SHELL": "/bin/sh", "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG": "C.UTF-8",
}
if command is None:
shell = account.pw_shell or "/bin/sh"
environment.update(SHELL=shell, TERM="xterm-256color", COLORTERM="truecolor")
os.execve(shell, [shell, "-i"], environment)
else:
os.execve("/bin/sh", ["sh", "-c", command], environment)
def run_terminal(connection: socket.socket, username: str, rows: int, cols: int, *, scope_factory=JobScope) -> None:
"""Bridge one real guest PTY until shell exit, explicit close, or host loss."""
validate_terminal_size(rows, cols)
scope = scope_factory()
process = None
master = slave = None
try:
master, slave = os.openpty()
fcntl.ioctl(slave, termios.TIOCSWINSZ, struct.pack("HHHH", rows, cols, 0, 0))
process = subprocess.Popen(
[sys.executable, "-I", str(Path(__file__).resolve()), "--terminal-child", str(scope.path), username],
stdin=slave, stdout=slave, stderr=slave, start_new_session=True,
env={"PATH": "/usr/sbin:/usr/bin:/sbin:/bin", "LANG": "C.UTF-8"},
)
os.close(slave)
slave = None
os.set_blocking(master, False)
send_frame(connection, {"type": "ready"}, MAX_TERMINAL_FRAME, time.monotonic() + CONNECT_TIMEOUT_SECONDS)
pending = bytearray()
with selectors.DefaultSelector() as selector:
selector.register(connection, selectors.EVENT_READ, "host")
selector.register(master, selectors.EVENT_READ, "pty")
finished = False
while not finished:
for key, events in selector.select(0.1):
if key.data == "host":
message = receive_frame(connection, MAX_TERMINAL_FRAME, time.monotonic() + CONNECT_TIMEOUT_SECONDS)
if message == {"type": "close"}:
finished = True
break
if set(message) == {"type", "data"} and message["type"] == "input":
pending.extend(terminal_data(message["data"]))
if len(pending) > 65536:
raise ProtocolError("terminal input backlog exceeded")
elif set(message) == {"type", "rows", "cols"} and message["type"] == "resize":
validate_terminal_size(message["rows"], message["cols"])
fcntl.ioctl(master, termios.TIOCSWINSZ, struct.pack("HHHH", message["rows"], message["cols"], 0, 0))
else:
raise ProtocolError("invalid terminal control message")
selector.modify(master, selectors.EVENT_READ | (selectors.EVENT_WRITE if pending else 0), "pty")
else:
if events & selectors.EVENT_READ:
try:
data = os.read(master, MAX_TERMINAL_BYTES)
except OSError as error:
if error.errno != errno.EIO:
raise
data = b""
if not data:
finished = True
break
send_frame(connection, {"type": "output", "data": base64.b64encode(data).decode("ascii")},
MAX_TERMINAL_FRAME, time.monotonic() + CONNECT_TIMEOUT_SECONDS)
if events & selectors.EVENT_WRITE and pending:
try:
del pending[:os.write(master, pending)]
except BlockingIOError:
pass
selector.modify(master, selectors.EVENT_READ | (selectors.EVENT_WRITE if pending else 0), "pty")
if _exited(process):
# Drain the leader's final output before terminating any background jobs.
for _ in range(16):
try:
data = os.read(master, MAX_TERMINAL_BYTES)
except (BlockingIOError, OSError):
break
if not data:
break
send_frame(connection, {"type": "output", "data": base64.b64encode(data).decode("ascii")},
MAX_TERMINAL_FRAME, time.monotonic() + CONNECT_TIMEOUT_SECONDS)
break
finally:
try:
_cleanup_job(process, scope)
finally:
for descriptor in (master, slave):
if descriptor is not None:
os.close(descriptor)
send_frame(connection, {"type": "exit", "exit_code": process.returncode}, MAX_TERMINAL_FRAME,
time.monotonic() + CONNECT_TIMEOUT_SECONDS)
def serve_connection(connection: socket.socket, config: dict, *, runner=run_command, terminal_runner=run_terminal) -> None:
def serve_connection(connection: socket.socket, config: dict, *, runner=run_command) -> None:
with connection:
try:
deadline = time.monotonic() + CONNECT_TIMEOUT_SECONDS
send_frame(connection, {"version": PROTOCOL_VERSION, **config}, 4096, deadline)
request = receive_frame(connection, MAX_REQUEST_BYTES, deadline)
if set(request) == {"version", "terminal"}:
if type(request["version"]) is not int or request["version"] != PROTOCOL_VERSION:
raise ProtocolError("unsupported guest protocol version")
size = request["terminal"]
if not isinstance(size, dict) or set(size) != {"rows", "cols"}:
raise ProtocolError("invalid terminal request")
validate_terminal_size(size["rows"], size["cols"])
terminal_runner(connection, config["username"], size["rows"], size["cols"])
return
if set(request) != {"version", "command", "sudo", "timeout_seconds"}:
raise ProtocolError("invalid execution request fields")
if type(request["version"]) is not int or request["version"] != PROTOCOL_VERSION:
@@ -429,13 +305,6 @@ def load_config(path: Path = CONFIG_PATH) -> dict:
def main() -> None:
if len(sys.argv) == 4 and sys.argv[1] == "--terminal-child":
try:
child_main(Path(sys.argv[2]), sys.argv[3], None)
except (OSError, KeyError) as error:
print(f"uvm-agent: could not launch guest terminal: {error}", file=sys.stderr)
sys.exit(126)
return
if len(sys.argv) == 5 and sys.argv[1] == "--child":
try:
child_main(Path(sys.argv[2]), sys.argv[3], sys.argv[4])
uvm/guest_exec.py
+2 -8
View File
@@ -71,8 +71,8 @@ class GuestSession:
if connection is not None:
connection.close()
raise GuestAgentUnavailableError(
"guest agent is unavailable or not ready; wait for guest startup, or restart the VM "
"to apply automatic vsock guest-agent setup (vsock-capable kernel required)"
"guest agent is unavailable or not ready; install it with POST /vms/{vm_id}/agent/install "
"while stopped, then start the VM (vsock-capable kernel required)"
) from error
def execute(self, command: str, *, sudo: bool, timeout_seconds: float) -> ExecResult:
@@ -111,10 +111,4 @@ class GuestSession:
) from error
def close(self) -> None:
# Wake a thread waiting for a long-running tool command on server shutdown.
# The guest still owns its bounded command and package-manager cleanup.
try:
self._connection.shutdown(socket.SHUT_RDWR)
except OSError:
pass
self._connection.close()
uvm/guest_terminal.py
-89
View File
@@ -1,89 +0,0 @@
"""Interactive, process-pinned guest terminal transport over the existing vsock agent."""
from __future__ import annotations
import asyncio
import base64
import json
import struct
import time
from .errors import GuestAgentUnavailableError, GuestExecutionError, ValidationError
from .guest_agent import (
CONNECT_TIMEOUT_SECONDS, MAX_TERMINAL_FRAME, PROTOCOL_VERSION, ProtocolError,
receive_frame, send_frame, terminal_data, validate_terminal_size,
)
from .guest_exec import GuestSession
def validate_size(rows: object, cols: object) -> None:
try:
validate_terminal_size(rows, cols)
except ProtocolError as error:
raise ValidationError(str(error)) from error
class GuestTerminal(GuestSession):
def start(self, rows: int, cols: int) -> None:
validate_size(rows, cols)
try:
deadline = time.monotonic() + CONNECT_TIMEOUT_SECONDS
send_frame(self._connection, {"version": PROTOCOL_VERSION, "terminal": {"rows": rows, "cols": cols}},
MAX_TERMINAL_FRAME, deadline)
response = receive_frame(self._connection, MAX_TERMINAL_FRAME, deadline)
if response != {"type": "ready"}:
raise ProtocolError("terminal support is unavailable")
self._connection.setblocking(False)
self._write_lock = asyncio.Lock()
except (OSError, EOFError, ProtocolError) as error:
self.close()
raise GuestAgentUnavailableError(
"Interactive terminal support is unavailable. Restart this VM to apply automatic guest-agent setup, "
"then wait for the agent to start and launch a new session."
) from error
async def _receive_exact(self, size: int) -> bytes:
data = bytearray()
while len(data) < size:
chunk = await asyncio.get_running_loop().sock_recv(self._connection, size - len(data))
if not chunk:
raise EOFError("guest terminal connection closed")
data.extend(chunk)
return bytes(data)
async def read(self) -> dict:
try:
# An idle shell has no output deadline; partial messages are bounded.
first = await self._receive_exact(1)
async with asyncio.timeout(CONNECT_TIMEOUT_SECONDS):
size, = struct.unpack("!I", first + await self._receive_exact(3))
if not 0 < size <= MAX_TERMINAL_FRAME:
raise ProtocolError("invalid terminal frame length")
message = json.loads(await self._receive_exact(size))
if isinstance(message, dict):
if set(message) == {"type", "data"} and message["type"] == "output":
return {"type": "output", "data": terminal_data(message["data"])}
if (set(message) == {"type", "exit_code"} and message["type"] == "exit"
and type(message["exit_code"]) is int and -64 <= message["exit_code"] <= 255):
return message
raise ProtocolError("invalid guest terminal response")
except (OSError, EOFError, ValueError, UnicodeError, RecursionError) as error:
raise GuestExecutionError("The guest terminal connection ended unexpectedly.", outcome_unknown=True) from error
async def _send(self, message: dict) -> None:
payload = json.dumps(message, separators=(",", ":")).encode("ascii")
if len(payload) > MAX_TERMINAL_FRAME:
raise ValidationError("terminal input exceeds its size limit")
async with self._write_lock:
async with asyncio.timeout(CONNECT_TIMEOUT_SECONDS):
await asyncio.get_running_loop().sock_sendall(self._connection, struct.pack("!I", len(payload)) + payload)
async def write(self, data: bytes) -> None:
await self._send({"type": "input", "data": base64.b64encode(data).decode("ascii")})
async def resize(self, rows: int, cols: int) -> None:
validate_size(rows, cols)
await self._send({"type": "resize", "rows": rows, "cols": cols})
async def end(self) -> None:
await self._send({"type": "close"})
uvm/guest_tools.py
-400
View File
@@ -1,400 +0,0 @@
"""Standalone guest-only tool installer, submitted through the execution agent.
Importing this module on the host has no side effects. It uses only Python 3.10
stdlib; all package commands, downloads, and filesystem changes run in the guest.
"""
from __future__ import annotations
import fcntl
import hashlib
import json
import os
import pwd
import re
import shlex
import shutil
import stat
import subprocess
import sys
import tarfile
import tempfile
import time
import urllib.request
from pathlib import Path
ROOT = Path("/opt/uvm-tools")
LOCK = Path("/run/uvm-tools.lock")
BIN_DIR = Path("/usr/local/bin")
PROFILE_DIR = Path("/etc/profile.d")
PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
LOG_LIMIT = 16384
PHP_KEY = "B8DC7E53946656EFBCE4C1DD71DAEAAB4AD4CAB6"
class ToolError(Exception):
pass
def atomic_write(path, data, mode=0o644):
descriptor, name = tempfile.mkstemp(prefix=".uvm-tools-", dir=path.parent)
try:
with os.fdopen(descriptor, "wb") as stream:
os.fchmod(stream.fileno(), mode)
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
directory = os.open(path.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(directory)
finally:
os.close(directory)
finally:
Path(name).unlink(missing_ok=True)
def safe_directory(path):
for parent in reversed((path, *path.parents)):
if not parent.exists():
parent.mkdir(mode=0o755)
metadata = parent.lstat()
if not stat.S_ISDIR(metadata.st_mode) or metadata.st_uid != os.geteuid() or metadata.st_mode & 0o022:
raise ToolError(f"tool installation needs a root-owned directory: {parent}")
def regular_file(path):
metadata = path.lstat()
if not stat.S_ISREG(metadata.st_mode) or metadata.st_uid != os.geteuid() or metadata.st_nlink != 1 or metadata.st_mode & 0o022:
raise ToolError(f"refusing to replace an unmanaged tool file: {path}")
def download(url, destination, digest=None, *, max_bytes=1024 * 1024 * 1024):
if not url.startswith("https://"):
raise ToolError("tool downloads require HTTPS")
checksum = hashlib.sha256()
size = 0
with urllib.request.urlopen(url, timeout=30) as response, destination.open("wb") as output:
if not response.geturl().startswith("https://"):
raise ToolError("tool download redirected away from HTTPS")
while True:
chunk = response.read(65536)
if not chunk:
break
size += len(chunk)
if size > max_bytes:
raise ToolError("tool download exceeds its size limit")
checksum.update(chunk)
output.write(chunk)
if digest is not None and checksum.hexdigest() != digest:
destination.unlink(missing_ok=True)
raise ToolError("tool download checksum did not match the pinned release")
def extract_archive(archive_path, destination):
"""Bound extraction and check each path, including previously extracted links."""
root = destination.resolve()
with tarfile.open(archive_path) as archive:
members = archive.getmembers()
if len(members) > 50000 or sum(member.size for member in members) > 4 * 1024**3:
raise ToolError("tool archive exceeds its extraction limit")
for member in members:
name = Path(member.name)
target = destination / name
if name.is_absolute() or ".." in name.parts or not target.resolve().is_relative_to(root):
raise ToolError("unsafe path in tool archive")
if not (member.isfile() or member.isdir() or member.issym() or member.islnk()):
raise ToolError("unsupported file in tool archive")
if member.issym() or member.islnk():
link = (target.parent if member.issym() else destination) / member.linkname
if Path(member.linkname).is_absolute() or not link.resolve().is_relative_to(root):
raise ToolError("unsafe link in tool archive")
member.mode &= 0o755
member.uid = member.gid = 0
member.uname = member.gname = "root"
options = {"filter": "data"} if hasattr(tarfile, "data_filter") else {}
archive.extract(member, destination, **options)
def platform_info():
values = {}
for line in Path("/etc/os-release").read_text().splitlines():
if "=" in line:
key, value = line.split("=", 1)
values[key] = value.strip('"')
arch = os.uname().machine
return {"os": values.get("PRETTY_NAME", "Unknown OS"), "architecture": arch,
"supported": values.get("ID") == "ubuntu" and values.get("VERSION_ID") == "22.04"
and arch in ("x86_64", "aarch64"),
"boot_id": Path("/proc/sys/kernel/random/boot_id").read_text().strip()}
def guest_busy():
try:
descriptor = os.open(LOCK, os.O_RDONLY | os.O_NOFOLLOW)
except FileNotFoundError:
return False
try:
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
return False
except BlockingIOError:
return True
finally:
os.close(descriptor)
def probe(tool, username):
account = pwd.getpwnam(username)
environment = {"PATH": PATH, "HOME": account.pw_dir, "USER": username, "LOGNAME": username,
"LANG": "C.UTF-8", "LC_ALL": "C.UTF-8", "PYTHONDONTWRITEBYTECODE": "1"}
result = {"key": tool["key"], "id": tool["id"], "version": tool["version"], "installed": False,
"detected_version": None, "error": None, "usage": tool["usage"]}
for index, command in enumerate(tool["checks"]):
if shutil.which(command[0], path=PATH) is None:
return result
try:
process = subprocess.run(command, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, timeout=3, env=environment, cwd=account.pw_dir,
user=account.pw_uid, group=account.pw_gid,
extra_groups=os.getgrouplist(username, account.pw_gid))
output = process.stdout[:4096].decode("utf-8", errors="replace")
except (OSError, subprocess.TimeoutExpired) as error:
result["error"] = f"could not verify {command[0]}: {error}"[:512]
return result
if process.returncode != 0:
result["error"] = f"{command[0]} verification failed: {output}"[:512]
return result
match = re.search(tool["match"], output)
if index == 0:
if not match:
result["error"] = f"could not determine the installed {tool['id']} version"
return result
result["detected_version"] = match.group(1)
if not result["detected_version"].startswith(tool["prefix_match"]):
return result
elif tool["id"] == "jdk" and (not match or not match.group(1).startswith("21.")):
return result
result["installed"] = True
return result
class Installer:
def __init__(self, username):
self.username = username
self.log = bytearray()
self.truncated = False
self.lock = None
self.managed = {}
def append(self, data):
self.log.extend(data)
if len(self.log) > LOG_LIMIT:
del self.log[:-LOG_LIMIT]
self.truncated = True
def run(self, command):
self.append(("+ " + shlex.join(map(str, command)) + "\n").encode())
environment = {**os.environ, "PATH": PATH, "LC_ALL": "C.UTF-8", "DEBIAN_FRONTEND": "noninteractive"}
with subprocess.Popen(command, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, env=environment, pass_fds=(self.lock,)) as process:
while True:
chunk = process.stdout.read(8192)
if not chunk:
break
self.append(chunk)
if process.wait() != 0:
raise ToolError(f"{command[0]} failed with exit code {process.returncode}; see the installation output")
def remember(self, path, value):
name = str(path)
self.managed[name] = list(dict.fromkeys([*self.managed.get(name, []), value]))[-2:]
atomic_write(ROOT / "managed.json", json.dumps(self.managed).encode(), 0o600)
def write_managed(self, path, data, mode=0o644):
safe_directory(path.parent)
digest = hashlib.sha256(data).hexdigest()
if path.exists() or path.is_symlink():
regular_file(path)
current = hashlib.sha256(path.read_bytes()).hexdigest()
if current == digest:
return
if current not in self.managed.get(str(path), []):
raise ToolError(f"customized file needs attention before installing: {path}")
# Record intent first so an interrupted publication can be retried.
self.remember(path, digest)
atomic_write(path, data, mode)
def link(self, name, target):
path = BIN_DIR / name
safe_directory(path.parent)
target = str(target)
if path.exists() or path.is_symlink():
if path.is_symlink() and os.readlink(path) == target:
return
if not path.is_symlink() or os.readlink(path) not in self.managed.get(str(path), []):
raise ToolError(f"customized executable needs attention before installing: {path}")
self.remember(path, target)
temporary = path.with_name(".uvm-" + name)
temporary.unlink(missing_ok=True)
temporary.symlink_to(target)
os.replace(temporary, path)
def apt(self, packages, *, refresh=False):
stamp = ROOT / "apt-updated"
if refresh or not stamp.exists() or time.time() - stamp.stat().st_mtime > 300:
self.run(["/usr/bin/apt-get", "-o", "DPkg::Lock::Timeout=300", "-o", "Acquire::Retries=2",
"-o", "APT::Update::Error-Mode=any", "update"])
atomic_write(stamp, b"updated\n", 0o600)
pending = ROOT / "apt-pending"
if pending.exists():
# Complete configuration left by an interrupted tool installation.
self.run(["/usr/bin/dpkg", "--configure", "-a"])
atomic_write(pending, b"pending\n", 0o600)
self.run(["/usr/bin/apt-get", "-o", "DPkg::Lock::Timeout=300", "-o", "Acquire::Retries=2",
"-o", "Dpkg::Options::=--force-confold", "-y", "--no-install-recommends", "install", *packages])
pending.unlink()
def php_repository(self, temporary):
self.apt(["ca-certificates", "gnupg"])
key = temporary / "php.asc"
download(f"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x{PHP_KEY}", key, max_bytes=65536)
result = subprocess.run(["gpg", "--batch", "--homedir", str(temporary), "--with-colons", "--show-keys", str(key)],
capture_output=True, check=True)
fingerprints = [line.split(":")[9] for line in result.stdout.decode().splitlines() if line.startswith("fpr:")]
if not fingerprints or fingerprints[0] != PHP_KEY:
raise ToolError("PHP repository signing key did not match its pinned fingerprint")
binary_key = temporary / "php.gpg"
self.run(["gpg", "--batch", "--homedir", str(temporary), "--dearmor", "--output", str(binary_key), str(key)])
self.write_managed(Path("/etc/apt/keyrings/uvm-php.gpg"), binary_key.read_bytes())
self.write_managed(Path("/etc/apt/sources.list.d/uvm-php.list"),
b"deb [signed-by=/etc/apt/keyrings/uvm-php.gpg] https://ppa.launchpadcontent.net/ondrej/php/ubuntu jammy main\n")
def payload(self, tool, temporary):
prefix = ROOT / tool["release"]
marker = prefix / ".uvm-tools.json"
identity = {"release": tool["release"], "sha256": tool["sha256"]}
if prefix.exists() or prefix.is_symlink():
safe_directory(prefix)
if not marker.exists():
raise ToolError(f"installation directory is already in use: {prefix}")
regular_file(marker)
previous = json.loads(marker.read_bytes())
if not isinstance(previous, dict) or any(previous.get(key) != value for key, value in identity.items()):
raise ToolError(f"installation directory belongs to a different release: {prefix}")
if tool["kind"] != "conda" or previous.get("complete"):
return prefix
artifact = temporary / "download"
self.append(f"Downloading {tool['release']} and verifying SHA-256…\n".encode())
download(tool["url"], artifact, tool["sha256"])
if tool["kind"] == "archive":
extraction = temporary / "extract"
extraction.mkdir()
extract_archive(artifact, extraction)
source = extraction / tool["archive_root"]
if not source.is_dir() or source.is_symlink():
raise ToolError("tool archive has an unexpected layout")
atomic_write(source / ".uvm-tools.json", json.dumps(identity).encode(), 0o600)
os.replace(source, prefix)
else:
prefix.mkdir(exist_ok=True, mode=0o755)
atomic_write(marker, json.dumps(identity).encode(), 0o600)
if tool["kind"] == "conda":
self.run(["/bin/bash", str(artifact), "-b", "-f", "-p", str(prefix)])
else:
artifact.chmod(0o755)
os.replace(artifact, prefix / "micromamba")
return prefix
def configure(self, tool, prefix):
for name, target in tool["links"].items():
self.link(name, Path(target) if target.startswith("/") else prefix / target)
if tool["id"] == "python":
executable = prefix / "bin" / f"python{tool['version']}"
self.write_managed(BIN_DIR / f"pip{tool['version']}",
f'#!/bin/sh\nexec {shlex.quote(str(executable))} -m pip "$@"\n'.encode(), 0o755)
elif tool["id"] == "jdk":
self.write_managed(PROFILE_DIR / "uvm-java.sh",
f"export JAVA_HOME={shlex.quote(tool['java_home'])}\n".encode())
elif tool["id"] == "conda":
for key, value in (("auto_activate_base", "false"),):
self.run([str(prefix / "bin/conda"), "config", "--system", "--set", key, value])
for key, value in (("envs_dirs", "~/.conda/envs"), ("pkgs_dirs", "~/.conda/pkgs")):
self.run([str(prefix / "bin/conda"), "config", "--system", "--prepend", key, value])
self.write_managed(PROFILE_DIR / "uvm-conda.sh",
f". {shlex.quote(str(prefix / 'etc/profile.d/conda.sh'))}\n".encode())
elif tool["id"] == "micromamba":
self.write_managed(PROFILE_DIR / "uvm-micromamba.sh",
b'export MAMBA_ROOT_PREFIX="${MAMBA_ROOT_PREFIX:-$HOME/micromamba}"\n'
b'eval "$(/usr/local/bin/micromamba shell hook --shell posix)"\n')
def install(self, tool):
safe_directory(LOCK.parent)
self.lock = os.open(LOCK, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600)
try:
metadata = os.fstat(self.lock)
if not stat.S_ISREG(metadata.st_mode) or metadata.st_nlink != 1 or metadata.st_uid != os.geteuid():
raise ToolError("unsafe guest tool installation lock")
try:
fcntl.flock(self.lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise ToolError("another installation is still running inside this guest; wait before retrying")
existing = probe(tool, self.username)
if existing["installed"]:
return {"status": "already_installed", "tool": existing}
safe_directory(ROOT)
manifest = ROOT / "managed.json"
if manifest.exists():
regular_file(manifest)
self.managed = json.loads(manifest.read_bytes())
if (not isinstance(self.managed, dict) or any(
not isinstance(values, list) or len(values) > 2 or any(not isinstance(value, str) for value in values)
for values in self.managed.values())):
raise ToolError("invalid managed tool file manifest")
with tempfile.TemporaryDirectory(prefix=".install-", dir=ROOT) as name:
temporary = Path(name)
if tool["kind"] == "php":
self.php_repository(temporary)
self.apt(tool["packages"], refresh=tool["kind"] == "php")
prefix = self.payload(tool, temporary) if "url" in tool else None
self.configure(tool, prefix)
installed = probe(tool, self.username)
if not installed["installed"]:
raise ToolError(installed["error"] or f"{tool['key']} did not pass its version check after installation")
if prefix is not None:
atomic_write(prefix / ".uvm-tools.json", json.dumps({"release": tool["release"],
"sha256": tool["sha256"], "complete": True}).encode(), 0o600)
return {"status": "installed", "tool": installed}
finally:
os.close(self.lock)
self.lock = None
def main(payload):
installer = Installer(payload["username"])
info = {}
try:
if os.geteuid() != 0:
raise ToolError("tool installation and inspection require guest root")
info = platform_info()
info["supported"] = info["supported"] and info["architecture"] == payload["architecture"]
if not info["supported"]:
raise ToolError("one-click tools require an Ubuntu 22.04 guest on the selected x86_64/ARM64 architecture")
if payload.get("boot_id") and payload["boot_id"] != info["boot_id"]:
raise ToolError("the guest rebooted during installation; retry on the current boot")
if payload["action"] == "probe":
result = {**info, "busy": guest_busy(), "tools": [probe(tool, payload["username"]) for tool in payload["tools"]]}
elif payload["action"] == "install":
result = installer.install(payload["tools"][0])
else:
raise ToolError("unsupported tool operation")
result["ok"] = True
except (ToolError, OSError, ValueError, KeyError, tarfile.TarError, subprocess.SubprocessError) as error:
result = {"ok": False, "error": str(error)[:2048], "supported": info.get("supported")}
result.update(output=installer.log.decode("utf-8", errors="replace"), truncated=installer.truncated)
print(json.dumps(result, ensure_ascii=True))
if __name__ == "__main__":
main(json.loads(sys.argv[1]))
uvm/images.py
+10 -49
View File
@@ -89,11 +89,6 @@ check_python
"""
# The original v4 template shipped this exact agent without an ownership
# manifest. Recognize that payload when creating from a cached v4 template.
_V4_TEMPLATE_AGENT_SHA256 = "dfdd4bd6ee3dc06fbdbda6402cb2c54921cff22190d8da889209559800f8b948"
def guest_agent_files() -> dict[str, bytes]:
"""The same standalone payload is used by the builder and offline upgrades."""
return {
@@ -288,7 +283,7 @@ class ImageStore:
)
if marker is None or marker.path.read_text(encoding="utf-8") not in {
"uvm-ubuntu-jammy-v1\n", "uvm-ubuntu-jammy-v2\n", "uvm-ubuntu-jammy-v3\n",
"uvm-ubuntu-jammy-v4\n", "uvm-ubuntu-jammy-v5\n",
"uvm-ubuntu-jammy-v4\n",
}:
return None
profile = self._read_guest_file(
@@ -304,12 +299,10 @@ class ImageStore:
return profile
def provision_guest_agent(self, disk: Path, vm_id: str, username: str, *, install: bool = False) -> bool:
"""Ensure every guest has its vsock agent before booting its private disk.
"""Configure new templates, or explicitly install on a proven-offline disk.
The lifecycle lock and confirmed VMM exit are required. A checked sparse
copy preserves the original disk on failure and makes retries atomic.
Return False only when the managed files and service are already current.
An explicit install request always stages and rechecks the installation.
"""
parse_username(username)
if re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,80}", vm_id) is None:
@@ -321,7 +314,10 @@ class ImageStore:
raise UvmError("private VM disk is unsafe for guest agent installation")
with tempfile.TemporaryDirectory(prefix="uvm-guest-agent-") as name:
files = Path(name)
if not install and self._guest_agent_current(disk, files, vm_id, username):
if not install:
marker = self._read_guest_file(disk, "/etc/uvm-image", files / "marker",
required=False, regular_only=True)
if marker is None or marker.path.read_text(encoding="utf-8") != "uvm-ubuntu-jammy-v4\n":
return False
descriptor, candidate = tempfile.mkstemp(prefix=".rootfs-agent-", suffix=".ext4", dir=disk.parent)
os.close(descriptor)
@@ -349,48 +345,16 @@ class ImageStore:
except (OSError, UnicodeError) as error:
raise UvmError(f"could not provision guest agent: {error}") from error
@staticmethod
def _agent_payloads(vm_id: str, username: str) -> dict[str, bytes]:
payloads = guest_agent_files()
payloads["/etc/uvm-agent.json"] = (json.dumps({"vm_id": vm_id, "username": username}) + "\n").encode("utf-8")
return payloads
def _guest_agent_current(self, disk: Path, files: Path, vm_id: str, username: str) -> bool:
"""Read-only fast path; changed installations are checked again after journal replay."""
payloads = self._agent_payloads(vm_id, username)
for index, (path, content) in enumerate(payloads.items()):
existing = self._agent_file(disk, path, files / f"current-{index}")
mode = 0o600 if path.endswith(".json") else 0o644
if (existing is None or (existing.mode, existing.gid) != (mode, 0)
or existing.path.read_bytes() != content):
return False
manifest = self._agent_file(disk, "/etc/uvm-agent-files.json", files / "current-manifest")
if manifest is None or (manifest.mode, manifest.gid) != (0o600, 0):
return False
try:
hashes = json.loads(manifest.path.read_bytes())
except (ValueError, UnicodeError):
return False
if hashes != {path: hashlib.sha256(content).hexdigest() for path, content in payloads.items()}:
return False
service = self._runner.run((
"debugfs", "-R", "stat /etc/systemd/system/multi-user.target.wants/uvm-agent.service", disk,
), capture=True)
return (re.search(r"Type:\s+symlink\b", service.stdout) is not None
and 'Fast link dest: "/etc/systemd/system/uvm-agent.service"' in service.stdout)
def _install_agent_files(self, disk: Path, files: Path, vm_id: str, username: str) -> None:
for binary in ("/usr/lib/systemd/systemd", "/usr/bin/apt-get"):
result = self._runner.run(("debugfs", "-R", f"stat {binary}", disk), capture=True)
if re.search(r"Type:\s+regular\b", result.stdout) is None:
raise UvmError(
"automatic vsock guest-agent setup requires a systemd/apt image (Ubuntu 22.04 or newer); "
"use the default Ubuntu rootfs or a compatible custom image"
)
raise UvmError("guest agent installation requires a systemd/apt image (Ubuntu 22.04 or newer)")
for directory in ("/etc", "/usr", "/usr/lib", "/usr/lib/uvm", "/etc/systemd",
"/etc/systemd/system", "/etc/systemd/system/multi-user.target.wants"):
self._agent_directory(disk, directory)
payloads = self._agent_payloads(vm_id, username)
payloads = guest_agent_files()
payloads["/etc/uvm-agent.json"] = (json.dumps({"vm_id": vm_id, "username": username}) + "\n").encode("utf-8")
manifest_path = "/etc/uvm-agent-files.json"
previous = self._agent_file(disk, manifest_path, files / "manifest")
old_hashes = {}
@@ -408,10 +372,7 @@ class ImageStore:
existing = self._agent_file(disk, path, files / f"existing-{index}")
if existing is not None:
digest = hashlib.sha256(existing.path.read_bytes()).hexdigest()
allowed = {hashes[path], old_hashes.get(path)}
if not old_hashes and path == "/usr/lib/uvm/agent.py":
allowed.add(_V4_TEMPLATE_AGENT_SHA256)
if digest not in allowed:
if digest not in {hashes[path], old_hashes.get(path)}:
raise UvmError(f"refusing to overwrite a customized guest agent file: {path}")
source = files / f"payload-{index}"
source.write_bytes(content)
uvm/install.py
+2 -9
View File
@@ -2,8 +2,6 @@
from __future__ import annotations
import hashlib
import json
import os
import shlex
import shutil
@@ -23,7 +21,7 @@ from .system import CommandRunner, check_kvm, ensure_data_directories, require_r
_UBUNTU_RELEASE = "jammy"
_ROOTFS_RECIPE = "uvm-ubuntu-jammy-v5\n"
_ROOTFS_RECIPE = "uvm-ubuntu-jammy-v4\n"
_GUEST_PACKAGES = (
"ca-certificates", "openssh-server", "sudo", "systemd-sysv", "udev",
"dbus", "libpam-systemd", "iproute2", "iputils-ping", "curl", "python3",
@@ -399,17 +397,12 @@ class Installer:
machine_id.symlink_to("../../../etc/machine-id")
for key in (root / "etc/ssh").glob("ssh_host_*_key*"):
key.unlink()
agent_payloads = guest_agent_files()
for guest_path, contents in agent_payloads.items():
for guest_path, contents in guest_agent_files().items():
path = root / guest_path.lstrip("/")
path.parent.mkdir(parents=True, exist_ok=True)
path.parent.chmod(0o755)
path.write_bytes(contents)
path.chmod(0o644)
manifest = root / "etc/uvm-agent-files.json"
manifest.write_text(json.dumps({name: hashlib.sha256(data).hexdigest() for name, data in agent_payloads.items()},
sort_keys=True) + "\n", encoding="utf-8")
manifest.chmod(0o600)
self._runner.run(("systemctl", "--root", root, "enable", "ssh.service", "systemd-networkd.service",
"uvm-agent.service"), capture=True)
self._runner.run(("systemctl", "--root", root, "set-default", "multi-user.target"), capture=True)
uvm/lifecycle.py
-68
View File
@@ -19,15 +19,12 @@ from .firecracker.config import build_config, write_config
from .firecracker.process import FirecrackerProcessManager, ProcessInfo
from .guest_agent import DEFAULT_TIMEOUT_SECONDS
from .guest_exec import ExecResult, GuestSession, validate_exec
from .guest_terminal import GuestTerminal, validate_size
from .images import ImageStore
from .integrity import load_manifest, verify_file
from .known_hosts import KnownHostsManager
from .network import NetworkManager
from .state import StateStore
from .system import check_kvm, ensure_data_directories, require_root
from .tool_catalog import normalize_tools
from .tool_store import ToolJobStore
from .validation import parse_name, parse_password, parse_ports, parse_timeout, parse_username
@@ -64,7 +61,6 @@ class LifecycleService:
known_hosts: KnownHostsManager,
client_factory: Callable[[Path, float], FirecrackerClient] = FirecrackerClient,
guest_connector: Callable[[Path, str, str, int], GuestSession] = GuestSession.connect,
terminal_connector: Callable[[Path, str, str, int], GuestTerminal] = GuestTerminal.connect,
clock: Callable[[], float] = time.time,
) -> None:
self._settings = settings
@@ -75,9 +71,7 @@ class LifecycleService:
self._known_hosts = known_hosts
self._client_factory = client_factory
self._guest_connector = guest_connector
self._terminal_connector = terminal_connector
self._clock = clock
self._tool_store = ToolJobStore(settings, state_store, clock=clock)
def create(self, spec: VmSpec) -> VmRecord:
require_root()
@@ -87,7 +81,6 @@ class LifecycleService:
if spec.name is not None:
parse_name(spec.name)
ports = self._validated_ports(spec.ports)
tools = normalize_tools(spec.tools, allow_empty=True)
ensure_data_directories(self._settings)
with self._state_store.operation_lock():
self._check_boot()
@@ -104,8 +97,6 @@ class LifecycleService:
self._images.provision_guest_agent(disk, vm.id, vm.username)
vm.disk_ready = True
self._save(vm)
if tools:
self._tool_store.enqueue_locked(vm.id, tools, source="create")
self._boot(vm, assets.kernel)
return vm
except BaseException as error:
@@ -204,7 +195,6 @@ class LifecycleService:
self._cleanup_runtime(vm)
self._apply_pending_resize(vm)
self._images.configure_guest_network(Path(vm.disk), vm.guest_ip)
self._images.provision_guest_agent(Path(vm.disk), vm.id, vm.username)
vm.network, vm.bridge = str(self._settings.network), self._settings.bridge
self._boot(vm, kernel)
return vm
@@ -512,8 +502,6 @@ class LifecycleService:
def execute(
self, vm_id: str, command: str, *, sudo: bool = False,
timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS,
expected_identity: tuple | None = None,
on_connect: Callable[[GuestSession], None] | None = None,
) -> ExecResult:
"""Pin a running VM's guest connection under the lock, then wait unlocked."""
require_root()
@@ -524,70 +512,14 @@ class LifecycleService:
if (vm.status != "running" or vm.operation is not None or vm.pending_resize is not None
or self._process.observe(vm) != "alive"):
raise ConflictError("guest execution requires a running VM with no pending lifecycle operation")
if expected_identity is not None and expected_identity != (vm.pid, vm.process_start_time, vm.boot_id):
raise ConflictError("the VM restarted during tool installation; retry on the current boot")
session = self._guest_connector(self._settings.guest_socket_path(vm.id), vm.id, vm.username, vm.pid)
try:
if on_connect is not None:
on_connect(session)
# This session stays attached to that VMM even if the VM is stopped,
# deleted, or restarted concurrently. Never reconnect/retry a command.
return session.execute(command, sudo=sudo, timeout_seconds=timeout_seconds)
finally:
session.close()
def _validate_tool_target(self, vm: VmRecord, *, start_if_stopped: bool, restart: bool) -> str:
self._not_deleting(vm)
observation = self._process.observe(vm)
if vm.operation is not None or vm.pending_resize is not None or not vm.disk_ready:
raise ConflictError("finish pending lifecycle operations before installing tools")
if vm.status == "running" and observation == "alive":
return "running"
if vm.status == "stopped" and observation == "gone" and (start_if_stopped or restart):
self._validate_resume(vm)
return "stopped"
raise ConflictError("tool installation requires a running VM; use Start & install for a stopped VM")
def queue_tools(self, vm_id: str, selections, *, start_if_stopped=False, restart=False, request_id=None) -> dict:
require_root()
selections = normalize_tools(selections)
with self._state_store.operation_lock():
vm = self._load_for_operation(vm_id, exact_id=True)
self._validate_tool_target(vm, start_if_stopped=start_if_stopped, restart=restart)
return self._tool_store.enqueue_locked(vm.id, selections, start_if_stopped=start_if_stopped,
restart=restart, request_id=request_id)
def prepare_tool_install(self, vm_id: str, *, start_if_stopped=False, restart=False) -> VmRecord:
"""Perform requested startup/agent refresh atomically before a tool worker connects."""
require_root()
with self._state_store.operation_lock():
vm = self._load_for_operation(vm_id, exact_id=True)
status = self._validate_tool_target(vm, start_if_stopped=start_if_stopped, restart=restart)
if status == "running" and restart:
self._stop_locked(vm)
if status == "stopped" or restart:
# The normal retained-disk boot also provisions the current agent.
vm = self._start_locked(vm)
return vm
def open_terminal(self, vm_id: str, *, rows: int = 24, cols: int = 80) -> tuple[GuestTerminal, str]:
"""Connect to the exact running VMM under the lock, then open its guest PTY."""
require_root()
validate_size(rows, cols)
with self._state_store.operation_lock():
vm = self._load_for_operation(vm_id, exact_id=True)
self._not_deleting(vm)
if (vm.status != "running" or vm.operation is not None or vm.pending_resize is not None
or self._process.observe(vm) != "alive"):
raise ConflictError("terminal sessions require a running VM with no pending lifecycle operation")
terminal = self._terminal_connector(self._settings.guest_socket_path(vm.id), vm.id, vm.username, vm.pid)
try:
terminal.start(rows, cols)
return terminal, vm.username
except BaseException:
terminal.close()
raise
def install_guest_agent(self, vm_id: str) -> VmRecord:
"""Explicitly install/upgrade the agent on a cleanly stopped private disk."""
require_root()
uvm/routers/tools.py
-46
View File
@@ -1,46 +0,0 @@
"""Authenticated tool catalog, installation jobs, and guest inventory."""
from __future__ import annotations
from fastapi import APIRouter, Depends, status
from ..api_models import ToolCatalogResponse, ToolInstallRequest, ToolJobResponse, VmToolsResponse
from ..app import Application
from ..errors import UvmError
from ..tool_catalog import CATALOG, normalize_tools
from .dependencies import get_authorized_application
router = APIRouter(tags=["tools"])
def tool_service(application):
service = getattr(application, "tools", None)
if service is None:
raise UvmError("guest tool installation is unavailable in this application")
return service
@router.get("/tools", response_model=list[ToolCatalogResponse])
def catalog(application: Application = Depends(get_authorized_application)):
return list(CATALOG)
@router.get("/vms/{vm_id}/tools", response_model=VmToolsResponse)
def vm_tools(vm_id: str, refresh: bool = False, application: Application = Depends(get_authorized_application)):
"""Return cached observations; refresh=true probes the running guest without installing."""
return tool_service(application).snapshot(vm_id, refresh=refresh)
@router.post("/vms/{vm_id}/tools/install", response_model=ToolJobResponse, status_code=status.HTTP_202_ACCEPTED)
def install_tools(vm_id: str, request: ToolInstallRequest,
application: Application = Depends(get_authorized_application)):
return tool_service(application).submit(
vm_id, normalize_tools([item.model_dump() for item in request.tools]),
start_if_stopped=request.start_if_stopped, restart=request.restart, request_id=request.request_id,
)
@router.get("/vms/{vm_id}/tools/jobs/{job_id}", response_model=ToolJobResponse)
def tool_job(vm_id: str, job_id: str, application: Application = Depends(get_authorized_application)):
return tool_service(application).store.get_job(vm_id, job_id)
uvm/routers/vms.py
-8
View File
@@ -22,7 +22,6 @@ from ..api_models import (
from ..app import Application
from ..domain import PortRule, VmRecord, VmSpec
from ..errors import ValidationError
from ..tool_catalog import normalize_tools
from ..validation import (
parse_cpu,
parse_name,
@@ -34,7 +33,6 @@ from ..validation import (
parse_username,
)
from .dependencies import get_authorized_application
from .tools import tool_service
router = APIRouter(prefix="/vms", tags=["vms"])
@@ -75,9 +73,6 @@ def create_vm(
ports = parse_ports([f"{rule.port}:{rule.protocol}" for rule in request.ports or ()])
if not request.no_default_ports and not any(rule.port == 22 for rule in ports):
ports = tuple(sorted((*ports, PortRule(22)), key=lambda rule: rule.port))
tools = normalize_tools([item.model_dump() for item in request.tools], allow_empty=True)
if tools:
tool_service(application).start()
vm = application.lifecycle.create(
VmSpec(
cpu=parse_cpu(str(request.cpu)),
@@ -88,11 +83,8 @@ def create_vm(
password=parse_password(request.password.get_secret_value()),
name=parse_name(request.name) if request.name is not None else None,
ports=ports,
tools=tools,
)
)
if tools:
tool_service(application).notify()
return vm_response(vm, policy_status=_mutation_policy_status(vm))
uvm/server.py
-16
View File
@@ -3,7 +3,6 @@
from __future__ import annotations
from collections.abc import Callable
from contextlib import asynccontextmanager
from ipaddress import ip_address
from pathlib import Path
import ssl
@@ -33,7 +32,6 @@ def create_api(application: Application | None = None, *, host: str | None = Non
try:
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from starlette.concurrency import run_in_threadpool
except ModuleNotFoundError as error:
raise UvmError(
"FastAPI server support is not installed. Install this project with its dependencies."
@@ -44,24 +42,11 @@ def create_api(application: Application | None = None, *, host: str | None = Non
from .routers.installation import router as installation_router
from .routers.vms import router as vms_router
from .routers.diagnostics import router as diagnostics_router
from .routers.tools import router as tools_router
@asynccontextmanager
async def lifespan(_api):
tools = getattr(resolved_application, "tools", None)
if tools is not None:
await run_in_threadpool(tools.start)
try:
yield
finally:
if tools is not None:
await run_in_threadpool(tools.shutdown)
api = FastAPI(
title="uvm",
version="0.1.0",
description="Local Firecracker microVM management API.",
lifespan=lifespan,
)
api.state.uvm_application = resolved_application
@@ -80,7 +65,6 @@ def create_api(application: Application | None = None, *, host: str | None = Non
api.include_router(installation_router)
api.include_router(vms_router)
api.include_router(diagnostics_router)
api.include_router(tools_router)
return api
uvm/tool_catalog.py
-162
View File
@@ -1,162 +0,0 @@
"""The allowlisted guest tool catalog and pinned, architecture-specific recipes.
Download digests come from the publishers' release SHA256SUMS/.sha256 files:
Node v24.21.0, python-build-standalone 20260924, Miniforge 26.5.3-0, and
micromamba-releases 2.9.0-0. Update versions, URLs, and digests together.
"""
from __future__ import annotations
from dataclasses import dataclass
from .errors import ValidationError
CATALOG = (
{"id": "git", "name": "Git", "description": "Version control for your projects.",
"version_label": "Ubuntu package", "versions": [], "source": "Ubuntu"},
{"id": "nodejs", "name": "Node.js", "description": "JavaScript runtime with npm and npx.",
"version_label": "24 LTS · 24.21.0", "versions": [], "source": "nodejs.org"},
{"id": "python", "name": "Python", "description": "Side-by-side Python runtimes with pip and venv.",
"version_label": "3.11 / 3.12 / 3.13", "versions": ["3.11", "3.12", "3.13"],
"source": "Astral python-build-standalone"},
{"id": "jdk", "name": "JDK", "description": "OpenJDK development kit: java and javac.",
"version_label": "OpenJDK 21 LTS", "versions": [], "source": "Ubuntu"},
{"id": "wget", "name": "wget", "description": "Download files from the command line.",
"version_label": "Ubuntu package", "versions": [], "source": "Ubuntu"},
{"id": "curl", "name": "curl", "description": "Transfer data and call HTTP APIs.",
"version_label": "Ubuntu package", "versions": [], "source": "Ubuntu"},
{"id": "htop", "name": "htop", "description": "Interactive process and resource monitor.",
"version_label": "Ubuntu package", "versions": [], "source": "Ubuntu"},
{"id": "php", "name": "PHP", "description": "PHP command-line runtime.",
"version_label": "8.5 CLI", "versions": [], "source": "Ondřej Surý PHP PPA"},
{"id": "conda", "name": "Conda", "description": "Miniforge with the conda-forge package channel.",
"version_label": "Miniforge 26.5.3-0", "versions": [], "source": "conda-forge"},
{"id": "micromamba", "name": "micromamba", "description": "Lightweight Conda-compatible environment manager.",
"version_label": "2.9.0", "versions": [], "source": "mamba-org"},
)
TOOL_IDS = frozenset(tool["id"] for tool in CATALOG)
MAX_SELECTIONS = 12
@dataclass(frozen=True, slots=True)
class ToolSelection:
id: str
version: str | None = None
def __post_init__(self) -> None:
if not isinstance(self.id, str) or self.id not in TOOL_IDS:
raise ValidationError("unknown tool; choose a tool from GET /tools")
if self.id == "python":
if self.version not in ("3.11", "3.12", "3.13"):
raise ValidationError("Python requires version 3.11, 3.12, or 3.13")
elif self.version is not None:
raise ValidationError("only Python accepts a version; other tools use the catalog default")
@property
def key(self) -> str:
return f"{self.id}-{self.version}" if self.version else self.id
def to_dict(self) -> dict:
return {"id": self.id, "version": self.version}
def normalize_tools(values: object, *, allow_empty: bool = False) -> tuple[ToolSelection, ...]:
if not isinstance(values, (list, tuple)) or len(values) > MAX_SELECTIONS:
raise ValidationError(f"tools must be a list of at most {MAX_SELECTIONS} catalog selections")
result = {}
for value in values:
if isinstance(value, ToolSelection):
selection = ToolSelection(value.id, value.version)
elif isinstance(value, dict) and "id" in value and not set(value) - {"id", "version"}:
selection = ToolSelection(value["id"], value.get("version"))
else:
raise ValidationError("each tool selection requires an id and optional Python version")
result[selection.key] = selection
if not result and not allow_empty:
raise ValidationError("select at least one tool to install")
# Stable ordering also makes duplicate requests independent of checkbox order.
order = {tool["id"]: index for index, tool in enumerate(CATALOG)}
return tuple(sorted(result.values(), key=lambda item: (order[item.id], item.version or "")))
def all_selections() -> tuple[ToolSelection, ...]:
return tuple(ToolSelection(tool["id"], version) for tool in CATALOG for version in tool["versions"] or [None])
_PYTHON_RELEASE = "20260924"
_PYTHON = {
"3.11": ("3.11.16", {
"x86_64": "68c6739376b65258dee5058ccf6777232fe38d31a578965ae8bda327ec7da3a8",
"aarch64": "763dae20c6abe982de86ceb7829cc050c5bda79a9824846d106cc1c86d57701b",
}),
"3.12": ("3.12.14", {
"x86_64": "269b2c99e4db15b242bf01832f4fea1e8f1a664f273cff519393f296e9820b41",
"aarch64": "c8499b61252c433280f134df954464d19811527b31cb920c35fc6967c1222e35",
}),
"3.13": ("3.13.15", {
"x86_64": "d0b640eed27fbdd6f5f2bd33444aee53df2c8863f8b2a96f4094717411e3de9c",
"aarch64": "5ad58156cbec94e5643c13caa792e92df72a23f33d3a6425d4cbff5c4b7a040c",
}),
}
_NODE_SHA256 = {
"x86_64": "fd8e59d5a511510f6a298afb548f18c7d2b1be404d8b4a27d94fbe49f56cb2d6",
"aarch64": "6ad1325edbdb5649c379b75a237147a666c95d4f9ae8d340fef2d1575d289ad2",
}
_CONDA_SHA256 = {
"x86_64": "14db468222ad564658656f769506056209b6dc375f5e7dfd31eb5ebbf08fa529",
"aarch64": "0391e42075a7632e9665d6e728387ee6b905f6c3e704d3513e1c1133d0d69b89",
}
_MICROMAMBA_SHA256 = {
"x86_64": "366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3",
"aarch64": "9f93b974adcb4d166996af969b6cd371287d1a3e52733704727884d9b74cb7a7",
}
def recipe(selection: ToolSelection, architecture: str) -> dict:
"""Return a JSON-serializable recipe; no request text becomes a shell program."""
if architecture not in {"x86_64", "aarch64"}:
raise ValidationError("tool installation supports x86_64 and ARM64 guests")
tool = selection.id
result = {**selection.to_dict(), "key": selection.key, "kind": "apt", "packages": [tool],
"checks": [[tool, "--version"]], "match": r"(\d+(?:\.\d+)+)", "prefix_match": "",
"links": {}, "usage": f"{tool} --version"}
if tool == "nodejs":
arch = "x64" if architecture == "x86_64" else "arm64"
result.update(kind="archive", release="nodejs-24.21.0", packages=["ca-certificates", "libstdc++6"],
url=f"https://nodejs.org/dist/v24.21.0/node-v24.21.0-linux-{arch}.tar.xz",
sha256=_NODE_SHA256[architecture], archive_root=f"node-v24.21.0-linux-{arch}",
checks=[["node", "--version"], ["npm", "--version"], ["npx", "--version"]], prefix_match="24.",
links={name: f"bin/{name}" for name in ("node", "npm", "npx")}, usage="node --version; npm --version")
elif tool == "python":
version, hashes = _PYTHON[selection.version]
binary = f"python{selection.version}"
result.update(kind="archive", release=f"python-{version}", packages=["ca-certificates", "libcrypt1"],
url=(f"https://github.com/astral-sh/python-build-standalone/releases/download/{_PYTHON_RELEASE}/"
f"cpython-{version}+{_PYTHON_RELEASE}-{architecture}-unknown-linux-gnu-install_only_stripped.tar.gz"),
sha256=hashes[architecture], archive_root="python", prefix_match=selection.version + ".",
checks=[[binary, "-c", "import sys, ssl, sqlite3, venv, ensurepip; print(sys.version.split()[0])"],
[binary, "-m", "pip", "--version"]], links={binary: f"bin/{binary}"},
usage=f"{binary} -m venv .venv; . .venv/bin/activate")
elif tool == "jdk":
arch = "amd64" if architecture == "x86_64" else "arm64"
home = f"/usr/lib/jvm/java-21-openjdk-{arch}"
result.update(packages=["openjdk-21-jdk-headless"], checks=[["javac", "-version"], ["java", "-version"]],
prefix_match="21.", java_home=home, links={name: f"{home}/bin/{name}" for name in ("java", "javac")},
usage="java -version; javac -version")
elif tool == "php":
result.update(kind="php", packages=["php8.5-cli"], prefix_match="8.5.",
links={"php": "/usr/bin/php8.5"}, usage="php --version")
elif tool == "conda":
result.update(kind="conda", release="miniforge-26.5.3-0", packages=["ca-certificates", "bzip2"],
url=("https://github.com/conda-forge/miniforge/releases/download/26.5.3-0/"
f"Miniforge3-26.5.3-0-Linux-{architecture}.sh"), sha256=_CONDA_SHA256[architecture],
links={"conda": "bin/conda", "mamba": "bin/mamba"},
usage='eval "$(conda shell.posix hook)"; conda create -n myenv python=3.13; conda activate myenv')
elif tool == "micromamba":
arch = "64" if architecture == "x86_64" else "aarch64"
result.update(kind="binary", release="micromamba-2.9.0", packages=["ca-certificates"],
url=f"https://github.com/mamba-org/micromamba-releases/releases/download/2.9.0-0/micromamba-linux-{arch}",
sha256=_MICROMAMBA_SHA256[architecture], links={"micromamba": "micromamba"},
usage='eval "$(micromamba shell hook --shell posix)"; micromamba create -n myenv -c conda-forge python=3.13; micromamba activate myenv')
return result
uvm/tool_store.py
-197
View File
@@ -1,197 +0,0 @@
"""Atomic per-VM tool job records, serialized with lifecycle filesystem changes."""
from __future__ import annotations
import copy
import json
import os
import re
import stat
import tempfile
import time
import uuid
from contextlib import contextmanager
from .config import Settings
from .errors import ConflictError, NotFoundError, StateError
from .state import StateStore
from .tool_catalog import normalize_tools
ACTIVE_JOB_STATUSES = frozenset({"queued", "starting", "waiting", "installing"})
MAX_RETAINED_JOBS = 10
MAX_DOCUMENT_BYTES = 16 * 1024 * 1024
class ToolJobStore:
"""Only short transactions hold operations.lock; guest execution never does."""
def __init__(self, settings: Settings, state_store: StateStore, *, clock=time.time) -> None:
self.settings = settings
self.state = state_store
self.clock = clock
def _path(self, vm_id):
vm = self.state.get(vm_id) # Exact ID only, including while creating a VM.
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,80}", vm.id):
raise StateError("invalid VM ID in tool history")
directory = self.settings.vm_dir(vm.id)
try:
metadata = directory.lstat()
except FileNotFoundError as error:
raise NotFoundError("VM runtime directory is no longer available") from error
if (not stat.S_ISDIR(metadata.st_mode) or metadata.st_uid != os.geteuid()
or metadata.st_mode & 0o077 or not vm.runtime_owned):
raise StateError("tool history requires a private, owned VM runtime directory")
return directory / "tools.json"
@staticmethod
def _empty():
return {"schema_version": 1, "jobs": [], "inventory": [], "checked_at": None,
"agent_ready": False, "guest_busy": False, "supported": None, "error": None}
def _read(self, vm_id):
path = self._path(vm_id)
try:
descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW)
except FileNotFoundError:
return self._empty()
except OSError as error:
raise StateError("could not open private tool installation history") from error
try:
with os.fdopen(descriptor, "rb") as stream:
metadata = os.fstat(stream.fileno())
if (not stat.S_ISREG(metadata.st_mode) or metadata.st_nlink != 1
or metadata.st_uid != os.geteuid() or metadata.st_mode & 0o077):
raise StateError("unsafe tool history file")
raw = stream.read(MAX_DOCUMENT_BYTES + 1)
if len(raw) > MAX_DOCUMENT_BYTES:
raise StateError("tool history exceeds its size limit")
data = json.loads(raw)
if (not isinstance(data, dict) or data.get("schema_version") != 1
or not isinstance(data.get("jobs"), list) or not isinstance(data.get("inventory"), list)
or len(data["jobs"]) > MAX_RETAINED_JOBS):
raise StateError("invalid tool history document")
for job in data["jobs"]:
if (not isinstance(job, dict) or job.get("vm_id") != vm_id
or not re.fullmatch(r"tools-[0-9a-f]{32}", str(job.get("id", "")))
or not isinstance(job.get("items"), list) or len(job["items"]) > 12):
raise StateError("invalid tool installation job")
return data
except (OSError, ValueError) as error:
raise StateError("could not read tool installation history") from error
def _write(self, vm_id, data):
path = self._path(vm_id) # Never recreate a deleted VM directory.
descriptor, name = tempfile.mkstemp(prefix=".tools-", suffix=".json", dir=path.parent)
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
os.fchmod(stream.fileno(), 0o600)
json.dump(data, stream, ensure_ascii=True, allow_nan=False)
stream.write("\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
self.state._fsync_parent(path)
except OSError as error:
raise StateError("could not save tool installation history") from error
finally:
try:
os.unlink(name)
except FileNotFoundError:
pass
@contextmanager
def transaction(self, vm_id):
with self.state.operation_lock():
data = self._read(vm_id)
yield data
self._write(vm_id, data)
def read(self, vm_id):
with self.state.operation_lock():
return self._read(vm_id)
def enqueue_locked(self, vm_id, selections, *, start_if_stopped=False, restart=False,
request_id=None, source="user"):
"""Called by lifecycle create/submit with operations.lock already held."""
selections = normalize_tools(selections)
data = self._read(vm_id)
wanted = [item.to_dict() for item in selections]
for previous in reversed(data["jobs"]):
same = (previous["selections"] == wanted and previous["start_if_stopped"] == start_if_stopped
and previous.get("restart", False) == restart)
if request_id is not None and previous.get("request_id") == request_id:
if not same:
raise ConflictError("this installation request ID was already used for different tools")
return copy.deepcopy(previous)
if previous["status"] in ACTIVE_JOB_STATUSES:
if same:
return copy.deepcopy(previous)
raise ConflictError("a tool installation is already queued or running for this VM")
now = self.clock()
job = {"id": "tools-" + uuid.uuid4().hex, "vm_id": vm_id, "status": "queued", "source": source,
"selections": wanted, "start_if_stopped": start_if_stopped, "restart": restart,
"request_id": request_id, "created_at": now, "updated_at": now, "completed_at": None,
"error": None, "outcome_unknown": False,
"items": [{**item.to_dict(), "key": item.key, "status": "queued", "detected_version": None,
"error": None, "output": "", "truncated": False} for item in selections]}
data["jobs"] = [*data["jobs"], job][-MAX_RETAINED_JOBS:]
self._write(vm_id, data)
return copy.deepcopy(job)
def get_job(self, vm_id, job_id):
for job in self.read(vm_id)["jobs"]:
if job["id"] == job_id:
return job
raise NotFoundError("tool installation job not found for this VM")
def update_job(self, vm_id, job_id, update):
with self.transaction(vm_id) as data:
for job in data["jobs"]:
if job["id"] == job_id:
update(job, data)
job["updated_at"] = self.clock()
return copy.deepcopy(job)
raise NotFoundError("tool installation job not found for this VM")
def claim(self):
with self.state.operation_lock():
for vm in self.state.load().vms.values():
# A failed/interrupted create may not own a runtime directory.
if not vm.runtime_owned or not self.settings.vm_dir(vm.id).is_dir():
continue
data = self._read(vm.id)
for job in data["jobs"]:
if job["status"] == "queued":
job["status"] = "starting" if vm.status == "stopped" or job.get("restart") else "waiting"
job["updated_at"] = self.clock()
self._write(vm.id, data)
return copy.deepcopy(job)
return None
@staticmethod
def interrupt(job, message, now, *, outcome_unknown=True):
job.update(status="interrupted", error=message, outcome_unknown=outcome_unknown,
updated_at=now, completed_at=now)
for item in job["items"]:
if item["status"] in {"installing", "queued"}:
item["status"] = "interrupted" if item["status"] == "installing" else "skipped"
item["error"] = message
def recover(self):
"""The worker owner lock proves that previous in-flight host workers exited."""
with self.state.operation_lock():
for vm in self.state.load().vms.values():
if not vm.runtime_owned or not self.settings.vm_dir(vm.id).is_dir():
continue
data = self._read(vm.id)
changed = False
for job in data["jobs"]:
if job["status"] in ACTIVE_JOB_STATUSES - {"queued"}:
self.interrupt(job, "Management server restarted. Refresh tool status before retrying; "
"the last installation may have completed inside the guest.", self.clock())
changed = True
if changed:
data.update(agent_ready=False, guest_busy=False)
self._write(vm.id, data)
uvm/tools.py
-338
View File
@@ -1,338 +0,0 @@
"""Bounded background installation workers using the existing guest exec channel."""
from __future__ import annotations
import fcntl
import json
import logging
import os
import re
import shlex
import stat
import threading
import time
from functools import lru_cache
from pathlib import Path
from . import guest_tools
from .errors import ConflictError, GuestAgentUnavailableError, GuestExecutionError, NotFoundError, UvmError, ValidationError
from .guest_exec import validate_exec
from .system import require_root
from .tool_catalog import all_selections, normalize_tools, recipe
from .tool_store import ToolJobStore
@lru_cache(maxsize=1)
def _guest_source():
return Path(guest_tools.__file__).read_text(encoding="utf-8")
def guest_command(payload: dict) -> str:
command = shlex.join(["/usr/bin/python3", "-I", "-c", _guest_source(), json.dumps(payload, ensure_ascii=True)])
validate_exec(command, True, 60)
return command
def _identity(vm):
return vm.pid, vm.process_start_time, vm.boot_id
def _text(value, limit=16384):
return str(value or "").encode("utf-8", errors="replace")[-limit:].decode("utf-8", errors="replace")
class ToolService:
def __init__(self, settings, state_store, lifecycle) -> None:
self.settings = settings
self.state = state_store
self.lifecycle = lifecycle
self.store = ToolJobStore(settings, state_store)
self._mutex = threading.RLock()
self._wake = threading.Condition()
self._stop = threading.Event()
self._threads = []
self._sessions = {}
self._owner = None
self._workers_left = 0
def start(self):
with self._mutex:
if self._stop.is_set():
raise ConflictError("the tool installer is shutting down")
if self._owner is not None:
return
with self.state.operation_lock():
descriptor = os.open(self.settings.base / "tool-worker.lock", os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600)
try:
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode) or metadata.st_nlink != 1 or metadata.st_uid != os.geteuid():
raise UvmError("unsafe tool worker ownership file")
os.fchmod(descriptor, 0o600)
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BaseException as error:
os.close(descriptor)
if isinstance(error, BlockingIOError):
raise ConflictError("another management server is already processing tool installations for this registry") from error
raise
self._owner = descriptor
try:
self.store.recover()
for index in range(self.settings.tool_workers):
thread = threading.Thread(target=self._worker, name=f"uvm-tools-{index}", daemon=True)
self._workers_left += 1
try:
thread.start()
except BaseException:
self._workers_left -= 1
raise
self._threads.append(thread)
except BaseException:
self._stop.set()
self.notify()
if not self._workers_left:
self._release_owner()
raise
def _release_owner(self):
if self._owner is not None:
os.close(self._owner)
self._owner = None
def notify(self):
with self._wake:
self._wake.notify_all()
def shutdown(self):
self._stop.set()
self.notify()
with self._mutex:
sessions = list(self._sessions.values())
for session in sessions:
session.close()
deadline = time.monotonic() + 8
for thread in self._threads:
thread.join(max(0, deadline - time.monotonic()))
# A worker finishing a host start keeps ownership until that start ends.
# Its guest command is never submitted after shutdown was requested.
with self._mutex:
if not self._workers_left:
self._release_owner()
def submit(self, vm_id, selections, *, start_if_stopped=False, restart=False, request_id=None):
require_root()
selections = normalize_tools(selections)
if type(start_if_stopped) is not bool or type(restart) is not bool:
raise ValidationError("start_if_stopped and restart must be booleans")
if request_id is not None and (not isinstance(request_id, str) or not re.fullmatch(r"[A-Za-z0-9_-]{1,80}", request_id)):
raise ValidationError("request_id must contain 1–80 letters, numbers, underscores, or hyphens")
self.start()
job = self.lifecycle.queue_tools(vm_id, selections, start_if_stopped=start_if_stopped,
restart=restart, request_id=request_id)
self.notify()
return job
def _execute(self, vm, payload, timeout):
if self._stop.is_set():
raise InterruptedError("management server is shutting down")
thread_id = threading.get_ident()
def connected(session):
with self._mutex:
self._sessions[thread_id] = session
if self._stop.is_set():
session.close()
try:
result = self.lifecycle.execute(vm.id, guest_command(payload), sudo=True, timeout_seconds=timeout,
expected_identity=_identity(vm), on_connect=connected)
finally:
with self._mutex:
self._sessions.pop(thread_id, None)
if result.timed_out:
return {"ok": False, "error": f"installation/verification timed out after {timeout:g} seconds; retry to finish it",
"output": _text(result.stderr), "truncated": result.truncated}
try:
value = json.loads(result.stdout)
if result.exit_code != 0 or result.truncated or not isinstance(value, dict) or type(value.get("ok")) is not bool:
raise ValueError("invalid result")
return value
except (ValueError, TypeError) as error:
raise GuestExecutionError("guest returned an incomplete tool result; refresh status before retrying",
outcome_unknown=payload["action"] == "install") from error
def _payload(self, vm, action, selections, *, boot_id=None):
return {"action": action, "username": vm.username, "architecture": self.settings.architecture,
"boot_id": boot_id, "tools": [recipe(selection, self.settings.architecture) for selection in selections]}
@staticmethod
def _inventory(result, selections):
expected = {item.key: item for item in selections}
records = result.get("tools")
if not isinstance(records, list) or len(records) != len(expected):
raise GuestExecutionError("invalid guest tool inventory")
inventory = []
for value in records:
if (not isinstance(value, dict) or value.get("key") not in expected
or type(value.get("installed")) is not bool):
raise GuestExecutionError("invalid guest tool inventory")
selection = expected.pop(value["key"])
inventory.append({**selection.to_dict(), "key": selection.key, "installed": value["installed"],
"detected_version": _text(value.get("detected_version"), 120) or None,
"error": _text(value.get("error"), 512) or None,
"usage": _text(value.get("usage"), 512)})
return inventory
def snapshot(self, vm_id, *, refresh=False):
require_root()
vm = self.state.get(vm_id)
data = self.store.read(vm_id)
running = self.lifecycle.inspect(vm.id).observed_status == "running"
if refresh and running:
supported = data["supported"]
try:
result = self._execute(vm, self._payload(vm, "probe", all_selections()), 90)
if type(result.get("supported")) is bool:
supported = result["supported"]
if not result["ok"]:
raise UvmError(_text(result.get("error"), 2048))
inventory = self._inventory(result, all_selections())
with self.store.transaction(vm_id) as current:
current.update(inventory=inventory, checked_at=time.time(), error=None, agent_ready=True,
guest_busy=result.get("busy") is True, supported=result.get("supported") is True)
data = self.store.read(vm_id)
except UvmError as error:
with self.store.transaction(vm_id) as current:
current.update(error=_text(error, 2048), agent_ready=not isinstance(error, GuestAgentUnavailableError),
supported=supported)
data = self.store.read(vm_id)
if not running:
data.update(agent_ready=False, guest_busy=False)
return {"vm_id": vm_id, **{key: data[key] for key in (
"inventory", "checked_at", "agent_ready", "guest_busy", "supported", "error")},
"stale": not running or data["checked_at"] is None or time.time() - data["checked_at"] > 60,
"latest_job": data["jobs"][-1] if data["jobs"] else None,
"jobs": [{key: job[key] for key in ("id", "status", "created_at", "completed_at")}
for job in reversed(data["jobs"])]}
def _worker(self):
try:
while not self._stop.is_set():
try:
job = self.store.claim()
if job is not None:
self._run_job(job)
continue
except (UvmError, OSError):
logging.getLogger(__name__).exception("Could not process a tool installation job")
with self._wake:
self._wake.wait(timeout=1)
finally:
with self._mutex:
self._workers_left -= 1
if self._workers_left == 0 and self._stop.is_set():
self._release_owner()
def _update(self, job, update):
return self.store.update_job(job["vm_id"], job["id"], update)
def _wait_ready(self, vm, selections):
deadline = time.monotonic() + self.settings.tool_ready_timeout_s
while not self._stop.is_set():
try:
result = self._execute(vm, self._payload(vm, "probe", selections), 90)
if not result["ok"]:
raise UvmError(_text(result.get("error"), 2048))
if result.get("busy"):
raise ConflictError("an earlier installation is still running inside this VM; wait and refresh before retrying")
if not isinstance(result.get("boot_id"), str) or not result["boot_id"]:
raise GuestExecutionError("guest did not report its boot identity")
return result
except GuestAgentUnavailableError:
if time.monotonic() >= deadline:
raise GuestAgentUnavailableError("Guest agent did not become ready. Restart & retry to apply agent setup, "
"then check the VM log if startup still fails.")
self._stop.wait(min(2, max(0, deadline - time.monotonic())))
raise InterruptedError("management server is shutting down")
def _run_job(self, job):
submitted = False
try:
if self._stop.is_set():
raise InterruptedError("management server is shutting down")
selections = normalize_tools(job["selections"])
vm = self.lifecycle.prepare_tool_install(job["vm_id"], start_if_stopped=job["start_if_stopped"],
restart=job.get("restart", False))
self._update(job, lambda current, data: current.update(status="waiting"))
info = self._wait_ready(vm, selections)
initial = self._inventory(info, selections)
def ready(current, data):
observed = {item["key"]: item for item in data["inventory"]}
observed.update({item["key"]: item for item in initial})
data.update(inventory=list(observed.values()), agent_ready=True,
guest_busy=False, supported=True, error=None)
self._update(job, ready)
for index, selection in enumerate(selections):
if self._stop.is_set():
raise InterruptedError("management server is shutting down")
def begin(current, data):
current["status"] = "installing"
current["items"][index]["status"] = "installing"
self._update(job, begin)
submitted = True
result = self._execute(vm, self._payload(vm, "install", [selection], boot_id=info["boot_id"]),
self.settings.tool_install_timeout_s)
observed = None
if result["ok"]:
observed, = self._inventory({"tools": [result.get("tool")]}, [selection])
if not observed["installed"] or result.get("status") not in {"installed", "already_installed"}:
raise GuestExecutionError("guest installation did not verify the selected tool", outcome_unknown=True)
def finish_item(current, data):
current["items"][index].update(
status=result["status"] if result["ok"] else "failed",
detected_version=observed["detected_version"] if observed else None,
error=None if result["ok"] else _text(result.get("error"), 2048),
output=_text(result.get("output")), truncated=result.get("truncated") is True)
inventory = {item["key"]: item for item in data["inventory"]}
if observed:
inventory[selection.key] = observed
elif selection.key in inventory:
inventory[selection.key].update(installed=False, error="Refresh to verify after the failed installation.")
data.update(inventory=list(inventory.values()))
self._update(job, finish_item)
submitted = False
def finish(current, data):
failures = sum(item["status"] == "failed" for item in current["items"])
current.update(status="failed" if failures else "succeeded", completed_at=time.time(),
error=f"{failures} tool installation(s) failed; see the per-tool output and retry." if failures else None)
self._update(job, finish)
except NotFoundError:
return # Deletion owns removal of the whole runtime, including history.
except (InterruptedError, GuestExecutionError, ConflictError) as error:
try:
self._update(job, lambda current, data: self.store.interrupt(
current, _text(error, 2048), time.time(),
outcome_unknown=getattr(error, "outcome_unknown", submitted)))
except NotFoundError:
pass
except (UvmError, OSError, ValueError) as error:
def failed(current, data):
self.store.interrupt(current, _text(error, 2048), time.time(), outcome_unknown=submitted)
current["status"] = "failed" if not submitted else "interrupted"
if isinstance(error, GuestAgentUnavailableError):
data.update(agent_ready=False, error=_text(error, 2048))
try:
self._update(job, failed)
except NotFoundError:
pass
except Exception:
logging.getLogger(__name__).exception("Unexpected tool installation failure")
try:
self._update(job, lambda current, data: self.store.interrupt(
current, "Unexpected installer failure; check the management server log and refresh before retrying.",
time.time(), outcome_unknown=submitted))
except NotFoundError:
pass