feat: add terminal session management with HTTP/WebSocket routes and session history
This commit is contained in:
+92
-7
@@ -71,6 +71,9 @@ Packet integration tests must remain inside disposable isolated namespaces.
|
||||
│ ├── __main__.py # `python -m nest`
|
||||
│ ├── cli.py # Nest workspace serve command
|
||||
│ ├── server.py # Nest composition of the dashboard and UVM API
|
||||
│ ├── sessions.py # Live guest terminals and browser-independent session lifecycle
|
||||
│ ├── session_store.py # Private SQLite session history and bounded transcripts
|
||||
│ ├── session_routes.py # VM-scoped session HTTP and WebSocket endpoints
|
||||
│ └── dashboard/
|
||||
│ ├── router.py # Nest dashboard document route
|
||||
│ └── static/ # Packaged HTML, CSS, JavaScript, and fonts
|
||||
@@ -91,6 +94,7 @@ Packet integration tests must remain inside disposable isolated namespaces.
|
||||
│ ├── known_hosts.py # Deletion-only cleanup of local SSH host-key caches
|
||||
│ ├── guest_agent.py # Standalone guest daemon, job scopes, and bounded wire protocol
|
||||
│ ├── guest_exec.py # Host-side guest sessions and execution results
|
||||
│ ├── guest_terminal.py # Async PTY transport over a process-pinned guest connection
|
||||
│ ├── network.py # IP/MAC/TAP/bridge/NAT/forward rules
|
||||
│ ├── firewall.py # Owned nft bridge tables and offline conntrack cleanup
|
||||
│ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port intent
|
||||
@@ -877,8 +881,8 @@ or secret manager when that matters.
|
||||
|
||||
### API Contract
|
||||
|
||||
The root redirect and dashboard document are Nest routes. Both server entry
|
||||
points expose the remaining management routes below.
|
||||
The root redirect, dashboard document, and terminal-session routes belong to
|
||||
Nest. Both server entry points expose the UVM management routes below.
|
||||
|
||||
| Method | Route | Router | Notes |
|
||||
|---|---|---|---|
|
||||
@@ -902,6 +906,11 @@ points expose the remaining management routes below.
|
||||
| `POST` | `/vms/{vm_id}/exec` | `vms.py` | Exact ID; `command`, strict `sudo`, bounded `timeout_seconds` |
|
||||
| `POST` | `/vms/{vm_id}/agent/install` | `vms.py` | Exact ID; install/upgrade the agent on a stopped disk |
|
||||
| `GET` | `/vms/{identifier}/logs` | `vms.py` | Bounded non-following log tail |
|
||||
| `GET` | `/vms/{vm_id}/sessions` | `nest/session_routes.py` | Saved session list (Nest only) |
|
||||
| `POST` | `/vms/{vm_id}/sessions` | `nest/session_routes.py` | Launch a guest PTY; optional rows/cols in a JSON object (Nest only) |
|
||||
| `GET` | `/vms/{vm_id}/sessions/{session_id}` | `nest/session_routes.py` | VM-scoped session metadata (Nest only) |
|
||||
| `POST` | `/vms/{vm_id}/sessions/{session_id}/close` | `nest/session_routes.py` | End a shell; retain history (Nest only) |
|
||||
| `WS` | `/vms/{vm_id}/sessions/{session_id}/terminal` | `nest/session_routes.py` | Authenticated replay/input/resize (Nest only) |
|
||||
|
||||
All management routes use `get_authorized_application()` and require:
|
||||
|
||||
@@ -969,7 +978,8 @@ The synchronous HTTP response exposes only exit code, stdout/stderr, timeout,
|
||||
and truncation. Captured text uses UTF-8 replacement. Nonzero process exits are
|
||||
HTTP 200 results; agent failures are 502, and failures before submission are 503.
|
||||
Lost/malformed responses after submission carry `outcome_unknown: true`. There
|
||||
is no command retry, durable job registry, streaming response, or reconnection.
|
||||
is no command retry, durable job registry, streaming response, or reconnection
|
||||
for these one-shot exec requests.
|
||||
|
||||
The root systemd agent accepts only host-CID connections and limits concurrent
|
||||
connections/jobs to eight. Each request creates a delegated cgroup v2 job scope.
|
||||
@@ -977,7 +987,8 @@ A fresh child interpreter joins the scope before initializing the chosen user's
|
||||
supplementary groups, GID, UID, home, and environment, then execs `/bin/sh -c`.
|
||||
`sudo` selects guest root directly; otherwise the recorded username is used.
|
||||
Password authentication is not involved, and a recorded `root` login remains
|
||||
root for both flag values. Input is closed and there is no PTY.
|
||||
root for both flag values. One-shot commands have closed stdin; Nest terminal
|
||||
sessions use the separate PTY channel described below.
|
||||
|
||||
Both output pipes are drained concurrently. The guest enforces the deadline,
|
||||
signals the process group, escalates via `cgroup.kill`, and confirms cleanup.
|
||||
@@ -986,9 +997,12 @@ PGID reuse before reap. Normal exits clean up remaining background descendants
|
||||
too. An unconfirmed cleanup is an unknown outcome. Stopping the agent service
|
||||
also kills its job scopes; dropping an HTTP connection does not cancel a job.
|
||||
|
||||
The v4 Ubuntu builder installs Python and the service. Creation provisions the
|
||||
VM ID/username in root-only `/etc/uvm-agent.json` before marking the disk ready.
|
||||
Older templates are left for an explicit stopped-disk `agent/install` request.
|
||||
The v5 Ubuntu builder installs Python, the terminal-capable service, and an
|
||||
ownership manifest. Creation provisions the VM ID/username in root-only
|
||||
`/etc/uvm-agent.json` before marking the disk ready. Cached v4 templates are also
|
||||
configured; their original manifest-less agent is recognized by its exact
|
||||
published source digest. Earlier templates require an explicit stopped-disk
|
||||
`agent/install` request.
|
||||
Provisioning and upgrades stage a sparse copy, replay/check ext4, validate guest
|
||||
path ownership and recorded agent-file hashes, enable the service, verify writes,
|
||||
and publish atomically. A root-only guest manifest records managed file hashes
|
||||
@@ -1003,6 +1017,59 @@ device and uses CID 3. The process adapter retains both sockets while identity
|
||||
is uncertain and removes them only after confirmed VMM exit. IPv4 port policies
|
||||
do not govern this host-only management channel.
|
||||
|
||||
### Nest Terminal Sessions
|
||||
|
||||
Nest owns session HTTP/WebSocket routes and history. UVM owns the process-pinned
|
||||
connection and guest PTY. `LifecycleService.open_terminal()` validates an exact
|
||||
running VM ID and connects under the operation lock, then opens the PTY outside
|
||||
the lock. It never reconnects that socket to a replacement VMM.
|
||||
|
||||
The existing version-1 hello and exec request stay compatible. A terminal request
|
||||
uses `{"version":1,"terminal":{"rows":24,"cols":80}}`. The guest replies with
|
||||
`{"type":"ready"}` and exchanges length-prefixed JSON output/input/resize/close
|
||||
messages. Bytes are base64 encoded; raw chunks are at most 8192 bytes, frames
|
||||
16384 bytes, and dimensions 2–200 rows / 2–400 columns. Partial frames have finite
|
||||
deadlines while idle shells can remain connected indefinitely. An old agent
|
||||
rejects the new request; the host reports the stopped-disk upgrade procedure.
|
||||
|
||||
The guest opens a real PTY and spawns a fresh interpreter in a new OS session.
|
||||
The child joins its delegated job cgroup, establishes the controlling terminal,
|
||||
assumes the recorded account, and execs that account's interactive shell with
|
||||
`TERM=xterm-256color`. Input, terminal echo, Ctrl+C, foreground job control, and
|
||||
window-size changes are handled by the guest terminal driver. Normal exit,
|
||||
explicit close, or host connection loss cleans up the job cgroup and processes.
|
||||
|
||||
`SessionManager` owns background output readers independently of WebSocket
|
||||
attachments. Creating a shell is shielded from an HTTP disconnect; its connection
|
||||
is either tracked or closed. Browser navigation, reload, and logout detach only
|
||||
the browser. Nest shutdown closes guest connections and marks live sessions
|
||||
interrupted. In-flight transcript writes finish before releasing store ownership.
|
||||
|
||||
`SessionStore` keeps metadata and raw output in
|
||||
`$UVM_BASE/nest/sessions.sqlite3`, separate from the VM registry. A private
|
||||
directory, mode-0600 files, an exclusive owner lock, and SQLite transactions
|
||||
protect history. On reopening after a server restart, unfinished records become
|
||||
interrupted. Session lookups always include the exact VM ID. Output is capped at
|
||||
the newest 8 MiB per session with byte offsets and an explicit truncation marker;
|
||||
session metadata is retained. Only PTY output is recorded, not a separate input
|
||||
or password log. Nest permits 32 active sessions; guest terminals share the
|
||||
agent's eight connection slots with one-shot jobs.
|
||||
|
||||
HTTP routes use the existing `X-UVM-Token` dependency. Browser WebSockets check
|
||||
the Origin and require `{"type":"auth","token":"...","offset":0}` as their
|
||||
first frame within ten seconds. Tokens never enter query strings. Each output
|
||||
message has its byte offset and base64 data; the browser acknowledges the next
|
||||
offset after xterm renders the chunk. Only one chunk is outstanding, providing
|
||||
backpressure. Reattachment resumes from an acknowledged offset; a truncated
|
||||
prefix resets the viewer. Closed sessions use the same channel for read-only
|
||||
replay. WebSocket transport support comes from the `websockets` dependency.
|
||||
|
||||
The dashboard's Sessions tab and nested hash routes belong to the selected VM.
|
||||
`sessions.js` owns attachment teardown, replay, reconnects, and the terminal
|
||||
widget. Vendored xterm 5.5.0 and fit addon 0.10.0 (with MIT licenses) are served
|
||||
locally under `static/vendor/`. Terminal keyboard input bypasses dashboard search
|
||||
shortcuts. The widget has a 10,000-line scrollback and a 64 KiB per-paste limit.
|
||||
|
||||
## 14. Configuration Reference
|
||||
|
||||
The environment-variable table is in [README.md](README.md#configuration);
|
||||
@@ -1088,6 +1155,24 @@ FastAPI route tests are conditionally enabled when both `fastapi` and `httpx`
|
||||
are installed. They should be enabled in CI using the `test` extra.
|
||||
`test_nest_server.py` checks the workspace launcher, relocated frontend assets,
|
||||
authenticated machine controls, API documentation, and graceful shutdown.
|
||||
`test_sessions.py` covers scoped persistence, interrupted-session recovery,
|
||||
authentication, reconnects, output acknowledgements, and cancellation cleanup.
|
||||
`test_guest_terminal.py` exercises actual PTYs, persistent shell state, resizing,
|
||||
Ctrl+C, and child cleanup using temporary account homes and fake cgroup scopes.
|
||||
|
||||
Optional browser regression checks use Playwright against an isolated fixture
|
||||
server with fake VMs and guest channels. With the project's test extra installed:
|
||||
|
||||
```sh
|
||||
npm install --prefix /tmp/nest-browser playwright
|
||||
/tmp/nest-browser/node_modules/.bin/playwright install chromium
|
||||
NODE_PATH=/tmp/nest-browser/node_modules \
|
||||
NEST_TEST_PYTHON="$(command -v python)" node tests/browser_sessions.cjs
|
||||
```
|
||||
|
||||
This checks launch/input, reload/reconnect, session switching, per-VM isolation,
|
||||
closed-session replay, and the mobile layout. It does not boot a real microVM.
|
||||
|
||||
When `nft` and `conntrack` are installed, command-parser tests append an invalid
|
||||
final command/option so parsing is checked without executing any mutation.
|
||||
These checks supplement fixture-based readback and cleanup tests; they do not
|
||||
|
||||
Reference in New Issue
Block a user