diff --git a/DEVELOPER.md b/DEVELOPER.md index 931f656..f5a7085 100644 --- a/DEVELOPER.md +++ b/DEVELOPER.md @@ -77,6 +77,7 @@ Packet integration tests must remain inside disposable isolated namespaces. │ ├── integrity.py # SHA-256 and integrity manifest logic │ ├── install.py # Apt, Firecracker, kernel, rootfs installation │ ├── images.py # Template lookup, private disks, and guest credentials +│ ├── known_hosts.py # Deletion-only cleanup of local SSH host-key caches │ ├── network.py # IP/MAC/TAP/bridge/NAT/forward rules │ ├── firewall.py # Owned nft bridge tables and offline conntrack cleanup │ ├── lifecycle.py # Lifecycle, bulk operations, recovery, port intent @@ -185,6 +186,7 @@ process: | CommandRunner | `CommandRunner` | Logged/testable subprocess execution | | StateStore | `StateStore` | Inventory, locks, JSON persistence | | ImageStore | `ImageStore` | Template verification, VM disks, and offline credentials | +| KnownHostsManager | `KnownHostsManager` | Scoped local SSH cache cleanup before deleted IPs are released | | NetworkManager | `NetworkManager` | Guest allocation and Linux networking | | FirewallManager | Constructed by `NetworkManager` | Owned native bridge policy and scoped offline conntrack cleanup | | FirecrackerProcessManager | `FirecrackerProcessManager` | VMM process control | @@ -415,7 +417,7 @@ stop: delete (destroy alias): persist delete intent -> same runtime cleanup -> remove owned runtime directory - -> remove state record -> release name/IP + -> clean local known_hosts entries -> remove state record -> release name/IP ``` Stop signals the **VMM**, not a guest shutdown agent or ACPI-coordinated @@ -428,6 +430,28 @@ Runtime cleanup retains reservations when process exit, TAP ownership, or connection cleanup is uncertain. A pending spawn cannot be declared gone merely because its PID was never persisted or its socket has not appeared. +`KnownHostsManager` is an explicit lifecycle dependency, also used by API and +bulk deletion. It runs after successful process/network/disk teardown and before +the registry entry is removed, under the same operation lock. Cleanup errors +retain `operation=delete` and the IP reservation; repeating delete retries it. +Stop, restart, and resize retain the existing guest's trusted host-key entries. + +Default caches are `known_hosts` and `known_hosts2` under the effective user's +and sudo invoker's account homes (`SUDO_UID`, with a `SUDO_USER` fallback). +`UVM_KNOWN_HOSTS_FILES` overrides this discovery for service deployments using a +colon-separated list of absolute paths. Only local files are managed. Matches +are explicit guest IPs or UVM's `uvm-` host-key alias, including OpenSSH's +HMAC-SHA1 hashed form. Plain bracketed ports are recognized directly; hashed +bracketed candidates use port 22 and the VM's recorded allowlist. Wildcard rules +and other aliases sharing a line are retained. + +File updates use pinned directory descriptors, regular/single-link checks, +an advisory lock, and a sibling temporary file. Owner, group, mode, and extended +attributes are preserved. A changed source snapshot prevents publication over +concurrent edits. A matching-free file is not rewritten and missing files are +not created. Tests inject the dependency to avoid accessing developers' caches, +and verify hashed-entry interoperability using real OpenSSH on temporary files. + ### Policy and Bulk Operations `set_ports()` takes a frozen registry ID, not a reusable name/IP; the CLI resolves diff --git a/README.md b/README.md index 3e21122..9066a2f 100644 --- a/README.md +++ b/README.md @@ -192,6 +192,15 @@ append a remote command after a literal `--`. `--insecure-host-key` disables verification for one connection. SSH does not automatically supply the stored password or open a closed port. +Deletion removes matching local SSH `known_hosts` entries for the guest IP and +UVM's VM-ID host-key alias before releasing the IP for reuse. It checks the +effective user's files and, under `sudo`, the invoking user's files, including +hashed entries. Unrelated hosts and file ownership/permissions are preserved. +For API services or custom SSH cache locations, set `UVM_KNOWN_HOSTS_FILES` to a +colon-separated list of absolute paths on the UVM host. See the SSH section in +[USAGE.md](USAGE.md#7-connect-with-ssh) for configuration and one-time cleanup of +entries left by older versions. + ### Private Ports ```sh @@ -448,6 +457,7 @@ Host and API settings can be overridden through environment variables: | `UVM_NETWORK` | Guest private IPv4 CIDR; defaults to `10.42.0.0/24` | | `UVM_GATEWAY` | Bridge/guest gateway; defaults to `10.42.0.1` | | `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` | +| `UVM_KNOWN_HOSTS_FILES` | Override local SSH cache files cleaned on deletion; colon-separated absolute paths | | `UVM_KERNEL_URL` | Override the host-architecture default guest kernel URL | | `UVM_ROOTFS_URL` | Override the host-architecture default guest rootfs URL | | `UVM_FIRECRACKER_SHA256` | Trusted Firecracker release archive SHA-256 | diff --git a/USAGE.md b/USAGE.md index 6cfd1fa..c9dd533 100644 --- a/USAGE.md +++ b/USAGE.md @@ -429,6 +429,33 @@ By default, UVM uses SSH `StrictHostKeyChecking=accept-new` and a stable host-key alias derived from the VM ID. This lets a guest IP be reused later without confusing it with the prior VM's known-hosts entry. +When a VM is deleted, UVM removes its explicit IP and `uvm-` entries +from local SSH caches. This also covers direct `ssh user@GUEST_IP` connections +and hashed hostnames. Plain `[host]:port` entries are removed for the VM; +hashed port entries are checked for TCP/22 and the ports in the VM's allowlist. +Other hosts (including other aliases on a shared line), wildcard trust rules, +comments, and file ownership/mode are preserved. Missing caches are harmless. + +The default files are `~/.ssh/known_hosts` and `~/.ssh/known_hosts2` for the +effective user and, when running under `sudo`, its invoking user. Homes are +resolved from the local account database. The API uses the user context that +launched its server. To select specific local files for a service or custom +SSH configuration, set this variable when launching UVM: + +```sh +export UVM_KNOWN_HOSTS_FILES='/root/.ssh/known_hosts:/home/dgxdev/.ssh/known_hosts' +# Pass it explicitly through sudo when starting the API or running deletion. +sudo env UVM_KNOWN_HOSTS_FILES="$UVM_KNOWN_HOSTS_FILES" ./uvm.py delete VM -y +``` + +These paths are local to the UVM host. An SSH client on another computer manages +its own cache there. For a stale entry left by a VM deleted with an older UVM +version, remove that entry once from the SSH client's account: + +```sh +ssh-keygen -f "$HOME/.ssh/known_hosts" -R '10.42.0.2' +``` + The key must already be authorized by the guest; UVM does not inject it. `--port` selects the guest SSH port (default 22), not a host mapping. The guest's sshd and private allowlist must allow that port. SSH does not feed the @@ -519,8 +546,10 @@ sudo ./uvm.py delete web -y # Equivalent destructive command: sudo ./uvm.py destroy web -y ``` -Delete performs the stop cleanup, then removes the runtime directory/private -disk and registry record, releasing the name and IP. `destroy` is the same +Delete performs the stop cleanup, removes the runtime directory/private disk, +cleans matching local SSH host-key entries, then removes the registry record, +releasing the name and IP. A cache cleanup error retains the pending deletion +and IP reservation so cleanup can be retried before the IP is reused. `destroy` is the same command with the same flags. Back up data first. A failed deletion remains pending; retry delete rather than trying to resurrect it with start. @@ -1023,6 +1052,7 @@ automatically detects every conflict with host routes or other managers. | `UVM_NETWORK` | Guest subnet; default `10.42.0.0/24` | | `UVM_GATEWAY` | Guest gateway / bridge IP; default `10.42.0.1` | | `UVM_BRIDGE` | Linux bridge name; default `uvm0` | +| `UVM_KNOWN_HOSTS_FILES` | Override deletion's local SSH cache files; colon-separated absolute paths | | `UVM_KERNEL_URL` | Override the host-architecture default kernel URL | | `UVM_ROOTFS_URL` | Override the host-architecture default rootfs URL | | `UVM_FIRECRACKER_SHA256` | Strict Firecracker archive checksum | diff --git a/tests/test_known_hosts.py b/tests/test_known_hosts.py index 8b7c434..226463f 100644 --- a/tests/test_known_hosts.py +++ b/tests/test_known_hosts.py @@ -1,7 +1,7 @@ from __future__ import annotations -import os import errno +import os import shutil import tempfile import unittest