feat: enhance guest network configuration and recovery process for UVM

This commit is contained in:
2026-09-23 08:20:35 +05:30
parent dca3cbde2f
commit 660ba9efee
9 changed files with 474 additions and 12 deletions
+43 -3
View File
@@ -336,7 +336,7 @@ a ruleset/attachment snapshot, not application health or a reachability test.
3. Reserve ID/name/IP/MAC/ports with `status=starting`, `operation=preparing`.
4. Create the owned mode-0700 runtime directory, sparse-copy a mode-0600 private
disk, resize ext4 to the requested storage capacity, provision credentials/host
keys, and persist `disk_ready=True`.
keys and the managed guest network profile, then persist `disk_ready=True`.
5. Ensure bridge/egress networking and prepare an owned, attached TAP **down**.
6. Apply/read back the nftables bridge policy, then activate the TAP.
7. Write Firecracker config; persist `operation=spawning`, `spawn_pending=True`,
@@ -373,6 +373,14 @@ not needed. It never recopies the rootfs, reprovisions credentials, or
regenerates guest SSH host keys. Name/IP/MAC, ports, and guest disk changes
survive stop/start; guest RAM does not.
After runtime cleanup confirms process exit and any pending resize finishes,
start specializes an original UVM Ubuntu network profile with the retained
guest address/gateway. This one-time migration repairs older guests that lost
their default route. It uses a checked sparse copy of the retained private
disk; it never replaces that disk with the shared template. Guest-edited
profiles are preserved. The image layer's network checks and writes run only
under the lifecycle operation lock, before the TAP and VMM are started.
Start on a healthy running VM validates its retained network, synchronizes
policy, then restores shared bridge/NAT/forwarding rules without spawning another
VMM. A shared-network setup failure retains the running VM for retry. An ambiguous
@@ -500,7 +508,9 @@ must prevent `InstanceStart`. Without guest entropy, systemd's random-seed
service and OpenSSH can remain in startup while the VM process and IP respond.
The guest receives a static IP via the kernel `ip=` boot argument. The netmask
comes from `Settings.network`, not a hard-coded `/24` value.
comes from `Settings.network`, not a hard-coded `/24` value. The Ubuntu
systemd-networkd profile also receives explicit `Address=` and `Gateway=`
settings so it can maintain the same configuration after boot or reconfiguration.
ARM64 guests also receive `keep_bootcon`, as recommended by Firecracker's
getting-started guide; both architectures use `console=ttyS0`.
@@ -566,6 +576,30 @@ UVM replaces conventional RSA, ECDSA, and Ed25519 host keys in each private
disk and removes the known public login key from Firecracker's bionic image.
A VM must never rely on reusable SSH private keys from a shared template.
### Guest Network Profile
`GUEST_NETWORK_TEMPLATE` in `images.py` is shared with the Ubuntu builder. It
uses `KeepConfiguration=yes`: in Ubuntu 22.04's systemd-networkd, `static`
retains permanent addresses and `proto static` routes, but the kernel's
`ip=`-configured default route uses `proto boot` and is otherwise removed.
This can leave SSH to the private IP working while all off-subnet destinations
fail locally with `Network is unreachable`.
`ImageStore.configure_guest_network()` fills in the assigned address/prefix and
gateway during creation and stopped-disk resume. It only specializes the exact
shipped profile from recognized UVM Jammy v1/v2/v3 images. Missing, customized,
symlinked, hard-linked, or unexpectedly owned/mode-set profiles are left intact.
An already-specialized profile is a no-op. Custom images retain responsibility
for their own guest networking.
For migration, the image layer sparse-copies the private disk and runs
`e2fsck -f -p` on the copy. It rechecks the profile after journal replay so
committed guest edits cannot be overwritten based on stale offline reads.
The updated profile is read back and the filesystem checked before fsync and
atomic disk replacement. Failed staging keeps the original disk available for
retry. The built template's recipe marker is v3; cached older templates are
also handled by per-VM provisioning without requiring an installation refresh.
### Integrity Model
[`integrity.py`](uvm/integrity.py) has two concepts:
@@ -985,6 +1019,10 @@ and stop-on-error behavior. Tests simulate Ubuntu removing `policy-rc.d` after
bootstrap. They do not run debootstrap or boot a VM.
Lifecycle resize tests use fake processes/networking to verify ordering,
single-lock ownership, identity preservation, pending intent, and failure recovery.
`test_guest_network.py` exercises real offline ext4 profiles, explicit guest
addresses/gateways, custom subnets, idempotent migration, journal-replay
rechecks, and retention of guest data, credentials and keys on failures.
Lifecycle tests require confirmed process exit before network-profile repair.
### Isolated Packet Tests
@@ -1017,7 +1055,8 @@ on an isolated Linux host:
1. Strict checked installation.
2. `create` and successful guest boot.
3. Guest IP reachability and egress.
3. Guest IP reachability and egress, including the guest's default route after
systemd-networkd startup and reconfiguration.
4. SSH with unique guest host keys, `sudo -i` for a custom login, and
`sudo apt update && sudo apt install htop` in a newly built Ubuntu guest.
5. Stop/start/restart with retained guest data, SSH keys, and credentials.
@@ -1055,6 +1094,7 @@ on an isolated Linux host:
| VM is `dead` in `list` | PID/start token in state, `/proc/<pid>`, VM log |
| Cannot stop legacy VM | Missing start/boot identity or failed protected binary/socket match; never guess a PID |
| Guest has no egress | Host connectivity/routes, bridge/TAP state, `uvm-egress` iptables rules; `start` refreshes networking on a running VM |
| Guest says `Network is unreachable` | Guest `ip -4 route`; the default route must use the UVM bridge gateway. Restart once with updated code to migrate an original Ubuntu profile. |
| Policy missing/drift/unknown | Actual nft JSON and ownership, STP/group mask, permissions; explicit sync after fixing cause |
| Retained-disk start fails | `disk_ready`, safe private path, kernel/VMM integrity, process identity, retained network, pending operation |
| API server refuses to start | `UVM_API_TOKEN`, requested host, TLS cert/key files |