2026-09-23 02:53:58 +00:00
3 changed files with 110 additions and 38 deletions
@@ -373,8 +373,10 @@ not needed. It never recopies the rootfs, reprovisions credentials, or
 regenerates guest SSH host keys. Name/IP/MAC, ports, and guest disk changes
 survive stop/start; guest RAM does not.
-Start on a healthy running VM synchronizes policy without spawning another
+Start on a healthy running VM validates its retained network, synchronizes
-VMM. An ambiguous process or interrupted live boot/stop is not adopted as a
+policy, then restores shared bridge/NAT/forwarding rules without spawning another
 VMM. A shared-network setup failure retains the running VM for retry. An ambiguous
 process or interrupted live boot/stop is not adopted as a
 successful start. Failed resume retains the private disk for retry. Restart
 performs stop and start under one operation lock, not two independent public
 calls with a race between them.
@@ -605,7 +607,7 @@ Linux bridge: uvm0 (10.42.0.1/24 by default)
    |
 iptables forwarding + MASQUERADE
    |
-host default uplink
+host IPv4 route for the destination (current uplink)
    |
 internet
 ```
@@ -620,7 +622,7 @@ internet
 - Bridge creation or validation that an existing interface is actually a
  Linux bridge.
 - Gateway address setup and IPv4 forwarding.
- One egress NAT rule and forward/return rules for the default uplink.
+- Egress NAT and forward/return rules scoped to the guest subnet/bridge.
 - TAP ownership aliases (`uvm:<id>`), preparation while down, activation,
  deactivation, cleanup, and error reporting.
@@ -629,6 +631,23 @@ it may belong to another VM. A create-time name collision is an error. Cleanup
 checks the TAP kind and ownership alias, with a narrow unaliased-TAP allowance
 for legacy records; it does not treat an arbitrary interface name as ownership.
 Internet sharing is configured even with no host default route. Rules never
 pin an uplink interface: Linux routing selects it, and MASQUERADE uses that
 interface's address. Offline-to-online transitions, reconnects and route changes
 therefore work without another UVM command or a monitoring daemon. Existing
 connections may need to reconnect after the upstream address changes. Guest
 DNS in the Ubuntu template uses the public resolvers documented in README.
 `uvm-egress`-commented iptables rules are checked before insertion, with a bounded
 xtables-lock wait. Scoped exceptions are inserted at the start of the host chains
 so pre-existing terminal drops cannot shadow them. Outbound forwarding requires
 the UVM bridge and source subnet; routed replies require the destination bridge,
 subnet and established/related conntrack state. Native bridge policy still checks
 identity, service ports and permitted protocols. Outbound forwarding also covers
 peer traffic if the host already uses `br_netfilter`, while MASQUERADE excludes
 the UVM bridge to retain private peer source addresses. Global chain policies
 and unrelated rules remain administrator-owned.
 ### Native Bridge Policy
 The supported path is a **software Linux bridge** with STP disabled. Each
@@ -685,16 +704,16 @@ including on conntrack-tools 1.4.8, where `--list` is not a supported option.
 ### Coexistence and Recovery
-Existing `iptables` egress MASQUERADE/forwarding remains unchanged. The new
+Shared `iptables` egress MASQUERADE/forwarding is separate from private bridge
-policy adds no DNAT, public listener, or host-port mapping and does not replace
+policy. The policy adds no DNAT, public listener, or host-port mapping and does not replace
-the host firewall manager. Public connectivity, DNS, guest reverse proxies/TLS,
+the host firewall manager. Public inbound connectivity, DNS publication, guest reverse proxies/TLS,
 and integration with UFW/firewalld/Docker/libvirt remain out of scope. An
 administrator's external routing/NAT can still affect reachability; do not
 infer source-CIDR restrictions or complete public-exposure protection from the
 use of private addresses. Operators must choose a non-conflicting topology.
-If an external actor loses or changes UVM rules, recovery is explicit
+If an external actor loses or changes private policy, recovery is explicit
-`ports sync`, not a promised daemon repair. An `enforced` snapshot cannot
+`ports sync`; `start` also restores shared networking for a running VM. An `enforced` snapshot cannot
 prevent later root-level changes or prove that other rules allow traffic.
 [FIREWALL.md](FIREWALL.md) retains the original proposal/backend evaluation;
 do not import its unimplemented topology checks into current documentation.
@@ -950,17 +969,18 @@ The opt-in packet harness must run **only in disposable isolated user/network
 namespaces**. It verifies namespace separation before creating links or
 changing rules, and never falls back to the parent host namespace. It may
 require privileges (or enabled unprivileged user namespaces) plus `unshare`,
-`ip`, `bridge`, `nft`, `conntrack`, and `sysctl`:
+`ip`, `bridge`, `nft`, `iptables`, `conntrack`, and `sysctl`:
 ```sh
 UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
 ```
-The harness uses bridge-connected veths in place of guest TAPs and routed
+The harness uses bridge-connected veths in place of guest TAPs and two simulated
-egress without NAT. It exercises host/peer service traffic, both-direction
+uplinks. It exercises NAT with a pre-existing terminal FORWARD drop, offline
-close of existing streams while preserving conntrack/egress, IPv6/spoof
+startup, uplink changes and reconnects for host/guest TCP and guest UDP/53.
-blocking, quarantine, and offline cleanup. It does not boot a microVM or
+It also checks host/peer service traffic, both-direction close of existing
-validate real TAP/NAT deployment.
+streams while preserving conntrack/egress, IPv6/spoof blocking, quarantine,
 and offline cleanup. It does not boot a microVM or validate a real TAP deployment.
 The packet test skipped in this workspace because namespace creation was
 denied (`Operation not permitted`). Document that permission limitation, not
@@ -1011,7 +1031,7 @@ on an isolated Linux host:
 | Firecracker exits early | VM `firecracker.log`, VM config, `/dev/kvm` access |
 | VM is `dead` in `list` | PID/start token in state, `/proc/<pid>`, VM log |
 | Cannot stop legacy VM | Missing start/boot identity or failed protected binary/socket match; never guess a PID |
-| Guest has no egress | `ip route show default`, bridge address, TAP state, iptables rules |
+| Guest has no egress | Host connectivity/routes, bridge/TAP state, `uvm-egress` iptables rules; `start` refreshes networking on a running VM |
 | Policy missing/drift/unknown | Actual nft JSON and ownership, STP/group mask, permissions; explicit sync after fixing cause |
 | Retained-disk start fails | `disk_ready`, safe private path, kernel/VMM integrity, process identity, retained network, pending operation |
 | API server refuses to start | `UVM_API_TOKEN`, requested host, TLS cert/key files |
@@ -1035,9 +1055,9 @@ The code is intentionally honest about current limits:
 - Guest health is inferred from VMM process state, not application readiness.
 - No quotas, DHCP, host-port mapping, UDP/IPv6 services, or network tenancy.
 - No guest snapshots, extra disks, production image builder, or SSH-key injection.
- Per-VM private TCP ingress policy exists, but public connectivity, DNS,
+- Per-VM private TCP ingress policy exists, but public inbound connectivity, DNS publication,
  guest reverse proxy/TLS, and external firewall integration remain out of scope.
- Existing egress `iptables` rules are incremental and unchanged; only the
+- Shared egress `iptables` rules are incremental and idempotent; only the
  private bridge policy is atomically rendered per VM.
 - No autostart, background reconciliation, or guest-coordinated shutdown.
 - The JSON inventory is suitable for one host, not distributed coordination.
@@ -3,8 +3,8 @@
 `uvm` is a small, local Firecracker microVM command-line manager for Ubuntu.
 It creates one Firecracker process per VM, attaches it to a host TAP device,
 and keeps a private disk and local registry for named VMs. The CLI supports
-create, start, stop, restart, resize, delete, inspection, SSH, logs, diagnostics, and
+create, start, stop, restart, resize, delete, inspection, SSH, logs, diagnostics,
-enforced private-IPv4 TCP port allowlists.
+host-shared IPv4 internet access, and enforced private-IPv4 TCP port allowlists.
 This directory is a self-contained project. All UVM code, tests, packaging
 metadata, launcher scripts, and documentation live here.
@@ -255,7 +255,7 @@ well as new connections. This is packet dropping, not a guaranteed TCP RST or
 immediate application disconnect.
 These rules govern access to the **guest private IP**. They do not start a
-guest service or provide public connectivity, host-port mappings, DNS,
+guest service or provide public inbound connectivity, host-port mappings, DNS publication,
 reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not
 yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required
 ARP/ICMP traffic remain allowed; managed TAPs block IPv6.
@@ -420,13 +420,33 @@ Bridge:   uvm0
 TAPs:     uvm-<unique suffix>
 ```
-It creates the bridge when absent, validates an existing bridge before using
+VMs share the host's IPv4 internet connection by default. When the host has
-it, enables IPv4 forwarding, adds an egress masquerade rule, and adds matching
+internet access, guests can initiate outbound connections through it. When
-forward rules for return traffic. Guest IPs, gateway, bridge name, and guest
+the host loses connectivity, guest internet access is lost too; the private
-boot netmask derive from the configured CIDR.
+VM network remains usable. Reconnecting the host restores guest egress.
-The existing `iptables` egress NAT/forwarding behavior is unchanged. Private
+UVM creates the bridge when absent, validates an existing bridge before using
-VM creation and start require an RFC 1918 subnet (`10.0.0.0/8`,
+it, enables IPv4 forwarding, and installs subnet-scoped `iptables` masquerade
 and forward/return rules. These rules are installed even while the host is
 offline and follow its current routing table, so switching uplinks does not
 require restarting VMs. Existing connections may need application retries
 after an outage or upstream address change. Guest IPs, gateway, bridge name,
 and guest boot netmask derive from the configured CIDR.
 Forwarding exceptions are inserted ahead of existing host-chain rules and
 allow guest-originated traffic and its replies. Masquerading excludes traffic
 back onto the UVM bridge, preserving private peer addresses.
 To apply updated sharing rules to an existing running VM, use:
 ```sh
 sudo ./uvm.py start web
 ```
 On a running VM, `start` refreshes its private policy and shared host networking
 without rebooting it. New VM boots set up the same networking automatically.
 Private VM creation and start require an RFC 1918 subnet (`10.0.0.0/8`,
 `172.16.0.0/12`, or `192.168.0.0/16`). Retained VMs prevent incompatible
 gateway, subnet, or bridge changes within the same registry. Private
 port enforcement uses separate per-VM native nftables `bridge` tables, not
@@ -443,8 +463,9 @@ firewalld, Docker, or other external firewall managers remains deferred.
 Do not offload traffic around the software bridge hooks.
 There is no autostart service or daemon reconciliation. After externally lost
-or changed UVM rules, inspect policy status and explicitly run `ports sync`.
+or changed private policy, inspect its status and explicitly run `ports sync`.
-Read-only commands do not repair rules; sync does not boot stopped VMs.
+Use `start` to restore shared bridge/NAT/forwarding rules as well. Read-only
 commands do not repair rules; port sync does not boot stopped VMs.
 Use one authoritative registry per bridge/subnet. Independent `UVM_BASE`
 directories do not coordinate IP or MAC allocation with each other.
@@ -568,8 +589,10 @@ UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration
 The packet test was skipped in this workspace because namespace creation was
 denied (`Operation not permitted`). A skip is not packet-path or real-host
-validation. The harness uses veths in place of TAPs and does not boot a real VM;
+validation. The harness uses veths in place of TAPs and simulated upstream
-validate KVM, TAP, NAT, SSH, and guest connectivity on an isolated Linux host.
+networks to check NAT, offline startup, reconnects, and uplink changes alongside
 private port policies. It does not boot a real VM; validate KVM, TAP, SSH, and
 guest connectivity on an isolated Linux host.
 ## Current Boundaries
@@ -29,6 +29,12 @@ Bridge:        uvm0
 Guest pool:    10.42.0.2 through 10.42.0.254
 ```
 Guest IPv4 internet access shares the host's connection automatically. Host
 offline means guest internet is offline; reconnecting the host restores guest
 egress. UVM installs sharing rules even when booting offline, and follows the
 host's current routes when uplinks change. The private network remains usable
 without internet. Applications may need to reconnect after a network interruption.
 The default locations on the host are under `/var/lib/uvm`. Stop/start and
 restart reuse the private disk; delete removes it. UVM is a local CLI, not an
 autostart service or a daemon that continuously repairs VM or firewall state.
@@ -1134,7 +1140,10 @@ Firecracker logs may include guest/application secrets; redact them before shari
 ## 15. Networking Troubleshooting
-The existing `iptables` egress MASQUERADE and forwarding rules are unchanged.
+UVM enables IPv4 forwarding and installs `uvm-egress`-commented iptables
 MASQUERADE and forward/return rules. These match the private subnet and bridge
 without fixing an uplink, so guest traffic follows the host's current routes.
 Scoped forwarding exceptions are inserted ahead of existing host-chain rules.
 Private port filtering is a separate native nftables `bridge` policy, not
 DNAT. It must be installed and read back before the TAP is active. The bridge
 must have STP disabled and each TAP must have bridge-slave `group_fwd_mask`
@@ -1162,11 +1171,30 @@ sudo iptables -t nat -S POSTROUTING
 sudo iptables -S FORWARD
 ```
 After upgrading UVM, apply the sharing rules to an already-running VM with:
 ```sh
 sudo ./uvm.py start web
 ```
 `start` on a running VM refreshes its private policy and shared host networking
 without a reboot. Use the updated CLI checkout, or restart the API server to
 load the updated code before using its start endpoint. Subsequent host
 disconnects, reconnects, and route changes require no UVM command.
 Check DNS and HTTPS from the guest:
 ```sh
 sudo ./uvm.py ssh web -- getent ahostsv4 example.com
 sudo ./uvm.py ssh web -- curl -4 --connect-timeout 5 --max-time 15 -I https://example.com
 ```
 Common issues:
 | Symptom | First checks |
 |---|---|
-| Guest cannot reach the internet | Bridge is up, TAP is up, host has default route, NAT and FORWARD rules exist. |
+| Guest cannot reach the internet | Confirm the host can reach the same destination, bridge/TAP are up, and `uvm-egress` NAT/FORWARD rules exist; use `start VM` to refresh sharing. |
 | IP connectivity works but DNS fails | Check guest `/etc/resolv.conf` and DNS reachability. The Ubuntu template uses `1.1.1.1` and `8.8.8.8`; host loopback resolvers such as `127.0.0.53` are not reachable as guest loopback addresses. |
 | Requested guest IP is rejected | Confirm it is inside the configured subnet and not used, gateway, network, or broadcast. |
 | Create says TAP already exists | Inspect the interface; do not delete it unless you know it is stale UVM state. |
 | Existing bridge is rejected | Verify it is a software Linux bridge with STP disabled. |
@@ -1187,7 +1215,7 @@ Keep these constraints in mind when deciding whether UVM fits a scenario:
 - No quotas or cgroup CPU, memory, I/O, or process limits are applied.
 - Fractional CPU requests are advisory only.
 - No autostart, background lifecycle/firewall reconciliation, or guest-coordinated stop.
- No UDP/IPv6 service policies, public connectivity setup, host-port mapping,
+- No UDP/IPv6 service policies, public inbound connectivity setup, host-port mapping,
  DNS publication, guest reverse proxy/TLS setup, DHCP, load balancing,
  snapshots, multi-network tenancy, or multi-host scheduling.
 - The API uses one shared bearer token, not individual users, roles, or tenants.
@@ -1220,7 +1248,7 @@ python3 -B -m unittest discover -s tests -v
 The opt-in packet test runs **only in disposable isolated user/network
 namespaces**, which may require privileges. It needs `unshare`, `ip`,
-`bridge`, `nft`, `conntrack`, and `sysctl`:
+`bridge`, `nft`, `iptables`, `conntrack`, and `sysctl`:
 ```sh
 UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v
@@ -1228,9 +1256,10 @@ UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration
 Namespace creation was denied in this workspace (`Operation not permitted`),
 so the packet test skipped. That is a permission limitation, not real-packet
-or real-host validation. The harness uses veths as TAP stand-ins and routed
+or real-host validation. The harness uses veths as TAP stand-ins and simulated
-egress without NAT; it does not boot Firecracker or validate the host's real
+upstreams to check NAT, offline startup, reconnects and uplink changes. It does
-TAP/NAT setup. Run actual VM/SSH/egress checks on an isolated Linux host before
+not boot Firecracker or validate the host's real TAP setup. Run actual
 VM/SSH/egress checks on an isolated Linux host before
 relying on the deployment.
 ## 19. Get Help