diff --git a/DEVELOPER.md b/DEVELOPER.md index f786681..2c94b58 100644 --- a/DEVELOPER.md +++ b/DEVELOPER.md @@ -373,8 +373,10 @@ not needed. It never recopies the rootfs, reprovisions credentials, or regenerates guest SSH host keys. Name/IP/MAC, ports, and guest disk changes survive stop/start; guest RAM does not. -Start on a healthy running VM synchronizes policy without spawning another -VMM. An ambiguous process or interrupted live boot/stop is not adopted as a +Start on a healthy running VM validates its retained network, synchronizes +policy, then restores shared bridge/NAT/forwarding rules without spawning another +VMM. A shared-network setup failure retains the running VM for retry. An ambiguous +process or interrupted live boot/stop is not adopted as a successful start. Failed resume retains the private disk for retry. Restart performs stop and start under one operation lock, not two independent public calls with a race between them. @@ -605,7 +607,7 @@ Linux bridge: uvm0 (10.42.0.1/24 by default) | iptables forwarding + MASQUERADE | -host default uplink +host IPv4 route for the destination (current uplink) | internet ``` @@ -620,7 +622,7 @@ internet - Bridge creation or validation that an existing interface is actually a Linux bridge. - Gateway address setup and IPv4 forwarding. -- One egress NAT rule and forward/return rules for the default uplink. +- Egress NAT and forward/return rules scoped to the guest subnet/bridge. - TAP ownership aliases (`uvm:`), preparation while down, activation, deactivation, cleanup, and error reporting. @@ -629,6 +631,23 @@ it may belong to another VM. A create-time name collision is an error. Cleanup checks the TAP kind and ownership alias, with a narrow unaliased-TAP allowance for legacy records; it does not treat an arbitrary interface name as ownership. +Internet sharing is configured even with no host default route. Rules never +pin an uplink interface: Linux routing selects it, and MASQUERADE uses that +interface's address. Offline-to-online transitions, reconnects and route changes +therefore work without another UVM command or a monitoring daemon. Existing +connections may need to reconnect after the upstream address changes. Guest +DNS in the Ubuntu template uses the public resolvers documented in README. + +`uvm-egress`-commented iptables rules are checked before insertion, with a bounded +xtables-lock wait. Scoped exceptions are inserted at the start of the host chains +so pre-existing terminal drops cannot shadow them. Outbound forwarding requires +the UVM bridge and source subnet; routed replies require the destination bridge, +subnet and established/related conntrack state. Native bridge policy still checks +identity, service ports and permitted protocols. Outbound forwarding also covers +peer traffic if the host already uses `br_netfilter`, while MASQUERADE excludes +the UVM bridge to retain private peer source addresses. Global chain policies +and unrelated rules remain administrator-owned. + ### Native Bridge Policy The supported path is a **software Linux bridge** with STP disabled. Each @@ -685,16 +704,16 @@ including on conntrack-tools 1.4.8, where `--list` is not a supported option. ### Coexistence and Recovery -Existing `iptables` egress MASQUERADE/forwarding remains unchanged. The new -policy adds no DNAT, public listener, or host-port mapping and does not replace -the host firewall manager. Public connectivity, DNS, guest reverse proxies/TLS, +Shared `iptables` egress MASQUERADE/forwarding is separate from private bridge +policy. The policy adds no DNAT, public listener, or host-port mapping and does not replace +the host firewall manager. Public inbound connectivity, DNS publication, guest reverse proxies/TLS, and integration with UFW/firewalld/Docker/libvirt remain out of scope. An administrator's external routing/NAT can still affect reachability; do not infer source-CIDR restrictions or complete public-exposure protection from the use of private addresses. Operators must choose a non-conflicting topology. -If an external actor loses or changes UVM rules, recovery is explicit -`ports sync`, not a promised daemon repair. An `enforced` snapshot cannot +If an external actor loses or changes private policy, recovery is explicit +`ports sync`; `start` also restores shared networking for a running VM. An `enforced` snapshot cannot prevent later root-level changes or prove that other rules allow traffic. [FIREWALL.md](FIREWALL.md) retains the original proposal/backend evaluation; do not import its unimplemented topology checks into current documentation. @@ -950,17 +969,18 @@ The opt-in packet harness must run **only in disposable isolated user/network namespaces**. It verifies namespace separation before creating links or changing rules, and never falls back to the parent host namespace. It may require privileges (or enabled unprivileged user namespaces) plus `unshare`, -`ip`, `bridge`, `nft`, `conntrack`, and `sysctl`: +`ip`, `bridge`, `nft`, `iptables`, `conntrack`, and `sysctl`: ```sh UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v ``` -The harness uses bridge-connected veths in place of guest TAPs and routed -egress without NAT. It exercises host/peer service traffic, both-direction -close of existing streams while preserving conntrack/egress, IPv6/spoof -blocking, quarantine, and offline cleanup. It does not boot a microVM or -validate real TAP/NAT deployment. +The harness uses bridge-connected veths in place of guest TAPs and two simulated +uplinks. It exercises NAT with a pre-existing terminal FORWARD drop, offline +startup, uplink changes and reconnects for host/guest TCP and guest UDP/53. +It also checks host/peer service traffic, both-direction close of existing +streams while preserving conntrack/egress, IPv6/spoof blocking, quarantine, +and offline cleanup. It does not boot a microVM or validate a real TAP deployment. The packet test skipped in this workspace because namespace creation was denied (`Operation not permitted`). Document that permission limitation, not @@ -1011,7 +1031,7 @@ on an isolated Linux host: | Firecracker exits early | VM `firecracker.log`, VM config, `/dev/kvm` access | | VM is `dead` in `list` | PID/start token in state, `/proc/`, VM log | | Cannot stop legacy VM | Missing start/boot identity or failed protected binary/socket match; never guess a PID | -| Guest has no egress | `ip route show default`, bridge address, TAP state, iptables rules | +| Guest has no egress | Host connectivity/routes, bridge/TAP state, `uvm-egress` iptables rules; `start` refreshes networking on a running VM | | Policy missing/drift/unknown | Actual nft JSON and ownership, STP/group mask, permissions; explicit sync after fixing cause | | Retained-disk start fails | `disk_ready`, safe private path, kernel/VMM integrity, process identity, retained network, pending operation | | API server refuses to start | `UVM_API_TOKEN`, requested host, TLS cert/key files | @@ -1035,9 +1055,9 @@ The code is intentionally honest about current limits: - Guest health is inferred from VMM process state, not application readiness. - No quotas, DHCP, host-port mapping, UDP/IPv6 services, or network tenancy. - No guest snapshots, extra disks, production image builder, or SSH-key injection. -- Per-VM private TCP ingress policy exists, but public connectivity, DNS, +- Per-VM private TCP ingress policy exists, but public inbound connectivity, DNS publication, guest reverse proxy/TLS, and external firewall integration remain out of scope. -- Existing egress `iptables` rules are incremental and unchanged; only the +- Shared egress `iptables` rules are incremental and idempotent; only the private bridge policy is atomically rendered per VM. - No autostart, background reconciliation, or guest-coordinated shutdown. - The JSON inventory is suitable for one host, not distributed coordination. diff --git a/README.md b/README.md index a258277..d223cc8 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,8 @@ `uvm` is a small, local Firecracker microVM command-line manager for Ubuntu. It creates one Firecracker process per VM, attaches it to a host TAP device, and keeps a private disk and local registry for named VMs. The CLI supports -create, start, stop, restart, resize, delete, inspection, SSH, logs, diagnostics, and -enforced private-IPv4 TCP port allowlists. +create, start, stop, restart, resize, delete, inspection, SSH, logs, diagnostics, +host-shared IPv4 internet access, and enforced private-IPv4 TCP port allowlists. This directory is a self-contained project. All UVM code, tests, packaging metadata, launcher scripts, and documentation live here. @@ -255,7 +255,7 @@ well as new connections. This is packet dropping, not a guaranteed TCP RST or immediate application disconnect. These rules govern access to the **guest private IP**. They do not start a -guest service or provide public connectivity, host-port mappings, DNS, +guest service or provide public inbound connectivity, host-port mappings, DNS publication, reverse proxies, or guest-service TLS. UDP and IPv6 service policies are not yet supported. Guest-initiated IPv4 TCP/UDP egress, its replies, and required ARP/ICMP traffic remain allowed; managed TAPs block IPv6. @@ -420,13 +420,33 @@ Bridge: uvm0 TAPs: uvm- ``` -It creates the bridge when absent, validates an existing bridge before using -it, enables IPv4 forwarding, adds an egress masquerade rule, and adds matching -forward rules for return traffic. Guest IPs, gateway, bridge name, and guest -boot netmask derive from the configured CIDR. +VMs share the host's IPv4 internet connection by default. When the host has +internet access, guests can initiate outbound connections through it. When +the host loses connectivity, guest internet access is lost too; the private +VM network remains usable. Reconnecting the host restores guest egress. -The existing `iptables` egress NAT/forwarding behavior is unchanged. Private -VM creation and start require an RFC 1918 subnet (`10.0.0.0/8`, +UVM creates the bridge when absent, validates an existing bridge before using +it, enables IPv4 forwarding, and installs subnet-scoped `iptables` masquerade +and forward/return rules. These rules are installed even while the host is +offline and follow its current routing table, so switching uplinks does not +require restarting VMs. Existing connections may need application retries +after an outage or upstream address change. Guest IPs, gateway, bridge name, +and guest boot netmask derive from the configured CIDR. + +Forwarding exceptions are inserted ahead of existing host-chain rules and +allow guest-originated traffic and its replies. Masquerading excludes traffic +back onto the UVM bridge, preserving private peer addresses. + +To apply updated sharing rules to an existing running VM, use: + +```sh +sudo ./uvm.py start web +``` + +On a running VM, `start` refreshes its private policy and shared host networking +without rebooting it. New VM boots set up the same networking automatically. + +Private VM creation and start require an RFC 1918 subnet (`10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`). Retained VMs prevent incompatible gateway, subnet, or bridge changes within the same registry. Private port enforcement uses separate per-VM native nftables `bridge` tables, not @@ -443,8 +463,9 @@ firewalld, Docker, or other external firewall managers remains deferred. Do not offload traffic around the software bridge hooks. There is no autostart service or daemon reconciliation. After externally lost -or changed UVM rules, inspect policy status and explicitly run `ports sync`. -Read-only commands do not repair rules; sync does not boot stopped VMs. +or changed private policy, inspect its status and explicitly run `ports sync`. +Use `start` to restore shared bridge/NAT/forwarding rules as well. Read-only +commands do not repair rules; port sync does not boot stopped VMs. Use one authoritative registry per bridge/subnet. Independent `UVM_BASE` directories do not coordinate IP or MAC allocation with each other. @@ -568,8 +589,10 @@ UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration The packet test was skipped in this workspace because namespace creation was denied (`Operation not permitted`). A skip is not packet-path or real-host -validation. The harness uses veths in place of TAPs and does not boot a real VM; -validate KVM, TAP, NAT, SSH, and guest connectivity on an isolated Linux host. +validation. The harness uses veths in place of TAPs and simulated upstream +networks to check NAT, offline startup, reconnects, and uplink changes alongside +private port policies. It does not boot a real VM; validate KVM, TAP, SSH, and +guest connectivity on an isolated Linux host. ## Current Boundaries diff --git a/USAGE.md b/USAGE.md index c4ae9ee..ca4a173 100644 --- a/USAGE.md +++ b/USAGE.md @@ -29,6 +29,12 @@ Bridge: uvm0 Guest pool: 10.42.0.2 through 10.42.0.254 ``` +Guest IPv4 internet access shares the host's connection automatically. Host +offline means guest internet is offline; reconnecting the host restores guest +egress. UVM installs sharing rules even when booting offline, and follows the +host's current routes when uplinks change. The private network remains usable +without internet. Applications may need to reconnect after a network interruption. + The default locations on the host are under `/var/lib/uvm`. Stop/start and restart reuse the private disk; delete removes it. UVM is a local CLI, not an autostart service or a daemon that continuously repairs VM or firewall state. @@ -1134,7 +1140,10 @@ Firecracker logs may include guest/application secrets; redact them before shari ## 15. Networking Troubleshooting -The existing `iptables` egress MASQUERADE and forwarding rules are unchanged. +UVM enables IPv4 forwarding and installs `uvm-egress`-commented iptables +MASQUERADE and forward/return rules. These match the private subnet and bridge +without fixing an uplink, so guest traffic follows the host's current routes. +Scoped forwarding exceptions are inserted ahead of existing host-chain rules. Private port filtering is a separate native nftables `bridge` policy, not DNAT. It must be installed and read back before the TAP is active. The bridge must have STP disabled and each TAP must have bridge-slave `group_fwd_mask` @@ -1162,11 +1171,30 @@ sudo iptables -t nat -S POSTROUTING sudo iptables -S FORWARD ``` +After upgrading UVM, apply the sharing rules to an already-running VM with: + +```sh +sudo ./uvm.py start web +``` + +`start` on a running VM refreshes its private policy and shared host networking +without a reboot. Use the updated CLI checkout, or restart the API server to +load the updated code before using its start endpoint. Subsequent host +disconnects, reconnects, and route changes require no UVM command. + +Check DNS and HTTPS from the guest: + +```sh +sudo ./uvm.py ssh web -- getent ahostsv4 example.com +sudo ./uvm.py ssh web -- curl -4 --connect-timeout 5 --max-time 15 -I https://example.com +``` + Common issues: | Symptom | First checks | |---|---| -| Guest cannot reach the internet | Bridge is up, TAP is up, host has default route, NAT and FORWARD rules exist. | +| Guest cannot reach the internet | Confirm the host can reach the same destination, bridge/TAP are up, and `uvm-egress` NAT/FORWARD rules exist; use `start VM` to refresh sharing. | +| IP connectivity works but DNS fails | Check guest `/etc/resolv.conf` and DNS reachability. The Ubuntu template uses `1.1.1.1` and `8.8.8.8`; host loopback resolvers such as `127.0.0.53` are not reachable as guest loopback addresses. | | Requested guest IP is rejected | Confirm it is inside the configured subnet and not used, gateway, network, or broadcast. | | Create says TAP already exists | Inspect the interface; do not delete it unless you know it is stale UVM state. | | Existing bridge is rejected | Verify it is a software Linux bridge with STP disabled. | @@ -1187,7 +1215,7 @@ Keep these constraints in mind when deciding whether UVM fits a scenario: - No quotas or cgroup CPU, memory, I/O, or process limits are applied. - Fractional CPU requests are advisory only. - No autostart, background lifecycle/firewall reconciliation, or guest-coordinated stop. -- No UDP/IPv6 service policies, public connectivity setup, host-port mapping, +- No UDP/IPv6 service policies, public inbound connectivity setup, host-port mapping, DNS publication, guest reverse proxy/TLS setup, DHCP, load balancing, snapshots, multi-network tenancy, or multi-host scheduling. - The API uses one shared bearer token, not individual users, roles, or tenants. @@ -1220,7 +1248,7 @@ python3 -B -m unittest discover -s tests -v The opt-in packet test runs **only in disposable isolated user/network namespaces**, which may require privileges. It needs `unshare`, `ip`, -`bridge`, `nft`, `conntrack`, and `sysctl`: +`bridge`, `nft`, `iptables`, `conntrack`, and `sysctl`: ```sh UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration -v @@ -1228,9 +1256,10 @@ UVM_FIREWALL_INTEGRATION=1 python3 -B -m unittest discover -s tests/integration Namespace creation was denied in this workspace (`Operation not permitted`), so the packet test skipped. That is a permission limitation, not real-packet -or real-host validation. The harness uses veths as TAP stand-ins and routed -egress without NAT; it does not boot Firecracker or validate the host's real -TAP/NAT setup. Run actual VM/SSH/egress checks on an isolated Linux host before +or real-host validation. The harness uses veths as TAP stand-ins and simulated +upstreams to check NAT, offline startup, reconnects and uplink changes. It does +not boot Firecracker or validate the host's real TAP setup. Run actual +VM/SSH/egress checks on an isolated Linux host before relying on the deployment. ## 19. Get Help