# UVM Local Firewall Design > Status: proposed design. This document describes a host-local firewall layer > to be implemented after the VM lifecycle and registry work. It does not > describe functionality currently available in `uvm`. ## 1. Decision UVM should use **native nftables**, managed locally through an internal Python adapter that invokes the `nft` command through the existing `CommandRunner`. The initial firewall layer will enforce per-VM **private IPv4 TCP allowlists** at the UVM bridge/TAP boundary. It will not require a cloud firewall, remote agent, external controller, or long-running UVM firewall daemon. This choice fits UVM because nftables provides: - Native Linux bridge filtering, which sees VM-to-VM traffic on `uvm0`. - Stateful connection tracking in the bridge family on supported kernels. - Atomic rule transactions through `nft --file -`. - Named sets and a dedicated table, allowing UVM to update only resources it owns. - A local CLI and JSON inspection interface that can be used safely through UVM's existing subprocess boundary. The recommended implementation is a focused adapter, not a general firewall framework or a public Python SDK. ## 2. Scope ### In scope - Enforce a per-VM TCP allowlist for traffic to the VM's private IPv4 address. - Cover host-to-VM and VM-to-VM traffic on the existing UVM bridge. - Preserve ordinary guest-initiated egress and replies to that egress. - Make `close` block subsequent packets on existing service connections as well as new connections. - Install the policy before a VM TAP becomes usable. - Let explicit lifecycle or policy-sync operations reconcile UVM-owned firewall state from the local registry after interruption or firewall-rule loss. - Work entirely on the Linux host that runs UVM. ### Explicitly out of scope - Public IP allocation, port forwarding, DNAT, reverse proxies, DNS, TLS termination, load balancing, or cloud security groups. - A remote firewall API, external controller, or multi-host coordination. - Automatic integration with Docker, UFW, firewalld, Kubernetes, libvirt, or a site-managed firewall. Those need separately tested integration profiles. - IPv6 service policies, UDP service policies, source-CIDR ACLs, rate limits, or egress restrictions in the first version. - A claim that the current Firecracker launch model is a hardened, multi-tenant isolation boundary. Opening TCP/80 in this design means that TCP/80 is reachable at the guest's private UVM address by permitted local paths. It does not create a public listener, start a guest service, configure TLS, or bypass a guest firewall. ### Supported topology profile The first version supports a UVM-dedicated private bridge only. Preflight must verify all of these conditions before activating a managed TAP: - `UVM_NETWORK` is wholly contained in one RFC1918 range and does not overlap another local address or non-default route, apart from the expected UVM bridge address and connected bridge route. The host's generic default route is expected and is not a conflict. - `UVM_BRIDGE` has only UVM-owned TAP member ports. It has no physical uplink, foreign veth, libvirt, container, or otherwise unknown bridge port. - New inbound service connections originate from the UVM bridge gateway or a valid address in `UVM_NETWORK`. This is a fixed private-path boundary, not a user-configurable source-CIDR feature. - UVM itself installs no DNAT, port-forward, or public-address rule. The source restriction permits host-to-guest traffic, whose source is normally the bridge gateway address, and VM-to-VM traffic. It does not affect replies to guest-initiated external connections, which are identified as reply traffic through connection tracking. A private RFC1918 address is not itself a complete public-exposure guarantee. An administrator can still add host routing, source NAT, DNAT, or another firewall rule that changes reachability. UVM does not discover or override such external policy; it only guarantees the scoped bridge policy after it has successfully applied and verified it. ## 3. Existing UVM Topology UVM currently creates one TAP per Firecracker VM and attaches every TAP to the same Linux bridge. The host bridge address is the guest default gateway. ```text guest eth0 | Firecracker virtio-net | per-VM TAP: uvm- | Linux bridge: uvm0 (10.42.0.1/24 by default) | host IPv4 routing and existing egress NAT | host uplink ``` Relevant current code: - [`uvm/network.py`](uvm/network.py) allocates guest addresses and MACs, creates the bridge/TAPs, enables IPv4 forwarding, and installs the current egress `iptables` rules. - [`uvm/lifecycle.py`](uvm/lifecycle.py) coordinates VM creation, rollback, stop, and destruction under the shared operation lock. - [`uvm/state.py`](uvm/state.py) owns the atomic registry and the operation lock. - [`uvm/system.py`](uvm/system.py) provides the testable `CommandRunner` boundary for local host commands. An IP-family `FORWARD` rule alone is insufficient for the future port policy. Traffic between two VM TAPs is normally switched by the Linux bridge at layer 2 and does not traverse an `inet`/`ip` forwarding hook. Host-to-guest traffic also follows bridge output rather than the ordinary routed-forward path. Native nftables bridge filtering is designed for these paths and does not require enabling the legacy `br_netfilter` module. The Linux bridge documentation specifically discourages `br_netfilter` for new filtering designs in favor of nftables. ## 4. Candidate Evaluation All options below are open source and can run entirely on the local Linux host. The comparison is about fit for UVM's intended lifecycle-managed policy, not whether the tool is generally useful. | Option | Local operation | Advantages | Limitations for UVM | Decision | |---|---|---|---|---| | Native nftables | Kernel Netfilter plus local `nft` CLI; no daemon required | Bridge-family hooks, conntrack, named sets, atomic batches, precise ownership | Requires a supported kernel and careful coexistence rules | Adopt | | `iptables-nft` compatibility frontend | Local CLI backed by nftables | Already used by existing UVM egress code; familiar syntax | Does not give a clean bridge-policy model and makes native ownership/order harder to reason about | Do not extend for the new layer | | `iptables-legacy` plus ebtables/ipset | Local CLI tools | Available on older systems | Separate legacy data path, incremental updates, more components, and bridge IP filtering commonly depends on legacy `br_netfilter` behavior | Do not use | | firewalld | Local daemon with D-Bus API | Zones, policies, IP sets, local programmatic control | Daemon ownership, runtime/permanent configuration, and policy interactions add operational complexity; it is not optimized for UVM's per-TAP hard-close contract | Optional future adapter only | | UFW | Local Ubuntu-oriented CLI | Simple operator workflow and routed rules | CLI/rule-number model is not a suitable authoritative dynamic policy API; reload/default-policy changes have broad effects | Do not use as UVM's backend | | eBPF/XDP or tc/eBPF | Local kernel programs and maps | High performance and dynamic maps | Requires a custom data plane, attachment/pinning lifecycle, verifier-compatible programs, and does not naturally provide UVM's conntrack semantics | Future performance option, not v1 | ### Why not reuse the current iptables implementation? The existing `NetworkManager._ensure_masquerade()` method incrementally adds egress NAT and forwarding rules. It does not manage a UVM-owned firewall table, does not filter bridge-forwarded VM-to-VM frames, and does not provide atomic whole-policy replacement. On systems where `iptables` points to the nftables compatibility backend, native nftables and `iptables-nft` share the kernel nftables data plane but still create separately managed rule structures. Adding the new policy through native nftables while leaving current egress rules unchanged is the smallest safe migration. A later, separately tested change may migrate UVM's existing egress rules into a dedicated nftables table. UVM must never change a system-wide `iptables` alternative, flush a global ruleset, or adopt unrelated rules just because they resemble UVM's topology. ### Why not use firewalld or UFW as the primary backend? firewalld is a valid local-only solution on hosts where it is already the authoritative firewall manager. Its D-Bus API and policy model make a future adapter possible. It adds a daemon and a separate permanent/runtime configuration model, however, and a firewalld reload intentionally preserves established connections. That does not by itself meet UVM's close-port contract. UFW is well suited to static host policy and can express routed rules. Its own manual states that its command interface is not intended to expose complete firewall functionality. Dynamic per-VM reconciliation would depend on parsing or manipulating a higher-level CLI-managed ruleset, and it still would not solve bridge-path filtering cleanly. If a host already uses firewalld or UFW, UVM must not write that manager's generated tables directly. The operator must choose and test a dedicated integration profile before using UVM's firewall layer on that host. ## 5. Recommended Security Contract The v1 policy is deliberately narrow: | Item | Contract | |---|---| | Protocol | IPv4 TCP only | | Inbound access | Default deny for managed VMs; only explicitly allowed guest TCP ports pass | | Default new-VM policy | Persist TCP/22 as an ordinary initial allowlist entry unless creation explicitly disables default ports | | Egress | Guest-initiated TCP, UDP, DNS, and required control traffic remain allowed | | Existing service connections after close | Subsequent packets are blocked in both directions after the policy update commits | | IPv6 | Drop IPv6 traffic on managed TAPs until an explicit IPv6 policy exists | | Guest identity | Bind each managed TAP to its recorded MAC and IPv4 address; reject spoofed source identity | | New inbound source | Only the UVM bridge gateway or an address in `UVM_NETWORK` may initiate a service connection | | Legacy records | A missing policy is `unmanaged`, not an implicit allow-all or deny-all policy | The policy must distinguish an explicit empty list from a legacy record: ```text open_tcp_ports = None -> legacy/unmanaged; do not claim enforcement open_tcp_ports = [] -> managed deny-all inbound policy open_tcp_ports = [22] -> managed policy allowing SSH only ``` HTTP and HTTPS are user-facing labels for TCP/80 and TCP/443. They do not change packet treatment. An active unmanaged VM may not share `uvm0` with managed VMs. Before managed enforcement is enabled, all active legacy records must be stopped and migrated to an explicit policy, or the operation must fail. This avoids treating an unfiltered legacy TAP as a trustworthy peer for MAC/IP/ARP identity checks. ### What close guarantees When a managed VM closes TCP/443, the resulting policy must match both: - Original-direction packets traveling to that VM's TCP/443 service. - Reply-direction packets leaving that service on an already established flow. The implementation must not place a broad `ct state established,related accept` rule before this service-policy check. Instead, it checks the connection's original destination address and destination port for every service-flow packet. This gives the following guarantee: > Once the atomic firewall update commits, later packets traversing the VM's > service path are blocked while the port remains closed. It cannot retract data already delivered, gracefully close a guest process, or guarantee an immediately visible close for a completely idle TCP socket. The design may use `reject with tcp reset` after packet-path tests establish that the behavior is appropriate; a drop rule is the conservative baseline. Connection tracking must remain available while enforcing a live close. A targeted `conntrack` cleanup is useful before a deleted VM IP is reused, but it is not the primary live-close mechanism. Deleting a conntrack entry too early can discard the original tuple needed to distinguish a service reply from a new guest-originated connection. ## 6. nftables Architecture UVM will own one dedicated bridge-family table, conceptually: ```text table bridge uvm ``` The table contains only UVM-generated rules, sets, and chains. It must never use `flush ruleset`, modify a built-in host policy, or clear tables owned by a different tool. ```mermaid flowchart LR LIFE[LifecycleService] --> FIREWALL[NftFirewall] LIFE --> NETWORK[NetworkManager] FIREWALL --> RUNNER[CommandRunner] RUNNER --> NFT[nftables bridge table] NETWORK --> BRIDGE[uvm0 bridge and TAPs] NFT --> BRIDGE BRIDGE --> VM1[Firecracker VM A] BRIDGE --> VM2[Firecracker VM B] ``` ### Hook Coverage | Bridge hook | UVM use | Required policy action | |---|---|---| | `prerouting` | Frame arrives from a managed TAP | Validate expected source MAC/IP before bridge learning; allow only necessary ARP; reject unsupported IPv6 | | `input` | VM frame enters the host stack, including the gateway path | Apply the VM's source-side policy and allow guest-originated traffic/relevant replies | | `output` | Host stack sends a frame toward a VM | Apply the destination VM's inbound policy or allow a reply to guest-originated traffic | | `forward` | Frame moves directly between bridge ports | Apply both source-VM and destination-VM policy; never let one VM's acceptance skip the other VM's check | The base chains have an `accept` policy so that UVM does not impose a global deny policy on non-UVM bridge ports. The UVM dispatch rules select only known, registry-backed TAP names. A denial occurs inside a selected VM's policy path. Once a frame is selected as to or from a managed TAP, an unmatched frame is a terminal denial. The v1 policy permits only untagged IPv4 and the narrowly required ARP traffic on that attachment. It drops IPv6, VLAN-tagged frames, and other unsupported EtherTypes in both directions. This is separate from the base-chain policy for non-UVM bridge ports. For VM-to-VM traffic, dispatch into source and destination policy chains must use `jump` and `return` semantics. A source-side success returns to the dispatcher so the destination VM's inbound policy still runs. It must not use an early terminal `accept` that bypasses the destination check. ### Per-VM Rule Model Each managed VM contributes a typed policy record: ```text VM ID TAP name guest IPv4 address guest MAC address open TCP ports runtime mode: active or quarantined ``` The generated rules maintain two logical checks: | Chain role | Original-direction traffic | Reply-direction traffic | |---|---|---| | Toward VM | Allow a new TCP flow only from the private-path sources and when its destination port is in the VM's current allowlist | Allow a reply only when it belongs to a guest-originated connection | | From VM | Allow guest-originated traffic | Allow a service reply only when the original destination VM/port is still in the allowlist | The implementation uses connection tracking's original tuple, rather than the packet's current source/destination port alone, for service replies. For example, a web server reply is sent *from* TCP/443 to a client ephemeral port; the original tuple still identifies TCP/443 as the guest service. The actual nft syntax is an implementation detail and must be generated from typed data. It will use `ct direction`, `ct original ip daddr`, and `ct original proto-dst` only after an isolated integration test verifies the target kernel's bridge conntrack behavior. ### Identity and Layer-2 Controls Filtering only TCP ports is not enough if a guest can spoof another guest's source identity. For every managed TAP, the policy must: - Accept a frame from the TAP only when its Ethernet source equals the persisted guest MAC. - Accept an IPv4 frame from the TAP only when its source equals the persisted guest IPv4 address. - Bind ARP sender hardware and protocol addresses from the TAP to that same MAC/IP pair; allow only the required target/address-probe forms after packet testing. - Drop IPv6, tagged Ethernet frames, and unsupported EtherTypes both to and from the managed TAP while they have no corresponding policy. The identity check belongs before or at the earliest bridge ingress point. It must be validated against the actual bridge forwarding database behavior on the target kernel. This is meaningful network-policy hardening, but it does not turn UVM into a complete hostile-tenant platform. Stronger tenant isolation may eventually require per-VM L2/L3 segments, bridge-port isolation, VLANs, or network namespaces. ### Ownership and Rule Rendering UVM owns only its dedicated table and resources carrying a fixed versioned UVM marker. The adapter must: 1. Refuse an unexpected pre-existing UVM table or an ownership-marker mismatch rather than adopting it. 2. Render the complete desired contents of its own table from current state. 3. Apply the generated table in one nftables transaction. 4. Inspect the owned table after a mutation when a stronger confirmation is required. 5. Remove a per-VM dispatch entry only after its TAP is confirmed absent or safely quarantined. The exact table replacement batch must be tested against the supported nft version. It may rebuild only `table bridge uvm`; it must never rebuild the host's whole nftables ruleset. ## 7. Persistence and Reconciliation The JSON registry remains the source of desired policy. Kernel firewall state is observed state and is never treated as durable truth. The future `VmRecord` model needs to persist at least: ```text name (if VM naming is added) open_tcp_ports: list[int] | None policy_intent / pending lifecycle phase policy generation ``` `None` preserves the legacy/unmanaged distinction. New VMs always have an explicit policy, including a deliberately empty policy. The registry also needs a topology fingerprint at the state level containing the configured network CIDR, gateway, and bridge name. Lifecycle and firewall preflight compare the current `Settings` against that persisted fingerprint before touching interfaces or rules. A legacy registry without this fingerprint requires an explicit migration; it must not infer that a changed `/24` and `/16`, for example, describe the same UVM topology. UVM should not persist an authoritative `policy_enforced: true` boolean. It would become stale after a reboot, manual firewall change, partial failure, or host firewall reload. Instead, `NftFirewall.reconcile()` renders the desired state from the registry and inspects the UVM-owned table when needed. Reconciliation has no continuous trigger in this design. It runs during create, start, port mutations, and an explicit future policy-sync operation. Read-only list/inspect commands may report an owned table as missing or unverified, but must not silently mutate the host. If an active VM loses its UVM firewall table between operations, its policy is unknown until a successful explicit reconciliation; UVM must not claim it remains enforced. ### Mutation Ordering All firewall mutations use the existing `operations.lock`. State and kernel updates cannot be one cross-resource transaction, so durable intent and observed enforcement must remain distinct: | Action | Required order | |---|---| | Create | Persist explicit desired policy, create the TAP down, install/verify policy, activate TAP, then launch Firecracker | | Start | Reconcile the saved policy before activating the retained TAP or launching a new VMM | | Open port | Persist desired policy first, atomically apply it, then clear the pending state | | Close port | Persist the more restrictive desired policy first, atomically apply it, then clear the pending state | | Stop | Persist stop intent, quarantine the TAP, confirm VMM exit, remove/deactivate TAP, retain saved policy | | Delete | Quarantine, confirm exit, remove TAP, remove UVM policy for the VM, clear stale flow state before IP reuse, then remove registry data | If an open operation cannot be applied, the desired state may say "open" while the host remains more restrictive. UVM reports failure and retries through reconciliation; it must not claim the port is open. If a close operation cannot be confirmed, UVM retains its restrictive intent and attempts to quarantine the attachment rather than restoring the old policy. There is an unavoidable crash window after restrictive intent is persisted and before the nftables transaction commits. During that window, an already-running VM can still have its old permissive kernel policy. An interrupted close stays pending and is not reported as enforced until an explicit reconciliation or quarantine successfully commits. The operation lock cannot close this window after the UVM process exits. ### Quarantine An empty inbound allowlist is not a quarantine because it still permits a VM to initiate outbound traffic. A quarantine mode drops traffic both to and from the owned TAP. It is required while a stop, delete, or failed boot leaves process/TAP ownership unresolved. Do not remove this guard merely because a cleanup operation was attempted. It remains until the TAP is gone or a successful start replaces it with the saved active policy. ## 8. Python Integration ### Internal Adapter Boundary Add a focused internal module such as `uvm/firewall.py`. It should be constructed in [`uvm/app.py`](uvm/app.py), similarly to the current `NetworkManager` and Firecracker process manager. ```python firewall = NftFirewall(settings, runner) lifecycle = LifecycleService(settings, state_store, images, network, process, firewall) ``` The exact constructor arrangement can vary, but these boundaries must remain: | Component | Responsibility | |---|---| | `LifecycleService` | Owns ordering, operation locking, persistence intent, rollback, and VM status transitions | | `NetworkManager` | Owns bridge validation plus TAP preparation, activation, and deletion | | `NftFirewall` | Owns only UVM's generated nftables policy and its local inspection/reconciliation | | `CommandRunner` | Runs local commands without a shell and makes command construction testable | | CLI/API routers | Validate user input and call services; never send raw firewall syntax or commands | The adapter should expose typed operations, for example: ```python @dataclass(frozen=True, slots=True) class VmFirewallPolicy: vm_id: str tap: str guest_ip: IPv4Address guest_mac: str open_tcp_ports: tuple[int, ...] quarantined: bool = False class NftFirewall: def preflight(self) -> None: ... def reconcile(self, policies: Iterable[VmFirewallPolicy]) -> FirewallObservation: ... def inspect(self) -> FirewallObservation: ... def purge_vm_flows(self, vm: VmFirewallPolicy) -> None: ... ``` The public surface should accept domain objects, not interface names or raw rules supplied by a CLI/API caller. There should be no `apply_raw`, `flush`, or generic arbitrary-command method. ### Command Transport The first implementation should call the local `nft` executable through `CommandRunner`, rather than add a Python package dependency: ```python runner.run( ("nft", "--check", "--file", "-"), input_text=ruleset, capture=True, timeout=settings.firewall_timeout_s, ) runner.run( ("nft", "--file", "-"), input_text=ruleset, capture=True, timeout=settings.firewall_timeout_s, ) ``` This follows the same engineering model as Firecracker's Unix-socket adapter: a narrow typed UVM boundary hides an external host interface. For nftables, the host interface is a local command/Netfilter transaction rather than HTTP. `nft --check` and `nft --file -` are separate operations. The checked batch is only a validation aid; UVM may claim a policy change only after the apply transaction succeeds and the adapter records its observation. Benefits of the CLI approach: - No shell interpolation or raw shell scripts. - Atomic batch application through `nft --file -`. - Syntax/preflight checking through `nft --check`. - Easy fake-runner unit tests and visible CLI traces. - No additional Python runtime dependency. Inspection should use JSON rather than human-formatted output: ```python runner.run( ("nft", "--json", "list", "table", "bridge", "uvm"), capture=True, timeout=settings.firewall_timeout_s, ) ``` `python3-nftables`/libnftables is a viable future transport if avoiding a subprocess becomes necessary. It must not be introduced casually: package API and kernel compatibility need a version-pinned integration suite. Direct Netlink programming is not justified for the first implementation. ### Errors Introduce a `FirewallError(UvmError)` for semantic firewall failures such as unsupported local capabilities, unexpected ownership, or an unsafe observed table. Preserve `CommandError` for an `nft` or `conntrack` command failure. Errors must include the failed operation and safe diagnostics, but never include raw untrusted rule input because the adapter has none. The CLI and API can retain their existing `UvmError` rendering conventions. ## 9. Installation and Host Requirements The installer remains the only component that invokes `apt-get`. The future package list should add: ```text nftables conntrack ``` `conntrack` is used for targeted stale-flow cleanup before a deleted VM IP is reallocated. It is not a substitute for the firewall's live close policy. On Ubuntu-compatible hosts, the installation step is conceptually: ```sh apt-get install --yes nftables conntrack ``` Do not automatically enable `nftables.service`, write `/etc/nftables.conf`, or install a system-wide persistence service. UVM dynamically owns only its dedicated table and recreates/reconciles it during lifecycle operations. ### Required Host Capabilities | Requirement | Reason | |---|---| | Linux with native nftables support | UVM uses local Netfilter tables | | Kernel with bridge-family connection tracking support | Required for original-tuple service checks; upstream support starts in Linux 5.3 | | `nft`, `conntrack`, and `ip` commands | Local policy, selected flow cleanup, and TAP/bridge management | | `CONFIG_BRIDGE` and `CONFIG_TUN` | Existing UVM bridge/TAP topology | | Netfilter nftables, conntrack, bridge, reject, and relevant protocol modules | Required to load/apply the generated table | | Root or `CAP_NET_ADMIN` in the bridge network namespace | Interface and firewall administration | | No flowtable/hardware offload that bypasses this filter path | Required for reliable close enforcement | Preflight should verify the actual generated ruleset with `nft --check` on the target host. Kernel-version checks alone are not enough; distributions can build or package Netfilter capabilities differently. UVM must not enable `br_netfilter` or change `net.bridge.bridge-nf-call-*` sysctls. Native bridge nftables works without that legacy compatibility layer. ## 10. Configuration Approach The first version should avoid a generic `UVM_FIREWALL_RULES` setting or any configuration value containing raw nftables syntax. The generated policy derives from existing UVM configuration and state: | Source | Firewall use | |---|---| | `UVM_NETWORK` | Validates guest IPv4 addresses and UVM subnet membership | | `UVM_GATEWAY` | Establishes the bridge/guest topology for ARP and gateway traffic | | `UVM_BRIDGE` | Selects the UVM bridge and its managed TAP attachments | | `VmRecord.tap`, `VmRecord.mac`, `VmRecord.guest_ip` | Binds policy to one owned VM attachment | | Persisted `open_tcp_ports` | Creates the per-VM TCP allowlist | One bounded command timeout, such as `UVM_FIREWALL_TIMEOUT_S`, is reasonable as a `Settings` value. It should default to a small value and apply only to UVM's local command invocations. Changing `UVM_NETWORK`, `UVM_GATEWAY`, or `UVM_BRIDGE` after VMs exist is not a firewall migration mechanism. UVM must fail rather than infer ownership of old interfaces or rules from changed environment values. ## 11. Lifecycle Integration The existing `NetworkManager.create_tap()` activates a TAP immediately. The firewall layer requires that operation to be split into preparation and activation: ```text create TAP down -> attach to bridge -> install policy -> bring TAP up ``` ### Create and Start ```mermaid sequenceDiagram participant L as LifecycleService participant S as StateStore participant N as NetworkManager participant F as NftFirewall participant P as Firecracker L->>S: persist VM and explicit policy intent L->>N: ensure bridge; prepare TAP down L->>F: reconcile active policy including new VM F-->>L: policy confirmed L->>N: activate TAP L->>P: spawn, configure, and start VMM L->>S: persist running state ``` A policy failure prevents TAP activation and Firecracker launch. A startup failure after policy installation leaves the policy in place until lifecycle cleanup confirms that the TAP is gone. This is safer than briefly booting an unfiltered guest. ### Stop, Delete, and Failure Cleanup ```text stop/delete intent -> quarantine owned TAP -> terminate verified Firecracker process and confirm exit -> deactivate/delete TAP -> stop: retain desired port policy for a later start -> delete: remove VM policy, purge stale flows, remove disk/state ``` If process identity or TAP ownership is unresolved, keep the quarantine guard, disk, and registry entry. Do not delete a possibly live VM disk or remove its network guard simply because a cleanup command raised an error. Shared table infrastructure may remain after the last VM is deleted. It is not a resource leak: it is a UVM-owned baseline that can be reconciled by the next lifecycle operation. It must contain no stale per-VM dispatch/policy entries after successful deletion. ## 12. Coexistence and Security Boundaries The initial supported profile is a host where UVM may own `table bridge uvm` and no other component modifies that table, using the dedicated private bridge profile described above. This does not mean UVM owns the host firewall. In particular: - A terminal `drop` in UVM's bridge table can protect a UVM TAP. - An `accept` in one nftables base chain does not promise that later hooks or another firewall manager will accept the packet. - UVM must not change global default policies, Docker chains, firewalld zones, UFW framework files, or a host's `nftables.service` configuration. - If Docker, Podman, UFW, firewalld, libvirt, or site firewall tooling is active, allowed traffic may still be blocked by that tooling. UVM should report its own enforcement state honestly rather than claim reachability. The present design controls the local private network path only. It does not protect against a privileged host process, a guest root exploit, raw host network administration, or future external DNAT rules. Source-IP allowlists and public exposure, if added, require a separate `inet`-family and ingress design. ## 13. Testing and Release Gates ### Unit Tests Add a focused `tests/test_firewall.py` with a stateful fake runner. Test: - Generated table ownership, no `flush ruleset`, and no built-in chain policy changes. - Exact TAP-scoped dispatch and VM policy rendering. - One atomic batch per policy update. - Validation failures before host mutation. - Idempotent reconciliation and recovery from missing UVM-owned rules. - Refusal to adopt an unexpected table or marker mismatch. - No broad `established,related` acceptance before service policy checks. - Close-policy rendering that blocks both request and reply service packets. - Quarantine and policy retention across stop/start/delete. - Lifecycle ordering: policy before TAP activation and Firecracker spawn. - Failure injection at state, nft, process, TAP, and cleanup boundaries. ### Isolated Network-Namespace Proof Before integrating with real Firecracker, build a disposable network-namespace test using a bridge and veth peers standing in for TAPs. It must prove: 1. Host-to-VM traffic reaches an allowed TCP port and fails at a closed port. 2. VM-to-VM traffic is governed by the destination VM's policy. 3. Guest-initiated TCP/UDP egress and replies still work. 4. Closing a port during one-way server streaming blocks subsequent reply packets as well as new requests. 5. A guest outbound connection using a source port equal to a closed service port is not falsely blocked as inbound service traffic. 6. ARP/gateway behavior remains functional. 7. IPv6 cannot bypass the IPv4-only policy. 8. MAC/IP spoofing from a managed TAP is rejected. 9. Tagged IPv4/IPv6 and unsupported EtherTypes are denied at managed TAPs. 10. An active unmanaged TAP prevents managed-policy activation. 11. Unrelated bridge ports and host firewall tables are unaffected. Only after that proof should an isolated Firecracker test verify real TAP/MAC behavior, default SSH access, an allowed guest listener, and live close. ## 14. References - [Linux bridge Netfilter documentation](https://docs.kernel.org/networking/bridge.html#netfilter) - [nftables bridge filtering](https://wiki.nftables.org/wiki-nftables/index.php/Bridge_filtering) - [nftables atomic rule replacement](https://wiki.nftables.org/wiki-nftables/index.php/Atomic_rule_replacement) - [nftables connection-tracking metadata](https://wiki.nftables.org/wiki-nftables/index.php/Matching_connection_tracking_stateful_metainformation) - [firewalld concepts and policies](https://firewalld.org/documentation/concepts.html) - [UFW manual](https://manpages.ubuntu.com/manpages/noble/man8/ufw.8.html) - [iptables manual](https://man7.org/linux/man-pages/man8/iptables.8.html) - [Linux `bpf(2)` manual](https://man7.org/linux/man-pages/man2/bpf.2.html)