feat: add host-backed entropy device support and update kernel configuration

DEVELOPER.md

@@ -490,7 +490,14 @@ The `firecracker/` package has three separate responsibilities.
 2. `PUT /boot-source`
 3. `PUT /drives/rootfs`
 4. `PUT /network-interfaces/eth0`
-5. `PUT /actions` with `InstanceStart`
+5. `PUT /entropy` with `{}` (a host-backed virtio RNG device)
+6. `PUT /actions` with `InstanceStart`
+
+Every generated config includes `entropy`, including retained-disk boots.
+An empty object enables the device with default settings; do not skip it with
+a truthiness check. A missing/invalid configuration or an entropy API failure
+must prevent `InstanceStart`. Without guest entropy, systemd's random-seed
+service and OpenSSH can remain in startup while the VM process and IP respond.
 
 The guest receives a static IP via the kernel `ip=` boot argument. The netmask
 comes from `Settings.network`, not a hard-coded `/24` value.
@@ -729,7 +736,14 @@ handling, downloads, and integrity-manifest publication.
 (`amd64` to `x86_64`, `arm64` to `aarch64`) and rejects unsupported architectures.
 The same architecture selects the Firecracker release archive, guest kernel,
 native rootfs packages, and architecture-specific boot arguments. The default
-kernel URL uses Firecracker's `img/quickstart_guide/<architecture>/kernels/vmlinux.bin`.
+kernel URL is pinned to Firecracker's published
+`firecracker-ci/v1.15/<architecture>/vmlinux-6.1.155` artifacts. The CI artifact
+series is independent of the VMM release. Both architecture configs include
+`CONFIG_HW_RANDOM=y` and `CONFIG_HW_RANDOM_VIRTIO=y`; custom kernels need these
+built-in drivers for UVM's entropy device. The previous ARM quickstart image
+was Linux 4.14.174 without the built-in virtio RNG driver. Updating the device
+configuration alone cannot repair that kernel; existing installs need a forced
+asset refresh with a matching new trusted kernel checksum, then a fresh VM boot.
 An empty `Settings.rootfs_url` selects the built Ubuntu 22.04 (Jammy) template.
 Explicit kernel/rootfs URLs and trusted digests must match the host architecture. ARM64
 uses an uncompressed Linux `Image` (PE format) and an ARM64 ext4 rootfs.

README.md

@@ -56,6 +56,17 @@ Spark, UVM uses ARM64 assets and packages.
 `x86_64`, respectively. Custom `UVM_KERNEL_URL` and `UVM_ROOTFS_URL` overrides
 must point to assets compatible with the host architecture.
 
+The default guest kernel is Linux 6.1.155 from Firecracker's published `v1.15`
+CI artifacts, with built-in `virtio-rng` support. UVM attaches a host-backed
+entropy device before each guest boot so services such as SSH can initialize
+their cryptographic random generator. Custom kernels must include
+`CONFIG_HW_RANDOM=y` and `CONFIG_HW_RANDOM_VIRTIO=y` for this device.
+
+Older installs used a Linux 4.14 ARM demo kernel and supplied no entropy
+device. If SSH stays at **Starting OpenBSD Secure Shell server** while ping
+works, see [SSH startup and guest entropy](USAGE.md#ssh-startup-and-guest-entropy)
+for the kernel-refresh and retained-VM recovery steps.
+
 ## Trusted Installation
 
 Installation requires trusted SHA-256 digests by default. Supply digests from

USAGE.md

@@ -169,7 +169,7 @@ UVM detects the host architecture and selects matching Firecracker and kernel
 downloads and native Ubuntu packages. NVIDIA DGX Spark uses `aarch64` artifacts
 and `arm64` packages; the `arm64` architecture alias is also accepted:
 
-- Kernel: `https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/aarch64/kernels/vmlinux.bin`
+- Kernel: `https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/aarch64/vmlinux-6.1.155`
 - Rootfs: Ubuntu 22.04 built with `debootstrap` from `ports.ubuntu.com/ubuntu-ports`.
 
 Use the installation commands below with checksums for the ARM64 kernel
@@ -180,6 +180,12 @@ On ARM64, the kernel must be an uncompressed ARM64 Linux `Image` (PE format),
 and the rootfs must be an ARM64 ext4 filesystem. UVM adds `keep_bootcon` to the
 guest boot arguments for ARM64 console output.
 
+Both architectures default to the pinned Linux 6.1.155 kernel from the published
+`v1.15` CI artifact series, independently of the Firecracker binary version.
+These kernels have `CONFIG_HW_RANDOM=y` and `CONFIG_HW_RANDOM_VIRTIO=y` built in.
+UVM configures Firecracker's entropy device before boot; custom kernels also
+need these drivers to receive host-backed randomness.
+
 ### Scenario: Strict First Installation
 
 Obtain trusted checksums independently, then run:
@@ -1222,6 +1228,58 @@ External firewall integration (including UFW, firewalld, Docker, and libvirt)
 is deferred. A UVM policy readback does not prove that external rules permit
 traffic, or that they will not later remove UVM's rules.
 
+### SSH Startup and Guest Entropy
+
+`running` checks the Firecracker process, and `enforced` checks the private
+firewall policy. Neither probes the guest's SSH listener. If ping works but
+TCP/22 returns `Connection refused`, inspect the guest boot log:
+
+```sh
+sudo ./uvm.py logs VM --tail 200
+sudo ./uvm.py inspect VM --json
+```
+
+An SSH service stuck at `Starting OpenBSD Secure Shell server...`, together
+with an unfinished `Load/Save Random Seed` service and no `random: crng init done`,
+strongly suggests entropy starvation. A login prompt can still appear while
+SSH is waiting for its cryptographic random generator. On the old ARM demo
+kernel, `random: fast init done` is not full cryptographic RNG initialization.
+
+Earlier UVM versions downloaded a Linux 4.14.174 ARM quickstart kernel without
+the built-in virtio RNG driver and did not attach an entropy device. Recovery
+requires **both** the updated UVM boot code and an RNG-capable kernel. A restart
+with the old cached kernel is insufficient.
+
+From the updated checkout, stop the affected VM before refreshing assets.
+Stop any other VMs using the same installed Firecracker binary before the
+forced reinstall as well. Use the Firecracker archive checksum for the configured
+release and a new trusted checksum for the selected architecture's Linux 6.1.155
+kernel; the old kernel checksum will not match:
+
+```sh
+sudo ./uvm.py stop VM
+sudo env -u UVM_KERNEL_URL \
+  UVM_FIRECRACKER_SHA256='<trusted-firecracker-archive-sha256>' \
+  UVM_KERNEL_SHA256='<trusted-new-kernel-sha256>' \
+  ./uvm.py install --force
+sudo ./uvm.py start VM
+sudo ./uvm.py logs VM --tail 100
+sudo ./uvm.py ssh VM
+```
+
+This uses the default kernel URL and boots the retained private disk with the
+updated device configuration. The VM's identity, files, credentials, and SSH
+host keys are retained. `install --force` also refreshes the shared rootfs
+template for future creates. If using a custom kernel, supply its URL and
+matching checksum with the required drivers instead. For an explicitly
+unverified development installation, retain the existing
+`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` setting as described in the installation section.
+
+Restart a running UVM API server to load the updated code before using dashboard
+start/restart actions. Look for `Started OpenBSD Secure Shell server` in the new
+boot log and confirm an actual SSH connection. `start` on an already-running
+VM only refreshes networking; use a stopped VM or `restart` to attach the device.
+
 ## 16. Important Limitations
 
 Keep these constraints in mind when deciding whether UVM fits a scenario:

tests/test_config.py

@@ -27,17 +27,17 @@ class ArchitectureSettingsTests(unittest.TestCase):
                     settings = factory()
 
                     base = (
-                        "https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/"
+                        "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/"
                         f"{architecture}"
                     )
                     self.assertEqual(settings.architecture, architecture)
-                    self.assertEqual(settings.kernel_url, f"{base}/kernels/vmlinux.bin")
+                    self.assertEqual(settings.kernel_url, f"{base}/vmlinux-6.1.155")
                     self.assertEqual(settings.rootfs_url, "")
 
     def test_preserves_individual_environment_url_overrides_on_arm64(self) -> None:
         kernel = "https://example.com/custom-arm64-kernel"
         rootfs = "https://example.com/custom-arm64-rootfs.ext4"
-        base = "https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/aarch64"
+        base = "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/aarch64"
         for overrides in (
             {"UVM_KERNEL_URL": kernel},
             {"UVM_ROOTFS_URL": rootfs},
@@ -52,7 +52,7 @@ class ArchitectureSettingsTests(unittest.TestCase):
 
                 self.assertEqual(
                     settings.kernel_url,
-                    overrides.get("UVM_KERNEL_URL", f"{base}/kernels/vmlinux.bin"),
+                    overrides.get("UVM_KERNEL_URL", f"{base}/vmlinux-6.1.155"),
                 )
                 self.assertEqual(
                     settings.rootfs_url,

tests/test_firecracker_config.py

@@ -3,9 +3,11 @@ from __future__ import annotations
 import unittest
 from ipaddress import IPv4Address, IPv4Network
 from pathlib import Path
+from unittest.mock import patch
 
 from uvm.config import Settings
 from uvm.domain import VmRecord
+from uvm.errors import FirecrackerError
 from uvm.firecracker.api import FirecrackerClient
 from uvm.firecracker.config import build_config, vcpu_count
 
@@ -45,6 +47,10 @@ class FirecrackerConfigTests(unittest.TestCase):
                 self.assertEqual("keep_bootcon" in arguments, architecture != "x86_64")
                 self.assertIn("console=ttyS0", arguments)
                 self.assertIn("ip=10.42.0.2::10.42.0.1:255.255.255.0::eth0:off", arguments)
+                client = RecordingFirecrackerClient()
+                client.configure_and_start(config)
+                self.assertEqual(client.calls[-2], ("PUT", "/entropy", {}))
+                self.assertEqual(client.calls[-1], ("PUT", "/actions", {"action_type": "InstanceStart"}))
 
     def test_builds_a_writable_vm_disk_configuration(self) -> None:
         vm = VmRecord(
@@ -101,6 +107,7 @@ class FirecrackerConfigTests(unittest.TestCase):
                 "/boot-source",
                 "/drives/rootfs",
                 "/network-interfaces/eth0",
+                "/entropy",
                 "/actions",
             ],
         )
@@ -110,6 +117,36 @@ class FirecrackerConfigTests(unittest.TestCase):
         )
         self.assertEqual(client.calls[-1][2], {"action_type": "InstanceStart"})
 
+    def test_entropy_failure_prevents_guest_boot(self) -> None:
+        config = {
+            "machine-config": {"vcpu_count": 1, "mem_size_mib": 512},
+            "boot-source": {},
+            "drives": [{}],
+            "network-interfaces": [{}],
+            "entropy": {},
+        }
+        client = RecordingFirecrackerClient()
+
+        def request(method: str, path: str, body: object) -> None:
+            client.calls.append((method, path, body))
+            if path == "/entropy":
+                raise FirecrackerError("entropy device unavailable")
+
+        with patch.object(client, "_request", side_effect=request):
+            with self.assertRaisesRegex(FirecrackerError, "entropy device unavailable"):
+                client.configure_and_start(config)
+
+        self.assertEqual(client.calls[-1], ("PUT", "/entropy", {}))
+        self.assertNotIn("/actions", [path for _method, path, _body in client.calls])
+
+        for entropy in (None, [], False):
+            with self.subTest(entropy=entropy):
+                client.calls.clear()
+                config["entropy"] = entropy
+                with self.assertRaisesRegex(FirecrackerError, "no valid entropy device"):
+                    client.configure_and_start(config)
+                self.assertEqual(client.calls, [])
+
     def test_uses_the_configured_network_netmask_in_guest_boot_arguments(self) -> None:
         settings = Settings(
             network=IPv4Network("10.50.0.0/16"),

tests/test_install.py

@@ -72,7 +72,8 @@ class InstallerArchitectureTests(unittest.TestCase):
                 payloads = {
                     "https://github.com/firecracker-microvm/firecracker/releases/download/"
                     f"{version}/firecracker-{version}-{architecture}.tgz": archive_bytes,
-                    f"{guest_base}/kernels/vmlinux.bin": kernel_bytes,
+                    "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/"
+                    f"{architecture}/vmlinux-6.1.155": kernel_bytes,
                     f"{guest_base}/rootfs/bionic.rootfs.ext4": rootfs_bytes,
                 }
                 runner = Mock(spec=CommandRunner)

uvm/config.py

@@ -10,7 +10,9 @@ from pathlib import Path
 from .errors import ConfigurationError
 
 
-DEFAULT_GUEST_ASSETS_URL = "https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide"
+# Pin published kernels with built-in virtio-rng support. The artifact series
+# is independent of the VMM release; the old ARM quickstart kernel lacks this driver.
+DEFAULT_GUEST_ASSETS_URL = "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15"
 MIN_STORAGE_MIB = 30 * 1024
 MAX_STORAGE_MIB = (2**63 - 1) // (1024 * 1024)
 
@@ -55,7 +57,7 @@ class Settings:
         # Resolve defaults at construction time, while preserving individual URL overrides.
         asset_base = f"{DEFAULT_GUEST_ASSETS_URL}/{architecture}"
         if not self.kernel_url:
-            object.__setattr__(self, "kernel_url", f"{asset_base}/kernels/vmlinux.bin")
+            object.__setattr__(self, "kernel_url", f"{asset_base}/vmlinux-6.1.155")
 
         if self.gateway not in self.network:
             raise ConfigurationError(

uvm/firecracker/api.py

@@ -35,12 +35,15 @@ class FirecrackerClient:
         boot = config["boot-source"]
         drives = config["drives"]
         interfaces = config["network-interfaces"]
+        entropy = config.get("entropy")
         if not isinstance(machine, Mapping) or not isinstance(boot, Mapping):
             raise FirecrackerError("invalid Firecracker configuration")
         if not isinstance(drives, list) or not drives:
             raise FirecrackerError("Firecracker configuration has no root drive")
         if not isinstance(interfaces, list) or not interfaces:
             raise FirecrackerError("Firecracker configuration has no network interface")
+        if not isinstance(entropy, Mapping):
+            raise FirecrackerError("Firecracker configuration has no valid entropy device")
 
         self._request(
             "PUT",
@@ -54,6 +57,7 @@ class FirecrackerClient:
         self._request("PUT", "/boot-source", boot)
         self._request("PUT", "/drives/rootfs", drives[0])
         self._request("PUT", "/network-interfaces/eth0", interfaces[0])
+        self._request("PUT", "/entropy", entropy)
         self._request("PUT", "/actions", {"action_type": "InstanceStart"})
 
     def _request(self, method: str, path: str, body: object) -> None:

uvm/firecracker/config.py

@@ -70,6 +70,8 @@ def build_config(
                 "host_dev_name": vm.tap,
             }
         ],
+        # Supply host-backed randomness before guest services such as sshd start.
+        "entropy": {},
     }