kstyagi/nest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Enhance UVM installation and configuration for Ubuntu 22.04 root filesystem

Browse Source 
- Update README and USAGE documentation to reflect changes in rootfs handling, including the default local build from Ubuntu 22.04 packages.
- Modify the installer to support building the rootfs locally with authenticated packages and to require SHA-256 checksums for prebuilt images.
- Implement checks for setuid permissions on sudo and update the provisioning logic for guest users to ensure proper sudo access.
- Adjust tests to validate the new rootfs building process and ensure that existing functionality is preserved.
- Refactor code to improve clarity and maintainability, including the introduction of constants for Ubuntu release and guest packages.
This commit is contained in:
kstyagi
2026-09-22 00:33:05 +05:30
parent 71425a98e3
commit 4bf4c0853c
10 changed files with 523 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DEVELOPER.md
+41 -7
View File
@@ -545,6 +545,13 @@ The new `/home/USERNAME` directory is mode `0700`, owned by the new UID/GID;
Directory ownership/modes and account-file writes are read back before the
disk can be marked ready. The default `root` account must already exist.
Custom login users are VM administrators. Provisioning requires a root-owned,
setuid `/usr/bin/sudo` and `/etc/sudoers`, and appends a user-scoped `PASSWD: ALL`
rule with root/target/runas-password defaults disabled. Sudo authenticates with
the login user's password; root's separate shadow entry is not changed. Global
sshd directives precede managed and existing `Match` blocks so standard Ubuntu
`UsePAM`/`Subsystem` settings remain valid.
UVM replaces conventional RSA, ECDSA, and Ed25519 host keys in each private
disk and removes the known public login key from Firecracker's bionic image.
A VM must never rely on reusable SSH private keys from a shared template.
@@ -557,14 +564,19 @@ A VM must never rely on reusable SSH private keys from a shared template.
- A local `integrity.json` manifest recording artifact hashes and whether the
installation was verified.
Strict installation requires all of these values:
Strict installation requires these values for downloaded artifacts:
```text
UVM_FIRECRACKER_SHA256
UVM_KERNEL_SHA256
UVM_ROOTFS_SHA256
```
The default rootfs is built from signature-verified Ubuntu packages and its
resulting digest is recorded in the same manifest. `UVM_ROOTFS_URL` explicitly
selects a prebuilt ext4 download and additionally requires `UVM_ROOTFS_SHA256`.
A rootfs checksum without a URL is rejected during installation to avoid using
a stale Bionic-image checksum for a local build.
`UVM_ALLOW_UNVERIFIED_DOWNLOADS=1` is a local-development escape hatch. It
writes a manifest marked `verified: false`; a later strict run will not treat
that manifest as publisher trust. Use `install --force` with trusted digests to
@@ -694,12 +706,30 @@ handling, downloads, and integrity-manifest publication.
`Settings` detects and normalizes the host architecture at construction time
(`amd64` to `x86_64`, `arm64` to `aarch64`) and rejects unsupported architectures.
The same architecture selects the Firecracker release archive, guest kernel,
rootfs, and architecture-specific boot arguments. Default guest URLs use
Firecracker's `img/quickstart_guide/<architecture>/` assets: `kernels/vmlinux.bin`
and `rootfs/bionic.rootfs.ext4`. Kernel and rootfs URL overrides are independent;
custom assets and trusted digests must match the host architecture. ARM64
native rootfs packages, and architecture-specific boot arguments. The default
kernel URL uses Firecracker's `img/quickstart_guide/<architecture>/kernels/vmlinux.bin`.
An empty `Settings.rootfs_url` selects the built Ubuntu 22.04 (Jammy) template.
Explicit kernel/rootfs URLs and trusted digests must match the host architecture. ARM64
uses an uncompressed Linux `Image` (PE format) and an ARM64 ext4 rootfs.
The default rootfs builder installs `debootstrap` and `ubuntu-keyring`, uses
`--force-check-gpg`, and includes main/universe plus updates/security suites.
amd64 uses `archive.ubuntu.com/ubuntu`; arm64 uses `ports.ubuntu.com/ubuntu-ports`.
Package scripts execute in private mount/PID namespaces with a temporary
`policy-rc.d` suppressing service startup. No mounts survive the build process.
The staged tree retains `/var/lib/dpkg` and apt metadata, validates installed
apt/sudo/sshd/systemd packages, enables SSH/networkd, preserves kernel-provided
static networking, and supplies guest DNS. Template host keys and machine IDs
are cleared for per-VM initialization. `mkfs.ext4 -d` creates a sparse 2 GiB
template with a guest-kernel-compatible feature set; `e2fsck` and fsync precede
atomic publication. Create still grows each private disk to at least 30 GiB.
Only a checksum-verified manifest and matching `/etc/uvm-image` recipe marker
permit reuse of a built template. Old demo images and unverified caches are
rebuilt, and `install --force` always rebuilds. A failed staging build retains
the old template. Installation holds the existing operation lock throughout;
existing VM disks are never upgraded or reprovisioned by installation.
The host package list includes `nftables` and `conntrack` as well as the
existing iproute2/iptables, ext4, OpenSSL, OpenSSH, download, and util-linux
tools. UVM adds no reconciliation daemon or tested external firewall-manager
@@ -892,6 +922,9 @@ sparse ext4 images when the tools are installed: default 30 GiB creation,
growth, shrinking, geometry, and retained data/password-hash/host-key fixtures.
`test_images.py` uses temporary ext4 images to verify custom-user creation,
UID/GID allocation, account/home metadata, existing accounts, and password hashes.
`test_install.py` simulates package bootstrap and exercises real ext4 formatting
and readback for retained dpkg metadata, sudo modes, template configuration,
staging failures, and checksum provenance. It does not run debootstrap or boot a VM.
Lifecycle resize tests use fake processes/networking to verify ordering,
single-lock ownership, identity preservation, pending intent, and failure recovery.
@@ -926,7 +959,8 @@ on an isolated Linux host:
1. Strict checked installation.
2. `create` and successful guest boot.
3. Guest IP reachability and egress.
4. SSH with unique guest host keys.
4. SSH with unique guest host keys, `sudo -i` for a custom login, and
`sudo apt update && sudo apt install htop` in a newly built Ubuntu guest.
5. Stop/start/restart with retained guest data, SSH keys, and credentials.
6. Live private-port close in both directions, empty policy, and explicit sync
after deliberately lost rules inside the isolated setup.
README.md
+40 -9
View File
@@ -48,8 +48,10 @@ before downloading VM artifacts. Boot operations additionally check that the
native nftables bridge policy is supported; installing packages alone is not
a connectivity or isolation test.
Firecracker, the guest kernel, and the Ubuntu rootfs are selected automatically
for the host architecture. On NVIDIA DGX Spark, UVM uses the ARM64 assets.
Firecracker and the guest kernel are selected automatically for the host
architecture. The default rootfs is built locally from Ubuntu 22.04 (Jammy)
packages with OpenSSH, sudo, and a complete apt/dpkg database. On NVIDIA DGX
Spark, UVM uses ARM64 assets and packages.
`arm64` and `amd64` architecture names are normalized to `aarch64` and
`x86_64`, respectively. Custom `UVM_KERNEL_URL` and `UVM_ROOTFS_URL` overrides
must point to assets compatible with the host architecture.
@@ -64,21 +66,35 @@ the privileged installer:
sudo env \
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
./uvm.py install
```
Use `--force` to redownload Firecracker and guest assets when replacing an
image or refreshing artifacts:
The Ubuntu build uses `debootstrap`, authenticated Ubuntu archive signatures,
and native packages. UVM installs the builder prerequisites and records the
built ext4 image's SHA-256 in the integrity manifest. Building requires root
with mount/PID namespace support and internet access to Ubuntu mirrors.
Use `--force` to refresh downloads and rebuild the Ubuntu template:
```sh
sudo env \
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
./uvm.py install --force
```
To use a prebuilt ext4 image instead, set both `UVM_ROOTFS_URL` and its trusted
`UVM_ROOTFS_SHA256`. Leave both unset for the default Ubuntu build.
### Upgrade from the Stripped Bionic Demo Image
Run `install --force` with the updated code and with `UVM_ROOTFS_URL` and
`UVM_ROOTFS_SHA256` unset, then create a new VM. The replacement template
includes sudo and the package database that the old demo image lacked.
Existing VM disks retain their original OS and credentials; installation
updates the template used for future VMs. Restart a running API server to load
the updated provisioner before creating VMs through the dashboard.
For local experimentation only, the old unverified quick-start behavior is
available through an explicit opt-out:
@@ -360,13 +376,28 @@ defaults are username `root` and password `root`. A custom image must provide:
- `sshd`
- `/etc/passwd`, `/etc/shadow`, and `/etc/ssh/sshd_config`
- `/etc/group` and `/bin/sh` when creating a new user
- A root-owned, setuid `/usr/bin/sudo` and `/etc/sudoers` for custom login users
Custom login users receive password-authenticated sudo access. Inside a new VM:
```sh
sudo -i # Enter the same password used for this user's SSH login.
sudo apt update
sudo apt install htop
```
`su -` asks for root's separate password; creating a custom login does not set
root's password. Use `sudo -i` for an administrative shell. The default Ubuntu
template retains apt's package metadata and enables `main` and `universe`,
including updates/security repositories. Guest DNS defaults to `1.1.1.1` and
`8.8.8.8` in `/etc/resolv.conf`.
Existing accounts retain their UID, groups, home directory, and shell. A new
user reuses a same-named group if present; otherwise UVM creates one with an
unused GID. `/etc/gshadow` is updated when present. The default `root` account
must already exist in the image.
The known public demo key bundled in Firecracker's default bionic image is
The known public demo key bundled in Firecracker's older Bionic image is
removed from each private disk. UVM also replaces the conventional RSA, ECDSA,
and Ed25519 SSH host keys in every private disk so cloned VMs have distinct
host identities. Never rely on SSH private keys baked into a shared template.
@@ -466,10 +497,10 @@ Host and API settings can be overridden through environment variables:
| `UVM_BRIDGE` | Linux bridge name; defaults to `uvm0` |
| `UVM_KNOWN_HOSTS_FILES` | Override local SSH cache files cleaned on deletion; colon-separated absolute paths |
| `UVM_KERNEL_URL` | Override the host-architecture default guest kernel URL |
| `UVM_ROOTFS_URL` | Override the host-architecture default guest rootfs URL |
| `UVM_ROOTFS_URL` | Optional prebuilt ext4 image URL; unset to build Ubuntu 22.04 locally |
| `UVM_FIRECRACKER_SHA256` | Trusted Firecracker release archive SHA-256 |
| `UVM_KERNEL_SHA256` | Trusted guest kernel SHA-256 |
| `UVM_ROOTFS_SHA256` | Trusted guest rootfs SHA-256 |
| `UVM_ROOTFS_SHA256` | Trusted prebuilt rootfs SHA-256; use with `UVM_ROOTFS_URL` |
| `UVM_FIRECRACKER_BINARY_SHA256` | Optional trusted local Firecracker binary SHA-256 |
| `UVM_ALLOW_UNVERIFIED_DOWNLOADS` | Set to `1` only for unverified local development |
| `UVM_API_TOKEN` | Required for every API server; sent as `X-UVM-Token` |
USAGE.md
+41 -14
View File
@@ -138,21 +138,22 @@ not disable confirmations, and errors may still be plain text on stderr.
## 4. Install UVM Assets
UVM installs host packages, Firecracker, Jailer, a guest kernel, and a guest
rootfs template. It requires SHA-256 checksums by default because it downloads
artifacts that will be used by a privileged process.
rootfs template. The default rootfs is built locally from authenticated Ubuntu
22.04 packages, including OpenSSH, sudo, and complete apt/dpkg metadata. The
downloaded Firecracker archive and kernel require trusted SHA-256 checksums.
### ARM64 Hosts (NVIDIA DGX Spark)
UVM detects the host architecture and selects matching Firecracker, kernel,
and rootfs downloads. NVIDIA DGX Spark uses the `aarch64` assets; the `arm64`
architecture alias is also accepted. The default ARM64 guest downloads are:
UVM detects the host architecture and selects matching Firecracker and kernel
downloads and native Ubuntu packages. NVIDIA DGX Spark uses `aarch64` artifacts
and `arm64` packages; the `arm64` architecture alias is also accepted:
- Kernel: `https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/aarch64/kernels/vmlinux.bin`
- Rootfs: `https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/aarch64/rootfs/bionic.rootfs.ext4`
- Rootfs: Ubuntu 22.04 built with `debootstrap` from `ports.ubuntu.com/ubuntu-ports`.
Use the installation commands below with checksums for those ARM64 artifacts
and the ARM64 Firecracker release archive. `UVM_KERNEL_URL` and
`UVM_ROOTFS_URL` can override either download independently. Custom images must
Use the installation commands below with checksums for the ARM64 kernel
and Firecracker release archive. `UVM_KERNEL_URL` overrides the kernel download;
`UVM_ROOTFS_URL` selects a prebuilt ext4 image instead of the local build. Custom images must
match the host architecture and the [guest image contract](README.md#guest-image-contract).
On ARM64, the kernel must be an uncompressed ARM64 Linux `Image` (PE format),
and the rootfs must be an ARM64 ext4 filesystem. UVM adds `keep_bootcon` to the
@@ -166,7 +167,6 @@ Obtain trusted checksums independently, then run:
sudo env \
UVM_FIRECRACKER_SHA256='<sha256-of-firecracker-archive>' \
UVM_KERNEL_SHA256='<sha256-of-vmlinux>' \
UVM_ROOTFS_SHA256='<sha256-of-rootfs>' \
./uvm.py install
```
@@ -174,6 +174,12 @@ Installation prints the Firecracker, kernel, and rootfs paths under
`/var/lib/uvm`. Checksums and verification provenance are recorded in
`/var/lib/uvm/integrity.json`.
The default build installs `debootstrap` and `ubuntu-keyring` on the host,
checks Ubuntu archive signatures, and builds inside private mount/PID
namespaces. It requires native root execution with namespace support. Leave
`UVM_ROOTFS_URL` and `UVM_ROOTFS_SHA256` unset for this mode. To use a prebuilt
ext4 image, supply both its URL and SHA-256 instead.
### Scenario: Refresh Firecracker or Guest Assets
Use `--force` after changing a URL, replacing an image, or rotating checksums:
@@ -182,12 +188,17 @@ Use `--force` after changing a URL, replacing an image, or rotating checksums:
sudo env \
UVM_FIRECRACKER_SHA256='<new-firecracker-archive-sha256>' \
UVM_KERNEL_SHA256='<new-kernel-sha256>' \
UVM_ROOTFS_SHA256='<new-rootfs-sha256>' \
./uvm.py install --force
```
`--force` downloads to temporary files and only replaces the existing artifact
after checksum verification succeeds.
`--force` stages downloads and builds in temporary paths. Downloaded artifacts
are checksum-verified; the built rootfs is checked for installed packages,
sudo permissions, and filesystem consistency before replacement.
To upgrade from the old stripped Bionic demo rootfs, run this refresh with
`UVM_ROOTFS_URL` and `UVM_ROOTFS_SHA256` unset, then create a new VM. Restart a
running API server to load the new provisioner. Existing VM disks retain their
installed OS, files, and credentials.
### Scenario: Local Experimentation Without Checksums
@@ -215,6 +226,7 @@ sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py start VM
| `checksum is required` | Provide the requested `UVM_*_SHA256` variable. |
| `SHA-256 mismatch` | Stop and obtain the correct digest or artifact URL. |
| `unsupported host architecture` | Use an x86_64 or ARM64 Linux host. |
| `unshare: Operation not permitted` | Run the builder as root on a host permitting private mount/PID namespaces. |
## 5. Create VMs
@@ -245,7 +257,8 @@ create DNS records, and identifiers are not matched by ID prefix.
`--username` selects the guest login account. If a custom username is missing,
UVM creates it on the private disk with a primary group, `/home/USERNAME`, and
the `/bin/sh` login shell. Existing accounts keep their identity and home.
the `/bin/sh` login shell. Custom logins receive sudo access using their guest
password. Existing accounts keep their identity and home.
A `--password` value is visible in process arguments and may enter shell
history. Prefer `--password-stdin`, which reads one line, strips the trailing
@@ -415,6 +428,20 @@ guest image must include:
- An SSH server.
- Standard passwd and shadow files, plus `/etc/ssh/sshd_config`.
- `/etc/group` and `/bin/sh` for new users; the default `root` account must exist.
- Root-owned setuid `/usr/bin/sudo` and `/etc/sudoers` for a custom login user.
For a custom login, run these commands inside the VM:
```sh
sudo -i # Authenticate with the custom user's SSH password.
sudo apt update
sudo apt install htop
```
`su -` requests root's password, which is separate from the custom user's
password. Use `sudo -i` for administrative access. The default Ubuntu image
includes an intact `/var/lib/dpkg` database; an old Bionic demo image missing
that database needs a replacement template and a new VM for package management.
UVM replaces conventional RSA, ECDSA, and Ed25519 SSH host keys in the private
disk during creation and removes Firecracker's publicly known demo login key.
tests/test_config.py
+2 -2
View File
@@ -32,7 +32,7 @@ class ArchitectureSettingsTests(unittest.TestCase):
)
self.assertEqual(settings.architecture, architecture)
self.assertEqual(settings.kernel_url, f"{base}/kernels/vmlinux.bin")
self.assertEqual(settings.rootfs_url, f"{base}/rootfs/bionic.rootfs.ext4")
self.assertEqual(settings.rootfs_url, "")
def test_preserves_individual_environment_url_overrides_on_arm64(self) -> None:
kernel = "https://example.com/custom-arm64-kernel"
@@ -56,7 +56,7 @@ class ArchitectureSettingsTests(unittest.TestCase):
)
self.assertEqual(
settings.rootfs_url,
overrides.get("UVM_ROOTFS_URL", f"{base}/rootfs/bionic.rootfs.ext4"),
overrides.get("UVM_ROOTFS_URL", ""),
)
def test_rejects_unsupported_hosts_even_with_custom_guest_assets(self) -> None:
tests/test_images.py
+42 -1
View File
@@ -130,6 +130,19 @@ class ImageStoreTests(unittest.TestCase):
self.assertIn("Match User service\n PasswordAuthentication yes", updated)
self.assertNotIn("ChallengeResponseAuthentication yes", updated)
def test_full_ubuntu_sshd_global_directives_stay_outside_match_blocks(self) -> None:
updated = _enable_ssh_password_authentication(
"Include /etc/ssh/sshd_config.d/*.conf\n"
"UsePAM yes\nSubsystem sftp /usr/lib/openssh/sftp-server\n"
"Match Group restricted\n X11Forwarding no\n PasswordAuthentication no\n",
"codemate-user",
)
self.assertLess(updated.index("UsePAM yes"), updated.index("Match User codemate-user"))
self.assertLess(updated.index("Subsystem sftp"), updated.index("Match User codemate-user"))
self.assertLess(updated.index("Match User codemate-user"), updated.index("Match Group restricted"))
self.assertIn("Match Group restricted\n X11Forwarding no", updated)
def test_removes_only_the_public_firecracker_demo_key(self) -> None:
own_key = "ssh-ed25519 AAAA-own-key developer@example"
@@ -241,6 +254,8 @@ class OfflineGuestCredentialsTests(unittest.TestCase):
),
"/etc/group": "root:x:0:\nshadow:x:42:\nubuntu:x:1000:\nreserved:x:1001:\nnogroup:x:65534:\n",
"/etc/gshadow": "root:*::\nshadow:*::\nubuntu:!::\nreserved:!::\nnogroup:*::\n",
"/etc/sudoers": "Defaults env_reset\nroot ALL=(ALL:ALL) ALL\n%sudo ALL=(ALL:ALL) ALL\n",
"/usr/bin/sudo": "sudo binary fixture\n",
"/etc/ssh/sshd_config": "PasswordAuthentication no\nPermitRootLogin no\n",
"/root/.ssh/authorized_keys": f"{_FIRECRACKER_DEMO_PUBLIC_KEY} demo\nssh-ed25519 AAAA-own-key\n",
}
@@ -251,7 +266,8 @@ class OfflineGuestCredentialsTests(unittest.TestCase):
path = root / guest_path.lstrip("/")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(contents, encoding="utf-8")
path.chmod(0o640 if guest_path in ("/etc/shadow", "/etc/gshadow") else 0o644)
mode = {"/etc/shadow": 0o640, "/etc/gshadow": 0o640, "/etc/sudoers": 0o440, "/usr/bin/sudo": 0o4755}
path.chmod(mode.get(guest_path, 0o644))
with self.template.open("wb") as image:
image.truncate(16 * 1024 * 1024)
self.runner.run(("mkfs.ext4", "-q", "-F", "-d", root, self.template), capture=True)
@@ -305,6 +321,10 @@ class OfflineGuestCredentialsTests(unittest.TestCase):
self.assertLessEqual(int(shadow[2]), int(time.time() // 86400))
self.assertEqual(shadow[3:], ["0", "99999", "7", "", "", ""])
self.assert_password("codemate-user", "custom-password")
self.assertIn("codemate-user ALL=(ALL:ALL) PASSWD: ALL\n", self.contents("/etc/sudoers"))
self.assertIn("Defaults:codemate-user !rootpw, !runaspw, !targetpw\n", self.contents("/etc/sudoers"))
self.assert_metadata("/etc/sudoers", 0o440, 0, 0)
self.assert_metadata("/usr/bin/sudo", 0o4755, 0, 0)
for guest_path in ("/etc/passwd", "/etc/group"):
self.assert_metadata(guest_path, 0o644, 0, 0)
for guest_path in ("/etc/shadow", "/etc/gshadow"):
@@ -332,6 +352,10 @@ class OfflineGuestCredentialsTests(unittest.TestCase):
self.assertEqual(self.contents(guest_path), self.files[guest_path])
self.assertEqual(self.entry("/etc/shadow", username)[2:], original_shadow[2:])
self.assert_password(username, "changed-password")
if username != "root":
self.assertIn(f"{username} ALL=(ALL:ALL) PASSWD: ALL\n", self.contents("/etc/sudoers"))
else:
self.assertEqual(self.contents("/etc/sudoers"), self.files["/etc/sudoers"])
for line in self.files["/etc/shadow"].splitlines():
if line.split(":", 1)[0] != username:
self.assertIn(line + "\n", self.contents("/etc/shadow"))
@@ -375,3 +399,20 @@ class OfflineGuestCredentialsTests(unittest.TestCase):
self.assertEqual(self.contents("/home/codemate-user"), self.files["/home/codemate-user"])
self.assertEqual(self.contents("/etc/passwd"), self.files["/etc/passwd"])
self.assertEqual(self.contents("/etc/shadow"), self.files["/etc/shadow"])
def test_custom_users_fail_with_an_actionable_error_on_stripped_images(self) -> None:
del self.files["/usr/bin/sudo"]
del self.files["/etc/sudoers"]
self.make_disk()
with self.assertRaisesRegex(UvmError, "guest image needs setuid-root sudo.*uvm install --force"):
self.store.provision_credentials(self.disk, "codemate-user", "custom-password")
self.assertEqual(self.contents("/etc/shadow"), self.files["/etc/shadow"])
def test_sudo_without_setuid_permission_does_not_produce_a_broken_administrator(self) -> None:
self.make_disk()
self.runner.run(("debugfs", "-w", "-R", "set_inode_field /usr/bin/sudo mode 0100755", self.disk), capture=True)
with self.assertRaisesRegex(UvmError, "guest image needs setuid-root sudo"):
self.store.provision_credentials(self.disk, "codemate-user", "custom-password")
tests/test_install.py
+158 -4
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import hashlib
import io
import os
import shutil
import tarfile
import tempfile
import unittest
@@ -12,11 +13,12 @@ from types import SimpleNamespace
from unittest.mock import Mock, patch
from uvm.config import Settings
from uvm.errors import UvmError
from uvm.install import Installer
from uvm.integrity import load_manifest
from uvm.errors import CommandError, UvmError
from uvm.images import ImageStore
from uvm.install import Installer, _ROOTFS_RECIPE
from uvm.integrity import load_manifest, write_manifest
from uvm.state import StateStore
from uvm.system import CommandRunner
from uvm.system import CommandResult, CommandRunner
class InstallerArchitectureTests(unittest.TestCase):
@@ -56,6 +58,10 @@ class InstallerArchitectureTests(unittest.TestCase):
firecracker_sha256=hashlib.sha256(archive_bytes).hexdigest(),
kernel_sha256=hashlib.sha256(kernel_bytes).hexdigest(),
rootfs_sha256=hashlib.sha256(rootfs_bytes).hexdigest(),
rootfs_url=(
"https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/"
f"{architecture}/rootfs/bionic.rootfs.ext4"
),
)
guest_base = (
"https://s3.amazonaws.com/spec.ccfc.min/img/quickstart_guide/"
@@ -111,3 +117,151 @@ class InstallerArchitectureTests(unittest.TestCase):
check_kvm.assert_not_called()
download.assert_not_called()
self.assertFalse(settings.integrity_manifest_path.exists())
def test_default_build_records_the_authenticated_local_image_digest(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
settings = Settings(
base=Path(temporary), firecracker_sha256="1" * 64,
kernel_sha256=hashlib.sha256(b"kernel").hexdigest(),
)
settings.bin_dir.mkdir()
settings.firecracker_binary.write_bytes(b"firecracker")
settings.jailer_binary.write_bytes(b"jailer")
installer = Installer(settings, Mock(spec=CommandRunner), StateStore(settings), emit=None)
with (
patch("uvm.install.require_root"), patch("uvm.install.check_kvm"),
patch.object(installer, "install_apt_packages"),
patch.object(installer, "install_firecracker_binary", return_value=settings.firecracker_binary),
patch.object(installer, "download", side_effect=lambda _url, path, **_kwargs: path.write_bytes(b"kernel")),
patch.object(installer, "_build_ubuntu_rootfs", side_effect=lambda path: path.write_bytes(b"full Ubuntu")) as build,
):
installer.install()
build.assert_called_once_with(settings.rootfs_image)
manifest = load_manifest(settings.integrity_manifest_path)
self.assertTrue(manifest.verified)
self.assertEqual(manifest.checksums["rootfs"], hashlib.sha256(b"full Ubuntu").hexdigest())
self.assertEqual(ImageStore(settings).installed_assets().rootfs, settings.rootfs_image)
def test_old_rootfs_checksum_requires_an_explicit_prebuilt_url_before_mutations(self) -> None:
settings = Settings(firecracker_sha256="1" * 64, kernel_sha256="2" * 64, rootfs_sha256="3" * 64)
runner = Mock(spec=CommandRunner)
installer = Installer(settings, runner, Mock(spec=StateStore), emit=None)
with self.assertRaisesRegex(UvmError, "unset UVM_ROOTFS_SHA256"):
installer._validate_integrity_policy()
runner.run.assert_not_called()
def test_only_verified_current_recipe_images_are_reused(self) -> None:
for verified, marker, force in (
(True, _ROOTFS_RECIPE, False), (False, _ROOTFS_RECIPE, False),
(True, "", False), (True, _ROOTFS_RECIPE, True),
):
with self.subTest(verified=verified, marker=marker, force=force), tempfile.TemporaryDirectory() as temporary:
settings = Settings(base=Path(temporary))
settings.images_dir.mkdir()
settings.rootfs_image.write_bytes(b"image")
write_manifest(settings.integrity_manifest_path,
{"rootfs": hashlib.sha256(b"image").hexdigest()}, verified=verified)
runner = Mock(spec=CommandRunner)
runner.run.return_value = CommandResult(("debugfs",), 0, stdout=marker)
installer = Installer(settings, runner, StateStore(settings), emit=None)
with patch.object(installer, "_build_ubuntu_rootfs") as build:
installer._install_ubuntu_rootfs(settings.rootfs_image, force=force)
self.assertEqual(build.called, not (verified and marker == _ROOTFS_RECIPE and not force))
class UbuntuBuildRunner:
"""Simulate package installation; exercise real ext4 formatting and readback."""
def __init__(self, *, failure: str | None = None) -> None:
self.calls: list[tuple[str, ...]] = []
self.failure = failure
self.runner = CommandRunner(emit=None)
def run(self, command, **options) -> CommandResult:
args = tuple(map(str, command))
self.calls.append(args)
if "debootstrap" in args:
root = Path(args[-2])
if self.failure == "bootstrap":
raise CommandError("bootstrap interrupted")
status = "\n\n".join(
f"Package: {package}\nStatus: install ok installed\nVersion: 1.0"
for package in ("apt", "sudo", "openssh-server", "systemd-sysv")
) + "\n"
files = {
"etc/passwd": "root:x:0:0::/root:/bin/bash\n",
"etc/shadow": "root:*:1:0:99999:7:::\n", "etc/group": "root:x:0:\n",
"etc/sudoers": "root ALL=(ALL:ALL) ALL\n",
"etc/ssh/sshd_config": "UsePAM yes\n",
"etc/ssh/ssh_host_rsa_key": "template host key",
"usr/bin/sudo": "sudo binary", "usr/bin/apt-get": "apt binary",
"usr/sbin/sshd": "sshd binary", "usr/lib/systemd/systemd": "init binary",
"var/lib/dpkg/status": status,
"var/lib/dpkg/info/apt.list": "/usr/bin/apt-get\n",
"var/lib/dpkg/info/sudo.list": "/usr/bin/sudo\n",
}
if self.failure == "metadata":
files["var/lib/dpkg/status"] = ""
for relative, contents in files.items():
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(contents, encoding="utf-8")
(root / "usr/bin/sudo").chmod(0o4755)
(root / "etc/resolv.conf").symlink_to("../run/systemd/resolve/stub-resolv.conf")
return CommandResult(args, 0)
if args[0] == "systemctl":
return CommandResult(args, 0)
return self.runner.run(command, **options)
@unittest.skipUnless(all(shutil.which(tool) for tool in ("mkfs.ext4", "e2fsck", "debugfs")),
"e2fsprogs is required for offline Ubuntu image checks")
class UbuntuRootfsBuildTests(unittest.TestCase):
def test_build_keeps_package_metadata_and_sudo_permissions_with_native_signed_bootstrap(self) -> None:
for architecture, deb_arch, mirror in (
("aarch64", "arm64", "https://ports.ubuntu.com/ubuntu-ports"),
("x86_64", "amd64", "https://archive.ubuntu.com/ubuntu"),
):
with self.subTest(architecture=architecture), tempfile.TemporaryDirectory() as temporary:
settings = Settings(base=Path(temporary), architecture=architecture)
runner = UbuntuBuildRunner()
installer = Installer(settings, runner, StateStore(settings), emit=None) # type: ignore[arg-type]
installer._build_ubuntu_rootfs(settings.rootfs_image)
bootstrap = next(call for call in runner.calls if "debootstrap" in call)
self.assertIn(f"--arch={deb_arch}", bootstrap)
self.assertEqual(bootstrap[-1], mirror)
self.assertIn("--force-check-gpg", bootstrap)
self.assertIn("--extra-suites=jammy-updates,jammy-security", bootstrap)
self.assertIn("--mount-proc", bootstrap)
self.assertIn("--kill-child", bootstrap)
self.assertEqual(settings.rootfs_image.stat().st_mode & 0o777, 0o600)
self.assertEqual(list(settings.images_dir.iterdir()), [settings.rootfs_image])
def guest(command):
return runner.runner.run(("debugfs", "-R", command, settings.rootfs_image), capture=True)
self.assertIn("Package: apt\nStatus: install ok installed", guest("cat /var/lib/dpkg/status").stdout)
self.assertEqual(guest("cat /var/lib/dpkg/info/sudo.list").stdout, "/usr/bin/sudo\n")
self.assertRegex(guest("stat /usr/bin/sudo").stdout, r"Mode:\s+0?4755")
self.assertEqual(guest("cat /etc/uvm-image").stdout, _ROOTFS_RECIPE)
self.assertIn("nameserver 1.1.1.1", guest("cat /etc/resolv.conf").stdout)
self.assertIn("KeepConfiguration=static", guest("cat /etc/systemd/network/10-uvm.network").stdout)
self.assertIn("jammy-security main universe", guest("cat /etc/apt/sources.list").stdout)
for path in ("/usr/sbin/policy-rc.d", "/etc/ssh/ssh_host_rsa_key"):
self.assertIn("File not found", guest(f"stat {path}").stderr)
def test_failed_bootstrap_or_missing_package_metadata_preserves_the_old_template(self) -> None:
for failure in ("bootstrap", "metadata"):
with self.subTest(failure=failure), tempfile.TemporaryDirectory() as temporary:
settings = Settings(base=Path(temporary))
settings.images_dir.mkdir()
settings.rootfs_image.write_bytes(b"old template")
runner = UbuntuBuildRunner(failure=failure)
installer = Installer(settings, runner, StateStore(settings), emit=None) # type: ignore[arg-type]
with self.assertRaises(UvmError):
installer._build_ubuntu_rootfs(settings.rootfs_image)
self.assertEqual(settings.rootfs_image.read_bytes(), b"old template")
self.assertEqual(list(settings.images_dir.iterdir()), [settings.rootfs_image])
self.assertFalse(any(call[0] == "mkfs.ext4" for call in runner.calls))
uvm/cli.py
+5 -4
View File
@@ -74,8 +74,9 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
install = subcommands.add_parser(
"install",
description=(
"Set UVM_FIRECRACKER_SHA256, UVM_KERNEL_SHA256, and UVM_ROOTFS_SHA256 "
"to trusted digests. UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 is an explicit "
"Set UVM_FIRECRACKER_SHA256 and UVM_KERNEL_SHA256 to trusted digests. "
"The default Ubuntu rootfs is built from signed packages. A custom "
"UVM_ROOTFS_URL also requires UVM_ROOTFS_SHA256. UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 is an explicit "
"development-only opt-out."
),
help="install dependencies and assets; SHA-256 values are required by default",
@@ -83,7 +84,7 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
install.add_argument(
"--force",
action="store_true",
help="redownload Firecracker, kernel, and rootfs even when local assets already exist",
help="refresh downloads and rebuild the default Ubuntu rootfs even when assets exist",
)
create = subcommands.add_parser("create", help="create and boot a microVM detached")
@@ -111,7 +112,7 @@ def build_parser(application: Application) -> argparse.ArgumentParser:
create.add_argument(
"--username",
default=application.settings.default_ssh_user,
help="guest login account; created if missing (defaults to root)",
help="guest administrator account; created if missing, with sudo access (defaults to root)",
)
credentials = create.add_mutually_exclusive_group()
credentials.add_argument(
uvm/config.py
+1 -3
View File
@@ -31,7 +31,7 @@ class Settings:
default_vcpu: float = 1
default_ssh_user: str = "root"
kernel_url: str = ""
rootfs_url: str = ""
rootfs_url: str = "" # Empty selects the locally built, package-managed Ubuntu image.
firecracker_sha256: str | None = None
firecracker_binary_sha256: str | None = None
kernel_sha256: str | None = None
@@ -56,8 +56,6 @@ class Settings:
asset_base = f"{DEFAULT_GUEST_ASSETS_URL}/{architecture}"
if not self.kernel_url:
object.__setattr__(self, "kernel_url", f"{asset_base}/kernels/vmlinux.bin")
if not self.rootfs_url:
object.__setattr__(self, "rootfs_url", f"{asset_base}/rootfs/bionic.rootfs.ext4")
if self.gateway not in self.network:
raise ConfigurationError(
uvm/images.py
+37 -3
View File
@@ -226,6 +226,8 @@ class ImageStore:
encoding="utf-8",
)
shadow.path.chmod(shadow.mode)
if username != "root":
self._provision_guest_sudo(disk, username, temporary)
sshd_config.path.write_text(
_enable_ssh_password_authentication(
@@ -244,6 +246,31 @@ class ImageStore:
except (OSError, UnicodeError) as error:
raise UvmError(f"could not provision guest credentials: {error}") from error
def _provision_guest_sudo(self, disk: Path, username: str, temporary: Path) -> None:
sudoers = self._read_guest_file(disk, "/etc/sudoers", temporary / "sudoers", required=False)
sudo = self._runner.run(("debugfs", "-R", "stat /usr/bin/sudo", disk), capture=True)
mode = _MODE_PATTERN.search(sudo.stdout)
owner = _OWNER_PATTERN.search(sudo.stdout)
if (
sudoers is None
or re.search(r"Type:\s+regular\b", sudo.stdout) is None
or mode is None or not int(mode.group(1), 8) & 0o4000
or owner is None or int(owner.group(1)) != 0
):
raise UvmError(
"guest image needs setuid-root sudo and /etc/sudoers for a custom administrator; "
"run uvm install --force to build the complete Ubuntu image"
)
contents = sudoers.path.read_text(encoding="utf-8").rstrip()
sudoers.path.write_text(
contents + "\n\n# Managed by uvm: guest administrator\n"
+ f"Defaults:{username} !rootpw, !runaspw, !targetpw\n"
+ f"{username} ALL=(ALL:ALL) PASSWD: ALL\n",
encoding="utf-8",
)
sudoers.path.chmod(sudoers.mode)
self._write_guest_file(disk, "/etc/sudoers", sudoers)
def _create_guest_user(
self,
disk: Path,
@@ -562,12 +589,16 @@ def _enable_ssh_password_authentication(contents: str, username: str) -> str:
}
if username == "root":
managed_directives.add("permitrootlogin")
unmanaged_lines = []
global_lines = []
match_lines = []
in_match = False
for line in contents.splitlines():
stripped = line.lstrip()
directive = stripped.split(None, 1)[0].lower() if stripped else ""
if directive == "match":
in_match = True
if stripped.startswith("#") or directive not in managed_directives:
unmanaged_lines.append(line)
(match_lines if in_match else global_lines).append(line)
directives = [
"HostKey /etc/ssh/ssh_host_rsa_key",
@@ -579,13 +610,16 @@ def _enable_ssh_password_authentication(contents: str, username: str) -> str:
]
if username == "root":
directives.append("PermitRootLogin yes")
# Global-only directives (e.g. UsePAM and Subsystem) must precede every Match block.
globals_text = "\n".join(global_lines).strip("\n")
return (
"# Managed by uvm\n"
+ "\n".join(directives)
+ (f"\n{globals_text}" if globals_text else "")
+ f"\nMatch User {username}\n"
+ " PasswordAuthentication yes\n"
+ "Match all\n"
+ "\n".join(unmanaged_lines).rstrip()
+ "\n".join(match_lines).rstrip()
+ "\n"
)
uvm/install.py
+156 -16
View File
@@ -14,11 +14,24 @@ from pathlib import Path
from .config import Settings
from .errors import UvmError
from .images import GuestAssets
from .integrity import require_checksum, sha256_file, verify_file, write_manifest
from .integrity import load_manifest, require_checksum, sha256_file, verify_file, write_manifest
from .state import StateStore
from .system import CommandRunner, check_kvm, ensure_data_directories, require_root
_UBUNTU_RELEASE = "jammy"
_ROOTFS_RECIPE = "uvm-ubuntu-jammy-v1\n"
_GUEST_PACKAGES = (
"ca-certificates", "openssh-server", "sudo", "systemd-sysv", "udev",
"dbus", "libpam-systemd", "iproute2", "iputils-ping", "curl",
)
# Keep the image mountable by the older Firecracker quickstart guest kernels.
_EXT4_FEATURES = (
"none,has_journal,ext_attr,resize_inode,dir_index,filetype,extent,64bit,"
"flex_bg,sparse_super,large_file,huge_file,dir_nlink,extra_isize,metadata_csum"
)
class Installer:
"""Install exactly the local host dependencies used by the uvm CLI."""
@@ -68,12 +81,18 @@ class Installer:
"UVM_KERNEL_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
require_checksum(
self._settings.rootfs_sha256,
"guest root filesystem",
"UVM_ROOTFS_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
if self._settings.rootfs_url:
require_checksum(
self._settings.rootfs_sha256,
"guest root filesystem",
"UVM_ROOTFS_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
elif self._settings.rootfs_sha256:
raise UvmError(
"UVM_ROOTFS_SHA256 requires UVM_ROOTFS_URL for a prebuilt image; "
"unset UVM_ROOTFS_SHA256 to build the default Ubuntu rootfs"
)
def _write_integrity_manifest(self, firecracker: Path, assets: GuestAssets) -> None:
write_manifest(
@@ -88,7 +107,7 @@ class Installer:
(
self._settings.firecracker_sha256,
self._settings.kernel_sha256,
self._settings.rootfs_sha256,
self._settings.rootfs_sha256 if self._settings.rootfs_url else True,
)
),
)
@@ -108,6 +127,8 @@ class Installer:
"openssl",
"openssh-client",
)
if not self._settings.rootfs_url:
packages += ("debootstrap", "ubuntu-keyring")
self._runner.run(("apt-get", "update"))
self._runner.run(("apt-get", "install", "-y", *packages))
@@ -183,7 +204,9 @@ class Installer:
label="guest kernel",
environment_name="UVM_KERNEL_SHA256",
)
if force or not rootfs.exists():
if not self._settings.rootfs_url:
self._install_ubuntu_rootfs(rootfs, force=force)
elif force or not rootfs.exists():
self.download(
self._settings.rootfs_url,
rootfs,
@@ -198,15 +221,132 @@ class Installer:
"UVM_KERNEL_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
verify_file(
rootfs,
self._settings.rootfs_sha256,
"guest root filesystem",
"UVM_ROOTFS_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
if self._settings.rootfs_url:
verify_file(
rootfs,
self._settings.rootfs_sha256,
"guest root filesystem",
"UVM_ROOTFS_SHA256",
allow_unverified=self._settings.allow_unverified_downloads,
)
return GuestAssets(kernel=kernel, rootfs=rootfs)
def _install_ubuntu_rootfs(self, destination: Path, *, force: bool) -> None:
if destination.is_file() and not force:
manifest = load_manifest(self._settings.integrity_manifest_path)
checksum = manifest.checksums.get("rootfs")
if manifest.verified and checksum:
verify_file(destination, checksum, "Ubuntu rootfs", "integrity manifest", allow_unverified=False)
marker = self._runner.run(
("debugfs", "-R", "cat /etc/uvm-image", destination), capture=True,
)
if marker.stdout == _ROOTFS_RECIPE:
return
# Rebuild old demo images and unverified caches rather than promoting their trust.
self._build_ubuntu_rootfs(destination)
def _build_ubuntu_rootfs(self, destination: Path) -> None:
architecture = "arm64" if self._settings.architecture == "aarch64" else "amd64"
mirror = (
"https://ports.ubuntu.com/ubuntu-ports" if architecture == "arm64"
else "https://archive.ubuntu.com/ubuntu"
)
if self._emit is not None:
self._emit("Building Ubuntu 22.04 rootfs with sudo, OpenSSH, and apt/dpkg metadata")
destination.parent.mkdir(parents=True, exist_ok=True)
try:
with tempfile.TemporaryDirectory(prefix=".ubuntu-build-", dir=destination.parent) as name:
temporary = Path(name)
root = temporary / "root"
policy = root / "usr/sbin/policy-rc.d"
policy.parent.mkdir(parents=True)
policy.write_text("#!/bin/sh\nexit 101\n", encoding="utf-8")
policy.chmod(0o755)
# Package scripts run natively, with private mounts/PIDs and service startup disabled.
self._runner.run((
"unshare", "--mount", "--propagation", "private", "--pid", "--fork",
"--kill-child", "--mount-proc", "env", "DEBIAN_FRONTEND=noninteractive",
"debootstrap", f"--arch={architecture}", "--variant=minbase", "--merged-usr",
"--components=main,universe", f"--include={','.join(_GUEST_PACKAGES)}",
f"--extra-suites={_UBUNTU_RELEASE}-updates,{_UBUNTU_RELEASE}-security",
"--force-check-gpg", "--keyring=/usr/share/keyrings/ubuntu-archive-keyring.gpg",
_UBUNTU_RELEASE, root, mirror,
))
policy.unlink()
self._configure_ubuntu_rootfs(root, mirror)
self._validate_ubuntu_rootfs(root)
disk = temporary / "rootfs.ext4"
with disk.open("wb") as image:
image.truncate(2 * 1024 * 1024 * 1024)
disk.chmod(0o600)
self._runner.run((
"mkfs.ext4", "-q", "-F", "-b", "4096", "-O", _EXT4_FEATURES,
"-E", "lazy_itable_init=0,lazy_journal_init=0", "-d", root, disk,
), capture=True)
self._runner.run(("e2fsck", "-f", "-n", disk), capture=True)
with disk.open("rb") as image:
os.fsync(image.fileno())
os.replace(disk, destination)
directory = os.open(destination.parent, os.O_RDONLY)
try:
os.fsync(directory)
finally:
os.close(directory)
except (OSError, UnicodeError) as error:
raise UvmError(f"could not build Ubuntu guest rootfs: {error}") from error
def _configure_ubuntu_rootfs(self, root: Path, mirror: str) -> None:
files = {
"etc/apt/sources.list": "".join(
f"deb {mirror} {suite} main universe\n"
for suite in (_UBUNTU_RELEASE, f"{_UBUNTU_RELEASE}-updates", f"{_UBUNTU_RELEASE}-security")
),
"etc/fstab": "/dev/vda / ext4 defaults 0 1\n",
"etc/hostname": "uvm\n",
"etc/hosts": "127.0.0.1 localhost\n127.0.1.1 uvm\n",
"etc/resolv.conf": "nameserver 1.1.1.1\nnameserver 8.8.8.8\n",
"etc/machine-id": "",
"etc/uvm-image": _ROOTFS_RECIPE,
"etc/systemd/network/10-uvm.network": (
"[Match]\nName=eth0\n\n[Network]\n"
"KeepConfiguration=static\nDHCP=no\nLinkLocalAddressing=no\nIPv6AcceptRA=no\n"
),
}
for relative, contents in files.items():
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.unlink(missing_ok=True) # In particular, replace a resolver symlink with guest DNS.
path.write_text(contents, encoding="utf-8")
path.chmod(0o644)
machine_id = root / "var/lib/dbus/machine-id"
machine_id.parent.mkdir(parents=True, exist_ok=True)
machine_id.unlink(missing_ok=True)
machine_id.symlink_to("../../../etc/machine-id")
for key in (root / "etc/ssh").glob("ssh_host_*_key*"):
key.unlink()
self._runner.run(("systemctl", "--root", root, "enable", "ssh.service", "systemd-networkd.service"), capture=True)
self._runner.run(("systemctl", "--root", root, "set-default", "multi-user.target"), capture=True)
@staticmethod
def _validate_ubuntu_rootfs(root: Path) -> None:
for relative in (
"etc/passwd", "etc/shadow", "etc/group", "etc/sudoers", "etc/ssh/sshd_config",
"usr/bin/sudo", "usr/bin/apt-get", "usr/sbin/sshd", "usr/lib/systemd/systemd",
"var/lib/dpkg/status", "var/lib/dpkg/info/apt.list", "var/lib/dpkg/info/sudo.list",
):
if not (root / relative).is_file():
raise UvmError(f"built Ubuntu rootfs is missing /{relative}")
installed = set()
for paragraph in (root / "var/lib/dpkg/status").read_text(encoding="utf-8").split("\n\n"):
fields = dict(line.split(": ", 1) for line in paragraph.splitlines() if ": " in line)
if fields.get("Status") == "install ok installed":
installed.add(fields.get("Package"))
if not {"apt", "sudo", "openssh-server", "systemd-sysv"}.issubset(installed):
raise UvmError("built Ubuntu rootfs has incomplete apt/dpkg package metadata")
if not (root / "usr/bin/sudo").stat().st_mode & 0o4000:
raise UvmError("built Ubuntu rootfs sudo is missing its setuid permission")
def download(
self,
url: str,