kstyagi/nest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement resource resizing for VMs with storage management

Browse Source 
- Added KnownHostsManager to handle cleanup of SSH known_hosts entries upon VM deletion.
- Introduced disk_size_bytes method in ImageStore to read logical disk capacity.
- Enhanced LifecycleService to manage VM resource resizing, including CPU, RAM, and storage.
- Updated application configuration to support known_hosts_files for SSH management.
- Modified dashboard to include resource resizing options in the UI.
- Improved error handling and validation for resource inputs in the dashboard.
- Removed obsolete .pyc files from the repository.
This commit is contained in:
kstyagi
2026-09-21 20:37:52 +05:30
parent fb80ce7c8a
commit 4a46685a4c
40 changed files with 1048 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
USAGE.md
+77 -14
View File
@@ -46,7 +46,7 @@ You need:
- Root access for installation and VM lifecycle operations.
- An x86_64 or ARM64 (`aarch64`) host, including NVIDIA DGX Spark.
- Internet access if downloading Firecracker and guest assets.
- `iproute2`, `iptables`, `nftables`, `conntrack`, `e2fsprogs`, `util-linux`,
- `iproute2`, `iptables`, `nftables`, `conntrack`, `coreutils`, `e2fsprogs`, `util-linux`,
`openssl`, and `openssh-client`; `install` installs the host packages.
- An STP-disabled software bridge without traffic offload around its hooks.
@@ -82,7 +82,7 @@ the full path to that environment's console script when necessary.
| Operation | Root required? |
|---|---|
| `install`, `create`, `start`, `stop`, `restart`, `delete`/`destroy` | Yes |
| `install`, `create`, `start`, `stop`, `restart`, `resize`, `delete`/`destroy` | Yes |
| `ports open/close/set/sync` | Yes |
| `list`, `inspect`, `ports list`, dry-run target lookup | Access to the default root-owned registry requires root |
| `ssh`, `logs` | Access to the protected registry/runtime files requires root |
@@ -102,7 +102,7 @@ and do not run every destructive example as a single script.
```text
uvm install [--force]
uvm create [--name NAME] [--cpu CPU] [--ram RAM] [--guest-ip GUEST_IP]
uvm create [--name NAME] [--cpu CPU] [--ram RAM] [--storage STORAGE] [--guest-ip GUEST_IP]
[--username USER] [--password PASSWORD | --password-stdin]
[--ports PORTS]... [--no-default-ports] [--json]
uvm list [--status STATUS]... [--json | -q | --quiet]
@@ -110,6 +110,7 @@ uvm inspect VM [--json]
uvm start (VM [VM ...] | --all) [--yes | -y] [--dry-run] [--json]
uvm stop (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm restart (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm resize VM [--cpu CPU] [--ram RAM] [--storage STORAGE] [--timeout SECONDS] [--json]
uvm delete (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm destroy (VM [VM ...] | --all) [--timeout SECONDS] [--yes | -y] [--dry-run] [--json]
uvm ssh VM [--user USER] [--key PATH] [--port PORT] [--insecure-host-key] [-- COMMAND ...]
@@ -199,7 +200,7 @@ sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py install
An unverified manifest is not publisher trust. Reinstall with trusted digests
to establish a verified installation. Until then, retain the explicit opt-out
on later create/start/restart commands and on an API server that creates VMs:
on later create/start/restart/resize commands and on an API server that creates VMs:
```sh
sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py create
@@ -219,7 +220,7 @@ sudo env UVM_ALLOW_UNVERIFIED_DOWNLOADS=1 ./uvm.py start VM
### Scenario: Create a Default VM
The default is one CPU, 512 MiB RAM, username `root`, password `root`, and an
The default is one CPU, 512 MiB RAM, 30 GiB storage, username `root`, password `root`, and an
allowlist containing private TCP/22:
```sh
@@ -281,10 +282,10 @@ sudo ./uvm.py create --name worker --no-default-ports
port mappings. See Private Port Policies below for live edits and network
scope. UVM does not install the services named by these labels.
### Scenario: Choose CPU and RAM
### Scenario: Choose CPU, RAM, and Storage
```sh
sudo ./uvm.py create --cpu 2 --ram 2G
sudo ./uvm.py create --cpu 2 --ram 2G --storage 50G
sudo ./uvm.py create --cpu 0.5 --ram 512
```
@@ -295,6 +296,14 @@ UVM enforces a minimum of 128 MiB. Firecracker uses whole vCPUs. A fractional
request such as `--cpu 0.5` is accepted but rounds the VM up to one Firecracker
vCPU; it does not currently apply a host CPU quota.
Storage defaults to **30 GiB**, with a **30 GiB minimum**. `--storage 50G` gives
the private ext4 root disk 50 GiB of capacity; bare numbers use GiB. Accepted
suffixes include `M`, `G`, `T`, `MiB`, `GiB`, and `TiB` (also `B` and `K`).
`GB` and `TB` use binary units, consistently with RAM. Fractional amounts round
up to whole MiB. The API and JSON output report capacity as `storage_mib`.
Sparse images consume host space as data is written, rather than reserving the
whole configured capacity up front.
### Scenario: Request a Specific Guest IP
```sh
@@ -312,7 +321,8 @@ UVM performs these operations in order:
1. Checks root access, KVM, artifact integrity, and nftables/conntrack support.
2. Reserves identity, name/IP, and initial port policy in the registry.
3. Copies the template into a private disk and provisions credentials and SSH host keys.
3. Copies the template into a private disk, resizes ext4 to the requested capacity,
and provisions credentials and SSH host keys.
4. Ensures bridge/egress networking and creates an owned TAP attached but down.
5. Installs and verifies the nftables bridge policy, then activates the TAP.
6. Starts Firecracker, persists process identity, and configures/boots the guest.
@@ -432,7 +442,7 @@ for one connection:
sudo ./uvm.py ssh web --insecure-host-key
```
## 8. Start, Stop, Restart, and Delete
## 8. Start, Stop, Restart, Resize, and Delete
### Scenario: Stop a VM and Keep Its Disk
@@ -473,6 +483,34 @@ disk, or pending deletion blocks a new boot; inspect the error rather than
editing registry flags. An interrupted boot with a live VMM must be stopped
before start.
### Scenario: Resize an Existing VM
```sh
sudo ./uvm.py resize web --cpu 2 --ram 4G --storage 80G
sudo ./uvm.py resize web --ram 2G --timeout 10 --json
sudo ./uvm.py resize web --storage 30G
```
Supply at least one resource; omitted fields retain their current values (or
the pending target when retrying an interrupted resize). UVM validates the
request and boot prerequisites, records the target, stops the VM, applies the
changes, and boots it under one operation lock. A stopped VM also boots after
resize. The operation retains the VM ID, IP/MAC, name, username/password, port
rules, guest data, and SSH host keys. The dashboard provides these controls
under **Resize resources** on a machine's detail page.
CPU and RAM can increase or decrease. Storage can grow or shrink to at least
30 GiB, subject to the existing data fitting in ext4. Disk resizing uses a
temporary sparse copy; the checked and resized filesystem replaces the original
only after success. This may require additional physical host space for the copy.
A failed stop prevents resource changes. A failed disk resize leaves the VM
stopped with its retained disk and a `pending_resize` target; correct the cause
and retry `resize` (use a larger storage target if needed). `start` on the stopped
VM also retries the pending target. A boot failure after the new resources were
applied retains those values for the next start. `stop` preserves a pending
resize; `delete` removes the VM and its disk normally.
### Scenario: Permanently Remove a VM
```sh
@@ -597,7 +635,7 @@ commands and starting the API server do not repair lost rules.
### Legacy VM Migration
The default registry is `/var/lib/uvm/state.json`, schema version `2`. Older
The default registry is `/var/lib/uvm/state.json`, schema version `3`. Older
registries are accepted and migrated on the next write. Missing/null `ports`
stays `None`/`unmanaged`, not an empty policy and not default SSH enforcement.
Open/close/sync/start do not silently adopt a policy for such a VM:
@@ -614,6 +652,11 @@ first be set while the legacy VM is stopped. Start also requires a safe,
completely provisioned private disk; very old records without one cannot be
resumed by copying a new template over their data.
VMs created before configurable storage retain `storage_mib: null` until their
first resize. That resize uses the existing disk capacity, rounded up to MiB and
at least 30 GiB, unless an explicit storage target is supplied. Merely reading
or migrating the registry does not resize disks.
A live legacy PID missing only the host boot ID can be adopted during a
locked mutation only if its saved start time, running Firecracker executable,
exact API-socket command line, and protected binary/socket paths match. A
@@ -712,6 +755,7 @@ curl --cacert /path/to/ca.pem \
| `GET` | `/vms/{identifier}` | Inspect by exact ID, name, or guest IP |
| `POST` | `/vms/{identifier}/start` | Start a retained VM |
| `POST` | `/vms/{identifier}/restart` | Restart a VM; accepts `timeout` query parameter |
| `POST` | `/vms/{identifier}/resize` | Stop, resize CPU/RAM/storage, and boot; accepts `timeout` query parameter |
| `POST` | `/vms/{identifier}/stop` | Stop and retain disk; accepts `timeout` query parameter |
| `DELETE` | `/vms/{identifier}` | Delete VM and private disk; accepts `timeout` query parameter |
| `POST` | `/vms/{vm_id}/ports` | Set, open, or close private ports for an exact VM ID |
@@ -720,7 +764,7 @@ curl --cacert /path/to/ca.pem \
Individual lifecycle requests do not expose a CLI dry-run flag and do not prompt
for confirmation. `timeout` is an optional finite, non-negative query parameter
on restart, stop, and delete; it has the same SIGTERM wait semantics as the CLI.
on restart, stop, resize, and delete; it has the same SIGTERM wait semantics as the CLI.
### Health, List, and Inspect
@@ -733,7 +777,7 @@ curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/vms/10.42.0.30"
# An exact VM ID from create/list can replace the name or IP.
```
Each response contains `id`, `name`, `cpu`, `ram_mib`, `guest_ip`, `gateway`,
Each response contains `id`, `name`, `cpu`, `ram_mib`, `storage_mib`, `pending_resize`, `guest_ip`, `gateway`,
`mac`, `username`, `status`, `observed_status`, `ports`, `policy_status`,
`operation`, `pid`, timestamps, and `last_error`. The response schema excludes
passwords and disk/socket/config/log path fields. Operational error messages
@@ -747,7 +791,7 @@ Create a named VM with a chosen guest address and private service ports:
curl -X POST \
-H "Content-Type: application/json" \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-d '{"name":"api-web","cpu":1,"ram":"1G","guest_ip":"10.42.0.30","username":"root","password":"replace-me","ports":[{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}],"no_default_ports":false}' \
-d '{"name":"api-web","cpu":1,"ram":"1G","storage":"50G","guest_ip":"10.42.0.30","username":"root","password":"replace-me","ports":[{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}],"no_default_ports":false}' \
"$UVM_API_URL/vms"
```
@@ -761,7 +805,7 @@ curl -X POST \
"$UVM_API_URL/vms"
```
Request fields are `cpu`, `ram`, `guest_ip`, `username`, `password`, `name`,
Request fields are `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`,
`ports`, and `no_default_ports`. The name has the same validation as the CLI.
`ports` is an array of objects (or null/omitted), not a CLI colon string.
Each object has a strict integer `port` from 1 to 65535 and an optional,
@@ -769,6 +813,8 @@ case-sensitive `protocol` of `TCP` (default), `HTTP`, or `HTTPS`.
`no_default_ports` is a strict boolean, default `false`; port 22 is added
unless that flag is true, even if `ports` is null, empty, or contains other
ports. Null create input does not create a legacy/unmanaged VM.
`storage` defaults to `"30G"`; numeric storage values use GiB, with the same
30 GiB minimum and suffixes as the CLI.
The response never includes the password. Omitting credentials still uses
`root` / `root`. Extra request fields, invalid ports/names, and conflicting
@@ -803,6 +849,23 @@ exact `id` and `status: "terminated"`. Start and restart return the updated VM
response. Start preserves the retained private disk and does not restore guest
memory.
### Resize CPU, RAM, or Storage
```sh
curl -X POST \
-H "Content-Type: application/json" \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-d '{"cpu":2,"ram":"4G","storage":"80G"}' \
"$UVM_API_URL/vms/api-web/resize?timeout=10"
```
The body accepts optional `cpu`, `ram`, and `storage` fields; at least one must
be non-null. For example, `{"ram":"2G"}` changes only memory. IP and credentials
are retained and cannot be edited through this endpoint. A successful response
is the updated running VM, including `storage_mib` and `pending_resize: null`.
During a failed/interrupted resize, `pending_resize` contains the full target
as `{ "cpu": 2, "ram_mib": 4096, "storage_mib": 81920 }` until it is applied.
### Bulk Lifecycle Actions
The dashboard batch toolbar uses this endpoint for selected VMs. The server