feat(tools): Enhance guest tools installation and management

- Added new API endpoints for tool catalog and installation management.
- Updated usage documentation to include new tool installation features.
- Implemented asynchronous tool installation with job tracking.
- Introduced integration tests for tool installation in disposable VMs.
- Enhanced dashboard UI to reflect tool installation status and usage instructions.
- Improved error handling and logging for tool installation processes.
- Added support for multiple Python versions and Conda environments.
- Refactored tool management logic to ensure better state handling and job persistence.
This commit is contained in:
2026-09-25 23:52:21 +05:30
parent a41aafce83
commit 3e17e3e72f
16 changed files with 729 additions and 34 deletions
+120 -1
View File
@@ -920,6 +920,10 @@ curl --cacert /path/to/ca.pem \
| `POST` | `/vms/{vm_id}/ports/sync` | Reapply the policy for an exact VM ID |
| `POST` | `/vms/{vm_id}/exec` | Execute a guest command/script over vsock, with optional root privileges |
| `POST` | `/vms/{vm_id}/agent/install` | Install/upgrade the agent on a stopped VM's disk |
| `GET` | `/tools` | Authenticated tool catalog and Python version options |
| `GET` | `/vms/{vm_id}/tools` | Cached tool inventory/history; `refresh=true` checks the running guest |
| `POST` | `/vms/{vm_id}/tools/install` | Queue tool installation; HTTP 202 with a job ID |
| `GET` | `/vms/{vm_id}/tools/jobs/{job_id}` | Installation progress, results, and captured output |
| `GET` | `/vms/{identifier}/logs` | Read a bounded, non-following log tail |
| `GET` | `/vms/{vm_id}/sessions` | List saved sessions for an exact VM ID (Nest) |
| `POST` | `/vms/{vm_id}/sessions` | Launch a terminal, with `{}` or optional `rows`/`cols` (Nest) |
@@ -973,7 +977,7 @@ curl -X POST \
```
Request fields are `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`,
`ports`, and `no_default_ports`. The name has the same validation as the CLI.
`ports`, `no_default_ports`, and `tools`. The name has the same validation as the CLI.
`ports` is an array of objects (or null/omitted), not a CLI colon string.
Each object has a strict integer `port` from 1 to 65535 and an optional,
case-sensitive `protocol` of `TCP` (default), `HTTP`, or `HTTPS`.
@@ -988,6 +992,106 @@ The response never includes the password. Omitting credentials still uses
labels return HTTP `422`. Example passwords above are placeholders; real
secrets in curl arguments can enter shell history and process listings.
### Install Guest Tools
The dashboard provides tool checkboxes during creation and a **Tools** tab on
each VM. Choose **Install** for a single tool or **Install selected** for a batch.
**Start & install** also boots a stopped VM. The catalog covers Ubuntu 22.04 on
x86_64 and ARM64, with guest internet access for package downloads.
Use the same API selections during creation:
```sh
curl -X POST \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"dev-tools","username":"developer","password":"replace-me","tools":[{"id":"git"},{"id":"nodejs"},{"id":"python","version":"3.12"}]}' \
"$UVM_API_URL/vms"
```
The VM response is HTTP 201 after creation. Selected tools install asynchronously;
read the returned VM's `/tools` endpoint for its job. Omitted `tools` defaults to
an empty list. An explicit null is rejected.
For an existing VM, use its exact ID from create/list:
```sh
export VM_ID='<exact-vm-id>'
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/tools"
curl -X POST \
-H "X-UVM-Token: $UVM_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{"tools":[{"id":"jdk"},{"id":"conda"},{"id":"micromamba"}],"start_if_stopped":true,"request_id":"dev-tools-1"}' \
"$UVM_API_URL/vms/$VM_ID/tools/install"
# Use the job ID from the HTTP 202 response.
export JOB_ID='<tools-job-id>'
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/tools/jobs/$JOB_ID"
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
"$UVM_API_URL/vms/$VM_ID/tools?refresh=true"
```
Selections are objects with `id` and an optional `version`. IDs are `git`,
`nodejs`, `python`, `jdk`, `wget`, `curl`, `htop`, `php`, `conda`, and
`micromamba`. Python requires `version: "3.11"`, `"3.12"`, or `"3.13"`;
the other tools use the curated catalog default and accept no version override.
Up to twelve selections are accepted, with identical selections deduplicated.
Multiple Python versions can be installed together.
Install requests require a nonempty `tools` list. `start_if_stopped` and `restart`
are strict booleans, both default false. Set `restart: true` to restart first and
refresh guest-agent setup. In the dashboard, the **Restart & install** recovery
button appears when agent readiness fails. Unknown tools/options return 422;
unavailable VM states or a different active install return 409.
`request_id` is an optional idempotency key of 1–80 letters, digits, underscores,
or hyphens. Reusing it with the same request returns the retained job; different
selections/options return 409. Use a new key for an intentional retry. Idempotency
keys last as long as their job in the ten-job history. One batch per VM is active
at a time, with two VM workers by default and sequential tools within a batch.
Jobs move through `queued`, `starting`, `waiting`, and `installing`, then
`succeeded`, `failed`, or `interrupted`. Each item reports `installed`,
`already_installed`, `failed`, `interrupted`, or `skipped`, plus its detected
version, error, bounded output, and truncation flag. Ordinary recipe failures
allow subsequent tools to run; uncertain execution outcomes end that batch.
Output is captured as a 16 KiB tail per tool and returned when the step finishes.
Default limits are 120 seconds for agent readiness and 1,200 seconds per tool.
`GET /vms/{vm_id}/tools` returns cached inventory, the latest job, and summaries
of the last ten jobs. `refresh=true` probes all tools in a running VM and updates
`checked_at`; it does not install software or start a stopped VM. `stale` marks
old or stopped-VM observations. Installed binaries and their version checks run
as the configured guest user. Matching installed runtimes are accepted, including
tools installed outside Nest. The install operation is not a general updater.
Tool jobs survive browser navigation, reload, and logout. The active management
server owns the queue through `$UVM_BASE/tool-worker.lock`. After server restart,
queued work resumes; in-flight jobs become interrupted with
`outcome_unknown: true`. The last command may still finish inside the guest.
Refresh inventory before retrying; a guest-side lock blocks overlapping installs.
Stopping/restarting the VM also interrupts a running batch. Successful tools and
the VM's private disk remain available. Deleting the VM removes its tool history.
Use versioned Python commands, for example `python3.12 -m venv .venv`. For Conda
or micromamba activation in a Nest terminal, initialize the current POSIX shell:
```sh
eval "$(conda shell.posix hook)"
conda create -n myenv python=3.13
conda activate myenv
# In another shell, for micromamba:
eval "$(micromamba shell hook --shell posix)"
micromamba create -n myenv -c conda-forge python=3.13
micromamba activate myenv
```
The installer also adds login-shell profiles under `/etc/profile.d`. See
[the catalog](README.md#one-click-guest-tools) for default versions and sources.
### Start, Restart, Stop, and Delete
```sh
@@ -1597,6 +1701,21 @@ exercise real child processes with fake cgroup scopes, local Unix-socket protoco
fixtures, and temporary ext4 upgrades; they do not prove the real vsock/cgroup
path inside a VM.
The tools smoke test installs the full catalog in a prepared **disposable Ubuntu
22.04 VM with a non-root login and no open TCP ports**. It downloads packages,
checks all runtimes, compiles Java, creates Python venvs, and creates/activates
temporary Conda and micromamba environments as the guest user. Run it separately
on x86_64 and ARM64 hosts:
```sh
UVM_TOOLS_INTEGRATION=1 UVM_TOOLS_VM_ID='<disposable-vm-id>' \
python3 -B -m unittest discover -s tests/integration -p test_guest_tools.py -v
```
Set `UVM_API_URL` and `UVM_API_TOKEN` as above. The test is opt-in and was not run
against a real microVM during this implementation. Automated API and browser
checks use simulated guest execution and real job persistence/workers.
## 19. Get Help
```sh