feat(tools): Enhance guest tools installation and management
- Added new API endpoints for tool catalog and installation management. - Updated usage documentation to include new tool installation features. - Implemented asynchronous tool installation with job tracking. - Introduced integration tests for tool installation in disposable VMs. - Enhanced dashboard UI to reflect tool installation status and usage instructions. - Improved error handling and logging for tool installation processes. - Added support for multiple Python versions and Conda environments. - Refactored tool management logic to ensure better state handling and job persistence.
This commit is contained in:
@@ -920,6 +920,10 @@ curl --cacert /path/to/ca.pem \
|
||||
| `POST` | `/vms/{vm_id}/ports/sync` | Reapply the policy for an exact VM ID |
|
||||
| `POST` | `/vms/{vm_id}/exec` | Execute a guest command/script over vsock, with optional root privileges |
|
||||
| `POST` | `/vms/{vm_id}/agent/install` | Install/upgrade the agent on a stopped VM's disk |
|
||||
| `GET` | `/tools` | Authenticated tool catalog and Python version options |
|
||||
| `GET` | `/vms/{vm_id}/tools` | Cached tool inventory/history; `refresh=true` checks the running guest |
|
||||
| `POST` | `/vms/{vm_id}/tools/install` | Queue tool installation; HTTP 202 with a job ID |
|
||||
| `GET` | `/vms/{vm_id}/tools/jobs/{job_id}` | Installation progress, results, and captured output |
|
||||
| `GET` | `/vms/{identifier}/logs` | Read a bounded, non-following log tail |
|
||||
| `GET` | `/vms/{vm_id}/sessions` | List saved sessions for an exact VM ID (Nest) |
|
||||
| `POST` | `/vms/{vm_id}/sessions` | Launch a terminal, with `{}` or optional `rows`/`cols` (Nest) |
|
||||
@@ -973,7 +977,7 @@ curl -X POST \
|
||||
```
|
||||
|
||||
Request fields are `cpu`, `ram`, `storage`, `guest_ip`, `username`, `password`, `name`,
|
||||
`ports`, and `no_default_ports`. The name has the same validation as the CLI.
|
||||
`ports`, `no_default_ports`, and `tools`. The name has the same validation as the CLI.
|
||||
`ports` is an array of objects (or null/omitted), not a CLI colon string.
|
||||
Each object has a strict integer `port` from 1 to 65535 and an optional,
|
||||
case-sensitive `protocol` of `TCP` (default), `HTTP`, or `HTTPS`.
|
||||
@@ -988,6 +992,106 @@ The response never includes the password. Omitting credentials still uses
|
||||
labels return HTTP `422`. Example passwords above are placeholders; real
|
||||
secrets in curl arguments can enter shell history and process listings.
|
||||
|
||||
### Install Guest Tools
|
||||
|
||||
The dashboard provides tool checkboxes during creation and a **Tools** tab on
|
||||
each VM. Choose **Install** for a single tool or **Install selected** for a batch.
|
||||
**Start & install** also boots a stopped VM. The catalog covers Ubuntu 22.04 on
|
||||
x86_64 and ARM64, with guest internet access for package downloads.
|
||||
|
||||
Use the same API selections during creation:
|
||||
|
||||
```sh
|
||||
curl -X POST \
|
||||
-H "X-UVM-Token: $UVM_API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"dev-tools","username":"developer","password":"replace-me","tools":[{"id":"git"},{"id":"nodejs"},{"id":"python","version":"3.12"}]}' \
|
||||
"$UVM_API_URL/vms"
|
||||
```
|
||||
|
||||
The VM response is HTTP 201 after creation. Selected tools install asynchronously;
|
||||
read the returned VM's `/tools` endpoint for its job. Omitted `tools` defaults to
|
||||
an empty list. An explicit null is rejected.
|
||||
|
||||
For an existing VM, use its exact ID from create/list:
|
||||
|
||||
```sh
|
||||
export VM_ID='<exact-vm-id>'
|
||||
curl -H "X-UVM-Token: $UVM_API_TOKEN" "$UVM_API_URL/tools"
|
||||
|
||||
curl -X POST \
|
||||
-H "X-UVM-Token: $UVM_API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"tools":[{"id":"jdk"},{"id":"conda"},{"id":"micromamba"}],"start_if_stopped":true,"request_id":"dev-tools-1"}' \
|
||||
"$UVM_API_URL/vms/$VM_ID/tools/install"
|
||||
|
||||
# Use the job ID from the HTTP 202 response.
|
||||
export JOB_ID='<tools-job-id>'
|
||||
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
|
||||
"$UVM_API_URL/vms/$VM_ID/tools/jobs/$JOB_ID"
|
||||
curl -H "X-UVM-Token: $UVM_API_TOKEN" \
|
||||
"$UVM_API_URL/vms/$VM_ID/tools?refresh=true"
|
||||
```
|
||||
|
||||
Selections are objects with `id` and an optional `version`. IDs are `git`,
|
||||
`nodejs`, `python`, `jdk`, `wget`, `curl`, `htop`, `php`, `conda`, and
|
||||
`micromamba`. Python requires `version: "3.11"`, `"3.12"`, or `"3.13"`;
|
||||
the other tools use the curated catalog default and accept no version override.
|
||||
Up to twelve selections are accepted, with identical selections deduplicated.
|
||||
Multiple Python versions can be installed together.
|
||||
|
||||
Install requests require a nonempty `tools` list. `start_if_stopped` and `restart`
|
||||
are strict booleans, both default false. Set `restart: true` to restart first and
|
||||
refresh guest-agent setup. In the dashboard, the **Restart & install** recovery
|
||||
button appears when agent readiness fails. Unknown tools/options return 422;
|
||||
unavailable VM states or a different active install return 409.
|
||||
|
||||
`request_id` is an optional idempotency key of 1–80 letters, digits, underscores,
|
||||
or hyphens. Reusing it with the same request returns the retained job; different
|
||||
selections/options return 409. Use a new key for an intentional retry. Idempotency
|
||||
keys last as long as their job in the ten-job history. One batch per VM is active
|
||||
at a time, with two VM workers by default and sequential tools within a batch.
|
||||
|
||||
Jobs move through `queued`, `starting`, `waiting`, and `installing`, then
|
||||
`succeeded`, `failed`, or `interrupted`. Each item reports `installed`,
|
||||
`already_installed`, `failed`, `interrupted`, or `skipped`, plus its detected
|
||||
version, error, bounded output, and truncation flag. Ordinary recipe failures
|
||||
allow subsequent tools to run; uncertain execution outcomes end that batch.
|
||||
Output is captured as a 16 KiB tail per tool and returned when the step finishes.
|
||||
Default limits are 120 seconds for agent readiness and 1,200 seconds per tool.
|
||||
|
||||
`GET /vms/{vm_id}/tools` returns cached inventory, the latest job, and summaries
|
||||
of the last ten jobs. `refresh=true` probes all tools in a running VM and updates
|
||||
`checked_at`; it does not install software or start a stopped VM. `stale` marks
|
||||
old or stopped-VM observations. Installed binaries and their version checks run
|
||||
as the configured guest user. Matching installed runtimes are accepted, including
|
||||
tools installed outside Nest. The install operation is not a general updater.
|
||||
|
||||
Tool jobs survive browser navigation, reload, and logout. The active management
|
||||
server owns the queue through `$UVM_BASE/tool-worker.lock`. After server restart,
|
||||
queued work resumes; in-flight jobs become interrupted with
|
||||
`outcome_unknown: true`. The last command may still finish inside the guest.
|
||||
Refresh inventory before retrying; a guest-side lock blocks overlapping installs.
|
||||
Stopping/restarting the VM also interrupts a running batch. Successful tools and
|
||||
the VM's private disk remain available. Deleting the VM removes its tool history.
|
||||
|
||||
Use versioned Python commands, for example `python3.12 -m venv .venv`. For Conda
|
||||
or micromamba activation in a Nest terminal, initialize the current POSIX shell:
|
||||
|
||||
```sh
|
||||
eval "$(conda shell.posix hook)"
|
||||
conda create -n myenv python=3.13
|
||||
conda activate myenv
|
||||
|
||||
# In another shell, for micromamba:
|
||||
eval "$(micromamba shell hook --shell posix)"
|
||||
micromamba create -n myenv -c conda-forge python=3.13
|
||||
micromamba activate myenv
|
||||
```
|
||||
|
||||
The installer also adds login-shell profiles under `/etc/profile.d`. See
|
||||
[the catalog](README.md#one-click-guest-tools) for default versions and sources.
|
||||
|
||||
### Start, Restart, Stop, and Delete
|
||||
|
||||
```sh
|
||||
@@ -1597,6 +1701,21 @@ exercise real child processes with fake cgroup scopes, local Unix-socket protoco
|
||||
fixtures, and temporary ext4 upgrades; they do not prove the real vsock/cgroup
|
||||
path inside a VM.
|
||||
|
||||
The tools smoke test installs the full catalog in a prepared **disposable Ubuntu
|
||||
22.04 VM with a non-root login and no open TCP ports**. It downloads packages,
|
||||
checks all runtimes, compiles Java, creates Python venvs, and creates/activates
|
||||
temporary Conda and micromamba environments as the guest user. Run it separately
|
||||
on x86_64 and ARM64 hosts:
|
||||
|
||||
```sh
|
||||
UVM_TOOLS_INTEGRATION=1 UVM_TOOLS_VM_ID='<disposable-vm-id>' \
|
||||
python3 -B -m unittest discover -s tests/integration -p test_guest_tools.py -v
|
||||
```
|
||||
|
||||
Set `UVM_API_URL` and `UVM_API_TOKEN` as above. The test is opt-in and was not run
|
||||
against a real microVM during this implementation. Automated API and browser
|
||||
checks use simulated guest execution and real job persistence/workers.
|
||||
|
||||
## 19. Get Help
|
||||
|
||||
```sh
|
||||
|
||||
Reference in New Issue
Block a user