diff --git a/AGENT-VM.md b/AGENT-VM.md index 1efdac5..918cf2c 100644 --- a/AGENT-VM.md +++ b/AGENT-VM.md @@ -179,7 +179,13 @@ replacing local instructions. This does not restart an already-running VM. DSH intentionally rejects `dsh web --host 0.0.0.0`. We don't patch around its browser protections: it listens on **guest `127.0.0.1:3080`**, and the shell launcher -publishes an **SSH local forward** on your chosen **host** IPv4 address/port. +publishes an **SSH local forward** on your chosen **host** IPv4 address. By default +it tries **3080 through 3100, in ascending order**, and keeps the first port SSH +successfully binds. There is no separate free-port probe that could race another +process. The printed login URL and `nix run .#agent -- url` use the chosen port, +recorded in the project's private runtime state. Each new launch starts at 3080 +again; shutdown clears the selection. If the range is full, launch fails clearly +and cleans up its VM/tunnel instead of leaving an inaccessible instance running. DSH's random launch-token → signed-cookie authentication and Host/Origin checks remain in use. Only SSH is forwarded by QEMU, always on host loopback. @@ -187,22 +193,31 @@ remain in use. Only SSH is forwarded by QEMU, always on host loopback. agentVM.network = { hostAddress = "127.0.0.1"; # default: this computer only sshPort = 2222; - webPort = 3080; + webPort = 3080; # First candidate. + webPortEnd = 3100; # Last candidate, inclusive. }; ``` +Set both values to the same port for a fixed listener. For compatibility, setting +only a nondefault `webPort` still means that single fixed port; give `webPortEnd` +explicitly to select a different range. Keep the SSH port outside the Web range. + - **One LAN/VPN interface:** set `hostAddress = "192.168.1.20"` (an IP actually - assigned to this host). Its browser authority is automatically trusted. + assigned to this host). Its exact authorities across the configured port range + are automatically trusted. - **All IPv4 interfaces:** set `hostAddress = "0.0.0.0"` and - `trustedHosts = [ "192.168.1.20:3080" "laptop.example:3080" ];`. + `trustedHosts = [ "192.168.1.20" "laptop.example" ];`. Port-less entries accept + that exact host on any port; an explicit `host:port` accepts only that port. Use a real address, not `0.0.0.0`, in your browser. The printed local URL can have its host replaced with one of those authorities. This includes public interfaces too; it is not shorthand for “LAN only.” -- Multiple project VMs need distinct host SSH/Web ports. +- Multiple project VMs still need distinct **SSH** ports. Web ports are selected + automatically from the range; SSH port selection is unchanged. - Binding a host IP controls the **incoming listener**, not outgoing routing or which NIC reaches DeepSeek. Outgoing traffic follows host routes/VPN policy. -- No host firewall is changed. For LAN access, explicitly allow only the Web port - on the intended host interface in your firewall. Do not open the SSH forward. +- No host firewall is changed. For LAN access, explicitly allow the selected Web + port (or intended Web range) on the intended host interface. Do not open the SSH + forward. Use a fixed port if a reverse proxy needs a stable upstream. **Use a VPN or a TLS reverse proxy for off-host access.** The forward is encrypted between host and guest, but browser → host remains HTTP. A token/cookie on an @@ -337,9 +352,11 @@ nix build path:/etc/nix/agent-vm#checks.x86_64-linux.config \ path:/etc/nix/agent-vm#checks.x86_64-linux.playwright --no-link # Actual offline microVM boot/mount/SSH/browser test, with a fake harness (no API calls): nix build path:/etc/nix/agent-vm#checks.x86_64-linux.boot --no-link +# Real port contention, range exhaustion and remembered-URL checks: +nix build path:/etc/nix/agent-vm#checks.x86_64-linux.ports --no-link ``` -The boot test requires KVM and nested user namespaces in the Nix build sandbox. +The boot and port tests require KVM and nested user namespaces in the Nix build sandbox. The Playwright check launches two real Firefox instances concurrently against a local HTTP fixture, checks headless/default browser selection, separate cookies, DOM and localStorage, screenshots, close-one/keep-one behavior and profile expiry. diff --git a/README.md b/README.md index 4214634..e489463 100644 --- a/README.md +++ b/README.md @@ -121,8 +121,8 @@ git credential-cache exit [AGENT-VM.md](AGENT-VM.md) documents the project template and `nix run .#agent`. The official `dsh@latest` runs as root in a rootless microVM, with the current project and your standard DSH home/shared skills mounted **read-write**. Its -host Web listener defaults to localhost; RAM, vCPUs, IPs and optional TAP -networking are configured through Nix modules. The VM is headless by default and +host Web listener defaults to localhost on the first free port in **3080–3100**; +RAM, vCPUs, IPs and optional TAP networking are configured through Nix modules. The VM is headless by default and includes **Playwright CLI + matching Firefox**, with a writable, once-seeded `playwright-firefox` skill for isolated named sessions across subagents. No host service is activated. diff --git a/agent-vm/launch.sh b/agent-vm/launch.sh index d16f16e..3aadab0 100644 --- a/agent-vm/launch.sh +++ b/agent-vm/launch.sh @@ -4,6 +4,7 @@ if [[ ${1:-} == --help ]]; then echo 'Usage: nix run .#agent -- [run | ssh [command ...] | url | stop]' echo 'Workspace = cwd. RW config/credentials/skills = DSH_HOME (default ~/.dsh)' echo 'Also shares DSH_AGENTS_HOME/skills (default ~/.agents/skills). RAM/CPU/network: flake.' + echo "Web UI tries $AGENT_WEB_PORT-$AGENT_WEB_PORT_END in order; url prints the selected port." exit 0 fi [[ $EUID != 0 ]] || { echo 'Run as your normal host user, not sudo/root.' >&2; exit 1; } @@ -26,13 +27,18 @@ ssh_cmd=(ssh -F /dev/null -i "$state/client-key" -p "$AGENT_SSH_PORT" -o StrictHostKeyChecking=yes -o HostKeyAlias=agent-vm -o ConnectTimeout=3 -o "UserKnownHostsFile=$state/known_hosts" -o GlobalKnownHostsFile=/dev/null) remote="root@$AGENT_SSH_HOST" +# Relative ControlPath avoids Unix-socket path limits with long state directories. +web_control() { (cd "$state" && "${ssh_cmd[@]}" -S web.sock "$@" "$remote"); } url() { - local found address=$AGENT_WEB_BIND + local found port address=$AGENT_WEB_BIND + [[ -f $state/web-port ]] && read -r port < "$state/web-port" || return 1 + [[ $port =~ ^[1-9][0-9]{0,4}$ ]] && (( port <= 65535 )) || return 1 + web_control -O check >/dev/null 2>&1 || return 1 [[ $address != 0.0.0.0 ]] || address=127.0.0.1 found=$("${ssh_cmd[@]}" "$remote" 'journalctl -u agent -b -o cat --no-pager' | grep -oE 'http://127\.0\.0\.1:3080/\?token=[a-zA-Z0-9_%.-]+' | tail -1) || return 1 [[ -n $found ]] || return 1 - printf '%s\n' "${found/http:\/\/127.0.0.1:3080/http:\/\/$address:$AGENT_WEB_PORT}" + printf '%s\n' "${found/http:\/\/127.0.0.1:3080/http:\/\/$address:$port}" } # Expand cwd inside the guest, not on the host. # shellcheck disable=SC2016 @@ -52,6 +58,9 @@ for dir in "$state" "$dsh"; do done exec 9>"$state/run.lock" flock -n 9 || { echo 'This project VM is already running.' >&2; exit 1; } +# Clean up only this project's stale forwarding state, after acquiring its lock. +web_control -O exit >/dev/null 2>&1 || true +rm -f "$state/web.sock" "$state/web-port" for key in client-key ssh-host-key; do [[ -f $state/$key ]] || ssh-keygen -q -t ed25519 -N '' -C agent-vm -f "$state/$key" done @@ -63,15 +72,15 @@ if [[ $AGENT_WEB_BIND != 127.0.0.1 ]]; then echo 'WARNING: off-host Web access is plaintext HTTP. Use a VPN/TLS; never expose directly to the Internet.' >&2 fi vm_pid='' -tunnel_pid='' cleanup() { trap - EXIT INT TERM + web_control -O exit >/dev/null 2>&1 || true + rm -f "$state/web-port" "$state/web.sock" if [[ -n $vm_pid ]] && kill -0 "$vm_pid" 2>/dev/null; then (cd "$state"; timeout 30 "$AGENT_RUNNER/microvm-shutdown") >/dev/null 2>&1 || true kill "$vm_pid" 2>/dev/null || true wait "$vm_pid" 2>/dev/null || true fi - if [[ -n $tunnel_pid ]]; then kill "$tunnel_pid" 2>/dev/null || true; wait "$tunnel_pid" 2>/dev/null || true; fi } trap cleanup EXIT trap 'exit 130' INT @@ -99,16 +108,28 @@ done $ready || { echo "SSH boot timeout; see $state/console.log" >&2; exit 1; } # DSH deliberately refuses --host 0.0.0.0. Keep its own authenticated browser # endpoint on guest loopback and publish an SSH forward on the chosen host IP. -"${ssh_cmd[@]}" -N -g -o ExitOnForwardFailure=yes -o ServerAliveInterval=10 \ - -o ServerAliveCountMax=3 -L "$AGENT_WEB_BIND:$AGENT_WEB_PORT:127.0.0.1:3080" \ - "$remote" >>"$state/console.log" 2>&1 & -tunnel_pid=$! +web_control -M -fN -g -o ExitOnForwardFailure=yes -o ServerAliveInterval=10 \ + -o ServerAliveCountMax=3 >>"$state/console.log" 2>&1 || { + echo "Cannot start Web SSH tunnel; see $state/console.log" >&2; exit 1; + } +# Ask SSH to actually bind each port: no probe-then-bind race or extra port helper. +for ((port=AGENT_WEB_PORT; port<=AGENT_WEB_PORT_END; port++)); do + if web_control -O forward -L "$AGENT_WEB_BIND:$port:127.0.0.1:3080" >>"$state/console.log" 2>&1; then + printf '%s\n' "$port" > "$state/web-port" + break + fi +done +[[ -f $state/web-port ]] || { + echo "No available Web UI port on $AGENT_WEB_BIND in $AGENT_WEB_PORT-$AGENT_WEB_PORT_END; see $state/console.log" >&2 + exit 1 +} +echo "Web UI selected host port $port." echo "Booted. DSH resolves npm @latest on startup; first launch may take a few minutes." echo 'Use another terminal: nix run .#agent -- url (or: ssh / stop)' printed=false while kill -0 "$vm_pid" 2>/dev/null; do # Normal guest poweroff can close SSH slightly before QEMU exits. - if ! kill -0 "$tunnel_pid" 2>/dev/null; then + if ! web_control -O check >/dev/null 2>&1; then timeout 30 tail --pid="$vm_pid" -f /dev/null || true if kill -0 "$vm_pid" 2>/dev/null; then echo "Web tunnel exited; see $state/console.log" >&2; exit 1; fi break diff --git a/agent-vm/lib.nix b/agent-vm/lib.nix index 5202694..9a3c0b9 100644 --- a/agent-vm/lib.nix +++ b/agent-vm/lib.nix @@ -35,6 +35,7 @@ let AGENT_NETWORK = net.mode; AGENT_WEB_BIND = net.hostAddress; AGENT_WEB_PORT = toString net.webPort; + AGENT_WEB_PORT_END = toString net.webPortEnd; AGENT_SSH_HOST = if net.mode == "user" then "127.0.0.1" else net.guestAddress; AGENT_SSH_PORT = toString (if net.mode == "user" then net.sshPort else 22); }; diff --git a/agent-vm/module.nix b/agent-vm/module.nix index c8a5243..b76496f 100644 --- a/agent-vm/module.nix +++ b/agent-vm/module.nix @@ -71,12 +71,18 @@ in webPort = mkOption { type = types.port; default = 3080; - description = "Host Web UI port; the guest DSH listener stays on 127.0.0.1:3080."; + description = "First host Web UI port to try; the guest listener stays on 127.0.0.1:3080."; + }; + webPortEnd = mkOption { + type = types.port; + # Preserve fixed-port behavior for existing nondefault webPort settings. + default = if net.webPort == 3080 then 3100 else net.webPort; + description = "Last host Web UI port to try, inclusive. Set equal to webPort for a fixed port."; }; trustedHosts = mkOption { type = types.listOf (types.strMatching "[a-zA-Z0-9.:-]+"); default = [ ]; - description = "Additional exact browser authorities for DSH's Host/Origin protection. Required for wildcard publication."; + description = "Additional host[:port] entries for DSH's Host/Origin protection. A port-less host matches any port. Required for wildcard publication."; }; tapName = mkOption { type = types.strMatching "[a-zA-Z0-9_-]{1,15}"; @@ -119,8 +125,12 @@ in message = "agentVM TAP mode requires network.guestAddress, gateway and dns."; } { - assertion = net.webPort >= 1024 && net.sshPort >= 1024 && net.webPort != net.sshPort; - message = "Rootless Web/SSH listeners need distinct unprivileged ports (>=1024)."; + assertion = + net.webPort >= 1024 + && net.webPortEnd >= net.webPort + && net.sshPort >= 1024 + && (net.sshPort < net.webPort || net.sshPort > net.webPortEnd); + message = "Use an ordered, unprivileged Web port range and an unprivileged SSH port outside that range."; } { assertion = net.hostAddress != "0.0.0.0" || net.trustedHosts != [ ]; @@ -369,12 +379,12 @@ in ]) ( net.trustedHosts - ++ lib.optional ( + ++ lib.optionals ( !builtins.elem net.hostAddress [ "127.0.0.1" "0.0.0.0" ] - ) "${net.hostAddress}:${toString net.webPort}" + ) (map (port: "${net.hostAddress}:${toString port}") (lib.range net.webPort net.webPortEnd)) ) ); Restart = "on-failure"; diff --git a/agent-vm/port-test.sh b/agent-vm/port-test.sh new file mode 100644 index 0000000..522cc21 --- /dev/null +++ b/agent-vm/port-test.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# Real SSH forwards in an offline VM; listeners belong only to this test. +set -euo pipefail +launcher=$1 +tmp=$(mktemp -d) +export HOME=$tmp/home DSH_HOME=$tmp/dsh DSH_AGENTS_HOME=$tmp/agents XDG_STATE_HOME=$tmp/state +mkdir -p "$HOME" "$tmp/project" +cd "$tmp/project" +state="$XDG_STATE_HOME/agent-vm/$(printf %s "$PWD" | sha256sum | cut -c1-16)" +vm_pid='' +listener='' +cleanup() { + status=$? + if (( status )); then + grep -h . "$tmp/launcher.log" "$tmp/listeners.log" "$state/console.log" | tail -100 || true + fi + "$launcher" stop >/dev/null 2>&1 || true + if [[ -n $vm_pid ]]; then kill "$vm_pid" 2>/dev/null || true; wait "$vm_pid" 2>/dev/null || true; fi + if [[ -n $listener ]]; then kill "$listener" 2>/dev/null || true; wait "$listener" 2>/dev/null || true; fi + rm -rf "$tmp" + return "$status" +} +trap cleanup EXIT +reserve_through() { + rm -f "$tmp/listeners-ready" + node - "$1" "$tmp/listeners-ready" > "$tmp/listeners.log" 2>&1 <<'JS' & +const http = require('node:http'); +const fs = require('node:fs'); +const listeners = []; +for (let port = 3080; port <= Number(process.argv[2]); port++) { + listeners.push(new Promise((resolve, reject) => { + http.createServer((req, res) => res.end('occupied\n')) + .on('error', reject).listen(port, '127.0.0.1', resolve); + })); +} +Promise.all(listeners).then(() => fs.writeFileSync(process.argv[3], 'ready')); +JS + listener=$! + for ((i=0; i<100; i++)); do + [[ ! -f $tmp/listeners-ready ]] || return 0 + kill -0 "$listener" + sleep 0.1 + done + echo 'Port fixture startup timed out' >&2; return 1 +} +release_listeners() { kill "$listener"; wait "$listener" || true; listener=''; } +expect_port() { + local expected=$1 login='' + "$launcher" run > "$tmp/launcher.log" 2>&1 & + vm_pid=$! + for ((i=0; i<120; i++)); do + kill -0 "$vm_pid" + if login=$("$launcher" url 2>/dev/null); then break; fi + sleep 1 + done + [[ $login == "http://127.0.0.1:$expected/?token=offline-test" ]] + [[ $(< "$state/web-port") == "$expected" ]] + [[ $(curl --fail --silent --max-time 5 "$login") == agent-vm-test ]] + if [[ -n $listener ]]; then + [[ $(curl --fail --silent --max-time 5 http://127.0.0.1:3080/) == occupied ]] + fi + "$launcher" stop + wait "$vm_pid" + vm_pid='' + [[ ! -e $state/web-port && ! -e $state/web.sock ]] + if "$launcher" url >/dev/null 2>&1; then echo 'Stale URL after shutdown' >&2; return 1; fi +} +# Pick the first hole, then exercise the inclusive upper endpoint. +reserve_through 3082 +expect_port 3083 +release_listeners +reserve_through 3099 +expect_port 3100 +release_listeners +# Exhaustion must fail, shut down its VM, and not leave a remembered URL. +reserve_through 3100 +if "$launcher" run > "$tmp/launcher.log" 2>&1; then echo 'Accepted a full port range' >&2; exit 1; fi +grep -Fq 'No available Web UI port on 127.0.0.1 in 3080-3100' "$tmp/launcher.log" +[[ ! -e $state/web-port && ! -e $state/web.sock ]] +if "$launcher" ssh true >/dev/null 2>&1; then echo 'VM survived failed launch' >&2; exit 1; fi +release_listeners +# A new run starts searching at 3080, not at the previously selected port. +expect_port 3080 +echo 'PASS: first available port, inclusive 3100 endpoint, full-range failure, real HTTP forwarding, URL persistence and cleanup, restart from 3080' diff --git a/agent-vm/tests.nix b/agent-vm/tests.nix index b9338d2..4f04166 100644 --- a/agent-vm/tests.nix +++ b/agent-vm/tests.nix @@ -16,8 +16,10 @@ let # Offline infrastructure test. A real DSH startup is tested separately; # @latest needs the network and is intentionally outside Nix reproducibility. agentVM.package = pkgs.writeShellScriptBin "dsh" '' - echo 'http://127.0.0.1:3080/?token=offline-test' - exec ${pkgs.coreutils}/bin/sleep infinity + exec ${pkgs.nodejs}/bin/node -e ' + require("node:http").createServer((req, res) => res.end("agent-vm-test")) + .listen(3080, "127.0.0.1", () => console.log("http://127.0.0.1:3080/?token=offline-test")); + ' ''; } ]; @@ -35,15 +37,38 @@ let guestAddress = "192.168.77.2"; gateway = "192.168.77.1"; dns = [ "192.168.77.1" ]; + hostAddress = "192.168.77.1"; + webPort = 3090; + webPortEnd = 3092; }; } ]; }; + fixed = inputs.self.lib.mkAgentVM { + system = pkgs.stdenv.hostPlatform.system; + project.packages = [ ]; + modules = [ { agentVM.network.webPort = 8080; } ]; + }; in { config = assert c.microvm.mem == 4096; assert c.microvm.vcpu == 4; + assert c.agentVM.network.webPort == 3080; + assert c.agentVM.network.webPortEnd == 3100; + assert fixed.nixos.config.agentVM.network.webPortEnd == 8080; + assert builtins.all + ( + port: + pkgs.lib.hasInfix "192.168.77.1:${toString port}" tap.nixos.config.systemd.services.agent.serviceConfig.ExecStart + ) + [ + 3090 + 3091 + 3092 + ]; + assert + !(pkgs.lib.hasInfix "192.168.77.1:3093" tap.nixos.config.systemd.services.agent.serviceConfig.ExecStart); assert builtins.length c.microvm.shares == 3; assert builtins.all (s: !s.readOnly && s.securityModel == "none") c.microvm.shares; assert c.microvm.storeOnDisk; @@ -74,7 +99,7 @@ in ]; } '' - shellcheck -s bash ${./launch.sh} ${./boot-test.sh} ${./playwright-test.sh} + shellcheck -s bash ${./launch.sh} ${./boot-test.sh} ${./playwright-test.sh} ${./port-test.sh} bash -n ${./launch.sh} touch "$out" ''; @@ -95,6 +120,23 @@ in bash ${./playwright-test.sh} ''; + ports = + pkgs.runCommand "agent-vm-port-check" + { + requiredSystemFeatures = [ "kvm" ]; + nativeBuildInputs = [ + pkgs.bash + pkgs.coreutils + pkgs.gnugrep + pkgs.nodejs + pkgs.curl + ]; + } + '' + bash ${./port-test.sh} ${testVM.package}/bin/agent-vm + touch "$out" + ''; + boot = pkgs.runCommand "agent-vm-boot-check" { diff --git a/templates/agent/README.md b/templates/agent/README.md index 1071bfe..e8bc89a 100644 --- a/templates/agent/README.md +++ b/templates/agent/README.md @@ -11,6 +11,11 @@ nix run .#agent -- ssh # root shell, starting in the same project cwd nix run .#agent -- stop ``` +The Web UI takes the first available port from **3080–3100**. The printed URL and +`url` command use the selected port; a full range fails cleanly. Configure +`agentVM.network.webPort` / `webPortEnd` to change the range (equal values mean a +fixed port). Multiple VMs still need distinct `sshPort` settings. + `$DSH_HOME` (default `~/.dsh`) and `${DSH_AGENTS_HOME:-~/.agents}/skills` are also mounted **read-write**. No other home directories or host sockets are shared. The first run creates missing DSH/skills directories. Existing DSH home must be diff --git a/templates/agent/flake.nix b/templates/agent/flake.nix index 8bb37d9..a4695e3 100644 --- a/templates/agent/flake.nix +++ b/templates/agent/flake.nix @@ -24,9 +24,11 @@ agentVM.network = { hostAddress = "127.0.0.1"; # Or a host LAN/VPN IPv4 address, or 0.0.0.0. sshPort = 2222; # Always host localhost in user-network mode. - webPort = 3080; - # For 0.0.0.0, list the actual browser authorities, not a wildcard: - # trustedHosts = [ "192.168.1.20:3080" ]; + webPort = 3080; # First available host Web port in this inclusive range. + webPortEnd = 3100; # Set equal to webPort to disable port hopping. + # For 0.0.0.0, list actual browser hosts, not a wildcard. + # A port-less host allows any selected port; host:port stays exact. + # trustedHosts = [ "192.168.1.20" ]; }; } ];