Run agent-desktop on a cloud desktop

agent-desktop lets an agent read a desktop app's accessibility tree as JSON and click or type on its elements. On a Prized box the apps run on the box's own desktop, not your laptop's: the agent can take the pointer and keyboard without taking yours, it keeps working with your laptop closed, and you watch it in the box's Desktop tab.

11. Start a box

Sign up and follow the tour, picking Small when it asks for a size: the command it shows installs the CLI and adds the box to your SSH config (Quickstart). Replace mango with your box's name.

22. Install agent-desktop on the box

prized ssh mango

On the box, paste this. It installs the accessibility bus (at-spi2-core), the agent-desktop release after checking its checksum, and an agent-desktop launcher that joins the desktop's session from any shell, since a shell over SSH, or your agent's, starts outside it.

sudo apt-get install -y -qq -o DPkg::Lock::Timeout=300 at-spi2-core
V=v1.1.0; A=agent-desktop-$V-x86_64-unknown-linux-gnu.tar.gz; D=~/.local/lib/agent-desktop; U=https://github.com/vercel-labs/agent-desktop/releases/download/$V
mkdir -p $D ~/.local/bin && cd $D && curl -fsSLO $U/$A && curl -fsSL $U/checksums.txt | grep " $A\$" | sha256sum -c - && tar xzf $A && rm $A && cd ~
cat > ~/.local/bin/agent-desktop <<'EOF'
#!/bin/sh
p=$(pgrep -u "$(id -u)" -x xfce4-session | head -n 1)
[ -n "$p" ] && eval "$(tr '\0' '\n' < "/proc/$p/environ" | grep -E '^(DISPLAY|DBUS_SESSION_BUS_ADDRESS|XDG_RUNTIME_DIR)=' | sed "s/^\([A-Z_]*\)=\(.*\)$/export \1='\2'/")"
exec "$HOME/.local/lib/agent-desktop/agent-desktop" "$@"
EOF
chmod +x ~/.local/bin/agent-desktop

For a newer release, change V to its tag from the releases page.

33. Open the desktop

On your machine:

prized desktop mango

It opens the box's Desktop tab in the dashboard; leave the command running. Open it after the install: an app that started before the accessibility bus never shows up to agent-desktop, so if the desktop was already open, run prized desktop mango --stop and open it again.

44. Check it sees the desktop

On the box:

agent-desktop status
agent-desktop list-apps
# "accessibility":{"state":"granted"}
# "apps":[{"name":"xfce4-session",...},{"name":"xfce4-panel",...},{"name":"thunar",...}

55. Let your agent drive it

Claude Code and Codex come preinstalled on the box; sign the one you use in there once (Agent sign-in). In the folder you start the agent from, tell it about agent-desktop (use AGENTS.md for Codex):

cat >> CLAUDE.md <<'EOF'
Use `agent-desktop` to see and operate desktop apps. Before the first use, run `agent-desktop skills get desktop` and `agent-desktop skills get platform`, and follow them. This desktop is a cloud box nobody else uses, so `--headed` input is allowed when a semantic action can't do the job.
EOF

Then start claude and give it a task. The box has Chrome, a terminal and the Thunar file manager; install other apps with apt, here the two this prompt uses:

sudo apt-get install -y -o DPkg::Lock::Timeout=300 gnome-calculator mousepad

Use the Calculator app to split a $184.50 dinner bill 6 ways with an 18% tip, then save the per-person amount to ~/Desktop/dinner.txt with Mousepad.

Apps that need a flag

An app shows up to agent-desktop only when its toolkit joins the accessibility bus.

For the content of web pages, a browser tool such as Playwright on the box is the better fit (Let an agent drive it).

Notes

• Headed input stays on the box. agent-desktop click <ref> --headed moves the desktop's pointer and focus, not yours, and you see it in the Desktop tab.
• Full-screen screenshots (agent-desktop screenshot --screen 0) are refused while the desktop's compositor is on; window screenshots work. To capture the whole screen, run xfconf-query -c xfwm4 -p /general/use_compositing -s false in a terminal on the desktop.
• Pause and resume keep the desktop as described on Desktop; a running desktop may count as activity for auto-pause.