VNC Access

View as Markdown

VNC (Virtual Network Computing) access provides a graphical desktop environment for your Daytona Sandbox directly in the browser. This allows you to interact with GUI applications, desktop tools, and visual interfaces running inside your sandbox.

VNC and computer use work together for manual and automated desktop interactions. VNC is the visual interface for users, while computer use is the programmatic API for AI agents to automate mouse, keyboard, and screenshot operations. Through VNC, you can watch agents perform computer use tasks in real time.

The desktop stack (D-Bus, Xvfb, the XFCE window manager, and the x11vnc server) ships with the sandbox runtime as a self-contained add-on, so it works on any Linux sandbox image, including custom images that carry no desktop packages. The virtual display runs at 1280x720.

Use cases

VNC access is used when a person needs to see or operate the desktop of a sandbox.

Computer-use agents: a person follows what an agent does on the desktop as it happens, from the browser.
Takeover from an agent: a person operates the desktop directly when an agent cannot continue, for example at a login screen.
Desktop setup before automation: a person signs in to applications or arranges the desktop by hand. An agent then continues on the same desktop through Computer Use.
GUI applications: desktop applications with no web or command-line interface run in the sandbox and are used from the browser.
Desktop test debugging: a person watches GUI tests run and inspects the desktop after a failure.
Several viewers: the VNC server runs in shared mode, so several viewers attach to the same desktop at once. A team watches one agent together.
Your own VNC client: any VNC client that supports WebSocket, such as noVNC, connects to the desktop through the Daytona proxy. An application embeds the desktop in its own interface.

Access the desktop from the Dashboard

Open the sandbox desktop directly from the Daytona Dashboard ↗.

Go to Daytona Sandboxes ↗
Open the sandbox you want to access. The sandbox must be in the running state.
Select the Desktop tab (also reachable via ?tab=desktop in the URL)
If the desktop is not started yet, click Start Desktop

The Dashboard polls the desktop status while it starts and then connects a VNC viewer in your browser, providing mouse and keyboard control over the sandbox’s graphical interface. Click into the frame to interact, and use Fullscreen for a larger view or Reconnect if the connection drops. If the desktop fails to start, the tab shows the failure cause and offers Restart Desktop.

Programmatic VNC management

Daytona provides methods to start, stop, monitor, and restart the desktop processes programmatically using the computer use references as part of automated workflows.

Start VNC

Start the desktop processes: D-Bus, Xvfb, the window manager, and the x11vnc server.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

result = sandbox.computer_use.start()
print("VNC processes started:", result.message)

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();

const result = await sandbox.computerUse.start();
console.log('VNC processes started:', result.message);

curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/start' \
  --request POST \
  --header 'Authorization: Bearer YOUR_API_KEY'

Stop VNC

Stop all desktop processes in the sandbox.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()

result = sandbox.computer_use.stop()
print("VNC processes stopped:", result.message)

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();

const result = await sandbox.computerUse.stop();
console.log('VNC processes stopped:', result.message);

curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/stop' \
  --request POST \
  --header 'Authorization: Bearer YOUR_API_KEY'

Get VNC status

Check whether the desktop is running. The status field is active or inactive, and the desktop field reports the lifecycle stage: not started, starting, ready, or failed.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

status = sandbox.computer_use.status()
print("VNC status:", status.status, status.desktop)

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();

const status = await sandbox.computerUse.status();
console.log('VNC status:', status.status, status.desktop);

curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/status' \
  --header 'Authorization: Bearer YOUR_API_KEY'

Restart VNC

Tear down the desktop processes and start a fresh desktop. Restart is valid from every state and is the only way out of the failed state. Application windows live inside the X server, so a restart returns an empty desktop. The response reports an active status and a ready desktop.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()
sandbox.computer_use.start()

result = sandbox.computer_use.restart()
print("VNC restarted:", result.message, result.desktop)

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();
await sandbox.computerUse.start();

const result = await sandbox.computerUse.restart();
console.log('VNC restarted:', result.message, result.desktop);

curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/restart' \
  --request POST \
  --header 'Authorization: Bearer YOUR_API_KEY'

Desktop states

The desktop field of the status response is the desktop lifecycle state. The status field is derived from it: active when the desktop is ready, inactive otherwise.

`desktop`	Meaning	Next step
`not_started`	No desktop is running. Also the state after a sandbox stop or start.	Call start.
`starting`	The desktop stack is coming up.	Poll status until `ready`.
`ready`	The desktop is running and the VNC server accepts connections.	Connect a viewer or run Computer Use operations.
`failed`	The desktop stopped working. `cause` carries `code` and `message`.	Call restart.

Only a failed desktop has a cause object, with the platform detail in its message.

Its code names what was lost:

`cause.code`	Meaning
`desktop_exited`	The X server (`xvfb`) exited. Windows live inside it, so the desktop is gone.
`addon_crashed`	The desktop supervisor died while a desktop existed.
`start_failed`	Starting the stack or probing its readiness failed.
`start_interrupted`	The supervisor died while the desktop was starting.
`state_corrupt`	The retained desktop state could not be read or validated.
`state_persistence_failed`	A desktop state transition could not be persisted.

Treat unknown codes as failures as well; the list grows additively.

Lifecycle rules:

Every Computer Use operation (mouse, keyboard, screenshot, display, accessibility, recording start) requires ready; otherwise it returns 409 DESKTOP_NOT_READY, raised by the SDKs as DesktopNotReadyError. Nothing retries it; a failed desktop cannot heal by retry.
On a ready desktop, start returns the process status; on starting or failed, 409 DESKTOP_NOT_READY. Retrying start cannot wipe a failed desktop.
stop kills a failed desktop’s processes, but only restart leaves failed.
dbus, wm, and x11vnc restart automatically with backoff if they exit; the desktop stays ready. xvfb never does, and restarting it per process returns 400; use restart.
Without the desktop add-on, computeruse endpoints return 503 ADDON_UNAVAILABLE.

from daytona import Daytona
from daytona.v2 import DesktopNotReadyError

daytona = Daytona()
sandbox = daytona.v2.create()

status = sandbox.computer_use.status()
if status.desktop == "failed":
    print("Desktop failed:", status.cause.code, status.cause.message)
    sandbox.computer_use.restart()

try:
    sandbox.computer_use.screen.capture()
except DesktopNotReadyError as e:
    print("Desktop not ready:", e)

import { Daytona } from '@daytona/sdk';
import { DesktopNotReadyError } from '@daytona/sdk/v2';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();

const status = await sandbox.computerUse.status();
if (status.desktop === 'failed') {
  console.log('Desktop failed:', status.cause.code, status.cause.message);
  await sandbox.computerUse.restart();
}

try {
  await sandbox.computerUse.screen.capture();
} catch (e) {
  if (e instanceof DesktopNotReadyError) console.log('Desktop not ready:', e.message);
}

# Failed desktop
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/status' \
  --header 'Authorization: Bearer YOUR_API_KEY'
# {"status":"inactive","desktop":"failed",
#  "cause":{"code":"desktop_exited","message":"Xvfb (:99) exited unexpectedly (signal 11)"}}

# Any action while not ready
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/computeruse/screenshot' \
  --header 'Authorization: Bearer YOUR_API_KEY'
# HTTP 409 {"statusCode":409,"code":"DESKTOP_NOT_READY",
#  "message":"desktop failed: Xvfb (:99) exited unexpectedly (signal 11); call POST /computeruse/restart", ...}

For additional process management operations including restarting individual processes and viewing logs, see the computer use reference.

Connect with your own VNC client

The sandbox’s VNC server is bridged through the Daytona proxy as a WebSocket at wss://toolbox.daytona.com/vnc/{sandboxId}, speaking the standard RFB protocol. Any WebSocket-capable VNC client (such as noVNC) can connect to it. Authenticate with your API key, either as an Authorization: Bearer <API_KEY> header or, for browser clients that cannot set headers, as an rlp_token query parameter:

wss://toolbox.daytona.com/vnc/{sandboxId}?rlp_token=<API_KEY>

The desktop processes must be started and the sandbox running. Before the WebSocket handshake completes, the proxy returns 401 for a missing or invalid credential, 404 for an unknown sandbox, 409 if it is not running, and 502 if the VNC server is unreachable.

The VNC server itself runs without a VNC password and in shared mode: the API key is the only credential, a viewer that prompts for a VNC password is not talking to the sandbox desktop, and several viewers can be attached to the same desktop at once. The bridge carries raw RFB in binary WebSocket frames and echoes the binary subprotocol when the client requests it.

VNC sessions and the idle timer

A connected VNC session counts as sandbox activity for the idle timeout. The proxy records activity when the session connects and whenever the viewer sends bytes. VNC viewers send protocol traffic while the user is idle, so an open session keeps the sandbox alive until it closes. Close the viewer, or the Dashboard’s Desktop tab, to let the idle timer run.

Automating desktop interactions

Once VNC is running, automate desktop interactions with Computer Use: AI agents can control the mouse and keyboard and capture screenshots within the VNC session.

Available operations:

Mouse: click, move, drag, scroll, and get cursor position
Keyboard: type text, press keys, and execute hotkey combinations
Screenshot: capture full screen, regions, or compressed images
Display: get display information and list open windows

For complete documentation on automating desktop interactions, see computer use.

Example: Automated browser interaction

Python
TypeScript

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

# Start VNC processes
sandbox.computer_use.start()

# Click to open browser
sandbox.computer_use.mouse.click(x=50, y=50, button="left")

# Type a URL
sandbox.computer_use.keyboard.type(text="https://www.daytona.io/docs/")
sandbox.computer_use.keyboard.key(key="enter")

# Take a screenshot
screenshot = sandbox.computer_use.screen.capture()

start() returns once the X server and the VNC server accept connections.
Later calls do not need to poll the status.
On a failed desktop, call restart() instead of start().

import { Daytona } from '@daytona/sdk';

const daytona = new Daytona();
const sandbox = await daytona.v2.create();

// Start VNC processes
await sandbox.computerUse.start();

// Click to open browser
await sandbox.computerUse.mouse.click({ x: 50, y: 50, button: 'left' });

// Type a URL
await sandbox.computerUse.keyboard.type({ text: 'https://www.daytona.io/docs/' });
await sandbox.computerUse.keyboard.key({ key: 'enter' });

// Take a screenshot
const screenshot = await sandbox.computerUse.screen.capture();

start() resolves once the X server and the VNC server accept connections.
Later calls do not need to poll the status.
On a failed desktop, call restart() instead of start().