ComputerUse

class ComputerUse()

Computer Use functionality for interacting with the desktop environment.

Provides access to mouse, keyboard, screenshot, and display operations for automating desktop interactions within a sandbox.

Attributes:

mouse Mouse - Mouse operations interface.
keyboard Keyboard - Keyboard operations interface.
screenshot Screenshot - Screenshot operations interface.
display Display - Display operations interface.

ComputerUse.start

@intercept_errors(message_prefix="Failed to start computer use: ")
def start() -> ComputerUseStartResponse

Starts all computer use processes (Xvfb, xfce4, x11vnc, novnc).

Returns:

ComputerUseStartResponse - Computer use start response.

Example:

result = sandbox.computer_use.start()
print("Computer use processes started:", result.message)

ComputerUse.stop

@intercept_errors(message_prefix="Failed to stop computer use: ")
def stop() -> ComputerUseStopResponse

Stops all computer use processes.

Returns:

ComputerUseStopResponse - Computer use stop response.

Example:

result = sandbox.computer_use.stop()
print("Computer use processes stopped:", result.message)

ComputerUse.get_status

@intercept_errors(message_prefix="Failed to get computer use status: ")
def get_status() -> ComputerUseStatusResponse

Gets the status of all computer use processes.

Returns:

ComputerUseStatusResponse - Status information about all VNC desktop processes.

Example:

response = sandbox.computer_use.get_status()
print("Computer use status:", response.status)

ComputerUse.get_process_status

@intercept_errors(message_prefix="Failed to get process status: ")
def get_process_status(process_name: str) -> ProcessStatusResponse

Gets the status of a specific VNC process.

Arguments:

process_name str - Name of the process to check.

Returns:

ProcessStatusResponse - Status information about the specific process.

Example:

xvfb_status = sandbox.computer_use.get_process_status("xvfb")
no_vnc_status = sandbox.computer_use.get_process_status("novnc")

ComputerUse.restart_process

@intercept_errors(message_prefix="Failed to restart process: ")
def restart_process(process_name: str) -> ProcessRestartResponse

Restarts a specific VNC process.

Arguments:

process_name str - Name of the process to restart.

Returns:

ProcessRestartResponse - Process restart response.

Example:

result = sandbox.computer_use.restart_process("xfce4")
print("XFCE4 process restarted:", result.message)

ComputerUse.get_process_logs

@intercept_errors(message_prefix="Failed to get process logs: ")
def get_process_logs(process_name: str) -> ProcessLogsResponse

Gets logs for a specific VNC process.

Arguments:

process_name str - Name of the process to get logs for.

Returns:

ProcessLogsResponse - Process logs.

Example:

logs = sandbox.computer_use.get_process_logs("novnc")
print("NoVNC logs:", logs)

ComputerUse.get_process_errors

@intercept_errors(message_prefix="Failed to get process errors: ")
def get_process_errors(process_name: str) -> ProcessErrorsResponse

Gets error logs for a specific VNC process.

Arguments:

process_name str - Name of the process to get error logs for.

Returns:

ProcessErrorsResponse - Process error logs.

Example:

errors = sandbox.computer_use.get_process_errors("x11vnc")
print("X11VNC errors:", errors)

Mouse

class Mouse()

Mouse operations for computer use functionality.

Mouse.get_position

@intercept_errors(message_prefix="Failed to get mouse position: ")
def get_position() -> MousePositionResponse

Gets the current mouse cursor position.

Returns:

MousePositionResponse - Current mouse position with x and y coordinates.

Example:

position = sandbox.computer_use.mouse.get_position()
print(f"Mouse is at: {position.x}, {position.y}")

Mouse.move

@intercept_errors(message_prefix="Failed to move mouse: ")
def move(x: int, y: int) -> MousePositionResponse

Moves the mouse cursor to the specified coordinates.

Arguments:

x int - The x coordinate to move to.
y int - The y coordinate to move to.

Returns:

MousePositionResponse - Position after move.

Example:

result = sandbox.computer_use.mouse.move(100, 200)
print(f"Mouse moved to: {result.x}, {result.y}")

Mouse.click

@intercept_errors(message_prefix="Failed to click mouse: ")
def click(x: int,
          y: int,
          button: str = "left",
          double: bool = False) -> MouseClickResponse

Clicks the mouse at the specified coordinates.

Arguments:

x int - The x coordinate to click at.
y int - The y coordinate to click at.
button str - The mouse button to click (‘left’, ‘right’, ‘middle’).
double bool - Whether to perform a double-click.

Returns:

MouseClickResponse - Click operation result.

Example:

# Single left click
result = sandbox.computer_use.mouse.click(100, 200)

# Double click
double_click = sandbox.computer_use.mouse.click(100, 200, "left", True)

# Right click
right_click = sandbox.computer_use.mouse.click(100, 200, "right")

Mouse.drag

@intercept_errors(message_prefix="Failed to drag mouse: ")
def drag(start_x: int,
         start_y: int,
         end_x: int,
         end_y: int,
         button: str = "left") -> MouseDragResponse

Drags the mouse from start coordinates to end coordinates.

Arguments:

start_x int - The starting x coordinate.
start_y int - The starting y coordinate.
end_x int - The ending x coordinate.
end_y int - The ending y coordinate.
button str - The mouse button to use for dragging.

Returns:

MouseDragResponse - Drag operation result.

Example:

result = sandbox.computer_use.mouse.drag(50, 50, 150, 150)
print(f"Dragged from {result.from_x},{result.from_y} to {result.to_x},{result.to_y}")

Mouse.scroll

@intercept_errors(message_prefix="Failed to scroll mouse: ")
def scroll(x: int, y: int, direction: str, amount: int = 1) -> bool

Scrolls the mouse wheel at the specified coordinates.

Arguments:

x int - The x coordinate to scroll at.
y int - The y coordinate to scroll at.
direction str - The direction to scroll (‘up’ or ‘down’).
amount int - The amount to scroll.

Returns:

bool - Whether the scroll operation was successful.

Example:

# Scroll up
scroll_up = sandbox.computer_use.mouse.scroll(100, 200, "up", 3)

# Scroll down
scroll_down = sandbox.computer_use.mouse.scroll(100, 200, "down", 5)

Keyboard

class Keyboard()

Keyboard operations for computer use functionality.

Keyboard.type

@intercept_errors(message_prefix="Failed to type text: ")
def type(text: str, delay: Optional[int] = None) -> None

Types the specified text.

Arguments:

text str - The text to type.
delay int - Delay between characters in milliseconds.

Raises:

DaytonaError - If the type operation fails.

Example:

try:
    sandbox.computer_use.keyboard.type("Hello, World!")
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

# With delay between characters
try:
    sandbox.computer_use.keyboard.type("Slow typing", 100)
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

Keyboard.press

@intercept_errors(message_prefix="Failed to press key: ")
def press(key: str, modifiers: Optional[List[str]] = None) -> None

Presses a key with optional modifiers.

Arguments:

key str - The key to press (e.g., ‘Enter’, ‘Escape’, ‘Tab’, ‘a’, ‘A’).
modifiers List[str] - Modifier keys (‘ctrl’, ‘alt’, ‘meta’, ‘shift’).

Raises:

DaytonaError - If the press operation fails.

Example:

# Press Enter
try:
    sandbox.computer_use.keyboard.press("Return")
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

# Press Ctrl+C
try:
    sandbox.computer_use.keyboard.press("c", ["ctrl"])
    print(f"Operation success")

# Press Ctrl+Shift+T
try:
    sandbox.computer_use.keyboard.press("t", ["ctrl", "shift"])
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

Keyboard.hotkey

@intercept_errors(message_prefix="Failed to press hotkey: ")
def hotkey(keys: str) -> None

Presses a hotkey combination.

Arguments:

keys str - The hotkey combination (e.g., ‘ctrl+c’, ‘alt+tab’, ‘cmd+shift+t’).

Raises:

DaytonaError - If the hotkey operation fails.

Example:

# Copy
try:
    sandbox.computer_use.keyboard.hotkey("ctrl+c")
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

# Paste
try:
    sandbox.computer_use.keyboard.hotkey("ctrl+v")
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

# Alt+Tab
try:
    sandbox.computer_use.keyboard.hotkey("alt+tab")
    print(f"Operation success")
except Exception as e:
    print(f"Operation failed: {e}")

Screenshot

class Screenshot()

Screenshot operations for computer use functionality.

Screenshot.take_full_screen

@intercept_errors(message_prefix="Failed to take screenshot: ")
def take_full_screen(show_cursor: bool = False) -> ScreenshotResponse

Takes a screenshot of the entire screen.

Arguments:

show_cursor bool - Whether to show the cursor in the screenshot.

Returns:

ScreenshotResponse - Screenshot data with base64 encoded image.

Example:

screenshot = sandbox.computer_use.screenshot.take_full_screen()
print(f"Screenshot size: {screenshot.width}x{screenshot.height}")

# With cursor visible
with_cursor = sandbox.computer_use.screenshot.take_full_screen(True)

Screenshot.take_region

@intercept_errors(message_prefix="Failed to take region screenshot: ")
def take_region(region: ScreenshotRegion,
                show_cursor: bool = False) -> ScreenshotResponse

Takes a screenshot of a specific region.

Arguments:

region ScreenshotRegion - The region to capture.
show_cursor bool - Whether to show the cursor in the screenshot.

Returns:

ScreenshotResponse - Screenshot data with base64 encoded image.

Example:

region = ScreenshotRegion(x=100, y=100, width=300, height=200)
screenshot = sandbox.computer_use.screenshot.take_region(region)
print(f"Captured region: {screenshot.region.width}x{screenshot.region.height}")

Screenshot.take_compressed

@intercept_errors(message_prefix="Failed to take compressed screenshot: ")
def take_compressed(
        options: Optional[ScreenshotOptions] = None) -> ScreenshotResponse

Takes a compressed screenshot of the entire screen.

Arguments:

options ScreenshotOptions - Compression and display options.

Returns:

ScreenshotResponse - Compressed screenshot data.

Example:

# Default compression
screenshot = sandbox.computer_use.screenshot.take_compressed()

# High quality JPEG
jpeg = sandbox.computer_use.screenshot.take_compressed(
    ScreenshotOptions(format="jpeg", quality=95, show_cursor=True)
)

# Scaled down PNG
scaled = sandbox.computer_use.screenshot.take_compressed(
    ScreenshotOptions(format="png", scale=0.5)
)

Screenshot.take_compressed_region

@intercept_errors(
    message_prefix="Failed to take compressed region screenshot: ")
def take_compressed_region(
        region: ScreenshotRegion,
        options: Optional[ScreenshotOptions] = None) -> ScreenshotResponse

Takes a compressed screenshot of a specific region.

Arguments:

region ScreenshotRegion - The region to capture.
options ScreenshotOptions - Compression and display options.

Returns:

ScreenshotResponse - Compressed screenshot data.

Example:

region = ScreenshotRegion(x=0, y=0, width=800, height=600)
screenshot = sandbox.computer_use.screenshot.take_compressed_region(
    region,
    ScreenshotOptions(format="webp", quality=80, show_cursor=True)
)
print(f"Compressed size: {screenshot.size_bytes} bytes")

Display

class Display()

Display operations for computer use functionality.

Display.get_info

@intercept_errors(message_prefix="Failed to get display info: ")
def get_info() -> DisplayInfoResponse

Gets information about the displays.

Returns:

DisplayInfoResponse - Display information including primary display and all available displays.

Example:

info = sandbox.computer_use.display.get_info()
print(f"Primary display: {info.primary_display.width}x{info.primary_display.height}")
print(f"Total displays: {info.total_displays}")
for i, display in enumerate(info.displays):
    print(f"Display {i}: {display.width}x{display.height} at {display.x},{display.y}")

Display.get_windows

@intercept_errors(message_prefix="Failed to get windows: ")
def get_windows() -> WindowsResponse

Gets the list of open windows.

Returns:

WindowsResponse - List of open windows with their IDs and titles.

Example:

windows = sandbox.computer_use.display.get_windows()
print(f"Found {windows.count} open windows:")
for window in windows.windows:
    print(f"- {window.title} (ID: {window.id})")

ScreenshotRegion

class ScreenshotRegion()

Region coordinates for screenshot operations.

Attributes:

x int - X coordinate of the region.
y int - Y coordinate of the region.
width int - Width of the region.
height int - Height of the region.

ScreenshotOptions

class ScreenshotOptions()

Options for screenshot compression and display.

Attributes:

show_cursor bool - Whether to show the cursor in the screenshot.
fmt str - Image format (e.g., ‘png’, ‘jpeg’, ‘webp’).
quality int - Compression quality (0-100).
scale float - Scale factor for the screenshot.