Computer Use

View as Markdown

Computer Use enables programmatic control of desktop environments within Daytona sandboxes. It provides mouse, keyboard, screenshot, and display operations for automating GUI interactions and testing desktop applications.

We recommend using this instance type if you need to interact with Linux, Windows, or macOS applications.

Common use cases

GUI Application Testing - Automate interactions with native macOS applications, click buttons, fill forms, and validate UI behavior
Visual Testing & Screenshots - Capture screenshots of applications, compare UI states, and perform visual regression testing
Desktop Automation - Automate repetitive macOS desktop tasks, file management through GUI, and complex workflows

Getting started

Once you have access to the Computer Use Alpha, you’ll be able to programmatically control desktop environments within your Daytona sandboxes. The functionality will include mouse operations, keyboard automation, screenshot capture, and display management specifically optimized for the operating system of your choice.

SDK References

Choose your preferred SDK to get started with computer use automation:

TypeScript SDK

Complete API reference for computer use operations in TypeScript.

View TypeScript SDK →

Python SDK

Complete API reference for computer use operations in Python (sync & async).

View Python SDK →

Quick Example

from daytona import Daytona, DaytonaConfig

# Initialize using environment variables (recommended)
daytona = DaytonaConfig()
# Or explicitly configure:
# daytona = Daytona(
#     api_key=os.getenv('DAYTONA_API_KEY'),
#     api_url=os.getenv('DAYTONA_API_URL', 'https://app.daytona.io/api')
# )

# Create sandbox from default snapshot (includes desktop environment)
sandbox = daytona.create()

# Start computer use processes
result = sandbox.computer_use.start()
print("Computer use started:", result.message)

# Take a screenshot
screenshot = sandbox.computer_use.screenshot.take_full_screen()

# Click and type
sandbox.computer_use.mouse.click(100, 200)
sandbox.computer_use.keyboard.type('Hello, Linux!')
sandbox.computer_use.keyboard.hotkey('ctrl+s')

import { Daytona } from '@daytonaio/sdk';

// Initialize using environment variables (recommended)
const daytona = new Daytona();
// Or explicitly configure:
// const daytona = new Daytona({
//     apiKey: process.env.DAYTONA_API_KEY,
//     apiUrl: process.env.DAYTONA_API_URL || 'https://app.daytona.io/api'
// });

// Create sandbox from default snapshot (includes desktop environment)
const sandbox = await daytona.create();

// Start computer use processes
await sandbox.computerUse.start();

// Take a screenshot
const screenshot = await sandbox.computerUse.screenshot.takeFullScreen();

// Click and type
await sandbox.computerUse.mouse.click(100, 200);
await sandbox.computerUse.keyboard.type('Hello, Linux!');
await sandbox.computerUse.keyboard.hotkey('ctrl+s');

require 'daytona'

# Initialize using environment variables (recommended)
daytona = Daytona::Daytona.new
# Or explicitly configure:
# config = Daytona::Config.new(
#   api_key: ENV['DAYTONA_API_KEY'],
#   api_url: ENV.fetch('DAYTONA_API_URL', 'https://app.daytona.io/api')
# )
# daytona = Daytona::Daytona.new(config)

# Create sandbox from default snapshot (includes desktop environment)
sandbox = daytona.create

# Start computer use processes
result = sandbox.computer_use.start
puts "Computer use started: #{result.message}"

# Take a screenshot
screenshot = sandbox.computer_use.screenshot.take_full_screen

# Click and type
sandbox.computer_use.mouse.click(100, 200)
sandbox.computer_use.keyboard.type('Hello, Linux!')
sandbox.computer_use.keyboard.hotkey('ctrl+s')

Environment Variables (.env file):

DAYTONA_API_KEY=your-api-key-here
DAYTONA_API_URL=https://app.daytona.io/api