# Use OpenAI Computer-Use Helpers with Daytona Desktops

This guide shows how to let an OpenAI model operate a browser running on a Daytona desktop. Daytona provides the desktop, and OpenAI's `ComputerToolset` helper runs the model's computer-use loop: it takes screenshots, sends them to the model, and executes the actions the model requests. The helper ships in the OpenAI Python SDK [3.31.0](https://github.com/openai/openai-python/releases/tag/v3.31.0) and the OpenAI Node SDK [7.33.0](https://github.com/openai/openai-node/releases/tag/v7.33.0). The toolset never creates or manages a sandbox, so your application stays in control of which actions the model may actually take.

Both SDKs document the driver contract in their "Connect a browser or desktop" provider pages: [Python](https://github.com/openai/openai-python/blob/v3.31.0/docs/browser-providers.md) and [Node](https://github.com/openai/openai-node/blob/v7.33.0/docs/browser-providers.md). This guide uses Python.

---

### 1. Workflow Overview

The [example](https://github.com/daytona/guides/tree/main/python/computer-use/openai-computer-use) is a single script. It creates a Daytona sandbox, starts its desktop, opens the Python documentation home page in Chromium, and connects a deliberately restrictive driver to OpenAI's native `computer` tool. The model may take screenshots and exactly one plain left click, and only after you answer a `y/N` consent prompt. The script then prints the model's answer, which should be the heading of the page that opened, and deletes the sandbox:

```bash
$ python main.py
Allow screenshots and one click? [y/N] y
The Python Tutorial
Deleted sandbox 05f343d2-e007-469e-8887-d221bf2f2737
```

The pieces that make this work:

- **Desktop (Daytona):** a sandbox with [Computer Use](https://www.daytona.io/docs/en/computer-use.md) started, serving screenshots and accepting mouse input.
- **Driver (your application):** a small class that implements the OpenAI driver methods and decides what each one is allowed to do.
- **Toolset (OpenAI):** `ComputerToolset`, which runs the screenshot-and-act loop against the model.

### 2. Project Setup

#### Clone the Repository

```bash
git clone https://github.com/daytona/guides.git
cd guides/python/computer-use/openai-computer-use
```

#### Configure Environment

The example reads three environment variables:

- **`OPENAI_API_KEY`**: your OpenAI key, from the [OpenAI Developer Platform](https://platform.openai.com/api-keys).
- **`OPENAI_MODEL`**: the model that drives the desktop. It must support the native `computer` tool.
- **`DAYTONA_API_KEY`**: your Daytona key, from the [Daytona Dashboard](https://app.daytona.io/dashboard/keys).

Copy `.env.example` to `.env` and add your keys:

```bash
OPENAI_API_KEY=your_openai_api_key
OPENAI_MODEL=gpt-6-astra
DAYTONA_API_KEY=your_daytona_key
```

Each run bills OpenAI model usage and Daytona sandbox runtime.

#### Install and Run

```bash
pip install "openai==3.31.0" "daytona==0.224.1" python-dotenv
python main.py
```

### 3. Understanding the Integration

#### Create the Desktop

The script creates a sandbox from the default snapshot, which already contains the Computer Use packages, then starts the desktop processes. `start()` returns before the desktop is usable, so the script polls `get_status()` until the status reports `active` and only then opens the browser. Chromium is launched on `DISPLAY=:0` in the background with its output redirected, so the call returns immediately, and the script waits for the page to load:

```python
daytona = Daytona()
sandbox = None
toolset = None
try:
    sandbox = daytona.create(
        CreateSandboxFromSnapshotParams(
            auto_stop_interval=20,
            auto_delete_interval=60,
        ),
        timeout=REQUEST_TIMEOUT,
    )
    sandbox.computer_use.start(request_timeout=REQUEST_TIMEOUT)

    deadline = time.monotonic() + 60
    while True:
        status = sandbox.computer_use.get_status(
            request_timeout=REQUEST_TIMEOUT
        ).status
        if status == "active":
            break
        if time.monotonic() > deadline:
            raise RuntimeError(
                "Computer use did not become active in time "
                f"(last status: {status})."
            )
        time.sleep(2)

    sandbox.process.exec(
        f"DISPLAY=:0 nohup chromium --no-sandbox --disable-dev-shm-usage "
        f"{START_URL} >/tmp/cu-browser.log 2>&1 </dev/null &",
        timeout=REQUEST_TIMEOUT,
    )
    time.sleep(10)
```

`auto_stop_interval=20` and `auto_delete_interval=60` bound the sandbox's lifetime, in minutes, if the script is interrupted before its cleanup runs.

#### Connect the Desktop

`DaytonaDriver` implements the nine methods `ComputerToolset` requires. Only two of them do anything: `screenshot` returns a base64 PNG in the `computer_screenshot` shape the toolset expects, and `click` allows a single plain left click inside the display bounds. Every other action name is aliased to `unsupported`, which raises:

```python
class DaytonaDriver:
    def __init__(self, computer, display):
        self.computer = computer
        self.display = display
        self.clicked = False

    def screenshot(self):
        image = self.computer.screenshot.take_full_screen(
            show_cursor=False,
            request_timeout=REQUEST_TIMEOUT,
        )
        if not image.screenshot:
            raise RuntimeError("Daytona returned no screenshot.")
        return {
            "type": "computer_screenshot",
            "image_url": f"data:image/png;base64,{image.screenshot}",
        }

    def click(self, action):
        # Limit the model to the single click required by this example.
        if self.clicked or action.button != "left" or action.keys:
            raise RuntimeError("Only one plain left click is allowed.")
        inside_x = 0 <= action.x < self.display.width
        inside_y = 0 <= action.y < self.display.height
        if not inside_x or not inside_y:
            raise RuntimeError("Click is outside the display.")
        self.clicked = True
        self.computer.mouse.click(
            action.x,
            action.y,
            button="left",
            request_timeout=REQUEST_TIMEOUT,
        )

    def unsupported(self, action):
        raise RuntimeError(f"Unsupported action: {action.type}")

    double_click = drag = keypress = move = unsupported
    scroll = type = wait = unsupported
```

#### Run the Model

The approval callbacks belong to `run()`, not to the `ComputerToolset` constructor. `before_action` is called before every action and returns `True` only for the two allowed types, and `approve_safety_checks` returns `False` so a pending safety check stops the run instead of being waved through:

```python
# Gate actions before dispatch even though unsupported driver methods also raise.
def before_action(action):
    return action.type in ("screenshot", "click")


# This example stops rather than approving pending model safety checks.
def approve_safety_checks(call):
    return False
```

The driver is bound to the one active display at `(0, 0)`, so its bounds check matches the screen the model sees. `ComputerToolset` then takes the driver and nothing else, and `run()` drives the loop:

```python
    displays = sandbox.computer_use.display.get_info(
        request_timeout=REQUEST_TIMEOUT
    ).displays or []
    active = [display for display in displays if display.is_active]
    if len(active) != 1 or active[0].x != 0 or active[0].y != 0:
        raise RuntimeError("Expected one active display at (0, 0).")
    if not active[0].width or not active[0].height:
        raise RuntimeError("Display dimensions are unavailable.")

    driver = DaytonaDriver(sandbox.computer_use, active[0])
    toolset = ComputerToolset(driver)
    request = {
        "model": os.environ["OPENAI_MODEL"],
        "store": False,
        "tools": [{"type": "computer"}],
        "tool_choice": "required",
        "input": TASK,
    }
    with OpenAI(timeout=30, max_retries=0) as client:
        response = toolset.run(
            client,
            request,
            max_steps=MAX_STEPS,
            before_action=before_action,
            approve_safety_checks=approve_safety_checks,
        )
    if not driver.clicked:
        raise RuntimeError("The model returned without clicking.")
    print(response.output_text)
```

`store: False` keeps the conversation out of OpenAI's storage, and `tool_choice: "required"` forces the model to use the computer tool rather than answering from memory. The script raises if `driver.clicked` is still `False`, which catches a model that answers without clicking.

#### Clean Up

A `finally` block closes the toolset and deletes the sandbox, so an error mid-run still leaves nothing behind. The deletion sits in its own nested `finally`, so it also runs if `toolset.close()` raises, and `wait=True` returns only once Daytona has removed the sandbox:

```python
finally:
    try:
        if toolset is not None:
            toolset.close()
    finally:
        if sandbox is not None:
            sandbox.delete(wait=True)
            print(f"Deleted sandbox {sandbox.id}")
```

### 4. Approvals and Safety

The consent prompt runs before anything else, including the first screenshot and the sandbox creation. Answer anything other than `y` and the script exits without contacting Daytona or OpenAI.

The driver, not the model, decides what happens on the desktop. It permits exactly one plain left click inside the display and raises on every other action, so a model that tries to type, scroll, or drag ends the run with an error. Pending model safety checks are never approved automatically; the callback returns `False` and the run stops. Review each check yourself before deciding to allow it.

Screenshots are sent to OpenAI as part of the model request, so treat everything visible on the desktop as shared with the model. The model's answer is not proof that it clicked the right thing. The example only checks, through `driver.clicked`, that the driver accepted one click; it does not check where the click landed, so verify the result independently before relying on it. Use a dedicated, disposable desktop with no real accounts, sessions, or credentials on it.

### 5. Adapt the Example

To let the model do more, extend the driver one method at a time with the same bounds checks, rather than loosening the safety callbacks. The callbacks are the last line of defence, and a permissive driver cannot be undone by a strict prompt.

For faster repeated runs, create a [Daytona snapshot](https://www.daytona.io/docs/en/snapshots.md) with the browser and any page state already in place, so each run skips the install and first-load wait. Restrict the sandbox's network to only the domains the task needs with the allow-list parameters on sandbox creation.

If you need more than screenshots and clicks, see Daytona's [Computer Use](https://www.daytona.io/docs/en/computer-use.md) page for the full mouse, keyboard, display, and screen-recording API.

**Key advantages:**

- Self-contained and disposable: the script creates and deletes its own desktop, with no manual setup
- Restrictive by default: the model can only take the actions your driver explicitly allows
- Decoupled: OpenAI owns the model loop, Daytona owns the desktop, and your application owns the policy