Log Streaming

View as Markdown

Log streaming lets you access and process logs of a running process in real time, or retrieve a snapshot of all logs produced so far. Use streaming for long-running or background commands. Use snapshots when you only need the current output.

Every process started through the process module writes its output to a cursor-addressed log ledger. Each log frame carries a channel (stdout and stderr for commands, pty for PTY processes), the data, and a cursor you can use to resume reading from where you left off. See log frames for the full list of fields and channels.

For entrypoint log streaming, see entrypoint process. To stream logs while sending input to a running command, see execute interactive commands.

Use cases

Log streaming is used when an application needs the output of a process while the process still runs.

Live output in a UI: build, test, or training output appears in a web application as the process writes it. Each frame carries a timestamp, so the UI shows when the output was written.
Agent feedback: an agent reads the output of a long command as it arrives and stops the command early on an error.
Reconnects: a client that loses its connection resumes from its last cursor. No frame is delivered twice, and no retained frame is missed.
Server logs: the output of a background web server or worker is followed for as long as the process runs.
Error handling: stdout and stderr arrive on separate channels, so an application handles errors apart from regular output.
Results of code runs: code executions stream charts, results, and exceptions on their own channels as JSON. An application renders each one as it arrives.
Binary output: base64 encoding returns the raw bytes of each frame. Output that is not valid UTF-8 text is read without loss.
Output after exit: a named process keeps its logs until it is deleted. An application reads the retained output of a finished or failed process later, page by page. Output beyond the retention cap evicts the oldest frames.

Stream logs

Stream the logs of a process. A stream replays the retained output from the beginning or from a cursor, then new output as it is written. It ends once the process exits and all output is delivered. While the process is silent, the daemon sends a keepalive every 15 seconds.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

handle = sandbox.process.start(
    'for i in {1..5}; do echo "Step $i"; echo "Error $i" >&2; sleep 1; done'
)

for frame in handle.stream_logs():
    print(f"[{frame.channel.upper()}]: {frame.data}", end="")

stream_logs() yields log frames only.
stream_logs() does not reopen a dropped connection.
Resume from the cursor of the last frame instead.

import asyncio

from daytona import AsyncDaytona


async def main() -> None:
    async with AsyncDaytona() as daytona:
        sandbox = await daytona.v2.create()

        handle = await sandbox.process.start(
            'for i in {1..5}; do echo "Step $i"; echo "Error $i" >&2; sleep 1; done'
        )

        async for frame in handle.stream_logs():
            print(f"[{frame.channel.upper()}]: {frame.data}", end="")


asyncio.run(main())

stream_logs() yields log frames only.
stream_logs() does not reopen a dropped connection.
Resume from the cursor of the last frame instead.

import { Daytona } from '@daytona/sdk'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

const handle = await sandbox.process.start(
  'for i in {1..5}; do echo "Step $i"; echo "Error $i" >&2; sleep 1; done',
)

for await (const event of handle.streamLogs()) {
  if (event.type === 'log') {
    process.stdout.write(`[${event.frame.channel.toUpperCase()}]: ${event.frame.data}`)
  }
}

streamLogs() yields every stream event, not only log frames.
Check event.type before reading event.frame.
streamLogs() yields a stream error event instead of throwing it.
The error field of a stream error event holds a DaytonaError.
A state event carries result, a WaitResult with reason, exitCode, signal, and rawReason, not the process record.
streamLogs() does not reopen a dropped connection.
Resume from the cursor of the last event instead.

# Run a command and stream its stdout and stderr until it exits
daytona sandbox exec {sandboxId} -- bash -c 'for i in {1..5}; do echo "Step $i"; echo "Error $i" >&2; sleep 1; done'

# Follow the logs of a running process in real time (SSE)
daytona sandbox process logs {sandboxId} {processId} --follow

The CLI writes stdout frames to its standard output and stderr frames to its standard error.
With --follow, the command exits with the exit code of the process.
If the global --timeout elapses first, the process keeps running.
The default timeout is 10m.
If retention evicts output that the stream has not delivered, the CLI prints a warning to standard error and continues from the oldest retained frame.
--follow supports only table and json output.

# Start a process
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes' \
  --request POST \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "kind": "exec",
  "shellCommand": "for i in {1..5}; do echo \"Step $i\"; echo \"Error $i\" >&2; sleep 1; done"
}'

# Follow process logs in real time (SSE)
curl -N 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?follow=true' \
  --header 'Authorization: Bearer YOUR_API_KEY'

The limit parameter does not apply to a follow stream.
If retention has evicted the beginning, the stream sends a warning event and continues from the oldest retained frame.

Stream events

Handle the events of a log stream. Each event carries a cursor, also sent as the event ID, so a client can resume after any event. A stream ends after its end event or its error event.

Event	Payload	Meaning
`log`	A log frame	One chunk of process output.
`state`	The process record with `exitCode`, `signal`, `reason`, and `cursor`	The process reached the `terminal` state. Sent once. Frames written before exit can still follow it.
`warning`	`cursor`, `message`, `firstAvailableCursor`	The requested position was evicted by log retention. The stream resumes from `firstAvailableCursor` on its own.
`eof`	`cursor`	The process is terminal and every frame has been delivered. The stream closes after this event.
`error`	`cursor`, `message`	The daemon failed to read the ledger. The stream closes after this event.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

handle = sandbox.process.start("for i in {1..3}; do echo \"Step $i\"; sleep 1; done")

# Only log frames are yielded; the generator returns after the eof event
for frame in handle.stream_logs():
    print(f"[{frame.channel.upper()}]: {frame.data}", end="")

# The exit result is read separately
result = handle.wait()
print(f"Exit code: {result.exit_code}, reason: {result.reason}")

import { Daytona } from '@daytona/sdk'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

const handle = await sandbox.process.start('for i in {1..3}; do echo "Step $i"; sleep 1; done')

for await (const event of handle.streamLogs()) {
  switch (event.type) {
    case 'log':
      process.stdout.write(`[${event.frame.channel.toUpperCase()}]: ${event.frame.data}`)
      break
    case 'state':
      console.log('Process finished:', event.result)
      break
    case 'warning':
      console.warn(`Older logs were evicted, resuming from ${event.firstAvailableCursor}`)
      break
    case 'eof':
      console.log('Stream complete at cursor', event.cursor)
      break
    case 'error':
      throw event.error
  }
}

curl -N 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?follow=true' \
  --header 'Authorization: Bearer YOUR_API_KEY'

# event: log
# id: djE6MQ
# data: {"cursor":"djE6MQ","channel":"stdout","timestamp":1757756400000,"data":"Step 1\n"}
#
# : keepalive
#
# event: state
# id: djE6Mw
# data: {"id":"p_...","kind":"exec","state":"terminal","exitCode":0,"reason":"exited","cursor":"djE6Mw",...}
#
# event: eof
# id: djE6Mw
# data: {"cursor":"djE6Mw"}

Resume a stream

Resume a stream from a cursor. A stream opened with a cursor starts strictly after that position, so no frame is delivered twice. An SSE client that reconnects sends the ID of the last event it received, and the daemon uses that ID in place of the cursor. If retention evicted the position, the stream warns and continues from the oldest retained frame.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

handle = sandbox.process.start("for i in {1..5}; do echo \"Step $i\"; sleep 1; done")

# Stop early, remembering the cursor of the last frame processed
last_cursor = None
for frame in handle.stream_logs():
    print(frame.data, end="")
    last_cursor = frame.cursor
    if frame.data.startswith("Step 3"):
        break

# Resume strictly after the last frame seen
for frame in handle.stream_logs(cursor=last_cursor):
    print(frame.data, end="")

import { Daytona } from '@daytona/sdk'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

const handle = await sandbox.process.start('for i in {1..5}; do echo "Step $i"; sleep 1; done')

// Stop early, remembering the cursor of the last event processed
let lastCursor: string | undefined
for await (const event of handle.streamLogs()) {
  lastCursor = event.cursor ?? lastCursor
  if (event.type !== 'log') continue
  process.stdout.write(event.frame.data)
  if (event.frame.data.startsWith('Step 3')) break
}

// Resume strictly after the last event seen
for await (const event of handle.streamLogs({ cursor: lastCursor })) {
  if (event.type === 'log') process.stdout.write(event.frame.data)
}

# --output json prints one frame per line, each with its cursor
daytona sandbox process logs {sandboxId} {processId} --follow --cursor {cursor} --output json

# Resume with the cursor query parameter
curl -N 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?follow=true&cursor={cursor}' \
  --header 'Authorization: Bearer YOUR_API_KEY'

# Resume with the SSE Last-Event-ID header; it takes precedence over cursor
curl -N 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?follow=true' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Last-Event-ID: {cursor}'

Retrieve all existing logs

Read the logs a process has produced so far. A read returns one page and does not wait for new output. Output longer than one page needs pagination.

# Logs produced up to the current point in time
page = handle.logs()
for frame in page.items:
    print(f"[{frame.channel.upper()}]: {frame.data}", end="")

A page holds up to 1000 frames by default.
The daemon clamps the page size to the range 1 to 5000.

// Logs produced up to the current point in time
const page = await handle.logs()
for (const frame of page.items) {
  process.stdout.write(`[${frame.channel.toUpperCase()}]: ${frame.data}`)
}

A page holds up to 1000 frames by default.
The daemon clamps the page size to the range 1 to 5000.

daytona sandbox process logs {sandboxId} {processId}

A page holds up to 1000 frames by default.
The daemon clamps the page size to the range 1 to 5000.

curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs' \
  --header 'Authorization: Bearer YOUR_API_KEY'

A page holds up to 1000 frames by default.
The daemon clamps the page size to the range 1 to 5000.

Paginate logs

Read logs page by page. Pass the next cursor of a page to read the page after it. An empty page means the read has caught up with the output; a process that is still running can produce more.

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

handle = sandbox.process.start("seq 1 5000")
handle.wait()

frames = []
cursor = None
while True:
    page = handle.logs(cursor=cursor, limit=500)
    frames.extend(page.items)
    if not page.items or page.next_cursor == cursor:
        break
    cursor = page.next_cursor

stdout = "".join(frame.data for frame in frames if frame.channel == "stdout")
print(f"{len(frames)} frames, {len(stdout)} bytes of stdout")

LogPage has no end-of-log flag.
Stop on an empty page, after the process has exited.

import { Daytona } from '@daytona/sdk'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

const handle = await sandbox.process.start('seq 1 5000')
await handle.wait()

const frames = []
let cursor: string | undefined
for (;;) {
  const page = await handle.logs({ cursor, limit: 500 })
  frames.push(...page.items)
  if (page.items.length === 0 || page.nextCursor === cursor) break
  cursor = page.nextCursor
}

const stdout = frames.filter((frame) => frame.channel === 'stdout').map((frame) => frame.data).join('')
console.log(`${frames.length} frames, ${stdout.length} bytes of stdout`)

LogsPage has no end-of-log flag.
Stop on an empty page, after the process has exited.

# First page, at most 500 frames; JSON output includes nextCursor and eof
daytona sandbox process logs {sandboxId} {processId} --limit 500 --output json

# Following pages, until eof is true
daytona sandbox process logs {sandboxId} {processId} --limit 500 --cursor {nextCursor} --output json

# First page, at most 500 frames
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?limit=500' \
  --header 'Authorization: Bearer YOUR_API_KEY'

# {
#   "frames": [
#     { "cursor": "djE6MQ", "channel": "stdout", "timestamp": 1757756400000, "data": "1\n2\n3\n" }
#   ],
#   "nextCursor": "djE6MQ",
#   "truncatedHead": false,
#   "eof": false
# }

# Following pages, until eof is true
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?cursor={nextCursor}&limit=500' \
  --header 'Authorization: Bearer YOUR_API_KEY'

Binary output

Read binary output without loss. The default text encoding replaces bytes that are not valid UTF-8. Base64 encoding returns the raw bytes of each frame as unpadded base64url, the URL-safe alphabet without trailing padding, so decode it with a decoder that accepts that form.

import base64

from daytona import Daytona

daytona = Daytona()
sandbox = daytona.v2.create()

handle = sandbox.process.start("head -c 1024 /dev/urandom")
handle.wait()

page = handle.logs(encoding="base64")
raw = b"".join(
    base64.urlsafe_b64decode(frame.data + "=" * (-len(frame.data) % 4))
    for frame in page.items
    if frame.channel == "stdout"
)
print(f"{len(raw)} bytes")

import { Daytona } from '@daytona/sdk'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

const handle = await sandbox.process.start('head -c 1024 /dev/urandom')
await handle.wait()

const page = await handle.logs({ encoding: 'base64' })
const raw = Buffer.concat(
  page.items
    .filter((frame) => frame.channel === 'stdout')
    .map((frame) => Buffer.from(frame.data, 'base64url')),
)
console.log(`${raw.length} bytes`)

# Frames carry unpadded base64url payloads; table output is not binary-safe, so use JSON
daytona sandbox process logs {sandboxId} {processId} --encoding base64 --output json

# Frames carry unpadded base64url payloads
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?encoding=base64' \
  --header 'Authorization: Bearer YOUR_API_KEY'

Log frames

Field	Description
`cursor`	Opaque position of this frame. Pass it as `cursor` to read strictly after this frame. `""` and `start` mean the beginning.
`channel`	Which stream the frame came from. See the channel table below.
`data`	The payload, as UTF-8 text or as an unpadded base64url string when `encoding` is `base64`.
`timestamp`	When the daemon recorded the frame, in Unix milliseconds.

Channel	Emitted by	Content
`stdout`	`exec` and `code` processes	Standard output. `exec` processes keep `stdout` and `stderr` on separate pipes, so ordering is preserved within a channel, not across channels.
`stderr`	`exec` and `code` processes	Standard error.
`pty`	PTY processes	The merged terminal byte stream, including escape sequences.
`setup`	The daemon	Progress messages emitted while the daemon provisions something the process needs, for example a language kernel before the first code execution.
`display`	Code executions	A kernel side output (`display_data` or `update_display_data`) as a JSON object keyed by media type.
`execute_result`	Code executions	The kernel’s primary result (`execute_result`) as a JSON object keyed by media type.
`error`	Code executions	A kernel exception as JSON: `{"ename", "evalue", "traceback"}`.

Log retention

Control how long a process record and its logs are kept. The retention policy is set when the process starts. Independent of the policy, retained output is capped in size; output beyond the cap evicts the oldest frames. A page read from an evicted position fails with an error that names the oldest retained position; a stream warns and continues from that position.

Policy	Default for	Behavior
`until_cleanup`	Processes started with a `name`, PTY processes started through the SDKs, and the entrypoint process	The record and logs stay until the process is deleted. A process that fails to start is downgraded to `on_exit_ttl` so failed records do not accumulate.
`on_exit_ttl`	Processes started without a `name`, and every `run()` call	The record and logs are removed 10 minutes after the process exits. Reading them afterwards fails with `404 PROCESS_NOT_FOUND`.
`none`		Logs are discarded the moment the process exits. The record stays for the same 10 minute window, so `wait()` still returns the exit result. `run()` never uses it.

from daytona import Daytona
from daytona.v2 import CursorExpiredError

daytona = Daytona()
sandbox = daytona.v2.create()

# Keep the logs of an unnamed process until delete()
handle = sandbox.process.start("seq 1 5000", keep_logs="until_cleanup")
handle.wait()

saved_cursor = "..."  # a cursor stored from an earlier read
try:
    page = handle.logs(cursor=saved_cursor)
except CursorExpiredError as e:
    # The position was evicted; restart from the oldest retained frame
    page = handle.logs(cursor=e.first_available_cursor)

handle.delete()

import { Daytona } from '@daytona/sdk'
import { CursorExpiredError } from '@daytona/sdk/v2'

const daytona = new Daytona()
const sandbox = await daytona.v2.create()

// Keep the logs of an unnamed process until delete()
const handle = await sandbox.process.start('seq 1 5000', { keepLogs: 'until_cleanup' })
await handle.wait()

const savedCursor = '...' // a cursor stored from an earlier read
let page
try {
  page = await handle.logs({ cursor: savedCursor })
} catch (error) {
  if (!(error instanceof CursorExpiredError)) throw error
  // The position was evicted; restart from the oldest retained frame
  page = await handle.logs({ cursor: error.firstAvailableCursor })
}

await handle.delete()

# Keep the logs until the record is deleted
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes' \
  --request POST \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "kind": "exec",
  "shellCommand": "seq 1 5000",
  "keepLogs": "until_cleanup"
}'

# Read from an evicted position
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}/logs?cursor={cursor}' \
  --header 'Authorization: Bearer YOUR_API_KEY'

# HTTP 409
# {
#   "code": "CURSOR_EXPIRED",
#   "message": "the requested position was evicted by log retention",
#   "firstAvailableCursor": "djE6NDA5Ng",
#   "source": "DAYTONA_DAEMON"
# }

# Delete the record and its logs
curl 'https://toolbox.daytona.com/toolbox/{sandboxId}/processes/{processId}' \
  --request DELETE \
  --header 'Authorization: Bearer YOUR_API_KEY'