Kubernetes Is the Wrong Primitive for Diverse Workloads

Introduction

Kubernetes is very good at maintaining and scaling replicas of a known service. Modern AI platforms often face a different problem: launching many distinct execution environments for short-lived jobs.

Consider three jobs arriving at the same compute platform.

1. A robotics simulation packaged in a custom container

2. A voice agent that depends on a precise combination of CUDA, Python, and model-serving software

3. A black-box container supplied by a customer

Each job may require a different image, runtime, filesystem, dependency set, and isolation boundary. To the platform, these are not replicas of one service but rather three distinct environments that must be materialized on demand.

This evaluation answers one main question:

When the environments are known ahead of time, how quickly can each platform launch them and run the first command?

Experiment Setup

We kept total launches fixed at 500 and varied environment diversity:

• 1 distinct environment launched 500 times

• 10 distinct environments launched 50 times each

• 100 distinct environments launched 5 times each

For Daytona, each environment was created as a snapshot before the timed evaluation. The timed path launched sandboxes from those snapshots.

For Kubernetes, each image was pulled onto the evaluation node before the timed evaluation. The timed path launched pods from local images, so image download was removed from the measured startup path.

This is the strongest practical comparison we have in the data: both systems know the working set ahead of time, and both avoid image download during the measured launch.

Results

The results show two consistent patterns. First, Daytona delivered much lower typical startup latency across every working-set size. Second, that advantage came with greater variability with most Daytona runs remaining fast, but with a small number taking longer.

Below shows startup times for a typical launch.

Daytona’s median time-to-first-command stayed close to one second:

• 1 environment: 1.05s

• 10 environments: 1.13s

• 100 environments: 1.10s

Kubernetes stayed in a higher but consistent band:

• 1 environment: 4.17s

• 10 environments: 4.27s

• 100 environments: 4.37s

At the median, Daytona was about 4x faster in all three cases.

Total Measured Launch Time

The median describes a typical launch, but small differences become more consequential when repeated across hundreds of jobs. The next chart sums the launch time across the same 500 sequential launches.

The median result also shows up in total measured launch time. Summing per-launch total_seconds across the 500 sequential launches:

• 1 environment: Daytona 16 minutes, Kubernetes 47 minutes

• 10 environments: Daytona 26 minutes, Kubernetes 48 minutes

• 100 environments: Daytona 31 minutes, Kubernetes 50 minutes

Daytona therefore spent substantially less cumulative time starting environments and running the first command. At platform scale, this translates into jobs beginning sooner, less compute time spent waiting on startup, and lower infrastructure costs.

Distribution At 100 Environments

To understand the variability in behavior, consider the full distribution for the 100-environment workload rather than a single summary statistic.

Most Daytona launches completed quickly, which produces its low median. A small fraction, however, took longer. Kubernetes showed the opposite shape with its launches were slower overall but clustered more tightly around four to five seconds.

This creates a clear tradeoff. Workloads that prioritize typical interactive latency benefit from Daytona’s much faster common case. Workloads with strict worst-case latency requirements must also account for its heavier tail.

Interpretation

The evaluation shows a workload boundary rather than a general weakness in Kubernetes.

Kubernetes is designed around pods, nodes, scheduling, and controllers. It is excellent for long-running services and repeated replicas. In this prepared evaluation, Kubernetes was predictable, but still had a startup floor of roughly four seconds before the first command completed.

Daytona is designed around prepared environments. With snapshots, it kept median time-to-first-command near one second even as the number of distinct environments grew. The tradeoff was a heavier tail, with occasional launches taking tens of seconds.

The takeaway:

Use Kubernetes to maintain and scale stable services. Use prepared sandboxes when the workload needs to launch many different short-lived environments on demand.

Limitations

These results apply to a specific workload shape: short-lived environments launched sequentially from a prepared working set. They do not measure concurrent launches, node autoscaling, scale from zero, model loading, application warm-up, or end-to-end request latency.

Kubernetes has many additional optimization paths. A production platform could use more nodes, registry mirrors, custom AMIs with common images preloaded, local image caches, lazy image loading, alternative container runtimes, different garbage-collection thresholds, or a scheduler strategy that places work on nodes where the image is already present. We did not try to exhaust that space. The tuning path is effectively endless, and each additional optimization makes the system more specialized to a particular workload and cluster shape.

The Kubernetes setup was intentionally simple: one EKS node, one launch at a time, and no node autoscaling. A larger cluster could improve some cases, but it also introduces cache-placement questions. If an image is present on one node but capacity is available on another, the scheduler and cache strategy become part of startup latency.

Conclusion

Diverse AI workloads increasingly need to launch exact environments rather than simply scale replicas of the same service. In this prepared-environment evaluation, Daytona maintained a median time-to-first-command of approximately 1.1 seconds across working sets of 1, 10, and 100 environments. Kubernetes, with images already present on the node, remained between 4.2 and 4.4 seconds.

Snapshot-based sandboxes reduced the typical startup floor for short-lived, diverse environments. Kubernetes remained predictable when images were already local, but incurred a higher per-launch cost and required more operational work to keep many distinct environments prepared.

The broader takeaway is simple:

Use Kubernetes to maintain and scale stable services. Use prepared sandboxes when the workload needs to launch many different short-lived environments on demand.

Methodology and Data

This post focuses on the primary results. The complete experimental configuration, measurement boundaries, infrastructure details, full metrics, and reproduction materials are available upon request.

Work on These Problems With Us

Launching an environment in one second is only one part of the problem. We are working on faster snapshot restoration, more predictable tail latency, efficient storage and distribution, stronger isolation, and infrastructure capable of running large numbers of agent workloads.

If these are the kinds of systems problems you want to work on, explore open roles here.