Snorkel was built to deliver frontier-grade datasets, training environments, and evaluations to leading AI teams that rely on agent simulations to improve models in real-world conditions. To produce reliable results, each simulation must run in its own configurable, isolated sandbox. 

For Rustem Feyzkhanov, AI Platform Engineering Manager, and his team, provisioning those sandboxes at a scalable cadence was a standing priority. Snorkel’s agent tasks range from writing Python code to navigating filesystems, and simulations often run for long periods depending on complexity. With thousands of examples in Snorkel’s evaluation pipeline, running these tasks sequentially wasn't an option.

This need for parallelization came with strict isolation requirements. By design, agents can take unpredictable execution paths and interfere with each other’s workloads, corrupting evaluation outputs. Because these cases are hard to spot in production, Rustem needed to ensure datasets wouldn’t require complex debugging later.

Each task also carried its own CPU, RAM, and dependency requirements, so sandboxes had to be configurable. A one-size-fits-all runtime could bottleneck complex workloads or skew simulation outcomes. Runtime performance wasn't the only consideration. Snorkel's customers scrutinize every layer of the infrastructure stack, including their partners’ tools. For a runtime platform to pass compliance reviews, its codebase had to be transparent.

Rustem knew building an in-house runtime solution to meet these needs would require complex, resource-intensive VM networking and workload scheduling. To keep Snorkel’s focus on producing frontier training datasets and environments, he started looking for a dedicated runtime platform that would deliver fast, scalable, and stateful infrastructure.

That search ultimately led him to Daytona. Their open-source runtime infrastructure provided the flexibility and performance Snorkel needed to parallelize agent simulations.