Soon after launching WolfBench, Wolfram Ravenwolf, AI Evangelist at CoreWeave, expanded the framework’s coverage to a wide range of models, agents, and scenarios. Running evaluations at a meaningful volume during that growth phase required dedicated sandbox infrastructure.

WolfBench tests a total of 89 scenarios across frontier AI model-agent combinations. Each combination is tested five times to control for statistical variance. On a local machine, these tests run one at a time, stretching a single model evaluation to hundreds of hours. With models being updated consistently, this delay would render WolfBench’s results irrelevant by the time they were ready.

Speed and scale weren’t the only reasons sandboxes were a necessity. Agents wrote files and modified system state as they worked, so they needed to operate in a clean, isolated environment to ensure valid test results. Once performance data was captured, sandboxes had to be torn down to avoid idle compute that would inflate costs.

To preserve the integrity of each evaluation cycle, every sandbox also had to be compatible with Harbor, an open-source evaluation harness that Wolfram used to orchestrate benchmark runs. Any compatibility gaps would require engineering work to keep the testing pipeline connected.

While WolfBench's inference was powered by CoreWeave, sandbox infrastructure wasn't part of the platform's offerings at the time. Building that capability internally would’ve diverted months of engineering time. So Wolfram set out to find a cloud sandbox provider to power WolfBench’s evaluation workflow in a period of rapid scale.

That’s when he discovered Daytona. The combination of their agent-native sandbox infrastructure and hands-on assistance convinced him he’d found what he was looking for.