Introducing Manifold

For robotics researchers, evaluation can be the slowest part of the research loop. Manifold makes it the fastest.

Today we're introducing Manifold, the platform for accelerating robotics research with failure analysis that explains itself.

The Loop Nobody Defends

The pattern is the same in almost every lab and frontier robotics team we talk to. You queue LIBERO-90 on a shared GPU before dinner, check it after breakfast, and find out around 9am whether yesterday's idea worked. If the node got preempted overnight, you find out nothing.

That works out to roughly one experiment per researcher per day, and the cap has nothing to do with how many ideas anyone has.

The cost doesn't stop at throughput. Getting to that one number means stitching configs, matching simulator builds, and rebuilding a harness someone wrote the week before a deadline. Six months later, even the original authors often can't regenerate their own results table. Everyone downstream cites it and hopes.

What Manifold Does

Manifold is the platform we've built to fix that:

• A month of experiments in a week.
• Simulate a thousand scenarios in 30 minutes.
• Dense labels on every atomic sub-goal and action.
• Agents that cluster failure modes across every rollout.

One eval harness for every benchmark, simulator, or world model, with native support for NVIDIA's Isaac Lab Arena.

One Harness, No Setup

Drop in a policy and run. No infrastructure to stand up, no simulator-specific configuration, no policy interface to write. Every simulator and benchmark ships pre-containerized, so the first run happens on day one instead of week three.

That matters most when you're comparing across suites. Today, running the same policy on LIBERO, RoboCasa, and an Isaac Lab task means three harnesses and three sets of assumptions about what counts as success. Manifold gives you one interface across all of them, which makes the numbers comparable by construction.

Sharded, Vectorized, Auto-Scaling

Rollouts run sharded and vectorized across GPUs, and the cluster scales itself to the size of the job. A thousand scenarios come back in about 30 minutes.

Because runs go out in parallel, testing 10 to 20 checkpoints costs roughly the same 30 minutes as testing one. Kick off a sweep before lunch, read the results with your coffee after.

The interesting part comes after the speedup. When an eval costs half an hour instead of a night, people stop rationing experiments. They run the ablation they'd normally skip and the full 90-task suite instead of a 10-task subset.

Failure Analysis That Explains Itself

A success rate tells you a policy is at 62%. It doesn't tell you that most of the misses are the same grasp failure on transparent objects, or that a ten-step task fails on step seven because step three quietly set up a state step seven could never recover from.

Manifold labels every atomic sub-goal and detected action across every rollout, then runs agents that cluster failures into modes: long-horizon drift, a bad handoff, a precision miss. You query failures instead of scrubbing hundreds of videos by hand, and you replay any rollout in 3D, powered by Rerun.

Knowing where a policy breaks is what feeds the next training run. That's the loop we're trying to close.

Results You Can Hand Someone Else

Every eval run gets a manifold:// URI that pins the policy checkpoint, simulator version, benchmark version, and seeds. Anyone with the link reruns the exact evaluation and gets the same numbers. Cite the URI and your baseline becomes something a reviewer, or a team three years from now, can check.

We've also precomputed verified baselines for pi0.5, gr00t-n1.5, openvla-7b, and octo-base across LIBERO and RoboCasa. Building a comparison table takes minutes instead of a week of resurrecting other people's eval code.

If you want the longer version of how we think about this, our guide on how to evaluate a VLA policy covers benchmark choice, rollout counts, and what to publish so someone else can reproduce your run.

Get Access

Manifold is in early access now, and it's free for researchers. We're onboarding research partners and frontier robotics teams in cohorts, and we're prioritizing groups working on VLA and manipulation policies.

Request access here.

We want the robotics research loop to move at the speed of compute. Running a thousand rollouts should be as easy as running one.

Bifrost builds simulation and evaluation infrastructure for physical AI. We've spent the last few years building this stack with teams like NASA and Honda.