ASPIRE: Can you zero-shot robotics with a coding agent?

A simulated robot has to pick up a red radio. It finds the radio, 1.91 m away, and starts to drive over. Then navigate_to_pose returns PLANNING_ERROR. It does it again, and again.

The camera works and the grasp code works. The problem is the goal. Every spot the program picked lies within about 20 cm of the table edge, inside the planner's collision buffer, so the planner rejects all of them. The fix is small: approach the radio from other sides of it, and keep the first side that works. (ASPIRE paper, Fig. 2)

Normally that fix lives in one program for one task. The next task starts cold. NVIDIA's ASPIRE asks a different question: what if the robot kept the fix, in a form the next task can read?

Why this one matters

A coding agent can now program a robot to do new tasks, and it gets better each time a better model arrives.

Coding agents already changed how software gets written: the agent writes code, runs it, reads the failure and tries again. Other knowledge work followed, with agents zero-shotting problems that used to need a specialist. ASPIRE puts a robot at the end of that loop. It works on a long list of simulated tasks and on a few real ones.

• No robot policy gets trained. The behavior is a Python program that a coding agent wrote and debugged. The language model is pretrained and ASPIRE leaves it frozen, so "no RL" means no policy trained for these tasks, not no learning anywhere.
• The trained baselines trail it. In the paper's LIBERO-Pro runs, π0.5, a trained vision-language-action policy, scores 0% to 38% across six conditions. ASPIRE with Claude Opus 4.6 scores 45% to 98% (Table 5).
• It is open. At launch Jim Fan wrote, "We'll open-source the full stack so your own robot library starts compounding from ours!" (post, 30 June 2026). The repo is Apache 2.0, and v1.0 shipped on 15 October.

The Bitter Lesson and the tension

Rich Sutton's Bitter Lesson (2019) says "general methods that leverage computation are ultimately the most effective, and by a large margin." He adds that "building in how we think we think does not work in the long run."

Jim Fan's v1.0 post (16 October 2026) applies that to robots: "On the day Astra arrived, our lab switched the backend model and saturated all the benchmarks to ~99% over night." He closes with "Now you can also feel the Physical Bitter Lesson at home like we did on that fateful day ;)". His framing is that the language model, trained on data and compute, is the general method. ASPIRE is the thin layer that lets a better model drive a robot.

That framing meets a real tension. ASPIRE is full of human-built structure: a fixed API of perception, planning and control primitives, hand-written prompts for the actor, verifier and coordinator, and hand-set admission rules. The paper's own numbers lean toward Sutton, and this part is my inference:

• Swapping the model moved LIBERO-Pro Goal success from 15% to 99%. The engine and search together moved Opus 4.6 from 14% to 72%.
• With strong models, search changes success by at most 2% on Object, Goal and Spatial.
• The post's "~99%" fits Opus 5.5. For GPT-6 Astra the paper reports 98% on Object, Goal and Spatial, and 87.9% on Long (Tables 3, 4, 17 and Sec. 3.8).

The structure still sets a ceiling: the API bounds what a program can express.

What ASPIRE is

ASPIRE moves learning out of the weights and into text. Four facts define it:

• No policy network and no gradient descent. A coding agent, Claude Code or Codex, writes Python against a fixed robot API for perception, grasping and motion planning.
• "Training" is debugging. The agent runs its program in simulation, reads what went wrong, and rewrites it.
• The "trained model" is a folder of text notes. Each note records one verified fix. Later tasks read the notes as in-context guidance.
• The model stays frozen. All the learning sits in the notes, the programs and the search around a language model.

The authors are from NVIDIA, UMich, UIUC, UC Berkeley and CMU. Yuke Zhu, Linxi "Jim" Fan and Guanzhi Wang are the project leads. The code is Apache 2.0.

The loop

Terms in the drawing:

• Debug seeds are the simulated scenes the loop learns on. A seed fixes object poses, distractors and the starting state.
• Held-out seeds are fresh scenes. The final program runs on them once, and that run gives the reported score.
• K is the number of candidate programs per search round.

Each of the four parts fixes one gap in earlier robot coding agents:

1. Execution engine replaces "it failed" with "this call failed". It records every perception, planning and grasp call: the API, inputs, outputs, return status, and RGB frames before and after.
2. Skill library replaces discarded fixes with kept ones. Skills are induced from validated repairs, and the authors did not fix the taxonomy in advance. The paper lists localization tricks, prompts, grasp rules, navigation recovery, motion primitives and debugging workflows.
3. Evolutionary search replaces patching one bad strategy with trying several. It samples K programs per round and conditions the next round on the best ones and their remaining failures.
4. Coordinator replaces an arbitrary task order with a curriculum. At the end it reruns low-scoring tasks once with the final library, so early tasks benefit from later skills.

What a skill looks like

The radio fix becomes a note with a failure signature, a trigger and a repair. This is the "Multi-Angle Approach" entry from the paper's Appendix A, shortened:

skill:  multi-angle approach (navigation)
signature:
  navigate_to_pose returns PLANNING_ERROR,
  or "succeeds" but ends > 1 m from the target
apply when:
  the object is near walls, furniture or a boundary
repair:
  for angle in [0, +90, -90, +45, -45] degrees:
    goal = object - dist * rotate(unit, angle)
    if navigate_to_pose(goal): break

The note is not a radio-pickup program. It is a recovery pattern that a bottle or a soda can can reuse.

In the released repo the library is plain markdown in .claude/skills/iterative-debugging/skills/: grasp.md, localize.md, transport.md, manipulation.md. This entry comes from grasp.md, with the markdown removed and the lines rewrapped:

Grasp with close_gripper(), never set_gripper(0.0): they are
not equivalent. close_gripper() accepts a contact stall as
success: the jaw stops on the object, the call returns
ok=True, and the harness records the commanded gripper as closed.

close_gripper()                  # correct: contact stall counts as closed
# set_gripper(0.0)               # WRONG when grasping: times out,
                                 # leaves stale open command

The file mixes a starting code template with entries added during runs. I can't tell from the file which lines a person wrote and which an agent did.

An entry arrives by one route:

• A task agent proposes the skill and reports the failure mode, the validated fix and the success rate on debug seeds.
• A fresh verifier agent gets the library and the proposal, but not the proposer's code. It must reach the fix on 5 seeds within 2 attempts.
• The coordinator promotes the entry only if the verifier reproduced it. Otherwise it is refuted and stays out.

Search, in pseudocode

This is a simplified version of the paper's Algorithm 1.

history = [(p0, score0, trace0)]   # debug seeds
for round in 1..R:                 # R = budget
    candidates = propose(agent, task,
                         top3(history), library)  # K programs
    for p in candidates:
        score, trace = execute(p, debug_seeds)
        history.add(p, score, trace)
    if best_score(history) >= tau:  # tau = threshold
        break
skills = extract_validated_patterns(history)
final  = execute(best_program(history),
                 heldout_seeds)     # run once

Symbols in the pseudocode:

• $p_0$ is the first program, with score0 and trace0 from its run on the debug seeds.
• R is the number of search rounds, and tau is the success score at which the search stops.
• top3(history) returns the three best programs so far with their traces. library is the set of skill notes.
• K is the number of programs the agent proposes per round.

The held-out seeds appear in the last line only. That is how the paper keeps the reported success rate honest. Every LIBERO-Pro number below is a macro-average:

$$\bar{s} = \frac{1}{T}\sum_{t=1}^{T}\frac{1}{S}\sum_{s=1}^{S}\mathbf{1}\big[\text{task } t \text{ succeeds on held-out seed } s\big]$$

Here $T = 10$ tasks per suite and perturbation, and $S = 50$ held-out seeds. A seed fixes object poses, distractors and the initial state.

What the numbers say

The model sets the ceiling, and the loop and the library lift a weaker model toward it. Four benchmarks and one baseline appear below:

• LIBERO-Pro: tabletop manipulation in simulation with four suites: Object, Goal, Spatial and Long. A position perturbation changes where objects sit, for example a bowl in front of the plate. A task perturbation changes the instruction wording.
• Robosuite: contact-rich one-arm and two-arm tasks such as stacking, nut assembly and a two-arm handover.
• BEHAVIOR-1K: long household tasks for a mobile robot, such as navigating to a radio and picking it up.
• CaP-Agent0: the baseline coding agent from the same lab. It uses visual differencing, a predefined skill library and retries on each episode.

Two notes on the headline numbers:

• The abstract's "72%" on handover is the gain in points, from 20% to 92%. It isn't a success rate.
• The 99% comes from the newer project PDF and the v1.0 repo release. The arXiv page still shows only v1, which has the Opus 4.6 numbers. The paper's own wording is "99% average success across all 80 LIBERO-Pro tasks". The repo news line adds "selected Robosuite, BEHAVIOR-1K tasks" to the same sentence, but the paper reports those separately, at 100% on its representative tasks.

Against the baselines

The skill library is what moves zero-shot

The same Opus 4.6 agent solves LIBERO-Pro Long at 4.7% with an empty library. Seeding the library from more LIBERO-90 tasks lifts it (paper Table 8):

The average rises smoothly. Single tasks don't. In the paper's per-task table (Table 9), "Cream cheese + butter" under position perturbation goes 0%, 0%, 6%, 94% as the library grows. "Soup + tomato sauce" under the same perturbation goes 0%, 14%, 20%, 0%. A bigger library helped one task a lot and removed the gain on another.

The model decides most of it

The paper holds the environment, API, seeds and protocol fixed and swaps only the language model. The setup is 10 LIBERO-Pro Goal tasks, position perturbation, 50 held-out seeds each, an empty library and no search (Table 18):

Success goes from 15% to 99% across models, and price does not track it. Opus 5 costs $597 for 90.6%. Sonnet 5.5 costs $121 for 98.8%. Search matters less as models improve. On Object, Goal and Spatial it changes success by at most 2% for Opus 5, GPT-6 Astra and Opus 5.5, against up to 34% for Opus 4.6.

Stronger models also change strategy. On "put the book under the low shelf", Opus 5.5 knocks the book flat without grasping it and pushes it under (68% success). Opus 4.6 carries it above the cabinet and stalls (0%). No model finds the human's move of tilting the book by its top edge. (Sec. 3.9, Fig. 10: one episode per row, so it shows strategies, not matched trials.)

On a real robot

Skills cut debugging cost on all three real tasks and raised success on two. The tests use a bimanual YAM station, a different embodiment and API from the Franka simulation. The agent is Codex with GPT-5.5. Skills found in simulation are given as in-context guidance. The agent still debugs through real execution (paper Table 1, 20 evaluation trials):

For the drawer, the no-skill run used up its token budget without producing a successful evaluation program.

The Rubik's Cube result is a different test. It is online learning on one real session of three episodes with GPT-6 Astra. Here is how it ran:

• What it was given. A human wrote text guidance first, and a symbolic solver (Kociemba) supplied the move sequence. The agent had to turn those moves into physical turns.
• What it learned. It wrote text skills for things like two-layer support, face-axis alignment and slip recovery. The paper says this is "context refinement rather than model-weight updates".
• Outcome. Episode 1 stayed unsolved after 50 min 54 s. Episodes 2 and 3 solved the cube, in 39 min and 77 min 10 s.
• Efficiency. The episodes completed 3, 5 and 25 face turns. Time per turn fell from 16 min 58 s to 7 min 48 s to 3 min 5 s. Tokens per turn fell from 11.27M to 2.01M, and estimated cost from $16.85 to $2.89.

Each episode had a different number of turns, so the per-turn drop is a trend within one session.

Drawbacks

ASPIRE is slow, and it has not been shown on tasks that need fast reflexes. It works when the model is strong and the API covers the task.

• Building a program is slow. The agent spends 31 to 139 minutes per task on the debugging loop, depending on the model (chart below). The paper also lists the search as compute-intensive, with many LLM calls and rollouts per task.
• Real-robot sessions are slow too. The Rubik's Cube episodes took 50 min 54 s unsolved, then 39 min and 77 min 10 s.
• Deliberation is slow and the program is fast, up to a point. In the simulated LIBERO-Pro and Robosuite runs, the finished program runs without a language model. In BEHAVIOR-1K the agent writes each next code block from the current trace, and in the Rubik's Cube setup the agent issues each action. There the language model sits in the loop and every step waits on it.
• Fast dynamic tasks are untested, so this is my inference. My bet is that juggling still needs a learned low-level policy. The paper tests nothing like it. Programs call primitives such as move_to_pose, which plan a trajectory and wait for a result, and the arm tracks position targets streamed to its motor drivers (Appendix D). A reflex that corrects within a fraction of a second has to live below that layer. A learned low-level policy is one way to get it, and I know of no result that lets ASPIRE do without one.

The other limits are about the model, the API and verification:

• It depends on the model. The authors write that ASPIRE "relies on a frozen LLM" and that their best results use proprietary frontier models. The best open-weight model in the study reaches 78%.
• Weak models are noisy. Three independent Opus 4.6 runs from an empty library score 59.2%, 67.4% and 62.8% on the same suite. On "open middle drawer" the same setup gave 0%, 100% and 0% (Table 25).
• A skill is verified on its source task. The repo says it directly: passing "shows the text transmits, not that it transfers". Transfer shows up only later, as other tasks grade the skill useful.
• The API bounds the robot. Programs use a predefined set of perception, planning and control primitives. A behavior outside that set needs a human to extend the API.
• Libraries go stale. Old entries can turn overly specific or misleading, and the authors say retrieval and pruning are not solved.
• The real world is not a loop yet. The paper says ASPIRE is "not yet a fully autonomous real-world lifelong learner". Real runs still need success detection, safe resets and calibration.
• It runs generated code. The repo warns that ASPIRE executes model-written Python with full import access. Run it on an isolated host.

Side note: people don't run fixed programs

This section is my inference, not a paper result. My view: the missing piece is a fast loop, and nothing in ASPIRE rules out the agent writing one.

• People run fast feedback loops. We don't execute a fixed program. We correct as we move, and slower thinking above that picks what to do.
• A program can hold its own loop. Code that an LLM writes can read a sensor, correct and repeat. ASPIRE's stored skills already do a little of this: grasp, lift, then re-localize the object to check it moved with the gripper.
• Those loops run at the program's pace. They tick in seconds. Underneath, the planner, the inverse kinematics and the streaming of position targets to the motors were written by people.
• Open question. Can the agent write the fast loop too, or does that layer stay a person's job or a learned policy's? This paper does not answer it. I would run that test first.

My read

• The unit of memory is right. Programs rarely repeat across tasks. Failures do: planner errors near furniture, bottles that tip, ambiguous "front bowl" language. A conditional note with a signature and a trigger matches how these failures recur.
• Admission is the weak spot. A verifier that reproduces a fix on its own source task checks that the note is readable. The zero-shot table shows that transfer is lumpy. The paper does not report an ablation of this verifier.
• The Bitter Lesson applies, with a floor. The model swap moves the numbers more than the harness, which is Sutton's pattern. The fixed API and the verifier are the human-built parts that remain.
• The model is the lever. With everything else fixed, success goes from 15% to 99% by swapping the language model. The paper's library-size curve uses Opus 4.6. I did not find the same curve for the newer models, so I can't say how much the library adds when the model is already strong.

What I like most is the format. A fix you can read, diff and revert is easier to trust than a change buried in weights. It is also exactly why the model and the API are the ceiling.

← All writing