How robot control is changing

Ask a robot arm to pick up a small green block and put it in a tray. The camera sees the block. The arm starts reaching. Then someone moves the block a few centimetres to the right.

The motors can follow their commands perfectly and still miss. Something has to notice the change and choose a different movement. So the real question of robot control isn't "how do I make the motor follow?" anymore. It's this: how does the robot choose its next movement?

Here's the short answer. Between 2023 and 2026, that choice moved out of hand-written planners and into one learned network, in five steps:

1. Copy a person (ACT, 2023): learn from about 50 demonstrations and predict a short chunk of future movements.
2. Borrow the web (VLA, 2023 to 2024): start from a model that already knows objects and words.
3. Train one model for many robots (π0 to π0.7, 2024 to 2026): one generalist policy, many bodies, many tasks.
4. Practice (reinforcement learning, 2024 to 2026): improve the generalist on the real robot until it stops failing.
5. Imagine (world action models, 2026): predict the future video and the action together.

What's still missing is memory and a cheap way to imagine. The timeline below shows which part of the loop each step changed. The rest of the post goes through them one at a time.

Before 2023: plan, then track

The classical answer splits the choice into separate programs. Perception finds the block and the obstacles. State estimation says where the block is relative to the arm. A task planner picks the step ("pick up the block"), a motion planner finds a path that doesn't hit the table, and a controller makes the joints follow that path. When the camera sees the block move, the estimate changes and the planner plans again.

This works, and it's still how most factory robots run. The trouble is that every piece is written by hand for a known world. Who writes the rule for a block that's half hidden, or slippery, or a sock instead of a block? Each new object asks for new code. That's the wall learning tries to get through.

ACT: copy a person, one chunk at a time

Suppose a person shows the arm how to pick up the block, about 50 times, from different positions. You record what the cameras see, the joint angles, and the commands the person gave. Can a network learn the next movement from that?

ALOHA made this cheap. A person moves two small "leader" arms, and two "follower" arms copy them. The leader joint angles become the action labels; the follower cameras and joint readings become the observations. ALOHA, pp. 3–5

The method that came with it, Action Chunking with Transformers (ACT), predicts a short sequence of future joint targets from the current images and joint angles. That sequence is an action chunk. A controller below tracks the targets. With only 10 minutes of demonstrations, ACT learned six fine tasks, like slotting a battery, with 80 to 90% success. ACT abstract

Why a chunk and not one action at a time? Copying one step at a time compounds errors: a small mistake puts the arm somewhere the person never was, and the next prediction is worse. Predicting a whole chunk keeps the motion coherent. ACT then predicts a new chunk at every step and averages the overlapping predictions for the same moment, which it calls temporal ensembling. ACT, §IV-A, p. 5

A chunk's length doesn't set how fast the robot reacts. Say a chunk holds $H$ targets sent at $f$ targets per second, and the robot executes $k$ of them before it looks again:

$$\begin{aligned} T_\text{chunk}&=\frac{H}{f}=\frac{10}{20}=0.5\ \mathrm{s},\\ T_\text{react}&=\frac{k}{f}=\frac{2}{20}=0.1\ \mathrm{s}. \end{aligned}$$

Units: targets ÷ (targets/s) = s. The plan looks half a second ahead, but the arm sees the moved block after a tenth of a second, plus the computing time.

Mobile ALOHA used the same recipe on a robot that drives around a house. Its authors trained with about 50 demonstrations per task, mixed with data from the static ALOHA tasks.

The limit shows up as soon as you change the task. An ACT policy knows one task, in one setup, on one robot. It has no idea what a "tray" is. It only knows the motions it saw.

Side path: when two routes are both right (Diffusion Policy)

Put an obstacle between the gripper and the block. A person goes around it on the left half the time and on the right half the time. A network trained to predict the average path goes straight through the obstacle.

Diffusion Policy (2023) learns the whole distribution of action sequences instead of their average. At run time it starts from random numbers and refines them, step by step, into one coherent route, left or right, never the average. Diffusion Policy, §§II-C, IV-A, pp. 3–4 The π models below use a close relative, flow matching, for the same reason. It matters again in the RL section: this kind of policy has no simple formula for the probability of an action, which breaks some classic RL methods.

From VLM to VLA: borrow what the web knows

A person knows what a tray is without a single robot demonstration. Could a robot borrow that knowledge?

A vision-language model (VLM) reads images and text and answers in text. It learned from billions of web images, so it knows objects, colors and words. A vision-language-action model (VLA) is a VLM that is trained further on robot data, so that it also outputs robot actions. The table compares the three.

RT-2 (2023) showed that the transfer is real. It wrote each action as tokens, the same discrete symbols the model uses for words: each action dimension was cut into 256 bins. RT-2 v1, pp. 5–6 On scenes with unseen objects, backgrounds and environments, success rose from 32% for RT-1 to 62%. DeepMind, RT-2

OpenVLA (2024) shows the recipe with open weights: two image encoders, a 7-billion-parameter language model, and a de-tokenizer that turns output tokens back into a small move of the gripper.

For our block, this means the instruction "put the green block in the tray" now means something to the model, even for a block color it never saw in the robot data. The cost is the action format: 256 bins per dimension, predicted one token at a time, is coarse and slow for a fast, precise arm.

Generalist models: one policy for many robots

Physical Intelligence (PI) took the next step: one model, trained on many robots and many tasks, that you can prompt or fine-tune.

π0 (October 2024) keeps a pretrained VLM but adds an action expert, a smaller network that outputs continuous action chunks by flow matching instead of tokens. It trained on data from single arms, two-arm robots and mobile manipulators, and it can fold laundry, bus a table and assemble a box. π0 abstract, π0, RSS 2025, §IV

π0.5 (April 2025) adds a step before the motion: the same model first predicts a subtask in words ("pick up the block"), then generates the actions for that subtask. It trains on many robots plus web data, which helps it work in homes it never saw. π0.5 v1, §§IV-A–D

π0.7 (April 2026) changes what the prompt can say. It reads not only what to do but how: metadata such as speed, quality and mistakes, and optional subgoal images. That lets it train on mixed data, including failed and slow attempts, because the labels tell good runs from bad ones. PI reports that π0.7 makes espresso out of the box about as well as specialist models fine-tuned with RL. π0.7 abstract

What does the generalist actually send to the robot? Still targets. In π*0.6, the output is joint angles and gripper commands at 50 Hz; in π0.7, joint targets or gripper poses, with inverse kinematics for poses, and a proportional-derivative (PD) controller that turns the targets into motor effort. π*0.6, §V-A, π0.7, §VIII The learned part chooses where the joints should go. A classical loop still makes them get there. I spent a whole series on that bottom loop on my $100 arm, and it's not a solved detail.

Practice: reinforcement learning on real robots

A generalist that succeeds 80% of the time is impressive in a video. In a home, it's a broken glass every few days. Perry Dong and Chelsea Finn put it this way in Towards Universal Post-Training for Robotics (September 2026): robot models today are where language models were around GPT-2 and GPT-3. They're capable enough for demos, not reliable enough to trust. Language models closed that gap with a standard post-training recipe, more training after the big pretraining: instruction tuning, then reinforcement learning from feedback. Their argument is that robots need the same thing.

Why not just collect more demonstrations? Because copying has a ceiling. The demonstrations never show how to recover from the robot's own mistakes, and the robot can't get better than the people who demonstrated. Reinforcement learning (RL) learns from the robot's own attempts and a score of how they went.

Why RL is hard on a robot

Dong and Finn list the differences with RL on language models:

• Data is expensive. A language model can try a thousand answers in parallel in a second. A robot tries once, in real time, and someone may have to reset the scene.
• Horizons are long. Picking up a glass is about 500 decisions, each a 7-number action every 20 ms. The reward ("success") comes at the end. Which of the 500 decisions deserves the credit?
• The world is random. The same command doesn't give the same result twice.
• The actions are continuous. This one is the most technical, so here it is with a picture.

RL usually trains a critic: a network $Q(s,a)$ that predicts how good action $a$ is in situation $s$. Then the policy should pick the best action:

$$a^\star=\arg\max_a\ Q(s,a).$$

In a game with four buttons, you score the four buttons and pick the best. That's what DQN does. A robot action is 7 real numbers. Cut each into 20 bins and you get $20^7 \approx 1.28$ billion actions to score, 50 times per second. You can't enumerate them.

Classic continuous-control RL answers this with an actor-critic: a second network, the actor, learns to output the action that the critic scores high, by following the slope of $Q$. DDPG, TD3 and SAC (soft actor-critic) all work this way. Play with the widget: the actor climbs the nearest hill. It's fast, but it can get stuck on a lower one.

Modern VLAs add a second problem. Their action expert is a flow or diffusion model: great at keeping the left and right detours separate, but it has no simple formula for the probability of an action. PPO, the RL workhorse of language models, needs exactly that probability ratio. So the field had to find other ways to plug a critic into a VLA.

Four ways to practice

These are the four methods I find most instructive. Each uses the critic differently.

Sources: HIL-SERL, π*0.6, RLT, EXPO-FT.

RECAP's trick is the one I found most surprising, because it turns RL back into supervised learning. A value model $V$ predicts the future reward from an observation. For a segment of $N$ steps, the advantage says whether the segment did better than expected:

$$\hat A_t=\sum_{k=t}^{t+N-1}r_k+V(o_{t+N})-V(o_t).$$

A worked example with made-up numbers. Each step costs $-1$, so a 10-step segment has rewards summing to $-10$. Before the segment, $V$ predicts $-40$; after, $-25$. Then $\hat A=-10+(-25)-(-40)=+5$: the segment moved the robot 15 units closer to done for a cost of 10. If $+5$ is above the threshold, the segment gets the label "Advantage: positive", even if the whole attempt failed later. The policy learns every segment under its label, and at run time you ask for "positive". π*0.6, §§IV-A–B

EXPO-FT is the method behind Dong and Finn's post, and its picture is the widget's third button. The big VLA proposes $N$ chunks $a^1,\dots,a^N$. A small edit policy proposes a bounded change $\hat a^i$ for each one. The robot runs whichever candidate the critic likes best:

$$a^\star=\arg\max_{a\,\in\,\{a^i,\ a^i+\hat a^i\}}\ Q(s,a),\qquad \|\hat a^i\|\le\varepsilon .$$

The VLA keeps the candidates in good regions. The edits stay small, so a wrong critic can't drag the robot far. Then the improvements are trained back into the VLA.

The comparison I care about is controlled: same base model, same tasks, same 30 trials. More copying, even with a person correcting the robot (HG-DAgger), stays around 20 to 22 out of 30. Practice reaches 30 out of 30.

Two cautions before this sounds solved. First, 30/30 is 30 trials in the authors' lab, not a deployment. Second, Dong and Finn are clear that the algorithm is only half of the recipe. Nobody has a standard answer yet for the other half:

1. Reward: who decides that the egg flip succeeded? Today, a trained success detector or a person.
2. Reset: who puts the egg back?
3. Human help: when should the person step in, and how much?
4. Defaults: RL on robots is still sensitive to settings, with no agreed defaults.

That's the difference with language models, where "check the answer" and "run it again" are almost free. RL also works when the robot practices in simulation instead: Mistral trained Robostral Navigate on simulated buildings, then ran online RL there and gained 4 points of success (more on it below).

World action models: imagine the video and the action together

A VLA maps an image to an action. It never has to say what the world will look like next. Does that matter?

NVIDIA's argument is yes. VLAs are good at semantics ("which object is the tray?") but weak at motions they never saw. A video model, trained on a huge amount of video, has seen how things fall, slide, open and pour. A world action model (WAM) is a policy built on such a video model: it predicts how the scene will change and the action that goes with it. NVIDIA, The rise of world action models

DreamZero (February 2026) is the clearest example. It starts from a 14-billion-parameter video model. During training, it adds noise to both the future video frames and the future action chunk, and learns to remove the noise from both together. On the robot, it imagines the next frames and the next actions, executes the actions, and then replaces its imagined frames with the real camera frames, so its errors don't pile up. DreamZero abstract

Here's what that looks like on tasks the model never trained on. The top row is what really happened; the bottom row is what the model imagined from the robot's cameras before acting.

The reported results:

• More than 2× better generalization to new tasks and environments than state-of-the-art VLAs, in real-robot tests.
• Learning from video only: 10 to 20 minutes of video of another robot or a person doing a task gave more than 42% relative improvement on that unseen task. No actions were needed for that data.
• A new robot in 30 minutes of play data, while keeping its zero-shot skills.
• On the public RoboArena ranking (April 2026 snapshot, reported by NVIDIA), DreamZero scored 1750 against 1622 for π0.5.

The price is compute. A 14B video model is slow: one action step first took 5.7 s. With system work and a trick called DreamZero-Flash, the authors brought it to 150 ms, 38× faster on NVIDIA's GB200 chips, enough for 7 Hz closed-loop control. DreamZero, real-time section

Limits and what comes next

Two limits stand out to me: the robot forgets, and imagining every pixel is expensive.

Memory: the same image, a different next step

Extend the job: inspect the block, then put it away. The camera image looks the same before and after the inspection. So which step comes next?

Every policy in this post, ACT, π0.5 and most VLAs, decides from the current images, maybe the last one or two. That's fine for "pick up the block". It fails for "did I already add salt?", for counting, and for an object the arm itself is hiding.

RoboMME (2026) measured this on 16 simulated memory tasks with the same π0.5 policy. With the current view only, it averages 17.9%. Giving it past actions barely helps (19.7%). Feeding it a sample of past frames reaches 44.5%. The authors' main finding is that the best kind of memory depends on the task. RoboMME abstract

Then there's Robostral Navigate, Mistral's first robotics model (July 2026). It's an 8-billion-parameter navigation model, not a manipulation one: it drives a robot through buildings it never saw, from a spoken instruction and one ordinary camera, with no map, no depth sensor and no special memory module. It simply keeps every frame of the trip in its input. It was trained only in simulation, and it's first on the standard R2R-CE benchmark: 77.4% success, ahead of systems that use depth sensors or several cameras. Robostral Navigate, §§2.1, 4.1

I find that striking: the simplest possible memory, "keep everything", wins a benchmark that used to need maps and depth. But it's a lucky case. A navigation episode is short, and the model only looks at its frames every 2 seconds. A 15-minute kitchen task at one frame per second is 900 frames. In the MEM paper, a VLA that reads raw frames already needs about 3.5 s for 16 frames, against a real-time limit of 0.3 s.

MEM (Physical Intelligence, 2026) splits memory in two, like people do. Recent seconds go through a compact video encoder, so the arm remembers the object it's now hiding. Older events become a short text summary that a higher-level policy rewrites as it goes ("I opened the drawer with the masher"). On recipe setup and kitchen cleanup tasks lasting up to 15 minutes, π0.6 without memory reaches about a third of the task; with MEM, about 70%. MEM abstract

Memory has its own failure mode, and it's the one I'd test first. Suppose the gripper closes beside the block. The command finished, but the pickup failed. If the text memory now says "block put away", every later decision builds on a false record. A memory that can't tell "I tried" from "it worked" is worse than no memory.

World models without all the pixels

DreamZero imagines every pixel of the next frames. That's what makes it slow and big. And sometimes it's what makes it wrong: when the imagined video goes off, the robot follows it.

Does the robot need the pixels at all? To choose between the detour and the collision, it needs to know where the gripper and the block will be, not the texture of the table.

V-JEPA 2 (Meta, 2025) predicts the future in features, learned numerical summaries of an image, not in pixels. To plan, it tries many candidate action sequences, predicts their final features, and keeps the one whose features are closest to the features of a goal image. Then it executes the first action and plans again. V-JEPA 2 v1, §3.2

It isn't free either. In the paper, one planning step took 16 seconds on one GPU, with 800 sampled action sequences and ten rounds of refinement. V-JEPA 2 v1, p. 15 The pixels are gone, but the search is expensive.

DreamZero's own speed-up points the same way, which I didn't expect. DreamZero-Flash trains the model to predict clean actions from very noisy video, so at run time one denoising step is enough. The actions stay good while the imagined video stays rough. The authors also note that dropping the video and generating only actions barely saved time at this size: the number of denoising steps and layers sets the cost, not the pixels. DreamZero, real-time section My reading: the useful part of imagining is the rough shape of the future, and the open question is how little of it you can predict and still choose well.

What I'd test next

The loop is the same as in 2010: observe, choose, act, check, revise. What changed is how much of "choose" is learned, and how many robots, tasks and minutes of practice feed it. Three checks would tell me whether the next steps are real:

1. Memory under failure. After a failed grasp, does a MEM-style text memory notice and correct a false "done"? Count complete jobs, retries and human interventions.
2. Imagination at contact. Does a world model rank the right action higher when contact matters, for example a grasp that slips, compared with the same policy without the imagined video?
3. Practice without a person. How much of RL's reliability gain survives when a learned success detector replaces the human judge and the robot resets its own scene?

← All writing