Why I am learning robotics

In 2018, Boston Dynamics posted a video of its Atlas humanoid jumping over a log and running up a set of boxes. A year earlier, Atlas did a backflip. For a lot of people, me included, those clips were what a robot was supposed to look like. So where are they? I went looking for one at work, and I couldn't find many. Spot, the robot dog, went on sale in 2020 for $74,500, and Boston Dynamics now says it has more than 1,500 Spots in customer hands. Over the same years, Amazon put one million robots into its warehouses, and almost none of them can walk.

That's the question that pulled me into robotics: why did the most capable robots in the world stay in the videos, and why does that seem to be changing now? My short answer is that the problem was never the body. It was the software, and more precisely the fact that the software couldn't generalize. Language models are changing that, and I think robotics today looks a lot like language models did in 2019.

Why the best robots stayed in the videos

Boston Dynamics explained how the parkour worked in a 2021 post. Atlas used model predictive control: a model of its own dynamics predicts how the body will move over the next moments, and an optimizer picks the forces that keep it on track. The jumps and vaults came from a small library of template behaviors that engineers designed offline. The controller adapted them to the terrain on the fly. It's great engineering. The catch is that each new behavior is a project for a team of experts.

That works when the task is fixed. Spot sells for inspection because inspection is easy to specify: walk this route, read these gauges, take this thermal photo. Now suppose you ask it to unload the dishwasher in a kitchen it has never seen. Who writes that behavior? And the next one? Change the cup, the lighting, the table height or the clutter, and the same instruction needs a different movement. The ability to carry a skill into a situation you didn't program for is called generalization, and a hand-engineered controller has very little of it.

You can see how hard it was to turn that hardware into a business. Boston Dynamics had three owners in eight years: Google bought it in 2013, SoftBank in 2017 and Hyundai in 2021. The robots were never the weak part.

Hardware is not the bottleneck

Here's a simple test for whether a robot's body is good enough for a task. Let a person drive it by remote control, which is called teleoperation. Can the operator do the task through the robot? Then the motors, joints and cameras are good enough. What's missing is a program that does what the operator did.

Cheap robots pass this test more often than you might expect. The 2023 ACT paper asked directly whether learning lets "low-cost and imprecise hardware" do fine manipulation. Its two-arm ALOHA setup threaded cable ties and slotted a battery from about 50 demonstrations per task. A year later, Mobile ALOHA put the same arms on a wheeled base and learned to sauté a shrimp, store heavy pots in a cabinet and call an elevator, again from 50 demonstrations per task. A Unitree G1 humanoid starts at $13,500. One SO-101 arm, the one I use, costs a little over $100 in parts.

The two Mobile ALOHA videos below make the teleoperation test visible. On the left, a person drives the robot through a three-course meal. On the right, the robot does chores on its own after learning from demonstrations.

This doesn't mean hardware is solved. Hands are still fragile, cables wear out, batteries run down, and a robot that breaks every week isn't useful (more on that in the drawers at the end). But compare two gaps. One is between the hardware and the task. The other is between what the hardware does with a human driving it and what it does on its own. The second gap is much bigger, and it's a software gap.

From language models to robot actions

Language models closed this kind of gap for text. A large language model, or LLM, is trained to predict the next piece of text, and in doing so it absorbs a huge amount of general knowledge. Two extensions brought that knowledge to robots:

A VLA usually starts from a pretrained VLM and is then trained on robot data, so the robot inherits what the VLM learned from the web. RT-2 showed in 2023 that this transfer is real: on scenarios with unseen objects, backgrounds and environments, success rose from RT-1's 32% to 62%. Systems like Gemini Robotics 1.5 add a reasoning model on top that breaks "put the groceries away" into steps and hands each one to the VLA.

OpenVLA shows the recipe clearly. Two vision encoders turn the camera image into tokens, a 7-billion-parameter language model reads them with the instruction, and the model's output tokens are decoded back into a small change of the gripper's position, rotation and opening.

Figure's Helix is the same idea on a humanoid. In the video, two robots run the same model and put away groceries they never saw in training.

For me, the clearest sign of the shift is what Boston Dynamics itself did. In August 2025, with Toyota Research Institute, it showed Atlas driven by a single large behavior model that controls the whole body, legs and hands, from demonstrations. Tasks like tying a rope or spreading a tablecloth, which would be miserable to hand-program, use the same training process as stacking blocks. In their words, "if you can demonstrate it, the robot can learn it." In January 2026, the company unveiled a production Atlas and a partnership with Google DeepMind to put Gemini models inside it, and Hyundai announced a factory that can build 30,000 robots per year. So the company that best stood for hand-engineered control is now betting on learned policies.

A reset button on classical robotics

The classical robot is a pipeline of separate modules, each written by experts: perception finds the objects, state estimation works out where everything is, a planner chooses a path and a controller follows it. Each module has its own equations and its own decades of research. A VLA replaces most of that pipeline with one network that goes from pixels and words to actions.

That's a reset button. A lot of what used to be an equations problem becomes a data problem. Which demonstrations do you collect? How do you cover the situations that matter? How do you know the robot got better? It also means someone who arrives late, like me, isn't decades behind. These questions are new for everybody.

It isn't a full reset, though, and this is the part I find most interesting. Underneath every VLA, a motor still runs a classical feedback loop. The network outputs a target angle, and the joint doesn't land exactly on it: gravity pulls it down, friction holds it back and the servo reacts late. That last mile is where my own research sits. I'm testing whether a controller can be more accurate than the classical ones on my SO-101 arm, and the series that starts with one equation is what I've learned so far.

Robotics is in its 2019

The timeline below puts the two fields side by side. Pick a stage to see what happened in each one.

The dates come from the papers and announcements: the Transformer, GPT-3, the scaling laws and InstructGPT for language, and SayCan, RT-1, RT-2, Open X-Embodiment, π0, Helix and π0.5 for robots.

In 2019, GPT-2 showed that a bigger model trained on more data generalizes better, and a whole industry grew up around producing training data: Scale, Labelbox and Snorkel AI all date from that period. I think robotics is at that point now. In August 2026, Figure launched Index, an app that pays people to record everyday activities, and reported more than 16 million videos from 44,000 weekly users. A month later it reported 56% task success in 30 homes the robot had never seen, compared with 9% without the Index pretraining. That's real generalization. It's also a long way from a robot you'd trust to finish the job.

The first big step was pooling. In 2023, Open X-Embodiment gathered robot data from 21 institutions into one dataset, so a single model could learn from 22 different robots.

The data can come from several places, and each one gives something different:

Variety matters as much as volume. DROID sent the same robot setup to 52 buildings and recorded 76,000 teleoperated episodes in bathrooms, bedrooms, kitchens and offices, because a robot that only saw one lab learns that lab.

Learning from people's point of view

There's a second way around the data problem: record people instead of robots. Egocentric data is video filmed from the person's own point of view, usually with a head-mounted camera, so the hands sit in the frame the way a robot's grippers would. People already do every task we care about, in every kind of place. The question is whether a robot can learn from hands that aren't its own.

In February 2026, NVIDIA's EgoScale tried it at a large scale. It pre-trained a VLA on 20,854 hours of egocentric human video, then trained it on a small set of matched human and robot play data (50 hours of human data, 4 hours of robot data), and finally on the target tasks. On a robot with a 22-degree-of-freedom hand, average success went up by 54% compared with no pre-training.

The matched human data comes from a light capture rig: a camera on the head, cameras on the wrists and tracked gloves, so each frame comes with the hand's motion.

But why would this work at all, when a human hand looks nothing like a gripper? A paper from Physical Intelligence, with Stanford's Chelsea Finn among the authors (Kareer et al., 2025), gives an answer I like. They co-trained π0.5 on robot data plus 3 to 5 hours of head-camera human video per task, with no special step to align humans and robots. Then they looked inside the model with a t-SNE plot. t-SNE squashes the model's high-dimensional internal vectors into two dimensions, so that points close together are inputs the model treats as similar.

Without robot pre-training, human and robot inputs form two separate clouds: the model learned them as two different worlds. With more diverse robot pre-training, the clouds merge, and the same task done by a human or by a robot lands in the same place. That's also when the transfer shows up. On scenes the robot only saw through human video, performance nearly doubles.

Where the analogy breaks

The parallel isn't perfect, and I want to be honest about where it breaks.

1. The data wasn't free. GPT-2 trained on text that already existed on the internet. Nobody wrote down what a robot saw, commanded and felt while opening a drawer. Every hour of robot data has to be produced, usually by a person.
2. The scaling law is only starting to appear. In 2020, language models got a curve that predicts how much better a model gets with more data and compute. That turned a bet into an investment plan. Robotics is just getting its first curves. EgoScale found that the validation loss falls in a straight line against the log of the hours of human video, and robot success rose with it, from 0.30 at 1,000 hours to 0.71 at 20,000 hours (average task completion). That's one recipe, one robot and five points, not a general law. So it's still possible we're really in 2017, not 2019.
3. The bar for reliability is higher. A wrong answer in a chat costs a retry. A wrong grasp can break a glass. And long tasks multiply errors: if a job has 20 steps that each succeed 95% of the time, the robot finishes only $0.95^{20} \approx 36\%$ of the time.

Mobile ALOHA's authors published their failures too, and they show what those lost percentage points look like.

That third point is also why measurement is so hard. Benchmarks like LIBERO and RoboCasa make it possible to compare policies in simulation, and even there the leading entry on RoboCasa scored 36.7% on held-out multi-step tasks in September 2026. In the real world, someone has to reset the scene, the lighting shifts and the cup is slightly different every time.

Waymo: a robot that tries not to touch the world

One robot already works at scale, and it's worth asking why. Waymo's cars drove more than 270 million fully autonomous miles through June 2026. Against human drivers in the same cities, Waymo reports 82% fewer injury crashes and 95% fewer crashes with a serious injury or worse.

So why does driving work before manipulation? My reading is that a car's job is to not touch anything. Every contact is a failure. The car has to predict where everything will be and stay away from it. A kitchen robot is the opposite. Its whole job is contact: push, grasp, squeeze, pour. And contact is exactly where physics gets hard to predict, with friction, slipping and soft objects that change shape.

How did Waymo get the extra nines of reliability? Simulation is a big part of the answer. Waymo runs closed-loop simulation, where the other cars and pedestrians react to what the Waymo does. In its 10 AI lessons, it describes an automated Critic that reviews the millions of road miles driven each week "and tens of billions in simulation." The simulator finds rare events before the car meets them on the road. But Waymo is also clear that simulation isn't enough: in the same post, it says a driving system only truly matures when it's solely responsible for the driving. So it's simulation plus hundreds of millions of real driverless miles, not simulation instead of them.

The other thing I find interesting is how much explicit structure Waymo keeps. Its foundation model doesn't just output a path. It also outputs structured representations: objects, their attributes and the road graph. And a separate onboard validation layer checks each planned trajectory against physics-based constraints and traffic laws before the car follows it. FieldAI, which builds autonomy for robots in unstructured environments, makes a similar choice: its Field Foundation Models combine learned models with physics-based reasoning and uncertainty awareness. That's the question my SO-101 series asks at a much smaller scale: where should physics stay inside a learned system?

World models: imagining the future before acting

A VLA maps what the robot sees to an action. A world model answers a different question: given what the robot sees now and an action, what will it see next? With one, a robot can try actions in its head, a company can generate training data, and you can test a policy without a real robot.

Waymo uses one for driving. The Waymo World Model, built on Google DeepMind's Genie 3, generates both camera and lidar data for scenes that are hard to film for real, like extreme weather or strange objects on the road. You can steer it with driving actions, a scene layout or a text prompt.

For robots, the clearest picture puts the real camera frame next to the imagined ones. NVIDIA's DreamGen starts from one real image from the robot's camera, asks a video model adapted to that robot to imagine the rest of the task, and then recovers the actions from the imagined video. With teleoperation data from only one pick-and-place task in one environment, a humanoid learned 22 new behaviors.

1X does this live on its NEO humanoid. With its world model, announced in January 2026, you give NEO a voice or text prompt, it generates a video of the future from what its camera sees, and an inverse dynamics model turns the imagined frames into movements.

Not every world model draws pixels. Meta's V-JEPA 2 predicts the features of the next frame instead of the frame itself. It first learned from over a million hours of video without action labels, then from less than 62 hours of robot data, and used its predictions to plan movements. Yann LeCun's company AMI is pursuing this direction.

Reindustrializing Europe and the United States

This is the part that excites me most. It's a vision, not a measurement.

Factories in Europe and the US already use a lot of robots. The International Federation of Robotics counts 449 robots per 10,000 manufacturing employees in Germany and 307 in the US, against 1,220 in South Korea. But those robots sit mostly where the task never changes: welding the same car body, placing the same chip. Every new task means an integrator reprogramming the cell, so automation only pays off at high volume. Small-batch work, with many different parts and frequent changes, stayed manual, and manual work moved to wherever labor was cheapest.

Now suppose a robot can learn a new task from a few dozen demonstrations in an afternoon. That calculation changes. Small-batch production becomes something a robot can do, and labor cost stops being the reason to build a factory far away. With an aging workforce in Europe and a political push in both regions to bring manufacturing back, general-purpose robots could make reindustrialization economically possible rather than a slogan. Hyundai building a factory for 30,000 robots a year is one company placing that bet.

What's blocking it, again, is software that generalizes and is reliable enough for a production line. That's the problem I want to work on.

Open questions I keep returning to

These questions keep pulling me in, but you don't need them for the argument above.

Learning quickly without skipping the thinking

I've just started learning robotics, and the reset makes this a good time to begin. With papers, simulators, cheap arms and coding agents, I can take a question to an experiment in an afternoon.

The speed has a trap, though. If I follow every interesting branch, I can explore for hours without making anything concrete. And when an agent builds the experiment for me, it can skip the design decisions that would have forced me to understand it. So I try to keep the parts that change my understanding: predicting a result before I run it, choosing the experiment, explaining a failure, and finding the example that makes an idea click.

Jacob Rothschild's writing about robot control helped inspire these posts. Writing gives the learning a finish line: if I can explain something clearly without the conversation that led there, I have something I can come back to and test.

The first technical post is about joint angles, hand poses and motor control. The SO-101 series is about the last mile: why the arm misses the target the policy asks for, and what each controller does about it.

← All writing