Robots are showing up in more real workspaces every quarter, but most still stumble on the small judgments a human handles without thinking: knowing when a task is finished, adapting when something slips, or coordinating with another machine in the same room. Google DeepMind just released Gemini Robotics ER 2, a new "embodied reasoning" model designed to sit above a robot's motor systems and act as its high-level brain. It plans multi-step tasks, watches its own video feed to verify progress, and can hand off work to a second robot when one machine cannot finish alone. For developers building physical AI agents, that shifts what a single deployment can realistically do. This post breaks down what Robotics ER 2 actually is, what it improved over ER 1.6, and how to start using it today.
This article covers:
What Gemini Robotics ER 2 does inside a robot stack
How it orchestrates multi-step tasks without stop-and-think pauses
Video understanding: progress tracking and moment finding
Multi-robot collaboration across different hardware
Safety gains and the new VLA-orchestrator benchmark
How to start building on Gemini Robotics ER 2 today
What Gemini Robotics ER 2 does inside a robot stack
Gemini Robotics ER 2 is a high-level reasoning model that talks to humans, interprets the physical world, plans multi-step tasks, and then hands motor execution to any vision-language-action (VLA) model below it. It can also call tools like Google Search or any user-defined function while the robot is moving.
Think of it as the planning layer, not the muscle. The lower-level VLA model still handles the actual arm and wheel movements. The model decides what to do, in what order, and adjusts on the fly when the video feed shows something unexpected. According to Google DeepMind, this design lets the robot "think" about the next step while its body is still executing the current one, which cuts the awkward pauses that plague scripted systems.
The model is a direct successor to Gemini Robotics ER 1.6. Google reports gains across spatial reasoning benchmarks, safety, tool orchestration, and video understanding. If you have worked with earlier generations of the Gemini Robotics model family, ER 2 slots into the same developer surfaces without a stack rewrite.
How it orchestrates multi-step tasks without stop-and-think pauses
In robotics, high-level reasoning is only useful if it runs fast enough to keep pace with a moving arm. Gemini Robotics ER 2 integrates with the Gemini Live API through a bidirectional streaming endpoint tuned for low latency. That means it can command action models and robotics APIs in a fluid loop, without the jarring "pause, think, act" cadence older setups produce.
To build an agentic setup, developers declare low-level control interfaces (VLA models, navigation APIs, custom scripts) as tools. They then stream multimodal video, audio, or text directly into the model. The model does the rest: it picks which tool to call, when to call it, and how to react to what it sees.
Google published a demo built with Boston Dynamics Spot, where the model orchestrates Spot's navigation and manipulator APIs to fetch a popcorn snack on a spoken command. According to Boston Dynamics, Spot has been a common research platform for this kind of high-level control experiment. The demo code is available on GitHub alongside other examples.
Across three control modes (real VLA hardware, simulated VLA, and human tele-operation), Google reports that Robotics ER 2 consistently outperforms ER 1.6 on tool orchestration.
[image: a quadruped robot in a bright modern office fetching a small package from a shelf, wide angle photo]
Video understanding: progress tracking and moment finding
One of the hardest problems in robotics is knowing when a job is actually done. Robotics ER 2 improves on two capabilities that address this directly: progress classification and moment finding. Together, they give the robot real-time situational awareness so it can retry a failed step without restarting the entire workflow.
Progress classification
Progress classification is the robot's ability to score how far along a task is. Google's evaluation bins each video frame into five progress ranges: 0 to 20 percent, 20 to 40, 40 to 60, 60 to 80, and 80 to 100. The model hits 57.4 percent accuracy on this benchmark, ahead of the previous generation and competing frontier models.
That number matters because it lets a robot judge, from raw video, whether tightening a bulb or tying a trash bag is halfway done or nearly finished. The robot can then decide whether to keep going, adjust, or move on.
Moment finding
Moment finding is the ability to spot the exact frame where something critical happens: the moment coffee reaches the rim of a cup, or the moment a bolt seats. Robotics ER 2 posts 91.3 percent accuracy on this task with a mean absolute distance of 0.96 seconds.
Precision at that level lets robots switch cleanly between steps, verify success, and suggest corrections. Google notes the model competes with much larger systems but runs at roughly four times the execution speed, at a fraction of the compute cost. Sub-second latency is what makes physical operation safe in the first place.
Multi-robot collaboration across different hardware
No single robot is right for every job. A wheeled rover works well indoors, a humanoid handles uneven terrain, a stationary dual-arm platform is best for a workbench. Robotics ER 2 introduces multi-robot collaboration, letting different machines share a common semantic understanding of the task and hand work back and forth.
Google demonstrated this with Apptronik's Apollo 2 humanoid working alongside a Franka F3 Duo stationary arm platform. According to Apptronik, Apollo is designed for exactly this kind of general-purpose collaboration in commercial settings. The two robots coordinate through the shared model, completing a workflow neither could finish alone. This is a departure from earlier setups, which typically required a single robot per task or heavy custom middleware to synchronize two machines.
For developers who follow releases across the Gemini family, this pattern will feel familiar. See our writeup on for how Google is layering specialized models across different workloads, and for the multimodal side of the same stack.
Safety gains and the new VLA-orchestrator benchmark
Robotics ER 2 is Google's safest robotics model to date. It posts significant gains on the Safety Instruction Following and Human Proximity benchmarks, which measure whether the model respects physical constraints while reasoning and whether it can detect nearby humans. In Google's tests, the model halts a humanoid robot when a person approaches, and autonomously resumes work only once the space is clear.
Google also released a new benchmark for evaluating foundation models as safe VLA orchestrators. It tests whether a model enforces safety constraints, monitors the environment, assesses whether an action is physically feasible, and knows when to ask a human for clarification. Details are in the safety technical report linked from the launch post.
The wider AI industry has been moving toward more rigorous physical-agent evaluation, and Google's release fits that pattern. For context on how other frontier labs are approaching model safety and openness, see.
How to start building on Gemini Robotics ER 2 today
Robotics ER 2 is publicly available to developers right now. Access points:
Gemini API: for direct programmatic use.
Google AI Studio: for prototyping and prompt design.
Gemini Enterprise Agent Platform: in private preview for larger deployments.
Google published starter examples showing how to configure the model and prompt it for physical AI tasks, including the Boston Dynamics Spot demo mentioned earlier.
Quick comparison of what changed from the last release:
Capability | ER 1.6 | Robotics ER 2 |
|---|---|---|
Progress classification accuracy | Lower | 57.4% |
Moment finding accuracy | Lower | 91.3% |
Multi-robot collaboration | Not supported | Supported |
Safety benchmarks | Baseline | Improved on both measured tests |
Execution speed vs comparable frontier models | Slower | ~4x faster |
For teams weighing whether to build on a hosted model like this versus something they run themselves, our guide on covers the tradeoffs.
Frequently Asked Questions
Is Gemini Robotics ER 2 a replacement for a VLA model?
No. The model sits above the VLA layer. It plans and orchestrates, then hands motor execution to whatever vision-language-action model or robotics API the developer configures underneath. The two work together.
Does Robotics ER 2 work with any robot hardware?
It is hardware-agnostic in principle. Google has demonstrated it with Boston Dynamics Spot, Apptronik Apollo 2, and Franka F3 Duo so far. Developers declare their robot's control interfaces as tools, so any platform that exposes an API can be wired in.
Is there a Gemini Robotics SDK?
Google's launch materials do not describe a standalone Gemini Robotics SDK product. Developer access is through the Gemini API, Google AI Studio, and the Gemini Enterprise Agent Platform preview, with example code on GitHub.
How does Robotics ER 2 compare to ER 1.6?
Google reports gains across spatial reasoning, tool orchestration, video-based progress tracking, moment finding, and safety benchmarks. ER 2 also introduces multi-robot collaboration, which ER 1.6 did not support.
Can the model call external tools like search?
Yes. It can natively call tools including Google Search, as well as any user-defined function a developer exposes to it. This is what lets a robot look up information mid-task rather than relying only on its trained knowledge.
What to do next
If you are prototyping a physical AI agent, the fastest path is to open Google AI Studio, load Robotics ER 2, and try wiring a single robotics API as a tool. The bigger prize is multi-robot workflows, but a solid single-robot orchestration loop is the right first milestone. The model gives you enough headroom on planning, video understanding, and safety to build something meaningful on day one.