Why Embodied AI Hits Hard Compute Limits at the Edge

The Power Budget That Sets the Real Ceiling

A robot's onboard computer runs on a fraction of the power a data-center AI chip gets. That power gap limits what embodied AI can do today more than the size of the AI model does. NVIDIA's Jetson Thor module is the compute board built for humanoid and mobile robots. It delivers up to 2,070 FP4 teraflops (TFLOPS, or trillion floating-point operations per second) of AI compute inside a 40 to 130 watt power envelope. A data-center graphics processing unit (GPU) built for the same class of AI workload commonly draws several hundred watts. It sits in a facility with industrial cooling and a wall-power connection. A robot's chip, by contrast, runs off a battery pack riding on its back.

That power gap comes down to how each system gets rid of heat. A robot has to shed the heat its chip produces through a small onboard heatsink and fan, or passive cooling fins. A server rack, by contrast, vents heat through a data center's own cooling system built for exactly that job. Push a robot's chip past its rated wattage. It throttles its own clock speed to avoid damaging itself. That throttling caps how much AI inference the robot can run per second, regardless of how capable the chip's architecture is on paper.

The generational jump from NVIDIA's prior Jetson Orin module to Jetson Thor shows how tight this constraint has been. Thor delivers roughly 7.5 times the AI compute of Orin and about 3.5 times the energy efficiency. It runs on NVIDIA's Blackwell GPU architecture, with 128GB of memory and a 14-core Arm Neoverse-V3AE CPU (central processing unit). Early adopters of the module named by NVIDIA include Agility Robotics, Amazon Robotics, Boston Dynamics, Caterpillar, Figure, Hexagon, Medtronic, and Meta. That spread crosses humanoid, warehouse, construction, and surgical robotics. The same watt ceiling applies whatever the robot's job is.

Why Sending the Job to the Cloud Doesn't Solve It

Moving a robot's AI workload to the cloud fixes the power problem. It creates a worse one: latency. A robot's low-level joint and balance control loop typically runs at 100 hertz (Hz) to 1 kilohertz (kHz). That means it needs a fresh decision every 1 to 10 milliseconds to keep a humanoid robot standing or stop a gripper from crushing what it holds. A round trip to a cloud server over Wi-Fi or cellular data commonly takes 20 to 100 milliseconds. That figure gets worse the moment a warehouse has spotty coverage or a robot walks out of range of its access point.

A missed or late reply from the cloud on a balance-control loop does not mean a slower response. It means the robot falls, drops what it is carrying, or stops mid-task waiting on a network packet that never arrives on time. That is why every fast, safety-critical control loop on a modern robot runs on the onboard chip, regardless of how much spare compute a cloud server has sitting idle.

Higher-level reasoning is a different story, and the cloud handles it well. Deciding which shelf to restock next, planning a multi-step task, and fine-tuning a model on the day's data can all tolerate a delay of a second or more. Robot makers push that kind of work off the robot and onto a server whenever the task allows it.

How Robot Makers Split the Job Between the Robot and the Network

Every production robot today runs a hybrid setup. Perception and low-level control run on the chip riding inside the robot. Higher-level planning and model updates run on a server the robot talks to when the network allows it. Figure and Boston Dynamics both build humanoid platforms that process camera and joint-sensor data locally for balance and immediate obstacle response. Once a task is done, each platform sends less time-sensitive data, like footage for retraining a grasping model, back to a data center over Wi-Fi.

Warehouse fleets split the job the same way for a different reason: scale. Amazon Robotics runs thousands of mobile robots off a centralized system that assigns tasks and routes traffic. Each individual robot still handles its own obstacle avoidance and motor control locally. A central server's reply for every wheel adjustment would be too slow across a fleet that size, and the whole system would stall the moment the warehouse's own network hiccupped.

Agility Robotics builds to the same split for its warehouse-oriented humanoid. It keeps the walking and manipulation control loop onboard while offloading fleet coordination and task assignment to a server. The pattern holds across every serious embodied AI deployment. The fast, physical loop stays on the chip riding on the robot. The slow, cognitive loop goes wherever there's spare compute.

Why More Onboard Compute Doesn't Move the Boundary

A faster chip does not change which tasks have to run on the robot instead of the cloud. The network's round-trip time sets that boundary, and the chip's own compute speed has no say in it. NVIDIA's Jetson Orin-to-Thor jump added roughly 7.5 times the AI compute in the same watt-class module. Even so, the balance-control loop on a humanoid robot still has to run onboard today, for the identical reason it did on the older chip. A 20-to-100-millisecond round trip is too slow for a 1-to-10-millisecond control cycle. That holds no matter how many extra teraflops sit inside the box making that decision.

A faster onboard chip buys a robot a bigger, more capable model running inside that same latency-bound loop. It does not buy permission to move the loop off the robot. A humanoid can run a larger vision-language-action model for grasping, or a richer scene-understanding model for navigation, on Thor than it could on Orin. Either way, it stays inside the same 1-to-10-millisecond control budget the robot's physics demands. Each new chip generation packs more computing into that fixed time window. It does not shrink the window itself.