A UC Berkeley team introduced SuperMimic, a framework that uses a single differentiable world model to scale humanoid whole-body control. The approach trains on a single NVIDIA H100 GPU in 3 hours to outperform the SONIC baseline, which required 128 NVIDIA L40 GPUs over 7 days, reducing GPU-hour costs by more than 1,000 times.

UC Berkeley Team Proposes SuperMimic: 3 Hours of Single-GPU Training Surpasses 128-GPU Baseline

How much training compute does it take for a humanoid robot to master walking, jumping, and even backflips?

The conventional approach involves continuous trial and error in simulation, using reinforcement learning to optimize motion control policies. As the motion library expands, the corresponding training costs scale accordingly.

On October 20, a team from the University of California, Berkeley, published the paper "SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model." The paper has seven authors, all affiliated with UC Berkeley. Qiayuan Liao is the first author, with Yiyang Shao and Junfeng Long as equal contributors. Pieter Abbeel, a UC Berkeley professor and co-founder of Covariant, and Koushil Sreenath, a UC Berkeley professor and director of the Hybrid Robotics Laboratory, serve as co-advisors on the paper.

This research targets a key training bottleneck: as the motion library continues to expand, is it possible to bypass cluster-level compute and still train a unified controller capable of handling a large-scale motion set?

According to the paper, SuperMimic achieved a normalized return exceeding that of the publicly available baseline SONIC model after just 3 hours of training on a single NVIDIA H100 GPU. SONIC, by comparison, was trained using 128 NVIDIA L40 GPUs for approximately 7 days. Based on the paper's metrics, SuperMimic reduces GPU-hour costs by more than 1,000 times.

The team also deployed the trained control policy on a Unitree G1 humanoid robot, demonstrating a variety of movements including jumping, lateral rolls, and backflips.

SuperMimic relies on a single differentiable world model to support learning across the entire motion library, eliminating the need to train separate, independent controllers for each individual movement.

First, learn how the body moves

Let us first break down the root cause of the high computational costs in previous approaches. Today's mainstream motion tracking mostly relies on model-free reinforcement learning, with Proximal Policy Optimization (PPO) as the representative algorithm. These methods treat the physics simulator as a black box, relying on massive sampling to evaluate the quality of actions. Because the sampling process is highly noisy, it requires large-scale simulation to reduce errors. The larger the action library, the more simulation is required, which ultimately forces a reliance on scaling up compute.

The core improvement in SuperMimic is the introduction of a world model. The world model is a neural network that takes the robot's current state and action as inputs to predict the next state, while simultaneously outputting the immediate reward, robot observations, and fall risk. Thanks to its differentiable nature, the policy can backpropagate gradients through the network to every action step, directly obtaining the direction for parameter updates. This method is known as Backpropagation Through Time (BPTT).

SuperMimic uses a single world model to support learning for the entire action library. All actions share the same robot body dynamics. Adding new action clips merely supplements the model with samples of body and contact interactions, requiring no expansion of the model's capacity.

The team did not modify the simulator. The world model learns from data generated by a standard rigid-body simulator, which does not need to support differentiable computation. Non-smooth physical processes, such as collisions and contacts, are still handled by the native simulator.

The entire training process consists of a three-stage loop.

In the first step, the current policy collects trajectories in the simulator and stores them in a replay buffer, where new data overwrites older samples;

In the second step, consecutive 8-step segments are sampled from the buffer. Starting from the initial state, the world model makes recursive predictions based on the historical actions and aligns them with the ground-truth trajectories;

In the third step, the world model is frozen while the policy performs a 32-step imagined rollout within it. The returns from this virtual rollout are then backpropagated to update the policy.

These three steps are iterated in alternation, jointly optimizing the world model and the policy. Data collected by older policies remains usable for training the world model because the robot's physical dynamics are independent of the data-collection policy, yielding higher data utilization.

During the data collection phase, noise is injected into the actions to broaden the action distribution observed by the world model. Randomized simulation variables—such as ground friction, joint calibration errors, and torso center-of-mass offsets—are fed to the model as additional inputs. This approach allows the world model to remain deterministic, yielding more stable predictions.

The virtual rollout predicts only 32 steps forward, while long-term returns are estimated by a critic network to evaluate the total subsequent rewards. This requires the world model to maintain prediction accuracy only within the 32-step horizon. The team also noted that the model autonomously corrects prediction biases during training, enabling stable rollouts over longer horizons in practice.

The policy inputs consist of the robot's proprioceptive observations: 29 joint positions and velocities, angular velocity from the IMU, gravity direction, and the previous action, using a 10-frame history. The optimization target is the subsequent 10-frame trajectory of the reference motion. All model networks are multilayer perceptrons; the world model dynamics network comprises three layers with 2,048 neurons each.

Two rounds filter down to 129,514 actions

With the training scheme established, the team still needed a sufficiently large and physically feasible motion library. They selected motion clips from the public BONES-SEED dataset that had been mapped to the G1 robot, starting with an original total of 142,220 clips. Many of these samples involved scenarios such as climbing stairs, carrying boxes, and sitting on chairs, or contained mapping errors. Because the G1 cannot execute these maneuvers in a flat environment, they required pre-filtering.

The first round applied a coarse keyword filter. Each motion is named after its behavior, and samples with names containing keywords such as stairs, ladders, boxes, chairs, sitting, and climbing were directly discarded. The keyword list was adapted from the public data processing code of SONIC, with the team adding any missing terms. This round removed 12,553 samples, leaving 129,667.

The second round conducts physical feasibility checks, specifically filtering out invalid motions with unrecognizable names, such as walking onto a box. The team runs each motion in MuJoCo for kinematic playback, monitoring the minimum height of the robot's body. If the lowest point remains above 0.20 meters for 0.6 consecutive seconds, it is classified as stepping on an object. If it drops below -0.12 meters, the legs are deemed to have clipped through the ground. If the robot is stationary with both feet more than 0.02 meters off the ground for 0.5 seconds, it is also marked as infeasible. This round removes another 153 motions, yielding a final set of 129,514 valid motions with a total duration of approximately 264 hours.

This motion library corresponds to the publicly available training subset of SONIC. SONIC uses a total of 611 hours of motion data, whereas SuperMimic requires only 264.56 hours, reducing the dataset size by a factor of approximately 2.3.

During training, the robot does not always restart from the motion's initial position. The team divides the motion timeline into 1-second segments and records the score for each. The environment reset strategy employs mixed sampling: a 50% probability of uniform random selection, a 40% probability of biased sampling toward low-scoring, difficult segments, and a 10% probability of sampling based on historical difficulty. This ensures that already-learned, high-difficulty motions are repeatedly reviewed.

Regarding the training configuration, a single GPU launches 4,096 simulation environments, using 4,096 virtual rollout trajectories for each policy update across a total of 320,000 training iterations. The simulator uses the GPU version of MuJoCo Warp with a control frequency of 50 Hz. All compute costs were measured on NVIDIA H100 GPUs on Lambda Cloud. The team repeated the experiments using multiple random seeds. The training process remained stable, a characteristic the paper attributes to the progressive reward shaping in the motion tracking task.

3 Hours on a Single GPU, Failures Drop from 179 to 25

First, consider the direct comparison between SuperMimic and SONIC. Both adopt the motion tracking setup from BeyondMimic, with SONIC currently serving as the most relevant public baseline. The evaluation metric is normalized return, which represents the ratio of the total score for the entire motion to the maximum possible score. If the robot falls midway through the motion, the score is penalized.

After just 3 hours of training on a single H100, SuperMimic's score surpasses that of the SONIC model, which required 128 L40 GPUs and 7 days of training to achieve a final score of 0.90. Wall-clock time is reduced by a factor of approximately 56, and GPU hours are reduced by a factor of approximately 7,200. The team's choice of the 3-hour mark is conservative; from this point onward, every saved model checkpoint outperforms SONIC in both score and the number of failed motions. This 3-hour figure accounts for the complete training loop and excludes subsequent evaluation time.

The team also notes the hardware differences between the two experimental setups. The H100's FP32 peak compute is 0.74 times that of the L40, while its tensor core peak compute is 5.5 times that of the L40; however, neither experiment saturated the hardware's peak performance. The team argues that the 2.3-fold difference in data volume and the up to 5.5-fold difference in hardware capability cannot explain the thousands-fold difference in GPU-hour costs.

When tested across all 129,514 motion clips sequentially, SuperMimic failed on 25 clips after 3 hours of training. The publicly available SONIC model failed on 179 clips, and SONIC v1.1 failed on 239 clips. SONIC's failure samples included all 44 backflips, all 22 cartwheels, 8 segments of the full dance routine, and 15 out of 24 standing long jump samples. All these samples belong to both parties' training sets and are used to measure the models' coverage completeness of the training motions.

SuperMimic's failures are concentrated in a few highly complex motion groups. The top five groups in the test contain a total of 195 motion clips, accounting for 14 of the 25 failures. There is no single motion group where SuperMimic's failure count exceeds that of SONIC. After training for 9.8 hours on two GPUs, the model achieved zero failures across all motions.

To eliminate interference from experimental settings, the team additionally built a strict control baseline, PPO-Sim. The baseline runs the PPO algorithm directly in the simulator, with actions, observations, rewards, fall detection, parameter randomization, network size, and evaluation rules all aligned with SuperMimic. The number of simulated environments per GPU is even twice that of SuperMimic. The test selected 40 motion clips from the LAFAN1 dataset, with a total duration of 2.45 hours.

Experimental results show that SuperMimic achieves a score of 0.82 after just 1.2 hours of single-GPU training. Within an 18-hour compute budget, PPO-Sim consistently fails to reach this score. Its final score after single-GPU training is 0.60, 0.70 with 4 GPUs, and only improves to 0.72 with 8 GPUs, showing almost no benefit from scaling compute from 4 to 8 GPUs.

The team identifies three core characteristics of the world model that significantly reduce compute requirements.

First, model predictions are stable, and performance does not degrade as data scales. With training data expanded from 1 hour to 264 hours, the 8-step prediction error remains stable in the 0.007 to 0.011 range, and the GPU hours required to reach the target increase only marginally.

Second, rollout speed is faster. Across 2,000 to 16,000 parallel environments, the rollout speed is more than 10 times faster than MuJoCo Warp, reaching up to 62 times faster in 64,000 parallel environments.

Third, gradient quality is higher. For walking, running, and sprinting motions in LAFAN1, within the same world model environment, the number of rollout steps required for SuperMimic to reach optimal PPO performance is reduced by approximately 64 times. All comparisons in this set run entirely within the world model under identical hardware speed conditions, with the performance gap stemming from the first-order gradient update mechanism.

The team also independently verified the role of the critic network. When the critic network was removed, training failed entirely at a maximum rollout length of 16 steps. With the critic network included, a 16-step rollout achieved the same performance as a 32-step rollout without it, which was the optimal configuration for training. Increasing the rollout length from 64 to 128 steps nearly doubled the time to convergence while simultaneously decreasing the final score; 128 steps also represents the maximum VRAM capacity of a single H100 GPU.

Even when achieving identical scores, the critic network fundamentally alters the robot's behavior. In simulation experiments, the robot was instructed to maintain a T-pose while subjected to lateral pushes at speeds ranging from 0.5 to 4 meters per second. The policy equipped with a critic network recovered its balance within 0.5 to 2.4 seconds. In contrast, the policy without the critic network failed to maintain stable balance; it stumbled under a 0.5 m/s push and fell completely at 4 m/s. The root cause is that the rewards associated with balance recovery typically fall outside the 32-step rollout window.

In real-robot experiments, the same universal tracking policy was deployed directly onto the Unitree G1 without fine-tuning parameters for individual movements. The four demonstrated maneuvers included hopping, lateral rolling with the torso and limbs on the ground, a 90-degree aerial flip, and an in-place 360-degree aerial flip, covering three categories of scenarios: locomotion, full-body ground contact, and acrobatic stunts. During the training phase, the team only applied domain randomization to the physical parameters and found that this was sufficient to achieve sim-to-real transfer.

The team also explored a downstream application: controlling the robot to walk by receiving velocity commands. Building on the same world model, they trained a velocity encoder using a velocity tracking reward, while the control group employed an action tracking reward for knowledge distillation. In tests using 672 step commands covering the full velocity range, the velocity tracking approach achieved a smaller error in 650 commands, whereas the distillation approach performed better in only 15. The median relative errors for the two groups were 0.16 and 0.49, respectively. Falls occurred only with the distillation approach, with failures concentrated in 7 commands that combined translation with rapid turning—commands for which corresponding samples were virtually nonexistent in the action library.

The authors also outlined current limitations. Research on joint training across multiple interfaces remains incomplete; the velocity reward currently updates only the velocity encoder and does not update the shared decoder. The world model learns exclusively from simulation data and has not yet incorporated data collected from real robots. Furthermore, the model was trained only on flat terrain, and work to extend it to complex environments and physical object interaction is still ongoing.