What is Rocky Learns to Walk Live?

It is a public SuperDroid Robots dashboard that runs Rocky’s latest locomotion policy in your browser with MuJoCo WASM and shows live training graphs from recent reinforcement-learning runs.

Is this a live camera feed of a physical robot?

No. The 3D view is a physics simulation. Policy weights and metrics are published from SuperDroid’s training systems; inference and physics run locally in your browser.

What do the graphs show?

The main charts show episode length at done, survival completion with EMA, and full-episode rate for the latest training run and the previous two. The header strips show live peak torque, peak speed, duty at peak (share of the last two seconds at ≥90% of that window’s peak torque), and Avg Power (time-average electrical watts over the last 8 seconds, with peak in smaller type; Σ|τ ω| / 0.98 at 98% motor efficiency). Torque, speed, and wattage include all 22 actuated joints, including PD-held arms. Hover a strip for the value at that time.

How is Rocky’s walking policy trained?

Rocky is trained with PPO in MuJoCo MJX (Warp) as a residual on a removable control scaffold: phase-locked gait, capture/stumble steering, and a recovery gate that restores swing authority after pushes. Difficulty stays at up to 250 N. See the methods note on this page for equations, curricula, and fade levers toward pure end-to-end control.

What is the walk-time counter?

It tracks how long Rocky stays upright in your browser session and compares your best to the global best reported by other visitors.

Rocky Learns to Walk Live | SuperDroid Robots

Methods note · SuperDroid Robots research

Learning locomotion with a removable control scaffold

How we train sim-concept-01 (“Rocky”) to walk and recover from large pushes: a phase-locked reference gait plus capture/stumble scaffolding, a residual PPO policy in MuJoCo MJX (Warp), coupled curricula that never lower the difficulty ceiling, and an explicit plan to fade every hand-written term to zero.

Source of truth: humanoid_loco/gait_reference.py, env/flat_walk_mjx.py, inference.action_to_q_ref, PIPELINE.md · public runs: Stage A recovery lineage (r51→r73)

1. Problem statement

End-to-end bipedal RL under impulsive disturbances is hard because a fixed gait clock and a throttled swing residual are excellent for clean walking and terrible for protective steps. A push can demand that the planted foot move immediately; a scripted schedule that still owns that leg makes ~100% survival at 250 N structurally unreachable. Our answer is not to soften the task. We keep 1500-step episodes and a 30–250 N push range, and instead introduce an explicit, measurable scaffold that assists recovery during Stage A, then fades to zero (Stages B–C) so the final artifact is network + PD only.

Diagram contrasting swing authority during walking versus recovery mode — Figure 1. The authority conflict that motivates scaffolding.

2. Control factorization

At 50 Hz the actor emits 22 residuals a. Joint targets are assembled identically in training (MJX) and deploy (CPU MuJoCo / this browser):

q_ref = q₀ + w_g·gait(φ)·move_gate(cmd) + w_s·steer(v,r) + w_u·stumble(·) + α(r)·tanh(a)·authority(φ,r)

Torques follow MIT PD, τ = clip(kp·(q_ref−q) − kd·q̇), held across physics substeps (~250 Hz). Phase 1 keeps feedforward torque at zero. Arms are PD-held (ARMS_MODE=held). The scripted gait uses cos-phased stride and large knee/ankle clearance (~0.68 / 0.38 rad) with no baked-in abduction (the policy owns frontal balance). Swing sagittal joints are throttled to SWING_POLICY_WEIGHT = 0.22 while walking so the residual cannot cancel the reference lift.

Block diagram of q_ref additive terms and fade weights — Figure 2. Additive scaffold terms and their fade levers.

3. Recovery gate, latch, and dual paths

A scalar recovery gate r ∈ [0,1] rises with torso tilt (|ĝ_xy|) and uncommanded horizontal velocity error. Thresholds were recalibrated after measuring that undisturbed gait can produce ~1 m/s velocity-error peaks—earlier settings made the gate a gait-phase detector. A gate latch (~0.75 s) holds recovery through the velocity→tilt hand-off after a push.

Recovery drives two different paths on purpose. The control path (steer, authority, optional gait fade / 2× phase rate) engages on tilt or velocity so feet can move. The reward path is anti-farmed: capture and recovery-shaped credit require push_recent or a large velocity kick—tilt alone is not paid, because policies otherwise learn to walk “tipsy” to harvest rewards.

Flowchart of recovery gate splitting into control path and reward path — Figure 3. Recovery gate: control assistance vs reward credit.

Capture steer biases foot placement (including crossover / stagger / relief modes: if the fall-side foot is loaded, briefly plant the other foot nearby so the loaded leg can take the wide recovery step). Stumble adds multi-step scaffolding in the re-armed post-capture window. Authoritative abduction override exists as a lever but is off on current public runs (CAPTURE_ABD_OVERRIDE_SCALE=0).

4. Observations and learning stack

Training uses PPO with an asymmetric actor-critic in MuJoCo MJX (Warp), typically 1024 environments on an RTX 3080. The shipped actor sees only deployable features:

Diagram of 81-dim instant features stacked five deep plus push-sense suffix — Figure 4. Actor observation packing (409 dims).

Network: dense 256→256→256 tanh MLP → 22-D Gaussian mean (fixed log-std on recent consolidation runs). Critic uses privileged state and is discarded at publish. Domain randomization includes mass, friction, sensor noise, and a hard tail (~30% of envs) with elevated push floors and mid-swing onset to concentrate rare failure modes (e.g. relief-plant misses) without easing the rest of the distribution.

5. Coupled curricula (difficulty never down)

Three ramps advance only when gates pass (survival, tracking, recovery bars, minimum update counts)—and the environment ceiling stays hard:

Command scale — from ~0.05 toward full command_ranges.
Push force ceiling — from a gentle start (e.g. 100 N) up to 250 N in steps, with interval 6–12 s and duration 0.05–0.20 s.
Action scale — residual range (e.g. 0.30→0.38) with backoff if actions saturate.

Schematic line chart of command scale, push ceiling, and action scale curricula — Figure 5. Coupled curricula schematic (Stage A).

Gold checkpoints and high-impulse probes (fixed seed suites) decide whether a rung is keepable. A representative Stage A deliverable recovered 47/48 probe events (r69@8500); later rungs (r72/r73) raise hard-tail force floors and add relief-mode steering rather than relaxing physics.

6. Pipeline: assist → fade → end-to-end

The standing directive is that scaffolding is temporary. Hard removal in one shot has been falsified (r62); Stage B fades levers rung-by-rung with probe gates between rungs.

Three-stage pipeline from scaffold-assisted training to pure end-to-end — Figure 6. Stages A → B → C.

Bar chart of scaffold lever weights in Stage A versus Stage C — Figure 7. Fade levers. When all weights are zero, `q_ref = q₀ + α·tanh(a)`.

Table 1. Scaffold fade levers
Lever	Stage A default	At 0
`GAIT_REF_WEIGHT`	1.0	No scripted walking reference
`STEER_WEIGHT`	1.0	No capture / relief steer
`STUMBLE_GAIN`	0.8	No multi-step stumble scaffold
`AUTHORITY_WEIGHT`	1.0	No swing-authority throttle logic
`RECOVERY_PHASE_RATE_GAIN`	1.0	Plain phase clock
`RECOVERY_GAIT_FADE`	0.0 (already)	Gait always on during recovery
`CAPTURE_ABD_OVERRIDE_SCALE`	0.0 (already)	No authoritative abd override

7. What the live dashboard measures

The charts above are the pinned TensorBoard scalars used to judge runs—not marketing proxies:

Episode length at done — survival in control steps (cap 1500).
Survival completion (+ EMA) — fraction of the horizon completed.
Full episode rate — push-deploy evaluations that finish intact.
Header actuator strips — live from your browser episode: peak torque (N·m and ft-lb), peak speed (rad/s and rpm), duty at peak, and Avg Power (electrical watts at 98% motor efficiency). Torque/speed are the max across all 22 actuated joints, including PD-held arms. Wattage is Σ|τ ω| / 0.98. Avg Power’s large number is the time-average over the last 8 s; Peak in smaller type is the max in that window. Hover a strip for the value at that time.

Policies and metrics publish on a cadence to this page; your browser runs MuJoCo WASM and refreshes about every 15 s. The Nudge control samples the training push force/duration ranges with a random heading. Browser recovery is still a simplified subset of the full Python scaffold path—useful for intuition, not a claim of probe parity.

Training publish stack from MJX to browser — Figure 8. Lab → publish → browser (no public TensorBoard/Viser).

8. Non-claims

This is not a physical robot livestream, not Stage C completion, and not a statement that every probe failure class is solved. It is a living methods note for the Stage A scaffold-assisted stack that currently feeds the public checkpoint, and a map of how that scaffold is designed to disappear.

Rocky learns to walk

Live dashboard