The Simulation Loop

The problem

You've seen simulated robots before — locomotion clips, manipulation demos, the works. Here's a question that separates watching from building: when a simulated box rests on a simulated table, what is the computer actually doing, hundreds of times a second, to keep it there?

If your answer is a shrug, everything downstream stays fuzzy. When a training run in chapter 2.4 produces a quadruped that vibrates through the floor, you need to know whether to suspect your reward or the physics. When your policy in chapter 1.1 acts at 10 Hz but the simulator steps at 500 Hz, you need to know what happens in the 49 steps between decisions. Every one of those debugging sessions bottoms out in the same two data structures and one function call.

So that's what we build: a complete, runnable simulation — load a world, step it through time, poke it mid-run, and read out what happened — in about 160 lines, and the loop at the heart of it is under twenty.

Build

MuJoCo splits the world into two objects, and the split is the single most important idea in this chapter. mjModel is everything that never changes while the simulation runs: geometry, masses, joint layout, actuator gearing. It's compiled once from a description file and then treated as read-only. mjData is everything that does change: positions, velocities, contact forces, the current time. mj_step reads the model, mutates the data, and that's the whole game. One consequence you'll cash in later: a single model can drive many independent datas — hold that thought until chapter 2.3, where it becomes 4096 robots training at once.

Setup

One thing to look for: there's a random number generator here, and it is the only source of randomness in the entire file.

import argparseimport jsonimport sysfrom pathlib import Path import mujocoimport numpy as npimport rerun as rr # Chapter artifacts run as loose scripts from the repo root; put the root on# sys.path so `curriculum.common` resolves (same pattern as ch1.1). device.py# keeps its torch import lazy, so this torch-free chapter stays torch-free.sys.path.insert(0, str(Path(__file__).resolve().parents[3]))from curriculum.common.device import banner  # noqa: E402 parser = argparse.ArgumentParser(description=__doc__)parser.add_argument("--seed", type=int, default=0, help="seeds the shove direction and strength")parser.add_argument("--steps", type=int, default=2000)  # any laptop: instantparser.add_argument("--timestep", type=float, default=0.002)  # sim seconds per mj_step; raise it and see Break Itparser.add_argument("--smoke", action="store_true", help="fixed 300-step run for CI; two runs must match byte-for-byte")parser.add_argument("--out", type=Path, default=Path("outputs/ch0.1-sim-loop"))parser.add_argument("--rerun", dest="rerun", action="store_true", default=True)  # recording is the default; opt OUT, not inparser.add_argument("--no-rerun", dest="rerun", action="store_false", help="skip the .rrd recording (CI smoke)")parser.add_argument("--no-perturb", dest="perturb", action="store_false", help="skip the shove — a baseline the tests compare against")args = parser.parse_args() banner("ch0.1-sim-loop")  # startup contract: every artifact prints tier + measured wall-clock (to stdout, not metrics.json)num_steps = 300 if args.smoke else args.steps  # smoke length is FIXED so CI can diff runs exactlyargs.out.mkdir(parents=True, exist_ok=True)rng = np.random.default_rng(args.seed)  # PCG64 — the only source of randomness in this file

The flags follow a convention you'll see in every chapter: free-tier defaults first, and a --smoke mode that runs short and fixed so CI can compare two runs byte-for-byte. The --seed flag controls exactly one thing in this file — the direction and strength of the shove we'll set up below. Physics itself needs no seed: given the same model and the same inputs, MuJoCo on CPU produces the same trajectory every time. Determinism isn't something we add; it's something we protect, by routing every random choice through one seeded generator.

Scene

The scene is handed to you this chapter — in 0.2 you'll write your own from scratch. Read it top to bottom anyway; it's short.

# The scene is handed to you this chapter; in 0.2 you write your own. Read it# top to bottom anyway: a floor plane, a red box attached to the world by a# free joint (all six degrees of freedom — loose on the table), and a blue# pusher that can ONLY slide along x, driven by a single motor.SCENE_XML = """<mujoco model="pusher_scene">  <worldbody>    <light pos="0 0 3" dir="0 0 -1"/>    <geom name="floor" type="plane" size="2 2 0.1" rgba="0.85 0.85 0.85 1"/>    <body name="box" pos="0.4 0 0.05">      <freejoint/>      <geom name="box_geom" type="box" size="0.05 0.05 0.05" mass="0.5" rgba="0.85 0.3 0.25 1"/>    </body>    <body name="pusher" pos="0 0 0.05">      <joint name="pusher_slide" type="slide" axis="1 0 0" damping="4"/>      <geom name="pusher_geom" type="box" size="0.04 0.12 0.05" mass="1.0" rgba="0.25 0.45 0.85 1"/>    </body>  </worldbody>  <actuator>    <motor name="push_motor" joint="pusher_slide" gear="10" ctrlrange="-1 1"/>  </actuator></mujoco>""" # mjModel: the compiled world — geometry, masses, joints, actuator gearing.# Nothing in it changes while the simulation steps. Build it once.model = mujoco.MjModel.from_xml_string(SCENE_XML)model.opt.timestep = args.timestep  # "never changes" means during stepping; between runs it's yours to edit # mjData: the state — positions, velocities, contact forces, time. Everything# mj_step writes lives here. One model can drive many independent datas# (that idea becomes 4096 parallel robots in chapter 2.3).data = mujoco.MjData(model) box = model.body("box")  # named lookups beat remembering integer idspusher = model.body("pusher")

Three things live in this world. A floor. A red box connected to the world by a freejoint — all six degrees of freedom, which is the XML way of saying "loose on the table". And a blue pusher that can only slide along x, because that's the single joint we gave it, driven by a single motor. That asymmetry is deliberate: the box can go anywhere physics sends it; the pusher can only do the one thing its actuator allows. Robots live on the pusher's side of that line.

Below the XML sit the two constructor calls that give this chapter its title: MjModel.from_xml_string compiles the description, MjData(model) allocates the state. Note the one honest asterisk on "never changes": we overwrite model.opt.timestep from a flag right after compiling. The model is frozen during stepping; between runs it's yours to edit. Break It below abuses exactly this.

Perturb

Before the loop starts, we decide how we're going to interfere with it.

# Mid-run we shove the box sideways with a force the pusher knows nothing# about. xfrc_applied is mjData's slot for exactly this: an external# (force, torque) on any body, added on top of whatever physics is already# doing. It PERSISTS until you clear it — MuJoCo never resets it for you,# and forgetting that is a classic bug (exercise 2 makes you find it).push_start = num_steps // 2push_steps = 50  # 0.1 s of shove at the default timesteppush_newtons = rng.uniform(6.0, 12.0)  # friction on the 0.5 kg box resists with ~5 N; this winspush_sign = rng.choice([-1.0, 1.0])  # +y or -y: sideways, across the pusher's line of travelpush_force = np.array([0.0, push_sign * push_newtons, 0.0])

xfrc_applied is mjData's slot for external forces: a force and torque you inject on any body, added on top of whatever physics is already doing. It's how a mouse-drag in the viewer works, and it's how we'll simulate disturbances for the rest of the book. The number is worth a look: the box weighs half a kilogram, so at MuJoCo's default friction coefficient of 1.0 the floor resists sliding with roughly 5 N (mass × g × μ), and we sample a shove between 6 and 12 N — enough to win, not enough to launch it.

One property of xfrc_applied matters more than all the others: it persists. MuJoCo does not clear it after a step. Set it once and it pushes forever, which is either exactly what you want or a bug you'll hunt for an afternoon. Exercise 2 makes you find that bug on purpose.

Loop

Here it is — the loop the whole book runs inside.

if args.rerun:    rr.init("zero2robot/ch0.1-sim-loop", spawn=False)    rr.save(str(args.out / "sim_loop.rrd"))    # Shapes and colors never change, so log them once as static; the loop    # then logs only the moving transforms.    rr.log("world/objects/box", rr.Boxes3D(half_sizes=[[0.05, 0.05, 0.05]], colors=[[217, 76, 64]]), static=True)    rr.log("world/robot/pusher", rr.Boxes3D(half_sizes=[[0.04, 0.12, 0.05]], colors=[[64, 115, 217]]), static=True) # mj_forward fills in the derived state — body poses, contact lists — from qpos# WITHOUT advancing time. mj_step would recompute all of it anyway, but calling# it here means the first frame we log below is the true rest pose at t=0, not# the state after a step has already run. It is how you inspect a world you# haven't stepped yet.mujoco.mj_forward(model, data) for step in range(num_steps):    in_shove = args.perturb and push_start <= step < push_start + push_steps    if step == push_start:        box_pos_at_push = data.xpos[box.id].copy()  # .copy(): xpos is a view into mjData and the next mj_step overwrites it in place     if args.rerun and step % 5 == 0:  # log the state ENTERING this step; logging every step quintuples file size and teaches nothing extra        rr.set_time("sim_time", duration=data.time)        # MuJoCo stores quaternions wxyz; rerun wants xyzw — hence the reindex.        rr.log("world/objects/box", rr.Transform3D(translation=data.xpos[box.id], quaternion=data.xquat[box.id][[1, 2, 3, 0]]))        rr.log("world/robot/pusher", rr.Transform3D(translation=data.xpos[pusher.id]))        shove_arrow = push_force * (0.02 if in_shove else 0.0)  # scaled to scene size; zero-length when inactive        rr.log("world/objects/box/shove", rr.Arrows3D(origins=[data.xpos[box.id]], vectors=[shove_arrow]))     # The rhythm every later chapter repeats: write intent, write interference, step.    data.ctrl[0] = 1.0  # full throttle on the slide motor — in chapter 1.1, a policy writes this line    data.xfrc_applied[box.id, :3] = push_force if in_shove else 0.0  # write 0 explicitly, or the shove sticks forever    mujoco.mj_step(model, data)  # one timestep: collision, contact, actuation, integration — the whole pipeline

Read the body of the for loop as a rhythm you'll repeat for 33 more chapters: write your intent into data.ctrl, write any outside interference into data.xfrc_applied, call mj_step, look at what changed. Today ctrl is a hardcoded 1.0 — full throttle on the slide motor. In chapter 1.1 a neural network writes that line instead, and nothing else about the loop changes.

Why mj_step and not our own integration? Because a step is not position += velocity * dt. Inside that one call MuJoCo detects collisions, solves for contact forces, applies actuator torques, and only then integrates. You could hand-roll the integration for a floating box; the moment two things touch, you couldn't. (In chapter 3.3 you will hand-roll it, and contact will take three chapters — that's the point.)

Two small things in the loop repay attention. First, mj_forward before the loop. mj_step computes the same derived quantities — body poses, contact lists — but only as a side effect of advancing time; mj_forward computes them without stepping, so the first frame we log is the true rest pose at t=0, before anything has moved. It's how you inspect a world you haven't stepped yet, and you'll reach for it whenever you need to read a freshly-loaded state. Second, the .copy() when we snapshot the box position: data.xpos is a view into mjData's memory, and the next mj_step overwrites it in place. Saving a reference instead of a copy is the other classic first-week bug.

The rerun calls log the two moving bodies against a sim_time timeline, into world/objects/box and world/robot/pusher — the same entity paths every chapter uses, so the debugging tool you learn here is the one you'll still be using at the capstone. MuJoCo hands us quaternions as wxyz and rerun wants xyzw; the reindex on that line is the first of many small format seams you'll cross in robotics, and not the last quaternion convention issue in this book. Chapter 0.3 is about the rest of them.

Inspect

The run is over. Where did everything end up?

# The box is declared first, so its free joint owns qpos[0:7] — xyz position# plus a wxyz quaternion. Its qvel slice is qvel[0:6]: 7 position numbers but# 6 velocity numbers, because a quaternion has 4 components and only 3# degrees of freedom. Chapter 0.3 lives inside that gap.final_box_pos = data.qpos[0:3]final_box_speed = float(np.linalg.norm(data.qvel[0:3]))lateral_drift = float(abs(final_box_pos[1] - box_pos_at_push[1]))  # how far the shove pushed it off the x axis metrics = {    "steps": num_steps,    "seed": args.seed,    "perturb": bool(args.perturb),    "box_final_pos": [round(float(v), 6) for v in final_box_pos],    "box_final_speed": round(final_box_speed, 6),    "pusher_final_x": round(float(data.qpos[7]), 6),  # the slide joint's single dof comes after the box's 7    "lateral_drift": round(lateral_drift, 6),    "shove_moved_box": bool(lateral_drift > 0.01),}(args.out / "metrics.json").write_text(json.dumps(metrics, indent=2, sort_keys=True) + "\n") print(f"stepped {num_steps} steps of {model.opt.timestep} s -> {data.time:.2f} s of sim time")print(f"box final position {np.round(final_box_pos, 3)}, speed {final_box_speed:.3f} m/s")if args.perturb:    print(f"shove: {push_force[1]:+.1f} N in y for {push_steps} steps -> {lateral_drift:.3f} m of sideways drift")else:    print("no shove (baseline run)")print(f"metrics: {args.out / 'metrics.json'}")if args.rerun:    print(f"recording: {args.out / 'sim_loop.rrd'} — open it with: rerun {args.out / 'sim_loop.rrd'}")

The box was declared first in the XML, so its free joint owns the first slots of the state vectors: qpos[0:7] and qvel[0:6]. Seven position numbers, six velocities. That mismatch isn't a bug — a quaternion carries four numbers but only three degrees of freedom, so positions need one more slot than velocities. For now, treat it as a rule; chapter 0.3 lives inside that gap. The pusher's single slide joint takes the next slot, qpos[7], and that's the entire state of this world: eight numbers.

The metrics file is deliberately boring: final positions rounded to six decimals, written with sorted keys. Boring is the feature — run the smoke config twice with the same seed and the two files are byte-identical, and CI holds every chapter in this book to that standard.

Run it

python sim_loop.py

The run prints where the box ended up and drops a recording at outputs/ch0.1-sim-loop/sim_loop.rrd. Open it:

rerun outputs/ch0.1-sim-loop/sim_loop.rrd

Scrub the sim_time timeline. You should see the pusher close the gap, the box slide ahead of it, and — halfway through — the sideways lurch of the shove, with the force arrow visible under world/objects/box/shove for exactly 0.1 s. With seed 0 the shove is +9.8 N in y and the box ends up about 13 cm off its original line. Change --seed and the shove changes with it; nothing else does.

If you don't see the lurch, first check you didn't pass --no-perturb; second, look at the entity tree — if world/objects/box isn't there, you're looking at an old recording.

Break it

The timestep is 0.002 s — five hundred mj_step calls per simulated second, two thousand for the four seconds we just ran. Steps cost wall-clock, so a bigger timestep is the obvious economy: fewer steps to buy the same span of simulated time. Let's take that economy 25× too far:

python sim_loop.py --timestep 0.05 --no-perturb

--no-perturb matters here: we strip out our own interference so that anything strange is the simulator's doing, not ours. And it is strange. Scrub the recording. The moment the pusher reaches the box, the box hops into the air off a flat, straight, horizontal push — z peaks around 0.11 m against 0.05 at rest — and by the end of the run it has drifted a quarter of a meter sideways, despite the fact that no lateral force exists anywhere in this scene. The energy came from nowhere. That's the signature to memorize: motion with no force to explain it means the integrator is manufacturing energy, and it manufactures it at contacts, where forces are stiffest and big steps hurt most.

The diagnosis walk in rerun: pick the box in the entity tree, watch its z track. At 0.002 s it's a flat line at 0.05 after settling. At 0.05 s it spikes at first contact. Same model, same scene, same code — the only thing that changed is how far each step extrapolates before physics gets a say. When a later chapter's robot jitters, dances, or launches itself, this is the first dial to check, and now you know its failure signature on sight.

(Why 0.002 and not 0.0002, then? Ten times the steps for the same simulated second. The timestep is a genuine trade — stability against wall-clock — and exercise 1 makes you find the edge yourself before you're allowed to trust the default.)

Exercises

Three, in exercises/: a bug-hunt where the metrics come out almost right and only your model of what persists across mj_step explains the gap, and two where you commit to a prediction before the run is allowed to answer.

What's next

You stepped a world, but you didn't build one — the XML at the top of this file was handed to you, and it's a toy: one box, one pusher, geometry chosen by someone else. The moment you want a different table, a goal region, or a T-shaped block instead of a cube, you're editing MJCF yourself, and MJCF has opinions about bodies, joints, and what's attached to what. Next chapter you build, from an empty file, the exact PushT scene the following fifteen chapters train on.