The problem
You ended 0.4 with a dataset and an uneasy question: is it any good? You recorded some episodes, but a directory of parquet files answers none of the things you actually care about. How many of your demonstrations reached the target? How long are they? Is the orientation you think you recorded the orientation that's actually in there? "I collected data" and "I know what's in my data" are different sentences, and the gap between them is where silent training failures live.
There is a specific trap hiding in that gap. The dataset does not store success. It does not store reward. Chapter 0.4 wrote exactly two things per frame — observation.state and action — because that is all a policy trains on. So the success rate you want is not a column you can read off; it is something you have to reconstruct from the observations, by applying the same rule the environment used. To know your data, you have to be able to decode it. And decoding has sharp edges — the block's orientation, for one, is stored not as an angle but as a (sin, cos) pair, and there is exactly one correct way to turn that back into an angle.
inspect.py is the tool for all of this: load a LeRobot v3 dataset back, read its structure, reconstruct the success hiding in it, and replay it to rerun so you can read an episode like a story.
Build
The shape of the file: load, reconstruct, replay. Load a dataset into plain numpy, reconstruct the errors and the success signal the dataset never stored, and log the whole thing to rerun on the canonical entity paths. There is also a small provisioning helper so the chapter runs even before you have data of your own.
Setup
One thing to notice before anything else — the file has to defend its own name.
import osimport sys # This file is named inspect.py, which collides with the STDLIB `inspect`# module: running `python inspect.py` seeds sys.path[0] with this file's own# directory, so numpy/mujoco's `import inspect` would find THIS file instead of# the standard library and crash. Drop our own dir from the path (FIRST, before# any heavy import) and add the repo root so `curriculum.common` resolves — the# same root-on-path pattern ch0.1/0.4 use, minus the self-shadowing._HERE = os.path.dirname(os.path.abspath(__file__))sys.path[:] = [p for p in sys.path if os.path.abspath(p or ".") != _HERE]sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(_HERE)))) import argparse # noqa: E402import json # noqa: E402import math # noqa: E402from pathlib import Path # noqa: E402 import numpy as np # noqa: E402 # The env is shared reference code (decision 004): we import its success# constants so the reconstruction below uses the SAME tolerances the recorder's# env used, never a second copy that could silently disagree.from curriculum.common.device import banner # noqa: E402from curriculum.common.envs.pusht.pusht_env import PushTEnv, wrap_angle # noqa: E402 # The three break modes are real misconceptions you can inject and MEASURE, not# toy typos. Each corrupts how you READ the data and leaves an honest signature.BREAK_MODES = ("none", "yaw-swap", "drop-boundaries") parser = argparse.ArgumentParser(description=__doc__)parser.add_argument("--seed", type=int, default=0, help="seeds the stand-in generator; two --seed 0 runs produce byte-identical metrics.json")parser.add_argument("--smoke", action="store_true", help="tiny fixed-length stand-in for CI; two --smoke runs must match byte-for-byte")parser.add_argument("--out", type=Path, default=Path("outputs/ch0.5-inspect"), help="run dir: metrics.json here, a provisioned stand-in in {out}/dataset")parser.add_argument("--dataset", type=Path, default=None, help="a LeRobot v3 dataset root to inspect (e.g. your ch0.4 output); default: provision a stand-in")parser.add_argument("--episodes", type=int, default=6, help="stand-in only: how many episodes to generate (ignored when --dataset is given)")parser.add_argument("--break", dest="break_mode", choices=BREAK_MODES, default="none", help="inject a reading misconception: yaw-swap | drop-boundaries")parser.add_argument("--rerun", dest="rerun", action="store_true", default=True) # replaying your episode to rerun is the default; opt OUT, not inparser.add_argument("--no-rerun", dest="rerun", action="store_false", help="skip the .rrd (CI smoke)")args = parser.parse_args() banner("ch0.5-inspect") # startup contract: tier + measured wall-clock to stdoutrng = np.random.default_rng(args.seed) # PCG64 — only used by the stand-in generatorThis artifact is called inspect.py, and inspect is a standard-library module. When you run python inspect.py, Python helpfully puts the script's own directory first on the import path — which means the next time anything (numpy, mujoco) does import inspect, it finds this file instead of the standard library and everything falls over. So the very first act of the file is to remove its own directory from the path and put the repo root there instead. It is a small, deliberate seam, called out here rather than hidden, because a debugging tool that trips over its own filename teaches the wrong lesson. With that done, we import the PushT environment for one reason: its success tolerances. We are going to reconstruct "did this reach the target," and we must use the same POS_TOL and ANG_TOL the recorder's world used, not a second copy that could drift.
Load
Reading a dataset back is two facts, and no more.
# Reading a dataset back is two facts: the SCHEMA (what every frame is) and the# EPISODE INDEX (where one demonstration ends and the next begins). lerobot's# meta.episodes carries per-episode [from, to) row ranges — an episode is a# contiguous run of frames, nothing more. We slice the frame table by those# ranges into plain numpy so the rest of the file never touches a tensor.def load_episodes(dataset_root: Path, break_mode: str): """Return (info, episodes). info: schema facts. episodes: a list of dicts with numpy `states` (N,10) and `actions` (N,2), one per demonstration.""" from lerobot.datasets.lerobot_dataset import LeRobotDataset dataset = LeRobotDataset(repo_id="zero2robot/inspect", root=dataset_root) frames = dataset.hf_dataset info = { "obs_dim": len(dataset.meta.features["observation.state"]["names"]), "act_dim": len(dataset.meta.features["action"]["names"]), "fps": int(dataset.meta.fps), "feature_keys": sorted(k for k in dataset.meta.features if k.startswith("observation") or k == "action"), } def slice_episode(lo: int, hi: int) -> dict: rows = frames[lo:hi] return {"states": np.stack([np.asarray(s, np.float32) for s in rows["observation.state"]]), "actions": np.stack([np.asarray(a, np.float32) for a in rows["action"]])} if break_mode == "drop-boundaries": # THE BUG: ignore episode_index and treat the whole table as ONE episode. # Everyone writes this once — you forget demonstrations have boundaries. episodes = [slice_episode(0, dataset.num_frames)] else: episodes = [slice_episode(int(ep["dataset_from_index"]), int(ep["dataset_to_index"])) for ep in dataset.meta.episodes] return info, episodesThe first fact is the schema: what every frame is. observation.state is ten floats, action is two — the contract you held in 0.4, read back from meta/info.json. The second fact is the episode index: lerobot's meta.episodes carries, for each demonstration, the [from, to) range of rows it occupies. That is the entire definition of an episode — a contiguous run of frames — and it is the fact everyone forgets exactly once (there's a --break for forgetting it, below). We slice the frame table by those ranges into plain numpy arrays so the rest of the file never has to touch a tensor.
Success — seeing like a robot
Here is the heart of the chapter.
# Seeing like a robot: the success signal is not stored, so decode it from the# state. tee_yaw lives as (sin, cos) at obs[4:6] to dodge the +/-pi wrap you met# in 0.3; the ONE correct way back to an angle is atan2(sin, cos) — arguments in# that order. Get them backwards and every orientation you read is reflected.def decode_tee_yaw(state: np.ndarray, break_mode: str) -> float: sin_yaw, cos_yaw = float(state[4]), float(state[5]) if break_mode == "yaw-swap": # THE BUG: atan2(cos, sin) reads the reflected angle (pi/2 - true). The # block will look rotated in rerun, and every ang_err below is wrong. return math.atan2(cos_yaw, sin_yaw) return math.atan2(sin_yaw, cos_yaw) def frame_errors(state: np.ndarray, break_mode: str) -> tuple[float, float]: """Reconstruct the env's (pos_err, ang_err) from one stored observation. Target is fixed at the origin with yaw 0 (obs[6:10]); the block is obs[2:6].""" pos_err = float(np.hypot(state[2] - state[6], state[3] - state[7])) ang_err = float(abs(wrap_angle(decode_tee_yaw(state, break_mode)))) # target yaw is 0 return pos_err, ang_err def episode_reached(states: np.ndarray, break_mode: str) -> bool: """Did this demonstration reach the goal? The recorder STOPS the instant the env latches success (it breaks on `done`), so the last frame it stored is the block sitting on the target — reading 'reached' is reading whether the episode ENDED in tolerance. The dataset never says 'success'; this decode IS the reading, and it uses the env's own POS_TOL / ANG_TOL, not a second copy.""" pos_err, ang_err = frame_errors(states[-1], break_mode) return pos_err < PushTEnv.POS_TOL and ang_err < PushTEnv.ANG_TOLThe orientation of the block is stored as sin(yaw) and cos(yaw) — the sin/cos trick from 0.3, which exists precisely so the value never jumps as the angle wraps past ±π. The one correct way back to an angle is atan2(sin, cos), arguments in that order. Swap them and you get the reflected angle, and every orientation you read from then on is wrong — the block looks rotated in the replay and the error you compute against it is garbage. That is not a hypothetical; it is --break yaw-swap, and it is the whole reason "seeing like a robot" is a skill and not a slogan.
With the decode in hand, frame_errors reconstructs the environment's pos_err (how far the block is from the target) and ang_err (how far its yaw is from the target's) from a single observation. And episode_reached answers the question you came for: did this demonstration reach the goal? Note how it answers. The recorder stops the instant the env latches success — it breaks on done — so the last frame it stored is the block sitting on the target. Reading "reached" is therefore reading whether the episode ended in tolerance. The dataset never says "success"; this decode is the reading.
Inspect
The walk turns a list of episodes into the handful of numbers you actually wanted.
# The walk: turn a list of episodes into the handful of numbers you actually# wanted — lengths, the reconstructed success rate, and the terminal error of# each demonstration. This is what "looking at your dataset" concretely means.def inspect_episodes(episodes: list, break_mode: str) -> dict: per_episode = [] for states in (ep["states"] for ep in episodes): pos_err, ang_err = frame_errors(states[-1], break_mode) # terminal frame = where the demo left the block per_episode.append({ "length": int(len(states)), "reached": bool(episode_reached(states, break_mode)), "final_pos_err": round(pos_err, 6), "final_ang_err": round(ang_err, 6), }) lengths = [rec["length"] for rec in per_episode] n_reached = sum(rec["reached"] for rec in per_episode) return { "n_episodes": len(per_episode), "n_frames": int(sum(lengths)), "episode_lengths": lengths, "reached": [rec["reached"] for rec in per_episode], "n_reached": int(n_reached), "success_rate": round(n_reached / len(per_episode), 6) if per_episode else 0.0, "mean_episode_length": round(float(np.mean(lengths)), 6) if lengths else 0.0, "final_pos_err": [rec["final_pos_err"] for rec in per_episode], "final_ang_err": [rec["final_ang_err"] for rec in per_episode], }Lengths, the reconstructed success rate, and each demonstration's terminal error. This is what "looking at your dataset" concretely means, and it is the same summary you will want for every dataset in the book — before you train on it, so a bad batch of demos fails your eyes here instead of your policy three chapters later.
Rerun
And then we replay it, so you can read it rather than tabulate it.
# Replay every stored observation onto the canonical entity paths (the SAME ones# every chapter uses, per .claude/skills/rerun-instrument), so you scrub your own# episodes on the sim_time timeline and watch the pusher worry the block toward# the green target. The reconstructed pos_err/ang_err ride along as scalars — and# under --break yaw-swap the tee visibly points the wrong way._TEE_CENTERS = [(0.0, 0.0, 0.0), (0.0, -0.06, 0.0)] # the T is two boxes (see pusht.xml)_TEE_HALF = [(0.06, 0.015, 0.015), (0.015, 0.045, 0.015)] def log_rerun(episodes: list, fps: int, break_mode: str, path: Path): import rerun as rr # lazy: --no-rerun (CI) never imports it rr.init("zero2robot/ch0.5-inspect", spawn=False) rr.save(str(path)) gx, gy, _ = PushTEnv.TARGET_POSE rr.log("world/objects/target", rr.Boxes3D(centers=[(gx + c[0], gy + c[1], 0.0) for c in _TEE_CENTERS], half_sizes=_TEE_HALF, colors=(90, 205, 100, 120)), static=True) rr.log("world/objects/tee", rr.Boxes3D(centers=_TEE_CENTERS, half_sizes=_TEE_HALF, colors=(115, 128, 242)), static=True) rr.log("world/robot/pusher", rr.Cylinders3D(lengths=[0.04], radii=[0.015], colors=(230, 102, 90)), static=True) frame_index = 0 for episode_index, ep in enumerate(episodes): for state, action in zip(ep["states"], ep["actions"]): rr.set_time("sim_time", duration=frame_index / fps) frame_index += 1 tee_yaw = decode_tee_yaw(state, break_mode) # the decode under test — reflected under yaw-swap pos_err, ang_err = frame_errors(state, break_mode) rr.log("world/objects/tee", rr.Transform3D(translation=(float(state[2]), float(state[3]), 0.0152), rotation=rr.RotationAxisAngle(axis=(0, 0, 1), radians=tee_yaw))) rr.log("world/robot/pusher", rr.Transform3D(translation=(float(state[0]), float(state[1]), 0.02))) rr.log("policy/action", rr.Scalars(np.asarray(action, dtype=np.float64))) rr.log("eval/pos_err", rr.Scalars([pos_err])) rr.log("eval/ang_err", rr.Scalars([ang_err])) rr.log("eval/episode_index", rr.Scalars([float(episode_index)]))Every stored observation is logged onto the same entity paths every chapter uses — world/objects/tee, world/robot/pusher — on the sim_time timeline, with policy/action and the reconstructed eval/pos_err / eval/ang_err riding along as scalars. Because it uses the same decode as the reconstruction, --break yaw-swap doesn't just change a number — it visibly rotates the block in the viewer, so the picture and the numbers disagree in front of you. That is the debugging instinct this chapter is trying to install.
Run it
With no dataset of your own yet, it makes a small one to inspect — six scripted-expert demonstrations, with a little noise turned on because real teleop is never clean:
python inspect.py --episodes 6
| cpu-laptop | expected wall-clock on cpu-laptop: ~0.06 min (measured) | measured |
| mps | wall-clock on mps: not yet measured | pending |
| t4 | expected wall-clock on t4: ~0.16 min (measured) | measured |
| 4090 | wall-clock on 4090: not yet measured | pending |
It prints the schema, the episode lengths, and the reconstructed reached target: N/6 — then drops an .rrd you can open with rerun. Pointing it at your real 0.4 dataset is the same command with --dataset. With seed 0 the six-episode stand-in records lengths [65, 89, 136, 52, 82, 143] and all six reach the target — a cleaner run than the one you recorded by hand will be, which is exactly why you learn to check.
Break it
Two ways to read the same, correct data wrongly — each with an honest signature you can measure:
python inspect.py --episodes 3 --break yaw-swap
python inspect.py --episodes 3 --break drop-boundaries
yaw-swap decodes the orientation as atan2(cos, sin) — the reflected angle. Nothing about the recorded data changes; only the reading. The signature is unmistakable: the schema and every episode length are byte-identical to a correct run, but the reconstructed success rate collapses from 3-of-3 to 0-of-3, because every block now reads as rotated ~90° off, and its ang_err sails past ANG_TOL. The block in the rerun replay points visibly wrong. This is the failure mode of trusting a decode you didn't check.
drop-boundaries forgets that demonstrations have boundaries — it ignores the episode index and treats the whole frame table as one long episode. The signature: your three episodes collapse into one long one, the "success rate" becomes a meaningless 1-out-of-1, and in the replay the block teleports between demonstrations because the timeline never resets. It is the bug you write the first time you load someone else's dataset and assume it's one trajectory.
Exercises
Three ways to make sure you can read a dataset, not just open one. Predict before you run.
What's next
You can now record a dataset (0.4) and read it (0.5) — which means you have, for the first time, everything a policy needs: examples, in a known format, that you've actually looked at. Phase 1 is where those examples become a policy. Chapter 1.1 takes this exact dataset and fits the dumbest thing that works — behavior cloning — and the very first thing it will do when the policy misbehaves is send you back here, to open the .rrd and ask what the policy saw. The inspector you just built is not a Phase-0 detour; it is the instrument you debug the rest of the book with.