Graduation Bridge ILeRobot for Real

Where you are

You have built ACT from scratch. In 1.3 you wrote the encoder that lets four entity tokens attend to each other, the decoder whose K query tokens each emit one action of the chunk, the temporal ensembler that averages overlapping predictions into a smooth trajectory — ~380 lines you can read top to bottom. That was the point of the whole exercise: you now know what ACT is, mechanically, with no magic left in it.

This chapter cashes that in. The same policy already lives in lerobot, the library the field actually ships and the one whose dataset format your gen_demos and your 0.4 recorder have been writing all along. Instantiating ACT there is not 380 lines. It is about twenty:

# THE ENTIRE OFFICIAL POLICY. Contrast this block with all of 1.3's act.py: the# encoder, decoder, chunking head, CVAE, and temporal ensembler are ALL in here,# behind three constructors. temporal_ensemble_coeff=0.01 (with n_action_steps=1)# is the real ACT ensembling we hand-rolled in 1.3. use_vae=False MATCHES 1.3 (which# cut the CVAE) so the comparison is apples-to-apples — flip it to True for the full# CVAE ACT. This is the payoff of having built it yourself: every argument is legible.config = ACTConfig(    input_features=input_features, output_features=output_features,    chunk_size=K, n_action_steps=1, temporal_ensemble_coeff=0.01,    dim_model=args.model_dim, n_heads=8, dim_feedforward=2 * args.model_dim,    n_encoder_layers=1, n_decoder_layers=1, use_vae=False,    dropout=0.1, optimizer_lr=args.lr, device=args.device)policy = ACTPolicy(config, dataset_stats=stats).to(device)preprocessor, postprocessor = make_act_pre_post_processors(config, dataset_stats=stats)n_params = sum(p.numel() for p in policy.parameters())print(f"official lerobot ACT: {n_params:,} params (use_vae={config.use_vae}, "      f"chunk_size={K}, dim_model={args.model_dim})")

That block is the entire policy: the encoder, the decoder, the chunking head, the CVAE, and the temporal ensembler — every piece you built by hand — behind three constructors. This is the first time in the course you are encouraged to import a framework that hides the loop, because you are no longer learning the loop. You are graduating past it.

The reversal, and why you earned it

Everywhere else in Phase 1 the doctrine is strict: no hydra, no pytorch_lightning, no wrappers that swallow the training step, because a hidden loop you have never written is a loop you do not understand. That rule was for the learner you were. The rule for the learner you are now is the opposite: reach for the maintained, tested, community implementation — and read its arguments like a menu you understand. use_vae=True turns the CVAE latent back on (1.3 cut it). temporal_ensemble_coeff=0.01 is the exact ensembling you hand-rolled. The ResNet image backbone is in there too, dormant only because we feed state, not pixels. You can see all of it because you have already written all of it.

Loading your data, and the one honest wrinkle

The dataset loads unchanged — it is the same LeRobot-v3 format throughout Phase 1:

# Reuse 1.3's exact demo source so the two ACTs train on bit-identical data.if args.data is None:    args.data = args.out / "demos"    if not (args.data / "meta" / "info.json").is_file():        gen_demos.main(["--episodes", str(args.num_demos), "--seed", str(args.seed),                        "--out", str(args.data), "--no-video"])if not (args.data / "meta" / "info.json").is_file():    sys.exit(f"no dataset at {args.data} — generate one first:\n"             f"  python curriculum/common/envs/aloha_cube/gen_demos.py "             f"--episodes 50 --seed 0 --out {args.data} --no-video") # Heavy imports AFTER the cheap failures — the whole official stack, in one place.from lerobot.configs.types import FeatureType, PolicyFeature  # noqa: E402from lerobot.datasets.lerobot_dataset import (  # noqa: E402    LeRobotDataset, LeRobotDatasetMetadata)from lerobot.datasets.utils import dataset_to_policy_features  # noqa: E402from lerobot.policies.act.configuration_act import ACTConfig  # noqa: E402from lerobot.policies.act.modeling_act import ACTPolicy  # noqa: E402from lerobot.policies.act.processor_act import make_act_pre_post_processors  # noqa: E402 ds_meta = LeRobotDatasetMetadata(REPO_ID, root=args.data)features = dataset_to_policy_features(ds_meta.features)# THE BRIDGE, in two lines. Our state IS the whole world (arms+cube+target), so# it is ACT's `environment_state`, not proprioceptive `observation.state`. ACT's# state-only schema requires that key; re-typing it to ENV is the entire remap.features["observation.environment_state"] = PolicyFeature(    FeatureType.ENV, features.pop("observation.state").shape)output_features = {k: v for k, v in features.items() if v.type is FeatureType.ACTION}input_features = {k: v for k, v in features.items() if k not in output_features}# Normalization stats travel WITH the data (this is what the --break skips in 1.6# spirit — here we always pass them). Re-key state stats to the bridged name.stats = dict(ds_meta.stats)stats["observation.environment_state"] = stats.pop("observation.state")print(f"dataset: {ds_meta.total_episodes} episodes / {ds_meta.total_frames} frames "      f"@ {ds_meta.fps} Hz; bridged observation.state -> observation.environment_state")

There is exactly one bridge to build, and it is a good bug to have met: ACT's state-only schema requires the input key observation.environment_state, while our recorder emits observation.state. The fix is not just a rename — it is a re-type. lerobot classifies every feature by its FeatureType, and ACT accepts a camera-less policy only when it finds a feature typed ENV. Rename the key but leave the type STATE and the policy is rejected (or, in a looser setup, the environment-state token is silently never built and you train on a truncated input). Exercise 3 is exactly this bug, isolated.

Training: the chunk you built by hand is now one declaration

In 1.3 you assembled the action chunk with an explicit loop over every episode, gathering the next K expert actions per frame and a pad mask. The official stack does it from a single declaration — delta_timestamps — and hands you the action_is_pad mask for free:

# lerobot builds the action CHUNK for us: delta_timestamps asks the dataset for K# future action frames per sample, plus an `action_is_pad` mask at episode ends —# the exact bookkeeping we wrote by hand in 1.3, now a one-liner. The preprocessor# normalizes with the dataset stats; policy.forward returns the L1(+KL) loss.delta_timestamps = {"action": [i / ds_meta.fps for i in range(K)]}dataset = LeRobotDataset(REPO_ID, root=args.data, delta_timestamps=delta_timestamps)loader = torch.utils.data.DataLoader(    dataset, batch_size=args.batch_size, shuffle=True, num_workers=0,    generator=torch.Generator().manual_seed(args.seed))  # seeded order => reproducible on CPUoptimizer = torch.optim.AdamW(policy.parameters(), lr=args.lr)policy.train()train_loss, global_step = float("nan"), 0for epoch in range(args.epochs):    epoch_loss, num_batches = 0.0, 0    for batch in loader:        # The device-probe inside ACT reads observation.state even when it uses        # env_state, so we carry both keys (same tensor); only env_state is a        # model input. preprocessor() normalizes, batches, and moves to device.        batch["observation.environment_state"] = batch["observation.state"]        batch = preprocessor(batch)        loss, _ = policy.forward(batch)        optimizer.zero_grad()        loss.backward()        optimizer.step()        epoch_loss, num_batches = epoch_loss + loss.item(), num_batches + 1        if args.rerun:            rr.set_time("step", sequence=global_step)            rr.log("official/loss/train", rr.Scalars([loss.item()]))        global_step += 1    train_loss = epoch_loss / num_batches    if epoch % 25 == 0 or epoch == args.epochs - 1:        print(f"epoch {epoch:4d}  loss {train_loss:.5f}")

Same bookkeeping, one line. The preprocessor normalizes with the dataset's own stats (this is why the stats must travel with the data — pass the wrong stats and the policy silently trains on mis-scaled inputs), and policy.forward returns the L1 loss (it adds a KL term when use_vae=True). On CPU the loss falls from about 0.51 to about 0.08 over 150 epochs.

Run it

python curriculum/phase1_imitation/ch1.9_bridge/bridge.py --seed 0

One command trains the official ACT, trains your from-scratch 1.3 ACT on the same demos as a subprocess, evaluates both on the same held-out seeds, and dry-runs the publish. It finishes in a few minutes on a CPU laptop — the free-tier floor, no GPU required.

The comparison, with error bars

Here is the question the chapter turns on: now that the official ACT is twenty lines, is it better than your 380-line one? bridge.py answers it honestly — it trains your 1.3 act.py as a subprocess on the same demos and evaluates both on the same held-out seeds, and — because 1.6 taught you that a bare success rate is a lie — it reports the official policy's rate with its Wilson interval:

# Booleans in, statistics out — the 1.6 discipline. A bare success rate hides its# own uncertainty; the Wilson interval is the honest error bar at these few# episodes. --break train_dist swaps the held-out seeds for the TRAINING seeds:# the policy has effectively memorized those exact starts, so the number inflates# and you would wrongly crown the official stack. That is the 1.6 sin, committed# on the real ACT — the eval, not the model, is what breaks.def wilson_ci(k: int, n: int) -> tuple[float, float]:    """95% Wilson score interval for k successes in n trials (no scipy)."""    if n == 0:        return (0.0, 0.0)    p, z2 = k / n, Z95 * Z95    center = (p + z2 / (2 * n)) / (1 + z2 / n)    half = (Z95 * np.sqrt(p * (1 - p) / n + z2 / (4 * n * n))) / (1 + z2 / n)    return (max(0.0, center - half), min(1.0, center + half))  @torch.no_grad()def rollout(seed: int, episode: int) -> tuple[bool, float]:    policy.reset()  # clears the temporal-ensembling buffer between episodes    env = AlohaCubeEnv()    observation = env.reset(seed)    done, episode_return, info = False, 0.0, {}    while not done:        obs_t = torch.from_numpy(observation)        batch = preprocessor({"observation.state": obs_t, "observation.environment_state": obs_t})        action = postprocessor(policy.select_action(batch))  # un-normalized env action        observation, reward, done, info = env.step(action.squeeze(0).cpu().numpy())        episode_return += reward        if args.rerun:            rr.set_time("sim_time", duration=episode * (AlohaCubeEnv.MAX_STEPS                        / AlohaCubeEnv.CONTROL_HZ) + env.data.time)            rr.log("eval/official/dist", rr.Scalars([info["dist"]]))    return bool(info["success"]), episode_return  # Held-out by default (base 20000, disjoint from demo seeds 0..num_demos-1);# --break train_dist evaluates on the demo seeds themselves (in-distribution).eval_base = args.seed if args.break_mode == "train_dist" else HELD_OUT_BASE + args.seedoutcomes = [rollout(eval_base + ep, ep) for ep in range(args.eval_episodes)]successes = sum(s for s, _ in outcomes)official_success = successes / args.eval_episodesofficial_return = float(np.mean([r for _, r in outcomes]))lo, hi = wilson_ci(successes, args.eval_episodes)if args.rerun:    rr.log("eval/official/success_rate", rr.Scalars([official_success]))print(f"official ACT:  success {official_success:.2f}  [{lo:.2f}, {hi:.2f}]  "      f"return {official_return:.3f}  ({'TRAIN-DIST' if args.break_mode else 'held-out'} "      f"seeds, n={args.eval_episodes})") # The from-scratch baseline: run 1.3's act.py as a subprocess on the SAME dataset# and seed, then read its held-out metrics.json. Same env, same seed formula# (base 20000), same episode count => an apples-to-apples comparison.scratch = Noneif args.compare and ACT_PY.is_file():    fs_out = args.out / "from_scratch"    cmd = [sys.executable, str(ACT_PY), "--data", str(args.data), "--seed", str(args.seed),           "--device", "cpu", "--no-rerun", "--out", str(fs_out)]    cmd += ["--smoke"] if args.smoke else [        "--chunk_size", str(K), "--model_dim", str(args.model_dim),        "--num_demos", str(args.num_demos), "--epochs", str(args.epochs),        "--eval_episodes", str(args.eval_episodes)]    print("\n[compare] training the from-scratch 1.3 ACT on the same data ...")    subprocess.run(cmd, check=True, cwd=ROOT)    scratch = json.loads((fs_out / "metrics.json").read_text())    print(f"from-scratch 1.3 ACT:  success {scratch['success_rate']:.2f}  "          f"return {scratch['mean_return']:.3f}  (held-out seeds, n={args.eval_episodes})")    # Read the BARS, not the point estimates (1.6): is the from-scratch rate inside    # the official Wilson interval? Inside => indistinguishable at this n; outside =>    # a real gap. Either way the durable headline is the CODE ratio, not a winner.    inside = lo <= scratch["success_rate"] <= hi    verdict = ("their intervals overlap — indistinguishable at this n"               if inside else "the from-scratch rate is OUTSIDE the official CI — a real gap at this n")    print(f"compare: {verdict}. The headline is the CODE ratio (~380 lines by hand "          f"vs ~20 official), not the winner.")

At the free-tier default (matched to 1.3: no CVAE, lr 1e-3, 150 epochs) the official ACT lands at 0.55 [0.34, 0.74] on 20 held-out seeds. That interval is wide on purpose — 20 episodes is thin, exactly the 1.6 point. The from-scratch 1.3 ACT at the matched budget lands at 0.95, and its point estimate sits above the official interval: this is a real gap, not sampling noise.

Why does the hand-rolled version win? Not because the framework is worse — it is the same algorithm. It is because in 1.3 you split the ten observation numbers into four semantic tokens (right arm, left arm, cube, target) and let them attend to each other. That decomposition is a task-specific inductive bias. The general lerobot config sees one undifferentiated state vector. Your domain knowledge is the difference — and it is portable: you could inject the same tokenization into a custom lerobot input processor. What the framework bought you, then, is not a higher number; it is a maintained, tested implementation, a dataset-and-model Hub, a real-robot deploy path, and a dozen other policies one import away. That trade — you keep your modeling insight, you gain the ecosystem — is the honest shape of graduating to a real stack. (Both rates also rise with the scale knobs; the honest way up is --epochs and --num_demos, not a louder claim.)

Break it: the 1.6 sin, one flag away

--break train_dist evaluates the official ACT on the training seeds instead of the held-out ones — the starts it has already seen. In a brand-new framework this is an easy accident: you grab whatever seed list is handy. The rate inflates to 0.60 [0.39, 0.78] against the held-out 0.55 [0.34, 0.74]. Here the gap is small — our held-out seeds are the same distribution as training (random resets, different draws), so the policy has little to memorize — but it is in the wrong direction to trust, and on a genuinely out-of-distribution held-out set (1.6's LIBERO-style annulus, the block spawned farther than any demo) the same sin costs far more. The invariant is not the magnitude; it is that you never report the train number. Graduating to a real stack does not graduate you out of held-out evaluation.

Publishing (when you have a token)

The last step of the real workflow is sharing: push the dataset and the trained policy to the Hugging Face Hub so others can reproduce and build on them. That step is human-gated — it needs a token and a network — so by default, and always in CI, bridge.py dry-runs it: it prints the exact push_to_hub calls it would make and serializes the policy locally instead.

# HUMAN-GATED. Offline or without a token this DRY-RUNS: it prints the exact# push_to_hub calls and serializes the policy locally (proving the save path)# WITHOUT any network. A real publish needs `huggingface-cli login` (HF_TOKEN)# and HF_HUB_OFFLINE unset — never CI, never the default path.import os  # noqa: E402 local_model = args.out / "lerobot_act"have_token = bool(os.environ.get("HF_TOKEN")) and os.environ.get("HF_HUB_OFFLINE") not in ("1", "true")if args.publish and have_token:    policy.push_to_hub(args.repo_id)          # real push (needs network + token)    dataset.push_to_hub(f"{args.repo_id}_dataset")    print(f"published policy -> hf.co/{args.repo_id} and dataset -> {args.repo_id}_dataset")else:    policy.save_pretrained(local_model)       # local serialization, always offline-safe    reason = "no --publish" if not args.publish else "offline / no HF_TOKEN"    print(f"\n[publish DRY-RUN — {reason}] saved policy to {local_model}. To publish for real:")    print("    huggingface-cli login   # sets HF_TOKEN")    print(f"    policy.push_to_hub({args.repo_id!r})")    print(f"    dataset.push_to_hub({args.repo_id + '_dataset'!r})")

To publish for real: huggingface-cli login, unset HF_HUB_OFFLINE, and pass --publish. Nothing in this repository ships a checkpoint or a dataset — those live on the Hub; the repo carries only the code that regenerates them.

What the framework hides (and what you can now see through)

lerobot is not magic, and after 1.3 it is not opaque to you either. The CVAE you dropped is back; the ResNet path you never needed is waiting; the normalization is a processor pipeline instead of two lines of your own. When one of those matters — when you need to change the objective, add a camera, or debug why the loss will not fall — you have the from-scratch mental model to open the file and read it. That is the real deliverable of this bridge: not that you can call ACTPolicy, but that you can call it and still know exactly what it is doing.

Exercises

Four, in exercises/. Two are predict-then-run and turn on the comparison: before you run, you predict whether the official ACT beats your from-scratch 1.3 ACT at the matched budget (it does not — the entity-token bias wins, above the official's Wilson interval), and whether evaluating on the training seeds inflates the rate over held-out (it does — the 1.6 sin, on the official stack). Two are fast and self-contained: a bug-hunt on the observation.environment_state re-type (the key is not enough; the FeatureType must be ENV), and a code-completion that rebuilds 1.3's action chunk as a one-line delta_timestamps declaration.

Read the real thing

The paired reading is lerobot itself, at the course-pinned v0.4.4 — the code your # --- region: official --- just called. Everything lives under src/lerobot/ at this tag; read each file against the piece of yours it replaces.

The policy — src/lerobot/policies/act/modeling_act.py. Your 1.3 act.py was the ~380 lines you can hold in your head. This is roughly 800, and every class in it is one you already wrote: ACTPolicy wraps ACT (the CVAE-plus-transformer core), ACTTemporalEnsembler is the ensembler behind the official region's temporal_ensemble_coeff=0.01, and ACTEncoder / ACTDecoder are your encoder and chunking decoder. The docstring credits Zhao et al.'s original ACT — same algorithm, same paper. What the extra ~400 lines buy is not accuracy (your entity-token 1.3 ACT beat this file's general config, above its Wilson interval); they buy the torchvision ResNet backbone, gated on config.image_features and dormant only because you feed state — the camera path you never needed, waiting.

The wiring — src/lerobot/policies/factory.py. make_policy(cfg, ds_meta=...) is the dispatch you skipped by importing ACTPolicy directly, and get_policy_class is a registry mapping "act", "diffusion", "smolvla", "pi0", and six more to their classes — a dozen policies one string away. It calls dataset_to_policy_features (defined in src/lerobot/datasets/utils.py, not the factory itself), the exact function whose output you re-typed in # --- region: data --- to slot observation.environment_state in.

The chunk — src/lerobot/datasets/lerobot_dataset.py. The delta_timestamps you passed in # --- region: train --- is checked by check_delta_timestamps and turned into delta_indices; then _get_query_indices builds the {key}_is_pad mask — your hand-written episode-boundary padding loop from 1.3, now a few lines behind a tolerance_s.

The stats — src/lerobot/processor/normalize_processor.py. NormalizerProcessorStep (and its inverse UnnormalizerProcessorStep) is the pipeline step behind make_act_pre_post_processors. Read its missing-stats branch: when a feature has no stats, it returns the input unchanged — the silent mis-scaling this chapter warned you about, sitting in production code. It supports MEAN_STD, MIN_MAX, QUANTILES, and QUANTILE10.

Read next: open modeling_act.py beside your own act.py. You built each of these once; the framework's gift is that you never have to again — and that you can still read every line when it matters. The number stayed yours; the ecosystem is what you gained.

What's next

That closes the imitation spine. Everything in Phase 1 learned from a fixed dataset of expert demonstrations — behavior cloning, ACT, diffusion, flow, the evaluation harness, the VLA — and this chapter delivered you onto the stack the field actually uses to do it. Phase 2 changes the question. Instead of imitating demonstrations you already have, you will generate the data by trial and error: chapter 2.1 opens with PPO from a blank file, a policy that improves against a reward instead of a teacher. The Hub publish path you dry-ran here returns for real at the 4.4 capstone, where you ship a trained policy, its dataset, and a writeup — the graduation this bridge was named for.