Recording your first LeRobot dataset with an SO-ARM101: 30 episodes
Share
Recording a LeRobot dataset means physically moving the leader arm while the follower arm mirrors the motion and cameras record the action, creating demonstrations that an AI model learns to replicate. With an SO-ARM101 robot arm kit or similar setup, you can collect 30 episodes of a pick-and-place task in about one hour, enough to start training your first policy. This guide assumes you have already run lerobot-calibrate and lerobot-teleoperate.
What a dataset episode contains
A LeRobot dataset is a directory of timestamped observations and actions. Each episode is one complete demonstration: move to a cube, close the gripper, lift, move to target, open gripper, return to neutral. The recording includes joint positions, torques, and camera frames.
LeRobot saves episodes as Parquet files. A 30-episode dataset occupies 2–5 GB, depending on camera resolution and episode length. Consistent demonstrations make policies easier to train.
| Dataset parameter | Recommended for first dataset | Why |
|---|---|---|
| Task | Pick cube, place in fixed target | Simple, repeatable, no ambiguity |
| Episodes | 30 | Enough for ACT or Diffusion Policy to show signs of learning |
| Episode length | 15–30 seconds | Covers motion without excessive idle time |
| Camera resolution | 480p wrist, 1080p overhead | Balances file size and visual detail |
| Frame rate | 30 fps | Standard for LeRobot, matches servo update rate |
Choosing a task and setting up the workspace
Your first dataset should use a single object, a fixed target, and no distractors. A colored wooden cube works well: clear edges, does not roll, and you can mark the target with tape. The training props set includes 100 cubes and a tray for consistent placement.
Place the cube in the same starting position for every episode. Mark it with tape. Place the target within reach, far enough to show visible motion but close enough to avoid joint limits. Test with lerobot-teleoperate first.
Mount your cameras before recording. Use an overhead camera desk arm to position the overhead camera consistently. Keep cameras fixed between episodes to reduce noise.
Recording the dataset with lerobot-record
Create a new dataset with the LeRobot CLI:
bash
lerobot-record \
--robot-path lerobot/configs/robot/so101.yaml \
--repo-id your-username/pick-cube-v1 \
--single-task pick_cube \
--num-episodes 30 \
--warmup-time-s 3 \
--episode-time-s 30 \
--reset-time-s 5
Replace your-username/pick-cube-v1 with your Hugging Face username and dataset name. For each episode, you have three seconds to position the cube, 30 seconds to complete the task, and five seconds to reset.
During warmup, place the cube and move the leader arm to neutral. When recording starts, perform the task smoothly: move to the cube, close gripper, lift, move to target, open gripper, return to neutral. Avoid sudden jerks; the policy learns from your motion style.
If you drop the cube or hit a joint limit, press Ctrl+C to skip that episode and record it again. Re-recording is faster than training on bad data.
How many episodes you need
Thirty episodes is a practical starting point for simple tasks. ACT and Diffusion Policy can learn from 30–50 demonstrations if the task is consistent. Complex tasks with multiple objects or variable positions require 50–200 episodes.
After training on 30 episodes, if the policy completes the task twice in a row, you have captured the pattern. If it fails every time, record 20 more episodes. Datasets below 20 episodes rarely work unless the task is trivial.
Recording speed improves with practice. Your first ten episodes take longer as you refine motion and camera angles. By episode 20, each demonstration should take under two minutes.
Common mistakes
Inconsistent starting positions cause policies to fail. If you place the cube slightly differently each time, the policy learns that location varies and does not converge. Use a reference mark and place the cube in the exact same spot.
Recording with the gripper already closed skips the approach and grasp sequence. Start every episode with the gripper open and the leader arm neutral.
Recording too fast causes motion blur. Move smoothly at the speed you would pick up a mug. If frames blur, slow down.
After recording
Push the dataset to Hugging Face with huggingface-cli upload. Check the dataset page to confirm all episodes uploaded correctly. Training takes 2–12 hours on a GPU. Start with ACT if unsure; it trains faster than Diffusion Policy and works well for pick-and-place.
If your policy does not learn after 30 episodes, record 20 more and check consistency: watch random episodes in a video player and confirm the motion path looks similar. Inconsistent demonstrations cause training failure.
For parts and accessories, visit the robot arm parts and accessories collection. For camera mounting, see the LeRobot camera setup guide.
FAQ
Can I record a dataset on a Raspberry Pi 5 or do I need a Linux desktop?
You can record on a Raspberry Pi 5, but the CPU struggles to encode 1080p camera frames in real time. Use 480p for both cameras if you record on a Pi, or offload recording to a Linux desktop with a faster CPU. The arm control itself works fine on a Pi; only the camera encoding is slow.
What happens if I run out of disk space during recording?
LeRobot stops recording and saves only the completed episodes. Check available disk space with df -h before starting. A 30-episode dataset with two cameras at 1080p and 480p uses about 3–5 GB. If you are recording to a microSD card, make sure you have at least 10 GB free.
Do I need to re-record the dataset if I change the camera positions later?
Yes, if you move the cameras significantly. Policies learn the spatial relationship between what the cameras see and where the arm moves. If you change the overhead camera angle by more than a few degrees, the learned policy will not generalize, and you need to record a new dataset with the updated camera setup.