LeRobot Camera Setup for the SO-ARM101: Wrist and Overhead Cameras Explained

Most imitation-learning policies for the SO-ARM101 train on two camera views: a wrist (eye-in-hand) camera on the gripper and a fixed overhead (third-person) camera above the workspace. Getting these two right matters more than any other hardware choice: a shaky or badly framed camera produces datasets that no policy can learn from.

1. Choose the lenses

Camera	Recommended field of view	Why
Wrist	90°, low-distortion	Sees both gripper fingers and the object without fisheye curvature
Overhead	110°–140°	Covers a 60 × 60 cm workspace from 50–60 cm above the desk

The 1080p USB camera module comes with a 60°, 90°, 110° or 140° M12 lens; buy one 90° for the wrist and one 110° or 140° for the overhead view.

2. Mount the wrist camera

The SO-101 gripper has a camera boss on the static jaw. A 32 × 32 mm camera board screws to a small printed bracket that tilts the lens about 15° toward the fingertips, so the object stays in frame during the last centimetres of a grasp. Route the USB cable along the inside of the forearm and leave a loop at the wrist joint; a tight cable is the most common cause of a wrist servo dropping out.

3. Mount the overhead camera

Clamp a desk arm with a 1/4" ball head to the back edge of the table so the camera looks down at roughly 45° from 50–60 cm above the workspace. Keep it off the same table leg the arm is clamped to if you can; vibration from the arm shows up as motion blur. Never move the overhead camera between recording and evaluation.

4. Tell LeRobot about the cameras

cameras = {
  "wrist": OpenCVCameraConfig(index_or_path=0, width=640, height=480, fps=30),
  "top":   OpenCVCameraConfig(index_or_path=2, width=640, height=480, fps=30),
}

Run lerobot-find-cameras opencv to get the indices, then record a 10-second test episode and scrub through it: both views must show the gripper reaching the object, with no dropped frames.

5. Lighting and background

• Use a plain, non-reflective mat; the 10 × 10 tray of our training cubes gives a fixed grid the policy can anchor to.
• Constant light: switch off windows-driven daylight changes with a desk lamp on both sessions.
• Fix the exposure in the camera driver if the auto-exposure pulses.

FAQ

What resolution should I record at?

640 × 480 at 30 fps is what most LeRobot tutorials and pretrained policies expect; it keeps datasets small. Record 1280 × 720 only if you plan to crop or need fine detail.

Why does LeRobot show the wrong camera index?

Linux assigns /dev/video numbers in plug-in order. Use lerobot's find_cameras script after plugging both cameras in, and plug them into the same USB ports every session, or address them by serial path.

Can I use a phone or a Logitech webcam?

Yes, anything UVC works. Webcams are bulkier on the wrist; a bare camera module on a bracket weighs 15 g and does not shift the arm's balance.