API reference¶
Generated from the source docstrings. The SO101Env lerobot config class is
not included here because importing it requires lerobot (and therefore torch);
its fields are documented in
Using it with lerobot.
Environments¶
lerobot_env_so101.SO101PickCubeGymEnv ¶
SO101PickCubeGymEnv(
seed: int = 0,
control_dt: float = 0.1,
physics_dt: float = 0.002,
render_spec: GymRenderingSpec = GymRenderingSpec(),
render_mode: Literal[
"rgb_array", "human"
] = "rgb_array",
image_obs: bool = False,
reward_type: str = "sparse",
random_block_position: bool = False,
action_scale: float = 1.0,
top_down_ik: bool = False,
jaw_pads: bool = False,
)
Bases: SO101GymEnv
Environment for an SO-101 robot picking up a cube.
Create the pick-cube environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed
|
int
|
Seed for the environment's own RNG, which drives block
placement when |
0
|
control_dt
|
float
|
Seconds of simulated time per |
0.1
|
physics_dt
|
float
|
MuJoCo integration timestep. |
0.002
|
render_spec
|
GymRenderingSpec
|
Frame height, width, and camera id used by
|
GymRenderingSpec()
|
render_mode
|
Literal['rgb_array', 'human']
|
|
'rgb_array'
|
image_obs
|
bool
|
Add a |
False
|
reward_type
|
str
|
|
'sparse'
|
random_block_position
|
bool
|
Randomize the block position on each reset instead of using a fixed pose. |
False
|
action_scale
|
float
|
Metres per unit of position action. Applies to
|
1.0
|
top_down_ik
|
bool
|
Keep the gripper pointing straight down and move the
grasp point between the jaws instead of |
False
|
jaw_pads
|
bool
|
Enable the fingertip collision pads. See |
False
|
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_env.py
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | |
reset ¶
Reset the environment.
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_env.py
step ¶
Take a step in the environment.
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_env.py
lerobot_env_so101.SO101PickPlaceGymEnv ¶
SO101PickPlaceGymEnv(
seed: int = 0,
control_dt: float = 0.1,
physics_dt: float = 0.002,
render_spec: GymRenderingSpec = GymRenderingSpec(),
render_mode: Literal[
"rgb_array", "human"
] = "rgb_array",
image_obs: bool = False,
random_block_position: bool = False,
action_scale: float = 1.0,
top_down_ik: bool = True,
jaw_pads: bool = True,
)
Bases: SO101PickCubeGymEnv
Environment for an SO-101 robot moving a cube to a goal position.
environment_state is the block position followed by the goal position.
Success needs the block to have been lifted off the table earlier in the
episode, so pushing it to the goal does not count.
The IK keeps the gripper pointing straight down by default, so the action
moves the grasp point between the jaws (ik_point_pos) rather than the
gripperframe site reported in agent_pos.
Create the pick-and-place environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed
|
int
|
Seed for the environment's own RNG. |
0
|
control_dt
|
float
|
Seconds of simulated time per |
0.1
|
physics_dt
|
float
|
MuJoCo integration timestep. |
0.002
|
render_spec
|
GymRenderingSpec
|
Frame height, width, and camera id used by
|
GymRenderingSpec()
|
render_mode
|
Literal['rgb_array', 'human']
|
|
'rgb_array'
|
image_obs
|
bool
|
Add a |
False
|
random_block_position
|
bool
|
Randomize the block and goal positions on each reset instead of using fixed ones. |
False
|
action_scale
|
float
|
Metres per unit of position action. Must be positive. |
1.0
|
top_down_ik
|
bool
|
Keep the gripper pointing straight down. Without it the jaws cannot straddle the block. |
True
|
jaw_pads
|
bool
|
Enable the fingertip collision pads. Without them the block is pinched at two points and swings out of the jaws. |
True
|
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_place_env.py
reset ¶
Reset the arm, the block, and the goal.
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_place_env.py
step ¶
Take a step in the environment.
Source code in packages/lerobot_env_so101/lerobot_env_so101/pick_place_env.py
lerobot_env_so101.SO101GymEnv ¶
SO101GymEnv(
xml_path: Path | None = None,
seed: int = 0,
control_dt: float = 0.02,
physics_dt: float = 0.002,
render_spec: GymRenderingSpec = GymRenderingSpec(),
render_mode: Literal[
"rgb_array", "human"
] = "rgb_array",
image_obs: bool = False,
home_position: ndarray = HOME_POSITION,
cartesian_bounds: ndarray = CARTESIAN_BOUNDS,
action_scale: float = 1.0,
top_down_ik: bool = False,
jaw_pads: bool = False,
)
Bases: MujocoGymEnv
SO-101 pick/place base environment: 4-DOF Cartesian position IK control.
Base class only: it implements robot control but not step()/reset().
Instantiate a task subclass such as SO101PickCubeGymEnv.
Native action space is [dx, dy, dz, grasp]. SO-101's 5-DOF arm cannot
independently track a 6-DOF Cartesian pose, so the IK controller solves for
position only and there are no orientation deltas to command. Use
lerobot_env_so101.wrappers.SevenDofToFourDofAdapter for teleop/HIL code
written against the upstream 7-dim convention.
Action deltas are unscaled metres, inherited from gym-hil's teleoperation convention: an action of 1.0 saturates the workspace in a single step. See "Known limitations" in the README before training a policy against this.
The grasp component is an absolute target in [0, 1] (0=closed, 1=open),
not a delta: it is written directly to the gripper actuator's ctrl each
step via lerobot_env_so101.gripper.normalized_to_ctrl.
Create the SO-101 base environment.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
xml_path
|
Path | None
|
MJCF scene to load. Defaults to the bundled
|
None
|
seed
|
int
|
Seed for the environment's RNG. |
0
|
control_dt
|
float
|
Seconds of simulated time per control step. |
0.02
|
physics_dt
|
float
|
MuJoCo integration timestep. |
0.002
|
render_spec
|
GymRenderingSpec
|
Frame height, width, and camera id used by
|
GymRenderingSpec()
|
render_mode
|
Literal['rgb_array', 'human']
|
|
'rgb_array'
|
image_obs
|
bool
|
Whether the task subclass should include a camera view in its observation. |
False
|
home_position
|
ndarray
|
Arm joint configuration applied by
|
HOME_POSITION
|
cartesian_bounds
|
ndarray
|
|
CARTESIAN_BOUNDS
|
action_scale
|
float
|
Metres per unit of position action. Must be positive. |
1.0
|
top_down_ik
|
bool
|
Solve the IK for the grasp point between the jaws
with the gripper pointing straight down, instead of for the
|
False
|
jaw_pads
|
bool
|
Enable collision on the box pads at the two fingertips. MuJoCo collides the jaw meshes as convex hulls, which touch a block at one point each; the pads give it flat faces to be held by. |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 | |
ik_point_pos
property
¶
World position of the point the action moves.
The gripperframe site by default, the grasp point between the jaws
with top_down_ik.
reset_robot ¶
Reset the arm to the home position and resynchronize the IK target.
The IK target (_target_ee_pos) is an accumulated Python attribute
rather than a MuJoCo body, so it must be re-read from the end-effector
sensor here; otherwise it would still hold the target from before the
reset.
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
apply_action ¶
Apply a native 4-dim action [dx, dy, dz, grasp] via position IK.
Position deltas are multiplied by action_scale (metres per unit
action); the grasp command is an absolute target in [0, 1]
(0=closed, 1=open), written directly to the gripper ctrl each step,
and is not scaled.
A near-zero position delta holds the current pose rather than the accumulated target, so repeated zero actions do not drift toward an overshot target.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
action
|
ndarray
|
Array of |
required |
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
get_robot_state ¶
Get the current state of the robot.
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
get_gripper_pose ¶
Gripper ctrl target normalized to [-1, 1]: -1 = fully closed, +1 = fully open.
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
render ¶
Render the front camera view.
The renderer is built on first use: constructing it eagerly would open an OpenGL context on every instantiation, which fails on a headless host even for state-only training that never renders.
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
lerobot_env_so101.MujocoGymEnv ¶
MujocoGymEnv(
xml_path: Path,
seed: int = 0,
control_dt: float = 0.02,
physics_dt: float = 0.002,
render_spec: GymRenderingSpec = GymRenderingSpec(),
)
Bases: Env
A MuJoCo model wrapped in the gymnasium interface.
Owns the model, data, and renderer, and derives the number of physics substeps per control step. Robot-specific control and the task loop belong to subclasses.
Load an MJCF model and configure the simulation timestep.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
xml_path
|
Path
|
Path to the MJCF scene. |
required |
seed
|
int
|
Seed for |
0
|
control_dt
|
float
|
Seconds of simulated time per control step. |
0.02
|
physics_dt
|
float
|
MuJoCo integration timestep. The number of substeps per
control step is |
0.002
|
render_spec
|
GymRenderingSpec
|
Frame height, width, and camera id used by
|
GymRenderingSpec()
|
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
render ¶
Render one offscreen frame.
Returns:
| Type | Description |
|---|---|
ndarray
|
An |
ndarray
|
camera. |
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
close ¶
Release graphics resources if they exist.
Source code in packages/lerobot_env_so101/lerobot_env_so101/mujoco_env.py
lerobot_env_so101.GymRenderingSpec
dataclass
¶
GymRenderingSpec(
height: int = 128,
width: int = 128,
camera_id: str | int = -1,
mode: Literal["rgb_array", "human"] = "rgb_array",
)
Offscreen frame size and camera used for rendering.
camera_id of -1 selects MuJoCo's free camera; pass a camera name
from the MJCF (for example "front") to render a fixed view.
Wrappers¶
lerobot_env_so101.SevenDofToFourDofAdapter ¶
Bases: ActionWrapper
Adapts 7-dim Cartesian actions to this package's native 4-dim action space.
Teleop and HIL tooling built on gym-hil's Panda convention emits 7-dim
actions [dx, dy, dz, drx, dry, drz, grasp]. SO-101 environments take a
native 4-dim [dx, dy, dz, grasp]; this wrapper discards the orientation
deltas (indices 3-5), which the position-only IK controller cannot act on.
grasp is passed through unchanged, so its meaning follows the wrapped
env's convention (an absolute target in [0, 1], 0=closed/1=open).
Wrap a native SO101GymEnv-based env to accept 7-dim actions from existing teleop/HIL code; new code should prefer the native 4-dim action space.
Wrap a native 4-dim env so it accepts 7-dim actions.
Emits a :class:UserWarning once at construction, rather than per
step, so that a caller porting 7-dim code learns why the arm never
rotates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
env
|
Env
|
An environment whose action space is a 4-dim |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the wrapped env's action space is not a 4-dim
|
Source code in packages/lerobot_env_so101/lerobot_env_so101/wrappers.py
action ¶
Drop the rotation deltas from a 7-dim action.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
action
|
ndarray
|
|
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
|
Source code in packages/lerobot_env_so101/lerobot_env_so101/wrappers.py
Gripper utilities¶
lerobot_env_so101.gripper.normalized_to_ctrl ¶
Map a normalized gripper command in [0, 1] (0=closed, 1=open) to ctrl radians.
Out-of-range input is clipped to [0, 1] before mapping.
Source code in packages/lerobot_env_so101/lerobot_env_so101/gripper.py
lerobot_env_so101.gripper.ctrl_to_normalized ¶
Inverse of normalized_to_ctrl: ctrl radians -> normalized [0, 1].
Source code in packages/lerobot_env_so101/lerobot_env_so101/gripper.py
Inverse kinematics¶
These back the position-only Cartesian control described in Action space. They are internal to the environment rather than part of the intended public surface, but are documented here because their behaviour explains several of the limitations.
lerobot_env_so101.ik_control.solve_ik ¶
solve_ik(
model: MjModel,
data: MjData,
site_id: int,
dof_ids: ndarray,
target_pos: ndarray,
ik_method: str = "levenberg_marquardt",
ik_damping: float = 0.1,
ik_iterations: int = 20,
seed_q: ndarray | None = None,
point_offset: ndarray | None = None,
approach_axis: int | None = None,
approach_dir: ndarray = DOWN,
orientation_weight: float = DEFAULT_ORIENTATION_WEIGHT,
) -> np.ndarray
Solve for the joint angles that place the end-effector at a Cartesian target.
The returned angles are the command for the arm's <position> actuators.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
MjModel
|
MuJoCo model |
required |
data
|
MjData
|
MuJoCo data |
required |
site_id
|
int
|
Site ID for end-effector |
required |
dof_ids
|
ndarray
|
DOF IDs for controlled joints |
required |
target_pos
|
ndarray
|
Desired Cartesian position (3,) |
required |
ik_method
|
str
|
"levenberg_marquardt" or "pseudoinverse" |
'levenberg_marquardt'
|
ik_damping
|
float
|
Damping for LM (if used) |
0.1
|
ik_iterations
|
int
|
Max IK iterations |
20
|
seed_q
|
ndarray | None
|
Joint angles the iteration starts from. Defaults to the
current |
None
|
point_offset
|
ndarray | None
|
Site-frame offset of the controlled point (3,) |
None
|
approach_axis
|
int | None
|
Site-frame axis index to point along |
None
|
approach_dir
|
ndarray
|
Unit world direction for the approach axis (3,) |
DOWN
|
orientation_weight
|
float
|
Metres of position error per radian of tilt |
DEFAULT_ORIENTATION_WEIGHT
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Target joint angles (len(dof_ids),) |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in packages/lerobot_env_so101/lerobot_env_so101/ik_control.py
250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 | |
lerobot_env_so101.ik_control.compute_ik_levenberg_marquardt ¶
compute_ik_levenberg_marquardt(
model: MjModel,
data: MjData,
site_id: int,
dof_ids: ndarray,
target_pos: ndarray,
current_q: ndarray,
damping: float = 0.1,
max_iterations: int = 20,
tolerance: float = 0.0001,
point_offset: ndarray | None = None,
approach_axis: int | None = None,
approach_dir: ndarray = DOWN,
orientation_weight: float = DEFAULT_ORIENTATION_WEIGHT,
) -> np.ndarray
Compute target joint angles using Levenberg-Marquardt IK.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
MjModel
|
MuJoCo model |
required |
data
|
MjData
|
MuJoCo data |
required |
site_id
|
int
|
Site ID for end-effector |
required |
dof_ids
|
ndarray
|
DOF IDs for controlled joints |
required |
target_pos
|
ndarray
|
Desired Cartesian position (3,) |
required |
current_q
|
ndarray
|
Current joint angles (n,) |
required |
damping
|
float
|
Damping factor (lambda in LM algorithm) |
0.1
|
max_iterations
|
int
|
Maximum IK iterations |
20
|
tolerance
|
float
|
Convergence tolerance |
0.0001
|
point_offset
|
ndarray | None
|
Offset of the controlled point from the site origin, in
the site frame (3,). |
None
|
approach_axis
|
int | None
|
Index (0, 1, 2) of the site-frame axis to point along
|
None
|
approach_dir
|
ndarray
|
Unit world direction for the approach axis (3,) |
DOWN
|
orientation_weight
|
float
|
Metres of position error equivalent to one radian of approach-axis tilt |
DEFAULT_ORIENTATION_WEIGHT
|
Returns:
| Type | Description |
|---|---|
ndarray
|
Target joint angles (n,) |
Source code in packages/lerobot_env_so101/lerobot_env_so101/ik_control.py
128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 | |
lerobot_env_so101.ik_control.compute_ik_pseudoinverse ¶
compute_ik_pseudoinverse(
model: MjModel,
data: MjData,
site_id: int,
dof_ids: ndarray,
target_pos: ndarray,
current_q: ndarray,
max_iterations: int = 20,
tolerance: float = 0.0001,
step_size: float = 1.0,
) -> np.ndarray
Compute target joint angles using pseudoinverse (Gauss-Newton) IK.
Faster but less robust than Levenberg-Marquardt.