Isaac Gym for Unitree G1

This blog guides through using Issac Gym for training reinforcement learning policies for humanoid robots using Unitree G1.

Repositories

My workspace structure is like this,

  |
  |-isaacgym
  |-rsl_rl
  |-unitree_rl_gym
  |-unitree_g1_doors

Installation

1. Install the base dependencies in a conda environment

     conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=12.1 -c pytorch -c nvidia
   

2. Go to each of the four repositories and run

   pip install -e .
   

   Before installing the rsl_rl library, run git checkout v1.0.2 for gym compatibility. Also note that isaacgym has to be installed by going in the isaacgym/python folder.

You can read more of specific repositories below or skip.

Understand the packages

Isaac Gym

Isaac Gym is a deprecated Nvidia python package for robot simulation. It uses Nvidia’s PhysX physics engine. It is the precursor of Isaac Sim.

Documentation

Isaac Gym allows you to run complex simulations using 1-2 script. Issac Gym can be used for RL policy implementations. We will only be using PhysX physics engine here.

rsl_rl

This is a python package that wraps Isaac Gym for robot learning applications. It provides package structure instructions and OOP for declaring your own robots, environments, policies etc. for simulation. It further improves accessibility to training for legged robots.

The package has 2 core classes: Task and Cfg. The Task class has methods that can setup simulation, viewer, safely start and stop training and playback. Cfg is a configuration class used to describe lengthy configurations.

These classes are further specialized for legged robots with the LeggedRobot, LeggedRobotCfg and LeggedRobotCfgPPO classes. They are child classes of Task and Cfg respectively. These classes provide torque, position, velocity control utils for controlling legged robots. They use base linear and angular velocity to teach legged robots how to walk.

A collection of a robot instance, env_cfg instance, and train_cfg instance defines one complete training set. Such collections are “registered” with a common task_registry. You can call registered tasks from the task registry directly from the command line. You can use various combination of these three instance to change robots/environments/training etc.

unitree_rl_gym

This package implements the above functionality for Unitree robots for walking task. This is a wrapper around the rsl_rl package.

unitree_g1_doors

This package implements the above functionality for Unitree g1 robot for door opening task. For our project, this is a wrapper around the rsl_rl package.

Nomenclature

Isaac Gym

• Environment - One exclusive space. Usually, one environment is used with one instance of a robot. Each simulation can have multiple environments
• Actors - Actors are instances of assets (objects) in an environment.
• Domain Randomizations - Randomizing properties of actors for each environment in a simulation

  rsl_rl

• PPO - RL training policy commonly used in robotics.
• BC - behavior cloning policy used as teacher-student distillation model for policy training

Build your own task

Coming Soon…