PR2: A Physics- and Photo-realistic Humanoid Testbed with Pilot Study in Competition

Hangxin Liu1, Qi Xie1, Zeyu Zhang1, Tao Yuan1, Xiaokun Leng2, Lining Sun3,4,
Song-Chun Zhu1,5,6, Jingwen Zhang1✉️ , Zhicheng He2✉️ , Yao Su1✉️ ,
1State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), 2Department of Computer Science, Harbin Institute of Technology, 3School of Mechatronics Engineering, Harbin Institute of Technology, 4Jiangsu Provincial Key Laboratory of Advanced Robotics, School of Mechanical and Electric Engineering, Soochow University, 5 Institute for Artificial Intelligence and School of Artificial Intelligence, Peking University,
6Department of Automation, Tsinghua University

✉️ indicates corresponding author

arXiv Code Video

Abstract

This paper presents the development of a Physics-realistic and Photo-realistic humanoid robot testbed, PR2, to facilitate collaborative research between Embodied Artificial Intelligence (Embodied AI) and robotics. PR2 offers high-quality scene rendering and robot dynamic simulation, enabling (i) the creation of diverse scenes using various digital assets, (ii) the integration of advanced perception or foundation models, and (iii) the implementation of planning and control algorithms for dynamic humanoid robot behaviors based on environmental feedback. The beta version of PR2 has been deployed for the simulation track of a nationwide full-size humanoid robot competition for college students, attracting 137 teams and over 400 participants within four months. This competition covered traditional tasks in bipedal walking, as well as novel challenges in loco-manipulation and language-instruction-based object search, marking a first for public college robotics competitions. A retrospective analysis of the competition suggests that future events should emphasize the integration of locomotion with manipulation and perception. By making the PR2 testbed publicly, we aim to further advance education and training in humanoid robotics.

Competition Trials

To comprehensively address humanoid locomotion, manipulation, and the integration of AI techniques while maintaining a moderate level of difficulty, we designed six distinct tasks; see Tab. for a detailed description.



...
, and indicate score assignments related to humanoid robot’s locomotion, manipulation, and perception, respectively.
Try It Out

System Architecture

PR2 is built on top of NVIDIA Isaac Sim , which leverages advanced GPU-enabled graphics and physics simulation with Nvidia PhysX 5, enabling users to investigate and test robotic skills in a physics-realistic and photo-realistic simulation environment. By extending the core components of NVIDIA Isaac Sim, PR2 consists of three main modules : (i) The task module and (ii) the physics simulation module support the import of various digital assets to create diverse task environments and simulate physical effects such as forces and system dynamics. Additionally, (iii) built-in controller module includes model predictive and whole-body controllers and high-level planners are provided for beginners but can be replaced by more advanced users as needed

Acknowledgments

We thank the engineering team from Leju Robotics for technical support, Ms. Zhen Chen from BIGAI for refining the figures, and all colleagues from the BIGAI TongVerse project for fruitful discussions and help on simulation developments.