We first introduce scene graphs to promote instruction driven robotic task planning via the graphs’ ability to understand wide-perspective and rich-semantic knowledge in the environment.
We design a novel GAT-based network named GRID,which takes instruction,robot graph & scene graph as inputs and outperforms GPT-4 by over 43.6% in task accuracy.
We build a synthetic dataset construction pipeline to generate datasets of scene graphs for instruction-driven robotic task planning.
GRID network is lightweight and can be deployed locally, without relying on large cloud models.
Thank you to the following units for their support and assistance.
.@article{ni2024grid,
title={GRID: Scene-Graph-based Instruction-driven Robotic Task Planning},
author={Zhe Ni and Xiaoxin Deng and Cong Tai and Xinyue Zhu and Qinghongbing Xie and Weihang Huang and Xiang Wu and Long Zeng},
journal={arXiv preprint arXiv:2309.07726},
year={2024}}