Learning to detect visual grasp affordance

Hyun Oh Song, Mario Fritz, Daniel Goehring, Trevor Darrell

Abstract

Appearance-based estimation of grasp affordances is desirable when 3-D scans become unreliable due to clutter or material properties. We develop a general framework for estimating grasp affordances from 2-D sources, including local texture-like measures as well as object-category measures that capture previously learned grasp strategies. Local approaches to estimating grasp positions have been shown to be effective in real-world scenarios, but are unable to impart object-level biases and can be prone to false positives. We describe how global cues can be used to compute continuous pose estimates and corresponding grasp point locations, using a max-margin optimization for category-level continuous pose regression. We provide a novel dataset to evaluate visual grasp affordance estimation; on this dataset we show that a fused method outperforms either local or global methods alone, and that continuous pose estimation improves over discrete output models. Finally, we demonstrate our autonomous object detection and grasping system on the Willow Garage PR2 robot.


Demo Video



References

Learning to detect visual grasp affordance
Hyun Oh Song, Mario Fritz, Daniel Goehring, Trevor Darrell
IEEE Transactions on Automation Science and Engineering (TASE), 2015
paper

Visual Grasp Affordances From Appearance-Based Cues
Hyun Oh Song, Mario Fritz, Chunhui Gu, Trevor Darrell
ICCV Workshop on Challenges and Opportunities in Robot Perception, 2011
paper / bibtex