Implicit Geometric-Semantic Fusion for Task-Oriented 6-DoF Grasp Detection via Visual Affordance and Stability Learning

The nature of design-based robotic tasks often results in the mechanical stability of robotic hands in one of two ways: hands either lose or gain their stability in regards to mechanical structure. There have been a number of methodologies created to tackle the issue of interpreting and generating stable robot grasps and affordances. However, tasks related to interpreting and manipulating objects in 3D space are often treated in isolation, which hinders the creation of task-specific robotic manipulation systems. In this paper, we present Implicit Geometric–Semantic Fusion (IGSF), a framework for stable grasp generation in 6-DoF from partial 3D point cloud representations. The IGSF architecture integrates the grasp generation of control points, backward learning of the spatial field and affordances, and the task-based control of the system. The grasp generation module of the framework makes predictions of the placement, orientation (as defined by unit quaternions), and stability without the need for utilizing any fixed grasp anchors. The affordance module uses an attention-augmented Dynamic Graph CNN to learn task-specific functional areas, while the refinement stage adjusts stable grasp candidates to be within the semantically relevant regions and within the constraints of kinematic feasibility. The architecture has been trained using ACRONYM’s characterized and validated grasp data alongside the 3D AffordanceNet’s subset fine-grained affordance annotations, utilizing a balanced multi-objective optimization technique. The experimental assessment of 50 distinct objects and 5 different tasks found an outcome of 100\% feasibility, a 95.1\% success for grasping, a 60\% error reduction with regards to pose (0.018 vs task-agnostic baselines 0.045 m2 with regards to pose error, p < 0.001), and semantic-type alignment. These findings illustrate that integrating geometric stability and semantic affordance within a constrained learning paradigm allows effective 3D task-oriented robotic manipulation systems.