Region-sensitive Scene Graph Generation Method

Aiming at that the granularity of the predicate feature extracted based on relation bounding box is relatively coarse, a region-sensitive scene graph generation method (RS-SGG) is proposed. The predicate feature extraction module divided the relationship bounding box into four regions and used the self-attention mechanism to suppress background regions that were irrelevant to relationship classification. The relationship feature decoder comprehensively employed the visual, semantic and the position features of object pairs for predicting the predicate relationships. Based on the publicly available visual genome (VG) dataset, RS-SGG was compared with some mainstream scene graph generation methods. The graph constraint recall and no graph constraint recall for three subtasks including scene graph detection, scene graph classification, and predicate classification were computed to evaluate the performance of the SGG models. Results show that graph constraint recall and no graph constraint of RS-SGG are better than that of the mainstream methods. In addition, the results of visualization experiments further demonstrate the effectiveness of the proposed method.