挑战性环境下基于双尺度CBAM的毫米波雷达与视觉特征融合目标检测

    Object Detection in Challenging Environments Using mmWave Radar and Vision Feature Fusion via Dual-scale CBAM

    • 摘要: 针对恶劣天气和低光照对基于深度学习的视觉目标检测算法带来的挑战,提出一种基于双尺度卷积注意力模块(dual-scale convolutional block attention module,DSCBAM)的双模态目标检测算法,旨在通过视觉与毫米波雷达数据的特征融合,提高目标检测算法在挑战性环境下的鲁棒性和准确性。该算法采用双分支的一阶段检测结构,图像分支采用预训练的CSPDarkNet53骨干网络提取图像特征,雷达分支采用基于体素的雷达特征生成网络提取雷达特征。然后,分别在颈部网络前后利用提出的基于DSCBAM的特征融合模块进行雷达-视觉特征融合。最后,使用解耦检测头实现目标的分类和定位。在nuScenes数据集上,对比实验和消融实验验证了该融合检测算法在挑战性环境下的有效性和优越性。

       

      Abstract: A dual-modality object detection algorithm, based on the dual-scale convolutional block attention module (DSCBAM), is addressed to tackle challenges posed by adverse weather conditions and low lighting for visual object detection algorithms based on deep learning.The algorithm aims to improve the robustness and accuracy of object detection in challenging environments by fusing features from vision and millimeter-wave radar.It utilizes a dual-branch one-stage architecture, with the image branch using a pre-trained CSPDarkNet53 backbone network to extract image features and the radar branch employing a voxel-based radar feature generation network to extract radar features.The proposed DSCBAM feature fusion module integrates radar and visual features before and after the neck network.Finally, a decoupled detection head is deployed to classify and locate objects.The effectiveness and superiority of the proposed fusion detection algorithm were validated by comparative and ablation experiments conducted on the nuScenes dataset in challenging environments.

       

    /

    返回文章
    返回