基于多重卷积和空谱注意力Transformer的双流高光谱图像分类网络
Dual-stream Hyperspectral Image Classification Network Based on Multiple Convolution and Spatial-spectral Attention Transformer
-
摘要: 针对现有的卷积神经网络(convolutional neural network, CNN)方法在高光谱图像分类过程中存在的空谱联合特征利用不充分, 对全局特征的关注度不足的问题, 设计了一种基于多重卷积和空谱注意力Transformer的双流高光谱图像分类网络, 通过CNN和Transformer相结合的双流结构, 实现局部和全局特征的充分利用。首先, 在CNN支路, 设计了一种基于多重卷积的空谱特征融合结构, 通过多重卷积实现空间和光谱维特征的充分挖掘和融合; 其次, 在Transformer网络支路则使用空谱注意力机制捕获整个图像的全局信息; 最后, 2条分支通过决策级融合实现了高性能的分类效果。基于4个典型数据集的测试结果表明, 该算法的分类结果与当前主流算法相比, 均有不同程度的提升。Abstract: To address the limitations of existing convolutional neural network (CNN) methods, which exhibit insufficient utilization of spatial-spectral joint features and insufficient attention to global features in the hyperspectral image classification process, a dual-stream hyperspectral image classification network based on multiple convolution and spatial-spectral attention Transformer is designed to achieve the full utilization of both local and global features through a dual-stream structure integrating CNN and Transformer network. First, in the CNN branch, a spatial-spectral feature fusion structure based on multiple convolutions was designed to fully mine and fuse spatial and spectral dimensional features; while in the Transformer network branch, the spatial-spectral attention mechanism was used to capture the global information of the whole image. Finally, two branches achieved high-performance classification results through decision-level fusion. The test results based on four typical datasets show that the classification results of this algorithm are improved to different degrees compared with the current mainstream algorithms.
下载: