Semi-supervised Video Target Segmentation Method Based on Attention Correction

FU Lihua; YANG Hanxue; ZHANG Bo; WANG Junxiang; WU Huixian; YAN Shaoxing

doi:10.11936/bjutxb2020110025

Journal of Beijing University of Technology > 2022 > 48(8): 822-829. > DOI: 10.11936/bjutxb2020110025

FU Lihua, YANG Hanxue, ZHANG Bo, WANG Junxiang, WU Huixian, YAN Shaoxing. Semi-supervised Video Target Segmentation Method Based on Attention Correction[J]. Journal of Beijing University of Technology, 2022, 48(8): 822-829. DOI: 10.11936/bjutxb2020110025

Citation:

PDF (3602 KB)

Semi-supervised Video Target Segmentation Method Based on Attention Correction

Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China

More Information

Received Date: November 18, 2020
Revised Date: February 01, 2021
Available Online: September 13, 2022

Graphical Abstract

Abstract

Abstract

To solve the problem that the existing semi-supervised video target segmentation methods cannot ensure segmentation accuracy and efficiency at the same time, an attention mechanism into the general semi-supervised video target segmentation method was introduced to modify segmentation results. First, an appearance feature extraction subnet was constructed to extract feature map of the first frame of video and it was used as appearance guidance information. Second, the segmentation result of the previous frame was obtained and used as position guidance information. Finally, a current frame feature extraction subnet was constructed, which combined position correction attention and appearance correction attention in a double branch structure, so as to integrate the position information and appearance information into the current frame feature map and accomplish the target segmentation. Experiments show that the target segmentation method can correct the propagation errors in video target segmentation and improve the segmentation accuracy.
- video object segmentation,
- attention mechanism,
- semantic information,
- channel attention,
- spatial attention,
- semi-supervised learning

FullText(HTML)

References (23)

References

[1]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137-1149. https://arxiv.org/abs/1506.01497
[2]	付利华, 赵宇, 孙晓威, 等. 基于孪生网络的快速视频目标分割[J]. 电子学报, 2020, 48(4): 625-630. doi: 10.3969/j.issn.0372-2112.2020.04.001 FU L H, ZHAO Y, SUN X W, et al. Fast video object segmentation based on siamese networks[J]. ACTA Electronica Sinica, 2020, 48(4): 625-630. (in Chinese) doi: 10.3969/j.issn.0372-2112.2020.04.001
[3]	张琳, 陆耀, 卢丽华, 等. 一种改进的视频分割网络及其全局信息优化方法[J]. 自动化学报, 2022, 48(3): 787-796. https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO202203011.htm ZHANG L, LU Y, LU L H, et al. An improved video segmentation network and its global information optimization method[J]. ACTA Automatica Sinica, 2022, 48(3): 787-796. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-MOTO202203011.htm
[4]	CAELLES S, MANINIS K K, PONT T J, et al. One-shot video object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 221-230.
[5]	HU Y T, HUANG J B, SCHWING A G. Videomatch: matching based video object segmentation[C]//Proceedings of European Conference on Computer Vision. Berlin: Springer, 2018: 54-70.
[6]	VOIGTLAENDER P, CHAI Y, SCHROFF F, et al. Feelvos: fast end-to-end embedding learning for video object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 9481-9490.
[7]	VOIGTLAENDER P, LEIBE B. Online adaptation of convolutional neural networks for video object segmentation[EB/OL]. [2019-07-23]. https://arxiv.org/pdf/1706.09364.pdf.
[8]	LUITEN J, VOIGTLAENDER P, LEIBE B. PReMVOS: proposal-generation, refinement and merging for video object segmentation[C]//Proceedings of Asian Conference on Computer Vision. Berlin: Springer, 2018: 565-580.
[9]	LI X, CHANGE L C. Video object segmentation with joint re-identification and attention-aware mask propagation[C]//Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 90-105.
[10]	PERAZZI F, KHOREVA A, BENENSON R, et al. Learning video object segmentation from static images[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2663-2672.
[11]	WUG O S, LEE J Y, SUNKAVALLI K, et al. Fast video object segmentation by reference-guided mask propagation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7376-7385.
[12]	JAMPANI V, GADDE R, GEHLER P V. Video propagation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 451-461.
[13]	CHENG J, TSAI Y H, HUNG W C, et al. Fast and accurate online video object segmentation via tracking parts[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7415-7424.
[14]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[15]	XU N, YANG L, FAN Y, et al. Youtube-VOS: sequence-to-sequence video object segmentation[C]//Proceedings of European Conference on Computer Vision. Berlin: Springer, 2018: 585-601.
[16]	PERAZZI F, PONT T J, MCWILLIAMS B, et al. A benchmark dataset and evaluation methodology for video object segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 724-732.
[17]	PONT T, PERAZZI F, CAELLES S, et al. The 2017 DAVIS challenge on video object segmentation[EB/OL]. [2019-09-12]. https://arxiv.org/pdf/1803.00557.pdf.
[18]	XIE S, TU Z. Holistically-nested edge detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1395-1403.
[19]	BERMAN M, RANNEN T A, BLASCHKO M B. The Lovász-Softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4413-4421.
[20]	MARKI N, PERAZZI F, WANG O, et al. Bilateral space video segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 743-751.
[21]	PERAZZI F, WANG O, GROSS M, et al. Fully connected object proposals for video segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 3227-3234.
[22]	YANG L, WANG Y, XIONG X, et al. Efficient video object segmentation via network modulation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 6499-6507.
[23]	SHIN Y J, RAMEAU F, KIM J, et al. Pixel-level matching for video object segmentation using convolutional neural networks[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2167-2176.

[1]	WANG Shufen, BI Yanqi, FANG Bonan, ZHANG Baohui, FANG Jingxuan. Significant Progress of Research on Green Space for People's Stress Relief[J]. Journal of Beijing University of Technology, 2023, 49(9): 1025-1038. DOI: 10.11936/bjutxb2022100022
[2]	LIU Danmin, SHI Feng. Research Progress on Doped and Composite Rare Earth Hexaboride[J]. Journal of Beijing University of Technology, 2022, 48(8): 869-877. DOI: 10.11936/bjutxb2021040023
[3]	LIU Danmin, SHI Zhan, ZHANG Guoqing, ZHANG Yongzhe. Doping Methods and Their Influences on Two-dimensional Photoelectric Device Materials[J]. Journal of Beijing University of Technology, 2022, 48(4): 430-442. DOI: 10.11936/bjutxb2020090011
[4]	CAO Yi, WEI Shou-yu, LIU Jiang, WANG Pu. Research Progress of High-power Yb-doped Superfluorescent Fiber Sources[J]. Journal of Beijing University of Technology, 2015, 41(12): 1789-1798. DOI: 10.11936/bjutxb2015070082
[5]	WANG Jin-shu, XING Peng-fei, LI Li-li, ZHOU Mei-ling. Synthesis and Characterization of Nano-sized Nitrogen Doped TiO₂ by Mechanochemical Method[J]. Journal of Beijing University of Technology, 2006, 32(7): 633-637. DOI: 10.3969/j.issn.0254-0037.2006.07.012
[6]	QIU Wen-ge, CHANG Xi-liang. Development of Polymeric Corrosion Inhibitors[J]. Journal of Beijing University of Technology, 2003, 29(2): 210-214. DOI: 10.3969/j.issn.0254-0037.2003.02.022
[7]	CHEN Zhi-lin, WANG Qun, ZHANG Xue-lian, ZUO Tie-yong, FU Feng, YE Ke-lin. Recent Development of Research on Wood / Inorganic Nonmetallic Composites[J]. Journal of Beijing University of Technology, 2003, 29(1): 116-121. DOI: 10.3969/j.issn.0254-0037.2003.01.027
[8]	Rong Jian, Liu Xiaoming, Ren Futian. Development of Highway Capacity Study[J]. Journal of Beijing University of Technology, 1998, 24(2): 121-129.
[9]	She Yuanbin, Yang Jinzong. Research Progress of Phthalocyanine-like Catalysts[J]. Journal of Beijing University of Technology, 1998, 24(2): 115-120.
[10]	She Yuanbin, Shi Fei. Study on a New Type of Styrene-Acrylate Emulsion Polymerization[J]. Journal of Beijing University of Technology, 1996, 22(3): 7-12.

Cited By

Cited by

Periodical cited type(2)

1.	李洁，杨济世，刘虎林，刘碧野，赵源，胡文波，吴胜利. 分离打拿极电子倍增器性能提升技术研究. 真空电子技术. 2023(01): 18-24 .
2.	邓晨晖，韩立，王岩，高召顺，牛耕. 二次电子产额影响因素的研究进展. 材料导报. 2023(24): 18-27 .

Other cited types(1)

Get Citation

PDF

XML

Article views PDF downloads Cited by(3)

Turn off MathJax

Article Contents

Abstract

References

Semi-supervised Video Target Segmentation Method Based on Attention Correction