基于深度学习的水下生物目标检测方法综述

doi:10.3976/j.issn.1002-4026.2023.06.001

摘要/Abstract

摘要：

水下生物目标识别对水产养殖、濒危生物保护、生态环境监测具有重要意义。综合分析了当前各种深度学习方法在水下生物目标检测中的应用情况。首先介绍了常用的水下生物目标检测数据集;然后,按照两阶段和单阶段对当前常用目标检测方法进行分类、分析和总结,详细阐述了各类检测方法的实际应用状况,并重点对上述各类检测方法优化策略的优势与不足进行了分析和总结;最后,对基于深度学习的水下生物目标检测提出今后的研究重点,为该领域的研究人员提供了资料性的参考依据。

关键词: 深度学习, 目标检测, 水下生物目标检测

Abstract:

Underwater biological object detection is crucial for aquaculture, endangered species protection,and ecological environment monitoring. This study comprehensively analyzes the applications of various deep learning methods in underwater biological object detection. The commonly used underwater biological object detection datasets are introduced. The state-of-the-art underwater biological object detection methods are classified, analyzed, and summarized by two stages and one stage. The actual applications of various detection methods are thoroughly described, and the advantages and disadvantages of their optimization strategies are analyzed and summarized. Future works in the field of underwater biological object detection based on deep learning are presented. This study provides a reference basis for researchers in the field of underwater biological object detection.

Key words: deep learning, object detection, underwater biological object detection

中图分类号:

TP181

于雨, 郭保琪, 初士博, 李恒, 杨鹏儒. 基于深度学习的水下生物目标检测方法综述[J]. 山东科学, 2023, 36(6): 1-7.

YU Yu, GUO Baoqi, CHU Shibo, LI Heng, YANG Pengru. Survey of underwater biological object detection methods based on deep learning[J]. Shandong Science, 2023, 36(6): 1-7.

图/表 5

表1

图1

表2

表3

表4

参考文献 36

[1]	林森, 赵颍. 水下光学图像中目标探测关键技术研究综述[J]. 激光与光电子学进展, 2020, 57(6): 26-37. DOI: 10.3788/LOP57.060002.
[2]	FAYAZ S, PARAH S A, QURESHI G J. Underwater object detection: architectures and algorithms-a comprehensive review[J]. Multimedia Tools and Applications, 2022, 81(15): 20871-20916. DOI: 10.1007/s11042-022-12502-1.
[3]	檀盼龙, 吴小兵, 张晓宇. 基于声呐图像的水下目标识别研究综述[J]. 数字海洋与水下攻防, 2022, 5(4): 342-353. DOI: 10.19838/j.issn.2096-5753.2022.04.010.
[4]	于红. 水产动物目标探测与追踪技术及应用研究进展[J]. 大连海洋大学学报, 2020, 35(6): 793-804. DOI: 10.16535/j.cnki.dlhyxb.2020-263.
[5]	董金耐, 杨淼, 谢卓冉, 等. 水下图像目标检测数据集及检测算法综述[J]. 海洋技术学报, 2022, 41(5): 60-72. DOI: 10.3969/j.issn.1003-2029.2022.05.007.
[6]	FISHER R B, CHEN-BURGERY H, GIORDANO D, et al. Fish4Knowledge: collecting and analyzing massive coral reef fish video data[M]. Berlin, Germany: Springer. DOI:10.10071978-3-319-3208-9.
[7]	PEDERSEN M, HAURUM J B, GADE R, et al. Detection of marine animals in a new underwater dataset with varying visibility[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, USA: IEEE, 2019:18-26.
[8]	Kaggle-NCFM[DB/OL].[2023-01-02]. https://www.kaggle.com/c/the-natureconservancy-fisheries-monitoring.
[9]	FishCLEF2015[DB/OL].[2023-01-02]. https://www.imageclef.org/lifeclef/2015/fish.
[10]	GARCIA-D'URSO N, GALAN-CUENCA A, PÉREZ-SÁNCHEZ P, et al. TheDeepFish computer vision dataset for fish instance segmentation, classification, and size estimation[J]. Scientific Data, 2022, 9(1): 1-7. DOI: 10.1038/s41597-022-01416-0.
[11]	URPC[DB/OL].[2023-01-02]. http://www.urpc.org.cn/index.html.
[12]	WANG Z H, LIU C W, WANG S J, et al. UDD: an underwater open-sea farm object detection dataset for underwater robotpicking[EB/OL]. [2023-01-03]. https://arxiv.org/abs/2003.01446.
[13]	LIU C W, LI H J, WANG S C, et al. A dataset and benchmark of underwater object detection for robot picking[C]// 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). Shenzhen, China: IEEE, 2021: 1-6. DOI: 10.1109/ICMEW53276.2021.9455997.
[14]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 580-587. DOI: 10.1109/CVPR.2014.81.
[15]	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2016: 1440-1448.DOI: 10.1109/ICCV.2015.169.
[16]	REN S Q, HE K M, GIRSHICKR, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI: 10.1109/TPAMI.2016.2577031. pmid: 27295650
[17]	袁红春, 张硕. 基于Faster R-CNN和图像增强的水下鱼类目标检测方法[J]. 大连海洋大学学报, 2020, 35(4): 612-619. DOI: 10.16535/j.cnki.dlhyxb.2019-146.
[18]	LIU J, LIU S, XU S J, et al. Two-stage underwater object detection network usingswin transformer[J]. IEEE Access, 2022, 10: 117235-117247. DOI: 10.1109/ACCESS.2022.3219592.
[19]	LIN W H, ZHONG J X, LIU S, et al. ROIMIX: proposal-fusion among multiple images for underwater object detection[C]// ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona, Spain:IEEE, 2020: 2588-2592. DOI:10.1109/ICASSP40776.2020.9053829.
[20]	SHI P F, XU X W, NI JJ, et al. Underwater biological detection algorithm based on improved faster-RCNN[J]. Water, 2021, 13(17): 2420. DOI: 10.3390/w13172420.
[21]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 779-788. DOI: 10.1109/CVPR.2016.91.
[22]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA. IEEE, 2017: 6517-6525. DOI: 10.1109/CVPR.2017.690.
[23]	REDMON J, FARHADI A. YOLOv3: An incrementalimprovement[EB/OL].[2023-01-03]. https://arxiv.org/abs/1804.02767.pdf.
[24]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of objectdetection[EB/OL].[2023-01-03]. https://arxiv.org/abs/2004.10934.pdf.
[25]	WANG M F, LIU M Y, ZHANG F H, et al. Fast classification and detection of fish images with YOLOv2[C]// 2018 OCEANS - MTS/IEEE Kobe Techno-Oceans (OTO). Kobe, Japan: IEEE, 2018: 1-4. DOI: 10.1109/OCEANSKOBE.2018.8559141.
[26]	AL MUKSIT A, HASAN F, HASAN BHUIYAN EMON M F, et al. YOLO-Fish: a robust fish detection model to detect fish in realistic underwater environment[J]. Ecological Informatics, 2022, 72: 101847. DOI: 10.1016/j.ecoinf.2022.101847.
[27]	CHEN L, LIU Z H, TONG L, et al. Underwater object detection using Invert Multi-ClassAdaboost with deep learning[C]// 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE, 2020: 1-8. DOI: 10.1109/IJCNN48605.2020.9207506.
[28]	ISA I S, ROSLI M S A, YUSOF U K, et al. Optimizing the hyperparameter tuning of YOLOv5 for underwater detection[J]. IEEE Access, 2022, 10: 52818-52831. DOI: 10.1109/ACCESS.2022.3174583.
[29]	ZHAI X Y, WEI H L, HE YY, et al. Underwater Sea cucumber identification based on improved YOLOv5[J]. Applied Sciences, 2022, 12(18): 9105. DOI: 10.3390/app12189105.
[30]	LEI F, TANG FF, LI S H. Underwater target detection algorithm based on improved YOLOv5[J]. Journal of Marine Science and Engineering, 2022, 10(3): 310. DOI: 10.3390/jmse10030310.
[31]	LI X, SHANG M, QIN H W, et al. Fast accurate fish detection and recognition of underwater images with Fast R-CNN[C]// OCEANS 2015-MTS/IEEE Washington. Washington, DC: IEEE, 2016: 1-5.
[32]	XU W W, MATZNER S. Underwater fish detection using deep learning for water power applications[C]// 2018 International Conference on Computational Science and Computational Intelligence (CSCI). Las Vegas, USA: IEEE, 2020: 313-318. DOI: 10.1109/CSCI46756.2018.00067.
[33]	刘萍, 杨鸿波, 宋阳. 改进YOLOv3网络的海洋生物识别算法[J]. 计算机应用研究, 2020, 37(S1): 394-397.
[34]	KNAUSGÅRD K M, WIKLUND A, SØRDALEN T K, et al. Temperate fish detection and classification: a deep learning based approach[J]. Applied Intelligence, 2022, 52(6): 6988-7001. DOI: 10.1007/s10489-020-02154-9.
[35]	LI A L, YU L, TIAN S W. Underwater biological detection based on YOLOv4 combined with channel attention[J]. Journal of Marine Science and Engineering, 2022, 10(4): 469. DOI:10.3390/jmse10040469.
[36]	张琳, 葛艳, 杜军威, 等. 改进FCOS网络的海洋鱼类目标检测[J]. 计算机系统应用, 2023, 32(3):309-315. DOI: 10.15888/j.cnki.csa.008965.

数据集	水下生物类别	数量/张	应用场景
Fish4Knowledge^[6]	鱼类	27 370	海洋生态研究
Brackish^[7]	鱼类、螃蟹、虾、水母和海星等	14 518	鱼类识别与分类
Kaggle-NCFM^[8]	吞拿鱼、鲨鱼等	3 777	捕捞船鱼类识别
FishCLEF2015^[9]	鱼类	20 000	鱼类识别与分类
DeepFish^[10]	鱼类	40 000	鱼类识别与分类
URPC^[11]	海参、海胆、扇贝、海星	3 701	水下机器人拾取
UDD^[12]	海参、海胆、扇贝、海星	2 227	水下机器人拾取
DUO^[13]	海参、海胆、扇贝、海星	7 782	水下机器人拾取

文献	改进方法	数据集	应用场景
[17]	二次迁移学习、图像增强	自定义数据集	鱼类识别
[18]	替换骨干网络、路径聚合网络、在线困难样本挖掘、感兴趣区域对齐	URPC2018	水下机器人拾取
[19]	数据增强	URPC2018、PASCAL VOC(07+12)	水下生物检测
[20]	替换骨干网络、双向特征金字塔、优化交并比函数、优化锚框生成	URPC2018	水下机器人拾取

文献	YOLO版本	改进方法	数据集	应用场景
[25]	v2	卷积层优化	Kaggle-NCFM	捕捞船鱼类识别
[26]	v3	上采样步长优化、特征金字塔	DeepFish、OzFish	鱼类识别与分类
[27]	v4	残差块、通道注意力机制、特征金字塔	Brackish	鱼类识别与分类
[28]	v5	模型参数优化	Brackish	鱼类识别与分类
[29]	v5	图像预处理、卷积注意力模块	自定义数据集	水下海参的识别
[30]	v5	替换骨干网络、改进多分辨率特征融合方法、改进置信损失函数	自定义数据集	水下机器人拾取

改进策略分类		相关文献	优势	不足
数据增强	像素级	[31]	提升水下拍摄图像的清晰度、对比度和质量	增大计算量,且仅适用于微调高清数据集训练出来的检测模型
	图像级	[25, 29, 30, 32⇓⇓⇓-36]	增加样本的多样性,操作简单	可能引入新的噪声干扰
	感兴趣区域级	[19]	将从多张图片中提取的感兴趣区域进行融合,以更好地模拟水下生物的重叠和遮挡	操作复杂,仅适用于具备感兴趣区域结构的模型
框架结构改进	替换骨干网络	[18, 20, 30, 36]	为模型带来更高的精度或效率	预训练模型不可复用
框架结构改进	多尺度特征融合	[18⇓-20, 26, 30, 35]	提高目标检测的准确率	增大计算量,效果提升不显著
样本平衡	在线困难样本挖掘	[18]	平衡正负样本数量	忽略了容易分类的样本,且执行效率低
样本平衡	样本重加权	[27]	平衡难易样本数量	需要对损失函数进行修改,通用性差
锚框优化	K-means++聚类	[20]	根据数据集生成更合适的锚点	尚未发现
	使用GIoU	[36]	解决预测框与真实框无法重叠造成的梯度消失问题	预测框与真实框相交时,在水平和垂直方向上收敛慢
	使用EIoU	[20]	将纵横比的损失项拆分成预测的宽高分别与最小外接框宽高的差值	尚未发现