基于深度学习的水下生物目标检测方法综述

doi:10.3976/j.issn.1002-4026.2023.06.001

Abstract

Abstract:

Underwater biological object detection is crucial for aquaculture, endangered species protection,and ecological environment monitoring. This study comprehensively analyzes the applications of various deep learning methods in underwater biological object detection. The commonly used underwater biological object detection datasets are introduced. The state-of-the-art underwater biological object detection methods are classified, analyzed, and summarized by two stages and one stage. The actual applications of various detection methods are thoroughly described, and the advantages and disadvantages of their optimization strategies are analyzed and summarized. Future works in the field of underwater biological object detection based on deep learning are presented. This study provides a reference basis for researchers in the field of underwater biological object detection.

Key words: deep learning, object detection, underwater biological object detection

CLC Number:

TP181

YU Yu, GUO Baoqi, CHU Shibo, LI Heng, YANG Pengru. Survey of underwater biological object detection methods based on deep learning[J].Shandong Science, 2023, 36(6): 1-7.

Figures/Tables 5

Table 1

Fig.1

Table 2

Table 3

Table 4

References 36

[1]	林森, 赵颍. 水下光学图像中目标探测关键技术研究综述[J]. 激光与光电子学进展, 2020, 57(6): 26-37. DOI: 10.3788/LOP57.060002.
[2]	FAYAZ S, PARAH S A, QURESHI G J. Underwater object detection: architectures and algorithms-a comprehensive review[J]. Multimedia Tools and Applications, 2022, 81(15): 20871-20916. DOI: 10.1007/s11042-022-12502-1.
[3]	檀盼龙, 吴小兵, 张晓宇. 基于声呐图像的水下目标识别研究综述[J]. 数字海洋与水下攻防, 2022, 5(4): 342-353. DOI: 10.19838/j.issn.2096-5753.2022.04.010.
[4]	于红. 水产动物目标探测与追踪技术及应用研究进展[J]. 大连海洋大学学报, 2020, 35(6): 793-804. DOI: 10.16535/j.cnki.dlhyxb.2020-263.
[5]	董金耐, 杨淼, 谢卓冉, 等. 水下图像目标检测数据集及检测算法综述[J]. 海洋技术学报, 2022, 41(5): 60-72. DOI: 10.3969/j.issn.1003-2029.2022.05.007.
[6]	FISHER R B, CHEN-BURGERY H, GIORDANO D, et al. Fish4Knowledge: collecting and analyzing massive coral reef fish video data[M]. Berlin, Germany: Springer. DOI:10.10071978-3-319-3208-9.
[7]	PEDERSEN M, HAURUM J B, GADE R, et al. Detection of marine animals in a new underwater dataset with varying visibility[C]// 2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, USA: IEEE, 2019:18-26.
[8]	Kaggle-NCFM[DB/OL].[2023-01-02]. https://www.kaggle.com/c/the-natureconservancy-fisheries-monitoring.
[9]	FishCLEF2015[DB/OL].[2023-01-02]. https://www.imageclef.org/lifeclef/2015/fish.
[10]	GARCIA-D'URSO N, GALAN-CUENCA A, PÉREZ-SÁNCHEZ P, et al. TheDeepFish computer vision dataset for fish instance segmentation, classification, and size estimation[J]. Scientific Data, 2022, 9(1): 1-7. DOI: 10.1038/s41597-022-01416-0.
[11]	URPC[DB/OL].[2023-01-02]. http://www.urpc.org.cn/index.html.
[12]	WANG Z H, LIU C W, WANG S J, et al. UDD: an underwater open-sea farm object detection dataset for underwater robotpicking[EB/OL]. [2023-01-03]. https://arxiv.org/abs/2003.01446.
[13]	LIU C W, LI H J, WANG S C, et al. A dataset and benchmark of underwater object detection for robot picking[C]// 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). Shenzhen, China: IEEE, 2021: 1-6. DOI: 10.1109/ICMEW53276.2021.9455997.
[14]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 580-587. DOI: 10.1109/CVPR.2014.81.
[15]	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2016: 1440-1448.DOI: 10.1109/ICCV.2015.169.
[16]	REN S Q, HE K M, GIRSHICKR, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI: 10.1109/TPAMI.2016.2577031. pmid: 27295650
[17]	袁红春, 张硕. 基于Faster R-CNN和图像增强的水下鱼类目标检测方法[J]. 大连海洋大学学报, 2020, 35(4): 612-619. DOI: 10.16535/j.cnki.dlhyxb.2019-146.
[18]	LIU J, LIU S, XU S J, et al. Two-stage underwater object detection network usingswin transformer[J]. IEEE Access, 2022, 10: 117235-117247. DOI: 10.1109/ACCESS.2022.3219592.
[19]	LIN W H, ZHONG J X, LIU S, et al. ROIMIX: proposal-fusion among multiple images for underwater object detection[C]// ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Barcelona, Spain:IEEE, 2020: 2588-2592. DOI:10.1109/ICASSP40776.2020.9053829.
[20]	SHI P F, XU X W, NI JJ, et al. Underwater biological detection algorithm based on improved faster-RCNN[J]. Water, 2021, 13(17): 2420. DOI: 10.3390/w13172420.
[21]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 779-788. DOI: 10.1109/CVPR.2016.91.
[22]	REDMON J, FARHADI A. YOLO9000: better, faster, stronger[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA. IEEE, 2017: 6517-6525. DOI: 10.1109/CVPR.2017.690.
[23]	REDMON J, FARHADI A. YOLOv3: An incrementalimprovement[EB/OL].[2023-01-03]. https://arxiv.org/abs/1804.02767.pdf.
[24]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of objectdetection[EB/OL].[2023-01-03]. https://arxiv.org/abs/2004.10934.pdf.
[25]	WANG M F, LIU M Y, ZHANG F H, et al. Fast classification and detection of fish images with YOLOv2[C]// 2018 OCEANS - MTS/IEEE Kobe Techno-Oceans (OTO). Kobe, Japan: IEEE, 2018: 1-4. DOI: 10.1109/OCEANSKOBE.2018.8559141.
[26]	AL MUKSIT A, HASAN F, HASAN BHUIYAN EMON M F, et al. YOLO-Fish: a robust fish detection model to detect fish in realistic underwater environment[J]. Ecological Informatics, 2022, 72: 101847. DOI: 10.1016/j.ecoinf.2022.101847.
[27]	CHEN L, LIU Z H, TONG L, et al. Underwater object detection using Invert Multi-ClassAdaboost with deep learning[C]// 2020 International Joint Conference on Neural Networks (IJCNN). Glasgow, UK: IEEE, 2020: 1-8. DOI: 10.1109/IJCNN48605.2020.9207506.
[28]	ISA I S, ROSLI M S A, YUSOF U K, et al. Optimizing the hyperparameter tuning of YOLOv5 for underwater detection[J]. IEEE Access, 2022, 10: 52818-52831. DOI: 10.1109/ACCESS.2022.3174583.
[29]	ZHAI X Y, WEI H L, HE YY, et al. Underwater Sea cucumber identification based on improved YOLOv5[J]. Applied Sciences, 2022, 12(18): 9105. DOI: 10.3390/app12189105.
[30]	LEI F, TANG FF, LI S H. Underwater target detection algorithm based on improved YOLOv5[J]. Journal of Marine Science and Engineering, 2022, 10(3): 310. DOI: 10.3390/jmse10030310.
[31]	LI X, SHANG M, QIN H W, et al. Fast accurate fish detection and recognition of underwater images with Fast R-CNN[C]// OCEANS 2015-MTS/IEEE Washington. Washington, DC: IEEE, 2016: 1-5.
[32]	XU W W, MATZNER S. Underwater fish detection using deep learning for water power applications[C]// 2018 International Conference on Computational Science and Computational Intelligence (CSCI). Las Vegas, USA: IEEE, 2020: 313-318. DOI: 10.1109/CSCI46756.2018.00067.
[33]	刘萍, 杨鸿波, 宋阳. 改进YOLOv3网络的海洋生物识别算法[J]. 计算机应用研究, 2020, 37(S1): 394-397.
[34]	KNAUSGÅRD K M, WIKLUND A, SØRDALEN T K, et al. Temperate fish detection and classification: a deep learning based approach[J]. Applied Intelligence, 2022, 52(6): 6988-7001. DOI: 10.1007/s10489-020-02154-9.
[35]	LI A L, YU L, TIAN S W. Underwater biological detection based on YOLOv4 combined with channel attention[J]. Journal of Marine Science and Engineering, 2022, 10(4): 469. DOI:10.3390/jmse10040469.
[36]	张琳, 葛艳, 杜军威, 等. 改进FCOS网络的海洋鱼类目标检测[J]. 计算机系统应用, 2023, 32(3):309-315. DOI: 10.15888/j.cnki.csa.008965.

数据集	水下生物类别	数量/张	应用场景
Fish4Knowledge^[6]	鱼类	27 370	海洋生态研究
Brackish^[7]	鱼类、螃蟹、虾、水母和海星等	14 518	鱼类识别与分类
Kaggle-NCFM^[8]	吞拿鱼、鲨鱼等	3 777	捕捞船鱼类识别
FishCLEF2015^[9]	鱼类	20 000	鱼类识别与分类
DeepFish^[10]	鱼类	40 000	鱼类识别与分类
URPC^[11]	海参、海胆、扇贝、海星	3 701	水下机器人拾取
UDD^[12]	海参、海胆、扇贝、海星	2 227	水下机器人拾取
DUO^[13]	海参、海胆、扇贝、海星	7 782	水下机器人拾取

文献	改进方法	数据集	应用场景
[17]	二次迁移学习、图像增强	自定义数据集	鱼类识别
[18]	替换骨干网络、路径聚合网络、在线困难样本挖掘、感兴趣区域对齐	URPC2018	水下机器人拾取
[19]	数据增强	URPC2018、PASCAL VOC(07+12)	水下生物检测
[20]	替换骨干网络、双向特征金字塔、优化交并比函数、优化锚框生成	URPC2018	水下机器人拾取

文献	YOLO版本	改进方法	数据集	应用场景
[25]	v2	卷积层优化	Kaggle-NCFM	捕捞船鱼类识别
[26]	v3	上采样步长优化、特征金字塔	DeepFish、OzFish	鱼类识别与分类
[27]	v4	残差块、通道注意力机制、特征金字塔	Brackish	鱼类识别与分类
[28]	v5	模型参数优化	Brackish	鱼类识别与分类
[29]	v5	图像预处理、卷积注意力模块	自定义数据集	水下海参的识别
[30]	v5	替换骨干网络、改进多分辨率特征融合方法、改进置信损失函数	自定义数据集	水下机器人拾取

改进策略分类		相关文献	优势	不足
数据增强	像素级	[31]	提升水下拍摄图像的清晰度、对比度和质量	增大计算量,且仅适用于微调高清数据集训练出来的检测模型
	图像级	[25, 29, 30, 32⇓⇓⇓-36]	增加样本的多样性,操作简单	可能引入新的噪声干扰
	感兴趣区域级	[19]	将从多张图片中提取的感兴趣区域进行融合,以更好地模拟水下生物的重叠和遮挡	操作复杂,仅适用于具备感兴趣区域结构的模型
框架结构改进	替换骨干网络	[18, 20, 30, 36]	为模型带来更高的精度或效率	预训练模型不可复用
框架结构改进	多尺度特征融合	[18⇓-20, 26, 30, 35]	提高目标检测的准确率	增大计算量,效果提升不显著
样本平衡	在线困难样本挖掘	[18]	平衡正负样本数量	忽略了容易分类的样本,且执行效率低
样本平衡	样本重加权	[27]	平衡难易样本数量	需要对损失函数进行修改,通用性差
锚框优化	K-means++聚类	[20]	根据数据集生成更合适的锚点	尚未发现
	使用GIoU	[36]	解决预测框与真实框无法重叠造成的梯度消失问题	预测框与真实框相交时,在水平和垂直方向上收敛慢
	使用EIoU	[20]	将纵横比的损失项拆分成预测的宽高分别与最小外接框宽高的差值	尚未发现

Survey of underwater biological object detection methods based on deep learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 36

Related Articles 1

Metrics

Comments

Recommended 0