| CNN-LSTM在日本鲭捕捞渔船行为提取中的应用 |
| |
| 引用本文： | 王书献,张胜茂,唐峰华,石永闯,范秀梅,樊伟,孙宇.CNN-LSTM在日本鲭捕捞渔船行为提取中的应用[J].农业工程学报,2022,38(7):200-209. |
| |
| 作者姓名： | 王书献 张胜茂 唐峰华 石永闯 范秀梅 樊伟 孙宇 |
| |
| 作者单位： | 1. 大连海洋大学航海与船舶工程学院,大连 116023; 2. 农业农村部渔业遥感重点试验室,中国水产科学研究院东海水产研究所,上海 200090; |
| |
| 基金项目： | 国家自然科学基金项目（61936014）;中国水产科学研究院基本科研业务费（2020TD82）;浙江省海洋渔业资源可持续利用技术研究重点试验室开放课题（2020KF001） |
| |
| 摘 要： | 目前,国内外部分自动化程度较高的捕捞渔船逐渐开始采用人类观察员与电子观察员相结合的捕捞监控方式。为简化电子监控系统数据处理流程、提高电子监控系统自动化程度、增强相关渔业公司及政府部门的管理效率,该研究提出将卷积神经网络（Convolutional Neural Network, CNN）与长短期记忆模块（Long Short Term Memory, LSTM）应用在传统电子监控系统数据中,用于灯光渔业日本鲭捕捞渔船行为提取。根据日本鲭捕捞渔船工作特点,划分出放网、收网、进鱼、转载等9种捕捞作业行为。设计4组平行试验,分别对比3层卷积神经网络、带有LSTM模块的3层卷积神经网络、8层卷积神经网络、带有LSTM模块的8层卷积神经网络在日本鲭捕捞渔船行为提取中的表现。试验结果显示,在测试集中该4种神经网络训练得到的模型综合评价指标F1分数分别达到了0.794、0.799,0.966及0.972,在配备NVIDIA Tesla V100 32GB高性能GPU的超算环境下5 000组数据平均每次检测耗时分别为34.66、34.50、37.41、37.61 ms。在一定范围内,网络深度的增加会显著提升模型的效果,但检测耗时也会有明显增加;LSTM模块的加入对网络模型效果有一定程度的提升,且不会显著影响检测耗时。因此,CNN-LSTM 模块在电子监控系统高实时性、高精度场景下均有较大的应用前景,能够提高电子监控系统自动化程度,提高相关部门的管理效率。
|
| 关 键 词： | 电子监控系统 卷积神经网络 长短期记忆 日本鲭 捕捞行为 |
| 收稿时间： | 2022/1/14 0:00:00 |
| 修稿时间： | 2022/3/22 0:00:00 |
Extracting the behavior of Scomber japonicus fishing vessel using CNN-LSTM |
| |
| Wang Shuxian,Zhang Shengmao,Tang Fenghu,Shi Yongchuang,Fan Xiumei,Fan Wei,Sun Yu.Extracting the behavior of Scomber japonicus fishing vessel using CNN-LSTM[J].Transactions of the Chinese Society of Agricultural Engineering,2022,38(7):200-209. |
| |
| Authors: | Wang Shuxian Zhang Shengmao Tang Fenghu Shi Yongchuang Fan Xiumei Fan Wei Sun Yu |
| |
| Institution: | 1. School of Navigation and Naval Architecture, Dalian Ocean University, Dalian 116023, China; 2. Key Laboratory of Fisheries Remote Sensing, Ministry of Agriculture and Rural Affairs, East China Sea Fisheries Research Institute, Chinese Academy of Fishery Sciences, Shanghai 200090, China; |
| |
| Abstract: | Fishing vessels have been ever-increasing to adopt the monitoring system for the combination of human and electronic observers, particularly with a high level of automation in the world. The data processing has required a simple configuration in the electronic monitoring system at present. It is urgent to improve the automation degree of the electronic monitoring system and the management efficiency of relevant fishing. In this study, the combined Convolutional Neural Network (CNN) with the Long Short Term Memory module (LSTM) was used to apply to the traditional electronic monitoring systems for the data extraction from the fishing vessel behavior of Japanese mackerel fishing in the light fishery. Nine types of fishing operations were divided, including sailing, putting net, waiting, pulling net, waiting for fish, organizing fish box, fish in, fish picking and reprinting, according to the operational characteristics of Japanese mackerel fishing boats. Four groups of parallel experiments were designed to compare the performance of a 3-layer CNN, a 3-layer CNN with an LSTM module (3-layer CNN-LSTM), an 8-layer CNN, and an 8-layer CNN with an LSTM module (8-layer CNN-LSTM) in a Japanese mackerel fishing boat during behavior extraction. Both CNN with LSTM modules were designed before the fully connected layers. The behaviors of nine fishing vessels presented the specific time characteristics, such as "pulling net" and "putting net". Therefore, the 100 consecutive frames were then stitched horizontally to design the dataset during data processing. The frames per second of the video data was 25, and 100 was determined as the length of each dataset. The reason was that the human eye was normally observed the EM video data for 2-4 s to distinguish the two behaviors. Each data sample also contained certain time dimension information in the LSTM module. The test results show that the comprehensive evaluation index, the F1 score of the models that trained by the 3-layer CNN, 3-layer CNN-LSTM, 8-layer CNN, and 8-layer CNN-LSTM in the test set reached 0.794, 0.799, 0.966, and 0.972, respectively. In addition, the average detection speed of each model on three devices was measured to fully detect the detection efficiency of each model, including a laboratory supercomputer, small mobile workstation, and personal computer. The test samples of 5 000 data sets were taken 34.66, 34.50, 37.41, and 37.61 ms to process each piece using 3-layer CNN, 3-layer CNN-LSTM 8-layer CNN, and 8-layer CNN-LSTM on the supercomputer, respectively. In a small workstation, they were 85.47, 86.10, 120.13, and 121.15 ms in PC2, respectively, whereas, 93.77, 102.27, 118.51, and 124.50 ms in PC3, respectively, in the personal computer. Consequently, the network depth significantly improved the efficiency of the model within the specific range, but the detection time also increased significantly. Specifically, the F1 score of the 8-layer CNN increased from 0.794 to 0.966, while the training time increased from 23.028 to 29.624 h, compared with the 3-layer CNN. The LSTM module enhanced the performance of the network model with the sound detection time. Therefore, the CNN-LSTM modules presented excellent application prospects in the high real-time and high-precision scenarios of electronic monitoring systems. The finding can be expected to improve the automation degree of electronic monitoring systems for the better management efficiency of fishing boats. |
| |
| Keywords: | electronic monitoring system convolutional neural network long short-term memory Scomber japonicus fishing behavior |
|
| 点击此处可从《农业工程学报》浏览原始摘要信息 |
| 点击此处可从《农业工程学报》下载免费的PDF全文 |
|
