
基于改进ENet的复杂背景下山药叶片图像分割方法
Image Segmentation Method of Chinese Yam Leaves in Complex Background Based on Improved ENet
[目的/意义] 作物叶面积是反映光合作用效率和生长状况的重要指标,建立一个品种丰富的山药图像数据集并提出一种基于深度学习的山药叶片图像分割方法,可以用于实时测定山药叶片面积,解决传统测量效率低的问题。 [方法] 基于改进ENet的轻量化分割网络,在ENet的基础上,裁剪掉第3阶段,减少模型中的冗余计算;将瓶颈结构里面的常规卷积用PConv替换,构成P-Bottleneck,减少模型参数量,加快推理速度;改进上采样模块中的转置卷积为双线性插值,提升模型分割精度,减少参数量;最后在模型编码阶段加入CA注意力机制模块,强化对叶片边缘语义特征的提取能力。训练时使用Adam优化器,根据历史梯度信息自适应地调节学习率,加速收敛过程,提高模型的泛化能力。 [结果和讨论] 改进的模型在包含40个品种的山药室内图像数据集和室外数据集上进行实验,平均交并比和均像素精度分别达到98.61%和99.32%,模型参数量下降51%,浮点运算量下降49%,并且网络运算速度提高38%。与原始模型相比,在保证分割精度的同时显著降低网络的参数量和浮点运算量,提升运行速度,减少资源占用,使其更加适合应用到农业监测设备。 [结论] 改进算法能够精准快速地分割山药叶片,为复杂背景下山药叶片面积的研究提供了参考依据。
[Objective] Crop leaf area is an important indicator reflecting light absorption efficiency and growth conditions. This paper established a diverse Chinese yam image dataset and proposesd a deep learning-based method for Chinese yam leaf image segmentation. This method can be used for real-time measurement of Chinese yam leaf area, addressing the inefficiency of traditional measurement techniques. This will provide more reliable data support for genetic breeding, growth and development research of Chinese yam, and promote the development and progress of the Chinese yam industry. [Methods] A lightweight segmentation network based on improved ENet was proposed. Firstly, based on ENet, the third stage was pruned to reduce redundant calculations in the model. This improved the computational efficiency and running speed, and provided a good basis for real-time applications. Secondly, PConv was used instead of the conventional convolution in the downsampling bottleneck structure and conventional bottleneck structure, the improved bottleneck structure was named P-Bottleneck. PConv applied conventional convolution to only a portion of the input channels and left the rest of the channels unchanged, which reduced memory accesses and redundant computations for more efficient spatial feature extraction. PConv was used to reduce the amount of model computation while increase the number of floating-point operations per second on the hardware device, resulting in lower latency. Additionally, the transposed convolution in the upsampling module was improved to bilinear interpolation to enhance model accuracy and reduce the number of parameters. Bilinear interpolation could process images smoother, making the processed images more realistic and clear. Finally, coordinate attention (CA) module was added to the encoder to introduce the attention mechanism, and the model was named CBPA-ENet. The CA mechanism not only focused on the channel information, but also keenly captured the orientation and position-sensitive information. The position information was embedded into the channel attention to globally encode the spatial information, capturing the channel information along one spatial direction while retaining the position information along the other spatial direction. The network could effectively enhance the attention to important regions in the image, and thus improve the quality and interpretability of segmentation results. [Results and Discussions] Trimming the third part resulted in a 28% decrease in FLOPs, a 41% decrease in parameters, and a 9 f/s increase in FPS. Improving the upsampling method to bilinear interpolation not only reduces the floating-point operation and parameters, but also slightly improves the segmentation accuracy of the model, increasing FPS by 4 f/s. Using P-Bottleneck instead of downsampling bottleneck structure and conventional bottleneck structure can reduce mIoU by only 0.04%, reduce FLOPs by 22%, reduce parameters by 16%, and increase FPS by 8 f/s. Adding CA mechanism to the encoder could only increase a small amount of FLOPs and parameters, improving the accuracy of the segmentation network. To verify the effectiveness of the improved segmentation algorithm, classic semantic segmentation networks of UNet, DeepLabV3+, PSPNet, and real-time semantic segmentation network LinkNet, DABNet were selected to train and validate. These six algorithms got quite high segmentation accuracy, among which UNet had the best mIoU and the mPA, but the model size was too large. The improved algorithm only accounts for 1% of the FLOPs and 0.41% of the parameters of UNet, and the mIoU and mPA were basically the same. Other classic semantic segmentation algorithms, such as DeepLabV3+, had similar accuracy to improved algorithms, but their large model size and slow inference speed were not conducive to embedded development. Although the real-time semantic segmentation algorithm LinkNet had a slightly higher mIoU, its FLOPs and parameters count were still far greater than the improved algorithm. Although the PSPNet model was relatively small, it was also much higher than the improved algorithm, and the mIoU and mPA were lower than the algorithm. The experimental results showed that the improved model achieved a mIoU of 98.61%. Compared with the original model, the number of parameters and FLOPs significantly decreased. Among them, the number of model parameters decreased by 51%, the FLOPs decreased by 49%, and the network operation speed increased by 38%. [Conclusions] The improved algorithm can accurately and quickly segment Chinese yam leaves, providing not only a more accurate means for determining Chinese yam phenotype data, but also a new method and approach for embedded research of Chinese yam. Using the model, the morphological feature data of Chinese yam leaves can be obtained more efficiently, providing a reliable foundation for further research and analysis.
山药 / 图像分割 / 深度学习 / ENet / 部分卷积 / CA注意力机制 {{custom_keyword}} /
Chinese yam / image segmentation / deep learning / ENet / partial convolution / CA mechanism {{custom_keyword}} /
表1 室内山药叶片采集数据分类Table 1 Classification of data collected from indoor Chinese yam leaves |
序号 | 品种名称 | 数量/张 | 序号 | 品种名称 | 数量/张 | 序号 | 品种名称 | 数量/张 | 序号 | 品种名称 | 数量/张 |
---|---|---|---|---|---|---|---|---|---|---|---|
1 | 砀山山药 | 45 | 11 | 嵩野2号 | 49 | 21 | 太和长芋 | 42 | 31 | 南京采药 | 34 |
2 | 梅岱山药 | 39 | 12 | 靳家岭山药 | 37 | 22 | 安顺山药 | 48 | 32 | 山东牛腿米 | 36 |
3 | 僵野1号 | 47 | 13 | 惠楼山药 | 48 | 23 | 山王庄铁棍 | 30 | 33 | 泌阳野山药 | 40 |
4 | 苏北淮山药 | 45 | 14 | 辉县太行山药 | 50 | 24 | 宿生野山药 | 46 | 34 | 太古8号 | 30 |
5 | 日本山药 | 34 | 15 | 临泉笨山药 | 37 | 25 | 平遥山药 | 30 | 35 | 新城细毛 | 33 |
6 | 太原8号 | 34 | 16 | 双胞山药 | 33 | 26 | 山西榆次山药 | 43 | 36 | 怀山药1号 | 34 |
7 | 温科3号 | 37 | 17 | 2018 -1号山药 | 46 | 27 | 四川雅山药 | 41 | 37 | 铁棍雌株 | 33 |
8 | 安顺2号 | 46 | 18 | 桑县10号 | 34 | 28 | 砀山山药2号 | 41 | 38 | 神农山山药 | 36 |
9 | 小白嘴山药 | 36 | 19 | 铁棍山药1号 | 31 | 29 | 白玉山药 | 39 | 39 | 日本白山药 | 35 |
10 | 安顺5号 | 32 | 20 | 丰县铁棍山药 | 41 | 30 | 白皮山药 | 36 | 40 | 陇山药1号 | 30 |
表2 山药叶片研究训练集数据增强统计Table 2 Data enhancement statistics for the training set of Chinese yam images |
类别 | 初始数量/张 | 数据增强 | 最终数量/张 |
---|---|---|---|
室内 | 1 077 | 是 | 1 500 |
室外 | 129 | 是 | 1 032 |
总计 | 1 206 | — | 2 532 |
表3 山药叶片图像分割消融实验Table 3 Ablation experiments on segmentation of yam leaves images |
Test No. | Model | mIoU/% | mPA/% | Accuracy/% | FPS/(f/s) | Inference time/ms | FLOPs/G | Params/ |
---|---|---|---|---|---|---|---|---|
0 | ENet | 98.58 | 99.24 | 99.57 | 50 | 20.00 | 2.178 | 0.3492 |
1 | C-ENet | 98.48 | 99.19 | 99.54 | 59 | 18.87 | 1.563 | 0.2046 |
2 | CB-ENet | 98.53 | 99.23 | 99.55 | 63 | 15.87 | 1.428 | 0.1995 |
3 | CP-ENet | 98.45 | 99.18 | 99.53 | 68 | 14.71 | 1.295 | 0.1758 |
4 | CPB-ENet | 98.49 | 99.25 | 99.53 | 71 | 14.08 | 1.112 | 0.1681 |
5 | CPBA-ENet | 98.61 | 99.32 | 99.62 | 69 | 14.49 | 1.114 | 0.1714 |
图11 不同模型FLOPs与mIoU关系对比图Fig. 11 Comparison of the relationship between FLOPs and mIoU of different models |
图12 不同模型Params与mIoU关系对比图Fig. 12 Comparison of the relationship between Params and mIoU of different models |
表4 不同模型的山药叶片分割性能比较Table 4 Comparison of segmentation performance of Chinese yam leaves using different models |
Model | mIoU/% | mPA/% | Accuracy/% | FPS/(f/s) | FLOPs/G | Params/M |
---|---|---|---|---|---|---|
UNet | 99.09 | 99.57 | 99.72 | 17 | 92.0 | 43.9 |
DeepLabV3+ | 98.58 | 99.35 | 99.57 | 31 | 83.4 | 54.7 |
PSPNet | 97.45 | 98.67 | 99.22 | 23 | 61.6 | 49.1 |
LinkNet | 98.65 | 99.41 | 99.63 | 58 | 12.1 | 11.5 |
DABNet | 98.23 | 99.26 | 99.05 | 62 | 5.3 | 0.8 |
CBPA-ENet | 98.61 | 99.32 | 99.62 | 69 | 1.1 | 0.2 |
1 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
2 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
3 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
4 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
5 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
6 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
7 |
温建荣. 山药传统生产与现代生产的区别与比较[J]. 江西农业, 2018(18): 14.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
8 |
王永乐. 让"科研之花"结出山药"产业之果"[N]. 河南日报, 2024-03-17(13).
{{custom_citation.content}}
{{custom_citation.annotation}}
|
9 |
郝雅洁, 张吴平, 史维杰, 等. 基于计算机视觉的小麦叶面积测量[J]. 湖北农业科学, 2019, 58(16): 129-132.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
10 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
11 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
12 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
13 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
14 |
李方一, 黄璜, 官春云. 作物叶面积测量的研究进展[J]. 湖南农业大学学报(自然科学版), 2021, 47(3): 274-282.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
15 |
崔世钢, 秦建华. 图像处理法测定油菜叶面积的研究[J]. 湖北农业科学, 2017, 56(14): 2756-2757, 2767.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
16 |
于东玉, 冯天祥, 李奕昕, 等. 基于植物图像的活体叶片面积测量方法研究与实现[J]. 智能计算机与应用, 2019, 9(4): 173-176.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
17 |
李秋洁, 杨远明, 袁鹏成, 等. 基于饱和度分割的叶面积图像测量方法[J]. 林业工程学报, 2021, 6(4): 147-152.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
18 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
19 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
20 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
21 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
22 |
陈从平, 钮嘉炜, 丁坤, 等. 基于深度学习的马铃薯病害智能识别[J]. 计算机仿真, 2023, 40(2): 214-217, 222.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
23 |
杜鹏飞, 黄媛, 高欣娜, 等. 基于语义分割的复杂背景下黄瓜叶部病害严重程度分级研究[J]. 中国农机化学报, 2023, 44(11): 138-147.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
24 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
25 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
26 |
管博伦, 张立平, 朱静波, 等. 农业病虫害图像数据集构建关键问题及评价方法综述[J]. 智慧农业(中英文), 2023, 5(3): 17-34.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
27 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
28 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
29 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
30 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
31 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
32 |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_ref.label}} |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
/
〈 |
|
〉 |