Recognition and positioning of fresh tea buds using YOLOv4-lighted + ICBAM Model and RGB-D sensing

Authors: GUO, SHUDAN - China Agricultural University; Yoon, Seung-Chul; LI, LEI - Zhanglou Town Government Of Chengwu County; WANG, WEI - China Agricultural University; Zhuang, Hong; WEI, CHAOLIE - China Agricultural University; LIU, YANG - China Agricultural University; LI, YUWEN - China Agricultural University

Submitted to: Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 2/17/2023
Publication Date: 2/21/2023
Citation: Guo, S., Yoon, S.C., Li, L., Wang, W., Zhuang, H., Wei, C., Liu, Y., Li, Y. 2023. Recognition and positioning of fresh tea buds using YOLOv4-lighted + ICBAM Model and RGB-D sensing. Agriculture. 13:518. https://doi.org/10.3390/agriculture13030518.
DOI: https://doi.org/10.3390/agriculture13030518

Interpretive Summary: A study was conducted to develop a computer vision technique adopting artificial intelligence (AI) technology and color and depth (RGB-D) sensing for the accurate detection and robotic harvest of tender tea buds which are often sold at premium prices. The objective of the study was to introduce and evaluate new deep learning-based object detection methods compared with a state-of-art object detection algorithm, YOLOv4, for which spatially registered color and depth images for model development and evaluation were acquired by a stereo vision camera. The YOLOv4 network for object detection was modified to obtain a lightweight model with a shorter inference time, called YOLOv4-light. Then, Squeeze-and-Excitation Networks (SENet), Efficient Channel Attention (ECA), Convolutional Block Attention Module (CBAM), and improved CBAM (ICBAM) were added to the output layer of the feature extraction network of the YOLOv4-light model for improving the performance on detecting small-sized and middle-sized tea buds. The performance of the baseline YOLOv4 and its improved models was evaluated in terms of accuracy and speed for the detection and localization of tea buds differentiated from other tea leaves. The YOLOv4-light with the ICBAM module was the best-performing model for detecting the bounding boxes of the tea buds with an F1-score of 0.94 and an average precision (AP) of 97% and for locating the positions for picking tea buds with an average success rate of 87.10% and an average positioning time of 0.12s. The results of the study suggest that the AI-based computer vision technology to detect and localize tea buds can lead to a vison-guided robotic picking machine that may increase the premium tea yield and product quality and solve a labor shortage problem.

Technical Abstract: This paper is concerned with the development of an improved You Only Look Once Version 4 (YOLOv4) object detection algorithm for the detection of tea buds and their picking points with tea-picking machines. The segmentation method based on color and depth data from a stereo vi-sion camera is proposed to detect the shapes of tea buds in 2D and 3D spaces more accurately than using 2D images. The YOLOv4 deep learning model for object detection was modified to obtain a lightweight model with a shorter inference time, called YOLOv4-light. Then, Squeeze-and-Excitation Networks (SENet), Efficient Channel Attention (ECA), Convolutional Block Attention Module (CBAM), and improved CBAM (ICBAM) were added to the output layer of the feature extraction network for improving the detection accuracy at small features. Finally, the Path Aggregation Network (PANet) in the neck network was simplified to the Feature Pyramid Network (FPN). The light-weighted YOLOv4 with ICBAM, called YOLOv4-light+ICBAM, was determined as the optimal recognition model for the detection of tea buds in terms of accuracy (94.19%), recall (93.50%), F1 score (0.94), and average precision (97.29%). Compared with the baseline YOLOv4, the model size of YOLOv4-light+ICBAM decreased by 75.18%, and the frame rate increased by 7.21%. In addition, the method for predicting the picking points of each detected tea bud was developed by segmentation of tea buds in each detected bounding box with filtering of each segment based on its depth (distance) from the camera. The test result showed that the average positioning success rate and the average positioning time were 87.10% and 0.12s respectively. In conclusion, the recognition and positioning method proposed in this paper would pave a theo-retical basis and method for the automatic picking of tea buds.