Deep learning image segmentation and extraction of blueberry fruit traits associated with harvestability and yield

Authors: NI, XUEPING - Zhejiang University; LI, CHANGYING - University Of Georgia; JIANG, HUANYU - Zhejiang University; Takeda, Fumiomi

Submitted to: Horticulture Research
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 4/14/2020
Publication Date: 7/1/2020
Citation: Ni, X., Li, C., Jiang, H., Takeda, F. 2020. Deep learning image segmentation and extraction of blueberry fruit traits associated with harvestability and yield. Horticulture Research. https://doi.org/10.1038/s41438-020-0323-3.
DOI: https://doi.org/10.1038/s41438-020-0323-3

Interpretive Summary: A rapid and reliable procedure to extract blueberry fruit traits such as fruit cluster compactness, maturity, and numbers by an automatic detection and segmentation is needed to help blueberry breeders with their genotype selection. It is also needed to help producers with making informed decisions on predicting yield, harvesting time and plant management. In this study, a data processing pipeline to count berries, measure maturity, and evaluate compactness of clusters using a deep learning image segmentation method was used to analyze four southern highbush blueberry cultivars and determine trait differences. The results of trait analyses showed that ‘Star’ had the lowest berry number per cluster, ‘Farthing’ was least mature in mid April, ‘Farthing’ had the most compact clusters, and ‘Meadowlark’ had the loosest clusters. Efficient image segmentation technique developed in this study is a useful analytical tool that can estimate yield, predict harvest time, and enable decisions on suitability for mechanical harvesting.

Technical Abstract: Extraction of blueberry fruit traits such as fruit cluster compactness, maturity, and berry numbers by automatic detection and segmentation can assist blueberry breeders and producers with making informed decisions on genotype selection, yield traits, harvesting, and plant management. The goal of this study was to develop a data processing pipeline to count berries, to measure maturity, and to evaluate compactness (cluster tightness) automatically using a deep learning image segmentation method. An iterative annotation strategy was used to label images that reduced the annotation time. A Mask R-CNN model was trained and tested to detect and segment individual blueberries with maturity. The mAP for the validation and test dataset was 78.3% and 71.6% under 0.5 IOU threshold, and the corresponding mask accuracy was 90.6% and 90.4%, respectively. The compactness was defined as the ratio of the berry mask area and the minimum bounding box area. Linear regression was conducted to compare the detected berry number and the ground truth with R2 of 0.886 and RMSE of 1.484 combining all cultivars. ANOVA was used to analyze the trait differences among four cultivars (Emerald, Farthing, Meadowlark, and Star). The results from trait analysis of these four cultivars indicated that ‘Star’ had a lower berry number, ‘Farthing’ had the least mature fruit in mid-April, ‘Farthing’ had the most compact clusters, and ‘Meadowlark’ had the loosest clusters. Efficient image segmentation techniques were developed for blueberry fruit detection and segmentation, machine harvestability trait extraction, and analysis to monitor blueberry fruit development. The deep learning technology is an analytical tool that can be used to conduct yield estimation, prediction of harvest time, and making decisions on the harvest method (manual harvest or mechanical harvest).