Deep Semantic Segmentation for the Quantification of Grape Foliar Diseases in the Vineyard

Authors: LIU, ERTAI - Cornell University; GOLD, KAITLIN - Cornell University; COMBS, DAVID - Cornell University; Cadle-Davidson, Lance; JIANG, YU - Cornell University

Submitted to: Frontiers in Plant Science
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 8/18/2022
Publication Date: 9/8/2022
Citation: Liu, E., Gold, K.M., Combs, D., Cadle Davidson, L.E., Jiang, Y. 2022. Deep semantic segmentation for the quantification of Grape Foliar Diseases in the vineyard. Frontiers in Plant Science. https://doi.org/10.3389/fpls.2022.978761.
DOI: https://doi.org/10.3389/fpls.2022.978761

Interpretive Summary: Plant disease evaluation is key for crop management and plant breeding. Field scouting by experts is commonly used to monitor diseases but is costly, laborious, subjective, and often imprecise. To improve disease evaluation, we used artificial intelligence (deep learning) analysis of images to calculate infection severity of grape foliar diseases. Five images per second were captured while driving a camera through the vineyard on an ATV. A custom stereo camera with strobe lighting was used along with real time kinematic (RTK) GPS for accurate location. The deep learning analysis identified healthy and infected grapevine to calculate disease severity. Fungicide trials for grape downy mildew and powdery mildew were used to evaluate these methods. The models were accurate for both downy mildew and powdery mildew. Because consecutive images had some overlap (redundancy), an analysis was developed to remove these redundant regions, to avoid double counting. Image-derived disease severity was highly correlated with expert ratings for both downy mildew and powdery mildew symptoms in the fungicide trial. Therefore, the developed approach can be used as an effective and efficient tool to quantify disease severity.

Technical Abstract: Grape foliar disease evaluation is crucial to vineyard management and grape breeding. Human field scouting has been widely used to monitor disease progress and provide qualitative and quantitative evaluation, which is costly, laborious, subjective, and often imprecise. To improve disease evaluation accuracy, throughput, and objectiveness, an image-based approach with a deep learning-based analysis pipeline was developed to calculate infection severity of grape foliar diseases. The image-based approach used a ground imaging system for field data acquisition, consisting of a custom stereo camera with strobe light for consistent illumination and real time kinematic (RTK) GPS for accurate localization. The deep learning-based pipeline used the hierarchical multiscale attention semantic segmentation (HMASS) model for disease infection segmentation, color filtering for grapevine canopy segmentation, and depth and location information for effective region masking. The resultant infection, canopy, and effective region masks were used to calculate the severity rate of disease infections in an image sequence collected in a given unit (e.g., grapevine panel). Fungicide trials for grape downy mildew (DM) and powdery mildew (PM) were used as case studies to evaluate the developed approach and pipeline. Experimental results showed that the HMASS model achieved acceptable to good segmentation accuracy of DM (mIoU > 0.84) and PM (mIoU > 0.74) infections in testing images, demonstrating the model capability for symptomatic disease segmentation. With the consistent image quality and multimodal metadata provided by the imaging system, the color filter and overlapping region removal could accurately and reliably segment grapevine canopies and identify repeatedly imaged regions between consecutive image frames, leading to critical information for infection severity calculation. Image-derived severity rates were highly correlated (r > 0.95) with human-assessed values, and had comparable statistical power in differentiating fungicide treatment efficacy in both case studies. Therefore, the developed approach and pipeline can be used as an effective and efficient tool to quantify the severity of foliar disease infections, enabling objective, high-throughput disease evaluation for research projects, breeding programs, and precision disease management.