Affordable high throughput field detection of wheat stripe rust using deep learning with semi-automated image labeling

Authors: TANG, ZHOU - Washington State University; WANG, MEINAN - Washington State University; SCHIRRMANN, MICHAEL - Leibniz Institute; Li, Xianran; BRUEGGEMAN, ROBERT - Washington State University; SANKARAN, SINGHUJA - Washington State University; VARTER, AARON - Washington State University; PUMPHREY, MICHAEL - Washington State University; HU, YANG - Washington State University; Chen, Xianming; ZHANG, ZHIWU - Washington State University

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 2/12/2023
Publication Date: 2/20/2023
Citation: Tang, Z., Wang, M., Schirrmann, M., Li, X., Brueggeman, R., Sankaran, S., Varter, A.H., Pumphrey, M.O., Hu, Y., Chen, X., Zhang, Z. 2023. Affordable high throughput field detection of wheat stripe rust using deep learning with semi-automated image labeling. Computers and Electronics in Agriculture. 207. Article 107709. https://doi.org/10.1016/j.compag.2023.107709.
DOI: https://doi.org/10.1016/j.compag.2023.107709

Interpretive Summary: Stripe rust is one of the most devastating diseases of wheat and causes large-scale epidemics and severe yield losses. Applying fungicides during early epidemic development is crucial to controlling the disease but is often challenged by resource-limited human visual scouting. Deep learning has the potential to process images and videos captured from affordable devices to empower high-throughput phenotyping for early detection of stripe rust for timely application of fungicides and improve control efficiency. Here, we developed RustNet, a neural network-based image classifier, for efficiently monitoring fields for stripe rust. RustNet was built on a ResNet-18 architecture pre-trained with ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) dataset using transfer learning. RGB (Red, Green, and Blue) images and videos of multiple wheat fields with different wheat types (winter and spring wheat), conditions (irrigated and non-irrigated), and locations were acquired using smartphones or unmanned aerial vehicles near the canopy. A semi-automated image labeling approach was conducted to improve labeling efficiency by combining automated machine labeling and human correction. Cross-validations across multiple categories (sensor platforms, wheat types, and locations) achieved Area Under Curve from 0.72 to 0.87. Independent validation on a published dataset from Germany achieved accuracies ranging from 0.79 to 0.86. The visualization of the last convolutional layer of RustNet demonstrated the identification of pixels with stripe rust. RustNet is freely available at https://zzlab.net/RustNet.

Technical Abstract: Stripe rust (caused by Puccinia striiformis f. sp. tritici) is one of the most devastating diseases of wheat and causes large-scale epidemics and severe yield losses. Applying fungicides during early epidemic development is crucial to controlling the disease but is often challenged by resource-limited human visual scouting. Deep learning has the potential to process images and videos captured from affordable devices to empower high-throughput phenotyping for early detection of stripe rust for timely application of fungicides and improve control efficiency. Here, we developed RustNet, a neural network-based image classifier, for efficiently monitoring fields for stripe rust. RustNet was built on a ResNet-18 architecture pre-trained with ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) dataset using transfer learning. RGB (Red, Green, and Blue) images and videos of multiple wheat fields with different wheat types (winter and spring wheat), conditions (irrigated and non-irrigated), and locations were acquired using smartphones or unmanned aerial vehicles near the canopy. A semi-automated image labeling approach was conducted to improve labeling efficiency by combining automated machine labeling and human correction. Cross-validations across multiple categories (sensor platforms, wheat types, and locations) achieved Area Under Curve from 0.72 to 0.87. Independent validation on a published dataset from Germany achieved accuracies ranging from 0.79 to 0.86. The visualization of the last convolutional layer of RustNet demonstrated the identification of pixels with stripe rust. RustNet is freely available at https://zzlab.net/RustNet.