Detecting intra-field variation in rice yield with UAV imagery and deep learning

Authors: BELLIS, EMILY - Arkansas State University; HASHEM, AHMED - Arkansas State University; CAUSEY, JASON - Arkansas State University; RUNKLE, BENJAMIN - University Of Arkansas; MOREGNO-GARCIA, BEATRIZ - University Of Arkansas; BURNS, BRAYDEN - Arkansas State University; GREEN, STEVE - Arkansas State University; BURCHAM, TIMOTHY - University Of Arkansas; Reba, Michele; HUANG, XIUZHEN - Arkansas State University

Submitted to: Frontiers in Plant Science
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 2/18/2022
Publication Date: 3/22/2022
Citation: Bellis, E., Hashem, A.A., Causey, J.L., Runkle, B.R., Moregno-Garcia, B., Burns, B., Green, S.V., Burcham, T.N., Reba, M.L., Huang, X. 2022. Detecting intra-field variation in rice yield with UAV imagery and deep learning. Frontiers in Plant Science. 13:716506. https://doi.org/10.3389/fpls.2022.716506.
DOI: https://doi.org/10.3389/fpls.2022.716506

Interpretive Summary: New technologies can be used to improve efficiency and sustainability of agricultural production. Monitoring crop stress with unmanned aerial vehicles (UAVs) allows for multiple measurements at high resolution. Pairing the data collected with UAVs with advanced machine learning models could improve real-time management of agricultural systems. However, guidance on the most effective strategy for this pairing is limited. Two deep learning-based strategies were tested for early warning detection of crop stress and how it related to yield. Root Mean Square Error describing differences in observed and predicted yield was less than 9% for both strategies, based on images of rice taken from UAVs throughout the growing season. The results highlight the promise of these strategies for UAV-based data collection to predict outputs. These findings will contribute to improvements in technology for precision agriculture. The work will also be of interest to growers and others seeking to improve sustainability of agricultural production by only using inputs (e.g., water, nutrients) when and where they are needed.

Technical Abstract: With increasing demand for crop production and rising environmental and economic challenges, there is an urgent need to harness new technologies for more efficient and sustainable agricultural production. To monitor crop stress at early stages of development, unmanned aerial vehicles (UAVs) equipped with multispectral sensors offer high spatial and temporal resolution imagery. Analysis of UAV-derived data with advanced machine learning models could improve real-time management in agricultural systems, but guidance for this integration is currently limited. Here we compare two deep learning-based strategies for early warning detection of crop stress, using multitemporal imagery throughout the growing season to predict field-scale yield in irrigated rice in eastern Arkansas. Both deep learning strategies showed improvements upon traditional statistical learning approaches including linear regression and gradient boosted decision trees. First, we explicitly accounted for variation across developmental stages using a 3D convolutional neural network (CNN) architecture that captures both spatial and temporal dimensions of UAV images from multiple time points throughout one growing season. 3D-CNNs achieved low prediction error on the test set, with a Root Mean Squared Error (RMSE) of 8.8% of the mean yield. For the second strategy, a 2D-CNN, we considered only spatial relationships among pixels for image features acquired during a single flyover. 2D-CNNs trained on images from a single day were most accurate when images were taken during boot stage or later, with RMSE ranging from 7.4 to 8.2% of the mean yield. A primary benefit of convolutional autoencoder-like models (based on analyses of prediction maps and feature importance) is the spatial denoising effect that corrects output (predicted yield) for an individual pixel based on observed input (vegetation index and thermal features) of nearby pixels. Our results highlight the promise of convolutional autoencoders for UAV-based yield prediction in rice. They also suggest that including data from more than one growth stage in a single model may not improve accuracy of yield prediction in rice.