Rapeseed stand count estimation at leaf development stages with UAV imagery and convolutional neural networks

Authors: ZHANG, JIAN - Huazhong Agricultural University; BIQUAN, ZHAO - Huazhong Agricultural University; Yang, Chenghai; YEYIN, SHI - University Of Nebraska; QINGXI, LIAO - Huazhong Agricultural University; GUANSCHENG, ZHOU - Huazhong Agricultural University; CHUFENG, WANG - Huazhong Agricultural University; TIANJIN, XIE - Huazhong Agricultural University; ZHAO, JIANG - Huazhong Agricultural University; DONGYAN, ZHANG - Anhui Agricultural University; WANNENG, YANG - Huazhong Agricultural University; CHENGLONG, HUANG - Huazhong Agricultural University; JING, XIE - Huazhong Agricultural University

Submitted to: Frontiers in Plant Science
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 11/20/2020
Publication Date: 11/30/2020
Citation: Zhang, J., Biquan, Z., Yang, C., Yeyin, S., Qingxi, L., Guanscheng, Z., Chufeng, W., Tianjin, X., Zhao, J., Dongyan, Z., Wanneng, Y., Chenglong, H., Jing, X. 2020. Rapeseed stand count estimation at leaf development stages with UAV imagery and convolutional neural networks. Frontiers in Plant Science. 11:617. https://doi.org/10.3389/fpls.2020.00617.
DOI: https://doi.org/10.3389/fpls.2020.00617

Interpretive Summary: Timely estimation of rapeseed stand count at early growth stages provides useful information for precision fertilization, irrigation, and yield prediction. This study aimed to detect and count rapeseed plant stands using high resolution imagery from an unmanned aerial vehicle and to determine the optimal timing for stand counting at leaf development stages. Object recognition models were developed to estimate plant stand count from recognized leaves. Results showed stand count achieved the best performance at the four- to six-leaf stage, indicating that it is feasible to estimate rapeseed stand count in the field rapidly and accurately using UAV-based imagery and image processing techniques. The methods and results from this study will be useful for crop phenotyping and precision management of rapeseed and other similar crops.

Technical Abstract: Timely estimation of rapeseed stand count at early growth stages provides useful information for precision fertilization, irrigation, and yield prediction. Based on the nature of rapeseed, the number of tillering leaves is strongly related to its growth stages. However, no field study has been reported on estimating rapeseed stand count by the number of leaves recognized with convolutional neural networks (CNNs) in unmanned aerial vehicle (UAV) imagery. The objectives of this study were to provide a case for rapeseed stand counting with reference to the existing knowledge of the number of leaves per plant and to determine the optimal timing for counting after rapeseed emergence at leaf development stages with one to seven leaves. A CNN model was developed to recognize leaves in UAV-based imagery, and rapeseed stand count was estimated with the number of recognized leaves. The performance of leaf detection was compared using sample sizes of 16, 24, 32, 40, and 48 pixels. Leaf overcounting occurred when a leaf was much bigger than others as this bigger leaf was recognized as several smaller leaves. Results showed CNN-based leaf count achieved the best performance at the four- to six-leaf stage with F-scores greater than 90% after calibration with overcounting rate. On average, 806 out of 812 plants were correctly estimated on 53 days after planting (DAP) at the four- to six leaf stage, which was considered as the optimal observation timing. For the 32-pixel patch size, root mean square error (RMSE) was 9 plants with relative RMSE (rRMSE) of 2.22% on 53 DAP, while the mean RMSE was 12 with mean rRMSE of 2.89% for all patch sizes. A sample size of 32 pixels was suggested to be optimal accounting for balancing performance and efficiency. The results of this study confirmed that it was feasible to estimate rapeseed stand count in field automatically, rapidly, and accurately. This study provided a special perspective in phenotyping and cultivation management for estimating seedling count for crops that have recognizable leaves at their early growth stage, such as soybean and potato.