Real-time stored product insect detection and identification using deep learning: System integration and extensibility to mobile platforms

Authors: BADGUJAR, CHETAN - Oak Ridge Institute For Science And Education (ORISE); Armstrong, Paul; Gerken, Alison; Pordesimo, Lester; Campbell, James - Jim

Submitted to: Journal of Stored Products Research
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/10/2023
Publication Date: 11/5/2023
Citation: Badgujar, C., Armstrong, P.R., Gerken, A.R., Pordesimo, L.O., Campbell, J.F. 2023. Real-time stored product insect detection and identification using deep learning: System integration and extensibility to mobile platforms. Journal of Stored Products Research. 104. Article 102196. https://doi.org/10.1016/j.jspr.2023.102196.
DOI: https://doi.org/10.1016/j.jspr.2023.102196

Interpretive Summary: Existing stored product insect monitoring methods can be time-consuming, costly, and require specialized equipment and personnel training to be performed accurately. To remove some of the barriers imposed by insect monitoring a system consisting of a simple RGB camera was developed to automatically recognize and identify insects using artificial intelligence (AI). Images of six common stored product insects (cigarette beetle, drug store beetle, red flour beetle, rice weevil, saw-toothed grain beetle and warehouse beetle) provided data for deep learning training, which is an emerging AI algorithm that is increasingly being used for insect species identification. State-of-the-art computer vision models, trained and evaluated using subsets of collected insect images, delivered impressive performance with high detection accuracy; over 76% of insects were accurately assigned to the correct species with fast detection times of, at most, 36 milliseconds. The best-performing model was integrated and deployed on a smartphone and desktop computer and real-time detection of insects appeared as a continuous video stream. Fast detection rates show good potential for the model to be employed on lower performing, and less expensive hardware, which is important for commercial applications. Overall, the study provided a framework for automatic and real-time insect identification and detection in a stored product environment that seems well suited for adaptability to a variety of hardware platforms.

Technical Abstract: Existing stored product insect monitoring methods are time-consuming, costly, and often require specialized equipment or training. This study proposed an integrated insect monitoring system that employs a simple RGB camera and data-driven deep-learning models to identify and detect stored product insect species in warehouses, food facilities, and retail environments. Top-down images of six common insect species were acquired with an established simulated setup with varying lighting and background conditions. These images were preprocessed and manually annotated, resulting in an insect dataset of 2,630 images with 14,509 labeled insects. A state-of-the-art computer vision model from the YOLO family was selected, and six YOLO variants (YOLOv5s/m/l and YOLOv5s/m/l) were trained and evaluated on the insect dataset. All trained YOLO models delivered an impressive performance in terms of high detection accuracy (above 76% for mAP@[0.50:0.95]) and fast inference time (12 to 36 ms range). Subsequently, the best-performing YOLOv8l model was integrated and deployed on mobile devices, achieving a good detection performance with aver-age detection speeds of 16 and 29 FPS on a desktop computer and smartphone, respectively. The study provided an end-to-end framework for automatic and real-time insect identification and detection in a stored product environment. The lightweight variant of YOLO can be deployed on low-cost edge hardware, mobile devices, or cloud computing. The proposed system is relatively fast, accurate, and inexpensive for insect monitoring and may prove an alternate solution to existing methods. The system would serve as a decision-making tool for stored product facilities managers and can be easily scaled and adapted in a variety of stored product environments.