A multimodal optical sensing system for automated and intelligent food safety inspection

Authors: Qin, Jianwei - Tony Qin; HONG, JEEHWA - National Agricultural Products Quality Management Service; CHO, HYUNJEONG - National Agricultural Products Quality Management Service; Van Kessel, Jo Ann; BAEK, INSUCK - Orise Fellow; Chao, Kuanglin - Kevin Chao; Kim, Moon

Submitted to: Journal of the ASABE
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 4/7/2023
Publication Date: 7/6/2023
Citation: Qin, J., Hong, J., Cho, H., Van Kessel, J.S., Baek, I., Chao, K., Kim, M.S. 2023. A multimodal optical sensing system for automated and intelligent food safety inspection. Journal of the ASABE. 66(4):839-849. https://doi.org/10.13031/ja.15526.
DOI: https://doi.org/10.13031/ja.15526

Interpretive Summary: Commercial integrated spectroscopy systems are usually bulky and not flexible for testing various food and agricultural products. There exists a lack of compact sensing devices and methods for quick and routine analysis of chemical and biological content of food samples. As an extension to our macro-scale Raman technologies, this study developed a new transportable multimodal (transmission, color, fluorescence, and Raman) optical sensing system with embedded artificial intelligence capabilities for automated and intelligent food safety inspection. By using machine vision and motion control techniques, the system can conduct automated Raman spectral acquisition for samples randomly scattered in Petri dishes or placed in customized well plates. Interesting targets in the samples can be identified and labeled using real-time image and spectral processing and machine learning functions integrated into the in-house developed software. The system is promising to be used by food safety regulatory agencies as an initial screening tool for quick species identification of common foodborne bacteria. The prototype is compact and easily transportable, which enables it to be suitable for field and on-site food safety inspection in potential regulatory and industrial uses.

Technical Abstract: A novel multimodal optical sensing system was developed for automated and intelligent food safety inspection. The system uses two pairs of compact point lasers and dispersive spectrometers at 785 and 1064 nm to realize dual-band Raman spectroscopy and imaging, which is suitable to measure samples generating low- and high-fluorescence interference signals, respectively. Automated spectral acquisition can be performed using a direct-drive XY moving stage for solid, powder, and liquid samples placed in customized well plates or randomly scattered in standard Petri dishes (e.g., bacterial colonies). Three LED lights (white backlight, UV ring light, and white ring light) and two miniature color cameras are used for machine vision measurements of samples in the Petri dishes using different combinations of illuminations and imaging modalities (e.g., transmission, fluorescence, and color). Real-time image processing and motion control techniques are used to implement automated sample counting, positioning, sampling, and synchronization functions. System software was developed using LabVIEW with integrated artificial intelligence functions able to identify and label interesting targets instantly. The system capability was demonstrated by an example application for rapid identification of five common foodborne bacteria, including Bacillus cereus, E. coli, Listeria monocytogenes, Staphylococcus aureus, and Salmonella spp.. Using a machine learning model based on a linear support vector machine, a classification accuracy of 98.6% was achieved using Raman spectra automatically collected from 222 bacterial colonies of the five species grown on nutrient nonselective agar in 90 mm Petri dishes. The entire system was built on a 30×45 cm2 breadboard, making it compact and easily transportable that can be used for field and on-site biological and chemical food safety inspection in regulatory and industrial applications.