Location: Quality and Safety Assessment Research Unit
2022 Annual Report
Objectives
1. Develop imaging technologies to detect and identify plastics during poultry processing with hyperspectral imaging and artificial intelligence.
1A. Develop hyperspectral imaging technology for detection and identification of plastic foreign objects during poultry processing.
1B. Develop AI technology for enhanced detection and smart robotic removal of foreign materials in hyperspectral imagery during poultry processing.
2. Detection and identification of foodborne bacteria and toxins in poultry products with high-throughput hyperspectral microscopy and surface plasmon resonance imaging.
2A. Rapid monitoring of indicator microorganisms in poultry processing.
2B. Develop advanced hyperspectral microscope imaging (HMI) methods and system for label-free detection and identification of pathogens at the cellular level with no enrichment.
2C. Develop high-sensitive and selective immunoassay method and system for foodborne bacteria and toxin detection with surface plasmon resonance imaging.
3. Eliminate the production of semicarbazide in non-nitrofurazone treated poultry by optimization of antimicrobial treatments and/or alternative antimicrobials during processing.
4. Develop safe and effective poultry processing strategies (scalding-picking-evisceration procedures) to reduce foodborne contaminants (pathogens/chemical) and enhance the sustainability of poultry processing.
Approach
Research on poultry safety will focus on: 1) developing and validating early, rapid, sensitive, and/or high-throughput optical sensing techniques for detecting physical and biological hazards in poultry products, and 2) eliminating semicarbazide in non-nitrofurazone treated poultry by optimization of antimicrobial treatments. In research on optical detection of physical hazards, spectroscopic and hyperspectral imaging (HSI) technologies will be developed for detection and identification of plastic foreign objects. A robot rejector and control software will be developed to eliminate foreign materials (FM) when detected by HSI. Artificial intelligence (AI) technology will be developed for enhanced detection and smart robotic removal of FM during poultry processing through the development and evaluation of customized deep learning algorithms based on hyperspectral imaging. A vision-guided smart robotic manipulator will be designed and built to remove FM by self-learning AI algorithms. To develop methods and techniques for detecting and identifying biological hazards, time-lapse image data on pure-culture indicator organisms and poultry carcass rinses from different processing locations will be collected to build a library, which will be used for on-line counting of microcolonies to build prototype systems. To detect foodborne pathogens, hyperspectral microscope imaging (HMI) methods will be developed with a spectral library of various pathogens using two HMI platforms of acousto-optical tunable filter (AOTF) and Fabry-Perot interferometer (FPI). In accordance with optimization of parameters on HMI and hypercubes, a transportable HMI system will be developed embedded with AI-based software for classification and identification. To identify foodborne bacteria and toxins, a highly-sensitive and selective immunoassay method and system will be developed using surface plasmon resonance imaging (SPRi). Microfluidic devices will be designed, simulated and fabricated for bacterial enrichment and separation. Both materials and parameters to develop a 3D printed biosensor for multiplex detection of pathogenic bacteria and toxins will be optimized and evaluated with food samples. Finally, a portable 3D printing platform for biosensor fabrication by integrating sample enrichment cartridge, biochip and SPRi detector will be developed. To develop techniques for eliminating the production of semicarbazide (SEM) in non-nitrofurazone treated poultry, a methodology for SEM analysis in chicken meat and a data library relating poultry processing conditions to SEM formation will be developed. Specifically, SEM in chicken leg quarters obtained from multiple processing facilities will be analyzed and methods to eliminate SEM production in poultry products under processing conditions will be developed.
Progress Report
During Fiscal Year (FY) 2022 significant progress was made on developing hyperspectral imaging technology for detection and identification of plastic foreign objects during poultry processing (Sub-Objective 1A). Hyperspectral images and spectra of common plastic materials such as polyethylene (PE), polyethylene terephthalate (PET), polyvinyl chloride (PVC), polystyrene (PS), polypropylene (PP), acrylonitrile butadiene styrene (ABS), and polyurethane (PUR) were collected and analyzed. An extended visible and near-infrared hyperspectral camera was used to collect hyperspectral images in the wavelength range 600 nm – 1,700 nm. A color camera and a spectrometer (400 nm – 2,500 nm) were also used. A classification technique was developed to identify the types of plastic pieces (foreign materials) found on breast fillets of broilers.
During FY 2022 progress was made on developing artificial intelligence (AI) technology for enhanced detection and smart robotic removal of foreign materials in hyperspectral imagery during poultry processing (Sub-objective 1B). A deep learning AI model based on a generative adversarial network was developed for non-destructive, hyperspectral image-based foreign material detection, where no foreign materials were required at all during the training for the detection model while only chicken meat data were required for training. A dataset including about 900,000 spectral data were obtained from chicken breast fillets in the wavelength range of 1,000 – 2,500 nm. Tests were conducted for breast fillets contaminated with 30 different types of foreign materials commonly found in processing plants, in two sizes: nominal 2 mm x 2 mm (actual dimension of the longer side: 2.2 ± 0.4 mm) and nominal 5 mm x 5mm (actual dimension of the longer side: 5.3 ± 0.5 mm). The model achieved an F1 score of over 95% while the detection of foreign materials was 100%.
During FY 2022 progress was made on the rapid monitoring of indicator microorganisms in poultry processing (Sub-objective 2A). Three dedicated incubators were modified for time-lapse imaging of colony growth in Petri dishes. Each incubator was modified to have a double-paned, sealed viewing port added to its top, through which a color camera can view and record bacterial growth in a single Petri dish. Each dish was back-illuminated by a specially modified light panel connected via fiber optic cable to an illuminator that resided outside of the incubator. Time-lapse images were captured at one-minute intervals at a resolution of 45.7 mega-pixels over a 48-hour period. The resulting high-resolution color images were assembled into a video at 8K resolution and analyzed using a change detection technique. Nine strains of Shiga toxin-producing E. coli have been imaged from serotypes O103, O121, O145, and O157.
During FY 2022 significant progress was made to develop advanced hyperspectral microscope imaging (HMI) methods and systems for label-free detection and identification of pathogens at the cellular level with no enrichment (Sub-objective 2B) in three projects. First, research was conducted on multi-task AI detection of live pathogenic bacteria. Identifying live bacterial cells is critical to investigating potential foodborne outbreaks. ARS researchers developed a multi-task model to recognize live bacterial cells with hyperspectral microscope imaging and deep learning (HMI-DL) methods. The AI-based model classified bacterial cells as dead or alive, and identified their species including E. Coli, Listeria, Staphylococcus, and Salmonella with 100% and 96.9% test accuracies. In addition, the live-cell identification of the model was highly accurate because the model adjusted its parameters based on the self-predicted species. Second, explainable artificial intelligence (XAI) methods to identify dead bacterial cells were researched. Building a transparent model is crucial to understanding what the model learned from data. However, it is challenging to understand the input-output relationship with an AI-based model because such a model involves multiple layers of computations with its input. ARS researchers faced this lack of transparency with an AI model (Fusion-Net) identifying live bacterial cells with 100% test accuracy. To interpret the trained model, a visualization method called Gradient-weighted class activation mapping (Grad-CAM) was employed. This technique provided visual explanations of Fusion-Net's decision with given inputs and allowed the identification of spectral-spatial features influential for the AI model to identify live cells with hyperspectral microscope image data. Furthermore, the transparency of the model helped to condense structure and improve performance of the model. Third, deep learning methods for bacterial detection were investigated with USDA SCINet. Hyperspectral microscope imaging with deep learning (HMI-DL) has accurately detected pathogenic bacterial cells. But the technique requires heavy computation to train an AI-based model with its complex architecture. To reduce this computational burden, ARS researchers built a new deep learning environment in USDA SCINet. The USDA-ARS initiative provides a high-performance computing system with a hundred computers connected to a high-speed network. The developed deep learning environment utilized more than 5,000 processing cores and 30 GB of memory in the graphics processing units of the system, allowed accelerated training and continuous experimentation of the AI model to detect pathogenic bacterial cells with HMI. The SCINet environment improved the AI-based model (Fusion-Net) to classify four bacterial species (Salmonella, E. coli, Listeria, and Staphylococcus) with the combined big dataset of 940 GB collected over the four years (2019-2022) and classified the species with 98.4% test accuracy.
During FY 2022 progress was made to develop a highly-sensitive and selective immunoassay method and system for foodborne bacteria and toxin detection with surface plasmon resonance imaging (Sub-objective 2C). Research on microfluidic sampling and biosensing systems for Escherichia coli and Salmonella was conducted. The development of portable biosensors for field-deployable detection has been increasingly important to control foodborne pathogens in the early stages of outbreaks. Since conventional cultivation and gene amplification methods require sophisticated instruments and highly skilled professionals, portable biosensing devices, which provide more flexibility for rapid detections even though their sensitivity and specificity are limited, are needed for high-throughput testing. Microfluidic methods have the advantage of miniaturizing instrumental size while integrating multiple functions and high-throughput capability into one streamlined system at low cost and minimal sample consumption to detect samples in different sizes and concentrations. ARS researchers investigated major active and passive microfluidic devices for bacteria sampling and biosensing focused on particle-based sorting/enrichment methods with or without external physical fields applied to the microfluidic devices for E. coli and Salmonella sampling.
During FY 2022 progress was made on research to eliminate the production of semicarbazide in non-nitrofurazone treated poultry by optimization of antimicrobial treatments and/or alternative antimicrobials during processing (Objective 3). Research was conducted to develop a data library relating poultry processing conditions to formation of semicarbazide. The pH range necessary for the chemical production of semicarbazide from chicken treated with peracetic acid, as occurs in a poultry processing chill tank, has been plotted. ARS researchers have found that semicarbazide is produced in the pH range of 9 through 13. Semicarbazide production was not detected in the pH range 4 through 8.
Accomplishments
1. Development of a novel deep learning technique to detect foreign materials during poultry processing. Foreign materials found in poultry products are a food safety concern for consumers and poultry processors. ARS researchers in Athens, Georgia, developed a semi-supervised deep learning model based on a technique called generative adversarial network from hyperspectral images such that the need for collecting massive amounts of foreign material data for training is eliminated completely. Using this approach, results indicated that the detection of foreign materials of relatively small size (~ 2 mm x 2 mm) in poultry meat could be achieved with hyperspectral imaging at >95% accuracy. These findings demonstrate that implementing this deep learning AI model may make it possible to utilize hyperspectral imaging as an accurate, high-throughput system for foreign material detection during poultry processing.
2. Label-free immunoassay for multiplex detection of foodborne bacteria in chicken carcass rinse with surface plasmon resonance imaging. Frequent outbreaks of foodborne pathogens have stimulated the demand for biosensors capable of rapidly detecting multiple pathogens in contaminated food. ARS researchers in Athens, Georgia, developed sensing technology with surface plasmon resonance imaging (SPRi) for simultaneous label-free detection of multiple foodborne pathogens with a low limit of detection, mainly Salmonella spp. and Shiga-toxin producing Escherichia coli (STEC), in commercial chicken carcass rinse. The injected samples with different bacteria (Salmonella Enteritidis, STEC, and Listeria monocytogenes) have been identified by the same biochip. Moreover, the SPRi signals revealed complex interference effects among coexisting bacteria species in heterogeneous bacteria solutions. This SPRi-based immunoassay demonstrated great potential in high-throughput screening of multiple pathogenic bacteria coexisting in chicken carcass rinse.
3. Rapid identification of foodborne bacteria with hyperspectral microscopic imaging and artificial intelligent classification methods. Early detection of foodborne pathogens is crucial to promote public health. ARS researchers in Athens, Georgia, developed a technique called artificial intelligence (AI) assisted hyperspectral microscopic imaging (HMI) to differentiate five foodborne bacteria including Campylobacter, E. coli, Listeria, Salmonella, and Staphylococcus simultaneously. An artificial recurrent neural network called long-short term memory (LSTM) was employed and optimized to directly process the spectra acquired from different regions of interest of bacterial cells. Compared to conventional machine learning methods with classification accuracies between 66 - 85 %, the newly developed AI-based classifier achieved an accuracy of 92 %. Furthermore, the AI-assisted HMI system can predict spectra instantly, making it an efficient tool for foodborne bacteria identification.
4. Development of integrated and high-throughput bacteria sampling cartridge. Most biosensing instruments for bacteria detection and identification require purification and enrichment of samples. Microfluidic systems are capable of bacteria sampling with miniaturized devices. However, sensitivity and selectivity must be improved for pathogen control in real-world applications. ARS researchers in Athens, Georgia, developed a microfluidic cartridge for bacteria sampling that contains a compact structure of multiple stacked layers which is reusable and requires no additional reagent. This bacteria sampling cartridge can be embedded into benchtop instruments or coupled onto portable biosensors. From this research, an invention disclosure has been approved for a patent application and can be fabricated with 3D printing techniques that provides high spatial resolution and effective mechanical properties for microchannels inside the cartridge.
Review Publications
Yao, L., Beibei, J., Yoon, S.C., Zhuang, H., Ni, X., Guo, B., Gold, S.E., Fountain, J.C., Glenn, A.E., Lawrence, K.C., Zhang, H., Guo, X., Zhang, F., Wang, W. 2022. Spatio-temporal patterns of Aspergillus flavus infection and aflatoxin B1 biosynthesis on maize kernels probed by SWIR hyperspectral imaging and synchrotron FTIR microspectroscopy. Food Chemistry. 382:132340. https://doi.org/10.1016/j.foodchem.2022.132340.
Chung, S., Yoon, S.C. 2021. Detection of foreign materials on broiler breast meat using fusion of visible near-infrared and short-wave infrared hyperspectral imaging. Applied Sciences. https://doi.org/10.3390/app112411987.
Park, B., Shin, T., Cho, J., Lim, J., Park, K. 2021. Characterizing hyperspectral microscope imagery for classification of blueberry firmness with deep learning methods. Agronomy Journal. https://doi.org/10.3390/agronomy12010085.
Kang, R., Park, B., Ouyang, Q., Ren, N. 2021. Rapid identification of foodborne bacteria with hyperspectral microscope imaging and artificial intelligence classification algorithms. Food Control. https://doi.org/10.1016/j.foodcont.2021.108379.
Wang, B., Park, B. 2022. Microfluidic sampling and biosensing systems for foodborne Escherichia coli and Salmonella. Foodborne Pathogens and Disease. https://doi.org/10.1089/fpd.2021.0087.
Wu, J., Ouyang, Q., Park, B., Kang, R., Wang, Z., Wang, L., Chen, Q. 2021. Physicochemical indicators coupled with multivariate analysis for comprehensive evaluation of matcha sensory quality. Food Chemistry. https://doi.org/10.1016/j.foodchem.2021.131100.
Zhang, H., Jia, B., Lu, Y., Yoon, S.C., Ni, X., Zhuang, H., Guo, X., Le, W., Wang, W. 2022. Detection of aflatoxin B1 in single peanut kernels by combining hyperspectral and microscopic imaging technologies. Sensors. 22(13):4864. https://doi.org/10.3390/s22134864.
Ouyang, Q., Wang, L., Park, B., Kang, R., Chen, Q. 2021. Simultaneous quantification of chemical constituents in matcha with visible near infrared hyperspectral imaging technology. Food Chemistry. https://doi.org/10.1016/j.foodchem.2021.129141.
Park, B., Wang, B., Chen, J. 2021. Label-free immunoassay for multiplex detections of foodborne bacteria in chicken carcass rinse with surface plasmon resonance imaging. Foodborne Pathogens and Disease. http://doi.org/10.1089/fpd.2020.2850.
Eady, M.B., Park, B. 2021. Unsupervised prediction model for Salmonella detection with hyperspectral microscopy: A Multi-year validation. Applied Sciences. https://doi.org/10.3390/app11030895.
Mitchell, T.R., Glenn, A.E., Gold, S.E., Lawrence, K.C., Berrang, M.E., Gamble, G.R., Feldner, P.W., Hawkins, J.A., Miller, C.E., Olson, D.E., Chatterjee, D., Mcdonough, C.M., Pokoo-Aikins, A. 2022. Survey of meat collected from commercial broiler processing plants suggests low levels of semicarbazide can be created during immersion chilling. Journal of Food Protection. 85(5):798-802. https://doi.org/10.4315/JFP-22-012.
