Classification of foodborne bacteria using hyperspectral microscope imaging technology coupled with convolutional neural networks

Authors: KANG, RUI - Nanjing Agricultural University; Park, Bosoon; Eady, Matthew; OUYANG, QIN - Jiangsu University; CHEN, KUNJIE - Nanjing Agricultural University

Submitted to: Applied Microbiology and Biotechnology
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 1/11/2019
Publication Date: 1/16/2020
Citation: Kang, R., Park, B., Eady, M.B., Ouyang, Q., Chen, K. 2020. Classification of foodborne bacteria using hyperspectral microscope imaging technology coupled with convolutional neural networks. Applied Microbiology and Biotechnology. https://doi.org/10.4315/0362-028X.JFP-19-311.
DOI: https://doi.org/10.4315/0362-028X.JFP-19-311

Interpretive Summary: A series of foodborne pathogen outbreaks increase the threat to the public, and major outbreaks of foodborne illness are often caused by several pathogens including Campylobacter spp., Escherichia coli, Listeria spp., Salmonella spp., and Staphylococcus spp. Although conventional bacterial detection methods are still the “gold standard” in bacterial detection, a hyperspectral microscope imaging (HMI) method combined with convolutional neural networks (CNN) is presented. The HMI method can acquire both images and spectral information of bacteria cells. Using these data, CNN methods were employed for bacterial cell segmentation and classification. Two CNN models were proposed in this research, a U-shape model called U-net was designed for cell-sized mask segmentation and a one-dimensional CNN (1D-CNN) model was designed for cell-based spectra classification. Comparing with conventional statistical data analysis methods, the proposed two CNN models performed better in both cell segmentation and classification. As a result, the U-net generated the most accurate mask for bacterial images with a fast processing time and the 1D-CNN method achieved higher accuracy for bacterial cell classification than conventional methods. In addition, these two CNN methods could be concatenated together to visualize the result for better understanding, which will enhance the detection ability of HMI technology.

Technical Abstract: Foodborne pathogens have become ongoing threats in the food industry, whereas their rapid detection and classification at an early stage are still challenging. To address early and rapid detection, hyperspectral microscope imaging (HMI) technology combined with convolutional neural networks (CNN) was proposed to classify foodborne bacterial species at the cellular level. HMI technology can simultaneously obtain both spatial and spectral information of different live bacterial cells, while two CNN frameworks, U-Net and one-dimensional CNN (1D-CNN) were employed to accelerate the data analysis process. U-Net was used for automated segmentation of cellular regions of interest (ROI), which generated accurate cell-ROI masks in a shorter timeframe than the conventional Otsu or Watershed methods. The 1D-CNN was employed for classifying the spectral profiles extracted from cell-ROI and resulted in a higher accuracy (90%) than k-nearest neighbor and support vector machine of which both had only 81% accuracy. Overall, the CNN-assisted HMI technology showed the potential for foodborne bacterial detection.