A novel machine learning framework based on a hierarchy of dispute models for the identification of fish species using multi-mode spectroscopy

Authors: SUEKER, MITCHELL - University Of North Dakota; DAGHIGHI, AMIRREZA - Safetyspect Inc; AKHBARDESH, ALIREZA - Safetyspect Inc; MACKINNON, NICHOLAS - Safetyspect Inc; BEARMAN, GREGORY - Safetyspect Inc; Baek, Insuck; HWANG, CHANSONG - US Department Of Agriculture (USDA); Qin, Jianwei - Tony Qin; TABB, AAMANDA - Chapman University; ROUNGCHUN, JIAHLEEN - Chapman University; HELLBERG, ROSALEE - Chapman University; VASEFI, FARTASH - Safetyspect Inc; Kim, Moon; TAVAKOLIAN, KOUHYAR - University Of North Dakota; KASHANI ZADEH, HOSSEIN - University Of North Dakota

Submitted to: Sensors
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/30/2023
Publication Date: 11/9/2023
Citation: Sueker, M., Daghighi, A., Akhbardesh, A., Mackinnon, N., Bearman, G., Baek, I., Hwang, C., Qin, J., Tabb, A.M., Roungchun, J.B., Hellberg, R.S., Vasefi, F., Kim, M.S., Tavakolian, K., Kashani Zadeh, H. 2023. A novel machine learning framework based on a hierarchy of dispute models for the identification of fish species using multi-mode spectroscopy . Sensors. 23(22): Article e9062. https://doi.org/10.3390/s23229062.
DOI: https://doi.org/10.3390/s23229062

Interpretive Summary: Seafood mislabeling poses risks for consumers’ health and gives rise to economic and environmental hazards. Mixing less expensive species into more expensive species is one of major fish fraudulent practices in the seafood industry. This study developed a method based on multi-mode spectroscopy and machine learning techniques for detecting mislabeling of fish fillets. Three modes of spectra, including fluorescence and reflectance in visible and near-infrared and short-wave infrared regions were extracted from hyperspectral images collected from 216 fish fillet samples of 43 species. Algorithms were developed to create a hierarchical decision process for higher classification performance. Based on a classifier incorporating global and dispute models, a classification accuracy of 89% was achieved using the fusion of the three spectroscopic modes. The method developed in this study can be used to develop a rapid and cost-effective spectral sensing device for on-site inspection of the fish fillet mislabeling, which can be used for authentication of the fish fillets and other related food products by the seafood industry and regulatory agencies.

Technical Abstract: Seafood mislabeling rates of approximately 20% have been reported globally. Traditional methods for fish species identification, such as DNA analysis and polymerase chain reaction (PCR), are expensive and time-consuming, and require skilled technicians and specialized equipment. An innovative solution combining spectroscopy and machine learning presents a promising approach to overcome these challenges. In our study, we took a comprehensive approach by considering a total of 43 different fish species and employing three modes of spectroscopy: fluorescence (Fluor), and reflectance in the visible near infra-red (VNIR), and short-wave near infra-red (SWIR). To achieve higher accuracies, we developed a novel machine learning framework where groups of similar fish types were identified, and specialized classifiers were trained for each group. The incorporation of global (single AI for all species) and dispute classification models created a hierarchical decision process, yielding higher performances. For Fluor, VNIR, and SWIR, accuracies increased from 80%, 75%, and 49% to 83%, 81%, and 58%, respectively. Furthermore, certain species witnessed remarkable performance enhancements of up to 40% in single mode identification. The fusion of all three spectroscopic modes further boosted the performance of the best single mode, averaged over all species, by 9%. Fish species mislabeling not only poses health-related risks due to contaminants, toxins, and allergens that could be life-threatening, but also gives rise to economic and environmental hazards, and loss of nutritional benefits. Our proposed method can detect fish fraud as a real-time alternative to DNA barcoding and other standard methods. The dispute model framework can be used in many applications, but its performance was validated with fish identification within this study.