Characterizing hyperspectral microscope imagery for classification of blueberry firmness with deep learning methods

Authors: Park, Bosoon; Shin, Tae-Sung; CHO, JEONG-SEOK - Korea Food Research Institute; LIM, JEONG-HO - Korea Food Research Institute; PARK, KI-JAE - Korea Food Research Institute

Submitted to: Agronomy Journal
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 12/27/2021
Publication Date: 12/30/2021
Citation: Park, B., Shin, T., Cho, J., Lim, J., Park, K. 2021. Characterizing hyperspectral microscope imagery for classification of blueberry firmness with deep learning methods. Agronomy Journal. https://doi.org/10.3390/agronomy12010085.
DOI: https://doi.org/10.3390/agronomy12010085

Interpretive Summary: Blueberry is one of the most popular and healthy fruits, because regular intake of blueberries offers a low-calorie meal and reduces the risk of cardiovascular disease and type 2 diabetes while boosting neuroprotection. The United States is the top blueberry growing country globally and produced 680 million pounds of blueberries worth $758 million in 2019. However, blueberry softening has been a significant problem in its postharvest shelf-life quality control. As the fruit is highly perishable, retailers should reject a large volume of blueberries with firmness lower than their standards. Since several studies concluded that the root cause could be moisture loss primarily through picking scar, a practical resolution against the problem will be maintaining proper storage humidity and cool storage temperature. There have been researches such as light microscopy and cold stage scanning electron microscopy to explain the softening with microstructural changes of blueberries. In this study, we examined spectral and spatial features of the microstructures together for blueberry firmness classification using hyperspectral microscope imaging (HMI) and deep learning (DL) methods. The main goal of this study was to develop HMI methods for understanding blueberry softening. Specific objectives were to acquire hyperspectral images of cells in blueberries, analyze the hypercubes to find spatial and spectral relationships between blueberry firmness and microstructural changes, and finally develop classification models with DL methods.

Technical Abstract: Firmness is an important quality indicator of blueberry. Firmness loss (or softening) of postharvest blueberry has posed a challenge in its shelf-life quality control and can be delineated with its microstructural changes. To investigate spatial and spectral characteristics of microstructures based on firmness, hyperspectral microscope imaging (HMI) was employed for this study. The mesocarp area with 20x magnification of blueberry was selectively imaged with a fabry-perot interferometer HMI system of 400-1000nm wavelengths, resulting in 281 hypercubes of parenchyma cells in a resolution of 968 x 608 x 300 pixels. After properly processing each hypercube of parenchyma cells in a blueberry, the cell image with different firmness was examined based on parenchyma cell shape, cell wall segment, cell-to-cell adhesion, and size of intercellular spaces. Spectral cell characteristics of firmness were also sought based on the spectral profile of cell walls with different image preprocessing methods. The study found that softer blueberry (1.96N- 3.92N) had more irregular cell shapes, lost cell-to-cell adhesion, loosened and round cell wall segments, large intercellular spaces, and cell wall colors more red shift in spectral images than firm blueberry (6.86N-8.83N). Even though berry-to-berry (or image-to-image) variations of the characteristics turned out large, the deep learning model with spatial and spectral features of blueberry cells demonstrated the potential for blueberry firmness classification with Matthew’s correlation coefficient of 73.4% and accuracy of 85% for test set.