Deep feature representation with stacked sparse auto-encoder and convolutional neural network for hyperspectral imaging-based detection of cucumber defects

Authors: LIU, ZIYI - Zhejiang University; HE, YONG - Zhejiang University; CEN, HAIYAN - Zhejiang University; Lu, Renfu

Submitted to: Transactions of the ASABE
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 12/16/2017
Publication Date: 3/31/2018
Citation: Liu, Z., He, Y., Cen, H., Lu, R. 2018. Deep feature representation with stacked sparse auto-encoder and convolutional neural network for hyperspectral imaging-based detection of cucumber defects. Transactions of the ASABE. 61(2):425-436.

Interpretive Summary: Hyperspectral imaging, an emerging optical imaging technique, enables to acquire three-dimensional image cubes (two represents the spatial information and one the spectral) from an object or a scene. The technique can provide superior capabilities in detecting properties and quality attributes of food and agricultural products. However, large amounts of image data acquired by the hyperspectral imaging system present real challenges for the technique to be used for rapid food quality and safety inspection. This paper reports on a new classification method, based on convolutional neural network, an artificial learning technique, for hyperspectral imaging-based detection of surface and subsurface defects in pickling cucumbers. Hyperspectral reflectance images in the visible region of 500-675 nm and transmittance images in the region of 675-1,000 nm were acquired for normal and defective pickling cucumbers at two conveyor speeds of 85 and 165 mm/s, using a hyperspectral imaging system developed by the USDA/ARS lab at East Lansing, Michigan. The test cucumbers were classified into six classes, i.e., normal, watery, split/hallow, shrivel, dirt/sand, and gouge/rot (the last two were considered as surface defect). New computer algorithms for different artificial learning methods were developed and compared for classifying the cucumbers. Results showed that the best algorithm was able to achieve 91.1% and 88.3% classification accuracies for six-class classification for the conveyor speeds of 85 mm/s and 165 mm/s, respectively. The algorithm is also fast, thus having the potential for online implementation with hyperspectral imaging technique for quality sorting of pickling cucumbers.

Technical Abstract: It is challenging to achieve rapid and accurate processing of large amounts of hyperspectral image data. This research was aimed to develop a novel classification method by employing deep feature representation with the stacked sparse auto-encoder (SSAE) and the SSAE combined with convolutional neural network (CNN-SSAE) learning for hyperspectral imaging-based defects detection of pickling cucumbers. Hyperspectral reflectance (500-675 nm) and transmittance (675-1,000 nm) images for normal and defective pickling cucumbers were acquired, using an in-house developed hyperspectral imaging system running at two conveyor speeds of 85 and 165 mm/s. The hyperspectral images were preprocessed, and then used in computing mean spectra for further feature representation and developing a region-based model with visual feature matching. An SSAE model was developed to learn the feature representation and to perform five-class (normal, watery, split/hallow, shrivel, and surface defect) classification using the softmax function. To deal with a more complicated task for different types of surface defects (i.e., dirt/sand and gouge/rot classes) in six-class classification, a CNN-SSAE system was developed by performing the classification based on the localization of the surface defective regions on the samples. The results showed that without CNN localization, the SSAE model gave the five-class classification accuracies of 85.6% and 78.3% for the conveyor speeds of 85 mm/s and 165 mm/s, respectively. The CNN-SSAE system improved the classification performance, compared with the SSAE, with the overall accuracies of 91.1% and 88.3% for six-class classification at the conveyor speeds of 85 mm/s and 165 mm/s, respectively. Additionally, the average running time of the CNN-SSAE system for each sample was less than 13.5 ms, showing the potential for implementing it in an automated online inspection system for cucumber sorting and grading.