Location: Sugarbeet and Bean Research
2018 Annual Report
Objectives
1. Enable new commercial imaging and spectroscopic methods to determine fruit and vegetable internal quality and maturity.
2. Enable new, economical, accurate, automated, in-orchard methods for commercial apple quality tracing and grading.
Approach
1) Improvements will be made in the method and technique for measuring the optical absorption and scattering properties of horticultural and food products that may be considered homogeneous or layered in the tissue structure. Factors affecting the optical property measurements, including light source configuration, the geometry and surface roughness of samples, and inverse algorithm, will be evaluated by using numerical simulation (e.g., Monte Carlo and finite element) and experiment for phantom tissues and real samples, so as to improve the measurement accuracy and reproducibility. New methods and algorithms will be developed for accurate measurement of the optical properties of layered food products. Experiments will be carried out to measure the optical properties of horticultural products like apple, orange, and pickling vegetable. The measured optical properties will be used to predict quality and condition of the products.
2) Research will be conducted on the development of a new sensing technique for more effective quality evaluation of horticultural and food products. Specifically, different light illumination and image acquisition methods will be investigated for detecting properties and characteristics of plant tissues at different depths. Light penetration characteristics in plant tissues will be studied through computer simulations and experimental tests. Image processing algorithms will be developed for extraction of important features from the reflectance images to characterize internal quality (including defect) of fruit and vegetable. A new sensing system that incorporates conventional imaging or hyperspectral imaging technique with the optimal lighting configuration and dedicated image processing algorithms will be assembled and evaluated for real time detection of internal quality for fruit and vegetable.
3) Research will be conducted to develop cost effective, automated in-orchard apple sorting technology. New and improved functions will be developed and incorporated into the machine vision system to allow more effective sorting and grading of different varieties of apple for color, size and defect. More efficient and reliable sorter and bin filler designs in modular format will be proposed, assembled and tested in laboratory and field. A new method for handling individual fruit bins will be proposed and implemented so that no fruit bins would be left half-filled and the possible down time for the harvest crew resulting from the bin handling would be eliminated or minimized. The new and improved sorting system will be integrated with either a self-propelled or tractor-driven harvest aid platform for automatically sorting and grading apples into two or three quality grades as well as enhancing harvest efficiency and worker safety. Laboratory and field tests and demonstrations will be carried out, in close collaboration with commercial equipment manufacturer, growers, and extension personnel, to facilitate the development and transfer of the technology to the end user.
Progress Report
Spatially resolved spectroscopy (SRS) allows acquiring spectral information at multiple spatial-resolved distances from the illumination of a focused light source, thus enabling better assessment of quality and condition of fruit and other food products. Using the recently developed SRS system, spectral data were collected from tomatoes of different maturity stages, along with conventional single-point spectroscopy (SPS) covering the visible and near-infrared spectral region (400-1,700 nm). Calibration models were then developed using individual SRS spectra and their combinations, as well as the computed optical absorption and scattering spectra, for assessing the firmness, soluble solids content and pH of tomatoes. Results showed that prediction of the quality parameters for tomatoes varied with the spatial position of acquired spectra. Overall, combination of multiple SRS spectra resulted in more consistent, better prediction of quality parameters, and SRS was also superior to SPS in assessing these quality parameters.
Conventional SPS is restricted in its ability of detecting internal defects that are distributed in small, discrete regions inside a fruit. A multichannel hyperspectral imaging system in semi-transmittance mode was constructed for detecting internally defective apples. The new multichannel system enables simultaneous acquisition of six spectra covering different sections of fruit in 360 degrees, thus having potential for more effectively detecting localized defects inside apple fruit. Experiments were conducted on detecting internal defect of ‘Honeycrisp’ apples using the multichannel system. Multi-spectra were acquired for each apple in three different orientations. Classification models were developed for individual spectra and their averaged spectra for each of the three fruit orientations. Results showed that defect detection results varied with the position of acquired spectra and fruit orientation. Combination of the six spectra overall resulted in better results for classification of defective and normal apples, with the overall accuracies of as high as 96%.
Spatial-frequency domain imaging (SFDI) provides a new means for measuring optical absorption and scattering properties of fruit and other food products, which are directly related to the chemical composition and physical properties of the products. Food products are generally heterogeneous in their structural and optical properties. For instance, many fruits are composed of skin and flesh, each of which has different properties. Hence it is desirable or necessary to be able to measure optical properties of each layer in order to better characterize the properties of samples. Measurement of optical properties for two-layer samples, however, presents many technical challenges due to the complexity in the mathematical model and a large number of optical parameters to be estimated. The conventional method with SFDI would estimate all four optical parameters (two for each layer) simultaneously, which often results in large, unacceptable estimation errors. To improve estimation accuracy for optical parameters with SFDI, a new stepwise method was proposed for estimating the optical parameters of two-layer samples. With this method, the optical properties of each layer are estimated separately and in multiple steps. Results for simulation samples demonstrated that the stepwise method greatly improved the accuracy of estimating optical properties of both layers, compared to the conventional one-step method. Moreover, the research also determined the constraining conditions on the appropriate range of top layer thicknesses, within which the optical properties of each layer can be estimated with acceptable accuracies.
Good progress has been made on the development of structured-illumination reflectance imaging (SIRI) technique as a new modality for enhanced defect detection of fruit. A fast image preprocessing algorithm, called bi-dimensional empirical mode (BEMD), was developed for removing noise and artifacts in the demodulated SIRI images. Both simulation and experiment results showed that the proposed image preprocessing algorithm was effective in enhancing the features of SIRI images. The proposed BEMD method was further implemented in conjunction with three machine learning algorithms (i.e., support vector machine, random forest, and convolutional neural network) for processing and analyzing SIRI images to detect both surface and subsurface defects of ‘Delicious’ and ‘Golden Delicious’ apples. Superior classification results (up to 98% accuracy) were obtained with the convolutional neural network algorithm. Further experiments also were conducted on using SIRI to detect bruises in peaches and for early detection of disease infection in peaches. Analysis of the acquired images showed that SIRI performed consistently better than conventional uniform-illumination imaging technique for early detection of disease symptoms in peaches. These studies showed that SIRI coupled with appropriate image processing algorithms, can provide an effective means for enhanced detection of defects on fruit. Furthermore, an initial exploration was conducted of two instrumentation configurations for real-time acquisition of SIRI images from moving samples. Preliminary analysis showed that it is feasible to implement SIRI for fast, real-time image acquisitions from moving fruit, which opens an opportunity for practical use of the technique for quality evaluation of horticultural and food products.
Progress has been made on the development of apple harvest and infield sorting technology. An improved version bin filler was constructed, tested and evaluated in both laboratory and field conditions. The performance of the bin filler in terms of fruit distributions in the bin was quantified using a 3-D depth imaging technique. Laboratory and field tests showed that the new bin filler performed much better than the previous version, in terms of fruit bruising and fruit distributions during the filling process. The 3-D depth imaging technique provides a new, quantitative means for evaluating the performance of bin fillers. In addition, laboratory tests were also conducted to evaluate the performance of the sorting system, in terms of singulating and rotating each fruit for imaging as well as sorting accuracies under different sorting speeds. During the 2017 harvest season, the apple harvest and infield sorting machine was tested in a commercial orchard. While the machine has met initial expectations, it also showed several areas needing improvement, which include improving the sorting system for more accurate, reliable sorting at a higher speed of up to 9 fruits/s, further improvement on the design and construction of bin fillers and their control/sensing system for better, more reliable operation, and improving the sensors and computer program for the automatic bin handling system.
Accomplishments
1. Development of new apple bin filling technology. Infield sorting or removal of inferior fruit at the time of harvest will help U.S. apple growers achieve cost savings in postharvest storage and packing, and improve postharvest disease/pest and inventory management. Bin filling is crucial in the successful development of apple harvest and infield sorting technology. After evaluating and comparing different commercial and research bin filler designs, ARS researchers at East Lansing, Michigan developed a new bin filler system for integration with the apple harvest. With several innovative features, this new bin filler is compact and simple in design and fully automated with sensors and controls. Laboratory and field tests demonstrated that the new bin filler is able to distribute apples in the bin evenly, while causing minimal bruising damage to the fruit, which is well within the industry’s requirement. The new bin filler has been integrated with the apple harvest and automated infield sorting machine and it can also be used with other commercial harvest machines, so as to help the apple industry enhance harvest productivity and reduce postharvest handling cost.
2. Development of image processing algorithms for enhanced defect detection of fruit. Detection of fruit defects, both surface and subsurface, is still challenging because there are a large variety of defects, some of which are difficult to identify by using current computer vision systems. A new structured-illumination reflectance imaging (SIRI) technique was developed recently for detection of surface and subsurface defects for apples, because it provides higher spatial resolutions and greater image contrasts and allows better control of light penetration in the fruit. However, accurate detection of defects by SIRI also depends on the development of effective image processing and classification algorithms for identifying defective tissues from normal ones. ARS researchers at East Lansing, Michigan developed a fast image preprocessing algorithm called bi-dimensional empirical mode decomposition (BEMD), for removal of noise and artifacts in the SIRI images. Simulation and experimental results showed that SIRI coupled with BEMD and a machine learning algorithm (i.e., convolutional neural network) significantly improved the accuracy of detecting surface and subsurface defects of apples and also enabled effective detection of early disease symptoms in peaches. Implementation of SIRI with these image processing algorithms will offer a new imaging modality for enhanced quality inspection of horticultural and food products, which can help the horticultural industry deliver better quality products to the consumer.
Review Publications
Lu, Y., Lu, R. 2018. Structured-illumination reflectance imaging coupled with phase analysis techniques for surface profiling of apples. Journal of Food Engineering. 232:11-20.
Huang, Y., Lu, R., Xu, Y., Chen, K. 2018. Prediction of tomato firmness using a spatially-resolved multichannel hyperspectral imaging probe. Postharvest Biology and Technology. 140:18-26.
Huang, Y., Hu, D., Lu, R., Chen, K. 2018. Quality assessment of tomato quality by optical absorption and scattering properties. Postharvest Biology and Technology. 143:78-85.
Huang, Y., Lu, R., Chen, K. 2018. Assessment of tomato soluble solids content and pH by spatially-resolved and conventional Vis/NIR spectroscopy. Journal of Food Engineering. 236:19-28.
Pothula, A., Zhang, Z., Lu, R. 2018. Design features and bruise damage evaluation of an apple harvest and infield sorting machine. Transactions of the ASABE. 61(3):1135-1144.
Li, R., Lu, Y., Lu, R. 2018. Structured illumination reflectance imaging for enhanced detection of subsurface tissue bruising in apples. Transactions of the ASABE. 61(3):809-819.
Lu, Y., Lu, R. 2018. Fast bi-dimensional empirical mode decomposition as an image enhancement technique for fruit defect detection. Computers and Electronics in Agriculture. 152:314-323.
Zhang, Z., Pothula, A., Lu, R. 2017. Development and preliminary evaluation of a new bin filler for apple harvesting and infield sorting. Transactions of the ASABE. 60(6):1839-1849.
Zhang, Z., Pothula, A.K., Lu, R. 2017. Economic evaluation of apple harvest and in-field sorting technology. Transactions of the ASABE. 60(5):1537-1550.
Lu, Y., Lu, R. 2017. Non-destructive defect detection of apples by spectroscopic and imaging technologies: A review. Transactions of the ASABE. 60(5):1765-1790.
Hu, D., Lu, R., Ying, Y. 2018. A two-step parameter optimization algorithm for improving estimation of optical properties using spatial frequency domain imaging. Journal of Quantitative Spectroscopy & Radiative Transfer. 207:32-40.
Mendoza, F., Cichy, K.A., Sprague, C., Goffnet, A., Lu, R., Kelly, J.D. 2017. Prediction of canned black bean texture (Phaseolus vulgaris L.) from intact dry seeds using visible/near-infrared spectroscopy and hyperspectral imaging data. Journal of the Science of Food and Agriculture. 98(1):283-290.
Lu, Y., Lu, R. 2017. Development of a multispectral structured-illumination reflectance imaging (SIRI) system and its application to bruise detection of apples. Transactions of the ASABE. 60(4):1379-1389.
Liu, Z., He, Y., Cen, H., Lu, R. 2018. Deep feature representation with stacked sparse auto-encoder and convolutional neural network for hyperspectral imaging-based detection of cucumber defects. Transactions of the ASABE. 61(2):425-436.
