Location: Sugarbeet and Bean Research
2020 Annual Report
Objectives
1. Enable new commercial imaging and spectroscopic methods to determine fruit and vegetable internal quality and maturity.
2. Enable new, economical, accurate, automated, in-orchard methods for commercial apple quality tracing and grading.
Approach
1) Improvements will be made in the method and technique for measuring the optical absorption and scattering properties of horticultural and food products that may be considered homogeneous or layered in the tissue structure. Factors affecting the optical property measurements, including light source configuration, the geometry and surface roughness of samples, and inverse algorithm, will be evaluated by using numerical simulation (e.g., Monte Carlo and finite element) and experiment for phantom tissues and real samples, so as to improve the measurement accuracy and reproducibility. New methods and algorithms will be developed for accurate measurement of the optical properties of layered food products. Experiments will be carried out to measure the optical properties of horticultural products like apple, orange, and pickling vegetable. The measured optical properties will be used to predict quality and condition of the products.
2) Research will be conducted on the development of a new sensing technique for more effective quality evaluation of horticultural and food products. Specifically, different light illumination and image acquisition methods will be investigated for detecting properties and characteristics of plant tissues at different depths. Light penetration characteristics in plant tissues will be studied through computer simulations and experimental tests. Image processing algorithms will be developed for extraction of important features from the reflectance images to characterize internal quality (including defect) of fruit and vegetable. A new sensing system that incorporates conventional imaging or hyperspectral imaging technique with the optimal lighting configuration and dedicated image processing algorithms will be assembled and evaluated for real time detection of internal quality for fruit and vegetable.
3) Research will be conducted to develop cost effective, automated in-orchard apple sorting technology. New and improved functions will be developed and incorporated into the machine vision system to allow more effective sorting and grading of different varieties of apple for color, size and defect. More efficient and reliable sorter and bin filler designs in modular format will be proposed, assembled and tested in laboratory and field. A new method for handling individual fruit bins will be proposed and implemented so that no fruit bins would be left half-filled and the possible down time for the harvest crew resulting from the bin handling would be eliminated or minimized. The new and improved sorting system will be integrated with either a self-propelled or tractor-driven harvest aid platform for automatically sorting and grading apples into two or three quality grades as well as enhancing harvest efficiency and worker safety. Laboratory and field tests and demonstrations will be carried out, in close collaboration with commercial equipment manufacturer, growers, and extension personnel, to facilitate the development and transfer of the technology to the end user.
Progress Report
Objective 1a: Measurement of optical absorption and scattering properties can provide a new means for assessing postharvest quality of horticultural products. The hyperspectral imaging-based spatially resolved technique developed by our laboratory in previous research enables nondestructive, fast measurement of the optical absorption and scattering properties of horticultural and food products. Research was conducted on improving the technique for more accurate measurement of optical properties of food products with layered structures (i.e., skin and flesh). Computer simulations and experiments were conducted to determine the optimal system configurations and inverse algorithms for optical property estimations. New, improved algorithms were developed, which performed significantly better compared to the conventional method for measuring horticultural and food products of homogenous and layered structures.
A new spatially resolved spectroscopy (SRS) system using a multi-channel hyperspectral imaging sensor as a platform, was developed for simultaneous acquisition of 30 spatially resolved spectra over the 550-1,650 nm for food samples. Mathematical methods were developed for estimating the optical absorption and scattering properties from the acquired spectra and for prediction of postharvest quality of tomatoes and other food products. The technique enables better quality assessment of horticultural and food products.
Spatial-frequency domain imaging (SFDI) is an emerging technique for measurement and spacial mapping of optical properties of biological tissues. Great challenges, however, still exist in accurate measurement of optical properties of food products using the SFDI technique. New, improved approaches to the SFDI data analysis were proposed for estimation of optical properties for both homogenous and layered food products. Computer simulations, followed with experimental validations, were conducted to determine the optimal mathematical procedures and system parameters for optical property estimations. The proposed new methods and algorithms significantly improved the accuracy of the SFDI technique for measurement of the optical absorption and scattering of food products.
Objective 1b: Machine vision technology is widely used for defects inspection of horticultural products, but its performance is still short of meeting industry expectations. Research was conducted on the development of a new imaging modality with substantially improved capabilities for detecting surface and subsurface defects of horticultural products. Two new structured-illumination reflectance imaging (SIRI) systems were built for acquiring images from food samples under illumination of sinusoidal patterns, compared to uniform illumination commonly used for conventional imaging systems. The first system allows acquiring broadband images in the visible spectral region, while the second system enables acquiring spectral images over the spectral region of 600-1,000 nm. Studies showed that SIRI can reveal some hidden features of horticultural products, which are difficult to ascertain by conventional imaging techniques, and it exhibited superior performance in detecting such defects as subsurface bruising in apples.
Two new demodulation methods were proposed, which is a critical procedure in implementing the SIRI technique, and they only require two patterned images instead of three by the conventional demodulation method. The methods enable faster acquisition of patterned images and would facilitate the implementation of the SIRI technique for online applications. In addition, a new image enhancement technique, called bi-dimensional empirical mode decomposition (BEMD), was proposed to remove artifacts in the demodulated images for improving subsequent image processing and defect detection. Light attenuation or penetration in biological tissues is related to the spatial frequency of illumination. By utilizing this important feature, our study demonstrated that it is feasible to acquire patterned images from food samples subjected to composite patterns of illumination with different spatial frequencies. This approach can be useful for more effective, simultaneous detection of different types of surface and/or subsurface defects.
Studies were conducted, using the SIRI systems, coupled with the new image processing methods, on detection of surface and subsurface defects (i.e., bruise, defects due to mechanical and/or physiological disorder, early disease infection, etc.) for apples, peaches, pickling cucumbers, and tomatoes. In all the studies, SIRI exhibited superior results over conventional imaging technique under uniform illumination.
Finally, a general-purpose, graphical user interfacing program, called siriTool, was developed for automatic image acquisition, demodulation and enhancement, object segmentation, image features extraction and selection, and classification. The program incorporates the two image demodulation algorithms, a new image segmentation technique, and the BEMD. Furthermore, it also enables using different methods for object classifications. An experimental study was carried out on the detection of yellowish subsurface spot defects in pickling cucumbers, which are difficult to ascertain by conventional imaging technique. siriTool was able to achieve 98% or higher classification accuracies, which were significantly better than that by using conventional imaging technique.
Objective 2: Harvest and postharvest handling (including storage, sorting, grading and packaging) are labor intensive operations, which account for about half of the total production cost for U.S. apple growers. A study was conducted of the economic benefits from adopting harvest assisting and infield sorting technologies. Results showed that significant savings in postharvest handling can be achieved if high quality apples for the fresh market can be segregated from inferior fruit that are only suitable for making juice or processed products at the time of harvesting in the orchard. Furthermore, greater economic benefits can be accrued by integration of infield sorting technology with a harvest assisting machinery system. While system cost is of great concern, there are several major technological challenges for automatic infield sorting, which include, but are not limited to, the development of a new machine vision inspection system that is compact, efficient and robust for orchard use, automatic bin filling technology for handling graded fruit in bins, and an automatic bin handling system. After several iterations in prototyping and testing, a cost-effective and compact machine vision-based system was constructed, and it consists of a digital color camera enabling acquisition of 30 images per second for inspecting fruit size and color or shape, an innovative, compact conveying module for fruit singulation, rotation and transporting, and a novel fruit sorting mechanism. Two versions of fruit sorting mechanisms were developed. The first version allows fruit to be sorted into two or three quality grades (i.e., fresh market, processing and cull or juice) at a rate up to 6 fruit per second, while the second, improved version, which is more compact and robust, sorts apples into two quality grades (i.e., fresh market and cull or processing) at a rate up to 12 fruit per second.
In collaboration with a commercial horticultural equipment manufacturer, ARS researchers at East Lansing, Michigan, designed and constructed a self-propelled apple harvest and infield sorting machine prototype. In addition to the innovative machine vision sorting module, this machine also has several other major innovations, which include automatic bin fillers for handling harvested fruit in bins, a computer controlled bin handling system, and adjustable harvest platforms with a special fruit receiving design for improving harvest efficiency and worker ergonomics. The bin filler is mainly composed of two pairs of foam rollers that allow sorted apples to drop freely for a maximum height of 1.5 m, a rotary fruit distributor, and a sensor and an onboard microprocessor for automatic monitoring and control of the fruit filling process. The computer-controlled hydraulic system minimizes the down times for fruit pickers caused by the handling of empty and full bins, thus enhancing overall harvest efficiency. Laboratory and field tests in a commercial orchard for three harvest seasons showed that the machine vision-based sorting system has fully met the performance expectations in terms of sorting accuracy and speed, and the bin fillers handled graded fruit in bins gently and evenly with minimum bruising damage to the harvested fruit.
In further support for Objective 2, a study was also conducted to evaluate three commercial harvest assisting methods (i.e., the traditional ladder and bucket method, a commercial vacuum-based harvester, and a commercial harvest platform system with use of picking buckets), so as to identify key technological gaps for these systems that need be addressed in future innovation and improvement. A time and motion method was used for analyzing the entire harvest process by fruit pickers under the three methods in commercial orchards in Michigan. The study showed that these harvest methods are still low in harvest efficiency due to lack of automation and that technological innovations should be focused on reducing non-picking activities for the traditional harvest method and development of new fruit receiving and bin filling technologies for the harvest-assisting platform systems.
Accomplishments
1. Automated image processing for enhanced defect detection of horticultural products. Currently, machine vision technology is widely used for automated inspection of horticultural products, but its capability of detecting defects, both surface and subsurface, still falls short of industry expectations. An ARS-led team of researchers at East Lansing, Michigan, first developed a new imaging technique based on the concept of structured lighting, which demonstrated superior capabilities of detecting surface and subsurface defects of horticultural products. A user-friendly computer program incorporating several new image processing algorithms was developed for automated image acquisition, enhancement and processing, and defect classification, a critical step in implementation of this new technique for online commercial use. The program provides an effective tool for fast, automated processing of images acquired under structured lighting as well as under conventional uniform lighting configurations. It would enable implementation of the new imaging technique for enhanced quality evaluation of horticultural products.
Review Publications
Hu, D., Lu, R., Huang, Y., Ying, Y., Fu, X. 2020. Effects of optical variables in a single integrating sphere system on estimation of scattering properties of turbid media. Biosystems Engineering. 194:82-98. https://doi.org/10.1016/j.biosystemseng.2020.03.012.
Lu, Y., Lu, R. 2019. Enhancing chlorophyll fluorescence imaging under sttructured illumination with automatic vignetting correction for detection of chilling injury in cucumbers. Computers and Electronics in Agriculture. 168:105145. https://doi.org/10.1016/j.compag.2019.105145.
Huang, Y., Lu, R., Chen, K. 2019. Detection of internal defect of apples by a multichannel Vis/NIR spectroscopic system. Postharvest Biology and Technology. 161:111065. https://doi.org/10.1016/j.postharvbio.2019.111065.
Hu, D., Lu, R., Ying, Y. 2020. Spatial frequency domain imaging coupled with frequency optimization for estimating optical properties of two-layered food and agricultural products. Journal of Food Engineering. 277:109909. https://doi.org/10.1016/j.jfoodeng.2020.109909.
Sun, Y., Lu, R., Wang, X. 2020. Evaluation of fungal infection in peaches based on optical and microstructural properties. Postharvest Biology and Technology. 165:111181. https://doi.org/10.1016/j.postharvbio.2020.111181.
Lu, R., Van Beers, R., Saeys, W., Li, C., Cen, H. 2019. Measurement of optical properties of fruits and vegetables: A review. Postharvest Biology and Technology. 159:111003. https://doi.org/10.1016/j.postharvbio.2019.111003.
