An automated apple harvesting robot—from system design to field evaluation

Authors: ZHANG, KAIXIANG - Michigan State University; LAMMERS, KYLE - Michigan State University; CHU, PENGYU - Michigan State University; LI, ZHAOJIAN - Michigan State University; Lu, Renfu

Submitted to: Journal of Field Robotics
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/28/2023
Publication Date: 11/14/2023
Citation: Zhang, K., Lammers, K., Chu, P., Li, Z., Lu, R. 2023. An automated apple harvesting robot—from system design to field evaluation. Journal of Field Robotics. 1-17. https://doi.org/10.1002/rob.22268.
DOI: https://doi.org/10.1002/rob.22268

Interpretive Summary: Growing labor shortage and rising cost in harvest pose a serious threat to the profitability and long-term sustainability of the U.S. apple and other special crop industries. Harvest automation is thus urgently needed to address these critical issues. A team of researchers from ARS and Michigan State University in East Lansing, Michigan recently developed a new robotic apple harvesting system. This system mainly consists of a specially designed perception component, a vacuum-based fruit picking component, and a fruit dropping/catching component for receiving and transporting picked fruits. Software algorithms were developed for detecting target fruits and determining their spatial locations. Planning and control algorithms were developed to coordinate the hardware components for efficient, automated harvesting of apples in challenging orchard environments. Field tests were conducted in two apple orchards with different tree architectures. The robot achieved more than 90% fruit detection rates and fruit localization accuracies of less than 7 mm. It was able to pick 82.4% of apples in an orchard with young and well-pruned trees, and 65.2% for an older orchard with dense, clustered tree canopies. The robot reached an average picking speed of 6 seconds per fruit. With further improvements, the technology is promising for commercial harvesting of apples and other tree fruit crops.

Technical Abstract: Decreased availability and rising cost in labor poses a serious threat to the long-term profitability and sustainability of the apple industry in the United States and many other countries. Harvest automation is thus urgently needed. In this paper, we present the unified system design and field evaluation of a new apple harvesting robot. The robot is mainly composed of a specially designed perception component, a four-degree-of-freedom manipulator, an improved vacuum-based soft end-effector, and a dropping/catching component to receive and transport picked fruits. Software algorithms are developed to enable synergistic coordination of the hardware components for efficient, automated harvesting of apples in challenging orchard environments. Specifically, by integrating modified triangulation and image processing and analysis algorithms, a novel perception strategy is developed to achieve robust apple detection and precise localization. Improved planning and control algorithms are developed to guide the robot to the target positions. The performance of the robotic system was evaluated through field tests in two apple orchards with different tree architectures and foliage conditions. In the orchard where trees were young and well-pruned, the robot achieved 82.4% successful harvesting rate. In a second, older orchard with dense, clustered branches and foliage, the robot had 65.2% successful rate. The average cycle time to harvest a fruit was approximately 6's, which included software algorithm processing and hardware execution. Moreover, through an in-depth analysis of the obtained results, limitations and planned future works are discussed.