Development and evaluation of a dual-arm robotic apple harvesting system

Authors: LAMMERS, KYLE - Michigan State University; ZHANG, KAIXIANG - Michigan State University; ZHU, KEYI - Michigan State University; CHU, PENGYU - Michigan State University; LI, ZHAOJIAN - Michigan State University; Lu, Renfu

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/23/2024
Publication Date: 11/14/2024
Citation: Lammers, K., Zhang, K., Zhu, K., Chu, P., Li, Z., Lu, R. 2024. Development and evaluation of a dual-arm robotic apple harvesting system. Computers and Electronics in Agriculture. 227. Article 109586. https://doi.org/10.1016/j.compag.2024.109586.
DOI: https://doi.org/10.1016/j.compag.2024.109586

Interpretive Summary: Harvesting labor is the single largest cost in apple production in the U.S. Increased cost and growing shortage of labor has forced the apple industry to seek automated harvesting solutions. Despite considerable progress in recent years, the existing robotic harvesting systems still fall short of performance expectations, lacking robustness and proving inefficient or overly complex for practical commercial deployment. Built on the promising performance of our 1-arm harvesting robot, we developed a dual-arm robotic harvesting system, which is mainly composed of a common perception component for fruit detection and localization, two four-degrees-of-freedom robot arms coupled with soft end effectors or grippers and a centralized vacuum system for fruit attachment and detachment, and a fruit handling and bin filling component for receiving and transporting harvested fruits to the bin. An advanced planning algorithm was developed to coordinate the picking operations of the two arms which share the same vacuum system for efficient fruit picking. Field trials in 2023 harvest season showed that the dual-arm robot improved the harvest efficiency by up to 34%, compared to the 1-arm robot developed earlier, and it achieved 60% successful picking rate in orchards with complex tree canopies. The new dual-arm robotic system is compact in design and dexterous in movement, and with further improvements in hardware and software, it holds great potential for providing a commercially viable harvesting automation solution for the apple industry.

Technical Abstract: Harvesting labor is the single largest cost in apple production in the U.S. Increased cost and growing shortage of labor has forced the apple industry to seek automated harvesting solutions. Despite considerable progress in recent years, the existing robotic harvesting systems still fall short of performance expectations, lacking robustness and proving inefficient or overly complex for practical commercial deployment. In this paper, we present the development and evaluation of a new dual-arm robotic apple harvesting system. The system hardware mainly consists of a perception component, two four-degree-of-freedom manipulators, a centralized vacuum system, and a fruit handling and bin filling component designed for the collection and transportation of picked fruits. Synergistic functionalities for automated apple harvesting were achieved through the development of software algorithms. In particular, an updated perception system based on dual-laser scanning was proposed to enable sequential localization of apples for the dual-arm robotic system. A sophisticated planning scheme was devised to coordinate the movement of the two manipulators, allowing them to approach the fruit effectively and share a centralized vacuum system for efficient fruit detachment. The robotic system has been evaluated through field trials in a challenging apple orchard with complex, dense canopy, and it achieved 60% successful picking rate. The dual-arm coordination algorithm resulted in 9% to 34% harvest time improvements, compared to the 1-arm robotic system design. The new dual-arm robotic system is compact in design and dexterous in movement, and with further improvements in hardware and software, it holds great potential for providing a commercially viable harvesting automation solution for the apple industry.