Skip to main content
ARS Home » Northeast Area » Kearneysville, West Virginia » Appalachian Fruit Research Laboratory » Innovative Fruit Production, Improvement, and Protection » Research » Publications at this Location » Publication #394759
Research Project: Integrated Production and Automation Systems for Temperate Fruit Crops

Location: Innovative Fruit Production, Improvement, and Protection

Title: Self-supervised learning for panoptic segmentation of multiple fruit flower species

Author
item SIDDIQUE, ABUBAKAR - Marquette University
item Tabb, Amy
item MEDEIROS, HENRY - University Of Florida

Submitted to: International of Electrical and Electronics Engineers (IEEE) Robotics and Automation Letters
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 9/29/2022
Publication Date: 10/25/2022
Citation: Siddique, A., Tabb, A., Medeiros, H. 2022. Self-supervised learning for panoptic segmentation of multiple fruit flower species. International of Electrical and Electronics Engineers (IEEE) Robotics and Automation Letters. 7(4):12387-12394. https://doi.org/10.1109/LRA.2022.3217000.
DOI: https://doi.org/10.1109/LRA.2022.3217000

Interpretive Summary: We explored whether improvements could be made to autonomous fruit flower detection systems that use images. One of the issues with image systems based on neural networks is the amount of labeled data that is needed, meaning parts of images where flower versus background is labeled by a person. We found that a new technique that “self-learns”, meaning without a lot of labeled data, worked better on detecting fruit flowers in images than past works. This is important as precision agriculture depends on correct, autonomous measurements of traits of interest.

Technical Abstract: Convolutional neural networks trained using manually generated labels are commonly used for semantic or instance segmentation. In precision agriculture, automated flower detection methods use supervised models and post-processing techniques that may not perform consistently as the appearance of the flowers and the data acquisition conditions vary. We propose a self-supervised learning strategy to enhance the sensitivity of segmentation models to different flower species using automatically generated pseudo-labels. We employ a data augmentation and refinement approach to improve the accuracy of the model predictions. The augmented semantic predictionsare then converted to panoptic pseudo-labels to iteratively train a multi-task model. The self-supervised model predictions can be augmented with existing post-processing approaches to further improve their accuracy. An evaluation on a multi-species flower dataset demonstrates that our method outperforms state-of-the-art models without computationally expensive post-processing steps, providing a new baseline for flower detection applications.