Rating Pome fruit quality traits using deep learning and image processing

Authors: NGUYEN, NHAN - Washington State University; MICHAUD, JOSEPH - US Department Of Agriculture (USDA); MOGOLLON, RENE - Washington State University; ZHANG, HUITING - Washington State University; Hargarten, Heidi; Leisso, Rachel; TORRES, CAROLINA - Washington State University; Honaas, Loren; FICKLIN, STEPHEN - Washington State University

Submitted to: bioRxiv
Publication Type: Pre-print Publication
Publication Acceptance Date: 4/4/2024
Publication Date: 4/5/2024
Citation: Nguyen, N.H., Michaud, J., Mogollon, R., Zhang, H., Hargarten, H.L., Leisso, R.S., Torres, C.A., Honaas, L.A., Ficklin, S. 2024. Rating Pome fruit quality traits using deep learning and image processing. bioRxiv. https://doi.org/10.1101/2024.04.03.588000.
DOI: https://doi.org/10.1101/2024.04.03.588000

Interpretive Summary: Visual assessments of pome fruit are commonly conducted to identify defects and other symptoms in postharvest research experiments and within stakeholder industries. These assessments include color, disorder incidence, and starch clearing, among others. However, two primary challenges persist within this practice: rater bias and lack of granularity. Rater bias occurs when subjective judgment led to inconsistencies among raters, resulting in different ratings of the same fruit. Moreover, many of these traits are evaluated using visual cues, such color or pattern cards that do not fully capture the range of variation. Consequently, these challenges hamper researchers by preventing comparisons across experiments due to rater bias and limiting the accuracy of outcome association due to inadequate granularity. To meet the demand of the pome fruit industry for a rapid and unbiased assessment of large amounts of fruit, the current human-based rating system is inadequate due to its tardiness, subjectivity, and lack of comparability. To address limitations of current visual assessment methodologies, we developed Granny, a freely available image analysis package that uses machine learning (ML) to rate pome fruit quality traits such as starch contents, peel defects, and peel color analyses. By providing rapid, fine-grained, and consistent data, Granny enables researchers to overcome person-to-person bias and variability inherent in manual assessments. Importantly, Granny’s performance has been validated with expert assessments, affirming its high accuracy.

Technical Abstract: Quality assessment of pome fruits (i.e. apples and pears) is used not only crucial for determining the optimal harvest time, but also the progression of those attributes during storage. Therefore, it is typical for a given lot of pome fruit to be repeatedly evaluated during the course of a postharvest experiment. This often includes careful visual assessments of fruit for apparent defects and other symptoms. General best practice in research and development settings is to have ratings done by the same individual or group of individuals to reduce confounding factors associated with person-to-person variability. However, such consistency across labs, facilities, and experiment duration is often not feasible or attainable. Moreover, while these visual assessments are critical empirical data, they are often coarse-grained and lack strict objective criteria. Here we present image-based analytics tools that attempt to mediate the above issues while also promoting program continuity by processing data according to long-established standards and references. Central to the approach is a machine learning algorithm that preprocesses fruit images for downstream analyses including starch assessment, presence of peel defects, and peel color analyses. Importantly, our approach provides rapid, fine-grained, consistent, and continuous data that we can relate to other data types of similar structure (namely global scale -omics technologies and other quantitative fruit quality metrics like soluble solids content and flesh firmness). Finally, these software tools are portable, updatable, and freely available at GitHub.