Reproducibility of the development and validation process of standard area diagram By two laboratories: An example using the Botrytis Cinerea/Gerbera Jamesonii Pathosystem

Authors: PEREIRA DE MELO, VILMA - University Of Maringa; DA SILVA MENDONCA, ANA - University Of Maringa; DE SOUZA, HUDSON - University Of Maringa; GABRIEL, LORRANT - University Of Maringa; Bock, Clive; EATON, MAHOGANI - Fort Valley State University; SCHWAN-ESTRADA, KATIA - University Of Maringa; DE CARVALHO NUNES, WILLIAM - University Of Maringa

Submitted to: Plant Disease
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 3/14/2020
Publication Date: 7/10/2020
Citation: Pereira De Melo, V., Da Silva Mendonca, A.C., De Souza, H.S., Gabriel, L.C., Bock, C.H., Eaton, M.J., Schwan-Estrada, K.R., De Carvalho Nunes, W.M. 2020. Reproducibility of the development and validation process of standard area diagram By two laboratories: An example using the Botrytis Cinerea/Gerbera Jamesonii Pathosystem. Plant Disease. 104:2440-2448. https://doi.org/10.1094/PDIS-08-19-1708-RE.
DOI: https://doi.org/10.1094/PDIS-08-19-1708-RE

Interpretive Summary: Standard area diagrams (SADs) are aids to improve the accuracy and reliability of visual estimates of the severity of plant diseases. Little is understood of the factors on that influence development of good SADs. This knowledge will help improve standard operating practice for development and validation for SADs. We investigated SAD validation in two different labs using the same images depicting severity of gray mold (caused by Botrytis cinerea) on gerbera daisy. The SADs had 6 diagrams. Results showed that actual severity on the SADs as measured at each lab varied by up to 5.18%. Furthermore, measurement of the test image actual values varied from 0 to up to 24.29%, depending on image. An equivalence test indicated no improvement of raters estimates in any measure of agreement with use of the SADs, at Lab 2, scale shift, generalized bias and agreement were significantly improved with use of the SADs. Inter-rater reliability was improved at both Lab 1 and Lab 2 when using the SADs. Absolute error was reduced at both labs when raters used SADs. The results confirm SADs are a useful tool, but components of the validation process may affect aspects of the outcome.

Technical Abstract: Standard area diagrams (SADs) are an accepted method to use as an aid to improve the accuracy and reliability of visual estimates of the severity of plant diseases. However, the effects of different factors on the outcome of the development and validation process for SADs are not well understood. This knowledge will help improve standard operating practice for development and validation for SADs. We investigated the outcome of two different labs validating a SADs for severity of gray mold (caused by Botrytis cinerea) on gerbera daisy. The SADs were developed in one lab (Lab 1) and had 6 diagrams. Severity measurements on the diagrams in the SADs and all the test images were performed at two labs (Lab 1 and Lab 2). A different group of 18 raters at each lab assessed the test images without and with SADs under independent instruction at Lab 1 and 2, respectively. Results showed that actual severity on the SADs as measured at each lab varied by up to 5.18%. Furthermore, measurement of the test image actual values varied from 0 to up to 24.29%, depending on image. Whereas at Lab 1 an equivalence test indicated no improvement in any measure of agreement with use of the SADs, at Lab 2, scale shift, generalized bias and agreement were significantly improved with use of the SADs. An analysis of variance indicated differences existed between labs, use of the SADs aid, and the interaction, depending on the agreement statistic. The inter-rater reliability was significantly improved at both Lab 1 and Lab 2 as a result of using SADs as an aid to severity estimation. Gain in measures of agreement and reliability tended to be greatest for the least able raters at both Lab 1 and Lab 2. Absolute error was reduced at both labs when raters used SADs. The results indicate that the SADs is a useful tool, and also that components of the validation process in different labs may affect aspects of the outcome of that process.