Skip to main content
ARS Home » Southeast Area » Byron, Georgia » Fruit and Tree Nut Research » Research » Publications at this Location » Publication #313813
Title: Assessing disease severity: accuracy and reliability of rater estimates in relation to number of diagrams in a standard area diagram set

Author
item Bock, Clive
item Hotchkiss, Michael - Mike
item Wood, Bruce

Submitted to: Plant Pathology
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 2/4/2015
Publication Date: 2/1/2016
Citation: Bock, C.H., Hotchkiss, M.W., Wood, B.W. 2016. Assessing disease severity: accuracy and reliability of rater estimates in relation to number of diagrams in a standard area diagram set. Plant Pathology. 65(2):261–272.

Interpretive Summary: Standard area diagrams (SADs) are figures of known disease severity that help improve accuracy and reliability. The effects of diagram number in a SAD set on accuracy and reliability is unknown. The objective of this study was to compare estimates of pecan scab severity made without SADs, and using 3, 5, 7 or 10-diagram SADs. SADs were used to interpolate disease severity to the nearest percent estimate (NPE), and as a category scale (scale) classifying severity to the closest reference diagram value. More diagrams using NPEs did not necessarily result in more accurate or reliable estimates, but did consistently improve inter-rater reliability. The least accurate estimates were improved the most when using SADs. When used as a scale, only estimates made with 10-diagram SADs had similar accuracy and inter-rater reliability to estimates made using SADs with NPEs. Maximum disease severity and the disease severity range are important factors and will likely influence resulting accuracy and reliability. The ramification of diagram numbers in a SADs, mean estimates and rater experience are discussed.

Technical Abstract: Error in rater estimates of plant disease severity occur, and standard area diagrams (SADs) help improve accuracy and reliability. The effects of diagram number in a SAD set on accuracy and reliability is unknown. The objective of this study was to compare estimates of pecan scab severity made without SADs, and using 3, 5, 7 or 10-diagram SADs. SADs were used to interpolate disease severity to the nearest percent estimate (NPE), and as a category scale (scale) classifying severity to the closest reference diagram value. Twelve experienced raters assessed 20 images of scabbed pecan valves without and with the SADs using NPEs and the scale method. Increments in diagram number using NPEs did not necessarily result in more accurate or reliable estimates. Inter-rater reliability was positively correlated with number of diagrams (r = 0.3288 (P = <0.0001)). The least accurate estimates were improved the most when using SADs with NPEs, but the gain did not relate to number of diagrams. As few as 3 diagrams could reduce error of inaccurate estimates as much as 5, 7 or 10 diagrams. When used as a scale, only estimates made with 10-diagram SADs had similar accuracy and inter-rater reliability to estimates made using SADs with NPEs. Maximum disease severity and the disease severity range are important factors and will likely influence resulting accuracy and reliability. The ramification of diagram numbers in a SADs, mean estimates and rater experience are discussed.