Effects of dataset curation on body condition score (BCS) determination with a vision transformer (ViT) applied to RGB+Depth images

Authors: WINKLER, ZACHARY - New Mexico State University; BOUCHERON, LAURA - New Mexico State University; UTSUMI, SANTIAGO - New Mexico State University; NYAMURYEKUNG'E, SHELEMIA - Norwegian Institute Of Bioeconomy Research(NIBIO); McIntosh, Matthew; Estell, Richard - Rick

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 6/1/2024
Publication Date: 6/5/2024
Citation: Winkler, Z., Boucheron, L.E., Utsumi, S.A., Nyamuryekung'e, S., McIntosh, M.M., Estell, R.E. 2024. Effects of dataset curation on body condition score (BCS) determination with a vision transformer (ViT) applied to RGB+Depth images. Computers and Electronics in Agriculture. 8. Article 100482. https://doi.org/10.1016/j.atech.2024.100482.
DOI: https://doi.org/10.1016/j.atech.2024.100482

Interpretive Summary: Body condition score (BCS) is a useful management tool for producers to evaluate the health condition of their cattle. Although BCS is useful for knowledgeable individuals, it is time consuming to perform, is subject to bias among observers, and is difficult to implement in outdoor cattle ranches/farms. Newly affordable digital color (RGB) and depth (D) cameras (RGBD) can capture color images with a 3-Dimentional depth map and could be adapted to record photographs of cattle in grassland settings. In this study, we used a language transformer to pre-train digital color and depth images of individual beef cattle from a herd in southern New Mexico and report the system’s capacity to accurately assess cattle body condition in relation to a team of expert scorers. We also detail challenges of this approach and opportunities to fine-tune our model for application in different operations with different types of cattle.

Technical Abstract: Body condition score (BCS) has been a useful tool in estimating the body energy reserves, reproductive performance, and health of cattle for many years now. This categorical metric, while useful, requires one or more experienced observers to visually inspect cows and assess variability of body fat deposits regularly, and this process can be very time consuming, subject to bias among observers, and difficult to implement in extensive ranching systems where the sighting and systematic inspection of cattle is not possible. Availability of low cost RGB+depth cameras has encouraged the use of computer vision and machine learning algorithms as a potential solution to those barriers. 'This approach shows promise, but requires further research and development. Obtaining BCS ground truth training data for herds of rangeland cattle can be difficult. Further-more, judging experts may not always agree on a cow's BCS categorical assessment, and the processing and curation of BCS data itself can pose problems for the successful training, validation, and testing of machine learning models. In this study we use the vision transformer (ViT) BEiT to assess the BCS of Raramuri Criollo cattle and discuss the challenges associated with the method, needs for dataset processing and curation, and similarities between BCS classes. Special consideration is given to procedures for BCS dataset curation, as this aspect of machine learning has significant influence on the overall performance of the network and present a unique challenge for image classification models.