Development of Family Level Assessment of Screen use in the Home for Television (FLASH-TV)

Authors: KUMAR VADATHYA, ANIL - Rice University; BARANOWSKI, TOM - Children'S Nutrition Research Center (CNRC); O'CONNOR, TERESIA - Children'S Nutrition Research Center (CNRC); BELTRAN, ALICIA - Children'S Nutrition Research Center (CNRC); MUSAAD, SALMA - Children'S Nutrition Research Center (CNRC); PEREZ, ORIANA - Children'S Nutrition Research Center (CNRC); MENDOZA, JASON - Fred Hutchinson Cancer Research Center; HUGHES, SHERYL - Children'S Nutrition Research Center (CNRC); VEERARAGHAVAN, ASHOK - Rice University

Submitted to: Multimedia Tools and Applications
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 12/12/2023
Publication Date: 1/12/2024
Citation: Kumar Vadathya, A., Baranowski, T., O'Connor, T.M., Beltran, A., Musaad, S.M., Perez, O., Mendoza, J.A., Hughes, S.O., Veeraraghavan, A. 2024. Development of Family Level Assessment of Screen use in the Home for Television (FLASH-TV). Multimedia Tools and Applications. https://doi.org/10.1007/s11042-023-17852-y.
DOI: https://doi.org/10.1007/s11042-023-17852-y

Interpretive Summary: Children's screen use, including TV, mobile devices, computers and gaming systems, has been associated with several health, developmental and psychosocial outcomes. However, to date almost all studies have relied on child- or parent-report of general estimates of the time children engaged with such screens. Passive, objective assessment tools are needed to more accurately measure children's exposure to different digital screens, including TVs and large gaming systems. We previously published on the development of the FLASH-TV 1.0 system to passively assess children's TV viewing or playing on large gaming systems. FLASH-TV 1.0 relies on video images captured in front of a TV and processes the images in three sequential steps: face detection, face recognition, and gaze detection, using machine learning algorithms (Kumar et al. JMIR Pediatrics and Parenting. 2022). The current paper presents a more detailed report of the computer science and machine learning methods in the development of FLASH-TV, which may be useful to those interested in modifying the current version, taking next steps in further developing it, or exploring new approaches to assessing screen media use.

Technical Abstract: Screen use, including TV viewing, among children is associated with their physical and mental development. The most common assessment of TV viewing are self-reports and these introduce significant error. Objective measures are needed to improve research approaches to inform screen use guidelines. We present an objective approach to assess TV viewing as participant's gaze on the screen. Family Level Assessment of Screen use in the Home (FLASH-TV) uses state-of-the-art computer vision methods for face detection, recognition, and gaze estimation to process images and estimate the amount of time a child in the family spends watching TV. We recruited 21 triads of participants for the development of the FLASH-TV system who took part in 1.5 h observation studies with 5 in participants' homes. We evaluated each step of FLASH-TV by comparing to human-labeled ground truth data. Face detection and recognition methods achieved more than 90% sensitivity in detecting the target child under the challenging conditions of low lighting and poor resolution on a subset of test frames. Our final step of gaze estimation achieved more than 70% sensitivity and 85% specificity when evaluated on all of 3 million gaze/no-gaze labeled frames from 21 triads. Finally, our combined three-step system achieved 4.68 min mean absolute error of the TV watching time with a mean ground-truth TV watching time of 21.72 min. This method offers an objective approach to measure a child's TV viewing, with validation studies underway.