Autonomous calibration of EFDC for predicting chlorophyll-a using reinforcement learning and a real-time monitoring system

Authors: HONG, SEOKMIN - Ulsan National Institute Of Science And Technology (UNIST); ABBAS, ATHER - Ulsan National Institute Of Science And Technology (UNIST); KIM, SOOBIN - Ulsan National Institute Of Science And Technology (UNIST); KWON, DO HUYCK - Ulsan National Institute Of Science And Technology (UNIST); YOON, NAKYUNG - Korea Institute Of Science And Technology; YUN, DAEUN - Ulsan National Institute Of Science And Technology (UNIST); LEE, SANGUK - Korea Institute Of Science And Technology; Pachepsky, Yakov; PYO, JONCHEOL - Pusan National University; CHO, KYUNG HWA - Ulsan National Institute Of Science And Technology (UNIST)

Submitted to: Journal of Environmental Modeling and Software
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 8/18/2023
Publication Date: 8/21/2023
Citation: Hong, S., Abbas, A., Kim, S., Kwon, D., Yoon, N., Yun, D., Lee, S., Pachepsky, Y.A., Pyo, J., Cho, K. 2023. Autonomous calibration of EFDC for predicting chlorophyll-a using reinforcement learning and a real-time monitoring system . Journal of Environmental Modeling and Software. 168. Article e105805. https://doi.org/10.1016/j.envsoft.2023.105805.
DOI: https://doi.org/10.1016/j.envsoft.2023.105805

Interpretive Summary: Chlorophyll a concentration in freshwater sources is the important indicator of water quality. Predictive modeling of chlorophyll concentrations is a fast-developing field. Existing models use constant values of model parameters over the entire simulation periods. However, model parameters reflect conditions that change during the period of simulations. Such changes may be significant, and it can be beneficial to vary model parameters during the simulation period trying to achieve the better model performance. There is no fixed recipe for the parameter variation, and machine learning appears to be efficient approach to carry on the parameter varying without a priori recipe. There are several machine learning methods suitable for that purpose. In this work, the reinforced machine learning application was applied to autonomously vary parameters of the EFDC model applied to the large section of a river in Korea. That approach provided much better modeling accuracy than the traditional constant parameter calibration and modeling. Results of this work will be of use for the water quality professionals in that they demonstrate the opportunities of substantial improvement of accuracy of an existing chlorophyll a concentration modeling tool.

Technical Abstract: Cyanobacterial blooms cause critical damage to aquatic ecosystems and water resources. Therefore, numerical models have been utilized to simulate cyanobacteria by calibrating model parameters for accurate simulation. While conventional calibration, which uses fixed water quality parameters throughout the simulation period, is commonly utilized, it may lead to inaccurate modeling results. To address it, this study proposed a reinforcement learning and environmental fluid dynamics code (EFDC-RL) model that uses real-time pontoon monitoring data and hyperspectral images to autonomously control water quality parameters. The EFDC-RL model showed impressive performance, with an R2 value of 0.7406 and 0.4126 for the training and test datasets, respectively. In comparison, the Chlorophyll-a simulation of conventional calibration had an R2 of 0.2133 and 0.0220, respectively. This study shows that the EFDC-RL model is a suitable framework for autonomous calibration of water quality parameters and real-time spatiotemporal simulation of cyanobacteria distribution.