Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling

Authors: Harmel, Daren; SMITH, PATRICIA - TEXAS A&M UNIV

Submitted to: Journal of Hydrology
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 1/31/2007
Publication Date: 3/15/2007
Citation: Harmel, R.D., Smith, P.K. 2007. Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling. Journal of Hydrology. 337:326-336.

Interpretive Summary: As hydrologic and water quality models are increasingly used to guide decisions regarding water resource policy, management, and regulation, it is no longer appropriate to disregard the uncertainties related to model use. In the present research, the method of calculating the error term, which is the difference between measured and predicted values, was modified to consider uncertainty in measured data. Our goal was to improve the process used to evaluate hydrologic and water quality models by including measurement uncertainty in the process. Specifically, the calculations of several goodness-of-fit statistical tools were modified based either on the maximum uncertainty (Modification 1) or the distribution of uncertainty (Modification 2). These modifications were based on the theory that hydrologic and water quality models should not be evaluated against the values of measured data, which are uncertain, but against the measurement uncertainty. Modification 1, which is necessary when only the uncertainty range is known, minimizes the error and thus improved the goodness-of-fit for each of the example data sets. The more practical Modification 2, which requires increased knowledge of the uncertainty, resulted in less pronounced improvement in the goodness-of-fit between measured and modeled values. The goodness-of-fit increased very little for measured data with little uncertainty along with poor model performance but showed modest improvement when data with substantial uncertainty were compared with both poor and good model predictions. These results are desirable because poor model performance, especially due to model problems, should not be judged as good simply because measurement uncertainty is considered.

Technical Abstract: As hydrologic and water quality models are increasingly used to guide water resource policy, management, and regulatory decision-making, it is no longer appropriate to disregard uncertainty in model calibration, validation, and evaluation. In the present research, the method of calculating the error term in pairwise comparisons of measured and predicted values was modified to consider measurement uncertainty with the goal of facilitating enhanced evaluation of hydrologic and water quality models. Specifically, the deviation calculations of several goodness-of-fit indicators were modified based on the uncertainty boundaries (Modification 1) or the probability distribution of measured data (Modification 2). The basis of this method was the theory that hydrologic and water quality models should not be evaluated against the values of measured data, which are uncertain, but against the inherent measurement uncertainty. Modification 1, which is necessary in the absence of distributional information, minimizes the calculated deviations and thus improved the goodness-of-fit indicators for each of the example data sets. The more practical deviation calculation of Modification 2, which requires distributional information or assumptions, resulted in less pronounced improvement in the goodness-of-fit indicators. The goodness-of-fit increased very little for measured data with little uncertainty and poor model performance but showed modest improvement when data with substantial uncertainty were compared with both poor and good model predictions. These results are desirable because poor model performance, especially due to model structure deficiencies, should not be judged as satisfactory simply due to measurement uncertainty.