Skip to main content
ARS Home » Research » Publications at this Location » Publication #304597
Title: Hydrologic and water quality models: Performance measures and evaluation criteria

Author
item Moriasi, Daniel
item GITAU, MARGARET - Florida A & M University
item PAI, NARESH - Stone Environmental Consulting
item DAGGUPATI, PRASAD - Texas Agrilife Research

Submitted to: Transactions of the ASABE
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 2/1/2015
Publication Date: 12/1/2015
Publication URL: http://handle.nal.usda.gov/10113/62083
Citation: Moriasi, D.N., Gitau, M.W., Pai, N., Daggupati, P. 2015. Hydrologic and water quality models: Performance measures and evaluation criteria. Transactions of the ASABE. 58(6):1763-1785.

Interpretive Summary: Hydrologic and water quality models are essential tools used to determine the impacts of land management, land use, climate, and conservation practices on water resources, ecology, and water related ecosystem services. Performance measures and corresponding criteria are essential metrics by which model simulation performance of the output of interest is determined. The ultimate goal is to minimize model output simulation errors. This paper provides recommendations for suitable graphical and statistical measures to determine simulation performance for most commonly used hydrologic and water quality models and establishes guidelines for the recommended performance measures for various simulated constituents at different spatial and temporal scales. This was accomplished by performing a synthesis of existing special collection of papers on model use, calibration, and validation. Based on the synthesis and a rigorous review of existing literature the time series, scatter plots, cumulative distribution, flow and load duration, and maps graphical measures and the coefficient of determination (R2) alongside the gradient b and the intercept a of the corresponding regression line, Nash Sutcliff efficiency (NSE), index of agreement (d), root mean square error (RMSE) alongside RMSE – observations standard deviation ratio (RSR), and percent bias (PBIAS) statistical measures be used for model performance evaluation. Model performance criteria for these measures are provided. The recommendations and guidelines developed in this paper will be used alongside those of other eight topic-specific papers to develop American Society of Agricultural and Biological Engineers (ASABE) standards for hydrologic and water quality models with regards to model calibration and validation.

Technical Abstract: Performance measures and corresponding criteria constitute an important aspect of calibration and validation of any hydrological and water quality (H/WQ) model. As new and improved methods and information are developed, it is essential that performance measures and criteria be updated. Therefore, the objectives of this paper were to: 1) synthesize the special collection papers (Moriasi et al, 2012) with respect to performance measures and criteria, and the corresponding performance ratings; 2) perform a meta-analysis of performance values as reported in literature both within and outside of the special collection considering calibration and validation periods, simulated components, as well as spatial and temporal scales; and, 3) establish guidelines for various the performance measures. Based on the synthesis and a rigorous review of existing literature the time series, scatter plots, cumulative distribution, flow and load duration, and maps graphical measures and the coefficient of determination (R2) alongside the gradient b and the intercept a of the corresponding regression line, Nash Sutcliff efficiency (NSE), index of agreement (d), root mean square error (RMSE) alongside RMSE – observations standard deviation ratio (RSR), and percent bias (PBIAS) statistical measures be used for model performance evaluation. The following criteria were established for four of the recommended statistical measures. In general, the performance of field-scale models to simulate flow can be judged as “satisfactory” if R2 > 0.70 and d > 0.75 at the monthly time step. For watershed-scale models, the performance of a given model to simulate flow can be evaluated as “satisfactory” if R2 > 0.60, NSE > 0.50, and PBIAS = ±15% at daily, monthly, or annual time step. In addition, model performance can be evaluated as “satisfactory” if R2 > 0.40 and NSE > 0.45 on monthly time step and PBIAS = ±20% at the daily, monthly, or annual time step for sediments; if R2 > 0.40 and NSE > 0.40 at monthly time step, and PBIAS = ±30% at the daily, monthly, or annual time step for phosphorus; and if R2 > 0.30 and NSE > 0.35 at a monthly time step and PBIAS = ±30% at the daily, monthly, or annual time step for nitrogen. Additional considerations, such as reproducibility and extendibility, residual analysis, quality and quantity of measured data, spatial and temporal scales, project scope and magnitude, calibration vs. validation performance criteria, and unacceptable performance measure values, which affect these guidelines, are also discussed. The general criteria presented for the recommended measures should be adjusted when appropriate to reflect these considerations. A case study and an excel spreadsheet were provided to illustrate the application of the recommended measures and the corresponding developed criteria guidelines.