A Monte Carlo based adaptive Kalman filtering framework for soil moisture data assimilation

Authors: GRUBER, A. - University Of Leuven; DELANNOY, G. - University Of Leuven; Crow, Wade

Submitted to: Remote Sensing of Environment
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 4/30/2019
Publication Date: 6/1/2019
Citation: Gruber, A., Delannoy, G., Crow, W.T. 2019. A Monte Carlo based adaptive Kalman filtering framework for soil moisture data assimilation. Remote Sensing of Environment. 228:105-114. https://doi.org/10.1016/j.rse.2019.04.003.
DOI: https://doi.org/10.1016/j.rse.2019.04.003

Interpretive Summary: Agricultural drought has enormous consequences for domestic economic interests and international food security concerns. These implications can be minimized by the effective detection and mitigation of drought impacts. Land data assimilation systems are increasingly being tasked with combining hydrologic information acquired from a variety of sources (e.g., hydrologic modeling, ground-based observations, and remotely-sensed retrievals) into integrated predictions of root-zone soil moisture availability. These predictions, in turn, form the basis of agricultural drought monitoring systems. To perform adequately, these systems require detailed information about errors present in various sources of soil moisture information. This paper describes a new mathematical technique for deriving soil moisture error information required by data assimilation systems to accurately predict the onset and evolution of drought. Results demonstrate the superiority of the approach versus existing techniques. It provides an important contribution to current USDA efforts aimed at improving our ability to monitor agricultural drought.

Technical Abstract: The main sources for global soil moisture information are remote sensing observations and land surface model estimates. Data assimilation (DA) aims at optimally combining these data sources through statistical merging. To properly parameterize such merging, one needs to obtain accurate knowledge of model and observation uncertainties, which is the crux of a successful DA system. In this paper, we propose a new Monte Carlo based adaptive Kalman Filtering framework (MadKF) that estimates model and observation uncertainties (Q and R) and updates soil moisture predictions simultaneously. Spatially distributed uncertainties are estimated by applying triple collocation analysis (TCA) to Monte Carlo simulations of the model open-loop, the model analysis, and the observation time series at each grid cell. Error cross-covariances, which are inevitable between these time series, are diagnosed from their ensembles and used to iteratively correct biases they cause in Q and R estimates, and hence, in the Kalman filter gain. The proposed MadKF is tested in a synthetic environment and by assimilating real satellite soil moisture retrievals into the Antecedent Precipitation Index model forced with daily aggregated satellite precipitation. Synthetic experiments indicate a good convergence of Q and R estimates. Internal DA diagnostics, i.e., the innovation auto-correlation (IAC) and the variance of the normalized innovations, asymptotically converge to their desired values, which indicates that the filter is operating near its optimum and reliably predicts analysis uncertainty. Real-data experiments further indicate that the MadKF is robust against observation error auto-correlations, which typically cause problems in conventional IAC-tuning based adaptive filtering approaches, and more self-consistent than a normal TCA-based KF framework, which is likely affected by non-zero error cross-correlations between active and passive microwave soil moisture retrievals violating standard TCA assumptions. The MadKF therefore also provides an attractive opportunity for the large-scale validation of remotely sensed satellite soil moisture products.