Evaluation of SMAP core validation site representativeness errors using dense networks of in situ sensors and random forests

Authors: WHITCOMB, J. - University Of Southern California; CLEWLEY, D. - Collaborator; COLLIANDER, A. - Jet Propulsion Laboratory; Cosh, Michael; POWERS, J. - Jet Propulsion Laboratory; FRIESEN, M. - Agriculture And Agri-Food Canada; MCNAIRN, H. - Agriculture And Agri-Food Canada; BERG, A. - University Of Guelph; Bosch, David - Dave; Coffin, Alisa; Holifield Collins, Chandra; Prueger, John; ENTEKHABI, D. - Massachusetts Institute Of Technology; MOGHADDAM, M. - University Of Southern California

Submitted to: IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/26/2020
Publication Date: 10/26/2020
Citation: Whitcomb, J., Clewley, D., Colliander, A., Cosh, M.H., Powers, J., Friesen, M., McNairn, H., Berg, A., Bosch, D.D., Coffin, A.W., Holifield Collins, C.D., Prueger, J.H., Entekhabi, D., Moghaddam, M. 2020. Evaluation of SMAP core validation site representativeness errors using dense networks of in situ sensors and random forests. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 13:6457-6472. https://doi.org/10.1109/JSTARS.2020.3033591.
DOI: https://doi.org/10.1109/JSTARS.2020.3033591

Interpretive Summary: In situ networks are used for calibration and validation of remote sensing products, but their accuracy for the entire landscape can often be biased, because of issues with siting and representation. By deploying temporary networks, it is possible to develop a sufficient dataset to develop a machine learning system for analyzing the permanent network quality. This machine learning technique is referred to as random forests and this provides an assessment of the potential errors in the permanent network. This was applied to four networks that are used for soil moisture satellite products and it was determined that some networks contained potential biases which would result in estimate errors beyond the performance criteria of the satellite mission. This technique points to the need for careful siting and performance analysis of in situ networks prior to use in a satellite calibration and validation program. This will be of value to space agencies as they prepare future missions.

Technical Abstract: In order to validate its soil moisture products, the NASA Soil Moisture Active Passive (SMAP) mission utilizes sites with permanent networks of in situ soil moisture sensors maintained by independent cal/val partners in a variety of ecosystems around the world. Soil moisture measurements from each of the core validation sites (CVSs) must be combined to produce an upscaled estimate of soil moisture at a 33–km scale that represents the SMAP’s radiometer-based retrievals. Each such upscaled estimate is typically calculated as a weighted average of individual sensor measurements within the SMAP grid. Since upscaled estimates produced in this manner are dependent on the particular weighting scheme applied, there is a need for an independent method of quantifying upscaled estimation biases. Here we present one such method. Our method uses reference soil moisture measurements, taken from a dense but temporary network of soil moisture sensors deployed at each CVS, to train a Random Forests regression that expresses soil moisture in terms of a set of spatial variables; the regression then serves as an independent source of upscaled estimates based on the temporary network measurements. The original upscaled soil moisture estimates based on the permanent network are then compared to the estimates based on the temporary network to calculate bias statistics. This method offers a straightforward, systematic, and unified approach to estimate bias across a variety of validation sites. In this study, it was applied to estimate biases at four SMAP CVSs for which the dense network reference data were available. The results showed that for at least some CVSs, the magnitude of the uncertainty in the CVS scaling function bias can be over 80% of the upper limit on SMAP’s entire allowable overall unbiased root mean squared error (ubRMSE). Such large CVS bias uncertainties could make it more difficult to assess biases in soil moisture estimates from SMAP.