The reliability of categorical triple collocation for evaluating soil freeze/thaw datasets

Authors: LI, H. - Beijing Normal University; CHAI, L. - Beijing Normal University; Crow, Wade; DONG, J. - Massachusetts Institute Of Technology; LIU, S. - Beijing Normal University; ZHAO, S. - Beijing Normal University

Submitted to: Remote Sensing of Environment
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 8/25/2022
Publication Date: 9/5/2022
Citation: Li, H., Chai, L., Crow, W.T., Dong, J., Liu, S., Zhao, S. 2022. The reliability of categorical triple collocation for evaluating soil freeze/thaw datasets. Remote Sensing of Environment. 281. Arcicle 113240. https://doi.org/10.1016/j.rse.2022.113240.
DOI: https://doi.org/10.1016/j.rse.2022.113240

Interpretive Summary: Accurate mapping of frozen soil is of value for a range of agricultural applications including fertilizer scheduling, flood monitoring and yield management. Within the past ten years, a range of remote sensing techniques have been developed for such mapping. However, before these techniques can be applied with confidence, they must first be evaluated against independent observations. However, such evaluation has been severely hampered by the cost and difficulty of ground-sampling dynamic spatial patterns of soil freeze/thaw within agricultural landscapes. As a result, many of these approaches have not yet been adequately evaluated. This paper describes and applies a novel evaluation procedure for remotely sensed freeze/thaw products that does not require the availability of ground data. Instead, it leverages the availability of multiple independent estimates of soil freeze/thaw datasets - acquired from a range of remote sensing and modelling sources - to estimate each product’s individual error statistics without reference to an error-free, ground-based benchmark. This approach will be used to better assess existing soil freeze/thaw data products and speed their incorporation into important agricultural applications.

Technical Abstract: Seasonal soil freeze/thaw (FT) state transition plays a critical role in a range of ecosystem, hydrological and biogeochemical processes. As a result, a thorough and large-scale validation of remote-sensed or model-based FT products is important. The categorical triple collocation (CTC) method can provide pixel-wise accuracy for categorical FT products, which is an essential supplement to ground-based verification, especially when there is a lack of in situ measurements or the spatial distribution of product accuracy is required. This paper aims to improve our knowledge and understanding of the reliability of the CTC method in accuracy reporting for the soil FT state. Four soil FT datasets over the period from March 31, 2015 to December 31, 2017 are examined, i.e., the L3 freeze/thaw product of the Soil Moisture Active Passive Mission (SMAP FT), the Freeze/Thaw Earth System Data Record product (ESDR FT), the ERA5 FT dataset derived from the first-layer (0-7 cm) soil temperature of ERA5-land (the land surface dataset produced from the fifth generation ECMWF atmospheric reanalysis) and the WFDE5 FT dataset derived from the 2-m air temperature of WFDE5 (bias adjusted ERA5 reanalysis based on WATCH Forcing Data methodology). They are organized into two triplets, i.e., SMAP-ESDR-WFDE5 and SMAP-ESDR-ERA5, during CTC application. The CTC reliability is then demonstrated via the consistency between the CTC results of the same FT product derived from two different triplets and the consistency between CTC results and the corresponding ground-based validation results against observations collected from dense in situ networks and sparse meteorological stations. Three main conclusions are drawn from this study. 1) CTC demonstrates overall stable performance and derives relatively reliable sensitivity and specificity assessments for the four FT products. In particular, CTC can generally give the correct rankings for all three products in a triplet or - in a worst case scenario – still identify the best or the worst product if failing to rank all three products correctly. 2) A GAM analysis reveals that sample size is a critical factor significantly impacting CTC performance. A sample size no less than a complete soil freeze-thaw cycle (365 days) is strongly recommended to ensure a reliable CTC result. 3) Based on verified CTC configurations, our analyses also show that sand content, snow depth, terrain complexity, and land surface temperature are four important confounding factors impacting the freeze/thaw classification accuracy of different FT products.