Skip to main content
ARS Home » Midwest Area » Bowling Green, Kentucky » Food Animal Environmental Systems Research » Research » Publications at this Location » Publication #220916

Title: On the Significance of Properly Weighting Sorption Data for Least Squares Analysis

Author
item Bolster, Carl

Submitted to: Soil Science Society of America Journal
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 9/30/2009
Publication Date: 2/1/2010
Citation: Bolster, C.H. 2010. On the Significance of Properly Weighting Sorption Data for Least Squares Analysis. Soil Science Society of America Journal. 74:670-679

Interpretive Summary: This study looks at assessing whether the use of unweighted least squares regression yields accurate parameter estimates when fitting the Langmuir model to phosphorus (P) sorption data. High replication experiments were performed to obtain accurate estimates of the error structure representative of P sorption studies. Measured and simulated P sorption data were fit using weighted and unweighted least squares regression and parameter estimates were compared using the two fitting methods. Results show that although weighted least squares regression provides slightly more accurate parameter estimates than unweighted least squares regression, this improvement is relatively minor. Therefore, based on this study it appears that even though an important assumption of least squares regression is violated in P sorption studies – the assumption of constant measurement variance – good parameter estimates can still be obtained through this commonly used approach for fitting the Langmuir model to sorption data.

Technical Abstract: One of the most commonly used models for describing phosphorus (P) sorption to soils is the Langmuir model. To obtain model parameters, the Langmuir model is fit to measured sorption data using least squares regression. Least squares regression is based on several assumptions including normally distributed and constant measurement errors. It is unclear, however, whether the error distribution representative of P sorption studies violates these assumptions. In addition, it is unknown what impact violating these assumptions has on model fits and fitted parameter estimates. Therefore, this study was undertaken to address the issue of measurement uncertainty on model fits and parameter estimates using the Langmuir model on P sorption data. First, the error structure for sorption data for five soils was determined through high replication (n = 10) P sorption studies. Based on this observed error structure, Monte-Carlo simulations were performed to compare parameter estimates between two nonlinear and four linear Langmuir equations to determine which equation provided the best parameter estimates for error structure representative of P sorption studies. Next, Monte-Carlo simulations were performed using weighted data to determine what effect ignoring measurement uncertainty had on parameter estimates. Finally, model fits and parameter estimates obtained for the five soils were compared using four different weighting schemes. Results of this study will provide useful information on the error structure representative of P sorption studies and the importance of accounting for measurement error when obtaining Langmuir constants through least squares regression analysis.