Skip to main content
ARS Home » Research » Publications at this Location » Publication #239042
Title: A graphical method to evaluate spectral preprocessing in multivariate regression calibrations: example with Savitzky-Golay filters and partial least squares regression

Author
item Delwiche, Stephen - Steve
item Reeves Iii, James

Submitted to: Applied Spectroscopy
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 11/3/2009
Publication Date: 1/1/2010
Citation: Delwiche, S.R., Reeves III, J.B. 2010. A graphical method to evaluate spectral preprocessing in multivariate regression calibrations: example with Savitzky-Golay filters and partial least squares regression. Applied Spectroscopy. 64(1):73-82.

Interpretive Summary: Near-infrared (NIR) spectroscopy is a widely used method in the agricultural, food, and chemical industries that allows for rapid and often non-destructive determinations of chemical constituents of many products. However, the NIR instrument must be very carefully calibrated for many variables related to the sample being tested. We tested several techniques on a common chemical constituent of wheat and determined how to make NIR determinations more accurate. This information is of direct benefit to the analytical laboratories that develop and maintain NIR calibrations for agricultural commodities, foods, pharmaceuticals, and other chemicals.

Technical Abstract: In multivariate regression analysis of spectroscopy data, spectral preprocessing is often performed to reduce unwanted background information (offsets, sloped baselines) or accentuate absorption features in intrinsically overlapping bands. These procedures, also known as pretreatments, are commonly smoothing operations or derivatives. While such operations are often useful in reducing the number of latent variables of the actual decomposition and lowering residual error, they also run the risk of misleading the practitioner into accepting calibration equations that are poorly adapted to samples outside of the calibration. The current study examined this effect through partial least squares (PLS) regression analysis on near-infrared (NIR) reflectance spectra of ground wheat meal with two analytes, protein content and sodium dodecyl sedimentation (SDS) volume (an indicator of the quantity of the gluten proteins that contribute to strong doughs). These two properties were chosen because of their differing abilities to be modeled by NIR: excellent for protein content and fair for SDS sedimentation volume. To further demonstrate the potential pitfalls of preprocessing, an artificial component, a randomly generated value, was included in PLS regression trials. Savitzky-Golay (digital filter) smooth-, first derivative-, and second derivative- preprocess functions (5 to 25 centrally symmetric convolution points) were applied to PLS calibrations of 1 to 15 factors. The results demonstrated the danger of an overreliance on preprocessing when a) the number of samples used in a multivariate calibration is low (< 50); b) the spectral response of the analyte is weak; and c) the goodness of the calibration is based on the coefficient of determination (R2) rather than a term based on residual error. The graphical method has application to the evaluation of other preprocess functions and various types of spectroscopy data.