Evaluating the use of questions and responses in a large national dietary data collection instrument

Authors: Steinfeldt, Lois; Clemens, John; Anand, Jaswinder

Submitted to: International Blaise Users Conference
Publication Type: Proceedings
Publication Acceptance Date: 2/6/2009
Publication Date: 6/2/2009
Citation: Steinfeldt, L.C., Clemens, J.C., Anand, J. 2009. Evaluating the use of questions and responses in a large national dietary data collection instrument. 12th International Blaise Users Conference Proceedings, June 2-4, 2009, Riga, Latvia. p.125-127. Available: www.blaiseusers.org/2009/papers/4b.pdf.

Interpretive Summary: Maintaining the Automated Multiple Pass Method (AMPM) dietary interview system to accommodate new foods and portion sizes is an ongoing challenge. Both the food supply and patterns of food consumption are constantly changing. The problem is most severe for large scale; long term food intake surveys such as What We Eat In America, the dietary component of the National Health and Nutrition Examination Survey, is conducted by the U.S. Department of Agriculture and the U.S. Department of Health and Human Services. The AMPM has been in continuous use since January of 2002 and in that time has been used to conduct over 60,000 dietary intake interviews. A large comprehensive update that includes major procedural changes is done for the beginning of each two-year survey period. The purpose of the update is to assure validity and completeness of the dietary data. These updates result from changes in the food supply and in food consumption patterns, and evaluation of the use of the questions and responses. Statistical analysis of the use of questions and responses provides a quantitative approach to evaluation which is particularly useful with a large and complex instrument such as AMPM. This approach can be used with other methods of prioritizing instrument review to substantially reduce the number of questions to be evaluated and focus scarce resources productively.

Technical Abstract: The USDA Automated Multiple Pass Method (AMPM) Blaise instrument collects 24-hour dietary recall data for the What We Eat In America, National Health and Nutrition Examination Survey. AMPM contains more than 2,500 questions and 25,000 responses about foods. Each year it is used in approximately 10,000 interviews which ask individuals to recall the foods and beverages that were consumed the day before the interview. On average, 13 food items are reported for each 24-hour dietary recall and 6 questions are asked about each food item. During each year of the survey, interviewers use AMPM to ask respondents a total of approximately 700,000 questions about foods. Yearly updates are made to questions, response values, and items-skipped patterns to improve the accuracy and completeness of the dietary data. These updates result from changes in the food supply and in food consumption patterns, and evaluation of the use of the questions and responses. Qualitative evaluation methods include direct observation of interviews, interviewer questionnaires, review of food coding, and informal feedback from interviewers and food coders. Statistical analysis of the questions and responses provides a quantitative approach to evaluation which is necessary with such a large and complex instrument. Question and response frequencies are used to prioritize annual reviews. Particular attention is paid to the percentages of “don’t know” and “other specified” responses. Statistical tests of differences in response percentages for the same question over time provide indicators of where changes are occurring that need to be evaluated.