Comparison of Validity of Food Group Intake by Food Frequency Questionnaire Between Pre- and Post- adjustment Estimates Derived from 2-day 24-hour Recalls in Combination with the Probability of Consumption

Validation of a food frequency questionnaire (FFQ) utilising a short-term measurement method is challenging when the reference method does not accurately reflect the usual food intake. In addition, food group intake that is not consumed on daily basis is more critical when episodically consumed foods are related and compared. To overcome these challenges, several statistical approaches have been developed to determine usual food intake distributions. The Multiple Source Method (MSM) can calculate the usual food intake by combining the frequency questions of an FFQ with the short-term food intake amount data. In this study, we applied the MSM to estimate the usual food group intake and evaluate the validity of an FFQ with a group of 333 Korean children (aged 3-6 y) who completed two 24-hour recalls (24HR) and one FFQ in 2010. After adjusting the data using the MSM procedure, the true rate of non-consumption for all food groups was less than 1% except for the beans group. The median Spearman correlation coefficients against FFQ of the mean of 2-d 24HRs data and the MSM-adjusted data were 0.20 (range: 0.11 to 0.40) and 0.35 (range: 0.14 to 0.60), respectively. The weighted kappa values against FFQ ranged from 0.08 to 0.25 for the mean of 2-d 24HRs data and from 0.10 to 0.41 for the MSM-adjusted data. For most food groups, the MSM-adjusted data showed relatively stronger correlations against FFQ than raw 2-d 24HRs data, from 0.03 (beverages) to 0.34 (mushrooms). The results of this study indicated that the application of the MSM, which was a better estimate of the usual intake, could be worth considering in FFQ validation studies among Korean children.


Introduction
An individual's long-term intake, which is defined as the 'usual intake', cannot be observed directly (Carriquiry and Camano-Garcia, 2006); thus, food frequency questionnaires (FFQ) are widely used to collect longterm data for dietary analysis in spite of some limitations (Byers, 2001;Kristal et al., 2005). To overcome the weaknesses of FFQ in describing usual intake, researchers use repeated short-term measurements, including 24-hour recall (24 HR) and food records (FR), to determine average daily food intake. Several studies (Oh and Hong, 1999;Mennen et al., 2002;Huybrechts et al., 2008) have already been conducted to determine the minimum number of days necessary to estimate usual intake; however, many of these studies have a relatively homogeneous study population and cannot be generalised to other populations. In addition, the time-consuming process involved, the large response burden, and the economic costs associated with short-term measurement make it unfeasible in large scale surveys.
Alternatively, some statistical methods estimating usual intake using multiple short-term measurements have been developed (National Research Council, 1986) and refined (Nusser et al., 1996) over the last 2 decades. These methods attempt to control within-individual variation of the individual intake, which may vary daily. However, most of these methods are suitable for nutrient analysis of the food consumed daily (Hoffmann et al., 2002;Waijers et al., 2006). Tooze and colleagues (Tooze et al., 2006) already illustrated some of the unique challenges for statistical modelling in relation to estimating the usual intake of episodically consumed food. Subsequently, to compensate for these challenges, more complex methods were developed, such as the National Cancer Institute (NCI) method and Multiple Source Method (MSM) that combine two concepts: the amount and the probability of consumption (Subar et al., 2006;Haubrock et al., 2011). The consumption amount can be determined from short-term measurements, and the probability of consumption can be deduced from longterm measurement data. Finally, the two instruments, which were developed for different purposes, can be merged to compensate for their mutual weaknesses.
In this study, employing the MSM concept, we used a frequency variable derived from the FFQ to adjust the 2-d 24HRs short-term measurement data so that they more closely approximate usual intake. In this way, we estimated usual food group intake for 16 food groups consumed by 333 Korean children. Finally, we compared the validity of food group intake by the FFQ between preand post-adjustment estimates derived from 2-d 24HRs in combination with the probability of consumption.

Subjects
The study participants were healthy children, aged 3 to 6 y old, who were enrolled at kindergarten, health centres, nurseries and hospitals. Five regions in South Korea participating were selected to take part in the study. The children's parents were invited to an orientation session about the study, and additional information was sent by postal mail and telephone calls. The FFQ was distributed to the children's mothers by mail before individual interview. The mothers of the children completed the FFQ at home, and the individual interview was scheduled by a trained dietician to verify the completion of the FFQ. During this face-to-face interview, the first 24HRs were also conducted. In addition, to account for within-individual variation in consumption, another non-consecutive 24HR was conducted by telephone within 1 week of the initial interview. A total of 349 children agreed to participate by their parents, and parents of 333 children completed both the FFQ and 2-d 24-hour recalls (24HRs) between June and August 2010. Demographic information was collected from participants, including sex and age. The children's height and weight were also measured and used to calculate the body mass index (BMI). This information was collected to be used as a covariate during MSM analysis. This study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Institutional Review Board (IRB approval number: DKUHIRB2010-04-0093). Written informed consent was obtained from patients of all subjects.

Dietary assessment
The FFQ used in the study was previously developed and validated with nutrients. The correlation coefficients of this FFQ ranged from 0.6 to 0.8 for reproducibility and from 0.3 to 0.6 for validity (Lim, 2001). A modified version of this FFQ was developed by one of the authors (S-Y Oh), and the composition of the FFQ was explained (Shin et al., 2007). For each item, the participating parents were asked about their child's frequency of consumption during the previous year using a nine-level scale, ranging from 'rarely eat' to 'three of more times per day'. The portion sizes of the FFQ items were categorised as small, average, and large, representing 0.5 times, 1.0 times, and 1.5 times than the average, respectively.
On the day of the face-to-face interview with the dietician, the children's mothers and caregivers were asked to write down every food the child consumed the previous day, as part of the 24HR procedure, including dishes, recipes, measurements based on twodimensional photography of utensil or volume unit, and the place and time of the meals at home (own home or neighbor/relative's home), cafeteria (pre-school, nursery, kindergarten), restaurant (street food stand, convenience store, bakery, or fast food), or others. Another 24HR via telephone was also conducted using the same procedure with the same dietician. After the data from the FFQ and the 2-d 24HRs were collected and confirmed, we calculated the food group intake for each child using CAN-PRO 3.0 (Computer Aided Nutritional analysis program, The Korean Nutrition Society, Seoul, Korea). For the analysis of food group intake of the FFQ and 2-d 24HRs, we used 16 food groups, as established by the Korean Nutrition Society in 2005 (Grains, potatoes, sugars, beans, seeds, vegetables, mushrooms, fruits, meats and poultry, eggs, fish and shells, seaweeds, milk and dairy products, oils, beverages, and seasonings).

MSM procedure
The MSM procedure was implemented as a webbased program built with open-source components and used to estimate the usual food group intake of the participants. Because MSM estimation was not possible when only one observation was available in the dataset, the participants who did not have either the FFQ or were missing 2-d 24HRs (n=16) were excluded. In addition, due to the small sample size, separate statistical analyses for boys and girls were not performed.
For each participant, the data reported in the FFQ were converted into daily frequency values. For example, "2 to 3 times a month" was coded as 2.5 (the mean of 2 and 3) and transformed to a daily frequency value of 0.083 (estimated by the formula 2.5/30). Next, these frequencies were summed at the food group level as based on the 16 previously defined food groups. The calculated daily intake in gram/day from the 2-d 24HR data was summed into the same 16 food groups. The two data sets were merged via three variables: participant, day, and food groups.
The MSM procedure comprises 3 steps based on at least 2 repeated short-term measurements. The MSM procedure is well-documented by Haubrock et al. (2011).
Briefly, a three-step process produced the results of the present study. 1. The individual probability of consuming a food group on one day was estimated by a logistic regression model. The model contained FFQ frequency, gender, age, and BMI as covariates assumed to be predictive of consumption.
2. The usual intake on reported days from the 2-d 24HR was estimated by a linear regression model with the observed food group intake as a function of the same covariates, as in step 1. Then, a measurement error model was assumed to allow for convergence of the mean food group intake of an individual toward a grand mean.
3. The MSM calculated the usual daily food group intake for each individual by multiplying step 1 (individual probability of consumption) and step 2 (the usual intake of an individual on a consumption day).
The percentiles (5th -95th), mean, SD, kurtosis, and skewness were all calculated to compare the usual food group intake distributions to the mean of 2-d 24HRs data.

Validation
We conducted a validation of the FFQ to evaluate the strength of correlation or agreement with shortterm measurements after adjusting with the MSM. Our validation proceeded with two short-term measurements: the mean of 2-d 24HRs data and the individual usual intake data, as adjusted by the MSM. For the validation, the Spearman correlation coefficients were computed to assess the associations between the FFQ and each of the short-term measurements before and after adjustment. In addition, weight kappa values were calculated to evaluate how well the MSMadjusted data categorised individuals compared to the mean of 2-d 24HRs data . For this agreement analysis, each food group intake estimated by FFQ and the two short-term measurements was categorised into tertiles. The proportions of participants categorised in the same tertile, in an adjoining tertile, and in contrary tertile by both methods were calculated. The statistical package SAS for Windows 9.2 (SAS Inc., USA) was used for all statistical analyses except the MSM procedure.

Results
A total of 333 children completed both the FFQ and 2-d 24HRs and were selected to analysis. Of those, 168 boys (50.5%) and 165 girls (49.5%) were included, and the mean age ± Standard deviation (SD) of the study population was 4.3 ± 1.1 y. The mean BMI ± SD was 17.9 ± 3.7, and 53.5% of children were below the 50th percentiles as set by the 2007 Korean National Growth Chart provided by the Korean Centres for Disease Control.

Estimated usual intake distributions
In both of the 2-d 24HRs, the groups with 20% or greater children reporting no consumption were beans, seeds, mushrooms, and beverages; however, all of the children reported consuming grains, vegetables, oils and seasoning groups during their 2-d 24HRs (Table 1). After adjusting the data using the MSM procedure and the daily frequency values derived from the FFQ, the true rate of non-consumption for all food groups was less than 1% except for the beans group. Table 2 lists the descriptive statistics using percentiles and 4 moments (mean, SD, kurtosis, and skewness) to compare the usual intake estimated by the MSM procedure and the mean of 2-d 24HRs data for the 16 food groups. As expected, the intake distribution variance for all food groups became smaller when adjusted by the MSM procedure and considering the within-individual variation, while the mean was not significantly different before and after adjustment. After the MSM procedure adjustment, the usual intake distribution showed no Table 3 estimated zero intakes in the 5th percentile except for seeds and mushrooms. On the other hands, the mean of 2-d 24HRs data indicated that non-consumption was up to the 75 th percentile for mushrooms and the 50 th percentile for seeds and beverages.

Validation
The weighted kappa value ranged from 0.08 (mushrooms) to 0.25 (milk and dairy products) and from 0.10 (oils) to 0.41 (milks and dairy products) for the mean of 2-d 24HRs data and MSM-adjusted data, respectively ( Table 3). The weighted kappa value was not significant for 6 food groups (potatoes, sugars, seaweeds, oils, beverages and seasonings) for the mean of 2-d 24HRs data, however only two insignificant weighted kappa value was found for the MSM-adjusted data (sugars and seasonings). Except for the seasonings and sugars, the weighted kappa value of all the other 14 food groups increased after the MSM procedure with a range of 0.02 (oils) to 0.26 (mushrooms).
For the mean of 2-d 24HRs data, the exact agreement using the tertile classification varied from 31% (potatoes) to 44% (milk and dairy products), and the exact agreement plus adjoining agreement varied from 71% (mushrooms) to 89% (milk and dairy products). For the MSM-adjusted data, the exact agreement increased in most of the food groups. The range was from 1% (oils) to 12% (mushrooms). The exact agreement did not increase in the sugars, seaweed and seasoning food groups (Table  3).
The calculated correlation coefficients were not significant for the sugars and seasonings groups for both data (mean of 2-d 24HRs data and MSM-adjusted data). In addition, fish and shellfish, seaweeds and oils groups showed the statistically non-significant correlation coefficients for mean of 2-d 24HRs data only. Median spearman correlation values were 0.20 (range: 0.11 to 0.40) for the mean of 2-d 24HRs data and 0.35 (range: 0.14 to 0.60) for MSM-adjusted data, respectively. For the MSM-adjusted data, the Spearman correlation coefficients of all 16 food groups increased after adjustment with the MSM procedure from 0.03 (Beverages) to 0.34 (Mushrooms).

Discussion
In the present study, the 2-d 24HRs that were insufficient to represent usual food group intake of the study participants were adjusted by a previously developed statistical method. The MSM-adjusted data and mean of 2-d 24HRs data were compared to evaluate the validity of an FFQ. Based on our results, the estimated usual food intake, as adjusted by the MSM, may provide more reliable data than the mean of 2-d 24HRs data and presented a stronger correlation with FFQ.
In a recent FFQ validation study (Fumagalli et al., 2008) involving 5 to 10 year-old Brazilian children, relatively higher correlation coefficients ranging from 0.2 to 0.81 were reported, even though they collected more days of data to validate the FFQ and conducted a nutrient analysis. In contrast, quartile classification indicated only weak to moderate agreement between the dietary assessment tools (Fumagalli, et al., 2008). The authors reported kappa statistics that ranged from 0.046 to 0.285, which were similar to our MSMadjusted data. Klohe and colleagues (Klohe et al., 2005) also validated an FFQ developed for children 1-to 3-years-old from low-income families. They applied 3-d diet records (DRs) as a reference method. With a 9 food group analysis, the authors reported Spearman correlation coefficients ranging from 0.53 (soups) to 0.84 (nonstarchy vegetables), which were clearly higher correlation coefficients than observed in our study. However, the quartile classification agreement was not notably different that the results of our study. Thirty-six percent of children were classified into the same quartile of food group intake, and 78% into the same or within one quartile. This level of agreement was also similar to that reported by some studies in neighbouring countries (Ogawa et al., 2003;Shu et al., 2004;Xu et al., 2004;Lee et al., 2006).
Adjusting the within-individual variation to validation the FFQ with short-term dietary measurements has been previously examined (Field et al., 1999;Carithers et al., 2009;Araujo et al., 2010;Fayet et al., 2011;Mouratidou et al., 2011), and several statistical approaches to estimate the usual intake distribution have been employed. A method using the ratio of the within-individual to between-individual variation to deattenuate the intake data and the Iowa State University method (ISU method) (Guenther et al., 1997), have been used widely. Field and colleagues (Field, et al., 1999) reported increased correlations between the FFQ and 24HRs among junior high school students after adjusting the within-individual variation using the ratio of the within-individual to between-individual variation. In contrast, in another study (Mouratidou, et al., 2011) deattenuated with the ISU method the correlation coefficients between the FFQ and the reference method (3-d mean of food record) indicated an average reduction of 22% for the deattenuated and energy-adjusted data. In our results, the coefficients and weighted kappa value were consistently higher in most of the food groups after adjustment with the MSM.
In this study, we attempted to combine 24HR data with an FFQ. This kind of method, which is called an NCI method, was already proposed and developed recently by NCI groups (Kipnis et al., 2009;Freedman et al., 2010;Tooze et al., 2010). This concept has emerged as an issue related to episodically-consumed foods, which are likely to present some challenges for statistical modelling (Tooze, et al., 2006), including an individual zero-intake problem from short-term measurements. In a nutrient analysis, an individual zero intake of a given nutrient rarely occurs as reported by 24HRs or other short-term measurements, however, at the food group level, the lower the number of days of dietary data was collected, the larger the probability of a participant reporting non-consumption of the food group. To overcome this challenge, Subar and colleagues (Subar, et al., 2006) developed and validated the Food Propensity Questionnaire for use as a covariate to estimate usual food intake. The authors found strong and consistent relationships between FFQ frequency and the 24HR consumption probability.
The MSM was proposed to determine the usual intake distribution from short-term measurements by combining the probability and the amount of consumption with the incorporation of covariates into models (Haubrock et al., 2011). Unlike the ISU or NCI method, which compute usual intake with simulated data, the MSM calculates usual intake at the individual level. Therefore, we could calculate the correlation coefficients and perform tertile classification analysis. Haubrock and colleagues applied the MSM to 383 participants who completed 2-d 24HRs and one FFQ. After two simulations, the authors concluded that the MSM is a useful method for estimating the usual food intake distribution. In our study, the MSMadjusted data showed relatively higher agreement than the mean of 2-d 24HRs data.
Many FFQ validation studies of FFQ have been previously conducted using various reference methods, and the majority of reference methods used in these studies were short-term measurements, such as 24HRs and diet records over the course of several days. When Molag and colleagues (Molag et al., 2007) reviewed 42 FFQ validation studies in 2007, the number of days for data sampled using the reference method ranged from 1 to 28. The authors conclude that the correlation coefficients in nutrient analysis were higher when the number of days for reference method was between 8 to 14 days than when range was 1 to 7 days. In this study, we used collected two days of data using our reference method. Two days is the minimal number of days that, conceptually, allowed us to control within-individual variation. Therefore, if more days of data were collected, the intake data would be more reliable overall. In addition, our two 24HRs could not be considered independent from each other because they were conducted within one week of each other. Another limitation of this convenience sample study was the lack of inclusion of useful covariates into the models, such as day of the week, season, sequence effects, interviewer and other general participant information. In this study, the only available individual covariates were age, sex, and BMI, which allows the direct evaluation of covariate effects on usual intake in addition to correction for measurement error .
To the best of our knowledge, this is the first study to evaluate the validity between FFQ and FFQ incorporated short-term measurement among Korean children. When the number of days for short-term measurement is relatively short, our approach could help to bridge the gaps between short-term measurement and longterm measurement. The results of this study indicate that the estimated usual food group intake of raw 2-d 24HRs using frequency values derived from the FFQ demonstrated a stronger validity with the FFQ after adjustment with the MSM in Korean children. When assessing an individual's usual intake with relatively less number of days, as well as the validity of the FFQ, multiple short-term measurements (24HRs) combined with the frequency variable of long-term measurements (FFQ) can be considered an alternative method.