Validity and Reliability of a Dish-based, Semi-quantitative Food Frequency Questionnaire for Korean Diet and Cancer Research

This study evaluated the validity and reliability of applying a newly developed dish-based, semiquantitative food frequency questionnaire (FFQ) for Korean diet and cancer research. The subjects in the present study were 288 Korean adults over 30 years of age who had completed two FFQs and four 3-day diet records (DRs) from May 2008 to February 2009. Student’s t-tests, Chi-square tests, and Spearman’s rank correlation coefficients were used to estimate and compare intakes from different dietary assessment tools. Agreement in quintiles was calculated to validate agreement between the results of the second FFQ (FFQ-2) conducted in February 2009 and the DRs. Median Spearman’s correlation coefficients between the intake of nutrients and foods assessed by the FFQ-1 and FFQ-2 were 0.59 and 0.57, respectively, and the coefficients between the intake of nutrients and foods assessed by the FFQ-2 and the DRs were 0.31 and 0.29, respectively. The quintile classifications of same or adjacent quintile for intake of nutrients and foods were 64% and 65%, respectively. Misclassification into opposite quintiles occurred in less than 5% for all dietary factors. Thus this newly-developed, Korean dish-based FFQ demonstrated moderate correspondence with the four 3-day DRs. Its reliability and validity are comparable to those reported in other studies.


Introduction
Cancer prevention research has been a primary focus of global health research (Bode & Dong, 2009). Many cohort research findings emphasize that environmental and lifestyle risk factors are preventable for reducing the large and increasing burden of cancers worldwide (Danaei et al., 2005). Although nutrition has long been known to be a major risk factor in cancer etiology (Doll & Peto, 1981), elucidating the relationship between diet and disease has been challenging, largely due to the multiple causes of cancer and complex dietary behavior. Therefore, accurate and reproducible dietary survey instruments are necessary for making advancements in diet and cancer research, particularly in populations with culturally sensitive dietary practices and a high prevalence of certain cancers, such the Koreans.
Many dietary assessment methods have been used to study how diet relates to cancers and chronic diseases. Of the many methods used, food frequency questionnaires (FFQ) have been the primary method for dietary assessment in epidemiological studies. FFQs have been particularly useful in situations that do not require quantitative assessment of the total diet or when financial resources are limited, while still focusing on the relationship between diet and such conditions as cancer and chronic diseases. The validity of FFQs has been challenged for their estimation of the usual intake in relation to usual and representative foods or portion sizes for target respondents and for grouping multiple foods within single listings (Lee & Nieman, 2007). These challenges are important, as the 100 to 150 foods and food groups in FFQs are expected to represent foods commonly consumed by the respondents. For these reasons, FFQs need to be validated and/or calibrated against other reference methods such as multiple-day diet records (DRs) or multiple 24-hour recalls, which give detailed and quantitative estimates of dietary intake (Block & Hartman, 1989;Feskanich et al., 1993;Pietinen et al., 1988;Willett et al., 1985). In this study,

546
we assessed the validity and reliability of a dish-based, semi-quantitative FFQ for diet and cancer research by comparing the results with 3-day DRs over four seasons in a sample of Korean adults over 30 years of age.

Participants and study design
The 288 participants (115 men and 173 women) in the present validation study were apparently healthy Korean adults over 30 years of age (range 30-66 years) ( Table 1). The mean age of the study population was 44.7 years (men: 43.4 years, women: 45.6 years). Of the 341 eligible participants initially recruited, only 288 had a complete data set of two FFQs and a four seasonal 3-day diet record (DR). The subjects were recruited with advertising flyers and website advertisements from Seoul and Gyeonggi provinces in Korea. Only subjects free from chronic diseases, such as diabetes, hypertension or cancer, and who had not modified their usual dietary practices during the previous year were included. The subjects participated in the study between May 2008 and February 2009. The study was approved by the institutional review board of the Seoul National University (IRB No. 0803/001-001), and all participants provided written informed consent.
At the onset of this validation study in May 2008, subjects completed a newly developed, dish-based, semiquantitative FFQ that was developed for cancer research in Korea (FFQ-1) and a 3-day DR. Two additional 3-day DRs were collected in August and November 2008. The final data collection was concluded with the second FFQ (FFQ-2) and a 3-day DR in February 2009. The four 3-day DRs were planned to collect data over four different seasons in Korea because the availability of seasonal produce and popularity of some dishes can vary by season. The recruitment and interviews were conducted by trained interviewers, all of whom had college degrees in food and nutrition.

The FFQ
The details of the dish-based, semi-quantitative  FFQ have recently been reported in a separate paper (Park et al., 2011). The comprehensive and current FFQ instrument consisted of 112 Korean dishes that contributed a significant number and amount of cancerrelated dietary factors and other nutrients based on the diet intake reported by participants in the Korean National Health and Nutrition Examination Survey (KNHANES). The cancer-related dietary factors were selected from the World Cancer Research Fund/ American Institute for Cancer Research's report, titled "Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective" (WCRF/AICR, 2007). The frequency categories varied among dish types to allow respondents to define their daily intake with greater specificity (Loma Linda University Adventist Health Studies, 2002). The portion sizes (small, medium or large) for each dish represented the amount that had been reported consumed in grams by the KNHANES subjects at the 25th, 50th, and 75th percentile levels, respectively. The FFQ asked respondents to report their intake over the previous year.

Data collection
The first FFQ (FFQ-1) was completed via a faceto-face interview by trained staff members. A portion of the second FFQ (FFQ-2) was conducted through a mail survey. Upon return of the FFQ questionnaires, the trained interviewers reviewed the questionnaires for completeness, and, if necessary, followed up by telephone to clarify any ambiguous or incomplete information. All completed FFQs were coded by a newlydeveloped computer program to estimate the intake of nutrients and foods. At the first interview, one preliminary 24-hour recall was practiced with all participants to provide instruction on how to complete the DRs using two-dimensional portion size aids consisting of circles and blocks and a full-size picture book of major dishes. Survey data were coded using WebDS24, which is a computer program developed at the Human Nutrition Lab at Seoul National University (2005).

Statistical analysis
The foods and nutrients intakes were estimated from the FFQ and the four 3-day DRs using the Korean food composition table, 6th revision (Rural Resouces Development Institue, 2001). Intakes estimated from the FFQ-1 and FFQ-2 were compared to one another using Wilcoxon rank sum tests for reliability, whereas intakes estimated from the FFQ-2 and the four 3-day DRs were compared through Wilcoxon rank sum tests for validity. The Spearman's rank correlation was calculated to compare the daily total intake and the daily total intake per 1,000 kcal collected by the FFQ-1 to those by the FFQ-2 for reliability and those collected by the FFQ-2 to those by the four 3-day DRs for validity. The Spearman's rank correlation was used because distributions of intakes were not normally distributed. The results of the FFQ-2 administered at the end of one year were used for validation because the time frame of "over the previous year" corresponded with the time when the four 3-day DRs over the four seasons were collected. Agreements in quintile classifications were calculated to validate the agreement on intakes between the FFQ-2 and the four 3-day DRs. All statistical analyses were carried out using Statistical Analysis Software (version 9.1.3, 2006, SAS Institute Inc, Cary, NC).

Results
The mean nutrients and foods intakes estimated by the FFQ-1, FFQ-2, and the four 3-day DRs are summarized in Table 2. Estimated intakes of most nutrients, fruit, vegetables, and carrots reported by the FFQ-1 were higher than those reported by the FFQ-2. The mean intake estimated by the FFQ-2 and DRs were different for most foods and nutrients.
The Spearman's correlation coefficients between the nutrients and foods intakes as estimated from the two FFQs are shown in Table 3. The Spearman's correlation Table 3

. Spearman's Correlation Coefficients 1 for Nutrients and Foods Iintakes between FFQ-1 2 and FFQ-2
Daily total intake Intake/1,000 kcal Total Men Women Total Men Women 548 coefficients for daily total intake varied from 0.41 (fruit) to 0.82 (alcohol). Except for fruit, beans, and fish, Spearman's correlation coefficients were over 0.5. Spearman's correlation coefficients for the nutrients and foods intakes estimated by the four 3-day DRs and the FFQ-2 are shown in Table 4. The correlation coefficients for daily total intakes varied from 0.15 (carrots) to 0.72 (alcohol). The median Spearman's correlation coefficients for daily total intake were 0.31 for nutrients and 0.29 for foods. Median Spearman's correlation coefficients for energy-adjusted intake were 0.29 for nutrients and 0.25 for foods.
The agreements in quintile classifications of nutrients and foods intakes between the FFQ-2 and the four 3-day DRs are also shown in Table 4. The percentage of subjects who were classified in the same or adjacent quintile was 64% and 65% for nutrients and foods, respectively. Consistent classifications of quintiles between the FFQ-2 and the four 3-day DR ranged from 47% (milk) to 18% (iron). Only less than 5% of the subjects were grossly misclassified for both foods and nutrients.

Discussion
The four 3-day DRs were obtained from the sample of 288 adults (115 men and 173 women). Our sample sizes were above the critical sizes as recommended by Willet (1998) (greater than 50 individuals in each demographic group; ideally between 100 to 200 persons) and Cade et al. (2002) (at least 100 participants with increasing numbers and larger age ranges when the number of replicates per participants is limited). The four 3-day DRs indicated the consumption of a variety of dishes and seasonal foods by the subjects. The collective data of 12 days of DRs collected over four seasons are an ideal means to collect usual intake data. Liu (1994) found that the attenuation factors for the FFQ were similar to those associated with a 2-or 3-day DR or recalls through reanalysis of the data in their validation studies. The authors insisted that findings based on intake data obtained by limited days of DR or recalls should be interpreted with caution. Using the Iowa State University method (Dodd et al., 2006) to calculate usual intake with the data from the four 3-day DRs, we found few differences in the raw intakes for foods and nutrients (data not shown). Therefore, we are confident that our data reflect usual intakes for our sample population. Many other studies reported overestimation by the FFQ compared to those by a 24-hour recall or DR (Bingham et al., 1997;Carithers et al., 2009;Deschamps et al., 2009;Jaceldo-Siegl et al., 2010;Papadopoulou et al., 2008;Presse et al., 2009). But in this study, several intakes of nutrients and foods estimated by the FFQ were underestimated. It was not previously clear from other studies whether the FFQ tends to overestimate or underestimate intake. Segovia-Siapco et al. (2008) found that in their FFQ, there was a tendency toward overestimation of foods considered to be healthy, such as fruits and vegetables, and underestimation of foods that are considered less healthy, such as sweets. Zemel et al. (2010) argue that this result may be due to a tendency for respondents to describe their ideal intake, rather than actual intake, in FFQ interviews. These effects might have contributed to an overestimation of vegetables and underestimation of fat reported in other studies (Brunner et al., 2001;Stiegler et al., 2010;Subar et al., 2001).
The overestimation of β-carotene by the FFQ-1 compared to the FFQ-2 may be due to the seasonal variation of fruit and vegetables intake. The intakes of fruit and vegetables estimated by the FFQ-1 (conducted in May, 2008) were higher than those estimated by the FFQ-2 (conducted in February, 2009). Β-carotene rich vegetables, such as red pepper, Perilla leaves, and sesame leaves, are frequently consumed by Koreans.
The reproducibility of this FFQ was reasonably acceptable. Other studies reported correlation coefficients for reliability ranging from 0.27 (fish/eggs/meat) to 0.87 (alcoholic beverages) (Matthys et al., 2007), 0.38 (vitamin C) to 0.64 (carbohydrates) (Villegas et al., 2007), and 0.42 (vitamin A) to 0.8 (cholesterol), which were obtained by a meal-based FFQ (Lyu et al., 2007). Descamps et al. (2009) reviewed previous studies and found that correlation coefficients for FFQs ranged from 0.4 to 0.8 for nutrient intake and from 0.3 to 0.8 for food groups. Similarly, this study reports a higher smallest correlation coefficient for nutrients than for foods.
The Spearman's correlation coefficients between FFQ-2 and DRs for milk, dairy, and alcohol intakes were 0.49, 0.54, and 0.72, as high as those of other studies (Brunner et al., 2001;Deschamps et al., 2009;Huybrechts et al., 2009;Roumelioti & Leotsinidis, 2009). According to Marks et al. (2006), the Spearman's correlation coefficients for alcohol and dairy were 0.85 and 0.75, respectively. In a study from Segovia-Siapco et al. (2008), the correlation coefficient for alcohol was 0.85, but those for vegetables, fruit, and meat/ fish/poultry were 0.25, 0.32, and 0.2, respectively, which match our results. Because the serving sizes of alcoholic beverages are fixed according to their types, the respondents probably easily recalled the amounts of alcoholic beverages they consumed. The high correlation coefficient for alcohol may also be due to the fact that information on alcoholic beverages was collected through an open-ended questionnaire in this FFQ, which obtained more detailed information on alcohol intake. In addition, milk and dairy products are usually consumed independently from meals in Korea, it would be easy to recall frequencies and portions of intake. However, the correlation coefficient for carrots was low; this may be because carrots are not usually consumed in isolation from other foods. The Spearman's correlation coefficient for calcium was similar or higher compared to other studies (Fregapane & Asensio-Garcia, 2000;Hacker-Thompson et al., 2009;Jaceldo-Siegl et al., 2010;Lyu et al., 2007;Toft et al., 2008;Watson et al., 2009;Zemel et al., 2010). The correlation coefficient for sodium was low, but this result was also found in many other studies (Carithers et al., 2009;Fumagalli et al., 2008;Jaceldo-Siegl et al., 2010). Many studies have reported an increase in correlation coefficients after adjusting for energy (Ahn et al., 2007;Brunner et al., 2001;Shim et al., 2002;Watson et al., 2009;Willett, 1998), but we found no increase in correlation coefficients after energy adjustment. Our finding was supported by other studies, which reported either a decrease or no improvement in correlation coefficients (Du et al., 2009;Fumagalli et al., 2008;Quandt et al., 2007;Toft et al., 2008). Quandt et al. (2007) postulated that this result comes from respondents' underreporting of some macronutrients.
The agreement proportion for this FFQ was similar to other FFQs in general. However, the percentage of opposite quintiles was under 5% in all nutrients and foods, and this percentage was lower than the results of other studies (Bolca et al., 2009;Carithers et al., 2009;Deschamps et al., 2009;Huybrechts et al., 2009;Villegas et al., 2007). The percentage of classifications in the same quintile for milk and alcohol was as high as that in other studies (Bolca et al., 2009;Brunner et al., 2001;Deschamps et al., 2009;Huybrechts et al., 2009), which may be because the classification of drinker and non-drinker in these foods was clearly distinguished.
The Spearman's correlation coefficients for the FFQ-2 and for the DRs were not as high as those for other FFQs. Thompson and Byers (1994) found that such correlations are generally in the range of 0.4-0.7 for most foods and nutrients, and Willet indicates that correlation coefficients fell within or above the common range of 0.5-0.7 (Willett, 1998). However, correlation coefficients for FFQs in Korea have fallen short of these results in many cases (Ahn et al., 2004;Ji et al., 2008;Shim et al., 2002).
The reason for low correlation coefficients between FFQ-2 and DRs may be due to typical characteristics of a dish-based FFQ. FFQs assume all respondents consume the same amount of nutrients and food ingredients because they have database made of a representative recipe and portion size for each dish item. Although the database of dishes in the FFQ consisted of recipes collected from a large representative sample (KNHANES), there were 550 limitations because food intakes estimated by the DRs were calculated from various recipes from the sample population and reflected more accurate portions. FFQs usually lose the details of diets (Carithers et al., 2009) and the variation in meals (O'Donnell et al., 1989). It is a systematic error of the FFQ that cannot reflect between-person differences of recipes for each dish item. The size of this error could be different according to dietary culture and get larger in dietary culture with more mixed dishes. In the countries with the dietary culture that people consume many mixed dishes with various ingredients like Korea, nutrient contents and amount of food ingredients in the dish item might be greatly different depending on individuals. Thus, FFQs to evaluate dietary intake of Korean may have a larger systemic error. Ahn et al. (2007) also pointed out that the low correlation coefficients for FFQs in Korea could be a result of the unique way of serving Korean foods, in which several people eat together from a large bowl or dish, and large inter-individual variations in nutrient intakes compared to those in the West. For meal-based FFQs, correlation coefficients ranged from 0.3 to 0.8 in Spain (Fregapane & Asensio-Garcia, 2000), and from -0.04 (not significant) to 0.53 in Taiwan (Lyu et al., 2007). Coefficients of variances (CVs) in nutrient intake estimated by the DRs in our data ranged from 19.9 (carbohydrates) to 69.8 (retinol). For food intakes, CVs ranged from 34.1 (vegetables) to 139.6 (processed meat), which were higher than those for the nutrient intakes. The high CVs in food intake could be the reason for low correlation coefficients in food intakes. These CVs were considerably higher than findings in any other studies, including Western, Korean, and Japanese studies (Ahn et al., 2007;Beaton et al., 1979;Ogawa et al., 1999).
We validated the FFQ with Korean adult population living in Seoul and Gyeonggi province which is located near Seoul. A separate validation study could be needed for target population living in other provinces with unique dietary culture. In addition, it is recommended to calibrate for using the absolute value of the intake of some nutrients and foods estimated the FFQ which have low validity. The limitations of the FFQ should be considered before applying the FFQ for the research. If the FFQ is calibrated with the reference data close to the usual intake, most limitations of the FFQ could be adjusted.
The results of this study showed that the newly developed dish-based, semi-quantitative FFQ is suitable for use in cancer research for Koreans. The reliability and validity is proven by comparison with the 3-day DRs collected over four seasons that were regarded as a usual intake. Although the intake of some nutrients and foods estimated from the FFQ did not highly correlate with those from DRs due to characteristics of Korean dietary culture and limitations of the FFQ, this could be improved by the calibration of the FFQ. Several FFQs have been developed and used for research of the relationship between chronic disease and diet for the Korean population. However, there is no specific FFQ for determining the relationship between diet and cancer, which is the primary cause of death in Korea. In the future cancer research and cohort studies, this FFQ can be an useful instrument for screening cancer-related dietary factors among Korean populations.