Article Text
PDF
Abstract
Introduction The Government of Thailand has drafted legislation to protect children from the harmful impact of unhealthy food (including beverages) marketing. Local evidence on Thai children’s exposure to, and the impact of, this marketing is necessary to, first, support the adoption of this Law and, second, to contribute to assessing policy implementation and effectiveness. This study aimed to develop and validate a questionnaire for examining Thai children’s exposure to unhealthy food marketing and its impact on diet-related outcomes.
Materials and methods To design the questionnaire, we first conceptualised the range of impacts of unhealthy food marketing on children’s diet-related outcomes based on published frameworks. These outcomes related to food brand loyalty, preference, purchase and consumption. We conducted a literature review to gather related questions used in earlier surveys to assess these outcomes. Using these questions, we assessed content validity with five experts. Face validity and reliability were assessed for 32 children. Validity was assessed using Content Validity Index (CVI) and Kappa statistics. Reliability was assessed using Cronbach’s alpha and intraclass correlation coefficients (ICC).
Results We identified 15 survey questions that had been used to assess the outcomes of interest. The CVI of all questions was 1.0, indicating perfect agreement with each question’s relevance by the experts. Most questions were perceived to be easy to read and comprehend, suggesting face validity. Cronbach’s alpha and ICC of all questions were both 0.75, demonstrating internal consistency across responses to questions about, separately, brand loyalty, preferences, purchase and consumption.
Conclusion The final 15-item questionnaire provides a valid and reliable survey instrument for measuring the impact of unhealthy food marketing on children’s diet-related outcomes. This instrument will be useful for gathering local evidence on the need for policy reform to protect children from unhealthy food marketing in Thailand. The instrument also provides a cost-effective approach for generating evidence in other jurisdictions to propel policy actions. This is a pilot study and the validity and reliability needs further testing after a larger-scale roll-out.
- Nutrition assessment
- Dietary patterns
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Unhealthy food marketing impact children’s diet-related behaviours.
WHAT THIS STUDY ADDS
There is no evidence from Thailand on the impact of unhealthy food marketing on children’s diet-related outcomes. To generate such evidence, we reviewed the evidence to develop and then test a questionnaire to assess important outcomes.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The final 15-item survey instrument had good face and content validity and internal consistency (reliability) for measuring the impact of unhealthy food marketing on children. This tool will be applied in a sample of children in Thailand to support the progression of draft legislation to protect Thai children from this marketing.
Introduction
Non-communicable diseases (NCDs) are responsible for the deaths of 41 million people annually.1 Among all behavioural risk factors for NCDs, dietary factors are the biggest contributor,2 including for cardiovascular diseases, cancers and diabetes linked to obesity.1 In Thailand, the consumption of unhealthy foods and beverages contributes to excess body weight and hypertension.3 The most recent population survey data from 2019 to 2020 indicated that the percentage of overweight or obese Thai persons aged 15 and older increased from 37.5% in 2019 to 42.2% in 2014.4
Food marketing is an important risk factor for unhealthy diets,5 particularly for children.6 In Thailand, a small number of studies have measured children’s potential exposure to unhealthy food marketing on television and in social media and video-sharing platforms, including YouTube and Facebook.7 8 These studies found that children have high exposures to unhealthy food and beverage marketing, which often use marketing techniques such as promotional characters and sales promotions.7 8 The large and accumulating global body of evidence shows that exposure to food marketing is significantly associated with children’s food intake, choices and preferences.9
Restricting children’s exposure to unhealthy food marketing is an important policy intervention to support healthy diets.10 In Thailand, the Bureau of Nutrition, which sits within the Department of Health in the Ministry of Public Health has developed a draft Act to regulate food and beverage marketing that affects children’s health. One component of the Act aims to reduce the exposure and power, or persuasive techniques, of food and beverage marketing to children aged under 18 years to improve children’s dietary intakes and health outcomes.11
To support the progression of the draft Act through to its endorsement, local evidence on Thai children’s exposure to unhealthy food marketing and its impact on children’s diet-related outcomes is needed. Such evidence provides a counter to the likely policy opposition from the food, media and advertising industries. Public reporting of evidence can also raise societal awareness of the issue and support for the policy. Leveraging local evidence in this way is important for policy progression.12 This study aimed to fill this gap by developing and testing a questionnaire for measuring impact of unhealthy food marketing to children in Thailand.
Materials and methods
Study design
The study was divided into four steps: (1) identification of existing questions related to the outcomes of interest from the UNICEF conceptual framework (described below); (2) drafting of the questionnaire; (3) finalising the questionnaire for testing by a project advisory group and five experts; and (4) pilot test of face validity and reliability with 32 children (see figure 1).
Overview of the three phases in the development and validation process of the unhealthy food marketing impact questionnaire.
Step 1: Identification of existing questions
Theoretical framework
To assess the relevant impacts of unhealthy food marketing on Thai children, our study used a conceptual framework developed for UNICEF to outline the range of impacts of food and non-alcoholic beverage marketing that effect children’s health13 (see figure 2). This conceptual framework was chosen as it illustrates the comprehensive impacts of food marketing, as given in the earlier WHO Set of Recommendation on the Marketing of Foods and Non-alcoholic Beverages to Children.14
Adapted from the UNICEF conceptual framework13.
The framework was used to identify important outcomes to be assessed in the questionnaire. Our study focused on unhealthy food marketing that increased brand loyalty, preferences, purchasing and consumption of unhealthy food. Other more distal outcomes in the framework, including weight, are less appropriate to be assessed by self-report due to misreporting biases.15 Measures of food literacy were excluded as instruments are typically lengthy.16
Literature review
A literature review was conducted to identify studies that had collected information about the impact of unhealthy food marketing on the outcomes of interest from the UNICEF conceptual framework (see figure 2). We were specifically interested in studies that had used cross-sectional quantitative surveys with children (aged up to 18 years). Searches of academic literature were conducted in PubMed, Thai Journal Online and Google Scholar, and of grey literature through Google search function, from 1 January 2011 to 15 April 2023. Search terms were ‘impact of food marketing’ and the search also included synonyms and related terms (using the Boolean operator ‘and’) AND (exposure*) AND (children* OR adolescent*). The search strategy was adjusted to each database. Studies were included if they met the following criteria1: primary and cross-sectional studies2; study participants were aged up to 18 years3; studies that examined the impact of unhealthy food marketing exposure or power across any media or setting; and4 outcomes included brand loyalty, preference, pestering or purchasing and consumption behaviours. The conceptual framework developed for UNICEF and thematic analysis were used to analyse data from the literature review. Information was extracted from eligible studies on the survey methodology, the questionnaire topics, phrasing and response categories and the process undertaken for question development.
Step 2: Drafting of questionnaire
Synthesis of literature review and theoretical framework
An initial 41 questions were identified that covered a diverse range of topics related to the impact of unhealthy food marketing on the outcomes of interest. Questions were then selected using the UNICEF conceptual framework. The questions were grouped according to the outcome of interest being measured. Then, the questions that were overlapping were removed or combined to ensure that key constructs were included. In addition, we selected questions that the words used for the questions were kept as simple as possible and technical terms were avoided.
Furthermore, the questions were adapted to align with the provisions for restricting food marketing exposure and power that were outlined in the draft Act.11 The nine provisions were1 cartoon characters on food packages2; use of celebrities and online influencers to endorse products3; sales promotions, such as giveaways, freebies, prize draw and free delivery4; discounts and price promotions5; sponsorship arrangements with schools, including the provision of gifts, giveaways and financial support6; child-targeted groups, clubs and online communities that are supported by foods brands7; sharing brand content through online communities8; in-store retail promotions, including direct marketing; and9 text (SMS) or email promotions. According to a narrative review of WHO on food marketing, the nine provisions were popular food marketing techniques that food industry used to appeal to children.17
The draft questionnaire comprised 20 items. Additional questions related to children’s sociodemographic characteristics were included, based on relevant questions identified in the literature in Step 1.
Step 3: Finalizing the questionnaire for testing
Advisory group meeting
Four advisory meetings were organised. A series of 3-hour online meetings were held. For content validity and face validity, the appropriateness, the readability and difficulty of the questionnaire were evaluated and adjusted by the advisory group. The advisory group consisted of five experts in the area of children, food marketing, health, health promotion and health policy. The selected questions and the response categories for the questions were reviewed by the advisory group at several stages during questionnaire validation. Based on the comments and suggestions of advisory group, the questions and the response categories for the questions were modified, changed and eliminated.
Validity assessment by experts
Purposive sampling was used to select five experts (not part of the advisory group) who each had more than 5 years’ research experience in communications (n2), advertising and marketing (n2) or food marketing policy (n1). These persons judged the relevance and clarity of the questions based on a 4-point scale (1=‘not relevant’ or ‘not clear’ to 4=‘very relevant’ or ‘very clear’).18
Step 4: Pilot test
Focus groups with children
Two schools were randomly selected across Thailand. All schools were listed and a random number generator was used to select the school samples. For face validity, 32 children aged 10–18 years, similar to the target group, were recruited through convenience sampling at two schools: one primary school and one secondary school. Three focus groups consisted of 10 children aged 10–12 years, 12 children aged 13–15 years and 10 children aged 16–18 years. In the recruitment process, the research team sought permission from the school principals to collect data in their schools. After obtaining the school permission, the team asked for written consent for research participation from children and their guardians. The focus group discussions were conducted by the lead researcher-led NJ. She was supported by one notetaker. Each discussion lasted 45–60 min. In the discussions, the children were asked to discuss each question and comment on any ambiguity and complexity, provide feedback and identify what they thought the question was seeking to measure.
Reliability assessment with children
Internal consistency of the questionnaire was assessed. The questionnaire was given to the same children as in step 4.1. Questions related to the same construct (outcome) were compared.
Data analysis
Content validity
The Content Validity Index (CVI) for survey items (I-CVI) is calculated as the proportion of experts (0%–100%) who judge the item as relevant or clear (rating 3 or 4).18 I-CVI values >0.79 indicate that the question is relevant, between 0.70 and 0.79, the question needs revisions, and if the value is below 0.70, the item is eliminated.18 CVI can also be given for an overall questionnaire, referred to as a scale-level CVI (S-CVI), which is defined as the proportion of items in a questionnaire that achieved a rating of 3 or 4 by the experts.18 A S-CVI/UA(A S-CVI/UA (UA referred to Universal Agreement) ≥0.8 and a S-CVI/Ave (S-CVI/Ave (Ave referred to Average CVI)) ≥0.9 represent excellent content validity.18
Kappa statistic is a Consensus Index of inter-rater agreement that adjusts for chance agreement19 and it is an important supplement to CVI because Kappa provides information about degree of agreement beyond chance.19 After calculating I-CVI for all questions, kappa was computed and kappa values of >0.74, 0.60–0.74 and 0.40–0.59 are considered as excellent, good and fair, respectively.18
Reliability
The internal consistency of the instrument, as well as the degree to which items in a scale are correlated, was determined by Cronbach’s alpha. Cronbach’s alpha values greater than 0.7 were considered acceptable.19 The correlations within a single class of data to measure reliability were evaluated using the intraclass correlation coefficient (ICC), where an ICC >0.7 was acceptable.20 SPSS 18.0 was used to calculate all statistics.
Results
Step 1: Identification of existing questions related to the outcomes of interest from the UNICEF conceptual framework
Literature review
The search identified 374 records. After removing duplicates (n=40), 334 titles and abstracts were screened. Of full-text articles assessed, 17 studies were included in the review. After checking the reference lists of included articles, one additional questionnaire entered the selection process (see figure 3). Most studies had been conducted in the USA21–26 and Australia.27–29 Survey questions that were identified in the document analysis were grouped as measuring either1 food and beverage marketing exposure and power,2 impact of unhealthy food marketing or3 sociodemographic characteristics of children.
Flow chart for the inclusion of reviewed studies.
Food and beverage marketing exposure and power
Exposure to unhealthy food marketing
There were two measures of exposure to unhealthy food marketing identified from previous surveys. This included indirectly asking children about their media use, as a proxy for marketing exposure. These types of questions asked about the type and frequency of media use (eg, ‘How often do you use(media) per weekday? And per weekend day?’).29–32 Other questions asked about exposure to food marketing directly, typically split across media types (eg, broadcast media, internet, print media, direct marketing and outdoor advertising)23 27–29 32–34 (online supplemental file A).
Supplemental material
Power of unhealthy food marketing
The ‘power’ of food marketing refers to the content, design and execution of the marketing message, influencing the impact of the marketing communication.17 For our study, power of unhealthy food marketing refers to the nine provisions of a draft Act to regulate food and beverage marketing that affects Thai children’s health.11 According to review, the response categories of power of unhealthy food marketing were measured as the frequency of children’s exposure to marketing techniques in the past 7 days.35 After considering the nine provisions for restricting marketing power in the draft Act, the draft questionnaire included questions on children’s reported exposure (Yes/No) to nine marketing techniques in the past week (online supplemental file A).
Impacts of food and beverage marketing
Questions related to four key outcomes from the conceptual framework adapted from the UNICEF conceptual framework were identified (online supplemental file A).
Brand loyalty
In previous surveys, popular food and beverage brands (eg, identified through sales data) were shown to children and used to measure brand loyalty. Variables to measure brand loyalty consisted of brand preference, brand attitude, brand purchasing intention and consumption of brand. There were two measures used to assess brand preferences.22 26 Brand attitudes measure assessed brand popularity using a 5-point scale (‘Advertising for this brand is aimed at someone like me.’). For brand purchasing intention and consumption of brand, how frequently they used their own money to buy and eat the brand.26 29
Purchasing and purchase requests
Questions asked children whether they had1 asked their parents to buy or2 bought unhealthy food they had seen advertised31 or in the last month.27 Response options were Yes/No.36
Eating
Food Frequency Questionnaires (FFQ) were used to measure consumption of foods with multiple choices responses21 24–27 31 33 or a 7-point scale.28 There were different timeframes used in FFQs. Six studies used the last 7 days21 24–28 and five studies used the past 30 days26 31 33 as a reference point. A 24-hour dietary recall was used to measure food consumption among children in one study.32 The five food groups of interest were ready-to-eat breakfast cereals, salty snacks, sweets and desserts, sugar-sweetened beverages, and milks and yoghurts. Consumption of ‘high-in’ (negative nutrient) foods and beverages was expressed both as absolute energy intake (kilocalories) and energy-adjusted (percent of total daily energy intake)32 (online supplemental file A).
Socio-demographic characteristics of children
Gender, age, race, mother’s education or parent’s education, socioeconomic status and family income were common variables used to measure sociodemographic characteristics of children.21–37 Measures of sociodemographic characteristics were included in the pool of questions but were not included in the validity and reliability assessment.
Step 2: drafting the questionnaire
An initial pool of 25 questions was generated to measure unhealthy food and beverage marketing exposure, power and impact. The 25 questions were divided into four parts: food and beverage marketing exposure and power; impact of unhealthy food marketing; eating behaviours; and sociodemographic characteristics of children (table 1).
The initial pool of 25 questions
Food and beverages in this study referred to eight unhealthy food groups that were classified by criteria from the Department of Health, Thailand.38
Step 3: Finalizing the questionnaire for testing
Advisory group meeting
The advisory group was asked to assess the questionnaire for appropriateness and readability. After redundant questions were eliminated, the final questionnaire included 15 questions, including part 1 unhealthy food marketing exposure and part 2 unhealthy food marketing exposure (table 1). Then all questions were assessed in order to examine validity and reliability.
Content validity
The content validity index (CVI)
CVI for each question was calculated. The 15 questions in parts 1 and 2 of the questionnaire (table 1) were sent to the five experts. The experts judged the questions’ relevance, clarity, length, simplicity and the need to delete or add questions. The experts indicted that the questionnaire covered the full range of topics and did not eliminate any questions. All questions had a CVI equal to one. As such, all questions were retained in the questionnaire.18 The S-CVI/UA for the questionnaire was 1.00, and S-CVI/Ave was 1.00. These results indicated the questionnaire had excellent content validity.18
Step 4: Pilot test
Face validity
Face validity was achieved through focus group discussion with 32 children to assess each item for ambiguity and complexity. All 32 children indicated that most of the questions were generally easy to read and comprehend, except for a few words that were changed to meet children’ considerations. For example, ‘format’ was replaced with ‘technique’ as recommended by the children.
Reliability
Table 2 shows the Cronbach’s alpha and ICC for the brand loyalty questions. The alpha value for the questionnaire indicated appropriate internal consistency.19 The ICC of the entire questionnaire was 0.745 (ranging from 0.567 to 0.863). This result revealed satisfactory stability.20
Cronbach’s alpha and ICC for brand loyalty domain of the questionnaire
Discussion
This study involved the design of a questionnaire based on a comprehensive review of available questionnaires and examined the psychometric properties of the proposed questionnaire. The questionnaire development process involved a review of previous questionnaires that included measures of unhealthy food marketing exposure, power and impact. The UNICEF conceptual framework was used to identify relevant outcomes.13 In addition, we adapted the questions related to marketing exposure and power to align these with the nine provisions outlined in the draft legislation from the Government of Thailand to control food marketing.11
According to published acceptable cut-off values, Cronbach’s alpha values greater than 0.7 are considered acceptable19 and ICC >0.7 are acceptable.20 The findings from our study showed appropriate internal consistency among the related questions, based on Cronbach’s alpha, and satisfactory stability, based on intraclass correlation. However, it was not possible to compare to other studies because some studies may not show the validation of the questionnaire.21–37
The questionnaire validation from our study showed strong content and face validity, and good inter-item reliability. While the sample of five experts and the advisory group used for assessing content validity was small, this was aligned with evidence that suggested a 5–10 expert panel is considered sufficient.39 The good face validity adds further confidence that children will find the questionnaire acceptable and understandable.
Our questionnaire is appropriate for measuring the impact of food marketing, with good psychometric properties as indicated by the statistical tests. However, there are other, more scientifically rigorous methods for assessing the impact of food marketing that should also be considered. This includes surveys that capture objective measures, such as sales data, and other study designs, such as longitudinal and experimental studies.6 A large and accumulating number of studies have experimentally manipulated children’s food marketing exposure and assessed the impact on their immediate food choice and intake.9 However, such experimental studies require technical expertise, while studies using sales data rely on the availability of such data. Questionnaires that ask children about their responses to food marketing offer a cost-effective and rapid approach for gathering information on the impact of food marketing, which can be compared across population groups and over time.
The strength of our study is in its rigorous examination of validity and reliability of survey measures. Our study examined both content and face validity, and reliability both internal consistency and intraclass correlation. Input from a range of experts, besides a high CVI builds confidence that this questionnaire accurately and reliably assesses impact of unhealthy food marketing. The number of children who participated in the reliability testing was 32 children, which is an appropriate sample size for this purpose.40There are some limitations in this study. First, for content validity, experts’ evaluation is subjective. Hence, our study is subjected to bias that may exist among the experts. However, experts are asked to suggest other questions for the questionnaire which may help minimise this limitation.19 20 Second, our study did not assess CVR and construct validity by using factor analysis because the number of questions was small. Finally, our study does not measure brand awareness. Therefore, the future study should be measured.
Conclusions
This study developed a valid and reliable questionnaire instrument to examine unhealthy food marketing impact on Thai children. The tool will be used and generate information on the impact of food marketing on children in Thailand, in order to support the draft Act. The questions were derived from questionnaires developed for children from mostly high-income Western countries but have demonstrated good psychometric properties with children in our sample. As such, researcher in other countries can consider applying the tool, with initial piloting, to examine outcomes in their population of interest.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Institutional Review Board of Institute for Population and Social Research at Mahidol University, Thailand (COA. No.2023/07-172). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would also like to thank the advisory group for their useful comments and Kasama Yakoh and Salakjit Chuenchom for their dedication and hard work during the data collection. We also are gratefully acknowledged to Thai Health Promotion Foundation for funding this project.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors NJ and SPu: Conceptualisation; NJ, BK and SPu: Methodology; NJ: Data collection and Formal analysis; NJ, SPu and SPh: Writing – original draft; NJ, BK, SPu and SPh: Writing – review and editing. All authors read and approved the final manuscript. We give the rights to the corresponding author to make necessary changes as per the request of the journal, do the rest of the correspondence on our behalf and NJ acts as the guarantor for the manuscript on our behalf.
Funding This study was funded by Thai Health Promotion Foundation [grant number 64-00-0197], Thailand.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.