Skip to main content

Scope and quality of Cochrane reviews of nutrition interventions: a cross-sectional study

Abstract

Background

All countries face significant challenges from complex manifestations of malnutrition, which affects one in three people globally. Systematic reviews provide ready-to-use syntheses of quality-appraised evidence to inform decision-making for actions. To enhance the utility and quality of future Cochrane nutrition evidence, we described the scope and quality of all nutrition systematic reviews in the Cochrane Database of Systematic Reviews (CDSR).

Methods

We screened all active CDSR records (31 July 2015) to identify reviews and protocols using pre-specified eligibility criteria and definitions. Duplicate, independent data extraction included criteria for inclusion of studies in completed reviews (PICOS). We assessed methodological quality (AMSTAR), use of GRADE, mapped reviews against 2013 Global Burden of Disease data, and categorised the paradigm (medical, lifestyle and socio-ecological) of the review question. We analysed our results using descriptive statistics.

Results

We screened 8484 records, and included 470 (8%) completed reviews (in 45 Cochrane Review Groups (CRGs)) and 169 (7%) protocols (in 41 CRGs) published by 47 of 53 CRGs with reviews. Most completed reviews were produced by the Pregnancy and Childbirth (n = 73), Neonatal (n = 64), Metabolic and Endocrine Disorders (n = 33), Developmental, Psychosocial and Learning Problems (n = 26), Kidney and Transplant (n = 18) and Heart (n = 18) CRGs. Only 27% (n = 129) of reviews had searches for new studies in 2013 or thereafter. Supplementation/supplement interventions were most common (50%; n = 235; majority with micronutrients; 73%, n = 173), followed by food interventions (20%; n = 95). All reviews included randomised controlled trials; about 5% included other designs; 25% used GRADE; the median AMSTAR score was 9 (interquartile range: 7 to 10), 51% were high (AMSTAR 9-11) and 49% moderate (AMSTAR 5-8) quality. More than 80% framed questions using a medical paradigm. For top causes of years-of-life-lost, most reviews addressed preterm birth, diabetes and ischaemic heart disease; for leading risk factors for disability-adjusted-life-years, most targeted childhood undernutrition and high body mass index.

Conclusions

Nutrition reviews comprised 8% of active CDSR records, were widely distributed across nearly all CRGs and reflected the double nutrition burden. This analysis presents a comprehensive description of the scope and quality of Cochrane nutrition reviews, and identifies gaps for future activities to support actions to address the nutrition burden, in line with the current nutrition agenda and impetus.

Peer Review reports

Background

The nutrition landscape has changed over the past 15 years and most countries are now faced with connected, complex and overlapping nutrition burdens. Multiple manifestations of malnutrition (undernutrition, overnutrition and micronutrient malnutrition) at community, household and individual levels, have become the ‘new normal’ [1]. Dealing with malnutrition, in all its forms, in the same place, at the same time is a problem for nearly half of all countries [2], and every nation faces a significant public health challenge from malnutrition, which affects one in three people globally [13].

Finding and implementing effective, scalable and sustainable solutions to address the complex, multi-sectoral nutrition burden is challenging for all stakeholders, particularly since decision-makers often have to deal with diverse and competing interests. Synthesised evidence in which scientifically defensible methods have been used (systematic reviews) can be a valuable tool for translating knowledge to action [4, 5]. Over the past 20 years, Cochrane – a global independent network of about 37 000 contributors from more than 130 countries - has helped transform the way health decisions are made by producing high-quality, relevant, up-to-date systematic reviews published in the Cochrane Library. However, attention has previously been drawn to the lack of nutrition-relevant Cochrane reviews [6, 7]. Furthermore, a recent assessment of Cochrane and non-Cochrane systematic reviews of nine pre-specified interventions from four nutrition areas, reported substantial variability in nutrition-specific methods, and that the scope of some interventions was broad, with poor focus on nutrition-specific indicators [8]. There are issues unique to nutrition topics that should be considered when preparing systematic reviews to aid interpretation of study findings, and enhance overall quality, applicability, and certainty of the evidence. These include background nutrient exposure, baseline nutritional status, multiple biologic functions of nutrients and the variable nature of nutrient interventions [9]. Other challenges in nutrition systematic reviews include lack of true placebo in controlled trials, blinding not always being possible, limitations of different methods or instruments used for assessing dietary intakes, inadequate duration of follow-up to observe a measurable effect over the study duration, as well as poor reporting or ill-defined diets, making it problematic to convincingly attribute an effect to a well-defined treatment compared to a well-defined control. Furthermore, there are the methodological challenges related to evaluating nutrition interventions implemented through sectors other than health, such as agriculture, social protection and trade.

Nutrition is a risk factor for most other dominant health burdens including infectious diseases, maternal and perinatal conditions and non-communicable diseases, especially in low and middle income countries. Nutrition is also an outcome of development processes in society and often connected to issues such as education, care, sanitation and hygiene, women’s empowerment and economic growth, and can therefore be viewed as both an input to and an outcome of sustainable development [10]. Various approaches have been used to conceptualize, describe and address nutritional challenges in society. The multiple interacting factors at many levels can create significant complexity. There may be different intervention components, non-linear pathways and feedback loops between the intervention and outcomes, and interactions between indirect and direct effects of an intervention [11]. Interventions themselves may be simple (e.g. a single component targeting an individual level outcome with a linear pathway) or complex (e.g. systems, components or processes targeting multiple health and non-health outcomes at community level). [11].

Since approaches to assessing interventions are closely linked to putative causal pathways, the perspectives adopted by researchers play an important role in evaluating cause-effect relationships. Lawrence has identified three paradigms (medical, lifestyle and socio-ecological) within which the relationship between food and health can be viewed, and which specifically influence how the causes of nutrition problems are framed [12]. The medical paradigm incorporates a linear view of single or multiple nutrient deficiencies and their impacts on health, without considering context or other components or factors affecting these relationships. The lifestyle paradigm goes wider, considering multi-functional, non-linear relationships between food, dietary patterns and other factors, such as behaviour. Broadest of all is the socio-ecological paradigm, which views the relationships between nutrition and health within social and ecological settings, with consideration of food and other relevant systems. We have graphically represented these three paradigms in Fig. 1, and overlaid the broad categories of actions that are likely relevant to each paradigm, namely nutrition- specific interventions, nutrition-sensitive interventions and an enabling environment for nutrition improvement [2, 13]. Nutrition-specific interventions address the immediate causes of undernutrition, such as inadequate dietary intake, and some of the underlying causes like feeding practices and access to food. Nutrition-sensitive interventions would address some of the underlying and basic causes of malnutrition by incorporating nutrition goals and actions throughout a wide range of sectors. They can also serve as delivery platforms for nutrition-specific interventions [13].

Fig. 1
figure 1

Graphic representation of the three paradigms (medical, lifestyle and socio-ecological) that conceptualize how the relationship between food and health is viewed and how the causes of nutrition problems are framed [12], along with the broad categories of nutrition interventions (nutrition-specific, nutrition-sensitive) and the enabling environment for nutrition improvement to support these interventions [2, 13]

In order to enhance the utility and quality of future Cochrane nutrition evidence and inform the work of the proposed Cochrane Nutrition Field (subsequently established in August 2016), we conducted an overview of the scope and quality of all nutrition-related systematic reviews published in the Cochrane Database of Systematic Reviews (CDSR). Our objectives were to (1) screen all completed and published reviews and review protocols (excluding withdrawn records) published in the CDSR to determine the number and percentage of nutrition-related reviews completed and in progress; (2) determine the criteria for inclusion of studies in completed reviews; (3) assess the methodological quality of completed reviews and document the use of the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) system [14]; and (4) categorise and describe the completed reviews in relation to the global burden of disease [15, 16] and the paradigm (medical, lifestyle and socio-ecological) of the review question [12].

Methods

Selection of reviews

Pairs of authors (CN, SD, AH and JV), using pre-specified eligibility criteria, independently screened all active titles and abstracts, and full-texts if necessary, in the CDSR up to 31 July 2015, to identify nutrition-related reviews and protocols. Nutrition reviews were defined as those that investigated the effectiveness of (1) diets and dietary patterns; foods; formulated, fortified or enriched foods or nutritional products and nutrients and bioactive non-nutrients naturally in foods, delivered orally, enterally or parenterally; (2) nutrition education, promotion, counselling, and programmes; coordination of care or delivery of foods or nutrients; and (3) any policies, programmes or systems which aimed to influence outcomes clearly distinguishable as nutrition-related. We included reviews and protocols that evaluated plant or other components used in food (e.g. cinnamon) and nutrition-related interventions combined with other types of interventions (e.g. diet and exercise). However, we excluded reviews or protocols that assessed pharmaceutical or herbal medicines and products only, those that focused on components of plant origin not used in food (e.g. Echinacea) and where diet or nutrition was not explicit in the title or abstract. Disagreements were resolved through discussion among the authors.

Data extraction and management

We obtained the full texts of all included reviews and protocols and two authors independently extracted the following information for entry into a piloted, standardised data extraction form [17]: unique Cochrane ID number, title, CRG that produced the protocol or review, and year of publication as stated next to the phrase “Publication status and date” in each full-text. For completed reviews, we also extracted year assessed as up-to-date by capturing the year specified next to the phrase “Review content assessed as up-to-date”. A Cochrane review should usually be updated about every two years and updating must involve a search for new studies, which if identified must be assessed for inclusion and if eligible, incorporated into the review [18]. The date at which the review content is assessed as up-to-date refers to the date of the most recent search for new studies. Other extracted domains were: use of the GRADE system, inclusion of a summary of findings table in the review [14], and pre-specified criteria for selecting studies for inclusion in the review, i.e. types of participants, interventions, comparators, outcomes and study designs (PICOS). We categorised and described the extracted PICOS information as shown in Table 1. Interventions in completed reviews were classified as either a primary nutrition intervention or as a nutrition intervention delivered in combination with, or as part of, other interventions.

Table 1 Categorisation of types of participants, interventions, comparators, outcomes and study types

We mapped the topics addressed by all completed reviews against the top 50 causes of global years of life lost (YLLs) in 2013 [16] and the 25 leading level three global risk factors for disability adjusted life years (DALYs) [15], according to the 2013 Global Burden of Disease Study. We also assessed the scope of reviews by categorising them into three paradigms: medical, lifestyle and socio-ecological [12] (Fig. 1).

Two authors independently assessed the methodological quality of all completed reviews that included at least one primary study using the validated AMSTAR tool [19] and resolved discrepancies by discussion. One point was awarded for each of the domains judged as meeting the recommended criterion, and points were summed to calculate a total score for each review. Total scores of 0 to 4 indicated low methodological quality, 5–8 moderate quality, and 9–11 high quality [20].

Analysis

We analysed the data using Stata/IC for Windows [21], and used descriptive statistics to explore and present the results. Tables, charts and descriptive summaries present the number of included reviews and protocols within each CRG, as well as data on participants, interventions/exposures, comparators, outcomes, study designs, and methodological quality.

Results

We screened 8484 active records in the CDSR at 31 July 2015 and included a total of 470 completed systematic reviews (in 45 CRGs) and 169 protocols for systematic reviews (in 41 CRGs) (Fig. 2). One of the included reviews was an overview of systematic reviews, and we extracted data for all applicable fields from the overview. Of the 470 reviews, 425 included at least one primary study and we assessed the methodological quality of these reviews using AMSTAR. We excluded from the AMSTAR assessment 44 ‘empty’ reviews that contained no eligible studies, and the overview, as the tool is applicable to systematic reviews.

Fig. 2
figure 2

Flowchart illustrating the search results and selection process of nutrition reviews and protocols in the Cochrane Database of Systematic Reviews, and methodological quality assessment of systematic reviews [19]

Nutrition reviews by Cochrane Review Groups

Table 2 details the number of records (reviews and protocols) screened and included per CRG in descending order of number of records included. Of the 470 completed nutrition reviews included, most were produced by the Pregnancy and Childbirth (n = 73), Neonatal (n = 64), Metabolic and Endocrine Disorders (n = 33), Developmental, Psychosocial and Learning Problems (n = 26), Kidney and Transplant (n = 18), and Heart (n = 18) groups (Table 2). The greatest numbers of included protocols (reviews in progress) were produced by the Public Health (n = 19), Pregnancy and Childbirth (n = 16), Neonatal (n = 15), Developmental, Psychosocial and Learning Problems (n = 11), Kidney and Transplant (n = 10), and Heart (n = 10) groups (Table 2). As a proportion of total active records produced by each CRG at the date of screening, the Public Health (45%), Metabolic and Endocrine Disorders (28%), Neonatal (19%), Developmental, Psychosocial and Learning Problems (19%) groups had the most nutrition reviews and protocols (Fig. 3a). However, groups with the highest proportion of completed nutrition reviews were the Metabolic and Endocrine Disorders (34%) and Hypertension (24%) groups, and those most nutrition protocols were Public Health (51%) and Developmental, Psychosocial and Learning Problems (20%) groups (Fig. 3a).

Table 2 Number of records (reviews and protocols) screened and included per Cochrane Review Group in descending order by number of records included
Fig. 3
figure 3

a) Proportions of nutrition reviews and protocols included of the total reviews and protocols screened in Cochrane Review Groups where 6% or more of all records screened per Group were included (numbers next to bars indicate the percentages of both reviews and protocols); b) Number of nutrition reviews and protocols published in the Cochrane Database per year; c) Percentage of nutrition reviews (n = 470) assessed as up-to-date by year

Year of publication and up-to-date status of completed reviews

The first Cochrane nutrition review was published in 2007, with 44% (n = 206) of reviews being published up to and including 2011, and the rest from 2012 to 31 July 2015 (56%; n = 264) (Fig. 3b). About a fifth (21%; n = 36) of nutrition protocols were published from 2008–2011, with the majority being produced from 2012 onwards (78%; n = 131) (Fig. 3b). Notably, the first nutrition protocol published in 2001 and the second published in 2004 have yet to be published as completed reviews. About 70% (n = 341) of completed reviews had their review content assessed as up-to-date prior to 2013 (Fig. 3c), thus their most recent searches for new studies were done prior to 2013. The remaining reviews had more recent searches for new studies (2013 to 31 July 2015).

Description of included reviews

Types of participants

Two third of reviews assessed interventions for participants with a specified condition(s) (67%; n = 316) and the remaining third (33%; n = 154) evaluated interventions for preventing or reducing risk for a specified condition(s) (Table 3). About a third of reviews did not define eligible participants by age or life-stage (32%; n = 151). Of reviews that pre-specified eligible participants by age or life-stage category, most included adults (22%; n = 104), followed by infants (14%; n = 66), pregnant women (11%; n = 53) and infants, preschool- and school-aged children (7%; n = 31). The remaining reviews included participants in other life stage categories (Table 3).

Table 3 Total number of included reviews by pre-defined participant categories, and stratified according to whether the review assessed treatment or prevention of a condition(s)

Types of interventions and comparators

Nutrition was the primary intervention in most included reviews (86%; n = 406), and was delivered in combination with or as part of other interventions in the remaining reviews (14%; n = 64). Reviews of nutritional supplementation/supplements were the most common (50%; n = 235), followed by food interventions (20%; n = 95), combined food and supplementation interventions (9%; n = 43) and nutrition education (4%; n = 19) (Fig. 4a). Policy/program/system interventions comprised 3% of reviews (n = 14). Combinations of food and policy/program/system interventions, food and education, and education and policy/program/system interventions each made up about 3% of reviews. The small proportion of reviews (5%) included different combinations of the specified intervention categories (Fig. 4a). Table 4 provides examples of reviews by type of intervention category.

Fig. 4
figure 4

a The percentage of completed nutrition reviews (n = 470) that examined the various categories of nutrition interventions b) The numbers of completed nutrition reviews that included the various categories of outcomes

Table 4 Examples of review titles for the main intervention categories

Further scrutiny of the supplementation reviews revealed that nearly three quarters (73%; n = 173) evaluated micronutrients (e.g. vitamins, minerals, trace elements), while others assessed alternative supplements, such as amino acid derivatives (e.g. carnitine), probiotics, food and spice extracts (e.g. garlic) and phytochemicals (e.g. flavonoids) (20%; n = 47) or combinations of these (7%) (e.g. antioxidant supplements including conventional micronutrients and phytochemical compounds).

With regards to food interventions, half of the reviews studied the effects of whole diets (50%; n = 66), nearly a third (31%; n = 41) investigated complete formulas (e.g. enteral nutrition, infant formulas, ready-to-use therapeutic foods), 8% (n = 11) single foods (e.g. sweet potato, honey, fermented milk), 4% (n = 5) food groups (e.g. wholegrain cereals), and the rest studied combinations of these.

Most reviews pre-specified more than one type of comparator. The most common comparator was placebo (63%; n = 297), followed by no intervention (51%; n = 238), a different intervention (51%; n = 237), or usual care (25%; n = 119). While these three were also the most common comparators in supplementation reviews, usual care was most commonly used as the comparator in reviews of food interventions.

Types of outcomes

Reviews most frequently assessed the following pre-specified outcomes: clinical or nutritional status assessments (82%; n = 386), frequency/severity of disease (76%; n = 359), adverse events, side-effects, and safety (57%; n = 266), and quality of life (23%; n = 109) (Fig. 4b). Fewer reviews included outcomes on economics or cost-effectiveness (12%); withdrawal, drop-out or adherence (11%); and diet-related behaviours (10%) (Fig. 4b). Less than 10% of reviews examined the effects of nutrition interventions on non-diet-related behaviours, food/nutrient/dietary intake, and dietary quality or variety (Fig. 4b).

Study designs

All nutrition reviews included randomised controlled trials (RCTs) and 3% (n = 16) included RCTs and experimental non-randomised studies. Very few reviews included RCTs and observational studies (2%; n = 8). Six reviews (1%) included RCTs, experimental non-randomised and observational studies. The single overview included only systematic reviews.

Methodological quality on included reviews and use of GRADE

The median AMSTAR score of the 425 eligible reviews assessed was 9 (interquartile range: 7–10), indicating high methodological quality. Half of the reviews were judged to be of high quality (51%; n = 218) (AMSTAR score 9–11), and the other half (49%; n = 207) were considered to be of moderate quality (AMSTAR score 5–8). AMSTAR domains most frequently judged to have shortcomings in the reviews assessed were those related to the appropriate use of the scientific quality of included studies in formulating conclusions (57%), the assessment of the likelihood of publication bias (51%), and the assessment of potential conflicts of interest in the primary studies included in the reviews (61%) (Fig. 5). All reviews declared their sources of support (n = 470). A quarter of nutrition reviews (25%; n = 118) used the GRADE system for rating quality of evidence and a Summary of Findings table for presenting results.

Fig. 5
figure 5

Percentages of reviews judged as meeting the recommended criterion specified for each of the 11 AMSTAR domains (n = 425 reviews methodological quality assessed using AMSTAR tool)

Reviews addressing the burden of disease and paradigms

Forty-two percent (n = 196) of nutrition reviews addressed one or more of 28 of the top 50 causes of global YLLs in 2013 [16] related to nutrition (Table 5). The remaining 22 causes are not known to have any direct nutritional link (e.g. road injuries, drowning, and syphilis). The YLL causes addressed by the greatest number of reviews were preterm birth (21%; n = 42), ranked as the 7th leading cause of YLLs, diabetes (10%; n = 26), ranked 17th. Ischaemic heart disease (rank 1), maternal disorders (rank 26), asthma (rank 32), diarrhoeal diseases (rank 4), and HIV/AIDS (rank 6) were addressed by between 5 and 10% of the 196 reviews (Table 5).

Table 5 Mapping of number of reviews addressing top 50 causes of global years of life lost in 2013 [16] and the top 25 global risk factors for disability adjusted life years (both sexes combined) [15], according to the 2013 Global Burden of Disease Study

Sixteen of the leading 25 risk factors for disability adjusted life years (DALYs) were addressed by 37% of reviews (n = 176). Some reviews addressed more than one risk factor. All except one of the remaining nine risk factors have no direct nutritional link (Table 5). The risk factor addressed by the greatest number of reviews was childhood undernutrition (27%; n = 48; rank 4), followed by high body mass index (15%; n = 27; rank 3), and low omega-3 fatty acid intake (14%; n = 24; rank 24). Suboptimal breastfeeding (rank 19), iron deficiency (rank 18), high blood pressure (rank 1), and high sodium intake (rank 11) were addressed by between 5 and 10% of the 176 reviews (Table 5).

The majority of included reviews (82%; n = 386) framed their questions using a medical paradigm, in which inadequate intake of single or multiple nutrients causes deficiency and physiological dysfunction in individuals and populations, with a linear, context-free relationship between nutrient intake, physiological status, and health outcomes (Fig. 1) [12]. Nearly one in five reviews (17%; n = 78) framed questions using the lifestyle paradigm, where behaviours or dietary imbalances cause nutrient deficiencies, elevated risk factor levels and physiological dysfunctions in individuals and populations, with a multifunctional, context-rich relationship between food/dietary patterns, physiological status, and health outcomes [12]. The socio-ecological paradigm, which incorporates social and ecological determinants of nutritional health and food system integrity, was used by only six reviews (1%).

Discussion

Summary of main findings

Similar to previous reports [8], we found that the methodological quality of Cochrane nutrition reviews was high (AMSTAR) [19]. Many reviews did not incorporate the scientific quality of included studies when formulating conclusions (AMSTAR domain 8). This domain is facilitated by the use of GRADE, where explicit criteria are used for rating the quality of evidence, including study design, risk of bias, imprecision, inconsistency, indirectness, and magnitude of effect [14]. GRADE is now used by most new and updated Cochrane reviews. In line with Cochrane’s strict policy on conflicts of interest [22], all reviews included clear disclosures from the authors. However, about 40% of reviews failed to report on potential conflicts of interest in the primary studies included in the reviews, as required for the AMSTAR domain 11.

Among the completed reviews, only about a quarter (27%; n = 129) had review content assessed as up-to-date between 2013 to 31 July 2015 indicating that the most recent searches for new studies had been conducted during this period. Thus, the large majority of reviews had searches which were potentially out-of-date. Few studies have been conducted to inform decisions about how and when to update systematic reviews [23]. This is an evolving area and Cochrane’s approach of updating every review every two years has become unfeasible, and is currently undergoing a process of adaptation to develop clear and sensible guidance on updating of systematic reviews that considers whether the review addresses a current question, uses valid methods, and is well conducted; and whether there are new relevant methods, new studies, or new information on existing included studies [23]

Nutrition reviews, completed or in progress, covered a wide range of conditions and were widely distributed across 47 of the 53 CRGs (Fig. 3). Cochrane nutrition reviews reflect the current double nutrition burden, covering conditions associated with undernutrition and obesity. Reviews addressing the top causes of global YLLs [16] mainly addressed preterm birth, diabetes and ischaemic heart disease, and those targeting leading risk factors for DALYs [15] predominantly addressed childhood undernutrition and high body mass index.

Paradigms of review topics

Although AMSTAR critically appraises the methodological quality of systematic reviews, it does not explicitly examine the perspective of the review question or its associated scope and complexity. We attempted to address this by categorising the completed nutrition reviews into the medical, lifestyle or socio-ecological paradigms.

Most review questions were formulated within a medical paradigm, with half of the reviews examining supplementation/supplement interventions, mostly with micronutrients. Micronutrient supplementation, using single or multiple nutrients, targets acute micronutrient deficiencies in high risk populations, and like other nutrition-specific interventions, mainly addresses the immediate causes of undernutrition. By contrast only one in five Cochrane nutrition reviews examined food-based interventions, which were mainly interventions to modify diets. Food-based interventions are generally more context-rich and complex than nutrient-based interventions, typically require behaviour change and have intended outcomes that are mostly not achievable in the short-term. These topics are usually located within a lifestyle paradigm. Fortification and supplementation programmes (e.g. vitamin A and zinc supplementation) are often prioritised over behaviour-change and health promotion-based approaches (e.g. exclusive breastfeeding and dietary diversity) as they are deemed more feasible [2]. An important reason for this may be that behaviour change is difficult to achieve without addressing underlying systemic and structural factors. It should, however, be kept in mind that in the absence of nutritional deficiency, effects of single nutrients and foods can be hard to demonstrate. Foods provide a complex mixture of nutrients and other components that may have synergistic, additive or antagonistic effects on health. Thus, in many instances, zooming in on a single dietary component runs the risk of ignoring the wider contexts and complex relationships between diet and health. This emphasises the importance of studying the effects of dietary patterns, a significantly neglected topic in the Cochrane nutrition reviews we assessed. Dietary patterns are defined as the quantities, proportions, variety, or combination of different foods, beverages, and nutrients (when available) in diets, and the frequency with which they are habitually consumed [24]. Evaluating dietary patterns can help identify links between the overall diet and its component beverages, foods and nutrients in relation to health outcomes, and thus reduces the multicollinearity among single foods and nutrients. Assessing dietary patterns can account for the intrinsic interactions between foods and nutrients that are relevant to effects on disease risk. It can also identify potential cumulative effects of single dietary components, and dietary exposure over time [24], in line with the lifestyle paradigm.

Factors unrelated to diet play an important role in the health impact of diets. Malnutrition is caused by the interaction between poor nutritional quality diets, unhealthy environments and health damaging behaviours. All three factors can be significantly shaped by underlying drivers, such as social disadvantage, income inequality, political instability and conflict, and some elements of globalization. It has been estimated that scaling up direct undernutrition interventions to 90% coverage rates will address only 20% of the stunting burden and that actions in other sectors (agriculture; health; education; social protection; water, sanitation, and hygiene) are vital to address the remaining 80% [2]. This complexity is encapsulated in the socio-ecological paradigm, which is currently under-represented in Cochrane nutrition reviews.

Methodological considerations

Only about 5% of nutrition reviews included study designs other than RCTs, likely because Cochrane reviews focus on questions related to effectiveness, where RCTs are considered the ‘gold standard.’ However, this may have the consequence of restricting the scope of review topics that can be considered [6, 2527]. Some questions about effects in the field of nutrition are difficult to answer using a trial design due to long time horizons for outcomes of interest, ethical issues where there is potential for harm, high cost, lack of feasibility, and complexity. We believe that evidence from non-randomised and observational studies should be considered for inclusion in future Cochrane nutrition reviews, despite their known limitations, as they may provide the best available evidence to guide decision-making in some circumstances. For example, interventions for improving access to food in low- and middle-income countries and interventions to improve the implementation of healthy eating and obesity prevention policies or programmes within childcare settings. This is especially warranted in light of the availability of instruments that can be utilized to evaluate limitations of various designs and thereby reduce the likelihood of readers being misled by problems such as bias or confounding [28, 29]. Indeed the Cochrane Handbook for Systematic Reviews allows for the inclusion of studies other than RCTs [30]. Cochrane Methods Groups are actively developing methods for including non-randomized studies in reviews [31], assessing risk of bias for non-randomized studies [32], and for synthesising qualitative evidence and integrating it in reviews of intervention effects [29]. This methodological support network can assist by extending the range of study designs used in Cochrane nutrition reviews thereby expanding the number of important and challenging questions that can be addressed.

Currently, two thirds of nutrition reviews examine treatment questions for specific conditions (e.g. vitamin A for treating measles in children) with the rest addressing risk factor questions (e.g. altered dietary salt for preventing pre-eclampsia). Many of the treatment questions for undernutrition have been addressed, and we also know a great deal about the drivers and the types of interventions needed to address undernutrition [2, 33]. For other forms of malnutrition, such as overweight and obesity, the evidence base is more complex and fractured and many “what works” questions remain [2]. For all forms of malnutrition and particularly the co-existence of different forms of malnutrition, less is known about what combinations of interventions work best. For example, how to implement the right mix of nutrition actions in different contexts, equitably and affordably, at a scale that matches the size of the problem, and in ways that connect nutrition-specific and nutrition-sensitive interventions [33]; how to scale up coverage of proven nutrition-specific interventions and integrate nutrition actions into health system platforms [2]. We believe that future Cochrane nutrition evidence can contribute to addressing these knowledge gaps, thereby informing public health nutrition policies and programmes focussing on nutrition problems across all relevant sectors. Reviews of more complex issues can help to identify appropriate questions for more targeted systematic reviews and primary studies, prioritize topics for future research, map the nature of best available evidence (location, intervention, study methods, and study quality), and establish the existence, nature, and direction of reported impact, including intermediate and adverse impacts [34]. We plan on progressing this work in the future to inform activities to strengthen methods for preparing nutrition systematic reviews. For example, this will include nutrition-focused appraisals of systematic reviews in order to help address nutrition-specific issues in evidence synthesis, such as assessing baseline exposures to nutrients, nutrient status of participants, bioequivalence of nutrients, multiple and related biological functions of nutrients and time-scale, including the plausibility of a measurable effect over the duration of the studies [9].

Lavis and colleagues [35] have called for more innovation in the preparation of systematic reviews to improve their utility, and have highlighted the need for improved relevance and accessibility of reviews, while maintaining the rigour that is foundational to a systematic approach. Seminal work has been published in a series of six technical reports by the Agency for Healthcare Research and Quality [9], seeking to facilitate a better understanding of the challenges involved in preparing and using nutrition-related systematic reviews, and there are ongoing capacity-building initiatives for nutrition systematic reviews [36]. Cochrane can build on these advances to promote better nutrition evidence synthesis and improve the use of this evidence. This is in line with the objectives of the recently established Cochrane Nutrition Field, which are to increase the coverage, quality, and relevance of Cochrane nutrition reviews, increase the impact of Cochrane nutrition reviews across all stakeholders and contribute to strengthening methods for conducting nutrition systematic reviews. Activities will include consultative efforts to explore and delineate priorities for nutrition evidence synthesis. In achieving its objectives, Cochrane Nutrition plans to collaborate and engage with stakeholders within and outside of Cochrane, including other evidence synthesis initiatives.

Conclusions

This analysis presents a comprehensive description of the scope and quality of Cochrane nutrition reviews, and identifies gaps for future activities to support actions to address the nutrition burden, in line with the current nutrition agenda and impetus. As the world seeks to accelerate and sustain recent nutrition gains, meet targets and reach places and people who have been left behind, a constant flow of new, good quality evidence is needed to fill knowledge gaps, deliver even greater impacts for existing resources, and to make the case for additional resources. Rigorous and relevant synthesised nutrition evidence, such as that from Cochrane systematic reviews, has a valuable role in informing nutrition decisions at local and international levels. It can also harness the political capital required from policymakers to reform policy and finance the scale up of interventions.

References

  1. International Food Policy Research Institute. Global Nutrition Report 2014: Actions and Accountability to Accelerate the World’s Progress on Nutrition. Washington DC: International Food Policy Research Institute & World Health Organization Dept. of Nutrition for Health and Development; 2014.

    Google Scholar 

  2. International Food Policy Research Institute. Global Nutrition Report 2016: From Promise to Impact, Ending Malnutrition by 2030. Washington DC: International Food Policy Research Institute & World Health Organization Dept. of Nutrition for Health and Development; 2016.

    Google Scholar 

  3. International Food Policy Research Institute. Global Nutrition Report 2015: Actions and Accountability to Advance Nutrition and Sustainable Development. Washington DC: International Food Policy Research Institute & World Health Organization Dept. of Nutrition for Health and Development; 2015.

    Google Scholar 

  4. Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, Durieux P, Legare F, Nielson CP, Adily A, et al. Health research funding agencies' support and promotion of knowledge translation: an international study. Milbank Q. 2008;86:125–55.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Summerbell CD, Chinnock P, O'Malley C, van Binsbergen JJ. The Cochrane Library: more systematic reviews on nutrition needed. Eur J Clin Nutr. 2005;59 Suppl 1:S172–178. discussion S195-176.

    Article  PubMed  Google Scholar 

  7. van Binsbergen JJ, Delaney BC, van Weel C. Nutrition in primary care: scope and relevance of output from the Cochrane Collaboration. Am J Clin Nutr. 2003;77:1083S–8S.

    PubMed  Google Scholar 

  8. Salam RA, Welch V, Bhutta ZA. Systematic reviews on selected nutrition interventions: descriptive assessment of conduct and methodological challenges. BMC Nutrition. 2015;1:1–12.

    Article  Google Scholar 

  9. Nutritional Research Series: Advancing the Role of Systematic Reviews in Nutrition Research and Applications. [http://www.ahrq.gov/research/findings/evidence-based-reports/tr17-series.html]. Accessed 10 June 2016.

  10. United Nations System Standing Committee on Nutrition. Nutrition and the Post-2015 Sustainable Development Goals. Geneva: World Health Organization; 2014.

    Google Scholar 

  11. Petticrew M. When are complex interventions 'complex'? When are simple interventions 'simple'? Eur J Public Health. 2011;21:397–8.

    Article  PubMed  Google Scholar 

  12. Lawrence M. Re-shaping evidence use for public health nutrition policy. In: Cochrane Colloquium. Vienna, Austria; 2015. [https://www.globalevidencesummit.org/sites/2015.colloquium.cochrane.org/files/uploads/CC_Programmbuch_int_151001.pdf]. Accessed 5 Apr 2016.

  13. Maternal and Child Nutrition Study Group. Executive summary of The Lancet Maternal and Child Nutrition Series. Lancet. 2013;1–12. [http://www.thelancet.com/pb/assets/raw/Lancet/stories/series/nutrition-eng.pdf]. Accessed 12 June 2016.

  14. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.

    Article  PubMed  Google Scholar 

  15. G. B. D. Risk Factors Collaborators. Global, regional, and national comparative risk assessment of 79 behavioural, environmental and occupational, and metabolic risks or clusters of risks in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;386(10010):2287–2323.

  16. G. B. D. Mortality Causes of Death Collaborators. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015;385:117–71.

    Article  Google Scholar 

  17. Microsoft. Microsoft Excel. Redmond: Microsoft Corporation; 2010.

  18. Higgins JPT, Green S, Scholten RJPM. Maintaining reviews: updates, amendments and feedback. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions, Version 510 [updated March 2011]. Hoboken: The Cochrane Collaboration; 2011. www.handbook.cochrane.org.

    Google Scholar 

  19. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–20.

    Article  PubMed  Google Scholar 

  20. van der Linde RM, Stephan BDM, Savva GM, Dening T, Brayne C. Systematic reviews on behavioural and psychological symptoms in the older or demented population. Alzheimer’s Res Ther. 2012;4:28. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3506942/pdf/alzrt131.pdf.

  21. Statacorp LP. Stata/IC Version 13 for Windows. College Station: Statacorp LP; 2014.

    Google Scholar 

  22. Cochrane. Editorial and publishing policy resource. http://community.cochrane.org/editorial-and-publishing-policy-resource. 2016. Accessed 26 Sept 2016.

  23. Garner P, Hopewell S, Chandler J, MacLehose H, Schunemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, et al. When and how to update systematic reviews: consensus and checklist. BMJ. 2016;354:i3507.

    Article  PubMed  PubMed Central  Google Scholar 

  24. United States Department of Agriculture. A Series of Systematic Reviews on the Relationship Between Dietary Patterns and Health Outcomes. Redmond: United States Department of Agriculture, Center for Nutrition Policy and Promotion, Evidence Analysis Library Division; 2014.

  25. Brannon PM, Taylor CL, Coates PM. Use and applications of systematic reviews in public health nutrition. Annu Rev Nutr. 2014;34:401–19.

    Article  CAS  PubMed  Google Scholar 

  26. Tovey D. The role of The Cochrane Collaboration in support of the WHO Nutrition Guidelines. Adv Nutr. 2014;5:35–9.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Blumberg J, Heaney RP, Huncharek M, Scholl T, Stampfer M, Vieth R, Weaver CM, Zeisel SH. Evidence-based criteria in the nutritional context. Nutr Rev. 2010;68:478–84.

    Article  PubMed  Google Scholar 

  28. ROBINS-I: a tool for assessing Risk Of Bias In Non-randomized Studies of Interventions. [https://sites.google.com/site/riskofbiastool/]. Accessed 10 Nov 2016.

  29. Noyes J, Popay J, Pearson A, Hannes K, Booth A, on behalf of the Cochrane Qualitative Research Methods Group. Qualitative research and Cochrane reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions, Version 510 [updated March 2011]. The Cochrane Collaboration http://handbook.cochrane.org/; 2011.

  30. Higgins D, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.1 [updated March 2011]. London: Wiley; 2011.

    Google Scholar 

  31. Reeves BC, Higgins JP, Ramsay C, Shea B, Tugwell P, Wells GA. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Res Synth Methods. 2013;4:1–11.

    Article  PubMed  Google Scholar 

  32. Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Gillespie S, Menon P, Kennedy AL. Scaling up impact on nutrition: what will it take? Adv Nutr. 2015;6:440–51.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Thomson H. Improving utility of evidence synthesis for healthy public policy: the three Rs (relevance, rigor, and readability [and resources]). Am J Public Health. 2013;103:e17–23.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2005;10.

  36. Summer Institute for systematic reviews in nutrition for global policy-making. [http://www.who.int/nutrition/events/2015_systematicreviews_globalpolicymaking_27julto7aug15/en/]. Accessed 9 June 2016.

Download references

Acknowledgements

Not applicable.

Funding

CN is partly supported by the Effective Health Care Research Consortium. This Consortium is funded by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). The views expressed in this publication do not necessarily reflect UK government policy. The funder was not involved in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Availability of data and materials

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Authors’ contributions

JV, CN, SD and conceived and designed the study; AH, CN, SD and JV performed the searching and screening; CN, AH and SD extracted data and performed AMSTAR; CN performed the data analysis with inputs from SD and AH, and all authors contributed to interpreting the data; CN wrote the initial draft of the manuscript, which was then added to and edited in subsequent iterations by SD, AH and JV. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests. CN, SD and JV are voluntary Cochrane contributors, and CN and SD are voluntary Co-Directors of Cochrane Nutrition.

Consent for publication

Not applicable.

Ethics approval

Not applicable.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Celeste E. Naude.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naude, C.E., Durao, S., Harper, A. et al. Scope and quality of Cochrane reviews of nutrition interventions: a cross-sectional study. Nutr J 16, 22 (2017). https://doi.org/10.1186/s12937-017-0244-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12937-017-0244-7

Keywords