• Weight management
• Metabolic syndrome
• Diabetes mellitus
• Nutritional treatment

Despite decades of research and huge amounts of money spent, we are still losing ground in what is a true pandemic. The situation is particularly bad in North America where 61.9% of adults are overweight. This leads to the grim situation where only 6.8% of Americans enjoy optimal cardiometabolic health.1

From 1986 until 2012 my clinical experience in tackling obesity was similarly disappointing. I handed out ‘standard advice’ to my patients around weight loss. This included ‘eat less, move more’, ‘everything in moderation’ and advice around calorie counting as part of a low-fat diet. It was rarely effective. For 26 years I blamed my patients for their poor results. Like so many doctors in primary care I came to believe it was a poor use of my time giving dietary advice to help people lose weight. It never once occurred to me that my poor advice was the common denominator. This changed suddenly 11 years ago when an angry patient asked me why I had never discussed reducing dietary carbohydrates as a way to lose weight and improve blood sugar. Since then, learning from that patient and many others, our 9900-patient practice has been offering the option of a low carbohydrate diet, particularly to our patients with T2 diabetes and pre-diabetes. We have audited our clinical data and published the results in this journal.2 3 We recorded a mean weight loss of over 10% body weight at 3 years. Since then, I have been looking out for other promising ‘real world’ work in obesity and weight loss which is why the paper by Dr Morten Dag Nilsen et al4 published this month in BMJ Nutrition, Prevention & Health is particularly interesting. Like me, Dr Nilsen is a family physician but working in Oslo, Norway. He and his colleagues are also interested in taking a pragmatic approach to help people with overweight and obesity, presenting where they so often do in primary care. 23 primary care physicians were cluster randomised to offer either ‘usual care’ or a fixed plan which in my judgement looked to be a low carb approach, starting as it did by not eating: potatoes, bread, rice, pasta, confectionery, fruit, dairy products, cereal products or nuts. These were replaced by: fish, meat, eggs, shellfish, vegetables and salad (ad libitum). By the end of the first year, 25% of the intervention group had lost >10% of their body weight.

The success of these interventions made me question where the evidence for the approach I used for 26 years came from? At the time I would proudly say that I was applying ‘The Guidelines’. But what underpins our guidelines? Most are based on a hierarchy of evidence with randomised controlled trials (RCTs), systematic reviews and meta-analyses of RCT’s firmly at the top5 (see figure 1). It has become obvious to me that we should question the utility of this model, particularly in the world of nutrition and health.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

The hierarchy of evidence (based on internal validity).

At present, most scientists would agree the progress of nutritional science goes something like this:

We begin with interesting questions. These can originate from clinically based audit, significant event analysis (n=1 cases) or epidemiological studies. All of which can suggest thought-provoking associations or hypotheses but cannot confirm causation. Often there are multiple variables involved so we make use of an RCT to reduce them, ideally to one. For example, if we hypothesise that a particular drug helps with blood pressure we need to exclude other variables from our study, things like other medications, change in weight and diet. In this way we are said to be maximising the ‘internal validity6’ of our study. Our real-world work and that of Dr Nilson is considered by some to be inferior because it is impossible to control all the variables in routine clinical work. However, when it comes to actual clinical implementation, RCTs have a problem. So many variables have been removed that the studies no longer represent real people leading ordinary lives outside of a tightly controlled trial. For example, in our National Health Service (NHS) clinic, very few patients with hypertension do not have a weight problem. Many of them are also on drugs for joint pain, reflux or depression. This is why the results seen in RCTs often do not roll out into real-world, clinical practice. For this to occur we need interventions that have a high degree of ‘external validity’—approaches that better represent the ‘ordinary’ people who populate our clinics. Away from the carefully controlled conditions of clinical trials, results can be very different in the messy, complex world of everyday general practice. An intervention like ours that works in general practice near Liverpool may also work in other areas. As it happens, a low carb approach similar to ours and the Swedish work has also shown promise in Essex, UK,7 New Zealand8 and in California, USA.9 I believe that internal and external validity are the ‘Yin and Yang’ of helping science progress. We need both, but perhaps internal validity has been over emphasised and valued at the cost of some real progress in nutritional science. Coming back to the hierarchy of evidence in figure 1. I suggest in reality it is the ‘hierarchy of internal validity in evidence’ and as such is missing the other 50% represented by external validity. It is possible that if we also incorporate external validity we are incorrect in the ranking and it should in fact be a square with no particular ranking. I feel this is particularly true as it relates to individual patients (see figure 2, you may notice I have added patient experience).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Possible sources of evidence (internal and external validity).

RCTs are expensive and time consuming. This and other practical considerations result in very few (if any) long-term RCTs looking at diet. In terms of diabetes guidelines, this has led to a recent paper10 concluding there is ‘no long-term evidence for the current guideline-driven approaches, so all long-term dietary strategies for diabetes management remain an ‘evidence-free zone’’.

The current research paradigm (or is it an industry?) is generating a huge number of published studies. A search in Google Scholar for ‘diet and obesity’ suggests 2 750 000 possible entries. It has become impossible to keep up! In any case most RCTs end up suggesting that ‘more and longer-term studies are needed’.

I suggest an alternative and possibly far more efficient line of enquiry. A bit like ‘reverse engineering’ why not start with ‘real world’ examples of clinics doing audit that show positive results in terms of weight loss and metabolic health. Then ask: Does what these clinics claim to do make sense in terms of physiology? Can their approaches be reproduced across different settings? What about costings and clinical drawbacks? Now we are well on our way to actual implementation.

I believe prioritising internal validity and the idea that proper science is about RCTs has distorted relationships between clinicians and academics. In 2008 Lawrence Green wrote about ‘the implicit assumption underlying much of the adoption, utilization and implementation of evidence-based guidelines is the characterization of a pipeline in which evidence is produced and delivered to practitioners’.11 In 2018 at the Food for Thought event in Zurich co-hosted by The BMJ and Swiss Re (a large insurance company). I asked what the panel thought clinicians could do to improve the science around nutrition? A well-known professor of nutrition explained how little clinicians could do, better to leave it to the scientists! I disagree, the best research is collaborative involving academics, clinicians and patients, all working together, cognisant of the pros and cons of both internal and external validity.

A crucial misunderstanding is around clinical audit and ethical approval. All research studies have to be scrutinised by an ethics committee (institutional review board in the USA), but most ethics committees specifically exclude audit studies from their remit.12 Its important to understand that the auditing and publishing of clinical services does not need ethics approval in the UK. Audit of your clinic would not normally be considered a study. The decision tree of the National Research Ethics Service of the Health Research Authority in England clearly states that this type of service evaluation is exempt from approval by an NHS Research Ethics Committee. Audit of service provision is encouraged and widely regarded as good practice for clinicians as long as participants are effectively anonymised. In my own practice, as an extra level of protection, we also seek informed consent from all patients involved in our audits.

Audit should not be considered as the poor cousin to RCTs. Research and audit have many similarities. They both start with a question, both expect the answer to inform, change or influence clinical practice, both require formal data collection on patients and both depend on using an appropriate method and design to reach sound conclusions.12

I would encourage clinicians working in the field of nutrition to think about publishing audit. In that way we could have vastly more information on interventions that are working well in the ‘real world’. Audit is the way to answer interesting questions about your clinic. For example, do you know the average weight loss, blood pressure improvement or other important clinical metrics achieved by your service?

It is interesting to note the very first medical RCT to be published was in 1948 on the use of Streptomycin in the treatment of Tuberculosis.13 I would point out that quite significant progress occured in medicine before that date. It was in 1996 that David Sacket ‘the Father of evidence-based medicine’ said ‘Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients’.14 Put another way we need both ‘Evidence based practice’ (RCTs, internal validity) and ‘Practice based evidence’ (audit, case studies, external validity)11 data is power. We clinicians have a lot of data, particularly around weight and other measures of metabolic health. Our computer systems make it more accessible than ever before, let us use it!