Article Text

Nutrition content of summative examinations within an Australian 4-year graduate entry medical course: 2013–2016
  1. Robyn Perlstein1,
  2. Janet McLeod1,
  3. Colin Bell2 and
  4. Caryl Nowson3
  1. 1 Faculty of Health, Deakin University, Geelong, Victoria, Australia
  2. 2 School of Medicine, Deakin University, Geelong, Victoria, Australia
  3. 3 School of Exercise and Nutrition Sciences, Deakin University-Geelong Campus at Waurn Ponds, Waurn Ponds, Victoria, Australia
  1. Correspondence to Robyn Perlstein, Deakin University Faculty of Health, Geelong, VIC 3125, Australia; robyn.perlstein{at}deakin.edu.au

Abstract

Background Poor nutrition is a major contributor to chronic disease, but the level of nutrition education in medical training is limited. Deakin University Medical School has been working to embed more nutrition into the curriculum since 2009.

Aim To assess the nutrition content of all summative examinations in the Bachelor of Medicine, Bachelor of Surgery over a 4-year period.

Methods The type, amount and scope of nutrition-related questions were assessed in all summative examinations delivered to all 4-year levels from 2013 to 2016. These were assessed independently and analysed for nutrition content. The amount of nutrition was quantified, and the nutrition topic areas and nutrition competencies addressed were documented.

Results Less than 10% of summative examination questions contained any nutrition content. For first-year and second-year students, these examinations included an average yearly total of 433 multiple choice questions (MCQs) (range 337–530) and 47 short answer questions (SAQs) (range 33–62). Third-year and fourth-year students had 150 MCQs on average per year and no SAQs. The percentage of nutrition-related questions across all 4 years ranged from 6% in 2013 to 10% in 2016. The proportion of SAQs with nutrition content ranged from 12% in 2013 to 19% in 2016. Basic nutritional sciences, accounted for 60% of nutrition content and, 25% addressed dietary strategies for prevention and treatment of disease, and skills-based nutrition competencies represented approximately 10% of all questions.

Summary and conclusions Minimal nutrition was included in the summative examinations. There did not appear to be any consistent increase in the nutrition content of MCQs over the 4-year period but there was some indication of an increase in nutrition content in SAQs. Longer term evaluation is required to confirm this trend. Only a small number of nutrition questions were skills based, most focused on basic nutritional science. Examinations included few skills-based nutrition questions, and consideration of setting a minimum level of nutrition in examinations could assist in ensuring the development of appropriate nutrition competencies in medical graduates.

  • nutrition assessment
  • nutritional treatment

Data availability statement

Data may be obtained from a third party and are not publicly available. As the data used here are current examination questions, these are kept in a secure location and are not accessible.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What this paper adds

  • Up to 10% of exam questions in the medical course at Deakin University were nutrition related

  • Nutrition related questions were mostly short-answer and asked in the preclinical years of the course

  • Questions mostly assessed knowledge of basic nutritional science and knowledge of dietary strategies for prevention and treatment of disease

Introduction

Poor nutrition is a major contributor to chronic disease1 2 and it is important for medical graduates to have an understanding of nutrition and modifiable dietary risk factors, as in Australia around 50% of patients who visit a general practitioner have one or more chronic conditions.3 However, recent studies indicate low levels of nutrition-related content in medical curricula around the world4–6 and a similar situation exists in Australia.7

The Australian Medical Council (AMC) is responsible for accrediting medical schools and their programmes of study for the medical profession. The AMC Accreditation Standards and Procedures8 indicate that the content of the curriculum should ‘prepare graduates to protect and advance the well-being of individuals, communities and populations’ which encompasses nutrition but does not specifically mention it. To guide nutrition teaching and learning in medicine, an Australian Nutrition Competencies Framework (NCF) has been developed that includes four knowledge and five nutrition-related skill goals for medical students7 9(see online supplemental appendix 1).

Supplemental material

Assessment is a primary driver of learning,10 and quality teaching and learning activities should have clear learning objectives and closely aligned assessment.11 Few studies have examined the nutrition content in summative assessment tasks, with no documentation of this for Australian Medical Schools. Hark et al 12 quantified the amount of nutrition information in step 1 of the US Medical School Licensing Examinations in 1986 and 1993. They found 11% of nutrition-related content in 1993 compared with 9% in the 1986 examinations. A more recent 2015 review, also in the USA, of the nutrition content of the test preparation material (but not quantification) found a large amount of information relating to vitamin and mineral deficiencies and gastroenterology, but not nutrition and lifestyle modification.13

Deakin University’s (DU) graduate entry Bachelor of Medicine, Bachelor of Surgery (BMBS) (now Doctor of Medicine) is based in Geelong, Victoria and has approximately 140 students in each of the 4 years of the course. DU School of Medicine has been actively integrating additional nutrition into the curriculum since 2009.

To assess the nutrition content of summative examinations, between 2013 and 2016, we documented the total number of examination questions in year levels 1–4, the number of examination questions covering nutrition topics, the types of nutrition-related questions, the proportion of marks allocated to nutrition content and the specific nutrition competencies addressed.

Methods

Design and setting of study

The DU School of Medicine was established in 2008 and has a strong focus on public health medicine. The school has an established collaboration with nutrition academics in the much longer established School of Exercise and Nutrition Science (ENS) at Deakin to facilitate the incorporation of nutrition-related learning objectives in the medical course. Collaborative initiatives have included reviewing nutrition-related learning objectives, lectures by ENS staff, development of nutrition competencies and convening student nutrition interest groups.

In years 1 and 2 (preclinical) of the BMBS, teaching is delivered via problem-based learning cases which are supplemented by lectures, workshops and practical classes focusing on human body systems. In the clinical years (years 3 and 4), learning is more experiential with students based at hospitals and general practices throughout Southwest Victoria. Student progress is assessed through written summative examinations, as well as Objective Structured Assessment Clinical Examinations (OSCEs), Workplace-Based Assessments (WBA) and written assignments.

Description of materials

Summative examinations in the BMBS incorporate multiple choice questions (MCQs) and short answer questions (SAQs), and the number of these varies by year level. An extensive question bank has been developed from which summative examination papers are assembled. The bank is continually expanded and reviewed to allow new items to be tested to preserve academic integrity and maintain quality assurance. The MCQs are composed of a question with five possible answers (one correct answer plus four distractors). The MCQs range in complexity from asking simple factual information to interpretative case-based study questions (clinical vignettes). SAQs comprise a brief prompt requiring a written answer, varying in length from one or two words to a short paragraph testing both factual and applied knowledge. All examinations are designed in accordance with the DU Assessment (Higher Education Courses) Procedure.14

Processes, interventions and comparisons

Two dietitians (RP, CN) independently undertook the following steps 1–4. Results were cross-checked and any discrepancies resolved by discussion.

Step 1: documented the total number of summative examinations and number and type of question (MCQ or SAQ) over 2013–2016 in year levels 1–4.

Step 2: identified the total number and type (MCQ or SAQ) of questions that covered or assessed nutrition using the Australian NCF9 (see online supplemental appendix 1).

Step 3: documented the marks allocated to nutrition-related SAQs and assessed the proportion of examination assessment marks allocated to nutrition in these SAQs. To determine the proportion of marks allocated to nutrition, the examiner-set marking scale was used. This ranged between 1 and 13 marks. Any SAQs that were entirely assessing nutrition knowledge/application were allocated the total marks for the question. The proportion of nutrition included in any SAQs that were only partly testing nutrition knowledge was allocated a proportion of nutrition-related marks (calculated as a percentage of the total marks for the question). MCQs were not included as they were all allocated 1 mark each.

Step 4: classified the nutrition-related examination questions according to which nutrition competencies they addressed.9 Each nutrition-related question was reviewed in detail so as to determine the number and type of nutrition-related competencies covered. Where questions were related to more than one competency, they were classified under all relevant competencies. Online supplemental appendix 2 gives an example of a question addressing a number of nutrition competencies. Questions that did not directly test nutritional knowledge were not included in the final analysis.

Results

Number of examinations and types of questions

First-year students in Deakin’s BMBS undertook an average of 14 summative examinations between 2013 and 2016, representing 66%–69% of their total assessment. For year-2 students, the contribution of summative examinations to total assessments was slightly lower (62%–68% of total assessment). Year-3 and 4 students had one final summative examination at the end of each year, representing 38% of their total assessment. For first-year and second-year students, these summative examinations included an average yearly total of 433 MCQs (range 337–530) and 47 SAQs (range 33–62). Third-year and fourth-year students had an average yearly total of 150 MCQs and no SAQs (see table 1).

Table 1

Total number of MCQs and SAQs in year levels 1–4 (2013–2016)

Questions with nutrition-related content

The percentage of nutrition-related questions across all 4 years ranged from 6% in 2013 to 10% in 2016. The number of nutrition-related questions in year level 1 ranged from 6% to 12%; year level 2 8%–10%; year 3 0%–7%; and year 4 1%–6% (see table 2, figure 1 and online supplemental appendix 3).

Table 2

Total number of nutrition-related MCQs and SAQs in year levels 1–4 (2013–2016)

Figure 1

Percentage of nutrition-related questions (MCQ and SAQ), 2013–2016. MCQ, multiple choice question; SAQ, short answer question.

In years 1 and 2, the percentage of nutrition-related content in the MCQs ranged from 7% in 2013 to 9% in 2016. In years 3 and 4, this range was 2% in 2013 to 7% in 2016 (see figure 2, table 3).

Figure 2

Average of all year levels’ nutrition-related MCQ versus SAQ (2013–2016). MCQ, multiple choice question; SAQ, short answer question.

Table 3

Comparison of amount of nutrition-related content versus marks allocated to nutrition in SAQs, year levels 1 and 2 (2013–2016)

Nutrition-related SAQs (only in years 1 and 2) ranged from 12% in 2013 to 19% in 2016 (see figure 2, table 2). The percentage of nutrition marks allocated to the nutrition content in SAQs varied from 2013 to 2016 in the range of 13%–21% (see online supplemental appendix 4).

Classification of questions according to the NCF

Evidence-based dietary strategies for prevention and treatment of disease (K3) were the most commonly covered competencies between 2013 and 2016 with over 60% of nutrition-related questions addressing Demonstrate understanding of the basic sciences in relation to nutrition (K1) and 25% addressing K3 (see figure 3). Almost no questions assessed prevention knowledge (K2) and only 11% or less of questions assessed skills-based competencies (S1–5) across the 4 years and S4 and S5 were not assessed at all (see online supplemental appendices 5 and 6).

Figure 3

Individual nutrition competencies addressed each year (2013–2016).

Over the 4 years, knowledge-based competencies were assessed 65–108 times in all year levels compared with skills-based competencies which were only assessed 7–10 times.

Discussion

Less than 10% of summative examination questions in the DU BMBS contained nutrition content over the period 2013–2016. There was some indication of an increase in nutrition content in SAQs, up from 12% in 2013 to 19% in 2016, but longer term evaluation is required to confirm this trend.

We cannot comment on the acceptability of a level of 10% nutrition content as we have no benchmarks against which to assess this. A review of studies investigating nutrition in medical courses found that nutrition has not adequately been incorporated into the medical curriculum and hence medical students are not adequately supported to provide high-quality, effective nutrition care to patients.4 Even in 1985 when the US National Research Council Committee on Nutrition in Medical Education15 recommended a minimum of 25 classroom hours devoted to nutrition in the preclinical years, most medical schools did not achieve this.16 Requirements such as formal hours of nutrition education may produce greater nutrition competency in graduates, but only if the nutrition-related curriculum is assessed accordingly.17 To our knowledge, this is the first time the nutrition-related content of summative examinations in a medical course has been documented.

In year levels 1 and 2, nutrition-related content was more common in SAQs than MCQs. This may be because most of the nutrition teaching and learning came through the Public Health Medicine theme and this theme tends to assess student knowledge and skills using SAQs more commonly than the other themes.18 Some studies indicate that SAQs could be a more effective assessment strategy than MCQs. A 2018 review of 20 UK medical courses indicated that while SAQs actually improved learning,19 MCQs gave a false impression of students’ competence and only indicated the ability to recall information.10 20

The number of MCQs administered decreased with increasing year level which represents the change in assessment from knowledge based to experiential (WBA and OSCE) assessments. In years 1 and 2, MCQ/SAQ style assessment comprised approximately 70% of total assessment, compared with approximately one-third of total assessment in years 3 and 4. The amount of nutrition contained in other types of assessments is unknown and needs to be further explored.21 Few Australian studies have looked at where and when nutrition education occurs in the latter years of medical school training, let alone the best modes of assessment. Many skills-based nutrition competencies would be better assessed outside of written summative assessment tasks: such as with the OSCE that has been widely adopted as a tool to assess students’ competencies as it measures outcomes and allows very specific feedback.

A higher number of knowledge-based nutrition-related questions were assessed in all year levels compared with only one-tenth being skills-based nutrition-related questions. Most of the knowledge-based nutrition questions addressed the basic sciences relating to nutrition such as biochemistry and physiology and fewer addressed the interactive role of nutrition in the prevention of disease and evidence-based dietary strategies for the treatment of disease such as heart disease and diabetes. While knowledge-based questions are necessary, skills-based questions are essential for the application of nutrition knowledge in practice.22

Of interest is that two skills-based nutrition competencies were not assessed over the 4-year period reviewed. These related to being able to ‘apply ethics in nutritional management’ (S4), and to ‘work as part of a team to provide adequate nutritional care’ (S5). Poor communication between caregivers and lack of interprofessional teamwork are causes of poor medical outcomes,23 while ethical and moral nutritional issues are essential considerations in patient care.24–26 These competencies may have been addressed as they may be elements more relevant to later professional practice and possibly not relevant to be assessed in case-based questions. Not all of the nutritional competencies can be addressed in non-clinical teaching.

Consideration of setting minimum levels of nutrition content in all examinations is worthy of discussion by key medical curriculum advisers as this could assist in ensuring adequate nutrition content. Increasing the level of assessment of nutrition competencies in summative assessment tasks begins with increasing the level of nutrition in learning objectives. Significant barriers do exist, such as medical school staff without adequate nutrition knowledge and experience; staff without specific training in drafting nutrition-related examination questions specific to the format required in medical courses and ensuring there are personnel with experience in translating evidence-based nutrition science to be relevant for medical students. Summative assessments for nutrition-related learning ideally would be developed by dietitians or nutritionists who could drive the implementation of nutrition competencies into medical curricula22 as this would facilitate their input on nutrition throughout the curriculum, which could enhance the nutrition education of medical students.27

Strengths and limitations

The strengths of this paper include the extensive documentation of the nutrition content of summative examination questions over a 4-year period and the type and content of questions. Information on competency categories (knowledge and skills based) was also provided. The limitations are that the summative assessment tasks examined here were subjectively assessed and hence we did not conduct a statistical analysis to determine if changes over time were significant.

Summary and conclusions

At DU, the School of Exercise and Nutrition and School of Medicine have worked together to incorporate more nutrition into the curriculum and to ensure nutrition is part of student assessment. This current study provides a ‘snapshot’ of the nutrition content of summative assessment tasks in an Australian medical school in year levels 1–4 between 2013 and 2016.

Incorporation of more assessment tasks focused on skills-based nutrition into the medical curriculum is likely to increase the nutrition competency, rather than just knowledge, of medical graduates.

Wider discussion and the development of guidelines on the minimum amount of nutrition in summative examinations is warranted as this could be an avenue to assist in ensuring the development of appropriate nutrition competencies in medical graduates.

Data availability statement

Data may be obtained from a third party and are not publicly available. As the data used here are current examination questions, these are kept in a secure location and are not accessible.

Ethics statements

Ethics approval

Exemption from ethics review granted 19 October 2016 (2016-332) by Deakin University Human Research Ethics Unit (DUHREC).

Acknowledgments

Thanks to Shani Watson who enabled research staff to access data.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @CarylNowson

  • Contributors CN and RP planned the study. RP, JM and CN conducted the study. CN, RP, JM and CB reported the study. RP submitted the study. CN and RP are responsible for the overall content as guarantors.

  • Funding The study was conducted by paid employees of the Deakin University School of Medicine and School of Exercise and Nutrition.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed by Dr Katharine Martyn, University of Brighton, UK.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.