Article Text
PDF
Abstract
Background Poor communication between healthcare professionals is recognised as accounting for a significant proportion of adverse patient outcomes. In the UK, the General Medical Council emphasises effective handover (handoff) as an essential outcome for medical graduates. Despite this, a significant proportion of medical schools do not teach the skill.
Objectives This study had two aims: (1) demonstrate a need for formal handover training through assessing the pre-existing knowledge, skills and attitudes of medical students and (2) study the effectiveness of a pilot educational handover workshop on improving confidence and competence in structured handover skills.
Design Students underwent an Objective Structured Clinical Examination style handover competency assessment before and after attending a handover workshop underpinned by educational theory. Participants also completed questionnaires before and after the workshop. The tool used to measure competency was developed through a modified Delphi process.
Setting Medical education departments within National Health Service (NHS) Lanarkshire hospitals.
Participants Forty-two undergraduate medical students rotating through their medical and surgical placements within NHS Lanarkshire enrolled in the study. Forty-one students completed all aspects.
Main outcome measures Paired questionnaires, preworkshop and postworkshop, ascertained prior teaching and confidence in handover skills. The questionnaires also elicited the student’s views on the importance of handover and the potential effects on patient safety. The assessment tool measured competency over 12 domains.
Results Eighty-three per cent of participants reported no previous handover teaching. There was a significant improvement, p<0.0001, in confidence in delivering handovers after attending the workshop. Student performance in the handover competency assessment showed a significant improvement (p<0.05) in 10 out of the 12 measured handover competency domains.
Conclusions A simple, robust and reproducible intervention, underpinned by medical education theory, can significantly improve competence and confidence in medical handover. Further research is required to assess long-term outcomes as student’s transition from undergraduate to postgraduate training.
- medical education & training
- change management
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A handover workshop was designed using various pedagogical theories.
A modified Delphi process added content and construct validity to the assessment tool.
The design of the workshop evaluation accessed the third level of the Kirkpatrick model, although with a short follow-up period postintervention.
The study had a small sample size and did not include a control group of students that underwent assessments without exposure to the educational workshop.
A single-assessor examining a single simulated patient handover has limitations in terms of assessment reliability.
Introduction
Patient handover (handoff) is defined as the transfer of information, professional responsibility and accountability for aspects of patient care to another healthcare worker, on a permanent or temporary basis.1 The General Medical Council (GMC), the medical regulator in the UK, has emphasised effective handover as a professional obligation for practising doctors, and views this as an essential outcome for medical graduates.2 Further afield, the Association of Faculties of Medicine of Canada and the Association of American Colleges expect graduates entering residency to be entrusted to undertake handoff/handover independently, and the Australian Medical Council deems it an essential competency it of Australian medical graduates.3–5
Poor communication between healthcare professionals is internationally recognised as accounting for a significant proportion of adverse patient outcomes.6–10 Inadequate handover can lead to lost, forgotten or misinterpreted information.11 Recent employment legislation has impacted on healthcare workers’ shift patterns.12–14 This has resulted in increasing frequency of patient handover between shifts, and therefore, ensuring effective and reliable communication is of particular importance.13
Structured verbal and written handovers are endorsed by international bodies such as WHO, as they allow for greater clarity in the transfer of information and higher information retention.15–17 A structured handover has been associated with further benefits to clinicians, such as enhanced multidisciplinary team working, reduced work related stress and improved job satisfaction.18 19
Several structured handover templates exist, including the Introduction, Medical complaint, Information related to the complaint, Signs and Symptoms, Treatment given/Trends noted, Allergies, Medications, Background history, Other information,20 and the Situation, History, Assessment, Risk, Expectation, Documentation tools.21 Another such structure, the Situation, Background, Assessment, Recommendation tool (SBAR) has demonstrated improvement in the quality and length of patient handover and is recommended by WHO.16 22
Delivering effective handover is a complex skill, incorporating technical and non-technical skills and requires practice to master.23 Teaching of handover skills at the undergraduate level remains variable and unstandardised internationally.24 25 Within the UK, it has previously been noted that a significant proportion of the medical schools do not teach the skill and there is no consensus among those that do teach it, as to the best methodology.26 Many medical schools view it as a postgraduate training need,25 26 despite handover skills being a requirement for graduation2 and an important factor in patient safety.27 It has, however, been noted that junior doctors receive little or no training in medical handover.28 Unsurprisingly, junior medical staff are noted to feel stressed and unprepared to engage in medical handover, with many handovers being task focused rather than aiming to preserve overall patient safety.22 29
We believe robust and consistent teaching of handover skills and emphasis of its importance needs to be built into all undergraduate medical school curricula to minimise the risk of harm to patients. This study had two aims: (1) demonstrate a need for formal handover training through assessing the pre-existing knowledge, skills and attitudes of medical students and (2) study the effectiveness of a pilot educational handover workshop on improving confidence and competence in structured handover skills.
Methods
Third-year and fourth-year medical students from the University of Glasgow were invited to participate in the study during their medical and surgical clinical attachments in National Health Service (NHS) Lanarkshire between January and May 2019. A visual summary of the process the study participants underwent is seen in figure 1.
Flow diagram illustrating the study participant process.
Preworkshop questionnaire
A questionnaire (online supplementary appendix 1) was developed to determine students’ prior experience of teaching on clinical handover. Students were asked to detail their experience in delivering handovers and asked to rate their self-efficacy in handover abilities. Finally, the questionnaire sought to gauge perceptions of the importance of handovers by asking the students to rank the incidence and significance of substandard handovers on patient harm compared with other forms of medical error.
Supplemental material
Preworkshop assessment exercise
Several sets of mock patient notes, formatted in the style of local admission documentation, were created by the investigators for the purpose of this project. These cases were subject to review by consultant physicians prior to being finalised. Two of the cases were chosen by a random number generator to be used in the assessment exercise.
The students underwent an individual handover assessment under examination conditions in the format of an Objective Structured Clinical Examination (OSCE). Students were assigned either case notes A or B and were allowed 10 min to familiarise themselves with the case and formulate a handover. They were then allocated a further 5 min to relay a verbal handover to a simulated senior colleague (one of the researchers). Verbal and written instructions were provided to the student to emphasise that this exercise was not a test of their clinical knowledge, but rather their ability to formulate a comprehensive yet concise structured handover.
The students were marked contemporaneously by the researcher receiving the handover. All researchers had completed formal OSCE training. The OSCE process used as it is a widely accepted validated assessment technique measuring behaviour and higher levels of clinical performance.30 An assessment tool (online supplementary appendix 2) that had been adapted from the ‘Clinical Handover Assessment Tool’ (CHAT)31 using a modified Delphi process was used.32 33 The CHAT tool has been validated and used to assess medical student handovers.34 The researchers followed the ISBAR structure of the CHAT tool, formatting the study’s assessment tool into the SBAR headings. However, simplified marking criteria of ‘observed’ and ‘not observed’ behaviours was used rather than the graded assessments of the CHAT tool, as the students would not be receiving direction during the handover from the examiner to maintain assessment reliability. Subsequently, feedback was sought in two rounds from clinical experts, then medical education experts, to determine the case-specific criteria against which the student’s performance was judged for each domain of the handover tool. The utilisation of the modified Delphi process to determine the essential components required for each domain aimed to enhance the content and construct validity of the modified tool.
Supplemental material
Following the assessment, there was an immediate debrief with the student which ensured psychological safety. Feedback on performance was not divulged at this point so as not to confound results.
Handover workshop
After completing the preworkshop questionnaire and handover assessment, the students attended a two and a half-hour interactive workshop underpinned by medical education theory.
The workshop was produced using the seven steps of Madeline Hunter’s direct instruction design and encompassed adult learning principles of self-direction and problem-based learning set out by Knowles et al.35 36 Madeline Hunter’s model35 uses teaching interventions to achieve student mastery of a subject through guided and independent practice (table 1). The workshop was based on authentic clinical scenarios and drew on the previous experiences of the students through discussion. Discussion was facilitated regarding the purpose and importance of handover, as well as critical elements of, and the barriers to, performing a good handover. The learning content was designed to be relevant to the undergraduate students’ stage of professional development and prompted the students to reflect on their abilities to perform the handover skill in the clinical environment
Application of Madeline Hunter’s seven steps to the study workshop design
The learning objectives of the workshop were designed to guide students through the hierarchical levels of Bloom’s modified taxonomy.37 Learners were encouraged to develop their own learning goals and students progressed through handover knowledge acquisition to being able to critically evaluate their own and their peers’, handover performances.
The researchers created video examples of good and bad handover practice for students to critique. The researchers developed deliberately discursive case vignettes which were distributed for students to analyse and present within small groups. This task required students to pick out the pertinent information and formulate a structured and concise handover which was subject to peer critique and facilitator feedback. Peer critique has been shown in previous studies to facilitate learning and provide cognitive benefits by: providing opportunity for constructive reflection, allowing increased time spent on the task, increasing awareness of what constitutes quality, providing a sense of responsibility and increasing a student’s confidence in clinical practice.38–41
Teaching focused on ensuring the learners could comprehend the purpose and importance of a handover as well as providing them with skills of handover, including use of a handover structure, further reducing cognitive load.42 It also formed a foundation on which students could build as they progress towards graduation, allowing a conceptual change and providing deep learning.43 44
Postworkshop
The students were asked to complete a questionnaire within 1 week of the workshop, to determine whether there had been any improvement in the students’ self-efficacy in presenting a structured medical handover, or any change to the students’ perception of the frequency and significance of substandard handover, compared with other forms of medical error. Additionally, students were asked to rate the specific aspects of the workshop on Likert scales.
Immediately following completion of the questionnaire, a postworkshop assessment exercise was undertaken, again in the format of an OSCE, with students being allocated the case that they had not been exposed to preworkshop. The students did not have the same researcher for the preworkshop and postworkshop assessments and the assessors were blinded to the score from the student’s preworkshop performance.
Data analysis
The questionnaire and handover assessment marksheet data were assigned a unique student number and no identifying information was used in order to preserve student anonymity in data analysis.
Quantitative data were analysed using STATA (V.16.0; Stata). The two sample Wilcoxon rank-sum (Mann-Whitney) test was used to compare preworkshop and postworkshop confidence. McNemar’s test was used to analyse the handover assessment performance data over the 12 domains on the marksheet: (1) introduction, (2) verification of recipient identity, (3) communication of main concern, (4) prioritisation of main issue(s), (5) communication of structured history, (6) communication of key factors of history, (7) communication of examination findings, (8) communication of key investigation findings, (9) communication of key aspects of treatment commenced, (10) clear recommendation to recipient, (11) logical flow of information and (12) delivery of concise handover. A p<0.05 was accepted as being statistically significant.
Patient and public involvement
Neither patients or the public were involved in the design, implementation or participation of this study. However, improving patient safety through improved patient handover practice was a fundamental driver of this project.
Results
Forty-two medical students participated in the study, 4 fourth-year medical students and 38 third-year medical students. One was unable to complete the postworkshop assessments due to illness, their data were excluded from the analysis. Out of the 41 students who completed the programme, 35 completed both the prestudy and poststudy questionnaires which were then used for paired data analysis.
Questionnaire data
Twenty-nine (83%) of the 35 students reported no previous teaching on medical handover. Eleven (31%) reported having prior experience of delivering a handover and eight of these (73%) reported having received feedback. One student commented in the postworkshop feedback ‘very helpful due to lack of prior teaching’.
Prior to the handover workshop, students rated handover error as the second most common medical error and fourth most significant error in causing patient harm. Following the workshop, the students rated it as the most frequent cause of medical error and the second most significant error in terms of potential to cause harm. A free-text comment highlighted ‘the workshop was very useful as it clearly pointed out why a good handover is important’.
There was a statistically significant improvement in self-assessed student confidence in delivering a medical handover postworkshop. All students agreed or strongly agreed that their ability to perform a structure handover had improved. One student commented ‘attending the workshop made a massive difference and improvement in how I would present a handover. It was extremely beneficial in highlighting the key points required when handing over a patient. Really enjoyed the workshop!’
With regard to specific components of the teaching workshop, all students agreed or strongly agreed that receiving feedback regarding their individual handovers was useful and 32 (91%) of students agreed or strongly agreed that watching video examples of good and bad handover practice was helpful. One student commented in feedback nice to have video examples and group work to practice and receive feedback’ and another ‘the opportunity to practise handover in a safe learning environment was very beneficial as I now know what will be expected of me in future and can start working on improving this skill’.
Handover assessment performance
Table 2 displays the proportion of students achieving the agreed criteria set out by the modified Delphi process in each assessed handover competency domain. There was statistical improvement in 10 of the 12 handover competency domains (see figure 2).
Student performance in handover assessment exercises analysed by competency domain
Illustration of student performance over the handover competency domains per patient case preworkshop and postworkshop. *Denotes statistically significant improvement in performance.
There was no significant improvement in the ‘communication of key aspects of treatment commenced’ or ‘communication of key factors of history’ domains. The order of patient Cases (case notes A and B) used in preworkshop and postworkshop assessment made no statistically significant difference to student performance.
Discussion
The first aim of this study was to demonstrate the need for formal handover training for undergraduate students from our local medical school. Student performance was poor across handover competency domains in the preworkshop assessments. The majority of students in the study reported limited prior experience and teaching on medical handover. Although eight students reported having had previous practice at delivering medical handovers and engaging in feedback, only four recognised this prior experience as teaching and learning. Students reporting previous experience rated themselves as slightly more confident in their ability to deliver a handover than those without, but their overall performance was statistically non-superior. Although only four fourth-year medical students partook in the study, these senior students also performed poorly, which may suggest that the experiential learning of medical handover during clinical attachments is not sufficient. Cleland et al 22 concluded that undergraduate and early postgraduate training in handover, which is largely experiential in nature, is viewed as inadequate by practising clinicians. The preintervention performances and lack of student confidence in delivering handover suggests a learning need among students and demonstrates a role for explicit teaching, in keeping with this previous work.
The second aim of the study was to assess the effectiveness of a pilot educational workshop in improving confidence and competence in structured handover skills. The workshop was well received by medical students, the majority finding the individual elements useful and reporting an improvement in their confidence and self-efficacy in delivering medical handover.
Other studies have demonstrated a positive student response to explicit handover teaching.45 However, the majority of interventions to improve the quality of handover are not underpinned by educational theory.26 Our study aimed to move beyond level 1 of Kirkpatrick’s educational intervention evaluation model.46 The tutor-facilitated group work not only allowed the students to appraise handover performances to critically reflect and improve their confidence and knowledge, but allowed faculty to assess whether learning objectives were being achieved (level 2 of Kirkpatrick model). Following the workshop, the students reported an increased appreciation of how significantly handover errors affected patient care. They also recognised that communication errors, particularly in handover situations, were the most common form of medical error. This change in attitude also reflects Kirkpatrick level 2. The study attempted to assess level 3 of the Kirkpatrick model by assessing student handover behaviour and competency in the simulated workplace when the repeat simulated handover assessments were undertaken 1 week after completion of the workshop.
There was a significant improvement in student performance across 10 of the 12 handover competency domains in the postworkshop assessments. Changing the order of patient cases that the students handed over in the preworkshop and postworkshop assessment ensured that any improvement in the domains was due to the teaching intervention rather than due to the variation in patient cases. The two domains which did not show significant improvement were ‘communication of key aspects of treatment started’ and ‘communication of key factors of the patient history.’ The patient cases were designed to minimise the confounding effects of students’ clinical knowledge on the handover performance as this would have undermined the construct validity of the assessment. Therefore, the cases had a clear treatment plan documented in the mock notes which the student was required to identify and then handover, along with any outstanding components, to the handover recipient. This is perhaps why the baseline student performance in this handover domain was relatively strong and the postworkshop performance showed no significant improvement.
With regard to the communication of the ‘key factors of patient history’ domain, there was a similarly poor performance preworkshop and postworkshop. The consensus from the modified Delphi rounds indicated that the treatment escalation status of both mock patients was a required element when presenting the key factors of the patient history. The escalation plan had been written on the front of the admission document in the identical place as to the real life NHS Lanarkshire document and both patients were identified for ‘full escalation.’ This would mean that the intensive care unit would be an option in the event of further clinical deterioration as both patients were described as being previously fit and healthy. In the UK, it is standard practice to assume that a patient is for ‘full escalation’ unless otherwise explicitly stated. Therefore, it is not surprising that students omitted this aspect from their handovers and as such could not be awarded the mark in this domain. This is likely the reason for less than 20% of students meeting the full criteria in this domain, and highlights a limitation with the modified Delphi process.
Gordon et al 23 noted that handover education often provides tools such as mnemonics and templates to provide structure without any education in their use. We believe that we have provided a robust and reproducible intervention that is not only founded on educational theory but provides education, guided practice and peer critique on using a structure that is endorsed by WHO. This will allow progression to mastery of learning as set out in Madeline Hunter’s seven steps.
Limitations
There are limitations to this study which we would like to acknowledge. There was no control group to assess whether students would acquire the same level of improvement by participating in their placements. However, the prestudy questionnaire suggested that the fourth-year students, although small in numbers, had not had any formal teaching during their earlier undergraduate placements. It should also be noted that the fourth-year students showed no superiority in the initial preworkshop assessment, suggesting that the third-year students would not have acquired these skills through experiential learning alone. Previous work also suggests that current experiential handover learning alone is inadequate to fulfil undergraduate handover learning needs.22 29
The sample size for the study was small, with reliance on student volunteers rotating through their undergraduate placements in NHS Lanarkshire. Forty-three students out of a possible 120 volunteered. Additionally, the authors note there is a small amount of missing data due to participant illness and questionnaire non-return. It is unclear if this would have had any impact on study results.
A single-assessor examining a single simulated patient handover has limitations in terms of assessment reliability. This was logistically necessary due to staff availability and to ensure minimal disruption to students personal studies and clinical placement. It was also necessary to enable different assessors to mark the preworkshop and postworkshop exercises to minimise bias. Assessors were blinded to the preworkshop assessment scores of the student participants. The handover assessors had received formal OSCE training from the University of Glasgow.
Finally, although this study has potentially addressed a curriculum gap, it has only demonstrated a short-term influence on student handover performance. It has not demonstrated acquisition of level 3 of Kirkpatrick’s evaluation model beyond 1-week postintervention; neither does it access level 4 of the model—impact on organisational outcomes. Handover is a clinical skill which is perfected through practice and is a skill that the GMC expect from all graduates. There is currently no long-term follow-up of these students to assess whether the skills and knowledge they have attained will have a lasting improvement in their confidence and competence as they start working as a postgraduate year 1 doctor. This is likely a future development for the project.
The authors are now in talks with a local medical school to integrate this workshop into the third-year undergraduate curriculum. Furthermore, a faculty training day programme is in development which aims to highlight the need for explicit teaching of handover as a clinical skill to local clinical educators, and ensure that teaching of this skill is standardised across the clinical sites associated with the medical school.
Conclusion
Quality handover is essential for patient care. Although the GMC requires doctors to be competent at medical handover, there is inconsistent teaching in undergraduate curricula across the UK and Europe. Using an assessment tool developed through a modified Delphi process, our study assessed baseline capabilities of students in delivering handover. It identified a curriculum gap, as the students had received minimal or no prior teaching and their competence and confidence in carrying out this skill was limited.
A simple, robust and reproducible intervention, underpinned by medical education theory, can equip students with the required tools to significantly improve student competence and confidence in medical handover. The researchers hope this will give them the building blocks to continue to develop this skill as they progress through their undergraduate training and will attain the expected standard set by the GMC on graduation and beyond.
Acknowledgments
We would like to thank our colleagues Peter Cordiner, Neera Gajree, Meadhbh Halpenny and David Shields for their assistance in delivering the competency assessments. Additionally, we would like to acknowledge Cordiner and Gajree for their assistance in the design of the handover workshop, and Shields for early statistical analysis support.
References
Footnotes
NH and KC are co-first authors.
Twitter @drnick_holt
NH and KC contributed equally.
Contributors NH: MBChB, MRCP, MRCPS(Glas), PGCert Med Ed, Honorary Clinical Lecturer (University of Glasgow), Clinical Teaching fellow in NHS Lanarkshire. I declare that I made substantial contributions to the conception and design of the work, the acquisition and interpretation of the data. I made a significant contribution to the drafting and revision of the work and give approval for the final piece to be published. I agree to be accountable for all aspects of the work and will investigate and resolve any questions that arise. KC: BMedSci (Hons), MBChB (Hons) MRCP, Honorary Clinical Lecturer (University of Glasgow), PGCert Med Ed, Clinical Teaching Fellow in NHS Lanarkshire. I declare that I made substantial contributions to the conception and design of the work, the acquisition and interpretation of the data. I made a significant contribution to the drafting and revision of the work and give approval for the final piece to be published. I agree to be accountable for all aspects of the work and will investigate and resolve any questions that arise. DL: MBChB (Hons), MRCP, MRCPS(Glas), Honorary Clinical Lecturer (University of Glasgow), PGCert Med Ed, Clinical Teaching Fellow in NHS Lanarkshire. I declare that I made substantial contributions to the conception and design of the work, the acquisition and interpretation of the data. I made a significant contribution to the drafting and revision of the work and give approval for the final piece to be published. I agree to be accountable for all aspects of the work and will investigate and resolve any questions that arise. ZH: BSc (Hons) Medicine, MBChB, MRCEM, PG Diploma Health Professions Education, Honorary Clinical Lecturer (University of Glasgow), Clinical Simulation Fellow in NHS LanarkshireI declare that I made substantial contributions to the conception and design of the work, the acquisition and interpretation of the data. I made a significant contribution to the drafting and revision of the work and give approval for the final piece to be published. I agree to be accountable for all aspects of the work and will investigate and resolve any questions that arise.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This project was granted ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee in December 2018, ID number 200 180 052.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request.