JPOSNA^® Special Edition
Advances in Pediatric Orthopaedic Education and Technical Training

REAL and SCOPE: Evolution from Simulation Team Training to Leadership Curriculum Development for Residents and Fellows

Peter M. Waters, MD, MMSc¹ and Peter Weinstock, MD, PhD²

¹Department of Orthopaedic Surgery, SIMPeds program, Boston Children’s Hospital, Harvard Medical School, Boston, MA; ²Department of Anesthesia, Critical Care, and Pain Medicine, SIMPeds program, Boston Children’s Hospital Harvard Medical School, Boston, MA

Correspondence: Peter M. Waters, MD, MMSc, Atrium Health, 2001 Vail Ave., Charlotte, NC. E-mail: [email protected]

Received: February 24, 2022; Accepted: April 7, 2022; Published: August 15, 2022

DOI: 10.55275/JPOSNA-2022-0055

Volume 4, Number S1, August 2022

Abstract:

Postgraduate medical training and post-training career continuing medical education in surgical subspecialty fields focus predominately on developing and mastering technical expertise and voluminous educational content, all of which are important and necessary. However, there is limited organized learning on leadership and non-technical skills (situational awareness, teamwork, communication, task management, decision-making) for surgeons in training, and thereafter as attending faculty, to gain expertise in leading the OR team of surgeons, anesthesiologists, and nurses during complex surgery. Aside from our simulation work on technical skills, we started and grew our training in team skills, leadership, and followership over the past decade-plus. We advanced to not only in-person professional assessments of simulated performances of case scenarios but also the use of audiovisual review with validated scoring (NOTSS, ANTS, SPLINTS). This led to the use of these tools in assessing live OR performance (REAL) and coaching (SCOPE) with educational debriefings for the betterment of the individual and the team of professionals. Now, we have a formal curriculum for orthopaedic surgery residents during their third year of training and pediatric surgical fellows (orthopaedic surgery, otolaryngology, plastic surgery, sports medicine, and urology) throughout their fellowship year. We use simulation, REAL, and SCOPE amongst other technics, with formative and summative assessments, to improve their leadership, non-technical skills, and emotional intelligence. Over the past 5 years, the leadership curriculum for surgical fellows and now residents has been highly successful. Our orthopaedic surgery residents and pediatric orthopaedic surgery and sports medicine fellows are better equipped to handle conflict, foster within and across profession collaboration, work in diverse environments, and build and lead a team both on a daily basis and over time. This increases patient safety, mitigates risk, and leads to better outcomes. We encourage others to use these or similar tools in their institutions and programs.

Key Concepts:

• Simulation can be successfully used to teach and assess non-technical skills of situational awareness, task management, decision-making, teamwork, communication, and leadership.
• Professional in-person assessment, audiovisual recording and analysis, use of validated tools (NOTSS, ANTS, SPLINTS), and post-learning event debriefings lead to improved individual and team performance through a better shared understanding and collaboration.
• Real event analysis and learning (REAL) and Surgical Coaching for Operative Performance Enhancement (SCOPE) are valuable in improving OR individual and team skill development during training and throughout a surgical career.
• A formal leadership curriculum using simulated case scenarios, simulation of disruptive professional behavior and high-risk OR situations, REAL and SCOPE in live OR cases, amongst other learnings, can improve the non-technical skills and emotional intelligence of orthopaedic surgery residents and subspecialty surgical fellows.
• This extended, focused curriculum of leadership development supplements the technical expertise and educational content learning in training and better enables trainees to manage professional conflict, work in diverse environments, mitigate risk and enhance patient safety through being a part of and leading OR teams.

Introduction

Orthopaedic surgery residents and pediatric orthopaedic surgery fellows, similar to their faculty attendings, focus their education intently on the acquisition of knowledge and technical expertise, all for their individual betterment, which ideally transfers to their patients through improved surgical decision-making and performance. During training and subsequent continuing orthopaedic surgical education, an abundance of formal and informal opportunities to learn knowledge content and technical expertise exist. These include institutional educational conferences, case discussions, post-surgical debriefings, annual meetings, focused subspecialty meetings, and technical execution training on specific procedures and equipment, to name a few. With post-graduate medical education training, both internal and external formative and summative assessments are required to determine progression and competency in these areas (examples include end-of-rotation assessments, ACGME milestones, OITE and ABOS exams, etc.). Ideally, young surgeons learn the WHY of each procedure, followed in importance by the WHAT and then HOW it is going to be performed.

This educational continuum focuses on the individual orthopaedic surgeon (the WHO) in relation to their peers and established standards. Implicit in this construct is that higher knowledge and technical skill for each surgeon leads to better outcomes and less error risk for their patients.

Similar educational programs and assessments exist for our surgical nursing and anesthesia colleagues. Each professional in the operating room is responsible for their knowledge and task management—their own professional WHY, WHAT, and HOW. These professional work tasks and decisions are often performed independently in the operating room, frequently in isolation (induction of anesthesia before prepping and draping, surgery while in anesthetic steady state, cessation of anesthesia and extubating after wound closure and dressing application, as examples). When cooperation is required to properly complete a task, it is usually in an OR professional dyad working together (circulating nurse and anesthesiologist during induction, scrub and surgeon during surgery, surgeon and anesthesia in preoperative planning). All the elements of a case can function reasonably well in most teams when there is no crisis or conflict. However, there is an increased risk that uncooperative or lower-performing teams will fail to execute as needed in an untoward event.

Surgery is both an individual and team act. Competent technical performance of individual professional activities is the minimum threshold for professional development. Error prevention and patient risk minimization require the ENTIRE team to be high functioning, especially in the arena of non-technical skills: communication, teamwork, leadership, situational awareness, and decision-making. Yet, how much time is spent during training and throughout a surgical career on formally teaching and assessing these skills by validated methods (Figure 1).

Figure 1. Inter-professional team skill development is taught and assessed in the simulation suite across all specialties at our institution.

We started simulation education of non-technical team skills when we (1) realized that wrong-site, wrong-patient, wrong-procedure events (WSPEs) were still occurring despite compulsory surgical safety checklists decades after COA, AAOS, WHO, and JACHO initiatives and mandates^1–5 (an estimated 25% of orthopaedic surgeons in their lifetime will perform a WSPE⁶; by anonymous self-reporting 50% of spine surgeons and 21% of hand surgeons already have⁷); (2) understood we still have equipment availability problems that prolong surgery and can impact outcome despite iterative, preventive improvements in electronic case scheduling and case cart management of inventory; (3) learned there was wide team performance variation in contingency planning and execution around in OR unexpected events. Each of these situations put our patients, first and foremost, but also our professionals and institution, at risk.⁸ Non-technical skill execution requires all the OR professionals to be in the same space, performing their individual professional tasks while communicating, alternating leadership roles, performing as a team, and being acutely aware and responding to changing situations.⁹

More than a decade ago, we started this work with a risk management-sponsored high-fidelity simulation initiative (see video #1, Figure 2) that focused on team skill development to reduce risk and error in the operating room.

Figure 2. High-fidelity femur fracture simulator case with unexpected bleeding and hypotension occurring and the entire OR team of nursing, anesthesia, and pediatric orthopaedic surgery having to respond. Surgical leadership and non-technical skills were assessed and debriefed with the entire team for future enhanced performance.

We progressed to scenario simulations (Figures 3, 4) for WSPEs: wrong site, wrong patient, wrong procedure (inaccurate consents, wrong equipment for procedure) and unexpected in OR events (bleeding, anaphylactic reaction to antibiotics) with immediate after event debriefing assessments of individual and team performances. (Video #2).

Figure 3. Short vignette team training simulations and debriefings on non-technical skills around at-risk situations, including wrong-site, wrong-consent, and wrong-equipment scenarios, amongst others.

Figure 4. The summative case and plan for development and institutionalization of REAL.

We progressed to in OR audiovisual recording and live observation of real surgical procedures from “wheels in to wheels out” with validated assessment tool scoring (REAL).

REAL (Real Event Analysis and Learning) evolved from our SimPEDS team skill simulation work; and SCOPE (Surgical Coaching for Operative Performance Enhancement) originated at the Brigham and Women’s STRATUS center for medical simulation and the Adriadne Labs at both the Brigham and Women’s Hospital and the Harvard T.H. Chan School of Public Health, led by their surgical coaching team (Douglas Smink, MD; Steven Yule, PhD; Jason Pradarelli, MD). REAL remained a Boston Childrens’ initiative and SCOPE evolved into a Harvard surgical initiative across four institutions including ours.^10–12 All of the simulation and live event analysis and coaching are now part of our leadership curriculum for Harvard orthopaedic surgery residents (HCORP) and Boston Children’s surgical fellows’ educational programs. This evolution of non-technical skill education and professional team training and analysis has been built on a 15-plus-year platform of psychological safety (Figure 5) that fosters trust in transparent communication and comfort with professional assessment.^13–15

Figure 5. Diagram of the evolution of simulation programs at Boston Children’s from skill development to team training to REAL.

Description of Simulation Exercises

I. Simulation of OR At-Risk Events for Team Performance Assessment and Learning

Simulation is used to assess individual and team performance in short vignettes. Case scenarios with specific disruptions are planned and executed in the simulation center. The simulation center is a near-exact replica of the hospital OR, including all the anesthesia equipment, nursing workstations, OR table, scrub nurse tables and equipment, overhead lights, wall monitors, and placards including the WHO Protocol for sign in, time out, and sign out. Room dimensions and paint colors are near identical to in-hospital ORs to create an aura of reality. Alternatively, these scenarios have been and can be done in an actual OR room if such a simulation center does not exist.

Teams of scrub, circulating nurse, surgeon, and anesthesiologist are established just prior to the simulation. Any operative scenario can be created with planned disruptions to assess individual and team performance. For example, we use a trauma scenario of closed versus open reduction and pinning, with possible neurovascular repair, of a displaced supracondylar humerus fracture in a pediatric “patient” (mannequin) (Figure 3). Another trauma patient scenario we use is unexpected bleeding and hypotension during ORIF of a femur fracture. A spine scenario used is a deformity fusion patient with neuromonitoring changes during instrumentation. You choose or create what fits your place and team’s needs.

Before the start of the scenarios, each team is informed that they are to proceed as if this was a live on-call trauma or elective surgical case. Each team is then confronted with different, impromptu combinations and permutations of simulated OR complications, such as (1) anesthesiologists’ consent form not signed, (2) anaphylaxis after prophylactic antibiotic administration, (3) surgical consent not matching the intended procedure or site, (4) loss of monitoring signals during spinal instrumentation; (5) uncontrolled bleeding and hypotension during surgery.

The scenarios are experienced in rapid sequence over the course of one hour. Interprofessional debriefing discussions after scenario completion are led by experienced members of the simulation center. High-risk and risk-mitigating behaviors are identified and discussed. Individual and team performance issues are shared for learning and professional development that hopefully transfers to real OR case performance.

II. Progress to Validated Scoring Assessments

To enhance and standardize simulation scenario performance assessment, audiovisual recordings were utilized. The scenarios were recorded by three Gorillapod mounted Axis M3046-V Network Cameras and a feed from the OR STERIS light camera, with cameras placed to ensure clear recording of individuals, teams, and activities. Audio included two Audio-Technica Pro45 ProPoint. Cardioid Condenser Hanging Microphones and two Audio-Technica ATW-T1006 Boundary Microphones. Recordings were stored on CAE Healthcare Intuity LearningSpace for review and scoring,

Simulation performance was scored by trained coders using validated non-technical skill assessment tools from the University of Aberdeen and Royal College of Surgeons of Edinburgh, surgeons by the Non-Technical Skills for Surgeons System Handbook v1.2 (NOTSS)¹⁶; anesthesiologists by the Framework for Observing and Rating Anesthetists’ Non-Technical Skills (ANTS)¹⁷; and nursing scrub by the Scrub Practitioners’ List of Intraoperative Non-Technical Skills (SPLINTS).¹⁸ There is extensive published work on the validation of NOTSS, ANTS, and SPLINTS and their use in simulation, in multiple-site operating rooms, and in educational settings, including by the American College of Surgeons.

Surgeons (NOTSS), anesthetists (ANTS), and nurses (SPLINTS) are respectively scored on the four-point scale (poor=1, marginal=2, acceptable=3, good=4) for each behavioral category per the validated tool: ANTS (task management, decision-making, team working, and situational awareness); NOTSS (situational awareness, teamwork, leadership, and decision-making) (Figure 6) and SPLINTS (task management, situational awareness, communication, and teamwork).

Figure 6. Non-Technical Skills for Surgeons (NOTSS) for structured, validated observations and scoring of surgeons’ behaviors in the operating room.

The audiovisual recordings of each simulated case are also scored on compliance with the Sign-in, Time Out, and Sign-out Checklist. The checklist includes all components of the WHO surgical safety protocol used for every surgical case in our live operating rooms.

The use of audiovisual recording simulation case validated tool scoring, compliance with checklist, team review, and education thereafter can be time consuming and resource heavy. When we looked at in-person assessment by simulation trainer versus audiovisual recording, they were statistically comparable.

III. Real Event Analysis and Learning (REAL)

Based on our experience with simulation for non-technical skill assessment and education, we were able to iteratively learn how to assess live OR performance.^19,20 Initially, this started with early adopter professionals and teams. Once “proof of concept” and reliability with all the technical aspects were achieved, then audiovisual and in OR assessment of situational awareness, teamwork, communication, task management, leadership, decision-making, and surgical safety checklist compliance was performed (Figure 7). Initially, this was for a subset of our hand and sports surgical teams. Most recently, our spine surgeons participated in non-technical skill assessment and learning with the added complexity of assessing compliance with spine specific checklists during induction, time out, instrumentation (pedicle screw and then bent rod placement), O-arm imaging and reading, deformity correction, neuromonitoring, including response to any changes, blood pressure desired zones and responses to changes, amongst others. REAL’s phased deployment drew on diffusion of innovation theory that encouraged adoption by increasing numbers of professionals until it became part of the norm.

Figure 7. Four audiovisual cameras and recorders were placed about the OR to record for non-technical skill scoring by ANTS, SPLINTS, and NOTSS from wheels in to wheels out of an operative procedure.

Debriefings²¹ after each surgery were performed similar to the simulation protocols, but now there was more risk of resistance to learning and improving. True performance assessment can be more threatening but also more effective in producing positive change. Such processes can be viewed like review of athletic game tapes and then practicing (simulation) to improve next game performance. Centering clinicians as key collaborators and champions facilitated alignment with team goals and forestalled potential perceptions of recording as adversarial or threatening. Solution-oriented debriefings allowed team members to recognize system vulnerabilities and assume joint responsibility for averting both adverse events and near misses across teams and hierarchies. Further, the continued application of psychologically safe debriefings enables sequential reduction in clinician resistance and barriers to change. Ongoing discussion and clearly articulated calls to action are critical; the hard work of achieving openness, adoption, and partnership acculturation is accomplished through repeated, stepwise exposure and reapplication of key guiding principles rather than a single, isolated introduction of a mandated initiative

IV. Surgical Coaching for Operative Performance Enhancement (SCOPE)

Surgical coaching is an evolving area of study, faculty development at all career stages, and as a training tool for residents and fellows to enhance learning and performance in both technical and non-technical domains. Initiated and organized by the Adriadne Labs at the Brigham and Women’s Hospital and the Harvard T.H. Chan School of Public Health, SCOPE was instituted at four Harvard academic medical centers including ours. The surgical coaching format was pre-education on coaching for all participants, pre-observation goal setting between coach and coachee, three observational surgical sessions, and structured debriefing meetings thereafter. During SCOPE implementation, important factors for building an effective surgical coaching program were identified: (1) self-identified goals by coachee and agreed upon by the coach; (2) collaborative analysis of each surgical session; (3) constructive feedback; and (4) action planning. Like REAL, this program was piloted at the attending surgeon level and then evolved into a training tool both in simulation scenarios and in REAL for residents and fellows.

V. Leadership Seminars

All of this non-technical education and assessment work is now embedded in our orthopaedic surgery residency program and institution-based pediatric surgical fellowships (orthopaedic surgery, sports medicine, plastic surgery, otolaryngology, urology) in leadership seminars. The premise is that leadership and emotional intelligence can be taught and that they are an essential part of a subspecialty surgical education and career development.

Resident seminars are held twice a month for 6 months (Figure 8) and fellows’ seminars once a month for 12 months. Each seminar lasts for an hour with individual and team-based pre-session reading and video reviews, summary presentations by teams of residents and fellows on a rotating basis, faculty support and feedback in session, and high-level seminar dialogue between all participants as emerging leaders (Figure 9). Case vignettes of professional disruption, conflict, high-risk scenarios, and potential surgical complications are studied and discussed. Videos of surgeon failures of situational awareness, task management, teamwork, leadership, communication, and decision-making are reviewed, scored by NOTSS by each participant, and then discussed in seminar. The fellows then participate in a simulation of OR at-risk events in which each trainee must act as an attending while OR nurses and anesthesiologists purposefully disrupt the OR environment in different scenarios. These short 15- to 20-minute scenarios are viewed by peers and faculty, scored, and the experience debriefed in a highly professional manner. Finally, the fellows serve as reciprocating coach and coachee for one another as they perform an appropriate for stage of training surgery with attending scrubbed but only observing unless they must intervene for patient safety and best outcome. Immediate post-surgery feedback is given adhering to the coaching principles outlined.

Figure 8. Title page of the residents’ reflections on the initial Orthopaedic Surgery Resident Leadership seminar and its essential importance to their HCORP training.

Figure 9. Image of page 1 of the education website for the “Leadership Seminar for Surgical Fellows,” now in its 5th year at our institution.

To reinforce, the main thread in the leadership seminar is fostering leadership development and improving emotional intelligence (social skills, self-awareness, self-regulation, motivation, empathy). A major component of the process involves progressive learning from (1) case scenarios reviewed and discussed, to (2) video review and NOTSS scoring of surgical failures of decision making, task management, teamwork, communication, situational awareness and leadership, into (3) simulation of OR professional disruptions and high-risk scenarios in which the trainee must manage the OR environment and case as an attending, and then finally (4) a REAL case in which trainees act as coach and coachee reciprocally with faculty observation and input. Deciding upon and performing a complex operation well is not easy. Leading an entire OR team while performing that operation, especially through at-risk moments, is very challenging (Figure 10).^20,22 To us, it makes sense to train for those moments to mitigate risk to the patient, to all the professionals, and to the institution where you work. For trainees to do this work over the course of their fellowship year, or as they transition to senior residency, better prepares them for the unexpected in their future.

Figure 10. The penultimate of leadership and team training: managing the surgical execution of a complex operation while educating and leading your teams to the best outcome for the patient.

Summary

We believe non-technical skills really matter. The evidence is overwhelming, including in our institution. Therefore, we teach and train to improve teamwork, communication, leadership, situational awareness, decision-making, and task management within and across the professional teams in the operating room. We understand our surgical residents and fellows are already leaders, but improving their leadership skills and emotional intelligence is vital to a successful career. Thus, we not only foster and assess their knowledge content and technical expertise sequentially during training but, for the past 5 years, we do the same for leadership development. We encourage you to do the same.

Acknowledgement

There are hundreds of professionals who contributed to this work over the past 15 years. There is no way to properly acknowledge them all except to simply say this manuscript represents each and every one of their contributions. We are merely the conduit of communication.

Disclaimer

The authors have no relevant conflicts of interest to report.

References

1. American Academy of Orthopaedic Surgeons Council on Education, Report of the Task Force on Wrong-Site Surgery. Rosemont, Ill American Academy of Orthopaedic Surgeons1998. Available at: http://www3.aaos.org/safety/ewsurgery.html. Accessed September 1, 2003.
2. Lewis BD. Initial evidence: reduced levels of wrong sided surgery [Internet]. c2009. cited 2012 Feb 5. Quebec, CA: Canadian Orthopaedic Association; Available at: http://www.coa-aco.org/library/practice-management/initial-evidence-reduced-levels-of-wrong-sided-surgery.html.
3. North American Spine Society. Prevention of wrong-site surgery: sign, mark & x-ray (SMaX). LaGrange, IL: North American Spine Society; 2001.
4. Stahel P. The 10th year of the “Universal Protocol”: are our patients safer today? Bone Joint. 2014;360(3):7-10.
5. Geraghty A, Ferguson L, McIlhenny C, et al. Incidence of WRONG-SITE Surgery LIST errors for a 2-year period in a single national health service board. J Patient Saf. 2020;16(1):79-83.
6. Santiesteban L, Hutzler L, Bosco 3rd JA, et al. Wrong-site surgery in orthopedics: Prevalence, risk factors, and strategies for prevention. JBJS Rev. 2016;4(1):e3.
7. Meinberg EG, Stern PJ. Incidence of wrong site surgery amongst hand surgeons. J Bone Joint Surg. 2003;85(2):193–197.
8. Greenberg CC, Regenbogen SE, Studdert DM, et al. Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg. 2007;204(4):533-540.
9. Berry JC, Davis JT, Bartman T, et al. Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J Patient Saf. 2020;16(2):130-136.
10. Pradarelli JC, Yule S, Panda N, et al. Surgeons’ coaching techniques in the the surgical coaching for operative performance enhancement (SCOPE) program. Ann Surg. 2022;275:e91-e98.
11. Pradarelli JC, Yule S, Panda N, et al. Optimizing the Implementation of Surgical Coaching through feedback from practicing surgeons. JAMA Surg. 2021;156:42-49.
12. Pradarelli JC, Yule S, Lipsitz SR, et al. Surgical coaching for operative performance enhancement (SCOPE): skill ratings and impact on surgeons’ practice. Surg Endosc. 2021;35:3829-3839.
13. Delizonna L. High-performing teams need psychological safety. Here’s how to create it. Harvard Business. 2017 Aug 24. Review. Available at: https://hbr.org/2017/08/high-performing-teams-need-psychological-safety-heres-how-to-create-it.
14. Rosenbaum, L. Cursed by knowledge-building a culture of psychological safety. N Engl J Med. 2019;380(8):786-790.
15. Roussin CJ, MacLean TL, Rudolph JW. The safety in unsafe teams: a multilevel approach to team psychological safety. J Manag. 2014;42:1409-1433.
16. Yule S, Flin R, Paterson-Brown S, et al. Development of a rating system for surgeons’ non-technical skills. Med Educ. 2006;40(11):1098-1104.
17. Fletcher G, Flin R, McGeorge P, et al. Anaesthetists’ non-technical skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth. 2003;90(5):580-588.
18. Mitchell L, Flin R, Yule S, et al. Evaluation of the scrub practitioners’ list of intraoperative non-technical skills (SPLINTS) system. Int J Nurs Stud. 2012;49(2):201-211.
19. Hu YY, Mazer LM, Yule SJ, et al. Complementing operating room teaching with video-based coaching. JAMA Surg. 2017;152(4):318-325.
20. Parker SH, Flin R, McKinley A, et al. Factors influencing surgeons’ intraoperative leadership: video analysis of unanticipated events in the operating room. World J Surg. 2014;38(1):4-10.
21. Rudolph JW, Simon R, Rivard P, et al. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25(2):361-376.
22. Edmondson A. Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. J Manag Stud. 2003;40:1419-1452.