Vol. 4 No. S1 (2022): Special Edition - August 2022
Competency Versus Proficiency in Pediatric Orthopaedic Training

JPOSNA® Special Edition
Advances in Pediatric Orthopaedic Education and Technical Training

Competency Versus Proficiency in Pediatric Orthopaedic Training

Benjamin A. Alman, MD1

1Department of Orthopaedic Surgery, Duke University, Durham, NC

Correspondence: Benjamin A. Alman, MD, DUMC 104002, 311 Trent Dr., Durham, NC 27710. E-mail: [email protected]

Received: April 18, 2022; Accepted: May 5, 2022; Published: August 15, 2022

DOI: 10.55275/JPOSNA-2022-0058

Volume 4, Number S1, August 2022

Abstract:

As pediatric orthopaedics has grown in size and scope, standardization of training is being considered as a way to ensure uniform standards. Competency is the knowledge, skill, and judgment to successfully complete a task but one that the trainee has not yet demonstrated that they can achieve on their own. Proficiency occurs when a task can be accomplished independently. Defining the standards for competency and proficiency are needed to develop such standards. In the North American context, residency is where competency is achieved and fellowship is a time in which pediatric orthopaedists achieve proficiency, although, for some highly specialized tasks, competency is also the goal of fellowship training. Attaining and assessing competency and proficiency can be accomplished using a structured curriculum and assessments. These do not necessarily require simulations, as observations in the clinical setting are useful for assessing proficiency. Defining the competencies and proficiencies required for pediatric orthopaedics and developing standard methods of assessment will ensure that pediatric orthopaedics fellowship trainees have acquired the base knowledge, judgment, and technical skills to be a safe and effective clinician.

Introduction

Training a pediatric orthopaedic surgeon requires many years of education, progressing through multiple training steps. Each step along the way has different goals. Defining the goals and quantifying their attainment is a long-standing quagmire amongst orthopaedic educators. Part of the reason for this is that while assessment of knowledge can be achieved using examinations, assessment of technical skills and judgment are more difficult. Furthermore, the critical factors from training that predict success in independent practice are poorly defined.1 Orthopaedic surgeons, even within subspecialties like pediatric orthopaedics, have various types of practices in which they may only see certain subpopulations of patients. Thus, defining the technical skills and knowledge base that all orthopaedists need to achieve can be problematic. For instance, it is not clear how much of the technical aspects of anterior cruciate ligament injury treatment are needed for a trainee who will become a spine surgeon.2 As pediatric orthopaedics becomes more subspecialized, these issues apply within our field as well. Despite these caveats, it is important that we define the training that is needed for a pediatric orthopaedist and develop methods to determine if fellowship trainees have acquired the base knowledge and technical skills to be a safe and effective clinician.

Competency, Proficiency, and Expertise

Over the past decades, multiple efforts were undertaken to define parameters that are important at different stages of training. An important principle is the differences between competency, proficiency, and expertise. While these terms frequently apply to the achievement of technical skills, they also apply to judgment and knowledge base. Competency is defined as having the knowledge, skill, and judgment to successfully complete a task but not yet demonstrating that one can achieve this on their own. Proficiency occurs when a task can be accomplished independently by a trainee. Expertise occurs when one has achieved a higher level of skill than the general practitioner and generally is used to identify individuals who teach others.3–5 In the North American orthopaedic educational context, during residency, one becomes competent in the skills necessary to become a pediatric orthopaedist but becomes proficient in these skills during fellowship training.6,7 This difference is central to how we develop a training curriculum and fellow assessment tools, as there are differences in teaching and measuring between competency and proficiency.

Competency-based education is an increasingly popular concept in curriculum development. At its core, a competency-based curriculum defines the goals that the learner must master to successfully complete the program and assesses the learner’s ability to achieve the goals. While these are termed “competency-based,” a better term might be “standards-based.” Standards are a list of learning goals that are required for completing a course or certification. They may or may not be relevant for the individual pediatric orthopedist’s ultimate goal for their clinical practice. Given the diversity in practices for an individual pediatric orthopaedist, their competency and proficiency requirements are likely different than “standards-based” requirements. Achieving these standards, however, is an important pragmatic goal, as without this, one generally cannot become certified to practice orthopaedics.5 In pediatric orthopaedics, defining such standards is an area of active ongoing work.

Defining Base Standards

With this as a background, how do we define what we need to teach pediatric orthopaedic fellows, and how will we assess if they are safe for practice? While competency and proficiency needs will vary according to a fellow’s plans for future practice, as a specialty, we will need to define base standards. As an example, while a pediatric orthopaedist who will not do spine surgery would not need to be proficient in performing a posterior spinal fusion and instrumentation, they would need to be proficient in the assessment and non-surgical management of spinal deformity. There are more complex spinal procedures, such as vertebral body resections, which the general pediatric orthopaedic fellow who will do spine surgery should be competent in but not necessarily proficient in. As a specialty, we may want to consider base standards and then add additional standards-based on the specific subspecialty that a fellow plans to undertake. The development of standards for competency and proficiency is important as we grow as a subspecialty to ensure that we have measures to determine if a fellow is safe for practice. Once such standards are developed, teaching and assessing for them using a competency and/or proficiency-based framework will ensure that the future pediatric orthopaedists we train have the skills to provide safe, effective, and quality care.

Acquiring procedural skills is an important area in fellowship training. Currently, the progress of fellows is assessed through faculty evaluations and case logs. Such analyses provide only surrogate measures of competency or proficiency. While case numbers are objective, they do not reflect competency or proficiency as the number of cases required to achieve this varies with each individual trainee and the training environment. Faculty evaluations are subjective and lack the rigor and reliability required for assessments on a large scale.8 In addition, teaching surgical skills on patients requires sufficient patient numbers and baseline technical competency. In some areas, like a circumferential release for clubfoot, far fewer cases are performed than decades ago,9 making sufficient case numbers problematic. An alternative is to incorporate structured surgical skill education, including surgical simulation, into fellowship training. Multiple studies have shown that structured surgical skill programs, which include surgical simulation, can be used to train, assess, and improve operative performance.10–12

Identifying Competency and Proficiency Skills

As we develop such programs, an early step is to identify the competency and proficiency skills required for successful fellowship completion. These skills can then become the standard that is required to become a pediatric orthopaedist. Broad consensus is needed to identify these. In this process, it will be critical to identify skills that are important for all pediatric orthopaedists, as well as those that are required for subspecialties, and it is necessary that we clearly define what the competency and proficiency expectations will be. Prior efforts used structured consensus building amongst educational leaders in the field to achieve this goal,13,14 and such a process might be a first step in pediatric orthopaedics.

The foundation of a competency or proficiency-based curriculum is that the teacher tells the learner what is expected, teaches them this, and then assesses for the ability to achieve competency in the specific knowledge, skills, and/or tasks. A structured approach to teach identified standards and then assessment of the learner is fundamental to this approach. Simulation is not necessarily needed in this process but rather is one tool that can be used in teaching and assessing competency and proficiency.13,14 Indeed, many such clinical programs use observation in the clinical setting and the operating room.15 However, observations in the clinical setting need to be structured, as traditional observations by faculty have a limited value in predicting future clinical proficiency and skills.8 A meaningful assessment of competence and proficiency is a critical component of training. Timely ongoing assessments along with comprehensive periodic reviews are tested methods to ensure that trainees continue to progress. Selecting an effective assessment program, along with a utilization of the right combination of a methods and appropriate assessor selection and training, is critical to successful implementation of such programs.16

One advantage of adopting such an approach is standardization so that no matter where one does their fellowship, it will be clear what standards a fellow has achieved. This will also ensure that an individual fellow will be able to obtain the training they need no matter which program they matriculate into. Standardization increases efficiency and effectiveness in multiple areas such as in industry and clinical care. While there is little data on this in the educational realm, it is likely that adopting such an approach will produce this outcome in fellowship training as well.

Simulation may be more helpful in the attainment and assessment of competency than in attaining and assessing proficiency. Studies found that structured surgical simulation is effective in training, assessing, and improving operative performance. While there are diverse tools that can be used to assess competence for technical skills, there remains a lack of consensus regarding the definition of technical proficiency. Further work is required to identify when and how often trainees should be assessed and to identify strategies to train faculty to ensure timely and accurate assessment.17 There is a connection between cognitive and technical skills in surgery. Teaching cognitive skills improves the achievement of technical competency.18 Given the types of skills required from orthopaedic fellows who have already completed residency training, pediatric orthopaedic fellowship training might rely less on simulation and more on other structured teaching and assessment approaches.12,15

Creating a Structured Assessment Program

Previous studies shed light on how to develop structured orthopaedic and assessment programs. There is a wealth of evidence that there is little crossover in skills.19 For instance, teaching k-wire proficiency does not generally crossover to proficiency in plate and screw use for fracture fixation. Objective assessments using simulation work best with multiple observers, reducing intraobserver and interobserver variability.20 Video of trainees performing surgery can be used to teach techniques and identify areas for improvement needed to achieve proficiency.21 Structured operative assessments and coaching will improve outcomes, even in experienced surgeons.22 Video can be used in the operating room to improve reliability of proficiency assessments.23 Such approaches can be successfully applied to non-operative treatments, such as Pavlik harness application.24 Taken together, this supports the notion that teaching and measuring proficiency can be accomplished with structured education and assessment programs in the clinical environment. Simulation might be best used for specific competencies, especially in areas in which patient numbers are limited.

Piloting a program and then using successful aspects of the pilot to develop a broader curriculum and assessment program is a pragmatic approach to developing a successful orthopaedic surgical curriculum. This was the case in orthopaedic resident education in Canada, where data from a pilot program instituted at one site14,25 was used in the development of a nationwide competency-based curriculum.26 This approach could be applied in pediatric orthopaedics to develop a competency and proficiency-based curriculum.

As with fellows, residents also have areas of pediatric orthopaedics where competency and proficiency are required. For most areas of pediatric orthopaedics, residents require competency rather than proficiency, and as such, simulation may play a larger role than for fellows. There are areas, however, in which proficiency is needed, for instance, in many types of fracture care. Many residency programs are structured to ensure competency for their trainees that includes pediatric orthopaedics.

Summary

With the growth of pediatric orthopaedics in numbers of providers and scope of the field, a competency and proficiency-based curriculum is an efficient and effective way to standardize what our fellows learn and the skills they acquire during training. The development of such an approach will require consensus among pediatric orthopaedic educators and perhaps some degree of centralized education and assessment (e.g., at a national meeting). Much of what fellows need to achieve is to develop proficiency, and data suggests that real-time structured clinical observation with feedback and assessment is an effective way to achieve this goal. This makes it easier to apply this curriculum in smaller programs, as multiple types of simulation equipment may not be needed. Such a curriculum will ensure that future pediatric orthopaedists have the knowledge base, judgment, and technical skills to provide safe, effective, and quality care.

Disclaimer

The author has no conflicts of interest to report.

References

  1. Pugh C, Plachta S, Auyang E, et al. Outcome measures for surgical simulators: is the focus on technical skills the best approach? Surgery. 2010;147(5):646–654.
  2. Black KP, Alman BA, Levine WN, et al. Orthopaedic resident education--it’s a whole new game: “if I’m going to be a spine surgeon, why do I need to learn how to reconstruct an anterior cruciate ligament?”: AOA critical issues. J Bone Joint Surg Am. 2012;94(13):e96.
  3. Jaffer A, Bednarz B, Challacombe B, et al. The assessment of surgical competency in the UK. Int J Surg. 2009;7(1):12-15.
  4. Carlson JN. Procedural competency and procedural proficiency. Harvard Macey Institute. 2018.
  5. Reznick RK, MacRae H. Teaching surgical skills--changes in the wind. N Engl J Med. 2006;355(25):2664-2669.
  6. Alman BA, Ferguson P, Kraemer W, et al. Competency-based education: a new model for teaching orthopaedics. Instr Course Lect. 2013;62:565-569.
  7. Sonnadara RR, Mui C, McQueen S, et al. Reflections on competency-based education and training for surgical residents. J Surg Educ. 2014;71(1):151-158.
  8. Putnam MD, Kinnucan E, Adams JE, et al. On orthopedic surgical skill prediction--the limited value of traditional testing. J Surg Educ. 2015;72(3):458-470.
  9. Zionts LE, Sangiorgio SN, Ebramzadeh E, et al. The current management of idiopathic clubfoot revisited: results of a survey of the POSNA membership. J Pediatr Orthop. 2012;32(5):515-520.
  10. Anetzberger H, Reppenhagen S, Eickhoff H, et al. Ten hours of simulator training in arthroscopy are insufficient to reach the target level based on the Diagnostic Arthroscopic Skill Score. Knee Surg Sports Traumatol Arthrosc. 2022;30(4):1471-1479.
  11. Moktar J, Bradley CS, Maxwell A, et al. Skill acquisition and retention following simulation-based training in pavlik harness application. J Bone Joint Surg Am. 2016;98(10):866-870.
  12. Sonnadara RR, Garbedian S, Safir O, et al. Orthopaedic Boot Camp II: examining the retention rates of an intensive surgical skills course. Surgery. 2012;151(6):803-807.
  13. Harris P, Bhanji F, Topps M, et al. Evolving concepts of assessment in a competency-based world. Med Teach. 2017;39(6):603-608.
  14. Ferguson PC, Kraemer W, Nousiainen M, et al. Three-year experience with an innovative, modular competency-based curriculum for orthopaedic training. J Bone Joint Surg Am. 2013;95(21):e166.
  15. Phillips DP, Zuckerman JD, Kalet A, et al. Direct observation: assessing orthopaedic trainee competence in the ambulatory setting. J Am Acad Orthop Surg. 2016;24(9):591-599.
  16. Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609-616.
  17. Fahim C, Wagner N, Nousiainen MT, et al. Assessment of technical skills competence in the operating room: a systematic and scoping review. Acad Med. 2018;93(5):794-808.
  18. Kohls-Gatzoulis JA, Regehr G, Hutchison C. Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg. 2004;47(4):277-283.
  19. VanHeest A, Kuzel B, Agel J, et al. Objective structured assessment of technical skill in upper extremity surgery. J Hand Surg Am. 2012;37(2):332-337.
  20. Dwyer T, Slade Shantz J, Kulasegaram KM, et al. Use of an objective structured assessment of technical skill after a sports medicine rotation. Arthroscopy. 2016;32(12):2572-2581.e3.
  21. Karam MD, Thomas GW, Koehler DM, et al. Surgical coaching from head-mounted video in the training of fluoroscopically guided articular fracture surgery. J Bone Joint Surg Am. 2015;97(12):1031-1039.
  22. Pradarelli JC, Yule S, Lipsitz SR, et al. Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons’ practice. Surg Endosc. 2021;35(7):3829-3839.
  23. Taylor LK, Thomas GW, Karam MD, et al. Developing an objective assessment of surgical performance from operating room video and surgical imagery. IISE Trans Healthc Syst Eng. 2018;88(2):110-116.
  24. Bradley CS, Moktar J, Maxwell A, et al. A reliable and valid objective structured assessment of technical skill for the application of a pavlik harness based on international expert consensus. J Pediatr Orthop. 2016;36(7):768-772.
  25. Nousiainen MT, Mironova P, Hynes M, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med Teach. 2018;40(10):1042-1054.
  26. Nousiainen MT, Bardana D, Gofton W, et al. Creating a national competency-based curriculum for orthopaedic surgery residency: The Canadian Experience. JB JS Open Access. 2022;7(1):e21.00131.
Copyright (c) 2022 JPOSNA®