Entrustable Professional Activities: An Analysis of Faculty Time, Trainee Perspectives and Actionability

Main Article Content

Victoria David
Michael Walsh
Jocelyn Lockyer
Marcy Mintz

Abstract

Abstract
The Royal College of Physicians and Surgeons of Canada introduced Competence by Design (CBD) as an educational model along with Entrustable Professional Activities (EPAs) as markers of achievement that could be directly observed on a frequent basis. In 2017, the University of Calgary Internal Medicine (IM) program piloted CBD. The purpose of this study was to (1) assess whether written feedback from EPAs were actionable, valuable, and disruptive to workflow and (2) assess the time required to complete an EPA.


Methods
Seven Foundations of Discipline EPAs were used with 31 PGY-1 Calgary IM residents. The study used quantitative and qualitative data. Following a discussion on an EPA and completion of both the quantitative and written feedback, residents were asked to comment on the value of the encounter and the degree of disruption to workflow. Assessors provided time to complete an EPA. Data were anonymized. Trainee comments were coded for value and disruption, and assessor’s written feedback was coded for actionability.


Results
One hundred and five EPA encounters were submitted. The majority of the comments provided to trainees were not actionable (94.3%, n = 99/105). While most residents did not comment on value (73.3%, n = 77/105) or disruption (44.8%, n = 47/105) of the encounter, those that did generally found the encounters valuable (25.7%, n = 27/105) and nondisruptive (35.2%, n = 37/105). A minority found the process nonvaluable (1%, n = 1/105) and disruptive (20%, n = 21/105). The mean time to complete an EPA form and provide feedback was 8.6 min.



Conclusion
Most written feedback was not actionable, suggesting a potential role for faculty development to guide assessors and help them coach trainees on EPAs.


 


RÉSUMÉ
Le Collège royal des médecins et chirurgiens du Canada a introduit la compétence par conception (CPC) comme modèle d’enseignement, ainsi que les activités professionnelles confiables (APC) comme marqueurs de réussite qui pourraient être observés directement et souvent. En 2017, le programme de médecine interne (MI) de l’Université de Calgary a testé la CPC. L’objectif de cette étude était : 1) d’évaluer si la rétroaction écrite des APC était exploitable, utile et perturbatrice pour le déroulement du travail; 2) d’évaluer le temps nécessaire pour mener à bien une APC.


Méthodes
Sept APC d’acquisition des fondements de la discipline ont été utilisées chez 31 résidents de première année en MI de l’Université de Calgary. L’étude a utilisé des données quantitatives et qualitatives. Après avoir discuté d’une APC et terminé la rétroaction quantitative et écrite, les résidents ont été invités à faire des commentaires sur l’utilité de la rencontre et le degré de perturbation du déroulement du travail. Les évaluateurs ont accordé du temps pour compléter une APC. Les données ont été anonymisées. Les commentaires des stagiaires ont été codés en fonction de l’utilité et du degré de perturbation, et la rétroaction écrite des évaluateurs a été codée en fonction de l’exploitabilité.


Résultats
Au total, 105 rencontres d’APC ont été soumises. La majorité des commentaires fournis aux stagiaires n’étaient pas exploitables (94,3 %, n = 99/105). Bien que la plupart des résidents n’aient pas fait de commentaires sur l’utilité (73,3 %, n = 77/105) ou la perturbation (44,8 %, n = 47/105) de la rencontre, ceux qui l’ont fait ont généralement trouvé les rencontres très utiles (25,7 %, n = 27/105) et non perturbatrices (35,2 %, n = 37/105). Une minorité a trouvé le processus inutile (1 %, n = 1/105) et perturbateur (20 %, n = 21/105). Le temps moyen pour remplir un formulaire d’APC et fournir une rétroaction était de 8,6 minutes.


Conclusion
La plupart des rétroactions écrites n’étaient pas exploitables, ce qui suggère un rôle que pourrait jouer le perfectionnement du corps professoral afin de guider les évaluateurs et de les aider à encadrer les stagiaires qui effectuent les APC.

Abstract 248 | pdf Downloads 94 HTML Downloads 177

References

1. Carraccio C, Englander R, Van Melle E, et al. International Competency-Based Medical Education Collaborators. Advancing Ccompetency-Bbased Mmedical Eeducation: A Ccharter for Cclinician-Eeducators. Acad Med. 2016; 91(5):645-–9. doi: http://dx.doi.org/10.1097/ACM.0000000000001048.
2. Lockyer J, Carraccio C, Chan MK, et al. ICBME Collaborators. Core principles of assessment in competency-based medical education. Med Teach. 2017; 39(6): 609-–616. doi:http://dx.doi.org/10.1080/0142159X.2017.1315082.
3. Harris P, Bhanji F, Topps M, et al. ICBME Collaborators. Evolving concepts of assessment in a competency-based world. Med Teach. 2017 Jun; 39(6): 603-–608. doi: http://dx.doi.org/10.1080/0142159X.2017.1315071.
4. Royal College of Physicians and Surgeons of Canada., Competence by Ddesign [Internet]. Ottawa, Canada. [cited 2020 Mar 21]. Available from: http://www.royalcollege.ca/rcsite/cbd/competence-by-design-cbd-e. Accessed: March 21, 2020
5. Royal College of Physicians and Surgeons of Canada., Understanding Wworkplace-Bbased Aassessment and CBD [Internet]. Ottawa, Canada. [cited 2020 Mar 21]. Available from: http://www.royalcollege.ca/rcsite/cbd/assessment/cbd-work-based-assessment-wbas-e. Accessed: March 21, 2020
6. Kwan BYM, Mbanwi A, Cofie N, et al. Creating a Ccompetency-Bbased Mmedical Eeducation Ccurriculum for Canadian Diagnostic Radiology Residency (Queen's Queen’s Fundamental Innovations in Residency Education)-Part 1: Transition to Ddiscipline and Ffoundation of Ddiscipline Sstages. Can Assoc Radiol J. 2020 Mar 4: 846537119894723. doi: http://dx.doi.org/10.1177/0846537119894723.
7. Stahl CC, Collins E, Jung SA, et al. Implementation of Eentrustable Pprofessional Aactivities into a Ggeneral Ssurgery Rresidency. J Surg Educ. 2020;77(4):739–48 Feb 8. pii: S1931-7204(20)30012-X. doi: http://dx.doi.org/10.1016/j.jsurg.2020.01.012. [Epub ahead of print] PubMed PMID: 32044326.
8. Martin L, Sibbald M, Brandt Vegas D, et al. The impact of entrustment assessments on feedback and learning: Trainee perspectives. Med Educ. 202019 Dec 15;54(4):328–36. doi: http://dx.doi.org/10.1111/medu.14047. [Epub ahead of print] PubMed PMID: 31840289.
9. Hatala R, Ginsburg S, Hauer KE, et al. Entrustment Rratings in Iinternal Mmedicine Ttraining: Capturing Mmeaningful Ssupervision Ddecisions or Jjust Aanother Rrating? J Gen Intern Med. 2019 May; 34(5): 740-–743. doi: http://dx.doi.org/10.1007/s11606-019-04878-y. PubMed PMID: 30993616; PubMed Central PMCID: PMC6502893.
10. Crawford L, Cofie N, McEwen L, et al. Perceptions and barriers to competency-based education in Canadian postgraduate medical education. J Eval Clin Pract. 2020;26(4):1124–31 Feb 27. doi: http://dx.doi.org/10.1111/jep.13371. [Epub ahead of print] PubMed PMID: 32106354.
11. Branfield Day L, Miles A, Ginsburg S, et al. Resident Pperceptions of Aassessment and Ffeedback in Ccompetency-Bbased Mmedical Eeducation: A Focus Group Study of One Internal Medicine Residency Program. Acad Med. 2020;95(11):1712–17 Mar. doi: http://dx.doi.org/10.1097/ACM.0000000000003315. [Epub ahead of print] PubMed PMID: 32195692.
12. Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency-based medical education: aAre we addressing the concerns and challenges? Med Educ. 2015; 49(11): 1086-–102. doi: http://dx.doi.org/10.1111/medu.12831.
13. Tomiak A, Braund H, Egan R, et al. Exploring Hhow the Nnew Eentrustable Pprofessional Aactivity Aassessment Ttools Aaffect the Qquality of Ffeedback Ggiven to Mmedical Ooncology Rresidents. J Cancer Educ. 2020 Feb;35(1):165-–177. doi: http://dx.doi.org/10.1007/s13187-018-1456-z.
14. Bing-You R, Hayes V, Varaklis K, et al. Feedback for learners in medical education: What is known? A scoping review. Academic Medicine. 2017 Sept; 92(9): 1346-–1354. doi: http://dx.doi.org/10.1097/ACM.0000000000001578. PubMed PMID: 28177958.
15. Nash RA, Winstone NE, Gregory SEA, et al. (2018). A memory advantage for past-oriented over future-oriented performance feedback. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2018 Mar; 44(12): 1864-–1879. http://dx.doi.org/10.1037/xlm0000549.
16. Roze des Ordons A, Cheng A, Gaudet J, et al. Adapting Ffeedback to Iindividual Rresidents: An Examination of Ppreceptor Cchallenges and Aapproaches. J Grad Med Educ. 2018 Apr; 10(2): 168-–175. doi: http://dx.doi.org/10.4300/JGME-D-17-00590.1. PubMed PMID: 29686756; PubMed Central PMCID: PMC5901796.
17. Ginsburg S, van der Vleuten CP, Eva KW, et al., Cracking the code: rResidents’' interpretations of written assessment comments. Med Educ. 2017 Apr;51(4):401-–410. doi: http://dx.doi.org/10.1111/medu.13158. Epub 2017 Jan 16.
18. Lefroy J, Watling C, Teunissen PW, et al. Guidelines: tThe do’s, don’ts and don’t knows of feedback for clinical education., Perspect Medical Education. 2015; 4:284–299. doi http://dx.doi.org/10.1007/s40037-015-0231-7.
19. Ajjawi R, and Regehr G., When I say… feedback., Medical Education. 2019;53(7):652–654, 21 October 2018,. http://dx.doi.org/10.1111/medu.13746.
20. Sargeant J, Lockyer JM, Mann K, et al. The R2C2 Mmodel in Rresidency Eeducation: How Ddoes Iit Ffoster Ccoaching and Ppromote Ffeedback Uuse? Acad Med. 2018 Jul; 93(7):1055-–1063. doi: http://dx.doi.org/10.1097/ACM.0000000000002131.
21. Lockyer J, Armson H, Könings KD, et al. In-the-Mmoment Ffeedback and Ccoaching: Improving R2C2 for a Nnew Ccontext. J Grad Med Educ. 2020 Feb;12(1):27-–35. doi:http://dx.doi.org/10.4300/JGME-D-1900508.1. PubMed PMID: 32089791; PubMed Central PMCID: PMC7012514.
22. Armson H, Lockyer JM, Zetkulic M, et al. Identifying coaching skills to improve feedback use in postgraduate medical education., Medical Education. 2019 May; 53(5):477-–493. doi: http://dx.doi.org/10.1111/medu.13818.
23. Crawford L, Cofie N, McEwen L, et al. Perceptions and barriers to competency-based education in Canadian postgraduate medical education. J Eval Clin Pract. 2020 Feb 27. doi: 10.1111/jep.13371. [Epub ahead of print] PubMed PMID: 32106354.
2423. Nousiainen MT, Caverzagie KJ, Ferguson PC, et al.; ICBME Collaborators. Implementing competency-based medical education: What changes in curricular structure and processes are needed? Med Teach. 2017 Jun; 39(6): 594-–598. doi: http://dx.doi.org/10.1080/0142159X.2017.1315077. PubMed PMID: 28598748.