Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Specific entrustable professional activities for undergraduate medical internships: a method compatible with the academic curriculum

BMC Medical Education volume 17, Article number: 143 (2017) Cite this article

Abstract

Background

Competency-based education has been considered the most important pedagogical trend in Medicine in the last two decades. In clinical contexts, competencies are implemented through Entrustable Professional Activities (EPAs) which are observable and measurable. The aim of this paper is to describe the methodology used in the design of educational tools to assess students´ competencies in clinical practice during their undergraduate internship (UI). In this paper, we present the construction of specific APROCs (Actividades Profesionales Confiables) in Surgery (S), Gynecology and Obstetrics (GO) and Family Medicine (FM) rotations with three levels of performance.

Methods

The study considered a mixed method exploratory type design, a qualitative phase followed by a quantitative validation exercise. In the first stage data was obtained from three rotations (FM, GO and S) through focus groups about real and expected activities of medical interns. Triangulation with other sources was made to construct benchmarks. In the second stage, narrative descriptions with the three levels were validated by professors who teach the different subjects using the Delphi technique.

Results

The results may be described both curricular and methodological wise. From the curricular point of view, APROCs were identified in three UI rotations within clinical contexts in Mexico City, benchmarks were developed by levels and validated by experts’ consensus. In regard to methodological issues, this research contributed to the development of a strategy, following six steps, to build APROCs using mixed methods.

Conclusions

Developing benchmarks provides a regular and standardized language that helps to evaluate student’s performance and define educational strategies efficiently and accurately. The university academic program was aligned with APROCs in clinical contexts to assure the acquisition of competencies by students.

Peer Review reports

Background

During the last decade, there has been an important development of the so called Entrustable Professional Activities (EPAs) to develop the medical competencies of students in clinical contexts. In his initial definition of EPAs Ten Cate describes them as “professional practice units defined as unsupervised tasks or responsibilities entrusted to students during their work after having acquired the competency level necessary to carry them out.” [1]. The intention is to connect the general doctor’s competencies (medical knowledge, clinical abilities and professional attitudes) with didactic activities and assessment during their clinical practices [2]. In the field of Health Sciences the Competency-based Education (CBE) movement has evolved internationally since the 90’s [3] from the mere enunciation of abstract principles to the detailed description of activities linked to compatible forms of assessment and feedback that complete the pedagogical circle of professional training.

The idea of EPAs has been welcome extensively in different medicine schools and departments throughout the world, and their theoretical, methodological, pedagogical and technical development has led to different educational experiences that have been documented in a variety of articles published in medical education literature. [1, 2, 4,5,6,7,8,9,10].

Since 2012, an interdisciplinary group of experts in medical education (MEDAPROC) at the Medicine School (MS) of the Universidad Nacional Autónoma de México (UNAM) has been working on operationalizing the CBE. In recent research [11] it was found that both teachers and students had difficulties to understand, implement and assess competencies in day to day educational activities.

Thus, the group went on to study a number of competency-based educational proposals such as the Bologna Pact [12], UNESCO [13] and OCDE [14] documents, as well as projects focused on medicine like the one made by the ACGME (Accreditation Council for Graduate Medical Education [15], the CanMeds (Canadian Medical Education Directives for Specialist, the Tuning project for Latin America [16] and others whose goal was to achieve learning results translated into competencies.

The following step was translating the abstract ideas of the competencies into concrete current academic programs. The group chose the Gynecology and Obstetrics 4th year course of the 2010 syllabus due both to its complexity (medical and surgical components) and the diversity of possible clinical scenarios [17]. They discussed the educational model (annex 1), and set its epistemological and pedagogical bases [18]. They also proposed a scheme to incorporate the courses’ content [17] and designed exercises to link practice with theory. The epistemological basis considered experiential theory and situational learning in the exercise of clinical reasoning that goes from inductive to deductive [19]: this supports active teaching strategies and the role of the educator as a guide for deliberate and reflexive practice from students [20].

In this model, the EPAs were translated as Actividades Profesionales Confiables (APROCs) term that, besides the translation, offers a particular way to operate the pedagogical model in clinical contexts in the Mexican National Health System.

In June 2014, when the AAMC (Association of American Medical Colleges) [21] published a document specifying 13 entrustable professional activities a graduate from medicine school should have in order to enter a residency program, the group decided to abide by this model to take advantage of it and align itself with the international CBE trends. Later that year the MEDAPROC group, after the experience of having designed and tested the model, allied with the medical internship area of the Clinical Teaching and Medical Internship Department (CTMID) of the UNAM MS to incorporate its pedagogical proposal into the undergraduate internship (UI).

The UI takes place during the tenth and eleventh semesters of the career, after having studied 2 years of theory at the MS facilities followed by two and a half years in clinical settings where they study several subjects. Interns go through six medical rotations spending 2 months in each: Surgery (S), Gynecology and Obstetrics (GO), Family Medicine (FM), Internal Medicine (IM), Pediatrics (P) and Emergency Medicine (ER). Unlike the previous semesters when learning was based on study, observation and performance of basic clinical procedures, during the UI the focus is on clinical practice, i.e., performing the acquired medical competencies under supervision. Thus, the main activities of their learning process happen within situational and reflexive learning.

The aim of this paper is to describe the methodology used in designing educational tools to assess competencies in the students’ clinical practice during their UI. Particularly the construction of specific APROCs in the S, GO and FM areas with three performance levels to be mastered by the end of the internship.

Method

The study’s design used an exploratory mixed method [22], i.e., a qualitative first phase followed by a quantitative study. In the qualitative stage, information was obtained on both the expected and actual activities of interns in three rotations (FM, GO and S) through focus groups (FG). Benchmarks were developed from this data and its triangulation with other sources. These were later validated by professors using the Delphi technique. Once the APROCs were translated into reliable scales for each rotation, guidelines were created so that they could be used in the formative assessment of the interns’ clinical practice.

This study is part of the investigation protocol “A new model for medical education: education modules for the development of clinical competencies (MEDEC)” approved on April 2, 2013 and registered under number 033-2013 in the Ethics and Investigation Committee of the UNAM MS. Both professors and students were asked to give their informed consent in audiotape for the group interviews and in writing for the Delphi technique. Their identities were kept anonymous.

Qualitative study

The qualitative study was divided into two steps; in the first one, FGs were completed to obtain the information about the activities that interns carry out in clinical contexts based on both theirs and their professors’ experience. In the second step the benchmarks were developed by triangulating various sources to compose performance level scales to evaluate the interns’ professional competencies in specific rotations.

Context

The field work was done by an interdisciplinary team of doctors, sociologists, students of the medical education master’s degree and social service interns in 2015 in the UNAM MS. That year, the Undergraduate Medical Internship Department (UMID) worked together with 52 clinics/hospitals from 5 different institutions both public and private around Mexico where 957 students were distributed into 48 groups in total.

From the 276 professors that collaborate with the UMID; those who participated in the FG were convened by the CTMID through an official written notice explaining the objective of such groups, i.e., to identify the contextual and pedagogical characteristics of the activities carried out by interns in their day to day clinical practice. In the case of students, social service interns who had completed their UI the year before and were assigned to the MS were convened.

Interview guidelines were developed to identify the compatibility between the academic (designed at the university) and operational program (designed at the clinic) with the activities an intern does in the FM, GO and S rotations. The guidelines were divided into three: 1) General questions regarding activities in each clinical space: What activities does an intern do in outpatient consultations? Which activities does he do on his own? Which require supervision? And from these, which must he be able to do on his own by the end of his outpatient services rotation? 2) Core topics of the academic curriculum. 3) Specific activities done by the intern during the rotation. This scheme was adjusted depending on whether it was aimed at teachers or students and according to the clinical areas each student went through in each rotation. The thematic contents also changed depending on each subject. Table 1 shows the interview guide for GO.

Table 1 Interview Guide
Full size table

Both the voluntary informed consent and the interviews were recorded and transcribed. Based on Grounded Theory [23], a category tree was made to code and categorize transcriptions according to the activities performed by interns in specific rotations within clinical services.

Once transcriptions were coded, testimonies were ordered by category into two columns, one with the teacher’s comments and one with the student’s (annex 2). The analysis of the similarities and differences in the experiences, plus the diversity of stories in different locations and institutions allowed to create an accurate image of what was happening in the clinics and hospitals of the National Mexican Health System where the 10° and 11° semester UNAM MS students study.

Second phase qualitative study

The objective of the second stage was to build scales with narrative descriptions of the level of performance achieved by the interns in the typical activities of the UI in each rotation (benchmarks). The idea was to use the testimonies of the activities at the clinics gathered in the FG and write up improved narrative descriptions for the benchmarks used for assessment. As Hanson et al. explain, [4] to move from words to figures when evaluating allows to reconstruct the feedback process signaling both strengths and weaknesses.

When formulating the narrative descriptions, different sources of information were considered in order to formulate or inform the narrative construction [24] in such a way that they would reflect the interns’ level of performance. The considered documents were:

  1. (1)

    2015 Internship Academic Programs (FM, GO and S) [25]

  2. (2)

    Operative Programs from some of the venues of the internship

  3. (3)

    ACGME Milestone Project from the FM, GO, and S residencies [15, 26, 27]

  4. (4)

    Document on the 13 EPA’s for the general physician from the AAMC titled “Core entrustable professional activities for entering residency AAMC: Developers Guide” [21]

  5. (5)

    Competencies of the 13 EPA’s outlined in the aforementioned document [21]

  6. (6)

    Testimonies by categories from the FG

  7. (7)

    Dreyfus and Dreyfus’s “Novice to expert” Model in the first three levels [28]

  8. (8)

    Table two from Ten Cate’s article “Nuts and Bolts of Entrustable Professional Activities” [1]

  9. (9)

    Miller’s Pyramid [29]

  10. (10)

    Bloom’s Taxonomy [30]

The procedure to develop the benchmarks started from the testimonies by category. Activities common to all rotations were eliminated to work only with the ones that were specific to each rotation. Once identified, they were both checked against the contents of each course’s 2015 academic program and revised considering the operative programs from some UI clinics. Keeping in mind the local scenario, we moved on to revise the AAMCs competencies per domain and their relationship to the APROCs.

While writing up the benchmarks for the now-called APROCs we considered three levels of performance according to Dreyfus and Dreyfus’s scale (novice, beginner and competent) and Miller’s Pyramid (know, know-how, and perform). Moreover, the recommendations in Ten Cate’s article describing EPAs were taken into consideration.

The next step in this iterative process to design the developmental benchmarks for each APROC was to use verbs from Bloom’s Taxonomy. These were chosen based on the expected performance for each level; for level 1 verbs from the knowledge and comprehension categories, for level 2 from the application category and for level 3 verbs from the analysis, synthesis and evaluation categories. There was flexibility in the use of verbs since some describe actions typical of medicine that are not found in Bloom’s taxonomy such as “diagnose,” “indicate”, “refer”, “prognosticate,” etc. [31].

The APROC’s titles refer to activities that translate into competencies and do not necessarily correspond to the topics in the programs. Once the APROCs were defined the narrative descriptors for each level were written using the present tense and avoiding verbs with negative connotation (lack, ignore, not know, overlook).

The APROCs for each rotation (FM, GO and S) were developed in subgroups and later revised and redesigned in plenary sessions until the final version was obtained. Twelve APROCs were created for GO, 14 for FM and 8 for S. They all reflect the expected learning outcomes in the specific internship rotations consistent with the graduate profile for the general physician as established in the 2010 Academic Curriculum. Table 2 shows an example of an APROC in each area.

Table 2 Examples of APROCs in each area
Full size table

Quantitative study

To validate the final benchmarks for the three rotations involved the subgroups used the Delphi technique to reach the experts’ consensus, expressed as 80% or more of the desired level. Teachers from sites that had not participated in the FG received a personalized invitation to collaborate through the online surveys in the Limesurvey® platform. The goal was to reach agreements regarding the level of development, 1 = novice, 2 = beginner, 3 = competent, that interns had to achieve in the APROCs of each rotation. These three rounds of surveys with 13 GO specialists, 11 family doctors and 12 S professors took place in November and December 2015.

First round

Professors were contacted through email and phone calls in November and December 2015. A link was provided to access the survey and once in it there was an example showing how to answer it.

Based on the comments and observations of the first Delphi round some benchmarks were rewritten.

Second round

The comments of the first round were included in the second survey so that participants were aware of them. It was detected that professors were putting the actual level interns were achieving and not the level they were expected to achieve at the end of the rotation. Therefore, instructions were rephrased to read “Choose the level the intern must reach at the end of the 8-week rotation. Keep in mind levels are both progressive and inclusive; level 2 includes level 1; level 3 includes both levels 2 and 1.” At the end a space for general comments was added since some professors wrote down general observations in the space provided for specific comments about each activity. To identify each teacher’s answers (day and time) and to send personalized reminders to those who had not replied passwords were created to limit access to the survey.

Third round

The third round consisted of a personalized survey for each professor showing the levels chosen both by him and others in the previous rounds as well as the percentages and comments received. In the end, data was systematized and analyzed. Whenever one of the APROC’s levels reached at least 80%, consensus was given and hence, the APROC validated. Otherwise the APROC was discarded [32].

Results

The results of the investigation can be described in two ways, curricular and methodological. From the curricular point of view, APROCs were identified in three UI rotations within clinical contexts in Mexico City and its surroundings; benchmarks were developed and validated by level. Table 3 shows the APROCs for FM, GO and S that resulted from this exercise.

Table 3 APP MEDAPROC in each rotation
Full size table

With the level-specific APROCs, guides were elaborated for both professors and students so that they could become familiar with the method and to define the results of the expected learning at the end of the rotation. Among other things, the guides explain how to assess the students’ performance during the course based on the narrative descriptions of each level. The “APP MEDAPROC” (Table 4) was designed to find and register the level reached by each student in his clinical practice; it is a useful tool for students, professors and education institutions for both the feedback and formative processes.

Table 4 “APP MEDAPROC” for Family Medicine
Full size table

From the methodological point of view this research fostered the development of a model to build APROCs from six mixed methods, summarized as follows:

  1. (1)

    Study of international literature about competency- based education and EPAs.

  2. (2)

    Analyze the MEDAPROC proposal and its compatibility with the specific curricular programs.

  3. (3)

    Make FG with professors and students to identify the real and the expected activities in the courses of interest.

  4. (4)

    Order and organize the information to obtain a document with testimonies for each category distinguishing between professors and students to identify the APROCs.

  5. (5)

    “Triangulate” the information obtained with other sources to make narrative descriptions for three levels (benchmarks) in each APROC; revise them individually and collectively to get a final version.

  6. (6)

    Validate the benchmark scales for the APROCs through the Delphi technique to achieve both experts’ consensus and reliability of the instruments for assessment.

Discussion

The original objective, to describe the methodology used in the design of specific APROCs as educational tools to assess competencies in the clinical practice of interns based on the MEDAPROC, was achieved. Moreover, benchmarks were developed for three rotations of the UI. The results went beyond explaining EPAs with definitions and description of activities, the APROCs were made compatible with the academic and operational UI programs (CTMID, MS, UNAM) and were adapted to the clinical contexts of the Mexican National Healthcare System.

A similar effort to link CBE to clinical scenarios was made in the Netherlands [2] where a method was developed to achieve such relationship by developing pedagogical designs. The difference is that, in our study, the exercise was made from the university to be implemented in multiple clinics and hospitals; and, in other studies described in literature, the development of EPAs was made in specific clinical environments. The question that arises is whether the UI’s APROCs work the same in different sites or whether each clinic or hospital needs specific criteria to apply the APROCs.

There are published works that describe the steps followed for the construction of EPAs, the most influential being that of Ten Cate [1]. Aylward et al. [7] also contribute to this topic in their article explaining how they planned and implemented an EPA related to the shifts of Pediatric residents. In Aylward’s EPA, as in this study, three-level scales were developed to measure the reliability achieved in the competencies. Recently in Australia, Kwan et al. [5], have described five steps to build EPAs and presented the development of two EPAs for the Emergency Room service, also constructed with three-level scales. As in our study, they used FG, but they did not validate the benchmarks through the Delphi technique as was done in this and other cases [33,34,35].

As experiences in the construction of EPAs are registered, the concern to ensure their quality has aroused and validation strategies have been put forward to ensure the reliability of the scales. Chen et al. [10], propose a methodology that considers seven aspects that must be attended when developing an EPA. Even though many of them were implemented in the construction of our APROCs, we still need to test the benchmarks’ functionality as a supervision tool and as a resource for formative assessment of the UI. Post et al. [8] presented an instrument to review EPAs that takes into consideration seven aspects that were useful when testing the viability of EPAs in pilot studies.

When work on the benchmarks started [36] emphasis was made on the specialties of medical residencies; but it has been proven that [6] EPAs can also be implemented in the undergraduate level as in the case of the UI, with the advantage that it gives continuity to both the contents and the methodology proposed for the acquisition of medical competencies. Benchmarks provide a standardized language that helps eliminate personal and institutional interpretations about the student’s performance. The process allows professors to assess knowledge, skills and abilities in each benchmark as well as define the strategies to evaluate students efficiently and accurately [31]. The APROCs constitute an attempt to align the learning results established by the current core curriculum of the MS at UNAM with international trends.

Even though the UNAM MS is the largest medicine school in Latin America and has more than 50 clinics and hospitals in its UI, this study was carried out in only one educational organization, and the investigation should be expanded to other universities. Another limitation was time. The UI-specific APROCs, unlike those of the medical residency, must be developed in a 2-month periods. This hastens its teaching in clinical situations that do not always promote learning. Among the pending tasks, we must test the proposal to assess how professors and students adapt to the new pedagogical model to later adjust the scales, formats and processes for a more efficient implementation.

In regards to the Delphi technique we found that there is no single definition of the term “consensus” and that there is a variety of methods to implement it [37]. In this work, due to the small samples and the logistic complications of assigning numerical scales to the three levels, we did not use validity coefficients to measure the degree of consistency of the results obtained (Pearson, Kappa) nor did we use descriptive statistics measures (mean, median, variance, standard deviation, confidence intervals) in the result analysis of the Delphi technique; we merely presented the percentages of the answers that were obtained.

Conclusions

To our knowledge, this study is the first formal effort to incorporate APROCs in an UI in Mexico as well as a methodology based on the CBE pedagogical model to develop useful benchmarks for the supervision of clinical practice and formative assessment. The APP MEDAPROC for the UI will enable the planning of the teaching-learning process through previous instructional design based on experiential theory, situational learning, deliberate practice and reflection. Making the APROCs operational in accordance with the 2010 Academic Curricula of the UNAM MS constitutes a powerful educational tool in clinical environments that contributes to the improvement of the quality of medical care and patient security. Nevertheless, the benchmarks should not be used as the only instrument to assess the acquisition of knowledge, skills and attitudes in the specific activities of the UI rotations; it is necessary to combine them with other assessment strategies that show the development of the interns’ professional competencies.

Abbreviations

CBE:

Competency-based education

CTMID:

Clinical Teaching and Medical Internship Department

EPAs:

Entrustable Professional Activities (APROCs for the Mexican version)

ER:

Emergency Medicine

FG:

Focus Groups

FM:

Family Medicine

GO:

Gynecology and Obstetrics

IM:

Internal Medicine

MS:

Medicine School

P:

Pediatrics

S:

Surgery

UI:

Undergraduate Internship

UMID:

Undergraduate Medical Internship Department

UNAM:

Universidad Nacional Autónoma de México

References

  1. Ten Cate O. Competency-based education, entrustable professional activities, and the power of language. J Grad Med Educ. 2013;5:6–7.

    Article  Google Scholar 

  2. Mulder H, Ten Cate O, Daalder R, Berkvens J, Ten CO, Daalder R, Berkvens J. Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training. Med Teach. 2010;32:e453–9.

    Article  Google Scholar 

  3. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226–35.

    Article  Google Scholar 

  4. Hanson JL, Rosenberg AA, Lane JL. Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States. Front Psychol. 2013;4:1–10.

    Article  Google Scholar 

  5. Kwan J, Crampton R, Mogensen LL, Weaver R, van der Vleuten CPM, Hu WCY. Bridging the gap: a five stage approach for developing specialty-specific entrustable professional activities. BMC Med Educ. 2016;16:117.

    Article  Google Scholar 

  6. Chen HC, van den Broek WES, ten Cate O. The case for use of Entrustable professional activities in undergraduate medical education. Acad Med. 2015;90:431–6.

    Article  Google Scholar 

  7. Aylward M, Nixon J, Gladding S. An Entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014;89:1335–40.

    Article  Google Scholar 

  8. Post JA, Wittich CM, Thomas KG, Dupras DM, Halvorsen AJ, Mandrekar JN, Oxentenko AS, Beckman TJ. Rating the quality of Entrustable professional activities: content validation and associations with the clinical context. J Gen Intern Med. 2016;5:1–6.

    Google Scholar 

  9. Teherani A, Chen HC. The next steps in competency-based medical education: milestones, entrustable professional activities and observable practice activities. J Gen Intern Med. 2014;29:1090–2.

    Article  Google Scholar 

  10. Chen HC, McNamara M, Teherani A, Cate O Ten, O’Sullivan P: Developing Entrustable professional activities for entry into clerkship. Acad Med. 2015;XX(X):1. http://www.project-watchme.eu/wp-content/uploads/2016/03/chen_et_al_2015_am_early_online.pdf.

  11. Hamui-Sutton A, Vives-Varela T, Gonzalez-Quintanilla E, Morales D, Uribe G. Evaluación Cualitativa de La Educación Clínica. Plan de Estudios, vol. 2014. México: Fase 2, Quinto, Sexto Y Séptimo Semestre; 2010.

    Google Scholar 

  12. Declaración de Bolonia [http://eees.umh.es/contenidos/Documentos/DeclaracionBolonia.pdf].

  13. Morin E. Los Siete Saberes Necesarios Para La Educación Del Futuro. UNESCO. Paris: Santillana; 1999.

    Google Scholar 

  14. OCDE. El programa PISA de la OCDE Que es y para que sirve; 2007. p. 34.

    Google Scholar 

  15. Accreditation Council for Graduate Medical Education (ACGME). The general surgery milestone project. J Grad Med Educ. 2014;6(1 Suppl 1):320–8.

    Google Scholar 

  16. Beneitone P, Cesar E, Gonzalez J, Maleta M, Siufi R, Wagenaar G. Reflexiones y perspectivas de la educación superior en América Latina. Inf Final Tuning-América Lat. 2004:1–432.

  17. Hamui-Sutton A, Varela Ruiz M, Ortiz Montalvo A, Torruco Garcia U. Modelo Educativo para desarrollar actividades profesionales confiables (MEDAPROC). Rev Med Inst Mex Seguro Soc. 2015;53:616–29.

    Google Scholar 

  18. Hamui-Sutton A, Varela Ruiz M, Ortiz Montalvo A, Torruco Garcia U. Las actividades profesionales confiables: un paso más en el paradigma de las competencias en educación médica. En prensa. 2015:24–39.

  19. Torruco Garcia U, Ortiz Montalvo A, Varela Ruiz M, Hamui-Sutton A. Desarrollo de competencias en el pregrado: un modelo esducativo para desarrollar actividades profesionales confiables. Aceptado en la Gac Médica México. 2015;53:3.

    Google Scholar 

  20. Hamui-Sutton A. Guías Educativas MEDAPROC. Profesores y Alumnos. Bloque Ginecología y Obstetricia. 8° Semestre. Plan de Estudios 2010. Facultad de Medicina, UNAM. México. 2015;1:1-252.

  21. Association of American Medical Colleges. Chen HC, van den Broek WES, ten Cate O: Core Entrustable professional activities for entering residency. Acad Med. 2013;90:431–6.

    Google Scholar 

  22. Hamui-Sutton A: Un acercamiento a los métodos mixtos de investigación en educación médica. Invest. Ed. Med. 2013;2:211–16.

  23. Jones D, Manzelli H, Pecheny M. Grounded Theory. Una aplicación de la teoría fundamentada a la salud. Cinta Moebio. 2004;19:38–54.

    Google Scholar 

  24. Cisterna Cabrera F. Categorización Y Triangulación Como Procesos de Validación del Conocimiento en Investigación Cualitativa categorization and triangulation as processes of validation of knowledge in qualitative investigations. Theoria. 2005;14:61–71.

    Google Scholar 

  25. Programa académico de internado médico, cirugía [http://paginas.facmed.unam.mx/deptos/cirugia/index.php/coordinacion-de-ensenanza/manual-depracticas].

  26. Ten Cate O, Scheele F, Abreu Hernández LF, Cid Garcia ÁN, Herrera Correa G, Lara Vélez JVM, Laviada Delgadillo R, Rodriguez Arroyo C, Sánchez Aparicio JA, Accreditation Council for Graduate Medical Education (ACGME), Acuerdo político nacional, Albanese MA, Mejicano G, Mullan PB, Kokotailo P, Gruppen LD, Aylward M, Nixon J, Gladding S, Boyce P, Spratt C, Davies M, McEvoy P, Hauff SR, Hopson LR, Losman E, Perry MA, Lypson ML, Fischer JP, Santen SA, et al.: Obstetrics and gynecology milestones. Med Educ 2015, XX:1.

  27. Allen S, Anim T, Anthony E, Araujo D, Beebe D, Dostal J, Elliott T, Green LA, McGaha A, Neill R, Nestler S, Pugno P, Quan M, Roise AJ, Shaughnessy AF, Tippy P: The family medicine milestone project. 2015.

    Google Scholar 

  28. Dreyfus SE. The five-stage model of adult skill acquisition. Bull Sci Technol Soc. 2004;24:177–81.

    Article  Google Scholar 

  29. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990:S63–7.

  30. Anderson LW, Sosniak L, Bloom B, Education NS for the S of: Bloom’s taxonomy: a forty-year retrospective. 1994.

    Google Scholar 

  31. Varela-Ruiz M, Díaz-Bravo L, García-Durán R. Descripción y usos del método Delphi en investigación del área de la salud. Investig en Eduación Médica. 2012;1:90–5.

    Google Scholar 

  32. Hauer KE, Kohlwes J, Cornett P, Hollander H, Ten Cate O, Ranji SR, Soni K, Iobst W, O’Sullivan P. Identifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013;5:54–9.

    Article  Google Scholar 

  33. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J. Entrustable professional activities in family medicine. J Grad Med Educ. 2013;5:112–8.

    Article  Google Scholar 

  34. Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011;11:96.

    Article  Google Scholar 

  35. Warm EJ, Mathis BR, Held JD, Pai S, Tolentino J, Ashbrook L, Lee CK, Lee D, Wood S, Fichtenbaum CJ, Schauer D, Munyon R, Mueller C. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med. 2014;29:1177–82.

    Article  Google Scholar 

  36. Carraccio C, Englander R, Gilhooly J, Mink R, Hofkosh D, Barone MA, Holmboe ES: Building a framework of Entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med 2016, XX(X):1.

  37. Hasson F, Keeney S. Enhancing rigour in the Delphi technique research. Technol Forecast Soc Change. 2011;78:1695–704.

    Article  Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the committed work of the following students of the Master in Health Education Degree: Adrián Garduño Vera, Samuel Gutiérrez Barreto and Oliver Gonzáles Gijón. We also want to thank social service undergraduate students who actively participated with us in 2015: Abraham Ortiz García, Lizbeth Cejudo Aparicio, Jorge Pereyra Arzate, Aketzali Flores Mayorga, Michel Sandoval Torres and Nancy Quintanilla Serrano. Authors want to thank Mrs. Josefina Bolado, Head of the Scientific Paper Translation Department, from the Investigation Division at the UNAM Medicine School, for translating the Spanish version of this manuscript, to Ms. Laura Amescua for revising the translation  and Manuel Millán for the final revision of the paper.

Funding

None.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

  1. Medical Education Secretary, Medicine School, Universidad Nacional Autónoma de México, Facultad de Medicina, Edificio B, 3er piso, Zip code, 04510, Coyoacán, Mexico

    Alicia Hamui-Sutton, Armando Ortiz-Montalvo & Uri Torruco-García

  2. Medical Internship Department, Clinical Teaching Secretary, Medicine School, Universidad Nacional Autónoma de México, Mexico City, Mexico

    Ana María Monterrosas-Rojas, Felipe Flores-Morones, Andrea Navarrete-Martínez & Araceli Arrioja-Guerrero

Authors
  1. Alicia Hamui-Sutton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ana María Monterrosas-Rojas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Armando Ortiz-Montalvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Felipe Flores-Morones
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Uri Torruco-García
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andrea Navarrete-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Araceli Arrioja-Guerrero
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AHS, AMMR, AOM, UTG, ANM and AAG all contributed to the design, fieldwork and data analysis. All authors read and approved the final manuscript

Corresponding author

Correspondence to Alicia Hamui-Sutton.

Ethics declarations

Ethics approval and consent to participate

This work is registered with number 033-2013 before the Ethics and Investigation Committee of the UNAM Medicine School. Consent was obtained from participants.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Annex 2 Testimonies by category
Full size table
Annex 1
figure 1

Model from MEDAPROC

Full size image

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamui-Sutton, A., Monterrosas-Rojas, A.M., Ortiz-Montalvo, A. et al. Specific entrustable professional activities for undergraduate medical internships: a method compatible with the academic curriculum. BMC Med Educ 17, 143 (2017). https://0-doi-org.brum.beds.ac.uk/10.1186/s12909-017-0980-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12909-017-0980-6

Keywords

BMC Medical Education

ISSN: 1472-6920

Contact us