Making it stick: use of active learning strategies in continuing medical education

Abstract

Background

Despite the known benefits of active learning (AL), the predominate educational format in higher education is the lecture. The reasons for slow adaptation of AL in medical education are not well understood. The purpose of this survey was to determine knowledge, usage, attitudes, and barriers to AL use in academic Continuing Medical Education (CME).

Method

A 20-item questionnaire was developed and sent with a link to an online questionnaire to the Society of Academic Continuing Medical Education (SACME) listserv of ~ 350 professionals representing academic medical centers, teaching hospitals, and medical specialty societies in the United States (U.S.) and Canada. Responses were collected with SurveyMonkey® from October–November, 2019. Data were analyzed using SPSS®.

Results

Responses from 146 SACME members in 91 CME units yielded a ~ 42% survey response rate. Many respondents reported their self-perceived knowledge of AL as high. Advanced training (e.g., certificate, Master of Education degree) was positively correlated with AL knowledge. AL methods were reportedly used in half of the CME activities in the majority (80%) of institutions. Higher levels of self-perceived knowledge were correlated with an increased percentage of AL-related CME activities. Commonly perceived barriers to use of AL were presenters’ lack of familiarity and a need for more time-consuming preparation.

Conclusions

More efforts are needed to increase innovation and incorporate evidence-based AL strategies in medical education, especially to foster learner engagement, critical thinking, and problem-solving ability.

Introduction

Teacher-centered lectures continue to be largely utilized in higher education overall. However, leaders in medical education have challenged the lecture format because active learning (AL) strategies have been shown to promote better retention and application of new knowledge than listening to passive, information-only imparting lectures [1,2,3]. Fostering curiosity and discovery while critically evaluating and discovering viable ways to address gaps in professional practice, change practice, and improve patient outcome are necessary for life-long learning in medicine. While biomedical information grows at an exponential rate and email inboxes are flooded with learning opportunities, learners seek relevant, efficient, and meaningful educational forums.

Active learning describes a broad range of learning activities (e.g., flipped classroom, think-pair-share, turn and talk, bulleted breaks during lecture) that require a learner to “construct”, understand, and comprehend the knowledge derived from their educational experience “while simultaneously improving knowledge gain and recall abilities” [4]. Although AL strategies were introduced nearly 40 years ago, reports of use in medical education are limited, but the benefits of AL have been well described. Increased content knowledge, critical thinking, problem-solving abilities, and positive attitudes towards learning have been noted when inquiry-based cooperative learning was compared to traditional lecture-based learning formats [5]. In addition, development of graduate capabilities such as critical and creative thinking, problem-solving, adaptability, communication and interpersonal skills were observed when AL strategies were used in graduate education [6].

Freeman et al. concluded, “If the experiments analyzed here had been conducted as randomized controlled trials of medical interventions, they may have been stopped for benefit—meaning that enrolling patients in the control condition might be discontinued because the treatment being tested was clearly more beneficial” [7].

Although AL is effective, there are opportunities to learn more about knowledge, usage, attitudes and barriers to use of AL methods. By improving content delivery, knowledge retention and self-directed learning are encouraged.

A knowledge gap about use of AL learning strategies extends across all areas of medical education, including continuing medical education (CME) [8]. Our research efforts emerged from the results of a 2018 survey of academic CME unit leaders in the U.S. and Canada suggesting that the lecture is the most common educational delivery method in CME [9]. An earlier report in the CME literature highlighted the importance of using learner-centric AL formats to engage the learner with the teacher and audience to increase self-awareness and critical thinking [3]. As in other learner groups, use of AL methods in CME has been shown to reinforce content and promote changes in physician practice with improvement in patient outcome [10]. Given the importance of these methods in changing practice, there is a substantial gap in the literature about the reasons surrounding the limited use of these newer learning strategies in CME. By understanding more about knowledge, usage, attitudes and barriers to use of AL methods, the results of this research will provide CME programs with an opportunity to learn and adapt so that CME units can improve the learning environment, using their unique position within medical education to influence physicians’ practice. By creating greater awareness of AL methods across the educational continuum, there is an opportunity to engage learners by making lessons “stickier” (more comprehensible and memorable) and utilizing “lecture halls without lecture” [1]. CME is just one area on this continuum of medical education, however similar questions and survey tools can be used to extend our knowledge and awareness of the benefits of AL in other areas of medical education.

Method

Potential survey respondents were leaders of CME units and members of Society of Academic Continuing Medical Education (SACME) at 144 U.S., 17 Canadian, and other CME medical education units (e.g., American College of Surgeons, American Academy of Pediatrics). SACME is the academic society for continuing medical and interprofessional education that advances continuing education through scholarship by promoting and advancing high-value healthcare. Respondents were identified through the SACME Communications Committee listserv.

While the 2018 Harrison Survey provided the AL construct for the study and reported that the lecture format is the most commonly used learning format in CME, the survey results provide a widesweeping “snapshot of the current structure and function of CME/CPD units at medical schools in the United States and Canada.” [9] and did not specifically address AL strategies. Consequently, the literature was searched extensively for publications about AL, but there have been no surveys published that represent the construct in its entirety. Questions were subsequently developed for this de novo survey based on 6 content areas that were identified through another comprehensive review of existing literature about AL use in medical education and higher education in general: 1) demographics; 2) knowledge; 3) usage; 4) attitudes about; 5) barriers to use; and 6) potential strategies to address those barriers [4, 7, 9, 11]. AMEE Guide No. 87 [12], a guideline publication outlining best practices for survey reporting [13], and an expert in survey design and administration from the University of Colorado School of Public Health influenced item development. Subsequently, 20 draft items consisting of multiple choice, matrix, yes/no, and free-text questions were pretested by content experts (including 15 members of the SACME communications and scholarship committees) as well as the expert in survey design. After this initial round of survey item revision, cognitive interviews were conducted with 7 randomly selected SACME members. This was followed by a final round of survey item revision, instrument formatting, and refinement. Altogether, pretesting, cognitive interviews, and expert reviews followed by further revision helped to establish content and response process validity.

The Active Learning Survey in CME was conducted under an IRB-approved protocol (#19–1274) through the University of Colorado School of Medicine. An introductory email announcing the upcoming survey was distributed to ~ 350 SACME members by the Content Manager of SACME. Survey invitations were then sent by this individual directing individual members to a specific SurveyMonkey® link (Active Learning Survey in CME). This email explained the purpose and objectives of the survey. They were informed that they were voluntarily consenting to participate in a research study when completing the survey. No compensation or incentives were provided. The 20-question survey took approximately 10 min to complete. Four email reminders were sent to non-respondents during the 6-week study period in October–November, 2019.

Demographic data were collected. Respondents were asked about their perceived level of knowledge about AL methods with a four-point Likert scale (very knowledgeable, mostly knowledgeable, somewhat knowledgeable, not at all knowledgeable). Respondents were also asked what type of, if any training they had received in AL methods (e.g., certificate or Master of Education). Their beliefs and the extent of agreement or disagreement (strongly agree, agree, neither agree or disagree, disagree, or strongly disagree) with attitudinal statements about AL were also assessed in the survey. CME unit leaders were asked about the use of AL compared to lecture in CME activities at their institution. AL strategies were defined for participants in a linked online Teaching Practice Terminology Tip Sheet. If respondents reported that AL strategies were used in their unit, they were prompted to identify which type of strategy was used. Survey respondents were also asked to indicate the degree to which various factors made it more difficult to use AL in their CME unit. This was followed by a question about how helpful various methods would be in facilitating the use of AL strategies in their unit. Respondents were also invited to complete open-ended questions about best practices for use of AL at their institution. A link to the Association of American Medical Colleges (AAMC) Faculty Roster 2018 was provided for respondents to identify the size of faculty at their institution.

Non-responders were identified by review of medical school IP addresses not represented in the data set. Ten non-responders were contacted by phone to participate in a brief scripted follow-up telephone survey focused on a subset of key survey questions, including demographics, use of AL in their CME unit, and barriers to usage. This telephone survey was conducted by the principal investigator to identify potential differences and extent of non-response bias between responders and non-responders. Sample size for this survey was limited by the fixed number of academic CME units in the U.S. and Canada.

Data from de-identified survey responses were collected electronically with SurveyMonkey® and analyzed using IBM® SPSS® Statistics 26.0 (publication date: April 9, 2019; https://www.ibm.com/analytics/spss-statistics-software) software platform. The primary outcomes analyzed were the overall knowledge, attitudes about, use, and barriers to use of AL in CME. All demographic data were summarized using descriptive statistics (means, standard deviations, frequencies, percentages). Spearman’s correlations were used to assess statistical associations between: 1) advanced training in AL (e.g., certificate, Master, or Doctor of Education degree) and self-perceived knowledge of AL; 2) self-perceived knowledge of AL and percentage of CME activities in their institution using AL; and 3) advanced training in AL (e.g., Certificate, Master’s, or Doctor of Education degree) and percentage of CME activities using AL in their institution. Internal consistency and reliability of the survey scores were assessed with Cronbach’s or Krippendorf’s alpha coefficients, which were calculated for each subconstruct measured.

Results

We were able to collect 146 responses. This represents ~ 42% of SACME unit leaders from 93 U.S. and Canadian medical schools as well as other medical specialty societies (e.g., American College of Surgeons, American Academy of Pediatrics). Since not all respondents answered all questions due to skip-logic survey methodology, not all data sets are complete. The number of full-time faculty in medical schools represented ranged from 150 to 8000 with a mean of 1406 full-time faculty members per responding institution. Seventy percent of the responding CME unit leaders were based in a medical school. More than 50% were CME Unit Directors and Assistant/Associate Deans of CME. Just over half of participating unit leaders had been in their positions for more than 6 years. Other demographics are described in Table 1.

Table 1 Respondent and Continuing Medical Education Unit Demographics for Continuing Medical Education Unit Leaders in the U.S. and Canada, 2019

Sixty percent of the survey participants described themselves as very or mostly knowledgeable about AL (see Fig. 1). Approximately 80% of respondents had participated in national continuing education programs (e.g., annual conferences) that included information about AL methods. In addition, ~ 50% had participated in a continuing education program at their institution and/or had engaged in self-study that included information about AL. Fewer respondents had participated in on-line courses. More than one-third of the survey respondents had completed a certificate, Master, or Doctor of Education degree that included information about AL methods. Spearman’s correlation revealed that advanced training was positively correlated with self-perceived knowledge of AL (Spearman’s correlation = 0.23 at P = .01).

Fig. 1

Respondents’ Self-perceived Knowledge about Active Learning in Continuing Medical Education Units in the U.S. and Canada, 2019. Data are presented as % (n) of respondents. Missing n = 5. U.S. = United States

More than 80% of respondents reported that AL methods were used in less than half of CME activities at their institutions. Simulation, audience response polling, case-based teaching, small group activities, and panel discussions were used most commonly (> 90%). Active learning strategies used in CME units, as reported by CME unit leaders are outlined in Table 2. Respondents also noted other examples of AL methods including gaming, group reflection with coaching between sessions, and role playing. While advanced training was not significantly correlated with an increased percentage of CME activities using AL (Spearman’s correlation = 0.14 at P = .06), higher levels of self-perceived knowledge were correlated with an increased percentage of CME activities using AL in respondents’ institutions (Spearman’s correlation = 0.16 at P = .04).

Table 2 Active Learning Strategies Currently Used in 2019 by Continuing Medical Education Units in the U.S. and Canada Using Descriptive Statistics and Krippendorff’s alpha from Survey Questions

More than 90% of respondents agreed or strongly agreed that AL combines engagement with reflection. They also agreed that AL requires cooperation of the instructor to engage learners by working together to increase knowledge and recall (see Table 3).

Table 3 Attitudes and Perceptions about Use of Active Learning in Continuing Medical Education Unit Leaders in the U.S. and Canada During 2019 Using Descriptive Statistics and Cronbach’s alpha from Survey Questions

Most commonly perceived barriers to use of AL are lack of familiarity of faculty/presenters with AL methods in addition to increased preparation and planning for AL. Other barriers are described in Table 4. Respondents suggested that guidelines for use of AL strategies, faculty development programs to include training in AL, and coaching course directors to discuss AL methods, are examples of how CME unit leaders could increase AL content in CME activities. Other resources that were noted by respondents in assisting them in developing AL strategies in their institutions are presented in Table 5.

Table 4 Perceived Barriers for Using Active Learning in Continuing Medical Education Units in the U.S. and Canada During 2019 Using Descriptive Statistics and Cronbach’s alpha from Survey Questions

Table 5 Suggested Resources for Increasing Use of Active Learning Methods in Continuing Medical Education Units in the U.S. and Canada in 2019 Using Descriptive Statistics and Cronbach’s Alpha from Survey Questions

Score validity and reliability of the survey instrument were assessed by applying Messick’s five sources of validity evidence [14]. While the new instrument was being developed, we collected evidence for content validity from an exhaustive literature review and consultation with content experts for question review and revision. Response process validity was also examined by testing the questions by experts and reviewers before distributing the survey. Since construct validity reflects the extent to which a measure ‘behaves’ the way that the underlying construct should behave in relation to other constructs, Spearman’s correlations were used to establish this type of validity. Score reliability of the survey instrument is reported in Tables 2, 3, 4 and 5.

When addressing potential non-response bias with telephone interviews, we found results similar to the online survey. Most of the 10 non-respondents that were questioned by phone were CME Unit Directors or Program Managers and 45–64 years of age. Less than 50% of these telephone interviewees used AL in CME activities at their institutions. Barriers included a lack of familiarity with AL methods, more time-consuming planning and preparation, and a lecture-based culture at their institutions.

Discussion

Increasing awareness of AL and its lack of use as well as extending the use of this AL survey instrument and methodology to other areas of medical education were important goals for publication of this study. However, the primary goal of this survey was to extend the results of the 2018 Harrison survey and address unanswered questions about the knowledge, usage, attitudes, and barriers to usage of AL strategies in academic CME units in U.S. and Canada. The results suggest that many CME unit leaders are well-trained and have education in AL as well as positive attitudes towards the use of AL methods, but the change from the lecture format to implementation of AL methods has been minimal. While the mission of the Accreditation Council for Continuing Medical Education (ACCME) is to “assume and advance quality learning for healthcare professionals that drives improvements in patient care” [15], the translation of knowledge into practice that improves patient outcome is complex [16]. Although much has been written about the quality of health care delivered in the U.S. [17, 18], the challenge for CME is to deliver impactful medical knowledge in a way that providers are able to retain and engage with the delivered knowledge in order to stay current and improve patient care.

In general, faculty members have not kept pace with pedagogical advancements and knowledge of medical education delivery methods [2]. Little is known about the actual training and knowledge of CME unit leaders in AL methods. In our study, we found that many of the CME unit leaders have both formal and informal training that included information about AL. More than half of the respondents consider themselves knowledgeable about AL methods. Learners in these institutions may have an edge since engagement and translation of the knowledge facilitates changes in practice and improved patient outcomes [10].

While the practice of medicine has changed dramatically during the last half century and knowledge and information have proliferated over the last several decades [19], medical classroom teaching has not changed substantially and is still driven by the lecture format [9]. Our study confirms that change in teaching methods comes slowly. Despite the fact that most of the respondents in our survey described themselves as knowledgeable about AL methods, 80% reported that AL is used in less than half of the CME activities at their institutions. When AL strategies were used, audience response polling, simulation, and group discussions were the most common. We asked survey respondents about the source of their responses to determine the accuracy of their reporting. They indicated that their answers were mostly based on factual CME unit data and frequent attendance at lectures, courses, or conferences at their institutions, validating the accuracy of our study findings. Taken together, the positive correlation of advanced training and self-perceived knowledge with use of AL suggest that many CME unit leaders are trained to use and facilitate these teaching methods in their institutions.

There are many AL strategies for instructors to consider when they design courses, but negative attitudes and pedagogical barriers must be overcome [4]. When exploring these potential attitudes of CME unit leaders, our survey demonstrates that survey participants recognize to a large degree the importance of learner engagement to increase knowledge and recall. They also agree that instructors who use AL are more concerned with facilitating reflection and working cooperatively with the learner. As a result, instead of merely conveying facts to the learner, knowledge can be applied in practice. While more than 60% of survey participants indicate that lectures are not difficult to adapt to AL methods, instructors at their institutions, nevertheless, do not incorporate AL methods to a large degree into their CME activities. Our study confirms that the benefits of AL are recognized but practical implementation still lags behind in many institutions.

While respondents have knowledge, positive attitudes, and perceptions about AL, we still needed to explore barriers to AL and describe resources needed to overcome these barriers to better understand the challenges of CME unit leaders in their institutions. In STEM disciplines, barriers to use of AL methods are described as “complex” [11] and include the triad of lack of time, training, and incentives to implement educational change. In medical education, however, barriers to use of AL have not previously been described. In our study, we found similar barriers to those identified by the STEM disciplines and the need for additional resources for preparation, planning, and faculty development. Other barriers include a lack of administrative support and change-averse cultures at institutions. Similar to the STEM disciplines, assistance from educational designers, institutional recognition, as well as additional training and faculty development in AL was found to be useful for increasing the use of AL in CME.

The strengths of this study are the methods used to develop the survey. Questions used to develop the construct, AL, were developed through an in-depth literature review of AL. Survey development was also guideded by Messick’s five sources of validity evidence [14]. Although content and response process validity are strong because of use of extensive pretesting and cognitive interviewing of the study population prior to dissemination, the lack of established criterion validity is a limitation of the study and will be an important next step in this area of research. Buy-in from SACME facilitated survey distribution to ensure that all members of the society had an opportunity to respond to the survey. Following survey distribution and data collection, we evaluated reliability of the survey instrument. Although one of our subconstructs, ‘Active learning strategies currently used in CME units’ had lower internal consistency reliability related to the yes/no question type and distribution of the data, the Likert scale survey questions demonstrated high internal consistency reliability as confirmed by Cronbach’s alpha.

Limitations of the study include low response rates as observed with on-line surveys. Reasons for this include potential incorrect email addresses, survey fatigue of the study population, and membership changes within SACME, whose membership is dynamic. The varying number of SACME listserv members, directly impacts the denominator. There were also more female compared to male respondents. Studies suggest that distribution of online survey responses by gender is variable and dependent upon the survey population and characteristics of that population. Drawing conclusions about these gender differences would be purely speculative. Although voluntary responses improve the quality of the responses, non-response bias must also be considered. Response rates to questionnaires have been declining over time regardless if the queries are Web-based or when other data collection technologies are used [20]. Multiple reasons for non-response have been recognized and despite utilization of recommended techniques and processes (e.g., prenotification of participants, publicizing the questionnaire) [12, 13, 20], we report an ~ 42% response rate. Our goal of performing follow up telephone surveys in non-respondents was to determine if respondents to the questionnaire were different from those who did not respond in terms of demographic or attitudinal variables. Since non-response bias also increases the likelihood of differences in the group of respondents compared to non-respondents, we followed up with telephone surveys of 10 randomly selected nonrespondents to validate the results of selected questions. Results of these telephone surveys were similar to online survey results, indicating a small effect of non-response bias on our results. Although some institutions may have been represented by several respondents, the anonymous nature of the survey and heterogeneity in responses limits this concern. The non-validated nature of self-reported items such as training, knowledge, and use of AL represent other recognized limitations. Finally, the use of AL cannot make up for poor event content, even if AL is used as the teaching method.

This study is the first to expand one conclusion of the 2018 Harrison survey by describing the knowledge, usage, attitudes, and barriers to implementation of AL strategies in academic CME units in the U.S. and Canada. Our results suggest that increased levels of self-perceived knowledge but not advanced training are associated with increased use of AL. Despite CME unit leaders’ training, education, and positive attitude toward AL, the use of AL is infrequent in many CME units, often resulting from limited resources such as faculty development and institutional recognition for AL teaching. Future directions include extending the survey and its methodology to other areas of medical education such as undergraduate and graduate medical education, incorporating qualitative data (e.g., in-depth narrative data) to gain better depth of understanding of the AL construct, as well as follow-up with similar surveys done in tandem with other aspects of the Harrison survey to trend the data. We would welcome collaboration by the AAMC and SACME to increase awareness of AL and the considerations of its use and implrmentation.

Conclusion

By improving dissemination of information about evidence-based AL strategies in medical education, the results of this study can be used to increase educational innovation that fosters learner engagement, critical thinking, and problem-solving ability.