Skip to main content

Advertisement

  • Research article
  • Open Access
  • Open Peer Review

Resident and program characteristics that impact performance on the Ophthalmic Knowledge Assessment Program (OKAP)

  • 1,
  • 1,
  • 1,
  • 1,
  • 1,
  • 1,
  • 1 and
  • 1Email author
BMC Medical Education201919:190

https://doi.org/10.1186/s12909-019-1637-4

  • Received: 21 December 2018
  • Accepted: 29 May 2019
  • Published:
Open Peer Review reports

Abstract

Background

To determine which resident and program characteristics correlate with ophthalmic knowledge, as assessed by resident Ophthalmic Knowledge Assessment Program (OKAP) performance.

Methods

An online survey was sent in June 2017 to all US ophthalmology residents who took the OKAP in April 2017.

Results

The survey response rate was 13.8% (192/1387 residents). The mean respondent age was 30.4 years, and 57.3% were male. The mean [SD] self-reported 2017 OKAP percentile was 61.9 [26.7]. OKAP performance was found to have a significant positive correlation with greater number of hours spent/week studying for the OKAPs (p = 0.007), with use of online question banks (p < 0.001), with review sessions and/or lectures arranged by residency programs (p < 0.001), and with OKAP-specific didactics (p = 0.002). On multivariable analysis, factors most predictive of residents scoring ≥75th percentile were, higher step 1 scores (OR = 2.48, [95% CI: 1.68–3.64, p < 0.001]), presence of incentives (OR = 2.75, [95% CI: 1.16–6.56, p = 0.022]), greater number of hours/week spent studying (OR = 1.09, [95% CI:1.01–1.17, p = 0.026]) and fewer hours spent in research 3 months prior to examination (OR = 1.08, [95% CI: 1.01–1.15, p = 0.020]. Lastly, residents less likely to depend on group study sessions as a learning method tended to score higher (OR = 3.40, [95% CI: 1.16–9.94, p = 0.026]).

Conclusions

Programs wishing to improve resident OKAP scores might consider offering incentives, providing effective access to learning content e.g. online question banks, and adjusting the curriculum to highlight OKAP material. Step 1 scores may help educators identify residents who might be at risk of not performing as well on the OKAP.

Keywords

  • Resident
  • Education
  • Knowledge
  • OKAP

Background

Medical knowledge is widely accepted to be positively associated with superior care and with better patient management. More knowledgeable physicians are more likely to adhere to evidence-based guidelines in the delivery of care and achieve better patient outcomes [13]. Indeed, one of the key elements of residency training is to give early career physicians sufficient knowledge for carrying out their appropriate roles. The Accreditation Council for Graduate Medical Education (ACGME) lists knowledge as one of the six core competencies required for all fields in resident training. Within the field of ophthalmology, the Ophthalmic Knowledge Assessment Program (OKAP), which is a multiple-choice, in-service examination, has been used as a measure to objectively assess core ophthalmic knowledge of residents and allows for comparison among peers, programs, and prior residents. Moreover, it is meant to serve as a guide towards preparation for the American Board of Ophthalmology Written Qualifying Examination (ABO-WQE) [4]. The WQE is an important component of board certification which is typically required for credentialing.

Given the importance of physician knowledge, it is important to provide guidance for ophthalmic trainees and their programs regarding the best methods and approaches to gain the requisite knowledge. Such guidance is increasingly important given the growing myriad of resources available to residents, including the Basic and Clinical Science Course (BCSC) books compiled by the American Academy of Ophthalmology (AAO), didactics by individual programs, review courses and books, and online question banks.

This study describes a survey of ophthalmology residents across the US to determine what resident characteristics and program attributes correlate with higher levels of ophthalmic knowledge, as judged by performance on the 2017 OKAP exam.

Methods

The study was approved by the Institutional Review Board of The Johns Hopkins University School of Medicine.

A 22-question anonymous survey (Qualtrics, Provo, UT) was emailed to all US ophthalmology program directors who were Association of University Professors of Ophthalmology (AUPO) members on June 2, 2017. The program directors were asked to forward the survey on to all residents, with response collection ending on June 30, 2017. Three reminder emails were sent during the response collection period. Program directors who forwarded the survey to their residents were asked to reply and indicate that they have done so. In the survey, participants were made aware that all data would be kept confidential and only aggregate data would be shared. Participants were informed that their completion of the survey indicated consent for participation in the study.

Data on demographics (age, sex, marital status, and year of training), geographic region of the residency training programs, and study habits (average number of months spent preparing for examination, average number of hours spent studying, average number of hours spent in research 3 month prior to exam, average work hours) were collected. Residents were asked to report their 2017 OKAP exam percentile in addition to their USMLE Step 1 scores in ranges of 10 (e.g. 210–220, 221–230, etc.). The utility of a variety of study resources was assessed with a four-point Likert scale (not at all useful, slightly useful, moderately useful, and extremely useful). The presence of program offerings such as incentives (e.g. awards, money, vacation), repercussions (e.g. remediation), and call coverage for the night prior to OKAP exam was also assessed using a 3-point scale (yes, no, and not sure). Residents’ opinion regarding the OKAP exam was examined using a 5-point Likert scale (strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, and strongly agree).

Pearson correlation coefficients were calculated between self-reported 2017 OKAP percentile and continuous variables. The two-sample t-test was used to compare OKAP percentiles with categorical variables, and analysis of variance (ANOVA) was used for variables with more than 2 categories. Additionally, a forward stepwise multivariable logistic regression model was constructed to determine which factors were related independently to higher resident medical knowledge (scoring on the 75th percentile or higher). Variables included in the model were participant demographics (age, sex, and marital status), residency program characteristics (geographic region, use of incentives or repercussions, call coverage for the night prior to examination), USMLE step 1 scores, and examination preparation (resources used, hours spent studying, work hours and hours spent in research). A linear regression was also performed to predict OKAP percentiles using the same variables. All data were analyzed using Stata. A p value of ≤0.05 was considered statistically significant for all analyses.

Results

Participant demographics

The survey was forwarded to residents by 19 of the ophthalmology programs (n = 272 residents), and 192 responses were received. This represented 13.8% of the ophthalmology resident pool in 2017 (192/1387). Of the 192 participants, 183 responded with their exact percentile, while 9 could only provide ranges because they did not recall their exact percentile. All subsequent correlation analyses were completed with the 183 participants who provided their exact percentile. Participant characteristics are summarized in Table 1.
Table 1

Baseline participant characteristics

Demographic

 Mean age, (SD), years

30.4 (2.7)

 Mean number of residents in program, (SD)

11.0 (4.6)

 Mean self-reported percentile score for 2017 OKAP exam, (SD)

61.9 (26.7)

Gender

 Female, No. (%)

82 (42.7)

 Male, No. (%)

110 (57.3)

Training Year

 PGY 2, No. (%)

71 (37.0)

 PGY 3, No. (%)

67 (34.9)

 PGY 4, No. (%)

54 (28.1)

Geographic Region of Residency Program

 Midwest, No. (%)

40 (20.8)

 Northeast, No. (%)

60 (31.3)

 South, No. (%)

48 (25.0)

 West, No. (%)

44 (22.9)

OKAP percentile and resident work

The mean (standard deviation [SD]) self-reported 2017 OKAP percentile from participants was 61.9 (26.7; range 4–100). The average number of hours per week spent on studying for the OKAP was significantly correlated with OKAP percentile (p = 0.007). OKAP percentile was not significantly correlated with program size, total number of months spent studying, average number of hours per week spent on research in the 3 months before the exam, or average number of duty hours per week logged for the ACGME (Table 2, p > 0.05 for all). The degree of use of online question banks (e.g., OphthoQuestions.com) was statistically significantly correlated with OKAP percentile (p < 0.001). Review sessions and/or lectures by programs were also correlated with OKAP percentile (p < 0.01). All other resources surveyed (review books, BCSC books, question books, group study sessions, and review courses) were not significantly correlated with OKAP percentile (Table 3, p > 0.05 for all). The write-in responses for the option of “Other” regarding study resources were “Anki cards,” “On call review for complex patients,” and “Self-made flash cards.”
Table 2

Resident Work and Correlation with OKAP Percentile

 

Pearson’s Correlation Coefficient

p-value

Mean number of months spent studying for OKAP

−0.026

0.727

Mean number of hours per week spent studying for OKAP

0.201

0.007

Mean number of hours per week spent on research

−0.101

0.180

Mean number of duty hours per week logged

−0.100

0.183

Table 3

Correlation of different study resources utilized with OKAP Percentile

Resource

OKAP Percentile, Mean ± SD, (No. of Respondents)

p-Value

Did Not Use

Not At All Useful

Slightly Useful

Moderately Useful

Extremely Useful

Review Books (e.g. Chern, Friedman)

60.5 ± 30.6 (51)

54.3 ± 30.0 (3)

63.5 ± 26.5 (42)

63.3 ± 23.9 (55)

63.7 ± 22.4 (26)

0.946

Basic Clinical science course (BCSC) books

57.7 ± 28.3 (19)

72.5 ± 27.1 (6)

55.9 ± 27.9 (52)

63.6 ± 24.3 (62)

69.3 ± 26.2 (38)

0.122

Question Books (e.g. ProVision, Mass Eye and Ear Review)

62.3 ± 27.0 (141)

26.0 ± 31.1 (2)

66.9 ± 22.6 (14)

67.7 ± 20.8 (13)

68.5 ± 27.6 (2)

0.308

Online Question Banks (e.g. OphthoQuestions.com)

59.6 ± 25.8 (5)

N/A (0)

49.3 ± 31.4 (8)

49.9 ± 27.2 (43)

68.4 ± 23.7 (120)

< 0.001

Group Study Sessions

60.2 ± 27.6 (107)

61.5 ± 21.4 (4)

71.8 ± 27.2 (25)

63.0 ± 19.0 (28)

68.0 ± 27.1 (10)

0.359

Review Sessions/Lectures by Program

47.1 ± 28.8 (13)

48.7 ± 28.8 (9)

55.4 ± 27.1 (65)

67.7 ± 24.0 (63)

77.7 ± 18.8 (26)

< 0.001

Review Courses (e.g. Wills Eye, Osler, San Antonio)

63.6 ± 26.4 (128)

48.0 ± 11.5 (3)

60.8 ± 28.5 (14)

58.6 ± 24.1 (18)

66.3 ± 28.8 (12)

0.770

OKAP percentile and program characteristics

When residents were asked how much they agree that their year-round didactics by their residency program emphasized OKAP material, those who strongly agreed with the statement had significantly higher OKAP scores than those who strongly disagreed (mean OKAP percentile 65.8 versus 37.8, p = 0.002). Residents’ opinion on whether their OKAP performance was important to their residency program or fellowship applications did not correlate with their 2017 OKAP performance (Table 4).
Table 4

Correlation between resident opinion on OKAP-specific statements and their OKAP percentile

Statement

OKAP Percentile, Mean ± SD

(No. of Respondents)

p-value

Strongly Disagree

Somewhat Disagree

Neither Agree Nor Disagree

Somewhat Agree

Strongly Agree

My residency program places importance on my OKAP performance

61.3 ± 38.5 (4)

45.8 ± 27.4 (12)

56.3 ± 23.9 (42)

60.8 ± 27.3 (81)

67.7 ± 24.1 (69)

0.075

My OKAP performance is important for fellowship applications

61.4 ± 31.4 (9)

59.0 ± 32.3 (25)

62.4 ± 23.5 (43)

60.8 ± 26.6 (81)

71.7 ± 21.9 (20)

0.537

My year round didactics emphasize OKAP material

37.8 ± 27.7 (15)

57.2 ± 28.9 (31)

64.5 ± 26.6 (34)

66.9 ± 24.3 (76)

65.8 ± 20.0 (20)

0.002

Furthermore, residents were asked whether their programs offered incentives for high OKAP performance, repercussions for poor OKAP performance, and/or call coverage for the night prior to OKAP exam. Those whose programs offered incentives had significantly higher OKAP scores than those whose programs did not (mean OKAP percentile 72.9 versus 57.6, p = 0.003). Residents whose programs had implemented repercussions for poor scores did not perform significantly differently from residents whose programs did not (mean OKAP percentile 59.7 versus 59.4, p = 0.98). OKAP performance also did not differ for residents whose programs had call coverage for the night before compared to those whose programs did not (mean OKAP percentile 63.2 versus 56.3, p = 0.59).

Regression analysis

Results of the univariable analysis are shown in Table 5. On multivariable analysis, factors that were most predictive of higher ophthalmic knowledge as predicted by residents scoring on the 75th percentile or higher were higher step 1 scores (defined as every 10-point increase in scores) (odds ratio (OR) = 2.48, [95% confidence interval (CI): 1.68–3.64, p < 0.001), presence of incentives (OR = 2.75, [95% CI: 1.16–6.6.56, p = 0.022]), spending more hours on average per week studying (OR = 1.09, [95% CI:1.01–1.17, p = 0.026]) and spending fewer hours in research three months prior to examination (OR = 1.08, [95% CI: 1.01–1.15, p = 0.020]. Lastly, residents less likely to depend on group study sessions as a learning method tended to score higher (OR = 3.40, [95% CI: 1.16–9.94, p = 0.026]) (Table 6). Higher Step 1 scores were associated with ophthalmology residents being less likely to score below the 30th percentile on their the OKAP examinations (Additional file 1: Table S1).
Table 5

Univariable regression analysis of factors predictive for scoring above the 75th percentile on the OKAPS

Variable

Odds Ratio

95% Confidence interval

P-value

Age

0.90

0.80–1.01

0.075

Sex (reference female)

 Male

1.89

1.04–3.45

0.037

Marital status (reference married)

 Single

1.01

0.55–1.86

0.972

Training year (reference PGY-2)

 PGY-3

0.90

0.45–1.78

0.755

 PGY-4

0.95

0.46–1.95

0.886

Geographic region (reference Midwest)

 Northeast

1.42

0.61–3.30

0.410

 South

0.91

0.36–2.62

0.837

 West

3.00

1.23–7.33

0.016

Incentives (reference no)

 Yes

2.40

1.27–4.54

0.007

Repercussions (reference no)

 Yes

1.05

0.52–2.10

0.896

Call coverage prior to OKAP (reference no)

 Yes

4.23

0.50–35.86

0.186

Review books (reference did not use)

 Not very useful

1.28

0.58–2.82

0.540

 Extremely/moderately useful

0.87

0.43–1.75

0.691

Basic Clinical science course (BCSC) books (reference did not use)

 Not very useful

1.24

0.44–3.54

0.683

 Extremely/moderately useful

1.55

0.58–4.16

0.383

Question books (reference did not use)

 Not very useful

1.50

0.56–4.00

0.418

 Extremely/moderately useful

1.17

0.41–3.31

0.772

Online question banks (reference did not use)

 Not very useful

2.40

0.18–32.88

0.660

 Extremely/moderately useful

2.87

0.31–26.21

0.930

Group study sessions (reference did not use)

 Not very useful

2.48

1.11–5.56

0.027

 Extremely/moderately useful

1.13

0.54–2.38

0.741

Table 6

Multivariable regression analysis of factors predictive for scoring above the 75th percentile on the OKAPS

Variable

Odds Ratio

95% Confidence interval

P-value

Sex (reference female)

 Male

2.14

0.94–4.90

0.070

Region (reference Midwest)

 Northeast

3.49

0.97–12.56

0.056

 West

3.67

0.99–13.50

0.051

 South

2.90

0.74–11.45

0.128

Lectures (reference did not use)

 Not very useful

0.60

0.10–3.61

0.578

 Extremely/moderately useful

1.42

0.13–4.85

0.795

Group study (reference did not use)

 Not very useful

3.40

1.16–9.94

0.026

 Extremely/moderately useful

0.67

0.53–3.78

0.481

Incentives (reference no)

 Yes

2.75

1.16–6.56

0.022

Average number of hours per week spent studying

1.09

1.01–1.17

0.026

Average number of hours spent in research 3 months prior to exam

0.93

0.87–0.99

0.020

Step 1

2.48

1.68–3.64

< 0.001

OKAP percentile was also evaluated as a continuous variable. We found OKAP percentile to significantly correlate with incentive-based performance (beta =13.72, [95%CI: 6.74, 20.70; p < 0.001)], Step 1 scores (beta = 8.56, [95% CI: 5.99, 11.13; p < 0.001)], and male gender (beta = 7.79, [95% CI: 1.24, 14.34; p = 0.020)]. Residents who found review sessions and lectures offered by their programs to have limited value were more likely to score lower (beta = − 8.18, [95% CI: − 15.19, − 1.17; p = 0.022)] compared to those who found them to be useful. Spending greater than 20 h per week in research was also negatively associated with OKAP performance (beta = − 17.42, [95% CI: − 2.45, − 32.39; p = 0.023)].

Discussion

In our study, we found resident OKAP performance, and by extension, ophthalmic knowledge, to significantly correlate with more hours per week spent studying as well as with incentive-based performance. Among educational resources, year-round didactics, online question banks, and review sessions were associated with better OKAP performance. To the best of our knowledge, resident ophthalmic knowledge has been evaluated in only 1 other study. The authors of that study found an association between resident knowledge, completion of clinical rotations and learning via multiple modalities. However, their study assessed resident knowledge specifically on the topic of glaucoma [5].

The results of our study have important implications for educators. In addition to being the only tool for assessing overall program effectiveness [6], multiple studies have shown an association between OKAP performance and the ABO-WQE pass rate [68]. Board certification has historically been considered a sign of physician competence. In many cases, it is a prerequisite for employment and academic appointment [6] and certification status has been shown to be an important predictor of patient quality of care among medical professionals [911]. Johnson et al. reported a 5.43-fold increased odds of passing the WQE among residents passing the OKAP examinations, (defined as overall score in the 30th percentile or above by the authors), in all 3 years of residency training. In contrast, failing all 3 OKAP examinations was associated with more than 9-fold lower odds of passing the WQE on the first attempt [5]. Similarly, a separate study by Lee et al. that involved 246 residents from 15 institutes found OKAP scores across all 3 years of residency to be predictive of first time WQE pass rate, with third year OKAP scores being the most predictive. Additionally, the authors found a correlation between the mean OKAP scores and WQE pass rate; passing the WQE was at least 80% for a score of 35 or higher, at least 90% for a score of 53 or higher, and at least 95% for a score of 72 or higher [6]. Furthermore, ophthalmic knowledge may have important implications for a graduating resident’s career, since OKAP performance is one of the factors currently used in the sub-specialty fellowship selection process [12]. As the OKAP is currently one of our best assessments of ophthalmic knowledge and because it serves as an important predictor of WQE performance, it is essential that residency programs place emphasis on improving study practices.

In our study, spending greater than 20 h per week in research-related activities closer to the scheduled examination was found to negatively affect resident OKAP performance. We believe that these residents likely preferred spending more time on research-related activities than studying for OKAPs. Furthermore, resident training years are commonly associated with long and unpredictable work hours. Physician burnout and emotional exhaustion have in fact, been shown to result in decreased medical knowledge [13, 14]. The challenge for medical education is therefore, to maintain knowledge while concurrently ensuring physician well-being and adequate patient safety. There is no question that the demands of surgical residency sometimes make reading and studying difficult. Possible changes can include allocating more time for resident self-study, offering review sessions which are tailored to meet the needs of residents and providing effective access to learning content. Residents may also consider maintaining a regular study schedule throughout the year, which has been shown to result in improved medical knowledge and better performance on standardized examinations [14, 15]. Although we did not assess the effectiveness of electronic or E-learning in our study, residency programs could also consider complementing traditional didactic methods with E-learning. E-learning has been shown to be as effective as other educational approaches for acquisition of knowledge [16]. Additionally, the greater flexibility offered by E-learning courses could make them particularly useful for graduate medical education [17]. We hypothesize that improved practices such as these will result in successful acquisition and retention of knowledge among residents. This is important because knowledge is currently one of the six core competencies that graduating residents are required to achieve competency in. [18]The OKAP can therefore be used as a benchmark for ophthalmology program effectiveness in medical knowledge [18] and additionally, as an educational tool in meeting the residency training requirement for board certification [19].

While increased emphasis of program policies towards in-training examinations as well as improved resident studying habits have been known to correlate with better knowledge acquisition [14, 2022], limited data currently exist on how this knowledge translates into clinical performance in the real world. A cross-sectional study conducted by Catalano et al. failed to demonstrate any relationship between OKAP scores and resident clinical performance [23] and a separate study by Scott et al. similarly failed to show an association between American Board of Surgery In-Training Examination (ABSITE) scores, technical skill and operative performance [24]. This lack of association however, may have been due to several different factors, including a small sample size [23, 24] and that clinical performance was judged subjectively [23]. Future studies are needed to assess whether resident knowledge levels are associated with other competency milestones such as patient care and procedural skills.

USMLE Step 1 scores are widely regarded to be a valid measure of underlying medical knowledge [25] and our study results are consistent with the notion that, medical students who are most successful with step 1 of the USMLE will be more able to acquire the knowledge needed to satisfactorily complete their ophthalmology training. We observed almost a 9-point increase in the OKAP percentile and 2.5 times higher odds of scoring above the 75th percentile, when USMLE scores moved up by every 10-point category. As such, this information benefits educators by allowing them to identify residents at risk of performing poorly, as early as possible. Identifying residents with insufficient medical knowledge is important for several reasons. Most importantly, physicians who lack adequate medical knowledge may be more likely to make errors in diagnosis and perform less well clinically, which can jeopardize patient safety [13, 26].

Interestingly, we found residents who were less likely to rely on group study for learning, performed better. A study involving 45 general surgery residents reported similar findings, with independent study methods identified as the most effective study method for successful American Board of Surgery examination performance [27]. Potential reasons for this include that, residents partaking in self-study may have greater focus and fewer distractions as well as the ability to study at one’s own pace while identifying areas of weakness that require greater reinforcement. Self-study may also be associated with higher levels of concentration, which has been shown to be an important predictor for learning [28]. However, it is important to approach these findings of group versus self-study with caution. It might be that respondents who preferred self-study generally do well on standardized tests. Furthermore, it might be that the responses we received might have been skewed from respondents who preferred self-study over group study. Incentive-based performance was found to be a very important motivator for medical knowledge acquisition in our study sample. Ophthalmic knowledge, as indicated by the OKAP scores, increased by 15 points for programs that offered incentives in the form of awards, money or vacation, compared to those that did not. Residents belonging to such programs were 3-times more likely to be in the top 25th percentile for medical knowledge concerning their field when compared with their peers. In contrast, no such relation was seen when repercussions were employed by residency programs for poor OKAP performance. Similar findings regarding incentive use and improved test performance have also been reported among school-going children [29]. In medicine, incentives have been used to improve trainee compliance with patient safety measures [30] and encourage faculty productivity in educational activities [31]. Furthermore, the recently introduced Merit-Based Incentive Payment System rewards U.S. physicians practicing higher-value care with higher fees [32]. Thus, by incentivizing examinations, residency programs might influence resident motivation and thereby improve knowledge levels.

One major limitation to our study was the limited response rate. Although we sent the survey to all training programs in the country, only 19 (15.7%) programs forwarded the survey to their residents. However, our program and respondent characteristics are comparable to that of the U.S. ophthalmology residency programs as well as the ophthalmology resident population. Based on our analysis of 121 U.S. ophthalmology residency programs, we found the average program size to be 12 residents, with the following regional distribution: Midwest 23.1%, Northeast 30.6%, South 28.9% and West 17.4%. Similarly, 240–250 was the most common Step 1 score range among our residents, which is again comparable to the average Step 1 score of 242–245 as reported by the San Francisco Match (SF Match) for the years 2014–2018. Furthermore, our study response rate, despite being at the lower end of the spectrum, is within the response rate range of surveys (10–51%) conducted in the field of ophthalmology [3335]. Given our limited response rate, it would be beneficial to assess our results using a large multi-center study, especially for understanding factors associated with residents scoring less than the 30th percentile. Secondly, the distribution of self-reported OKAP percentile is positively skewed, indicating sampling bias, as those who scored higher on the OKAP might be more willing to complete the survey. Since residents were asked by report their OKAP percentiles, recall bias may be present, although the survey was distributed shortly after the 2017 OKAP results were released. Future studies assessing exact individual scores reported to programs (as opposed to self-reported) and correlations with objective clinical performance may be useful.

Conclusion

In conclusion, our study demonstrates several factors that correlate with OKAP performance and ways programs may strengthen resident medical knowledge as measured by OKAP performance. Programs wishing to improve resident learning and education might consider implementing strategies designed to increase knowledge acquisition. Strategies supported by our results include implementing incentives and effective access to learning content, with scheduling didactics dedicated to OKAP material and making online question banks available to residents. It may be worthwhile to consider allocating more time for resident self-study. Furthermore, Step 1 scores can help educators identify early on, residents who might be at risk for attaining insufficient medical knowledge and, for not performing as well on the OKAP. Future studies are needed to investigate whether improved resident knowledge, predicts clinical acumen after training or the Oral Exam pass rate, which is administered after the successful completion of the WQE and whether medical knowledge is correlated with other ACGME-mandated competencies.

Abbreviations

AAO: 

American Academy of Ophthalmology

ABO-WQE: 

American Board of Ophthalmology Written Qualifying Examination

ABSITE: 

American Board of Surgery In-Training Examination

ACGME: 

Accreditation Council for Graduate Medical Education

ANOVA: 

Analysis of Variance

AUPO: 

Association of University Professors of Ophthalmology

BCSC: 

Basic and Clinical Science Course

OKAP: 

Ophthalmic Knowledge Assessment Program

SF Match: 

San Francisco Match

Declarations

Acknowledgments

We would like to thank the residency program directors across the US who distributed our survey to their residents. We would also like to thank Jiangxia Wang MS at the Johns Hopkins Biostatistics Center, for her assistance with data analysis.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors’ contributions

SZ manuscript draft, data analysis and interpretation; XW data collection, manuscript draft; DS study design, critical revision of manuscript; SS study design, manuscript draft and critical revision; MB study design, critical revision of manuscript; PR study design, data analysis, critical revision of manuscript; ES data analysis and interpretation, critical revision of manuscript; FW study design, data interpretation and critical revision of manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Johns Hopkins Institutional Review Board; resident’s completion of survey was considered as consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Wilmer Eye Institute, Johns Hopkins University School of Medicine, 600 North Wolfe Street, Wilmer B29, Baltimore, MD, Baltimore, MD 21287, USA

References

  1. Richardson DP, Porter GA, Johnson PM. Surgeon knowledge contributes to the relationship between surgeon volume and patient outcomes in rectal cancer. Ann Surg. 2013;257(2):295–301.View ArticleGoogle Scholar
  2. Tamblyn R, Abrahamowicz M, Brailovsky C, et al. Association between licensing examination scores and resource use and quality of care in primary care practice. JAMA. 1998;280(11):989–96.View ArticleGoogle Scholar
  3. Tamblyn R, Abrahamowicz M, Dauphinee W, et al. Association between licensure examination scores and practice in primary care. JAMA. 2002;288(23):3019–26.View ArticleGoogle Scholar
  4. Liesegang TJ. New directions for the ophthalmic knowledge assessment program (OKAP) examination. Ophthalmology. 1994;101(1):194–8.View ArticleGoogle Scholar
  5. van Landingham SW, Srikumaran D, Siddiqui AA, Swenor B, Ramulu PY. Characteristics that predict ophthalmic knowledge among ophthalmology residents. Ophthalmology. 2015;122(5):1063–5.View ArticleGoogle Scholar
  6. Lee AG, Oetting TA, Blomquist PH, et al. A multicenter analysis of the ophthalmic knowledge assessment program and American board of ophthalmology written qualifying examination performance. Ophthalmology. 2012;119(10):1949–53.View ArticleGoogle Scholar
  7. Johnson GA, Bloom JN, Szczotka-Flynn L, Zauner D, Tomsak RL. A comparative study of resident performance on standardized training examinations and the American board of ophthalmology written examination. Ophthalmology. 2010;117(12):2435–9.View ArticleGoogle Scholar
  8. Chen PP, Bhandari A. Resident performance on the ophthalmic knowledge assessment program and the written qualifying examination for board certification. J Acad Ophthalmol. 2010;3(1):11–4.Google Scholar
  9. Ramsey PG, Carline JD, Inui TS, Larson EB, LoGerfo JP, Wenrich MD. Predictive validity of certification by the American Board of Internal Medicine. Ann Intern Med. 1989;110(9):719.View ArticleGoogle Scholar
  10. Silber JH, Kennedy SK, Even-Shoshan O, et al. Anesthesiologist board certification and patient outcomes. Anesthesiology. 2002;96(5):1044–52.View ArticleGoogle Scholar
  11. Kelly JV, Hellinger FJ. Physician and hospital factors associated with mortality of surgical patients. Med Care. 1986;24(9):785–800.View ArticleGoogle Scholar
  12. Kempton JE, Shields MB, Afshari NA, Dou W, Adelman RA. Fellow selection criteria. Ophthalmology. 2009;116(5):1020–1020.e1024.View ArticleGoogle Scholar
  13. West CP, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306(9):952–60.View ArticleGoogle Scholar
  14. Smeds MR, Thrush CR, McDaniel FK, et al. Relationships between study habits, burnout, and general surgery resident performance on the American Board of Surgery in-Training Examination. J Surg Res. 2017;217:217–25.View ArticleGoogle Scholar
  15. Kim JJ, Kim DY, Kaji AH, et al. Reading habits of general surgery residents and association with American Board of Surgery in-Training Examination Performance. JAMA Surg. 2015;150(9):882–9.View ArticleGoogle Scholar
  16. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300(10):1181–96.View ArticleGoogle Scholar
  17. Wittich CM, Agrawal A, Cook DA, et al. E-learning in graduate medical education: survey of residency program directors. BMC Med Educ. 2017;17(1):114.View ArticleGoogle Scholar
  18. Lee AG. The new competencies and their impact on resident training in ophthalmology. Surv Ophthalmol. 2003;48(6):651–62.View ArticleGoogle Scholar
  19. O'Day DM. Examinations and ophthalmic education. Ophthalmology. 2012;119(10):1947–8.View ArticleGoogle Scholar
  20. Wade TP, Kaminski DL. Comparative evaluation of educational methods in surgical resident education. Arch Surg. 1995;130(1):83–7.View ArticleGoogle Scholar
  21. Godellas CV, Hauge LS, Huang R. Factors affecting improvement on the American Board of Surgery in-Training Exam (ABSITE). J Surg Res. 2000;91(1):1–4.View ArticleGoogle Scholar
  22. Miyamoto RG Jr, Klein GR, Walsh M, Zuckerman JD. Orthopedic surgery residents' study habits and performance on the orthopedic in-training examination. Am J Orthop (Belle Mead NJ). 2007;36(12):E185–8.Google Scholar
  23. Catalano RA, Jenkins PL. Correlating factors to resident performance in an ophthalmology program. Ophthalmology. 1989;96(4):552–4.View ArticleGoogle Scholar
  24. Scott DJ, Valentine RJ, Bergen PC, et al. Evaluating surgical competency with the American Board of Surgery in-Training Examination, skill testing, and intraoperative assessment. Surgery. 2000;128(4):613–22.View ArticleGoogle Scholar
  25. Margolis MJ, Clauser BE, Winward M, Dillon GF. Validity evidence for USMLE examination cut scores: results of a large-scale survey. Acad Med. 2010;85(10 Suppl):S93–7.View ArticleGoogle Scholar
  26. Kohn LT, Corrigan JM, Donaldson MS. Errors in health care: a leading cause of death and injury. 2000.Google Scholar
  27. Coe NW, Rowland-Morin PA. Comparative evaluation of educational methods in surgical resident education. Arch Surg. 1995;130(9):1022.View ArticleGoogle Scholar
  28. West C, Kurz T, Smith S, Graham L. Are study strategies related to medical licensing exam performance? Int J Med Educ. 2014;5:199–204.View ArticleGoogle Scholar
  29. Allan BM, Fryer RG. The power and pitfalls of education incentives. Brookings Institution, Hamilton Project; 2011.Google Scholar
  30. Hussain SA, Arsene C, Hamstra C, Woehrlen TH, Wiese-Rometsch W, White SR. Successful resident engagement in quality improvement: the Detroit medical center story. J Grad Med Educ. 2016;8(2):214–8.View ArticleGoogle Scholar
  31. Rahim A, Anderson RA. Strategies for residency programs that improve medicine departments and teaching hospitals. Am J Med. 2008;121(5):450–5.View ArticleGoogle Scholar
  32. Rathi VK, McWilliams JM. First-year report cards from the merit-based incentive payment system (MIPS): what will be learned and what next? JAMA. 2019;321(12):1157–8.View ArticleGoogle Scholar
  33. Chiang MF, Boland MV, Margolis JW, Lum F, Abramoff MD, Hildebrand PL. Adoption and perceptions of electronic health record systems by ophthalmologists: an American Academy of ophthalmology survey. Ophthalmology. 2008;115(9):1591–7 quiz 1597.e1591-1595.View ArticleGoogle Scholar
  34. Gedde SJ, Budenz DL, Haft P, Tielsch JM, Lee Y, Quigley HA. Factors influencing career choices among graduating ophthalmology residents. Ophthalmology. 2005;112(7):1247–54.View ArticleGoogle Scholar
  35. Macaluso DC, Andre M, Caroline PJ, Suhler EB, Rich LF. Assessment of ophthalmology residents' contact lens training. CLAO J. 2000;26(4):221–4.Google Scholar

Copyright

© The Author(s). 2019

Advertisement