Table 7 Key take home messages for improving AI-generated test item quality

• Compare multiple AI platforms to evaluate the output fidelity.

• Link course syllabus, SLOs, expected competency, and learner’s stage in the program.

• Use unambiguous and specific search prompts to refine the search iteration strategy.

• Decide whether test items sought are for formative or summative purpose.

• Clarify the expected test items match on Bloom’s taxonomy.

• Seek high fidelity clinical vignette to promote context-based learning.

• Define the level of integration appropriate to learner’s stage in the program.

• Integrate the complexity of OSPE clinical scenarios to patient-instructions.

• Recognize the limitations of AIs such as a limited access to all treatment guidelines.

• Ensure the validity of AI generated test items by content experts.

• Evaluate simulation- based standard setting guidance offered by AIs to real world situation.