Table 1 Preliminary conceptual framework for establishing content validity of AI-generated test items
Domains | Items assessed |
|---|---|
Technical | • Are the test items/explanation technically accurate and free from empirical or clinical mistakes? |
Comprehensiveness | • Do the test items /explanations sufficiently address relevant topics/subtopics? • Is the within-topic variation (range of examples, patient characteristics, scenario descriptions) at the desired level? |
Education level | • Are the test items /model answers appropriate to the education level of the learner? • Is the structure of the questions/explanations aligned to learning outcomes (as per Bloom’s taxonomy)? |
Free of construction defects | • Are the test items /answers framed in a way to present a clear “best response” with appropriate and unambiguous distractors? • Does the test item/explanation avoid therapeutic controversies. • Are the test items integrated with the case vignette (without being standalone)? |