Authors: Mirza Aroosa Beg , Afifa Tabassum , Sobia Ali

Abstract

Background and Objective:  Multiple-choice questions (MCQs) are a frequently used method of assessment in medical education. MCQs have to be well-constructed and have minimal flaws to achieve higher reliability and validity. Developing high quality MCQs is difficult but formal faculty training has found to have a positive impact on MCQ writing skills and quality. The purpose of the study was to assess the effect of a one-day training session on the quality of MCQs and the change in faculty’s self-rating of their competency in MCQ construction.

Methods:  This was a quasi-experimental study. A targeted workshop was conducted to train 17 faculty members in item writing skills and to improve the quality of existing item bank. Training was provided on the process of developing high quality assessment items and methods to avoid item flaws. Analysis of 48 MCQs from the existing MCQ bank was done for critique and improvement in quality. The MCQs were analyzed for level and type of questions and presence of item writing flaws. Faculty self-rating of their competency before and after the workshop session was also collected. Wilcoxon rank sum test was used to determine any change in the pre and post workshop quality of MCQs. Paired t-test was applied to analyze change in participants’ perception of their competency before and after the workshop session.

Results:  Analysis showed that after the workshop, the number of MCQs testing recall reduced significantly (N 42, 30, P ≤ 0.05). Similarly, questions testing application of knowledge improved significantly (N 6, 17,
P ≤ 0.05). Analysis of faculty’s self-rating of “change in competency” showed a significant improvement (P = 0.00) in their ability to improve the stem, lead-in and options list with appropriate and logically sequenced distractors.

Conclusion:  Targeted faculty development workshops improves the quality of MCQs construction. These trainings should be well structured and conducted on regular basis for better outcome.

Keywords: Multiple choice questions, Faculty development, Educational assessment, Workshops.