Designing Exam Questions
Designing Exam Questions
Criteria for Questions
Writing exam questions can be challenging. Best practices for designing and grading exams suggest that good exams should have the following characteristics:
- Valid: They should provide useful information about the concepts they were designed to test.
- Reliable: They should allow consistent measurement and should be able to discriminate between different levels of performance.
- Recognizable: Instruction has prepared learners for the exam and aligns with the exam content.
- Realistic: The time and effort required to complete the exam is realistic.
Exams can comprise a variety of question types, including true/false, multiple choice, matching, short answer/fill-in-the-blank, and essay. Each of these question types has advantages and disadvantages.
| Exam Question Type | Advantages | Disadvantages |
|---|---|---|
| True/false | Many items can be administered in a short period of time Moderately easy to write Easily scored | Limited primarily to testing knowledge/recollection Easy to guess correctly on many items even if the material is not mastered |
| Multiple choice | can be used to assess a broad range of content in a brief period skillfully written items can measure higher-order cognitive skills easily scored | Difficult and time-consuming to write good items Possible to assess higher-order cognitive skills, but most items assess only knowledge/recollection Some correct answers can be guessed |
| Matching | Items can be written quickly A broad range of content can be assessed Easily scored | Higher-order cognitive skills are difficult to assess
|
| Short answer/fill-in-the-blank | Can be administered in a brief period of time Moderately easy to write Relatively easy to score | Sometimes challenging to identify criteria for correct answers Limited to questions that can be answered or completed in very few words |
| Essay | Can be used to measure higher-order cognitive skills Relatively easy to write Difficult for respondents to get correct answers by guessing | Time-consuming to administer and grade Challenging to identify criteria for scoring Only a limited range of content can be assessed during any one testing period |
If exams, especially closed book/notes exams, are a significant part of your assessment strategy, it is important to intentionally design your course in a way that supports learners' ability to succeed and demonstrate their learning on your exams. By leveraging cognitive science principles that improve recall, building in repeated low-stakes practice quizzes, polling questions, homework assignments, and answering verbal questions in class, learners can improve recall, understanding, and comprehension. Writing exam questions for learners throughout a semester and receiving feedback from learners can also help instructors improve their own skills in developing good exams.
more information on organizing course material, enhancing memory, and metacognitive principles
Developing Exams to Evaluate Higher-Order Thinking
Exam questions can be written at different levels of Bloom’s Taxonomy, though some question types are better suited for assessing certain levels of cognitive complexity than others. For example, multiple-choice questions are best at assessing skills related to remembering or understanding, while short answers and essays are better suited to allow learners to demonstrate their ability to evaluate, analyze and create. Bloom's Taxonomy is a useful framework that allows instructors to assure that exam questions are developed at the appropriate cognitive level to align with the cognitive complexity associated with learning tasks and course learning objectives.
Things to Consider When Developing an Exam
- Align exam questions with course objectives and work with objectives that are well-suited to testing.
- Construct exam questions that reflect the emphasis placed on the material during the course.
- Questions should be stated clearly; don't write vague or ambiguous questions.
- Develop instructions that preview the exam and give examples if needed for clarity.
- Enlist other instructors or your teaching assistant to review your test questions, take the exam informally, and provide feedback on accuracy, clarity, and length.
- Maintain learner attention and engagement in longer exams, which have more reliable outcomes with a variety of question types.
- Consider the point values of different question types and align them with the emphasis on and importance of the concepts and material.
- Establish transparent grading criteria and publish those criteria.
- Include questions that have a difficulty level appropriate to the learning objectives.
- Review policies and procedures for accommodations and make learners aware of them.
- Evaluate the test by also reviewing the cognitive (thinking) level of questions, the meaningfulness and value of what is tested, the clarity and inclusiveness of the language used in the questions, the fairness of the questions, and the reliability of the test outcomes over time
Tips on Writing Individual Exam Questions
- Write questions with one best answer.
- Test a single idea per question.
- Incorporate common learner misconceptions and errors as wrong answers.
- Write more questions than you need, in case you want to create different exam versions.
- Avoid extreme or vague modifiers or qualifiers (e.g. generally, all, frequently).
- Randomize answers and avoid patterns in answer selections.
- Keep the length of answer choices as short as possible.
- Sparingly write questions with a negative statement.
- Organize questions by question type (multiple-choice, true-false, matching, fill-in-the-blank).
- Write explicit directions for each question type.
- Eliminate grammatical clues that point to the correct answer.
- Don't overuse “all of the above” and “none of the above” in responses.
References
Worthen, B.R., Borg, W.R., & White, K.R. (1993). Measurement and evaluation in the schools. New York: Longman.
Resources
Kansas State University: A Short Guide to Writing Effective Test Questions
Center for Research on Learning and Teaching, University of Michigan: Framework for Designing Effective Exams
Center for Research on Learning and Teaching, University of Michigan: Best Practices for Designing and Grading Exams
Center for Research on Learning and Teaching, University of Michigan: Writing Questions