Multiple Choice Quizzes Do you ever have accreditation standards that require students to do tests? Do you have a lot of content that students must learn to recall? Do you have students who need grades to motivate them to complete required readings or activities? Do you need a way to quickly measure and gauge if students are grasping new material? Have you ever considered multiple choice quizzes?
10
One of the most common types of selected response quizzes is the multiple choice test. A multiple choice question consists of a question stem and item shell, a keyed answer - the best answer, and one or more, usually three, distractors - possible answers (DiBattista & Kurzawa, 2011). With large classes, multiple choice tests are a quick and easy way to measure student progress. Statistically, a well-written multiple choice test is a reliable measurement. However, despite their widespread use, most multiple choice tests lack quality questions (Asim, Ekuri,& Eni, 2013; DiBattista & Kurzawa, 2011).
Variations There are two main types of quizzes: selected response or constructed response. In a selected response quiz, students select from a choice of answers. In a constructed response quiz, students generate a unique answer. Types of Selected-Response Quizzes •True and False •Ordering •Matching •Multiple Choice Types of Constructed Response Quizzes •Fill-in-the-Blank •Short Answer •Essay
Examples Use your imagination. What kind of multiple choice quizzes could your students complete? What would be an appropriate quiz for your students? Only you know what would be authentic to your field of study, your curriculum, and your students. To get started, consider these ideas: Visual Arts •Art period classification •Colour theory •Lighting analysis •Composition comparison Health & Community Studies •Pharmacy compounding 11
•Behaviour theory •Graph analysis •Case scenarios Science & Technology •Equations •Theoretical physics •Derivative interpretation •Patent study Business Administration •Financial statements •Economic theory •Statistical analysis •Case Study Humanities & Social Sciences •TOEFL test •Literary theory •Poem analysis •Adaptation comparison
Merits Multiple choice tests can be a good way to test memory recall. A multiple choice quiz may be necessary when there is an accreditation requirement. Recall tests can provide the student and the professor with meaningful feedback about what still needs mastery. In addition to recall, a multiple choice quiz can also have higher order problem solving type questions. Finally, multiple choice quizzes can be quick and easy for grading. Accreditation. Multiple choice tests reflect the reality of many accreditation exams. For programs that have accreditation requirements related to quizzes and tests, we must use these types of examinations. Multiple choice tests can be good way to measure recall. Remembering is the first order in Bloom’s taxonomy (Anderson & Krathwol, 2001). Likewise, in the SOLO taxonomy, developed by Biggs and Collis, learners identify, name, list, and recall items or tasks at the unistructural level (Biggs & Tang, 2011). Higher Order Thinking. While short quizzes may be good for testing recall, a course built around content memorization is not as effective as one built on experiential learning, applied projects, and community involvement. For an assessment “to tap higher-order 12
cognitive processes,” students must do more than simply memorize (Anderson & Krathwohl, 2001, p. 71). Even a multiple choice test can have a complicated case study question, an excerpt from a play to analyze, or a statistical chart to interpret. We can have multiple sets of questions for one case scenario, poem, or graph. These questions can require students to implement, apply, analyze, interpret, compare and contrast, or evaluate course content. Formative Assessment. When we use quizzes for formative assessment, they contribute to the “cycle of studying, testing, and feedback” that leads to mastery learning (McGuire & MacDonald, 2009). Consider allowing students to retake a formative test multiple times as part of the learning process. Use review and practice tests to prepare for a cumulative final exam. Take up quizzes, to make sure students understand questions they missed. Allow open book and collaborative quizzes. Open Book Quizzes. Not only do students prefer open book quizzes, but they also do slightly better on open book quizzes. They have less anxiety with open book or a study aid sheet (Gharib, Phillips, & Mathew, 2012). An open book quiz can actually be quite difficult with well-crafted questions. They are also authentic. In the real world, we all sometimes use a dictionary, search the Web for a fact, or consult a reference book to do our job. Collaborative Quizzes. Alternatively, students can work collaboratively on a quiz. The quiz can be recall or open book. It can even be a game or competition between teams! Collaborative quizzes improve engagement. However, they make no difference on final exam grades (Slusser & Erikson, 2006) (Berg, Plovsing, & Damgaard, 2012). Quick Feedback. With scanned response sheets like Scantron grading is quick and easy, especially in large courses where there is a need to broadly cover topics. “Wellconstructed” and “well-written” multiple choice test questions can yield reliable test scores on par with constructed-response questions (DiBattista & Kurzawa, 2011). Multiple Choice tests also provide quick feedback to the professor about which topics students are struggling with and what to review. While we cannot use pop quizzes for final grades or summative assessment, we can use a pop quiz to gauge what students remember as a diagnostic needs assessment. With Scantron, we can get a detailed report about which questions students missed so that we know what to review or what questions might not have been well-written.
13
Challenges Writing a high quality multiple choice test is difficult. It takes time to sit down and craft wellwritten and relevant questions. For an electronic quiz, we may also want to draft hints and feedback responses. Maintaining integrity for a quiz may be paramount for accreditation. To effectively out smart cheaters, we may need to have multiple versions of quizzes, randomized question sets, and proctored exams. Writing Quality Questions. Unfortunately, most professors do not write high quality questions for multiple choice tests (Asim, Ekuri,& Eni, 2013; DiBattista & Kurzawa, 2011). DiBattista and Kurzawa found that almost half of all test questions are discriminatory due to poor item shells or distractors. A poorly written test question lacks reliability and validity in terms of measuring student recall or ability. They also advocate that professors receive training on how to write good test questions as large scale multiple choice testing will likely continue to be prevalent in large institutions (DiBattista & Kurzawa, 2011). Writing Relevant Questions. Questions should also be applicable to the field of study. According to Mancini (2013), avoid writing test questions like quiz show trivia. Instead, anchor questions in the career experience. For example, a travel sales question about the biggest lake in Ontario is too general. A better question would be in which city might you book a guest who wants to sail the Thousand Islands. Question Format. A good multiple choice test uses question format rather than sentence completion, using item shells with “hollow” phrases in the stem such as “which best defines x?” or “which is the cause of x?” (DiBattista, 2011, p.10; Haladyna, 2004). Create distractors from the most common errors that students make; these distractors should be plausible in “content, length, and complexity”. More than one question may even be true but not necessarily correct. Avoid negative wording, patterns in “length and location of correct answers”, and phrases such as “none of the above” or “all of the above” (DiBattista, 2011, p.2). Make sure spelling and grammar are correct, and balance the position of answers to the same number of times (DiBattista, 2011). Giving Feedback. It is important to provide students with feedback on their quiz. Not only their grade, but also where they missed questions and what they need to do to master the learning. Consider using the feedback tools in the learning management system (LMS) to provide answers to students or to use the quiz as a formative teaching and learning tool. An electronic quiz can give immediate feedback to a student with hints or even the correct answer. If you build a quiz with a large bank of questions, students can keep practicing with
14
randomized questions until they grasp the content. Again, giving feedback is part of the cycle of learning that leads to mastery (McGuire & McDonald, 2009). Cheat Proof. To avoid cheating, an online test can have limiters and randomization. Use a tool like Respondus 4.0 to convert paper exams to a learning management system (LMS) or to print an online exam from an LMS. Use a tool like Respondus Lockdown Browser, to prevent students from opening a new browser window or Respondus Monitor for automated online proctoring. For more security, we can require that students write tests in a proctored test centre. There are even online proctored exams where the proctor observes the student during the test via web conferencing; for an example, SmartServe uses secure online testing for their certification. Most professors take up tests, but keep the originals, and rotate versions, to prevent students sharing the exam with other cohorts.
Instructional Design Use a backwards design approach. First, start with your learning outcomes. Second, create your assessments. Third, plan your instructional activities. Finally, choose a technology to enhance the lesson. Design. Consider your curriculum; look carefully at your accreditation standards, graduate attributes, program learning outcomes, course learning outcomes, and lesson learning outcomes. Will a multiple choice quiz fulfill the learning outcomes? How will you prepare students for a quiz? What technologies will you and your students need to learn in order to take an online quiz? Will quizzes enhance the learning experience? Develop. Review the kinds of tests and exams used in your field of study. Reflect on your own testing experience in your academic and professional career. Research what kinds of test questions are common. Look at what types of technologies you can use to build quizzes. Use the quiz tools in the learning management system (LMS). There are also drag and drop, image, and audio type quiz formats that would appeal to a variety of learners like kinaesthetic and visual as well as auditory or musical-rhythmic learners. Write your multiple choice test questions. Draft a higher order thinking question that targets problem solving, analysis, or interpretation; the question should express “a single problem” “related to significant content in the course” (Weimer, 2014). Write the correct answer. Think about common student errors, and draft 2-4 distractors - wrong but believable answers. Make sure that all distractors have a similar length, grammar, and content as well as proper grammar and spelling. When you have finished all of your
15
questions, take your test and ensure that the correct answers appear randomly without a pattern (DiBattista, 2011; Haladyna, 2004; Weimer, 2014). Format 1.Use an item shell - a hollow phrase with a syntactic structure but no content 2.Use a question stem - question format 3.Use a keyed answer - the best answer 4.Use one to three distractors - plausible and possible answers Item Shell What is the relationship between X and Y? Hollow Phrase Which grass X is resistant to which disease Y ? Stem Tall fescue grass is resistant to Dollar Spot (Sclerotinia homoeocarpa) Question Stem Which grass type is resistant to Dollar Spot ? Keyed Answer tall fescue Plausible Distractors a. creeping bentgrass b. fine fescue c. perennial ryegrass d. tall fescue Tips • Use a verb in the stem • Balance length and location of answers • Use proper grammar and spelling • Avoid negative wording (cognitive overload) • Avoid generalizations “always”, “never”, “only”, “all” • Avoid “All of the above” • Avoid “None of the above” • (DiBattista, 2011; Haladyna, 2004; Weimer, 2014) 16
Deliver. Decide what students will do before, during, and after the quiz. Before the quiz, students might read an article, listen to a podcast, or watch a video. During the quiz, students may use open book, work collaboratively, use a lockdown browser, or sit in a proctored environment. After the quiz, students should receive feedback on their answers to learn from the missed questions. The quiz should be authentic, active, and applied.
Technology Scantron
Quizlet
Apple iBooks Author
We can build quizzes in the learning management system (LMS) or use external tools like Respondus to import into the LMS. Some rapid authoring tools like Adobe Captivate and TechSmith Camtasia Studio can incorporate interactive quizzes. Likewise, books made with Apple iBooks Author can have interactive quizzes including drag and drop image type quizzes. To help students prepare for tests, consider online tools that students can use to organize text, image, and audio into study notes. Popular clicker technologies include Kahoot and Socrative which work with most mobile devices and allows students to interact in polling type questions or quizzes. We can also use Scantron to quickly score paper tests.
References Anderson, L.W. & Krathwohl D.R. (eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York, NY: Longman. Asim, A. E., Ekuri, E. E., & Eni, E. I. (2013). A diagnostic study of pre-service teachers' competency in multiple-choice item development. Research In Education, 89, p.13-22. Berg, R. G., Plovsing, R. R., & Damgaard, M. (2012). Teaching baroreflex physiology to medical students: A comparison of quiz-based and conventional teaching strategies in a laboratory exercise. Advances In Physiology Education, 36(2), 147-153. Biggs, J. & Tang, Catherine. (2011). Teaching for quality learning at university (4th ed.). Berkshire, UK: McGraw-Hill. DiBattista, D., & Kurzawa, L. (2011). Examination of the quality of multiple-choice items on classroom tests. Canadian Journal For The Scholarship Of Teaching And Learning, 2(2).
17
DiBattista, D. (2011). Designing effective multiple-choice questions. Proceedings from the 6th Annual Curriculum Development Affinity Group (CDAG) Conference, Niagara College, Niagara Falls, ON. DiBattista, D. (2010). Tips for constructing multiple-choice items. Scarborough, ON: Nelson Education Teaching Advantage (NETA). Gharib, A., Phillips, W., & Mathew, N. (2012). Cheat Sheet or Open-Book? A Comparison of the Effects of Exam Types on Performance, Retention, and Anxiety. Psychology Research, 2(8), p.469-478. Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed). Mahwah, N.J.: Routledge. Mancini, M. (2013). Selling destinations: Geography for the travel professional (5th ed.). [Instructor’s Manual]. USA: Nelson Education Ltd. McGuire, M. J., & MacDonald, P. M. (2009). Relation of early testing and incentive on quiz performance in introductory psychology: An archival analysis. Journal Of Instructional Psychology, 36(2), p. 34-141. Slusser, S. R., & Erickson, R. J. (2006). Group quizzes: An extension of the collaborative learning process. Teaching Sociology, 34(3), p.249-262. Weimer, M. (2014, March 5). Tips for writing good multiple-choice questions. Faculty Focus. Retrieved from http://www.facultyfocus.com/articles/teaching-professor-blog/writing-goodmultiple-choice-questions/
18
EMERGING TRENDS COLLECTION
Paula Ogg © 2020 Photography by Jonathan Eger