Writing a good nursing test isn’t a one-time feat. It’s an ongoing process. A well-written exam provides useful assessment analytics that can benefit both students and the instructor. That data can help you shape better courses, improve your assessment measurement processes, and provide stronger feedback to students. Nursing faculty should therefore make it a habit after every test to analyze exam performance with a careful eye for improvements that can be made to better align the test with assessment goals.
Make sure the topics covered reflect what was taught in the classroom.
When constructing an exam meant to test the knowledge of what students have been learning during class, the number of questions assigned to each topic should correlate to the amount of time you’ve spent on each topic during the course.
To help get the right distribution, you can use the tagging feature in ExamSoft to stay on top of how many questions you have for each topic. ExamSoft allows you to create as many categories as you want, so use them liberally. Categorize all your questions according to the topics covered, and the exam blueprint will track the breakdown so that you can ensure they are all weighted appropriately for the exam.
Decide the best number of questions to use.
A lot of exam best practices can be generalized, but there’s no one right number of questions to use in every case. The total number of questions that should be on an exam will depend on the program and specific course being taught. In most cases, providing more exams with a higher number of items on each exam does increase the odds of a student passing the course.
Also, the higher number of questions a student is exposed to during the program, the more skilled they will become at taking exams, which will ultimately enhance their opportunity for success when taking state boards.
Include a mix of question types.
Any question that tests straight memorization is considered a low-level question in Bloom’s Taxonomy of the Cognitive Domain. Most students are great at answering these types of questions and frequently score 100 percent on item analysis because the questions do not require critical thinking skills. That’s not a problem, as long as they’re not the only questions you use.
Just because an item scores 100 percent does not mean it should be disposed of. You do want to set the expectation that your students need to memorize facts alongside the critical thinking skills you teach them. These facts will play an important role when they’re providing patient care, as long as they can combine them with the other crucial skills nurses need.
Evaluate question performance so you know which questions should be reused, and which should be updated.
When running an item analysis, you have the option of selecting Performance History. This statistic shows you how the performance of an item on the most recent exam compares to its performance when used on past exams.
If you have a high-level item that has usually scored 78 percent on item analysis and suddenly on your last exam scored 100 percent, you should consider that this item has been compromised and should be retired or revised. ExamSoft allows you to duplicate the question, so you can easily make changes to it rather than starting from scratch.
That same Performance History statistic can help you make informed decisions on the best questions to reuse for future tests. When you’re working on a new exam, look at the history and choose items based on their previous performance. Include a fair number of questions proven to be challenging, as well as some that are easy for the student to answer correctly.
ExamSoft item analysis also provides information on how many students selected each potential answer for an individual question so you can analyze the quality of your distractors. Consider why the student selected specific answers. If 55 percent selected the correct answer of B and 35 percent selected the wrong answer of C, perhaps the question needs revision so students can more clearly understand what you are asking. In that case, change the wording of the question and/or the distractors to provide clarity.
Use the analysis of question performance to make classroom improvements, in addition to test updates.
Sometimes the question performance will tell you about the quality of the question; other times it may provide insights into how well topics are being covered in your course. When examining the poor performance history of a question, do not instantly dispose of it. Instead, consider if the identified topic needs more coverage in the classroom. Review the lecture content and consider revisions. Perhaps you should revise the objectives to better guide students in their studies. Keep the question in use while you try improving the course, and only consider retiring it if it continues to perform poorly after all other options have been considered.
Identify which questions should be discarded entirely, and mark them accordingly.
Ultimately, some questions simply cannot be improved to the point of usefulness. If they are deleted from the question bank, a faculty member may make the same mistake of using the question (or a variant) again a couple of years down the line. Instead of deleting, consider adding a statement to the question, such as “see question performance history—do not use” or perhaps create a trash folder to move poor performing items from the question bank of well-performing questions.
Exam analytics don’t just tell us about a student’s performance; they tell us how well the exams are testing the knowledge we want them to cover and how successfully courses are covering the information being tested. Analytics will help nursing educators improve their courses and exams only if you take the time to review the available data and work to make the necessary changes.
About the Author
Sherri Graves, MNSc, RN, is a Clinical Instructor at the University of Arkansas for Medical Sciences College of Nursing Baccalaureate Program. Prior to accepting her first position as faculty in nursing education, she worked at Arkansas Children's Hospital for 18 years, 5 of those in Nursing Informatics training, testing, building, implementing, and supporting electronic health records projects.More Content by Sherri Graves