The phrase “hindsight is 20/20” always seems to come up in the months following a recently launched project. We certainly had those moments after rolling out ExamSoft, but rather than focus on what could have or should have been done, we used these moments to learn how to improve the future of our college’s computer-based testing (CBT) procedures.
ExamSoft rolled out in January 2015 to a few of our courses and has since expanded greatly in our College of Pharmacy & Health Sciences. Our fall 2016 semester shows ExamSoft being used in 36 courses, many with multiple sections, across four programs. As one of two key ExamSoft administrators, I can attest to the challenges we faced rolling it out at a college encompassing multiple programs. More important are the solutions we developed to guarantee all our different programs could use ExamSoft effectively.
It’s important to recognize all the key stakeholders involved in CBT early on. Students and faculty were the most obvious; however, we soon identified other individuals who needed to be involved. To anyone at the beginning stages of ExamSoft use, consider asking yourself a few questions.
Will IT be involved? Are there any administrators or committees that should be aware of CBT growth? Collaborating with others in your college, as needed, promotes ideas on CBT from all angles.
Are there any facility or infrastructure factors to consider? Perhaps it’s worth getting those involved with scheduling/room placement to ensure your exam rooms support CBT-sufficient outlets, that there’s adequate space for laptops, etc.
Are there interested instructors who cannot yet use ExamSoft? Faculty buy-in is important, so anytime an instructor conveyed interest in ExamSoft (but their course was not yet approved for its use), we provided them with all we could to preserve that interest. Consider creating ExamSoft accounts on their behalf, so they can at least use the ExamSoft Portal for its question-banking benefit in the meantime.
As we defined our universal ExamSoft protocols, we recognized that providing consistent best practices was necessary. However, we also wanted to respect each program’s unique differences. In an effort to do so, it may be tempting to initially allow users to utilize the ExamSoft Portal however they deem appropriate. A better option may be to introduce a universal protocol from the beginning. It’s easier to collaborate with instructors later on to introduce improved practices than to break old (and potentially flawed) ExamSoft Portal habits.
Initially, we offered one-on-one trainings on an as needed basis, but soon we realized that the most efficient way to support all our new (and returning) users was to offer formalized training opportunities as often as possible. Increasing faculty buy-in and building a culture of CBT was reinforced by our new, multifaceted User Development Initiative. More details on this initiative are outlined in the second part of this blog series.
We also received great feedback on ExamSoft use from all our programs, but it could have been more efficiently tracked from its onset. Consider developing a system in which concerns, issues, or ideas on ExamSoft implementation can be tracked or shared. Perhaps developing a committee, promoting an online forum, or holding scheduled meetings focused on your college’s use of ExamSoft may be helpful.
New challenges will always arise in a college with many programs, but working with instructors and students of all disciplines will certainly help you better predict and solve CBT challenges that may lie ahead.
About the Author
Carla Hernandez is a Data Administrator with the Office of Assessment and a Key Examsoft Administrator in the College of Pharmacy and Health Sciences at St. John’s University in Queens, New York. Carla facilitates data management and exam administration analysis for the Health Sciences programs in the Department of Clinical Pharmacy Practice. Her research interests include higher education assessment, improvements in student learning, and quantitative statistical analysis of student learning.More Content by Carla Hernandez