Transforming Testing at the University of Rhode Island College of Pharmacy
The University of Rhode Island College of Pharmacy (URI) in Kingston, Rhode Island, offers a Doctor of Pharmacy or Pharm.D. program, as well as bachelor, master, and doctoral level degrees in the pharmaceutical sciences. It has more than 700 enrolled students, including 572 students in the Pharm.D. program, alone. In 2011, Pharm.D. graduates had a 100% pass rate for first-time takers of the NAPLEX and MPJE exams. The school has a demonstrated commitment to advancing student education, research, and effective administration.
URI had historically used paper exams, with Scantron answer forms, as the primary method of assessing its undergraduate students. However, this long-time method posed a number of significant challenges, which prompted two faculty members, Dr. Jayne Pawasauskas and Dr. Kelly Matson, to lead an effort to investigate and implement computer-based testing.
According to their analysis, the paper-based system not only created inconvenience and consumed precious time, but it also presented substantial financial costs. Faculty had to “lug papers across campus” to have them scored on a central campus Scantron machine, wait for an available time-slot to scan the answer forms, manually fill out answer keys, fix answer key mistakes, and redo the entire process if an issue arose (e.g. if post-exam analysis demonstrated there was a problem with a question or the accepted responses, or if a student’s form was not completed correctly).
Moreover, to preserve proper documentation, the school required faculty to store all exam results for at least two semesters after each exam session. According to Dr. Pawasauskas, “Clinical faculty don’t all have clean, dedicated workspace, so storing so much paper becomes really difficult – and costly.” Dr. Pawasauskas and Dr. Matson estimated that the costs associated with paper exams totaled approximately $30,000 for roughly 300 students.
The professors were also very concerned about security issues associated with paper testing. Dr. Pawasauskas said, “Students look at each other’s papers, but the problem is you can’t really prove they did anything unless a second proctor sees the same thing.” Adding proctors, of course, adds expense. “Even then, proctors are naturally concerned about the ramifications of pulling a paper away from a student … and what if they’re wrong.”
A core group of faculty implemented the ExamSoft software system and identified very substantial benefits. The system allows faculty to create and store both items and exams in an electronic format, within an item bank, and deliver exams securely on student-owned laptops. Pre-exam paper handling is eliminated, and so is the need to manually process, re-process or store answer forms after the exams. Grading is instantaneous and re-grading is done with the click of a mouse.
The faculty’s logistical issues were addressed, but just as importantly, exam security was enhanced. “One of the things I love about the software,” said Dr. Pawasauskas, “is proctors don’t need to pull papers away from students at the end of an exam to prevent them from exceeding the allotted time (for finishing their exam), and they don’t need to make undocumented allegations.”
The software includes an extremely extensive set of cheating prevention measures, and if a proctor suspects any improper student behavior, administrators or faculty can review post-exam auditing reports and tools to investigate their suspicions.
“We felt there had to be a better way to assess students’ strengths and weaknesses, while reducing the potential for cheating…We found it.”
– Dr. Jayne Pawasauskas
Why is it important that ExamSoft doesn’t require Internet connectivity during testing? “Internet connectivity is the #1 concern for students. Your solution came to the front because we wouldn’t have to rely on internet connectivity during the exam.”
How are you setting up categories within ExamSoft?
“We created categories within the ExamSoft system based on ACPE, but we know we can really expand these to do a lot more.”
What were the challenges you experienced providing exam feedback to students before using ExamSoft?
“With the paper based testing, it seemed the test results were very freestanding – you’d get an email with exam results, and you could export and manipulate the data. However it didn’t really make it easy to help students identify specific strengths and weaknesses – over time.”
What do ExamSoft’s security features mean to you?
“If I can stop worrying about potential cheating, I can focus 100% of my efforts on teaching better.”
How has ExamSoft made the exam process easier for you?
“ExamSoft allowed me to give my students extensive feedback on exam questions that I previously would have had to go through an exam review during class time and/or office hours.”
How does ExamSoft help you with student assessment and remediation?
“By modeling our exam question categories on ACPE curriculum guidelines, we will be able to provide longitudinal assessment and remediation for our students, as well as help meet accreditation standards for our college in the future.”