Implementation of a new assessment process – From paper to data

March 20, 2015 Beth Ennis

Dr. Beth Ennis is an ExamSoft client, and graciously agreed to share her experiences with implementing and using our testing and analytics platform with our blog audience.

Somehow, I became the technology person in our department. I still don’t know how that happened, although I am a bit of a geek and love all things tech. I flipped classes, used apps, and tried to find ways to enhance the learning of students in our Doctor of Physical Therapy program. We were using a Scantron system for testing, but had issues with test security, and struggled with the amount of paper generated with three classes of up to 72 students.

While scanning blogs and tech boards, I noted a post on a testing system that did not require the students to be online during testing and was intrigued. As I dug further into the examsoft system, I thought it had possibilities. A brief discussion with my program director gave me the encouragement to investigate further. I attended webinars and conference calls with current users discussing their implementation and issues they had, discussed possibilities and cost with examsoft staff, and then began to discuss the possibilities with faculty.

Most of the faculty were intrigued with the possibility of maintaining a test bank online with running statistics on items. They were also excited about the ability to track student issues, course goals, and program goals, especially as this related to our accreditation process. We made the decision as a program to try it for a year, with several classes piloting for the fall, and the rest joining in during the spring and summer. Courses in both the first and second year were chosen to pilot, and the process of uploading questions, formatting exams and delivering content began. We created categories (or, an organized system of learning objectives we wanted to track), set up question and exam folders and subfolders, and moved forward.

Although there was great excitement and the pilot was successful, we found ourselves in the middle of accreditation and this stretched the adoption process over several semesters. We began in the fall of 2013 and are just getting the last of the courses into the system. Although we have a few faculty who are less comfortable with technology than others, the fact that we often team teach made this process doable – there was always one faculty member who was comfortable with using technology, and generally, they took charge of the question upload and exam formatting process. This made the process more palatable for those less comfortable with technology.

We did have some challenges with test deployment that needed to be overcome. Our students all have iPads, so policies were developed related to required preparation for the test, making sure downloads were done prior to class, and coming prepared with adequate space and power to take the test. Making sure faculty are comfortable with posting requirements, and upload procedures, as well as use of the crash code in case a student was accidentally booted from the test continue to be ongoing processes, as new security measures are deployed and new features are released.

examsoft-categorization-tree

However, we’ve already seen benefits from the system including test security, monitoring of student progress, and monitoring program goals. The randomization feature releases a different test to each student, accommodation features can be built into the system for students who require them, and we are able to use categories to make sure test content is distributed across the topics needed, course objectives are being addressed, and students are progressing in their ability to problem solve.

This is an ongoing process, which has recently included the addition of rubrics, and the ongoing changes can be a challenge for faculty to keep up with, but we are developing a video library housed on our LMS that complements the help features of the program to keep the faculty up to date with changes.

About the Author

Beth Ennis

Dr. Ennis is currently an Associate Professor at Bellarmine University, Department of Physical Therapy, and is involved in courses in Pediatrics, Teaching and Learning, Professional and Legal Issues and Technology. Research interests include use of technology in education, as well as using therapy to re-engage children with disabilities into the community.

Follow on Twitter More Content by Beth Ennis
Previous Article
The Challenges of Supporting BYOD Culture on College Campuses
The Challenges of Supporting BYOD Culture on College Campuses

BYOD on college campuses is a growing trend in higher education - but how can institutions support the grow...

Next Article
Infusing Technology in a School of Nursing – An Education Dive Article
Infusing Technology in a School of Nursing – An Education Dive Article

Infusing technology into a school of nursing, the decision to implement a computer-based testing software a...