Software testing is both an art and a science. Throughout our teams at ExamSoft, we have different types of testers who execute different types of tests.
The first type of Quality Assurance (QA) engineer that we have focuses on testing automation. They create scripts that are run against our releases, simulating how our software is used. The goal of these automated tests is to ensure that our software is continuously tested against various hardware and operating system configurations. We also have QA testers who focus on manual testing—hands on keyboards covering both the functional and usability aspects of our products. Additionally, our operations team members execute regression tests, checking that both existing and new features are working together as designed before each release. Our teams run thousands of test scenarios against each release for all our products.
One of the challenges our product teams face is simulating the unknown—how our software is used in the day-to-day assessment processes across our customer base.
This month we wanted to share with you one of the ways we are simulating real-world product usage as we continue to fortify Examplify for Mac, Windows, iPad, and Surface Pro exam takers.
Earlier this year we expanded our large-scale testing initiatives from about 100 friends and family testers to more than 300 random individuals. The exam takers took a mock exam before arriving, then brought their own devices and registered on the testing day. As you can imagine, we saw every type of laptop, operating system, and anti-virus software at these testing events. Multiple tests were given at multiple locations across multiple days.
These proctored simulations expanded our testing beyond our development, automated testing, QA, and operations teams.
The testers split into groups so that we could test and observe the before, during, and after exam experiences for:
- long-running essay exams
- multiple exams with uploads within one testing session
- good, bad, and no Internet access
- multiple-choice exams with all the bells and whistles
These test sessions provided invaluable information from a software usability and performance perspective. This initiative has helped the team identify and resolve issues before the software is released and impacts your exam takers. Additionally, our QA team adds and adjusts our core testing suites as we “test and learn” at these simulations.
The next round of these large-scale tests just days away, and we are excited to observe the results from our latest releases and confirm our next set of product and development priorities as we continue to optimize the exam-taking experience.
About the Author
John-Paul Gaconnier is the Vice President of Product for ExamSoft.More Content by John-Paul Gaconnier