What Happens When Assessment Affects The Bottom Line

March 6, 2014 Beth Donahue

Why do teachers test their students? Traditional reasons for testing are largely summative in nature–testing is done at the end of the learning process to measure student achievement and assign grades or ranks. In more recent years, researchers have found that tests are far more versatile than traditionally supposed. Tests can drive student achievement, not just measure it. The research shows that short-term formative testing (within and between lessons) is one of the most cost-effective ways to increase student achievement.

Legal educators today are aware of this research and increasingly interested in how they might incorporate more formative testing into their courses. But they remain hesitant, and their hesitation typically comes in the form of one of the following two “objections”: The first objection is that it would be far too time consuming to grade essays every week, and the idea of multiple-“guess” questions… well, perish the thought! I plan to address this objection at length in my upcoming webinar. The second objection goes something like this: “I understand why teachers need to give little kids constant feedback, but my students are adults and need to be responsible for learning the material without constant hand-holding along the way.”

My answer to this second objection is that the embrace of formative testing goes far beyond education, and is in fact the most radicalized in professions, like law, where systems must process different (and sometimes unexpected) inputs and contexts. You don’t just teach “the law,” after all. You teach your students to “think like lawyers.” You want each one of them to be able to process different client questions (inputs and contexts) within a cognitive system that you help them build.

What I’m describing here sounds similar to someone writing a computer program, and while students and their ability to process knowledge are far more complex than any program a software developer might write, there are some parallels. One of the most interesting to me is the growing use of automated quality-assurance testing in software development during the last decade. Software development teams have moved testing from the end of the project to the beginning, sometimes even before any actual “code” is written. They’ve done so not because they adhere to any assessment-related dogma, but because they’ve observed that testing a system early and often improves a company’s bottom line both by cutting costs and increasing quality. This is the ultimate win-win change in the real world of business. As the chart below shows, the earlier flaws can be detected in an information system, the cheaper it is to fix them:


This makes intuitive sense. The more code you add after a flawed line of code or an erroneous assumption, the more dependencies and links you’ll need to rewrite or redesign once you discover the flaw. What could have been a quick fix can turn into a major headache as the system matures, and major headaches cost more money to fix. The process of testing early in the development life-cycle leads to better systems, too. That is, although the coding stage of development takes 15-35% longer because it incorporates regular testing, the system will have 60-90% fewer flaws than a similar system built by a team that delayed testing until after the code was completed. http://research.microsoft.com/en-us/news/features/nagappan-100609.aspx

So what does this mean for law professors interested in formative testing? The issue is whether what you’re teaching is something significantly more complicated than a store of knowledge that can be memorized and corrected at any time, without escalating costs, if an error is found. If what you’re teaching involves not just comprehension of readings done in class but the sophisticated transfer of those readings to other kinds of problems through complex connections and reasoning, then you should test your students early and often. A misconception that is built into a student’s “codebase” could cost her 30% of her grade if it isn’t detected until the final exam, and the costs for her and her clients will multiply if the error persists beyond law school and into practice. In contrast, the same misconception caught and corrected at the time a concept is introduced in class will have only a negligible effect on the overall cost and quality of her ability to “think like a lawyer,” which is your bottom line.

About the Author

Beth Donahue

Beth Donahue is a former editor and program director of the Multistate Bar Examination, as well as a former manager of a large technical team developing online assessment programs for K-12 standardized tests. She is the co-founder of BarCrowd LLC, a company that develops crowd-sourced assessment software for use as a teaching and learning tool in law schools.

More Content by Beth Donahue
Previous Article
Multiple-Choice Questions. Don’t We Have More Important Things to Worry About?
Multiple-Choice Questions. Don’t We Have More Important Things to Worry About?

Law schools may not think it’s the right time to explore the tedious construction of multiple-choice questi...

No More Articles