The culture of assessment is in full swing. Its purpose is, in fact, quite wonderful. Namely, we should articulate what we hope to accomplish, we should look into how well we are accomplishing it and into what factors are helping or hindering us, and then we should improve our practice to better accomplish our aims.

But it seems like some of those involved in assessment miss why we need assessment and just know that we do. This results in “assessments” that are disconnected from any meaningful purpose, such as understanding how well we and our students are doing and improving our teaching practices. It is often as if generating rubrics and numbers were ends in and of themselves.

I wish that more people involved in assessment would read some of the scholarship on assessment and on rubrics. I suggest one excellent article in particular: Chris Anson et al.’s “Big Rubrics and Weird Genres: The Futility of Using Generic Assessment Tools Across Diverse Instructional Contexts,” The Journal of Writing Assessment 5.1 (Jan. 2012).

Standardized rubrics are prized by some in the assessment movement because they are good for generating numbers. On both theoretical and empirical grounds, however, Anson et al. argue against the use of standardized rubrics across different contexts, whether across an entire program, across different types of assignments in different classes, or across different sections of the same class taught by different teachers.

What’s wrong with standardized rubrics? To begin with, standardized rubrics have some detrimental effects, such as inhibiting the pedagogical creativity, freedom, and investment of individual professors. As a case in point, Anson et al. note how it is important for teachers to be able to use “weird genres,” that is, “mixed, hybrid, invented, and creatively orchestrated assignments [that] may call for unique or blended genres representing . . . localized pedagogical goals.” Standardized rubrics rule out the use and ongoing development of such assignments.

But, one might ask, would it be worth giving up some flexibility for the benefits of standardization, such as the confidence of knowing that everyone is doing the same thing? No. First, that benefit in itself is not all that valuable. More importantly, standardized rubrics only produce the appearance of standardization while concealing the actual variety of practice.

In the study, Anson et al. find that different faculty do different things under the same labels (i.e. “research paper”), look for different criteria using the same terms (i.e. “organization” or “analysis”), and connect the actual details of student work to those criteria in different ways.

But faculty can and should still collaborate in authentic kinds of assessment. Instead of standardized rubrics, Anson et al. recommend having professors who are teaching a common course or major meet to discuss criteria and standards in the context of desired outcomes and concrete pieces of student work.