Quality and Innovation

According to dictionary.com, one of the definitions for rubric is “any established mode of conduct; protocol.” But the context you’ve probably heard this word in is education – where a grading rubric or a scoring rubric is used to evaluate a complex artifact like a student essay.

In my opinion, it’s time to move the concept of the rubric from the classroom into the mainstream, because it can be applied as a very practical general purpose quality tool! (Hear that, Nancy Tague? I think you should write about rubrics in your next edition of the very excellent book The Quality Toolbox. Let me know if you’d like me to help make this happen.)

A rubric is basically a grid with 1) levels of performance indicated along the top row, and 2) criteria or dimensions of performance listed down the leftmost column. Each cell of the grid contains a descriptive statement that explains how the level of performance in that column might be achieved for a specific dimension:

For example, here’s a rubric that one group constructed to evaluate the quality of the mind maps that they were producing. The performance levels are organized from high performance in the top left (smiley face giving a thumbs up) to low performance in the top right (smiley face that looks like he’s about to pass out):

The dimensions of performance are neatness and presentation, use of images/symbols, and use of color. The descriptive statements in each cell provide specific examples of how the performance level might be achieved, e.g. “has failed to include color in the mind map” is an indicator of a low performance level for the dimension of “use of color” – which is very understandable!

The concept of the rubric as a performance assessment tool is relatively new! Griffin (2009), in a brief history of the rubric, notes that since its introduction in 1981, “the scoring rubric has evolved into a more precise, technical, scientific-looking document. It carries a tone of certainty, authority, and exactitude.” However, she notes, the utility of a rubric will depend upon the thought and consideration that goes into its construction. “A rubric is a product of many minds working collaboratively to create new knowledge. It will, almost by definition, be more thoughtful, valid, unbiased and useful than any one of us could have conceived of being as we worked in isolation.”

Advantages of applying a well developed rubric include:

• Provides a common language for sharing expectations and feedback
• Helps to clarify and distinguish the differences between various performance levels
• Helps to focus an individual or group’s ATTENTION on relevant aspects of each desired quality characteristic or skill area
• Provides a mechanism to more easily identify strengths and opportunities for improvement
• Helps lend objectivity to an evaluation process that might otherwise be subjective

Disadvantages:

• Different rubrics may need to be devised for the different activities or artifacts that are to be evaluated using the rubric
• Not all evaluators will apply the rubric in exactly the same way – there is a subjective element at work here – so people may need to be trained in the use of a rubric, or perhaps it would be more effective in a group consensus context where inter-rater variability can be interactively discussed and resolved
• Creating a rubric can be time consuming
• The rubric may limit exploration of solutions or modes of presentation that do not conform to the rubric

Using Rubrics for Quality Improvement

Rubrics are already applied in the world of quality, although I’ve never heard them go by that name. The process scoring guidelines for the Baldrige Criteria are essentially rubrics (although the extra dimension of ADLI and LeTCI has to be considered in the mind of the examiner). The International Team Excellence Award (ITEA) criteria in the Team Excellence Framework (TEF) also forms a rubric in conjunction with the performance levels of missing, unclear, meets expectations or exceeds expectations.

I see a lot of ways in which rubrics can be developed and applied in the quality community to help us establish best practices for some of our most common project artifacts, such as Project Charters. Nancy Tague includes a Project Charter Checklist in The Quality Toolbox to help us create better and more complete charters… but what if we added a second dimension, which includes performance levels, and turned this checklist into a rubric? Any checklist could be transformed into a rubric. Furthermore, to develop a good rubric, we can brainstorm and rank all of the potential criteria in the left hand column, using a Pareto chart to separate the vital few criteria from the trivial many.

Are any of you already using rubrics for purposes outside training or education? I would love to start a list of resources to share with the quality community.

Reference: Griffin, M. (2009). What is a rubric? Assessment Update, 21(6), Nov/Dec 2009.

Note: There is a comprehensive site containing many examples of rubrics at http://www.web.virginia.edu/iaas/assess/tools/rubrics.shtm – however, they won’t open in Google Chrome.