Quality and Innovation

According to dictionary.com, one of the definitions for rubric is “any established mode of conduct; protocol.” But the context you’ve probably heard this word in is education – where a grading rubric or a scoring rubric is used to evaluate a complex artifact like a student essay.

In my opinion, it’s time to move the concept of the rubric from the classroom into the mainstream, because it can be applied as a very practical general purpose quality tool! (Hear that, Nancy Tague? I think you should write about rubrics in your next edition of the very excellent book The Quality Toolbox. Let me know if you’d like me to help make this happen.)

A rubric is basically a grid with 1) levels of performance indicated along the top row, and 2) criteria or dimensions of performance listed down the leftmost column. Each cell of the grid contains a descriptive statement that explains how the level of performance in that column might be achieved for a specific dimension:

For example, here’s a rubric that one group constructed to evaluate the quality of the mind maps that they were producing. The performance levels are organized from high performance in the top left (smiley face giving a thumbs up) to low performance in the top right (smiley face that looks like he’s about to pass out):

The dimensions of performance are neatness and presentation, use of images/symbols, and use of color. The descriptive statements in each cell provide specific examples of how the performance level might be achieved, e.g. “has failed to include color in the mind map” is an indicator of a low performance level for the dimension of “use of color” – which is very understandable!

The concept of the rubric as a performance assessment tool is relatively new! Griffin (2009), in a brief history of the rubric, notes that since its introduction in 1981, “the scoring rubric has evolved into a more precise, technical, scientific-looking document. It carries a tone of certainty, authority, and exactitude.” However, she notes, the utility of a rubric will depend upon the thought and consideration that goes into its construction. “A rubric is a product of many minds working collaboratively to create new knowledge. It will, almost by definition, be more thoughtful, valid, unbiased and useful than any one of us could have conceived of being as we worked in isolation.”

Advantages of applying a well developed rubric include:

• Provides a common language for sharing expectations and feedback
• Helps to clarify and distinguish the differences between various performance levels
• Helps to focus an individual or group’s ATTENTION on relevant aspects of each desired quality characteristic or skill area
• Provides a mechanism to more easily identify strengths and opportunities for improvement
• Helps lend objectivity to an evaluation process that might otherwise be subjective

Disadvantages:

• Different rubrics may need to be devised for the different activities or artifacts that are to be evaluated using the rubric
• Not all evaluators will apply the rubric in exactly the same way – there is a subjective element at work here – so people may need to be trained in the use of a rubric, or perhaps it would be more effective in a group consensus context where inter-rater variability can be interactively discussed and resolved
• Creating a rubric can be time consuming
• The rubric may limit exploration of solutions or modes of presentation that do not conform to the rubric

Using Rubrics for Quality Improvement

Rubrics are already applied in the world of quality, although I’ve never heard them go by that name. The process scoring guidelines for the Baldrige Criteria are essentially rubrics (although the extra dimension of ADLI and LeTCI has to be considered in the mind of the examiner). The International Team Excellence Award (ITEA) criteria in the Team Excellence Framework (TEF) also forms a rubric in conjunction with the performance levels of missing, unclear, meets expectations or exceeds expectations.

I see a lot of ways in which rubrics can be developed and applied in the quality community to help us establish best practices for some of our most common project artifacts, such as Project Charters. Nancy Tague includes a Project Charter Checklist in The Quality Toolbox to help us create better and more complete charters… but what if we added a second dimension, which includes performance levels, and turned this checklist into a rubric? Any checklist could be transformed into a rubric. Furthermore, to develop a good rubric, we can brainstorm and rank all of the potential criteria in the left hand column, using a Pareto chart to separate the vital few criteria from the trivial many.

Are any of you already using rubrics for purposes outside training or education? I would love to start a list of resources to share with the quality community.

Reference: Griffin, M. (2009). What is a rubric? Assessment Update, 21(6), Nov/Dec 2009.

Note: There is a comprehensive site containing many examples of rubrics at http://www.web.virginia.edu/iaas/assess/tools/rubrics.shtm – however, they won’t open in Google Chrome.

For the past few months, I’ve been working on an article to describe and define quality consciousness. Someone recently told me that there have been a lot of people asking about this concept lately (which I find really cool because as far as I know, I’m the only one actively studying it under this banner), and that I should blog about what quality consciousness is ahead of the publication. (That said, if you’re also researching quality consciousness, let me know in the comments section below! Let’s play with this idea together.)

So here’s a synopsis of the story of quality consciousness:

• The existential question that motivated this line of inquiry: If ISO 8402:1994 says that quality is the “totality of characteristics of an entity that bear upon its ability to satisfy stated and implied needs,” then what if that entity is YOU? What are the totality of characteristics of YOU that bear upon YOUR ABILITY to satisfy the stated and implied needs of your stakeholders?
• The term “quality consciousness” was first used, from what I can find, in a 1947 keynote by C.R. Sheaffer to the first convention of the American Society for Quality Control (ASQC), the predecessor to ASQ. To answer the question “what does top management expect from quality control [people and organizations]” he notes that a change in quality consciousness is expected. Attitudes must shift from an acceptance of what’s good enough to the constant pursuit of making things better. People must be able to take pride in their high-quality work. (from Borawski, 2006)
• Consciousness, according to the Random House dictionary, is 1) awareness of one’s own thoughts feelings, and surroundings, 2) the full activity and engagement of the senses, and 3) the thoughts and feelings of individuals and groups.
• Based on this definition, I believe that quality consciousness can be summed up by the “3 A’s” – Awareness, Alignment, and Attention. Quality consciousness implies awareness of yourself and the environment around you (including what constitutes quality and high performance for people, processes and products – most importantly, YOU). It also suggests that you must achieve alignment of your consciousness with the consciousness of the organization, which will aid in full activity and engagement of the senses. Your attention must be selectively focused onto what you can accomplish in the present moment according to that alignment (which implies that you are able to effectively filter the rapid and voluminous streams of information coming at you).
• From reviewing the literature, I find that there are four elements that contribute to developing awareness, finding alignment, and focusing attention. These are Action, Reflection, Interaction, and Education. I’ll go into more detail in the article on how these are all related.
• I think that quality consciousness is exactly what Deming was after… and that it’s the moral of the story of his 14 points. But whereas the unit of analysis for his 14 points was the organizational level, we need to internalize those points within ourselves. What if Deming’s 14 points were geared towards YOU developing your quality consciousness… what do you think he would have said differently?
• The absence of focus on developing a quality consciousness is, I believe, the distinguishing factor between companies that have implemented the Toyota Production System successfully (ie. Toyota) and companies that have implemented the Toyota Production System with limited results (e.g. pretty much everyone else).
• A personal path for developing quality consciousness might include asking yourself the following questions: What do YOU need to expand your awareness? To enhance your mood and affect so that you’re aware of the vast landscape of innovative potentials available to you (e.g. http://qualityandinnovation.com/2011/09/29/why-positive-psychology-is-essential-for-quality/)? What do YOU need to align yourself with your organization? What do YOU need to be able to focus your attention on the most productive thing you can do at any given moment – resulting in effortless action, optimal flow and productivity, and positive affect that will cycle back to expanding your awareness even more?

Borawski, P. (2006). The state of quality: 1947 and 2006. Journal for Quality and Participation, Winter 2006, p 19-24.

(Image Credit: Doug Buckley of Hyperactive Multimedia at http://www.hyperactive.to)

I call this Part I not because I know what Part II will be… but because I know there will be at least one followup to this post sometime in the future.

Last weekend I finished preparing the report that will be used for my Annual Performance Appraisal (or whatever they’re calling it). To do this, I had to reflect on (and in some cases, remember) all the contributions I made to my job and to society between May 15, 2010 and May 14, 2011. Our department head will use this report, and presumably his interactions with me over the year, to determine whether I’m worthy and where I need to improve. As a result, I’ve been thinking about the nature of performance appraisals over the past few days.

One of Deming’s “Seven Deadly Diseases” is the practice of performance appraisals, which naturally includes merit reviews and annual reviews. Such reviews are ubiquitous in most industries, and often, pay raises are tied to the results. Humorously, at my last job, the HR department routinely ensured the workforce that our performance reviews were NOT in any way related to our pay increases – even though as a manager, I was aware of many cases where the information did indeed influence the numbers (consciously or subconsciously).

I believe, from reading Deming’s books and a whole host of journal articles on Deming-related topics, that he had two reasons for feeling this way: 1) performance appraisals are anathema to “driving out fear,” one of his 14 points, and 2) the practice of performance appraisal assumes the duality of manager vs. employee; (s)he who wields power vs. (s)he who does not; the active evaluator vs. the passive evaluated. By instituting a performance review process, we are establishing a power structure whether we want to or not. Progressive managers might use the performance review as an opportunity for two-way dialogue and a cooperative exploration of strengths and opportunities for improvement. Progressive organizations might use a 360-degree approach, a la Jones & Bearley, but the underlying dynamic is the same: I’m telling you what I think about you and that’s my evaluation. I’m not familiar with any managers or organizations who can pull this off with impartiality and avoid the many sources of bias that can creep into the process.

What if I’m just a really bad observer? What if I’m observing you based on the wrong criteria – or worse, criteria that I just don’t have the experience to honestly evaluate you against? According to quantum physics, there’s no such thing as an observer. The presence of an observer makes him or her a participant in the dynamics – and thus outcomes – of the system. There’s no way to sit outside the system and just watch without affecting what happens.

So what’s the solution? Here’s my idea for today, and I don’t think it’s that revolutionary: 1) Each individual should actively take responsibility for his or her own performance to standards and performance improvement (yet another one of Deming’s 14 points), and 2) Teams should engage in continuous dialogue about individual and collective performance meaning that everyone is responsible for making sure the following questions are asked repeatedly by every member of the team:

• What are we trying to accomplish together?
• How are we doing? Are we making progress, and if so, what should we keep doing?
• If so, what’s working right? If not, what can we do to fix it?
• Then commit to doing whatever it is you decided to do.

Like I said, not really revolutionary. The only new idea here is that the individual should aggressively manage his or her own contribution to the objectives of the team or organization, seek out others’ opinions of his or her strengths and weaknesses, seek out quality standards to measure one’s self by, and take responsibility for helping the team ask the important questions above. If the individual is responsibly managing his or her own performance and demonstrating continuous improvement, the system of performance appraisals becomes moot.

It turns into a pull system rather than the push system than it is now.

The obvious problem? Not everyone cares about their own individual continuous performance improvement. As a manager for over a decade, I’ve had employees who were intellectual lazy, physically lazy, intellectually weak and trying hard to cover it up, or just in the job for the paycheck so they would do anything to avoid doing real work. Performance appraisals serve the purpose, in my opinion, of forcing dialogue with these employees who otherwise might not talk about their performance at all. Sometimes it works, sometimes it doesn’t. In many cases managers and employees trudge through the performance review process on autopilot. It becomes a drawn out paper exercise; an infrequent opportunity to shield a company from liability in case a disgruntled employee feels they’ve been wrongly terminated.