The Rubric as a General Purpose Quality Tool
According to dictionary.com, one of the definitions for rubric is “any established mode of conduct; protocol.” But the context you’ve probably heard this word in is education – where a grading rubric or a scoring rubric is used to evaluate a complex artifact like a student essay.
In my opinion, it’s time to move the concept of the rubric from the classroom into the mainstream, because it can be applied as a very practical general purpose quality tool! (Hear that, Nancy Tague? I think you should write about rubrics in your next edition of the very excellent book The Quality Toolbox. Let me know if you’d like me to help make this happen.)
A rubric is basically a grid with 1) levels of performance indicated along the top row, and 2) criteria or dimensions of performance listed down the leftmost column. Each cell of the grid contains a descriptive statement that explains how the level of performance in that column might be achieved for a specific dimension:
For example, here’s a rubric that one group constructed to evaluate the quality of the mind maps that they were producing. The performance levels are organized from high performance in the top left (smiley face giving a thumbs up) to low performance in the top right (smiley face that looks like he’s about to pass out):
The dimensions of performance are neatness and presentation, use of images/symbols, and use of color. The descriptive statements in each cell provide specific examples of how the performance level might be achieved, e.g. “has failed to include color in the mind map” is an indicator of a low performance level for the dimension of “use of color” – which is very understandable!
The concept of the rubric as a performance assessment tool is relatively new! Griffin (2009), in a brief history of the rubric, notes that since its introduction in 1981, “the scoring rubric has evolved into a more precise, technical, scientific-looking document. It carries a tone of certainty, authority, and exactitude.” However, she notes, the utility of a rubric will depend upon the thought and consideration that goes into its construction. “A rubric is a product of many minds working collaboratively to create new knowledge. It will, almost by definition, be more thoughtful, valid, unbiased and useful than any one of us could have conceived of being as we worked in isolation.”
Advantages of applying a well developed rubric include:
- Provides a common language for sharing expectations and feedback
- Helps to clarify and distinguish the differences between various performance levels
- Helps to focus an individual or group’s ATTENTION on relevant aspects of each desired quality characteristic or skill area
- Provides a mechanism to more easily identify strengths and opportunities for improvement
- Helps lend objectivity to an evaluation process that might otherwise be subjective
Disadvantages:
- Different rubrics may need to be devised for the different activities or artifacts that are to be evaluated using the rubric
- Not all evaluators will apply the rubric in exactly the same way – there is a subjective element at work here – so people may need to be trained in the use of a rubric, or perhaps it would be more effective in a group consensus context where inter-rater variability can be interactively discussed and resolved
- Creating a rubric can be time consuming
- The rubric may limit exploration of solutions or modes of presentation that do not conform to the rubric
Using Rubrics for Quality Improvement
Rubrics are already applied in the world of quality, although I’ve never heard them go by that name. The process scoring guidelines for the Baldrige Criteria are essentially rubrics (although the extra dimension of ADLI and LeTCI has to be considered in the mind of the examiner). The International Team Excellence Award (ITEA) criteria in the Team Excellence Framework (TEF) also forms a rubric in conjunction with the performance levels of missing, unclear, meets expectations or exceeds expectations.
I see a lot of ways in which rubrics can be developed and applied in the quality community to help us establish best practices for some of our most common project artifacts, such as Project Charters. Nancy Tague includes a Project Charter Checklist in The Quality Toolbox to help us create better and more complete charters… but what if we added a second dimension, which includes performance levels, and turned this checklist into a rubric? Any checklist could be transformed into a rubric. Furthermore, to develop a good rubric, we can brainstorm and rank all of the potential criteria in the left hand column, using a Pareto chart to separate the vital few criteria from the trivial many.
Are any of you already using rubrics for purposes outside training or education? I would love to start a list of resources to share with the quality community.
Reference: Griffin, M. (2009). What is a rubric? Assessment Update, 21(6), Nov/Dec 2009.
Note: There is a comprehensive site containing many examples of rubrics at http://www.web.virginia.edu/iaas/assess/tools/rubrics.shtm – however, they won’t open in Google Chrome.
Should You Get Your Six Sigma Black Belt (SSBB) from ASQ?
A couple of weeks ago, I got this email:
Comment: I am basker from NJ, and I have a PMP certification, I want to get a six sigma black belt certification — are there other certifications out there other than one from ASQ ? that you would suggest and what is the cheapest and quickest way to pass the black belt certification ?
Kind Regards,
Basker.
Basker would like to know the quickest and cheapest way to get a Black Belt certification. Here are the routes I know about to obtain the certification: 1) you could go to http://sixsigmaonline.org, whose training and certification program costs about $1000, 2) you could attend the 4-week program offered by http://www.6sigma.us which runs about $8000, 3) you could go to Villanova and pay $7000 for their three-course package, or 4) you can study on your own, complete two projects and get a Project Champion to sign an affidavit, become an ASQ member and pay $299 to take the exam, or don’t become an ASQ member and pay $449 to take the exam.
The ASQ option is probably the quickest and cheapest if you’re good at self study, and dedicated to the task of Six Sigma problem solving. However, with the ASQ certification you also get a lot of clout. I don’t think any other organization has more than 10,000 Black Belts supporting the program, working on updates to the new exams, and keeping the curriculum current. I just personally think it’s a much more viable Black Belt designation than the other programs.
I also asked Jeanine Becker, who works at ASQ in Milwaukee and is responsible for the Six Sigma Forum, what she thought. Jeanine says:
ASQ certification is a formal recognition by ASQ that an individual has demonstrated a proficiency within, and comprehension of, a specific body of knowledge. Nearly 150,000 certifications have been issued to dedicated professionals worldwide.
Anyone can give you a “certificate” for simply attending a course, but an ASQ certification is the recognized gold standard for certification for quality professionals. ASQ is the only third-party provider of certifications for the quality professional. Those that are ASQ certified have the potential to earn a higher salary than their counterparts. The majority of positions companies recruit for that require certifications specify an ASQ certification.
I totally agree. And as a result, Basker, I’d say go with the ASQ cert.
Nicole
SPIE Chairs: Here’s Help Processing Your Reviewer Ratings
The Problem: You are a Chair for a SPIE conference, your review team has provided numerical ratings for all your oral and poster submissions, and now you have to sort through all the numbers and prepare a draft program.
The Solution: Use R!
1. Go into MySPIE into the “Review Presentations” section
2. Click on the Excel icon for “Reviewer Results” (with comments)
3. Save that file as a CSV into some directory on your machine. I saved mine to C:/SPIE12/PresRevs.csv
4. Download the R Statistical Software from http://www.r-project.org if you don’t already have it.
5. Open R and cut and paste the following code onto your R command line. (Note: Use YOUR OWN directory name or it won’t work… mine is SPIE12 on my hard drive for this year’s conference.)
setwd("C:/SPIE12")
spie <- read.csv("PresRevs.csv",header=TRUE)
names(spie)[1] = "tracking"
names(spie)[2] = "conference"
names(spie)[3] = "papernum"
names(spie)[4] = "title"
names(spie)[5] = "reviewer"
names(spie)[6] = "rating"
names(spie)[7] = "recommendation"
names(spie)[8] = "comments"
names(spie)[9] = "more.comments"
papermeans <- aggregate(spie$rating,by=list(papernum),FUN=mean,na.rm=TRUE)
t <-unique(spie$title)
all <- cbind(papermeans,t)
names(all)[1] = "papernum"
names(all)[2] = "mean.rating"
names(all)[3] = "title"
sorted <- all[rev(order(papermeans$mean.rating)),]
write.csv(all,file="ProcessedReviews.csv")
write.csv(sorted,file="SortedProcessedReviews.csv")
6. This will export two CSV files, one with all your processed reviews in order of the paper number, and the other in order of the highest ranked paper first.
7. Have fun preparing your program.
My 2012 Resolution is Myopium
The dictionary defines myopia as narrow mindedness, the inability to see things that aren’t right in front of you. However, the positive spin on myopia is that you can choose to hold only those things in your vision that support what you want to achieve, and where you want to go in life.
As a result, I propose my 2012 resolution: the choice to be blissfully afflicted by myopium: that is, becoming singularly focused on those things that provide me with ecstasy, beatitude, buoyancy, euphoria, joy and accomplishment – and turning away from grief, disappointment, and sorrow. This does not mean that grief and its cousins might not pay me visits occasionally, as is their nature – it just means that I’m going to minimize the time I spend with them this year.
Fortunately, there are many constructive things that I do (many related to my job teaching science and technology to college students) that provide me with feelings of joy. My goal is to do MORE of the things that make me feel good while I’m doing them, and LESS of the things that make me feel bad or make me feel nothing. Feeling good can only contribute to increased productivity… so that’s where I’m headed.
Happy 2012!
Collins and Hansen’s Great By Choice: A Story of Quality Consciousness
Jim Collins, author of Built to Last (2004) and Good to Great (2001), released a new compendium of his research this fall entitled Great by Choice: Uncertainty, Chaos, and Luck – Why Some Thrive Despite Them All. I was pleasantly surprised to discover that these authors have also stumbled upon the importance of quality consciousness – awareness, alignment, and selectively focused attention! These are the keys to developing a highly successful “ten-X” (10X) organization (one that outperforms its industry index by at least ten times, especially during times of great volatility in the business environment).
Collins and his co-author, Morten Hansen, don’t call it quality consciousness, though – they call it “Level 5 Ambition.” And Level 5 Ambition consists of three traits: fanatic discipline, empirical creativity, and productive paranoia. Each of these traits demonstrates one or more aspects of quality consciousness. Here’s how (using excerpts from p. 35 and 36 of the book):
Fanatic discipline: 10Xers display extreme consistency of action – consistency with values, goals, performance standards, and methods. They are utterly relentless, monomaniacal, unbending in their focus on their quests [emphasis added].
Consistency of action is enabled by awareness of quality standards, and unrelenting attention towards achieving them.
Empirical creativity: When faced with uncertainty, 10Xers do not look primarily to other people, conventional wisdom, authority figures, or peers for direction; they look primarily to empirical evidence. They rely upon direct observation, practical experimentation, and direct engagement with tangible evidence. They make their bold, creative moves from a sound empirical base.
By aligning the actions of an organization and its players with what the evidence shows will work, everyone is more confident and able to engage fully in the pursuit of shared goals. A data-driven approach, familiar to anyone who understands quality improvement practice, allows an organization to test its ideas on a smaller scale before committing to major changes.
Productive paranoia: 10Xers maintain hypervigilance, staying highly attuned to threats and changes in their environment, even when – especially when – all’s going well. They assume conditions will turn against them, at perhaps the worst possible moment. They channel their fear and worry into action, preparing, developing contingency plans, building buffers, and maintaining large margins of safety.
Hypervigilance is heightened awareness of the external environment, even during times of peace and productivity. The aspect of productive paranoia that I think is most instructive, however, is that it involves a choice of where to focus your attention: instead of harboring worry and panic about what might happen, the productively paranoid manager will focus on understanding failure modes, developing contingency plans, identifying backup strategies, and planning to branch off on alternative paths, if necessary. The attention is purposefully and positively diverted from unproductive emotions (worry and panic) to productive emotions (the positive feelings associated with being prepared).
Even though nearly 40% of the end of the book is an “Epilogue” containing more detail about Collins and Hansen’s research methodology and results, this is still a very substantial read, and one with very practical advice for businesses aiming to succeed through a challenging economy. My graduate students in technology management enjoyed it too.
Quality Consciousness: Turn On, Tune In, Drop Out!
(Image Credit: Doug Buckley of http://hyperactive.to)
In a previous article, I described the notion of quality consciousness that I’m currently preparing an article about.
To achieve quality consciousness, we ask the very important question (cf. ISO 8402) “What are the totality of characteristics of YOU that bear upon YOUR ABILITY to satisfy the stated and implied needs of your stakeholders?”
The reason we WANT quality consciousness is because we know that the more in tune with the essence of quality that we are, within ourselves, the better we will attune to the needs of our customers and clients – to be able to help them achieve their goals for making things better, more streamlined, and more cost effective.
I summarized quality consciousness as the “3 A’s” – Awareness, Alignment, and Attention:
Quality consciousness implies awareness of yourself and the environment around you (including what constitutes quality and high performance for people, processes and products – most importantly, YOU). It also suggests that you must achieve alignment of your consciousness with the consciousness of the organization, which will aid in full activity and engagement of the senses. Your attention must be selectively focused onto what you can accomplish in the present moment according to that alignment (which implies that you are able to effectively filter the rapid and voluminous streams of information coming at you).
It struck me today how similar this whole notion is to Timothy Leary’s appeal to the counterculture of the late 1960’s, to achieve breakthrough innovation in individual and collective perception of the world to “Turn On, Tune In, and Drop Out”! The message, according to the summary on Wikipedia, was intended to “urge people to embrace cultural changes… detaching themselves from the existing conventions and hierarchies in society.”
So if you want to improve a product, a process, or yourself, embrace the breakthrough innovation that is promised by quality consciousness!
- TURN ON = Become aware of quality standards and the true meaning of excellence, for you and for the domain you work in.
- TUNE IN = Align yourself personally and professionally with your goals, and those of your organization!
- DROP OUT! Focus your attention on the essentials… don’t be distracted by the down economy, by social upheaval, or the perils of ever-increasing competition.
Deliver value… to yourself and those around you! Make it a personal imperative and watch the avalanche of breakthrough innovations begin to cascade around you and your inspirational attitude.
How I Passed My ASQ Certified Six Sigma Black Belt (CSSBB) Exam
I very recently took my ASQ CSSBB exam and passed! Here’s what I think helped me:
1. I studied for about 4 weeks (2 weeks very gently, 1 week much-more-work-because-the-exam-is-getting-closer, and 1 week of panicked, freaked out all nighters) using these great references that I wrote up tons of comments about.
2. I took about 10 pages of really good, concise notes. (I’ll share those with you sometime before the end of the year… want to write them up for public consumption.)
3. I brought about 15 super sharp #2 pencils just in case 14 of them broke. I made sure all the pencils actually SAID #2 on them, so the Scantron machine wouldn’t fail me.
4. I brought my SMART RULER. I’ve had this ruler since the late 1980′s, and every time I’ve taken a tough test, I’ve had my smart ruler with me in case I need to underline anything, or draw dividers between notes. I usually never have to USE the ruler. Usually, its presence is enough to make me do better on any exam.
5. They (the people who say such things) say that peppermint makes you smarter. So I got a new pack of Orbit peppermint gum and chewed it like I had obsessive compulsive disorder for all four hours. (Afterwards I found out that the peppermint thing isn’t really backed up by research, but I didn’t know that going into the exam, so I believed that the peppermint would make my brain work better, and that belief probably helped me out. Got to stack the deck in my favor… didn’t want those 4 weeks of studying NOT to pay off.)
6. When I wasn’t chewing gum, I was nibbling on a Reese’s peanut butter bar. Best 300 calorie investment ever made… the protein made my stomach stop growling so it wouldn’t bother the other test takers.
7. I also brought a couple very cold Diet Cokes, to wash down the peanut butter and the gum taste.
8. To appropriately address my superstitious nature, I wore my Ganesh necklace. In Hindu parlance, Ganesh helps break through obstacles, and I figured the exam that stood between me and CSSBB-hood was definitely an obstacle I wanted broken. (Hey, whatever works, right??)
Nicole
