Quality and Innovation

(Image Credit: Doug Buckley of http://hyperactive.to)

In his February post, ASQ CEO Paul Borawski writes about the importance of encouraging today’s youth to pursue careers in science, technology, engineering and math (STEM). In particular, he asks what we can do to help young people get more interested. This first post is part of a three-part series that I’ll present in February to explain some of my thoughts on how to do this: I) Rethink the Educational System, II) Get Rid of Grades, and III) Develop Better Gateway Drugs.

(The first theme is general, and applies to both STEM and non-STEM disciplines; the third one will be particularly fun, but you’re going to have to wait until the end of the month to find out what I mean by this!)

This post, Part I, is about rethinking the educational system. I know that I’m not the first person to do this, nor the last, and my goal here is not to be comprehensive or justify my opinions – but to give you a sense of how I feel as a STEM educator at the college level.

I’d like to do this from the quality perspective. So first, it’s important to recognize that there is a difference between perceived quality and perceived value (according to Mitra’s Model). Here’s the difference:

• Perceived quality is your assessment of how well a product, service or experience will satisfy your expectations before you buy, adopt, or experience it.
• Perceived value depends on how well the product, service or experience meets your expectations after you buy, adopt or experience it.
• Perceived quality and perceived value are moderated by your expectations. Your expectations can (and often do!) change after you buy, adopt or experience something. Perceived value is NOT invariant, nor is it independent – your perception of value can change after you buy, adopt or experience similar products or participate in similar activities, because then you have a more rich basis for comparison. Perceived value can also change over time.
• Actual quality is the totality of characteristics of the product, service or experience that enable it to meet the stated and implied needs of all stakeholders (ISO 9000: para 3.1.5).

If the perceived value of your higher education remains strong over time (e.g. 10, 20 or 30 years after you graduate), this is a good indication that the quality of the program was high – that it enabled you to meet your needs, the needs of your employers as stakeholders, and/or the needs of your communities and society in general. Even if one of these three classifications of needs is satisfied, perceived value will be preserved over time. If we improve the quality of the educational system now, our personal perceived value of our own educations should be high – and remain high – over the course of our careers and lives. So that’s what I think we should aim for. But how? Here are some ideas:

#1 Institute a Kanban Educational System. You don’t actually learn something – like really learn something – until you need to use it. As educators, we need to change our “push” system of education to a “pull” system, where students can signal for new knowledge and resources as the problems demand. Quality Bob’s recommendation of combining Deming’s System of Profound Knowledge with statistical thinking (http://roberthmitchell.blogspot.com/2012/02/stem-and-quality-statistical-thinking.html) would be helpful here. So would a more widespread adoption of team-based learning, which has become established as a pedagogy.

#2 Abolish “Throughput” as the Key System Performance Metric. Granted, it’s more difficult to set up Socratic exercises (like those that would be required to drive a “pull” educational system) that will lead students to discover the principles and techniques that drive solutions. It’s not difficult, however, to set up the kind of environment that you might encounter in any office: we have a project that needs to be done, and someone’s going to have to sit down and figure out how to do it. But even this approach takes time, effort, and a lot of interaction between the students and their educators, and between the students and other students – it does not align with the dynamic duo of performance metrics, the production of student credit-hours and the number of degrees granted.

Higher education has become more commoditized over the past few decades, which (I believe) is eroding the overall quality of the institution itself. But “getting rid of the urge to push people through” is a tricky suggestion, because it also implies that we may have to reconsider the notion of higher education as a profit center. I would love to keep students in my class until they achieve a minimum level of competence, even if it takes years. But that’s not necessarily practical, and as I think back on my own experience as a STEM undergraduate, there are plenty of things I didn’t “get” until years later when I NEEDED them to get my job done. Refer to point #1 above.

#3 Admit that the Customer is Not Always Right. A recent study indicated that increasing patient satisfaction in hospitals raises healthcare costs and leads to more patient deaths.

What the patients think is best, and what makes them the happiest as consumers of the healthcare system, is not what keeps them alive. And so it is with students in higher education. Students have a financial and emotional incentive to just get through a class. Particularly since they’ve been conditioned to speculate vacuously about how ANY of the stuff they learn could POSSIBLY be useful once they get out of school, it will be difficult for them to anticipate – let alone appreciate – the value of what they are (in many cases) being forced to think about. Some students, as well, will not be satisfied unless they get an easy A without having to do any work. So I advocate tempering the notion of student satisfaction as a measure of how well we’re doing as educators.

#4 Abandon Grades. One of Deming’s Seven Deadly Diseases was the practice of performance appraisals, and grading in academia is no different than what he was rallying about decades ago. There are tons of reasons why grading is not an appropriate – or even an adequate – practice for assuring credentials or credibility. My friend Mary Pat has more to say about this, in particular, because she advocates standardized testing as a “minimum hurdle” that we should expect students to be able to accomplish (or else, not give them the “magic piece of paper”).

Some teachers are hard, some teachers are easy. An A in my class is not the same as an A in another professor’s class. Furthermore, how can I say that what I allocated the most points to on that last exam was really the most important thing my students needed to know? For STEM courses, the recent ASQ survey indicated that there is a perception that way too much work is required, and it would be hard to get the grades that would lead to a good job. This “perception of difficulty” leads many to steer clear of STEM fields, and is entirely rooted in the whole grading nightmare.

More on this particular can-o-worms in Part II.

My valentine is unique. It will not provide me with flowers, or chocolates, or a romantic dinner tonight, and will certainly not whisper sweet nothings into my good ear. And yet – I will feel no less loved. In contrast, my valentine will probably give me some routines for identifying control limits on control charts, and maybe a way to classify time series. I’m really looking forward to spending some quality time today with this great positive force in my life that saves me so much time and makes me so productive.

Today, on Valentine’s Day, I am serenading one of the loves of my life – R. Technically, R is a statistical software package, but for me, it’s the nirvana of data analysis. I am not a hardcore geek programmer, you see. I don’t like to spend hours coding, admiring the elegance of the syntax and data structures, or finding more compact ways to get the job done. I just want to crack open my data and learn cool things about it, and the faster and more butter-like the better.

Here are a few of the reasons why I love R:

• R did not play hard to get. The first time I downloaded R from http://www.r-project.org, it only took about 3 minutes, I was able to start playing with it immediately, and it actually worked without a giant installation struggle.
• R is free. I didn’t have to pay to download it. I don’t have to pay its living expenses in the form of license fees, upgrade fees, or rental charges (like I did when I used SPSS). If I need more from R, I can probably download a new package, and get that too for free.
• R blended into my living situation rather nicely, and if I decide to move, I’m confident that R will be happy in my new place. As a Windows user, I’m accustomed to having hellacious issues installing software, keeping it up to date, loading new packages, and so on. But R works well on Windows. And when I want to move to Linux, R works well there too. And on the days when I just want to get touchy feely with a Mac, R works well there too.
• R gets a lot of exercise, so it’s always in pretty good shape. There is an enthusiastic global community of R users who number in the tens of thousands (and maybe more), and report issues to the people who develop and maintain the individual packages. It’s rare to run into an error with R, especially when you’re using a package that is very popular.
• R is very social; in fact, it’s on Facebook. And if you friend “R Bloggers” you’ll get updates about great things you can do with the software (some basic techniques, but some really advanced ones too). Most updates from R Bloggers come with working code.
• Instead of just having ONE nice package, R has HUNDREDS of nice packages. And each performs a different and unique function, from graphics, to network analysis, to machine learning, to bioinformatics, to super hot-off-the-press algorithms that someone just developed and published. (I even learned how to use the “dtw” package over the weekend, which provides algorithms for time series clustering and classification using a technique called Dynamic Time Warping. Sounds cool, huh!) If you aren’t happy with one package, you can probably find a comparable package that someone else wrote that implements your desired functions in a different way.
• (And if you aren’t satisfied by those packages, there’s always someone out there coding a new one.)
• R helps me meditate. OK, so we can’t go to tai chi class together, but I do find it very easy to get into the flow (a la Csikzentmihalyi) when I’m using R.
• R doesn’t argue with me for no reason. Most of the error messages actually make sense and mean something.
• R always has time to spend with me. All I have to do is turn it on by double-clicking that nice R icon on my desktop. I don’t ever have to compete with other users or feel jealous of them. R never turns me down or says it’s got other stuff to do. R always makes me feel important and special, because it helps me accomplish great things that I would not be able to do on my own. R supports my personal and professional goals.
• R has its own journal. Wow. Not only is it utilitarian and fun to be around, but it’s also got a great reputation and is recognized and honored as a solid citizen of the software community.
• R always remembers me. I can save the image of my entire session with it and pick it up at a later time.
• R will never leave me. (Well, I hope.)

The most important reason I like R is that I just like spending time with it, learning more about it, and feeling our relationship deepen as it gently helps me analyze all my new data. (Seriously geeky – yeah, I know. At least I won’t be disappointed by the object of MY affection today : )

<3

Note: This post is NOT about soup. If you’re searching for really good soup to eat, you will not find it here.

This post is, in contrast, about something that @ASQ tweeted earlier today: “QP Perspectives Column: Is the quality profession undermining ISO 9000?”

In this February 2012 column, author Bob Kennedy examines reflected on a heated discussion at a gathering of senior-level quality practitioners regarding the merit of various tools, methodologies and themes in the context of the quality body of knowledge – what I refer to as “quality soup”. These paragraphs sum up the dilemma captured at that meeting:

Next came the bombshell from a very senior quality consultant: “No one is interested in ISO 9000 anymore; they all want lean.” In hindsight, I think he was speaking from a consultant’s perspective. In other words, there’s no money to be made peddling ISO 9000, but there is with lean and LSS.

I was appalled at this blatant undermining of a fundamental bedrock of quality that is employed by more than 1 million organizations representing nearly every country in the world. The ISO 9000 series is Quality 101, and as quality practitioners, we should never forget it.

If we don’t believe this and promote it, we undermine the impact and importance of ISO 9000. We must ask ourselves, “Am I interested in ISO 9000 anymore?”

When I see articles like this, and other articles or books that question whether a tool or technique is just a passing fad (e.g. there’s a whole history of them presented in Cole’s 1999 book) my visceral reaction is always the same. How can so many quality professionals not see that each of these “things we do” satisfies a well-defined and very distinct purpose? (I quickly and compassionately recall that it only took me 6 years to figure this out, 4 of which were spent in a PhD program focusing on quality systems – so don’t feel bad if I just pointed a finger at you, because I’d actually be pointing it at past-me as well, and I’m still in the process of figuring all of this stuff out.)

In a successful and high-performing organization, I would expect to see SEVERAL of these philosophies, methodologies and techniques applied. For example:

• The Baldrige Criteria provide a general framework to align an organization’s strategy with its operations in a way that promotes continuous improvement, organizational learning, and social responsibility. (In addition to the Criteria booklet itself, Latham & Vinyard’s users guide is also pretty comprehensive and accessible in case you want to learn more.)
• ISO 9000 provides eight categories of quality standards to make sure we’re setting up the framework for a process-driven quality management system. (Cianfrani, Tsiakals & West are my two heroes of this system, because it wasn’t until I read their book that I realized what ISO 9001:2000, specifically, was all about.)
• Thus you could very easily have ISO 9000 compliant processes and operations in an organization whose strategy, structure, and results orientation are guided by the Baldrige Criteria.
• Six Sigma helps us reduce defects in any of those processes that we may or may not be managing via an ISO 9000 compliant system. (It also provides us with a couple of nifty methodologies, DMAIC and DMADV, that can help us structure improvement projects that might focus on improving another parameter that describes system performance OR design processes that tend not to yield defectives.)
• The Six Sigma “movement” also provides a management philosophy that centers around the tools and technologies of Six Sigma, but really emphasizes the need for data-driven decision making that stimulates robust conclusions and recommendations.
• Lean helps us continuously improve processes to obtain greater margins of value. It won’t help you reduce defects like Six Sigma will (unless your waste WAS those defects, or you’re consciously mashing the two up and applying Lean Six Sigma). It won’t help you explore alternative designs or policies like Design of Experiments, part of the Six Sigma DMAIC “Improve” phase, might do. It won’t help you identify which processes are active in your organization, or the interactions and interdependencies between those processes, like an ISO 9000 system will (certified or not).
• ISO 9000 only guarantees that you know your processes, and you’re reliably doing what you say you’re supposed to be doing. It doesn’t help you do the right thing – you could be doing lots of wrong things VERY reliably and consistently, while keeping perfect records, and still be honorably ISO certified. The Baldrige process is much better for designing the right processes to support your overall strategy.
• Baldrige, ISO 9000, and lean will not help you do structured problem-solving of the kind that’s needed for continuous improvement to occur. PDSA, and possibly Six Sigma methodologies, will help you accomplish this.

Are you starting to see how they all fit together?

So yeah, let’s GET LEAN and stop wasting our energy on the debate about whether one approach is better than another, or whether one should be put out to pasture. We don’t dry our clothes in the microwave, and we don’t typically take baths in our kitchen sink, but it is very easy to apply one quality philosophy, methodology or set of practices and expect a result that is much better generated by another.

Bob Kennedy comes to the same conclusion at the end of his column, one which I fully support:

All quality approaches have a place in our society. Their place is in the supportive environment of an ISO 9000-based QMS, regardless of whether it’s accredited. Otherwise, these approaches will operate in a vacuum and fail to deliver the improvements they promise.

Sunday afternoon, as I sat at my kitchen table drinking tea and flipping through the Sunday Washington Post, I came across an article about using process improvement principles in everyday life.  Aha!  My first thought was that I need to share this article with Nicole, followed immediately by another thought – I really need to share this with the students in our HON 300/ISAT 680 process improvement class!

I’m not sure of the exact date when Nicole and I hatched the idea of a process improvement course.  Since we both arrived at JMU in the fall of 2009, we have had numerous conversations about all the cool things we would love to share with students about quality and process improvement.  Those conversations inspire me, and honestly, sometimes they leave me feeling a little overwhelmed.

Whenever it was that we hatched the idea of this course, I definitely had some concerns about the reality of fitting a process improvement project into the time limitations of a single semester.  My concerns were heightened when we decided to structure the course using the DMAIC approach.  All the Six Sigma projects I led as a Black Belt certainly took longer than the approximately 15 weeks allowed for a semester, and I was a full-time Black Belt who (supposedly) knew what I was doing!

Wouldn’t the data collection and analysis overwhelm our students?  Shouldn’t we require some background in statistics as a prerequisite for enrollment in the course?  How would the relationships work with our clients?  Would the students really be able to deliver results in one semester?

Yet here we are, in early February with four outstanding project teams that are quickly moving into the Measure phase.  What’s all that got to do with the Post article?

Process improvement doesn’t have to require mountains of data and highly sophisticated statistical analysis.  The basic principles of process improvement can be used to effect change by anyone, and in almost any situation.  That is really what our course is all about, and the Post article just provided an example we can all relate to.  Who doesn’t wish for more time in their day?

I think we have an outstanding group of students enrolled in our course.  Nicole and I want them to learn, understand, and apply process improvement principles, but not just within the confines of this specific course.  The continuous pursuit of improvement is a lifestyle, and today I was reminded of just how true that can be.

– Rebecca Simmons

(Image Credit: Doug Buckley of http://hyperactive.to)

Are you an entry-level Young Quality Professional (YQP) who has participated or led a project that resulted in tangible benefits for your organization or community? Are you a student in high school or college who has worked individually, or with a team, to apply quality tools to solve practical problems in your school, or community, or at home? If so, WE WANT YOU to tell us about it!

On Wednesday, May 23rd, we will be conducting a session at the ASQ World Conference on Quality and Improvement (WCQI) to demonstrate the impact that the newest members of our community – students and entry-level professionals – have had on their organizations, businesses, and communities. This session will consist of vignettes from up to 15 quality or process improvement projects performed by students or entry-level professionals. We’ll be inviting all WCQI attendees to come celebrate the accomplishments of the youngest members of our community, and support them as they progress in their education and in their careers!

HOW TO APPLY: Send an email to Jacob Mayiani Loorimirim (Graduate Assistant, JMU) at loorimjm@dukes.jmu.edu (and cc: simmo2ra@jmu.edu and radziwnm@jmu.edu) with short answers to each of the 6 questions below. Project review started on February 1 and will continue until all slots are filled, or February 29 at the latest.

If your project is selected to be part of the Showcase, we’ll work with you to put together a few slides, audio recordings, and/or video clips that describe the problem you solved, the quality tools and approaches you used, your results, and the impact of your solution on your stakeholders. We plan to spend between 3 and 5 minutes showcasing each project. If you are a student who will be attending the WCQI in person, we would love for you to submit a project that you completed individually or as a team – and talk about it yourself for 2 to 3 minutes during our session!

QUESTIONS:

1. Project completed by: (List your names and ages, and specify whether this was part of a school/university project, for a client, or was done in service to your community; if you had a teacher or faculty advisor, please list them too! Also, let us know if you plan to attend the WCQI in person in Anaheim, CA this May.)

2. Project title:

3. Project start and completion dates:

4. Provide a brief problem statement (1-3 sentences) that summarizes the problem, your stakeholders, and your goals:

5. Provide a brief description of your hands-on performance while completing the project, providing specific examples of the methodologies (e.g. PDSA, DMAIC) and/or tools (e.g. process maps, fishbone diagrams, Pareto charts, affinity diagrams, multivoting/nominal group technique) that you used to solve the problem.

6. In one sentence, describe your project’s RESULTS and the impacts on its stakeholders.

Thank you for your interest! Please forward this announcement by email, Facebook, Twitter, or any other mechanism if you know of a Young Quality Professional (YQP) whose work should be noticed and recognized – or where they might be hanging out.

Sincerely,

Nicole Radziwill, College of Integrated Science & Technology, James Madison University
Rebecca Simmons, College of Business, James Madison University

According to dictionary.com, one of the definitions for rubric is “any established mode of conduct; protocol.” But the context you’ve probably heard this word in is education – where a grading rubric or a scoring rubric is used to evaluate a complex artifact like a student essay.

In my opinion, it’s time to move the concept of the rubric from the classroom into the mainstream, because it can be applied as a very practical general purpose quality tool! (Hear that, Nancy Tague? I think you should write about rubrics in your next edition of the very excellent book The Quality Toolbox. Let me know if you’d like me to help make this happen.)

A rubric is basically a grid with 1) levels of performance indicated along the top row, and 2) criteria or dimensions of performance listed down the leftmost column. Each cell of the grid contains a descriptive statement that explains how the level of performance in that column might be achieved for a specific dimension:

For example, here’s a rubric that one group constructed to evaluate the quality of the mind maps that they were producing. The performance levels are organized from high performance in the top left (smiley face giving a thumbs up) to low performance in the top right (smiley face that looks like he’s about to pass out):

The dimensions of performance are neatness and presentation, use of images/symbols, and use of color. The descriptive statements in each cell provide specific examples of how the performance level might be achieved, e.g. “has failed to include color in the mind map” is an indicator of a low performance level for the dimension of “use of color” – which is very understandable!

The concept of the rubric as a performance assessment tool is relatively new! Griffin (2009), in a brief history of the rubric, notes that since its introduction in 1981, “the scoring rubric has evolved into a more precise, technical, scientific-looking document. It carries a tone of certainty, authority, and exactitude.” However, she notes, the utility of a rubric will depend upon the thought and consideration that goes into its construction. “A rubric is a product of many minds working collaboratively to create new knowledge. It will, almost by definition, be more thoughtful, valid, unbiased and useful than any one of us could have conceived of being as we worked in isolation.”

Advantages of applying a well developed rubric include:

• Provides a common language for sharing expectations and feedback
• Helps to clarify and distinguish the differences between various performance levels
• Helps to focus an individual or group’s ATTENTION on relevant aspects of each desired quality characteristic or skill area
• Provides a mechanism to more easily identify strengths and opportunities for improvement
• Helps lend objectivity to an evaluation process that might otherwise be subjective

Disadvantages:

• Different rubrics may need to be devised for the different activities or artifacts that are to be evaluated using the rubric
• Not all evaluators will apply the rubric in exactly the same way – there is a subjective element at work here – so people may need to be trained in the use of a rubric, or perhaps it would be more effective in a group consensus context where inter-rater variability can be interactively discussed and resolved
• Creating a rubric can be time consuming
• The rubric may limit exploration of solutions or modes of presentation that do not conform to the rubric

Using Rubrics for Quality Improvement

Rubrics are already applied in the world of quality, although I’ve never heard them go by that name. The process scoring guidelines for the Baldrige Criteria are essentially rubrics (although the extra dimension of ADLI and LeTCI has to be considered in the mind of the examiner). The International Team Excellence Award (ITEA) criteria in the Team Excellence Framework (TEF) also forms a rubric in conjunction with the performance levels of missing, unclear, meets expectations or exceeds expectations.

I see a lot of ways in which rubrics can be developed and applied in the quality community to help us establish best practices for some of our most common project artifacts, such as Project Charters. Nancy Tague includes a Project Charter Checklist in The Quality Toolbox to help us create better and more complete charters… but what if we added a second dimension, which includes performance levels, and turned this checklist into a rubric? Any checklist could be transformed into a rubric. Furthermore, to develop a good rubric, we can brainstorm and rank all of the potential criteria in the left hand column, using a Pareto chart to separate the vital few criteria from the trivial many.

Are any of you already using rubrics for purposes outside training or education? I would love to start a list of resources to share with the quality community.

Reference: Griffin, M. (2009). What is a rubric? Assessment Update, 21(6), Nov/Dec 2009.

Note: There is a comprehensive site containing many examples of rubrics at http://www.web.virginia.edu/iaas/assess/tools/rubrics.shtm – however, they won’t open in Google Chrome.

A couple of weeks ago, I got this email:

Comment: I am basker from NJ, and I have a PMP certification, I want to get a six sigma black belt certification — are there other certifications out there other than one from ASQ ? that you would suggest and what is the cheapest and quickest way to pass the black belt certification ?

Kind Regards,

Basker.

Basker would like to know the quickest and cheapest way to get a Black Belt certification. Here are the routes I know about to obtain the certification: 1) you could go to http://sixsigmaonline.org, whose training and certification program costs about $1000, 2) you could attend the 4-week program offered by http://www.6sigma.us which runs about $8000, 3) you could go to Villanova and pay $7000 for their three-course package, or 4) you can study on your own, complete two projects and get a Project Champion to sign an affidavit, become an ASQ member and pay $299 to take the exam, or don’t become an ASQ member and pay $449 to take the exam.

The ASQ option is probably the quickest and cheapest if you’re good at self study, and dedicated to the task of Six Sigma problem solving. However, with the ASQ certification you also get a lot of clout. I don’t think any other organization has more than 10,000 Black Belts supporting the program, working on updates to the new exams, and keeping the curriculum current. I just personally think it’s a much more viable Black Belt designation than the other programs.

I also asked Jeanine Becker, who works at ASQ in Milwaukee and is responsible for the Six Sigma Forum, what she thought. Jeanine says:

ASQ certification is a formal recognition by ASQ that an individual has demonstrated a proficiency within, and comprehension of, a specific body of knowledge. Nearly 150,000 certifications have been issued to dedicated professionals worldwide.

Anyone can give you a “certificate” for simply attending a course, but an ASQ certification is the recognized gold standard for certification for quality professionals.  ASQ is the only third-party provider of certifications for the quality professional.   Those that are ASQ certified have the potential to earn a higher salary than their counterparts.    The majority of positions companies recruit for that require certifications specify an ASQ certification.

I totally agree. And as a result, Basker, I’d say go with the ASQ cert.

Nicole