It can be difficult to focus on strategy when your organization has to comply with standards and regulations. Tracking and auditing can be tedious! If you’re a medical device manufacturer, you may need to maintain ISO 13485 compliance to participate in the supply chain. At the same time, you’ve got to meet all the requirements of 21 CFR 820. You’ve also got to remember other regulations that govern production and postmarket. (To read more about the challenges, check out Wienholt’s 2016 post.) There’s a lot to keep track of!
I have not shared all the commonalities of or differences between ISO 9001:2015 and the Baldrige Excellence Framework. Instead, I have tried to show the organizational possibilities of building on conformity assessment to establish a holistic approach for achieving excellence in every dimension of organizational performance today, with a look to the strategic imperatives and opportunities for the future. Baldrige helps an organization take this journey with a focus on process (55% of the scoring rubric) and results (45% of the rubric), recognizing that great processes are only valuable if they yield the complete set of results that lead to organizational sustainability… I encourage organizations that have not gone beyond conformity to take the next step in securing your future.
My Favorite (#10, Firing Line), from http://www.telegraph.co.uk/sport/horseracing/11574821/Kentucky-Derby-Simon-Callaghan-has-Firing-Line-primed.html. Apr 29, 2015; Louisville, KY, USA; Exercise rider Humberto Gomez works out Kentucky Derby hopeful Firing Line trained by Simon Callaghan at Churchill Downs. Mandatory Credit: Jamie Rhodes-USA TODAY Sports
I love horse racing. More specifically, I love betting on the horses. Why? Because it’s a complex exercise in data science, requiring you to integrate (what feels like) hundreds of different kinds of performance measures — and environmental factors (like weather) — to predict which horse will come in first, second, third, and maybe even fourth (if you’re betting a superfecta). And, you can win actual money!
I spent most of the day yesterday handicapping for Kentucky Derby 2015, before stopping at the track to place my bets for today. As I was going through the handicapping process, I realized that I’m essentially following the analysis process that we use as Examiners when we review applications for the Malcolm Baldrige National Quality Award (MBNQA). We apply “LeTCI” — pronounced like “let’s see” — to determine whether an organization has constructed a robust, reliable, and relevant assessment program to evaluate their business and their results. (And if they haven’t, LeTCI can provide some guidance on how to continuously improve to get there).
LeTCI stands for “Levels, Trends, Comparisons, and Integration”. In Baldrige parlance, here’s what we mean by each of those:
Levels: This refers to categorical or quantitative values that “place or position an organization’s results and performance on a meaningful measurement scale. Performance levels permit evaluation relative to past performance, projections, goals, and appropriate comparisons.”  Your measured levels refer to where you’re at now — your current performance.
Trends: These describe the direction and/or rate of your performance improvements, including the slope of the trend data (if appropriate) and the breadth of your performance results.  “A minimum of three data points is generally needed to begin to ascertain a trend.” 
Comparisons: This “refers to establishing the value of results by their relationship to similar or equivalent measures. Comparisons can be made to results of competitors, industry averages, or best-in-class organizations. The maturity of the organization should help determine what comparisons are most relevant.”  This also includes performance relative to benchmarks.
Integration: This refers to “the extent to which your results measures address important customer, product, market, process, and action plan performance requirements” and “whether your results are harmonized across processes and work units to support organization-wide goals.” 
Here’s a snapshot of my Kentucky Derby handicapping process, using LeTCI. (I also do it for other horse races, but the Derby has got to be one of the most challenging prediction tasks of the year.) Derby prediction is fascinating because all of the horses are excellent, for the most part — and what you’re trying to do is determine on this particular day, against these particular competitors, how likely is a horse to win? Although my handicapping process is much more complex than what I lay out below, this should give you a sense of the process that I use, and how it relates to the Baldrige LeTCI approach:
Levels: First, I have to check out the current performance levels of each contender in the Derby. What’s the horse’s current Beyer speed score or Bris score (that is, are they fast enough to win this race)? What are the recent exercise times? If a horse isn’t running 5 furlongs in under a minute, then I wonder (for example) if they can handle the Derby pace. Has this horse raced on this particular track, or with this particular jockey? I can also check out the racing pedigree of the horse through metrics like “dosage”.
Trends: Next, I look at a few key trends. Have the horse’s past races been preparing him for the longer distance of the Derby? Ideally, I want to see that the two prior races were a mile and a sixteenth, and a mile and an eighth. Is their Beyer speed score increasing, at least over the past three races? Depending on the weather for Louisville, has this horse shown a liking for either fast or muddy tracks? Has the horse won a race recently?
Comparisons: Is the horse paired with a jockey he has been successful with in the past? I spend a lot of time comparing the horses to each other as well. A horse doesn’t have to beat track records to win… he just has to beat the other horses. Even a slow horse will win if the other horses are slower. Additionally, you have to compare the horse’s performance to baselines provided by the other horses throughout the duration of the race. Does your horse tend to get out in front, and then burn out? Or does he stalk the other horses and then launch an attack in the end, pulling out in front as a closer? You have to compare the performance of the horse to the performance of the other horses longitudinally — because the relative performance will change as the race progresses.
Integration: What kind of story do all of these metrics tell together? That’s the real trick of handicapping horse races… the part where you have to bring everything together in to a cohesive, coherent way. This is also the part where you have to apply intuition. Do I really think this horse is ready to pull off a victory today, at this particular track, against these contenders and embedded in the wild and festive Derby environment (which a horse may not have experienced yet)?
And what does this mean for organizational metrics? To me, it means that when I’m formulating and evaluating business metrics I should take a perspective that’s much more like handicapping a major horse race — because assessing performance is intricately tied to capabilities, context, the environment, and what’s bound to happen now, in the near future.
I had a great experience in 2006 using the Baldrige Criteria to develop a Workforce Management Plan for the National Radio Astronomy Observatory (NRAO). We were tasked by the National Science Foundation (NSF) to prepare this report, which was definitely going to require us to dig deep and reflect on how we were managing our workforce, both at the operational level and in service of our strategic priorities. Unfortunately, none of us had ever done this before, so we were pretty much clueless as to what elements such a report would require, and what sorts of questions we might have to answer to ensure that we were approaching the question of workforce management strategically. The NSF wasn’t really able to provide guidance to us other than “you should use best practices from business and industry.” Fortunately, because I had been involved in the quality community for several years, I knew that the Baldrige Criteria might help us accomplish our goal. And it did!
“…share with us some of the ways the practice of quality is changing to meet the needs of faster, faster, faster. “
Fortunately, there has been a ton of research on this over the past decade or so, on the topics of dynamic capabilities and environmental dynamism. The notion of environmental dynamism just means things are changing pretty fast out there, and we have to respond to it. So I’ll focus on what dynamic capabilities are, one way to get them, and some resources and references where you can find out more.
The cornerstone for developing dynamic capabilities seems to be a culture of intentional learning with a dual focus on gaining tacit knowledge (learning by doing) as well as explicit knowledge (learning by reading, conceptualizing, categorizing information) – just like what was recommended by Nonaka & Takeuchi (1995). Using system dynamics modeling, however, Romme et al. (2010) figured out that “there is no linearrelationship between tacit knowledge, deliberate learning and dynamic capability” so really understanding how to leverage your learning capabilities to become more agile needs a few more years of research, it seems.
What this means is – don’t just rush out and start a giant initiative involving deliberate learning in your organization. Although this research uncovered a relationship between a learning orientation and dynamic capabilities, the investigators also found that positive outcomes are very sensitive to the level of environmental dynamism and the initial conditions of the organization (ie. its culture).
However, we have several models for quality systems that honor deliberate learning as a core value, such as the Baldrige Criteria! To keep up with the changing pace of the external environment, the best course of action is to commit to a proven system for continuous improvementThere are also resources like Senge’s classic (1995) book The Fifth Discipline that illuminate the characteristics of a learning organization. While researchers are exploring these links to help us understand how to meet the pace of change more effectively, tried and true systems for continuous improvement with learning as a key component can provide a useful foundation for dealing with these challenges.
If you have at least 5 years broad experience in quality, please help us validate the “Quality Systems Development Roadmap” originally published in Quality Progress in 2008 (http://asq.org/quality-progress/2008/09/basic-quality/starting-from-scratch.html). This is part of an expert systems project developed by Doug Jin, a student at James Madison University, under the guidance of Nicole Radziwill, JMU faculty member and ASQ member leader.
The 5-page, 13-question survey will be available until January 15, 2011 or 1500 responses are received, whatever comes first, so contribute now at http://www.surveymonkey.com/s/W2LYFYP!
Today’s Washington Post has an article by Minnesota senator Tim Pawlenty on the effective design of national health care reform, entitled “To Fix Health Care, Follow the States”. He argues that the federal government should model its initiatives after successful state-based systems that link outcomes to value:
In Minnesota, our state employee health-care plan has demonstrated incredible results by linking outcomes to value. State employees in Minnesota can choose any clinic available to them in the health-care network they’ve selected. However, individuals who use more costly and less-efficient clinics are required to pay more out-of-pocket.
Not surprisingly, informed health-care consumers vote wisely with their feet and their wallets. Employees overwhelmingly selected providers who deliver higher quality and lower costs as a result of getting things right the first time. The payoff is straightforward: For two of the past five years, we’ve had zero percent premium increases in the state employee insurance plan.
Minnesota has also implemented an innovative program called QCARE, for Quality Care and Rewarding Excellence. QCARE identifies quality measures, sets aggressive outcome targets for providers, makes comparable measures transparent to the public and changes the payment system to reward quality rather than quantity. We must stop paying based on the number of procedures and start paying based on results.
Pawlenty also notes that healthcare reform should not focus solely on access to health care, but also the cost and quality of the service – that is, the value that is delivered. The Malcolm Baldrige National Quality Award (MBNQA) Criteria for Performance Excellence provides a framework that has been tailored over 20 years by a huge collaboration of experts to help business, industry and the government better solve this kind of “wicked problem”. The Minnesota solution sounds like it has applied concepts very similar – if not identical – to those presented by the Baldrige Criteria.
When will the government employ the successful problem-solving frameworks it has developed itself (e.g. MBNQA) to solve its most pressing problems?