All of us should be starting to feel it now… we’re entering a new AI-ra. Which is why I’m expanding horizons! Today I’m launching the Digital Quality Institute at dxquality.com. That’s where I’ll be offering my old PDF eBooks that have been available for years, new PDF eBooks yet to be written, and high-impact courseware that I produce or curate.
For years, I’ve imagined what it would be like if I could connect people with the resources I wish I had, especially in the first 10 or 20 years of my career – to help us balance purpose, performance, and personal transformation. How to build critical thinking skills in technology-intense environments. How to make digital transformation actionable. How to navigate times when the technology is moving so fast we feel like we’ll be left behind. I’ve wistfully browsed through my Google Drive, packed with 14 years of content, much of which has only been shared with a relatively small group of people.
Since 2016, lots of people I’ve interacted with in work environments, or after talks and keynotes, have said “That’s SUCH useful content… you should put that on Udemy.” But the quality of courseware on Udemy and Coursera varies a LOT, and I’ve never felt comfortable adding my work to that giant haystack. And while I love TEDx, it’s become too diluted to me, it doesn’t feel as special as it did years ago when researchers were sharing their new insights with the world in a sparkly new way.
That’s why today, I’m launching the Digital Quality Institute at dxquality.com.
My vision for DQI is that it provides a marketplace of unique, compelling, and inventive ideas from leaders and emerging leaders who care about quality and excellence – people at all stages of their careers who have been inspired by the rich tradition of quality. Leaders who can carry those messages forward in relevant and engaging new ways that younger generations can be inspired by.
While the first two courses are derived from instruction and mentorship I’ve been providing students and direct reports since the late 90s, I’m in discussions with insightful leaders with great ideas who might also contribute to the offerings. (If you’ve got material grounded in quality and you’re looking for a platform to share it, let’s talk & see if it’s suitable for DQI).
I’m excited to be the curator of dynamic stories that help us connect with our power… so we can help each other transform our work, transform our lives, and realize our potential as individuals and innovative communities in the digital-first world.
Agreeing on simple things (for example, where and when to meet) can be anything but simple. There can be miscommunications, incomplete communications, desires can change along the way, or people can learn things that change their availability or ability to get together. How do you make sure that you stay on the same page, even as you (and the things around you) change?
If you search YouTube for videos on expectation setting, though, you’ll get lots of results that talk about one-way communication around making sure someone (often someone with less organizational power) knows what you want from them. Lots of these videos are great, but they are limited to helping you build your skills around clear, concise, meaningful, tangible communication. And sometimes, expectation setting is indeed one-way, like for example when a company explains its return policy before you purchase something, or lets you know that there’s a real person behind its chat rather than a bot.
More often, expectation setting requires dialogue, and can’t just be unilaterally set by one party. When two parties (or a group of people) establish or update their shared understanding of a problem, situation, or scenario, then confirm that shared understanding, expectations are set. But that doesn’t mean you’re done.
Expectations erode over time. People learn, situations change, and relationships change. Especially in a work context, we might be setting expectations with a role, and people can flow in and out of roles. It’s not uncommon that we won’t have all the information or context we need going into a discussion, or that the information has become stale since the last time the parties reviewed it. Sometimes, new people are at the table, and we can’t make assumptions about how much understanding they have (or whether they’ve been introduced to the problem at hand at all).
For example, it’s common to scope a services engagement with the decision maker and project champion (who signs the contract), but be handed off to more operational contacts for the day-to-day work. Sometimes these operations people will not have had the opportunity to be briefed by the decision maker, who may or may not have prioritized getting their buy-in. You’ll have to deal with this challenge during the first opportunity you get to engage in expectation setting with this group of people.
The dialogue around expectations is ongoing, and you can tell it’s happening when you hear things like:
“What we think we’re going to do next is [this]. Has anything changed since the last time we reviewed this?”
“What I’m hearing is… [this]. Is that your understanding too?”
“So what I think you just said is… [thing I think you just said]. Is that accurate?”
“Let me reflect that back to you, and you can assess how close I am.”
“So if we do [this], does everyone here agree that’s the next step?”
I’d rather be disappointed now than disappointed and overwhelmed later by having to pivot. It’s good to be agile and be able to pivot, but it’s bad to have to pivot over and over again. (You have to regain your balance after every pivot. Sometimes this rebalancing is quick, and other times it never quite happens. Pivoting always comes with a risk, just like not pivoting can be risky.)
Expectation setting is one of the most valuable skills you can develop, especially if you are working as part of an agile team. Not only can expectation setting help you become a badass at work, but it will also help you build better relationships with friends and family.
Don’t assume that the world is the same as it was last time. Set expectations early and often.
This morning, while I was checking my email, the power went out.
There was no storm, the neighbors still had power… did I forget to pay the bill? Surely not… I get an email every month & immediately click through to process the payment. But I decided to check anyway.
I went to our local electric company’s web site to check on the status of my account. It’s not a big town, and their development budget has got to be tiny… it looks like they haven’t updated their web site in more than a decade. And the “Pay Your Bill” form is even more extreme… that form is straight outta 1997, complete with a little moving text, like ants marching across the bottom of your screen. It looks like Peoplesoft used to look like, at the dawn of the millennium.
Not my power company’s web form. But close. You get the idea.
Turns out, I did make a mistake last month. I was traveling when bill time came around, and instead of clicking the button to pay all accounts, I accidentally only paid one and not the one that corresponds to the house where I live, where I work, where I absolutely positively need internet all the time.
There was a text box at the top of my screen with a phone number, and a message saying that power could usually be restored by 8pm. I started mentally arranging my work day… how much connectivity I’d be able to get by tethering to my phone… how much battery life I had left in my laptop… how much crow I’d have to eat, as the impacts impacted other people and other meetings.
I quickly filled out all the little boxes to let the electric company know that I really wanted them to take my money and turn my power back on as soon as they could. Like usual, their clunky looking web form was smooth to actually fill in.
But then magic happened.
I clicked the “Pay Now” button with the stylus on my phone. TAP! Before my fingers returned to the position they were in when they started the downward trip to click the button (literally milliseconds later) I heard whirring all around me. The printer was going through its startup sequence. The TV started to flicker on.
That 1997-era web form was like a lightswitch. When I flipped it, my power went back on. For whatever the power company didn’t invest in making their web site look slick and exciting, they sure did invest in what means the most to me: automatically, instantaneously, magically getting my power back in the blink of an eye.
Why don’t all companies invest, like this, in delivering meaningful value over delivering the appearance of value?
I’m thinking of a company I know, right now, that’s invested several million dollars over the past year trying to get a web app in place to perform a basic revenue-driving function of their business. They even “built an MVP”! But they’re still falling short of their goal. I wonder if they, like so many others, are falling into the most nefarious trap that exists:
The Illusion of Progress… showing that you’re moving forward without showing that you know what really matters, and then surgically focusing on that.
Are you making progress on what really matters? Or what looks like it matters?
Agility can be great (even though for many neurodivergent people, it’s the opposite of great). But when teams go through the motions to be “Agile” they often end up overcomplicating the work, adding stress to interactions, and achieving less agility.
If your agile processes are “working”, then over the next iteration (typically 1-4 weeks):
⬜ You have a clearunderstanding, as a team, of the value you’re working to demonstrate. (Note that this is not a promise or a commitment, but a shared purpose, direction and intention.)
⬜ You have a clearunderstanding, as individuals, of the actions that will generate that value. You know what to work on (and think about) when you’re not in meetings or with clients.
⬜ You don’t feel alone – you have teammates to communicate with and collaborate with, and resources you can use, as you move forward to advance project objectives.
⬜ You are working at a comfortable, sustainable pace.
⬜ Your team is improving every week – you continually do things a little differently to help improve quality, productivity, collaboration, or other outcomes.
None of these things require meetings (or “ceremonies”) – just communication and inclusion. We should always be asking ourselves and our team members: what’s the easiest, smoothest, least intrusive way of building this shared understanding and maintaining it every day?
When teams work with agility:
⬜ Everyone understands the overall goal and the next increment of value to demonstrate.
⬜ Everyone has clear things to do to contribute to that value.
⬜ Each person (and the team as a whole) works at a sustainable pace.
⬜ Nobody feels alone. There’s a group of collaborators to share the work with.
⬜ The client or project champion has visibility into the work and is happy with progress.
⬜ There’s an opportunity to rest, reflect, and adjust every week or two.
⬜ It’s not ridiculously hard to adjust to changes in scope, tools, or personnel.
But I regularly see “agile” teams flailing… unhappy… in constant panic mode, with stress that just won’t end. They are very busy and always seem to be scrambling to show their client or project champion some kind of value… with the feeling that they have to defend their existence. They have a nagging sense of confusion and impostor syndrome may be creeping in. They can tell you exactly what tasks are on their JIRA boards, but they can’t tell you why those tickets exist or how their task is contributing to overall project value. They have lost sight of the forest (value) because they have planted so many trees (tickets).
For these teams, the Scrum process and Agile ceremonies may be adding layers of stress and bureaucracy rather than helping the team work sustainably to consistently drive value. They are “doing Agile ” but have actually made themselves less agile… less able to flow and adapt and respond to changes in scope, tools, or participants.
Agile methods first emerged in response to the slow, painful, unsatisfying, unhappy practice of software development. It was really depressing to spend months building software, only to have customers and users complain how bad it was when they got it (especially when you developed exactly what they specified). It felt isolating to have to figure out how to deliver a piece of the software without any input from other engineers, and it was distressing when others were blocking your work and you had to convince them to listen to you. The whole endeavor was inefficient, and people were often tense and stressed.
The realizations in red (in the 90s) led to the Agile Manifesto items (2001) in blue
We’re applying processes and tools without really examining why we need them
So let’s prioritize…
Individuals and interactions over processes and tools
Why this is part of the Agile Manifesto: Since the beginning of time, development teams tend to get hung up on the details of using tools (e.g. MS Project, JIRA, kanban boards) rather than why the tools are there in the first place: to make sure people are getting the information and context they need – on a regular, routine basis – to make continuous progress on the team’s overall deliverables.
What we should do in 2022: Focus on information sharing, context building, and working arrangements that help people get work done. This applies to interactions within the team as well as interactions with the client.
We’re doing a lot of work that doesn’t actually contribute to our goal
So let’s prioritize…
Working software[ie. tangible stuff that’s valuable]over comprehensive documentation
Why this is part of the Agile Manifesto: Software development in the 1990s was documentation-heavy. I even remember, in 2018, throwing away a few hundred pounds of paper (in about 20 4” binders) containing requirements and design documents for a really big software project that we did between 2002 and 2004. Often, development teams wouldn’t even produce software that matched the documentation because we’d learn about what was feasible and what was not as we were doing it. Everyone tended to deliver software that the business needed a year or two ago. We didn’t learn together and co-create the software.
What we should do in 2022: We provide value in the form of information, shared understanding, and working software (which may be in the form of quality controls) – that’s it. Anything we produce that doesn’t directly contribute to making these things happen shouldn’t be done.
We’re unable to deliver something we don’t know how to define or describe yet
So let’s prioritize…
Customer collaboration over contract negotiation
Why this is part of the Agile Manifesto: Because deciding what you’re on the hook to deliver at the beginning of a project – and exactly what the deliverables are going to look like – is dangerous. You and the customer rarely have enough understanding of the problem (or of each other) to get it right.
What we should do in 2022: Establish shared accountability. (Note: this is rare. How many times has a customer been on your team and just as accountable to the project champion for the end result as you are?) A workaround is to set expectations with the client that we are discovering the shared understanding together, and we will get as close to the desired deliverables as we can, given the fact that we are embarking on a process of learning together.
We’re unable to commit to a plan when we might learn that our plan isn’t feasible
So let’s prioritize…
Responding to change over following a plan
Why this is part of the Agile Manifesto: As you learn about what’s possible, what’s not possible, and what the client actually values… plans will change. Instead of establishing a timeline that won’t end up working out (and that will cause you a lot of stress when you start deviating from), just start with the understanding that your Gantt Chart and your intermediary milestones will probably not be achieved when you think they will – or maybe even at all.
What we should do in 2022: Always keep the final goal in mind, but revisit and adjust the plan as you learn more every week. Iterate!
Presentations can be boring. Talking about your work can be boring (to other people). When you’re sitting in a talk or a session that you find boring (and you can’t figure out WIIFM – what’s in it for me)… you learn less.
Although we shouldn’t have to use clickbait techniques to satisfy societally decreasing attention spans, it is easier to learn and retain information when it’s interesting. So I encourage all of you to apply the “Discovery Channel Test” to every presentation, talk, or session you contribute to or lead.
The reason I call it the “Discovery Channel Test” is that there used to be a program called City Confidential that was on all the time. Even though City Confidential was a show about murder mysteries, the first half of the one-hour show was about the city or region where the crime occurred. They talked about when the town was first settled, and key families who made the town what it is today. You heard about stories of intrigue, and challenges the town was facing today. They made the story about the town itself so captivating that EVERY SINGLE TIME I was caught off guard when they ended the first half-hour segment with “You’d never believe a murder could happen here.” Through the hundreds of times I watched this show, I was always shocked when this point came. “Oh, yeah… this is where that murder mystery happened, and we’re about to find out about it.”
Any production crew that can spend a half hour off-topic, keep me interested, and give me a dopamine burst right before the main point of the show… has achieved something great. And you, too, can achieve this same greatness when you’re talking about your tech stack or a new architectural initiative or that project you did last year that improved customer sat. Make it interesting by applying….
The Discovery Channel Test (* = could also be called The Good Podcast Test)
Rule 1: Use an emotional hook up front. Why should any of the people in your audience not leave after the first five minutes? Don’t make them guess! Tell them specifically why this topic might be interesting, or surprise them with an initial feeling of novelty or unexpectedness. Jonathan Lipnicki’s character in Jerry Maguire (1996) was great at this, with his “did you know?” questions. The dopamine surge you create with an emotional hook will keep them engaged for long enough to get hooked on your story.
Rule 2: Find the wow nugget(s). This is one of the things the Discovery Channel has always been really good at: getting me interested in things that I didn’t think were interesting to me. Remember that your projects and initiatives, no matter how cool they are to you, will be boring to other people unless you TRY to make them interesting. I try to tell my high schooler that even in the most boring classes you should be able to find some nugget, some angle, some insight that helps you see the subject matter in a way that grabs you. Find that angle for your audience, and then spoon feed it to them.
Rule 3: Use examples, screen shots, visuals, diagrams. If your presentations are full of slides with words, people will start yawning immediately and may or may not actually hear those words. You can also reduce ambiguity with examples and screen shots. For example, saying “we used Python for this project” is far less compelling than showing the tree structure of the code (or a simplified diagram) and annotating it with what each piece did to get the overall job done. Saying “we used Confluence” is less compelling than saying “we set up a confluence site at [this location] and agreed to put [this kind of information in there]” because if someone has a need for [that information] – at least they know a first place they can look for it.
Rule 4: Spoon feed the closing thoughts. At the very end, remind people what you want them to remember when they leave. Remind people why that’s interesting, and how it might benefit them in the future. Make it concrete and tangible. For example, can you give them reference material, or an infographic, or a checklist that they can use in the future? Don’t assume that people will get something out of your presentation just by attending… spoon feed what you WANT them to get out of it.
One of the heuristics we use at Intelex to guide decision making is former US President Truman’s advice that “imperfect action is better than perfect inaction.” What it means is — don’t wait too long to take action, because you don’t want to miss opportunities. Good advice, right?
When I share this with colleagues, I often hear a response like: “that’s dangerous!” To which my answer is “well sure, sometimes, but it can be really valuable depending on how you apply it!” The trick is: knowing how and when.
Here’s how it can be dangerous. For example, statistical process control (SPC) exists to keep us from tampering with processes — from taking imperfect action based on random variation, which will not only get us nowhere, but can exacerbate the problem we were trying to solve. The secret is to apply Truman’s heuristic based on an understanding of exactly how imperfect is OK with your organization, based on your risk appetite. And this is where loss functions can help.
Along the way, we’ll demonstrate how to do a few important things related to plotting with the ggplot package in R, gradually adding in new elements to the plot so you can see how it’s layered, including:
Plot a function based on its equation
Add text annotations to specific locations on a ggplot
Draw horizontal and vertical lines on a ggplot
Draw arrows on a ggplot
Add extra dots to a ggplot
Eliminate axis text and axis tick marks
What is a Loss Function?
A loss function quantifies how unhappy you’ll be based on the accuracy or effectiveness of a prediction or decision. In the simplest case, you control one variable (x) which leads to some cost or loss (y). For the case we’ll examine in this post, the variables are:
How much time and effort you put in to scoping and characterizing the problem (x); we assume that time+effort invested leads to real understanding
How much it will cost you (y); can be expressed in terms of direct costs (e.g. capex + opex) as well as opportunity costs or intangible costs (e.g. damage to reputation)
Here is an example of what this might look like, if you have a situation where overestimating (putting in too much x) OR underestimating (putting in too little x) are both equally bad. In this case, x=10 is the best (least costly) decision or prediction:
plot of a typical squared loss function
# describe the equation we want to plot
parabola <- function(x) ((x-10)^2)+10
# initialize ggplot with a dummy dataset
library(ggplot)
p <- ggplot(data = data.frame(x=0), mapping = aes(x=x))
p + stat_function(fun=parabola) + xlim(-2,23) + ylim(-2,100) +
xlab("x = the variable you can control") +
ylab("y = cost of loss ($$)")
In regression (and other techniques where you’re trying to build a model to predict a quantitative dependent variable), mean square error is a squared loss function that helps you quantify error. It captures two facts: the farther away you are from the correct answer the worse the error is — and both overestimating and underestimating is bad (which is why you square the values). Across this and related techniques, the loss function captures these characteristics:
Not all loss functions have that general shape. For classification, for example, the 0-1 loss function tells the story that if you get a classification wrong (x < 0) you incur all the penalty or loss (y=1), whereas if you get it right (x > 0) there is no penalty or loss (y=0):
# set up data frame of red points
d.step <- data.frame(x=c(-3,0,0,3), y=c(1,1,0,0))
# note that the loss function really extends to x=-Inf and x=+Inf
ggplot(d.step) + geom_step(mapping=aes(x=x, y=y), direction="hv") +
geom_point(mapping=aes(x=x, y=y), color="red") +
xlab("y* f(x)") + ylab("Loss (Cost)") +
ggtitle("0-1 Loss Function for Classification")
Use the Loss Function to Make Strategic Decisions
So let’s get back to Truman’s advice. Ideally, we want to choose the x (the amount of time and effort to invest into project planning) that results in the lowest possible cost or loss. That’s the green point at the nadir of the parabola:
And time+effort grows as we move along the x-axis (we might spend minutes on a problem at the left of the plot, or weeks to years by the time we get to the right hand side):
What this means is — if we don’t plan, or we plan just a little bit, we incur high costs. We might make the wrong decision! Or miss critical opportunities! But if we plan too much — we’re going to spend too much time, money, and/or effort compared to the benefit of the solution we provide.
The trick is to FIND THAT CRITICAL LEVEL OF TIME and EFFORT invested to gain information and understanding about your problem… and then if you’re going to err, make sure you err towards the left — if you’re going to make a mistake, make the mistake that costs less and takes less time to make:
The moral of the story is… imperfect action can be expensive, but perfect action is ALWAYS expensive. Spend less to make mistakes and learn from them, if you can! This is one of the value drivers for agile methodologies… agile practices can help improve communication and coordination so that the loss function is minimized.
## FULL CODE FOR THE COMPLETELY ANNOTATED CHART ##
# If you change the equation for the parabola, annotations may shift and be in the wrong place.
parabola <- function(x) ((x-10)^2)+10
my.title <- expression(paste("Imperfect Action Can Be Expensive. But Perfect Action is ", italic("Always"), " Expensive."))
arrow.x <- c(10, 10, 10, 10)
arrow.y <- c(35, 50, 65, 80)
arrow.x.end <- c(6, 6, 6, 6)
arrow.y.end <- arrow.y
d <- data.frame(arrow.x, arrow.y, arrow.x.end, arrow.y.end)
p + stat_function(fun=parabola) + xlim(-2,23) + ylim(-2,100) +
xlab("Time Spent and Information Gained (e.g. person-weeks)") + ylab("$$ COST $$") +
annotate(geom="text", x=10, y=5, label="Some Effort, Lowest Cost!!", color="darkgreen") +
geom_point(aes(x=10, y=10), colour="darkgreen") +
annotate(geom="text", x=0, y=100, label="$$$$$", color="green") +
annotate(geom="text", x=0, y=75, label="$$$$", color="green") +
annotate(geom="text", x=0, y=50, label="$$$", color="green") +
annotate(geom="text", x=0, y=25, label="$$", color="green") +
annotate(geom="text", x=0, y=0, label="$ 0", color="green") +
annotate(geom="text", x=2, y=0, label="minutes\nof effort", size=3) +
annotate(geom="text", x=20, y=0, label="months\nof effort", size=3) +
annotate(geom="text",x=3, y=85, label="Little (or no) Planning\nHIGH COST", color="red") +
annotate(geom="text", x=18, y=85, label="Paralysis by Planning\nHIGH COST", color="red") +
geom_vline(xintercept=0, linetype="dotted") +
geom_hline(yintercept=0, linetype="dotted") +
geom_vline(xintercept=10) +
geom_segment(data=d, mapping=aes(x=arrow.x, y=arrow.y, xend=arrow.x.end, yend=arrow.y.end),
arrow=arrow(), color="blue", size=2) +
annotate(geom="text", x=8, y=95, size=2.3, color="blue",
label="we prefer to be\non this side of the\nloss function") +
ggtitle(my.title) +
theme(axis.text.x=element_blank(), axis.ticks.x=element_blank(),
axis.text.y=element_blank(), axis.ticks.y=element_blank())
Now sometimes you need to make this investment! (Think nuclear power plants, or constructing aircraft carriers or submarines.) Don’t get caught up in getting your planning investment perfectly optimized — but do be aware of the trade-offs, and go into the decision deliberately, based on the risk level (and regulatory nature) of your industry, and your company’s risk appetite.
It can be difficult to focus on strategy when your organization has to comply with standards and regulations. Tracking and auditing can be tedious! If you’re a medical device manufacturer, you may need to maintain ISO 13485 compliance to participate in the supply chain. At the same time, you’ve got to meet all the requirements of 21 CFR 820. You’ve also got to remember other regulations that govern production and postmarket. (To read more about the challenges, check out Wienholt’s 2016 post.) There’s a lot to keep track of!
In Is Good, Good Enough for You? Taking the Next Step After ISO 9001:2015, former Baldrige Program Executive Director Harry Hertz outlines similarities and differences between ISO 9001:2015 and Baldrige. After examining complements, Harry shows how Baldrige helps organizations grow beyond the conformance mindset:
I have not shared all the commonalities of or differences between ISO 9001:2015 and the Baldrige Excellence Framework. Instead, I have tried to show the organizational possibilities of building on conformity assessment to establish a holistic approach for achieving excellence in every dimension of organizational performance today, with a look to the strategic imperatives and opportunities for the future. Baldrige helps an organization take this journey with a focus on process (55% of the scoring rubric) and results (45% of the rubric), recognizing that great processes are only valuable if they yield the complete set of results that lead to organizational sustainability… I encourage organizations that have not gone beyond conformity to take the next step in securing your future.
It can be difficult to focus on strategy when your organization has to comply with standards and regulations. Tracking and auditing can be tedious! If you’re a medical device manufacturer, you may need to maintain