Quality and Innovation

In July, Paul Graham posted an article called “Maker’s Schedule, Manager’s Schedule“. He points out that people who make things, like software engineers and writers, are on a completely different schedule than managers – and that by imposing the manager’s schedule on the developers, there is an associated cost. Makers simply can’t be as productive on the manager’s schedule:

When you’re operating on the maker’s schedule, meetings are a disaster. A single meeting can blow a whole afternoon, by breaking it into two pieces each too small to do anything hard in. Plus you have to remember to go to the meeting. That’s no problem for someone on the manager’s schedule. There’s always something coming on the next hour; the only question is what. But when someone on the maker’s schedule has a meeting, they have to think about it.

For someone on the maker’s schedule, having a meeting is like throwing an exception. It doesn’t merely cause you to switch from one task to another; it changes the mode in which you work.

I find one meeting can sometimes affect a whole day. A meeting commonly blows at least half a day, by breaking up a morning or afternoon. But in addition there’s sometimes a cascading effect. If I know the afternoon is going to be broken up, I’m slightly less likely to start something ambitious in the morning.

This concept really resonated with us – we know about the costs of context switching, but this presented a nice concept for how a developer’s day can be segmented such that ample time is provided for getting things done. As a result, we attempted to apply the concept to achieve more effective communication between technical staff and managers. And in at least one case, it worked extremely well.

Case: Ron DuPlain (@rduplain) and I frequently work together on technical projects. I am the manager; he is the developer. More than we like, we run into problems communicating, but fortunately we are both always on the lookout for strategies to help us communicate better. We decided to apply the “makers vs. managers” concept to meetings, to see whether declaring whether we were having a maker’s meeting or a manager’s meeting prior to the session would improve our ability to communicate with one another.

And it did. We had a very effective maker’s meeting today, for example… explored some technical challenges, worked through a solution space, and talked about possible design options and background information. It was great. As a manager, I got to spend time thinking about a technical problem, but temporarily suspended my attachment to dates, milestones and artifacts. As a developer, Ron got the time and attention from me that he needed to explain his challenges, without the pressure of knowing that I was in a hurry and just needed the bottom line. As a result, Ron felt like I was able to understand the perspectives he was presenting more effectively, and get a better sense of the trade-offs he was exploring.

We had the opportunity to meet on the same terms, all because we declared the intent of our meeting up front in terms of “makers” and “managers”. Thanks Paul – this common language is proving to be a powerful concept for achieving a shared and immediate understanding of technical problems.

Today, I spent some time in a remote visualization tutorial presented by John Clyne of NCAR. He referenced a 2005 answer to the question “What is meant by interactive analysis?” by Mark Rast of the University of Colorado:

Definition: A system is interactive if the time between a user event and [the system's] response to that event is short enough to maintain my full attention.

If the response time is…

1-5 seconds: I’m engaged
5-60 seconds: I’m tapping my foot
1-3 minutes: I’m reading email
>3 minutes: I’ve forgotten why I asked the question!

I liked this because it defines a quality attribute: a high-quality interactive system maintains the user’s attention.

If you are part of the software development world, no doubt you are familiar with “not invented here” (NIH) syndrome. It is the scourge of the software development culture, the unfortunate tendency within a group of software-minded people to attribute value to the code that members of the group or the group itself has written, while devaluing code, modules or COTS packages that have not been written by members of the group.

“Not invented here” is so prominent that it has a wikipedia entry, with text that assures us that this tendency is indeed a facet of many a social, corporate or institutional culture. Bloggers and even Harvard Business Review have touted its benefits, suggesting that this characteristic of a culture may catalyze innovation.

Today, I attended a meeting where I had an even bigger revelation about NIH. About 10 people attended, and we talked about how to search a large archive of metadata across multiple data sources. Attendees spoke of the problem as something that really needs to be done, as something that our organization really needs to spend time on – and we need resources to do it. There’s only one problem with this picture – a couple of people within the organization have been working on this problem for the past 18 months, have produced a prototype that’s consistently getting about 100 hits a day (which is substantial given the problem domain), and have received positive reviews and helpful suggestions for moving forward from the user community. The releases have been published in inter-organizational emails, the company newsletter, and other venues where it would be very easy for everyone in this meeting to have learned of the new functionality and used it. But apparently no one has bothered to pay attention!

The moral of the story: when a NIH culture is observed, perhaps the resources and opportunities that are available to a group or an organization that could use them are truly invisible to the people who need them. The people can not see the opportunities because they are not looking; they are not paying attention.

Is paying attention to opportunities a value within your software development organization? It requires conscious effort.

(I originally wrote this article for the ASQ Software Division Newsletter compiled in the first quarter of 2009. I’m reproducing it here because I’ve found the ITEA criteria to be remarkably useful for all kinds of planning since I was introduced to it last year.)

For software professionals, particularly those of us who manage product development or development teams, it is important to track progress towards our goals and to justify the results of our efforts. We have to write effective project charters for software development just to get things moving, evaluate improvement alternatives before making an investment of time and effort in a process change, and ultimately validate the effectiveness of what we have implemented.

This past fall, I had the opportunity to serve as a preliminary round judge for the ASQ International Team Excellence Award (ITEA). My subgroup of judges met at the Bank of America training facility in Charlotte, North Carolina, where we split up into teams to evaluate almost 20 project portfolios. A handful of other events just like ours were held at the same time across the country, giving many people the opportunity to train and serve as judges. Before we evaluated the portfolios, we were all trained on how to use and understand the ITEA criteria, a 37-point system for assessing how well a project had established and managed to its own internal quality system. The ITEA criteria can be applied to any development project or process improvement initiative in the same way that the Baldrige criteria might be applied to an organization‘s strategic efforts. For software, this might include improving the internal processes of a software development team, using software improvements and automation to streamline a production or service process, and improving the performance or quality of a software product. (For example, I can envision the ITEA criteria being used to evaluate the benefits of parallelizing all or part of a software system to achieve a tenfold or hundredfold performance improvement.)

You can review these criteria on the web at http://wcqi.asq.org/2008/pdf/criteria-detail.pdf yourself. There are five main categories in the ITEA criteria: project selection and purpose, the current situation (prior to improvement), solution development (and evaluation of alternatives), project implementation and results, and team management and project presentation. An important distinction is in the use of the words Identify/Indicate, Describe and Explain within the criteria. To identify or indicate means that you have enumerated the results of brainstorming or analysis, which can often be achieved using a simple list of bullet points. To describe means that you have explained what you mean by each of these points. To explain means that you have fully discussed not only the subject addressed by one of the 37 points, but also your rationale for whatever decisions were made. Sustainability of the improvements that a project makes is also a major component of the ITEA criteria. Once your project is complete, how will you ensure that the benefits you provided are continued? How can you make sure that a new process you developed will actually be followed? Do you have the resources and capabilities to maintain the new state of the system and/or process?

The ITEA criteria can serve as a useful checklist to make sure you‘ve covered all of the bases for your software development or process improvement project. I encourage you to review the criteria and see how they can be useful to your work.

I was talking to a group of professors from James Madison University yesterday, when the topic shifted to “discrete event simulation”. They asked me if I knew anything about it – I said no. I don’t think I had ever heard of those words strung together in the same phrase, and so immediately assumed that this domain was just something I had never been exposed to. They also told me that they were using ProModel to do a lot of their discrete event simulation.

I’m addicted to learning new things, so I checked out to see what ProModel was all about so I would know what “discrete event simulation” was. And guess what! That’s the term for plant layout, for simulating new manufacturing processes, for designing kanban systems, for designing supply chains and other networks, for setting up queueing systems, for doing SPC and for managing your Six Sigma analytics. So I guess I do know something about “discrete event simulation”! I’m also wondering why I’ve never seen this product advertised in Quality Progress – it looks like it’s pretty good, and pretty comprehensive.

This reminded me that jargon can impede or enhance communication, depending upon the capabilities and understanding of the communicators. Neil Ward-Dutton, in a December 2006 post, collected some articles on this theme and reflected on them in “On jargon, and creating a common language.” He contrasts a message about the benefits of Web 2.0 presented in two ways: one filled with technical jargon, explaining the way it came to be, and one explaining the same thing but from the perspective of how Web 2.0 influences and affects people. The use of jargon – or the avoidance of jargon – can either communicate competence in a field or alienate people who need to know more about it.

Awareness of whether a term or a phrase is jargon can help us understand whether we are communicating accurately.

If I was aware of the nature of the term “discrete event simulation” I would have said “Sure! I really like discrete event simulation. In fact, I really enjoy designing plant layouts (which can be useful for designing software systems too), I am insanely enthusiastic about inventory models, and these are the kinds of analytical things that we do in Six Sigma projects all the time.” But no, I missed an opportunity to communicate – and maybe even to learn new things about modeling – because of jargon.

It reminds me of when I was a meteorology student several years ago. In one of our dynamics classes, I was dumbfounded by the number of times the professor referred to “zonal” and “meridional”. I had no idea what these two words meant – I could guess, but I might be wrong – so I searched all through our textbooks to find anything that would tell me about these two words. They were NOWHERE. And the dictionary was no help either. So one day I asked the professor, in class, what “zonal” and “meridional” meant. Her response is etched in my psyche forever: “If you don’t know what those words mean, then you shouldn’t be in this class.” Now this was in the days before Google, so I couldn’t just go look it up. What do you think I did? I felt totally embarrassed, crushed because I didn’t know something that was apparently so easy, decided to hide my lack of knowledge, and struggled through the class. I was even too embarrassed to ask my classmates. Years later, when I figured out the answer to my simple question, everything fell in place and I understood what went on!
The far more constructive answer from my professor would have been: “Zonal refers to the east-west direction and meridional refers to the north-south direction. So if we have zonal flow, it’s oriented predominantly east to west, and if we have meridional flow, it’s oriented primarily north to south.” My response would have been “Excellent! That’s simple! Now I understand what the equations are trying to say!”

The lesson here: no questions are stupid. Sometimes, a stupid question just reflects that someone’s trying to break through the barrier of jargon. This is a positive thing – it means they’re trying to figure stuff out! After this experience with my dynamics professor, I vowed that I would never think someone was totally stupid if they were asking (what I thought was) a simple question. I hope my coworkers and staff members feel that I’ve followed through on this.

(It reminds me of another time in that same course. We did a lot of multivariate calculus and differential equations, and the professor kept referring to “zed,” but for the life of me I couldn’t find the “zed symbol” in any of the equations. And none of my books would tell me what the “zed symbol” looked like. I’ll leave this joke as an exercise for the reader.)

How you perceive quality is influenced by your expectations. And sometimes, your expectations are subconscious or emotionally driven.

For example, a product may have all the features you, as a consumer, could possibly want and need – and it might perform well too! But it still might not satisfy everyone, or generate the magnitude of sales that were originally projected. How could this be?

Understanding the psychology of quality and value, based on affect, provides insight into how this can happen. Merriam Webster’s Medical Dictionary defines affect as “the conscious subjective aspect of an emotion considered apart from bodily changes.” In short, affect describes how something makes you feel. For example, working on a task that you really enjoy promotes positive affect. Spending time with “de-energizers” who are negative, critical, and generally unhappy can create negative affect.

Research in psychology indicates that positive affect corresponds with the ability to solve problems more readily and effectively, while negative affect can impede problem solving, even for simple tasks. As a result, usability can be considered a function of the positive or negative affect that is generated when a user interacts with a product. This applies to all products, including software and web-based applications.

These studies also suggest that effective design translates to positive affect – meaning that before use, perceived quality and perceived value are more closely related to the perceived quality and value that will be experienced after use. Aesthetics thus play a role in promoting positive affect. As interpreted by Don Norman (2004) in Emotional Design, where many of the aforementioned studies are referenced,

“the emotional system changes how the cognitive system operates… [it is] easier for people to find solutions to the problems they encounter… [there is a] tendency to repeat the same operation over again is especially likely for those who are anxious or tense.”

An entertaining example is the ATM case, which I’ll write about tomorrow.

I’ve been thinking a lot about management fads lately, and ran into this 2005 article by Nick Carr, titled “Does Not Compute”. Here’s the part that caught my eye:

“A look at the private sector reveals that software debacles are routine. And the more ambitious the project, the higher the odds of disappointment. It may not be much consolation to taxpayers, but the F.B.I. has a lot of company. Software hell is a very crowded place.”

Carr continues by describing two examples of failed projects: a massive systems integration effort at Ford Motor Company, and a overzealous business intelligence initiative embarked upon by McDonald’s. Both projects were cancelled when the price tags got too big: $200M for Ford, $170M for McDonald’s. The catch is that failure is good, because when we fail we at least know one solution path that’s not workable – we just need to 1) understand that it doesn’t have to be expensive, and 2) have more courage to allow ourselves and our colleagues to fail without getting depressed or thinking our coworkers are idiots. This is often expressed as “fail early, fail often“. (But note that the assumption is that you persist, and as a result of the learning experience, ultimately meet your goals.)

Without an effective team culture, rational managers, healthy relationships with stakeholders, and capable programmers dedicated to continually improving their skills, all roads can lead to software hell. The process of getting there – which is hellish in and of itself – is the famed death march. This is where a software-related project, doomed to fail, sucks up more time, people, resources, and emotional energy at an ever increasing rate until the eventual cataclysm.

Carr also cites The Standish Report, which in 1994, asserted that only 16% of projects were completed on time, and budget, and meeting specifications. By 2003 the percentage had grown to 34% in a new survey. Other projects that were still completed ran, on average, 50 percent over budget. (And this is for the survey respondents who were actually telling the truth. I know a few people who wouldn’t admit that their project was quite so grossly over budget.)

One way to solve this problem is by focusing on sufficiency and continuous learning, starting the blueprint for a system based on these questions:

• What features represent the bare minimum we need to run this system?
• What are the really critical success factors?
• What do we know about our specifications now? What do we not know?
• What do we know about ourselves now? What do we want to learn more about?

Software development is a learning process. It’s a process of learning about the problem we need to solve, the problem domain, and ourselves – our interests and capabilities. It’s a process of recognizing what parts of building the solution we’re really good at, and what parts we’re not so good at. Let’s start small, and grow bigger as we form stronger relationships with the systems that we are developing. Having a $170M appetite sure didn’t get McDonald’s anywhere, at least in this case.