Category Archives: Measurement

Eight Ways to Deal with Complexity

There is not one person I know who doesn’t have to deal with complexity in their work or personal lives. Either the subject matter they work with is complex or specialized, the politics are stressful, or there’s just too much information to process and the information overload becomes oppressive.

Fortunately, dealing with complexity has been the subject of recent research and there are some lessons to report. These lessons revolve around the importance of “sensemaking” – a term coined by Karl Weick to reflect concerted effort made to understand the relationships between people, technologies, places and events, how they behave, and how to respond to their current and future behaviors.

Weick and Sutcliffe (2001) studied the environments of aircraft carriers, nuclear power plants, and fire fighting teams – situations where the stakes are high, and on-the-job errors can be costly and dangerous. These are the workplaces “that require acute mindfulness if they are to avoid situations in which small errors build upon one another to precipitate a catastrophic breakdown.” Snowden (2003), who worked at IBM, said that most environments just weren’t like that – so it would be difficult to generalize how those workers dealt with complexity to the typical office.

Browning & Boudes (2005) compared and contrasted these two studies to try and define some guiding principles for how people make sense of complex scenarios. Here’s a synopsis of what they found:

1: “Acknowledging and accepting complexity is better than placating it with planning models. There are simply too many situations where the standard tools and techniques of policy-making and decisionmaking do not apply.” The lesson: move away from a “training culture of obedience” and towards a “learning culture of understanding and action”. (This will be a challenge, because it requires humility and trust.)

2: “It is important to acknowledge failure and learn from instances of it.” Self-explanatory, but as above, this requires humility and ego-transcendence. For people to learn from failure, they must first confront it. I can think of some “death march” software projects where failure “never” occurred!

3: “Self-organization is an order that has no hierarchical designer or director.” Browning & Boudes cite Peter Drucker’s idea that “in the Knowledge Economy everyone is a volunteer.” Volunteerism means that everyone is fundamentally qualified to do some part of a project, that roles shift to accommodate existing talents and the development of new talents, and that if a person isn’t working out in one role, they can move to another. In volunteer contexts, leadership is often dynamic, where everyone serves for a time as the leader and then moves on.

4: “Narratives are valuable for showing role differentiation and polyvocality.” There are many voices, and many perspectives, and these can be effectively communicated when people relate to one another through stories and examples. Diversity of opinion and distance from a problem can help raise solutions – but the people who know the situation best, and who are closest to it, must be open to the possibilities. (Easier said than done.)

5: “Conformity carries risks, and thus we need diverse inputs when responding to complexity.” The authors suggest that learning should be valued over order and structure. This does not mean that order is unnecessary, but that any order that is established should be viewed as temporary – a framework to serve as the basis for new learning.

6: “Action is valuable under conditions of complexity.” Acting, learning, and adjusting is more effective and more productive than trying to identify the right solution, plan it, and then do it. Action builds identity and creates meaning; “the most useful information is contextual and need-driven.”

7: “The focus is properly on small forces and how they affect complex systems.” The authors suggest that focusing on small wins and keeping things simple is a strategy for success. I love the following story that they describe, so I have to include it here:

Snowden relates a story of two software development groups – one expert, the other a lesser group – whose experience in programming was limited to the fairly routine requirements of payroll systems. In a competitive exercise between these two groups for learning purposes, the experts created a plan for an elegant piece of code that would take two months to develop. The payroll programmers, meanwhile, downloaded a “good enough” list from the Internet that cost five dollars (Snowden, 1999). Thus one feature for smallness for Snowden is the decisions that can be made that allow the group to move on – to accept “good enough,” implement it, and then see what that action means.

8: “It is important to understand the irony of bureaucratic control.” Producing data and information can be overwhelming; innovative achievements can be suffocated by measurement, evaluation and control. We assume that organizations are deterministic and behavior can be programmed, using carrots and sticks. But people are neither rational nor linear, and this can be both the strength of the organization and its Achilles heel.

New organizational models are needed to be able to follow this advice. So many of the projects I see today are like solving mysteries: you know what needs to happen (because you have requirements documents), there are motivated people all around you who really want the mystery solved, someone is footing the bill (at least for a while), and everyone wants to make progress. But because the process is full of learning new information, finding clues, and relating to people’s needs – it’s impossible to put a timeline on it. This frustrates project managers (and their bosses) immensely, because their jobs revolve around bringing some order to complexity and setting expectations in terms of time and budget.

Can you imagine a crime show like Law & Order starting out with a murder – and then you cut to the scene where the police chief says: “We need to solve this murder immediately… all the clues need to be found by next Friday, all the suspects interviewed by the following Wednesday, and then I want a report on my desk with the solution three weeks from today!”

It sounds funny, but this is exactly what plenty of managers are trying to do right now in their own organizations. The “corporate consciousness” will support this kind of behavior until a rational alternative is found.


Browning, L. & Boudes, T. (2005). The use of narrative to understand and respond to complexity: a comparative analysis of the Cynefin and Weickian models. E:CO, 7(3-4), 32-39.
Snowden, D. J. (1999). “Story telling: An old skill in a new context,” Business Information Review, 16 (1): 30-37.
Weick, K. E. and Sutcliffe, K. M. (2001). Managing the Unexpected: Assuring High Performance in an Age of Complexity, San Francisco, CA: Jossey-Bass.

Quality in your Genes

Is your body programmed to get cancer? How about Parkinson’s, or multiple sclerosis, or diabetes? With 23andme.com, you could find out if you are predisposed to these or 19 other diseases simply by spitting in a cup. In addition, you can easily donate your data to further medical research. These are the same kinds of tests used to diagnose cystic fibrosis and Down’s syndrome prenatally, and have earned a measure of trust within the medical community. Now, genetic testing is a service offered to just about anyone – even outside the hospital or doctor’s office. (It’s not free, but what’s $399 for a little peace of mind – assuming that’s what you end up with?)

That’s not where the fun ends, though. If you’d like to meet other people who share similar genetic characteristics, no problem – one someone writes a mashup between a social networking site like Facebook and your 23andme genome data, it will be as easy as adding a new tab to “My Networks”.

Diagnosing genetic profiles is a high-stakes game for two reasons: quality and ethics. The quality of the genetic testing procedures, the calibration of the lab instruments, and the handling of the samples are all aspects of the process that must be carefully controlled. Failures can result in minor consequences (e.g. a sample is lost and the subject has to spit in a second cup, then wait longer to get the results) or major consequences (someone is incorrectly told that they are predisposed to a life-threatening condition, and because they can’t bear to get it, they commit suicide). Fortunately, quality control is taken very seriously in this domain, as the QC process from Kalman et al. (2005) indicates.

The ethical issues can be a direct result of the capability of the process to produce accurate and reliable results. But is it appropriate to offer this service on such a large scale? What are the consequences of knowing versus not knowing, on both a personal scale and on the scale of society? Without probing these issues in more depth, and understanding how laws and regulations could counterbalance potentially austere social consequences, it’s a risky service to offer. Some people might be able to handle the information; some might not. (Imagine being in high school and getting dumped by someone who checked your genome, found out you’re predisposed to psoriasis, and thinks that’s gross.)

I’m not sure how I would respond personally to having such information, assuming that it was accurate and my sample wasn’t swapped in the lab with someone else’s. But the concept is interesting enough that I just might try their demo and explore the genomes of the Mendels, a hypothetical family.


Kalman, L., Chen, B., & Boone, J. (2005). The genetic testing quality control materials program (GTQC) – a sustainable community process to improve availability of appropriate, verified quality control (QC) materials for genetic testing.
Kotler, S. (2008). Spitomics: Spit-in-a-cup genomics has arrived: now what? Science Progress, 10/28/2008.

What are Standards?

Standards represent “accepted ways of doing things” and can be defined as sets or features of characteristics that describe products, processes, services or concepts. (National Research Council, 2005) Alternatively, standards can be described as “rules, regulations, and frameworks intended to enable individuals, groups, organizations, countries and nations to achieve goals.” (Spivak & Brenner, 2001) According to these authors, there are five types of standards: 1) physical or measurement standards; 2) standards for symbols, meanings and language; 3) standards for products and processes, including test validation, quality assurance, and operating procedures; 4) standards for systems (including management systems and regulatory systems); and 5) standards for health, safety, the environment and for ensuring other consumer interests. Standards may be compulsory (as in the case of legal and regulatory standards, such as the Sarbanes-Oxley requirements for financial reporting) or voluntarily adopted (as in the case of product interface standards such as USB for computer peripherals).

Because standards are a mechanism used by social groups to promote norms, and to facilitate the creation of artifacts that advance human capabilities, standards are technologies. (Recall that Random House defines technology as “The sum of the ways in which social groups provide themselves with the material objects of their civilizations.”)

What can standards do for you and your projects? The importance of standards can be explained in terms of the eight goals that standards can be used to achieve (Blind, 2004):

  • Compatibility – facilitating cooperation between people, processes, systems, or technologies
  • Communication – facilitates information flow between people, processes, systems or technologies
  • Conformance – provides a common basis to establish competency or define excellence
  • Continuous Improvement – helps organizations leverage the “lessons learned” that are imbued within the standards
  • Establish Order – promotes consistency, reliability and effective communications
  • Prescribe Behavior – enables organizations to reduce variability between people and systems
  • Promote Social Order – establishes a basis for legal and ethical behavior
  • Establish Shared Meaning – provides a context for communication, compatibility and conformance

A standard is “working” if it accomplishes one or more of these goals (depending, of course, on how relevant and pertinent the goals are to the project that is being pursued). For example, it’s probably not very important for two computer devices to “promote social order” if they need to communicate. But it’s definitely important for people.


Blind, K. (2003). The economics of standards. Kluwer Academic Press.
National Research Council. (1995). Standards, conformity assessment and trade. National Academy Press.
Spivak, L. & Brenner, F.C. (2001). Standardization essentials: principles and practice. Marcel Dekker.

Extreme Measurement and the Trash Guy

“You can only manage what you measure.” Yeah, yeah, yeah. I’ve heard this statement so often that I’m not sure whether it’s a valuable heuristic or just a cliché. What I do know is that I (personally) tend to like measuring for two reasons: 1) I’m not good at remembering things that happened in the past, and so measurement provides me with an objective line in the sand that I can use to gauge improvement, and 2) for some unknown visceral reason, it just feels wrong not to do it.

But then I read the story of Ari Derfel, who I call the “Trash Guy” – and this completely shifted the way I look at measurement. Motivated by a dinner conversation with friends, he decided that he was going to “keep” his trash for one year to see what he was throwing away. Starting in December 2006, he stashed his trash in the basement (cleaned, of course) just to see what refuse he (personally) was producing. After some dedicated time measuring and observing his trash-producing habits, he found that his consumption habits and spending habits changed!

Here’s what he says about his “lessons learned”:

Some of the things that I learned are interesting. I learned what I spent most of my money on because by watching a pile of trash grow over a year, I really began to see, “Wow! I spent it on that food, on this electronic, on that item,” and my consumption habits and spending habits became really clear.

The second thing that I learned really powerfully, in addition to what I spend my money on, is what I put in my body. I started to see things pile up. The most commonly talked about are little stacks of pints of non-dairy ice cream that I would eat: pint one, pint two, pint three, pint 12, pint 15. I started to see what lives in my body, and what kind of fuel I’m choosing to put in my body.

Then, I learned where most trash seems to be made, food packaging. Of all the different things that could be making trash, that was really profound to me because I realized that it’s not that big of a problem. I mean, we’ve only been packaging food for 50, 60 or 75 years. So, if that’s the small amount of time in which the problem was created. We should be able to undo the problem. Those are three of the primary things that I learned.

Then I learned that if I composted everything organic, which I did, trash doesn’t smell. That’s an awesome thing to learn because most folks think of the dump or, trash and they think it smells really disgusting. I realized, “Wow! That’s not the case.” If we properly treat all of our organic matter, that’s not going to be a problem.

Measurement can motivate people – in a profound way – to change their behavior. After reading about Ari’s trash escapades, I started thinking about the trash I produce. I’m looking at all those plastic bottles I drink water from with a little disdain now. I’m saving a lot more of those empty Country Crock containers. I’m far more conscious of the proliferation of plastic bags in modern food packaging. I’m really looking fondly at that burlap bag with the 15 lbs of rice in it… I can use that for something else later, when the rice is gone. I’m actually cooking dinner more – from scratch. And I haven’t even started seriously measuring yet.

Recent Entries »