Quality and Innovation

This morning for breakfast, I chose the “nutritious” option of a slice of buttered rye. After the obligatory ninety seconds, my freshly toasted bread popped out of the toaster. It was hot, with a little steam coming off the sides – pretty tantalizing for a frigid winter’s morning. But I had forgotten to take the butter out of the refrigerator earlier – arrhgrhh, I thought, now I’m going to have to cut cold butter and try to get it to spread on my rapidly cooling toast.

I hate cutting cold butter. The butter itself is unwieldy – it tends to act like a magnet (at least for me), consistently tipped the wrong way, repelling the knife at every attempt. It’s an altogether unpleasant experience, certainly not a complement to a rushed morning where you’re trying to get yourself ready for work and kids ready for school.

But — microwave that butter for about 15 seconds and everything changes. All of a sudden, the butter is at your command. The knife slices through effortlessly, like an airy ballet. The pat of butter, liberated from its stick, conforms to the shape of your toast with only a few strokes.

It’s my opinion that the software you use should feel the same as the experience of cutting the warm butter. But all too often, software feels far more like the cold butter. You try to take a decisive step with the tool, but the application jumps out from under your control. It falls on the ground, sometimes things get dirty, and it can become a struggle to work with the tool thereafter. That is, until you’ve had a while to adjust to the software, and it’s had time to adjust to room temperature and become more malleable. Just as the butter eventually adjusts to a temperate environment, with extended use, a user will adapt to software and be able to work with it (not always due to changes in the software, but to changes in the human-software sociotechnical system nonetheless).

Enter the Butter Test. The Butter Test is the equivalent of the “5-Second” test for user interface navigability – but for software applications, web pages, web applications, APIs, or any other software-related design. (There’s even a web app that helps you conduct a formal 5-Second test.) Whereas the 5-Second test gauges your user’s first impressions when they visit your web page, the Butter Test assesses how malleable the software is upon a user’s first encounter. To do the Butter Test, spend about five minutes with a new application. You don’t have to be alone; you can get a walkthrough from someone who’s more familiar with it. How does it feel? Do you feel like you’re struggling with a cold stick of butter? Does the software respond in a jagged, unpredictable way – forcing you to catch it before it falls on the floor? Or alternatively, is your first cut at using the software smooth? Do the results feel trustworthy, interpretable, and extendable (meaning you’re left feeling empowered to do more)? Using the Butter Test, your first impressions count.

The Butter Test is not just useful for subjectively evaluating full software applications. Just today, I used it to determine whether a taxonomy for a directory structure made sense. We needed a file structure for holding different types of data (with different levels of “ease to reproduce”) from different instruments. After being walked through the new structure, I could immediately figure out where new data would go, how we would adapt to novel data types and processed data products, and how to access the data without an a priori knowledge of the full directory structure. By learning a few rules, I could work easily with the entire collection of data – and I learned all this in less than five minutes. The new taxonomy passed the Butter Test with flying colors.

I have been using the Butter Test for about 15 years and it does not disappoint. Trust your instincts. If your software was toast and butter, would you be content or frustrated?

Tom Erickson, in his introduction to one of the sections in the forthcoming Handbook of Research on Socio-Technical Design and Social Networking Systems, explains socio-technical design well:

“Socio-technical design is not just about designing things, it is about designing things that participate in complex systems that have both social and technical aspects. Furthermore, these systems and the activities they support are distributed across time and space. One consequence of this is that the systems that are the sites for which we are designing are in constant flux. And even if we were to ignore the flux, the distributed nature of the systems means that they surface in different contexts, and are used by different people for different (and sometimes conflicting) purposes.”

Socio-technical design is, understandably, related to sociotechnology. There is much work to be done to develop the processes and techniques that will be required to manage quality and continuous improvement in the context of socio-technical design!

Technology is the “sum of ways in which social groups construct the material objects of their civilizations.” The things that we use – the “design artifacts” of the processes used to build them – are socially constructed to the same extent that they are technically constructed. The convergence of technological and social insights in the creation, construction and use of artifacts is sociotechnology.

For example, we typically build a bridge when there’s some expectation that people need to get from Point A to Point B, and there’s something they need to bypass along the way (e.g. a river, a canyon, another road). Failure to consider the social factors as well as the technical factors could lead to a “bridge to nowhere” – and we all know at least one person who’s had a problem with those. Non-technical factors pertaining to the environment in which an idea is created and implemented are crucial.

According to Bunge (1998), sociotechnology is the process of applying insights from the social sciences to design policies and programs. More specifically, this is how Gingras & Niosi (1990) explain Bunge’s perspective:

Ten years ago we were talking about the convergence of customer touch points: phone, fax, web, cell phones, email and regular mail. With handhelds and mobile devices becoming more and more ubiquitous, and services like Facebook becoming more integrated into our daily lives, the next convergence is between people and the technologies we use. The boundaries are becoming increasingly blurred, and the impact of this convergence on business must be explored. Reviewing research by pioneers like Tom Erickson is a good place to start. We are all becoming sociotechnical.

(Note: Sociotechnology is an important part of socio-technical design.)

---

Bunge, M. (1985). Philosophy of science and technology. Vol. 7 of Treatise on basic philosophy, Dordrecht: Holland.
Bunge, M. (1998), Social Science under debate. A Philosophical Approach. Toronto University Press: Toronto.
Gingras, Y. & Niosi, J. (1990). Technology and society: a view from sociology, in Georg Dorn and Paul Weintgartner (eds.) Studies on Mario Bunge’s Treatise, Poznan Studies in the Philosophy of Science, Amsterdam and Atlanta, 421-430. Retrieved from http://www.archipel.uqam.ca/506/01/On_Bunge.PDF
Nieto, C. C., Neotropica, F., & Durbin, P. T. (1995). Sustainable development and philosophies of technology. Society for Philosophy and Technology, Vol. 1, Fall 1995. Retrieved from http://scholar.lib.vt.edu/ejournals/SPT/v1n1n2/nieto.html – (Note: I added this one simply because I really like it, and it’s related to the discussion on sociotechnology.)

If you’re a software developer, software reuse is kind of like the Holy Grail.
You probably think it’s a good idea already. Plus, managers always want more of it.

But achieving software reuse in practice can be difficult. It’s one thing if you can use really stable, trusted code libraries and treat them as black boxes in your code. Unfortunately this only works if those black boxes don’t run into problems, even when you deploy them on hardware that’s different than what the original writers used. (Good luck.)

Here’s why I think software reuse is difficult.

First, we need to note the difference between explicit knowledge and tacit knowledge. Explicit knowledge is what you get from books, or memos, or hearing a story or experience that someone else shares with you. Tacit knowledge is what you get from experimentation, mentorship, or exploration. (For example, I have no explicit knowledge of the vi editor – but I have extensive tacit knowledge of it, because when I open the editor my fingers know what to do, darting back and forth between ESC and : and other punctuation marks that my brain is not consciously processing. Perl is kind of the same way for me.)

If software is the executable representation of knowledge, then developing software is the process of codifying (literally!) tacit and explicit knowledge to make it executable. That’s a huge challenge that requires the developer to learn, reflect, experiment, learn, reflect, experiment… and so on. It’s a process of learning that ultimately results in software – a tool that hopefully will earn a life of its own, perhaps extending beyond the time that the original developer works on it.

What do you get by reading and exploring someone else’s code? Explicit knowledge. What do you get by writing code yourself, by uncovering each new function one at a time, by realizing you didn’t know what you were doing the first time, by refactoring, by trying out a new design (or maybe two or three), by bouncing ideas off of your colleagues, or by going back to the first design you had a long time ago? That’s tacit knowledge. You need both to write good software.

Unless there’s some way to get that tacit knowledge of someone else’s code, or you don’t need the tacit knowledge anyway (ie. in the case of the “black box”), successful reuse will be challenging… if not impossible.

(Related Post –> Software Reuse Antipatterns)

In 2000, Scott Ambler wrote  an excellent article on the organizational aspects of software reuse. He talked specifically about patterns and antipatterns:

“A pattern is a common approach to solving a problem that is proven to
work in practice. Conversely, an antipattern is a common approach to solving a problem that leaves us worse off than when we started.”

Long (2001) built on Ambler’s work and made it more fun. (This is, in my opinion, one of the most interesting and entertaining articles about software reuse in existence. You can get the full text from the ACM if you have an account there.) Long calls antipatterns obvious, but wrong, solutions to recurring problems and characterizes four organizational approaches that don’t support successful software reuse. See if your organization is one of these:

• Abracadabra Model: A high-level manager is frustrated with a perceived lack of reuse and declares that “reuse will happen”. What Happens: Lots of talk, no action, silo development continues, managers start to panic, then the organization “de-evolves” into the next model.
• High Noon Model: A high-level manager is REALLY frustrated with a perceived lack of reuse and declares that “reuse will happen”. What Happens: Finger pointing, as everybody has a lot of reasons (many of them very good, and very accurate) about why reuse can’t possibly work. The de-evolution continues.
• Cost Cutter Model: A high-level manager is REALLY, REALLY frustrated with a perceived lack of reuse and declares that “reuse must happen to cut costs”. What Happens: Software people start to “force” reuse, immediate costs go up, upper management gets nervous, and more finger pointing happens (as everybody finds even more reasons now – including higher cost – for why reuse can’t possibly work.)
• Used Car Fiasco Model: Software group says “OK, we’ll try reuse.” One group has software it thinks about group can use, so it is made available as-is and with no support. What Happens: There are lots of bugs to fix. Reusers have to fix them because the originators don’t have the time or resources to solve the new group’s problems. The reusers get frustrated and then write the code themselves.

Note that in all models, the expectations and behavior of the managers doesn’t change. In the fourth model, the behavior of the software developers changes. At no point do the expectations of the software developers change – their mission is to do what they need to do to get the software written.

Tomorrow I’ll write about why I think software reuse is difficult. The antipatterns above provide a good foundation for that discussion.

---

Ambler, S. (2000). Reuse Patterns and Antipatterns. Dr. Dobbs Journal, February 1, 2000.
Long, J. (2001). Software Reuse Antipatterns. ACM SIGSOFT, Software Engineering Notes, 26(3), 68-76.

Software is the executable representation of knowledge. [1] As a result, I find that software development provides a fruitful basis for exploring how problem solving is done by diverse team members in a cooperative (or even combative!) context.

Here is one example. In June 1997, Tom Gilb wrote an article for Crosstalk on “Requirements-Driven Management”. He noted that his purpose was intended to discuss, among other things, “some of the current major problems in systems engineering.” I stumbled upon this article again over the weekend, and it’s still as relevant now as it was a decade ago.

Standards are a prerequisite for systematic continuous improvement; the means to achieve an improvement process, not the ends, according to Glib. Furthermore, he remarks that Deming’s perspective on continuous improvement establishes, in part, that the reason we should adopt standards is to normalize our project to other projects . Once we do that, we’ve opened the doors to be able to use industry best practices, derived from other projects who also applied the standards – so that we can “clearly see the effect of any changes experimentally introduced into a process and not have to worry too much about other potential factors that impact the results.”

Learning, learning, learning. It’s all about continuous improvement through continuous learning, and in presenting this, Gilb is essentially promoting the same philosophy as Alistair Cockburn’s Cooperative Game Manifesto for Software Development. Both see the learning process as the key to successful software development. (So why do we not focus on this aspect of development more?) Glib addresses the issue of learning directly:

“The time has come to recognize that projects are so large, so complex, so unpredictable, and so state-of-the-art new that we have no practical alternative except to maximize our learning process during the project and as early as possible in the project life. “

In this article, Gilb also presents “Evo” – short for “Evolutionary Project Management” – which has been used at IBM since 1970. It’s an implementation of Deming’s PDCA (Plan-Do-Check-Act) approach, and the author likens it to Humphrey’s Personal Software Process (PSP).

---

[1] This definition is credited to Eric Sessoms, who I consider a true artisan of software development. See some of his work at libraryh3lp.com. He likes the elegance of simplicity in his software.

The Center for Environmental Journalism (CEJ) recently posted an interview with Roger Pielke, Jr., an authority on (as CEJ calls it) “the nexus of science and technology in decision making”. The interview seeks to provide a perspective on how journalists can more accurately address climate change in the context of public policy over the next several years.

I was really intrigued by this part:

Reporters could help clarify understandings by asking climate scientists: “What behavior of the climate system over the next 5-10 years would cause you to question the IPCC consensus?” This would give people some metrics against which to evaluate future behavior as it evolves.

Similarly, you could ask partisans in the political debate “What science would cause you to change your political position on the issue?” This would allow people to judge how much dependence partisans put on science and what science would change their views. I would be surprised if many people would give a concrete answer to this!!

For the first question, Pielke is recommending is that we take an approach conceptually resembling statistical process control to help us figure out how to evaluate the magnitude and potential impacts of climate change. (Could we actually apply such techniques? It would be an interesting research question. Makes me think of studies like Khoo & Ariffin (2006), for example, who propose one method based on Shewhart x-bar charts to detect process shifts with a higher level of sensitivity – only tuned for a particular policy problem.) For the second question, I’m reminded of “willingness to pay” or “willingness to recommend” or other related marketing metrics. I’m sure that one of these established approaches could be extended to the policy domain (if it hasn’t been done already).