Category Archives: Software Quality

How Usability and (Software) Quality are Related

ISO 9241-11 defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” The four elements that define usability within this context are as follows:

  • both the users and goals must be explicitly identified,
  • the intended context of use must be identified and understood, and
  • the user can use the system in question
  • to meet those stated goals.

These same four elements are implied by the ISO 8402 definition of quality: stated and implied needs are relative to specific users with specific goals, are dependent upon a context of use, and the entity in question is the system being defined and developed in response.

Usability is the extent, or the degree, to which the above criteria are satisfied. Here’s an example from software development to make this a little more concrete. The software development lifecycle, regardless of what incarnation you’re using (even waterfall),  inherently addresses usability through these four elements:

  • the requirements process outlines the specified users, their goals, and the context of use
  • the design process defines a specific technical solution to meet those needs, and
  • the finished product provides evidence that the system can be used to achieve the goals of its users.

As a result, usability can be considered an implicit factor in software quality, ultimately reflecting how well the design process interpreted requirements within a specified context of use.

Why Software Reuse is Hard – More Perspectives

A couple weeks ago I presented my perspective on Why Software Reuse is Hard. I also posted a link on LinkedIn, and wanted to capture the comments that were received there. I think it’s interesting to note how many people support the concept of continuous learning for competitive advantage in software development – without actually saying it!

Mike Duskis – Software Quality Lead at CaridianBCT says:

I like the distinction between “black box” code that we can simply plug in on one hand and code that we need to understand “tacitly” on the other. I think that, deep down, most of us software geeks would love to experiment with the innards of every interesting gadget that we can get our hands on. That’s what makes us geeks.

In the real world, we can’t let our geek nature control us. We also need to be pragmatic engineers, producing optimal solutions with minimal waste. This is what drives us to adopt code written by someone else.

But the issue cannot be boiled down to a primal conflict between the inner geek and the outer engineer. As Nicole’s headline implies, software reuse is hard. Even after we make the decision to reuse existing code, we face all sorts of obstacles. There is a whole lot of lousy code floating around out there. In fact, I would argue that most code available for reuse is poorly designed, badly documented, and ripe for mishap or exploit. Doubters might tour a few random projects at . The good ones are very good, but they are vastly outnumbered. The only way to know for sure that a component is not lousy is to tear it open and examine its guts, but doing so would expend the very resources that we are trying to save. Even if we assume that the code is not lousy outright, its developer was probably motivated by different problems and circumstances than we face. Ironically, the more efficient the original developer was, the less likely it is that he produced something that can simply be plugged in to a completely different environment.

Over the years, languages, patterns, and paradigms have evolved to help us to resolve these difficulties, but there is always a cost. Java almost forces us to produce reusable components but at the cost of a cumbersome paradigm that encourages over-engineering. At the other end of the spectrum, Perl and Python are easy to use. They also encourage the development of quick-and-dirty code that can be very hard to reuse. I suspect I’ll get some flack for this, but I’ve struggled with both extremes enough to develop “tacit” knowledge of them. If there is a happy medium, I think it lies in a way of thinking rather than a language, but every step that we take from the extreme represented by orthodox JEE (with beans) is a step away from the “black box” that allows us to simply plug a component in and forget about it.

On balance, I think that reuse is not only desirable but necessary. Who has time to design new systems from the operating system on up to the code that actually solves our business problem? The trick is figuring out how to do it well.

Greg Zimmerman, Owner, Applied Quality Solutions LLC & Regional Councilor, ASQ Software Division says:

There’s no magic way to make reuse easier, but you can attempt to create a normalizing factor with service frameworks that define the structures, relationships, and communications. On top of that, build code generation tools that automactically create the common components within and around the objects.

I say this as though it’s simple. Nothing can be farther from the truth. It’s taken us the better part of three years to reach the point where this has become standard practice for new services. Integrating / updating existing services is still a challenge.

I guess I’ve just reiterated the point that reuse is hard, but as Nicole said, it’s not impossible. It requires committment from both developers and managers, and the patience (and funding) to lay the groundwork.

Steven Rakitin – Software Consulting, Training, Facilitating, and Auditing says:

Nicole there are several other reasons why software reuse is hard that have nothing to do with the forms of knowledge that you talked about. Imagine trying to come up with a reusable design for a new product. In order for the design to be truly reuseable, the designer would need to know the possible ways the design will be used in the future. If that information or knowledge does not exist, it becomes very difficult to design something that is truly reusable…

Reuse was a vary popular issue back in the 1980s and people envisioned “software factories” cranking out reusable, modular components that could be designed and tested once and reused many times. About the closest we’ve ever come to achieving that goal is defining standard interfaces (APIs) that designers can design to…

It’s hard because everytime we design a new app, we want to start from scratch… just the nature of software engineering and software engineers…

-Steve Rakitin-

Bryce Bolton – Electrical Engineer, Prof. Research Assistant at LASP / University of Colorado

Personalities and architecture are part of the difficulty.

Personalities: Everyone wants to do things their own way, and if they can’t quickly understand the architecture, or don’t like it, they will re-create it into something they understand.

Architecture: Even the best architectures create ongoing debates. Consider the battles between languages like Perl and Python. Verilog and VHDL, C and ADA. In each case, the first language is more freestyle & flexible while the second is more structured. This creates endless preference debates on a language level. Within teams there could be even more differences.

Good system architects within the projects lead the team toward a standard which is adopted by the other team members. So lack of reuse, or spaghetti code could indicate a lack of leadership in the team.

To converge, some team members must force their structure on others, or management must resolve to come up with a standard to overcome the “style battles”. Getting team members working together on this, and putting team members with different styles together on a project may help smooth differences.

Or, the organization could suggest a 3rd party platform or OS from which the project-specific standard is derived.

The goal isn’t to make everything reused or rigid, except where project requirements mandate that. The goal is to meet the system requirements, including reliability. So, if reuse means designing to a coding style that increases quality, then the effort matches programmatic requirements. If the code will not be reused, it is wasteful of project resources to design to something outside the scope of the project, even if it is an ideal.

We typically working within a project, not fully on ancillary goals or ideals. Yet, if functional teams do not gel and create standards across projects, all projects will be less effective.

Reuse is of particular interest in product platforms and successive projects where teams are re-using each others code anyhow. At some point it will pay off to converge on a coding style. Seeing this and pushing for such commonality is up to management or enlightened team members. In the beginning, though, it may be less likely to invest in reuse. The situation may be one of more rhetoric and less consensus.

Bryce Bolton

Alex Elentukh – Quality Assurance Director, Program Manager, Process Architect, Consultant, Writer

At my last engagement we had some significant progress with reusable test components. These are fully supported by BPT feature of Quality Center of HP. Component-based test design allows to curb the number of tests. Otherwise tests were growing into an unmanageable mass of many thousands. A predictable lesson from this experience is that reuse must be supported by tools and deployed with great care and attention. For example, there should be provision to regularly review, communicate and measure reusable components

Emmett Dignan – Experienced systems and software professional

How many of the issues of reuseability are simply issues of poor documentation, overly complex or overly large code blocks?

I have seen an organization use an internally developed library of functional modules for regularly reused needs. But it required a dragon of a software manager devoted to the concept.

But development time dropped substantially if you could just drop a known and tested module into the framework for the next program.

Would UML help with this?

The Butter Test

butterThis morning for breakfast, I chose the “nutritious” option of a slice of buttered rye. After the obligatory ninety seconds’ wait, my freshly toasted bread popped out of the toaster. It was hot, with a little steam coming off the sides – pretty atractive for a frigid winter’s morning. But I’d forgotten to take the butter out of the refrigerator earlier – arrhgrhh, I thought, now I’m going to have to wrestle with the cold butter and try to get it to spread on my rapidly cooling toast.

I hate cutting cold butter. The butter itself is unwieldy – it tends to act like an anti-magnet (at least for me), consistently tipping the wrong way, repelling the knife at every attempt. It’s an altogether unpleasant experience, certainly not a complement to a rushed morning where you’re trying to get yourself ready for work and kids ready for school.

But — microwave that butter for about 15 seconds and everything changes. All of a sudden, the butter is at your command. The knife slices through effortlessly, like an airy ballet. The pat of butter, liberated from its stick(!) conforms to the shape of your toast with only a few strokes.

But I spend more time managing software development than I do buttering toast.

So it’s become my opinion that the software you use should feel the same as the experience of cutting the warm butter. But all too often, software feels far more like the cold butter. You try to take a decisive step with your newly coded tool, but the application jumps out from under your control. It falls on the ground, sometimes things get dirty, maybe lands face down or picks up some hair or dust, and then it becomes a struggle to work with the tool thereafter. That is, until you’ve had a while to adjust to the software, and it’s had time to adjust to the “room temperature” of the deployed environment and become more malleable. Just as the butter eventually adjusts to a temperate environment, with extended use, a user will adapt to software and be able to work with it. This is not always due to changes in the software, but to changes in the human-software sociotechnical system – that is, you just develop an all-around better relationship with the app and it becomes easier to work with.

Enter the Butter Test.

The Butter Test is the equivalent of the “5-Second” test for user interface navigability – but for software applications, web pages, web applications, APIs, or any other software-related design. (There’s even a web app that helps you conduct a formal 5-Second test.) Whereas the 5-Second test gauges your user’s first impressions when they visit your web page, the Butter Test assesses how malleable the software is upon a user’s first encounter. To do the Butter Test, spend about five minutes with a new application. You don’t have to be alone; you can get a walkthrough from someone who’s more familiar with it. How does it feel? Do you feel like you’re struggling with a cold stick of butter? Does the software respond in a jagged, unpredictable way – forcing you to catch it before it falls on the floor? Or alternatively, is your first cut at using the software smooth? Do the results feel trustworthy, interpretable, and extendable (meaning you’re left feeling empowered to do more)? Using the Butter Test, your first impressions count.

The Butter Test is not just useful for subjectively evaluating full software applications. Today, I used it to determine whether a taxonomy for a directory structure made sense. We needed a file structure for holding different types of data (with different levels of “ease to reproduce”) from different instruments. After learning more about the proposed new structure, I could immediately figure out where new data would go, how we would adapt to novel data types and processed data products, and how to access the data without an a priori knowledge of the full directory structure. By learning a few rules, I could work easily with the entire collection of data – and I learned all this in less than five minutes. The new taxonomy passed the Butter Test with flying colors.

I’ve been using the Butter Test for about 15 years and it does not disappoint. Trust your instincts. If your software was toast and butter, would you be content or frustrated?

Why Software Reuse is Hard

If you were a software developer in the 1990’s or 2000’s, software reuse was kind of like the Holy Grail.

You probably thought it was a good idea in theory from the start. Plus, managers were always enthusiastic about wanting to get more of it.

But achieving software reuse in practice was, and still can be, difficult. It’s one thing if you can use really stable, trusted code libraries and treat them as black boxes in your code. Unfortunately this only works if those black boxes don’t run into problems, even when you deploy them on hardware that’s different than what the original writers used. (Good luck.)

Here’s why I think software reuse is difficult.

First, we need to note the difference between explicit knowledge and tacit knowledge. Explicit knowledge is what you get from books, or memos, or hearing a story or experience that someone else shares with you. Tacit knowledge is what you get from experimentation, mentorship, or exploration. (For example, I have no explicit knowledge of the vi editor – but I have extensive tacit knowledge of it, because when I open the editor my fingers know what to do, darting back and forth between ESC and : and other punctuation marks that my brain is not consciously processing. Perl is kind of the same way for me.)

If software is the executable representation of knowledge, then developing software is the process of codifying (literally!) tacit and explicit knowledge to make it executable. That’s a huge challenge that requires the developer to learn, reflect, experiment, learn, reflect, experiment… and so on. It’s a process of learning that ultimately results in software – a tool that hopefully will earn a life of its own, perhaps extending beyond the time that the original developer works on it.

What do you get by reading and exploring someone else’s code? Explicit knowledge. What do you get by writing code yourself, by uncovering each new function one at a time, by realizing you didn’t know what you were doing the first time, by refactoring, by trying out a new design (or maybe two or three), by bouncing ideas off of your colleagues, or by going back to the first design you had a long time ago? That’s tacit knowledge. You need both to write good software.

Unless there’s some way to get that tacit knowledge of someone else’s code, or you don’t need the tacit knowledge anyway (ie. in the case of the “black box”), successful reuse will be challenging… if not impossible.

(Related Post –> Software Reuse Antipatterns)

Software Reuse Antipatterns

coderIn 2000, Scott Ambler wrote  an excellent article on the organizational aspects of software reuse. He talked specifically about patterns and antipatterns:

“A pattern is a common approach to solving a problem that is proven to
work in practice. Conversely, an antipattern is a common approach to solving a problem that leaves us worse off than when we started.”

Long (2001) built on Ambler’s work and made it more fun. (This is, in my opinion, one of the most interesting and entertaining articles about software reuse in existence. You can get the full text from the ACM if you have an account there.) Long calls antipatterns obvious, but wrong, solutions to recurring problems and characterizes four organizational approaches that don’t support successful software reuse. See if your organization is one of these:

  • Abracadabra Model: A high-level manager is frustrated with a perceived lack of reuse and declares that “reuse will happen”. What Happens: Lots of talk, no action, silo development continues, managers start to panic, then the organization “de-evolves” into the next model.
  • High Noon Model: A high-level manager is REALLY frustrated with a perceived lack of reuse and declares that “reuse will happen”. What Happens: Finger pointing, as everybody has a lot of reasons (many of them very good, and very accurate) about why reuse can’t possibly work. The de-evolution continues.
  • Cost Cutter Model: A high-level manager is REALLY, REALLY frustrated with a perceived lack of reuse and declares that “reuse must happen to cut costs”. What Happens: Software people start to “force” reuse, immediate costs go up, upper management gets nervous, and more finger pointing happens (as everybody finds even more reasons now – including higher cost – for why reuse can’t possibly work.)
  • Used Car Fiasco Model: Software group says “OK, we’ll try reuse.” One group has software it thinks about group can use, so it is made available as-is and with no support. What Happens: There are lots of bugs to fix. Reusers have to fix them because the originators don’t have the time or resources to solve the new group’s problems. The reusers get frustrated and then write the code themselves.

Note that in all models, the expectations and behavior of the managers doesn’t change. In the fourth model, the behavior of the software developers changes. At no point do the expectations of the software developers change – their mission is to do what they need to do to get the software written.

Tomorrow I’ll write about why I think software reuse is difficult. The antipatterns above provide a good foundation for that discussion.

Ambler, S. (2000). Reuse Patterns and Antipatterns. Dr. Dobbs Journal, February 1, 2000.
Long, J. (2001). Software Reuse Antipatterns. ACM SIGSOFT, Software Engineering Notes, 26(3), 68-76.

Continuous Improvement Begins With Standards

apple-21Software is the executable representation of knowledge. [1] As a result, I find that software development provides a fruitful basis for exploring how problem solving is done by diverse team members in a cooperative (or even combative!) context.

Here is one example. In June 1997, Tom Gilb wrote an article for Crosstalk on “Requirements-Driven Management”. He noted that his purpose was intended to discuss, among other things, “some of the current major problems in systems engineering.” I stumbled upon this article again over the weekend, and it’s still as relevant now as it was a decade ago.

Standards are a prerequisite for systematic continuous improvement; the means to achieve an improvement process, not the ends, according to Glib. Furthermore, he remarks that Deming’s perspective on continuous improvement establishes, in part, that the reason we should adopt standards is to normalize our project to other projects . Once we do that, we’ve opened the doors to be able to use industry best practices, derived from other projects who also applied the standards – so that we can “clearly see the effect of any changes experimentally introduced into a process and not have to worry too much about other potential factors that impact the results.”

Learning, learning, learning. It’s all about continuous improvement through continuous learning, and in presenting this, Gilb is essentially promoting the same philosophy as Alistair Cockburn’s Cooperative Game Manifesto for Software Development. Both see the learning process as the key to successful software development. (So why do we not focus on this aspect of development more?) Glib addresses the issue of learning directly:

“The time has come to recognize that projects are so large, so complex, so unpredictable, and so state-of-the-art new that we have no practical alternative except to maximize our learning process during the project and as early as possible in the project life.

In this article, Gilb also presents “Evo” – short for “Evolutionary Project Management” – which has been used at IBM since 1970. It’s an implementation of Deming’s PDCA (Plan-Do-Check-Act) approach, and the author likens it to Humphrey’s Personal Software Process (PSP).

[1] This definition is credited to Eric Sessoms, who I consider a true artisan of software development. See some of his work at He likes the elegance of simplicity in his software.

Inspection, Abstraction and Shipping Containers

On my drive home tonight, a giant “Maersk Sealand” branded truck passed me on the highway. It got me thinking about how introducing a standard size and container shape revolutionized the shipping industry and enabled a growing global economy. At least that’s the perspective presented by Mark Levinson in The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger.

A synopsis of the story and a sample chapter are available; Wikipedia’s entry on containerization also presents a narrative describing the development and its impacts.

Here’s how describes it:

Indeed, it is hard to imagine how world trade could have grown so fast—quintupling in the last two decades—without the “intermodal shipping container,” to use the technical term. The invention of a standard-size steel box that can be easily moved from a truck to a ship to a railroad car, without ever passing through human hands, cut down on the work and vastly increased the speed of shipping. It represented an entirely new system, not just a new product. The dark side is that these steel containers are by definition black boxes, invisible to casual inspection, and the more of them authorities open for inspection, the more they undermine the smooth functioning of the system.

Although some people like to debate whether shipping containers were an incremental improvement or a breakthrough innovation, I’d like to note that a single process improvement step generated a multitude of benefits because the inspection step was eliminated. Inspection happened naturally the old way, without planning it explicitly; workers had to unpack all the boxes and crates from one truck and load them onto another truck, or a ship. It would be difficult to overlook a nuclear warhead or a few tons of pot.

To make the system work, the concept of what was being transported was abstracted away from the problem, making the shipping container a black box. If all parties are trustworthy and not using the system for a purpose other than what was intended, this is no problem. But once people start using the system for unintended purposes, everything changes.

This reflects what happens in software development as well: you code an application, abstracting away the complex aspects of the problem and attaching unit tests to those nuggets. You don’t have to inspect the code within the nuggets because either you’ve already fully tested them, or you don’t care – and either way, you don’t expect what’s in the nugget to change. Similarly, the shipping industry did not plan that the containers would be used to ship illegal cargo – that wasn’t one of the expectations of what could be within the black box. The lesson (to me)? Degree of abstraction within a system, and the level of inspection of a system, are related. When your expectations of what constitutes your components changes, you need to revisit whether you need inspection (and how much).

« Older Entries Recent Entries »