Tag Archives: software

Why Software Estimation is (Often) Useless

Posted on by Leave a comment

For 12 years, I blogged and wrote a whole bunch. For the past year and a half, I’ve let myself be pulled away from so many of the things that make me me… including writing. Today I heard one of the best anecdotes ever, and it’s the spark that will be pulling me back in. (Thanks, John.)

So here it is! It has to do with software estimation. Not only is it difficult to accurately estimate how long it’s going to take someone to do a programming (or similarly technical) task, the act of estimating often does not add any value at all. Estimation is a bad thing, especially if you’re trying to be Agile.

(And if you’re in a client/contractor relationship, you’ll have discovered that estimates are the lifeblood of the relationship, even as they drain all the life and all the blood out of the relationship, slowly and deliberately.)

Sometimes I get caught asking engineers for estimates even when the task we’re embarking on is new, unknown, uncertain, and requiring lots of learning and exploration and discovery. I should know better. But I cave, because the concept of the software estimate is so enticing: with a good estimate, I’ll know exactly how much time someone will need to spend working on a task that’s still kind of nebulous and mostly unknown. (That was a joke.)

My friend John shared the best anecdote ever today about why software estimation is so frustrating (liberally embellished):

Imagine that you’re standing on a hill looking down at a labyrinth, or a corn maze. It’s reasonably small… you can see that the corn maze is definitely doable, you can see a couple paths in and out, and the entire maze is a similar size to other mazes you’ve successfully found your way out of. So you’re pretty sure once you get to the entrance, you’ll find your way out.

But there’s no way you can say exactly how long it will take you to escape. Maybe you’ll run right through from start to finish, and it will be smooth. Maybe you’ll get stuck in the beginning, and spend a long time before winding your way out. Maybe you’ll run right close to the end, but have no idea you’re a few feet away from the exit, and you’ll get stuck there for a while.

And maybe you’ll make it halfway through, get lost, go in circles, and eventually just die in the maze.

Problem is, I can tell you how long it’s taken me to get across comparable mazes, but I have no way of knowing how long it will take me to escape from this maze, and just having another engineer in the maze to pair with me and see things I’m maybe not seeing is no guarantee at all that either of us will get us. Statistically, we’ll probably make it out, but the estimate I give you is just a guess.

Unfortunately, it’s a guess that’s going to make a lot of people unhappy, no matter what. Because even if I make it out of the maze fast this time, then they’ll expect that I’ll zoom through the next maze.

– John

Productivity Hack: Thinking in layers

Posted on by Leave a comment

I just finished reviewing a colleague’s latest project. It is absolutely beautiful. It’s a collection of very pretty looking documents and forms that you can use to keep track of your professional accomplishments and portfolio. The idea is that each person can use it to cultivate more agency in the professional development process — and it will definitely help people as individuals achieve that goal.

So why would it make my heart sink? Because it’s perfect for one person (or a handful of people) to more easily manage an individual (and isolated) process, but will make it difficult for us to gain visibility into the collection as a whole. It’s a personal management tool, not an organizational management tool. What it won’t help us do is:

  • Search across 10s or 100s of portfolios
  • Scale to 100s or 1000s of employees
  • Keep the content relevant and up to date

So when I see this gorgeous looking thing, I feel sad: so much work went into this, and although it’s going to help solve one problem, it is going to create a maintenance nightmare. In addition, now we’re locked into a single UI and any change will require a ton of effort (and manual changes in each employee’s worksheet). How could we have avoiding painting ourselves into this box?

Thinking in layers is a habit I’ve developed from decades of working in software. By thinking in layers, you can create more maintainable, systematic, and repeatable solutions for solving operations problems. If we had solved this problem in layers, here’s what the solution components would have looked like:

  • Data layer – store all the data in CSVs, with one observation per row and one variable per column
  • Processing layer – a way to create, read, update, and delete documents, records or fields & perform calculations
  • Presentation layer – a way to display the data and make what the user sees pretty

(One of the reasons I love R Markdown is that it gives you a way to easily combine the processing and the presentation in a way that still doesn’t break the layers. If you want to change what people see, you change that in one spot in your Rmd, then re-knit.)

The moral of the story is: you can’t build a scalable system without layers. Think in layers.

Top 10 Business Books You Should Read in 2020

Posted on by Leave a comment

I read well over a hundred books a year, and review many for Quality Management Journal and Software Quality Professional. Today, I’d like to bring you my TOP 10 PICKS out of all the books I read in 2019. First, let me affirm that I loved all of these books — it was really difficult to rank them. The criteria I used were:

  1. Is the topic related to quality or improvement? The book had to focus on making people, process, or technology better in some way. (So even though Greg Satell’s Cascades provided an amazing treatment of how to start movements, which is helpful for innovation, it wasn’t as closely related to the themes of quality and improvement I was targeting.)
  2. Did the book have an impact on me? In particular, did it transform my thinking in some way?
  3. Finally, how big is the audience that would be interested in this book? (Although some of my picks are amazing for niche audiences, they will be less amazing for people who are not part of that group; they were ranked lower.)
  4. Did I read it in 2019? (Unfortunately, several amazing books I read at the end of 2018 like Siva Vaidhyanathan’s Antisocial Media.)

#10 – Understanding Agile Values & Principles (Duncan)

Duncan, Scott. (2019). Understanding Agile Values & Principles. An Examination of the Agile Manifesto. InfoQ, 106 pp. Available from https://www.infoq.com/minibooks/agile-values-principles

The biggest obstacle in agile transformation is getting teams to internalize the core values, and apply them as a matter of habit. This is why you see so many organizations do “fake agile” — do things like introduce daily stand-ups, declare themselves agile, and wonder why the success isn’t pouring in. Scott goes back to the first principles of the Agile Manifesto from 2001 to help leaders and teams become genuinely agile.

#9 – Risk-Based Thinking (Muschara)

Muschara, T. (2018). Risk-Based Thinking: Managing the Uncertainty of Human Error in Operations. Routledge/Taylor & Francis: Oxon and New York. 287 pages.

Risk-based thinking is one of the key tenets of ISO 9001:2015, which became the authoritative version in September 2018. Although clause 8.5.3 from ISO 9001:2008 indirectly mentioned risk, it was not a driver for identifying and executing preventive actions. The new emphasis on risk depends upon the organizational context (clause 4.1) and the needs and expectations of “interested parties” or stakeholders (clause 4.2).

Unfortunately, the ISO 9001 revision does not provide guidance for how to incorporate risk-based thinking into operations, which is where Muschara’s new book fills the gap. It’s detailed and complex, but practical (and includes immediately actionable elements) throughout. For anyone struggling with the new focus of ISO 9001:2015, this book will help you bring theory into practice.

#8 – The Successful Software Manager (Fung)

Fung, H. (2019). The Successful Software Manager. Packt Publishing, Birmingham UK, 433 pp.

There lots of books on the market that provide technical guidance to software engineers and quality assurance specialists, but little information to help them figure out how (and whether) to make the transition from developer to manager. Herman Fung’s new release fills this gap in a complete, methodical, and inspiring way. This book will benefit any developer or technical specialist who wants to know what software management entails and how they can adapt to this role effectively. It’s the book I wish I had 20 years ago.

#7 – New Power (Heimans & Timms)

Heiman, J. & Timms, H. (2018). New Power: How Power Works in Our Hyperconnected World – and How to Make it Work For You. Doubleday, New York, 325 pp.

As we change technology, the technology changes us. This book is an engaging treatise on how to navigate the power dynamics of our social media-infused world. It provides insight on how to use, and think in terms of, “platform culture”.

#6 – A Practical Guide to the Safety Profession (Maldonado)

Maldonado, J. (2019). A Practical Guide to the Safety Profession: The Relentless Pursuit (CRC Focus). CRC Press: Taylor & Francis, Boca Raton FL, 154 pp.

One of the best ways to learn about a role or responsibility is to hear stories from people who have previously served in those roles. With that in mind, if you’re looking for a way to help make safety management “real” — or to help new safety managers in your organization quickly and easily focus on the most important elements of the job — this book should be your go-to reference. In contrast with other books that focus on the interrelated concepts in quality, safety, and environmental management, this book gets the reader engaged by presenting one key story per chapter. Each story takes an honest, revealing look at safety. This book is short, sweet, and high-impact for those who need a quick introduction to the life of an occupational health and safety manager.

# 5 – Data Quality (Mahanti)

Mahanti, R. (2018). Data Quality: Dimensions, Measurement, Strategy, Management and Governance. ASQ Quality Press, Milwaukee WI, 526 pp.

I can now confidently say — if you need a book on data quality, you only need ONE book on data quality. Mahanti, who is one of the Associate Editors of Software Quality Professional, has done a masterful job compiling, organizing, and explaining all aspects of data quality. She takes a cross-industry perspective, producing a handbook that is applicable for solving quality challenges associated with any kind of data.

Throughout the book, examples and stories are emphasized. Explanations supplement most concepts and topics in a way that it is easy to relate your own challenges to the lessons within the book. In short, this is the best data quality book on the market, and will provide immediately actionable guidance for software engineers, development managers, senior leaders, and executives who want to improve their capabilities through data quality.

#4 – The Innovator’s Book (McKeown)

McKeown, M. (2020). The Innovator’s Book: Rules for Rebels, Mavericks and Innovators (Concise Advice). LID Publishing, 128 pp.

Want to inspire your teams to keep innovation at the front of their brains? If so, you need a coffee table book, and preferably one where the insights come from actual research. That’s what you’ve got with Max’s new book. (And yes, it’s “not published yet” — I got an early copy. Still meets my criteria for 2019 recommendations.)

#3 – The Seventh Level (Slavin)

Slavin, A. (2019). The Seventh Level: Transform Your Business Through Meaningful Engagement with Customer and Employees. Lioncrest Publishing, New York, 250 pp.

For starters, Amanda is a powerhouse who’s had some amazing marketing and branding successes early in her career. It makes sense, then, that she’s been able to encapsulate the lessons learned into this book that will help you achieve better customer engagement. How? By thinking about engagement in terms of different levels, from Disengagement to Literate Thinking. By helping your customers take smaller steps along this seven step path, you can make engagement a reality.

#2 – Principle Based Organizational Structure (Meyer)

Meyer, D. (2019). Principle-Based Organizational Structure: A Handbook to Help You Engineer Entrepreneurial Thinking and Teamwork into Organizations of Any Size. NDMA, 420 pp.

This is my odds-on impact favorite of the year. It takes all the best practices I’ve learned over the past two decades about designing an organization for laser focus on strategy execution — and packages them up into a step-by-step method for assessing and improving organizational design. This book can help you fix broken organizations… and most organizations are broken in some way.

#1 Story 10x (Margolis)

Margolis, M. (2019). Story 10x: Turn the Impossible Into the Inevitable. Storied, 208 pp.

You have great ideas, but nobody else can see what you see. Right?? Michael’s book will help you cut through the fog — build a story that connects with the right people at the right time. It’s not like those other “build a narrative” books — it’s like a concentrated power pellet, immediately actionable and compelling. This is my utility favorite of the year… and it changed the way I think about how I present my own ideas.

Hope you found this list enjoyable! And although it’s not on my Top 10 for obvious reasons, check out my Introductory Statistics and Data Science with R as well — I released the 3rd edition in 2019.

An Easy Way to Make Minimum Viable Product (MVP) Totally Not Viable

Posted on by Leave a comment
The classic viral MVP cartoon from Henrik Kniberg (https://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp)

5 minute read

The Minimum Viable Product (MVP) concept has taken off over the past few years. Indeed, its heart is in the right place. MVP encourages product managers to scope features and functionality carefully so that customer needs are satisfied at every stage of development — not just in a sweeping finale at the end of development.

It’s a great way to shorten time-to-value and test new market concepts before committing. Zappos, for example, started by posting pictures of shoes on the internet without having an inventory. They wanted to quickly test to see whether people would even consider buying shoes without trying them on.

Unfortunately, adhering to MVP won’t guarantee success thanks to one critical caveat. And that is: if your product already exists, you have to consider your product’s base state. What can your customers do right now with your product? Failure to take this into consideration can be disastrous.

An Example: Your Web Site

Here’s what I mean: let’s say the product is your company’s web site. If you’re starting from scratch, a perfectly suitable MVP would be a splash page with one or two sentences about what you do. Maybe you’d add some contact information. Customers will be able to find you and communicate with you, and you’ll be providing greater value than without a web presence.

But if you already have a 5000-page site online, that solution is not going to fly. Customers and prospects returning to your site will wonder why it vaporized. If they’re relying on the content or functionality you previously provided, chances are they will not be happy. Confused, they may choose to go elsewhere.

The moral of the story is: in defining the scope of your MVP, take into consideration what your customers can already do, and don’t dare give them less in your next release.

Perception of Value & Today’s Cryptocurrency “Crash”

Posted on by Leave a comment

Artist’s rendering of Bitcoin. THERE ARE NO ACTUAL COINS THAT LOOK LIKE THIS. Don’t ever let anyone sell you one.

Today, many cryptocurrencies lost ~35-50% of their value. Reddit even posted contact information for the National Suicide Prevention Hotline in /r/cryptocurrency, knowing how emotional investors were bound to be today. Bitcoin, which was nearly $20K in mid-December and has been hovering near $14K this past week, dropped nearly $4K and almost sunk below the $10K milestone. I usually track the price of Bitcoin at http://bitcointicker.co, which can show the posted prices from several exchanges (web locations where people go to buy and sell, like Ebay). There are hundreds of cryptocurrencies and many of them dropped in value today.

Why did the prices drop so much on Tuesday? Here are some likely influences:

Market prices are usually driven by supply and demand — for example, if there aren’t that many lobsters available in a particular area at a particular time, and you go to a restaurant hoping to order one — you’ll pay a premium. But that price is also influenced by the quality of the product, the image of the product, which influences your perception of its value. Quality reflects how well something satisfies stated and implied needs or expectations.

Value, however, is quality relative to price, and influenced by image. And people are not always rational: they’ll pay a premium for image, even if the value of a product isn’t particularly high. Just think of all the Macs on display at schools, coffee shops, and airports. Price is related to value… usually, price goes up as value goes up.

Where’s the value of cryptocurrency? A Bitcoin does not, on its own, have any inherent value — just like a dollar or a Euro (a “fiat currency”). But the prospect of an asset that will increase in perceived value — where you can buy low, hold (sometimes just for a few days), and sell high because there are lots of people willing to buy it from you — will have perceived value. Hundreds of early adopters — or “Bitcoin millionaires” — are getting people excited about the prospect of making small investments and reaping huge rewards. That this has happened so recently lends a mystique to ownership of cryptocurrencies and Altcoins (or “alternatives to Bitcoin,” like Ether) in addition to the novelty.

Value is attributed to things by people, and cryptocurrencies are no exception. The quality of the currency itself, and the technical solidity of the platform upon which one is based, isn’t really tied to the cryptocurrency price right now — although this will probably change as knowledge and awareness increases.

Is this the end of Bitcoin? That’s doubtful — there are too many innovators who insist on exploring the technological landscape of cryptocurrencies and blockchain technology, and lots of investors willing to fund them. In the meantime, there are unlikely benefits: because cryptocurrencies are not yet mainstream, a “crypto crash” is not as likely to ripple through the whole economy (no pun intended) like the subprime mortgage crisis of 2008. But if you do decide to buy cryptocurrency, don’t invest any more than you can afford to lose.

How to Assess the Quality of a Chatbot

Posted on by Leave a comment

Image Credit: Doug Buckley of http://hyperactive.to

Quality is the “totality of characteristics of an entity that bear upon its ability to meet stated and implied needs.” (ISO 9001:2015, p.3.1.5) Quality assurance is the practice of assessing whether a particular product or service has the characteristics to meet needs, and through continuous improvement efforts, we use data to tell us whether or not we are adjusting those characteristics to more effectively meet the needs of our stakeholders.

But what if the entity is a chatbot?

In June 2017, we published a paper that explored that question. We mined the academic and industry literature to determine 1) what quality attributes have been used by others to determine chatbot quality, we 2) organized them according to the efficiency, effectiveness, and satisfaction (using guidance from the ISO 9241 definition of usability), and 3) we explored the utility of Saaty’s Analytic Hierarchy Process (AHP) to help organizations select between one or more versions of chatbots based on quality considerations. (It’s sort of like A/B testing for chatbots.)

“There are many ways for practitioners to apply the material in this article:

  • The quality attributes in Table 1 can be used as a checklist for a chatbot implementation team to make sure they have addressed key issues.
  • Two or more conversational systems can be compared by selecting the most significant quality attributes.
  • Systems can be compared at two points in time to see if quality has improved, which may be particularly useful for adaptive systems that learn as they as exposed to additional participants and topics.”

Analytic Hierarchy Process (AHP) with the ahp Package

Posted on by 11 comments

On my December to-do list, I had “write an R package to make analytic hierarchy process (AHP) easier” — but fortunately gluc beat me to it, and saved me tons of time that I spent using AHP to do an actual research problem. First of all, thank you for writing the new ahp package! Next, I’d like to show everyone just how easy this package makes performing AHP and displaying the results. We will use the Tom, Dick, and Harry example that is described on Wikipedia. – the goal is to choose a new employee, and you can pick either Tom, Dick, or Harry. Read the problem statement on Wikipedia before proceeding.

AHP is a method for multi-criteria decision making that breaks the problem down based on decision criteria, subcriteria, and alternatives that could satisfy a particular goal. The criteria are compared to one another, the alternatives are compared to one another based on how well they comparatively satisfy the subcriteria, and then the subcriteria are examined in terms of how well they satisfy the higher-level criteria. The Tom-Dick-Harry problem is a simple hierarchy: only one level of criteria separates the goal (“Choose the Most Suitable Leader”) from the alternatives (Tom, Dick, or Harry):

tom-dick-harry

To use the ahp package, the most challenging part involves setting up the YAML file with your hierarchy and your rankings. THE MOST IMPORTANT THING TO REMEMBER IS THAT THE FIRST COLUMN IN WHICH A WORD APPEARS IS IMPORTANT. This feels like FORTRAN. YAML experts may be appalled that I just didn’t know this, but I didn’t. So most of the first 20 hours I spent stumbling through the ahp package involved coming to this very critical conclusion. The YAML AHP input file requires you to specify 1) the alternatives (along with some variables that describe the alternatives; I didn’t use them in this example, but I’ll post a second example that does use them) and 2) the goal hierarchy, which includes 2A) comparisons of all the criteria against one another FIRST, and then 2B) comparisons of the criteria against the alternatives. I saved my YAML file as tomdickharry.txt and put it in my C:/AHP/artifacts directory:

#########################
# Alternatives Section
# THIS IS FOR The Tom, Dick, & Harry problem at
# https://en.wikipedia.org/wiki/Analytic_hierarchy_process_%E2%80%93_leader_example
#
Alternatives: &alternatives
# 1= not well; 10 = best possible
# Your assessment based on the paragraph descriptions may be different.
  Tom:
    age: 50
    experience: 7
    education: 4
    leadership: 10
  Dick:
    age: 60
    experience: 10
    education: 6
    leadership: 6
  Harry:
    age: 30
    experience: 5
    education: 8
    leadership: 6
#
# End of Alternatives Section
#####################################
# Goal Section
#
Goal:
# A Goal HAS preferences (within-level comparison) and HAS Children (items in level)
  name: Choose the Most Suitable Leader
  preferences:
    # preferences are defined pairwise
    # 1 means: A is equal to B
    # 9 means: A is highly preferable to B
    # 1/9 means: B is highly preferable to A
    - [Experience, Education, 4]
    - [Experience, Charisma, 3]
    - [Experience, Age, 7]
    - [Education, Charisma, 1/3]
    - [Education, Age, 3]
    - [Age, Charisma, 1/5]
  children: 
    Experience:
      preferences:
        - [Tom, Dick, 1/4]
        - [Tom, Harry, 4]
        - [Dick, Harry, 9]
      children: *alternatives
    Education:
      preferences:
        - [Tom, Dick, 3]
        - [Tom, Harry, 1/5]
        - [Dick, Harry, 1/7]
      children: *alternatives
    Charisma:
      preferences:
        - [Tom, Dick, 5]
        - [Tom, Harry, 9]
        - [Dick, Harry, 4]
      children: *alternatives
    Age:
      preferences:
        - [Tom, Dick, 1/3]
        - [Tom, Harry, 5]
        - [Dick, Harry, 9]
      children: *alternatives
#
# End of Goal Section
#####################################

Next, I installed gluc’s ahp package and a helper package, data.tree, then loaded them into R:

devtools::install_github("gluc/ahp", build_vignettes = TRUE)
install.packages("data.tree")

library(ahp)
library(data.tree)

Running the calculations was ridiculously easy:

setwd("C:/AHP/artifacts")
myAhp <- LoadFile("tomdickharry.txt")
Calculate(myAhp)

And then generating the output was also ridiculously easy:

> GetDataFrame(myAhp)
                                  Weight  Dick   Tom Harry Consistency
1 Choose the Most Suitable Leader 100.0% 49.3% 35.8% 14.9%        4.4%
2  ¦--Experience                   54.8% 39.3% 11.9%  3.6%        3.2%
3  ¦--Education                    12.7%  1.0%  2.4%  9.2%        5.6%
4  ¦--Charisma                     27.0%  5.2% 20.1%  1.7%        6.1%
5  °--Age                           5.6%  3.8%  1.5%  0.4%        2.5%
> 
> print(myAhp, "weight", filterFun = isNotLeaf)
                        levelName     weight
1 Choose the Most Suitable Leader 1.00000000
2  ¦--Experience                  0.54756924
3  ¦--Education                   0.12655528
4  ¦--Charisma                    0.26994992
5  °--Age                         0.05592555
> print(myAhp, "weight")
                         levelName     weight
1  Choose the Most Suitable Leader 1.00000000
2   ¦--Experience                  0.54756924
3   ¦   ¦--Tom                     0.21716561
4   ¦   ¦--Dick                    0.71706504
5   ¦   °--Harry                   0.06576935
6   ¦--Education                   0.12655528
7   ¦   ¦--Tom                     0.18839410
8   ¦   ¦--Dick                    0.08096123
9   ¦   °--Harry                   0.73064467
10  ¦--Charisma                    0.26994992
11  ¦   ¦--Tom                     0.74286662
12  ¦   ¦--Dick                    0.19388163
13  ¦   °--Harry                   0.06325174
14  °--Age                         0.05592555
15      ¦--Tom                     0.26543334
16      ¦--Dick                    0.67162545
17      °--Harry                   0.06294121

You can also generate very beautiful output with the command below (but you’ll have to run the example yourself if you want to see how fantastically it turns out — maybe that will provide some motivation!)

ShowTable(myAhp)

I’ll post soon with an example of how to use AHP preference functions in the Tom, Dick, & Harry problem.

« Older Entries