Quality and Innovation

Every year on this day, statistics and data people are REQIURED to advance data literacy by making sure as many people see this as possible (thanks to Scott for being a first mover on Halloween 2024):

But you know what’s REALLY scary? Not understanding distributions!

While individual metrics on your Power BI or Tableau dashboards show performance at a specific point in time or space, distributions give you a quick way to see LOTS of values over time or space.

Not looking at distributions means you MISS important signals… because sometimes means just aren’t as meaningful!!

For example:

Customer Lifetime Value

• Point metric: Our customers have an average CLV of $1000
• Distribution reveals: Two bumps on the distribution, like a camel! That means we’ve got two distinct groups of customers… maybe casual customers have a CLV of $200, but our really REALLY loyal customers have $1800 CLV
• Business impact: If we know the distribution, we can formulate targeted retention strategies for each segment. Or maybe just focus our effort on pushing the center of the bigger bump to the right.

Delivery Times

• Point metric: 30-minute average delivery
• Distribution reveals: A long tail. Our most egregious deliveries show up 12 hours after they’re promised… customer satisfaction suffers. Those customers get LOUD on social media.
• Business impact: The distribution can help identify systematic issues causing delays

Product Ratings

• Point metric: We’ve got a 4.2/5 average rating… YAY! Right?
• Distribution reveals: No. you could have polarized reviews, suggesting product controversy. Studying one group will reveal what delights customers, and studying the other group will reveal dissatisfiers.
• Business impact: Makes it possible to address specific customer concerns instead of ignoring them because you’re happy you got all those stars

Employee Performance

• Point metric: Average sales per employee is $20K a month… excellent!
• Distribution reveals: Top performers bring in $200K a month. Struggling staff may not sell anything.
• Business impact: Better targeting of training and support resources. You can help your top performers soar, while helping those who are struggling grow (or find a better role to play).

Web Site Load Times

• Point metric: Average 2-second load time. That’s great… right?!
• Distribution reveals: Spikes during peak hours or specific pages. Between 9am and 11am ET on Mondays, load time is 3 minutes and there are lots of timeouts. You’re losing customers because your competitor’s web site worked when they needed it to.
• Business impact: Identifies specific technical problems and makes it possible to repair them in a cost effective way. If you’d told your software engineers to “go fix this bug” – they’d spend ages on it and still not solve the problem. There’s probably no bug… but rather, an infrastructure issue.

Customer Support Response Times

• Point metric: Average 30 minute response time across the whole team. Fantastic!
• Distribution reveals: Time-of-day variations and outliers. Your staff is providing quick responses during business hours, but customers who have to wait overnight for answers are upset.
• Business impact: Helps optimize staff scheduling & customer sat.

Frequency of Online Product Usage

• Point metric: Average 10 uses per week
• Distribution reveals: Super-users vs occasional users. Some people are staying online all the time.
• Business impact: Guides feature development priorities & helps you discover a bug that’s keeping people logged in, artificially decreasing the frequency metric.

Purchase Order Values

• Point metric: Average order is $500!
• Distribution reveals: Bulk buyers vs individual purchasers. Individuals are only only buying $20, on average.
• Business impact: Influences inventory policies, ricing strategies, and helps you decide which part of the business to focus your effort.

Customer Churn

• Point metric: 15% monthly churn
• Distribution reveals: Early vs late-stage customer departure patterns. Some people are getting irritated fast, but we don’t know why (yet).
• Business impact: Enables proactive retention efforts.

Marketing Campaign Response

• Point metric: 2% conversion rate. That seems low!
• Distribution reveals: Demographic-specific response patterns, or maybe a campaign or three where you didn’t set your conversion tracking up right and you’re showing 0% conversions when in fact you ARE getting customers through those channels.
• Business impact: Allows for more targeted campaign optimization. Helps you catch technical problems that have a material impact on your metrics.

Remember, MEANS are not always MEANINGFUL. You’ve got to look under the surface for patterns revealed by distributions – or guaranteed, you will have ghosts in your data. (And not fun ones.)

[If you need to amplify data literacy at your org so your staff has ghost-vision, contact me by DM at https://www.linkedin.com/in/nicoleradziwill/. We can get you started on a path to progress.)