• “The Bear” Season 2: A Poignant Story About Building Quality Culture

    Confession: When I originally viewed The Bear (Season 1) last year, I thought it was “just OK.” It’s a drama that centers around a group of 8 people who work

    Read more »

Accountability: Getting Real About What We CAN Do

Posted on by Leave a comment

When you hear the word accountability, what do you think of?

In cultures with low psychological safety, accountability can be a scary word. Who’s going to whip the horses harder to get them to go faster? (This is particularly frightening if you’re one of the horses.) As long as there are people assigned to the positions of whipper and whipped, you’ve got accountability. Right? No way.

True accountability – that is, accountability with psychological safety – is not only different, but tremendously beneficial to everyone in an organization! Most of us, at our core, want to do a good job. Without clear lines of accountability (and expectations set early and often around it) performance will suffer, systems will fail, and trust will erode.

When no one is clearly and specifically accountable for anything, everyone becomes accountable for everything. This drives a stake into the heart of prioritization, and gets personally exhausting very quickly. It’s a recipe for distress, disillusionment, and burnout. Why?

  • Accountability is a belief. Anyone can believe that you are accountable for a particular task, initiative, or outcome… but if you don’t share their belief, or if you aren’t aware that someone else believes you are accountable for something, one (or both) of you will end up very disappointed.
  • If you’re accountable for something you have no control over, you’re bound to disappoint someone. (Think about how you’d feel right now if I told you that you were accountable for fixing global warming this year, and you’d be measured on how successful you were at reducing the average surface temperature of the planet by a degree.)
  • If you’re accountable for an outcome but not held to accountability, the rest of your colleagues will notice… and you will lose credibility and respect. If you’re a leader who doesn’t hold others accountable, you’ll also lose credibility and respect.
  • If you’re not accountable for something you’re the most qualified to be accountable for, your organization will not benefit in the ways it could… and you’re more likely to feel marginalized, dissatisfied, or left out of opportunities to add value.

Accountability helps us get real about what we CAN do for one another. For everyone in your organization to get the most value out of accountability, remember these elements of the CAN acronym:

  • Conversations are not commitments. You stopped somebody in the hall, let them know you’re waiting on a new feature, and that it’s really, really important because you need it next week! Next week comes and goes, your feature is still missing, and now you’re mad (and losing business). Did the accountable party let you down? No, they didn’t… because accountability requires commitments to be formalized and prioritized in some way. While not all accountability requires legal or contractual arrangements between parties, there needs to be a way to clearly identify the basis for accountability.
  • Accountability aligned with leverage catalyzes results. In contrast, accountability without ownership leads to disappointment. Have you ever been accountable for something you have no way at all (even indirectly!) have a way to control or influence? It feels the same way as trying to cook dinner but discovering that you’re missing a key ingredient! If all the ingredients are available to you, and you have control over how they’re used and the tools to use them, you’re set up for success. 
  • No expectations without consent. Accountability is not unilateral: it means that expectations are set, negotiated, and agreed upon by the involved parties. (Accepting a written job offer is a great example of giving consent to expectations… your employer has defined your role and responsibilities, and your acceptance indicates that you consent to those expectations.)

Accountability means I know the vital few outcomes I’m expected to ensure, and I’ve consented to realizing them. Accountability means I know what resources to ask for, what tradeoffs to make, and what boundaries have to be set and pushed. Accountability is a shared belief – an understanding of what is needed from multiple parties so that a goal can be achieved, and the consent to move forward in that agreement. 

Bottom line, accountability is the order that slices through organizational chaos.

“Us vs. Them”: There’s A Better Choice

Posted on by 2 comments

Over the past 25 years (at least), I’ve routinely noticed two (and only two!) patterns of interaction emerge between people, and one of those patterns is wildly superior to the other. This post is my attempt to make you aware of them and recognize there’s a better choice than letting “us vs. them” dynamics take root.

Scenario 1 – The Transactional Style: Party A has a problem to solve. Party B is assigned to solve that problem. Party B aims to detect, elucidate, and satisfy the requirements and needs underlying that problem, and hopefully deliver something that Party A approves of (and maybe even likes). Party B is accountable to Party A to get the job done right. At some point, Party A evaluates the conduct and the output of Party B and determines whether they provided value.

Scenario 2 – The Collaborative Style: Party A has a problem to solve, and asks their trusted colleague B1 to help them convene Party B to solve the problem. Party A and Colleague B1 collaboratively design the team that becomes Party B. Together, Party B team members seek to understand the problem, and work on outlining several different paths that Party A could take to solve the problem. Party B helps Party A understand what options for moving forward are available, and Party A feels like they are actively contribute to the solution. The parties come to an agreement on the best way to move forward, and Party B engages Party A at every opportunity, resulting in a solution that everyone feels they have contributed to.

What’s the difference? Scenario 1 puts people on opposite sides of the problem to solve, and Scenario 2 assumes they’re partners in a quest. In Scenario 1, Party A and Party B might be teacher and student, manager and employee, client and contractor, or customer and provider. There is a convoluted and invisible fence set up between them, because one has a need and the other is required to address and fulfill that need according to one person’s judgment (often subjective). In Scenario 2, Party A and Party B are partners, accountable to each other to solve a problem that lies between them.

While the transactional style is perfectly appropriate for short-term or low-stakes interactions (like going to a restaurant; it would be kind of weird to collaborate with the kitchen staff) it’s damaging in situations where building relationships is ultimately critical to success.

It takes humility to be part of a collaborative team. Usually, the problem the team is working on is bigger than any single person on the team; and typically, the answer lies between the participants and not within one of the participant’s heads. Anyone who walks into the team thinking they’re going to carry the solution is wrong. No matter how many years (or decades) of experience you have, you don’t get to rest on your laurels. You get to solve this problem right now, with these people.

It takes discernment to be part of a collaborative team. Know what you know, know what you don’t know, seek out people who do know when you need them, and figure out how you’ll help each other discover and achieve quality standards. It takes the courage to admit when you don’t know something (pro tip: the more you do this, the easier it gets). It takes courage to say… “Are we sure of that? How do we know?” and hold not only your team, but yourself, to higher standards.

It takes openness to be part of a collaborative team. If you’re hoarding knowledge, controlling the narrative, or resting on laurels you’re working against openness, but likely creating little pockets of power for yourself that feel rewarding (at least temporarily… until people figure you out). Openly sharing information, evidence, thoughts and ideas can be risky – people might find out you’re not as smart as they thought, or you might feel put on the spot. You will be forced to confront the limits of your own understanding. At the same time, they might find out how much progress you can make, together… even if you’re just asking the questions that get everyone to the next step.

The bottom line: there are two modes of interaction. One is transactional and confrontational, focused on each person attempting to make their own worth and value visible. The other is collaborative, emphasizes discovery and problem solving, and focuses on a problem (and solution) that sits in the middle of multiple stakeholder groups.

The transactional mode forces people to categorize themselves and colleagues into “us and them”. In the collaborative mode, there’s only us… facing a shared problem, and committed to discovering a shared solution. Our new “them” is the collection of questions we pose, the investigations we launch, the paths we have to we have to go down, the trails we have to blaze, and the challenges we’re yet to overcome… together.

A Solid Data Architecture Pays Off

Posted on by Leave a comment

Earlier this year, the people who (rightly) decided the “collective we” needed better tools to manage all our spaghetti SQL (dbt) acquired the “semantic layer” company, Transform. In one of the merger announcements, I read this:

Imagine instead that you could disentangle metric definition from visualization. In this world, the teams that own metrics would be able to define them once, in a way that’s consistent across dashboards, automation tools, sales reporting, and so on. Let’s call this “Headless BI”.

https://www.getdbt.com/blog/dbt-acquisition-transform

Whoaaaa…. I thought. Imagine!! I can’t imagine, because to me, disentangling happens in the architecture and design stages, and then governance keeps things disentangled over time. But adopting solid habits and practices takes time and dedication that many don’t have (or get bored with).

Just how many “data stack companies” exist because people who work in data have (probably accidentally) ignored separation of concerns, and over a course of years, backed into a problematic architecture? How many of these companies have hit the “non-signal $10M” by convincing prospects that they can buy way out of technical debt by signing just one check?

Probably a lot. But they’ll soon find out that “there is no instant pudding.” — W. Edwards Deming

Enterprise Data and the Holy Grail

Posted on by Leave a comment

TL;DR – A core principle of systems engineering and software engineering isn’t readily applied (or even recognized!) among people who build, operate, and maintain enterprise data ecosystems… leading us to inadvertently make fragile systems more fragile, and confusing data flows even more confusing.

The Holy Grail is a legendary artifact in Christian mythology. Typically imagined to be the cup used by Jesus at the Last Supper, it is said to possess miraculous powers, and represents an ultimate object of achievement or desire. Over the centuries, it’s been the subject of numerous literary works, from medieval romances to modern novels. King Arthur launched a quest to find it… so did Indiana Jones. The Holy Grail has enduring appeal as a symbol of the ultimate treasure, the solution to the most vexing of problems.

And from what I’ve observed, there’s a Holy Grail in enterprise data management. It’s so elusive that it’s sitting in reach of nearly everyone, and yet, it shimmers in and out of existence and is only rarely spotted even in the most mature of organizations. It’s separation of concerns (SoC). (I talked about this in my presentation at the MIT Chief Data Officer conference last week, illustrating how quality and triage information can “pop” from the business perspective when your underlying system adheres to SoC.)

Separation of concerns is an essential principle in software and systems engineering that promotes modular design. The idea is to break a program into distinct sections that each address a specific concern or function. The modules should be loosely coupled (to reduce the risk that a change made in one component, like a schema or a UI, creates unexpected changes in other components) and highly cohesive (related functions are grouped together).

Every time (except one) that I’ve asked a data team how they’re implementing SoC, I get blank stares. Then I ask “where do you keep your raw data?” and “where do you keep your clean data?” and “how do you know the difference?” and you can see the gears start to turn when people realize that they pretty much don’t distinguish what’s essential from what’s trash.

While it’s table stakes in software and network architecture, the Frankenstein systems we’ve evolved to for managing data and producing analytics tend to mash together all the steps in a data flow (in varied and very creative ways). Products that manage data delivery to the end user, like Tableau and PowerBI, often embed scripts that acquire, transform, clean, test, and produce new information… instead of just packaging up data and information so that users can slice, dice, and visualize it. Our data ecosystems are often tightly coupled and not cohesive at all.

Without separation of concerns, we lose two critical things: 1) the ability to reuse raw or clean data to generate new information that becomes data products, which leads to tremendous duplication of effort across a company (especially over time), and 2) the ability to collect diagnostics that quickly illuminate the root cause of each failure by pinpointing the stage at which they occur. By applying separation of concerns, we can look independently at each stage:

  • Arrival/acquisition tasks generate raw data
  • Integration tasks structure, clean, and transform raw data into clean data
  • Analysis and processing tasks use clean data to produce entirely new information (data products)
  • Clean data and data products are delivered to end users in a multitude of ways, using a variety of styles (reports, dashboards, UIs)
  • Stakeholders engage with those delivery mechanisms to generate insights and business value

Are your pipelines failing? The way you respond will be totally different (and require a lot less effort) if you know that those pipelines share a common task or element. Why is one business user happy with the data and another one totally unhappy with the same data? Being able to look at diagnostics through a lens of what’s important to each person can illuminate the reason.

Postscript: Someone asked me “who was the ONE? what was unique about THEM?” – and the answer is, they were a huge fan of Delta Lake (that comes with a built-in SoC conceptual model) and the Cube semantic layer (which does too). They were specifically looking for ways to keep the layers in their data ecosystem distinct.

ETL is So Much Easier Now

Posted on by Leave a comment

When I think about the rich landscape of ETL (Extract, Transform & Load) tools that organizations can choose from in 2023 to reduce the time to access and manipulate data, I feel like a fossil. Back in the 90s, we didn’t have it that easy. We had to walk to and from school every day, uphill both ways, in five feet of snow… even in the summer. Our ETLs were bash scripts and DSNs, binary files and makefiles, and lots of lots of subdirectories. We spent a ton of time staring at schemas and ERDs and trying to get tables third-order-normalized because there wasn’t enough storage space to be anything other than minimalist. Want to connect to a new data source? Better set aside at least four weeks so you can figure out how to pull data from it.

If time traveler Ethan Aaron, CEO of Portable, had magically appeared in my office and said “In 25 years, there will be libraries of connectors to any data source you can imagine, and if a source you need isn’t in the library we’ll code it for you for free“ I would have immediately started worrying about job security and what this portended for a future that was clearly more shocking than I could imagine. (Don’t even mention the whole magical cloud thing.) If he showed me the Top 100 ETL tools list, I’d think I was hallucinating. I also would have started thinking about Kermit.

(Not the frog… but a really, really yucky kermit-related ETL problem I had in 1997.)

A coworker and I had to analyze some data, but one of the inputs we needed was on a hard drive we had retrieved from some instrumentation out “in the field.” (It was literally in a locked box in a field, next to the instrument.) We found some cables that fit into some ports, and were able to peek into the elaborate directory structure that the instrument had created to store the data. The files were right there… we could see them. But we couldn’t tell what format they were in, or how to open them.

For weeks, we tried everything. Nothing worked. Those files with the super valuable data were just impenetrable.

But then, we decided to ask around the office. Someone had to have collected this same kind of data at some point in time, or maybe had used a similar kind of data collection instrument. We poked our head into people’s offices, but no one had any clues. A week later, a woman came to see us. “I can’t help you with your data,” she said, “but you might want to try Dave’s office, down the hall.” Dave had retired a few years ago, but there were still a couple of Dave’s cabinets in his old room that had been serving as a storage room.

At the bottom of one of Retired Dave’s old cabinets, we found a dusty box with some diagrams of a data collection instrument a lot like the one we were trying to use and we got excited. There was also a book called “Kermit” and a few pages of Unix commands that looked like they might be related to the book. I’ll spare you the gory details (mainly because I can’t remember all of them, this many years in the future…) but long story short, we were able to use the kermit file transfer and management protocol to crack open our data… and do the analysis we’d needed.

Time-to-value in 1997 to get our data: About four elapsed months.

Time-to-value in 2023 with ETL tools like Portable: Days, I bet. I can’t express what a joy it is to not have to build frameworks from scratch, any more. And I’m thankful that companies like those on the Top ETL Tools list are making this joy possible… even if early career programmers can’t feel the gratitude quite as intensely 🙂

“The Bear” Season 2: A Poignant Story About Building Quality Culture

Posted on by Leave a comment

Confession: When I originally viewed The Bear (Season 1) last year, I thought it was “just OK.” It’s a drama that centers around a group of 8 people who work in a restaurant, and I’ve never worked in a restaurant. I had little to relate to. Plus, the excellent cinematography helps viewers feel the panic, the chaos, the always-on tension of playing an active role in an industrial kitchen… to be honest, the show kind of stressed me out.

But there’s one subject I think about All. The. Time. and that’s quality and performance excellence. So I’m particularly relieved that I dove in to watch Season 2, because all I can say is: this season is perfection.

(If you haven’t watched Season 1 or 2 yet, there will be spoilers in the remainder of this post. Go watch it on Hulu, come back and finish this post, and then we can talk about what you learned in the comments.)

Season 1 starts as Carmy, the main character, returns to Chicago after the unexpected suicide of his brother Mikey, who ran a gritty neighborhood restaurant called “The Original Beef of Chicagoland” – a long-time community favorite. Mikey has left the restaurant to his brother, an award-winning chef from a Michelin-starred restaurant in New York, entrusting him to take up the helm at the head of the family business. While the family is proud of his accomplishments and happy he’s returned, there are clearly tense family dynamics and a lingering sense of betrayal that he had left Chicago while they remained stuck there.

As soon as Carmy arrives at The Beef, there’s a palpable power differential: While Carmy is in charge, he’s new. The entire rest of the staff has been there for months already, maybe years, and have habits and routines of their own. They’re accustomed to doing their own thing, figuring things out as they go along, and bristle at Carmy’s attempt to manage. On his own, he’s brought in an unpaid intern, a talented young chef named Sydney who’s trained at some of the most reputable places in town and, admiring Carmy’s reputation, wants to work with him… even if it’s at the sandwich shop her father’s been going to for years, and not a fine dining establishment.

Carmy and Sydney are accustomed to an entirely different mode of operation than the old timers at The Beef; his quality and performance standards are much, much higher. The performance differential sets up a cultural rift: the previous staff, in particular “Cousin Richie” (who’s not actually a cousin), get the sense that Carmy and Sydney think they’re better than everyone else (even though they just want the staff to learn and grow). The tension is particularly high with Tina, a long-time restaurant worker who’s not particularly enthusiastic about change, but had been committed to brother Mikey’s establishment and wanted it to succeed.

Soon, they notice Carmy doing strange things: being particular about his, and their, methods for prepping and cooking; cleaning with a toothbrush; rigorous routines, for things they didn’t see the value in. “It’s about consistency,” he says, “You can’t operate at a higher level without it.” His experiences training under world class chefs has opened his eyes to what is possible, and he aspires to the excellence that he observed.

They [Michelin-starred restaurants] teach you how to operate on a level you didn’t know you could operate at.

Carmy, in Season 1 of “The Bear”

Carmy sees the performance gap between his team and himself, but also sees the performance gap between his own level of performance and where he aspires to be. He meets them where they are, because he knows they’re not emotionally ready for breakthrough improvements, and The Beef isn’t exactly a candidate for Michelin stars. He models the desired behaviors, and never relents: each day, every day, in his interactions with every person, he calls out performance gaps and sets the expectation that he won’t compromise on quality. Meanwhile, they get to observe Sydney’s higher standards, and how Carmy responds – and builds on – her work.

Gradually, and with much pain and friction, the team works out practices that are acceptable enough for Carmy, at least for the time being. He finds he’s got bigger problems: the financial state of the business wasn’t as stable as he was originally led to believe, and the restaurant is struggling. They try experimenting: new recipes, new menu items, and “take out” – but each potential new path fails. He finds out that he’s deep in debt to the Mob, and even if he could pay it off, the restaurant is unlikely to survive. Tension rises – and in a moment of exhaustion, Carmy berates Sydney for minor issues, reminiscent of the abusive treatment he received from his primary mentor. Sydney’s not going to put up with that… she quits, leaving him with even more tasks to do to keep the restaurant afloat.

While Season 1 culminates with Carmy finding hundreds of thousands of dollars stashed in tomato sauce cans that he can use to repay his debt to the Mob, there’s still not enough liquidity for him to keep The Beef in operation. But the team has an idea: they’re ready for the next step, and they want to do it together. Sydney returns at just the right moment to rejoin the crew, who hangs a sign on the door announcing that “The Bear” is coming soon.

Postscript: Since I started watching this show a year ago, I’ve been confused about why they call this show “The Bear” when the restaurant is called “The Beef” – and who wants to go to a restaurant thinking about eating bears? But last night, I figured out that “Bear” must be a nickname based on Carmy’s surname, “Berzatto” (bear-zotto).

Season 2 is all about transformation: building habits and practices that reflect a new understanding of quality standards, and a dedication to excellence and continuous improvement. Marcus, the pastry chef, heads to Copenhagen to study under a master. Tina heads to chef school. And cantankerous Richie (who, it appears, feels like he’s so battle-worn and experienced he doesn’t need any chef school) is assigned to one of Chicago’s best restaurants (called “Ever” – it actually exists) for shadowing. He resists, complains, lashes out… considers quitting. After a few days of rudely resisting his mentors’ attempts to level him up, he decides to accept his “punishment” – but in a moment of “wax on, wax off” has an epiphany and sees how good it feels to embrace quality in a new way. He finds his best self, and becomes a warm, attentive colleague, tuning into a talent of connecting with people that we hadn’t yet seen (but Carmy was clearly aware of, and waiting for its resurrection).

Meanwhile, the building itself transforms in ways that mirror the psychological transformations: walls are torn down, and walls fall down on their own (forcing a vulnerability that the characters aren’t quite ready for). Mold pours out of the ceiling, and all over Richie. The fire suppression system has been tampered with and is no longer functional, and the team has to reverse the damage to get the license to operate. Floors are shined; cozy new lighting is installed. Tables are tightened. A gift arrives in the mail from Denmark, and Marcus hangs it – an “Every Second Counts” sign – under the clock in the kitchen to remind everyone of their shared strategy for banding together in times of crisis to remember their goals.

The team manages to band together for a “Friends and Family” opening night that ends up being… mostly successful. Carmy gets locked in the walk-in refrigerator when the handle breaks off, clearly some karma for his resistance to handing off the “call fridge guy” task to literally anyone else for three months. In a fit of shame and self-flagellation he vocally expresses his regret about prioritizing a new girlfriend over his restaurant, but she’s been listening to his diatribe the whole time…

So we’re well set up for another season, where the characters realize that even though they’ve been through training, they don’t have things figured out just yet, and building a quality culture takes lots of time and a ton of shared dedication. This is probably the best series with a quality theme that I’ve ever seen, and certainly the best of any restaurant show I can think of (Alice, SpongeBob, and the sequel to Three’s Company).

I’m looking forward to Season 3, where I can follow the stories of each person connected to The Bear as they get to know the new quality standards more intimately, and recognize that excellence is a journey, not a destination.

A+ for ML with R: Lantz’s 4th Edition

Posted on by Leave a comment

TL;DR – This book is a great value. I’d pay $100 for it, even though I already have a well worn 3rd edition (and even a 1st edition that I started using with undergrads in 2015).

Want to learn how to write R code to develop and deploy machine learning models in R? If so, use caution: ANYONE can “turn the crank” and write code to build machine learning models, especially now that tools like Github Copilot are so easy to access and use. Companies like mine will increasingly be looking for data analysts and data scientists who can articulate the context of a problem and understand when methods and models are appropriate to apply. While hundreds of books focus on the mechanics of turning the crank, or the details of the packages used to build the models, Brett Lantz takes a first principles approach.

Several years ago, I adopted an early edition of Lantz’s book for a course I regularly taught for undergraduate juniors and seniors in STEM rather than deciding to write my own book. I never regretted that decision. Students routinely evaluated my choice of textbook highly, and said they appreciated the clear examples. This book is ideal to support an introductory two-course sequence for upper level undergrads or graduate students who need a solid foundation in machine learning with R. It is also ideal for entry level and mid-level data analysts and data scientists who want to build solid competencies. The book does not cover topics like tidymodels, deployment issues (especially to the cloud), model maintenance, or integrating R and Python through R Markdown and Quarto, BUT I think if it had, it would have taken away from the simple style and approach that makes this book so unique and powerful – I’m glad the table of contents covers what it does, and nothing more.

The 4th edition maintains the same standard for excellence as in previous editions. His prose is clear, his examples have no missing steps, and his discussions are straightforward and comprehensive. For each method, he lays out the background and rationale for using it, illustrates the inputs, process, and outputs with simple yet compelling examples, and explains the results in the context of how you might evaluate them. He also includes modernized deeper dives on topics like lift, model tuning, feature engineering, stacking, and outliers (what exactly ARE outliers in a group of images?) Chapters 11-15 are largely new, and make up the bulk of the additional 300 pages, making this a 700+ page reference that will surely find its permanent home in a prominent position on your desk.

« Older Entries