This example is drawn from “A Painless Q-Learning Tutorial” at http://mnemstudio.org/path-finding-q-learning-tutorial.htm which explains how to manually calculate iterations using the updating equation for Q-Learning, based on the Bellman Equation (image from
Overview: Reinforcement learning uses “reward” signals to determine how to navigate through a system in the most valuable way. (I’m particularly interested in the variant of reinforcement learning called “Q-Learning”
Image Credit: Shutterstock, from http://asq.org/blog/2015/02/why-should-quality-go-global/
Do you have, or have you had, a supplier selection problem to solve? I have some algorithms I’ve been working on to help you make better decisions about what suppliers to choose — and how to monitor performance over time. I’d like to test and refine them on real data. If anyone has data that you’ve used to select suppliers in the past 10 years, or have data that you’re working with right now to select suppliers, or have a colleague who may be able to share this data — that’s what I’m interested in sourcing.
Because this data can sometimes be proprietary and confidential, feel free to blind the names or identifying information for the suppliers — or I can do this myself (no suppliers, products, or parts will be named when I publish the results). I just need to be able to tell them apart. Tags like Supplier A or Part1SupplierA are fine. I’d prefer if you blinded the data, but I can also write scripts to do this and have you check them before I move forward.
Desired data format is CSV or Excel. Text files are also OK, as long as they clearly identify the criteria that you used for supplier selection. Email me at myfirstname dot mylastname at gmail if you can help out — and maybe I can help you out too! Thanks.
When I was younger, I felt like I was pretty smart. Then I turned 23, was thrown into the fast-faced world of helping CxOs try to straighten out their wayward enterprise software implementations, and realized just how little I knew. My turning point came around 6pm on a hot, sticky, smelly evening on Staten Island in a conference room where a director named Mike Davis was yelling at a bunch of us youngster consultants. I thought he was mad at us, but in retrospect, it’s pretty clear that he just wanted something simple, and no matter how clearly he explained it, no one could hear him. Not even me, not even when I was being smart.
The customer was asking for some kind of functionality that didn’t make sense to me. It seemed excessive and unwieldy. I knew a better way to do it. So when Mike asked us to tell him, step by step, what user scenario we would be implementing… I told him THE RIGHT WAY. After about five attempts, he blew up. He didn’t want “the right way” — he wanted “the way that would work.” The way that would draw the most potential out of those people working on those processes. The way that would make people feel the most engaged, the most in control of their own destiny, the way that they were used to doing (with maybe a couple of small tweaks to lead them in a direction of greater efficiency). He knew them, and he knew that. He was being a leader.
Now I’m in my 40s and I have a much better view of everything I don’t know. (A lot of that used to be invisible to me.) It makes me both happier (for the perspective it brings) and unhappier (because I can see so many of the intellectual greenfields and curiosities that I’ll never get to spend time in — and know that more will crop up every year). I’m limited by the expiration date on this body I’m in, something that never used to cross my mind.
One of the things I’ve learned is that the best things emerge when groups of people with diverse skills (and maybe complementary interests) get together, drive out fear, and drive out preconceived notions about what’s “right” or “best”. When something amazing sprouts up, it’s not because it was your idea (or because it turned out “right”). It’s because the ground was tilled in such a way that a group of people felt comfortable bringing their own ideas into the light, making them better together, and being open to their own emergent truths.
I used to think leadership was about coming up with the BEST, RIGHT IDEA — and then pushing for it. This week, I got to see someone else pushing really hard for her “best, most right, more right than anyone else’s” idea. But it’s only hers. She’s intent on steamrolling over everyone around her to get what she wants. She’s going to be really lonely when the time comes to implement it… because even if someone starts out with her, they’ll leave when they realize there’s no creative expression in it for them, no room for them to explore their own interests and boundaries. I feel sorry for her, but I’m not in a position to point it out. Especially since she’s older than me. Hasn’t she seen this kind of thing fail before? Probably, but she’s about to try again. Maybe she thinks she didn’t push hard enough last time.
Leadership is about creating spaces where other people can find purpose and meaning. No pushing required.
Thanks to @maryconger who posted the image on Twitter earlier today. Also thanks to Mike Davis, wherever you are. If you stumble across this on the web one day, thanks for waking me up in 2000. It’s made the 18 years thereafter much more productive.
What’s Quality 4.0, why is it important, and how can you use it to gain competitive advantage? Did you know you can benefit from Quality 4.0 even if you’re not a manufacturing organization? That’s right. I’ll tell you more next week.
Since then, hundreds more have followed to help people understand more about quality and process improvement in theory and in practice. I started writing because I was in the middle of my PhD dissertation in the Quality Systems program at Indiana State, and I was discovering so many interesting nuggets of information that I wanted to share those with the world – particularly practitioners, who might not have lots of time (or even interest) in sifting through the research. In addition, I was using data science (and some machine learning, although at the time, it was much more difficult to implement) to explore quality-related problems, and could see the earliest signs that this new paradigm for problem solving might help fuel data-driven decision making in the workplace… if only we could make the advanced techniques easy for people in busy jobs to use and apply.
We’re not there yet, but as ASQ and other organizations recognize Quality 4.0 as a focus area, we’re much closer. As a result, I’ve made it my mission to help bring insights from research to practitioners, to make these new innovations real. If you are developing or demonstrating any new innovative techniques that relate to making people, processes, or products better, easier, faster, or less expensive — or reducing risks and building individual and organizational capabilities — let me know!
I’ve also learned a lot in the past decade, most of which I’ve spent helping undergraduate students develop and refine their data-driven decision making skills, and more recently at Intelex (provider of integrated environment, health & safety, and quality management EHSQ software to enterprises and smaller organizations). Here are some of the big lessons:
People are complex. They have multidimensional lives, and work should support and enrich those lives. Any organization that cares about performance — internally and in the market — should examine how it can create complete and meaningful experiences. This applies not only to customers, but to employees and partners and suppliers. It also applies to anyone an organization has the power and potential to impact, no matter how small.
Your data are your most valuable assets. It sounds trite, but data is becoming as valuable as warehouses, inventory, and equipment. I was involved in a project a few years ago where we digitized data that had been collected for three years — and by analyzing it, we uncovered improvement opportunities that when implemented, saved thousands of dollars a week. We would not have been able to do that if the data had remained scratched in pencil on thousands of sheets of well-worn legal paper.
Self-awareness must be cultivated. The older you get, and the more experience you gain, the more you know what you don’t know. Many of my junior colleagues (and yours) haven’t reached this point yet, and will need some help from senior colleagues to gain this awareness. At the same time, those of you who are senior have valuable lessons to learn from your junior colleagues, too! Quality improvement is grounded in personal and organizational learning, and processes should help people help each other uncover blind spots and work through them — without fear.
Most of all, I discovered that what really matters is learning. We can spend time supporting human and organizational performance, developing and refining processes that have quality baked in, and making sure that products meet all their specifications. But what’s going on under the surface is more profound: people are learning about themselves, they are learning about how to transform inputs into outputs in a way that adds value, and they are learning about each other and their environment. Our processes just encapsulate that organizational knowledge that we develop as we learn.
Want to find out what Quality 4.0 really is — and start realizing the benefits for your organization? If so, check out the October 2018 issue of ASQ’s Quality Progress, where my new article (“Let’s Get Digital“) does just that.
Quality 4.0 asks how we can leverage connected, intelligent, automated (C-I-A) technologies to increase efficiency, effectiveness, and satisfaction: “As connected, intelligent and automated systems are more widely adopted, we can once again expect a renaissance in quality tools and methods.” In addition, we’re working to bring this to the forefront of quality management and quality engineering practice at Intelex.
Quality 4.0 Evolution
The progression can be summarized through four themes. We’re in the “quality as discovery” stage today:
Quality as inspection: In the early days, quality assurance relied on inspecting bad quality out of items produced. Walter A. Shewhart’s methods for statistical process control (SPC) helped operators determine whether variation was due to random or special causes.
Quality as design: Next, more holistic methods emerged for designing quality in to processes. The goal is to prevent quality problems before they occur. These movements were inspired by W. Edwards Deming’s push to cease dependence on inspection, and Juran’s Quality by Design.
Quality as empowerment: By the 1990’s, organizations adopting TQM and Six Sigma advocated a holistic approach to quality. Quality is everyone’s responsibility and empowered individuals contribute to continuous improvement.
Quality as discovery: Because of emerging technologies, we’re at a new frontier. In an adaptive, intelligent environment, quality depends on how:
quickly we can discover and aggregate new data sources,
effectively we can discover root causes and
how well we can discover new insights about ourselves, our products and our organizations.”
This article originally appeared on the Intelex Community on 9/14/2018 at https://community.intelex.com/explore/posts/why-fema-monitoring-waffle-house-weekendSometimes the most informative metrics show up in the strangest of places.Case in point: with a hurricane making landfall today in North Carolina, and the prospect for catastrophic flooding over the weekend and into next week, emergency managers are mobilizing for action – and if you’re in the path of the storm, you may be doing the same. Have you started monitoring the Waffle House Index? The US Federal Emergency Management Agency (FEMA) has.Originally devised by W. Craig Fugate, former FEMA Director, the Waffle House Index is based on the observation that the popular 24-hour breakfast chain has historically been unusually well prepared for disasters. Part of their business model is to be the spot for emergency personnel to rely on for their coffee and nourishment – a valuable role when power crews, rescue teams, and debris removal workers are working long, hard hours.To do this, they make sure all employees have disaster training and stock all their restaurants with generators, and have a reduced menu specifically to be offered in the aftermath of a disaster. Over time, this even led to a more formal partnership between the organizations. FEMA first responders are known to set up initial operations in Waffle House locations. Waffle House now reports the status of each location to FEMA after a disaster to facilitate data collection.The Waffle House Index is a red, yellow, or green marker placed on a map wherever a Waffle House location is found. Under normal conditions, the marker is green. If the restaurant has shifted into emergency operations and is offering their limited menu, the marker is yellow. If the marker is red, that means that the Waffle House is closed – either the site itself is damaged or destroyed, emergency staff can not reach the site, the emergency generators are down or out of fuel, or there is a food shortage. When FEMA sees one or more reds, they know an area is in particularly bad shape – and they’ll need to help.What can you learn about risk-based thinking from the Waffle House index? Three things: first, that you can (and should) look outside your organization for risk indicators that might help you make better (and faster) decisions, particularly when those risks are activated. Second, that you should explore crowdsourced risk data as a source of up-to-date information.And finally – if Waffle House is closed, there’s a serious problem.AdditionalReading: McKnight, B., & Linnenluecke, M. K. (2016). How firm responses to natural disasters strengthen community resilience: A stakeholder-based perspective. Organization & Environment, 29(3), 290-307.
Walter, L. (2011, July 6) What do waffles have to do with risk management? EHS Today. Available from https://www.ehstoday.com/fire_emergencyresponse/disaster-planning/waffles-risk-management-0706