Category Archives: Uncategorized

A 15-Week Course to Introduce Machine Learning and Intelligent Systems in R

lantz-ml-in-rEvery fall, I teach a survey course for advanced undergraduates that covers one of the most critical themes in data science: intelligent systems. According to the IEEE, these are “systems that perceive, reason, learn, and act intelligently.” While data science is focused on analyzing data (often quite a lot of it) to make effective data-driven decisions, intelligent systems use those decisions to accomplish goals. As more and more devices join the Internet of Things (IoT), collecting data and sharing it with other “things” to make even more complex decisions, the role of intelligent systems will become even more pronounced.

So by the end of my course, I want students to have some practical skills that will be useful in analyzing, specifying, building, testing, and using intelligent systems:

  • Know whether a system they’re building (or interacting with) is intelligent… and how it could be made more intelligent
  • Be sensitized to ethical, social, political, and legal aspects of building and using intelligent systems 
  • Use regression techniques to uncover relationships in data using R (including linear, nonlinear, and neural network approaches)
  • Use classification and clustering methods to categorize observations (neural networks, k-means/KNN, Naive Bayes, support vector machines)
  • Be able to handle structured and unstructured data, using both supervised and unsupervised approaches
  • Understand what “big data” is, know when (and when not) to use it, and be familiar with some tools that help them deal with it

My course uses Brett Lantz’s VERY excellent book, Machine Learning with R (which is now also available in Kindle format), which I provide effusive praise for at

One of the things I like the MOST about my class is that we actually cover the link between how your brain works and how neural networks are set up. (Other classes and textbooks typically just show you a picture of a neuron superimposed with inputs, a summation, an activation, and outputs, implying that “See? They’re pretty much the same!”) But it goes much deeper than this… we actually model error-correction learning and observational learning through the different algorithms we employ. To make this point real, we have an amazing guest lecture every year by Dr. Anne Henriksen, who is also a faculty member in the Department of Integrated Science and Technology at JMU. She also does research in neuroscience at the University of Virginia. After we do an exercise where we use a spreadsheet to iteratively determine the equation for a single layer perceptron’s decision boundary, we watch a video by Dr. Mark Gluck that shows how what we’re doing is essentially error-correction learning… and then he explains the chemistry that supports the process. We’re going to videotape Anne’s lecture this fall so you can see it!

Here is the syllabus I am using for Fall 2015. Please feel free to use it (in full or in part) if you are planning a similar class… but do let me know!

Getting Deep With Value Creation

Image Credit: Doug Buckley of

Image Credit: Doug Buckley of

In his November post, ASQ CEO Paul Borawski asks “What new fields or disciplines could most reap the benefits of quality tools and techniques?”

He notes that although the tradition of quality assurance, control, and improvement emerged from manufacturing, the techniques are now widely acknowledged and applied in many fields such as healthcare, education, and service. So what’s next?

One of the things I like to do when I’m trying to be a futurist is to go back to first principles. Explore the basis for why we do what we do… what makes us tick… why we like improving quality and making processes more effective, more efficient. And in doing so, while reflecting on Paul’s question, I think what’s next is…

Getting Deep with Value Creation.

As quality professionals we spend most of our time and energy figuring out how to create value. Either we’re improving the systems we work with to tweak out additional value, or we’re working with customers and stakeholders to figure out how to provide them with more value, or we’re focusing on innovation – figuring out how to create value for the future — reliably, consistently, and according to new and unexpected business models.

To me, this starts with me. How can I improve myself so that I’m a kickass vessel for the delivery of value? How can I use quality principles and quality tools to find – and align myself – with what I’m supposed to be doing at any given time? How can I become most productive in terms of the deep, meaningful value I add to those around me?

I know that others feel the same way. Marc Kelemen, a member of the ASQ Board of Directors, is leading a charge to develop a Body of Knowledge for Social Responsibility. He recognizes that the personal element is crucial if we’re trying to become socially responsible as teams, and organizations, and communities. So we’ll be working on this over the next few months… figuring out how to get deep with the notion of value creation, and how we can do it within ourselves so that we are better positioned to help others do it too.

Peripheral Visioning


Image Credit: Doug Buckley of

Somehow, some way, over the course of too many years growing up staring into a computer screen — my eyesight became much-less-than-perfect.

Only I didn’t know it. I thought everyone lived in a slightly hazy, cloudy world, where all the colors naturally blended into postmodern mosaics of distant trees and mountains. It was never a problem for me until the day about ten years ago that I was headed east on I-64 into Charlottesville, and coming over the hill into town, struggled to identify what that giant number on the speed limit sign was. I squinted, closed one eye at a time, and figured that the number was probably 55… so I slowed down. Then I realized:

They probably make those speed limit signs big enough for anyone to see.

I got scared, and drove straight to the walk-in eyeglass clinic, where I explained my predicament. They quickly made room in their schedule for an emergency appointment. Usually afterwards, they make you wait 24 hours to pick up your new glasses, but with my 20/400 vision, they wouldn’t let me leave without them. Fortunately, my eyesight could be corrected to almost 20/20, which was nice. I walked out of the store with my new glasses on — and into an amazing, sparkly new world! The trees all had individual leaves on them!! Cars were so shiny! I could read license plates — from MY driver’s seat!

But immediately, I recognized how I’d managed to drive for all those years with bad vision!

Because I couldn’t really see what was ahead of me, I just focused my vision off and to the right side of the road, on the ground. I kept the road and the cars in my peripheral vision, so I could easily sense where they were, and make accommodations. If I tried to look straight ahead, I got frustrated quickly, emotionally wrapped around my own axle, because I couldn’t see any of the detail… and ultimately, that state of being wasn’t safe for driving. I couldn’t focus on what I was worried about, or I’d be a danger on the road.

Not long after that, I realized how effective a strategy this was in my work — because there’s so much change and uncertainty, it’s impossible to look directly ahead of you and see clearly. And that can be scary and unsettling! My solution was: if there was some big goal I was trying to achieve, the best way to reduce my angst and qualm my (sometimes very subtle) emotional stranglehold on myself — was to focus on something else. Something just as important, maybe even something that contributed to the main goal, but something I was not quite so emotionally wrangled by!

I starting calling this my “peripheral visioning” technique. It actually helped me achieve my primary goals – because by consciously setting my primary goal to the side, and focusing on something related to it (or maybe in support of it), I was still making progress but I wasn’t experiencing as much stress. And as a result, I was more open to the serendipity and the chance encounters – with people and with information – that helped me make progress on the primary goal!

Set an intention, get your ducks in a row, and then get out of your own way by focusing on something else!

2012 Management Improvement Carnival – Part 2

doug-jan-e(Image Credit: Doug Buckley of

I am pleased once again to host ASQ Influential Voices blogger John Hunter’s Management Improvement Carnivalfeaturing some interesting or noteworthy articles that have been posted over the past year. Be sure to check out previous installations of the Carnival to get a broad sample of the most recent blog posts that are relevant to managers who are interested in quality, innovation and process improvement.

This post covers the final two of four blogs that I reviewed for the Management Improvement Carnival: Design Thinking (Thoughts by Tim Brown), and Business901 by Joe Dager (who I also follow on Twitter at @business901).

(In Part 1, I reviewed 2012 posts in StatsMadeEasy and the Harvard Business Review’s (HBR) collection of insights from Peter Bregman.)

Any student of innovation is bound to be familiar with IDEO, a design firm that has a long history of uniting the power of the creative culture with practical, tangible results. IDEO’s President, Tim Brown, blogs at In addition to sharing their creative prototypes online, IDEO has also produced a Human-Centered Design Toolkit which can help nonprofits and community agencies apply design thinking to innovate in their domains. It’s a powerhouse of a company with a long-lived reputation for pushing the limits when it comes to thinking about innovation.

In “Does the Media Have a Negative Effect on Innovation?” Tim asks whether news media’s focus on the risks of certain technologies and new developments might blind us to potentially positive – and breakthrough – innovations related to those technologies. He encourages positive storytelling as a means of stimulating innovation, and challenges us to embark on “optimistic and helpful journalism” when we share technical stories. These, he suggests, will encourage even greater innovation.

In October, Tim shared the secret phrase all innovators use… and it’s rather simple and straightforward! Since reading this post last year, I’ve tried his approach when I’m presented with challenging situations… even those in my own life, where I’m required to find creative solutions to time management issues all my own. It’s been a valuable technique… I encourage you to find out what it is – and try it!

I was really inspired by the November post on Creative Confidence. In addition to having great ideas, you have to connect with your stakeholders in an authentic, emotional way – and not be afraid to really follow through, even when you’re opposed in one way or another. He links to a TED talk where you can hear more about his idea, and summarizes what creative thinking means to him:

“Creative thinking in business begins with having empathy for your customers (whether they’re internal or external), and you can’t get that sitting behind a desk.”

In the quality world, we’re used to seeing how concepts from the visual factory can be quickly and easily applied to generate high value. In “Make it Visual,” Tim supports this idea, but notes that it can be just as straightforwardly applied to idea generation as to operations. He encourages us to try, for a week, to “record observations and ideas visually” to bring abstract ideas into being. This is definitely an experiment I’m planning for sometime in 2013.

Last but definitely not least is Joe Dager‘s Business901 blog. I follow Joe on Twitter – and his feed is always buzzing with cool thoughts and ideas that keep me on my feet. Along with Saul Kaplan and Valdis Krebs’ OrgNet, it’s one of my favorite ways to keep up with ideas and insights that will help me understand innovation in greater depth. Joe is one of my primary sources for figuring out which cool TED talk to watch next.

In February, Joe introduced me to Appreciative Inquiry… which I had never heard of before. This approach to organizational development, which emerged from knowledge management research in the 1990’s, seeks to integrate positive thinking (and thus positive psychology in general) into the design and management of organizations. Quality researchers, take note of Appreciative Inquiry!! In my January 2013 article in the Quality Management Journal, my analysis of QMJ research indicates that integrating positive psychology into quality management is a gap that we must all seek to fill.

As a fan of Jane McGonigal, I’m also interested in Joe’s posts on how gaming can enhance various aspects of business and quality management. He introduces us in one post to Dave Gray’s Gamestorming technique, which I’d like to try. In another post, “How Gaming Teaches You to Plan,” he suggests that games provide excellent training for navigating your way through messy (real-life) situations, where your ability to change and adapt can be paramount to survival:

Understanding when to deviate from your plan through adjusting or even discarding it entirely can be learned and simulated through gaming.

I also think Joe and I are similarly oriented in our thinking in many ways… so many of his articles truly resonate with me. For example, I strongly support Deming’s notion that the underlying purpose of the 14 Points is to help people be able to work with joy… and articles like April’s “The Show Business Side of Service Design” seem to present the same encouragement. In this post, he summarizes his podcast with Adam Lawrence of Work Play Experience, who asserts that service design is theatrical… and thus should be fun. When asked by Joe whether that means all interactions should be scripted (a common practice in service scenarios to ensure consistency) Adam responded that each person must interpret those words in their own way… and that improvisation centered around the core message is, of course, not excluded. I’d never considered service occupations to require a dramatic flair… but many of them do, and indeed, leveraging this as a feature of those kind of jobs could make the work environment more fun.

Joe’s got tons of great posts, and a frequent update schedule. Follow him on Twitter to get real-time updates about when new articles are posted.

That’s it for my contribution to this year’s Management Improvement Carnival. See you next year!

The December 2012 End of the World IMPROVEMENT CHALLENGE

lucy-dec3(Image Credit: Lucy Glover of Lucy Glover Photography, San Francisco CA. Used with permission.)

Hey everybody, remember last month when everyone was posting things they were thankful for in the twenty-odd days leading up to Thanksgiving? (They might still be doing it… I don’t know.) I thought that was a great idea. So I started doing something similar this month that I’ll tell you all about now!!

But as many of you know, the Mayan Calendar is coming to an end, and we’re moving into a new world of completely undetermined proportion. Some predict a doomsday scenario, which means it will be very easy to see what’s changed. Others predict a BIG NOTHING, a non-event kind of like Y2K (well… that one actually had some ripple effects for me. But that’s another story that I’ll post later. I diverge.)

A non-event means IT’S UP TO US TO CHANGE THINGS. So my challenge to all of you for December 2012 is: let’s get in the habit of improving a least ONE thing a day between now and the much hyped “end of the world”. If the world does end, it will end being just a little better than it was at the beginning of December. And if it doesn’t end, we might have 1) developed a new habit or mode of self-reflection that will serve us well moving ahead into 2013, and/or 2) built some very useful social capital that will enhance the resilience of our individual communities.

(Disappointed that you didn’t get to the party on December 1st? Don’t worry! Make your improvement for today to START IMPROVING ONE THING A DAY, starting NOW!)

I’ll toss out some ideas for your own DAILY IMPROVEMENT CHALLENGE at the end of this post.

But in the meantime, let’s broadly consider what would happen to our sociotechnical systems (composed of people, products, processes, and projects) in the event of a massive shift or change (of any variety, “new age” or “old age”!) The products will change. The projects will change. The processes will be adapted to make projects to create the new products, and since we don’t know what the environment will be like, or what the new products we’ll need will be…

… the only STABLE element in this mix is the PEOPLE.

When the world disruptively changes around us without killing us, we’re still left behind. Which means our personal capabilities and our capabilities working together in groups and communities – our social capital – becomes increasingly more important.

My friend Daniel Aldrich, who’s been seriously researching this for several years, has determined that social capital is the number one thing that helps communities revitalize after disasters. So if you think there’s a possibility of a major change, you could prepare by stockpiling food and fuel, or you could just work on building your own self-reliance and the social capital within your community.

So I challenge you to DO ONE THING EVERY DAY between now and December 21, 2012 to accomplish one of the following improvement goals, all of which are related to increasing positive feelings:

  • improve how YOU feel
  • improve how someone ELSE feels
  • improve something about your ENVIRONMENT, as long as it make YOU or someone else feel good/better
  • do something courageous to improve your SELF-CONFIDENCE or self-image (or someone else’s!)
  • improve your AWARENESS of other peoples’ beliefs, situations, circumstances, or beliefs
  • improve your BURDEN by getting rid of a grudge or negative feelings… even if only for a day

Think about the many sources of waste, or maybe read about 5S, to get you started with ideas for where you might begin. Scott Rutherford (@srlean6) also recommends this post  as well as this one for some background on 5S.

(For example, yesterday, I decided to improve someone’s day! We went to a restaurant grand opening, and the place was packed like sardines. Our server was rushing around from table to table, sweating profusely but still maintaining an admirably positive vibe. When he got to our table, smiling with enthusiasm, and asked us what we wanted – I told him I wanted him to close his eyes for a minute, and take three DEEP breaths! He thought this was a bizarre request, but he did it. After all, I was the customer… right? After his third deep breath he said “Wow! I really do feel better. Just that minute of standing still is really going to help me get through this big grand opening night.” He was visibly more relaxed with everyone the rest of the evening. See how easy it can be?)

The world changes when we change. So let’s go!! Let’s start some improvement habits that will spread good feelings and inspire ourselves and others. Let’s use this time to learn how to make it a daily practice.

And post in the comments – tell us what you have chosen to improve from day to day!

Access METAR Weather Data in R Statistical Software

Although this is neither quality nor innovation related, as a multi-decade weather geek and degreed meteorologist, I still really love my weather data. Today I wanted to learn how to retrieve historical weather data from within the R statistical software. I managed to get a list within R that contains METAR observations for an entire day for one observing station. Here’s how I did it!

1. First, I signed up for a KEY to use the Weather Underground API at – I’m not going to tell you what my personal key is, but it has 16 characters and looks kind of like this: d7000XXXXXXXXXXX

2. Next, I installed the rjson package into R

3. Then, I used this code to find out that there were 46 observations for August 11, 2012 (the date of interest). You’ll have to try it with YOUR new Weather Underground API key in place of the d7000XXX… :


x <- fromJSON("


daily.metars <- rep(NA,length((x$history)$observations))
for (n in 1:length((x$history)$observations)) {
daily.metars[n] <- (x$history$observations)[[n]]$metar

4. Now you have a list in R called daily.metars that contains strings holding all of your METARs for the day! Here’s the header from the list that I produced:

> head(daily.metars)
[1] "SPECI KCHO 110408Z AUTO 00000KT 5SM VCTS -RA BR SCT003 BKN030 OVC075 21/19 A2983 RMK AO2 LTG DSNT SW P0001"
[2] "SPECI KCHO 110433Z AUTO 20005KT 6SM -TSRA BR FEW003 BKN033 OVC110 21/19 A2984 RMK AO2 LTG DSNT NE AND S AND SW TSE11B27 P0002"
[3] "METAR KCHO 110453Z AUTO 00000KT 5SM -TSRA BR SCT018 SCT046 OVC100 21/19 A2983 RMK AO2 LTG DSNT NE-S TSE11B27 SLP093 P0003 T02060194 402940200"
[4] "SPECI KCHO 110529Z AUTO 00000KT 8SM -RA FEW070 SCT095 BKN110 21/19 A2982 RMK AO2 LTG DSNT NE-SE TSE23 P0001"
[5] "METAR KCHO 110553Z AUTO 00000KT 8SM FEW085 21/19 A2982 RMK AO2 LTG DSNT NE-SE TSE23RAE30 SLP090 P0001 60127 T02060194 10250 20200 58004"
[6] "METAR KCHO 110653Z AUTO 18005KT 8SM BKN090 21/19 A2983 RMK AO2 SLP093 T02060194"

« Older Entries Recent Entries »