Tag Archives: intelligent systems

A 15-Week Course to Introduce Machine Learning and Intelligent Systems in R

lantz-ml-in-rEvery fall, I teach a survey course for advanced undergraduates that covers one of the most critical themes in data science: intelligent systems. According to the IEEE, these are “systems that perceive, reason, learn, and act intelligently.” While data science is focused on analyzing data (often quite a lot of it) to make effective data-driven decisions, intelligent systems use those decisions to accomplish goals. As more and more devices join the Internet of Things (IoT), collecting data and sharing it with other “things” to make even more complex decisions, the role of intelligent systems will become even more pronounced.

So by the end of my course, I want students to have some practical skills that will be useful in analyzing, specifying, building, testing, and using intelligent systems:

  • Know whether a system they’re building (or interacting with) is intelligent… and how it could be made more intelligent
  • Be sensitized to ethical, social, political, and legal aspects of building and using intelligent systems 
  • Use regression techniques to uncover relationships in data using R (including linear, nonlinear, and neural network approaches)
  • Use classification and clustering methods to categorize observations (neural networks, k-means/KNN, Naive Bayes, support vector machines)
  • Be able to handle structured and unstructured data, using both supervised and unsupervised approaches
  • Understand what “big data” is, know when (and when not) to use it, and be familiar with some tools that help them deal with it

My course uses Brett Lantz’s VERY excellent book, Machine Learning with R (which is now also available in Kindle format), which I provide effusive praise for at http://qualityandinnovation.com/2014/04/14/the-best-book-ever-on-machine-learning-and-intelligent-systems-in-r/

One of the things I like the MOST about my class is that we actually cover the link between how your brain works and how neural networks are set up. (Other classes and textbooks typically just show you a picture of a neuron superimposed with inputs, a summation, an activation, and outputs, implying that “See? They’re pretty much the same!”) But it goes much deeper than this… we actually model error-correction learning and observational learning through the different algorithms we employ. To make this point real, we have an amazing guest lecture every year by Dr. Anne Henriksen, who is also a faculty member in the Department of Integrated Science and Technology at JMU. She also does research in neuroscience at the University of Virginia. After we do an exercise where we use a spreadsheet to iteratively determine the equation for a single layer perceptron’s decision boundary, we watch a video by Dr. Mark Gluck that shows how what we’re doing is essentially error-correction learning… and then he explains the chemistry that supports the process. We’re going to videotape Anne’s lecture this fall so you can see it!

Here is the syllabus I am using for Fall 2015. Please feel free to use it (in full or in part) if you are planning a similar class… but do let me know!

The Best Book Ever on Machine Learning (and Intelligent Systems) in R

lantz-ml-in-rDear Brett (Lantz),

In short: your book, Machine Learning with R, is the book I’ve been dreaming about for years. Everyone who applies machine learning techniques for their work, teaches applied machine learning at a university, or just loves R and wants to know more about these super cool algorithms should buy and use your book.

I’ve been teaching a course called “Intelligent Systems” (ISAT/CS 344 at JMU) for the past few years. I inherited a syllabus and course description from professors who had taught the course from the mid-1990’s until 2009, so I started out following their lead and broadly covering expert/knowledge-based systems, simple neural networks for regression, and some elements of robotics. We used a commercial package to build the expert systems (rather than a declarative language like Prolog), which was fine, but we also used a commercial package for the neural networks. I was unsatisfied for two reasons: first, I knew that far more “stuff” was going on in the world of intelligent systems which we weren’t sharing with our students, and second, I knew there were tons of free packages in the R Statistical Software that could perform the same tasks… and more. I started a yearlong process of soul-searching and creating new materials… determined to bring R to the classroom, along with neural networks for both classification and regression, classification using k-nearest neighbors and Naive Bayes approaches, clustering with k-means, and some text mining and analysis to show students what you could do with unstructured data.

I also wanted to compare and contrast neural network regression with simple linear regression, classification algorithms in general with logistic regression, and share how to evaluate and improve model performance using metrics like precision, recall, and F1. (I mean, who cares about developing an intelligent software system if you can’t evaluate and continually improve its performance?) In addition, I’ve dreamed about adding a module on decision trees, in particular focusing on the C5.0 algorithm. But I haven’t found the time to explore or create new course materials on this topic. So I knew it would be even harder to compile all of my course materials into a book for my students to reference.

But you, in the meantime, have saved my life. I’ve explored tons of books on machine learning and intelligent systems that focus more on the practical applications of the techniques rather than the theory… and I have not found one that meets my standards, until now. In a friendly and conversational manner (that’s not overfriendly, condescending, or flippant) you have managed to cover pretty much all of the topics I want to share in my intelligent systems class — in a way that I’m comfortable with.

Chapters 1 (Introduction to Machine Learning) and 2 (Managing and Understanding Data) provide a great, simplified introduction to what machine learning is all about and highlights the data structures and R commands that might be the most useful for these purposes. Chapters 3 and 4 cover classification… first with k-nearest neighbors, then with Naive Bayes. Chapter 5 covers decision trees and C5.0. Chapter 6 covers regression in general, but with applications to decision trees (yeah!) In Chapter 7 (Black Box Methods – Neural Networks and Support Vector Machines) there’s a great example based on Optical Character Recognition (which will pair nicely with the lab exercise I already use). Chapter 8 covers Apriori, Chapter 9 introduces clustering with k-means, and Chapters 10 and 11 specifically deal with evaluating and improving model performance.

As a cherry on top of the cake that is this book, Chapter 12 provides an overview of most-used ways to acquire data (e.g. using RCurl, XML, and JSON) and even introduces parallel computing.

I am eternally grateful to you for writing the book that’s been in my head in a way I (think I!) would have written it. It’s not PERFECT (I would have spent more time on concepts like overfitting, and maybe given examples… and maybe some prose on the Turing Test and Reverse Turing Test) — but I can easily use your book as a required text and then provide supplemental materials on the side.

Thank you Brett!

Sincerely and with a world of gratitude,