Category Archives: Book Reviews

Dean Meyer’s “How Organizations Should Work” – an A+ Reference on Intentional Organizational Design

Posted on November 21, 2022 by Nicole Radziwill Leave a comment

The organization that you design designs you back.

Similarly, the organization that you fail to intentionally design will also design you back. And you probably won’t like the pain it inflicts… whether that pain manifests as political battles, conflicts of interests, or just plain stonewalling or slow-walking. People tend to perform based on how well their goals and objectives are defined, how effectively their roles (and the relationships between their roles and other roles) are defined, and how rigorously the organization monitors and reinforces desired behaviors and outcomes.

Unfortunately, it’s rare to get all those pieces in place and functioning at the same time. But thanks to Dean Meyer’s new 2022 book, How Organizations Should Work, you’ll have a head start on lessons learned. Based on his multi-decade career in organizational design, he provides simple, tangible, and meaningful explanations that will help you learn how to intentionally design your organization.

I brought this book to Burning Man because I wanted to read it in a place where I could take my time, where I could allow my mind to expand and take in Dean’s lessons, where I could feel the inspiration all around me while figuring out how to bake more inspired organizational design into my own workplace. And I did start reading it there.

But this book is so packed with wisdom and insights that you’ll want to read it slowly. Plan adequate time for it. After getting through the first 80 pages or so at the event, I read one or two sections every weekend. It took me a full three months to get through cover to cover… and I’m someone who can read a whole book in one sitting. This one, though, will make you think, and you’ll need time to pause and reflect on each of the stories and narratives that Dean uses to make his points.

Here’s a small sample of the highlights (and there are more; you can find this book on Amazon for $37-40).

Book Summary (p. 465-494). The best way to develop a reading plan for this book is to start at the end. Dean provides a helpful synopsis of each chapter that you can use to figure out which sections are most relevant for your organization now. The book is structured so that you don’t have to read it from start to finish, but can pick the sections that are the highest priority for continuous improvement in your organization. Some sections (like Chapter 19, on managing workers in the field) may be less relevant to your needs than other sections (like Chapter 15, on aligning sales and marketing). I recommend starting with Chapters 1 through 6 (and if your time is limited, start with Chapters 5 & 6).

Chapter 5 – Market Organization & Chapter 6 – Empowerment (p. 33-57). These two chapters provide the best (and most concentrated) view of the main thesis of the book, which is that every organization should be structured as a market. Each functional area should have customers, and services they provide to those customers. Each functional area should be empowered to make decisions about how the business of that unit is conducted, and should also have the skills to do it (and to build trust within their business unit and between their business unit and others). Organizing according to a market allows people to specialize, makes it possible to monitor performance in a more modular way, and simplifies the cognitive complexity of an organization.

Appendix 4 – Culture: Examples of Behavioral Principles (p. 376-380). Dean notes that companies are great at defining their culture, but often not so great at explaining how people can embody that culture and monitor their behaviors to ensure that they are living it and reinforcing it. In this section, he provides specific examples of behaviors that can make goals like “maintain a high ethical standard” actionable. For example, you can say “we do not permit personal conflicts of interest”. You’ll have to test your processes against these behavioral expressions of culture, too: if you don’t permit personal conflicts of interest, you should identify areas where conflicts of interest might arise and anticipate ways to identify them, prevent them, or resolve them quickly.

There are so many things I like about this book, and it embodies so many principles of good organizational design in a light, conversational way. Although there are a lot of books I really like, I don’t typically gush over them… but this one meets my high standards. Coupled with Dean’s 2017 book, Principle-Based Organizational Structure (which has a less conversational and more academic style), these are the only two references I really need to remind me of the essentials for intentionally building healthy organizations that are aligned internally and externally.

Alignment is the best way to reduce friction between people, and accelerate real progress towards tangible goals.

#10 – Understanding Agile Values & Principles (Duncan)

Duncan, Scott. (2019). Understanding Agile Values & Principles. An Examination of the Agile Manifesto. InfoQ, 106 pp. Available from https://www.infoq.com/minibooks/agile-values-principles

The biggest obstacle in agile transformation is getting teams to internalize the core values, and apply them as a matter of habit. This is why you see so many organizations do “fake agile” — do things like introduce daily stand-ups, declare themselves agile, and wonder why the success isn’t pouring in. Scott goes back to the first principles of the Agile Manifesto from 2001 to help leaders and teams become genuinely agile.

#9 – Risk-Based Thinking (Muschara)

Muschara, T. (2018). Risk-Based Thinking: Managing the Uncertainty of Human Error in Operations. Routledge/Taylor & Francis: Oxon and New York. 287 pages.

Risk-based thinking is one of the key tenets of ISO 9001:2015, which became the authoritative version in September 2018. Although clause 8.5.3 from ISO 9001:2008 indirectly mentioned risk, it was not a driver for identifying and executing preventive actions. The new emphasis on risk depends upon the organizational context (clause 4.1) and the needs and expectations of “interested parties” or stakeholders (clause 4.2).

Unfortunately, the ISO 9001 revision does not provide guidance for how to incorporate risk-based thinking into operations, which is where Muschara’s new book fills the gap. It’s detailed and complex, but practical (and includes immediately actionable elements) throughout. For anyone struggling with the new focus of ISO 9001:2015, this book will help you bring theory into practice.

#8 – The Successful Software Manager (Fung)

Fung, H. (2019). The Successful Software Manager. Packt Publishing, Birmingham UK, 433 pp.

There lots of books on the market that provide technical guidance to software engineers and quality assurance specialists, but little information to help them figure out how (and whether) to make the transition from developer to manager. Herman Fung’s new release fills this gap in a complete, methodical, and inspiring way. This book will benefit any developer or technical specialist who wants to know what software management entails and how they can adapt to this role effectively. It’s the book I wish I had 20 years ago.

#7 – New Power (Heimans & Timms)

Heiman, J. & Timms, H. (2018). New Power: How Power Works in Our Hyperconnected World – and How to Make it Work For You. Doubleday, New York, 325 pp.

As we change technology, the technology changes us. This book is an engaging treatise on how to navigate the power dynamics of our social media-infused world. It provides insight on how to use, and think in terms of, “platform culture”.

#6 – A Practical Guide to the Safety Profession (Maldonado)

Maldonado, J. (2019). A Practical Guide to the Safety Profession: The Relentless Pursuit (CRC Focus). CRC Press: Taylor & Francis, Boca Raton FL, 154 pp.

One of the best ways to learn about a role or responsibility is to hear stories from people who have previously served in those roles. With that in mind, if you’re looking for a way to help make safety management “real” — or to help new safety managers in your organization quickly and easily focus on the most important elements of the job — this book should be your go-to reference. In contrast with other books that focus on the interrelated concepts in quality, safety, and environmental management, this book gets the reader engaged by presenting one key story per chapter. Each story takes an honest, revealing look at safety. This book is short, sweet, and high-impact for those who need a quick introduction to the life of an occupational health and safety manager.

# 5 – Data Quality (Mahanti)

Mahanti, R. (2018). Data Quality: Dimensions, Measurement, Strategy, Management and Governance. ASQ Quality Press, Milwaukee WI, 526 pp.

I can now confidently say — if you need a book on data quality, you only need ONE book on data quality. Mahanti, who is one of the Associate Editors of Software Quality Professional, has done a masterful job compiling, organizing, and explaining all aspects of data quality. She takes a cross-industry perspective, producing a handbook that is applicable for solving quality challenges associated with any kind of data.

Throughout the book, examples and stories are emphasized. Explanations supplement most concepts and topics in a way that it is easy to relate your own challenges to the lessons within the book. In short, this is the best data quality book on the market, and will provide immediately actionable guidance for software engineers, development managers, senior leaders, and executives who want to improve their capabilities through data quality.

#4 – The Innovator’s Book (McKeown)

McKeown, M. (2020). The Innovator’s Book: Rules for Rebels, Mavericks and Innovators (Concise Advice). LID Publishing, 128 pp.

Want to inspire your teams to keep innovation at the front of their brains? If so, you need a coffee table book, and preferably one where the insights come from actual research. That’s what you’ve got with Max’s new book. (And yes, it’s “not published yet” — I got an early copy. Still meets my criteria for 2019 recommendations.)

#3 – The Seventh Level (Slavin)

Slavin, A. (2019). The Seventh Level: Transform Your Business Through Meaningful Engagement with Customer and Employees. Lioncrest Publishing, New York, 250 pp.

For starters, Amanda is a powerhouse who’s had some amazing marketing and branding successes early in her career. It makes sense, then, that she’s been able to encapsulate the lessons learned into this book that will help you achieve better customer engagement. How? By thinking about engagement in terms of different levels, from Disengagement to Literate Thinking. By helping your customers take smaller steps along this seven step path, you can make engagement a reality.

#2 – Principle Based Organizational Structure (Meyer)

Meyer, D. (2019). Principle-Based Organizational Structure: A Handbook to Help You Engineer Entrepreneurial Thinking and Teamwork into Organizations of Any Size. NDMA, 420 pp.

This is my odds-on impact favorite of the year. It takes all the best practices I’ve learned over the past two decades about designing an organization for laser focus on strategy execution — and packages them up into a step-by-step method for assessing and improving organizational design. This book can help you fix broken organizations… and most organizations are broken in some way.

#1 Story 10x (Margolis)

Margolis, M. (2019). Story 10x: Turn the Impossible Into the Inevitable. Storied, 208 pp.

You have great ideas, but nobody else can see what you see. Right?? Michael’s book will help you cut through the fog — build a story that connects with the right people at the right time. It’s not like those other “build a narrative” books — it’s like a concentrated power pellet, immediately actionable and compelling. This is my utility favorite of the year… and it changed the way I think about how I present my own ideas.

Hope you found this list enjoyable! And although it’s not on my Top 10 for obvious reasons, check out my Introductory Statistics and Data Science with R as well — I released the 3rd edition in 2019.

Writing a Great Article Review

Posted on January 16, 2018 by Nicole Radziwill One comment

We’re teaching a class on blockchain and cryptocurrencies this semester, and since the field is so new and changing rapidly, we’ve asked our students to make finding and reviewing articles part of their learning practice this semester. This is a particularly challenging topic for this task because there’s so much hype, marketing, and fluff around these topics. We want to slice through that, and improve the signal-to-noise for people new to learning about blockchain and cryptocurrencies. Here are some tips I just prepared for our students — they may be helpful to anyone writing article reviews, especially for technology-related areas.

0 – Type of Source. Reviews or articles from from arXiv, Google Scholar were strong; reviews from Coindesk, CNN were weak; reviews from WSJ and Hacker Noon went both ways. Here are two submissions that were publishable with only minor edits:

1 – Spelling & Grammar. Most of you are college seniors, and the few who aren’t… are juniors. Please use complete sentences that make sense, with words that are spelled correctly! If this is hard for you, remember that every one of you has spell check. One way to remember this is to draft your posts in Word, and then perform spell check before you copy and paste what you wrote into WordPress.

1 – Your job is to create the TL;DR. What’s the essential substance of the source you’re reviewing? What are the main lessons or findings? If you were taking notes for an exam, what elements would you capture? (Using this perspective, commentary about how good or bad you think the article was, or what it didn’t cover well, would not help you on an exam.)

2 – Choose solid source material — primary sources, e.g. research papers, if possible. If the article is less than ~400-500 words, it’s probably not detailed enough to write a 250-300 word summary/analysis. Your job in this class is to break down complex topics & help people understand them. If your article is short and already very easy to understand, there’s nothing for you to do.

3 – Avoid “weasel words” (phrases or sentences that sound like marketing or clickbait but actually say nothing) and words/sentences that sound like you’re writing a Yelp or Amazon review rather than a critical academic review. Here are a couple weaselly examples drawn from this week’s draft posts (see if you can spot what’s wrong):

It is clear how beneficial blockchain can be to smaller businesses.
Blockchain has the potential to change the world.
Each other the topics covered in the article deserve their own piece and could be augmented upon greatly.
There is a degree of uncertainty that comes with an emerging technology.
Blockchain can bring them into the 21st century to compete with larger corporations.
Many people are scared of the changes, and governments will seek to regulate it.

4 – Answer the “so what” question. Why is this topic interesting or compelling?

5 – Choose information-rich tags. For example, in our class, don’t include blockchain as a tag… pretty much everything we do will be related to blockchain, and everyone will tend to use it, so there won’t be much information contained in the tag.

What Protests and Revolutions Reveal About Innovation

Posted on July 31, 2016 by Nicole Radziwill Leave a comment

The following book review will appear in an issue of the Quality Management Journal later this year:

The End of Protest: A New Playbook for Revolution. 2016. Micah White. Toronto, Ontario, Canada. Alfred A. Knopf Publishing. 317 pages.

You may wonder why I’m reviewing a book written by the creator of the Occupy movement for an audience of academics and practitioners who care about quality and continuous improvement in organizations, many of which are trying to not only sustain themselves but also (in many cases) to make a profit. The answer is simple: by understanding how modern social movements are catalyzed by decentralized (and often autonomous) interactive media, we will be better able to achieve some goals we are very familiar with. These include 1) capturing the rapidly changing “Voice of the Customer” and, in particular, gaining access to its silent or hidden aspects, 2) promoting deep engagement, not just in work but in the human spirit, and 3) gaining insights into how innovation can be catalyzed and sustained in a truly democratic organization.

This book is packed with meticulously researched cases, and deeply reflective analysis. As a result, is not an easy read, but experiencing its modern insights in terms of the historical context it presents is highly rewarding. Organized into three sections, it starts by describing the events leading up to the Occupy movement, the experience of being a part of it, and why the author feels Occupy fell short of its objectives. The second section covers several examples of protests, from ancient history to modern times, and extracts the most important strategic insight from each event. Next, a unified theory of revolution is presented that reconciles the unexpected, the emotional, and the systematic aspects of large-scale change.

The third section speaks directly to innovation. Some of the book’s most powerful messages, the principles of revolution, are presented in Chapter 14. “Understanding the principles behind revolution,” this chapter begins, “allows for unending tactical innovation that shifts the paradigms of activism, creates new forms of protest, and gives the people a sudden power over their rulers.” If we consider that we are often “ruled” by the status quo, then these principles provide insight into how we can break free: short sprints, breaking patterns, emphasizing spirit, presenting constraints, breaking scripts, transposing known tactics to new environmental contexts, and proposing ideas from the edge. The end result is a masterful work that describes how to hear, and mobilize, the collective will.

Reviewed by

Dr. Nicole M. Radziwill

Course Materials for Statistics (The Easier Way) With R

Posted on January 3, 2016 by Nicole Radziwill One comment

Are you an instructor with a Spring 2016 intro to statistics course coming up… and yet you haven’t started preparing? If so, I have a potential solution for you to consider. The materials (lecture slides, in-class practice problems in R, exams, syllabus) go with my book and are about 85% compiled, but good enough to get a course started this week (I will be finishing the collection by mid-January).

There is also a 36MB .zip file if you would like to download the materials.

Whether you will be using them or just considering them, please fill in the Google Form at https://docs.google.com/forms/d/1Z7djuKHg1L4k7bTtktHXI7Juduad3fUW9G69bxN6jRA/viewform so I can keep track of everyone and provide you with updates. Also, I want to make sure that I’m providing the materials to INSTRUCTORS (and not students), so please use the email account from your institution when you sign up. Once I get your contact information, I will email you the link to the materials.

If you would like permission to edit the materials, I can do that as well — I know a couple of you have expressed that you would like to add to the collection (e.g translate them to another language). If you see any issues or errors, please either fix and/or email to tell me to fix!

Thanks for your interest and participation! Also, Happy New Year!

Simulation for Data Science With R

Posted on July 28, 2015 by Nicole Radziwill 8 comments

Image Credit: Doug Buckley of http://hyperactive.to

Hey everyone! I just wanted to give you the heads up on a book project that I’ve been working on (which should be available by Spring 2016). It’s all about using the R programming language to do simulation — which I think is one of the most powerful (and overlooked) tools in data science. Please feel free to email or write comments below if you have any suggestions for material you’d like to have included in it!

Originally, this project was supposed to be a secret… I’ve been working on it for about two years now, along with two other writing projects, and was approached in June by a traditional publishing company (who I won’t mention by name) who wanted to brainstorm with me about possibly publishing and distributing my next book. After we discussed the intersection of their wants and my needs, I prepared a full outline for them, and they came up with a work schedule and sent me a contract. While I was reading the contract, I got cold feet. It was the part about giving up “all moral rights” to my work, which sounds really frightening (and is not something I have to do under creative commons licensing, which I prefer). I shared the contract with a few colleagues and a lawyer, hoping that they’d say don’t worry… it sounds a lot worse than it really is. But the response I got was it sounds pretty much like it is.

While deliberating the past two weeks, I’ve been moving around a lot and haven’t been in touch with the publisher. I got an email this morning asking for my immediate decision on the matter (paraphrased, because there’s a legal disclaimer at the bottom of their emails that says “this information may be privileged” and I don’t want to violate any laws):

If we don’t hear from you, unfortunately we’ll be moving forward with this project. Do you still want to be on board?

The answer is YEAH – of COURSE I’m “on board” with my own project. But this really made me question the value of a traditional publisher over an indie publisher, or even self-publishing. And if they’re moving forward anyway, does that mean they take my outline (and supporting information about what I’m planning for each chapter) and just have someone else write to it? That doesn’t sound very nice. Since all the content on my blog is copyrighted by ME, I’m sharing the entire contents of what I sent to them on July 6th to establish the copyright on my outline in a public forum.

So if you see this chapter structure in someone ELSE’S book… you know what happened. The publisher came up with the idea for the main title (“Simulation for Data Science With R”) so I might publish under a different title that still has the words Simulation and R in them.

I may still publish with them, but I’ll make that decision after I have the full manuscript in place in a couple months. And after I have the chance to reflect more on what’s best for everyone. What do you think is the best route forward?

Simulation for Data Science With R

Effective Data-Driven Decision Making for Business Analysis by Nicole M. Radziwill

Audience

Simulation is an essential (yet often overlooked) tool in data science – an interdisciplinary approach to problem-solving that leverages computer science, statistics, and domain expertise. This easy-to-understand introductory text for new and intermediate-level programmers, data scientists, and business analysts surveys five different simulation techniques (Monte Carlo, Discrete Event Simulation, System Dynamics, Agent-Based Modeling, and Resampling). The book focuses on practical and illustrative examples using the R Statistical Software, presented within the context of structured methodologies for problem solving (such as DMAIC and DMADV) that will enable you to more easily use simulation to make effective data-driven decisions. Readers should have exposure to basic concepts in programming but can be new to the R Statistical Software.

Mission

This book helps its readers 1) formulate research questions that simulation can help solve, 2) choose an appropriate problem-solving methodology, 3) choose one or more simulation techniques to help solve that problem, 4) perform basic simulations using the R Statistical Software, and 5) present results and conclusions clearly and effectively.

Objectives and achievements

The reader will:

Learn about essential and foundational concepts in modeling and simulation
Determine whether a simulation project is also a data science project
Choose an appropriate problem-solving methodology for effective data-driven decision making
Select suitable simulation techniques to provide insights about a given problem
Build and interpret the results from basic simulations using the R Statistical Software

SECTION I: BASIC CONCEPTS

Introduction to Simulation for Data Science
Foundations for Decision-Making
SECRET NEW CHAPTER THAT YOU WILL BE REALLY EXCITED ABOUT

SECTION II: STOCHASTIC PROCESSES

Variation and Random Variable Generation
Distribution Fitting
Data Generating Processes

SECTION III: SIMULATION TECHNIQUES

Monte Carlo Simulation
Discrete Event Simulation
System Dynamics
Agent-Based Modeling
Resampling Methods
SECRET NEW CHAPTER THAT YOU WILL BE REALLY EXCITED ABOUT

SECTION IV: CASE STUDIES

Case Study 1: Possibly modeling opinion dynamics… specific example still TBD
Case Study 2: A Really Practical Application of Simulation (especially for women)

Chapter 1: Introduction to Simulation for Data Science – 35 pages

Description

This chapter explains the role of simulation in data science, and provides the context for understanding the differences between simulation techniques and their philosophical underpinnings.

Level

BASIC

Topics covered

Variation and Data-Driven Decision Making

What are Complex Systems?

What are Complex Dynamical Systems? What is systems thinking? Why is a systems perspective critical for data-driven decision making? Where do we encounter complex systems in business or day-to-day life?

What is Data Science?

A Taxonomy of Data Science. The Data Science Venn Diagram. What are the roles of modeling and simulation in data science projects? “Is it a Data Science Project?” — a Litmus Test. How modeling and simulation align with data science.

What is a Model?

Conceptual Models. Equations. Deterministic Models, Stochastic Models. Endogeneous and Exogenous Variables.

What is Simulation?

Types of Simulation: Static vs. Dynamic, Stochastic vs. Deterministic, Discrete vs. Continuous, Terminating and Non-Terminating (Steady State). Philosophical Principles: Holistic vs. Reductionist, Kadanoff’s Universality, Parsimony, Sensitivity to Initial Conditions

Why Use Simulation?

Simulation and Big Data

Choosing the Right Simulation Technique

Skills learned

The reader will be able to:

Distinguish a model from a simulation
Explain how simulation can provide a valuable perspective in data-driven decision making
Understand how simulation fits into the taxonomy of data science
Determine whether a simulation project is also a data science project
Determine which simulation technique to apply to various kinds of real-world problems

Chapter 2: Foundations for Decision Making – 25 pages

Description

In this chapter, the reader will learn how to plan and structure a simulation project to aid in the decision-making process as well as the presentation of results. The social context of data science will be explained, emphasizing the growing importance of collaborative data and information sharing.

Level

BASIC

Topics covered

The Social Context of Data Science

Ethics and Provenance. Data Curation. Replicability, Reproducibility, and Open Science. Open, interoperable frameworks for collaborative data and information sharing. Problem-Centric Habits of Mind.

Selecting Key Performance Indicators (KPIs)

Determining the Number of Replications

Methodologies for Simulation Projects

A General Problem-Solving Approach

DMAIC

DMADV

Root Cause Analysis (RCA)

PDSA

Verification and Validation Techniques

Output Analysis

Skills learned

The reader will be able to:

Plan a simulation study that is supported by effective and meaningful metadata
Select an appropriate methodology to guide the simulation project
Choose activities to ensure that verification and validation requirements are met
Construct confidence intervals for reporting simulation output

Chapter 3: Variability and Random Variate Generation – 25 pages

Description

Simulation is powerful because it provides a way to closely examine the random behavior in systems that arises due to interdependencies and variability. This requires being able to generate random numbers and random variates that come from populations with known statistical characteristics. This chapter describes how random numbers and random variates are generated, and shows how they are applied to perform simple simulations.

Level

MEDIUM

Topics covered

Variability in Stochastic Processes

Why Generate Random Variables?

Pseudorandom Number Generation

Linear Congruential Generators

Inverse Transformation Method

Using sample for Discrete Distributions

Is this Sequence Random? Tests for Randomness

Autocorrelation, Frequency, Runs Tests. Using the randtests package

Tests for homogeneity

Simple Simulations with Random Numbers

Skills learned

The reader will be able to:

Generate pseudorandom numbers that are uniformly distributed
Use random numbers to generate random variates from a target distribution
Perform simple simulations using streams of random numbers

Chapter 4: Data Generating Processes – 30 pages

Description

To execute a simulation, you must be able to generate random variates that represent the physical process you are trying to emulate. In this chapter, we cover several common statistical distributions that can be used to represent real physical processes, and explain which physical processes are often modeled using those distributions.

Level

MEDIUM

Topics covered

What is a Data Generating Process?

Continuous, Discrete, and Multivariate Distributions

Discrete Distributions

Binomial Distribution

Geometric Distribution

Hypergeometric Distribution

Poisson Distribution

Continuous Distributions

Exponential Distribution

F Distribution

Lognormal Distribution

Normal Distribution

Student’s t Distribution

Uniform Distribution

Weibull Distribution

Chi² Distribution

Stochastic Processes

Markov. Poisson. Gaussian, Bernoulli. Brownian Motion. Random Walk.

Stationary and Autoregressive Processes.

Skills learned

The reader will be able to:

Understand the characteristics of several common discrete and continuous data generating processes
Use those distributions to generate streams of random variates
Describe several common types of stochastic processes

Chapter 5: Distribution Fitting – 30 pages

Description

An effective simulation is driven by data generating processes that accurately reflect real physical populations. This chapter shows how to use a sample of data to determine which statistical distribution best represents the real population. The resulting distribution is used to generate random variates for the simulation.

Level

MEDIUM

Topics covered

Why is Distribution Fitting Essential?

Techniques for Distribution Fitting

Shapiro-Wilk Test for Normality

Anderson-Darling Test

Lillefors Test

Kolmogorov-Smirnov Test

Chi² Goodness of Fit Test

Other Goodness Of Fit Tests

Transforming Your Data

When There’s No Data, Use Interviews

Skills learned

The reader will be able to:

Use a sample of real data to determine which data generating process is required in a simulation
Transform data to find a more effective data generating process
Estimate appropriate distributions when samples of real data are not available

Chapter 6: Monte Carlo Simulation – 30 pages

Description

This chapter explains how to set up and execute simple Monte Carlo simulations, using data generating processes to represent random inputs.

Level

ADVANCED

Topics covered

Anatomy of a Monte Carlo Project

The Many Flavors of Monte Carlo

The Hit-or-Miss Method

Example: Estimating Pi

Monte Carlo Integration

Example: Numerical Integration of y = x²

Estimating Variables

Monte Carlo Confidence Intervals

Example: Projecting Profits

Sensitivity Analysis

Example: Projecting Variability of Profits

Example: Projecting Yield of a Process

Markov Chain Monte Carlo

Skills learned

The reader will be able to:

Plan and execute a Monte Carlo simulation in R
Construct confidence intervals using the Monte Carlo method
Determine the sensitivity of process outputs and interpret the results

Chapter 7: Discrete Event Simulation – 30 pages

Description

What is this chapter about?

Level

ADVANCED

Topics covered

Anatomy of a DES Project

Entities, Locations, Resources and Events

System Performance Metrics

Queuing Models and Kendall’s Notation

The Event Calendar

Manual Event Calendar Generation

Example: An M/M/1 system in R

Using the queueing package

Using the simmer package

Arrival-Counting Processes with the NHPoisson Package

Survival Analysis with the survival Package

Example: When Will the Bagels Run Out?

Skills learned

The reader will be able to:

Plan and execute discrete event simulation in R
Choose an appropriate model for a queueing problem
Manually generate an event calendar to verify simulation results
Use arrival counting processes for practical problem-solving
Execute a survival analysis in R and interpret the results

Chapter 8: System Dynamics – 30 pages

Description

This chapter presents system dynamics, a powerful technique for characterizing the effects of multiple nested feedback loops in a dynamical system. This technique helps uncover the large-scale patterns in a complex system where interdependencies and variation are critical.

Level

ADVANCED

Topics covered

Anatomy of a SD Project

The Law of Unintended Consequences and Policy Resistance

Introduction to Differential Equations

Causal Loop Diagrams (CLDs)

Stock and Flow Diagrams (SFDs)

Using the deSolve Package

Example: Lotka-Volterra Equations

Dynamic Archetypes

Linear Growth

Exponential Growth and Collapse

S-Shaped Growth

S-Shaped Growth with Overshoot

Overshoot and Collapse

Delays and Oscillations

Using the stellaR and simecol Packages

Skills learned

The reader will be able to:

Plan and execute a system dynamics project
Create causal loop diagrams and stock-and-flow diagrams
Set up simple systems of differential equations and solve them with deSolve in R
Predict the evolution of stocks using dynamic archetypes in CLDs
Convert STELLA models to R

Chapter 9: Agent-Based Modeling – 25 pages

Description

Agent-Based Modeling (ABM) provides a unique perspective on simulation, illuminating the emergent behavior of the whole system by simply characterizing the rules by which each participant in the system operates. This chapter provides an overview of ABM, compares and contrasts it with the other simulation techniques, and demonstrates how to set up a simulation using an ABM in R.

Level

ADVANCED

Topics covered

Anatomy of an ABM Project

Emergent Behavior

PAGE (Percepts, Actions, Goals, and Environment)

Turtles and Patches

Using the RNetLogo package

Skills learned

The reader will be able to:

Plan and execute an ABM project in R
Create specifications for the ABM using PAGE

Chapter 10: Resampling – 25 pages

Description

Resampling methods are related to Monte Carlo simulation, but serve a different purpose: to help us characterize a data generating process or make inferences about the population our data came from when all we have is a small sample. In this chapter, resampling methods (and some practical problems that use them) are explained.

Level

MEDIUM

Topics covered

Anatomy of an Resampling Project

Bootstrapping

Jackknifing

Permutation Tests

Skills learned

The reader will be able to:

Plan and execute a resampling project in R
Understand how to select and use a resampling technique for real data

Chapter 11: Comparing the Simulation Techniques – 15 pages

Description

In this chapter, the simulation techniques will be compared and contrasted in terms of their strengths, weaknesses, biases, and computational complexity.

Level

ADVANCED

Topics covered

TBD – at least two simulation approaches will be applied

Skills learned

The reader will learn how to:

Think about a simulation study holistically
Select an appropriate combination of techniques for a real simulation study

My New Favorite Statistics & Data Analysis Book Using R

Posted on April 20, 2015 by Nicole Radziwill 7 comments

NOTE: The 3rd Edition (Purple Swan) was released in April 2019. There is a companion book that presents end-to-end examples of each of the methods.

As of today, I now have a NEW FAVORITE introductory statistics textbook… the one I’ve always dreamed of having. I’ve been looking for a book to use in my classes for undergraduate sophomores and juniors, but none of the textbooks I considered over the past three years (and I’ve looked at over a hundred!) had all of the things I really, really wanted. So I had to go make it happen myself. These things are:

download the preview here (first ~100 pages)

1) An integrated treatment of theory and practice. All of my stats textbooks have a lot of formulas, and no information about how to do what the formulas do in the R statistical software. All of my R textbooks have a lot of information about how to run the commands, but not really much information about what formulas are being used. I wanted a book that would show how to solve problems analytically (using the equations), and then show how they’re done in R. If there were discrepancies between the stats textbook answers and the R answers, I wanted to know why. A lot of times, the developers of R packages use very sophisticated adjustments and corrections, which I only became aware of because my analytical solutions didn’t match the R output. At first, I thought I was wrong. But later, I realized I was right, and R was right: we were just doing different things. I wanted my students to know what was going on under the hood, and have an awareness of exactly which methods R was using at every moment.

2) An easy way to develop research questions for observational studies and organize the presentation of results. We always do small research projects in my classes, and in my opinion, this is the best way for students to get a strong grasp of the fundamental statistical concepts. But they always have the same questions: Which statistical test should I use? How should I phrase my research question? What should I include in my report? I wanted a book that made developing statistical research questions easy. In fact, I know a lot of people I went to PhD school with that would have loved to have this book while they were proposing, conducting, and defending their dissertations.

3) A confidence interval cookbook. This is probably one of the most important things I want my students to leave my class remembering: that from whatever sample you collect, you can construct a confidence interval that will give you an idea of what the true population parameter should be. You don’t even need to do a hypothesis test! but it can be difficult to remember which formula to use… so I wanted an easy reference where I’d be able to look things up, and find out really easily how to use R to construct those confidence intervals for me. Furthermore, some of the confidence intervals that everyone is taught in an introductory statistics course are wildly inaccurate – and statisticians know this. But they hesitate to scare away novice data analysts with long, scary looking equations, and so students keep learning those inaccurate methods and believing they’re good. Since so many people never get beyond introductory statistics and still turn into researchers in other fields, I thought this was horrible. I want to make sure my students know the best way to do each confidence interval in their first class… even if the equations are not as friendly.

4) An inference test cookbook. I wanted a book that stepped me through each of the primary parametric inference tests analytically (using the equations), and then showed me how it was done in R. If there were discrepancies, I wanted to know why. I wanted an easy way to remember the assumptions for each test, and when to use a pooled standard deviation versus an unpooled one. There’s a lot to keep track of! I wanted a reference that it would make it easy to keep track of all of it: assumptions, tests for assumptions, equations, R code, and diagnostic plots.

5) No step left behind. It’s really frustrating to me how so many R books assume you can do a psychic fill-in-the-blank for missing code. Since I’ve been using R for several years now, I’ve gotten to the point where my psychic abilities are pretty good, and at least 60% of the time I can figure out the missing pieces. But wow, what a waste of time! So I wanted a book that had all of the steps for each example. Even if it was a little repetitive. I may have missed this in a few places, but I think beginners will have a much easier time with this book. Also, I put all my data and functions on GitHub for people to run the examples with. I’m growing this slowly, but I don’t want people to be left in the lurch.

6) An easy way to produce any of the charts and graphs in the book. One of my pet peeves about R books is that the authors generate beautiful charts and graphs, and then you’re reading through the book and say “Yes!! Yes!! That’s the chart I need for my report… I want to do that… how did they do that?” and they don’t tell you anywhere how they did it. I did not want there to be any secrets in this book. If I generated a page of interesting looking simulated distributions, I wanted you to know how I did it (just in case you want to do it later).

GRANTED… I am sure it will not be perfect – no book is. (For example, Google Forms changes a lot and there are a couple examples that use it that will probably be outdated when the book gets to press… and I just found out this morning that you don’t need the source_https trick in R 3.2.0 and beyond.) [Note: data access has been fully updated in the 2nd Edition.] However, I will keep updating my blog with posts about useful things as they evolve.

In any case, I hope you enjoy my book as much as I’ve been enjoying using it as a reference for myself… it really is all my most important notes, neatly organized into just over 500 pages of everything I want to remember. And everything I want to make sure my students take with them after they leave my class.

[Note: Any errors and omissions from earlier printings (which have been taken care of in later printings) are being recorded at https://qualityandinnovation.wordpress.com/errata/.]