Quality Metrics for Policy Evaluation?
The Center for Environmental Journalism (CEJ) recently posted an interview with Roger Pielke, Jr., an authority on (as CEJ calls it) “the nexus of science and technology in decision making”. The interview seeks to provide a perspective on how journalists can more accurately address climate change in the context of public policy over the next several years.
I was really intrigued by this part:
Reporters could help clarify understandings by asking climate scientists: “What behavior of the climate system over the next 5-10 years would cause you to question the IPCC consensus?” This would give people some metrics against which to evaluate future behavior as it evolves.
Similarly, you could ask partisans in the political debate “What science would cause you to change your political position on the issue?” This would allow people to judge how much dependence partisans put on science and what science would change their views. I would be surprised if many people would give a concrete answer to this!!
For the first question, Pielke is recommending is that we take an approach conceptually resembling statistical process control to help us figure out how to evaluate the magnitude and potential impacts of climate change. (Could we actually apply such techniques? It would be an interesting research question. Makes me think of studies like Khoo & Ariffin (2006), for example, who propose one method based on Shewhart x-bar charts to detect process shifts with a higher level of sensitivity – only tuned for a particular policy problem.) For the second question, I’m reminded of “willingness to pay” or “willingness to recommend” or other related marketing metrics. I’m sure that one of these established approaches could be extended to the policy domain (if it hasn’t been done already).