Week 3: RCTs and Their Critics

Instructions

Critics of RCTs voice concerns about the ethics of ‘randomizing policies’ and raise doubts about the ability of RCTs to inform policy making. We look at both of these critics this week. After watching Nancy Cartwright’s 5-minute teaser video, read her paper with noble prize winner Angus Deaton. What is their main concern with RCTs? Next, read the paper by Dawn Langan Teele on the ethnics of RCTs. Are their concerns the same as those of Chris Pottas when he critized an RCT in Nairobi’s slums by Coville, Galiani, Gertler, and Yoshida?

Required readings

• Deaton, Angus and Cartwright, Nancy (2018). Understanding and Misunderstanding Randomized Controlled Trials. Social Science & Medicine, 210, pp. 2–21.
• Teele, Dawn Langan (2014). Reflections on the Ethics of Field Experiments. In Teele, Dawn Langan (Ed.), Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences. Yale University Press.
• Coville, Aidan and Galiani, Sebastian and Gertler, Paul and Yoshida, Susumu (2020). Enforcing Payment for Water and Sanitation Services in Nairobi's Slums. National Bureau of Economic Research, Working Paper No. 27569.
  • The authors. A Comment on the Ethical Issues Twitter Discussion in Enforcing Payment for Water and Sanitation Services in Nairobi's Slums.
  • Prottas, Chris. Commentary on Enforcing Payment for Water and Sanitation Services in Nairobi's Slums.

Further reading (Ethics)

• Phillips, Trisha (2021). Ethics of Field Experiments. Annual Review of Political Science, 24, pp. 277–300.
• Johnson, Jeremy B (2018). Protecting the Community: Lessons From the Montana Flyer Project. PS: Political Science & Politics, 51(3), pp. 615–619.
• Kramer, Adam DI and Guillory, Jamie E and Hancock, Jeffrey T (2014). Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks. Proceedings of the National Academy of Sciences, 111(24), pp. 8788–8790.
• Bond, Robert M. and Fariss, Christopher and Jones, Jason J. and Kramer, Adam D. I. and Marlow, Cameron and Settle, Jaime E. and Fowler, James H. (2012). A 61-Million-Person Experiment in Social Influence and Political Mobilization. Nature, 489, pp. 295–298.

Further reading (Generalizability)

• Cartwright, Nancy and Hardie, Jeremy (2012). Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford: Oxford University Press.
• Dunning, Thad and Grossman, Guy and Humphreys, Macartan and Hyde, Susan D and McIntosh, Craig and Nellis, Gareth (2019). Information, Accountability, and Cumulative Learning: Lessons from Metaketa I. Cambridge: Cambridge University Press.

Suggested media

• A Generation of Bad Blood, The Atlantic, June 17, 2016