Evaluative Research Methods

Managing the Complexities of Judgment in the Field

By:
Saville Kushner

A volume in the series: Evaluation and Society. Editor(s): Stewart I. Donaldson, Claremont Graduate University. Katrina L. Bledsoe, Education Development Center.

Published 2016

How do research students and their supervisors respond in a world of ‘fake news’, the destabilisation of public institutions and the rise of populism? The very foundations of our liberal democracies seem to be under threat, and this implicates social inquiry. Postgraduate research remains one of the few information spaces which are still free of politicisation and committed to validation.

This book focuses on democracy in inquiry, and on the role of inquiry in a democracy – how research helps us to deliberate over what counts as of public value. It is a research methods book, but methods shaped by political and ethical purposes, and by the challenge of making judgements about what, in the public sphere, is worthy. We may be looking at a police training program, the siting of a clean energy project, a new school curriculum, maternal health program or an environmental adaptation project – in each case and in others like them we have to negotiate perspectives and claims, forge and justify a consensus, support competing stakeholders with the best information and analyses possible.

And we have to make our work defensible – undeniable in the forum of public debate and exchange, examination and accountability. This book, full of examples from contemporary research projects, is designed to help navigate our way through the complexities of social research which focuses on judgements about public action.

The book was written with research students and includes examples of their work. It recognises that supervisors often struggle as much as students in meeting the challenges of inquiry that involves some element of evaluative judgement – inquiry that potentially carries consequences. Where there are no quick-and-ready recipes, check-lists or theoretical frameworks – where we confront the particularities of the context in which the research takes place, we are all forced back onto good methodological thinking, and this is the pedagogical framing of the book.

CONTENTS
Acknowledgements. Outline of the Book. PART I: INTRODUCTIONS CHAPTER 1: What is Evaluative Research and How Are You Positioned Within It? CHAPTER 2: Evaluative Enquiry and Democracy: Where It Came From and Where It Might Go. CHAPTER 3: Social Change and Evaluative Research. CHAPTER 4: Theorizing, Contingency, and Evaluative Cognition: Conceptual Tools to Help You Read the Book. PART II: THE CRAFT OF EVALUATIVE RESEARCH CHAPTER 5: The ‘Self’ and the ‘Person’: Democratic Individualism. CHAPTER 6: Evaluative Interviewing. CHAPTER 7: Observation-Based Interviewing, Triangulation and the Redistribution of Power. CHAPTER 8: On Bullfighting, the Fragile Self, and an Introduction to Evaluative Case Study. CHAPTER 9: Analyzing Organization as an Evaluative Case. CHAPTER 10: Writing Democratically, and Finding Your Own Voice. PART III: THE POLITICS OF EVALUATIVE RESEARCH: DANGERS IN THE FIELD CHAPTER 11: Quieting Reform: When Evaluative Research Can Be Dangerous, with Robert Stake and Acacia Cochise. CHAPTER 12: Politics and Negotiation in Evaluative Enquiry. CHAPTER 13: Evaluation Inside the Political System, With Barry Kushner. CHAPTER 14: Radical Manifestos for Evaluative Research. PART IV: CRITIQUES OF CURRENTLY POPULAR EVALUATION PRACTICES, THE WAY FORWARD CHAPTER 15: Democracy, Theory, and Logic in Evaluative Research: Beware of Rationalism. CHAPTER 16: Democracy and Method in International Development Evaluation. CHAPTER 17: Arriving Backwards: Evaluative Enquiry, Innovation, and Democracy. References.