10 Semi-structured Interview

Abstract

A widely used qualitative research technique, the semi-structured interview consists of a verbal interaction solicited by the interviewer from a respondent, based on a grid of questions used in a very flexible manner. The interview aims both to collect information and to give an account of the person’s experience and view of the world, from a comprehensive perspective. It is useful for various types of public policy evaluations, including clarifying the objectives of a policy, analysing its implementation or studying its reception.

Keywords: Qualitative methods, semi-structured interview, induction, empathy, case study, ideal-type, realist evaluation

I. What does this method consist of?

The semi-structured interview is a data collection technique widely used in qualitative research in the social sciences. In very general terms, it is radically different from a questionnaire survey, which aims to produce standardised data on a vast population in order to search for regularities in the variation of opinions or attitudes between groups of individuals by statistical processing. The practice of interviewing, whatever its specific form, is used to produce data that allow us to better understand the singularity of the experience that individuals or groups of individuals have of their relations with others, with institutions, or more broadly of social phenomena. While the qualitative and in-depth study of a singular case may in itself give rise to knowledge with a certain degree of generalisation, this knowledge is usually derived from data processing using several case studies and ideal-types, as well as from cross-checking with data collected by means of the other two classic qualitative techniques, namely observation and the analysis of written sources. Qualitative techniques can also be used in mixed method research.

The practice of interviewing emerged in the nineteenth century in the field of clinical psychology and social enquiry for medical and political purposes respectively. It developed as a research technique in its own right during the 20th century in the United States and then in Europe in a comprehensive sociological approach in the wake of the work of Max Weber. The function of the interview is to gather the words of individuals, the general theoretical postulate being that social phenomena cannot be understood and therefore explained independently of the meaning that individuals give to their actions. On this common basis, several scientific interview practices have been progressively formalised, the main ones being the ethnographic interview, the non-directive interview and the semi-structured interview. However, it is the latter that has become the most widely used technique in policy analysis in recent decades, particularly in France (Pinson, Sala Pala, 2017). In this context, it is often used, if not as an exclusive mode of data collection, at least as a privileged one, on the grounds that it allows for the production of data with intrinsic value (and not only by cross-checking with observations or documentation).

Like other forms of social science interviewing, the semi-structured interview is a verbal interaction solicited by the interviewer from a respondent. However, in the case of the semi-structured interview, the interaction situation is special in that the respondent is initially placed in the role of informant, the holder of valuable (common, non-scientific) knowledge on the topic of interest to the interviewer.

Epistemologically, the semi-structured interview is part of a scientific mode of reasoning in which the fieldwork is not simply meant to verify pre-existing theories developed in the abstract, but rather the basis for developing the research question and hypotheses: the theory is produced by induction from the field data, according to the principle of grounded theory popularised by Anselm Strauss.

What is meant by a ‘semi-structured’ interview? Although the interviewer should prepare an organised grid of questions to guide the interview, the use of this grid is not rigid. The challenge is for the respondent to provide as much information as possible, both objective (on the phenomena, institutions or processes studied) and subjective (on his/her representations, value system, beliefs). It is therefore necessary to interact with the interviewee in such a way that he or she actively assumes the role of informant, in a conversational manner rather than a questionnaire administered “from above”. The quality of a semi-structured interview thus depends to a large extent on the interviewer’s attitude of empathy and attentive listening, which will enable him/her to make the most appropriate use of his/her grid of questions in the situation (Kaufmann, 2016).

The application of these methodological principles will never have the effect of cutting short the debates specific to the field of qualitative research and the different forms of interviewing, whether these debates concern the validity of the data collected (their degree of objectivity/subjectivity, their veracity/factuality, their partiality, etc.), or between scientific paradigms (constructivism/critical realism), so that there is no good use of semi-structured interviewing that is not reflected upon, methodically elaborated, and explained.

II. How is this method useful for policy evaluation?

Semi-structured interviews can be used to address three main types of evaluation questions. First, it can help to make the often complex set of initial objectives of a public policy understandable. Secondly, semi-structured interviews can be used in an evaluation process that aims to trace the processes of policy implementation, to understand how its objectives are concretely translated into the interventions and practices of administrative agents. Finally, although less recognised for this purpose in the French context, the semi-structured interview survey can contribute to producing evaluations by documenting the reception of a policy by its beneficiaries and, more broadly, by the individuals it targets. If these three uses can be combined in the same evaluative research, we will specify their respective contributions in turn.

From the perspective of clarifying the objectives of a policy, the semi-structured interview appears to be one of the rare means of empirically approaching the work of the government and, more precisely, the decision-making processes involved in putting public problems on the agenda and defining policies to deal with them. Because of their highly political nature, governmental spheres remain difficult to access for observation. Written sources, because of their official and consensual character, remain poor in terms of information for capturing debates and controversies between policymakers driven by ideologies, institutional logics and particular interests. The semi-structured interview is therefore used as a technique for retrospectively accessing first-hand information that is indispensable for deciphering the issues that presided over the formation of compromises and trade-offs that are only very implicitly expressed in the official formulation of policy objectives.

In an evaluation approach centred on the study of the means effectively deployed (outputs) in application of a policy, the use of the semi-structured interview appears at first sight to be less central. On the one hand, since the necessary data are by definition of a pronounced administrative and technical nature, they are often available in written form. Moreover, as the agents’ practices are considered more ordinary, they lend themselves more to observation, which can be a useful technique at this stage, in order to grasp the practices of adapting the rule to the diversity of situations and publics concerned (see separate chapter on direct observation). However, the semi-structured interview can be used as a complement to cross-check the explanatory hypotheses concerning the agents’ practices with the accounts they give of their work situations and the expert representations they develop about the publics they interact with.

The use of semi-structured interviews in the study of the effects (outcomes) of a policy is conceivable if we do not reduce this study to the only (quantitative) measurement of impacts but seek to understand (qualitatively) the process of producing these effects. This type of analysis, formalised in the 1990s by the pioneers of qualitative evaluation such as Michael Patton, emphasises that the same policy can have different meanings depending on the populations concerned, and that this diversity produces significant variation in its effects. The concept of reception (Revillard, 2019) helps to analyse the interactions between the logics of appropriation (cognitive and practical) and the effects (symbolic and material) of a policy. The empirical study of reception involves conducting semi-structured interviews, the particularity of which is to give primacy to the comprehension dimension rather than the information dimension, the examination focusing primarily on the subjectivity of the recipients. Another, less subjectivist, practice of semi-structured interviews is also developed in the realist evaluation. We present it in the next section.

III. An example of the use of this method in the evaluation of educational policies

Theorised by the sociologist Ray Pawson, realist evaluation is now well recognised in international scientific literature and is used by many governmental organisations (see separate chapter on realist evaluation). Its main characteristic is to replace the ordinary question “does this policy work? (in the sense of does it produce the intended effects?) with a more detailed questioning of “what effects does it produce? for whom? in what contexts? under what conditions? The (critical) realism of this approach lies in the postulate that measuring the impact of a policy is insufficient to grasp its effects, and that these are so different depending on the target audience and the context that it is essential, in order to evaluate it, to understand the variety of processes that it activates. Evaluating a policy is therefore a matter of formulating and empirically examining hypotheses about the way in which contexts, mechanisms and outcomes interact (the ”contexts-mechanisms-outcomes“ analysis scheme – CMO).

The work of formulating and examining hypotheses is based centrally on the conduct of semi-structured interviews designed according to a logic described as a teacher-learner function (Pawson, 1996), halfway between the structured and unstructured interview. The informational dimension of the interview is dominant, with the exchange with the interviewee focusing less on his or her experiences and representations than on a reflection on research hypotheses (theory-driven). This interview practice cannot, however, be described as directive insofar as, depending on the phase of the survey, the interviewer and the respondent will alternately play the roles of teacher and learner. In order to help anticipate and control this role switching, Ana Manzano (2016) distinguishes three phases in interview uses. The first set of interviews performs a theory gleaning function, i.e. it identifies provisional hypotheses from the actors about the effects of contextual circumstances on the functioning of the programme studied. In a second phase, certain theories are discarded and the selected theories are examined in greater detail by means of less standardised interviews in order to question the interviewees in a variety of ways with a view to refining the theory (theory refining). In the third phase of theory consolidation, the evaluator acts as a teacher by presenting his or her contextualised understanding of the programme to the respondent, to which the respondent can react by using examples in a logic of verification or falsification.

A recent example of an evaluation conducted in the field of educational policy illustrates this practice of semi-structured interviews particularly well. In order to evaluate the Colombian policy aimed at reducing regional inequalities in educational success by extending the length of the school day universally (Jordana Unica programme), Juan David Parra (2022) carried out a qualitative study consisting of 31 interviews (11 with officials from central and deconcentrated state services, 20 with school headmasters and educators), 20 focus groups (10 with parents, 10 with students) and 40 hours of non-participatory observations in schools. He also administered a questionnaire to a representative sample of school headmasters (N = 681). This survey enabled him to formulate, refine and then consolidate hypotheses on the implementation, reception and effects of this policy by emphasising the importance of reasoning at three levels: the decentralisation of educational policies, the well-being of children and adolescents, and the motivation of pupils.

IV. What are the criteria for judging the quality of the mobilisation of this method?

A first element conditioning the quality of a study based on semi-structured interviews concerns the number and choice of interviewees. As the representativeness of the sample is not a criterion of validity, the principle is rather to carry out a sufficient number of interviews (generally estimated at between 20 and 30) to gather the testimony of people who, from a formal or informal point of view, occupy different positions and are in different situations with regard to the object studied, so that they may have different points of view, in other words, varied experiences, practices and representations about it.

A second quality criterion is the way the interviews are conducted. The semi-structured interview must alternate between moments intended to collect narratives or stories freely produced by the respondent (generally at least at the beginning of the interview) and moments of greater directivity aimed at collecting information previously targeted by the interviewer. This art of interviewing is prepared beforehand by drawing up an interview guide, which evolves over the course of the research and can be adjusted according to the interviewees. This guide not only includes the formulation of initial instructions and general themes for discussion, but also establishes a series of follow-up questions that make it possible to obtain the information sought. Conducting interviews also depends on the posture the interviewer and respondent adopt in the situation and the follow-up techniques used by the interviewer.

A third set of issues lies in the processing of the data collected by the interviews. This decisive stage aims to analyse the content of the interviews in a cross-referenced and comparative manner so as not only to synthesise and cross-check the information collected, but also to produce an interpretation that is both global and detailed of the object studied, with reference to the theoretical framework and the research hypotheses initially formulated. This phase of the work requires the data collected in each interview to be relatively decontextualised by analysing their content in terms of the categories of analysis relating to the functioning of the action system and/or the processes studied and the experience of the various actors concerned.

V. What are the strengths and limitations of this method compared to others?

The main advantage of semi-structured interviews is that they provide essential data for understanding the processes by which a public policy produces its effects, from the genesis of the multiplicity of its objectives and its content (means devoted, instruments developed), to the actual methods of implementation and the various ways in which it is received. These data relate to the practices and representations of all the actors involved or more widely concerned (a priori) by the same policy. Depending on the research stages and the types of respondents solicited (decision-makers, implementers, beneficiaries, recipients), the use of the semi-structured interview can be modulated to activate its informative or comprehensive dimension first.

Its main limitations are twofold. Firstly, in the context of a strictly qualitative evaluation, it is required that the administration of proof operates by crossing the use of interviews with other data collection techniques, namely observation and the analysis of written sources. Secondly, as a qualitative method, it is clear that the use of the semi-structured interview does not in itself allow for the production of quantitative evaluations, evaluations which are otherwise very useful in providing contextual data for the design of the questioning of a qualitative evaluation.

Finally, it should be noted that in the current context of quantitative impact evaluation development, semi-structured interviews can find their place in the framework of research adopting a mixed methodology (Pin, Barone, 2021). Semi-structured interviews can thus contribute to the design (upstream) and interpretation (downstream) of a randomised experiment. In this case, as in others, the use of the interview will be modulated according to the research stages. The semi-structured interview technique will initially be used in a “qualitative instrumentalized” way to help identify the various contextual conditions of implementation of a programme whose impact we are trying to measure and thus refine its implementation methods. The semi-structured interview can then be used in an ’empowered qualitative’ logic to construct ideal-types that provide a posteriori explanatory elements of a qualitative nature to understand the causal processes that led to the measured impacts.

Some bibliographical references to go further

Kaufmann, Jean-Claude. 2016. L’entretien compréhensif. Armand Colin.

Manzano, Ana. 2016. “The craft of interviewing in realist evaluation”. Evaluation, n°22: 342-360.

Parra, Juan David. 2022. “Decentralisation and school-based management in Colombia: An exploration (using systems thinking) of the Full‐Day Schooling programme”. International Journal of Educational Development, n°91: 102579.

Pawson, Ray. 1996. “Theorizing the interview”. British Journal of Sociology, n°47: 295-314.

Pin, Clément. and Barone, Carlo. 2021. “L’apport des méthodes mixtes à l’évaluation”. Revue française de science politique, n°71: 391-412.

Pinson, Gilles. and Sala Pala, Valérie. 2007. “Peut-on vraiment se passer de l’entretien en sociologie de l’action publique?”. Revue française de science politique, n°57: 555-597.

Revillard, Anne. 2018. “Saisir les conséquences d’une politique à partir de ses ressortissants: la réception de l’action publique”. Revue française de science politique, n°68: 469-492.