12 Group Interviews

Abstract

The group interview^[1] is a qualitative method through which semi-structured interviews are conducted with several people at the same time. This method intends to artificially recreating a set of social interactions between a selected number of participants, for example different policy stakeholders. It is useful to various forms of policy evaluation, such as ex ante, ex post and process evaluation.

Keywords: Qualitative methods, interview, group interview, elites, plurality, constructivism, interpretivism

I. What does this method consist of?

Interviews conducted with several actors (or stakeholders) at the same time refer to a diverse set of well-known qualitative methods in social science research. Their specific use depends on the role and function they hold in a research strategy (Knott et al., 2022), as well as their properties (Duchesne, Haegel 2008). Among them, group interviews are of particular relevance for policy evaluation research. They are not to be confused with other techniques such as group discussions, focus groups and pre-tests, mainly because they do not require being tethered unto a common experience, nor for participants to share homogeneous professional and social statuses (Marier et al., 2020). In artificially creating a set of social interactions between a selected number of participants, they differ from ethnographic methods, including observations.

Group interviews are understood as a technique used in public policy research, to launch an informal group discussion with a small group of knowledgeable stakeholders and experts – also referred to as “elites” (Glas 2021) – whose contribution is thought relevant for the understanding of the issue under study, including the evaluation of a public policy programme. The added value of group interviews does not lie in the time saved by interviewing several people at the same time – this view is erroneous as group interviews require considerable preparatory work and data processing than a series of one-to-one interviews (see separate chapter on semi-structured interviews) – but in providing the opportunity to artificially generate social interactions among a diversity of stakeholders. It helps identify and make sense of a plurality of perspectives, interests, and values, as well as shedding light on contradictions and ambiguities. Following Frey and Fontana (1991, 183), group interviews « take advantage of group dynamics to produce new and additional data. In addition to the respondent-interviewer relationship, the evolving relations among group members can be a stimulus to elaboration and expression. »

Group interviews may play a decisive role in qualitative research designs in different ways. First, when introduced in an exploratory perspective in the earliest stage of the research, group interviews are particularly useful in the case of a little-studied subject, for which the sources are scarce and insufficiently diversified. Second, by drawing on a “group effect”, group interactions may foster insightful perspectives on a given topic that would have remained hidden in observations or one-to-one interviews. As such, group interviews provide an opportunity to artificially generate a set of social interactions to express shared views or disagreements on a given topic (Morgan 1997), while leaving the possibility for additional one-to-one interviews with a selected number of participants. For those practitioners at the very top of their organisational structure, joining a group discussion constitutes a decisive factor for making time for the interview (Glas 2021). Third, during the earliest stage of research, they can be used to examine the robustness of the set of hypotheses stemming from the literature review and to refine them accordingly. Group interviews are of relevance in the context of a comparative research framework, with the same interview guide being applied across the cases under study to provide a first general comparative overview and generate some hypotheses on a case-by-case basis.

Furthermore, in selecting 8-12 participants, the organiser aims at bringing together a set of knowledgeable stakeholders and experts, representing a diversity of views on the object under study due to their respective background, roles and functions in their own organisations. Diversity may vary according to the policy context and the research question. It may refer to different training and professional backgrounds to ensure some cross-disciplinary discussions, to different roles and functions^[2] to allow for a variety of concerns, contexts, and priorities to be addressed, or to reflect the large range of organisations and institutions that characterises this policy context. In case studies that cover a longer period of 40 to 50 years, diversity may refer to different generations of stakeholders and experts.

Depending on the evaluative research question, data availability and whether the data is collected in the same language, this qualitative dataset can be coded for analysis using a qualitative analysis software such as InVivo (see also Knott et al., 2022). It can thus be used for text or discourse analysis, but also to produce a stakeholders’ mapping or a policy timeline, providing a strong basis for further developing the dataset and deepening the analysis through more targeted evaluative questioning.

II. How is this method useful for policy evaluation?

The extent to which group interviews may play a decisive role in a qualitative research design has already been addressed. In the context of public policy evaluation, it offers an opportunity to re-examine the boundaries of well-known policy problems as well as causal relations (Zittoun et al., 2021). Drawing on the constructivist-interpretative school of thought, this method takes a critical view on rationalist premises and highlights the constraints resulting from the various factors that may complicate evaluation activities (Wollman, 2006). It acknowledges that policy goals (as intended consequences) are often vague, ambiguous, potentially contradictory, or mutually exclusive. Public policy goals are understood in different ways by key policy actors, let alone by stakeholders, and while not necessarily accurate, these various understandings of policy problems and solutions are nonetheless fed back into the policy process, influencing its direction and future developments. This raises significant causality problems, more so for policy issues that are characterised by complexity and uncertainty, and at a time of crisis (Voss and Kemp, 2006). Based on these observations, group interviews seek to artificially generate a set of social interactions to critically examine causal relations between expected or observed changes and a given policy programme or measure.

Having this in mind, group interviews are useful to a variety of evaluative questions, such as ex ante, ex post and process evaluation, whether in combination with other evaluation methods or as a stand-alone. When it comes to ex ante evaluations, it can be drawn upon as an opportunity to examine (more or less) explicit causal relations between stated goals, the proposed selection of means, as well as expected results (see separate chapter on theory-based evaluation). In addition, it helps make sense of how alternative policy options are debated, what worldviews and arguments are being used, and what risk mitigation strategies are being developed to overcome expected resistances. This may, in turn, feed into decision-making and shed light on existing contradictions and ambiguities. Group interviews have also been particularly useful to feed into process evaluation, also in an accompanying (running in parallel) or an intervening mode. In this case, its function is to identify and make sense of interim effects while implementation is underway. Lastly, in the case of ex post evaluations, whether focusing on methods or findings, group interviews shed a complementary light to targeted evaluative questioning, often helping to make sense of potential disconnects between stated policy goals and their un(intended) effects, to discuss the use of a given set of indicators, and to spark a debate about future policy programmes. This, in turn, may contribute to examining learning processes, either as an object of research or of intervention.

III. An example of the use of this method

Group interviews have been used in a diversity of public policy research contexts, including evaluative questions. In the background of the climate crisis, it opens new avenues for the evaluation of transition and adaptation policies. As policies aim at achieving long-term goals, transition and adaptation policies refer to the change from the possible to the desirable, and progress is assessed in relation to policy futures that are not unequivocal. By contrast to technically clear problems, transition policy problems do not draw on a clear definition or solution, they are characterised by uncertain causal-effects relationships, and they bring together a wide range of stakeholders with conflicting values or interests, thus accounting for constant disagreements over the means to address the problems (Van der Steen et al. 2016). This fosters the need to draw on evaluative research designs in which degrees of divergence in values are purposefully examined and debated (Delahais et al., 2020).

Focusing on sustainable mobility transitions, Hickman and Banister (2014) examined the extent to which the future constitutes a challenge for policymakers, as well as the shortcomings of dominant methods as identified in the literature, such as forecasting and modeling in particular, or classic approaches used in scenario analysis. Reflecting on the work achieved together under the Urban Buzz Project^[3], they account for how a backcasting approach to transport planning in London was set up with the explicit goal to assess the existing strategy’s carbon efficiency and contribute to the development of a new strategy aimed at a 60% reduction in transport emissions by 2025 and 2050. The research design drew on a combination of methods, including group interviews, which took the format of workshops with policy-makers and, alternatively, with policy makers and stakeholders, at each stage of the process. The research design explicitly sought to bring the role of values back in the analytical framework, to assess the diversity of representations about transition futures, the hierarchy of values associated with transition processes, the range of implementation strategies at hand and the extent to which such choices were debatable. By contributing to the development of a backcast scenario closely articulated with an implementation pathway, the project confirmed the relevance of examining stakeholders’ values to address transport futures and fed into a changed approach to mobility in London.

IV. What are the criteria for judging the quality of the mobilisation of this method?

The simultaneous interview of stakeholders is not necessarily a timesaving research strategy. The logistics require a considerable amount of preparatory work and data analysis (Duchesne, Hagel, 2008). Being exploratory in nature, group interviews are, indeed, grounded in extensive preliminary research, such as a literature review, an assessment of data availability – grey literature, public reports, press clippings, political party manifestos, etc. – and a mapping of main stakeholders. This feeds into the production of an interview guide, which contributes to structuring the discussion while at the same time serving an exploratory purpose. It may include a small number of purposive questions to guide the discussion. In addition, small-group discussions may be encouraged through dedicated sequences, to produce a detailed and/or context specific understanding of working relationships across different organisations or to generate a precise understanding of a policy timeline, to be reflected on a paperboard.

The group interview organiser should also be aware that bringing together such a diverse group of stakeholders can be a perilous exercise, especially if the topic is contentious. While seeking to foster an informal and lively discussion, group interviews should take place in a formal framework. Also, participants may be reluctant to attend a group interview, fearing that it may only lead to a general and informal discussion. It is thus critical to clearly introduce it as a research method and to provide a (light) structure to avoid overly general and trivial discussions. While the discussion should not last more than 3-4 hours, accommodating time for a break will offer some opportunities for small talk. To avoid putting participants in a difficult position, participants must be informed in advance of the interview’s main features and the list of participants, and must provide their informed consent. Decisions about anonymity or confidentiality, data storage and dissemination, are to be addressed when asking the participants’ informed consent, whether in written or oral. Depending on the chosen approach for analysing the data, group interviews can be audio recorded and detailed notes can be taken during the discussion for the purpose of the research team. No public external to the interviews’ organisers and participants should be admitted.

Group interviews thus require important preparatory work to decide on the selection of participants, the interview guide and whether accommodating small group discussions might be useful to explore a specific issue into more depth.

V. What are the strengths and limitations of this method compared to others?

To conclude, group interviews present several advantages to policy evaluation research and practice. When used in an exploratory perspective, at the earliest stage of research, they help examine the robustness of the set of hypotheses resulting from the literature review, to provide a first general comparative overview and to generate context specific hypotheses. By artificially generating a set of social interactions or “group effect”, they provide an opportunity for participants to express shared views or disagreements on a given topic. As such, they are a powerful data collection technique, which provides a fresh look on a given topic that would have remained hidden in observations or one-to-one interviews.

By artificially generating a set of interactions, the “group effect” produces a highly original dataset, consisting of new information and evidence. By sharing their views and potential disagreements about a specific policy issue, its narrative, causal relations, and effects become debatable again, thus contributing to open new avenues for evaluative research or to inform policy making. Moreover, group interviews help generate a robust set of general and case-by-case assumptions, to question the relevance of external and internal drivers of change, to identify the effects of a given policy measure while at the same time taking into consideration wider policy considerations (and questioning its (unintended) effects).

Yet, they are ill adjusted for a targeted evaluative questioning. Other qualitative methods, such as focus groups would be better suited, mainly because group interviews do not require participants to share a common experience, homogeneous professional and social statuses. Also, group interviews seek to artificially create a set of social interactions between a selected number of participants in which they are encouraged to express their disagreements on a given topic, whether the diagnosis of the problem, the hierarchy of values to select a course for action, or its effects. As such, they also differ from ethnographic methods, including observations, and from one-to-one interviews.

Some bibliographical references to go further

Delahais, Thomas. and Sage, Kate. and Honoré, Vincent. 2020. Evaluators in Transition. Zeitschrift für Evaluation (ZfEv), 2: 239-260.

Duchesne, Sophie. and Haegel, Florence. 2008. L’enquête et ses méthodes: l’entretien collectif. Paris: Armand Colin.

Frey, James H.. and Fontana, Andrea. 1991. The group interview in social science research. Social Science Journal, 28(2): 175-187.

Glas, Aarie. 2021. Positionality, power and positions of power: reflexivity in elite interviewing. PS, Political science & politics, 54(3): 438-442.

Hickman, Robin. and Banister, David. 2014. Transport, climate change and the city. London: Routledge.

Knoll, Eleanor. and Hamid Rao, Aliya. and Summers, Kate. and Teeger, Chana. 2022. Interviews in the social sciences. Nat Rev Methods Primers, 2(73). https://doi.org/10.1038/s43586-022-00150-6.

Marier, Patrick. and Dickson, Daniel. and Dubé, Anne-Sophie. 2020. Using focus groups in comparative policy analysis. In: Peters, B. Guy. and Fontaine, Guillaume. Eds. Handbook of Research Methods and Applications in Comparative Policy Analysis. Cheltenham: Edward Elgar Publishing, 297-310.

Morgan, David L. 1997. Focus Groups as Qualitative Research, London: Sage.

Steen, Martijn van der. and Chin-A-Fat, Nancy. and Vink, Martinus. and Twist, Mark van. 2016. Puzzling, powering and perpetuating: Long-term decision-making by the Dutch Delta Committee. Futures, 76: 7-17.

Voß, Jan-Peter. and Kemp, René. 2006. Sustainability and reflexive governance: introduction. In: Voß, Jan-Peter. and Bauknecht, Dieter. and Kemp, René. Eds. Reflexive Governance for Sustainable Development. Cheltenham: Edward Elgar.

Wollmann, Hellmut. 2006. Evaluation and evaluation research. In Fischer, Frank, Miller, Gerald J., Sidney Mara S. Handbook of Public policy analysis. London: Routledge.