18 Macro comparisons

Abstract

Macro comparisons is an approach that exploits variation and similarities across large macro-social units of analysis (e.g. states, regions, provinces) to investigate different social phenomena. Studies can be undertaken at different scales and for various purposes, for example describing macro differences among different states, or evaluating the influence of a different welfare state structure on individual outcomes (such us levels of unemployment, life expectancy etc…).

Keywords: Mixed methods, macro-social units, variation, similarities, welfare state

I. What does this approach consist of?

All scientific inquiry is inherently comparative, and this is clearly observable when considering the logic applied to the most common methods in social sciences. To provide some examples: experiments are comparative because they need a control group to measure against a null case the effect of a treatment; regression analyses control for the effect of several variables comparing their effect on a range of cases. Hence, while all research methods are comparative in a broad sense, in the social sciences the idea of comparative inquiry often refers to research involving the use of large macro-social units of analysis (Ragin, 2014). Research in this sense is comparative when it exploits the variation or similarity of macro social units of analysis, e.g. a state, a region, a province^[1]. This can give way then to studies that are based upon different levels and scales, but all include the use of macro-units of analysis. The goal of these macro comparisons is to understand causal complexity and describe the relation between macro and micro units of analysis and between macro units of analysis among each other. The literature provides different examples, e.g. the comparisons between different social security models, or the evaluation of how a specific configuration of family policy impacts on female employment and fertility rates. The analysis of macrosocial units is a ‘meta-theoretical category’, which basically distinguishes comparative social scientists from the others, because they use ‘macrosocial units in explanatory (and descriptive) statements’ (Ragin, 2014: 5). Indeed, the vast majority of scholars working in the field (including the author of this chapter!), often do not define the nature and the role of the macrosocial units, but rather use them implicitly as ‘observations’ and/or ‘explanatory’ units of analysis (Ragin, 2014: 8).

Accordingly, the use of macro comparisons is more a way of thinking than a method stricto sensu. Macro comparisons can be set employing different techniques at the quantitative, qualitative and historical level, e.g. descriptive statistics, case studies and comparative historical analysis (CHA), qualitative comparative analysis (QCA)/fuzzy-sets, regression techniques, structural equation modelling (SEM) and factorial analyses, and cluster analysis. Other techniques used less frequently are diagonal reference models, sequence analysis, scale construction, thematic analysis, propensity score matching (PSM), optimal matching, Krippendorff’s alpha (KA) and event history analysis (for a systematic review of methods used in macro comparative research see Ferragina and Deeming 2022). This means that macro comparisons are not bounded to specific techniques, but rather need to be viewed as structuring ‘thinking about thinking’ (Sartori 1970) in order to increase the inference (the broader conclusions that may be drawn) we gauge from the study of specific cases.

II. How is this approach useful for policy evaluation?

Macro comparisons are extremely useful for the evaluation of public policy both ex ante and ex post. In particular macro comparisons have an important role in helping to contextualise the evidence provided by specific case studies or experimental evaluations of public policies. Key to advancing the debate about the relation between specific policies and their effects is the ability of comparative macro comparisons and national case studies to learn from each other (Ferragina 2020). National case studies – e.g. the evaluation of a specific policy within a country – are often plagued by a lack of external validity (the capacity to generalise the conclusions beyond the case under study). On the other hand, when using experiments scholars are able to test the effect of incremental reforms, but not the overall effect of a policy component on a specific outcome. So for example, in the field of family policy, macro comparisons can help to disentangle how the joint effect of explicit family policies differently (i.e. childcare, leave and child income support) impact on female employment across countries, while experiment can allow to disentangle the specific effect of an increase in the number of childcare facility on women’s’ employment elasticity in a specific case. For this reason, we need more studies that interact systematically with policy measures and the context in which they are implemented. In this sense macro comparisons can not only offer interesting insights about the effects of different policies cross-nationally or cross-regionally, but also allow us to critically evaluate the results from specific evaluations. Moreover, from an explanatory point of view, the existence of consolidated macro comparative evidence can help to interpret the results from studies run at the national level. This is the case of one of the most famous macro comparative works ever published, namely The Three Worlds of Welfare Capitalism by Gøsta Esping-Andersen (1990).

III. The three worlds of welfare capitalism: A famous example of how the comparative method can inform different types of policy evaluations

The Three Worlds of Welfare Capitalism is part of a long-standing academic tradition in sociology and political science rooted in deductive reasoning^[2] and the use of ideal types^[3]. As Max Weber (1904: 87) highlighted, ‘the construction of a system of abstract and therefore purely formal propositions …, is the only means of analysing and intellectually mastering the complexity of social life’. In this vein, Esping-Andersen (1990) constructed the welfare regime typology acknowledging the ideational importance and power of the three dominant political movements of the long 20th century in Western Europe and North America, that is, social democracy, Christian democracy (conservatism) and liberalism.

The ideal social-democratic welfare state is based on the principle of universalism, granting access to benefits and services based on citizenship. Such a welfare state is said to provide a relatively high degree of autonomy, limiting the reliance on family and market. In order to achieve autonomy, social-democratic welfare states are characterised by a high level of decommodification and a low degree of stratification. Social policies are perceived as ‘politics against the market’ (Esping-Andersen, 1985). Christian-democratic welfare states are based on the principle of subsidiarity and the dominance of social insurance schemes, offering a medium level of decommodification^[4] and a high degree of social stratification. The liberal regime is based on the notion of market dominance and private provision; ideally, the state only interferes to ameliorate poverty and provide for basic needs, largely on a means-tested basis. Hence, the decommodification potential of state benefits is low and social stratification high. However, these models are not pure and in each real national case different features are mixed. In this sense Esping-Andersen clearly shows how the comparative device is a way to classify and understand differences and clusters of countries, but needs to be considered with caution:

We show that welfare state clusters, but we must recognise that there is no single pure case. The Scandinavian countries may be predominantly social democratic, but they are not free of crucial Liberal elements. Neither are the Liberal regimes pure types. The American social-security system is redistributive, compulsory and far from actuarial. At least in its early formulation, the New Deal was as social democratic as was contemporary Scandinavian social democracy. And European conservative regimes have incorporated both Liberal and social democratic impulses. Over the decades, they have become less corporatist and less authoritarian (Esping-Andersen, 1990: 28-29).

Various contributions have confirmed his typology, while others have challenged, and expanded it, from substantive and methodological perspectives (see Ferragina and Seeleib-Kaiser 2011; Ferragina and Filetti 2022 for a discussion). However, despite this lengthy debate and important controversies in the literature, one cannot deny the fundamental role this work has assumed in the structuration and understanding of an important segment of public policy, namely social policy. In particular, The Three Worlds of Welfare Capitalism offers a plastic representation of the utility of macro comparisons and the framework developed by Esping-Andersen has been used as a departure point for thousands of studies (at the 26th of October 2022 the book has been cited 44086 times!).

Concerning public policy evaluation Esping-Andersen’s work has been used:

• To select different studies for analysis. The selection of at least one social democratic case, one Christian democratic case and one liberal case has allowed scholars to draw more insights from the study of a few countries.
• As a heuristic device to interpret the effects of different policies across countries.
• To understand and describe the different trajectories of countries over time.
• To contextualise the results obtained when comparing different countries.

IV. What are the strengths and limitations of this approach compared to others?

Macro comparisons are used to test hypotheses, infer causation, illustrate and gain in depth understanding of specific patterns, and interpret social change. They allow greater interpretative power in comparison to single case studies. This implies a strong heuristic power. It is not a random coincidence that highly cited works like the Three Worlds of Welfare Capitalism provided researchers in public policy with important insights on a large number of developed countries, which remain still valid more than 30 years after the publication of Esping-Andersen’s work. Macro comparisons allow us to pay attention to the context and the potential effects that this context might exert on specific outcomes. However, the fact of looking at ‘the forest’ instead of ‘the trees’ impose on the one hand high costs for the researcher (in terms of expertise about multiple cases), and on the other it requires a simplification of the analysis to accommodate the comparisons between different macro units of analysis. This can generate several issues, such as misclassification (the creation of pseudo-classes that incorrectly simplify the universe of cases analysed) and ‘conceptual stretching’, that is the erroneous application of theories and concepts to cases other than the ones that have been analysed.

Often scholars tend to include a lot of countries in their comparison by broadening the categories they have developed on the basis of direct knowledge acquired through few cases. However, this broadening can be problematic in many respects. On the one hand it is useful to have more countries in order to provide a better test of a series of hypotheses, but on the other, with fewer cases one can be more precise in the definition of concepts. This trade-off is not always considered in modern social sciences, with comparisons that end up over-stretching concepts. Therefore, concepts and insights extracted from macro comparisons need to be used with a grain of salt. As an approach more than a method, macro comparisons allow a critical approach to social sciences and historically raised important questions on the results obtained from researchers. In conclusion, macro comparisons are a double edge sword, they can inform in a meaningful way public policy evaluation, but they have also to be considered with caution.

Cited references

Andreski, Stanislav. 1965. The Uses of Comparative Sociology. Berkeley: University of California Press.

Armer, Michael. 1973. “Methodological problems and possibilities in comparative research”. In: Armer, Michael. and Grinmshaw, Qllen. (eds). Comparative Social Research. New York: Wiley, 49–79.

Esping-Andersen, Gosta. 1985. Politics against markets. Princeton: Princeton University Press.

Esping-Andersen, Gosta. 1990. The Three Worlds of Welfare Capitalism. Princeton: Princeton University Press.

Ferragina, Emanuele. 2020. Family policy and women’s employment outcomes in 45 high-income countries: A systematic qualitative review of 238 comparative and national studies. Social Policy & Administration, 54(7): 1016-1066.

Ferragina, Emanuele. and Deeming, Christopher. 2022. “Methodologies for comparative social policy analysis”. In: Yerkes Mara A. and Nelson, Keneth. and Nieuwenhuis, Rense (eds). Changing European Societies: The Role for Social Policy Research. Cheltenham: Edward Elgar, 218–235.

Ferragina, Emanuele. and Filetti, Federico Danilo. 2022. Labour market protection across space and time: a revised typology and a taxonomy of countries’ trajectories of change. Journal of European Social Policy, 32(2): 148–165.

Ferragina, Emanuele. and Seeleib-Kaiser, Martin. 2011. Welfare regime debate: past, present, futures? Policy & Politics, 39(4): 583–611.

Moore, Barrington. Jr. 1966. Social Origins of Dictatorship and Democracy. London: Penguin.

Przeworski, Adam. and Teune, Henry. 1970. The Logic of Comparative Social Enquiry. New York: Wiley.

Ragin, Charles. 2014. The Comparative Method. Moving beyond Qualitative and Quantitative Strategies. Oakland: University of California Press.

Rokkan, Stein. 1966. “Comparative cross-national research”. In: Merritt, Richard. and Rokkan, Stein. (eds). Comparing Nations. New Haven: Yale University Press, 3–26.

Sartori, Giovanni. 1970. Concept misformation in comparative politics. American Political Science Review, 64(4): 1033–1053.

Skocpol, Theda. 1979. States and social revolutions. Cambridge: Cambridge University Press.

Skocpol, Theda. 1992. Protecting soldiers and mothers: The political origins of social policy in the United States. Harvard: Harvard University Press.

Weber, Max. 1904. On the methodology of the social sciences. Glencoe: The Free Press.

Some bibliographical references to go further

Ferragina, Emanuele. and Deeming, Christopher. (Forthcoming). ‘Comparative mainstreaming? Mapping the uses of the comparative method in social policy, sociology and political science since the 1970s’. Journal of European Social Policy. An analysis of 50 years of comparative research based on a database including thousands of comparative articles from top journals in sociology, political science and sociology. The quantitative analysis of the main trends in the use of the comparative method is complemented with a qualitative analysis of the most cited articles in the comparative field.

Kohn, Melvin L.. 1987. ‘Cross-national research as an analytic strategy. American Sociological Association, 1987 presidential address.’ American sociological review, 52(6): 713-731. This presidential address of the American Sociological Association suggests that cross-national comparative research is an essential tool to generate, test, and develop sociological theory. The comparative method is costly and hard to apply, and it can also generate some interpretative problem. However, despite its limitations it is a fundamental tool of social science research.

Lijphart, Arend. 1971. Comparative politics and the comparative method. American Political Science Review, 65(3): 682-693. The article offers a systematic analysis of the comparative method. Its emphasis is on both the limitations of the method and the ways in which, despite these limitations, it can be used to maximum advantage. Lijphart focuses on the role of case studies (in their different forms) as the main way to undertake macro comparisons. In the article, he contrasts case-based comparisons to experimental and statistical methods.

Przeworski, Adam. and Teune, Henry. 1970. The Logic of Comparative Social Inquiry. New York: John Wiley and Sons. In this ground-breaking book the authors proposed insights and views about comparative research that have profoundly shaped political science research. The book focuses mostly on quantitative analysis. A must read for all students interested in the comparative method and what can be done with it.

Ragin, Charles. 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press. This book offers considerable insights for the understanding and use of comparative analysis. Originally written to present the utility of the Qualitative Comparative Analysis (QCA) in comparison to qualitative and quantitative techniques, it also provides theoretical and substantive reasons for the use of the comparative method and macro comparisons in the public policy field.