Aljoscha Albrecht and Isabelle Werenfels

Measuring the Maghreb

What Do Rankings and Indexes Really Tell Us?

SWP Comment 2021/C 37, June 2021

Impressum

© Stiftung Wissenschaft und Politik, 2021

All rights reserved

This Comment reflects the authors’ views.

SWP Comments are subject to internal peer review, fact-checking and copy-editing. For further information on our quality control procedures, please visit the SWP website: https://www.swp-berlin.org/en/about-swp/ quality-management-for-swp-publications/

SWP

Stiftung Wissenschaft und Politik

German Institute for International and Security Affairs

Ludwigkirchplatz 3–4
10719 Berlin
Telephone +49 30 880 07-0
Fax +49 30 880 07-100
www.swp-berlin.org
swp@swp-berlin.org

ISSN (Print) 1861-1761

ISSN (Online) 2747-5107

doi: 10.18449/2021C37

Translation by Meredith Dale

(English version of SWP‑Aktuell 40/2021)

International rankings and indexes, such as the World Press Freedom Index published in mid-April 2021, are increasingly influential in the Maghreb. The region’s governments promote improvements in their own performance, protest where they score poorly, and celebrate outranking their rivals. Rankings also allow opposition groups and activists to spotlight grievances in their own country. External cooperation partners, above all the European Union (EU) and its member states, use them as a basis for decision making and policies towards Algeria, Morocco and Tunisia. While rankings and indexes insinuate objectivity and comparability, their sources, validity and utilisation are frequently problematic. Only if they are embedded in qualitative research on the Maghreb and their downsides reflected can they make a meaningful contribution to identifying reform needs and addressing deficits.

Measuring the Maghreb

What Do Rankings and Indexes Really Tell Us?

Aljoscha Albrecht and Isabelle Werenfels

When the latest World Press Freedom Index (PFI) came out in April 2021, it made headlines in the Maghreb and was picked up by prominent government critics. At the same time, the Maghreb states’ press agencies cite rankings and indexes to highlight their neighbours’ deficits: While the Moroccan state press agency points to the PFI to underline the lack of press freedom in Algeria, its Algerian counterpart prefers to publicise Algeria’s own position in the United Nations Development Programme’s (UNDP) Human Development Index (HDI).

The March 2021 Freedom in the World Index (FWI) created similar waves, as did the January 2021 Corruption Perceptions Index (CPI). The latter was eagerly awaited and immediately politically exploited, with the Tunisian press celebrating the country’s best positioning in a decade, the Moroccan and Algerian media each pointing to the other’s poor score.

Other, newer, indexes have also become touchstones. These include the Bertelsmann Transformation Index (BTI), the Democracy Index of the Economist Intelligence Unit (EIU) and the World Bank’s Ease of Doing Business Index. They play a growing role in foreign and security as well as economic and development policy, including in significant country analyses of the German Ministry for Economic Cooperation and Development (BMZ), in status reports of the European Commission on the European Neighbourhood Policy, for example on Algeria, and in mandatory Sustainability Impact Assessments, which the EU requires for new trade agreements, specifically with Morocco and Tunisia.

What all these rankings and indexes have in common is their claim to provide reliable objective assessments. Closer examination of these instruments and their application in the Maghreb context reveals how unrealistic that is in practice.

The Maghreb in Figures

The position of the three Maghreb states in the 35 rankings and indexes selected for this analysis largely mirrors their geographical location – between Europe and Sub-Saharan Africa, and thus between the extremes of the respective scales. In other words, the instruments reflect the inequality gradient between “the West”, the Maghreb states on its “threshold” and the “Global South”. Almost half the rankings that compare internationally place all three Maghreb states in the middle third. None have all three in the bottom third, while only the Global Militarisation Index (GMI) by the Bonn International Center for Conversion (BICC) and Arton Capital’s Passport Index see all three in the top third. The Mo Ibrahim Foundation’s Ibrahim Index of African Governance (IIAG) indicates that the Maghreb is ahead in the regional context, placing all three Maghreb states in the top third of its ranking of 54 African countries.

A direct comparison between the Maghreb states also reveals a clear pattern of strengths and weaknesses: Tunisia does best overall, Algeria worst, Morocco is in the middle.

Democratic Front Runner Tunisia

Tunisia comes out first of the three Maghreb states in more than half the examined rankings and indexes, scoring especially well in the areas of statehood and human and civil rights. It performs exceptionally well in indexes assessing political systems, freedoms and governance (see Table 1). This applies for example to the BTI, which covers “political transformation”, “economic transformation” and “governance”. Algeria and Morocco lag far behind. Tunisia comes fourth overall in the IIAG ranking of African states, behind only the island states of Mauritius, Cape Verde and the Seychelles. Its public sector also makes a relatively solid impression in the CPI.

Nevertheless, the democracy assessments do also reveal deficits. The BTI classifies Tunisia as a “defective democracy” with a “limited” economic transformation and “moderate” governance performance. The EIU’s Democracy Index confirms the first point, classifying the country as a “flawed democracy”. All the same, Tunisia rates as the only democracy in the Maghreb.

The human and civil rights situation is similar. In the FWI, which rates “political rights” and “civil liberties”, Tunisia scores 71/100; neither Morocco nor Algeria even manage 40 (see Table 1). Tunisia is thus the only Maghreb state that Freedom House judges “free” and in addition the only Arab state and one of only five African states. The FWI classes Morocco as “partly free” (but without Western Sahara), Algeria as “not free”. Tunisia is also well ahead of Morocco and Algeria in the PFI – despite having once occupied, with 164th place in 2010, the lowest position of any Maghreb state since the PFI was first created in 2002.

Tunisia also leads on visa-free travel and academic freedom. According to Henley & Partners Tunisians can travel visa-free to 72 countries, and Tunisia is the best-placed Maghreb country in Arton Capital’s Passport Index. It does especially well in the Academic Freedom Index (AFi) by Berlin-based Global Public Policy Institute (GPPi) where it is the only Maghreb state in the best of the five categories. This is also reflected in its top placing in the UNDP’s Global Knowledge Index (GKI). Tunisia also leads the Maghreb in the Digital Quality of Life Index (DQL Index).

The same cannot be said of security and conflict potential. To its credit Tunisia comes last among Maghreb states in both the GMI and the Fragile States Index (FSI) – meaning it is the least fragile and the least militarised. Less positively, it leads Algeria and Morocco in the Global Terrorism Index (GTI), which measures the “impacts of terrorism” in terms of incidents, fatalities, injuries and property damage. Tunisia also fares worse than Morocco on the Political Terror Scale (PTS) and in the Global Peace Index (GPI), which aggregates “domestic and international conflict”, “safety and security” and “militarisation”.

Table 1

Democracy, freedom, corruption

Index

Tunisia

Morocco

Algeria

Bertelsmann Transformation Index (BTI)

44th of 137

94th

83rd

EIU Democracy Index

54th of 165

96th

115th

Freedom in the World Index (FWI)

71/100 (“free”)

37/100 (“partly free”)

32/100 (“not free”)

Corruption Perceptions Index (CPI)

69th of 180

86th

104th

World Press Freedom Index (PFI)

73rd of 180

136th

146th

Sources: Bertelsmann Stiftung, BTI 2020; The Economist Intelligence Unit, Democracy Index 2020; Freedom House, FWI 2021; Transparency International, CPI 2020; Reporters sans frontières, PFI 2021.

Tunisia’s worst deficits are found in the economic sphere, where the free-market poster child Morocco sweeps the field.

Economic Champion Morocco

Morocco is close on Tunisia’s heels overall, placing first of the three Maghreb states in 14 of the 35 rankings and indexes. The kingdom has a clear lead in the economic indexes, coming 53rd of 190 countries in the World Bank’s prominent Ease of Doing Business Index, thus outclassing Tunisia by 25 places and Algeria by 104.

The positive assessment of Morocco as a place to do business is attributable above all to the indicators “dealing with construction permits” (16th), “paying taxes” (24th) und “getting electricity” (34th). Morocco’s competitiveness is further underlined by the World Economic Forum’s Global Competitiveness Index (GCI), where it lies 12 places ahead of Tunisia and 14 ahead of Algeria (see Table 2). Morocco also holds a clear lead on budgetary transparency, economic freedom and connectivity: In the Open Budget Index (OBI) it holds 62nd place, 20 ahead of Tunisia and 50 ahead of Algeria. The gaps are even bigger in the Heritage Foundation’s Index of Economic Freedom and Brand Finance’s Global Soft Power Index. Morocco is also best of the three in ETH Zürich’s KOF Globalisation Index, in which the “economic globalisation” component accounts for one-third.

Morocco leads on ecological sustainability, taking seventh place of the 57 countries ranked in the Climate Change Performance Index (CCPI). In reality that means fourth because its creator, the environmental NGO Germanwatch, demonstratively leaves the first three places unfilled. In contrast Algeria’s extensive oil and gas industry leaves it at the bottom of the list (43rd). Morocco also does passably in Dual Citizen’s Global Green Economy Index (GGEI), coming 59th of 130.

In Yale University’s renowned Environmental Performance Index (EPI) it scores considerably less well. Here Morocco comes 100th of 180 countries, behind Tunisia (71st) and even Algeria (84th). The main reason for this is that the index also includes the development-related category “environmental health”, in which Algeria is 61st overall and best in the Maghreb. And although Morocco leads the Maghreb in the economic indexes, the BTI includes a pointer suggesting that its lead is not unassailable: The Bertelsmann Stiftung regards Morocco’s “economic transformation” as “limited”, putting it 62nd, 17 places behind Tunisia.

Laggard Algeria

Table 2

Economics and climate

Index

Tunisia

Morocco

Algeria

Ease of Doing Business Index

78th

53rd of 190

157th

Global Competitiveness Index (GCI)

87th

75rd of 141

89th

Index of Economic Freedom

119th

81st of 178

162nd

Climate Change Performance Index (CCPI)

n/a

7th of 57

43rd

Sources: World Bank, Ease of Doing Business Index 2020; World Economic Forum, GCI 2019; Heritage Foundation, Index of Economic Freedom 2021; Germanwatch, CCPI 2021.

Overall Algeria lags far behind; in just three of the 35 investigated rankings and indexes does it come ahead of Tunisia and Morocco, namely the GMI, the Human Capital Index (HCI) and the HDI. Especially noteworthy are Algeria’s good showings in the HCI, introduced by the World Bank in 2018 to assess how well countries are harnessing the economic potential of their citizens, and in the UNDP’s HDI. Algeria’s performance in the HCI is not exactly spectacular – 98th of 174 – but still ahead of the other two Maghreb states, particularly in the education components: on average Algerians attend school for more than a year longer than their Tunisian and Moroccan peers.

Where the Maghreb’s largest state really stands out is the HDI (see Table 3), where it comes third in Africa (after Mauritius and the Seychelles). Since 2010 the HDI comprises “life expectancy at birth”, “expected” and “mean years of schooling” and “gross national income (GNI) per capita”. Algeria scores better than the other two in three indicators, but the one that stands out most is per capita GNI. This is especially significant because the Cost of Living Index attributes Algeria a lower cost of living than Morocco, which Numbeo calculates as having the highest in the Maghreb.

However, Algerian women appear to profit less than their male compatriots from the country’s development successes. Both UNDP gender indexes reveal significant deficits in Algerian gender parity. Algeria receives the worst possible rating in the Gender Development Index (GDI) and places 103rd of 162 in the Gender Inequality Index (GII). That said, neither Tunisia still less Morocco do especially well in this area either.

Despite Algeria’s decent showing in certain development-related areas, there is no denying that it comes last of the three Maghreb states in more than two-thirds of the quantitative instruments overall. And it has most catching up to do in the fields where Tunisia and Morocco do best: statehood, human and civil rights, and economic and technological development. Algeria is the weakest of the Maghreb states in the DQL Index (and second-last of all the 85 included countries). The same also applies to the EF English Proficiency Index.

Finally, Algeria also gives cause for concern in the sphere of security and conflict potential. The FSI rates it as the most fragile Maghreb state (and 71st of 178 overall), the GPI as the least peaceful (117th of 163) and the GMI as by far the most militarised (see Table 3).

The Downsides: Plausibility Problems of Rankings and Indexes

At first glance these rankings and indexes might appear to create a largely plausible profile of the Maghreb states and their relative international and regional status that tallies with qualitative observations. Closer examination reveals the downsides: These revolve around questions of consistency between instruments and compatibility with the findings of qualitative research, spanning the entire process from methodological issues to potential uses and general validity.

Methodological Question Marks

Table 3

Development and militarisation

Index

Tunisia

Morocco

Algeria

Human Development Index (HDI)

95th

121st

91st of 189

Human Capital Index (HCI)

102nd

110th

98th of 174

Global Militarisation Index (GMI)

47th

23rd

15th of 151

Sources: UNDP, HDI 2020; World Bank, HCI 2020; Bonn International Center for Conversion, GMI 2019.

Comparison of different indexes and rankings operating in the same or very similar areas quickly raises multiple questions concerning their design. Take for example the area of development: Algeria leads the Maghreb in the HDI on the basis of education and a rentier state based on oil and gas wealth. What the HDI fails to pick up is the development realities on the ground and especially the opportunities associated – in an age of global connectivity and digitalisation – more with English language skills and access to high-speed internet than with traditional schooling. Those are areas where Morocco and especially Tunisia do better. Although Algeria leads in the HDI, Tunisia does considerably better in five other development-related indexes (and Morocco in four): As well as the two UNDP gender indexes (excepting Morocco in the GII) this also applies to the Global Hunger Index (GHI), the GKI and the DQL Index.

Even greater contradictions appear among the rankings of political freedom and classifications of political systems. The BTI classifies Algeria as a “moderate autocracy”, Morocco considerably less favourably as a “hard-line autocracy”. The EIU’s Democracy Index on the other hand classes Morocco as a “hybrid regime”, Algeria as an “authoritarian regime”. The FWI also regards Morocco (“partly free”) as much “freer” than Algeria (“not free”). One obvious explanation for such discrepancies would be different underlying understandings of democracy. But in fact both the Bertelsmann Stiftung and the EIU base their work on five broadly congruent criteria.

This would suggest that even minor deviations in individual criteria, their operationalisation, standardisation and weighting, can cause wide divergences or even contradictions in the classifications. In effect, the instruments selected by European and other external cooperation partners ultimately decide how a state is judged. With regard to cooperation projects – whether economic or developmental in nature – it certainly does make a difference whether a state is regarded as an “authoritarian regime” or a “moderate autocracy” (example Algeria). In Morocco’s case the spectrum is especially broad, with partners able to choose between a “hybrid regime” and a “hard-line autocracy” as the basis for developing policies, making decisions and tailoring justifications. This illustrates the danger inherent to quantitative instruments: They insinuate comprehensive objectivity and precise measurement. It tends to get forgotten that their subjectivity begins with those who design them, and that judgements are already being made in the process of designing the instruments.

And if the indicators are then applied selectively or instrumentalised, the result is frequently a simplistic and distorted image of a considerably more complex reality – and in the worst case policies based on false assumptions. For example Morocco’s relatively uncritical elevation to a “darling” of international donors and investors over the past decade, alongside Tunisia, is likely attributable in part to its improving rankings. The danger here is that governments may tend to design reforms to optimise their rankings rather than enhancing their actual performance. Comparisons over time should also be treated with caution because improvements in rankings do not necessarily indicate progress. They can result simply from deterioration elsewhere, and vice versa of course.

Further problems include the difficulty of data gathering in authoritarian states, and outdated data or estimates underlying certain indexes; these include the UNDP’s Multidimensional Poverty Index (MPI) and Gender Social Norms Index (GSNI).

Flawed Comparisons and Blind Spots

A comparison with the qualitative literature introduces another set of problems. For instance, political freedom indexes tend to obscure the nature of the underlying systems. The examples of Morocco and Algeria illustrate the difficulties of placing countries with very different political systems on a single scale. One is ruled by a monarch with a firm hand and almost absolute constitutional power. The other is a highly opaque system controlled by competing actors primarily from the military, operating behind the scenes via clientelistic networks.

Both systems are equally problematic for political activists. The Algerian possibly more so, as restricted access for international human rights organisations can result in information deficits for example on torture. So the question of which system deserves to be labelled as more “authoritarian” is debatable. Such judgements are potentially crucial for the Maghreb governments, because of their repercussions for international agreements and cooperation. The aforementioned analyses and reports on which the German government and the European Commission base their decisions fail to address this aspect.

Ultimately, the comparability of such structurally very different economies is very limited. How to meaningfully compare a rentier economy like Algeria with diversified systems like Tunisia or Morocco? Morocco’s economy is strongly regulated – one could say dominated – by the royal family, and is more attractive to foreign investors, especially from Europe. But the royal family’s economic monopoly, as well as the corruption in public administration, are detrimental to the domestic private sector with less good connections to the royal court. In other words, the local perspective on Morocco’s good showing in the Ease of Doing Business Index will be different and more sceptical. This is underlined by a comparison with other business indexes: Morocco lags far behind Tunisia in the BTI component “economic transformation”, which unlike the World Bank index includes criteria like “social security and compensation”, “equality of opportunity” and “sustainability”.

Concentrating exclusively on rankings risks oversimplification, because many of the Maghreb states’ profound problems are difficult or impossible to grasp quantitatively. For example, the individual indicators that form the instruments cannot comprehend phenomena such as informality and political parallel structures (the “deep state”). That is simply outside the scope of the quantitative approach. Even if the CPI does represent an effort to address these issues, it is still quantifying corruption as perceived by a section of the business elites rather than the experience across society. Nor do the bare numbers reveal much about the quality and structures of corruption. Consequently, rankings and indexes supply little or no meaningful basis for determining what kind of reforms would be needed. The same can be said about the GMI. It measures a country’s militarisation but says nothing about actual or intended use of armed force, nor about the relationship between military and society. Is a country militarising because the monarch wishes it to (Morocco), or has the military long been operating outside of (or even controlling) politics (Algeria)? The indexes reveal little or nothing about the actors, whose qualitative nature largely precludes quantitative analysis.

No less problematic is the loss of local detail. The blind spots include intra-national differences in poverty and unemployment – for example in Morocco between the economic centres of Casablanca and Rabat and the marginalised Rif, or in Tunisia between the affluent Tunis suburb of La Marsa and disadvantaged regions like Tataouine and Jendouba. It is a real question whether it would not be more productive to gather detailed data on regions and actors – for example on the performance of political parties, ministries and political functions such as head of state – rather painting a whole country with the same broad brush.

Furthermore, in almost all cases (CPI apart) it remains unclear whether the country’s citizens share the assessment. According to surveys from 2020 the authoritarian regimes of Morocco and Algeria enjoy a great deal more public confidence than Tunisia’s democratically elected government.

Added Value and Recommendations

The weaknesses illustrated by the example of the Maghreb states demonstrate the ambivalence of rankings and indexes. Unreflected application can lead to problematic assessments and decisions. But rankings and indexes can contribute to identifying reform needs and rectifying deficits, from which both the Maghreb states themselves and their external cooperation partners stand to benefit. This makes it all the more important that the German government and the European Commission are aware of the weaknesses of the instruments they use. Otherwise the risk is that they will, as Alexander Cooley and Jack Snyder put it, “dumb down global governance”.

The following points should be observed: Generating a comprehensive understanding requires holistic and prudent consideration of different rankings and indexes and awareness of their contradictions. Even then they certainly cannot substitute the profound knowledge of the local context that is a precondition for identifying omissions and distortions, avoiding misconceptions and identifying the real causes of grievances.

The extent to which it is useful or legitimate to compare states with fundamentally different priors is also an issue policy makers should carefully scrutinise in each case. If we are to draw conclusions about the validity of rankings and indexes we must also factor in their origins and composition. These include the selected criteria, their operationalisation and weighting, the data sources and the collection methods. Not least, it is worth taking a look at the funding, interests and worldviews of the institutions producing them.

Aljoscha Albrecht is Research Assistant in the Middle East and Africa Division at SWP. Dr Isabelle Werenfels is Senior Fellow in the Middle East and Africa Division at SWP.