Here we go again…
Nobody promotes better the think tank sector among ‘non-think tank’ audience than Jim McGann! Kudos for his ability to spread the study and his results! The coverage is ubiquitous: from my friends in international organizations, to local NGOs, to some governmental officials, everyone has received the announcement for the launch of the study. And this is the good part… people who are not acquainted with the think tanks and those who have little contact with this type of organizations get to know about them – if nothing else as a checklist of ‘Who Won the Oscars in this world this year?’.
The value of this promotional effort notwithstanding, I cannot stay indifferent to the mistakes and persistent blind sports of these rankings. So, I am probably foolish to fall in the last year’s trap and comment on this year’s ranking :-). Anyway, here is what needs to get out of my chest :-).
I pick up the outdated definition of think tanks, the number of identified think tanks and a myriad of factual mistakes and inconsistencies in the rankings this year.
About the definition of think tanks
The definition is so outdated (bridging knowledge and power). Even the originators of this definition such as Diane Lesley Stone no longer subscribe to it. The world has moved on in the last 10-15 years and with it the understanding of the think tanks. Moreover, this definition worked well in developed countries where the academia is well developed and ‘produces knowledge’. In the developing world this assumption is far from being correct. Then, in many places the policy makers are no longer in the same locus. For example, in many policy areas throughout the member-states of the European Union, policy making has been divided between national governments and Brussels. if I try to interpret the definition in a somewhat funny way, to simply claim that now think tanks have to build two bridges, one at the national level and the second one at the EU is overly simplistic :-). One has to go at various definitions of think tanks today. One my earlier posts addresses this issue.
The number of think tanks included in the study
The sheer number of ‘think tanks’ identified in the study is surprising, to say the least. I am aware that McGann uses the broadest take to include everyone from government think tanks to political parties to consultancies to academy of sciences to private think tanks. Yet, knowing a bit about think tanks in Central and Eastern Europe, I am simply shocked at the numbers that the study operates with. It seems that every organization that has done at least one policy-relevant research study could now call itself a think tank. Any university-based research center addressing economic, social, political or international affairs could immediately ‘check in’ this ranking as a think tank. The fact that they publish only in academic journals and have not carried out an ‘a’ of an advocacy plan for any of their studies apparently does not matter. How otherwise could one explain that there are 47 think tanks in Ukraine (25th by numbers of think tanks in the world), or 39 in Hungary or 25 in Czech Republic? I went on to count Serbia one by one any NGO that has done more than 3 surveys/policy studies that I could think of, then any political party think tank to all ‘real’ think tanks, university based research centers, governmental institutes and still did not arrive to 24, number listed in this study :-). I could go on…
Some amusing parts:
Nominations for leading think tanks in the world: I amused to see that F.A. Hayek Foundation, (Slovakia) was nominated as one of the best world think tanks :-). This organization is struggling to regain its reputation and resume regular work after last year’s scandal that implied two of its founders in embezzlement of government funds.
I am equally surprised/amused that Open Society Institute (OSI), the organization that I work for listed in the rankings. Twice. OSI is not a think tank. Full stop. Even some of us want to see OSI moving in that direction. OSI is predominantly a donor.
European Stability Initiative (Turkey, they have offices in Berlin and representative in Brussels) is ranked as 12th in the ‘Top 25 Think Tanks in the Middle East and North Africa (MENA)’. With all the respect for their work on Turkey, but ESI is a think tank that is mainly working on EU affairs, EU accession, socio-economic development of Balkan countries (including Turkey; ESI’s work on the Turkey is at most 20-25% of their total work), South Caucasus, but nothing in MENA countries. This example illustrates the problem of ranking organized as ‘a beauty contest’. Who checked the votes and nominations and who decides that ESI’s work on Turkey and only Turkey suffices to be ranked under MENA countries.
Even more worrying parts:
I am really worried in the type of the organizations that are listed as think tanks in the list. Only in the top 5 of the Top50 Worldwide – non US (page 25) two are not think tanks: Amnesty International and Transparency International. While I am grateful to AI and TI for all the policy research and excellent advocacy they are doing regularly, their mandate and type of activities are much broader than those of think tanks. And here is the danger of really blurring the lines and confusing those leaders approaching this ranking lightly. One could go on and wonder about the ‘thinktank-ness’ of Human Rights Watch …
What if I nominate Coalition of Youth NGOs ‘Now’ – Macedonia as a leading think tank for next year? The Coalition regularly carries out research about the rends among the young population, published reports, commissions public opinion polls and carries out surveys. Does that make them a think tank when 80% of their work has nothing to do with policy research? Shall the body overseeing this ranking exclude my proposal? I think they have to
My point is that the team in charge of this ranking should have clear criteria to weed out those nominated organizations that are not think tank (even if they claim or are nominated by one or two ‘experts’).
Ignorance on the verge of turning into an insult
DEMOS – UK has really tried in the last two years (particularly last year) to distance itself from the Labour Party. It has diversified its Board, established strong links with the Lib-Dems, so much that the former director accepted a position in the current UK government. To rank DEMOS as a party affiliated think tank next to the German Stiftungs is just a sign of ignorance and failure to update the rankings from one year to the other (In case that the ‘experts’ of this ‘beauty contest’ perceive that DEMOS is a party affiliated think tank, then something is wrong with their own competence and update. To be blunt and maybe vulgar, if the competition is for the best blonde… and a blonde had dyed her hair to brunette last year, now it has to be judged in the category of brunettes. With apologies to DEMOS for this comparison, but their remodelling and repositioning was so obvious even to someone who follows them from a distance as I do.
And an embarrassment:
Is Razumkov Center Ukraine ranked 16th or 25th in the Top 25 Central and Eastern European Think Tanks? It is listed on both positions! The mean result suggests 20,5th! ;-).
If the quality control of the presentation of the rankings is so weak (I could imagine a typing mistake), how can one trust the analysis of the date derived from the complex questionnaire sent to the experts?
And even a bigger embarrassment
National Endowment for Democracy is a respected donor, not a think tank. One needs to check them on the internet. Any intern with a BA degree will recognize the difference between a donor and a think tank. Who vets the interns who help out the ranking? If OSI was a mistake that one could forgive, this one should not be.
….And let’s check the big picture:
The thematic rankings are even more dubious. What is a development think tank? Is this a think tank that deals with development aid, with development as defined by Wikipedia or… something else? How the ranking distinguishes between think tanks that analyze international relations (e.g. Brookings) from those that specialize on development aid ( e.g. Center for Global Development). Or if it compares what are the criteria to compare those?
Similarly, in the rankings for Best Think Tanks in Social Research, what are the criteria based on which USA based Brookings Institution is better than let’s say Center for Economic and Policy Research (CEPR) in the UK. The two think tanks address completely different issues and context (one in USA the other one in the UK)? Is this because Brookings has more publications? Or does it have a better design? Or methodologically Brookings is more advanced than the other think tanks? Or because it is more recognized (and if yes, recognized for what)? I do not want to mention impact J
To end, because I bored myself already, let alone you who have made it so far in my blog post :-). My favorite category is “The Greatest Impact on Public Policy”. I suggest the authors of this ranking to simply patent the way how they measured the impact and the differences between the organizations. Then, they should bottle it and start selling it to donors (especially those who support think tanks – I know plenty who would gladly pay for it), think tanks and other organizations that aim to influence public policy. The authors will retire filthy rich J
These bits reveal that year after year, the few of us who observe and critically comment this ranking struggle to go through the factual errors and basic inconsistencies in the various tables. Last year, in my comment to the 2009 edition, I focused on the conceptual problems with the methodology and improvements of the rankings altogether. This year, for those who care about ranking at all ( Not my cup of tea), I suggest alternatives.
Instead of conclusion:
I thank THE THINK TANKS AND CIVIL SOCIETIES PROGRAM, International Relations Program at the University of Pennsylvania for promoting think tanks in the world. However, apart from making more people sensitive of the world think tank (Let’s hope not oversensitive), this study has little added value.
As someone who works with think tanks, studies think tanks, writes about think tanks, I see very little value in it. Therefore, it is high time to move to alternatives to this study:
– Best advocacy campaign by a think tanks [consisted of a series of policy products (from op-ed to book), events (briefings, debates, seminars, conferences, training events etc.],
– Best online presentation
– Best design and communication strategy