Mirror, mirror on the wall… tell me who is the best think tank in the world?

THE THINK TANKS AND CIVIL SOCIETIES PROGRAM

Here we go again…

THE GLOBAL “GO-TO THINK TANKS”2010 edition is out!

Nobody promotes better the think tank sector among ‘non-think tank’ audience than Jim McGann! Kudos for his ability to spread the study and his results! The coverage is ubiquitous: from my friends in international organizations, to local NGOs, to some governmental officials, everyone has received the announcement for the launch of the study. And this is the good part… people who are not acquainted with the think tanks and those who have little contact with this type of organizations get to know about them – if nothing else as a checklist of ‘Who Won the Oscars in this world this year?’.

The value of this promotional effort notwithstanding, I cannot stay indifferent to the mistakes and persistent blind sports of these rankings. So, I am probably foolish to fall in the last year’s trap and comment on this year’s ranking :-). Anyway, here is what needs to get out of my chest :-).

I pick up the outdated definition of think tanks, the number of identified think tanks and a myriad of factual mistakes and inconsistencies in the rankings this year.

About the definition of think tanks

The definition is so outdated (bridging knowledge and power). Even the originators of this definition such as Diane Lesley Stone no longer subscribe to it. The world has moved on in the last 10-15 years and with it the understanding of the think tanks. Moreover, this definition worked well in developed countries where the academia is well developed and ‘produces knowledge’. In the developing world this assumption is far from being correct. Then, in many places the policy makers are no longer in the same locus. For example, in many policy areas throughout the construction company
outpatient drug treatment
boombox
best wireless speakers
ankle injury
acl surgery recovery
construction management
alcohol withdrawal
member-states of the European Union, policy making has been divided between national governments and Brussels. if I try to interpret the definition in a somewhat funny way, to simply claim that now think tanks have to build two bridges, one at the national level and the second one at the EU is overly simplistic :-).  One has to go at various definitions of think tanks today. One my earlier posts addresses this issue.

The number of think tanks included in the study

The sheer number of ‘think tanks’ identified in the study is surprising, to say the least. I am aware that McGann uses the broadest take to include everyone from government think tanks to political parties to consultancies to academy of sciences to private think tanks. Yet, knowing a bit about think tanks in Central and Eastern Europe, I am simply shocked at the numbers that the study operates with. It seems that every organization that has done at least one policy-relevant research study could now call itself a think tank. Any university-based research center addressing economic, social, political or international affairs could immediately ‘check in’ this ranking as a think tank. The fact that they publish only in academic journals and have not carried out an ‘a’ of an advocacy plan for any of their studies apparently does not matter. How otherwise could one explain that there are 47 think tanks in Ukraine (25th by numbers of think tanks in the world), or 39 in Hungary or 25 in Czech Republic? I went on to count Serbia one by one any NGO that has done more than 3 surveys/policy studies that I could think of, then any political party think tank to all ‘real’ think tanks, university based research centers, governmental institutes and still did not arrive to 24, number listed in this study :-). I could go on…

Some amusing parts:

Nominations for leading think tanks in the world: I amused to see that F.A. Hayek Foundation, (Slovakia) was nominated as one of the best world think tanks :-). This organization is struggling to regain its reputation and resume regular work after last year’s scandal that implied two of its founders in embezzlement of government funds.

I am equally surprised/amused that Open Society Institute (OSI), the organization that I work for listed in the rankings. Twice. OSI is not a think tank. Full stop. Even some of us want to see OSI moving in that direction. OSI is predominantly a donor.

European Stability Initiative (Turkey, they have offices in Berlin and representative in Brussels) is ranked as 12th in the ‘Top 25 Think Tanks in the Middle East and North Africa (MENA)’.  With all the respect for their work on Turkey, but ESI is a think tank that is mainly working on EU affairs, EU accession, socio-economic development of Balkan countries (including Turkey; ESI’s work on the Turkey is at most 20-25% of their total work), South Caucasus, but nothing in MENA countries. This example illustrates the problem of ranking organized as ‘a beauty contest’. Who checked the votes and nominations and who decides that ESI’s work on Turkey and only Turkey suffices to be ranked under MENA countries.

Even more worrying parts:

I am really worried in the type of the organizations that are listed as think tanks in the list. Only in the top 5 of the Top50 Worldwide – non US (page 25) two are not think tanks: Amnesty International and Transparency International. While I am grateful to AI and TI for all the policy research and excellent advocacy they are doing regularly, their mandate and type of activities are much broader than those of think tanks. And here is the danger of really blurring the lines and confusing those leaders approaching this ranking lightly. One could go on and wonder about the ‘thinktank-ness’ of Human Rights Watch

What if I nominate Coalition of Youth NGOs ‘Now’ – Macedonia as a leading think tank for next year? The Coalition regularly carries out research about the rends among the young population, published reports, commissions public opinion polls and carries out surveys. Does that make them a think tank when 80% of their work has nothing to do with policy research? Shall the body overseeing this ranking exclude my proposal? I think they have to

My point is that the team in charge of this ranking should have clear criteria to weed out those nominated organizations that are not think tank (even if they claim or are nominated by one or two ‘experts’).

Ignorance on the verge of turning into an insult

DEMOS – UK has really tried in the last two years (particularly last year) to distance itself from the Labour Party. It has diversified its Board, established strong links with the Lib-Dems, so much that the former director accepted a position in the current UK government. To rank DEMOS as a party affiliated think tank next to the German Stiftungs is just a sign of ignorance and failure to update the rankings from one year to the other (In case that the ‘experts’ of this ‘beauty contest’ perceive that DEMOS is a party affiliated think tank, then something is wrong with their own competence and update. To be blunt and maybe vulgar, if the competition is for the best blonde… and a blonde had dyed her hair to brunette last year, now it has to be judged in the category of brunettes. With apologies to DEMOS for this comparison, but their remodelling and repositioning was so obvious even to someone who follows them from a distance as I do.

And an embarrassment:

Is Razumkov Center Ukraine ranked 16th or 25th in the Top 25 Central and Eastern European Think Tanks? It is listed on both positions! The mean result suggests 20,5th! ;-).

If the quality control of the presentation of the rankings is so weak (I could imagine a typing mistake), how can one trust the analysis of the date derived from the complex questionnaire sent to the experts?

And even a bigger embarrassment

National Endowment for Democracy is a respected donor, not a think tank. One needs to check them on the internet. Any intern with a BA degree will recognize the difference between a donor and a think tank. Who vets the interns who help out the ranking? :-) If OSI was a mistake that one could forgive, this one should not be.

….And let’s check the big picture:

The thematic rankings are even more dubious. What is a development think tank? Is this a think tank that deals with development aid, with development as defined by Wikipedia or… something else? How the ranking distinguishes between think tanks that analyze international relations (e.g. Brookings) from those that specialize on development aid ( e.g. Center for Global Development). Or if it compares what are the criteria to compare those?

Similarly, in the rankings for Best Think Tanks in Social Research, what are the criteria based on which USA based Brookings Institution is better than let’s say Center for Economic and Policy Research (CEPR) in the UK. The two think tanks address completely different issues and context (one in USA the other one in the UK)? Is this because Brookings has more publications? Or does it have a better design? Or methodologically Brookings is more advanced than the other think tanks? Or because it is more recognized (and if yes, recognized for what)? I do not want to mention impact J

To end, because I bored myself already, let alone you who have made it so far in my blog post :-). My favorite category is “The Greatest Impact on Public Policy”. I suggest the authors of this ranking to simply patent the way how they measured the impact and the differences between the organizations. Then, they should bottle it and start selling it to donors (especially those who support think tanks – I know plenty who would gladly pay for it), think tanks and other organizations that aim to influence public policy. The authors will retire filthy rich J

—-

These bits reveal that year after year, the few of us who observe and critically comment this ranking struggle to go through the factual errors and basic inconsistencies in the various tables. Last year, in my comment to the 2009 edition, I focused on the conceptual problems with the methodology and improvements of the rankings altogether. This year, for those who care about ranking at all ( Not my cup of tea), I suggest alternatives.

Instead of conclusion:

I thank THE THINK TANKS AND CIVIL SOCIETIES PROGRAM, International Relations Program at the University of Pennsylvania for promoting think tanks in the world. However, apart from making more people sensitive of the world think tank (Let’s hope not oversensitive), this study has little added value.

As someone who works with think tanks, studies think tanks, writes about think tanks, I see very little value in it. Therefore, it is high time to move to alternatives to this study:

–          Best national think tanks (see the suggestion of Enrique Mendizabal) modeled on the UK’s ranking done by Prospect magazine. Note: Thematic categories could be also established

–          Best Policy study ( for example see the Policy Association for Open Society (PASOS) award for the best study penned by their members)

–          Best advocacy campaign by a think tanks [consisted of a series of policy products (from op-ed to book), events (briefings, debates, seminars, conferences, training events etc.],

–          Best online presentation

–          Best design and communication strategy

http://goranspolicy.com/wp-content/plugins/sociofluid/images/stumbleupon_48.png http://goranspolicy.com/wp-content/plugins/sociofluid/images/delicious_48.png http://goranspolicy.com/wp-content/plugins/sociofluid/images/google_48.png http://goranspolicy.com/wp-content/plugins/sociofluid/images/facebook_48.png http://goranspolicy.com/wp-content/plugins/sociofluid/images/yahoobuzz_48.png http://goranspolicy.com/wp-content/plugins/sociofluid/images/twitter_48.png

12 comments ↓

#1 Enrique Mendizabal on 01.23.11 at 1:50 pm

Brilliant Goran, that type of analysis for your region is exactly what I feel is missing -and is far more useful.

It would be great if others -in Africa, Western Europe, Latin America, etc. provided a similar analysis.

If you don’t mind I’ll republish some of your comments on my blog and link back to this post.

#2 Goran’s recommendations on think tank rankings « on think tanks on 01.23.11 at 2:12 pm

[...] Goran Buldioski offers another take on the rankings in his blog Goran’s musings and some very interesting recommendations that I republish below: [...]

#3 Sonja on 01.24.11 at 6:49 pm

Thank you for speaking up! After looking into findings I could not be more disappointed to find organisation I am leading – Belgrade Centre for Security Policy among best party affiliated think-tanks. In all evaluations that both we took, as well as independent consultants paid by our donors, the first thing interviewees say about us is that we are non-partisan. During Milosevic’s regime and after 5th October, the Centre has on numerous occasions confronted in public domain parties from all sides of political specter, ruling parties most frequently. All policy advise that we have produced on a demand by particular party was published and distributed to other parties too, so that it is used as a public good and not a party good. See e.g. Team of the Centre for Civil-Military Relations prepared comments on the proposals of laws regulating defence and security

#4 Goran Buldioski on 01.24.11 at 11:25 pm

Thanks Sonja for your comment. The misrepresentation of BCSP is a clear example of how a Ranking is as god as it is accurate for every single organizations listed in the ranking. It seems three people (Enrique, you and I) already found more than 6 inaccurate representations. These all could have ben avoided with an additional step of quality control of the rankings.

#5 Yevhen with Razumkov Centre on 01.25.11 at 8:06 am

Hi, Gorane, check this out: Mr. McGann issued an updated version of the report, where Razumkov Centre is not mentioned twice (I was also surprised to notice it), but what do we see instead: now ICPS gets two places on the list — not fair, I think :)

http://www.gotothinktank.com/wp-content/uploads/2010GlobalGoToReport_ThinkTankIndex_UNEDITION_15_.pdf

#6 Yevhen on 01.25.11 at 8:12 am

Oh… Sorry for my ignorance, the Macedonians don’t have a vocative case, do you? Addressing to you as Gorane instead of Goran wasn’t proper, I guess.

#7 Goran Buldioski on 01.25.11 at 10:12 am

Interesting that the report was corrected :-). And this is a good sign, this is positive step.

I do not like to criticize without solid grounds. My preference would be to be constructive and maybe contribute to the design of the study. By now, unfortunately the study has taken a life on its own and the authors are hesitant to change their design and methodology.
No problem about the vocative. Macedonian does not have cases, but this particular form has remained. So you were correct. :-)

#8 Aytan Hachiyeva on 01.27.11 at 2:53 pm

Ooo, interesting discussions. To be honestly, I found this page in the result of searching “think tanks in the world”, however, I really enjoyed reading Goran Buldiosk’s article and others’ feedbacks. I am from Azerbaijan and working for UN Institute in Baku. I do agree that methodology used by UPen Think Tanks Program might be weak, not too strong, but I think it is still the best report on think tanks in the world. By “googling” I found other similar reports developed by different organizations, however, UPen’s report, in any case, is better than others in my opinion… I do agree Goran Buldiosk’s comments and I think his feedback can recommended to be used by UPen to improve its methodology. In the report, I found that Center for Economic and Social Development (CESD) from Azerbaijan also ranked in the top CEE think tanks. No doubt that CESD is the top think tank in Azerbaijan and the center has produced many high quality papers and many of them used by UN as well. Meanwhile, I am almost familiar with activities of all top 25 think tanks in CEE, particularly Carnegie Moscow Center, CASE (Poland), Center for Eastern Studies (Poland), Lithuanian Free Market Institute and I think the report was able to wrap all major think tanks in the region…

#9 Jeff Lovitt on 02.18.11 at 1:48 pm

Yes, the old mistakes are repeated year after year, despite our having repeatedly alerted Jim McGann. Not only Sonja’s BCSP is there as a party-affiliated think-tank, but also CEPS in Brussels – doubtless linked to one of the Flemish parties :) , even though their focus is the EU, not Belgium. The inconsistencies and mistakes – and the addition of political foundations, as well as TI and AI, as think-tanks, is particularly silly. Atyan (above) is probably not aware of the previous feedback consistently provided with no improvement over the years. The list of top 25 think tanks in the CEE region omits many think-tanks that should be there – e.g. the Institute of Public Affairs in Poland. The whole rankings are a standing joke in the think-tank community – for the simplistic, popular vote methodology. And I represent PASOS, many of whose members are highlighted in the rankings – so no sour grapes on my part :)

#10 Envelope, Please: Seeking a New Way to Rank Think Tanks | David Roodman | Global Development: Views from the Center on 11.19.12 at 3:11 pm

[...] it hard to learn from CFR, or even be sure that it is the right role model. (Enrique Mendizabal, Goran Buldioski, and Christian Seiler and Klaus Wohlrabe have said similar things. Also see David’s [...]

#11 David Roodman on 11.19.12 at 6:18 pm

Goran, my colleague Julia Clark and I just posted some quantitative metrics of think tank profile, which might interest you:
http://blogs.cgdev.org/globaldevelopment/?p=9872
It doesn’t approach the grounded quality of the commentary you offer here, but I think it does share your motivation, the belief that we can do better. See what you think.

#12 Cato Tops New CGD Index of Think Tank Profile | David Roodman | Global Development: Views from the Center on 01.14.13 at 9:28 pm

[...] be one reason expert perceptions of the GGTTT index itself have tended to be highly critical (here, here, here, here, here). Among the concerns: the opacity of the ranking process, the inclusion of [...]

Leave a Comment

buy robaxin onlineHerpes Zostergenital herpes buy isotretinoin onlinebuypillsonline2014.combuy Propecia online valacyclovir onlinehydrochlorothiazide onlineorder robaxinBuy albendazole order ataraxBuy ataraxbuypillsonline2014.combuy combivent online orofacial herpesHerpes Zosterorder citalopramcheap valtrex buy hydrochlorothiazide onlinePropecia without prescriptionvaltrex without prescription Buy aciclovirorder robaxinbuy buspar onlinebuy aciclovir online order vermox onlinepharmacy onlinebuypillsonline2014.comcheap valacyclovir rosuvastatinataraxisotretinoin without prescriptionBuy phenergan buy paxil over the counterbuy sertraline in the uk buy sertraline in the ukbuy baclofen with paypal buy strattera ukbuy antabuse online cheap buy augmentin duo fortebuy paxil 20 mg buy baclofen onlinebuy augmentin xr online can you buy zoloft in mexicocan you buy strattera over the counter buy antabuse in canadabuy clopidogrel 75 mg cymbalta buy online ukbuy generic augmentin online buy strattera online cheaptopamax to buy buy paxil cheaporder cymbalta 60 mg online Buy cheap propecia Buy cheap propecia online Cheap propecia online no prescription Buy propecia 24h Cheap propecia no prescription Propecia online Buy propecia online 24h Cheap propecia online Propecia 24h Buy propecia online Propecia 24h Cheap propecia no prescription Propecia 24h Buy propecia online Buy cheap propecia online no prescription Buy propecia buy proscarbuy letrozole online cheap indocinorder lasix online order letrozole onlinegeneric estrace order metronidazoleestrace online buy disulfiramorder metronidazole ariceptcytotec buy augmentin onlinecheap natrexone order lasixbuy diflucan online order retin acheap nexium y>