A Survey from Thomson Reuters
A survey was conducted recently for Thomson Reuters to provide input for the forthcoming Times Higher Education World University Rankings. The results of the survey can be accessed at the Global Institutional Profiles Project set up by Thomson Reuters.
The results of the survey are important since they might provide a clue to what the new ranking will look like.
There were 350 respondents from the “global academic community” This is apparently more than the numbers that answered similar surveys by QS but it does not seem very large especially when THE has raised justified concern about the low and possibly unrepresentative numbers participating in the THE-QS survey of academic opinion.
Of those 350, 107 were from the UK (31%), 90 from the US, 30 from Australia, nine from Canada and seven from New Zealand. Thirty three were from the rest of Europe, 32 from Asia and 42 from the others, i.e. not from North America, Europe, Asia and Australasia. With nearly a third of the respondents coming from the UK and over two thirds from just five English speaking countries this is a distinctly Anglo-Saxon-centric affair.
The first question was the level of familiarity with various rankings. The THE ranking was the one with which the largest number of respondents were familiar. This is a slightly odd result since from 2004 to 2009 THE published rankings under the name THE (S) - QS World University Rankings. The Times Higher Education World University Rankings (minus QS) have yet to appear. No doubt, THE would claim that it was they who published the ranking and can retrospectively rename them if they wish.
Summarizing the responses to the survey, it seems that the respondents believe that university rankings
• are useful
• have methodological problems
• are biased
• encourage the manipulation of data
• encourage a focus on numerical comparisons
• use data that is not transparent or reproducible
• do not not include appropriate metrics
• favour research institutions
Among the information that respondents need or would like to have are
• Publications and citations
• Research awards
• Patents
• Faculty student ratio
• Faculty activity ratios (teaching income/research grants/publications per staff)
• Number of faculty by gender, international, ethnicity or race
• Number of graduate programs and degrees
• Collaboration
• Community engagement
• Perceptions of researchers, employers, alumni and community
Some of this, community engagement for instance, is too vague to be useful. Other items contradict the stated objectives of the developing ranking system: including patents and research collaboration in a general ranking would add more bias in favour of the natural and applied sciences. Others betray the American or European concerns of the respondents: alumni have little significance outside the USA. It is also noticeable that nobody seems interested in student perceptions.
2 comments:
I think it is worth pointing out the distinction between the Thomson Reuters' opinion survey you refer to, which was a short exercise to gather views on the strenghts and weaknesses of existing ranking systems to help inform the new methodology for the Times Higher Education World University Rankings, and the much, much larger reputation survey that Thomson Reuters is going to carry out which will contribute to the actual rankings results themselves. The massive reputation survey will gather properly targetted and sampled responses from thousands of academics across the world. More information is available at: http://science.thomsonreuters.com/globalprofilesproject/gpp-reputational/
Phil Baty, editor, Times Higher Education World University Rankings
I wouldn't say that high awareness of the THE ranking was especially odd in this context: the survey was heavily promoted by THE with links from their pages which discuss the proposed new system.
Post a Comment