If you ask me

Why the UK is taking a fresh look at how it assesses the quality of academic research. Plus, why the UK should focus on its strengths and achievements.

A measure of success

How should you measure the quality of research being carried out by a nation's universities? This question has vexed the UK since the inception of the Research Assessment Exercise in the 1980s. The RAE approach has been to use subject-specific expert panels to assess the quality of research in each university. Universities that were judged to perform world-class research received a larger chunk of funding to support their development.

In 2006 the government decided that the 2008 RAE would be the final instalment as the expert panels were seen as too time-intensive and could be replaced by metrics. The replacement to the RAE is called the Research Excellence Framework. At the heart of the original proposal for the REF was metrics-based assessment of research quality for science and engineering subjects. The main metric was a bibliometric indicator. Research that had more citations would be deemed of better quality.

The science and engineering community responded to the proposal with a number of fundamental and practical concerns. The fundamental concerns included whether or not citations provide a good indicator of research quality. In some instances it would, but citations are a better indicator of impact, which could be positive or negative.

There was also concern, especially among engineers, that assessing only published articles is short-sighted, since high-quality research results in a variety of outputs, particularly in applied research. A focus on citations
could also discourage researchers from taking risks in forging new areas, as there would not be a community of researchers to cite their work.

A practical concern was the impact that metrics could have on the behaviour of those being assessed. Researchers may change their publishing behaviour in an attempt to maximise the citations of their papers; this could include publishing fewer papers and only in higher-impact journals. It could also encourage a move away from applied research in order to increase publications.

Most of the science and engineering community called for maintaining an element of peer-review so that a variety of outputs could be assessed and that any metrics used would be put in context. The government and Funding Council responded by saying there would be a "variable geometry of assessment". The method of assessment could include bibliometrics, other quantitative indicators (research income and number of PhD students) and qualitative assessment. Subject panels would be responsible for deciding what approach to use and the final quality profile.

The debate about how to assess is by no means over. There will be pilot exercises to assess the practicality of the bibliometric indicator and then the assessment will start in full. All along the way there will be discussions about the practicalities of assessment.

What we should not lose among these practical discussions are questions about why we are assessing research quality. If we do lose this, we will end up with a system of assessment for its own sake and not one that aims to improve the state of science and engineering in the UK.

Nick Dusic, director, Campaign for Science & Engineering

UK should keep aiming high

Let's paint a typically gloomy scenario about the future of British manufacturing. After the decimation of its industrial base, Britain has become rejuvenated by its expertise in research, development, science and innovation.

However, along comes China which, not content with providing the world's cheap-labour force, is threatening to take even this away from the UK. So British government ministers' exhortations about the country's bright future as a 'high-level' manufacturer will prove to be hot air.

This concern is certainly legitimate. China, never afraid of ambition, has set itself the task of doubling its research and development by 2020. It is already the second-largest R&D spender behind the US. With its access to a large engineering workforce as well as to US and European companies' technological know-how, China does appear to pose a threat to Western innovation.

Moreover, the nation has proved itself capable of taking Western specialist technology and turning it into a mass-market product in its own country. And I'm not referring to the theft of intellectual property rights from Western companies - a practice for which China has rightly been criticised in the past, but which it is now trying to address because to do otherwise would undermine its attempts to be seen as a credible global centre of innovation.

According to Professor Peter Williamson of Cambridge University, who spoke at a recent 'Horizon' seminar at the university, China has shown that it can turn high-cost innovations from overseas companies into 'low-cost innovation'. One example he gives is a 'digital-direct' X-ray machine developed by General Electric and Philips, which sold for $150,000 per unit. A (legitimate) Chinese version of this technology was offered at just $20,000. Why? Because China is adept at bringing high technology to a wider market at low cost.

Chinese companies are also buying Western brands as a way of accessing global markets. This access has been limited to date in the face of the Western-led globalisation of the past 30 years. Chinese firms have also been able to benefit and learn from the many manufacturing partnerships that these Western behemoths have established in the country.

So far so bad. Or is it? According to another Horizon speaker, Professor Chris Cullen of the Needham Research Institute, it is "unambiguously in Britain's interests" that China succeeds in becoming a research powerhouse. History shows, he says, that the development of a global industrial power, whether it be the US, Germany, or the capitalist-communist China that has now emerged, can only be good for the economic prosperity of Britain and the West.

Cullen admits to being an optimist on China, and some may see his analysis as downplaying the impact on the West. But his is still a refreshingly positive approach that also presents a challenge to Britain to stop looking over its shoulder at overseas threats and to focus on driving forward the high-quality technology and innovation for which it is lauded.

Bob Cervi, Editor, Manufacturing

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them