Is Italian Science Declining?

Embedded Scribd iPaper - Requires Javascript and Flash Player
Is Italian Science Declining?
Cinzia Daraio Department of Management University of Bologna Via U. Terracini, 28 40131 Bologna - Italy Tel. +390512090210 Fax. +390512090222 E-mail: cinzia.daraio@unibo.it Henk F. Moed * Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands. * Current address: Elsevier, Radarweg 29, 1043 NX Amsterdam, The Netherlands. E-mail: H.Moed@elsevier.com This version: 14 December 2010
Abstract
The paper analyses the Italian contribution to the world scientific production, its relative citation impact, its international collaborations and scientific productivity compared with the most productive EU countries over the period 1980-2009. It shows that despite the fact that the level of funding has been dramatically low during the past decades, Italian science has been able to increase its performance up to 2007. Italian science is a "cathedral in the desert". However, a recent reduction in the level of scientific production, the lagging behind in international scientific collaboration (highly correlated with the relative citation impact) and the great heterogeneity of researchers’ productivity (absence of correlation of number of researchers with quality and quantity of scientific production) may mark the start of a decline of Italian science. The paper concludes that the increased funding must go hand-in-hand with reform of autonomy and governance and calling for a sound system of internal quality control and performance enhancement.
Keywords: Italian science, public research organizations, bibliometric indicators, international collaboration, R&D policy
1. Introduction and research questions
Basic research carried out at universities and public research organizations is a crucial important driver for innovation, economic progress and social welfare (e.g. Adams, 1990; Griliches 1998; Henderson Jaffe and Trajtenberg 1998; Mansfield 1995; Rosenberg and Nelson 1994) and could be managed in a recession period, like the current one, in an anti-cyclical perspective. Both in the literature and in the political and public debate there is an increasing recognition of the role of universities as strategic actors in knowledge creation and diffusion (Etzkowitz et. al., 2000; Bonaccorsi and Daraio, 2007). Universities’ scientific production concerns especially basic research, but the results which are generated are not only long-term ones but produce spillovers that have short and medium term effects on industrial innovation (Mansfield,1991). Recent trends in the growth of international collaboration - as evidenced by co-publication, the emergence of international collaborative programs and increased mobility of scientists- and the growth of international comparison of scientific performance -as reflected in the frequent publication of benchmarking comparisons and ranking of scientific institutions (see Harvey, 2008) give evidence of the growing internationalization of scientific activity. The increasing use of economic rationales to support increased public funding for research has its natural corollary in the desire for evaluations to ascertain whether the promised benefits are actually being delivered. Despite the methodological problems that may arise in estimating the economic returns to public investment in basic research, according to Martin et al. (1996), the main contributions that publicly funded research makes to economic growth are: increasing the stock of useful knowledge; training skilled graduates; creating new scientific instrumentation and methodologies; forming networks and stimulating social interaction; increasing the capacity for scientific and technological problem solving; and creating new firms. Salter and Martin (2001) critically reviewed the three main methodological approaches adopted by the literature on the economic benefits of publicly funded basic research: econometric studies, surveys and case studies. Econometric studies are subject to certain methodological limitations, such as the assumption of a simple production function model of the science system, but they suggest that the economic benefits are very substantial. From the literature based on surveys and on case studies, it emerged that the benefits from public investment in basic research can take a variety of forms. The relative importance of these different forms of benefit apparently varies with scientific field, technology and industrial sector. Consequently, no simple model of the economic benefits from basic research is possible. They concluded their review stating that
“The literature available has shown that there are considerable differences across areas of research and across countries and that additional research is needed to better define and understand these differences. This limitation in current science policy research should not be seen as implying a need for less government funding of science. Rather, it indicates that public funding for basic research is, like many areas of government spending (e.g. defence), not easy to justify solely in terms of measurable economic benefits.”
Carillo and Papagni (2006) put forward a model of basic research and long-run economic growth in which the incentives of social reward to scientific work may produce increasing returns and multiple equilibria. 2
Rich empirical evidence shows that the governance and design of research institutions matters for economic growth and development (see Guiso et al. 2004; Bennedsen et al. 2005; Persson and Tabellini, 2006; Bauwens, Mion and Thisse, 2007). Hanushek and Woessmann (2008), reviewing the role of education quality in promoting economic growth, conclude that there is strong evidence that cognitive skills are powerfully related to longrun economic growth. They found that the relationship between skills and growth proves extremely robust in empirical applications. They interestingly showed that the effect of skills is complementary to the quality of economic institutions. They concluded that the long-run rewards to educational quality are large but also require patience. Aghion, David and Foray (2009) consider that the increasing awareness of the intimate and multiple connections of technological change and innovation with advances in science, on the one hand, and of the set of socio-economic institutions operating in a given context, on the other, encourages the conceptualization of “science, technology, innovation and growth systems” as appropriate subjects for policy-oriented research. Policy complementarities should be hence pursued in a larger dynamic system perspective among education, competition, macroeconomics and labour market. In a system driven by public funding, evaluating research is a key preliminary requirement. This is becoming more and more important given the broader changes in public sector management and the needs for accountability required by stakeholders. In such a context, it is imperative to define and implement effective evaluation systems that, in support of the allocation processes, stimulate adoption of a strong strategy and practices for increased productivity, both in quality and quantity, by universities and public research organizations. Evaluation is fundamental to allocate incentives to scientific excellence and as instruments for strategic choices on the part of political decisionmakers (Van Raan, 2004; Narin and Hamilton, 1996; Moed et al., 1995). Compared to other sectors, the university sector in Italy has the largest public human capital employed to produce R&D. According to the data from the General Accounting Office of the State (Ragioneria Generale dello Stato), in 2008, 89 per cent of R&D full time equivalent funded by the state, persons with a permanent position worked in universities as assistant professors, associate professors and full professors, whilst the remaining 11 per cent work in public research centres. The evaluation of the Italian R&D system has been analysed in the literature (see e.g. Silvani and Sirilli, 1995). In particular, Woolf (2003) studies previous attempts towards a university reform in Italy that proved dismal in the context of higher education policy in Western Europe, due to the pervasive power of academic mandarins, technocratic methods of reform, and the recurrent expectations that import of foreign models will resolve contradictions that are deeply rooted in Italian power relations. Biggeri and Bini (2001) examine the relationships between the State (the Ministry of the Universities) and each university in Italy, and the evaluation system established in 1996 and revised by the law of 1999. They discuss the system of indicators to be used for the evaluation and for the allocation of specific funds in terms of incentives, and to their possible effects on the decisions of the universities’ management. In 2003 Italy started up its first national research evaluation, a Triennial Research Evaluation, which referred to the period 2001–2003, with the aim to evaluate, using the peer review method, the excellence of the national research production. The evaluation involved 20 disciplinary areas, 102 research structures, 18,500 research products and 6,661 peer reviewers (1,465 from abroad); it had a direct cost of 3.55 millions Euros and a time length spanning over 18 months. 3
Using the data on the research assessment exercise of 2003, based on peer review, some papers have analysed them and compared with bibliometric evaluation (see Abramo, D’Angelo and Caprasecca, 2009; Franceschet and Costantini, 2009). A second evaluation exercise, assessing the time period 2004-2008, is currently being prepared. With the Decree no. 76 of the 1st February 2010 it has been approved the functioning and organizational structure of the Italian National Agency for the Evaluation of the University System and of Research (ANVUR, Agenzia Nazionale per la Valutazione del Sistema Universitario e della Ricerca) established 4 years ago with the law no. 286 of the 24 November 2006. According to the Decree no. 76/2010 the ANVUR is lead by a Committee (Consiglio Direttivo) composed of seven members with at least two men and two women, that are selected (chosen) among experts, also foreigners, with an high and recognised experience in the research and higher education sectors, and in particular in the evaluation of these activities, coming from different disciplinary and professional fields. The submission to a Selection Committee of proposals for experts was closed on 20 September 2010. Currently1 the Selection Committee is examining the CVs of the proposed experts and will nominate between 10 and 15 experts to the Ministry of Education and Research that will be in charge of choosing among these names the seven members of the Board of Directors (Consiglio Direttivo) that will run the ANVUR, The Selection Committee applies the following criteria: a) consolidated experience in evaluation, at a national and/or international level; b) consolidated experience in the direction of structures with high complexity, at a national and/or international level; c) a high international scientific profile. The Italian government has decided to carry out a plan, according to which the budgets of all Italian universities will be reduced by 7 per cent (this percentage has to be increased in the next years up to 30 per cent). This 7 per cent is put in one single basked, and re-distributed to universities on the basis of demonstrated research quality. Research quality is measured mainly on the basis of peer review, by external, mostly foreign reviewers who review the submitted "best" papers of each researcher. There is a current debate in Italy on the university reform. Some of the recurrent points of view in the debate appeared also in the journal Nature. Some believe that the Italian university system is not competitive, so that no more money should be spent on it until appropriate reforms have been carried out. But reform will not be possible without a sustained increase in research investment. At present, the research budget covers only staff salaries and there is no tool to encourage the best scientists with increased funding. (Nature, 452; 2008)”. What is needed is an “unregulated system of research funding allocation to reform the allocation criteria for funding and start applying across the board the selection and evaluation rules of peer review. Such a system would acknowledge meritocracy and free researchers from the virtual slavery under which they have been kept by old academicians ” (Nature, 543, 449; 2008) . And in Nature, 456; 2008, it was stated: “Another problem is that research resources are taken up by academics who only teach, rather than doing internationally recognised research; there is a marked resistance to the evaluation of scientific output, particularly from the unproductive areas. In the rare cases evaluation is carried out, this is done entirely on impact factors…”
1
At the moment we submit the paper, 15 December 2010.
4
The lively debated university reform received the approval of the Italian Conference of Rectors and has been approved by a branch of the Parliament in November 2010; it will most probably be approved by the other branch of the Parliament by the end of 2010. Given the crucial role of a nation's R&D for its development, thorough and critical analyses of the performance of national R&D systems are highly policy relevant. The measurement of Italian scientific standing is crucial for government and policy makers that have to decide on scientific priorities and funding. The main issue addressed in the paper is the assessment of Italian scientific standing within the European context, from 1980 to 2009. In order to address this issue thoroughly, the following detailed research questions have been formulated.  What is the Italian contribution to the word’s scientific production?  As regards the quality of Italian science: how is Italian science doing in terms of relative citation impact?  What is the standing of international collaboration in Italy? And how is it related to the quality of scientific production?  How many researchers are at work to produce Italian science? How many resources have been spent in the last three decades for science in Italy?  How is the Italian scientific system performing in terms of partial productivity measures (i.e. number of publications per researcher) and in terms of structural capacity of the system (i.e. number of publications per 1000 inhabitants) compared with most productive European countries?  What are the relationships among inputs and outputs of the Italian research activity?
The paper unfolds as follows. Section 2 outlines data analysed and methodology applied in the study. Section 3 presents bibliometric output indicators of Italian science in the European context, namely the world share of published papers, its relative citation impact, its scientific collaborations and its role in bilateral collaborations. Section 4 deals with the input size of scientific research, i.e. human and financial resources of the scientific systems in the various countries. Section 5 analyses scientific productivity, roughly defined as output divided by input. It combines bibliometric output indicators from Section 3 and input indicators presented in Section 4. Finally, Section 6 summarizes the main findings of the paper and provides an evidence based interpretation.
5
2. Data and methodology
This paper combines bibliometric indicatoes of research output (publication output, citation impact, and international scientific collaboration), input indicators of human and financial resources, and productivity indicators relating output to input. Table 1 presents an overview of the indicators calculated in this paper: their labels used in the paper, a short definition, technical specifications and the data source(s) from which they were derived.
Label Definition Output indicators(bibliometric) Number of research papers Publication published in peer reviewed journals output or scientific production (Relative) Citation impact Actual citation impact per paper published from a country, divided by the world average in the subfields in which a country is active Technical specification Expressed as a percentage of the world total (world share); counts articles, reviews and proceedings papers only; Use of a 4-year citation window (e.g., for a paper from 2005 cites are counted during 20052008) Based on institutional affiliations of publishing authors Bi-lateral ISC is ISC in which authors from precisely 2 countries collaborate Data source ThomsonReuters Web of Science
Web of Science
International scientific collaboration (ISC) A country's role in ISC (primary / secondary)
The share of papers by authors located in institutions in a country co-published with authors located in foreign countries The percentage of a country's bilateral ISC articles to which it contributed the first author (primary role) or only secondary authors (secondary role) Input indicators (human and financial resources) GERD All Total R&D expenditures, in all sectors sectors GERD Gov. R&D expenditures of Government & HE and Higher Education Sector No. Number of researchers in all sectors researchers per 1000 inhabitants All sectors No. Number of researchers in researchers Government and Higher Education Gov. & HE sector per 1000 inhabitants Productivity indicators (output/ input) Nr papers / Number of published papers per 1000 1000 inhabitants inhabitants Nr papers / Number of published papers per researcher researcher
Web of Science
Web of Science
Expressed in Euro per Inhabitant or as a percentage of Gross Domestic Product (GDP)
EUROSTAT EUROSTAT EUROSTAT
EUROSTAT
Fractional publication counts, i.e., a paper by authors from two counties contributes 0.5 paper to the publication output of each country
Web of Science/ EUROSTAT Web of Science / EUROSTAT
The data analysis applied a locally weighted least squares (loess) technique that fits 75 per cent of data, using an Epanechnikov kernel to show the trend in the indicators analysed. This approach was complemented with the calculation of non-parametric correlations among research output measures, namely indicators of publication output, relative citation impact and international scientific collaboration on the one hand, and resources used, R&D investments and number of researchers, 6
on the other. . Loess combines the simplicity of linear least squares regression with the flexibility of nonlinear regression. This is done by fitting simple models to localized subsets of the data to build up a function that describes the deterministic part of the variation in the data, point by point. One of the main advantages of this method is that it is not required to specify a global function of any form to fit a model to the data, but only to fit segments of the data. For more details the reader is referred to Cleveland (1979), and Cleveland and Devlin (1988). Nonparametric correlation measures, namely Kendall (1938, 1955) and Spearman (see Siegel, 1956) correlations are widely used in the social sciences. These measures are often considered robust with respect to outlying observations as opposed to Pearson correlation. Croux and Dehon (2008) have studied their robustness by means of their influence functions and conclude that both Spearman and Kendall correlation measures combine good robustness properties with high efficiency. Therefore these are used in this paper.
7
3. Italian scientific production in the European context
3.1 Trend of the Italian contribution to the world’s scientific production
The publication output of Italian science constantly increased during the past decades (1980-2007); the annual growth rates tend to be higher than those for other major European countries except Spain. This shows that Italy and Spain do not suffer from a “displacement effect” produced by the exponential increase of China and other scientifically emerging countries such as India and Brazil on the main European producers, namely Germany, France and UK (see Figure 1). In 2007 the percentage of articles from Italian institutions is around 3.5 per cent, close to that for France which is around 4 per cent, even though Italy fell below other large countries such as Germany and UK. However, after 2007, a slight decrease in the Italian scientific production can be observed: it was 3.4 per cent in 2008 and 3.3 in per cent in 2009 (see Figure 1).
Figure 1. Evolution of the percentage of articles published from eight major countries.. Lines are loess fit at 75 per cent obtained using an Epanechnikov kernel. CH: Switzerland; CN: China; DE: Germany; ES: Spain; FR: France; IT: Italy; NL: Netherlands; Source: Authors' elaborations on Thomson Reuters’ Web of Science (WoS).
8
3.2 Evolution of the quality of Italian scientific production
Figure 2 shows that Italy has reached in 2000 the world average in terms of relative citation impact of its scientific production; however its level is lower than the other main European scientific producers (CH, NL, DE, FR). In 2007 Spain has caught up with Italy in terms of citation impact. The converging trend in citation impact across countries over time reflects the globalization of science.
Figure 2. Evolution of the relative citation impact of eight major countries. The dotted reference line represents the world’s average. For China only 2005 and 2006 are reported. For country codes see the legend of Figure 1. Source: WoS.
3.3 Evolution of international collaborations
Italy is lagging behind in international collaboration as measured by co-publications, as illustrated by Figure 3 and may therefore loose a connection to the international research front. It was second during 1980s, but it is semi-last from 2003. In 2009 only Spain, with 41 per cent, has a percentage of co-publication with international co-authors lower than that for Italy (42 per cent), whilst Switzerland has a percentage of 65, and Germany, France, Netherlands and UK pergentags of 48, 49.5, 52 and 47, respectively.. 9
Figure 3. Percentages of articles (fractional count) from international collaboration relative to country total. Lines are loess fit at 75 per cent obtained using an Epanechnikov kernel. Data from WoS. Figure 4 illustrates the bilateral international collaborations that represent almost 85 per cent of all international collaborations for the year 2007. For each of a country's papers resulting from bilateral collaboration it was determined whether or not the paper's first author was affiliated with an institution in that country. If one assumes that in most subject fields the leading researcher or group in a collaboration tends to obtain the first position in the author list, one can obtain an indication whether the role of a particular country in a collaboration is leading or secondary. Inspecting Figure 4 it appears that Italy and Spain show international collaboration practices that are similar to those of developing countries.
10
% 1st address papers in bi-lat collab
60 CN TR TW IN 50 JP US KR BR ES IT
RU
SE AU DE NL BE CA UK FR CH
40 0 40 % Int collab articles (in 2007) 80
Figure 4: Percentage of first address papers in bilateral collaborations against the percentage of international collaborative articles in 2007. Data from WoS.
CorreCountry lation DE ES FR IT NL UK a) b) a) b) a) b) a) b) a) b) a) b)
GERD Per cent GOV+HE PUB int. coll. euro/inh. 0.930** 0.986** 0.946** 0.991** 0.876** 0.966** 0.879** 0.969** 0.606** 0.798** 0.235 0.320 0.918** 0.983** 0.917** 0.984** 0.816** 0.948** 0.771** 0.926** 0.626** 0.815** 0.385* 0.530**
TOT GERD euro/inh. 0.918** 0.983** 0.888** 0.972** 0.847** 0.960** 0.728** 0.892** 0.659** 0.839** 0.355* 0.496*
No. RES (all sect) per 1000 inhab. 0.818** 0.935** 0.925** 0.987** 0.859** 0.960** 0.268 0.365 0.574** 0.808** 0.269 0.376
RES GOV+HE per 1000 inhab. 0.901** 0.968** 0.912** 0.985** 0.809** 0.945** 0.268 0.266 0.349* 0.525* 0.378* 0.502*
RES BUS per 1000 inhab 0.822** 0.935** 0.881** 0.968** 0.853** 0.958** 0.224 0.374 0.657** 0.855** -0.033 -0.055
Notes:
a) Kendall's tau_b correlation. b) Spearman's rho correlation. ** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). ------------------------------------------------------------------
Table 2. Nonparametric correlations between relative citation impact on the one hand, and international collaboration and 5 input measures on the other. Table 2 shows that international collaborations (as measured by the percentage of articles in copublication) are highly correlated (more than 85 per cent) with relative citation impact for Germany, 11
Spain, France and Italy; it is modestly correlated (more than 60 per cent) for Netherlands and not correlated at all for UK. For most EU countries, including Italy, the quality of scientific production goes hand-in-hand with international collaborations. Hence, the Italy's lagging behind in international collaborations may mark the start of a decline of the quality of Italian scientific production. Strikingly, Table 2 illustrates that only for Italy and UK the number of researchers is not correlated with the relative citation impact. However, as showed by Bonaccorsi and Daraio (2009), UK is a differentiated system (in terms of teaching and research) whilst Italy is a non-differentiated nation. Hence, while the non-correlation of UK may be the result of different specialization of researchers either in teaching or in research, this is not the case for Italy2. The last two columns in Table 2 analyse more in dept the situation and show that for UK and Netherlands, two differentiated systems, there is a modest correlation of relative citation impact with the number of researchers in government and higher education sectors. Contrary to this finding, the number of researchers in the business enterprises sector for the UK is not related to relative citation impact, while it is for the Netherlands. Strikingly, for Italy neither the number of researchers in government and higher education sectors, nor the number of researchers in business enterprises sectors are related to the relative citation impact. Our interpretation of this puzzling evidence is presented in Section 6.
4 Input indicators: Human and financial resources
In this section we illustrate the evolution of the inputs of the research activities, namely financial and human resources.
4.1 Human resources
Figure 5 shows that the number of researchers in public research organizations (PROs) and higher education institutions in Italy is dramatically low: from the 1990s onwards it is the lowest in Europe: it was of 0.75 fte researchers per 1000 inhabitants in 1991 and it is the only country with a decreasing trend as from the 1990s. Spain has an exponential increase as from the 1990s onwards, reaching the highest value of 1.75 in the most recent years (2006-2007). The same dramatic scenario applies to total R&D personnel in these two countries (Figure not reported to save space).
Bonaccorsi and Daraio (2009) showed that the European landscape as a whole is poorly differentiated. Differentiation along the research dimension emerges only in UK, Switzerland and Netherlands, while it is totally absent in Italy and Spain. They conclude arguing that countries in which universities are more differentiated according to research or teaching dimensions have implemented differentiation policies through a variety of policy instruments. In turn, these countries also are ranked high in international ranking s of universities. They suggest that there may be a structural linkage between the poor performance of European universities in research-based rankings and the lack of differentiation. Daraio et al. (2010) point out that only a few European countries encourage differentiation according to university research output and competitive funding. In most countries universities are characterized by the absence of correlation (concentration) between research, funding and top researchers: excellent researchers do not receive better structural funding (although they probably win more competitive funding), thus the universities they belong to do not necessarily come at the top of the international rankings.
2
12
Figure 5 Evolution of the number of researchers(in full time equivalent) –sector Government and Higher Education - per 1000 inhabitants, over the period 1981-2009. Data from EUROSTAT.
4.2 R&D Investments
It is well known that Europe suffers from a ‘double deficit’ in higher education and research in comparison with the United States: as a percentage of GDP, there is the often debated deficit in terms of research funding, but there is also a sizable deficit in terms of higher education funding. The level of funding of European universities varies across countries but, on average, it is insufficient for a satisfactory discharge of its teaching and research missions. In particular, differences across countries in R&D spending become even more pronounced when the public versus the private source of this funding is considered; the gap in private funding is particularly important. Figure 6 focuses on investments in R&D. Italy spends much less than the other European countries: the total annual investment in R&D (Figure not reported here) is stable at a low level of 1 per cent of the GDP: the same trend is found for the investment in public research organizations and higher education institutions of around 0.40 per cent of GDP (Figure 6). The situation is even more dramatic if we consider the R&D expenditures of the business enterprise sector: also here, Italy is with a poor 0.6 per cent last in the set of European countries analyzed in Figure 6.
13
Figure 6. Evolution of the R&D expenditure (GERD) –sector Government and Higher Education as a percentage of the GDP over the period 1981-2009. Data from EUROSTAT.
4.3 Rate of growth of R&D investments
Figure 7 illustrates the rates of growth of R&D expenditure of Government and Higher Education sectors for main EU scientific producers. In the last decade Italy shows a trend of R&D growth close to zero, as Germany and France do. This is unfortunate because it shows that public funding is low as well in Italy and is not able to balance the low level of R&D investments of the business and enterprises sectors. By contrast, please note the constant increase of Spanish R&D expenditure in the last decade, that -as showed above – correlated with the Spanish increase of quality and quantity of scientific publications.
14
Figure 7 Evolution of the rates of growth in the percentage of R&D expenditure of Government and Higher Education sectors as a percentage of GDP over the period 1982-2009. Source: EUROSTAT.
5 Scientific productivity
Considering a simple productivity indicator given by the number of publication per researcher3 Italy appears to be the most productive compared with other major European countries. See Figure 8. The effects of a lack of investments during the past decades are visible in Figure 9: Italy is the last in terms of number of publications per 1000 inhabitants. Table 3 presents the nonparametric correlations of the number of of publications per 1000 inhabitants versus R&D investments of government and higher education sector (expressed in Euro per 1000 inhabitants). For all European countries the correlations are quite high indicating that the increase of R&D investments is positively correlated with scientific production. However, the nonparametric correlations of the number of publications per 1000 inhabitants versus Total number of researchers (all sectors) per 1000 inhabitants reported in Table 3 shows for Italy a different pattern compared with other European countries; the kendall’s tau-b and the Spearman rank correlations are the lowest with modest values of 0.418 and 0.535, respectively, significant at the 90 per cent.probability level.
See Bonaccorsi and Daraio (2004) for the main limitations of this simple measure and Daraio and Simar (2007) for more advanced productivity indicators.
3
15
Figure 8 Evolution of the number of publications (fractional count) per researcher over the period 1981-2009. Source: WoS and EUROSTAT.
Figure 9 Evolution of the number of publications (fractional count) per 1000 inhabitants over the period 1981-2009. Source: WoS and EUROSTAT. 16
Country DE ES FR IT NL UK
Correlation a) b) a) b) a) b) a) b) a) b) a) b)
Tot res per 1000 inhab. 0.743** 0.906** 0.968** 0.996** 0.744** 0.877** 0.418** 0.535** 0.752** 0.883** 0.540** 0.741**
GERD gov+he (euro per inhab) 0.836** 0.955** 0.951** 0.992** 0.735** 0.881** 0.878** 0.964** 0.872** 0.965** 0.662** 0.826**
Notes:
a) Kendall's tau_b correlation. b) Spearman's rho correlation. ** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed). ------------------------------------------------------------------
Table 3. Nonparametric correlations between the number of publications per 1000 inhabitants and two input measures
6. Discussion and Conclusions
This paper analysed the standing of Italian science and its evolution over the last three decades compared with the main scientific producers in Europe. At this purpose, the paper presented a bibliometric macro analysis of the Italian scientific production, analysing the evolution of the number of international publications in the WoS database of Italy, France, Germany, Netherlands, Spain, Switzerland and UK over the period 1980-2009 and comparing their relative citation impact, as well as their levels of international scientific collaboration. Human and financial resources have also been investigated as well as rates of growth of R&D investments and scientific productivity. It focuses on the position of Italy. This section first summarizes the main outcomes of the study. Next, it discusses the outcomes from a broader perspective. The main empirical findings are summarized in Table 4.
Table 4. Summary of the empirical evidence presented in Sections 4 and 5.
Aspect Empirical results measured Output indicators Publication The publication output of Italian science constantly increased during the past decades (1980output (Figure 1) 2007); the annual growth rate tends to be higher than that for other major European countries except Spain. However, after 2007, a slight decrease in the Italian scientific production can be observed (Relative) Italy has reached in 2000 the world average in terms of relative citation impact of its citation impact scientific production; however its level is lower than that of other main European scientific (Figure 2) producers (Switzerland, Netherlands, Germany, France). In 2007, Spain has caught up with Italy
17
Scientific collaboration (Figure 3) Role in bi-lateral collaboration (Figure 4) Correlations (Table 2)
Italy is lagging behind in international scientific collaboration and may therefore loose a connection to the international research front. It was second during 1980s; it is semi-last from 2003. In terms of the extent to which a country's role in bi-lateral international scientific collaboration is primary as opposed to secondary, both Italy and Spain show a pattern that is different from that of major European countries and similar to that of scientifically developing nations Contrary to the findings for other major European countries, for Italy neither the number of researchers in government and higher education sectors, nor the number of researchers in business enterprises sectors shows a significant, positive relationship with relative citation impact.
Input indicators Human resources (Figure 5)
R&D expenditures (Figure 6)
Annual growth rate in R&D expenditures (Figure 7) Productivity indicators (output / input) Number of Compared with other major European countries, Italy appears like the most productive papers per country in terms of number of papers per researcher,. researcher (Figure 8) Number of In the set of countries analysed in this paper, Italy ranks last in terms of number of publications per publications per 1000 inhabitants. 1000 inhabitants (Figure 9) Correlations - 2 For all European countries the increase of R&D investments is positively correlated with (Table 3) scientific production. Despite the fact that the level of funding has been dramatically low during the past decades, Italian science has been able to increase its performance. However, Italy shows a low correlation between the number of publications per 1000 inhabitants versus the total number of researchers (all sectors) per 1000 inhabitants.
The number of researchers in public research organizations and higher education in Italy is dramatically low: from the 1990s onwards it is the lowest in Europe. Spain shows an exponential increase from the 1990s onwards reaching in recent years a level more than twice that of Italy (1.75 versus 0.75. fte research per 1000 inhabitants. The same dramatic scenario applies to the total R&D personnel. Italy spends much less on R&D than the other European countries. The total investment in R&D remains stable at a low level of 1 per cent of the GDP. The same trend is found for the investment in public research organizations and higher education institutions at a level of around 0.4 per cent of GDP. The situation is even more dramatic if one considers the R&D expenditures of the business enterprise sector: also here, Italy is the last in Europe with a poor 0.6 per cent Italy's annual rate of growth of R&D expenditure of Government and Higher Education sectors is close to zero, as in Germany and France. By contrast, Spanish R&D expenditure shows a constant increase in the last decade.
It was found that both the Italian scientific production and its quality are highly correlated with R&D expenditures of government and higher education sectors. The paper has shown that despite the fact that the level of funding has been dramatically low during the past decades compared with most EU science producers, Italian science has been able to increase its performance up until 2007. Italian science is a “cathedral in the desert”, as the scientific system is productive even if very few resources are allocated to it. In presence of few resources the Italian scientific system reacts improving its productivity. This can be interpreted as an “overcompensation effect” of publication production increase in the phase of reduction of funds. The reader is referred to Braun, Glanzel and 18
Schubert (1989) who use this concept in relation to the decline in British science. Nevertheless, the productivity of the system is often used in the political debate to justify further cut on funding. The Italian contribution to the scientific production (as measured by the percentage of articles in the WoS database) decreased from 2008: it was 3.5 per cent in 2007 it went to 3.4 per cent in 2008 and to 3.3 in per cent in 2009. Italy, as the other main scientific producers in Europe suffers from the “displacement effect” generated by the globalization of science and in particularly by the exponential increase of Chinese scientific activities. However, the results revealed signs of decline, that should be carefully taken into account. Italy is lagging behind in international collaboration as measured by co-publications. In 2009 Italy had a percentage of co-publication with international co-authors of 42 per cent: compared to the main EU scientific producers it ranks semi-last (followed only by Spain with 41 per cent). Given the high correlation of international collaborations with relative citation impact as a proxy of the quality of scientific output, this lagging behind may mark the start of decline in the scientific quality of Italian science. It appears that the “bucket (container)” of (human and financial) R&D resources of Italian science is empty. Moreover, as pointed out by Bonaccorsi et al (2005), the bucket contains holes, meaning that the microprocesses that have to transmit the virtuous effects of knowledge, namely qualified human capital (especially in S&T), university-industry collaborations, public incentives, ICT technologies adoption, creation of new innovative firms, are blocked or do not work well. The observed low level of private investment in Italy can be conceived as a result of differences between EU countries in tuition fees, in the share of private institutions, in philanthropic funding, contributions by alumni and in the level of funding provided by enterprises. The reader is referred to Daraio et al. 2010 for a quantification of differences between 11 European countries. This paper revealed large difference in research policies/interventions carried out in Italy and Spain. The relevant factors of the Spanish growth (see also Cruz-Castro and Sanz-Menéndez, 2008) are related to the implementation of a national system of evaluation of researchers performance, the introduction of incentives based on individual performance (i.e., CNEAI sexenios) and to the promotion of the regionalisation of higher education policy. Moreover, it is well known that Italian institutions, including universities, are not able to get back the funding paid by the Italian Republic to have access to the European Research Framework programs. As regards proposals submitted recently to the European Research Council (ERC) according to a table reported in Nature (“Small countries are unexpected winners in ERC grant tables”, Nature, 454, 690; 2008) Italy ranks 15th in terms of grants per capita or 14th in terms of grants per overall GDP. The situation for Italy is even worse if one considers the rate of success of Italian proposals that is the lowest of all European countries. This result is related to the excessive number of proposals presented by Italian scholars (the highest in Europe) which in turn is related to the lack of funding available for research at national level, but includes also a kind of “brain-drain” effect4 in that the number of granted proposals are reported by host country and not by the nationality of the applicant.. Both the decline of international collaboration and the low success rate of ERC proposals may be due to the lack of “qualified” administrative support for the preparation of proposals which includes the lack of foreign language skills, the unwillingness to take responsibility by the universities or
4
For a quantification of Italian “brain drain” see Becher, Ichino and Peri (2004).
19
PROs’ administrators and lack of administrative assistance in the accountability of granted research projects very often lamented about by Italian researchers. Bonaccorsi and Daraio (2007) showed that the ratio of academic to non-academic staff is heterogeneous in the Italian university system and follows “political consensus” rather than being based on qualified support to increase international scientific productivity. A clear policy implication is that governments and large public research organisations should increase the “qualified” administrative support for the preparation, management and accountability of research proposals to enhance international scientific collaborations. A paradox emerged when considering the productivity of Italian researchers. In terms of number of publications per researcher Italy is the first in the EU comparison, but when correlating the number of publications and their relative citation impact with the total number of researchers (all sectors and sub-divided into government and higher education sector against business enterprise sector), it was found that the correlation for Italy is the lowest of all EU countries analysed, and is, as far as the number of publications is concerned, totally absent as regards the relative citation impact. The authors' interpretation is that Italian researchers are highly heterogeneous in terms of research quality and productivity. The peculiar correlation pattern found for Italy could be due to the selection process that followed political instead of merit-based competition; and to the evaluation based on non-quality related criteria. In this respect, an editorial of the journal Nature (“Situations vacant”, Nature 456, 142; 2008) has rightly emphasised that “Italy’s universities should be allowed to recruit whom-ever and however they want –with the all-important proviso that they also be evaluated on their academic performance. If the best performing universities received more state support, and the underperformers received less, the incentive to play politics when hiring would be plummet”. Indeed letting Italian institutions free in selecting personnel is essential, but at the same time, the timing and volume of hiring should become less dependent upon political cycles. Moreover, promotion of appointed researchers should be more performance based too, also because this instrument could tackle at least partly the problem of ageing of research staff. Our interpretation is supported by Bonaccorsi and Daraio (2003) who found that hiring policies follow the upturn and downturn of political cycles, rather than the intrinsic needs of scientific development. In fact, the flow of talented graduate and post-graduate students can be considered steady over time around a trend, apart from sectoral shifts due to the rise of interest for particular scientific areas (e.g. computer science in the ‘70s, or biotechnology in the ‘90s). If their hypothesis is true, hiring policies should follow the supply of talented people by opening opportunities at a steady rate. If not, talented people may be discouraged and uncertainty over the timing and volume of hiring may induce biases in the planned investment in human capital (see also “Acceptance of peer review will free Italy’s research slaves”, Nature, 453, 449; 2008). In addition, when hiring is massive and concentrated in a few years, the rate of hiring may be larger than the rate of supply of talented people and low quality people have better opportunities to enter. The process of recruitment of young researchers, which could have reduced the average age, was found to be waveform and lead to a significant increase of the entry age. Bonaccorsi and Daraio (2003) suggest then that the appropriate recruitment policy for scientific institutions is based on a steady flow of job opportunities, that encourage the investment of human capital and reduce the time interval between the graduate degree and a permanent position. Bonaccorsi and Daraio (2007) showed that the strong increase of full professors from 2000 had a negative impact on the average scientific productivity of Italian universities. A clear policy implication is that governments and large public research organisations should decide a steady 20
state rate of growth and plan recruitment campaigns within short, regular and reliable time intervals. Moreover, promotion based on scientific productivity and not seniority could address the aging of academic staff without being detrimental for Italian scientific productivity. A well developed system of academic performance measurement combining advanced metrics and peer review (see Moed, Glanzel and Schmoch, 2004; Moed, 2005) is absolutely essential to build up a political basis for a substantial increase of the level of public funding. Given the Italian situation illustrated in the paper, such an increase seems highly desirable. If Italy has to make an effort to bridge its funding gap, which is highly desirable, this can only be realized if at the same time the governance of public research organizations and in particular of universities is tackled. This is necessary to increase the efficiency of spending by these organizations, thereby delivering results. To attract more funding, universities and public research organizations first need to convince stakeholders—governments, companies, tax payers, students— that existing resources are efficiently used and would produce added value for them. Higher funding have to go hand-in-hand with a sound system of internal quality control and performance enhancement.
Acknowledgements
A previous version of this paper has been presented at the ESOF2010 (European Open Science Forum) workshop “Towards Criteria For The Evaluation Of Research And Researchers: State Of The Art Four Years After The European Charter For Researcher And The Minerva Codex”, 2 July 2010, Torino Lingotto, Italy. We thank Laura Teodori and conference participants for useful comments and discussion. The usual disclaimers apply.
References
Alberto Silvani – Giorgio Sirilli: "R&D evaluation in Italy: a science and technology policy view" Research Evaluation, vol.5 n.1, pp.69-77(April 1995). Etzkowitz H. , Webster A. , Gebhardt C., Cantisano Terra B.R. (2000), The future of the university and the university of the future: evolution of ivory tower to entrepreneurial paradigm, Research Policy 29, 313–330. Abramo G., D’Angelo C.A., Caprasecca A. (2009), Allocative efficiency in public research funding: Can bibliometrics help?, Research Policy, 38 (2009) 206–215. Adams, J. (1990), ‘‘Fundamental Stocks of Knowledge and Productivity Growth’’, Journal of Political Economy 98, 673–702. Aghion P., David P.A. , Foray D., (2009), Science, technology and innovation for economic growth: Linking policy research and practice in ‘STIG Systems’, Research Policy, 38, 681–693. Aghion, P. Mathias Dewatripont, Caroline Hoxby, Andreu Mas-Colell, André Sapir (2008), Higher aspirations: An agenda for reforming European universities, Bruegel Blueprint Series, Bruegel. Bauwens, L., G. Mion and J-F Thisse (2007) The Resistible Decline of European Science, CORE Working Paper 2007/92. 21
Becker S. O., Ichino A., Peri G. (2004), How large is the “brain drain” from Italy, Giornale degli Economisti e Annali di Economia, 63 (1), 1-32. Biggeri L. M. Bini (2001), Evaluation at University and State Level in Italy: Need For a System of Evaluation and Indicators, Tertiary Education and Management 7: 149–162. Bonaccorsi A., Daraio C. (2003), Age effects in scientific productivity. The case of the Italian National Research Council (CNR), Scientometrics, Vol. 58, No. 1, 47-88. Bonaccorsi A., Daraio C. (2004), “Econometric approaches to the analysis of productivity of R&D systems. Production functions and production frontiers”, in H.F. Moed, W. Glanzel and U. Schmoch (eds.). Bonaccorsi, A., Daraio C. (2007), University autonomy and structural constraints in the Italian system, in Bonaccorsi A., Daraio C., eds., (2007), Universities and Strategic Knowledge Creation. Specialization and Performance in Europe, Edward Elgar Publisher, Cheltenham (UK). Bonaccorsi A., Daraio C. (2009), “Characterising the European university system: a preliminary classification using census microdata”, Science and Public Policy, 36(10), 763-775. Bonaccorsi A., Daraio C., Merito M., Moneta A., Rossi C., Giannangeli S. (2005), Il Secchio Bucato. L’impatto Della Spesa Pubblica In Ricerca Sulla Crescita Economica Delle Regioni, Analisi di efficacia degli interventi in ricerca nelle aree sottoutilizzate (1999-2002), Ministero dell’Economia, Dipartimento per le Politiche dello Sviluppo. Bornmann, L.; Daniel, H-D. (2005) Does the h-index for ranking of scientists really work? Scientometrics, vol. 65, no. 3, pp. 391-392. Branstetter, L. (2003), ‘‘Measuring the Impact of Academic Science on Industrial Innovation: The Case of California’s Research Universities’’, Columbia Business School Working Paper. Braun, T., Glanzel, W., and Schubert, A. (1989). Assessing assessments of British science: Some facts and figures to accept or decline. Scientometrics, 155, 165-70. Carillo M.R. and Papagni E. (2006) Social rewards in Science and Economic Growth, DES DP no. 10, University of Naples “Parthenope”. Cleveland, W.S. (1979). “Robust Locally Weighted Regression and Smoothing Scatterplots”. Journal of the American Statistical Association 74 (368): 829–836. Cleveland, W.S.; Devlin, S.J. (1988). “Locally-Weighted Regression: An Approach to Regression Analysis by Local Fitting”. Journal of the American Statistical Association 83 (403): 596–610. Croux C., C. Dehon, (2008), Robustness versus Efficiency for Nonparametric Correlation Measures, ECARES working paper 2008_002, Brussels, Belgium. Cruz-Castro L. and Sanz-Menéndez L. (2008), Research evaluation in transition: Individual versus organisational assessment in Spain, in The Changing Governance of the Sciences. The Advent of the Research Evaluation Systems, Richard Whitley and Jochen Gläser, eds., vol 26, pp. 205-223, Springer.
22
Daraio, C. et al. (2010), The European University landscape: A micro characterization based on evidence from the Aquameth project, forthcoming on Research Policy, DOI: 10.1016/j.respol.2010.10.009 Daraio C., Simar L. (2007), Advanced Robust and Nonparametric Methods in Efficiency Analysis. Methodology and Applications, Springer, New York (USA). Franceschet M., Costantini, A. (2009), The first Italian research assessment exercise: a bibliometric perspective, arXiv:0912.2601v1 [cs.CY] 14 Dec 2009, downloadable at http://arxiv.org/abs/0912.2601v1 Georghiou, L. (1995), Research evaluation in European and National science and technology Systems, Research Evaluation, 5(1), 3-10. Geuna A., and Martin B. R. (2003), University Research Evaluation and Funding: An International Comparison”, Minerva 41(4), 277-304. Griliches, Z. (1998), R&D and Productivity, Chicago University Press, Chicago. Hanushek E.A. Woessmann L. (2008), Education and Economic Growth in International Encyclopedia of Education, 3rd Ed. Harvey, L. (2008) ‘Editorial: Rankings of Higher Education Institutions: A Critical Review’, Quality in Higher Education, 14(3): 187-207. Henderson, R., A. Jaffe and M. Trajtenberg (1998), ‘‘Universities as a Source of Commercial Technology: A detailed Analysis of University Patenting, 1965–1988’’, Review of Economics and Statistics 65, 119–27. Kendall, M.G.(1938). A new measure of rank correlation, Biometrika, 30, 81-93. Kendall, M. G., (1955), Rank correlation methods, Charles Griffin, London. Liefner I., (2003) Funding, resource allocation, and performance in higher education Systems, Higher Education, 46(4), 469-489. Mansfield, E. (1995), ‘‘Academic Research Underlying Industrial Innovations: Sources, Characteristics, and Financing’’, Review of Economics and Statistics 77, 55–65. Martin, B., Salter, A., Hicks, D., Pavitt, K., Senker, J., Sharp, M., Von Tunzelmann, N., (1996). The Relationship Between Publicly Funded Basic Research and Economic Performance: A SPRU Review. HM Treasury, London. Moed, H.F., Glänzel, W., and Schmoch, U. (2004) (eds.), Handbook of Quantitative Science and Technology Research. The Use of Publication and Patent Statistics in Studies of S&T Systems. Dordrecht (the Netherlands): Kluwer Academic Publishers. Moed, H.F. (2005). Citation Analysis in Research Evaluation, Springer, Dordrecht (Netherlands). OECD, (2006), Evaluation of Publicly Funded Research. Recent Trends and Perspectives. (Report prepared by Luke Georghiou and Philippe Laredo), OECD, Paris [DSTI/STP(2006)7]. 23
Rosenberg, N. and R. Nelson (1994), ‘‘American Universities and Technical Advance in Industry’’, Research Policy 23, 323–48. Salter A., Martin, B. R. (2001), “The Economic Benefit of Publicly Funded Basic Research: a Critical Review”, Research Policy, 30, 509–532 Siegel, S., (1956), Nonparametric statistics for the behavioral sciences, McGraw-Hill, New York. Silvani A., Sirilli:G., (1995), R&D evaluation in Italy: a science and technology policy view, Research Evaluation, vol.5 n.1, pp.69-77. Van Raan, A.F.J. (2004). Measuring Science. Capita Selecta of Current Main Issues.In: H.F. Moed, W, Glänzel and U.Schmoch, editors: Handbook of Quantitative Science and Technology Research, Dordrecht: Kluwer Academic Publishers, 2004, p.19-50 Whitley R. and J. Gläser, eds, (2008), The Changing Governance of the Sciences. The Advent of the Research Evaluation Systems, vol 26, Springer. Woolf S. (2003), On University Reform in Italy: Contradictions and Power Relations In Structure and Function, Minerva 41: 347–363.
24