A Non-Parametric Delphi Approach to Foster Innovation Policy Debate in Spain

The aim of this paper is to identify some changes needed in Spain’s innovation policy to fill the gap between its innovation results and those of other European countries in lieu of sustainable leadership. To do this we apply the Delphi methodology to experts from academia, business, and government. To overcome the shortcomings of traditional descriptive methods, we develop an inferential analysis by following a non-parametric bootstrap method which enables us to identify important changes that should be implemented. Particularly interesting is the support found for improving the interconnections among the relevant agents of the innovation system (instead of focusing exclusively in the provision of knowledge and technological inputs through R and D activities), or the support found for “soft” policy instruments aimed at providing a homogeneous framework to assess the innovation capabilities of firms (e.g., for funding purposes). Attention to potential innovators among small and medium enterprises (SMEs) and traditional industries is particularly encouraged by experts.


Conceptual Framework
Innovation is a complex process where agents produce knowledge based on other pieces of disperse and fragmented knowledge.As a consequence, we can reasonably expect that agents will almost never go through this process in isolation, but interact with other agents to acquire and exchange knowledge, information, and other resources [1].Social and technological changes in past decades induced more complexity on the innovation process, increasing the potential connections among actors [2].Indeed, it has been argued that innovation increasingly depends on external knowledge sources [3].
The theoretical approaches to assess innovation have reacted to such changes, shifting from a "linear" and autonomous perspective on the production of knowledge [4] to a comprehensive approach where the innovation process is the result of the co-evolution of society and technology, where agents do not innovate in isolation but within a "system", and where interactions and feedback processes are key to explaining innovation paths [5][6][7][8].This holistic vision of the innovation process gave rise to new research agendas, such as "open innovation", in which the discussion is centered at acknowledging the value of using external knowledge [9][10][11].There has been an increasing interest in the research of empirical evidence relating open innovation and innovation performance [12], but its implications for public policy-making have not yet been analyzed in detail (with few exceptions [13]).
These changes in innovation practice and theory also induced changes in innovation policy [14,15].For example, they showed the increasing need to manage interfaces between users and producers of innovations beyond the pure transfer of knowledge and technologies (e.g., stimulating demand articulation and bridging gaps between actors).These changes also urged policy-makers to embed innovation policies in a broader socio-economic context (i.e., the need to broaden the policy domain), to be more consistent with the management of a network.Finally, these new approaches allowed for the possibility of legitimizing innovation policy not only as a solution to market failures, but also to tackle system imperfections [16].
Even within this complex environment, innovation is considered a necessary path for growth and increasing competitiveness [17] and, thus, careful design of innovation policy has been a matter of concern for governments and managers that aim for sustainable development through technology changes.On that regard, most innovation-related indicators show that Spain is below the average of OECD countries [18], which suggests that it is not using its scientific and technological potential to adequately generate innovation.This might be seriously damaging the country's possibilities of overcoming the economic crisis and modifying its production model.In this paper we focus on the analysis of problems and policy instruments of the Spanish innovation system.According to the Innovation Union Scoreboard of the European Commission [19], Spain is below the European Union (EU) average for most indicators on innovation performance, and is considered a "moderate innovator" (among other countries such as Croatia, Cyprus, Czech Republic, Estonia, Greece, Hungary, Italy, Lithuania, Malta, Poland, Portugal, and Slovakia).
The primary objective of this paper is, therefore, to identify the changes needed in Spanish innovation policy, in the belief that innovation should be a strategic component of a new growth model in Spain; a model that should be adopted by all the stakeholders involved in its sustainable development.More specifically, the efficiency of the innovation policy is tightly linked to its four main activities [20]: (a) provision of knowledge and technological inputs to the innovation process, which include the provision of R and D and competence-building (e.g., education and training); (b) demand-side activities, directed to the formation of new product markets; (c) provision of constituents for systems of innovation, such as the creation and change of organizations, innovation, networking, and the creation and change of innovation-related institutions (e.g., regulatory instruments); and (d) support services for innovating firms, like financing of innovation, incubation activities (start-ups, entrepreneurship, and small firms), provision of consultancy, etc.
These four types of activities can be used to identify specific problems within the innovation system that might be (or are being) tackled through policy.In this sense, this classification can be used for both diagnosis and evaluation purposes of innovation policy, and sets the theoretical framework through the following general research question: R1.Taking into account the past and current Spanish situation, which are the main lessons to be learned in order to improve innovation policy efficiency?
The activities of the innovation system might also be affected by policy instruments.Even though they can be very diverse, the most accepted classification of such instruments in the literature [21] group them in three: (a) regulatory instruments, like the regulation of intellectual property rights, the regulation of research and higher education organizations; competition (anti-trust) policy regulations concerning R and D, etc.; (b) economic and financial instruments, providing pecuniary incentives to innovation activities (primarily through public support to research organizations); and (c) soft instruments, which are non-coercive and can be used to complement financial and regulatory instrument (like public-private partnerships, voluntary standardization of information, campaigns and public communication instruments, etc.).This last type of instrument is changing the role of the policy-maker, to become not only a provider and regulator, but also a coordinator and facilitator [22].
There is a recent body of literature that can be linked to the discussion of emerging soft instruments for innovation policy; that is the case of literature on knowledge management and intellectual capital, which deals with the importance of intangible capital of the firm [23,24], and the need for companies to manage and report on this capital [25].With a few exceptions [26,27] this literature is not sufficiently linked to innovation studies and this is damaging innovation measurements [28] which, in turn, hampers the design of an appropriate innovation policy.
What this line of research tells us is that the most important assets of companies are not bricks and equipment, but human capital, know-how, organizational capabilities, networking capacities, or technological knowledge [29,30].These are of an intangible nature and are not well reflected, if at all, in firms' annual accounts [31].This produces an important asymmetry in capital markets [32] because, despite being valuable firms with strong potential, their accounts reflect less than their real value, so they are then forced to pay more for financing in capital markets than others of less value.Up to now there has been no standard procedure to report on the above-mentioned intangibles and although financial analysts are taking into account the real situation of the companies, particularly the public ones, banks are still reluctant to accept information on intellectual capital which is not audited.This is especially damaging for SMEs, the largest group of companies within the Spanish economy.Therefore, one of our objectives in this study is to fill the gap between the two bodies of literature (innovation policy studies and intellectual capital analysis) by investigating whether encouraging standard and reliable reports on intangibles would be a good innovation "soft" policy instrument.This is stated in our second research question: R2.Would encouraging a firm's management and reporting of intellectual capital be an effective innovation policy instrument?
The importance of activities and instruments should be discussed by all the agents involved and taking into consideration the full complexity of the innovation system.One popular classification of relevant agents within the innovation system comes from the Triple Helix approach [33][34][35], where the results of the innovation process are seen as a consequence of the co-evolution of three main sub-sets of social systems: Industry, University, and Government.This approach emphasizes the role of linkages between the industrial, academic, and governmental agents.Of course, the goals of each type of agent might be different (e.g., academic excellence, or popular support), and the evolution of innovation policy might be deemed as a result of policy discussion among the relevant actors of the system (i.e., academics, businessmen, and policy-makers).In this sense, our analysis on this topic will be guided by the following research question: R3.Are there significant differences among the main players in the Spanish national innovation system with respect to the variables to be tackled and the direction that innovation policy should take?
To address these three research questions, we use a structured method to quantify the discussion of expert groups called the Delphi technique.Information is gathered iteratively through questionnaires where experts have to position themselves towards certain problems related to the activities of the Spanish innovation system and the feasible policy instruments to solve them.Their statistical opinion, and the degree of consensus of the group, will throw light regarding our first research question.The analysis on the opinion of experts towards certain "soft" instruments of innovation policy, related with intellectual capital reporting, will provide information on our second research question.Finally, the analysis of opinion by groups of agents (academics, businessmen and policy-makers) will answer our third question.
The rest of the paper is structured as follows.In Section 2 the non-parametric Delphi method introduced in this study is presented.Section 3 is devoted to presenting and discussing the research results.Section 4 includes the conclusions of the paper and the contribution to addressing the research questions.Section 5 relates to final remarks and future research.Appendix A describes the bootstrap Delphi method in algorithmic form while Appendix B details the results of its application to the case in hand.Appendix C summarizes the analysis by highlighting the most significant identified trends.

Design
Delphi is very useful whenever traditional statistical models of forecasting are not appropriate due to the complexity of the problem [36] or in the absence of adequate sets of data (historic, economic, or technical), which is when expert judgment is necessary [37].
As is well known, the Delphi method is used to help in the decision-making process whenever an expert opinion is gathered, mostly through the use of questionnaires with propositions [38].Its key characteristics are: (1) Iteration, by having the experts answer the questionnaire in at least two rounds.
(2) Controlled feedback, by providing the experts with information about the answers in-between rounds (for example, averages, modes, medians, etc.).(3) Anonymity, by not including information that might bias the results.(4) Group opinion, by aggregating the answers of the individual experts, which might be categorized according to a critical variable.
The selection of experts [38,39] and the development of the questionnaire are, therefore, key issues in Delphi analysis.In this case we followed a snowball process to select the experts and develop the questionnaire.In the first place four experts from the two sponsoring institutions were called in to define the scope of the analysis and the precise topics the propositions should explore.An initial set of topics was defined and discussed through in-depth interviews with five additional experts from different backgrounds and positions.As a result of the discussions and having in mind the above-mentioned conceptual framework, a preliminary questionnaire addressing the topics was tested with all these experts and a further 10 who joined them (names and positions available at [40]).These 19 experts provided the names of those who would be participating in the Delphi and then went on to review and refine the whole list, which resulted in a total of 315 people (40% in business, 39% in academia, and 21% in public administration).From these, a total of 82 experts were involved in the two rounds (38% from firms (29 experts), 41% from academia (37 experts), and 21% high officials in public administration (16 experts)).
The questionnaire addresses the following final list of issues, which are all relevant within the conceptual framework, with the number of propositions included within brackets: (1) Investments in physical capital and in education (13) (2) Information society (14) (3) R and D investments (9) (4) Disruptive innovation/Entrepreneurship (9) (5) R and D and innovation funding (9) (6) R and D and innovation fiscal incentives (4) (7) Social value of innovation (8) (8) Cooperation for R and D and innovation between academia and industry (10) Even though these proposed topics were obtained from the discussion with the experts, each problem or policy identified in the propositions of the questionnaire can be related to the theoretical framework based on activities of the innovation system [20].Taking a broad perspective, Investments in physical capital and in education, and R and D investments can be related with the activities for the provision of knowledge and technological inputs to the innovation process, since education and R and D are the main sources of competence building and new knowledge creation.Demand-side activities are studied through topics such as promotion (demand-side) of Information society and Social value of innovation.Although Information society might be also viewed as a promoter of networking for innovation, our focus in the questionnaire was on policies directed to increase the diffusion rate of innovation and communication technologies.That is why it is related here with demand-side activities (and policies).Cooperation for innovation and Disruptive innovation/Entrepreneurship are related with the provision of constituents for systems of innovation capturing, respectively, the links between agents (collaboration agreements, industry-university research, etc.) and the generation of variety within the spectrum of innovation plans.Finally, R and D Funding and R and D incentives for innovation activities can be clearly identified as financial support services for innovating firms.Of course, the list of topics included in the questionnaire do not pretend to be an exhaustive list of problems and policies related with these activities, but specific cases that can contribute to discussion with a systemic view of innovation process.
In addition to the theoretical frameworks, three criteria were taken into consideration when finally including propositions dealing with specific issues.First, the existence of data for Spain, comparable with those of other OECD countries (for example, Education investments); second, the use of the variable in the scientific literature (for example, R and D firm expenditures), and third, the interaction of at least two of the three helices (for example, Fiscal incentives are designed by the government and affect firms; R and D cooperation needs both industrial and academic involvement).
Taking into account the path dependent nature of technology and innovation [41], the experts were provided with a set of graphs and figures, mostly based on OECD data, which reflected the Spanish situation in comparison with other countries.
The result of the design is a questionnaire of 70 propositions that had to be scored on a five-point Likert scale, with score 1 representing strong disagreement with a given proposition and score 5, strong agreement: Each expert gives a score on a Likert scale for each proposition in the questionnaire in each round.Experts are also asked to provide additional comments to all propositions.This is particularly important when, in the second (and possibly subsequent) round, the expert disagrees with the group opinion.Very often the explanation of discrepancies is an interesting source of knowledge [36,42].
We have tested the validity of the questionnaire, demonstrating that the results of the survey are jointly reliable (Cronbach's alpha coefficient = 0.8818).We also have assessed how individual items are related to the overall scale, and all individual coefficients of Cronbach's alpha are above 0.80.
As for the traditional analysis of the results of the Delphi experiment, the following descriptive statistics are readily calculated: (1) Average score per round, group and proposition, to assess opinion.
(2) Variance of the score per round, group and proposition, to determine consensus in opinion.
(3) Difference in the variance of the score in-between rounds or groups per proposition, to evaluate persistence in consensus.(4) Difference in average score in-between rounds or groups per proposition, to quantify the differences of opinion.
Simply graphing the results or comparing them against thresholds is usually enough to derive useful information.In the case at hand, and due to the importance and novelty of the topics, a more robust inference analysis is sought.

Non-Parametric Bootstrap Analysis
The main reasons for being unable to carry out a sound and robust inferential analysis of a Delphi experiment are usually the small size of the sample and the Likert scale in which the propositions are quantified.In these circumstances, the use of traditional parametric tools is questionable and other techniques have proved more reliable.Resampling methods are a good alternative [43].Among them, the non-parametric bootstrap [44,45] should be highlighted, not only due to its widespread use but also because it is used to analyze the results of Delphi experiments, although as mentioned before, only in medical studies [46][47][48].However, it was only a matter of time before the bootstrap would be used in other areas, like technology and innovation, as we do in this paper.To the best of our knowledge, the non-parametric bootstrap Delphi is methodologically applied for the first time in the field of technology and innovation policy forecasting, providing a sound basis for a thorough discussion on innovation policy issues.
The idea behind bootstrap is both simple and statistically robust.The aim of bootstrap is to generate a lot of data by resampling from the original sample.This data intensive strategy has proven reliable in estimating statistics like averages and their confidence intervals [49], quantiles [50], or even p-values [51].Non-parametric bootstrap is also appropriate when analyzing the results based on a Likert scale and different sample sizes (for example, smaller than 100 [52], larger than 1000 [53]).
Methodologically, bootstrap works as follows.Out of the original sample of I values, a new sample, called the bootstrap sample is obtained.Each original value is assigned a probability of 1/I, and I samples are obtained by sampling with replacements.The procedure might be repeated r = 1,..., R times to obtain R bootstrap samples.Each time, sample statistics might be calculated, arriving at their sampling distributions.
Then, to compare the sample statistic with the underlying sampling distribution, resampling methods use the frequency distribution of the statistics obtained from the newly generated samples instead of using a theoretical parametric distribution.The p-values might then be calculated by just using relative frequencies corresponding to the null hypothesis or the predefined thresholds.As usual, p-values lower than the significance level call for the rejection of the null hypothesis that is set to evaluate a predefined research question.The non-parametric bootstrap method used in this paper is mathematically specified in Appendix A.
Therefore, by resampling the experts' scores which were given to the statements proposed in the questionnaire, a discussion on policy issues can be fostered based on the following assumptions, as in usual Delphi analysis: (1) Propositions with extreme scores (strong agreement or disagreement) and higher levels of consensus provide stronger messages on policy directions which might either encourage innovation or hamper it, than propositions with lower levels of agreement or consensus.(2) Propositions with higher levels of agreement among the three groups (academics, entrepreneurs, and policy makers) indicate policy directions that are more well-defined than those where there are greater differences among groups.
The initial descriptive results of the two-round Delphi experiment showed the huge potential for discussion of innovation policy topics but were not considered robust enough to define policy directions and it was then that the resampling method was developed.
As mentioned before, 82 experts from three sectors participated: 37 from academia, 29 from the business sector, and 16 from public administration.They gave scores for each of the 70 propositions.

Aggregate Analysis of Opinions
The descriptive results of the Delphi can be seen in [40].We will concentrate here on the more robust results drawn from the inferential analysis.The latter do not contradict the former, but allow us to see, for example, whether a strong agreement is statistically significant.
The paragraphs below show the results obtained when individually testing each proposition looking for statistical significance in which to base the discussion on innovation in Spain: For a proposition to be considered as policy-prone the expected score is 4 or higher, or 2 or lower.
For a proposition to have demonstrated solid opinions, the variance of the scores between rounds decreases.‚ Test 3.For a proposition where expert groups differ, the pair-wise difference in the expected score is other than 0.
The results of the bootstrap analysis by proposition are detailed in Appendix B. Appendix C summarizes them and helps highlight the significant propositions in terms of extreme scores, higher consensus, and differences among expert groups.The tables included in this section summarize the results by topic.

Test 1: Looking for Extreme Average Scores
Initially we focus on those propositions with high or low average scores.In the case of strong agreements, the null hypothesis is that the true expected score for the proposition is 4 (Ho: µ j = 4), and is tested against the alternative hypothesis H 1 : µ j > 4. Accordingly, in the case of strong disagreement, the null hypothesis is Ho: µ j = 2, and the alternative is H 1 : µ j < 2.
More than half of the 70 propositions show statistical significance for strong agreement or disagreement at 95% confidence (see Table 1 for a summary of the detailed results provided in Appendix C).In addition, we have carried out an additional test for controversial results, to see the number of propositions for which the average score is 3 (i.e., H 0 : µ j = µ 0j = 3, H 1 : µ j ‰ 3), which turns out to be 6 (8.6% of the 70 propositions).Thus, the number of propositions with weak agreement or disagreement is 26 (37.1%).Some controversial topics appear for the following issues: Information society, Disruptive innovation, and Social value of innovation.Strong agreement (or disagreement) is the typical stance in most of the topics, except for Information society and R and D and innovation funding, where most of the propositions receive weak agreement (or disagreement).It should provide a concise and precise description of the experimental results, their interpretation as well as the experimental conclusions that can be drawn.

Test 2: Looking for Solid Opinions
An expected outcome of Delphi is an increase in consensus through rounds.Such an increase could be interpreted as a more solid opinion and would therefore provide a more solid basis for identifying policy directions.The test we have used in this case is based on the differences in the variances of the scores through rounds: The results show that, with 95% confidence, consensus increases in 60% of the propositions and only decreases in 11.4% of them (Table 2).Increasing consensus dominates in all topics, and decreasing consensus is present only in three of them.The presence of stability in consensus is particularly important in Information society and Disruptive innovation.It is interesting to note that in the case of Disruptive innovation, the dominance of a non-increasing consensus is in line with the low proportion of strong agreements or disagreements (see Table 1).It is important to study differences in opinion among institutional agents (i.e., researchers (r), businessmen (b) and policy-makers (p)).First, the results of the second round for all of the groups are simultaneously compared: Then, the average score from each group of experts is individually compared against the average score of the whole sample: pH 0 : µ g2 " µ 2 , for g " r, b, pq Table 3 summarizes these results by topic (since we are comparing the opinion of groups only in the second round, subscripts indicating the round are omitted to simplify notation).
The fact that the proportions of significant differences are "low" shows homogeneity of opinion.One interesting aspect is that businessmen appear to be more favorable to the propositions than the other two groups, since in 18.6% of propositions they were significantly above the average score of the whole sample.On the other hand, researchers look more cautious than the other groups, since it is the group that scores below average in more propositions (7.1%).
There are no significant differences in opinion among groups in the topics R and D investment and R and D and innovation fiscal incentives, but there seems to be more discrepancies in topics like Investment in physical capital and education and Information society.

Results of the Experiment
Particularly interesting are the findings detailed below.They have been grouped first according to the list of issues given to the experts.A final subsection is included to summarize the results.

Investments in Physical Capital and Education
The propositions of this topic can be related to the activities of provision of knowledge and technological inputs to the innovation process.Although experts identify a problem of the innovation system regarding these activities, there is less consensus surrounding the specific instrument to improve performance.
We raised two propositions related to investment in equipment.As could be expected, the experts as a whole are not happy either with the amount of current investment or the choice of industries which have received investment in the past.Although agreement is weak, the consensus increased during the process and no discrepancies appear among the three groups.We must say, however, that policy-makers (partly responsible for such investments) are more comfortable with the current status of physical capital than the other two groups.
Notwithstanding the importance of physical capital, we were particularly interested in the part of the questionnaire which referred to the main changes needed in education.The agreement in favor of increasing investment in education is strong and stable.Consensus increases between rounds when agreeing that there is room for improvement.
The three groups of experts agree on the need to develop competences for innovation in both secondary and tertiary education, while only businessmen think that this is also a must in primary education.
To develop a culture of effort and raise social awareness of teachers' value gains strong agreement from all groups with an increase of consensus throughout the process.Already in the first round all the experts agreed there was a clear need to pay more attention to vocational training.Some controversy arises, however, on how suitable it is to foster institutions of excellence at the different education levels.Agreement is strong when talking about the tertiary education level but consensus decreases in the second round and there is only weak agreement on secondary schooling.

Information Society
The propositions in this set were looking for areas where public incentive would be most useful in fostering the spread of the information society as a framework condition for innovation.In general, these propositions intended to capture demand-side policy to promote the use of information technologies.Large Spanish companies compare very well with others at European level, while the opposite is true of Spanish SMEs and households.Weak agreement is found in all the groups for supporting investment in the information society with subsidies.As could be expected the businessmen are more in favor of supporting companies, whatever size, than the other experts.
There is no clear support either in favor of the other incentives suggested (supporting equipment acquisitions, reducing taxes for on-line buying, etc.).In some of these direct support incentives there are clear discrepancies among the three expert groups.
In contrast, there is a very wide support for improving the teaching skills needed to introduce the information society at primary level in schools.Again, education is considered a key issue.
With respect to e-government, one of the main indicators for measuring the scope of the information society in a particular economy [54], the Spanish tax agency has been a benchmark in using on-line procedures to communicate with taxpayers and allowing on-line presentations and tax payments.Strong support, already shown in the first round, is given to extending this example into other areas of Government (health-care, education, etc.).

R and D Efforts
This topic is one of the most relevant issues for innovation policy, as it is clearly directed to increasing the provision of knowledge inputs to the innovation process.
There is weak agreement in favor of increasing public R and D. However, it becomes stronger when referring to an increase in the private sector.Reflected here are the experts' concerns about R and D expenses and their sectorial distribution in Spain.Not only is the ratio R and D/GDP smaller in Spain than the OECD average (1.35% against 2.34% in 2008) but also most of the R and D spending rests on the Government's shoulders.Industry spends 0.74% in relation to GDP, while in the OECD this figure is 1.63% [55].
There is also strong agreement about the need to evaluate and follow up public R and D policy, although, as could be expected, policy-makers are less inclined to support this opinion.
The particular comments added to this set of propositions highlight the need to take into account SMEs when designing new R and D policies.A significant increase in the number of Spanish companies undertaking R and D efforts should necessarily come from increasing SMEs' share.

Disruptive Innovation/Entrepreneurship
With regard to constituents, once again, experts show their preference for improving framework conditions (such as reducing red tape) to encourage disruptive innovation rather than using direct subsidies.
Their opinions, however, differ on whether there should be a size threshold to be a "disruptive innovator".Businessmen clearly reject this suggestion and there is agreement, albeit weak, on the possibilities of innovation in traditional industries and on the existence of "gazelle" entrepreneurs in these areas.Businessmen are particularly convinced of it.
The importance of traditional industries and the ways of improving their innovation capacity through the use of high technology is mentioned by many experts in their additional comments.A provocative proposition (number 41) suggesting that business success derived from innovation occurs mainly in high-tech industries raises an interesting degree of controversy.Businessmen particularly disagree with this assumption, even those whose business is actually high-tech.
On the whole, the experts draw attention to SMEs and traditional industries, highlighting how important they may be in improving innovation performance in Spain.It must be noted that these two groups have been very much ignored by the Spanish innovation policy up to now.

R and D and Innovation Funding
The lack of financial support services is a clear barrier for R&D and innovation in Spain [56].All the experts agree, with nuances, on the idea that the increase in funding would be more efficient if made by companies rather than by government.
As mentioned before, one of our objectives was to explore the possible benefits of reporting intellectual capital in order to increase innovation funding and thus to have it encouraged through policy mechanisms.Accordingly, this part of the questionnaire was a long and important one where the experts were particularly asked to give opinions on the propositions related to this issue.
Six propositions confronted them with the fact that annual company accounts do not clearly show their intangible investments and assets (like R and D and other innovation efforts) and that this creates a serious barrier to obtaining external funds, especially for knowledge-intensive small firms [57].The experts were asked their opinions in relation to taking into account the potential of those investments by having reliable (e.g., verified by external experts) additional information on them.
A current characteristic of the Spanish innovation policy makes these questions particularly interesting.Not only are banks reluctant to provide loans when they are not backed up by physical assets, but public sector institutions, the loan providers of R and D and innovation activity funding, require bank guarantees whenever the main assets are intangibles.Potentially innovative companies, intensive in intangibles, are thus trapped in a vicious circle when needing funds.
There is no controversy around this issue.Regardless of any small discrepancy which may stem from the novelty of connecting intangibles reporting with innovation funding, all the experts agree on the need to change the current status, in order to accurately measure and value the intangibles.By achieving this, companies will be in a better position when demanding innovation funding.This is an interesting result since, to the best of our knowledge, this topic -the relationship between reporting on intangibles and better performance in capital markets-has not been properly linked to innovation policy analysis in the literature.

R and D and Innovation Fiscal Incentives
Spain is the OECD country with the largest tax credits for R and D [58].With this in mind, the fact that it lags behind the OECD average in R and D and innovation investments may be considered a proxy for the inefficiency of government fiscal sacrifice.
Looking for explanations to such poor results from the fiscal incentives and support services, especially in SMEs, experts agreed clearly on: (a) tax incentives are distributed badly and are not well-known to company managers; and (b) fiscal norms should be drafted more clearly and have continuity to be more effective.The consensus increases around these ideas in the three groups of experts.

Social Value of Innovation
Through a yearly survey exploring public interest (demand-side) in science and technology produced by the Spanish Foundation for Science and Technology [59], it can be seen that even among the well-educated population, scientific, and technological issues attract little interest.As could be expected, this is considered very damaging to innovation by the experts.
They coincide on the need for companies to report more on the beneficial consequences of their innovation efforts, the importance of scientists' publicizing their work, the need for reporting on scientific and technological developments and the value of innovative managers for society.These are all necessary conditions for growth and welfare.

University-Industry Cooperation for R and D and Innovation
The experts strongly agree on the need for companies to play a more important role in university governance and on the need for universities to be more receptive to ideas from business (constituents).
They are also very clearly in favor of rewarding researchers fairly for improving business success as a result of collaborative projects.

Summary for Spain
It is interesting to emphasize two specific recommendation that emerged from the analysis of the topics and that summarize the results, as specified by the experts concerning the situation of Spain: (a) Framework conditions might be more effective than subsidies or direct support for encouraging innovation; and (b) SMEs and traditional industries must be drawn into research and innovation policy.
(a) Framework conditions might be more effective than subsidies or direct support for encouraging innovation.This argument, clearly backed up in some literature [60], is particularly clear in several issues such as education policy, e-government, and private firms' involvement in innovation.
The strong support for continuing to increase investment in education is in line with OECD recommendations, but this is not all that matters.Spain is showing bad results according to the Pisa report [61] which calls for developing a culture of effort and a greater social awareness of teachers' worth [62].Developing competences for the introduction of innovation in both secondary and tertiary education is to be encouraged, however, among the three groups, only businessmen think that this should also apply to primary education.In this regard, they are closer to recent postulates that suggest that cognitive capabilities and stimulating intellectual curiosity, which serve as a lifelong motivator for continued learning, take place at primary, and even at pre-primary, school level [62].The importance of fostering institutions of excellence at the different education levels is controversial among the experts, particularly when talking about the secondary level.It is an issue that also causes controversy in the Pisa report [61], and deserves closer future attention.On the whole, experts are considering the education institutions in a holistic way, which is how any system of innovation should be analyzed [63].
Framework conditions are also highly relevant when spreading the Information Society.Actions where the government leads by example (using on-line procedures in its dealings with the public) would have a stronger effect than direct subsidies for specific information technology use.E-government is considered one of the main indicators for measuring the scope of the information society in any economy [54].
Moreover, framework conditions should be changed so as to allow companies to really participate in the knowledge economy.Reducing red tape would facilitate their engagement in innovation activities, while direct measures, such as tax credits to R and D and innovation expenses, should be well-diffused and stable over time.Finally, companies should have a real say in teaching and research in higher education institutions bringing the governance model closer to the entrepreneurial and managerial system [64,65].
Framework conditions are especially important in critical periods when deficits must be reduced because, very often, they do not require additional or significant public investment [56].This is the case of many of the actions supported by our experts; for example, the general use of e-government will reduce the cost of public services in the medium term.
However, direct subsidies continue to be in use and are particularly encouraged for SMEs in some cases.This can be seen, for example, in some Baltic countries [66,67] or in the OECD recommendations to Serbia [68,69].Nevertheless, when looking at these recommendations in detail, the SMEs implicitly or explicitly considered are those which are already conscious of R and D relevance and are involved in these activities to some extent.
Regarding the framework conditions, our results are also consistent with some recent work on the implications of open innovation practices for innovation policy.It has been argued [13] that policies for open innovation should not target only firms' behaviors, but also need to secure the key external conditions of open innovation.These conditions can be classified in three topics: (1) strong public knowledge base; (2) mobile and educated labor force; and (3) good access to finance.The strong support of our experts to policies affecting education and public expenditure on R and D activities, points also to the need for a stronger public knowledge base and a more educated labor force.The same can be said about the consensus towards improving the access of SMEs to finance.We suggested before that these consensus is aligned with the need to improve Spanish innovation system, but we can also state that this support might be also connected with the need for a more "open" system [70].
(b) SMEs and traditional industries must be drawn into research and innovation policy.The experts identify niches in those two groups (e.g., efficient use of water in the energy industries; tourist industry and health; the development of logistics centers in transport and communication) where support may be particularly effective.
SMEs and traditional industries have not been targeted in past or present innovation policy "soft" instruments and this has been an oversight recognized by even current policy makers.When SMEs are targeted as happens in the case of tax credits, the actual impact of the subsidy is small.This may be due to the fact that government institutions are not sufficiently aware of how these firms work [71], and that most Spanish SMEs are potential innovators or non-innovators and do not respond to these types of instruments.The European Commission is very clear about this: When dealing with "SMEs (that) do not practice any R and D activities, it is in general not useful to have an R and D policy for them that consists purely of financial support" [72] but the policy should call for a proactive attitude to address their specificities.
The lack of conclusive results in the literature with respect to the relationship between company size and innovation capacity is also reflected in the experts' opinions.Their replies denote disagreement when referring to the need for a size threshold for developing disruptive innovation.The businessmen clearly reject any such need which is in line with the idea that SMEs are actually more agile and responsive to change [73].Markides [74] develops Christensen's disruptive innovation concept [75] and suggests that the early pioneers that create "new-to-the-world" products are rarely the ones that scale them from little to big markets.They are usually small, even start-up firms, which nurture the network large firms need to extend their business.
Examples showing how business-model disruptive innovation may happen in traditional industries are found in the literature.This is the case of Amazon or Swatch [76].
Thus, the suggestion is that when targeting SMEs to increase innovation, their size is not the most important consideration but their role within the Clarysse and Duchêne classification [77] and that particular attention should be paid to the large group of potential innovators.

Conclusions
R1. Taking into account the past and current Spanish situation, which are the main lessons to be learned in order to improve innovation policy efficiency?
Regarding our first question, we follow the framework based on activities [20] to discuss our policy recommendations in the light of their relation with one of the four types of activities within an innovation system.
Regarding the provision of knowledge and technological inputs to the innovation process, the main lessons or policy recommendations are: R2.Would encouraging a firm's management and reporting of intellectual capital be an effective innovation policy instrument?
Regarding our second research question, we find support for the idea that R and D and innovation funding may be increased if intangible assets and competencies are duly measured and reported.This would require standardized and reliable intangibles valuation procedures.
The need for adequate funding to improve innovation capabilities [4] is clearly acknowledged by the experts.They also state that funding for innovation is hampered by the characteristics of the current accounting model which fails to show the company's intangibles and intellectual capital in the annual accounts [29].
Some recent international initiatives [77,78] are insisting on the need to have standards for valuation and reporting on intangibles and the need to follow the guidelines suggested in the literature [22].The strongest recommendation is to produce a statement on intellectual capital which supplements the financial information provided by the annual accounts [38].The experts support both standardization of reports and external validation.
Should companies produce this type of report, the financial markets would have access to most relevant information which would, in turn, decrease information asymmetries and reduce capital costs [32].Many of these incentives do not need to be costly for the public sector [53].
Being able to rely on information about R and D and innovation directly provided by companies instead of capturing it through surveys (for example, the Community Innovation Survey [79]) as is currently done, would improve the knowledge available to public authorities and allow a much more accurate policy design [28].However, the precise mechanisms to be designed and the way to incorporate them into actual innovation policy are yet to be tackled in future research projects.
R3. Are there significant differences among the main players in the Spanish National Innovation system with respect to the variables to be tackled and the direction that innovation policy should take?
Regarding our third research question, it is interesting to note that, with few exceptions, there is big consensus in the recommendations made by the three groups (researchers, businessmen, and policy-makers), which means that they are talking "the same language" and that the research and innovation policy agenda can be developed in an interactive way.This is considered an advantageous situation by Leydesdorff and Etzkowitz who drew attention to it when they first wrote about the Triple Helix framework [33].

Final Remarks
Our intention has been to identify the main trends and changes needed in Spanish innovation policies to catch up with other better performing economies in this respect.Given the relevant role that innovation plays in growth and competitiveness, a shift in policies in the right direction should help to overcome the current economic crisis.
Having in mind the complexity of the system being dealt with, the multiplicity of agents involved and the path-dependency that characterizes the system, a Delphi methodology was found to be the most appropriate to deal with our research questions.In order to have inferential analysis and arrive at more robust conclusions we have applied for the first time to our knowledge a non-parametric bootstrap method to analyze the results of a Delphi exercise concerning innovation.
Our research has contributed to the literature in several ways.We have identified necessary changes in innovation policy to be addressed in Spain (R1), such as focusing attention on SMEs' potential innovators and traditional industries and on framework conditions through open innovation.High levels of consensus have been reached among the three main players of the innovation system (R3).Particularly interesting is the need for companies to manage and report their intangibles as a way to improve the likelihood of innovation funding (R2).This issue has not received enough attention in the literature and deserves further research.A promising line would be to test the hypothesis that firms that manage their intellectual capital are more productive than those that do not, making them more attractive to potential investors.
The main limitations are related to the scope of variables and issues which have an influence on innovation and may be affected by policy action.Choices had to be made to arrive at questionnaires of reasonable length and because of this, important elements of the national innovation system have not been completely addressed, maybe hampering its full validity, but clearly establishing a promising line of research.This is the case for instance of interface institutions which favor knowledge exchange between academia and industry or the research and innovation priority setting mechanisms in the public sector.We also have to put a word of caution on the applicability of the results since they are related to Spain, which also has forced for the sample to be somewhat small.Finally, the paper shows that it is possible to improve the Delphi methodology and, thus, allow inferential analysis to increase the reliability of the results.Step 5: The algorithm The algorithm for the non-parametric bootstrap is further specified in pseudocode format: 1.
Define the null H 0 and alternative hypothesis H 1 .

2.
Calculate the statistic t j out of the observed sample and H 0 .a For each bootstrap sample i Sample with replacement after assigning a probability of 1/I to each value of the observed sample ii Calculate the statistic T j r (or, alternatively F j r ) out of the bootstrap sample, assuming that the null hypothesis is true (e.g., ensuring it is centered on zero).

3.
Assign significance to the p-value (The p-value is defined as the probability of obtaining a statistic at least as extreme as the observed value, given that the null hypothesis is true.In quantitative terms, the p-value is the probability of being inside the interval [t j ,8) if t j > 0, or the probability of being in the interval (´8,t j ] if t j < 0) if p-value < α; for, α = 0.01, 0.05, 0.1.
The procedure has to be adapted therefore according to the null hypothesis that is being tested.The specific calculation of the statistics t j and T j r follows accordingly.
iii Calculate the indicator by comparing the bootstrap statistic and the observed threshold: I r (‚).The indicator will be equal to one if the condition within the parenthesis is verified and 1 otherwise[10]: I r p‚q "

Table 1 .
Distribution of level of agreement/disagreement (% of propositions).

Table 2 .
Changes of consensus over Delphi rounds (% of propositions).

Table 3 .
Differences of opinion among groups of experts.
and D activities, but primarily in private sector;The main focus of the questionnaire regarding demand-side activities fostering the use of information technologies in firms and households.The main results are:‚weak agreement towards establishing incentives for SMEs and households to invest more in the information society, but no agreement on extending this policy to large firms; and ‚ no agreement towards fiscal incentives (such as VAT or income tax)From the point of view of the provision of constituents for systems of innovation, there are interesting lessons regarding entrepreneurship and networking.The main results are: ‚fostering institutions of excellence at the different education levels is controversial among the experts, particularly when talking of the secondary level; and ‚ increasing investment in technological equipment is more controversial.‚weakagreement towards specific incentives for spin-offs;‚ risk analysis of financial institutions should go beyond the financial statements of firms, to consider variables that capture the capacity to generate profits derived from the innovative potential of the applicants credit.