The Dysfunction of Mission-Oriented Innovation Policy: Impeding the Accumulation of Scientific Knowledge in the Japanese Academic Sector

Abstract

Since the Organisation for Economic Cooperation and Development presented its comprehensive Innovation Strategy in 2010, numerous countries have been updating their innovation policies. Subsequent to the promulgation, the innovation policies of Japan shifted the focus from discipline-specific to social issue-oriented approaches. This study investigates the response of the Japanese academic sector to this policy shift and the characteristics of the research projects associated with innovation policy by utilizing descriptive statistics from policy documents and the database of Grants-in-Aid for Scientific Research. The findings reveal that Japanese researchers have increasingly aligned their efforts with government-proposed research themes in recent years, with a notable shift toward short-term research projects. Moreover, Japanese universities are undergoing reforms that are transforming them into entrepreneurial institutions by altering incentive structures. Although these reforms may yield short-term research outcomes, they may not always address long-term societal needs. The narrowing focus on research themes could restrict the potential impact of research and impede the development of innovative solutions to societal challenges. From this viewpoint, assessing the relationship between government-proposed research themes and the research productivity of Japan is critical. Universities and public research institutions play a vital social role in broadening the foundational knowledge base through basic research, while private enterprises may lack the motivation to invest in research and development with low appropriability. These results may be beneficial for policymakers in reconsidering the division of labor in industry-academic collaboration in a knowledge-intensive economy.

1. Introduction

The Innovation Strategy 2010 of the Organisation for Economic Cooperation and Development (OECD) has increased the importance of the demand side of innovation policy. Although it emphasized that science remains an important component of innovation, the plan presented a systematic approach that includes not only supply-side policies focused on the development of specific technologies but also demand side ones, including economic and social issues and the broad range of actors that influence innovation (OECD 2010). In response to these trends, countries worldwide are developing mission-oriented innovation policies (MOIPs) that aim to create markets through the resolution of societal challenges in collaboration with various actors (Mazzucato 2016, 2018). Examples of MOIP include the Community Wealth Building program introduced in Preston, the United Kingdom, in 2015 (The Lancet Public Health 2023), policies affecting the demand and supply of Responsible Innovation in Health in Brazil and Canada (Lehoux et al. 2023), the Nutrient Recycling policy in Finland (Nylén et al. 2023), and the Vision Zero initiative of Sweden as a mission-oriented innovation policy for traffic safety (Craens et al. 2022).

The innovation policy in Japan, which has been formulated every five years since 1996, is also transitioning toward a mission-oriented approach (Larrue 2021; Carraz and Harayama 2018). The Fourth Science and Technology Basic Plan (2011–2015) shifted the focus of the agenda from discipline-specific to social and environmental challenges after noting that the outcomes of science and technology (S&T) were not being effectively utilized for the creation of new industries, employment, national welfare, or the resolution of social issues (Cabinet Office 2011).

Policymakers of MOIPs argue that the government should not only fund but also play an active role in the formation of market directions and the creation of new markets (Mazzucato 2014). Thus, the role of academia in bridging science and society to address social challenges and goals has attracted attention. However, debates over the approach of the government to evaluating university research remain active (Smith et al. 2011; Torrance 2020; Martin-Sardesai et al. 2017; Morrish 2020; Hudson and Williams 2016; Varghese and Martin 2014). Government interventions in academic institutions may have several potential consequences for their autonomy. For instance, if public and private funding bodies prioritize short-term results, then researchers may favor projects that yield quick outcomes, which may not always address long-term societal needs. Moreover, the lack of diversity in research themes could limit the potential impact of research and hinder the development of innovative solutions to societal challenges. Therefore, critically assessing the relationship between government-proposed research themes and research productivity is important.

The current study aims to explore the alignment between Japanese researchers and government research themes and their implications for potential societal impacts. Specifically, how did the academic sector in Japan respond to the shift in innovation policy? What are the characteristics of a research project for which researchers voluntarily apply? By seeking the answer to these research questions, this study proposes potential issues inherent in Japan’s MOIPs within the academic sector.

For government-led MOIPs to successfully address complex social and economic challenges, cooperation and coordination with various nongovernment stakeholders are essential. In particular, ensuring effective coordination and communication across sectors presents a significant practical challenge (Sabel and Zeitlin 2008). This study focuses on the responses of universities to MOIPs among key actors in the national innovation system (NIS) in Japan. By presenting challenges faced in the process of establishing and achieving joint goals among multiple stakeholders, it intends to offer valuable insights for policymakers and researchers.

The remainder of the paper is structured as follows: Section 2 reviews the previous literature on topics related to MOIPs and their intervention in academic freedom. Section 3 describes the Japanese context with regard to the transformation of innovation policy and university reform. Section 4 describes the materials and methods for data analysis. Section 5 provides an overview of the results derived from descriptive statistics. Section 6 features a discussion based on the fact finding. Finally, Section 7 concludes and presents implications.

2. Literature Review

2.1. Evolution of Innovation Policy

Innovation policy has evolved significantly over time, and such an evolution has been closely linked to the broad systemic understanding of innovation. In recent years, scholars in the field of innovation policy have discussed it in three phases (Diercks et al. 2019; Schot and Steinmueller 2018).

The first phase was the S&T policy, which occurred after World War II. Based on the idea of a linear model of innovation, the S&T policy was designed to promote scientific research and technological development and to strengthen the S&T base of the nation, which translates into economic and social benefits. Economic growth through the provision of new knowledge to the private sector and by addressing market failures and externalities justified policy interventions to disseminate science to society (Schot and Steinmueller 2018). The specifics of this policy include increasing investment in research and development (R&D), strengthening S&T education, promoting cooperation between public and private research institutions, and developing a legal and regulatory framework that promotes innovation (Diercks et al. 2019; Schot and Steinmueller 2018). These initiatives were institutionalized as key elements for enhancing the international competitiveness of the country and supporting sustainable economic growth.

The second phase is the NIS, which was conceptualized during the 1980s under the influence of neoliberal thought and whose policy agenda focused on economic goals. The NIS is generally defined as an entire system that consists of various organizations and institutions of actors involved in innovation activities in a country (e.g., firms, universities, and government agencies) and their interactions (Freeman 1987; Lundvall 1995; Nelson 1993). The major focus has expanded to peripheral areas, such as building cooperative R&D networks, facilitating knowledge dissemination and technology transfer, developing infrastructure to support innovation, strengthening linkages across regions and industries, and facilitating learning and knowledge spillovers in developing countries (Mowery 1998; Block and Keller 2009; Hall and Lerner 2010; Cooke 2005; Malerba 2004; Lundvall et al. 2006).

Since the 2000s, innovation policy has transitioned into the third phase, which is considered a paradigm change in innovation policy due to the diversification of policy issues. For example, it has been labeled MOIPs (Mazzucato 2016), a novel policy paradigm (Kemp and Mainguy 2011), and transformative innovation policy (Diercks et al. 2019; Schot and Steinmueller 2018) and positioned as broad social issue-oriented innovation policies. Thus, the policy agenda at this phase emphasizes a shift from economic to contemporary social and environmental issues such as the SDGs, which emphasize a distinction from previous innovation policies (Schot and Steinmueller 2018).

However, as Diercks et al. (2019) argued, this new policy paradigm does not entirely replace previous policy paradigms such as the S&T policy and innovation system policies. As demonstrated by the evolution of debates on the NIS, different countries exhibit varied performance with respect to innovation due to their diverse institutions. Formal and informal constraints, institutional complementarities, and path dependence operate within the NIS, which underlies the reason why innovation performance does not converge over the process of globalization (North 1990). Therefore, a historical perspective that focuses on institutional continuities and discontinuities is essential for the analysis of how institutional change leads to outcomes.

2.2. Mission-Oriented Innovation Policy

Innovation policy issues have diversified over time. The focus of policy agendas has shifted from the production of scientific and technological knowledge based on a linear model of innovation in the 1960s to one that is economic, social, and environmental, as the understanding of innovation has expanded (Diercks et al. 2019). MOIPs, advocated in conjunction with this paradigm shift in innovation policy, constitute a policy approach that establishes clear goals and measurable targets to address societal challenges and promote innovation. This approach aims to mobilize a wide range of actors, including governments, businesses, and communities, to work together to achieve specific goals (Mazzucato 2016, 2018). However, the exploration of the effectiveness and scalability of MOIP continues, along with the challenge of integrating these policies into a broad policy framework and overcoming administrative, legal, and governance barriers (The Lancet Public Health 2023; OECD 2023). The MOIP concept is applied to various contexts that promote transformative change and address complex social issues, such as economic development, sustainability, and health (Nylén et al. 2023; Kirchherr et al. 2023; Janssen et al. 2023).

MOIPs face several major problems. The first is the selection and definition of goals. In the mission-oriented approach, determining the mission to choose is critical. Mazzucato (2018) provides guidance on how governments and policymakers should set specific, measurable, and achievable missions and how to incorporate these goals into the actual policy design. However, the selection and definition of specific goals are complex and typically require political and social consensus building. If goals are extremely broad and abstract, then identifying specific action plans and deliverables becomes difficult. Conversely, if goals are very narrow, then they may limit the potential for innovation. Furthermore, strong missions can lead to reduced technological diversity in the innovation system (Hekkert et al. 2020). The second is the direction of innovation. Mazzucato (2016) argues that governments should play an active role in the formation of market direction and the creation of new markets instead of merely serving as funders. This context denotes an entrepreneurial government. However, when determining the direction of innovation, governments need to consider the risk of government failure as well as market failure (Kärnä et al. 2023). Inappropriate goal setting and implementation plans can lead to wasted resources and inefficiencies. Thus, overcoming these problems and making MOIP successful will require ongoing evaluation and adjustment through a cumulative series of case studies on policy design and implementation.

2.3. Changing the Academic Sector

The trend in market co-creation by entrepreneurial states and various private sectors has led to increased demand and pressure on the academic sector from the economy and society. As a result, the academic sector is expected to support economic development, innovation, and social progress (Rostan 2010). Universities have been encouraged to develop new venture partnerships with the industry as an important driver of a knowledge-intensive economy (Olssen and Peters 2005; Godin and Gingras 2000). Furthermore, the interaction among industry, academia, and government (triple helix) has emphasized the role of universities as entrepreneurial universities, along with the government and industry, in technology transfer and the creation of knowledge-based startups (Etzkowitz and Leydesdorff 2000; Etzkowitz 2003; Lerman et al. 2021).

This trend is perceived as academic capitalism. In this context, a particular conception of the world was constructed in which knowledge is only valid if it is marketable, if researchers are entrepreneurs, and if higher education and research are in the service of global capitalism and neoliberal rationality (Bori and Block 2024). Moreover, public universities in the United States adopted the practices and norms of a business mindset that are more closely consistent with corporate goals and values instead of social institutions responsible for knowledge creation and dissemination (Spinrad et al. 2022).

The discussions on academic freedom and autonomy are related to the intervention of public institutions in the scholarly sciences, which leads to serious consequences for the creation, dissemination, and application of knowledge. By prioritizing marketable knowledge, recognizing researchers as entrepreneurs, and shifting toward subordinating higher education and research under global capitalism and neoliberal rationality, academic capitalism has severely affected the advancement of scientific knowledge and posed a major threat to the academic freedom of individual scholars (Walsh et al. 2007; Wieczorek and Muench 2023; Reichman 2022).

In addition, however, the prioritization of marketable knowledge under a strong mission can also hinder the accumulation of scientific knowledge. As Lin et al. (2010) point out, technological development that is dependent on innovation policies based on technological forecasts will not necessarily lead to national economic growth unless it is coordinated with basic research and the long-term economy (Lin et al. 2010). The reason is that basic research has high scientific value and knowledge spillover effects. Nevertheless, basic research explores the frontiers of knowledge; thus, the actual emergence of its results can take a long time. Therefore, examining the relationship between innovation policy and the accumulation of S&T is important. However, previous studies have provided insufficient discussion on this point. For this reason, the current study focuses on the behavior of researchers involved in research activities through their organizational incentives prior to patenting.

3. Japanese Innovation Contexts

The following text addresses the development of innovation policy in Japan using materials from the Cabinet Office and also describes the characteristics of university reforms at private and national universities.

3.1. Transition of the Science and Technology Basic Plan and Competitive Research Funds

The Basic Act on Science and Technology was enacted in 1995 (and revised in 2020) with the objective of improving S&T, economic, and social developments in Japan. Since 1996, the Science and Technology Basic Plan (Basic Plan) has been formulated and implemented every five years; the Sixth Basic Plan started in 2021 (Cabinet Office 2021). Figure 1 illustrates the shifts in the national initial budgets for the Basic Plan and competitive research funds and their fluctuation each year, which reflects consistent increases over time. Although the amount of competitive funds peaked under the Third Basic Plan, the ratio of the initial budgets has generally been 10%.

Figure 1. Shifts in science and technology budgets and competitive research funds between Basic Plans. (Source: Processed and created by the author based on the National Institute of Science and Technology Policy (NISTEP) and Ministry of Education, Culture, Sports, Science, and Technology (MEXT), “Digest of Japanese Science and Technology Indicators 2022” (NISTEP and MEXT 2022) and JSPS, “KAKENHI: New knowledge creation for the formation and inheritance of world-leading intellectual assets” (MEXT and JSPS 2021)).

Although several competitive research funds exist under the jurisdiction of ministries and agencies, the majority are short-lived. One of the major funds in Japan is KAKENHI, which was integrated into several research grants in 1965 with the aim of subsidizing creative and pioneering academic research that will contribute to the foundation of an affluent society. It is administered by the Japan Society for the Promotion of Science (MEXT and JSPS 2018). KAKENHI aims to fund all academic research, from basic to applied, in all fields, which accounts for approximately 50% of the total competitive research funds in Figure 1. This study examines the impacts of each Basic Plan with a focus on the Third and Fourth Basic Plans, on national research activities.

3.2. Transformation of S&T Basic Plans

The First Basic Plan (1996–2000) focused on the creation of intellectual assets through research by supporting 10,000 postdoctoral fellows and enhancing research in private universities, which comprise 80% of higher education in Japan. It also introduced measures for facilitating the transfer of university research to businesses and established technology licensing organizations (TLOs) to foster industry-academia collaboration. As such, TLOs play a central role as the driving force of the cycle of knowledge creation, which creates new industries that originate from universities and returns a part of the profits to researchers. In this manner, research funds are generated, and university research is further activated (METI 1998). The Second Plan (2001–2005) targeted specific areas in R&D, such as life sciences and nanotechnology. The Third Plan (2006–2010) expanded priorities to include energy and manufacturing, which promoted researcher mobility and gender diversity in research. The Fourth Plan (2011–2015) shifted the focus from academic disciplines to societal issues and emphasized green innovation and continued support for the tenure-track system. The Fifth Plan (2016–2020) aimed to promote innovation to address societal challenges and stressed the importance of collaboration among industry, academia, and government.

The innovation policy in Japan has long developed basic research capabilities that target key technologies that will drive future industrial growth. Until the Third Basic Plan, the national policy gradually expanded innovation efforts to incorporate diverse actors within industry-academia-government collaboration (Cabinet Office 2006a). The innovation policy then seemingly radically shifted to the mission-oriented approach since the Fourth Basic Plan (Larrue 2021; Diercks et al. 2019). Notably, however, each successive plan has consistently assumed that universities will play a central role in knowledge production for innovation, despite the diversification of the sites of knowledge production (Godin and Gingras 2000).

3.3. Japanese University Reforms

Since the 1990s, the environment for Japanese universities has significantly shifted. A report in 1991 highlighted the challenges posed by the declining youth population on university capacity, which had led to increased competition among universities, especially private ones, to attract students. This competition prompted schools to revamp faculties, admissions, and curricula (Morikawa 1994). In response, MEXT updated the University Establishment Standards to enable educational restructuring while maintaining control over the predominantly private higher education sector. The 2013 Comprehensive Support Program for Private University Reform aimed to stabilize finances and promote quality education by offering targeted subsidies for reformative efforts, which intensified grant competition among universities.

In 2004, the transformation of national universities into corporations (NUC) aimed to foster entrepreneurial activity, which enabled them to patent inventions and engage in dual employment. This reform resulted in increased patent applications and citations, which signifies success in enhancing the social impact of academic research (Motohashi et al. 2022). However, despite these reforms, MEXT retained significant control over universities and required them to align with policy agendas, especially in securing competitive external grants. A few critiques of the reform pointed to its negative impact on research infrastructure and activity due to the emphasis on external funding and competition (Yoshida 2007).

Recent reforms have likely shifted the focus of researchers toward short-term projects to secure funding, with an expected alignment with the keywords and trends of the Third and Fourth Basic Plans. This study aims to identify the trends in the responses of researchers to the keywords in the Third and Fourth Basic Plans by dividing them into five-year periods before, during, and after each plan.

4. Materials and Methods

4.1. Data Collection

Many methods can be used to achieve innovation policy objectives including economic growth, climate change prevention, improvement in population health (Fagerberg 2017), and the relationship between patents and innovation (Jaffe 2000; Koléda 2008). However, the current study focuses on research grants. A number of studies have used research grants as data. Research grants influence the number of subsequent publications (Hottenrott and Lawson 2017; Jacob and Lefgren 2011). Furthermore, research grants also affect the number of patents (Munari et al. 2024). Considering these studies, a potential pathway for innovation policy could be “government direction → research project → paper/patent → economic and social outcomes”. This study focuses on the first step, “government direction → research project”, because the reaction of researchers strongly influences the outcomes.

We selected two major innovation initiatives in Japan, namely, the Basic Plan and KAKENHI, as the research object, because the first forms the basis of Japanese innovation policy, and the second is one of the most major competitive research funds in the academic sector in Japan. In particular, we analyzed the five-year periods before and after the Third (2006–2010) and Fourth (2011–2016) Basic Plans, because the focus of the policy agendas between the two plans has shifted from specific disciplines (e.g., life sciences and manufacturing technology) to issues (e.g., green innovation). For the Third Basic Plan, we captured the trends in keywords for policy issues across a total period of 15 years: 2001–2005 (before), 2006–2010 (during), and 2011–2015 (after); for the Fourth Basic Plan, a total period of 15 years: 2006–2010 (before), 2011–2015 (during), and 2016–2020 (after; Figure 2).

Figure 2. Data collection period for the Third and Fouth Basic Plans.

The first step of data collection is described below. The materials we used are those of the Third and Fourth Basic Plans published by the Cabinet Office. Supplementary materials for each priority area, such as life sciences, information and telecommunications, environmental sciences, and nanotechnology/materials, were prepared at the same time as the S&T Basic Plan was announced up to the Third Basic Plan. Therefore, we mainly extracted keywords from the supplementary materials, referring to Chapter 2 of the Third Basic Plan entitled “Strategic Priority Setting on S&T” of the Third Basic Plan (Cabinet Office 2006a, 2006b). As for the Fourth Basic Plan, supplementary materials are no longer produced. Therefore, we derived keywords from Chapter 2 of the Fourth Basic Plan, “Achieving Sustainable Growth and Social Development in the Future.” (Cabinet Office 2011).

We extracted the keywords using the following procedure: First, the first author derived the keywords from the source documents. Then, the second author checked the keywords against the source documents. We carefully discussed the items that did not match or needed interpretation and determined the keywords. The first type of keyword can be considered an as-is keyword. This is a case where the keywords are the same words as in the original documents (e.g., “nanomedical technology” and “hydrogen transportation technology” in the Third Basic Plan, “smart grid” and “nanocarbon materials” in the Fourth Basic Plan). The second type of keywords are slightly modified keywords. These are keywords whose content is clear but can be created by simply removing particles or rearranging the order of words (e.g., “technology to support software development” → “software development support technology” in the Third Basic Plan, and “making lighting more efficient” → “high efficiency lighting” in the Fourth Basic Plan). The third type of keywords, which constituted the primary focus of our discussion, are those that must be determined from the context. In this study, we refer to these as contextual reading keywords. For example, we derived “local industry revival” from the original text: “The Great East Japan Earthquake caused extensive damage to primary industries such as agriculture, forestry, and fisheries, which are local industries, over a wide area along the coast of the Tohoku and Kanto regions. In light of this, the Basic Plan promotes R&D related to the investigation and improvement of contaminated soil and water quality, the restoration of marine ecosystems, the improvement of productivity and safety of agricultural, forestry and fishery products, as well as the use and application of the results of such R&D, with the objective of achieving the reconstruction, revitalization, and further growth of these industries” (Cabinet Office 2011). In this context, we could have extracted multiple synonyms, such as local industry reconstruction or local industry growth. However, we extracted only one keyword from each context to avoid biasing the data. In cases where the source text comprised multiple contexts, we extracted as many keywords as the number of contexts. Finally, 91 and 97 keywords were extracted from the Third and Fourth Basic Plans, respectively. Table A1 and Table A2 provide a comprehensive list of sources and all extracted keywords. Figure 3 illustrates the percentages of as-is keywords, slightly modified keywords, and contextual reading keywords in the Third and Fourth Basic Plans. The figure demonstrates that the keywords discussed in depth by the authors account for approximately 4.1% of the keywords included in the Fourth Basic Plan.

Figure 3. The percentages of extracted keywords by type in the Third and Fourth Basic Plans.

In the next step of the data collection, we extracted research projects from the KAKEN database that contained any of the keywords over the study period of the 15 years around the Third and Fourth Basic Plans. All data were collected in August 2022, and we searched for not only the titles and keywords of the research project but also summaries and full studies. The objective of this search was to capture data from research that mentioned the keywords, even if they were not the key topics of the research. Reports can also be searched in the KAKEN database. However, research projects whose keywords appear only in the reports were excluded from the sample in lieu of the abovementioned objective.

A few projects contained multiple keywords; however, we observed that not all of the keywords from the Third Basic Plan were incorporated into research. Specifically, researchers used all 97 keywords from the Fourth Basic Plan but failed to incorporate 15 out of 91 from the Third. We identified a total of 116,938 research projects that incorporated at least one of the keywords from either of the Basic Plans (Third: 42,456; Fourth: 74,482). Figure 4 presents the breakdown of the KAKENHI grants into five-year periods that reflect those from the Third and Fourth Basic Plans.

Figure 4. Number of KAKENHI projects for each five-year period.

4.2. Data Analysis

By comparing the number of projects adopted by KAKENHI during the Third and Fourth Basic Plan periods with those of the five years before and after the relevant Basic Plan period, we intended to elucidate the response of the Japanese academic sector toward the shift in innovation policy. First, we respectively compared the numbers of research projects before, during, and after the Third and Fourth Basic Plans by calculating the highest average number of KAKENHI projects during each five-year period to calculate the research volume per period. However, the average spanned 5 years and did not reflect individual peaks in the number of projects. Therefore, we identified the peak year for research projects related to the keywords that showed the highest average in each five-year period and organized the location of the research projects for each period. The abovementioned cross-tabulation of the research volume and peak year locations yielded a 3 × 3 matrix (Table 1).

Table 1. Cross-tabulation of the research volumes and peak years for KAKENHI projects.

	Peak in the Number of Projects
Highest average number of projects		Five years before Basic Plan	Five years during Basic Plan	Five years after Basic Plan
	Five years before Basic Plan
	Five years during Basic Plan
	Five years after Basic Plan

Second, by comparing periods with the peaks in the number of projects across the 15-year period, including the five-year period before and after each Basic Plan, we examined the degree of progress of the research projects. An S-shaped relationship between the amount of resources invested in technology and its consequent technological performance empirically represented technological progress (Foster 1986). In particular, technological progress was initially slow due to a lack of knowledge about the technology but progressed rapidly with the accumulation of knowledge. However, technology will eventually face its limits, and the pace of technological progress will slacken as large investments no longer produce the expected results. In other words, if KAKENHI-funded research projects have advanced over the long-term, then the number of projects should have peaked after each Basic Plan period.

Based on the abovementioned discussion and expectations, if the government has misspecified the research keywords of the Basic Plan, then many studies would be observed in the upper-left corner (before × before). If the Basic Plan had set keywords for research that would only produce long-term outcomes, such as the basic sciences, then the research volume would be higher in the lower-right corner (after × after). On the contrary, if researchers myopically responded to the keywords, then the research volume would settle in the middle (during × during).

5. Results

5.1. Responses of Researchers to the Innovation Policy

To explore the responses of Japanese researchers to the transformation of Japanese innovation policy, we calculated descriptive statistics. The numbers on the vertical axes of Table 2 and Table 3 indicate the number (percentage) of keywords in the Basic Plan and research projects adopted by KAKENHI within each period when the five-year average of KAKENHI-adopted projects related to the keywords was the highest over a total of 15 years, including the five years before and after the Basic Plan period. Simply put, if researchers agilely responded to the keywords in the policy agenda, then the number (percentage) of KAKENHI adoption should have been larger during the relevant period of the Basic Plan compared with those in the periods before and after.

Table 2. Responses of researchers to 76 out of the 91 keywords in the Third Basic Plan.

	Peaks in the Number of Projects
Highest average number of projects		Before the Third (2001–2005) 8 (10.5%) 62 (0.1%)	During the Third (2006–2010) 21 (27.6%) 4035 (9.5%)	After the Third (2011–2015) 47 (61.8%) 38,359 (90.4%)
	Before the Third (2001–2005) 6 (7.9%) 32 (0.1%)	6 (7.9%) 32 (0.1%)	0	0
	During the Third (2006–2010) 18 (23.7%) 2219 (5.2%)	1 (1.3%) 25 (0.1%)	15 (19.7%) 521 (1.2%)	2 (2.6%) 1673 (3.9%)
	After the Third (2011–2015) 52 (68.4%) 40,205 (94.7%)	1 (1.3%) 5 (0.0%)	6 (7.9%) 3514 (8.3%)	45 (59.2%) 36,686 (86.4%)

Note: In the upper row, each cell presents the number of keywords in the Basic Plan and their percentages. The lower row represents the number of KAKENHI-adopted projects and their percentages.

Table 3. Responses of researchers to 97 out of 97 keywords in the Fourth Basic Plan.

	Peaks in the Number of Projects
Highest average number of projects		Before the Fourth (2006–2010) 9 (9.3%) 639 (0.9%)	During the Fourth (2011–2015) 49 (50.5%) 16,311 (21.9%)	After the Fourth (2016–2020) 39 (40.2%) 57,532 (77.2%)
	Before the Fourth (2006–2010) 7 (7.2%) 568 (0.8%)	5 (5.2%) 505 (0.7%)	1 (1.0%) 21 (0.0%)	1 (1.0%) 42 (0.1%)
	During the Fourth (2011–2015) 72 (74.2%) 48,832 (65.6%)	3 (3.1%) 128 (0.2%)	47 (48.5%) 16,248 (21.8%)	22 (22.7%) 32,456 (43.6%)
	After the Fourth (2016–2020) 18 (18.6%) 25,082 (33.7%)	1 (1.0%) 6 (0.0%)	1 (1.0%) 42 (0.1%)	16 (16.5%) 25,034 (33.6%)

Note: In the upper row, each cell presents the number of keywords in the Basic Plan and their percentages. The lower row denotes the number of KAKENHI-adopted projects and their percentages.

Table 2 indicates that KAKENHI-adopted 40,205 research projects that pertained to 52 out of the 76 keywords in the five years after the Third Basic Plan period; that is, 68.4% of the total number of keywords and 94.7% of the total number of KAKENHI projects were highly concentrated in the five-year period after the Third Basic Plan. In other words, researchers did not always quickly respond to the keywords in the policy agenda within the period of the Third Basic Plan. In contrast, Table 3 implies that during the Fourth Basic Plan period, a convergence occurred between the keywords and KAKENHI research projects, which indicates that researchers responded more quickly to the keywords in the Fourth Basic Plan.

If the outcomes of research conducted on a particular keyword in the Basic Plan are positive, then the research could continue over the long-term, such that the peak in the number of KAKENHI adoptions should occur after the relevant Basic Plan period. For instance, in the Third Basic Plan, 45 out of 76 keywords (e.g., “next-generation network technology”, “biomass utilization technology”, “innovative materials technology”, “hydrogen transportation technology”, and “X-ray free electron laser”) were continued in 36,686 (86.4%) KAKENHI projects even after the Third Basic Plan period (Table 2). This finding implies that progress in research related to discipline-specific keywords takes longer.

However, a certain degree of controversy exists regarding this part of the Fourth Basic Plan. Specifically, 47 out of 97 keywords (e.g., “medical care in disaster areas”, “medical institution networks”, “nursing care in disaster areas”, “advanced water treatment technology”, “information and communication technology in disaster areas”, “local industry revival”, and “highly insulated housing”) reached the peak number of KAKENHI adoption during the period, and 32,456 (43.6%) KAKENHI research projects related to 22 (22.7%) keywords (e.g., “imaging technology”, “renewable energy”, “nucleic acid medicine”, “telemedicine technology”, “power semiconductors”, “green sustainable chemistry”, and “hydrogen station”) were continued after the Fourth Basic Plan period (Table 3). We compared the aforementioned keywords that peaked during and after the Fourth Basic Plan period and found that those that peaked during the period were mainly related to the social issues at the time. Alternatively, those that peaked after the period were related to relatively general-purpose technologies. This trend suggests quicker outcomes from research based on keywords related to social issues than those based on specific fields. A potential reason is that the former is designed to be implemented as quickly as possible. Table 2 and Table 3 demonstrate that researchers responded more quickly to research projects on policy issues during the Fourth Basic Plan period than those in the Third Basic Plan period. Moreover, the periods of individual research tended to be shorter during the Fourth Basic Plan period than those during the Third Basic Plan period. Thus, the study infers that the behavior of researchers who immediately responded to the keywords of the Fourth Basic Plan became increasingly myopic.

5.2. Progress of KAKENHI-Adopted Research Projects

Figure 5 and Figure 6 illustrate the changes in the total numbers of KAKENHI adoption within the quadrants of during × during and after × after, which cover five years before and after the relevant Basic Plan period based on Table 2 and Table 3. In particular, we focused on the research trends in each quadrant, which indicated the rapid responses of researchers to the keywords and the progress of KAKENHI projects.

Figure 5. Trends in KAKENHI research projects related to the keywords in the Third and Fourth Basic Plans (during × during). Note: Left: Third Basic Plan; right: Fourth Basic Plan.

Figure 6. Trends in KAKENHI research projects related to the keywords in the Third and Fourth Basic Plans (after × after). Note: Left: Third Basic Plan; right: Fourth Basic Plan.

Figure 5 indicates the progress of research in which researchers rapidly reacted to the keywords during the relevant Basic Plan period and the number of KAKENHI-adopted projects that peaked during the same period. In the figure, the results for both Basic Plan periods denote an approximate inverted U-shape.

In contrast, Figure 6 presents a notable difference: more researchers reacted to the keywords after the relevant period, and the number of KAKENHI-adopted projects peaked after the relevant period. However, although the number of researchers that responded to the keywords of the Third Basic Plan continued to increase, the number of studies during the Fourth Basic Plan began to decrease. In other words, the number of KAKENHI-adopted projects in the Third Basic Plan, which stressed an academic field-specific approach, increased even after the period. However, the number of such projects during the Fourth Basic Plan, which prioritized the social issue-based approach, began to decline after the period. This result can be interpreted in two ways.

The first is that research projects related to academic field-specific keywords hold more potential for scientific research than for social implementation. In other words, researchers applied for research projects related to the Fourth Basic Plan in their application areas to obtain funding to continue basic research. As a result, they lost interest in projects related to social issues, which are oriented toward social implementation after achieving their objectives. The second interpretation is that research projects related to solving social problems are short-lived because they end with the development of practical knowledge. Hence, the speed of responses to policy challenges becomes significant.

6. Discussion

We analyzed the change in the focus of the Basic Plan of Japan, that is, from the academic field (Third Basic Plan) to solutions to social issues (Fourth Basic Plan) and ongoing university reforms toward entrepreneurship. We inferred from such changes in the environment surrounding the academic sector that recent university reforms have likely shifted the focus of researchers toward short-term projects to secure funding, with an expected alignment with keywords and trends from the Third and Fourth Basic Plans. We reasoned that although the increased mobility of human resources and the introduction of competition in the academic sector would lead to improved research performance, researchers would be more inclined to select research that would yield results in the short-term. Therefore, this study aimed to identify trends in the responses of researchers to the keywords in the Third and Fourth Basic Plans by dividing them into five-year periods before, during, and after each plan. The result of tracking the keywords of the Basic Plans in the KAKENHI-funded projects demonstrated that keywords in the Third Basic Plan exhibited a tendency to continue over the long-term after the plan, whereas those of the Fourth Basic Plan depicted short-term trends.

6.1. Contributions

This study draws on three contributions from previous research on MOIPs and academic capitalism. The first is the importance of long-term research at the frontier of knowledge. The study found that researchers increasingly tended to select research projects that would produce results in the short-term under the Fourth Basic Plan, which sought to address social issues. The shift toward entrepreneurial universities could devalue and hinder the production and accumulation of new knowledge as a public good. Salter and Martin (2001) explained the justification for the promotion of industry-academia collaboration as follows: profit-seeking companies lack the incentives to invest in nonmarketable basic research; therefore, universities are expected to be responsible for creating new knowledge as a public good (Salter and Martin 2001). However, the shift in R&D activities to applied research and projects designed for social implementation could discourage the accumulation of basic scientific knowledge through industry-academia collaboration, as evidenced by the short-term research progress under the Fourth Basic Plan. This finding suggests that policy intervention in academic freedom not only restricts the range of research choices but also hinders the accumulation of long-term research, as in the basic sciences. The current results are consistent with this argument.

The second is the importance of the impact of changes in the incentive structure on the behavioral changes of people. The results demonstrate that the agenda under the Fourth Basic Plan has promoted the transformation of higher institutions of education in Japan from primarily academic to entrepreneurial universities. The introduction of NUCs, expanded competitive research funds, and tenure-track systems have provided incentives—and consequently changed the behaviors of—the Japanese academic sector to select research with likely short-term outcomes (Suzuki et al. 2021). If these changes in the institutional incentive structure did not accompany the transformation of innovation policy, then researchers might have responded differently. Nevertheless, the Japanese academic sector has been responsive to changes in innovation policy due to these factors. This result indicates that the NIS not only poses institutional complementarities, such as changing innovation policies, but also provides incentives for scholars to change their behavior to obtain external research funding (North 1990).

The third is the path-dependent nature of the institutions. The Fourth Basic Plan shifted the focus of the policy agenda from academic discipline-specific to social issue-based approaches but retained the traditionally narrow understanding of the innovation process as deriving from technological revolutions driven by universities and companies. As Larrue (2021) pointed out, MOIPs in Japan need to be complemented with additional bottom-up, issue-based initiatives. The aim of the MOIPs in Japan since the Fourth Basic Plan has been to resolve the fragmentation among scientific knowledge, social issues, and the lack of coordination across the innovation system of Japan to proactively address social challenges. However, the current scenario is that of a top-down MOIP built on a long history of goal-oriented policy interventions in Japan. This aspect demonstrates the diversity of MOIPs, which reflects the path-dependent nature of the institutions, including norms and practices that structure repetitive interactions among people (North 1990). The current results are supportive of the arguments of previous studies.

6.2. Limitations

This study has its limitations. The first is survivorship bias. The KAKEN database only displays research projects that were adopted. Thus, identifying the entire trend in the Japanese academic sector, including the number of rejected projects, is impossible. Therefore, another limitation is that the study only displays the characteristics of adopted research projects.

The second is consistency between the plans and results of KAKENHI, which amount to a large number of adopted KAKENHI projects per year. Researchers use catchy words as rhetoric in their applications to obtain research funding. However, the actual research content and results do not always match the keywords. Moreover, analyzing the consistency between the contents of the research plan and the results of the research is extremely difficult.

6.3. Future Research

This study has three future research directions. The first is to adopt a qualitative content analysis approach (Krippendorff 2004; Harwood and Garry 2003). This study aimed to examine the changes in the immediate responses of researchers to the policy change. In future research, we will create second-order constructs based on the keywords in this paper, using the criteria of research area, basic and applied research, and research topic, and analyze the speed of response for each second-order construct.

Second, we intend to determine the path dependence of changes in the transformative Japanese innovation system by examining not only the formal rules of the systems but also the roles of informal constraints and their complementarities across institutions (Hall and Soskice 2001; North 1990, 2005; Suzuki et al. 2021). For instance, we intend to explore the effect of public opinion as an informal constraint on innovation on policies and institutions.

We also aim to conduct a network analysis of collaborative research in the KAKENHI-adopted research projects examined in this paper. If solving social challenges entails diverse collaborations, then a possibility exists that the collaborative research network of the Fourth Basic Plan (social issue-based approach) is wider than that of the Third Basic Plan (academic field-specific approach). For this future research, we will use the KAKEN data on the researchers involved in each research project and their affiliated institutions, then compare the research communities in both Basic Plan periods.

7. Conclusions and Implications

This study aimed to explore the alignment between Japanese academics and the research agenda of the government and its implications for potential social impacts. Specifically, we investigated the responses of the Japanese academic sector to changes in innovation policy and the characteristics that can be observed in KAKENHI research projects that are consistent with the Basic Plan. This section concludes our findings on these research questions and poses potential issues inherent to MOIPs in Japan within the academic sector.

This study was conducted with descriptive statistics of a total of 188 keywords and 116,938 research projects extracted from policy documents (Basic Plan) and the KAKENHI database to investigate the responses of the Japanese academic sector to the shift in innovation policy and the characteristics of research projects related to the innovation policy. The findings reveal that Japanese scholars in recent years have become increasingly consistent with the research themes proposed by the government, especially in their focus on short-term research projects. Moreover, Japanese universities are in the process of transforming into entrepreneurial institutions by changing incentive structures. Although these series of transformations may yield short-term research results, they do not necessarily address long-term social challenges. Moreover, policy intervention in research themes that target social implementation can not only narrow the focus of research projects but also impede the accumulation of research at the knowledge frontier. Thus, from a long-term perspective, the MOIPs may dysfunction, which may limit the potential impact of research and hinder the development of innovative solutions to societal challenges.

As a practical implication, the findings may help policymakers reconsider the division of labor between industry and academia in a knowledge-intensive economy. Although MOIPs advocate market co-creation by an entrepreneurial state and various private sectors, they will not lead to sustainable development of the innovation ecosystem if mistakes emerge in the establishment of the division of labor among universities and public research institutions, whose social role is to expand the fundamental knowledge pool of the nation.