Uncertain Public R&D Project Portfolio Selection Considering Sectoral Balancing and Project Failure

Abstract

In order to promote scientific and technological innovation and sustainable development, public funding agencies select and fund a large number of R&D projects every year. To guarantee the performance of the resulting project portfolio and the government’s investment benefits, the decision maker needs to select appropriate projects and determine a reasonable funding amount for each selected project. In the process of project selection, it is necessary to consider the balance of funding allocated to different scientific sectors as well as the failure probability of the projects in future execution, so that the expected performance of the project portfolio is maximized as much as possible. In view of this, we propose and study the uncertain public R&D project portfolio selection problem considering sectoral balancing and project failure. We formulate a stochastic programming model for the problem to support the portfolio decisions of the funding agencies. We also transform the model into an equivalent deterministic second-order cone programming model that can be directly solved by exact solvers. We generate datasets reflecting different scenarios through simulation and perform computational experiments to validate our model. The impacts of various factors (i.e., the number of project proposals, project failure probability, the upper limit of the budget allocated to each project, and the decision maker’s tolerance for project failure) on the project portfolio performance are analyzed.

1. Introduction

Science and technology play a key role in socially sustainable development. Governments of various countries have invested a lot of money in the form of public R&D projects to promote scientific and technological innovation [1]. For example, the financial investment of the National Natural Science Foundation of China (NSFC) has increased from 80 million RMB in 1986 to 28 billion RMB in 2018 [2]; China’s total R&D investment reached 2786.4 billion RMB in 2021 [3]. The amount of capital investment received by each approved project has an impact on its performance, which in turn affects the effectiveness of government capital investment [4]. Since the total amount of funds available to the project funding agencies is limited, when the funding agencies fund R&D projects, they need to select a portfolio of projects with high potential from a large number of project proposals and allocate a reasonable budget for each selected project. As a result, the sustainability and efficiency of public funds are improved, and the project portfolio’s performance is maximized [5].

When the government funding agencies select the portfolio of R&D projects, it is necessary to ensure the balance of budget allocation among different sectors or scientific disciplines, such that a certain degree of fairness is ensured. Typical ways of balancing budget allocation include limiting the proportion or amount of funds allocated to different sectors [6,7] and limiting the number of projects funded in different sectors [8]. Karsu and Morton propose an imbalance indicator that measures the difference of a distribution from an ideally balanced distribution and define a bi-criteria framework for trading balance off against efficiency in resource allocation problems [9]. However, these studies are based on the assumption that the decision maker knows the optimal balanced allocation results. In practice, it is also difficult for the decision maker to determine the balanced allocation in advance. Çağlar and Gürel propose a two-stage model in which they make sectoral budget allocation decisions in the first stage to maximize the total impact of the budget while ensuring the relative balance among sectors, and in the second stage they maximize the total score of the supported projects under allocated sectoral budgets [10]. Both Mavrotas and Özpeynirci et al. develop interactive approaches that require the decision maker to perform pairwise comparisons among alternative allocation schemes [11,12]. In addition, the budget allocation can also be improved from the perspective of project resource allocation by using the fuzzy sets theory [13,14,15].

The projects are selected before they are executed, and some of the selected projects may fail during the long time period from the fund allocation to their completion. Therefore, when constructing the project portfolio, the government funding agencies may benefit from considering the probability of different projects failing in the future. In fact, in the project environment, researchers have developed many methods to estimate different types of likelihood [16]. For example, Kahraman and İhsan employ the concept of probability of a fuzzy event to perform investment analysis [17]. Fuzzy numbers are used to model uncertain activity times and their distributions [18,19]. Balali et al. use artificial neural networks to predict the cost performance indices of a project [20]. In the field of public R&D project portfolio selection, only a few studies have considered project failure. To solve a public R&D project portfolio selection problem with cancellations, Çağlar and Gürel develop a robust optimization model for the case that the cancellation probabilities are unknown, and a stochastic optimization model for the case that the cancellation probabilities can be assessed [21]. Mohagheghi et al. propose project resilience as a key criterion in the large-scale construction project selection problem [22]. Gouglas and Marsh study the optimal portfolio solutions of the vaccine technologies by estimating their probability of success for rapid response to emerging infections [23]. Some studies also consider project failure in the private R&D project portfolio selection field. Zhao et al. build a mixed-integer nonlinear programming model to maximize the expected net present value for the selection of projects under variable task failures [24]. Gökalp and Branke consider a pharmaceutical R&D pipeline management problem under two significant uncertainties, i.e., the outcomes of clinical trials and their durations, and they present an approximate dynamic programming approach to solve the problem [25]. Tamošaitienė et al. discover that project risks might cause firms to end up facing big losses or even failure, hence they use the risk and return trade-off in choosing their portfolios of projects [26].

In addition, in the field of project portfolio selection, whether it takes public or private R&D projects as the research object, there are many studies that consider the interrelationships and interactions between projects. There are often substitutional or complementary relationships between some projects, and the outcomes or technologies of the projects influence each other. The interrelationships make the performance of the project portfolio a nonlinear function of the selected projects and further increase the complexity of the R&D portfolio selection. In the field of public project portfolio selection, Fernandez et al. consider the interactions and synergies between projects and the decision maker’s preferences when solving the problem of allocating public funds to competing programs or projects [27]. Arratia-Martinez et al. take into account portfolio balancing rules and interdependence between tasks and/or projects and solve the R&D project portfolio selection problem under uncertainty [28]. Wei et al. construct a co-citation network to simulate project interactions and study the ways in which project interactions influence the final values of project portfolios [29]. Delouyi et al. use network mapping to visualize project interdependencies and improve the quality of the dynamic portfolio selection decision in cross-country gas transmission projects [30]. In the private project portfolio selection field, both Pérez et al. and Kumar et al. believe that uncertain synergies and incompatibilities exist between projects, and they handle these uncertainties using fuzzy parameters [31,32]. Ghasemi et al. believe project interdependencies and cause–effect relationships between risks create complexity for portfolio risk analysis, so they present a model using a Bayesian network for modeling and analyzing portfolio risks [33].

Therefore, in project portfolio selection, there are relatively few studies on public R&D projects.

Table 1. Representative studies in project portfolio selection.

Studies	Sectoral Balancing	Project Failure	Interrelationships	Uncertainty	Project Type
[27,29,30]			√		Public
[31] *, [32] **, [33] **			√	√	Private
[6,8,9,10,11,12]	√				Public
[21] **, [22] *, [23] **		√		√	Public
[7] *, [28] *	√		√	√	Public
[24] **, [25] **, [26] **		√		√	Private

*: Fuzzy numbers [13,14,15,17,19] are used to model uncertain parameters. **: Stochastic variables [18] are used to model uncertain parameters.

summarizes the representative studies mentioned above. It can be seen that, both in deterministic and uncertain environments, no research has yet considered sectoral balancing and project failure. This paper aims to fill this gap. The main contributions of this paper are as follows: (1) We propose and study the uncertain public R&D project portfolio selection problem, considering both sectoral balancing and project failure (URDPS). (2) We formulate a stochastic programming model for the URDPS. We then transform the stochastic programming model into an equivalent deterministic second-order cone programming model such that it can be solved by commercial solvers (e.g., CPLEX). (3) We perform computational experiments to validate our model and analyze the impacts of the number of project proposals, project failure probability, the upper limit of the budget allocated to each project, and the decision maker’s tolerance for project failure on the portfolio performance.

The remainder of this article is structured as follows. In Section 2, we describe the uncertain public R&D project portfolio selection problem considering both sectoral balancing and project failure. Section 3 presents the stochastic optimization model and its equivalent deterministic second-order cone programming model. In Section 4, we analyze our model based on computational experiments. Section 5 offers conclusions and suggestions for future work.

2. Problem Statement

The funding organization receives $\left|J\right|$ project proposals ($J=\left\{1,2,\dots ,\left|J\right|\right\}$) and needs to select a subset of the projects to fund. The selected projects form a portfolio. There are $K$ sectors, and each project belongs to a single sector. For example, the NSFC’s funding covers nine sectors. Let $S\left(k\right)$ denote the set of project proposals under sector $k$. The funding organization has a budget 𝐵. The variable $B_k$ represents the budget for sector $k$, $k=1,2,\dots ,K$. $B_k$ is not greater than the sum of the required budgets for all project proposals, i.e., $B_k\le {\displaystyle \sum }_{j\in S\left(k\right)}{r}_j$. There are $K^{\prime }\le K$ sectors whose numbers of funded projects are limited to the interval $\left[m,n\right]$. Let $J_{k^{\prime }}^{\prime }$ denote the set of project proposals in this type of sector $k^{\prime }$ ($k^{\prime }=1,2,\dots ,K^{\prime })$.

Peer reviewers assess each project proposal $j$ and assign a score $w_j\in \left[w_{min},w_{max}\right]$ for $j$. If $w_j$ is lower than a pre-given threshold, project $j$ will not be funded. If $w_j$ is not lower than the threshold, the decision maker of the funding organization needs to further determine the actual fund allocated to project $j$. We use $x_j\in \left(0,1\right]$ to denote the proportion of the actual allocated fund to the fund $r_j$ claimed by the principal investigator, which is also used as an estimation for the future expected performance after project $j$ is finished. Therefore, the actual obtained fund for project $j$ is $x_j{r}_j$ and it has an upper limit of $R_j$. In addition, to ensure that project $j$ is funded enough to be executed, we provide a lower limit $b_j$ to $x_j$. For the convenience of subsequent modeling, we also introduce an auxiliary variable $f_j$ to indicate whether project $j$ is funded or not: if $x_j>0$, then let $f_j=1$; otherwise let $f_j=0$. It can be seen that the nonzero elements in the set $\left\{f_1,f_2,\dots ,f_{\left|J\right|}\right\}$ correspond to a project portfolio.

To ensure fairness, the funding organization considers sectoral balancing when selecting projects. A proper capital allocation [14] is essential for balancing the budget allocation. Sectoral balancing avoids situations where some sectors receive most of the funds and others receive little. We quantify the balance of budget allocation based on each sector’s historical contribution and impact ($e_k$) on society [10]. It is a trend in public R&D funding to assess the impact of each sector’s outcomes and decide the budget allocation accordingly. For example, Graham and Mackie shift 3.4% of the agency budget from lower-impact areas to higher-impact areas, which improves the public health impact of the resources [34]. Following Çağlar and Gürel [10], the impact $I$ of a portfolio is calculated as $I={\displaystyle \sum }_{k=1}^K{(e_k{B}_k)}^{1-\alpha }/\left(1-\alpha \right)$, where $\alpha \in \left(0,1\right)$ represents the disfavor of funding agencies to the inequities that result from disparate budgets across disciplines. For a given $\alpha$, the maximum value $I_{max}$ of $I$ can be obtained by solving the following nonlinear programming model [10]:

$$\mathrm{Maximize} I={{\displaystyle \sum }}_{k=1}^K\frac{{(e_k{B}_k)}^{1-\alpha }}{\left(1-\alpha \right)}$$

(1)

$\mathrm{Subject} \mathrm{to}:$

$${{\displaystyle \sum }}_{j\in S\left(k\right)}{r}_j\ge B_k, k=1,\cdots ,K,$$

(2)

$${{\displaystyle \sum }}_{k=1}^K{B}_k\le B,$$

(3)

$$B_k\ge 0, k=1,\cdots ,K.$$

(4)

where Constraints (2) are used to determine the total budget of each sector. Constraint (3) ensures the total allocated budget to all sectors does not exceed $B$. Constraints (4) define the range of the decision variables.

It is not uncommon for a project to be faced with uncertainties [13,17]. In our problem, due to the uncertainty of scientific research, each project $j$ has a probability ${\theta }_j$ of failure when executed in the future. If project $j$ fails, the actual budget used is $a_j{r}_j$, where $a_j$ is the proportion of the funds used at the time of failure to its grant amount. Let random variable $\widetilde{r_J}$ denote the actual budget used by project $j$ in its future execution, then we obtain

$$\widetilde{r_J}=\{\begin{array}{c}{r}_j\hspace{1em}\hspace{1em}\hspace{0.5em}w.p.\left(1-{\theta }_j\right)\hfill \\ a_j{r}_j\hspace{1em}\hspace{0.5em} w.p.{\theta }_j\hfill \end{array}\hspace{0.5em}\forall j\in J.$$

(5)

The funding organization wants to limit the number of failed projects in the portfolio, so the total number ${\displaystyle \sum }_{j\in J}{\theta }_j{f}_j$ of failed projects should not exceed $\zeta {\displaystyle \sum }_{j\in J}{f}_j$, where $\zeta \in \left[0,1\right]$ is set by the funding agency according to its needs and determines the maximum expected number of failed projects.

There are different relationships between projects. The relationships have different impacts on the ultimate performance of the corresponding portfolio. For example, co-funding complementary projects helps to increase the expected performance of the portfolio, whereas co-funding competing projects tends to decrease the expected performance of the portfolio. We consider the two types of relationships mentioned above when constructing the project portfolio. When the interrelated projects $u$ and $v$ are chosen simultaneously, the additional impact on the total score of the project portfolio is expressed as $o_{u,v}$. When $o_{u,v}>0$, it means that the selection of both projects $u$ and $v$ has a positive impact on the portfolio performance; otherwise, there is a negative impact. Due to the existence of the project failure probability, the expected additional impact of projects $u$ and $v$ on the total score of the project portfolio is $\left(1-{\theta }_u\right)\left(1-{\theta }_v\right)f_u{f}_v{o}_{u,v}$.

The public R&D project portfolio selection problem studied in this paper aims at constructing a portfolio with the maximum expected total score $z$ of the project portfolio while considering sectoral balancing, project failure, and project interrelationships. We express $z$ as $z={\displaystyle \sum }_{j\in J}\left(1-{\theta }_j\right)w_j{x}_j+{\displaystyle \sum }_{\left(u,v\right)\in S_l}\left(1-{\theta }_u\right)\left(1-{\theta }_v\right)f_u{f}_v{o}_{u,v}$, where ${\displaystyle \sum }_{j\in J}\left(1-{\theta }_j\right)w_j{x}_j$ is the expected total score under the condition of project failure without considering project interrelationships, ${\displaystyle \sum }_{\left(u,v\right)\in S_l}\left(1-{\theta }_u\right)\left(1-{\theta }_v\right)f_u{f}_v{o}_{u,v}$ is the added value of the expected score of the portfolio caused by the interrelationships between the projects, and $S_l\subseteq J\times J$ is the set of all possible pairwise projects with interrelationships.

3. Optimization Models

3.1. Stochastic Programming Model

We formulate the following stochastic programming model (SP) for the above problem:

$$(\mathrm{SP}) \mathrm{Maximize} z={{\displaystyle \sum }}_{j\in J}\left(1-{\theta }_j\right)w_j{x}_j+{{\displaystyle \sum }}_{\left(u,v\right)\in S_l}\left(1-{\theta }_u\right)\left(1-{\theta }_v\right)f_u{f}_v{o}_{u,v}$$

(6)

$\mathrm{Subject} \mathrm{to}$:

(2)–(4).

Constraints on the number of funded projects:

$${{\displaystyle \sum }}_{j\in J_{k^{\prime }}^{\prime }}{f}_j\ge m, k^{\prime }=1,2,\dots ,K^{\prime },$$

(7)

$${{\displaystyle \sum }}_{j\in J_{k^{\prime }}^{\prime }}{f}_j\le n, k^{\prime }=1,2,\dots ,K^{\prime }.$$

(8)

Constraints on the funding proportion:

$$f_j\ge x_j, \forall j\in J,$$

(9)

$$1-f_j+x_j\ge b_j, \forall j\in J.$$

(10)

Funding amount constraints:

$$x_j{r}_j\le R_j, \forall j\in J.$$

(11)

Sectoral balancing constraints:

$${{\displaystyle \sum }}_{k=1}^K\frac{{(e_k{B}_k)}^{1-\alpha }}{\left(1-\alpha \right)}\ge ϵI_{max}.$$

(12)

Chance constraints on budget:

$$P\left({{\displaystyle \sum }}_{j\in S\left(k\right)}\widetilde{r_J}{x}_j\le B_k\right)\ge \beta , k=1,\cdots ,K.$$

(13)

Constraints on the number of failed projects:

$${{\displaystyle \sum }}_{j\in J}{\theta }_j{f}_j\le \zeta {{\displaystyle \sum }}_{j\in J}{f}_j.$$

(14)

Range of decision variables:

$$f_j\in \left\{0,1\right\}, \forall j\in J,$$

(15)

$$0\le x_j\le 1, \forall j\in J.$$

(16)

In model SP, the objective function (6) maximizes the expected total score of the portfolio. Constraints (7) and (8) restrict the lower and upper limits of the number of funded projects in different sectors, respectively. Constraints (9) express the logical relations between decision variables $f_j$ and $x_j$. Constraints (10) indicate the lower limit of the funding proportion for each project. Constraints (11) are the upper limit for the funded amount of each project. Constraint (12) ensures that the degree of balance in budget allocation between sectors is not less than $ϵI_{max}$, where the balancing preference parameter $ϵ\in \left[0,1\right]$ reflects the degree to which the funding organization wants the balance of the budget allocation to reach its theoretical maximum value $I_{max}$. Chance Constraints (13) ensure that when the project is executed, the probability that the budget of each sector does not exceed the planned quota is greater than or equal to the confidence level $\beta$. Constraint (14) limits the number of failed projects. Finally, Constraints (15) and (16) define the range of the decision variables.

It should be noted that both Constraints (12) and (13) in SP are nonlinear. Therefore, SP is a nonlinear optimization model. To solve the model with exact solvers (e.g., CPLEX), we will transform SP into its equivalent deterministic second-order cone programming model in the next subsection.

3.2. Second-Order Cone Programming Model

By introducing auxiliary variables and second-order cone inequalities, we transform SP into an equivalent deterministic second-order cone programming model. First, proposition 1 transforms the nonlinear constraints in (13) into deterministic second-order cone constraints.

Proposition 1. 

The constraints in (13) are equivalent to the following constraints:

$$y_k=\frac{B_k-{{\displaystyle \sum }}_{j\in S\left(k\right)}{r}_j\left[1-{\theta }_j\left(1-a_j\right)\right]x_j}{{\Phi }^{-1}\left(\beta \right)}, k=1,\cdots ,K,$$

(17)

$${{\displaystyle \sum }}_{j\in S\left(k\right)}\left[r_j^2{\theta }_j\left(1-{\theta }_j\right)+a_j{r}_j^2{\theta }_j\left[a_j-2\left(1-{\theta }_j\right)-a_j{\theta }_j\right]\right]x_j^2\le y_k^2, k=1,\cdots ,K,$$

(18)

$$y_k\ge 0, k=1,\cdots ,K.$$

(19)

Proof. 

The expectation of $\widetilde{{\mathrm{r}}_{\mathrm{J}}}$ is $\mathbb{E}\left(\widetilde{{\mathrm{r}}_{\mathrm{J}}}\right)={\mathrm{r}}_{\mathrm{j}}\left(1-{\mathsf{\theta }}_{\mathrm{j}}\right)+{\mathrm{a}}_{\mathrm{j}}{\mathrm{r}}_{\mathrm{j}}{\mathsf{\theta }}_{\mathrm{j}}={\mathrm{r}}_{\mathrm{j}}\left[1-{\mathsf{\theta }}_{\mathrm{j}}\left(1-{\mathrm{a}}_{\mathrm{j}}\right)\right]$. The variance of $\widetilde{{\mathrm{r}}_{\mathrm{J}}}$ is $\mathrm{Var}\left(\widetilde{{\mathrm{r}}_{\mathrm{J}}}\right)=\mathbb{E}\left({\widetilde{{\mathrm{r}}_{\mathrm{J}}}}^2\right)-{[\mathbb{E}\left(\widetilde{{\mathrm{r}}_{\mathrm{J}}}\right)]}^2={\mathrm{r}}_{\mathrm{j}}^2\left(1-{\mathsf{\theta }}_{\mathrm{j}}\right)+{\mathrm{a}}_{\mathrm{j}}^2{\mathrm{r}}_{\mathrm{j}}^2{\mathsf{\theta }}_{\mathrm{j}}-{({\mathrm{r}}_{\mathrm{j}}\left[1-{\mathsf{\theta }}_{\mathrm{j}}\left(1-{\mathrm{a}}_{\mathrm{j}}\right)\right])}^2{=\mathrm{r}}_{\mathrm{j}}^2{\mathsf{\theta }}_{\mathrm{j}}\left(1-{\mathsf{\theta }}_{\mathrm{j}}\right)+{\mathrm{a}}_{\mathrm{j}}{\mathrm{r}}_{\mathrm{j}}^2{\mathsf{\theta }}_{\mathrm{j}}\left[{\mathrm{a}}_{\mathrm{j}}-2\left(1-{\mathsf{\theta }}_{\mathrm{j}}\right)-{\mathrm{a}}_{\mathrm{j}}{\mathsf{\theta }}_{\mathrm{j}}\right]$.

Based on the properties of the normal distribution, the chance constraints in (13) can be transformed into Constraints (20), where $\Phi$ denotes the cumulative distribution function of the normal distribution:

$$P\left({{\displaystyle \sum }}_{j\in S\left(k\right)}\widetilde{r_J}{x}_j\le B_k\right)\ge \beta \Rightarrow P\left(\frac{{{\displaystyle \sum }}_{j\in S\left(k\right)}\left[\widetilde{r_J}-\mathbb{E}\left(\widetilde{r_J}\right)\right]x_j}{\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}}\right)\le \frac{B_k-{{\displaystyle \sum }}_{j\in S\left(k\right)}\mathbb{E}\left(\widetilde{r_J}\right)x_j}{\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}})\ge \beta ,$$

$$\Rightarrow \Phi \left(\frac{B_k-{{\displaystyle \sum }}_{j\in S\left(k\right)}\mathbb{E}\left(\widetilde{r_J}\right)x_j}{\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}}\right)\ge \beta , k=1,\cdots ,K.$$

(20)

Let ${\Phi }^{-1}$ denote the inverse function of $\Phi$. Since ${\Phi }^{-1}$ is monotonically increasing, constraints (20) can be further transformed into the following constraints:

$$\Phi \left(\frac{B_k-{{\displaystyle \sum }}_{j\in S\left(k\right)}\mathbb{E}\left(\widetilde{r_J}\right)x_j}{\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}}\right)\ge \beta \Rightarrow \frac{B_k-{{\displaystyle \sum }}_{j\in S\left(k\right)}\mathbb{E}\left(\widetilde{r_J}\right)x_j}{\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}}\ge {\Phi }^{-1}\left(\beta \right),$$

$$\Rightarrow {{\displaystyle \sum }}_{j\in S\left(k\right)}\mathbb{E}\left(\widetilde{r_J}\right)x_j+{\Phi }^{-1}\left(\beta \right)\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}Var\left(\widetilde{r_J}\right)x_j^2}\le B_k, k=1,\cdots ,K.$$

(21)

We then obtain Constraints (22) by plugging the expectation $\mathbb{E}\left(\widetilde{r_J}\right)$ and variance $Var\left(\widetilde{r_J}\right)$ into Constraints (21):

$$\begin{array}{c}{\displaystyle {\displaystyle \sum }_{j\in S\left(k\right)}}{r}_j\left[1-{\theta }_j\left(1-a_j\right)\right]x_j+\hfill \\ {\Phi }^{-1}\left(\beta \right)\sqrt{{{\displaystyle \sum }}_{j\in S\left(k\right)}\left[r_j^2{\theta }_j\left(1-{\theta }_j\right)+a_j{r}_j^2{\theta }_j\left[a_j-2\left(1-{\theta }_j\right)-a_j{\theta }_j\right]\right]x_j^2}\le B_k,k=1,\cdots ,K. \hfill \end{array}$$

(22)

Since ${\Phi }^{-1}\left(\beta \right)>0$, Constraints (22) are convex and equivalent to Constraints (17)–(19), where Constraints (18) are second-order cone constraints and $y_k$ is a continuous auxiliary variable. □

Next, proposition 2 transforms the nonlinear constraint in (12) into the equivalent Constraints (23)–(27).

Proposition 2. 

Model SP is equivalent to model MISOCP:

(MISOCP) Maximize (6)

Subject to

(2)–(4), (7)–(11), (14)–(19)

$${{\displaystyle \sum }}_{k=1}^K{t}_k\ge \left(1-\frac{g_1}{g_2}\right)ϵI_{max},$$

(23)

$$\left|\left|2q_{1k},e_k{B}_k-t_k\right|\right|\le e_k{B}_k+t_k, k=1,\cdots ,K,$$

(24)

$$\left|\left|2q_{2k},q_{1k}-1\right|\right|\le q_{1k}+1, k=1,\cdots ,K,$$

(25)

$$\left|\left|2t_k,q_{1k}-q_{2k}\right|\right|\le q_{1k}+q_{2k}, k=1,\cdots ,K,$$

(26)

$$q_{1k},q_{2k}\ge 0, k=1,\cdots ,K.$$

(27)

Proof. 

In Constraint (12), $\alpha \in \left(0, 1\right)$ can be replaced by two positive integers $g_1$ and $g_2$, i.e., $\alpha =g_1/g_2$. Then Constraint (12) can be converted to Constraint (28):

$${{\displaystyle \sum }}_{k=1}^K\frac{{(e_k{B}_k)}^{1-\alpha }}{\left(1-\alpha \right)}\ge ϵI_{max}\Rightarrow {{\displaystyle \sum }}_{k=1}^K\frac{{(e_k{B}_k)}^{1-\frac{g_1}{g_2}}}{1-\frac{g_1}{g_2}}\ge ϵI_{max}\Rightarrow {{\displaystyle \sum }}_{k=1}^K{(e_k{B}_k)}^{\frac{g_2-g_1}{g_2}}\ge \left(1-\frac{g_1}{g_2}\right)ϵI_{max}.$$

(28)

By introducing an auxiliary variable $t_k$, Constraint (28) can be split into Constraints (23) and (29):

$$t_k\le {(e_k{B}_k)}^{\frac{g_2-g_1}{g_2}}, k=1,\cdots ,K.$$

(29)

where Constraint (23) is linear and Constraints (29) are nonlinear. We need to further transform Constraints (29) that are equivalent to the following constraints:

$$t_k^{g_2}\le {(e_k{B}_k)}^{g_2-g_1}, k=1,\cdots ,K.$$

(30)

There exist positive integers $g_3$ and $g$ that satisfy $2^g=g_2-g_1+g_3+g_1$. We can transform Constraints (30) into Constraints (31):

$$t_k^{2^g}\le {(e_k{B}_k)}^{g_2-g_1}\times t_k^{g_3}\Rightarrow t_k^{2^g}\le {(e_k{B}_k)}^{g_2-g_1}\times t_k^{g_3}\times 1^{g_1}, k=1,\cdots ,K.$$

(31)

Following Çağlar and Gürel [10], we set $\alpha =0.4$. In this case, $g_1=2,g_2=5$. We substitute the values of $g_1$ and $g_2$ into Constraints (29) and obtain Constraints (32) based on Constraints (29)–(31).

$$t_k\le {(e_k{B}_k)}^{\frac{3}{5}}\Rightarrow t_k^5\le {(e_k{B}_k)}^3\Rightarrow t_k^{2^3}\le {(e_k{B}_k)}^3\times t_k^3\times 1^2, k=1,\cdots ,K.$$

(32)

Alzalg [35] shows that for any inequality of the form $t^{2^l}\le s_1{s}_2\dots s_{2^l}$, if $t\ge 0,s_1\ge 0,\dots ,s_{2^l}\ge 0$, then the inequality can be expressed by at most $2^{l-1}$ inequalities of the form $q^2\le uv$, where $q\ge 0,u\ge 0,v\ge 0$. Based on this, we transform Constraints (32) into a set of inequalities (33)–(35). Constraints (27) specify the range of the auxiliary variables $q_{1k}$ and $q_{2k}$.

$$q_{1k}^2\le e_k{B}_k\times t_k, k=1,\cdots ,K,$$

(33)

$$q_{2k}^2\le q_{1k}\times 1, k=1,\cdots ,K,$$

(34)

$$t_k^2\le q_{1k}\times q_{2k}, k=1,\cdots ,K.$$

(35)

For any inequality of the form $q^2\le uv$, it can be converted to conic quadratic inequality $\left|\left|2q,u-v\right|\right|\le u+v$. Therefore, the inequality set (33)–(35) can be further transformed into the conic quadratic inequality set (24)–(26). □

MISOCP is a mixed-integer second-order cone program that can be directly solved by CPLEX.

4. Computational Experiments

In this section, we investigate the following four research questions (RQs) based on computational experiments.

RQ1: How does the number of project proposals affect the total score of the portfolio?

RQ2: The failure probability of a project affects the chance of the project being selected. Then, how does the proportion of projects with high and low failure probability in all candidate projects affect the portfolio performance?

RQ3: How do different upper limits of the budget allocated to each project affect the performance of the project portfolio?

RQ4: The decision maker’s tolerance for project failure affects the chance of a project being selected. Then, how do different failure tolerances affect the portfolio performance?

In our experiments, the above RQs are answered by solving the MISOCP model with CPLEX. Our code is programmed in MATLAB 2016a, and CPLEX is also called by MATLAB 2016a. Our experiments were conducted on a computer equipped with a Ryzen5 5500U 2.1 GHz CPU and Windows 11 64-bit.

4.1. Benchmark Dataset

There is no dataset designed specifically for our problem. Therefore, we generate the benchmark dataset SET using a full factorial design. We call the situation corresponding to the benchmark dataset the baseline scenario. The parameter values of the benchmark dataset are shown in

Table 2. Parameter values for the benchmark dataset $SET$.

Parameters	Values	Notes
$\left|J\right|$	2000	The number of project proposals
$K$	7	The number of sectors. Its value is based on [10]
$\left|S_k\right|/\left|J\right|$	0.074, 0.071, 0.175, 0.128, 0.153, 0.235, 0.164	Proportion of the number of projects in each sector
$r_j$	[21, 36], [9, 36], [7, 36], [23, 36], [7, 22], [11, 36], [14, 36]	The range of the project budget in each sector
$B/{\displaystyle \sum }_{j\in J}{r}_j$	0.5	The proportion of the available budget to the total required budget of all project proposals
$\left|J_{k^{\prime }}^{\prime }\right|/\left|J\right|$	10%	The proportion of projects in the set that have quantity constraints compared to the total number of proposals
$K^{\prime }$	1	The number of project sets that have quantity constraints
$m$	1	The lower limit of the number of funded projects in $J_{k^{\prime }}^{\prime }(k^{\prime }=1,2,\dots ,K^{\prime })$
$n$	5	The upper limit of the number of funded projects in $J_{k^{\prime }}^{\prime }(k^{\prime }=1,2,\dots ,K^{\prime })$
$w_j$	[2, 10]	The range of the score for each project
$\eta$	85%	The upper limit of the budget allocated to each project $\left(\eta =R_j/\underset{j\in J}{\mathit{max}}\left\{r_j\right\}\right)$
$b_j$	0.5	The lower limit of the budget allocated to each project
$e_k$	0.86, 1.07, 1.01, 0.68, 0.81, 1.05, 0.73	The impact value for each sector [10]
$ϵ$	0.8	Decision maker’s balancing preference parameter
${\theta }_j$	[0.1, 0.2]	The range of failure probability for each project
$a_j$	0.3	The estimated fraction of the budget spent by each failed project
$\beta$	0.95	The confidence level for budget constraints
$\zeta$	50%	The upper limit for the proportion of failed projects
$o_{u,v}$	[−5, 5]	The range of the performance score fluctuations for the projects with interrelationships

. For each parameter combination in Table 2, 10 instances are generated. Since each parameter in Table 2 has only one value, this leads to 10 instances in the benchmark dataset. The dataset used in this paper has been uploaded to https://github.com/RuiChen-329/URDPS, accessed on 18 October 2022.

4.2. Performance Measure

In our experiments, we generate multiple variants of the baseline scenario by changing the parameter values in the baseline scenario. The number of instances contained in each scenario variant is the same as the number of instances in the baseline scenario. Based on the computational results obtained by comparing the scenario variants with the baseline scenario, we answer the four RQs. In the comparisons, we use the average relative deviation (ARD) of the total score as the performance measure. ARD measures the average gap in the optimal objective function value between the scenario variants and the baseline scenario. ARD is calculated as follows:

$$ARD=\frac{{{\displaystyle \sum }}_{i=1}^{\left|SET\right|}\left[\left(z_i^{VS}-z_i^{BS}\right)/z_i^{BS}\right]}{\left|SET\right|},$$

(36)

where $z_i^{VS}$ ($z_i^{BS})$ is the optimal objective function value of instance $i$ in the scenario variants (baseline scenario). The value of the ARD reveals the impacts of the factors (i.e., the number of project proposals, the funding amount, and the decision maker’s tolerance for project failure) on the performance of the project portfolio. In other words, when the decision maker chooses an alternative scenario instead of the baseline scenario, the value of the ARD indicates the gain or loss of the decision. The smaller the ARD value, the weaker the impact of the factors on the portfolio selection result.

When using CPLEX to solve each instance in different scenarios, we set the time limit to 30 s. The results show that 180 out of the 410 instances are solved to optimality. For the remaining 230 instances, only feasible solutions are obtained, and the estimated optimality gaps given by CPLEX are within 0.02%.

4.3. Experimental Results

We perform four experiments to answer the four RQs, respectively. As shown in

Table 3. Parameter values for the scenario variants.

Experiment #	Parameter Changed	Scenario Variants
		1	2	3	4	5	6	7	8	9	10
Experiment 1	$\left|J\right|$	1000	1200	1400	1600	1800	2200	2400	2600	2800	3000
Experiment 2	$\% of {\theta }_j$	10%	20%	30%	40%	50%	10%	20%	30%	40%	50%
Experiment 3	$\eta$	70%	73%	76%	79%	82%	85%	88%	91%	94%	97%
Experiment 4	$\zeta$	11%	12%	13%	14%	15%	16%	17%	18%	19%	20%

, in each experiment, only one parameter’s value is changed compared to the baseline scenario, and we generate 10 scenario variants (other parameters are kept unchanged). Note that in Experiment 2, instances of scenario variants 1–5 and 6–10 are obtained by extracting the failure probabilities of 10–50% of the baseline scenario instances from the intervals (0, 0.1) and (0.2, 0.3), respectively.

The computational results are shown in Figure 1. We can see from Figure 1 that the number of project proposals, the upper limit of the budget allocated to each project, and the decision maker’s tolerance for project failure have positive impacts on the portfolio performance. This means that the greater the values of these parameters, the higher the performance of the project portfolio. For the number of project proposals, this result is obvious: more candidate projects tend to contain more potential projects that can contribute more to the portfolio. In addition, the impact of the number of project proposals is linear.

The impacts of the upper limit of the budget allocated to each project and the decision maker’s tolerance for project failure are non-linear and follow a similar pattern. Specifically, before the thresholds (88% for the former factor and 14% for the latter) are reached, the performance of the portfolio increases rapidly as the above two factors increase. After the thresholds are reached, the impacts of both factors become steady. This means that for the upper limit of the budget allocated to each project, after most of the selected projects are allocated to large enough funds, increasing the funding amount of each project benefits fewer. For the decision maker’s tolerance for project failure, it reveals that the number of failed projects should be controlled within a certain range, which can achieve higher portfolio performance.

Compared with the baseline scenario, when the probability of project failure is small (large), the performance of the project portfolio is higher (lower). Increasing the number of projects with a low failure probability helps enhance the portfolio performance.

5. Conclusions and Future Research

In this paper, considering sectoral balancing, project failure probability, and interrelationships between projects, we have studied the uncertain public R&D project portfolio selection problem to maximize the expected performance of the project portfolio. We formulate a stochastic programming model for the problem. We transform the stochastic programming model into an equivalent deterministic second-order cone programming model that can be directly solved by exact solvers. Computational experiments are conducted to analyze the impacts of different factors on the project portfolio performance. The results show that the number of project proposals, the funding amount, and the decision maker’s tolerance for project failure have positive impacts on the portfolio performance, whereas the probability of project failure has a negative impact.

Our results reveal that the proposed method provides an effective tool to aid the decision maker’s project portfolio selection decisions. By incorporating real-world characteristics that appear in public R&D project portfolio selection such as sectoral balancing, project failure, and project interrelationships, our method is able to generate satisfactory project portfolios for different decision scenarios. Our method is easily embeddable into funding agency information systems, resulting in portfolio selection solutions that are automatically obtained to serve as a reference for decision-making and the effective utilization of public investment.

Future work will consider more factors in public R&D portfolio selection, such as the robustness of the solution. Additionally, since project funding agencies make project selection decisions dynamically or at regular intervals, it is necessary to study the problem from the perspective of a multi-stage decision-making process.