Risk Aversion Based Inexact Stochastic Dynamic Programming Approach for Water Resources Management Planning under Uncertainty

: In this study, a dual interval robust stochastic dynamic programming (DIRSDP) method is developed for planning water resources management systems under uncertainty. As an extension of the existing interval stochastic dynamic programming (ISDP) method, DIRSDP can deal with two-stage stochastic programming (TSP)-based planning problems associated with dynamic features, input uncertainties, and multistage concerns. Compared with other optimization methods dealing with uncertainties, the developed DIRSDP method has advantages in addressing uncertainties with complex presentations and reﬂecting decision makers’ risk-aversion attitudes within its optimization process. Parameters in the DIRSDP model can be represented as probability distributions as well as single and / or dual intervals. Decision makers’ risk-aversion attitudes can be reﬂected through restricting the deviation of the recourse costs to a tolerance level. Water-allocation plans can then be developed based on the analysis of tradeo ﬀ s between the system beneﬁt and solution robustness. The developed method is applied to a case of water resources management planning. The solutions are reasonable, indicating applicability of the developed methodology. The developed method is applied to a case of water resources management planning. The solutions are reasonable, indicating applicability of the developed methodology. The obtained results are further analyzed for identifying signiﬁcant factors that a ﬀ ect the system dynamics. This study is an attempt to introduce the RSP technique and the dual-interval concept into an interval SDP framework. Compared with other optimization methods dealing with uncertainties, the developed DIRSDP method has advantages in addressing uncertainties in multiple forms and reﬂecting decision makers’ attitude toward risks. DIRSDP can improve the existing TSP, RSP, and ISDP methods by its e ﬀ ectiveness in reﬂecting dual uncertainties and subjective information. The results suggest that the modeling methodology and its solution algorithm are also applicable to many other resources and environmental management problems that are of dynamic and uncertain features. DIRSDP can also be further improved through integrations with methods of multistage stochastic programming and nonlinear dynamic programming.


Introduction
Controversial and conflict-laden water allocation issues between competing water users (e.g., municipal, industrial, and agricultural) have challenged water resource managers for many decades. An imbalance between increasing consumption from the users and decreasing acquisition of water resources is one of major challenges [1]. Such an imbalance will be exacerbated by climate change, population growth, urbanization, environmental contamination, technology development, and other impact factors. These impacts introduce uncertainty into water resources management decision making, leading to potential risks associated with economic losses and eco-environment degradation and thereby moving backwards sustainable development [2]. Therefore, effective methods for water resources management planning are desired in order to balance water demand and supply under uncertainty as well as balance economic return and environmental safe considering risk management [3][4][5][6][7][8].
Previously, two-stage stochastic programming (TSP) was reported as an effective tool for water resources planning under uncertainty [9][10][11][12][13][14][15][16][17][18]. The TSP methods reported are characterized by two essential features: the uncertainty (i.e., random river flows) is but not only expressed with a certain probabilistic distribution and the sequence of decisions. The local authority has to determine first, before the uncertain water availability is revealed, how much water the multiple competing users can expect (e.g., water allocation targets) for their various activities and development plans with an objective to maximize their net benefits. After the random water availability is realized, the authority might correct the preceding decisions at the second-stage due to water shortage, where the users might take recourse actions to obtain water from expensive alternatives or curtail their development plans. The second-stage decisions made will result in economic penalties (i.e., increased costs, economic losses, or negative consequences), denoting a reduction of the net benefit resulting from the first-stage decisions made [19]. Ji, L. et al. [16] developed an inexact TSP modelfor supporting regional water resource allocation management under uncertainties. The research handled the uncertainties of future water demand through generating a set of representative scenarios and analyzed different scenarios to discuss their effects on the water distribution patterns, water shortages, total benefits, and system cost. Khosrojerdi, T. et al. [18] developed optimal allocation strategies of water resources using a TSP method with interval and fuzzy parameters. In the abovementioned TSP approaches, the total expected value of the second-stage costs (e.g., economic penalties or recourse costs) is measured without considering and controlling the variabilities of the costs under different possible water-delivery scenarios. An unacceptably high second-stage cost might be incurred from water shortage, leading to a system-disruption risk, while the total expected second-stage cost over all possible scenarios remains relatively low. Accordingly, robust stochastic programming (RSP), as an extension of TSP, was developed to control the variabilities of the second-stage costs. A number of related studies have been reported [20][21][22][23][24]. For example, Mulvey, J.M. et al. [25] discussed a measure of the variability and introduced the concept of robustness. Ahmed, S. et al. [20] introduced a concept of upper partial mean (UPM) as a new measure of the variability of the cost.
Although TSP-related methods can assist in developing decision plans in an uncertain environment, they could not adequately support sequential decisions in long-term water resources planning problems. Such problems with stochastic and dynamic features are often tackled through stochastic dynamic programming (SDP) methods [26][27][28][29][30][31][32][33]. Most of the previous SDP methods are effective for state problems with dynamic and sequential structures but are limited for providing recourse actions in order to minimize the expected costs or maximize the expected net system benefits over the whole states [1]. To address this, a number of TSP-based SDP researches were reported for dealing with problems where recourse decisions are desired, dynamic variations of system conditions are to be reflected and/or the available data information cannot be expressed with precision [34][35][36][37][38][39]. For example, Luo, B. et al. [36] developed an interval stochastic dynamic programming model (ISDP) with embedded TSP submodels and applied it to a long-term water resources planning problem. ISDP can reflect dynamic complexities and uncertainties (expressed as probability distributions and intervals) through communicating them into simplified interval TSP submodels within a dynamic linear optimization framework.
However, some deficiencies could exist in TSP-based ISDP that limit its application. For example, there is a chance that the second-stage costs, resulting from high water-allocation targets determined at the first-stage and thereafter insufficient supplies delivered, may be unacceptably high and lead to a high system-disruption risk. On the other hand, deterministic values for lower and upper bounds of the input intervals may be unavailable. Normally, a single interval (a, b) has deterministic bounds a and b. If these bounds cannot be expressed as crisp values (i.e., they can only be quantified as distribution and/or membership functions), intervals (a, c) and (d, b) would be used to quantify the two bounds; this leads to dual uncertainties. Thus, a concept of dual interval (i.e.,((a, c), (d, b))) can be proposed [40,41].
Therefore, the aim of this study is to develop a dual interval robust stochastic dynamic programming (DIRSDP) approach for water resources management planning. Figure 1 shows the methodological framework of DIRSDP. DIRSDP is an extension of the existing interval stochastic dynamic programming (ISDP), where the concept of dual interval and upper partial mean measure (a robust analysis technique) are introduced into the ISDP to account for system complexities and risks. The developed method can deal with system uncertainties expressed as probability distributions and single/dual intervals, and reflect system dynamics and multistage decision concerns, as well as reflect subjective risk-aversion attitudes. Additionally, it can help decision-makers to identify desired management plans with risk control. To demonstrate applicability of the proposed method, DIRDSP will be applied to a case of water resources management planning. The solutions are analyzed to identify significant factors that affect the system's performance and to gain insights into the tradeoffs between environmental and economic objectives. risks. The developed method can deal with system uncertainties expressed as probability distributions and single/dual intervals, and reflect system dynamics and multistage decision concerns, as well as reflect subjective risk-aversion attitudes. Additionally, it can help decisionmakers to identify desired management plans with risk control. To demonstrate applicability of the proposed method, DIRDSP will be applied to a case of water resources management planning. The solutions are analyzed to identify significant factors that affect the system's performance and to gain insights into the tradeoffs between environmental and economic objectives.

Modeling Formulation
Consider the planning for a water resources system under uncertainty (i.e., random water availability). Multiple users (e.g., municipal, industrial, and agricultural) extract water from the same water body (e.g., lake, reservoir, and river) over a planning horizon of T time periods. In each planning period, the users will expand their activities (e.g., conducting municipal, industrial or irrigation infrastructures) according to their demands with an objective to maximize their net benefits. They are determined by random water availability. When water demand of a user is high, it would lead to a raised net benefit if the promised water is delivered; however, it would incur penalties (including extra costs or economic losses) if the expected water is not delivered. On the contrary, when the demand is low, it would result in reduced net benefits (with a decreased risk of water shortage) and, in addition, would incur wastage of resources when the available water is high. Thus, decisions of each user should be properly made in each planning period so that the expected value of net-system benefit over the entire planning period can be maximized.
Such a long-term water resources system planning problem can be formulated as an interval stochastic dynamic programming (ISDP) model. In each planning stage, decisions of capacity expansion based on water demands at a stage will influence water-allocation targets at the beginning of the next stage. Meanwhile, the allocation targets at the end of each time period will depend solely on the entering state variables and the decisions made at that stage; they are independent of decisions made at the previous stages [36]. To effectively reflect the system's uncertainty and dynamic features, an inexact multistage optimization problem can be formulated with an objective of maximizing the expected net system benefit. Assume discrete distributions for the random parameters to avoid complicated computational requirements, thus, a deterministic equivalent formulation for ISDP can

Modeling Formulation
Consider the planning for a water resources system under uncertainty (i.e., random water availability). Multiple users (e.g., municipal, industrial, and agricultural) extract water from the same water body (e.g., lake, reservoir, and river) over a planning horizon of T time periods. In each planning period, the users will expand their activities (e.g., conducting municipal, industrial or irrigation infrastructures) according to their demands with an objective to maximize their net benefits. They are determined by random water availability. When water demand of a user is high, it would lead to a raised net benefit if the promised water is delivered; however, it would incur penalties (including extra costs or economic losses) if the expected water is not delivered. On the contrary, when the demand is low, it would result in reduced net benefits (with a decreased risk of water shortage) and, in addition, would incur wastage of resources when the available water is high. Thus, decisions of each user should be properly made in each planning period so that the expected value of net-system benefit over the entire planning period can be maximized.
Such a long-term water resources system planning problem can be formulated as an interval stochastic dynamic programming (ISDP) model. In each planning stage, decisions of capacity expansion based on water demands at a stage will influence water-allocation targets at the beginning of the next stage. Meanwhile, the allocation targets at the end of each time period will depend solely on the entering state variables and the decisions made at that stage; they are independent of decisions made at the previous stages [36]. To effectively reflect the system's uncertainty and dynamic features, an inexact multistage optimization problem can be formulated with an objective of maximizing the expected net system benefit. Assume discrete distributions for the random parameters to avoid complicated computational requirements, thus, a deterministic equivalent formulation for ISDP can be expressed as follows (assuming zero initial capacities and stage values without loss of generality) [36,42]: where B ± ti is the net benefit per unit of water allocated to user i in stage t; C ± ti is the reduction of net benefit per unit of water not delivered to user i in stage t (C ± ti > B ± ti ); d 1i is the minimum water demand in stage 1 to user i; f ± t is the net-system benefit over the entire planning horizon and f 0 is the zero initial stage value; k is the expansion option of water demand in each stage t; w tik denotes the amount of expanded water to user i in stage t under an expansion option k; x tik are state variables, denoting the water-allocation target to user i (i = 1, . . . , m) in stage t on a given expansion option k (k = 1, . . . , k); y ± tijk is the amount by which water-allocation target x tik is not met in stage t when total available water is q tj with probability level p tj under a given expansion option k in stage t; and x t−1i and y ± t−1ij are fixed water-allocation targets and the amounts of water shortage in stage t − 1 under an expansion option k. The objective of Model (1)- (6) is to maximize f ± T over the entire planning horizon, where constraints in (3) are related to water availability, ensuring that water consumption of the system should be less than total water availability in each planning stage; constraints in (4) are related to state transformations enforcing the dependency of each user's water-allocation target in stage t on the user's expansion options in that stage, as well as on the water-allocation target in stage t − 1; constraints in (5) are related to allowable water allocation, guaranteeing that the water-allocation target of each user in each stage is greater than the water shortage; constraints in (6) reflect the initial minimum water demand for each user.
Model (1)-(6) is a stochastic dynamic linear program. In each planning stage, once an expansion option is chosen, a decision regarding water-allocation targets needs to be made before the realization of uncertain water availability; then, a recourse action can be taken after such uncertainties are presented to mitigate that losses occurred due to water shortage. Such a problem can be analyzed through interval two-stage stochastic programming (ITSP). Thus, Model (1)-(6) can be solved by a dynamic linear program with embedded ITSP sub-problems (Model (7)-(11)), which can be developed as follows [36]: subject to: where x * t−1i is the optimized water-allocation targets of user i in stage t − 1 under different expansion options, and x * 0i = d 1i ; h ± tk is the system benefit in stage (t). Such an ITSP sub-problem should be solved in each stage (t) to determine state variables (x tik ) in the hierarchy tree of the dynamic problem (1); then, a backward recourse can be taken to determine the optimal expansion options (k) in all planning stages so that the optimum of Model (1)-(6) can be obtained.
However, the before-mentioned ITSP sub-problem is based on an assumption that decision makers are risk-neutral with measuring the expected value of the second-stage costs and without controlling the degree of variability of the costs. Such an assumption might lead to an unacceptably high cost incurred from water shortage. To address this, a measure of the deviation is needed to quantify the variability of the recourse costs. Through limiting the deviation of the recourse cost to a tolerance level, the degree of variability of the cost can be controlled and the risk-aversion from the decision makers can then be reflected. Such a measure can be formulated as follows [43] (Equation (12)): where ε is a given tolerance level for the variability (v y ti1k , y ti2k , . . . , y tijk ). Usually, as a symmetric risk measure, variance is the natural choice of the variability metric, which can penalize cost both above and below the expected recourse cost. However, it may cause the inclusion of nonlinearities into the formulation, leading to computational difficulties. Moreover, for water resources management problems, water shortages will incur penalties (recourse cost), while excess water should not be penalized. Thus, according to Ahmed, S. et al. [20], the upper partial mean (UPM) as an asymmetric measure of the variability of the recourse costs can be proposed as follows (Equations (13) and (14)): ∆ tj y ti1k , y ti2k , . . . y tijk = max In each planning stage, for scenario j (j = 1, 2, . . . , n), ∆ tj is the positive deviation of the cost from the expected recourse cost; in definition, ∆ t is the expected cost of the positive deviations over all scenarios. UPM can be used as the measure of variance in the restricted recourse formulations (through linear constraints) by introducing additional variables (∆ tj ) as follows (Equation (15)): Introducing (15) to the above TSP framework, a TSP submodel with restricted recourse can be obtained. Mulvey, J.M. et al. [25] named such a modeling methodology as robust stochastic programming (RSP) because the deviation of the recourse costs was limited (close to the expected value over all possible scenarios) and the resulting solutions would not be significantly sensitive to the variations of random events [13].Thus, one potential extension of Model (1)-(6) would be to integrate interval-parameter RSP into a dynamic linear programming framework. This leads to an interval robust stochastic dynamic programming (RSDP) model as follows: f 0 x 0i , y 0ij , w 0i = 0 (17) and f ± t are interval parameters or variables. Here, an interval is defined as a real number with known lower and upper bounds but unknown distribution information [44,45], where values of the bounds are deterministic.
However, dual uncertainties may exist when the available information for the boundaries of interval parameters cannot be acquired as deterministic values. For example, the reservoir outflows . Nevertheless, the lower and upper bounds of the intervals may not be known with certainty, leading to dual uncertainties. Thus, the concept of a dual interval (i.e., ((a 1 , ) is proposed to reflect such dual uncertainties. With uncertain boundaries, an infinite number of intervals (single) derived from each dual interval could exist. For example, the lower and upper bounds of a dual interval ((a 1 , c 1 ), (d 1 , b 1 )) may be valued as intervals (a 1 , c 1 ) and (d 1 , b 1 ), respectively. Then, the lower bound would take any value between a 1 and c 1 , and the upper one between d 1 and b 1 . Therefore, there would be infinite single intervals derived from this dual interval. When the reservoir outflows are expressed as dual intervals ((a 1 , c 1 ), (d 1 , b 1 )), ((a 2 , c 2 ), (d 2 , b 2 )), and ((a 3 , c 3 ), (d 3 , b 3 )) (each with a probability of occurrence), complexities will then exist. The difference between single and dual intervals for the reservoir outflows can be visualized in Figure 2. Joslyn, C. [41] proposed a method to simply simulate a dual interval ((a, c), (d, b)) in a class of random intervals (interval-valued random variables). According to the available information, ten types of potential conditions can be derived from combinations of four endpoints (a, b, c, and d) of the dual interval (Figures 3 and 4). Each of them is a collection of intervals with corresponding probabilities (e.g., random intervals). For example, when a and b (i.e., outliers of a dual interval) are considered to be more reliable than c and d, the interval can be simulated by random intervals (a, d), (a, b) and (c, b) with each having an identical probability of 0.33 [40]. According to Langewisch, A.T. et al. [46], the mean value of a random interval is (0.67a + 0.33c, 0.33d + 0.67b). The bounds for the mean have a general form (Equations (19) and (20)): where E − and E + are bounds for the mean of the random intervals, m l is assigned probability, and q − l and q + l are lower and upper bounds of each interval, respectively ( Figure 5). Thus, if q ± tj in Model (16)- (18) are expressed as dual intervals, they can be simulated by a class of random intervals based on the available information of their endpoints.         Considering the dual uncertainties (in reservoir outflows) in Model (16)-(18), a dual interval robust stochastic dynamic programming (DIRSDP) model can be developed as follows: subject to: where q ± tj ± denotes dual interval flows. Thus, under a given expansion option, Model (21)-(27) can be decomposed into many stages; for each stage, a dual interval RSP sub-problem can be formulated. In order to obtain the solution of Model (21)-(27), the dual interval RSP sub-problem for each stage should firstly be solved. Figure 6 illustrates a general flowchart of the solution for Model (21)-(27), and the detailed solution approach is presented in the next section.

Solution Method
In Model (21)-(27), when water-allocation targets ( ) in stage t − 1 are determined under given expansion options, in stage t under option k can be approached by the following dual interval robust stochastic programming (DIRSP) submodel:

Solution Method
In Model (21)-(27), when water-allocation targets (x t−1ik ) in stage t − 1 are determined under given expansion options, x tik in stage t under option k can be approached by the following dual interval robust stochastic programming (DIRSP) submodel: subject to: According to Huang, G.H. et al. [44,45], the solution for Model (28)-(33) can be obtained through a two-step method based on the transformation of dual intervals into a corresponding class of random intervals. A submodel corresponding to f + (when the objective function is to be maximized) can be first formulated, and then the relevant submodel corresponding to f − can be formulated based on the solution of the first submodel [44,45]. In this case, the solutions of the above sub-problem can be expressed as follows (Equation (34)): where h ± tkopt are optimum objective-function values of Model (28)-(33); y ± tijkopt are optimized second-stage decision variables; x tikopt are optimized water-allocation targets in stage t. Thus, the optimal water-allocation plans are (Equation (35)): where A ± tijkopt is the optimal water allocation to user i in stage t under expansion option k and flow level j. Thus, an optimal water-allocation scheme (including an optimal capacity-expansion option) can be determined in stage t to maximize the expected value of system benefit (objective-function value in stage t); also, the optimized water-allocation targets (x * tik ) in stage t will work on the next stage (stage t + 1) as an entering state variable. In stage t + 1, another DIRSP sub-problem will be confronted in such a way that an optimal expansion option, water-allocation scheme and system benefit can be found. Over the entire planning horizon, a set of optimal solutions can be obtained through dynamically solving Model (21)- (27). Finally, the best route in the network can be found and the multistage system benefit can be maximized.
Based on the solutions of Model (28)-(33), we have h + tkopt > h − tkopt for any given k. Therefore, there exists f + t > f − t for any planning stage from stage 1 to stage T. The solution algorithm of the DIRSDP model can be presented as follows: (1) Formulate the DIRSDP problem with the objective being maximized; (2) Decompose the DIRSDP problem into sub-problems as defined in Model (28) (36)):   (37) and (38)): where x * tik are optimized water-allocation targets in each planning stage (t); y ± tijkopt are amounts of water shortage when the optimized allocation targets are not met in planning stage; A ± tijkopt are optimized allocation schemes over the entire planning horizon; h ± tkopt are stage-t objective-function values when Model (21)

Case
A water resources planning problem will be studied to demonstrate applicability of the developed DIRSDP approach. A local authority is in charge of delivering water from a reservoir during three periods (five years for each) to three users: municipal, industrial, and agricultural sectors. All users are planning to expand their capacities and want to know how much water they can expect in the future. Three options of expected water availability are provided for the three users. If sufficient water is unavailable (e.g., in dry seasons), the users will confront a reduction in their net benefits. Table 1 presents the initial water demand and the economic data to each user in the three planning periods. Economic coefficients B ± ti and C ± ti are represented as intervals. Table 2 shows the expansion options (k) and the amounts of expected water for capacity expansion (w tik ) for the three users in each planning period. Table 3 lists regulated outflows from the reservoir (expressed as dual intervals) and their associated probabilities. Assume that the probability distributions of the outflows are the same in each planning period, and the out-layer values of the dual intervals are more reliable than interiors (more confidence from the stakeholders). According to Joslyn, C. [41], each dual interval can be transformed into a class of random intervals with relevant probabilities. The bounds for the mean of these random intervals can then be obtained ( Figure 5).

Flow Level (j) Flow Volume Probability
Very-low (VL) (j  Table 4 lists the solutions obtained through Model (21)- (27), under restricted-recourse consideration, for the objective function and decision variables. Most of them are presented as intervals in response to the input uncertainties. During period 1, expansion option 3 (i.e., (0.11, 0.14) × 10 6 , (0.24, 0.32) × 10 6 , and (2.4, 3.2) × 10 6 m 3 for users 1, 2, and 3, respectively) would be a choice for the three users. Nevertheless, the relationship between net-system benefit and water-allocation targets is not monotonic in each period. The users would select option 1 (i.e.,(0.04, 0.07) × 10 6 , (0.08, 0.16) × 10 6 , and (0.8, 1.6) × 10 6 m 3 for users 1, 2, and 3, respectively) during periods 2 and 3 since larger expansion may result in higher penalties when water availability is insufficient in dry seasons. A tree notation for the expansion plan is illustrated in Figure 7. Table 4. Solutions of the DIRSDP model.

Solution
Optimized expansions option:     Figure 8 shows the optimized water-allocation targets for the three users in different planning periods. In period 1, the municipal sector would set its water-allocation target at 2.14 × 10 6 m 3 according to the initial (minimum) water demand of 2.00 × 10 6 m 3 and the expansion need of (0.11, 0.14) × 10 6 m 3 . It can be seen that the water demand from this sector would be satisfied to the maximum (i.e., 0.14 × 10 6 m 3 ) since it can bring about high benefits when the demand is met and can be subject to high penalties if the promised water is not delivered. The industrial and agricultural sectors would have their targets be 2.82 × 10 6 and 5.90 × 10 6 m 3 , respectively. These targets may not always be satisfied due to the lower benefit and penalty levels. Since x tikopt is related to (k + 1) w tik (decision of expansion options at the present stage) and x * t−1i (optimized water-allocation target set at the previous stage), the optimized water-allocation targets for periods 2 and 3 would be: x 211opt = 2.21, x 221opt = 2.98, x 231opt = 6.70, x 311opt = 2.28, x 321opt = 3.14, and x 331opt = 7.50.  Table 4 also provides the optimized water-allocation scheme over the entire planning horizon, where interval solutions ± (in reference to the optimized water-allocation targets) reflect potential variations in the system conditions due to the uncertain inputs ( ± , ± , and [ ± ] ± ). In period 1, the solution of = (0.61, 1.67) for user 1 (i.e., municipality) indicates that a shortage of 0.61 × 10 6 to 1.67 × 10 6 m 3 would exist under very-low flow (with a probability of 2.5%), and there would be no water shortage under the other flow levels. In periods 2 and 3, the shortage amounts would be increased to (0.68, 1.74) × 10 6 and (0.75, 1.81) × 10 6 m 3 , respectively.
For user 2 (i.e., industry) in period 1, the solutions of = 2.82 and = (0.93, 2.82) imply that a shortage would definitely occur under very-low flow since the shortage (2.82 × 10 6 m 3 ) equals the target ( * = 2.82 × 10 6 m 3 ); when the flow is at a low level (with a probability of 5%), the shortage would be relatively low (0.93 × 10 6 m 3 ) under advantageous conditions (e.g., the other users do not consume their full amounts of targeted demands), and would be raised to 2.82 × 10 6 m 3 under demanding conditions. In periods 2 and 3, the shortage amounts would be 2.98 × 10 6 and 3.14 × 10 6 m 3 under very-low flow and reach (1.16, 2.98)× 10 6 and (1.39, 3.14)× 10 6 m 3 under low flow.
The results of = 5.90, = 5.90, = (3.33, 5.39), = (0, 2.79), and = = = 0 mean that, for user 3 (i.e., agriculture) in period 1, 5.90 unit of deficiency would definitely occur when the reservoir outflow is very-low or low; shortages of 3.33 × 10 6 to 5.39 × 10 6 m 3 and 0 to 2.79 × 10 6 m 3 would exist under low-medium and medium flows, respectively; there would be no water deficiency under medium-high to very-high outflows. In the case of shortage (i.e., in dry seasons), water would be firstly guaranteed to municipality, secondly to the industrial sector, and then to the agricultural one (since it can bring about low net benefits when the water demand is satisfied, and has low penalties if the promised water is not delivered). Moreover, the agricultural sector is projected to have larger expansion in the entire planning horizon. Therefore, its water-allocation targets in periods 2 and 3 may become more difficult to be satisfied, compared to that in period 1 when the available water is insufficient. The shortage amounts would be increased to 6.70 × 10 6 , 6.70 × 10 6 , (4.36, 6.42)× 10 6 , (0.96, 3.82)× 10 6 , and (0, 0.32)× 10 6 m 3 in period 2, and be 7.50 × 10 6 , 7.50 × 10 6 , (5.39, 7.45)× 10 6 , (1.99, 4.85)× 10 6 , and (0, 1.35)× 10 6 m 3 in period 3, when the reservoir outflows are at very-low to medium-high levels, respectively.  Table 4 also provides the optimized water-allocation scheme over the entire planning horizon, where interval solutions y ± tijkopt (in reference to the optimized water-allocation targets) reflect potential variations in the system conditions due to the uncertain inputs (B ± ti , C ± ti , and q ± tj ± ). In period 1, the solution of y 1113opt = (0.61, 1.67) for user 1 (i.e., municipality) indicates that a shortage of 0.61 × 10 6 to 1.67 × 10 6 m 3 would exist under very-low flow (with a probability of 2.5%), and there would be no water shortage under the other flow levels. In periods 2 and 3, the shortage amounts would be increased to (0.68, 1.74) × 10 6 and (0.75, 1.81) × 10 6 m 3 , respectively. For user 2 (i.e., industry) in period 1, the solutions of y 1213opt = 2.82 and y 1223opt = (0.93, 2.82) imply that a shortage would definitely occur under very-low flow since the shortage (2.82 × 10 6 m 3 ) equals the target (x * 12 = 2.82 × 10 6 m 3 ); when the flow is at a low level (with a probability of 5%), the shortage would be relatively low (0.93 × 10 6 m 3 ) under advantageous conditions (e.g., the other users do not consume their full amounts of targeted demands), and would be raised to 2.82 × 10 6 m 3 under demanding conditions. In periods 2 and 3, the shortage amounts would be 2.98 × 10 6 and 3.14 × 10 6 m 3 under very-low flow and reach (1.16, 2.98) × 10 6 and (1.39, 3.14) × 10 6 m 3 under low flow.
The results of y 1311opt = 5.90, y 1323opt = 5.90, y 1133opt = (3.33, 5.39), y 1143opt = (0, 2.79), and y 1253opt = y 1163opt = y 1173opt = 0 mean that, for user 3 (i.e., agriculture) in period 1, 5.90 unit of deficiency would definitely occur when the reservoir outflow is very-low or low; shortages of 3.33 × 10 6 to 5.39 × 10 6 m 3 and 0 to 2.79 × 10 6 m 3 would exist under low-medium and medium flows, respectively; there would be no water deficiency under medium-high to very-high outflows. In the case of shortage (i.e., in dry seasons), water would be firstly guaranteed to municipality, secondly to the industrial sector, and then to the agricultural one (since it can bring about low net benefits when the water demand is satisfied, and has low penalties if the promised water is not delivered). Moreover, the agricultural sector is projected to have larger expansion in the entire planning horizon. Therefore, its water-allocation targets in periods 2 and 3 may become more difficult to be satisfied, compared to that in period 1 when the available water is insufficient. The shortage amounts would be increased to 6.70 × 10 6 , 6.70 × 10 6 , (4.36, 6.42) × 10 6 , (0.96, 3.82) × 10 6 , and (0, 0.32) × 10 6 m 3 in period 2, and be 7.50 × 10 6 , 7.50 × 10 6 , (5.39, 7.45) × 10 6 , (1.99, 4.85) × 10 6 , and (0, 1.35) × 10 6 m 3 in period 3, when the reservoir outflows are at very-low to medium-high levels, respectively. Figure 9 and Table 4 show the objective-function value in each planning period. It can be noticed that the lower bound of h ± tkopt would decrease and the upper bound would increase over the entire planning horizon; this implies an increase in fluctuation range of the objective-function value ( f ± t ). For example, when the upper bound of h ± tkopt increases from $521.88 × 10 6 in period 1 to $600.08 × 10 6 in period 3, the lower bound would decrease from $256.54 × 10 6 in period 1 to $235.42 × 10 6 in period 3. This is mainly because the water-allocation targets in each period are set to correspond with h + tk in the dual interval RSP sub-problem, making the system take more penalties when water availability is insufficient in dry seasons. At the same time, values of f − 3 and f + 3 provide two extremes of the expected net system benefit over the entire planning horizon.
Sustainability 2019, 11, x FOR PEER REVIEW 17 of 23 Figure 9 and Table 4 show the objective-function value in each planning period. It can be noticed that the lower bound of ℎ ± would decrease and the upper bound would increase over the entire planning horizon; this implies an increase in fluctuation range of the objective-function value ( ± ). For example, when the upper bound of ℎ ± increases from $521.88 × 10 6 in period 1 to $600.08 × 10 6 in period 3, the lower bound would decrease from $256.54 × 10 6 in period 1 to $235.42 × 10 6 in period 3. This is mainly because the water-allocation targets in each period are set to correspond with ℎ in the dual interval RSP sub-problem, making the system take more penalties when water availability is insufficient in dry seasons. At the same time, values of and provide two extremes of the expected net system benefit over the entire planning horizon. Without the restricted-recourse consideration, Model (21)- (27) can be viewed as a dual interval SDP based on Luo, B. et al. [36] and Huang, G.H. and Loucks, D.P. [19]. Table 5 presents the solution under such a consideration, assuming that the decision makers are risk-neutral. In the case of water shortage, allocation to the municipal sector would always be guaranteed; in comparison, allotments to the industrial and agricultural sectors would be reduced, especially for the latter. For example, the optimized water-allocation target for user 3 in period 1 would be 5.97 × 10 6 m 3 (without restricted recourse consideration) other than 5.90 × 10 6 m 3 (with restricted recourse consideration). This is because the decision makers' preference towards risk is involved in Model (21)- (27), leading to a relatively low water-allocation target with a reduced water shortage. This will finally result in a decrease in the system benefit (Tables 4 and 5). The decision makers would rather accept a decrease in the system benefit through restricting the second-stage costs in advance than take a risk to accept an extremely high cost incurred after the realization of an uncertain water shortage. As a measure of the variability of the second-stage cost, the upper partial mean (UPM) is introduced to Model (21)- (27). Without the restricted-recourse consideration, Model (21)- (27) can be viewed as a dual interval SDP based on Luo, B. et al. [36] and Huang, G.H. and Loucks, D.P. [19]. Table 5 presents the solution under such a consideration, assuming that the decision makers are risk-neutral. In the case of water shortage, allocation to the municipal sector would always be guaranteed; in comparison, allotments to the industrial and agricultural sectors would be reduced, especially for the latter. For example, the optimized water-allocation target for user 3 in period 1 would be 5.97 × 10 6 m 3 (without restricted recourse consideration) other than 5.90 × 10 6 m 3 (with restricted recourse consideration). This is because the decision makers' preference towards risk is involved in Model (21)- (27), leading to a relatively low water-allocation target with a reduced water shortage. This will finally result in a decrease in the system benefit (Tables 4 and 5). The decision makers would rather accept a decrease in the system benefit through restricting the second-stage costs in advance than take a risk to accept an extremely high cost incurred after the realization of an uncertain water shortage. As a measure of the variability of the second-stage cost, the upper partial mean (UPM) is introduced to Model (21)- (27).    Table 6 compares the solutions related to UPM in different planning periods with and without the restricted-recourse consideration. For example, the UPM value in period 1 would be $(44.57, 69.36) × 10 6 with the restricted-recourse consideration, and $(45.15, 69.60) × 10 6 without the consideration. Figure 10 illustrates the deviation of the recourse costs from their expected values (i.e.,∆ ± tj ) under different probabilities (under the two scenarios). It is indicated that the degree of variability of the second-stage cost would be controlled through the proposed UPM measure. The risks of system-disruption will be reduced though it will finally result in a decrease in the system benefit. Accordingly, the system robustness will be enhanced for dealing with various uncertainties and risks. Therefore, the most desirable water-allocation scheme should be based on the analysis of tradeoff between the benefit and the robustness, where the objective is to identify a sound water-resource-management strategy with a relatively low deviation but a high benefit [47].  Table 6 compares the solutions related to UPM in different planning periods with and without the restricted-recourse consideration. For example, the UPM value in period 1 would be $(44.57, 69.36) × 10 6 with the restricted-recourse consideration, and $(45.15, 69.60) × 10 6 without the consideration. Figure 10 illustrates the deviation of the recourse costs from their expected values (i.e., ∆ ± ) under different probabilities (under the two scenarios). It is indicated that the degree of variability of the second-stage cost would be controlled through the proposed UPM measure. The risks of system-disruption will be reduced though it will finally result in a decrease in the system benefit. Accordingly, the system robustness will be enhanced for dealing with various uncertainties and risks. Therefore, the most desirable water-allocation scheme should be based on the analysis of tradeoff between the benefit and the robustness, where the objective is to identify a sound waterresource-management strategy with a relatively low deviation but a high benefit [47].  Figure 10. Deviation of recourse cost over its expected value. Figure 10. Deviation of recourse cost over its expected value.

Discussions
More scenarios can be examined under different UPM tolerance levels in each planning period. In this case, a condition can be specified for limiting the variability of recourse costs to a targeted level. If the obtained solutions are beyond the decision makers' expectations, new target levels can be further considered and a series of decision alternatives can thus be generated. Meanwhile, the best expansion route in the network could be varied by considering limitations on the deviation of the recourse cost in each planning stage. However, such a consideration could increase computational requirements.
The dual interval RSP sub-problem (Model (28)-(33)) in each planning stage can be solved through conventional TSP method with restricted-recourse by allowing all interval parameters to be equal to their mid values. The obtained solutions form a set of deterministic values, which represent a decision under deterministic inputs. However, such results may not be obtained from the DIRSDP model. The range of water-allocation target in one stage is partly determined by the allocation targets in its previous stages. Thus, the recourse action in the dynamic program can be different when the DIRSDP model is solved through the conventional robust stochastic dynamic programming (RSDP) method. This would drive the water-allocation targets and the objective-function values out of their ranges. Accordingly, the RSDP is not likely to produce the variety in decision scenarios that the dual interval SDP does.
DIRSDP has an advantage in addressing uncertainties with complex presentations (probability distributions and intervals). It can handle not only probabilistic uncertainties in the model's right-hand sides but independent uncertainties, expressed as intervals, of the model's left-hand sides and cost coefficients. In most of the cases, the information is unavailable to access probability distributions; even if the distributions are available, it could be difficult to reflect them in large-scale SDP models. Taking intervals as uncertain inputs, DIRSDP can address such an embarrassment and communicate uncertainties (in interval formats) into the optimization process. Moreover, DIRSDP can quantify impacts of uncertainties represented as dual intervals on water availability. When highly uncertain information for boundaries of intervals is unavailable to establish membership or probability functions, the dual-interval concept can be presented to reflect the dual uncertainties. In a real-world water resources planning problem, such uncertainties may result from temporal variations in river flows and climatic conditions.
Another advantage of DIRSDP is its capability in incorporating subjective information within the optimization framework through controlling the degree of variability for the recourse cost. Assuming that the decision makers are risk-neutral, DIRSDP can be viewed as dual interval SDP without restricted-recourse consideration. The resulting recourse cost could be unacceptably high and be very sensitive to random variations of water availability. DIRSDP can also provide a spectrum of robust options through progressively decreasing the tolerance level of the deviation degree for the recourse cost. The decision makers' preference towards risk management can be reflected and their choice of water allocation can then be facilitated. A scheme with a relatively low deviation but a high benefit may be desired by risk-aversion decision makers.

Conclusions
A dual interval robust stochastic dynamic programming (DIRSDP) method has been developed for planning water resources systems under uncertainty. As an extension of the existing interval stochastic dynamic programming (SDP) method, DIRSDP can deal with uncertainties with complex presentations and incorporate subjective information within the optimization process. Parameters in the DIRSDP model can be represented as probability distributions as well as single and/or dual intervals. Through limiting the recourse-cost variability, decision makers' preference can be reflected. Reasonable water-allocation plans can be developed based on the analysis of tradeoffs between the system benefit and solution robustness. The developed method is applied to a case of water resources management planning. The solutions are reasonable, indicating applicability of the developed methodology. The obtained results are further analyzed for identifying significant factors that affect the system dynamics.
This study is an attempt to introduce the RSP technique and the dual-interval concept into an interval SDP framework. Compared with other optimization methods dealing with uncertainties, the developed DIRSDP method has advantages in addressing uncertainties in multiple forms and reflecting decision makers' attitude toward risks. DIRSDP can improve the existing TSP, RSP, and ISDP methods by its effectiveness in reflecting dual uncertainties and subjective information. The results suggest that the modeling methodology and its solution algorithm are also applicable to many other resources and environmental management problems that are of dynamic and uncertain features. DIRSDP can also be further improved through integrations with methods of multistage stochastic programming and nonlinear dynamic programming.