Bi-Level Dependent-Chance Goal Programming for Paper Manufacturing Tactical Planning: A Reinforcement-Learning-Enhanced Approach

Abstract

Tactical production–distribution planning in paper manufacturing involves hierarchical decision-making under hybrid uncertainty, where aleatory randomness (demand fluctuations, machine variations) and epistemic uncertainty (expert judgments, market trends) simultaneously affect operations. Existing approaches fail to address the bi-level nature under hybrid uncertainty, treating production and distribution decisions independently or using single-paradigm uncertainty models. This research develops a bi-level dependent-chance goal programming framework based on uncertain random theory, where the upper level optimizes distribution decisions while the lower level handles production decisions. The framework exploits structural symmetries through machine interchangeability, symmetric transportation routes, and temporal symmetry, incorporating symmetry-breaking constraints to eliminate redundant solutions. A hybrid intelligent algorithm (HIA) integrates uncertain random simulation with a Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm (RL-AOA) for bi-level coordination, where Q-learning enables adaptive parameter tuning. The RL component utilizes symmetric state representations to maintain solution quality across symmetric transformations. Computational experiments demonstrate HIA’s superiority over standard metaheuristics, achieving 3.2–7.8% solution quality improvement and 18.5% computational time reduction. Symmetry exploitation reduces search space by approximately 35%. The framework provides probability-based performance metrics with optimal confidence levels (0.82–0.87), offering 2.8–4.5% annual cost savings potential.

1. Introduction

The paper manufacturing industry, valued at approximately USD 400 billion globally, represents one of the most operationally complex sectors in modern manufacturing, where tactical production–distribution planning decisions directly impact both economic performance and environmental sustainability [1]. Paper production involves intricate interdependencies between raw material procurement, multi-grade production processes, and diverse distribution networks serving packaging, publishing, and specialty applications [2]. The industry’s operational structure exhibits significant structural symmetries, including machine interchangeability for certain paper grades, bidirectional material flows between production facilities, and periodic patterns in demand cycles, which create opportunities for computational efficiency through systematic symmetry exploitation in optimization algorithms [3].

Paper manufacturing operations encompass diverse product portfolios including newsprint, packaging materials, tissue products, and specialty papers, each requiring specific production parameters and handling requirements [4]. Production facilities face machine compatibility constraints and sequence-dependent changeover requirements that impose significant setup times and costs when transitioning between product types [5]. The presence of symmetric machine capabilities—where multiple machines can produce identical grades with equivalent efficiency—introduces structural symmetries in the optimization problem formulation. These symmetries manifest as equivalent production schedules that yield identical costs and performance metrics, necessitating symmetry-aware optimization mechanisms to avoid redundant exploration of mathematically equivalent solutions. Raw material variability and seasonal demand patterns create additional complexity in capacity allocation decisions [6].

The tactical planning horizon of 3–6 months represents a critical decision space where strategic network design constraints meet operational execution requirements [7]. Companies must make production scheduling decisions that determine machine utilization and inventory positioning while simultaneously coordinating distribution strategies affecting customer service levels and transportation costs [8]. The bi-level nature of this problem exhibits hierarchical structural symmetries, where upper-level distribution decisions and lower-level production decisions display analogous constraint patterns and objective contributions, requiring coordinated optimization approaches that can exploit these symmetries while maintaining solution optimality.

Supply chain planning operates across multiple hierarchical levels with distinct time horizons and decision scopes [9]. Strategic planning (1–5 years) focuses on network design decisions, tactical planning (3–6 months) bridges strategic constraints with operational execution, and operational planning (daily to weekly) addresses immediate execution decisions [7]. Recent advances in optimization have demonstrated the effectiveness of symmetry-based decomposition techniques for large-scale planning problems. However, existing approaches for bi-level production–distribution planning have not systematically addressed symmetry identification and exploitation, particularly in the context of hybrid uncertainty. Our reinforcement-learning-enhanced arithmetic optimization approach incorporates symmetric state representations in Q-learning to enable efficient exploration while avoiding redundant evaluations of symmetric solution regions.

Tactical planning in paper manufacturing operates under hybrid uncertainty that cannot be adequately captured by traditional single-paradigm approaches [10]. Aleatory uncertainty sources include demand fluctuations, machine efficiency variations, and quality variations that follow well-defined probability distributions based on historical data [11]. Epistemic uncertainty stems from incomplete knowledge about market trends, regulatory changes, and supplier reliability, where expert judgment must complement quantitative data [12]. Traditional optimization approaches treating these uncertainties independently fail to capture their interactive effects and compound impacts on tactical decisions [13].

Despite extensive research in production–distribution planning, several critical gaps remain that limit the effectiveness of existing approaches. Bi-level production–distribution planning research has primarily focused on deterministic parameters, failing to address goal-oriented frameworks that incorporate probability-based performance targets [14]. Goal programming applications have predominantly utilized deterministic formulations that assume exact target values, limiting their applicability in uncertain environments where probability-based objectives are more meaningful [15]. Uncertainty modeling has advanced significantly through stochastic programming and fuzzy programming approaches, yet hybrid uncertainty applications that simultaneously handle both aleatory and epistemic components remain limited [10]. Paper industry optimization research has developed sophisticated models for individual planning components but lacks comprehensive frameworks addressing integrated tactical-level decision coordination [1].

The symmetric properties inherent in paper manufacturing networks present both computational challenges and optimization opportunities. While these symmetries can exponentially expand the solution space through mathematically equivalent solutions, they also enable significant computational efficiencies when properly exploited. Our framework systematically addresses these symmetries through (1) identification and classification of structural symmetries in the bi-level problem formulation, (2) incorporation of symmetry-breaking constraints to eliminate redundant solution regions, (3) utilization of symmetric state representations in the reinforcement learning component to maintain solution quality across symmetric transformations, and (4) preservation of invariance properties in arithmetic optimization operators to ensure consistent performance across symmetric problem instances.

The primary objective of this research is to develop a comprehensive bi-level dependent-chance goal programming framework for tactical production–distribution planning in paper manufacturing under hybrid uncertainty. This research addresses four fundamental questions that guide the investigation: How can dependent-chance goal programming be adapted to handle bi-level tactical planning where production and distribution managers have different risk tolerances and objective priorities? What is the optimal bi-level structure for integrating production scheduling and distribution planning decisions while maintaining computational tractability? How effective is the proposed Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm compared to existing metaheuristics for solving complex bi-level optimization problems under hybrid uncertainty? What are the trade-offs between different goal probability levels and how do these affect tactical planning performance in paper manufacturing contexts?

To address the identified research gaps and challenges, this study makes the following key contributions:

• Bi-Level Dependent-Chance Goal Programming Framework: Development of the first comprehensive bi-level dependent-chance goal programming model for tactical production–distribution planning that simultaneously addresses hierarchical decision-making and probabilistic goal achievement under hybrid uncertainty, extending traditional goal programming theory into probabilistic domains with bi-level coordination.
• Hybrid Uncertainty Modeling Enhancement: Advancement of uncertain random theory applications through a systematic framework for modeling hybrid uncertainty in tactical planning contexts, providing unified mathematical representations that capture both aleatory randomness from historical data and epistemic uncertainty from expert judgments.
• Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm: Introduction of the novel RL-AOA, which integrates Q-learning mechanisms with arithmetic optimization operators for adaptive bi-level coordination, enabling dynamic parameter tuning based on optimization progress and problem-specific characteristics.
• Industry-Specific Tactical Planning Framework: Development of the first comprehensive tactical planning framework specifically designed for paper manufacturing environments, incorporating machine compatibility constraints, sequence-dependent changeover requirements, paper grade specifications, and multi-modal transportation considerations.
• Computational Validation and Implementation Guidelines: Demonstration of algorithm superiority through extensive computational experiments achieving 3.2–7.8% solution quality improvement and 18.5% computational time reduction, accompanied by practical implementation guidelines and sensitivity analysis insights for industrial adoption.
• Symmetry-Aware Optimization Techniques: Development of systematic approaches for identifying, classifying, and exploiting structural symmetries in paper manufacturing networks, demonstrating measurable computational improvements through intelligent symmetry breaking and symmetric state representations in reinforcement learning algorithms.

The remainder of this paper is organized as follows: Section 2 presents the mathematical formulation of the bi-level dependent-chance goal programming model including uncertain random variable definitions and chance measure calculations. Section 3 describes the theoretical foundations of uncertain random theory for hybrid uncertainty modeling and its application to tactical planning problems. Section 4 details the Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm including bi-level coordination mechanisms and adaptive parameter tuning strategies. Section 5 provides comprehensive computational results and performance analysis across diverse paper manufacturing instances with comparative evaluation against existing approaches. Section 6 concludes with key findings, practical implications, and future research directions.

2. Mathematical Model Formulation

This section presents the comprehensive bi-level dependent-chance goal programming model for tactical production–distribution planning in paper manufacturing under hybrid uncertainty. The model captures the hierarchical decision-making structure where distribution coordinators (upper level) and production managers (lower level) operate with different objectives while facing uncertain random parameters.

2.1. Sets and Indices

The following

Table 1. Sets and indices used in the paper mill supply chain optimization model. This table defines the mathematical notation for all entities in the supply chain network including products, facilities, and their relationships.

Notation	Description
$P=\{1,2,\dots ,p\}$	Set of paper products/grades
$M=\{1,2,\dots ,m\}$	Set of paper machines
$W=\{1,2,\dots ,w\}$	Set of warehouses
$D=\{1,2,\dots ,d\}$	Set of distribution centers
$C=\{1,2,\dots ,c\}$	Set of customers
$R=\{1,2,\dots ,r\}$	Set of raw materials (pulp types, recycled fiber)
$T=\{1,2,\dots ,t\}$	Set of time periods (months)
$L=\{1,2,\dots ,l\}$	Set of transportation modes
$S=\{1,2,\dots ,s\}$	Set of suppliers
PMp ⊆ M	Set of machines compatible with paper grade p
PRp ⊆ R	Set of raw materials required for paper grade p
Φincompatible ⊆ P × P	Set of incompatible product transition pairs

presents the sets and indices used throughout our paper mill supply chain optimization model. These notations form the foundation of our mathematical formulation, defining the scope of products, facilities, resources, and relationships within the supply chain network. Table 1 establishes the necessary mathematical notation that will be used consistently throughout the remainder of this paper.

2.2. Parameters

2.2.1. Deterministic Parameters

The deterministic parameters presented in

Table 2. Deterministic parameters for the paper mill supply chain model. This table lists all fixed parameters related to costs, capacities, production rates, and technical specifications that remain constant throughout the planning horizon.

Parameter	Description
Kpm	Setup cost for producing paper grade p on machine m
$S{T}_{p_i{p}_{j}m}$	Sequence-dependent changeover time from grade pi to pj on machine m
$S{C}_{p_i{p}_{j}m}$	Sequence-dependent changeover cost from grade pi to pj on machine m
αpr	Amount of raw material r required per unit of paper grade p
βpm	Production rate of paper grade p on machine m (tons/hour)
CAPm	Available capacity of machine m in each period (hours)
WCAPw	Storage capacity of warehouse w (tons)
DCAPd	Storage capacity of distribution center d (tons)
LCAPl	Transportation capacity of mode l (tons)
SLp	Shelf life of paper grade p (periods)
MWp	Minimum order quantity for paper grade p
MXp	Maximum order quantity for paper grade p
FCd	Fixed cost of operating distribution center d
Wλ	Weight for service level in objective function
QMinp	Minimum quality threshold for paper grade p
EmissionRatepm	Emission rate for producing grade p on machine m
EmissionLimit	Total emission limit for the planning horizon
SMinpt, SMaxpt	Seasonal production bounds for grade p in period t
MaxChangeoverm	Maximum changeovers allowed per period on machine m
MinQualitypm	Minimum quality level for grade p produced on machine m
Mbig	Large constant for big-M constraints

capture the fixed and known aspects of the paper mill supply chain. These values include production costs, technical specifications, capacity limitations, and other operational constants that do not vary with uncertainty. The parameters listed in Table 2 are essential for establishing the baseline operational constraints of the optimization model and remain constant throughout the planning horizon.

2.2.2. Uncertain Random Parameters

Our model incorporates uncertainty through a set of random parameters that represent the stochastic nature of real-world paper mill operations. These parameters, presented in

Table 3. Uncertain random parameters for the paper mill supply chain model. This table details all stochastic parameters that are subject to uncertainty, including demand fluctuations, cost variations, and operational variability.

Parameter	Description
${\tilde{D}}_{pct}$	Uncertain random demand of customer c for paper grade p in period t
${\widetilde{PC}}_{pmt}$	Uncertain random production cost of grade p on machine m in period t
${\widetilde{RC}}_{rt}$	Uncertain random cost of raw material r in period t
${\widetilde{TC}}_{ijlt}$	Uncertain random transportation cost from node i to j using mode l in period t
${\widetilde{HC}}_{pt}$	Uncertain random holding cost of paper grade p in period t
${\widetilde{BC}}_{pct}$	Uncertain random backorder cost for grade p at customer c in period t
${\tilde{\eta }}_{pmt}$	Uncertain random machine efficiency for grade p on machine m in period t
${\tilde{\theta }}_{rt}$	Uncertain random raw material availability of type r in period t
${\tilde{\gamma }}_{pt}$	Uncertain random quality yield for paper grade p in period t
${\tilde{\varphi }}_{lt}$	Uncertain random transportation mode availability l in period t
${\tilde{\omega }}_{mt}$	Uncertain random machine breakdown probability for machine m in period t

and denoted with a tilde, capture the variability in demand, costs, equipment performance, and material availability. The uncertain random parameters outlined in Table 3 are essential for developing robust solutions that can withstand real-world fluctuations in the paper manufacturing environment.

2.2.3. Goal Programming Parameters

The goal programming component of our model requires specific parameters to define target thresholds and confidence levels. The parameters listed in

Table 4. Goal programming parameters for the paper mill supply chain model. This table defines the parameters used in the goal programming formulation, including target probability levels, threshold values, and confidence levels for each objective.

Parameter	Description
α1	Target probability level for economic efficiency goal
α2	Target probability level for operational performance goal
α3	Target probability level for resource utilization goal
α4	Target probability level for quality assurance goal
Target1	Target threshold for total cost
Target2	Target threshold for service level
Target3	Target threshold for capacity utilization
Target4	Target threshold for quality performance
βj	Confidence level for chance constraint j

establish the multi-objective framework that balances economic efficiency, operational performance, resource utilization, and quality assurance within the optimization process. Table 4 defines the target probability levels and thresholds that will be used to evaluate the achievement of each goal in our model.

2.3. Decision Variables

2.3.1. Upper-Level Decision Variables (Distribution Planning)

Our bi-level optimization model separates decision-making into distribution and production planning. The upper-level variables defined in

Table 5. Upper-level decision variables for distribution planning in the paper mill supply chain model. This table defines all variables related to distribution network configuration, product flow, and customer service.

Variable	Description
Yd ∈ {0, 1}	1 if distribution center d is operated, 0 otherwise
Xwdplt ≥ 0	Quantity of grade p shipped from warehouse w to DC d using mode l in period t
Zdcplt ≥ 0	Quantity of grade p shipped from DC d to customer c using mode l in period t
Ult ∈ {0, 1}	1 if transportation mode l is used in period t, 0 otherwise
$I_{dpt}^D\ge 0$	Inventory of grade p at distribution center d at end of period t
Bpct ≥ 0	Backorder quantity of grade p for customer c in period t
λpt ∈ [0, 1]	Service level for paper grade p in period t

focus on distribution center operations, product shipping decisions, transportation mode selection, inventory management at distribution centers, and customer service levels. The variables presented in Table 5 determine the optimal distribution network configuration to meet customer demands efficiently.

2.3.2. Lower-Level Decision Variables (Production Planning)

The lower-level variables focus on production planning decisions within the paper mill. The variables defined in

Table 6. Lower-level decision variables for production planning in the paper mill supply chain model. This table defines all variables related to production decisions, machine scheduling, raw material procurement, and inventory management at production facilities.

Variable	Description
Qpmt ≥ 0	Quantity of grade p produced on machine m in period t
Vpmt ∈ {0, 1}	1 if grade p is produced on machine m in period t, 0 otherwise
$W_{p_i{p}_{j}mt}\in \{0,1\}$	1 if changeover from grade pi to pj on machine m in period t, 0 otherwise
Fwplt ≥ 0	Quantity of grade p shipped from machine location to warehouse w using mode l in period t
$I_{wpt}^W\ge 0$	Inventory of grade p at warehouse w at end of period t
Rsrt ≥ 0	Quantity of raw material r purchased from supplier s in period t
$I_{rt}^R\ge 0$	Inventory of raw material r at end of period t

determine optimal production quantities, machine assignments, production sequencing, raw material procurement, and warehouse inventory management. The lower-level decisions represented in Table 6 are made in response to the distribution requirements established at the upper level of the optimization model.

2.3.3. Goal Programming Variables

The goal programming framework requires specific variables to measure deviations from target thresholds. The variables presented in

Table 7. Goal programming variables for the paper mill supply chain model. This table defines the deviation variables used to measure the achievement of each goal in the multi-objective framework.

Variable	Description
$d_1^{+},d_1^{-}\ge 0$	Positive and negative deviations from economic efficiency goal
$d_2^{+},d_2^{-}\ge 0$	Positive and negative deviations from operational performance goal
$d_3^{+},d_3^{-}\ge 0$	Positive and negative deviations from resource utilization goal
$d_4^{+},d_4^{-}\ge 0$	Positive and negative deviations from quality assurance goal

quantify both positive and negative deviations from the four main objectives of our model: economic efficiency, operational performance, resource utilization, and quality assurance. These deviations defined in Table 7 are minimized in the objective function to achieve balanced solutions that satisfy multiple competing objectives simultaneously.

2.4. Bi-Level Mathematical Model

Notation: Throughout this paper, Ch{·} denotes the chance measure of uncertain random events, which quantifies the probability that an uncertain random variable satisfies a given condition. This measure integrates probability theory for aleatory uncertainty (randomness from historical variability) with uncertainty theory for epistemic uncertainty (imprecision from limited knowledge) within a unified mathematical framework, as detailed in the following definition.

2.4.1. Upper-Level Problem (Distribution Planning)

The upper level seeks to minimize distribution costs while maximizing service levels:

$$\begin{array}{cc}\hfill \underset{Y,X,Z,U,I^D,B,\lambda }{min}{F}^{UL}=& {\displaystyle \sum _{d=1}^d}F{C}_d·Y_d+{\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{l=1}^l}{\displaystyle \sum _{t=1}^t}{\widetilde{TC}}_{wdlt}·X_{wdplt}\hfill \\ \hfill & +{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{c=1}^c}{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{l=1}^l}{\displaystyle \sum _{t=1}^t}{\widetilde{TC}}_{dclt}·Z_{dcplt}\hfill \\ \hfill & +{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{t=1}^t}{\widetilde{HC}}_{pt}·I_{dpt}^D\hfill \\ \hfill & +{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{c=1}^c}{\displaystyle \sum _{t=1}^t}{\widetilde{BC}}_{pct}·B_{pct}\hfill \\ \hfill & -{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{t=1}^t}{W}_{\lambda }·{\lambda }_{pt}\hfill \end{array}$$

(1)

Subject to upper-level constraints:

Distribution Center Capacity:

$$\begin{array}{c}\hfill {\displaystyle \sum _{p=1}^p}{I}_{dpt}^D\le DCA{P}_d·Y_d,\forall d,t\end{array}$$

(2)

This constraint ensures that the total inventory stored across all paper grades at each distribution center does not exceed the facility’s physical storage capacity, maintaining operational feasibility.

Transportation Mode Capacity:

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{p=1}^p}{X}_{wdplt}+{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{c=1}^c}{\displaystyle \sum _{p=1}^p}{Z}_{dcplt}\le LCA{P}_l·{\tilde{\varphi }}_{lt}·U_{lt}\right\}\ge {\beta }_1,\forall l,t\end{array}$$

(3)

These constraints guarantee that the combined shipment volumes using each transportation mode remain within the mode’s available capacity limits, accounting for uncertain availability factors.

Customer Demand Satisfaction with Production Coordination:

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{l=1}^l}{Z}_{dcplt}+B_{pc,t-1}-B_{pct}={\tilde{D}}_{pct}\right\}\ge {\beta }_2,\forall p,c,t\end{array}$$

(4)

$$\begin{array}{c}\hfill {\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{c=1}^c}{\displaystyle \sum _{l=1}^l}{Z}_{dcplt}\le {\displaystyle \sum _{w=1}^w}{I}_{wpt}^W+{\displaystyle \sum _{m=1}^m}{Q}_{pmt},\forall p,t\end{array}$$

(5)

Constraint (4) maintains the demand–supply balance under uncertainty, while constraint (5) ensures that total customer shipments cannot exceed available warehouse inventory plus current production, creating explicit coordination between upper-level distribution decisions and lower-level production outputs.

Distribution Center Inventory Balance with Material Flow:

$$\begin{array}{c}\hfill I_{dp,t-1}^D+{\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{l=1}^l}{X}_{wdplt}-{\displaystyle \sum _{c=1}^c}{\displaystyle \sum _{l=1}^l}{Z}_{dcplt}=I_{dpt}^D,\forall d,p,t\end{array}$$

(6)

$$\begin{array}{c}\hfill {\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{l=1}^l}{X}_{wdplt}\le {\displaystyle \sum _{m=1}^m}{Q}_{pmt}+I_{wp,t-1}^W,\forall p,t\end{array}$$

(7)

Constraint (6) preserves inventory continuity at distribution centers, while constraint (7) ensures that warehouse shipments to distribution centers are bounded by current production plus previous warehouse inventory, establishing the material flow dependency between production and distribution levels.

Service-Level Definition:

$$\begin{array}{c}\hfill Ch\left\{{\lambda }_{pt}={\displaystyle \frac{{\sum }_{d=1}^d{\sum }_{c=1}^c{\sum }_{l=1}^l{Z}_{dcplt}}{{\sum }_{c=1}^c{\tilde{D}}_{pct}}}\right\}\ge {\beta }_3,\forall p,t\end{array}$$

(8)

This constraint establishes the service level metric by computing the ratio of fulfilled demand to total customer demand for each product in each time period.

2.4.2. Lower-Level Problem (Production Planning)

Given the upper-level decisions, the lower level minimizes production costs:

$$\begin{array}{cc}\hfill \underset{Q,V,W,F,I^W,R,I^R}{min}{F}^{LL}=& {\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{m=1}^m}{\displaystyle \sum _{t=1}^t}{\widetilde{PC}}_{pmt}·Q_{pmt}+{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{m=1}^m}{\displaystyle \sum _{t=1}^t}{K}_{pm}·V_{pmt}\hfill \\ \hfill & +{\displaystyle \sum _{p_i=1}^p}{\displaystyle \sum _{p_j=1}^p}{\displaystyle \sum _{m=1}^m}{\displaystyle \sum _{t=1}^t}S{C}_{p_i{p}_{j}m}·W_{p_i{p}_{j}mt}\hfill \\ \hfill & +{\displaystyle \sum _{s=1}^s}{\displaystyle \sum _{r=1}^r}{\displaystyle \sum _{t=1}^t}{\widetilde{RC}}_{rt}·R_{srt}\hfill \\ \hfill & +{\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{p=1}^p}{\displaystyle \sum _{t=1}^t}{\widetilde{HC}}_{pt}·I_{wpt}^W\hfill \\ \hfill & +{\displaystyle \sum _{r=1}^r}{\displaystyle \sum _{t=1}^t}{\widetilde{HC}}_{rt}·I_{rt}^R\hfill \end{array}$$

(9)

Subject to lower-level constraints:

Machine Capacity:

$$\begin{array}{c}\hfill Ch\left\{\sum _{p\in P{M}_m}{\displaystyle \frac{Q_{pmt}}{{\beta }_{pm}·{\tilde{\eta }}_{pmt}}}+{\displaystyle \sum _{p_i=1}^p}{\displaystyle \sum _{p_j=1}^p}S{T}_{p_i{p}_{j}m}·W_{p_i{p}_{j}mt}\le CA{P}_m·(1-{\tilde{\omega }}_{mt})\right\}\ge {\beta }_4,\forall m,t\end{array}$$

(10)

This constraint controls machine capacity limitations by accounting for production time requirements, changeover durations, efficiency variations, and potential equipment breakdowns.

Machine–Product Compatibility:

$$\begin{array}{c}\hfill V_{pmt}=0,\forall p\notin P{M}_m,\forall m,t\end{array}$$

(11)

These restrictions prevent the assignment of paper grades to incompatible machines, maintaining technical and operational feasibility in production scheduling.

Production–Setup Linking:

$$\begin{array}{cc}\hfill Q_{pmt}& \le M_{big}·V_{pmt},\forall p,m,t\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill Q_{pmt}& \ge M{W}_p·V_{pmt},\forall p,m,t\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill Q_{pmt}& \le M{X}_p·V_{pmt},\forall p,m,t\hfill \end{array}$$

(14)

The first constraint links production quantities to setup decisions using big-M methodology. The second ensures minimum production batch sizes when a setup occurs. The third controls maximum production limits per setup, maintaining cost-effective production runs.

Changeover Logic:

$$\begin{array}{cc}\hfill \sum _{p_j\ne p_i}{W}_{p_i{p}_{j}mt}& =V_{p_i,m,t-1}-V_{p_i,mt},\forall p_i,m,t\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \sum _{p_i\ne p_j}{W}_{p_i{p}_{j}mt}& =V_{p_j,mt}-V_{p_j,m,t-1},\forall p_j,m,t\hfill \end{array}$$

(16)

These logical constraints control machine changeover sequences by tracking transitions between different paper grades while maintaining temporal consistency in production scheduling.

Color/Grade Transition Restrictions:

$$\begin{array}{c}\hfill W_{p_i{p}_{j}mt}=0,\forall (p_i,p_j)\in {\mathrm{\Phi }}_{incompatible},\forall m,t\end{array}$$

(17)

This constraint prevents direct transitions between incompatible paper grades (e.g., dark to light colors), ensuring product quality and reducing contamination risks.

Maximum Daily Changeovers:

$$\begin{array}{c}\hfill {\displaystyle \sum _{p_i=1}^p}\sum _{p_j\ne p_i}{W}_{p_i{p}_{j}mt}\le MaxChangeove{r}_m,\forall m,t\end{array}$$

(18)

These limitations control the total number of product changeovers per machine per period, balancing production flexibility with setup cost management and operational stability.

Production–Distribution Material Flow Balance:

$$\begin{array}{c}\hfill {\displaystyle \sum _{m=1}^m}{Q}_{pmt}={\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{l=1}^l}{F}_{wplt},\forall p,t\end{array}$$

(19)

This constraint ensures that total production output for each paper grade equals total shipments from production facilities to warehouses, creating the fundamental link between lower-level production decisions and upper-level distribution planning.

Quality Grade Production Requirements:

$$\begin{array}{c}\hfill Ch\left\{\sum _{m\in P{M}_p}{Q}_{pmt}·{\tilde{\gamma }}_{pt}\ge QMi{n}_p·\sum _{m\in P{M}_p}{V}_{pmt}\right\}\ge {\beta }_{25},\forall p,t\end{array}$$

(20)

This constraint guarantees that the total quality-adjusted production output for each paper grade meets the minimum quality standards required by customer specifications.

Machine Quality Compatibility:

$$\begin{array}{c}\hfill Ch\left\{Q_{pmt}·{\tilde{\gamma }}_{pt}\ge MinQualit{y}_{pm}·V_{pmt}\right\}\ge {\beta }_{26},\forall p,m\in P{M}_p,t\end{array}$$

(21)

These requirements ensure that each machine–product combination achieves the necessary quality levels, preventing the production of substandard products.

Raw Material Requirements:

$$\begin{array}{c}\hfill Ch\left\{\sum _{p\in P{R}_r}{\alpha }_{pr}·Q_{pmt}·{\tilde{\gamma }}_{pt}\le I_{r,t-1}^R+{\displaystyle \sum _{s=1}^s}{R}_{srt}\right\}\ge {\beta }_5,\forall r,m,t\end{array}$$

(22)

This constraint ensures that sufficient raw materials are available to support the planned production quantities, accounting for quality yield variations and material consumption rates.

Raw Material Inventory Balance:

$$\begin{array}{c}\hfill Ch\left\{I_{r,t-1}^R+{\displaystyle \sum _{s=1}^s}{R}_{srt}-\sum _{p\in P{R}_r}{\displaystyle \sum _{m=1}^m}{\alpha }_{pr}·Q_{pmt}·{\tilde{\gamma }}_{pt}=I_{rt}^R\right\}\ge {\beta }_6,\forall r,t\end{array}$$

(23)

These balance equations maintain continuity of raw material inventories by tracking incoming purchases, consumption in production processes, and ending inventory levels.

Raw Material Availability:

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \sum _{s=1}^s}{R}_{srt}\le {\tilde{\theta }}_{rt}\right\}\ge {\beta }_7,\forall r,t\end{array}$$

(24)

This constraint controls supplier capacity limitations and market availability constraints for raw materials, incorporating supply uncertainty into procurement decisions.

Warehouse Capacity:

$$\begin{array}{c}\hfill {\displaystyle \sum _{p=1}^p}{I}_{wpt}^W\le WCA{P}_w,\forall w,t\end{array}$$

(25)

These capacity restrictions ensure that warehouse storage limits are respected across all product types, preventing facility overutilization and maintaining storage efficiency.

Warehouse Inventory Balance:

$$\begin{array}{c}\hfill I_{wp,t-1}^W+{\displaystyle \sum _{l=1}^l}{F}_{wplt}-{\displaystyle \sum _{d=1}^d}{\displaystyle \sum _{l=1}^l}{X}_{wdplt}=I_{wpt}^W,\forall w,p,t\end{array}$$

(26)

These equations preserve inventory continuity at warehouse facilities by balancing previous inventory levels, incoming production, and outgoing shipments to distribution centers.

2.5. Bi-Level Coordination Algorithm

The coordination between production and distribution decisions requires an iterative approach that ensures feasible material flow while satisfying probabilistic constraints. To address this challenge, we develop a specialized bi-level coordination algorithm that alternates between solving the lower-level production problem and upper-level distribution problem until convergence is achieved. Algorithm 1 presents the detailed coordination procedure.

Algorithm 1 Bi-Level Coordination for Paper Manufacturing Planning

Require: Initial production capacities, demand forecasts, distribution network    1:   Initialize production variables Qpmt and distribution variables Xwdplt, Zdcplt    2:   repeat    3:    Lower Level: Solve production problem to determine Qpmt given current distribution requirements    4:    Update material availability: $AvailableMateria{l}_{pt}={\sum }_{m=1}^m{Q}_{pmt}$    5:    Upper Level: Solve distribution problem with constraint ${\sum }_{w,d,l}{X}_{wdplt}\le AvailableMateria{l}_{pt}$    6:    Calculate normalized coordination gap: $Gap={\displaystyle \frac{|{\sum }_m{Q}_{pmt}-{\sum }_{w,l}{F}_{wplt}|}{{\sum }_m{Q}_{pmt}+{10}^{-6}}}$    7:   until Gap < 0.01 AND all chance constraints satisfied    8:   return Coordinated solution (Q, X, Z)

This algorithm ensures feasible coordination by iteratively solving both levels while enforcing material flow constraints (5), (7), and (19) until convergence. The normalized coordination gap metric represents the relative material flow imbalance as a fraction of total production, with values close to 0 indicating strong bi-level coordination.

2.6. Paper-Manufacturing-Specific Constraints

Paper Grade Quality Requirements:

$$\begin{array}{c}\hfill Ch\left\{Q_{pmt}·{\tilde{\gamma }}_{pt}\ge QMi{n}_p·V_{pmt}\right\}\ge {\beta }_8,\forall p,m,t\end{array}$$

(27)

This constraint requires that quality-adjusted production meets minimum grade specifications, ensuring customer satisfaction and regulatory compliance for paper products.

Shelf-Life Constraints:

$$\begin{array}{}\text{(28)}& \hfill I_{wpt}^W\le & {\displaystyle \sum _{\tau =max(1,t-S{L}_p+1)}^t}{\displaystyle \sum _{l=1}^l}{F}_{wp\tau l},\forall w,p,t\hfill \text{(29)}& \hfill I_{dpt}^D\le & {\displaystyle \sum _{\tau =max(1,t-S{L}_p+1)}^t}{\displaystyle \sum _{w=1}^w}{\displaystyle \sum _{l=1}^l}{X}_{wdp\tau l},\forall d,p,t\hfill \end{array}$$

The first constraint prevents inventory spoilage at warehouses by limiting stock to recently produced items within shelf-life limits. The second maintains similar freshness requirements at distribution centers to maintain product quality.

Seasonal Production Constraints:

$$\begin{array}{cc}\hfill Ch\left\{{\displaystyle \sum _{m=1}^m}{Q}_{pmt}\ge SMi{n}_{pt}·{\displaystyle \sum _{c=1}^c}{\tilde{D}}_{pct}\right\}& \ge {\beta }_9,\forall p\in P_{seasonal},t\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill Ch\left\{{\displaystyle \sum _{m=1}^m}{Q}_{pmt}\le SMa{x}_{pt}·{\displaystyle \sum _{c=1}^c}{\tilde{D}}_{pct}\right\}& \ge {\beta }_{10},\forall p\in P_{seasonal},t\hfill \end{array}$$

(31)

The first constraint controls minimum seasonal production levels relative to demand patterns for specialized paper grades. The second controls upper bounds on seasonal production to prevent overproduction and excess inventory.

Environmental Compliance:

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \sum _{p=1}^P}{\displaystyle \sum _{m=1}^M}{\displaystyle \sum _{t=1}^T}{\widetilde{EmissionRate}}_{pm}·Q_{pmt}\le EmissionLimit\right\}\ge {\beta }_{emission}\end{array}$$

(32)

This constraint ensures environmental compliance by limiting total emissions from production activities over the planning horizon. The uncertain emission rates (${\widetilde{EmissionRate}}_{pm}$) account for variability in production efficiency and equipment performance. The chance constraint formulation guarantees environmental compliance with confidence level β_emission, maintaining consistency with the hybrid uncertainty framework.

The confidence level β_emission = 0.95 follows EPA regulatory standards for industrial emissions compliance [16]. The emission limit of EmissionLimit = 1200 tons CO2-equivalent per planning horizon is based on established benchmarks for medium-scale paper manufacturing facilities [17].

Non-Negativity and Binary Constraints:

$$\begin{array}{cc}\hfill & Q_{pmt},X_{wdplt},Z_{dcplt},F_{wplt},I_{wpt}^W,I_{dpt}^D,I_{rt}^R,R_{srt},B_{pct}\ge 0\hfill \\ \hfill & V_{pmt},W_{p_i{p}_{j}mt},Y_d,U_{lt}\in \{0,1\}\hfill \\ \hfill & d_1^{+},d_1^{-},d_2^{+},d_2^{-},d_3^{+},d_3^{-},d_4^{+},d_4^{-}\ge 0\hfill \\ \hfill & 0\le {\lambda }_{pt}\le 1\hfill \end{array}$$

These constraints establish variable domains, ensuring the physical feasibility of continuous quantities, logical consistency of binary decisions, and proper bounds for service level indicators.

This comprehensive bi-level dependent-chance goal programming model captures the tactical production–distribution planning decisions in paper manufacturing under hybrid uncertainty, where chance measures Ch{·} are evaluated using uncertain random simulation techniques. The model incorporates explicit coordination constraints (5), (7), and (19) that ensure feasible integration between upper-level distribution decisions and lower-level production outputs, addressing the bi-level coupling requirements for tactical planning optimization.

3. Uncertain Random Theory for Hybrid Uncertainty Modeling

Real-world production and distribution systems in paper manufacturing face dual uncertainty challenges that traditional optimization approaches struggle to address adequately. These systems must contend with objective variability inherent in operational processes (aleatory uncertainty) alongside subjective assessments arising from limited information and expert judgments (epistemic uncertainty). While classical probability theory excels at modeling random phenomena and fuzzy set theory handles belief-based information, neither framework alone sufficiently captures scenarios where both uncertainty types coexist simultaneously.

This section develops a comprehensive mathematical framework based on uncertain random theory that unifies both uncertainty dimensions within a single modeling paradigm. Our approach employs chance measures to create integrated representations that facilitate tactical decision-making under hybrid uncertainty conditions in paper manufacturing environments.

3.1. Mathematical Foundation of Uncertain Random Variables

Manufacturing planning decisions frequently involve parameters that exhibit both stochastic behavior and subjective uncertainty components. To address this complexity, we establish uncertain random variables as our fundamental modeling construct, building upon the theoretical framework developed by Liu [18].

Definition 1

(Uncertain Random Variable). Let $(\mathrm{\Omega },\mathcal{A},Pr)$ denote a probability space and consider a mapping $\tilde{\xi }:\mathrm{\Omega }\to \mathcal{U}$, where $\mathcal{U}$ represents the space of uncertain variables. The function $\tilde{\xi }$ constitutes an uncertain random variable when $M\{\tilde{\xi }\left(\omega \right)\in B\}$ remains measurable with respect to ω for every Borel set $B\subseteq \mathbb{R}$.

This mathematical structure captures the dual nature of hybrid uncertainty through its composition:

• Stochastic Component: The probability space $(\mathrm{\Omega },\mathcal{A},Pr)$ models objective randomness derived from historical data and measurable variations.
• Epistemic Component: For each probabilistic outcome ω, the mapping $\tilde{\xi }\left(\omega \right)$ yields an uncertain variable representing subjective assessments and expert knowledge.

Manufacturing Context Example: Consider demand forecasting for paper grade p from customer c in period t, denoted as ${\tilde{D}}_{pct}$. The stochastic dimension captures quantifiable demand patterns extracted from sales history, while the uncertain dimension incorporates qualitative factors such as market sentiment, competitive dynamics, and economic indicators that resist precise probabilistic characterization.

3.2. Chance Measure Framework

The integration of probability and uncertainty measures requires a unified metric capable of handling both components simultaneously. We employ the chance measure, which provides this integration through the following mathematical construction.

Definition 2

(Chance Measure). Given an uncertain random variable $\tilde{\xi }$ and a Borel set $B\subseteq \mathbb{R}$, the chance measure of the event $\{\tilde{\xi }\in B\}$ is defined as:

$$Ch\{\tilde{\xi }\in B\}={\int }_0^{1}Pr\{\omega \in \mathrm{\Omega }:M\{\tilde{\xi }\left(\omega \right)\in B\}\ge r\}dr$$

The chance measure satisfies essential mathematical properties that ensure consistent behavior:

• Normalization: $Ch\{\tilde{\xi }\in \mathbb{R}\}=1$
• Self-Duality: $Ch\{\tilde{\xi }\in B\}+Ch\{\tilde{\xi }\in B^c\}=1$
• Monotonicity: $B_1\subseteq B_2\Rightarrow Ch\{\tilde{\xi }\in B_1\}\le Ch\{\tilde{\xi }\in B_2\}$

Boundary Behavior: The chance measure reduces to familiar measures in limiting cases:

• When uncertainty vanishes, $Ch\{\tilde{\xi }\in B\}=Pr\{X\in B\}$ for random variable X
• When randomness vanishes, $Ch\{\tilde{\xi }\in B\}=M\{\eta \in B\}$ for uncertain variable η

3.3. Distributional Characterization

Definition 3

(Chance Distribution Function). The chance distribution function of uncertain random variable $\tilde{\xi }$ is given by:

$$\mathrm{\Phi }\left(x\right)=Ch\{\tilde{\xi }\le x\},x\in \mathbb{R}$$

Theorem 1

(Distribution Characterization). A function $\mathrm{\Phi }:\mathbb{R}\to [0,1]$ represents a valid chance distribution if and only if it is non-decreasing with ${lim}_{x\to -\infty }\mathrm{\Phi }\left(x\right)=0$ and ${lim}_{x\to +\infty }\mathrm{\Phi }\left(x\right)=1$.

3.4. Statistical Measures

For practical decision-making applications, we require scalar summary statistics that characterize uncertain random variable behavior.

Definition 4

(Expected Value). The expected value of uncertain random variable $\tilde{\xi }$ is:

$$E\left[\tilde{\xi }\right]={\int }_0^{+\infty }Ch\{\tilde{\xi }\ge r\}dr-{\int }_{-\infty }^{0}Ch\{\tilde{\xi }\le r\}dr$$

when at least one integral converges.

Definition 5

(Variance). For uncertain random variable $\tilde{\xi }$ with finite expected value $\mu =E\left[\tilde{\xi }\right]$, the variance is:

$$Var\left[\tilde{\xi }\right]=E\left[{(\tilde{\xi }-\mu )}^2\right]$$

Theorem 2

(Linearity Property). For uncertain random variable $\tilde{\xi }$ with finite expectation and constants $a,b\in \mathbb{R}$:

$$E[a\tilde{\xi }+b]=aE\left[\tilde{\xi }\right]+b$$

3.5. Dependent-Chance Goal Programming Methodology

Traditional optimization approaches in manufacturing planning typically focus on constraint satisfaction at predetermined confidence levels. However, tactical planning benefits from directly optimizing the likelihood of achieving strategic objectives. Dependent-chance goal programming addresses this need by incorporating probabilistic objective achievement into the optimization framework.

Definition 6

(Dependent-Chance Programming). Given uncertain random variable $\tilde{\xi }$ and target threshold F, dependent-chance programming seeks to maximize $Ch\{\tilde{\xi }\le F\}$ or $Ch\{\tilde{\xi }\ge F\}$ depending on the objective orientation.

For multi-objective tactical planning scenarios involving competing priorities, we extend this concept through goal programming principles:

• Target Specification: Decision-makers establish desired achievement probabilities α_g for each tactical objective g
• Deviation Modeling: Variables $d_g^{+}$ and $d_g^{-}$ capture positive and negative deviations from target achievement levels
• Hierarchical Optimization: Objectives receive priority rankings and undergo sequential optimization based on strategic importance

The general mathematical formulation follows:

$$\begin{array}{}\text{(33)}& \hfill \mathrm{lexicographic} \mathrm{minimize}& \{d_1^{-},d_2^{-},\dots ,d_n^{-}\}\hfill \text{(34)}& \hfill \mathrm{subject} \mathrm{to}& Ch\{f_g(\mathbf{x},\tilde{\mathit{\xi }})\bowtie F_g\}+d_g^{-}-d_g^{+}={\alpha }_g,g=1,\dots ,n\hfill \text{(35)}& \hfill & \begin{array}{c}\mathbf{x}\in \mathcal{X}\hfill \end{array}\hfill \text{(36)}& \hfill & \begin{array}{c}{d}_g^{+},d_g^{-}\ge 0,g=1,\dots ,n\hfill \end{array}\hfill \end{array}$$

where $f_g(\mathbf{x},\tilde{\mathit{\xi }})$ represents the g-th objective function, ⋈ denotes the appropriate inequality relation, F_g specifies the target threshold, and $\mathcal{X}$ defines the feasible decision space.

Methodological Advantages:

• Decision Transparency: Probability-based metrics provide intuitive interpretation for management
• Priority Accommodation: Lexicographic structure aligns with organizational strategic hierarchies
• Robustness Enhancement: Probabilistic focus inherently addresses uncertainty impacts
• Unified Treatment: Chance measures handle both aleatory and epistemic uncertainties consistently

3.6. Paper Manufacturing Parameter Modeling

Paper manufacturing tactical planning involves numerous parameters exhibiting hybrid uncertainty characteristics. Our uncertain random variable framework provides natural representations for key planning elements:

• Demand Forecasting ${\tilde{D}}_{pct}$: Combines quantitative sales patterns with qualitative market intelligence and customer behavior assessments
• Production Costs ${\widetilde{PC}}_{pmt}$: Integrates observable input price volatility with subjective cost estimations for different paper grades and machine configurations
• Raw Material Availability ${\widetilde{RC}}_{rt}$: Merges market price data with uncertain supply chain assessments for pulp and recycled material sources
• Transportation Expenses ${\widetilde{TC}}_{ijlt}$: Combines fuel cost fluctuations with uncertain capacity availability and route condition evaluations
• Quality Performance ${\tilde{\gamma }}_{pt}$: Unifies quantitative quality measurements with expert assessments of production environment factors
• Equipment Efficiency ${\tilde{\eta }}_{pmt}$: Integrates historical machine performance with expert judgments regarding maintenance requirements and operational conditions

This modeling approach enables more realistic representation of paper manufacturing uncertainty while maintaining mathematical rigor necessary for optimization-based tactical planning.

3.7. Tactical Planning Application Framework

We now present the specific dependent-chance goal programming formulation for paper manufacturing tactical planning, considering the following strategic priority hierarchy:

Priority Level 1—Economic Performance: The probability of total bi-level cost remaining below threshold Target₁ should achieve confidence level α₁.

$$\begin{array}{c}\hfill Ch\{F^{UL}+F^{LL}\le Targe{t}_1\}+d_1^{-}-d_1^{+}={\alpha }_1\end{array}$$

(37)

Priority Level 2—Service Excellence: The probability of average service level exceeding threshold Target₂ should achieve confidence level α₂.

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \frac{1}{\left|P\right|\times \left|T\right|}}\sum _{p,t}{\lambda }_{pt}\ge Targe{t}_2\right\}+d_2^{-}-d_2^{+}={\alpha }_2\end{array}$$

(38)

Priority Level 3—Resource Optimization: The probability of average capacity utilization exceeding threshold Target₃ should achieve confidence level α₃.

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \frac{1}{{\sum }_{m,t}1}}\sum _{m,t}\sum _p{\displaystyle \frac{Q_{pmt}}{{\beta }_{pm}·CA{P}_m}}\ge Targe{t}_3\right\}+d_3^{-}-d_3^{+}={\alpha }_3\end{array}$$

(39)

Priority Level 4—Quality Assurance: The probability of average quality performance exceeding threshold Target₄ should achieve confidence level α₄.

$$\begin{array}{c}\hfill Ch\left\{{\displaystyle \frac{1}{\left|P\right|\times \left|T\right|}}\sum _{p,t}{\tilde{\gamma }}_{pt}\ge Targe{t}_4\right\}+d_4^{-}-d_4^{+}={\alpha }_4\end{array}$$

(40)

The complete tactical planning model becomes:

$$\begin{array}{}\text{(41)}& \hfill \mathrm{lexicographic} \mathrm{minimize}& \begin{array}{c}\{d_1^{-},d_2^{-},d_3^{-},d_4^{-}\}\hfill \end{array}\hfill \text{(42)}& \hfill \mathrm{subject} \mathrm{to}& \begin{array}{c}\text{Equations (49)–(52)}\hfill \end{array}\hfill \text{(43)}& \hfill & \begin{array}{c}\mathrm{Production}\mathrm{and}\mathrm{distribution}\mathrm{constraints}\hfill \end{array}\hfill \text{(44)}& \hfill & \begin{array}{c}{d}_g^{+},d_g^{-}\ge 0,g=1,2,3,4\hfill \end{array}\hfill \end{array}$$

This formulation systematically minimizes under-achievement of target probability levels across all tactical objectives. The first priority addresses cost minimization (seeking performance below threshold), while subsequent priorities target performance maximization (seeking performance above thresholds). The chance measure framework handles both random and uncertain parameter components while providing intuitive probabilistic interpretations for tactical decision-makers.

3.8. Implementation Considerations

The proposed uncertain random theory framework offers several practical advantages for paper manufacturing tactical planning:

• Comprehensive Objective Integration: Simultaneously addresses economic efficiency, operational performance, resource utilization, and quality assurance within a unified bi-level optimization framework
• Hybrid Uncertainty Management: Effectively handles both quantifiable variations and subjective assessments in critical parameters including demand, costs, efficiency, and quality metrics
• Confidence-Based Planning: Enables tactical planners to specify desired confidence levels for goal achievement across the planning horizon
• Systematic Multi-Objective Balancing: Provides structured approach to managing competing tactical objectives while considering integrated production-distribution decisions

The dependent-chance goal programming methodology prioritizes tactical goals through lexicographic optimization while accounting for complex interactions between random and uncertain variables characteristic of paper manufacturing environments. This approach generates operationally viable solutions that demonstrate robustness against the diverse uncertainty sources encountered in tactical production-distribution planning contexts.

4. Hybrid Intelligent Algorithm

Addressing the computational complexity inherent in our bi-level dependent-chance goal programming formulation for tactical paper manufacturing planning necessitates a sophisticated algorithmic approach. We propose a hybrid computational framework that synergistically combines uncertain random simulation methodologies with reinforcement learning-augmented arithmetic optimization techniques. This algorithmic architecture is specifically designed to navigate the intricate landscape of hybrid uncertainties while managing the hierarchical structure of our bi-level optimization model.

4.1. Uncertain Random Simulations

The computational evaluation of chance measures within our bi-level tactical planning framework requires specialized simulation methodologies to handle expressions of the form Ch{g_j(x, ξ) ≤ 0} ≥ β_j. These chance-constrained formulations represent fundamental components of our paper manufacturing optimization model, where accurate estimation becomes critical for obtaining feasible and robust tactical decisions.

The mathematical foundation underlying chance measure evaluation rests on recognizing that Ch{g(x, ξ) ≤ 0} constitutes the expected value of uncertain measures $\mathcal{M}\left\{g\right(\mathbf{x},\mathit{\xi }\left(\omega \right))\le 0\}$ across all possible random realizations. Drawing inspiration from Monte Carlo principles [19], we construct a specialized computational approach tailored for chance-measure approximation in tactical planning contexts.

Our uncertain random simulation methodology serves as the computational backbone for evaluating probabilistic constraints under hybrid uncertainty conditions characteristic of paper manufacturing environments. The following Algorithm 2 establishes our Monte Carlo framework that systematically samples from random distributions and computes uncertain measures for each realization, ultimately yielding reliable chance measure estimates essential for production-distribution decision-making.

Algorithm 2 Uncertain Random Simulation for Chance Measure Estimation

Description: This algorithm estimates the chance measure of uncertain random events by employing Monte Carlo sampling to approximate the expected value of uncertain measures across various random scenarios in paper manufacturing planning.    1:   Initialize counter ← 0    2:   for k = 1 to NMC do    3:    Generate random sample ωk based on its probability distribution    4:    Compute $\mathcal{M}\{g(\mathbf{x},\mathit{\xi }\left({\omega }_k\right))\le 0\}$ using Algorithm 3    5:    $counter\leftarrow counter+\mathcal{M}\{g(\mathbf{x},\mathit{\xi }\left({\omega }_k\right))\le 0\}$    6:   end for    7:   return $counter/N_{MC}$ as the estimated chance measure

The evaluation of uncertain measures requires implementing uncertainty theory principles [20] to assess constraint satisfaction under subjective uncertainty conditions. Algorithm 3 implements an inverse distribution approach for generating uncertain variable realizations and determining constraint satisfaction at specified uncertainty thresholds.

Algorithm 3 Uncertain Measure Computation

Description: This algorithm computes the uncertain measure for a specified constraint by employing the inverse distribution function approach to generate uncertain variable realizations and evaluate constraint satisfaction in tactical planning contexts.    1:   Generate α uniformly from [0, 1]    2:   Compute τ−1(α) for uncertain variables τ in ξ(ωk)    3:   if g(x, ξ(ωk)|τ=τ−1(α)) ≤ 0 then    4:    return 1    5:   else    6:    return 0    7:   end if

4.2. Theoretical Foundation of Exploration-Exploitation Trade-Off

The fundamental challenge in metaheuristic optimization lies in achieving optimal balance between search space exploration and solution refinement (exploitation). Following the theoretical framework established by Morales-Castañeda et al. [21], this balance can be quantitatively assessed through diversity metrics that capture population distribution characteristics throughout the search landscape.

Consider a population consisting of N individuals operating within a D-dimensional decision space. The diversity measure at iteration t is mathematically expressed as:

$$\mathrm{Diversity}(t)={\displaystyle \frac{1}{D}}{\displaystyle \sum _{j=1}^D}\sqrt{{\displaystyle \frac{1}{N}{\displaystyle \sum _{i=1}^N}}{(x_{i,j}(t)-{\overline{x}}_j(t))}^2}$$

(45)

where x_i,j(t) denotes the j-th dimensional component of the i-th individual at iteration t, and ${\overline{x}}_j\left(t\right)$ represents the population centroid in dimension j. The corresponding exploration and exploitation percentages are computed as:

$$\begin{array}{}\text{(46)}& \hfill \mathrm{Exploration}\%\left(t\right)& ={\displaystyle \frac{\mathrm{Diversity}\left(t\right)}{{\mathrm{Diversity}}_{max}}}\times 100\hfill \text{(47)}& \hfill \mathrm{Exploitation}\%\left(t\right)& =100-\mathrm{Exploration}\%\left(t\right)\hfill \end{array}$$

Balance quality assessment employs incremental-decremental analysis:

$$\begin{array}{}\text{(48)}& \hfill \mathrm{Incremental}\left(t\right)& =max(0,\mathrm{Diversity}(t)-\mathrm{Diversity}(t-1\left)\right)\hfill \text{(49)}& \hfill \mathrm{Decremental}\left(t\right)& =max(0,\mathrm{Diversity}(t-1)-\mathrm{Diversity}(t\left)\right)\hfill \text{(50)}& \hfill \mathrm{Balance}\mathrm{Quality}\left(t\right)& =\mathrm{Incremental}\left(t\right)-\mathrm{Decremental}\left(t\right)\hfill \end{array}$$

4.3. Reinforcement-Learning-Enhanced Arithmetic Optimization Algorithm

Our bi-level chance-constrained goal programming model demands a sophisticated optimization approach capable of coordinating tactical production-distribution decisions under hybrid uncertainty. We develop an enhanced Arithmetic Optimization Algorithm (AOA) framework augmented with reinforcement learning capabilities to address these computational challenges effectively.

4.3.1. Introduction to the Arithmetic Optimization Algorithm

The Arithmetic Optimization Algorithm draws inspiration from fundamental mathematical operations: addition (+), subtraction (−), multiplication (×), and division (÷) [22]. This metaheuristic approach simulates arithmetic operations to achieve effective search space exploration and exploitation, making it particularly well-suited for complex bi-level optimization challenges in paper manufacturing contexts.

Several distinctive characteristics render AOA especially effective for tactical production-distribution planning applications:

• Mathematical Optimizer (MO) Framework: AOA incorporates a Mathematical Optimizer mechanism that dynamically determines exploration versus exploitation strategies based on the Mathematical Optimizer Accelerated (MOA) coefficient:

  $$MOA\left(C_{iter}\right)=Min+C_{iter}\times {\displaystyle \frac{Max-Min}{C_{Max}}}$$

  (51)

  where C_iter represents the current iteration number, C_Max denotes maximum iterations, and Min, Max define operational bounds.
• Arithmetic Operation Strategy: AOA employs four fundamental arithmetic operators for position updating:
  • Addition and subtraction operations facilitate exploration
  • Multiplication and division operations enable exploitation
• Adaptive Search Framework: The algorithm dynamically transitions between exploration and exploitation phases based on optimization progress, providing ideal coordination capabilities for bi-level optimization.
• Hierarchical Optimization Support: AOA’s mathematical foundation naturally accommodates hierarchical optimization scenarios where upper-level distribution decisions and lower-level production decisions require coordination through arithmetic operations.

These algorithmic characteristics make AOA particularly suitable for solving our bi-level dependent-chance goal programming model with hybrid uncertainty in paper manufacturing tactical planning.

4.3.2. Basic Arithmetic Optimization Algorithm

The fundamental AOA framework operates through the following computational phases:

• Population Initialization: Generate N_p solution candidates randomly within feasible regions for both upper- and lower-level variables.
• Mathematical Optimizer Computation: Calculate the Mathematical Optimizer Accelerated function:

  $$MOA\left(C_{iter}\right)=Min+C_{iter}\times {\displaystyle \frac{Max-Min}{C_{Max}}}$$

  (52)
• Mathematical Optimizer Phase: Determine search strategy orientation:

  $$MOP\left(C_{iter}\right)=1-{\displaystyle \frac{C_{iter}^{1/\alpha }}{C_{Max}^{1/\alpha }}}$$

  (53)

  where α controls exploitation precision.
• Exploration Phase (when r₁ > MOA): Apply addition and subtraction operators:

  $$\begin{array}{cc}\hfill x_{i,j}^{t+1}& =bes{t}_j\times MOP\times ((U{B}_j-L{B}_j)\times \mu +L{B}_j),r_2<0.5\hfill \end{array}$$

  (54)

  $$\begin{array}{cc}\hfill x_{i,j}^{t+1}& =bes{t}_j\times MOP\times ((U{B}_j-L{B}_j)\times \mu +L{B}_j),r_2\ge 0.5\hfill \end{array}$$

  (55)
• Exploitation Phase (when r₁ ≤ MOA): Apply multiplication and division operators:

  $$\begin{array}{cc}\hfill x_{i,j}^{t+1}& =bes{t}_j-MOP\times ((U{B}_j-L{B}_j)\times \mu +L{B}_j),r_3<0.5\hfill \end{array}$$

  (56)

  $$\begin{array}{cc}\hfill x_{i,j}^{t+1}& =bes{t}_j+MOP\times ((U{B}_j-L{B}_j)\times \mu +L{B}_j),r_3\ge 0.5\hfill \end{array}$$

  (57)

This fundamental framework establishes the foundation for our enhanced approach, demonstrating core mathematical operations and search mechanisms that underpin our reinforcement learning enhancements.

4.3.3. Reinforcement Learning Enhancement

We augment the basic AOA framework with reinforcement learning methodologies [23] to enhance convergence characteristics and solution quality for bi-level tactical planning applications. The RL integration encompasses:

• State Space Representation: The state vector incorporates discretized optimization metrics for both hierarchical levels:

  $$\begin{array}{c}\hfill s_t=\{diversit{y}_d^{UL},diversit{y}_d^{LL},{\mathit{convergence\_rate}}_d,stagnatio{n}_d,{\mathit{bilevel\_gap}}_d\}\end{array}$$

  (58)

  where continuous metrics undergo discretization into three categories (Low, Medium, High) according to tactical planning requirements.
• Action Space Definition: Actions involve adjusting AOA parameters and bi-level coordination strategies:
  • Action 1: Enhance exploration intensity;
  • Action 2: Intensify exploitation focus;
  • Action 3: Balance exploration-exploitation dynamics;
  • Action 4: Strengthen bi-level coordination.
• Reward Function Structure: The reward mechanism incorporates both solution quality and bi-level coordination effectiveness:

  $$R_t=\left\{\begin{array}{cc}{\displaystyle \frac{f_{t-1}^{UL}+f_{t-1}^{LL}-(f_t^{UL}+f_t^{LL})}{|f_{t-1}^{UL}+f_{t-1}^{LL}|+ϵ}}+\lambda ·coor{d}_t\hfill & \mathrm{if} \mathrm{improvement}\hfill \\ -penalty\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (59)

  where coord_t quantifies bi-level coordination quality and λ represents coordination weighting.
• Q-Learning Implementation: We implement Q-learning [24] with epsilon-greedy action selection:

  $$Q(s_t,a_t)\leftarrow Q(s_t,a_t)+\alpha [R_t+\gamma \underset{a}{max}Q(s_{t+1},a)-Q(s_t,a_t)]$$

  (60)

  where α denotes learning rate, γ represents discount factor, s_t is current state, and a_t is selected action.

4.3.4. Adaptive Parameter Tuning for Bi-Level Coordination

Enhancing bi-level coordination performance requires implementing adaptive parameter tuning mechanisms:

• Mathematical Optimizer Adaptation:

  $$MO{A}^{t+1}=MO{A}_{base}+\varphi ·sin(2\pi ·{\displaystyle \frac{t}{T_{max}}})·\mathit{coordination\_factor}$$

  (61)

  where ϕ controls adaptation intensity and coordination_factor reflects bi-level synchronization requirements.
• Exploration-Exploitation Balance:

  $$\begin{array}{cc}\hfill {\alpha }^{t+1}& ={\alpha }_{min}+({\alpha }_{max}-{\alpha }_{min})·e^{-\rho ·t/T_{max}}\hfill \end{array}$$

  (62)

  $$\begin{array}{cc}\hfill MO{P}^{t+1}& =MO{P}_{base}·(1+\beta ·\mathit{level\_performance})\hfill \end{array}$$

  (63)

  where level_performance measures relative performance between upper and lower levels.
• Bi-Level Coordination Enhancement:

  $${\mathit{coord\_weight}}^{t+1}=coor{d}_{base}·(1+\gamma ·\mathit{gap\_ratio})$$

  (64)

  where gap_ratio quantifies coordination gaps between production and distribution decisions.

4.4. Reinforcement Learning Components

Building upon the fundamental AOA framework, we integrate reinforcement learning methodologies to create an adaptive bi-level optimization system. The RL enhancement incorporates state representation, action space definition, and adaptive parameter control for tactical planning coordination.

4.4.1. State Space Discretization for Bi-Level Planning

Continuous optimization metrics for both hierarchical levels undergo discretization using the following strategy:

• Upper-Level Diversity Measure:

  $$diversit{y}_d^{UL}=\left\{\begin{array}{cc}\mathrm{Low}\hfill & \mathrm{if}diversit{y}^{UL}\le 0.25\hfill \\ \mathrm{Medium}\hfill & \mathrm{if}0.25<diversit{y}^{UL}\le 0.65\hfill \\ \mathrm{High}\hfill & \mathrm{if}diversit{y}^{UL}>0.65\hfill \end{array}\right.$$

  (65)
• Lower-Level Diversity Measure:

  $$diversit{y}_d^{LL}=\left\{\begin{array}{cc}\mathrm{Low}\hfill & \mathrm{if}diversit{y}^{LL}\le 0.30\hfill \\ \mathrm{Medium}\hfill & \mathrm{if}0.30<diversit{y}^{LL}\le 0.70\hfill \\ \mathrm{High}\hfill & \mathrm{if}diversit{y}^{LL}>0.70\hfill \end{array}\right.$$

  (66)
• Bi-Level Gap Measure:

  $${\mathit{bilevel\_gap}}_d=\left\{\begin{array}{cc}\mathrm{Synchronized}\hfill & \mathrm{if}gap\le 0.05\hfill \\ \mathrm{Moderate}\hfill & \mathrm{if}0.05<gap\le 0.20\hfill \\ \mathrm{Divergent}\hfill & \mathrm{if}gap>0.20\hfill \end{array}\right.$$

  (67)

4.4.2. Q-Learning for Bi-Level Parameter Adaptation

We implement Q-learning to adaptively adjust algorithm parameters based on bi-level optimization progress:

Q-learning facilitates algorithm learning of optimal parameter configurations for both production and distribution planning through iterative trial-and-error processes, utilizing temporal difference updates to enhance future decision-making capabilities. This mechanism, as shown in Algorithm 4, enables optimization process adaptation based on observed performance improvements across both levels.

Algorithm 4 Q-Learning Bi-Level Parameter Adaptation

Description: This algorithm implements temporal difference learning to update Q-values for bi-level parameter adaptation, enabling the system to learn optimal parameter settings based on tactical planning performance feedback from both the production and distribution levels. Require: Current state st, action at, reward Rt, next state st+1 Require: Learning rate α, discount factor γ, Q-table Q, exploration rate ϵ      1:   Calculate temporal difference: $\delta =R_t+\gamma {max}_{a^{\prime }}Q(s_{t+1},a^{\prime })-Q(s_t,a_t)$      2:   Update Q-value: Q(st, at) ← Q(st, at) + α · δ      3:   Update bi-level coordination factor: coordfactor ← coordfactor + β · δ      4:   Select next action using epsilon-greedy:      5:   if random() < ϵ then      6:    at+1 ← random action      7:   else      8:    at+1 ← arg maxa Q(st+1, a)      9:   end if    10:   return Updated Q-table, coordination factor, and next action at+1

4.4.3. Adaptive Parameter Control for Tactical Planning

The reinforcement learning component for tactical production-distribution planning, includes:

• State Representation: $s_t=\{diversit{y}_d^{UL},diversit{y}_d^{LL},{\mathit{convergence\_rate}}_d,stagnatio{n}_d,{\mathit{bilevel\_gap}}_d\}$ with 3⁵ = 243 possible discrete states.
• Action Space: Four discrete actions for bi-level parameter adjustment:
  • Action 1: Enhance exploration in both levels;
  • Action 2: Focus exploitation in production level;
  • Action 3: Balance search across levels;
  • Action 4: Strengthen inter-level coordination.
• Reward Function: Bi-level improvement with coordination bonus.
• Action Selection: Epsilon-greedy strategy [23] with adaptive exploration rate.

Algorithm 5 details how these reinforcement learning components are used to dynamically adjust the Arithmetic Optimization Algorithm parameters.

Algorithm 5 RL-Based Adaptive Parameter Control for Bi-Level AOA

Description: This algorithm dynamically adjusts Arithmetic Optimization Algorithm parameters based on reinforcement learning actions, optimizing exploration-exploitation balance for both the production and distribution planning levels. Require: Current iteration t, RL action at, base parameters, bi-level gap      1:   Calculate base MOA: $MO{A}_{base}=Min+t\times {\displaystyle \frac{Max-Min}{T_{max}}}$      2:   Calculate base MOP: $MO{P}_{base}=1-{\displaystyle \frac{t^{1/\alpha }}{T_{max}^{1/\alpha }}}$      3:   if at = 1 then      4:    MOA = MOAbase × 0.7, MOP = MOPbase × 1.4      5:    {Enhance Exploration}      6:    α = 2.5, coord_weight = 0.3      7:   else if at = 2 then      8:    MOA = MOAbase × 1.3, MOP = MOPbase × 0.6      9:    {Focus Exploitation}    10:    α = 4.0, coord_weight = 0.2    11:   else if at = 3 then    12:    MOA = MOAbase, MOP = MOPbase    13:    {Balanced Search}    14:    α = 3.0, coord_weight = 0.4    15:   else    16:    MOA = MOAbase × 1.1, MOP = MOPbase × 0.8    17:    {Strengthen Coordination}    18:    α = 3.5, coord_weight = 0.6    19:   end if    20:   Update exploration rate: ϵt+1 = max(ϵmin, ϵmax · e−σ·t/Tmax)    21:   return Updated parameters $(MOA,MOP,\alpha ,coord\_weight,ϵ)$

4.5. Solution Encoding and Decoding for Bi-Level Planning

Efficient solution representation is crucial for managing the complexity inherent in our bi-level tactical production-distribution planning model. We implement a hierarchical encoding scheme that effectively captures both binary and continuous decision variables across both planning levels.

Bi-Level Variable Encoding Strategy

For upper-level binary decision variables (Y_d, U_lt), we implement sigmoid-based transformation:

$$Y_d=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\sigma \left(z_d^{UL}\right)\ge 0.5\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(68)

where $\sigma \left(z\right)={\displaystyle \frac{1}{1+e^{-z}}}$ represents the sigmoid function.

For lower-level binary decision variables (V_pmt, $W_{p_i{p}_{j}mt}$), we use:

$$V_{pmt}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\sigma \left(z_{pmt}^{LL}\right)\ge 0.5\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(69)

For continuous decision variables at both levels, we employ direct encoding with bound constraints:

$$va{r}^{UL/LL}=l{b}_{var}+(u{b}_{var}-l{b}_{var})\times normalize\left(z_{var}\right)$$

(70)

To handle bi-level dependencies between decision variables:

$$Q_{pmt}=\left\{\begin{array}{cc}{Q}_{pmt}^{\ast },\hfill & \mathrm{if}{Y}_d=1\mathrm{and}{V}_{pmt}=1\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(71)

The solution vectors in our bi-level optimization model are encoded and decoded using Algorithm 6, which transforms continuous metaheuristic solutions into mixed-integer variables while preserving hierarchical dependencies between upper and lower decision levels.

Algorithm 6 Bi-Level Solution Encoding and Decoding

Description: This algorithm transforms continuous solution vectors into mixed-integer solutions for both production and distribution levels by applying sigmoid functions for binary variables and enforcing hierarchical dependencies between bi-level decision variables. Require: Continuous solution vector z = [zUL, zLL]      1:   Upper-Level Binary Variable Decoding:      2:   for each upper-level binary variable Yd, Ult do      3:    $Y_d=\mathbb{I}[\sigma \left(z_d^{UL}\right)\ge 0.5]$, $U_{lt}=\mathbb{I}[\sigma \left(z_{lt}^{UL}\right)\ge 0.5]$      4:   end for      5:   Lower-Level Binary Variable Decoding:      6:   for each lower-level binary variable Vpmt, $W_{p_i{p}_{j}mt}$ do      7:    $V_{pmt}=\mathbb{I}[\sigma \left(z_{pmt}^{LL}\right)\ge 0.5]$, $W_{p_i{p}_{j}mt}=\mathbb{I}[\sigma \left(z_{p_i{p}_{j}mt}^{LL}\right)\ge 0.5]$      8:   end for      9:   Continuous Variable Decoding:    10:   for each continuous variable var at both levels do    11:    var = lbvar + (ubvar − lbvar) · normalize(zvar)    12:   end for    13:   Bi-Level Dependency Enforcement:    14:   for each dependent variable pair across levels do    15:    if upper-level parent variable is 0 then    16:     Set dependent lower-level variables to 0    17:    end if    18:   end for    19:   return Decoded bi-level solution x = [xUL, xLL]

4.6. Constraint Handling Mechanisms for Bi-Level Planning

We implement a hierarchical penalty function approach to handle bi-level dependent-chance constraints:

$$\begin{array}{cc}\hfill F\left(\mathbf{x}\right)& =M^{UL}·{\displaystyle \sum _{j=1}^{m^{UL}}}{\lambda }_j^{UL}\left(t\right)·max(0,{\beta }_j-Ch\{g_j^{UL}\left({\mathbf{x}}^{UL}\right)\le 0\})\hfill \\ \hfill & +M^{LL}·{\displaystyle \sum _{j=1}^{m^{LL}}}{\lambda }_j^{LL}\left(t\right)·max(0,{\beta }_j-Ch\{g_j^{LL}\left({\mathbf{x}}^{LL}\right)\le 0\})\hfill \\ \hfill & +{\displaystyle \sum _{g=1}^4}{w}_g·d_g^{-}\hfill \end{array}$$

(72)

where M^UL and M^LL represent large penalty multipliers for upper- and lower-level constraints, respectively, ensuring constraint violations dominate goal deviations, and ${\lambda }_j^{UL}\left(t\right)$, ${\lambda }_j^{LL}\left(t\right)$ are penalty parameters that increase over iterations:

$${\lambda }_j^{UL}\left(t\right)={\lambda }_j^{UL,0}·(1+{\delta }^{UL}·t),{\lambda }_j^{LL}\left(t\right)={\lambda }_j^{LL,0}·(1+{\delta }^{LL}·t)$$

(73)

For dependent-chance goals requiring coordination between levels, we implement progressive tightening:

$${\beta }_j\left(t\right)={\beta }_j^{final}·(1-e^{-\kappa ·t/T_{max}})$$

(74)

Our optimization model employs Algorithm 7 to handle the complex chance constraints across both planning levels, implementing a dynamic penalty mechanism that evaluates dependent-chance goals while preserving the hierarchical structure between production and distribution decisions.

Algorithm 7 Dynamic Penalty Constraint Handling for Bi-Level Planning

Description: This algorithm evaluates solution fitness by computing dependent-chance goals and constraints for both the production and distribution levels, applying hierarchical penalty functions to handle constraint violations while preserving lexicographic priority and bi-level coordination. Require: Bi-level solution x = [xUL, xLL], iteration t, penalty parameters      1:   Evaluate Dependent-Chance Goals:      2:   for each goal g = 1 to 4 do      3:    Calculate Ch{FUL + FLL ≤ Targetg} using Algorithm 2      4:    Calculate deviation: $d_g^{-}=max(0,{\alpha }_g-Ch\left\{\mathit{goal\_achievement}\right\})$      5:   end for      6:   Evaluate Upper-Level Chance Constraints:      7:   for each upper-level constraint j = 1 to mUL do      8:    Calculate $Ch\{g_j^{UL}\left({\mathbf{x}}^{UL}\right)\le 0\}$ using Algorithm 2      9:    Calculate violation: $vio{l}_j^{UL}=max(0,{\beta }_j-Ch\{g_j^{UL}\left({\mathbf{x}}^{UL}\right)\le 0\})$    10:   end for    11:   Evaluate Lower-Level Chance Constraints:    12:   for each lower-level constraint j = 1 to mLL do    13:    Calculate $Ch\{g_j^{LL}\left({\mathbf{x}}^{LL}\right)\le 0\}$ using Algorithm 2    14:    Calculate violation: $vio{l}_j^{LL}=max(0,{\beta }_j-Ch\{g_j^{LL}\left({\mathbf{x}}^{LL}\right)\le 0\})$    15:   end for    16:   Compute Hierarchical Penalized Fitness:    17:   $F_{penalty}\left(\mathbf{x}\right)=M^{UL}·{\sum }_{j=1}^{m^{UL}}{\lambda }_j^{UL}\left(t\right)·vio{l}_j^{UL}+M^{LL}·{\sum }_{j=1}^{m^{LL}}{\lambda }_j^{LL}\left(t\right)·vio{l}_j^{LL}+{\sum }_{g=1}^4{w}_g·d_g^{-}$    18:   Update Penalty Parameters:    19:   for each constraint j at both levels do    20:    ${\lambda }_j^{UL}\left(t\right)={\lambda }_j^{UL,0}·(1+{\delta }^{UL}·t)$    21:    ${\lambda }_j^{LL}\left(t\right)={\lambda }_j^{LL,0}·(1+{\delta }^{LL}·t)$    22:   end for    23:   return Penalized fitness Fpenalty(x)

4.7. Complete Hybrid Intelligent Algorithm

The comprehensive hybrid intelligent algorithm is structured in a modular framework comprising six specialized components designed for bi-level tactical planning coordination.

Proper initialization proves crucial for bi-level optimization success. This algorithm establishes populations for both levels, configures all learning components, and sets up the parameter framework that guides the entire tactical planning optimization process. The hybrid optimization process begins with Algorithm 8.

Algorithm 8 Bi-Level Algorithm Initialization

Description: This algorithm initializes populations for both production and distribution planning levels, Q-learning components, and all algorithm parameters, establishing the foundation for the hybrid bi-level optimization process.    1:   Initialize upper-level population of $N_p^{UL}$ solutions randomly in feasible region    2:   Initialize lower-level population of $N_p^{LL}$ solutions randomly in feasible region    3:   Initialize Q-table for reinforcement learning with small random values    4:   Set RL parameters: α = 0.1, γ = 0.9, ϵstart = 0.9, ϵend = 0.1    5:   Set AOA parameters: Min = 0.2, Max = 1.0, α = 5.0, μ = 0.5    6:   Set penalty parameters: ${\lambda }_j^{UL,0}=1.0$, ${\lambda }_j^{LL,0}=1.0$, δUL = 0.1, δLL = 0.1    7:   Evaluate initial bi-level solutions using Algorithm 7    8:   Identify global best solutions ${\mathbf{x}}_{gbest}^{UL}$, ${\mathbf{x}}_{gbest}^{LL}$ and initialize $f_{best}^{UL}$, $f_{best}^{LL}$    9:   Initialize stagnation counters: stagUL = 0, stagLL = 0

Effective reinforcement learning for bi-level optimization requires continuous monitoring of both production and distribution planning states. Algorithm 9 captures key performance indicators for both levels and implements intelligent action selection for coordinated parameter adjustment.

Algorithm 9 RL State Observation and Action Selection for Bi-Level Planning

Description: This algorithm observes the current optimization state for both the production and distribution levels, calculates key performance metrics, and selects actions using an epsilon-greedy strategy to balance exploration and exploitation in bi-level parameter adaptation.      1:   Calculate upper-level diversity: $diversit{y}^{UL}={\displaystyle \frac{1}{N_p^{UL}}}{\sum }_{i=1}^{N_p^{UL}}\Vert {\mathbf{x}}_i^{UL}-{\mathbf{x}}_{gbest}^{UL}\Vert$      2:   Calculate lower-level diversity: $diversit{y}^{LL}={\displaystyle \frac{1}{N_p^{LL}}}{\sum }_{i=1}^{N_p^{LL}}\Vert {\mathbf{x}}_i^{LL}-{\mathbf{x}}_{gbest}^{LL}\Vert$      3:   Calculate convergence rate: $con{v}_{rate}={\displaystyle \frac{{(f_{best}^{UL}+f_{best}^{LL})}^{t-5}-{(f_{best}^{UL}+f_{best}^{LL})}^t}{5}}$      4:   Calculate bi-level gap: $gap={\displaystyle \frac{|{\mathit{coordination\_measure}}^{UL}-{\mathit{coordination\_measure}}^{LL}|}{{\mathit{coordination\_measure}}^{UL}+{\mathrm{coordination\_measure}}^{LL}}}$      5:   Define state: st = {diversityUL, diversityLL, convrate, stag, gap}      6:   Update ϵ: ϵ = max(ϵend, ϵstart · 0.995t)      7:   if random() < ϵ then      8:    Select random action at ∈ {1, 2, 3, 4}      9:   else    10:    at = arg maxaQ(st, a)    11:   end if    12:   Update AOA parameters using Algorithm 5

The core of the Arithmetic Optimization Algorithm lies in its mathematical-operation-based position updates. This algorithm implements the exploration and exploitation phases for both production and distribution planning levels through coordinated arithmetic operations.

Solution improvement and learning coordination are essential for bi-level optimization success. This algorithm manages global best solutions for both levels, quantifies performance improvements, and facilitates Q-learning for adaptive parameter coordination.

Managing algorithm convergence and coordination between levels is critical for bi-level optimization quality. Algorithm 10 monitors progress for both production and distribution planning, implements coordination strategies, and ensures robust termination.

Algorithm 10 Bi-Level Arithmetic Optimization Position Update

Description: This algorithm updates positions for both upper-level (distribution) and lower-level (production) populations using arithmetic operations, coordinating exploration and exploitation across both planning levels.      1:   Calculate $MOA=Min+t\times {\displaystyle \frac{Max-Min}{T_{max}}}$      2:   Calculate $MOP=1-{\displaystyle \frac{t^{1/\alpha }}{T_{max}^{1/\alpha }}}$      3:   Update Upper-Level Population (Distribution):      4:   for i = 1 to $N_p^{UL}$ do      5:    for j = 1 to DUL do      6:     Generate r1, r2, r3 ∼ U(0, 1)      7:     if r1 > MOA then      8:      {Exploration Phase}      9:      if r2 < 0.5 then    10:       $x_{i,j}^{UL,t+1}=bes{t}_j^{UL}+MOP\times ((U{B}_j^{UL}-L{B}_j^{UL})\times \mu +L{B}_j^{UL})$    11:      else    12:       $x_{i,j}^{UL,t+1}=bes{t}_j^{UL}-MOP\times ((U{B}_j^{UL}-L{B}_j^{UL})\times \mu +L{B}_j^{UL})$    13:      end if    14:     else    15:      {Exploitation Phase}    16:      if r3 < 0.5 then    17:       $x_{i,j}^{UL,t+1}=bes{t}_j^{UL}\times MOP\times ((U{B}_j^{UL}-L{B}_j^{UL})\times \mu +L{B}_j^{UL})$    18:      else    19:       $x_{i,j}^{UL,t+1}=bes{t}_j^{UL}÷MOP\times ((U{B}_j^{UL}-L{B}_j^{UL})\times \mu +L{B}_j^{UL})$    20:      end if    21:     end if    22:    end for    23:   end for    24:   Update Lower-Level Population (Production):    25:   for i = 1 to $N_p^{LL}$ do    26:    for j = 1 to DLL do    27:     Generate r1, r2, r3 ∼ U(0, 1)    28:     Apply similar arithmetic operations as upper level with coordination factor    29:     $x_{i,j}^{LL,t+1}=\mathit{arithmetic\_operation}(bes{t}_j^{LL},\mathit{coordination\_factor}\times x_{i,j}^{UL,t+1})$    30:    end for    31:   end for    32:   Apply boundary constraints and decode solutions using Algorithm 6    33:   Evaluate fitness using Algorithm 7

The main execution framework, implemented in Algorithm 11, orchestrates all bi-level algorithm components in a coordinated manner. This algorithm provides the high-level control structure that manages the iterative optimization process, coordinates the interaction between AOA and RL components across both levels, and ensures proper termination.

The comprehensive methodology ensures the development of a hybrid tactical planning framework that combines uncertain random simulation, reinforcement learning for adaptive coordination, and bi-level dependent-chance goal programming for multi-objective optimization. The approach is specifically designed to operate in hybrid uncertain environments where both data-driven and expert-based information guide tactical production-distribution decisions in paper manufacturing. This modular structure ensures the framework is scalable, interpretable, and adaptable to various tactical planning scenarios while maintaining coordination between production scheduling and distribution allocation decisions.

Algorithm 11 Main RL-Enhanced AOA Bi-Level Execution Framework

Description: This is the main execution framework that orchestrates all bi-level algorithm components, coordinating the Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm process for tactical production-distribution planning from initialization to termination.      1:   Execute Algorithm 8 for bi-level system initialization      2:   for t = 1 to Tmax do      3:    Execute Algorithm 9 for RL state observation and action selection      4:    Execute Algorithm 10 for bi-level arithmetic optimization position updates      5:    Execute Algorithm 12 for bi-level solution updates and RL learning      6:    Execute Algorithm 13 for stagnation management and termination check      7:    if Algorithm 13 returns True then      8:     break {Bi-level optimization completed}      9:    end if    10:   end for    11:   return Optimal bi-level solution ${\mathbf{x}}_{gbest}=[{\mathbf{x}}_{gbest}^{UL},{\mathbf{x}}_{gbest}^{LL}]$ with performance metrics

Algorithm 12 Bi-Level Solution Update and RL Learning

Description: This algorithm updates global best solutions for both production and distribution levels, calculates rewards based on bi-level fitness improvement, and performs Q-learning updates to enhance future parameter selection decisions.      1:   $f_{prev}^{UL}=f_{best}^{UL}$, $f_{prev}^{LL}=f_{best}^{LL}$      2:   Update Upper-Level Solutions:      3:   for each upper-level solution i do      4:    if $f^{UL}\left({\mathbf{x}}_i^{UL}\right)<f^{UL}\left({\mathbf{x}}_{gbest}^{UL}\right)$ then      5:     ${\mathbf{x}}_{gbest}^{UL}={\mathbf{x}}_i^{UL}$, $f_{best}^{UL}=f^{UL}\left({\mathbf{x}}_i^{UL}\right)$      6:    end if      7:   end for      8:   Update Lower-Level Solutions:      9:   for each lower-level solution i do    10:    if $f^{LL}\left({\mathbf{x}}_i^{LL}\right)<f^{LL}\left({\mathbf{x}}_{gbest}^{LL}\right)$ then    11:     ${\mathbf{x}}_{gbest}^{LL}={\mathbf{x}}_i^{LL}$, $f_{best}^{LL}=f^{LL}\left({\mathbf{x}}_i^{LL}\right)$    12:    end if    13:   end for    14:   Calculate bi-level reward: $R_t={\displaystyle \frac{(f_{prev}^{UL}+f_{prev}^{LL})-(f_{best}^{UL}+f_{best}^{LL})}{f_{prev}^{UL}+f_{prev}^{LL}+{10}^{-8}}}+\lambda ·\mathit{coordination\_bonus}$    15:   Observe next state st+1    16:   Update Q-table using Algorithm 4

After each position update, Algorithm 12 tracks solution improvements and provides feedback for reinforcement learning, ensuring continuous adaptation of the bi-level optimization process.

To prevent premature convergence and ensure solution quality, Algorithm 13 implements intelligent stagnation detection, strategic population reinitialization, and robust termination criteria for the bi-level optimization process.

Algorithm 13 Bi-Level Stagnation Management and Termination

Description: This algorithm monitors optimization progress for both levels, handles stagnation through coordinated reinitialization, progressively tightens constraints, and checks termination criteria for bi-level convergence.      1:   if $f_{best}^{UL}==f_{prev}^{UL}$ AND $f_{best}^{LL}==f_{prev}^{LL}$ then      2:    stag = stag + 1      3:   else      4:    stag = 0      5:   end if      6:   if stag > 0.15 × Tmax then      7:    Reinitialize worst 30% of upper-level population randomly      8:    Reinitialize worst 30% of lower-level population with coordination to upper level      9:    Reset: stag = 0    10:   end if    11:   Progressive Constraint Tightening:    12:   for each chance constraint j at both levels do    13:    ${\beta }_j\left(t\right)={\beta }_j^{final}·(1-e^{-\kappa ·t/T_{max}})$    14:   end for    15:   Check Bi-Level Convergence:    16:   Calculate coordination measure: $coord={\displaystyle \frac{|{\mathit{obj\_improvement}}^{UL}-{\mathit{obj\_improvement}}^{LL}|}{{\mathit{obj\_improvement}}^{UL}+{\mathit{obj\_improvement}}^{LL}}}$    17:   if convergence criteria met OR t == Tmax OR coord < threshold then    18:    Evaluate ${\mathbf{x}}_{gbest}^{UL}$, ${\mathbf{x}}_{gbest}^{LL}$ with high-precision chance simulation (NMC = 5000)    19:    Verify constraint satisfaction and goal achievement for both levels    20:    return True {Termination signal}    21:   end if    22:   return False {Continue optimization}

5. Numerical Results and Analysis

This section presents comprehensive numerical experiments to evaluate the performance of our proposed Reinforcement-Learning-enhanced Arithmetic Optimization Algorithm (RL-AOA) for solving the bi-level dependent-chance goal programming model for tactical production–distribution planning in paper manufacturing under hybrid uncertainty.

5.1. Experimental Setup

5.1.1. Algorithm Parameter Settings

Algorithm parameters were determined through systematic preliminary experiments using the Taguchi method for parameter optimization. The final RL-AOA parameters are presented in

Table 8. Parameter settings for RL-AOA.

Parameter	Symbol	Value
Population size (upper level)	$N_p^{UL}$	80
Population size (lower level)	$N_p^{LL}$	60
Maximum iterations	Tmax	400
Number of Monte Carlo simulations	NMC	8000
Math Optimizer bounds	[Min, Max]	[0.2, 1.0]
Exploitation accuracy parameter	α	5.0
Mutation factor	μ	0.5
Initial learning rate	α0	0.15
Learning rate decay	ϕ	0.6
Discount factor	γ	0.85
Exploration probability (initial)	ϵmax	0.8
Exploration probability (final)	ϵmin	0.05
Penalty factor (initial upper level)	${\lambda }_j^{UL,0}$	150
Penalty factor (initial lower level)	${\lambda }_j^{LL,0}$	100
Confidence level tightening rate	κ	2.5

.

5.1.2. Parameter Calibration Methodology

The algorithm parameters in Table 8 were calibrated using a systematic two-phase approach:

Phase 1: Taguchi Orthogonal Array Design

We employed an L16(4⁵) orthogonal array to evaluate key parameters:

• Population sizes: $N_p^{UL}\in \{60,80,100,120\}$, $N_p^{LL}\in \{40,60,80,100\}$;
• Learning parameters: α₀ ∈ {0.1, 0.15, 0.2, 0.25};
• Exploration bounds: ϵ_max ∈ {0.6, 0.8, 0.9, 1.0};
• Penalty factors: ${\lambda }_j^{UL,0}\in \{100,150,200,250\}$.

Phase 2: Fine-tuning Validation

Selected parameters underwent validation across three representative instances (Small-1, Medium-1, Large-1) with 15 independent runs each. The Signal-to-Noise ratio analysis confirmed optimal settings, i.e., $N_p^{UL}=80$ (S/N = 24.3 dB) and α₀ = 0.15 (S/N = 22.8 dB), achieving a 12.4% improvement over default values.

5.1.3. Test Instance Characteristics

To evaluate the algorithm comprehensively, we generated a diverse set of paper manufacturing test instances with varying network sizes, complexity levels, and uncertainty characteristics.

Table 9. Characteristics of paper manufacturing test instances.

Instance	Machines	WHs	DCs	Customers	Grades	Periods	Suppliers
Small-1	4	3	5	8	3	3	4
Small-2	6	4	6	12	4	3	5
Medium-1	8	5	8	15	5	4	6
Medium-2	10	6	10	20	6	4	8
Large-1	12	8	12	25	7	5	10
Large-2	15	10	15	30	8	5	12
Very Large	20	12	18	40	10	6	15

summarizes the key characteristics of these instances.

These instances represent paper manufacturing networks of varying complexity, ranging from small regional operations to very large international paper manufacturing facilities. Each instance incorporates specific characteristics reflecting different operational contexts:

• Small instances: Regional paper mills with limited product portfolio.
• Medium instances: Multi-grade paper manufacturing with moderate complexity.
• Large instances: Multi-facility paper manufacturing networks with diverse product lines.
• Very Large instance: Global paper manufacturing network with maximum complexity.

5.1.4. Uncertain Random Parameter Specifications

The hybrid uncertainty in our paper manufacturing model is represented through uncertain random parameters following different distributions.

Table 10. Uncertain random parameter distributions for paper manufacturing.

Parameter	Random Component	Uncertain Component
Customer demand (${\tilde{D}}_{pct}$)	Normal (μpct, ${\sigma }_{pct}^2$)	Linear (0.85μpct, 1.15μpct)
Production cost (${\widetilde{PC}}_{pmt}$)	Lognormal (μPC, $0.08{\mu }_{PC}^2$)	Linear (0.9μPC, 1.2μPC)
Raw material cost (${\widetilde{RC}}_{rt}$)	Normal (μRC, $0.1{\mu }_{RC}^2$)	Linear (0.8μRC, 1.25μRC)
Transportation cost (${\widetilde{TC}}_{ijlt}$)	Uniform ($0.85{\widehat{TC}}_{ijlt}$, $1.15{\widehat{TC}}_{ijlt}$)	Linear ($0.75{\widehat{TC}}_{ijlt}$, $1.3{\widehat{TC}}_{ijlt}$)
Machine efficiency (${\tilde{\eta }}_{pmt}$)	Beta (7, 2)	Linear (0.85, 1.0)
Raw material availability (${\tilde{\theta }}_{rt}$)	Triangular (0.8, 1.0, 1.1)	Linear (0.9, 1.15)
Quality yield (${\tilde{\gamma }}_{pt}$)	Beta (8, 1.5)	Linear (0.92, 1.0)
Transportation mode availability (${\tilde{\varphi }}_{lt}$)	Beta (3, 1)	Linear (0.7, 1.0)
Machine breakdown probability (${\tilde{\omega }}_{mt}$)	Beta (1.5, 10)	Linear (0.4, 2.2)
Emission rate (${\widetilde{EmissionRate}}_{pm}$)	Lognormal (μER, $0.12{\mu }_{ER}^2$)	Linear (0.85μER, 1.25μER)

presents the uncertain random parameter settings used in our computational experiments.

The hybrid uncertain random parameters were generated following a rigorous two-stage calibration process:

• Random Component Calibration: Historical data from five major paper manufacturing companies in North America and Europe were analyzed to determine appropriate probability distributions and their parameters.
• Uncertain Component Calibration: Expert elicitation sessions with 12 paper industry professionals were conducted to establish uncertainty bounds, representing epistemic uncertainty in tactical planning parameter estimation.

5.1.5. Computational Environment

All experiments were conducted on a workstation equipped with Intel Core i9-12900K processor and 32GB RAM, implemented in MATLAB R2023b. Each test instance was solved 30 times with different random seeds to ensure the statistical significance of the results.

5.2. Bi-Level Dependent-Chance Goal Programming Results

The core contribution of our approach lies in optimizing probability achievement for tactical planning goals through lexicographic bi-level dependent-chance programming. We define target probability levels based on paper industry benchmarks and operational requirements: Economic Efficiency (α₁ = 0.90), Operational Performance (α₂ = 0.85), Resource Utilization (α₃ = 0.80), and Quality Assurance (α₄ = 0.90).

5.2.1. Probability Achievement Analysis

Table 11. Tactical planning goal probability achievement with confidence intervals.

Instance	Economic	Operational	Resource Util.	Quality
	(Target: 0.90)	(Target: 0.85)	(Target: 0.80)	(Target: 0.90)
Small-1	0.887 ± 0.012	0.836 ± 0.015	0.783 ± 0.018	0.878 ± 0.014
Small-2	0.883 ± 0.013	0.831 ± 0.016	0.779 ± 0.019	0.874 ± 0.015
Medium-1	0.876 ± 0.014	0.823 ± 0.017	0.771 ± 0.020	0.867 ± 0.016
Medium-2	0.871 ± 0.015	0.817 ± 0.018	0.765 ± 0.021	0.862 ± 0.017
Large-1	0.865 ± 0.016	0.810 ± 0.019	0.758 ± 0.022	0.856 ± 0.018
Large-2	0.859 ± 0.017	0.804 ± 0.020	0.752 ± 0.023	0.850 ± 0.019
Very Large	0.852 ± 0.018	0.797 ± 0.021	0.745 ± 0.024	0.843 ± 0.020
Average	0.870	0.817	0.765	0.861
Gap from Target	−3.3%	−3.9%	−4.4%	−4.3%

Note: Bold formatting in summary rows distinguishes aggregate statistics from individual instance results for enhanced readability.

presents the achieved probability levels for each tactical planning dimension across all test instances, with 95% confidence intervals computed using bootstrap resampling (1000 replications). Figure 1 illustrates the tactical planning goal achievement with confidence intervals, demonstrating the systematic performance trends across different problem scales.

The results demonstrate consistent performance across all tactical planning objectives, with economic efficiency and quality assurance achieving the highest rates. As shown in Figure 1, there is a clear downward trend in probability achievement as problem size increases, with all objectives showing statistically significant gaps from their targets (paired t-test, p < 0.01 for all objectives). This confirms the increasing challenge of simultaneously achieving all tactical planning goals under hybrid uncertainty as network complexity grows.

5.2.2. Bi-Level Coordination Analysis

Table 12. Bi-level coordination effectiveness.

Instance	Coordination Gap a	Iterations to Converge	Upper-Level Cost (USD K)	Lower-Level Cost (USD K)
Small-1	0.024	28	45.3	23.2
Small-2	0.031	32	52.5	27.8
Medium-1	0.038	36	68.8	35.4
Medium-2	0.045	41	84.3	43.7
Large-1	0.052	47	112.6	58.9
Large-2	0.059	53	138.5	72.3
Very Large	0.067	61	189.7	98.8
Average	0.045	42.6	98.8	51.4

^a Normalized coordination gap = ${\displaystyle \frac{|\sum Q_{pmt}-\sum F_{wplt}|}{\sum Q_{pmt}}}$, representing relative material flow imbalance (0 = perfect coordination, 1 = complete mismatch). Note: Bold formatting in the average row distinguishes summary statistics from individual instance results for enhanced readability.

shows the coordination effectiveness between upper-level distribution decisions and lower-level production decisions across iterations. Figure 2 illustrates the convergence behavior of the bi-level optimization process, demonstrating how both upper- and lower-level costs converge simultaneously while maintaining coordination balance.

As illustrated in Figure 2, the coordination gap decreases systematically as the bi-level algorithm progresses, with both distribution and production costs converging to stable values. The coordination gap increases with problem complexity, indicating that the bi-level structure becomes more necessary for complex tactical planning problems. The convergence times are reasonable across all instances, with the largest problem requiring 61 iterations on average.

5.3. Paper Manufacturing Instance Details

To demonstrate the practicality of our approach, we present detailed results for a representative medium-sized paper manufacturing instance (Medium-1), as shown in Table 9. This instance includes 8 paper machines, 5 warehouses, 8 distribution centers, 15 customers, 5 paper grades, and a 4-period tactical planning horizon.

5.3.1. Paper Machine and Grade Information

Table 13. Production and setup costs for Medium-1 instance (USD/ton, USD).

Machine	Production Cost (USD/ton)					Setup Cost (USD)
	G1	G2	G3	G4	G5	G1	G2	G3	G4	G5
M1	285	320	245	–	380	1200	1450	950	–	1680
M2	295	–	255	420	365	1350	–	1100	1890	1580
M3	–	310	240	405	375	–	1380	980	1820	1620
M4	280	315	–	415	–	1180	1420	–	1860	–
M5	290	325	250	–	385	1280	1480	1050	–	1720
M6	–	–	235	395	360	–	–	920	1750	1540
M7	275	305	245	410	–	1150	1350	970	1840	–
M8	285	–	–	400	370	1220	–	–	1780	1590

presents the production and setup costs for different paper grades on each machine, while

Table 14. Machine capacity information for Medium-1 instance.

Machine	Daily Capacity (tons)	Efficiency (%)	Availability (%)
M1	45	92.3	95.2
M2	38	89.7	94.8
M3	42	91.5	96.1
M4	50	88.9	93.7
M5	35	90.2	95.5
M6	40	93.1	96.8
M7	48	87.4	94.2
M8	44	91.8	95.9

shows the machine capacities for the Medium-1 instance.

5.3.2. Customer Demand Information

Table 15. Customer demand information with uncertain parameters (Medium-1).

Customer	Type	Mean Demand (tons/period)					Uncertainty Parameters
		G1	G2	G3	G4	G5	CV (%)	α-cut	Service Level
C1	Packaging	450	380	0	520	0	15	[0.9, 1.1]	0.95
C2	Newsprint	0	680	420	0	0	18	[0.85, 1.15]	0.92
C3	Tissue	0	0	320	0	460	22	[0.8, 1.2]	0.90
C4	Publishing	350	550	280	0	0	16	[0.9, 1.1]	0.94
C5	Specialty	0	0	0	380	420	25	[0.75, 1.25]	0.88
C6	Packaging	480	0	0	560	0	14	[0.92, 1.08]	0.96
C7	Newsprint	0	720	390	0	0	19	[0.85, 1.15]	0.91
C8	Tissue	0	0	340	0	480	21	[0.8, 1.2]	0.89

presents stochastic demand parameters and service requirements for different customer segments in the Medium-1 instance.

5.3.3. Baseline Algorithm Specifications

The comparative algorithms were adapted for bi-level optimization as follows:

• Bi-GA: Two-population genetic algorithm with tournament selection (size 3), single-point crossover (p_c = 0.8), and uniform mutation (p_m = 0.05). Upper-level population = 80, lower-level population = 60.
• Bi-PSO: Hierarchical particle swarm optimization with w = 0.9 → 0.4 (linear decrease), c₁ = c₂ = 2.0, velocity clamping at ± 0.2× search range.
• Bi-DE: Differential evolution with DE/rand/1/bin strategy, F = 0.5, CR = 0.8, coordinated through best solution sharing between levels every 10 iterations.
• Bi-SSO: Social spider optimization with female ratio = 0.65, vibration attenuation r_a = 1.0, bi-level coordination through pheromone information exchange.

All algorithms used identical constraint handling (dynamic penalty), chance constraint evaluation (8000 Monte Carlo samples), and termination criteria (400 iterations) for fair comparison.

5.4. Algorithm Performance Comparison

We compare our RL-AOA with several state-of-the-art metaheuristics adapted for bi-level optimization.

Table 16. Algorithm performance comparison for bi-level optimization.

Algorithm	Best	Average	Worst	Std. Dev.	Time (min)	Success Rate
Bi-GA	0.821	0.826	0.839	0.006	24.3	71.2%
Bi-PSO	0.834	0.839	0.851	0.005	21.7	75.8%
Bi-DE	0.847	0.852	0.863	0.005	19.4	78.3%
Bi-SSO	0.855	0.860	0.869	0.004	17.9	81.7%
RL-AOA	0.870	0.875	0.882	0.003	15.2	86.4%

Wilcoxon Signed-Rank Test (RL-AOA vs. others): all p < 0.01. Note: Bold formatting highlights the proposed RL-AOA method to distinguish it from baseline algorithms for comparison clarity.

presents comprehensive performance metrics.

The results demonstrate that RL-AOA consistently outperforms other bi-level optimization approaches, achieving 3.2–7.8% improvement in solution quality and 18.5% reduction in computational time compared to the best competing algorithm. The convergence behavior is illustrated in Figure 2, which shows that RL-AOA achieves faster convergence and better final solutions compared to traditional metaheuristics.

5.5. Empirical Validation of Balance Strategy

To validate our exploration–exploitation trade-off strategy, we conducted comprehensive analysis of RL-AOA balance evolution. Figure 3 illustrates the dynamic adaptation capability.

Key empirical findings:

• Optimal Balance Progression: Systematic transition from 25% exploration to 2% exploitation over 400 iterations.
• Stable Balance Quality: Incremental–decremental analysis shows consistent balance near zero.
• Adaptive Capability: Algorithm successfully adapts exploration–exploitation ratio based on optimization progress.
• Convergence Stability: Balance evolution demonstrates stable convergence without premature stagnation.

5.6. Reinforcement Learning Component Analysis

We analyze the effectiveness of the RL component in adaptive parameter tuning for bi-level coordination.

Table 17. RL action selection patterns in bi-level optimization.

Optimization Phase	Enhance Exploration	Focus Exploitation	Balanced Search	Strengthen Coordination
Early (0–25%)	42.8%	12.6%	28.4%	16.2%
Middle (25–65%)	24.3%	21.7%	35.8%	18.2%
Late (65–100%)	8.9%	35.4%	29.3%	26.4%
Performance Gain	+3.1%	+2.4%	+1.6%	+3.8%

Note: Bold formatting in the performance gain row distinguishes performance metrics from action selection frequencies, highlighting the effectiveness of each RL action type for enhanced readability.

shows action selection patterns across different optimization phases.

The RL component demonstrates intelligent adaptive behavior, emphasizing exploration and coordination in early phases while shifting to exploitation and refined coordination in later iterations, contributing to a 5.2% average performance improvement over static parameter settings.

5.7. Sensitivity Analysis

We conducted comprehensive sensitivity analysis to evaluate the robustness of our bi-level dependent-chance goal programming approach under varying parameter settings.

5.7.1. Confidence Level Impact

Table 18. Impact of confidence levels on bi-level performance metrics.

Confidence	Goal Achievement	Upper-Level Cost	Lower-Level Cost	Coordination Gap	CPU Time
0.70	0.892	84.3	42.2	0.038	11.2
0.75	0.881	87.5	43.8	0.041	12.4
0.80	0.870	91.3	45.7	0.045	13.8
0.85	0.858	95.8	47.9	0.049	15.2
0.90	0.844	101.2	50.6	0.054	17.9
0.95	0.827	108.5	54.3	0.061	22.1

presents the detailed results of confidence level sensitivity analysis for the bi-level model.

The analysis shows how increasing confidence levels affect both cost and goal achievement, demonstrating the trade-off between reliability and cost efficiency in tactical paper manufacturing planning.

5.7.2. Production–Distribution Coordination Analysis

Table 19. Production–distribution coordination under uncertainty.

Uncertainty Level	Coordination Efficiency	Information Exchange	Decision Consistency	Overall Performance
Low (0.5×)	0.924	0.957	0.889	0.923
Medium (1.0×)	0.870	0.912	0.845	0.876
High (1.5×)	0.823	0.868	0.798	0.830
Very High (2.0×)	0.778	0.821	0.752	0.784

analyzes the coordination effectiveness between the production and distribution levels under different uncertainty scenarios.

5.8. Paper Manufacturing Specific Analysis

5.8.1. Paper Grade Production Allocation

Table 20. Optimal paper grade production allocation (Medium-1 instance).

Machine	Period 1					Period 2					Period 3					Period 4
	G1	G2	G3	G4	G5	G1	G2	G3	G4	G5	G1	G2	G3	G4	G5	G1	G2	G3	G4	G5
M1	125	0	89	–	0	0	134	76	–	98	118	0	92	–	0	0	128	0	–	105
M2	0	–	0	156	87	98	–	0	143	0	0	–	67	148	94	112	–	0	0	89
M3	–	142	0	0	76	–	0	89	134	0	–	156	0	0	82	–	0	78	145	0
M4	89	0	–	0	–	0	125	–	167	–	104	0	–	0	–	0	138	–	158	–
M5	0	0	76	–	123	87	0	0	–	0	0	0	89	–	134	96	0	0	–	0
M6	–	–	94	0	0	–	–	0	0	118	–	–	104	0	0	–	–	0	0	127
M7	0	89	0	124	–	76	0	98	0	–	0	112	0	135	–	88	0	87	0	–
M8	134	–	–	0	97	0	–	–	148	0	125	–	–	0	106	0	–	–	142	0

shows the optimal production allocation across different paper grades and planning periods for the Medium-1 instance. Figure 4 provides a visual representation of the production distribution across planning periods, highlighting the balanced allocation strategy employed by our optimization framework.

As illustrated in Figure 4, the production allocation strategy demonstrates effective balance across different paper grades and planning periods. Grade G4 (Specialty) shows the highest total production volume (1600 tons), followed by Grade G1 (Packaging) with 1252 tons, reflecting market demand patterns and profitability considerations. The temporal distribution shows relatively stable production across periods, with some variation to accommodate demand fluctuations and machine availability constraints.

5.8.2. Machine Changeover Analysis

Table 21. Machine changeover analysis for tactical planning.

Machine	Total Changeovers	Changeover Cost (USD)	Changeover Time (hrs)	Efficiency Loss (%)	Utilization (%)
M1	6	8450	23.5	3.2	87.4
M2	7	9680	27.2	3.8	85.1
M3	5	7320	19.8	2.7	89.2
M4	8	11,240	31.6	4.3	82.7
M5	6	8750	24.8	3.4	86.8
M6	4	5890	16.2	2.2	91.5
M7	7	9450	26.4	3.6	84.9
M8	5	7680	20.9	2.9	88.6
Average	6.0	8557	23.8	3.3	87.0

Note: Bold formatting in the average row distinguishes aggregate statistics from individual machine data for enhanced readability and quick identification of summary metrics.

presents the changeover patterns and their impact on production efficiency.

The analysis shows the trade-off between production flexibility and efficiency, with machines experiencing 4–8 changeovers per planning horizon and maintaining an average utilization of 87.0% despite changeover penalties.

5.9. Cost–Benefit Analysis for Paper Manufacturing

Table 22. Cost–benefit analysis for paper manufacturing implementation.

Mill Size	Implementation Cost (USD K)	Annual Savings (USD K)	Payback Period (Months)
Small (1–2 machines)	150–250	180–320	8–14
Medium (3–8 machines)	280–450	350–680	9–15
Large (9+ machines)	520–850	750–1400	8–13

presents the estimated costs and benefits of implementing the bi-level DCGP framework in paper manufacturing environments. Figure 5 illustrates the return on investment analysis across different mill sizes, demonstrating favorable economic prospects for implementation.

As demonstrated in Figure 5, the expected return on investment is favorable across all mill sizes, with the 3-year ROI ranging from 35 to 55%. The payback periods are particularly attractive, ranging from 8 to 15 months, with larger mills achieving faster payback due to economies of scale. The analysis indicates that medium and large mills benefit most from implementation, with annual savings significantly exceeding implementation costs.

5.10. Target Probability Level (α_g) Impact on Lexicographic Feasibility

The lexicographic optimization structure requires careful consideration of how target probability levels affect feasibility across priority levels.

Table 23. Impact of target probability levels on lexicographic feasibility.

Scenario	Econ.	Oper.	Res.	Qual.	Econ.	Oper.	Res.	Qual.
	Target αg Values				Goal Achievement
Relaxed	0.80	0.75	0.70	0.80	0.892	0.848	0.811	0.879
Current	0.90	0.85	0.80	0.90	0.870	0.817	0.765	0.861
Stringent	0.95	0.90	0.85	0.95	0.831	0.762	0.689	0.798
Cascade Effect:					0.045	0.085	0.167	-

analyzes the impact of varying α_g on goal achievement and feasibility.

Critical insights:

• Feasibility Threshold: Target levels above α_g = 0.85 create significant feasibility issues;
• Cascade Effects: Higher-priority goals progressively constrain lower-priority goals (0.045 to 0.167);
• Current Settings Validation: Our α_g values operate near feasibility boundary, explaining observed gaps.

5.11. Expert Opinion Impact and Sensitivity Analysis

Expert opinions define epistemic uncertainty bounds (alpha-cuts) for uncertain random variables, directly influencing chance measure calculations and goal achievement probabilities. Figure 6 presents comprehensive sensitivity analysis examining how expert-defined uncertainty ranges affect model performance.

The sensitivity analysis reveals critical insights:

• Quantified Sensitivity Range: Goal achievement probabilities vary within ±3.0% bounds across expert opinion scenarios. Economic efficiency shows ±0.9%, operational performance ±0.8%, resource utilization ±2.5%, and quality assurance ±1.3% variations.
• Differential Impact Analysis: Resource utilization goals demonstrate 2.8× higher sensitivity than economic efficiency, reflecting operational complexity under epistemic uncertainty. This identifies critical areas requiring careful expert calibration.
• Robustness Validation: The bounded sensitivity range (maximum 3.0%) confirms framework stability. Even with ±100% variation in expert uncertainty estimates (from ±5% to ±30%), goal achievement varies by less than 3.5%, validating practical applicability.
• Optimal Calibration Confirmation: Current ±15% uncertainty ranges represent optimal balance, positioned at the inflection point where sensitivity transitions from steep (conservative side) to gradual (optimistic side), maximizing information value while maintaining stability.

These findings demonstrate that while expert opinions measurably influence outcomes, the framework maintains robust performance (coefficient of variation < 0.035) across reasonable uncertainty variations. The moderate sensitivity validates our approach for industrial application without requiring excessive precision in expert assessments, supporting practical implementation in paper manufacturing environments where expert consensus may vary.

5.12. Comparative Analysis: Deterministic vs. Hybrid Uncertainty Framework

To demonstrate the added value of hybrid uncertainty modeling, we compared four approaches: deterministic baseline, stochastic-only, fuzzy-only, and our hybrid framework. Figure 7 presents a comprehensive comparative analysis of these approaches.

Key findings:

• Cost Realism: The hybrid framework shows a 17.8% cost increase over deterministic baseline;
• Goal Achievement Stability: Deterministic approaches exhibit overoptimistic variability while hybrid maintains stable probabilities;
• Single-Paradigm Limitations: Stochastic-only and fuzzy-only fail to capture complete uncertainty spectrum.

This analysis demonstrates when hybrid uncertainty complexity is justified for tactical planning decisions.

5.13. Managerial Insights and Implementation Guidelines

Based on our comprehensive analysis, we provide actionable insights for paper manufacturing tactical planning:

• Tactical Goal Probability Setting:
  • Economic efficiency: Target 87–90% (achievable with moderate resource allocation);
  • Operational performance: Target 82–85% (balances service level with operational flexibility);
  • Resource utilization: Target 76–80% (realistic given machine compatibility constraints);
  • Quality assurance: Target 86–90% (essential for paper grade specifications).
• Implementation Roadmap for Paper Manufacturing:
  • Phase 1: Implement production scheduling optimization (2–4 months)
  • Phase 2: Integrate distribution coordination (4–6 months)
  • Phase 3: Add quality and changeover optimization (6–8 months)
  • Expected operational improvement: 15–25% within 12 months
• Machine-Grade Compatibility Management:
  • Maintain compatibility matrices for all machine-grade combinations;
  • Optimize changeover sequences to minimize setup times and costs;
  • Consider machine specialization for high-volume paper grades;
  • Update efficiency parameters monthly based on production data.
• Uncertainty Management in Paper Manufacturing:
  • Monitor demand patterns by paper grade and customer type monthly;
  • Conduct quarterly expert assessments for market trend uncertainties;
  • Maintain confidence levels between 0.82 and 0.87 for optimal performance;
  • Implement adaptive inventory policies based on demand uncertainty;
  • Dynamic Shelf Life Modeling: The current model assumes fixed shelf-life periods for computational tractability. Future extensions could incorporate storage-condition-dependent deterioration models, particularly valuable for moisture-sensitive paper grades where humidity and temperature significantly affect product quality over time.

5.14. Limitations and Future Research Directions

While our bi-level DCGP framework demonstrates significant improvements for paper manufacturing tactical planning, several limitations should be acknowledged:

• Real-World Validation: Future work should include partnerships with paper mills for comprehensive industrial validation across different paper grades and manufacturing processes.
• Dynamic Grade Specifications: The current model assumes static quality requirements; extensions should incorporate evolving customer specifications and regulatory changes.
• Multi-Mill Coordination: Framework extension to coordinate multiple paper mills within a corporate network represents a promising research direction.
• Environmental Regulations: Integration of time-varying environmental constraints and carbon pricing mechanisms would enhance practical applicability.
• Supply Chain Resilience: Incorporation of supply disruption scenarios and recovery strategies would improve robustness.

The proposed framework provides a solid foundation for tactical production–distribution planning in paper manufacturing, offering both theoretical contributions and practical implementation guidelines for industry adoption. The results demonstrate that the RL-AOA with bi-level dependent-chance goal programming can effectively handle the complexity of hybrid uncertainty while maintaining computational efficiency for real-world applications.

6. Conclusions

This research developed a bi-level dependent-chance goal programming framework for tactical production–distribution planning in paper manufacturing under hybrid uncertainty. The framework successfully coordinates upper-level distribution decisions with lower-level production decisions while handling both aleatory randomness and epistemic uncertainty through uncertain random theory. The Reinforcement Learning-enhanced Arithmetic Optimization Algorithm achieved 3.2–7.8% improvements in solution quality and 18.5% computational time reduction compared to existing bi-level optimization methods.

The dependent-chance goal programming formulation effectively balanced four tactical objectives, achieving average probability levels of 87.0% for economic efficiency, 81.7% for operational performance, 76.5% for resource utilization, and 86.1% for quality assurance. The bi-level coordination mechanism maintained average coordination gaps of 4.5% across all test instances, demonstrating effective integration between production and distribution planning levels. The reinforcement learning component provided intelligent adaptive behavior, contributing to a 5.2% performance improvement over static parameter configurations.

Computational experiments across seven test instances validated the framework’s scalability and practical applicability for paper manufacturing operations. Industry-specific analysis confirmed effective handling of machine-grade compatibility constraints, sequence-dependent changeovers, and quality requirements while maintaining 87.0% average machine utilization. Cost–benefit analysis indicates favorable economic prospects with payback periods of 8–15 months and 35–55% three-year return on investment.

The framework provides both theoretical contributions to bi-level optimization under hybrid uncertainty and practical tools for tactical planning in paper manufacturing. Future research should focus on real-world validation, dynamic uncertainty modeling, multi-mill coordination, and integration of environmental constraints. The approach demonstrates that sophisticated optimization techniques can successfully address complex industrial planning problems while maintaining computational efficiency and providing actionable insights for hierarchical decision-making under uncertainty.