Search Results (10)

In contextual multi-armed bandits, the relationship between contextual information and rewards is typically unknown, complicating the trade-off between exploration and exploitation. A common approach to address this challenge is the Upper Confidence Bound (UCB) method, which constructs confidence intervals to guide exploration. However, the UCB method becomes computationally expensive in environments with numerous arms and dynamic contexts. This paper presents an adaptive noise exploration framework to reduce computational complexity and introduces two novel algorithms: EAD (Exploring Adaptive Noise in Decision-Making Processes) and EAP (Exploring Adaptive Noise in Parameter Spaces). EAD injects adaptive noise into the reward signals based on arm selection frequency, while EAP adds adaptive noise to the hidden layer of the neural network for more stable exploration. Experimental results on recommendation and classification tasks show that both algorithms significantly surpass traditional linear and neural methods in computational efficiency and overall performance. Full article

This study discusses the planning of a regional-scale water–food nexus (WFN) system using an inexact fuzzy chance constraint programming (IFCCP) method. The IFCCP approach can handle uncertainties expressed as interval and fuzzy parameters, as well as the preferences of decision makers. An inexact fuzzy chance constraint programming-based water–food nexus (IFCCP-WFN) model has been developed for the City of Jinan with the consideration of various restrictions related to water and land availability, as well as food and vegetable demands. Solutions for the planting areas for different crops in different periods have been generated under the different preferences of decision makers. The water resource availability would be the priority factor affecting the WFN system under demanding conditions, in which wheat cultivation would be dominated by this factor under fuzzy confidence levels of 0.2 and 0.5, and the planting area of corn would be determined by this factor under high fuzzy confidence levels (e.g., 0.8). In addition, the reliability of irrigation would decrease with increasing fuzzy confidence levels under demanding conditions, limiting the planting areas for crops and leading to a decreasing trend of the system benefit. Adequate water resources would be available for irrigation under optimistic conditions, implying no significant contributions to the planting schemes. Nevertheless, increasing food loss rates would result in more planting areas to satisfy food requirements and thus a greater system benefit under advantageous conditions. Compared with the developed IFCCP-WFN model, the interval-linear-programming-based water–food nexus (ILP-WFN) model can merely reflect the lower and upper bounds of uncertain parameters and neglects the inherent distributional information within the fuzzy parameters. Thus, the ILP-WFN model is unable to reveal the inherent impacts of the fuzzy parameters on the resulting planting strategies. Full article

Downside risk measures play a very interesting role in risk management problems. In particular, the value at risk (VaR) and the conditional value at risk (CVaR) have become very important instruments to address problems such as risk optimization, capital requirements, portfolio selection, pricing and hedging issues, risk transference, risk sharing, etc. In contrast, expectile risk measures are not as widely used, even though they are both coherent and elicitable. This paper addresses the bidual representation of expectiles in order to prove further important properties of these risk measures. Indeed, the bidual representation of expectiles enables us to estimate and optimize them by linear programming methods, deal with optimization problems involving expectile-linked constraints, relate expectiles with VaR and CVaR by means of both equalities and inequalities, give VaR and CVaR hyperbolic upper bounds beyond the level of confidence, and analyze whether co-monotonic additivity holds for expectiles. Illustrative applications are presented. Full article

As a key 5G technology, massive multiple-input multiple-output (MIMO) can effectively improve system capacity and reduce latency. This paper proposes a user scheduling and spectrum allocation method based on combinatorial multi-armed bandit (CMAB) for a massive MIMO system. Compared with traditional methods, the proposed CMAB-based method can avoid channel estimation for all users, significantly reduce pilot overhead, and improve spectral efficiency. Specifically, the proposed method is a two-stage method; in the first stage, we transform the user scheduling problem into a CMAB problem, with each user being referred to as a base arm and the energy of the channel being considered a reward. A linear upper confidence bound (UCB) arm selection algorithm is proposed. It is proved that the proposed user scheduling algorithm experiences logarithmic regret over time. In the second stage, by grouping the statistical channel state information (CSI), such that the statistical CSI of the users in the angular domain in different groups is approximately orthogonal, we are able to select one user in each group and allocate a subcarrier to the selected users, so that the channels of users on each subcarrier are approximately orthogonal, which can reduce the inter-user interference and improve the spectral efficiency. The simulation results validate that the proposed method has a high spectral efficiency. Full article

Kriging has some problems such as ignoring sample values in giving weights to them, reducing dependence structure to a single covariance function, and facing negative confidence bounds. In view to these problems of kriging in this study to estimate Cu in the Iju porphyry Cu deposit in Iran, we used a convex linear combination of Archimedean copulas. To delineate the spatial dependence structure of Cu, the best Frank, Gumbel, and Clayton copula models were determined at different lags to fit with higher-order polynomials. The resulting Archimedean copulas were able to describe all kinds of spatial dependence structures, including asymmetric lower and upper tails. The copula and kriging methods were compared through a split-sample cross-validation test whereby the drill-hole data were divided into modeling and validation sets. The cross-validation showed better results for geostatistical estimation through copula than through kriging in terms of accuracy and precision. The mean of the validation set, which was 0.1218%, was estimated as 0.1278% and 0.1369% by the copula and kriging methods, respectively. The correlation coefficient between the estimated and measured values was higher for the copula method than for the kriging method. With 0.0143%² and 0.0162%² values, the mean square error was substantially smaller for copula than for kriging. A boxplot of the results demonstrated that the copula method was better in reproducing the Cu distribution and had fewer smoothing problems. Full article

Contextual bandits can solve a huge range of real-world problems. However, current popular algorithms to solve them either rely on linear models or unreliable uncertainty estimation in non-linear models, which are required to deal with the exploration–exploitation trade-off. Inspired by theories of human cognition, we introduce novel techniques that use maximum entropy exploration, relying on neural networks to find optimal policies in settings with both continuous and discrete action spaces. We present two classes of models, one with neural networks as reward estimators, and the other with energy based models, which model the probability of obtaining an optimal reward given an action. We evaluate the performance of these models in static and dynamic contextual bandit simulation environments. We show that both techniques outperform standard baseline algorithms, such as NN HMC, NN Discrete, Upper Confidence Bound, and Thompson Sampling, where energy based models have the best overall performance. This provides practitioners with new techniques that perform well in static and dynamic settings, and are particularly well suited to non-linear scenarios with continuous action spaces. Full article

District heating networks make up an important public energy service, in which leakage is the main problem affecting the safety of pipeline network operation. This paper proposes a Leakage Fault Detection (LFD) method based on the Linear Upper Confidence Bound (LinUCB) which is used for arm selection in the Contextual Bandit (CB) algorithm. With data collected from end-users’ pressure and flow information in the simulation model, the LinUCB method is adopted to locate the leakage faults. Firstly, we use a hydraulic simulation model to simulate all failure conditions that can occur in the network, and these change rate vectors of observed data form a dataset. Secondly, the LinUCB method is used to train an agent for the arm selection, and the outcome of arm selection is the leaking pipe label. Thirdly, the experiment results show that this method can detect the leaking pipe accurately and effectively. Furthermore, it allows operators to evaluate the system performance, supports troubleshooting of decision mechanisms, and provides guidance in the arrangement of maintenance. Full article

The unique features of millimeter waves (mmWaves) motivate its leveraging to future, beyond-fifth-generation/sixth-generation (B5G/6G)-based device-to-device (D2D) communications. However, the neighborhood discovery and selection (NDS) problem still needs intelligent solutions due to the trade-off of investigating adjacent devices for the optimum device choice against the crucial beamform training (BT) overhead. In this paper, by making use of multiband (μW/mmWave) standard devices, the mmWave NDS problem is addressed using machine-learning-based contextual multi-armed bandit (CMAB) algorithms. This is done by leveraging the context information of Wi-Fi signal characteristics, i.e., received signal strength (RSS), mean, and variance, to further improve the NDS method. In this setup, the transmitting device acts as the player, the arms are the candidate mmWave D2D links between that device and its neighbors, while the reward is the average throughput. We examine the NDS’s primary trade-off and the impacts of the contextual information on the total performance. Furthermore, modified energy-aware linear upper confidence bound (EA-LinUCB) and contextual Thomson sampling (EA-CTS) algorithms are proposed to handle the problem through reflecting the nearby devices’ withstanding battery levels, which simulate real scenarios. Simulation results ensure the superior efficiency of the proposed algorithms over the single band (mmWave) energy-aware noncontextual MAB algorithms (EA-UCB and EA-TS) and traditional schemes regarding energy efficiency and average throughput with a reasonable convergence rate. Full article

Wind power intervals with different confidence levels have an impact on both the economic cost and risk of dispatch plans for power systems with wind power integration. The higher the confidence level, the greater the bandwidth of corresponding intervals. Thus, more reserves are needed, resulting in higher economic cost but less risk. In order to balance the economic cost and risk, a unit commitment model based on the optimal wind power confidence level is proposed. There are definite integral terms in the objective function of the model, and both the integrand function and integral upper/lower bound contain decision variables, which makes it difficult to solve this problem. The objective function is linearized and solved by discretizing the wind power probability density function and using auxiliary variables. On the basis, a rolling dispatching model considering the dynamic regulation costs among multiple rolling plans is established. In addition to balancing economic cost and risk, it can help to avoid repeated regulations among different rolling plans. Simulations are carried on a 10-units system and a 118-bus system to verify the effectiveness of the proposed models. Full article

The nonuniformity of the spatial response to surface radiation is a fundamental characteristic of all airborne and spaceborne sensors that inevitably introduces uncertainty into the estimation of object proportions by the spectral unmixing of mixed pixels. Simulated data of the surface radiation distribution and a TM (thematic mapper) response matrix were developed and utilized to imitate the generation of mixed pixels and the extraction of the object proportion via a Monte Carlo simulation, and then, the nonuniformity effect of a sensor’s PSF (point spread function) was explored. The following conclusions were drawn: (1) given a nonuniform spatial response of a sensor to a surface scene with a constant object proportion and various object distribution patterns, the mixed pixel DN (digital number) of a remotely-sensed image becomes a random variable, which causes a PSF nonuniform effect on the object proportion extraction; (2) for the estimated object proportion, the corresponding true object proportion appears with a random variation; its upper and lower bounds take on an asymmetrical spindle shape; and models of these bound curves at any probability level were established; (3) there exists a negative linear relationship between the bias of the spectral unmixing and the estimated proportion; the bias is zero at an estimated proportion of 50%, and when the estimated proportions are approximately 100% and 0%, the object proportion is overestimated by 0.78% and underestimated by 0.78%, respectively; (4) the relationship between the standard deviation of the spectral unmixing and the estimated proportion follows a symmetrical polynomial function opening downward; the standard deviation reaches a maximum of 4.4% at the estimated proportion of 50%, and when the estimated proportion is approximately 100% or 0%, the standard deviation is a minimum, 1.05%. The above findings contribute to a comprehensive understanding of the PSF nonuniformity effect, have the potential to compensate for the bias of proportion estimation and present its confidence interval at any probability level. Full article