Two-Dimensional Thompson Sampling for Joint Beam and Power Control for Uplink Maritime Communications

Abstract

In a cellular maritime communication system, ocean buoys are essential to enable environmental monitoring, offshore platform management, and disaster response. Therefore, energy-efficient transmission from the buoys is a key requirement to prolong their operational time. A fixed uplink beamforming can be considered to save energy by leveraging its beam gain while managing the target link reliability. However, the dynamic condition of ocean waves causes buoys’ random orientation, leading to frequent misalignment of their predefined beam direction aimed at the base station, which degrades both the link reliability and energy efficiency. To address this challenge, we propose a wave-adaptive beamforming framework to satisfy data-rate demands within limited power budgets. This strategy targets scenarios where sea state information is unavailable, such as in network-assisted systems. We propose a Two-Dimensional Thompson Sampling (2DTS) scheme that jointly selects beamwidth and transmit power to satisfy the target-rate constraint with minimal power consumption and thus achieve maximal energy efficiency. This adaptive learning approach effectively balances exploration and exploitation, enabling efficient operation in uncertain and changing sea conditions. In simulation, under a moderate sea state, 2DTS achieves an energy efficiency of $1.26 \times {10}^4$ bps/Hz/J at round 600, which is $73.7\%$ of the ideal ($1.71 \times {10}^4$), and yield gains of $96.9\%$ and $447.8\%$ over exploration-based TS and conventional TS, respectively. Under a harsh sea state, 2DTS attains $3.09 \times {10}^4$ bps/Hz/J ($85.6\%$ of the ideal $3.61 \times {10}^4$), outperforming the exploration-based and conventional TS by $83.9\%$ and $113.1\%$, respectively. The simulation results demonstrate that the strategy enhances energy efficiency, confirming its practicality for maritime communication systems constrained by limited power budgets.

1. Introduction

The growing demand for advanced maritime applications, such as surveillance, environmental monitoring, offshore platform management, and disaster response services, requires buoy-based ocean communication networks [1,2]. In particular, lighthouses and maritime buoys now play a critical role, going beyond simple signaling to transmitting real-time environmental and situational data including wave height, surrounding visibility, and environmental video feeds. These high data-rate requirements necessitate more efficient wireless transmission strategies than the low-rate, omnidirectional links traditionally used for basic telemetry (e.g., wave height, air temperature).

Unlike terrestrial networks, maritime communication systems must deal with a highly dynamic ocean environment. Waves, wind and currents induce buoy motions that tilt, heave and rotate, increasing the probability of outage and energy consumption [3]. Although these effects are governed by the sea state, real-time sea state information is often unavailable. As a result, static power allocation schemes often result in inefficiencies in spectral and energy resources [4]. To address this, recent studies have explored adaptive techniques such as beamforming and multi-hop relaying to strengthen maritime links [5,6]. While directional beamforming can focus energy toward the intended receiver [7], most existing methods presume prior knowledge of sea conditions at the buoys, limiting their utility in the use cases when the sea conditions or channel state information are not available [8,9]. Buoys generally possess only partial and delayed knowledge of the prevailing sea state for uplink beamforming, whereas the base station (BS) can infer sea-state dynamics more reliably from satellite-based, wide-area forecasts. Moreover, it is noted that the instantaneous beamforming that adapts to the changing orientation of the buoy requires channel state information feedback from the base station, where cellular maritime communication such as Long-Term Evolution for Maritime (LTE-M) lacks a Multiple-Input Multiple-Output (MIMO) framework.

To address this challenge with respect to the uncertainty of the sea state and its related feedback channel unavailability in the current marine cellular system, energy-efficient uplink transmission becomes a crucial design goal, since maritime buoys operate under limited battery capacity and must maintain reliable connectivity under harsh sea-state variations. Recent studies have emphasized the importance of optimizing power efficiency and robustness in underwater and backscatter communication scenarios, such as in the Anti-Jamming Colonel Blotto Game for Underwater Acoustic Backscatter Communication [10]. Motivated by these challenges, we aim to minimize power consumption while satisfying the target rate constraint in uncertain sea-state environments. Therefore, we propose an adaptive beam and power control method based on Thompson Sampling (TS)-based reinforcement learning [11] without knowledge on the sea state directly from the base station. Because the base station cannot have knowledge on the sea state directly, we exploit the feedback information to implicitly infer the current sea state. To optimize beamwidth and power allocation under uncertain conditions, we employ the Thompson Sampling (TS) framework, a Bayesian algorithm widely used in multi-armed bandit (MAB) problems [12,13,14]. TS is known to efficiently balance exploration and exploitation by sampling from the posterior belief distributions of each action. In particular, the posterior is updated based on binary feedback success or failure, while allowing the algorithm to gradually converge to the most promising actions. TS has been successfully applied in areas such as online advertising, recommendation systems, and clinical trials. In communication systems, it has been used for channel selection, interference management, and power control, including underwater acoustic communications and dynamic spectrum access [14,15].

Unfortunately, most of the above-mentioned applications focus on maximizing performance metrics like throughput or reliability, with limited attention paid to energy efficiency [16]. Unlike these existing applications, in our scenario, beamforming performance is highly sensitive to sea-state conditions, where dynamic wave motion causes variations in signal alignment and channel quality. Alongside strict power constraints at buoys, this creates a multi-dimensional optimization problem involving both beamwidth and transmit power, which makes standard TS not suitable for our problem. This limitation arises because conventional Thompson Sampling (TS) is designed to maximize the expected reward of a single action rather than to handle constrained or multi-dimensional objectives. If the reward is defined as the achieved rate, TS tends to converge toward high-power actions, since higher power monotonically increases the rate. Even when the reward is represented as a binary indicator of meeting the target rate, the algorithm still favors the high power that maximizes success probability, without accounting for power minimization. By contrast, our problem follows an objective: minimize transmit power subject to satisfying a predefined target rate, which cannot be directly encoded in the standard TS formulation. Moreover, unlike traditional TS that optimizes a single action, our setting requires joint exploration and exploitation over two coupled decision variables, beamwidth and transmit power, which form a two-dimensional action space. To address this challenge, we develop a Two-Dimensional Thompson Sampling (2DTS) approach that maintains beta posteriors for each beamwidth–power pair and applies a dominance-aware selection rule. Among all feasible pairs whose sampled success probabilities exceed the target reliability, 2DTS selects the configuration with the lowest power, thereby ensuring energy-efficient yet reliable transmission under uncertain sea-state conditions. The goal of 2DTS is to jointly select the optimal beamwidth and the minimum transmission power required to meet a predefined target data rate. This is achieved by updating the shape parameters $\alpha$ and $\beta$ of the posterior beta distribution for each beamwidth and power pair selection actions, using binary feedback from the base station indicating whether the target rate was successfully achieved. This approach enables adaptive learning that accounts for underlying sea-state variations while minimizing power consumption. By integrating 2DTS into our framework, we provide an efficient solution that balances exploration and exploitation, ensuring reliable and energy-efficient communication in dynamic and uncertain maritime environments.

The paper is organized as follows. Section 2 models wave-induced fluctuations in the uplink transmit beam gain by incorporating buoy motion and sea-surface dynamics. Section 3 presents the Two-Dimensional Thompson Sampling (2DTS) algorithm, which jointly selects beamwidth and transmit power to meet the target-rate constraint with minimal power, thereby maximizing energy efficiency. Section 4 reports simulation results that validate the modeling and quantify the gains of the proposed approach. Finally, Section 5 concludes the paper.

2. System Model

As illustrated in Figure 1, we consider an uplink maritime communication scenario where a buoy equipped with N transmit antennas communicates with a single-antenna base station (BS). The buoy’s location and orientation are assumed to be known, so the buoy employs predefined fixed transmit beamforming aiming at the base station without dynamic adjustments. The buoy transmits the scalar information symbol $x\sim \mathcal{CN}(0,P)$ to the BS through a linear beamforming vector $\mathbf{f}\in {\mathbb{C}}^{N\times 1}$. Accordingly, the received signal, denoted as y at the BS, is represented as

$$y=\mathbf{h}\mathbf{f}x+n,$$

(1)

where $\mathbf{h}\in {\mathbb{C}}^{1\times N}$ is the channel between the buoy and BS, and $n\in \mathbb{C}$ follows an additive white Gaussian noise (AWGN) distribution [17]. For simplicity, $\mathbf{f}$ is chosen from a predefined vertical beam vector set $\mathcal{F}=\{{\mathbf{f}}_1,\dots ,{\mathbf{f}}_S\}$, where they are the arrays steering to BS with different vertical beamwidths. It is assumed that the uplink beam vectors are designed to be horizontally isotropic and vertically directional because the buoy can frequently be rotating due to the ocean current. ${\mathbf{f}}_1$ indicates the beamforming vector with beamwidth ${\theta }_1={\displaystyle \frac{\pi }{4}}$, and subsequent beamwidths are progressively halved, given by

$${\theta }_s={\displaystyle \frac{\pi }{2^{s+1}}},s=1,2,\dots ,5,$$

(2)

corresponding to ${\displaystyle \frac{\pi }{4}},{\displaystyle \frac{\pi }{8}},{\displaystyle \frac{\pi }{16}},{\displaystyle \frac{\pi }{32}},$ and ${\displaystyle \frac{\pi }{64}}$ [18]. For modeling the channel $\mathbf{h}$, following [19], we assume

$$\mathbf{h}=\sqrt{{\displaystyle \frac{K}{K+1}}}{\mathbf{h}}_{\mathsf{LoS}}+\sqrt{{\displaystyle \frac{1}{K+1}}}{\mathbf{h}}_{\mathsf{NLoS}},$$

(3)

where ${\mathbf{h}}_{\mathsf{LoS}}$ and ${\mathbf{h}}_{\mathsf{NLoS}}$ represent the Line-of-Sight (LoS) and the Non Line-of-Sight (NLoS) components of the channel, respectively. For the NLoS component, we assume that each element in ${\mathbf{h}}_{\mathsf{NLoS}}$ is drawn from $\mathcal{CN}(0,1)$. This part captures the scattered multipath components. The Rician factor K represents the ratio of the power in the LoS component to the power in the NLoS component. The LoS component, ${\mathbf{h}}_{\mathsf{LoS}}$, reflects the deterministic nature of the direct transmission path, which is particularly dominant in the maritime environment due to the lack of obstacles between the BS and the buoy [20]. Although the buoy is placed at known locations, the sea surface moves dynamically and causes buoy orientation changes, resulting in misalignment of the direction of the transmit array of the buoy and degradation of the gain at BS [21].

Statistical Modeling of Ocean Beam Channel

Incorporating the dynamic ocean environment into the channel model, the modified channel is expressed as

$${\mathbf{h}}_{\mathsf{dyn}}=\sqrt{{\displaystyle \frac{K}{K+1}}}{\widehat{\mathbf{h}}}_{\mathsf{LoS}}+\sqrt{{\displaystyle \frac{1}{K+1}}}{\mathbf{h}}_{\mathsf{NLoS}},$$

(4)

where

$${\widehat{\mathbf{h}}}_{\mathsf{LoS}}\left[i\right]=e^{j\pi (i-1)sin({\mathrm{\Theta }}_{\mathrm{AoD}}+{\mathrm{\Omega }}_0)}$$

(5)

where ${\mathrm{\Theta }}_{\mathrm{AoD}}$ is the angle of departure at the buoy and ${\mathrm{\Omega }}_0$ is derived from the buoy’s angular displacement caused by ocean wave dynamics modeled via the ISSC spectrum [3]. To account for the impact of the beamwidth on the overall channel performance, the effective channel gain is defined as the projection of the channel vector onto the beamforming vector. Specifically, the effective channel gain $H_G$ is expressed as follows:

$$H_G={\parallel {\mathbf{h}}_{\mathsf{dyn}}\mathbf{f}\parallel }^2,$$

(6)

where $H_G$ denotes the effective channel gain after considering the effects of the dynamic ocean environment, and $\mathbf{f}$ represents the beamforming vector corresponding to a specific beamwidth [22]. In high sea states, large wave heights induce substantial angular variations in the buoy, thereby increasing the probability of steering misalignment, particularly when employing narrower transmit beams. While a narrower beam can provide larger array gain, its susceptibility to misalignment may severely degrade system performance under adverse conditions. Consequently, the buoy must adapt its beam vector to preserve robustness against wave-induced motion in the current sea state. The optimal beamwidths and the corresponding minimum transmit powers required to meet the target across different sea states are summarized in

Table 1. Optimal beamwidth and minimum power required per sea state.

Sea State	Optimal Beamwidth (°)	Minimum Power (W)
1	2.81	0.0013
2	5.62	0.0031
3	11.25	0.0106
4	11.25	0.0304
5	22.50	0.0416
6	22.50	0.0563
7	22.50	0.0779
8	45.00	0.1373
9	45.00	0.1625

Note: Values are computed for a target rate of 6 bps/Hz.

.

3. Joint Beamwidth and Power Selection Using 2DTS

This section introduces a joint beamwidth and power selection strategy for maritime communication using 2DTS. In scenarios where the sea-state information is known at the buoy, effective adaptation is crucial. Unlike the Upper Confidence Bound (UCB) algorithm, which struggles under dynamic conditions due to its reliance on confidence intervals, TS achieves a better balance between exploration and exploitation through probabilistic sampling. The proposed approach extends standard TS by jointly selecting the best beamwidth and minimum transmit power, enabling network-assisted real-time adaptation to varying maritime environments. Simulation shows that 2DTS outperforms standard TS, leading to lower power consumption, higher reliability, and robust operation under dynamic and uncertain sea states.

3.1. Standard Thompson Sampling

Thompson Sampling (TS) is a popular Bayesian decision-making algorithm tailored for uncertain environments, particularly effective in MAB problems. The fundamental objective in a MAB setting is to maximize cumulative rewards by choosing among K possible actions (or arms), each with an unknown reward distribution. TS accomplishes this by sampling from posterior distributions of the reward means and updating these distributions as new data (rewards) are observed. Over time, it allocates resources to actions believed to yield higher returns while still periodically exploring other actions to gather new information. Below, we provide an expanded description of TS, emphasizing the Bayesian structure and adaptive update rules.

3.1.1. Bayesian Multi-Armed Bandit Formulation

Consider a MAB with K arms, where each arm $a_k$ is associated with an unknown reward distribution. Let ${\mu }_k$ denote the (unknown) mean reward of arm k. The agent, or decision-maker, seeks to maximize the cumulative reward obtained by sequentially selecting arms over T rounds. At each round t, the agent selects an arm $a_{k^{*}}$, observes a reward $r_{k^{*}}$ drawn from the distribution with mean ${\mu }_{k^{*}}$, and subsequently updates its posterior belief about ${\mu }_{k^{*}}$ based on the observed reward. The central challenge lies in balancing exploration of insufficiently understood arms with exploitation of arms already identified as yielding high rewards.

3.1.2. Bayesian Perspective and Thompson Sampling Logic

From a Bayesian standpoint, each mean reward ${\mu }_k$ is modeled with a prior distribution that reflects initial uncertainty. As rewards are observed, Bayes’ rule is applied to update this belief, yielding a posterior distribution $P({\mu }_k\mid \mathrm{data})$. Thompson Sampling exploits this posterior by sampling a candidate value of ${\mu }_k$ for each arm and selecting the arm with the highest sampled value. This naturally balances exploration and exploitation, as arms with uncertain but potentially high means can still be chosen. The procedure can be described step by step as follows:

(1)

Initialization.

Define a prior distribution $P\left({\mu }_k\right)$ for each arm $k\in \{1,\dots ,K\}$. The choice of prior depends on the reward type:

• For Bernoulli rewards (i.e., rewards are either 0 or 1), a beta distribution is typically selected:

  $${\mu }_k\sim \mathrm{Beta}({\alpha }_k,{\beta }_k),$$

  (7)

  where ${\alpha }_k$ and ${\beta }_k$ are positive hyperparameters. This is because the beta distribution is the conjugate prior for the Bernoulli likelihood, ensuring a simple form of posterior updates.
• For Gaussian rewards, a normal prior is often chosen:

  $${\mu }_k\sim \mathcal{N}\left({\mu }_{0,k},{\sigma }_{0,k}^2\right),$$

  (8)

  where ${\mu }_{0,k}$ is the initial mean, and ${\sigma }_{0,k}^2$ is the initial variance. The normal distribution is a conjugate prior for the Gaussian likelihood, which keeps posterior updates tractable.

These priors capture the agent’s initial beliefs about the reward means before any data is observed.

(2)

Sampling.

In each round t, for each arm $a_k$, sample a reward estimate ${\widehat{\mu }}_k$ from the posterior distribution $P\left({\mu }_k\right|\mathrm{data})$. This posterior reflects the agent’s updated knowledge after previous selections. Concretely,

• Bernoulli rewards (Beta prior):

  $${\widehat{\mu }}_k\sim \mathrm{Beta}({\alpha }_k+S_k,{\beta }_k+F_k),$$

  (9)

  where $S_k$ is the total number of observed “successes” (reward = 1) and $F_k$ the total number of observed “failures” (reward = 0) for arm k. Each time arm k is played, the parameters ${\alpha }_k,{\beta }_k$ are updated accordingly.
• Gaussian rewards (Normal prior):

  $${\widehat{\mu }}_k\sim \mathcal{N}\left({\displaystyle \frac{{\mu }_{0,k}/{\sigma }_{0,k}^2+\sum r_k/{\sigma }^2}{1/{\sigma }_{0,k}^2+n_k/{\sigma }^2}},{\displaystyle \frac{1}{1/{\sigma }_{0,k}^2+n_k/{\sigma }^2}}\right).$$

  (10)
  Here, $\sum r_k$ is the sum of all rewards observed from arm k, $n_k$ is the number of times arm k has been selected, and ${\sigma }^2$ is the known variance of the underlying reward distribution.

By drawing a sample ${\widehat{\mu }}_k$ from each posterior, TS naturally incorporates both the estimated mean and associated uncertainty.

(3)

Action Selection.

After sampling one candidate ${\widehat{\mu }}_k$ for each arm k, the algorithm chooses the arm

$$a_{k^{*}}=arg\underset{k}{max}{\widehat{\mu }}_k.$$

(11)

Thus, the selected arm is the one that this particular sample indicates has the largest mean reward. Over many rounds, arms with higher posterior means or lower uncertainty will be chosen more often, but less-known arms still have a chance to be sampled if their posterior draws occasionally exceed those of well-explored arms.

(4)

Update.

The chosen arm $a_{k^{*}}$ is played, and the resulting reward $r_{k^{*}}$ is observed. The posterior for ${\mu }_{k^{*}}$ is then updated according to Bayesian rules:

• Bernoulli rewards: If $r_{k^{*}}\in \{0,1\}$, update

  $${\alpha }_{k^{*}}\leftarrow {\alpha }_{k^{*}}+r_{k^{*}},{\beta }_{k^{*}}\leftarrow {\beta }_{k^{*}}+(1-r_{k^{*}}).$$

  (12)
• Gaussian rewards: Update the mean and variance parameters of the normal posterior using standard Gaussian conjugate updating formulas, incorporating the new data point $r_{k^{*}}$.

• This feedback loop is repeated in each subsequent round, enabling the algorithm to progressively refine its understanding of each arm’s reward potential.

3.2. Problem Formulation

In this paper, the objective is to select the beamwidth and the transmit power level so that the achieved rate meets or exceeds a predefined threshold while minimizing the power used. To achieve this, we utilized 2DTS to select the best beamwidth and minimum power level.

Let ${\theta }_i$ denote the available beamwidth options, where $i\in 1,2,\dots ,N$, and $P_j$ denote the available power levels, where $j\in 1,2,\dots ,M$. The objective is to maximize the cumulative number of successful transmissions over multiple trials while minimizing energy consumption.

3.3. Thompson Sampling with Joint Beamwidth and Power Selection

We apply 2DTS to jointly select the beamwidth and the power level in each trial to find the beamwidth and the minimum power that meet a given rate threshold r. The algorithm operates as follows:

• Initialize the parameters $\alpha$ and $\beta$ for the beta distribution for each beamwidth i and each power level j, with ${\alpha }_i=1$, ${\beta }_i=1$, ${\alpha }_{i,j}=1$, and ${\beta }_{i,j}=1$ for all i and j.
• For each trial t

  –

  Sample ${\theta }_i$ from $\mathrm{Beta}({\alpha }_i,{\beta }_i)$ for each beamwidth i.

  –

  Select the beamwidth $b_t$ with the highest sampled value:

  $$b_t=arg\underset{i}{max}{\theta }_i$$

  (13)

  –

  For the selected beamwidth $b_t$, establish the power value corresponding to the maximum sampled probability from the beta distribution $P_{i,j}$ from $\mathrm{Beta}({\alpha }_{i,j},{\beta }_{i,j})$ for each power level j, then select the power level $p_t$ with the highest sampled value:

  $$p_t=arg\underset{j}{max}{P}_{i,j}$$

  (14)

  The algorithm selects the power level $p_t$, aiming to minimize power usage by choosing the lowest possible power level that still satisfies the target threshold r. Here, $R_t$ denotes the instantaneous channel capacity, while the threshold r is set strictly below this maximum capacity to represent the actual transmission rate. Thus, the algorithm checks whether the achievable rate $R_t$ exceeds r, treating r as the success criterion for transmission.

  –

  Calculate the SNR for the selected beamwidth and power level:

  $${\mathrm{SNR}}_t={\displaystyle \frac{p_t{H}_G}{N}},$$

  (15)

  where $H_G$ is the effective channel gain associated with beamwidth $b_t$ and N is the noise power. Using this SNR, compute the rate $R_t$ as follows:

  $$R_t={log}_2(1+{\mathrm{SNR}}_t),$$

  (16)

  –

  Determine the reward $r_t$ based on whether the achievable rate $R_t$ meets or exceeds the target threshold r:

  $$r_t=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{R}_t\ge r\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (17)

  –

  Update the beta distribution parameters for the selected beamwidth and power level based on the observed reward:

  *

  For the selected beamwidth $b_t$

  $${\alpha }_{b_t}={\alpha }_{b_t}+r_t,$$

  (18)

  $${\beta }_{b_t}={\beta }_{b_t}+(1-r_t),$$

  (19)

  *

  For the selected power level $p_t$ of the selected beamwidth $b_t$

  • If $r_t=1$, this means success (the rate threshold is met or exceeded); increase $\alpha$ for the selected power level and the level immediately below it (if it exists). This update encourages the selection of lower power levels that can meet the threshold, improving power efficiency:

    $${\alpha }_{b_t,p_t}={\alpha }_{b_t,p_t}+r_t,$$

    (20)

    $${\alpha }_{b_t,p_{t-1}}={\alpha }_{b_t,p_{t-1}}+r_t\mathrm{if}{p}_{t-1}\mathrm{exists},$$

    (21)
  • If $r_t=0$, this means fail (the rate threshold is not met); increase $\beta$ for the selected power level and all power levels below it. This penalizes lower power levels that fail to meet the threshold, encouraging the selection of higher power levels when necessary:

    $${\beta }_{b_t,1:p_t}={\beta }_{b_t,1:p_t}+(1-r_t),$$

    (22)

In this approach, 2DTS iteratively updates its estimates of the success probabilities for each power level, allowing it to identify the minimum power level that achieves the required rate with high confidence. Adjusting $\alpha$ and $\beta$ for each beamwidth and power level based on the observed rate, the algorithm efficiently balances exploration and exploitation, focusing on power levels that maximize energy efficiency while meeting the communication threshold Algorithm 1.

Algorithm 1: 2DTS for Beamwidth and Power selection
Require: Threshold r, Noise power $N_0$, Beamwidth options $\theta$, Power levels P, Rate R .
1:	Initialize ${\alpha }_i=1$, ${\beta }_i=1$ for all $i\in b$, and ${\alpha }_{i,j}=1$, ${\beta }_{i,j}=1$ for all $i\in \theta$, $j\in P$.
2:	for $t=1$ to T do
3:	Sample ${\theta }_i\sim \mathrm{Beta}({\alpha }_i,{\beta }_i)$ for each $i\in \theta$.
4:	Select beamwidth $b_t=arg{max}_i{\theta }_i$.
5:	For the selected $b_t$, sample $P_{i,j}\sim \mathrm{Beta}({\alpha }_{i,j},{\beta }_{i,j})$ for each $j\in P$.
6:	Select power level $p_t=arg{max}_j{P}_{i,j}$.
7:	Compute SNR: ${\displaystyle \frac{p_t{H}_g}{N}}$.
8:	Compute rate: $R_t={log}_2(1+{\mathrm{SNR}}_t)$.
9:	Compute reward $r_t=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{R}_t\ge r\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$.
10:	if $r_t=1$ then
11:	Update ${\alpha }_{b_t}\leftarrow {\alpha }_{b_t}+r_t$.
12:	Update ${\alpha }_{b_t,p_t}\leftarrow {\alpha }_{b_t,p_t}+r_t$.
13:	if $p_{t-1}$ exists then
14:	Update ${\alpha }_{b_t,p_{t-1}}\leftarrow {\alpha }_{b_t,p_{t-1}}+r_t$.
15:	end if
16:	else
17:	Update ${\beta }_{b_t}\leftarrow {\beta }_{b_t}+r_t$.
18:	Update ${\beta }_{b_t,1:p_t}\leftarrow {\beta }_{b_t,1:p_t}+r_t$.
19:	end if
20:	end for

3.4. Energy Efficiency

We confirmed the proposed 2DTS joint beamwidth–power selection over T trials. At each trial t, 2DTS samples a beam and a power. The cumulative number of successful transmissions at trial t is

$$R_T=\sum _{t=1}^T{r}_t.$$

(23)

where $r_t\in \{0,1\}$ denotes the binary reward at round t. The cumulative power used over T trials is

$$P_{\mathrm{total}}=\sum _{t=1}^T{p}_t,0<p_t\le P_{max},b_t\in \theta ,p_t\in P.$$

(24)

Accordingly, the overall energy efficiency after T trials is

$${\mathrm{EE}}_T={\displaystyle \frac{R_t}{P_{\mathrm{total}}}}={\displaystyle \frac{{\sum }_{t=1}^T{r}_t}{{\sum }_{t=1}^T{p}_t}}.$$

(25)

Equivalently, 2DTS seeks to maximize ${\mathrm{EE}}_T$ over the following sequence:

$$\begin{array}{cccc}& \underset{{\left\{(b_t,p_t)\right\}}_{t=1}^T}{\mathrm{maximize}}\hfill & & {\mathrm{EE}}_T={\displaystyle \frac{R_t}{P_{\mathrm{total}}}}\hfill \\ & \mathrm{subject}\mathrm{to}\hfill & & r_t=\left\{\begin{array}{cc}1,\hfill & {log}_2(1+{\mathrm{SNR}}_t)\ge r,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\hfill \\ & & & 0<p_t\le p_{max},b_t\in \theta ,p_t\in P,t=1,\dots ,T.\hfill \end{array}$$

The results indicate that the algorithm successfully converges to the optimal beamwidth and power level, achieving reliable communication with minimum energy consumption.

4. Simulation Results

In this section, we present simulation results that quantitatively demonstrate the effectiveness of the proposed 2DTS algorithm in selecting the optimal beamwidth and transmit power under varying sea states. Performance is measured in terms of cumulative energy efficiency ($r/p$) in iterative rounds. Here, r represents the achieved data rate (in bps/Hz) obtained from the simulation, rather than the binary success indicator used within the Thompson Sampling process. Accordingly, the energy efficiency (EE) metric has the physical unit of bits/Joule/Hz, consistent with the standard definition of spectral energy efficiency. For clarity, all relevant figures are labeled as “Energy Efficiency (bps/Hz/J)”. In buoy-based maritime uplinks, energy efficiency (EE) is a primary design objective due to strict power budgets. Throughout this section, we use $\mathrm{EE}=r/P_t$, where r denotes the achieved rate in bps/Hz (i.e., spectral efficiency, SE). Hence, EE implicitly captures SE in its numerator while simultaneously reflecting the rate–power trade-off that is central to battery-constrained buoy operation. In our target-rate setting, methods that satisfy the reliability constraint yield comparable SE at the threshold, so the observed EE gains of 2DTS primarily arise from using lower transmit power to meet the same target. Qualitatively, therefore, SE trends are consistent with the reported EE improvements. Figure 2 and Figure 3 show the energy efficiency achieved by four schemes: (i) an ideal case that assumes perfect prior knowledge of the optimal beamwidth and power at each round and (ii) the proposed 2DTS algorithm. (iii) The “modified TS with a dedicated exploration phase” (Exploration TS) follows an explore–then–commit protocol over the joint action space of beamwidth and power. During the first $T_{\exp }$ rounds, actions are sampled uniformly (no exploitation) and beta posteriors are updated from binary success/failure. At $t=T_{\exp }+1$, the algorithm commits to a single pair: among arms whose posterior-mean success probability meets the target reliability, it picks the minimum-power configuration; if none is feasible, it selects the arm with the largest posterior mean and uses it for the remaining rounds and (iv) a conventional TS that independently selects beamwidth and power using standard TS. As shown in Figure 2, under a moderate sea state (sea state 4), the proposed 2DTS achieves an energy efficiency of approximately $1.26\times {10}^4$ (bps/Hz/J) at round 600, which is 73.5% of the ideal case performance ($1.71\times {10}^4$ (bps/Hz/J)). This result clearly outperforms both the exploration-based variant ($0.64\times {10}^4$) and the conventional TS ($0.23\times {10}^4$), indicating relative gains of 96.9% and 447.8% over the exploration-based and conventional TS baselines, respectively. As shown in Figure 3, for a harsh sea state (sea state 9), the performance gap narrows further. In round 600, 2DTS achieves $3.09\times {10}^4$ (bps/Hz/J), corresponding to 85.6% of the ideal case ($3.61\times {10}^4$), while exploration TS and conventional TS reach $1.68\times {10}^4$ and $1.45\times {10}^4$, respectively. This translates to a performance gain of 84.0% over conventional TS, again highlighting the robustness of the 2DTS framework in dynamically fluctuating maritime environments. The superior performance of 2DTS arises from its joint optimization structure.

5. Discussion

5.1. Key Findings

Across moderate (Sea State 4) and harsh (Sea State 9) sea states, the proposed 2DTS consistently achieves high spectral energy efficiency (EE) while meeting the target-rate constraint. At round 600, 2DTS attains $1.26\times {10}^4$ bps/Hz/J in sea state 4 (73.5% of the ideal) and $3.09\times {10}^4$ bps/Hz/J in sea state 9 (85.6% of the ideal), outperforming exploration–then–commit TS and conventional TS by large margins. These results indicate that feasibility-first selection combined with minimum-power control is effective under sea-state uncertainty.

5.2. Joint vs. Independent Selection

Unlike conventional TS that treats beamwidth and power independently, 2DTS jointly updates and selects (beamwidth, power) pairs. This joint treatment captures the coupling between directional gain (beamwidth) and energy usage (power), reducing both misalignment-induced losses and unnecessary transmit power. The observed gains over conventional TS in Figure 2 and Figure 3 are consistent with this mechanism.

5.3. Effect of Explore–Then–Commit

The explore–then–commit baseline improves naive independent selection by dedicating an initial exploration budget; however, its one-shot commitment is vulnerable to subsequent sea-state shifts. In contrast, 2DTS continuously re-balances exploration and exploitation, which leads to higher cumulative EE across rounds.

5.4. Sensitivity and Limitations

Performance depends on the reliability threshold, the discretization of the (beamwidth, power) action grid, and the duration of the exploratory budget in the explore–then–commit baseline ($T_{\exp }$). While our setup requires only binary success/failure feedback (compatible with LTE-M systems lacking full MIMO/Channel State Information), extremely severe sea states or overly coarse action grids may slow convergence. Incorporating coarse contextual signals (e.g., forecasted sea state) or adaptive action refinement could further improve sample efficiency.

5.5. Practical Implications and Future Work

The results suggest that lightweight, feedback-driven learning can provide reliable and energy-efficient operation for buoy-based uplinks without explicit sea-state information. Future work includes context-aware 2DTS (e.g., integrating satellite-based forecasts as side information), multi-buoy cooperation, and relay-assisted links.

Summary and Novelty

Unlike standard TS, which treats each parameter selection problem independently, our 2DTS algorithm simultaneously selects the optimal beamwidth–power pair through a joint posterior update that captures both directional-gain dynamics (via beamwidth) and power–efficiency trade-offs. This design is tailored to maritime systems where beam misalignment and power constraints are tightly coupled due to wave-induced platform motion. To the best of our knowledge, no existing TS-based algorithm directly addresses the joint selection of beamwidth and power levels under unknown and dynamic sea-state conditions. Although advanced variants of TS (e.g., contextual or neural TS) have been proposed in other domains, they are generally ill-suited to the maritime scenario considered here. In our setting, the decision space is discrete and multidimensional and is tightly coupled with physical constraints such as misalignment and energy efficiency. Therefore, our approach constitutes a novel TS formulation tailored to this setting, offering a practical solution when sea-state information is unavailable. In summary, 2DTS achieves up to 85.6% of the ideal energy efficiency across varying sea states while significantly outperforming baseline methods, demonstrating its suitability for energy-constrained maritime communication environments.

6. Conclusions

In this work, we proposed an adaptive approach for buoy-embedded maritime communication systems, focusing on the selection of beamforming and power control under dynamic sea conditions. For unknown sea states, we developed a novel 2DTS algorithm that simultaneously adjusts beamwidth and transmission power, achieving robust communication even without direct sea-state information. This adaptive framework balances exploration and exploitation to optimize communication parameters over time, closely approximating ideal performance and ensuring energy-efficient operation. Our simulation results validate the effectiveness of this method, demonstrating significant improvements in energy efficiency and reliability in dynamic maritime environments. Our approach advances the field of maritime communications by providing a robust, energy-efficient strategy for dynamic and unpredictable sea conditions.