Movable Wireless Sensor-Enabled Waterway Surveillance with Enhanced Coverage Using Multi-Layer Perceptron and Reinforced Learning

Abstract

Waterway networking environments present unique challenges due to their dynamic nature, including vessel movement, water flow, and varying water quality. These challenges render traditional static surveillance systems inadequate for effective monitoring. This study proposes a novel wireless sensor-enabled surveillance and monitoring framework tailored to waterway conditions, integrating a two-phase approach with a Movement Phase and a Deployment Phase. In the Movement Phase, a Multi-Layer Perceptron (MLP) guides sensors efficiently toward a designated target area, minimizing travel time and computational complexity. Subsequently, the Deployment Phase utilizes reinforcement learning (RL) to arrange sensors within the target area, optimizing coverage while minimizing overlap between sensing regions. By addressing the unique requirements of waterways, the proposed framework ensures both efficient sensor mobility and resource utilization. Experimental evaluations demonstrate the framework’s effectiveness in achieving high coverage and minimal overlap, with comparable performance to traditional clustering algorithms such as K-Means. The results confirm that the proposed approach achieves flexible, scalable, and computationally efficient monitoring tailored to waterway environments.

1. Introduction

Waterways are spaces that play an important role in modern society and are used for various purposes, such as ship operation, ecosystem maintenance, and water quality management. A waterway is a dynamic environment that is a mixture of physical and ecological diversity, and there are complex elements such as ships going on and off the route, activities of various aquatic organisms, and water quality that changes over time and region. These characteristics mean that waterways have more than just a passageway and are an important axis of natural ecosystems and an essential foundation for human activities. Therefore, understanding and monitoring the environment of waterways is essential in various fields such as logistics management, multipath routing, environmental protection, and water quality improvement [1,2,3,4]. In addition to spatial monitoring, recent research highlights the importance of temporal data validity, particularly through the concept of AoI (Age of Information) [5,6]. In dynamic environments like waterways, delayed or outdated environmental data can significantly reduce the effectiveness of decision-making. Incorporating AoI into surveillance design ensures that the system not only captures wide-area data but also prioritizes data freshness in real-time operations.

However, the surveillance and monitoring work of waterways presents challenges that are inherently different from existing land-based systems. The surveillance system in the land environment is designed based on a relatively fixed sensor network and stable monitoring conditions in combination with AI(Artificial Intelligence), 5G-6G communications, WSN (Wireless Sensor Networks), IoT (Internet of Things), UAVs (Unmanned Aerial Vehicles) [7,8,9,10,11,12,13]. On the other hand, the limitations of the traditional monitoring system are clearly revealed due to the ever-changing fluid environment of the waterways. For example, changes in the moving path or water quality of a ship are difficult to predict, and a fixed sensor network cannot guarantee sufficient coverage and efficient data collection. Figure 1 depicts the waterway surveillance for a specific region detected by a movable sensor. In particular, due to the nature of the waterways, a flexible system design that can efficiently cover a large monitoring area and reflect dynamic changes in real time is required [14,15].

Existing barrier-based sensor systems are designed for surveillance-oriented purposes [16,17,18,19]. However, in a waterway environment, it is important to efficiently collect environmental data over a wide area to detect intruders. To this end, a new approach is needed to monitor a wide range by minimizing overlap when deploying sensors and maximizing the sensing radius. For example, a sensor network with reduced overlap is more suitable for collecting environmental data such as water temperature or pollution level, which enables broader and more even data collection.

This study deals with the problem of using a movable sensor network for monitoring work in a waterway. Unlike the existing static sensor arrangement, in this study, sensors are designed to move to the target area and effectively perform sensing in the area. To this end, this study proposes a system composed of the following two steps. First, the moving step of the sensors to the target area, and second, the arranging step of the sensors being effectively arranged within the target area to maximize sensing efficiency. In the first step, the sensors are learned to move efficiently to the target area using MLP. This reduces the computational complexity of the movement path extraction process and allows a simple movement task to be performed quickly. In the second step, sensors find the optimal arrangement within the target area using RL. In particular, it aims to minimize the overlap of sensing areas between sensors and maximize the sensing radius by reflecting the characteristics of the water channel environment. Unlike conventional security-oriented surveillance, it is based on design goals for more effectively collecting environmental data, such as water temperature and pollution level.

The main contributions of this study are summarized as follows.

• First, we design a movable sensor-assisted waterway surveillance model with enhanced coverage using a Multi-Layer Perceptron and Reinforced Learning. Also, we present the system overview and settings, as well as make a formal problem definition for a dynamic waterway environment.
• To resolve the problem, we propose a scheme consisting of Movement Phase and Deployment Phase which provides enhanced coverage and overlapped area minimization. Then, the performances of the proposed scheme are evaluated based on numerical results through extensive simulations with various scenarios and factors, including loss, ratio, train time, traveled distance, and overlapped area.

The remainder of this paper is organized as follows. Section 2 introduces the proposed system with overview, settings and problem definition. Section 3 describes the proposed schemes consisting of Movement Phase and Deployment Phase. Then, in Section 4, the performances of the proposed algorithms are evaluated with detailed discussions through extensive simulations. Finally, in Section 5, we conclude the paper.

2. Proposed System

In this section, we explain the system overview, settings, and problem definitions that are used in the proposed waterway surveillance model.

2.1. System Overview

A waterway is an environment with dynamic characteristics and is used for various purposes such as ship movement, ecosystem activities, and environmental monitoring. These characteristics are essentially different from the environment in which the existing monitoring and monitoring system is designed. While the existing monitoring system is designed to respond to dynamic intruders for a relatively fixed environment, the waterway environment is constantly changing due to dynamic factors such as water flow, ship movement, and water quality changes.

The proposed system consists largely of two stages: the Movement Phase and the Deployment Phase. Those stages work complementarily for efficient operation of the sensor network, with specialized techniques utilized at each stage. In the Movement Phase, the sensor is moved to the target region using an MLP. This step serves to learn and predict the movement direction so that the sensor may reach the target region from the current position. The goal is to effectively move the sensor to the target region while reducing computational complexity. In the Deployment Phase, RL is used to efficiently arrange sensors within the target area. In a waterway environment, the main task is to minimize the overlap of sensing areas between sensors and to maximize the overall coverage. In the Deployment Phase, to achieve this goal, the RL model is trained to optimize the sensor placement. Figure 2 and Figure 3 briefly present an overview of the Movement Phase and Deployment Phase.

2.2. System Setting

• All simulations were implemented in Python 3.8.
• The MLP and RL models used in the proposed framework were implemented using PyTorch 2.0.
• The size of the waterway environment consists of a 2D environment of 100 (m) × 100 (m).
• The size of the target area, where sensors are intensively conducting sensing operations, is 10 (m) × 10 (m).
• The sensor used in the proposed system is a movable sensor in the waterway.
• In a waterway, the movable sensor can only move in four directions: up, down, left, and right, based on the environmental 2D coordinates.

2.3. Problem Definition

This study addresses the problem of achieving efficient surveillance and monitoring in dynamic waterway environments by utilizing a mobile sensor network. Unlike traditional static environments, waterways are characterized by continuous changes due to factors such as water flow, vessel movement, and environmental variability. These dynamic characteristics require sensors to not only respond to moving targets but also monitor static or semi-static phenomena across a constantly shifting landscape. To tackle these challenges, this study proposes a system that ensures efficient sensor movement and deployment within a designated target region.

The problem is divided into two phases: the Movement Phase, which focuses on guiding sensors from random initial positions to the target region, and the Deployment Phase, where sensors are strategically positioned within the target region to optimize sensing coverage while minimizing overlap. The two steps serve complementary roles and aim to propose an effective sensor network that provides computationally efficient movement within the dynamic characteristics of the waterway environment and less overlap of sensing radii in the waterway.

3. Proposed Schemes

The proposed system in this study is divided into two key phases: the Movement Phase and the Deployment Phase. The Movement Phase leverages an MLP to navigate sensors from their initial random positions to a designated target region, while the Deployment Phase employs RL to optimize sensor placement within the target region by maximizing coverage and minimizing overlap.

3.1. Movement Phase with MLP

The Movement Phase serves as the first step in the proposed system, aiming to guide sensors from their initial random positions to a predefined target region. This phase is essential to ensure that sensors are effectively positioned for subsequent deployment while maintaining computational efficiency. Due to the relatively straightforward nature of the movement problem compared with deployment, this phase adopts a Multi-Layer Perceptron (MLP) as a lightweight and fast machine learning model. Figure 4 shows the MLP overview, which is applied to the Movement Phase. The MLP model, a type of feedforward neural network, is trained to map the current sensor position (x,y) to the optimal movement direction (up, down, left, or right) that minimizes the distance to the target region. The input to the model comprises the current coordinates of the sensor, while the output is a probability distribution over the four possible directions. Each probability represents the likelihood that a specific direction will reduce the sensor’s distance to the target region. The model employs a Softmax function to normalize these probabilities and selects the direction with the highest probability, thereby ensuring that the sensor moves closer to the target region with every step.

The proposed architecture with MLP consists of multiple fully connected layers with ReLU activation functions as follows:

$$h_1=\mathrm{ReLU}(W_1·x+b_1)$$

(1)

$$h_2=\mathrm{ReLU}(W_2·h_1+b_2)$$

(2)

$$h_3=\mathrm{ReLU}(W_3·h_2+b_3)$$

(3)

$$\mathrm{output}=\mathrm{Softmax}(W_4·h_3+b_4)$$

(4)

where x represents the input coordinates ($x,y$), $W1$, $W2$, $W3$, $W4$ are the weights, and $b1$, $b2$, $b3$, $b4$ are the biases of each layer. The hidden layers increase and then decrease the number of neurons, enabling efficient feature extraction and decision-making. The output layer produces probabilities for the four movement directions.

$${\mathcal{L}}_{\mathrm{classification}}=-\sum _{i=1}^N{y}_{i}log\left({\widehat{y}}_i\right)$$

(5)

$${\mathcal{L}}_{\mathrm{distance}}=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{d}_{\mathrm{current}}-d_{\mathrm{next}}>0\hfill \\ {(d_{\mathrm{current}}-d_{\mathrm{next}})}^2\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(6)

$${\mathcal{L}}_{\mathrm{total}}=\alpha ·{\mathcal{L}}_{\mathrm{classification}}+\beta ·{\mathcal{L}}_{\mathrm{distance}}$$

(7)

The training process of the MLP incorporates two loss functions to achieve optimal movement. The first is the classification loss, where Equation (5), based on cross-entropy, ensures that the predicted movement direction aligns with the optimal direction. Here, $y_i$ denotes the ground truth label (one-hot encoded) for the i-th training sample, and ${\widehat{y}}_i$ represents the predicted probability for each possible direction.

The second is the distance-based loss where the Equation (6) encourages the sensor to minimize its Euclidean distance to the target region ($d_{\mathrm{current}}$) at each step. The term $d_{\mathrm{next}}$ indicates the predicted distance after the next move, and the loss penalizes cases where this value is not smaller than $d_{\mathrm{current}}$.

By combining these two losses as in Equation (7), with $\alpha$ and $\beta$ representing the weighting factors for classification and distance losses, respectively, the MLP learns to navigate sensors efficiently toward the target region.

This combination leverages the complementary strengths of each loss. The classification loss enhances directional prediction accuracy, and the distance-based loss promotes efficient physical movement toward the target region. Together, they are expected to improve both convergence stability and the model’s ability to generalize across diverse initial positions and dynamic environmental conditions.

The MLP-based Movement Phase provides a computationally efficient and accurate solution for sensor navigation, even in large-scale waterway environments. This phase sets the foundation for the subsequent Deployment Phase, where sensor placement is optimized to maximize coverage and minimize overlap within the target region. Algorithms 1 and 2 describes the procedures in detail.

Algorithm 1 MLP Functions

1: Function InitializeWeights($input\_dim$, $output\_dim$): 2:     Return random weights for given dimensions 3: Function InitializeBias($dim$): 4:     Return zeros for given dimension 5: Function ReLU(x): 6:     Return max$(0,x)$ 7: Function Softmax(x): 8:     $exp\_x\leftarrow exp(x-max(x\left)\right)$ 9:     Return $exp\_x/\sum (exp\_x)$ 10: Function EncodeInput($position$): 11:     Return position as the $input\_vector$ 12: Function SelectMove($probabilities$): 13:     Return the $index$ of the sample probabilistically 14: Function UpdatePosition($current\_position$, $move$): 15:     $move\_mapping$ 16:          $\leftarrow \{0:(0,1),1:(0,-1),2:(-1,0),3:(1,0\left)\right\}$ 17:     $(dx,dy)$ 18:          $\leftarrow move\_mapping\left[move\right]$ 19:     Return $(current\_position.x+dx,$ 20:                                     $current\_position.y+dy)$ 21: Function CalculateDistanceToRegion($position$): 22:     $target\_region\_x\leftarrow [x\_min,x\_max]$ 23:     $target\_region\_y\leftarrow [y\_min,y\_max]$ 24:     If $position.x<x\_min$ 25:          $x\_distance\leftarrow x\_min-position.x$ 26:     Else if $position.x>x\_max$ 27:          $x\_distance\leftarrow position.x-x\_max$ 28:     Else 29:          $x\_distance\leftarrow 0$ 30:     If$position.y<y\_min$ 31:          $y\_distance\leftarrow y\_min-position.y$ 32:     Else if $position.y>y\_max$ 33:          $y\_distance\leftarrow position.y-y\_max$ 34:     Else 35:          $y\_distance\leftarrow 0$ 36:     Return $x\_distance+y\_distance$

Algorithm 2 MLP and Movement Phase with MLP

1: Class MLP: 2:     Initialize weights $(W1,W2,W3,W4)$ 3:                      and biases $(b1,b2,b3,b4)$ 4: Function forward(x): 5:       $h1\leftarrow \mathrm{ReLU}(W1\times x+b1)$ 6:       $h2\leftarrow \mathrm{ReLU}(W2\times h1+b2)$ 7:       $h3\leftarrow \mathrm{ReLU}(W3\times h2+b3)$ 8:       $output\leftarrow \mathrm{Softmax}(W4\times h3+b4)$ 9:       Return $output$ 10: Function MovementPhase($current\_position$, 11:                                          $mlp\_model$): 12:     $input\_vector$ 13:          $\leftarrow \mathrm{EncodeInput}(current\_position)$ 14:     $movement\_probabilities$ 15:          $\leftarrow \mathrm{mlp\_model.forward}(input\_vector)$ 16:     $next\_move$ 17:          $\leftarrow \mathrm{SelectMove}(movement\_probabilities)$ 18:     $initial\_distance$ 19:          $\leftarrow \mathrm{CalculateDistanceToRegion}(current\_position)$ 20:     $next\_position$ 21:          $\leftarrow \mathrm{UpdatePosition}(current\_position,next\_move)$ 22:     $current\_distance$ 23:          $\leftarrow \mathrm{CalculateDistanceToRegion}(next\_position)$ 24:     $diff\_init\_cur$ 25:          ←$initial\_distance$ - $current\_distance$ 26:     $target\_distances$ 27:          $\leftarrow \mathrm{CalculateDistanceToRegion}$        $\left(\mathrm{UpdatePosition}\right(current\_position,move)$ 28:                         $\mathbf{for }move\mathbf{ in }[0,1,2,3])$ 29:     $target\_direction$ 30:         $\leftarrow \mathrm{IndexOfMinimum}(target\_distances)$ 31:     $loss$          $\leftarrow \mathrm{CalculateLoss}(diff\_init\_cur,$ 32:               $movement\_probabilities,target\_direction)$ 33:     Backpropagate loss and update model 34:     Return $next\_move$

3.2. Deployment Phase with RL

The Deployment Phase is the second step in the proposed system, designed to position sensors optimally within the target region after their arrival. This phase aims to maximize the overall sensing coverage while minimizing overlap between the sensing ranges of different sensors. Given the dynamic and constrained nature of the waterway environment, reinforcement learning (RL) is employed to solve this optimization problem. The RL framework defines three key elements: state, action, and reward. The state represents the current positions of the sensors and the coverage map, reflecting the areas already monitored within the target region. The action corresponds to adjustments in the sensors’ positions, allowing them to explore new configurations within their movement constraints. The reward function is carefully designed to evaluate the effectiveness of sensor placements. It incorporates three components: total coverage, overlap penalty, and boundary penalty. The total reward can be expressed as

$$\mathrm{Coverage}=\sum _{i=1}^M\mathrm{coverage\_map}\left[i\right]$$

(8)

$$\mathrm{Overlap}\mathrm{Penalty}=\sum _{i=1}^N\sum _{j=i+1}^{N}max(0,r_i+r_j-d_{ij})$$

(9)

$$\begin{array}{c}\mathrm{Boundary}\mathrm{Penalty}\\ ={\displaystyle \sum _{i=1}^N}(max(0,x_i-x_{\max })+max(0,x_{\min }-x_i)\\ +max(0,y_i-y_{\max })+max(0,y_{\min }-y_i))\end{array}$$

(10)

$$\mathrm{Reward}=\mathrm{Coverage}-(\mathrm{Overlap}\mathrm{Penalty}+\mathrm{Boundary}\mathrm{Penalty})$$

(11)

Equation (8) represents the total coverage, calculated as the sum of all covered cells $\mathit{coverage}\_\mathit{map}\left[i\right]$, where M is the total number of cells in the coverage grid and each element $\mathit{coverage}\_\mathit{map}\left[i\right]$ is 1 if covered and 0 otherwise.

Equation (9) accounts for the overlap penalty by summing the penalty for overlapping sensing ranges among N sensors, where $r_i$ and $r_j$ denote the sensing radii of sensors i and j, and $d_{ij}$ is the Euclidean distance between these sensors.

Equation (10) defines the boundary penalty, which penalizes sensors placed outside the defined target boundaries, with $x_{\max },x_{\min },y_{\max },y_{\min }$ indicating the limits of the operational area.

These three components are combined in Equation (11), where the coverage area is adjusted by subtracting the two penalty values. The weighting of these penalties is critical to maintaining a balance between maximizing coverage and preventing undesirable configurations such as excessive overlap or boundary violations. Through this formulation, the final reward reflects both the quantity of covered area and the quality of the deployment layout.

The RL model implemented in this phase utilizes the Proximal Policy Optimization (PPO) algorithm [20], which balances exploration and exploitation during training. PPO is a reinforcement learning algorithm that incorporates a policy network, responsible for determining actions, and a value network, which estimates future rewards for given states. In the Deployment Phase, the policy network defined as part of the PPO model takes the state as input and outputs actions, allowing iterative adjustments to sensor positions. Additionally, the PPO algorithm uses the value network to guide the learning process effectively and maintain stability. The environment is initialized with key parameters, including map dimensions, the number of sensors, and the sensing radius. The system updates the coverage map dynamically as sensors are repositioned, ensuring that sensor placements reflect the latest configuration. The step function evaluates the reward for each configuration by calculating the total coverage, overlap penalty, and boundary penalty, optimizing sensor placement iteratively based on these metrics. By continuously improving the placement of sensors, the RL model learns strategies that maximize sensing coverage and minimize redundancy. The DeploymentPhaseWithRL function integrates the trained PPO model with the environment, visualizing the final sensor placements and coverage map for evaluation.

Overall, the RL-based Deployment Phase ensures efficient utilization of sensors within the target region, addressing the dynamic and complex challenges of waterway environments. This phase complements the preceding Movement Phase by optimizing sensor placement for effective surveillance and monitoring. Algorithms 3 and 4 describes the procedures of RL functions and RL-based deployment in detail.

Algorithm 3 RL Functions

1: Function InitializeEnvironment(): 2:     $map\_dimensions,num\_sensors,sensing\_radius$ 3:          ← defined 4:     $action\_space,observation\_space$ 5:          ← set 6:     Return $initialized$ $environment$ $state$ 7: Function Reset(): 8:     $sensor\_coordinates$ 9:          ←$randomly\_initialized\_within\_map\_bounds$ 10:     $coverage\_map$ 11:          ←$initializedasempty$ 12:     $coverage\_map$ 13:          ←$updatedwithinitialsensorplacement$ 14:     Return $initial$ $observation$ 15: Function Step($action$): 16:     $reward$← 0 17:     $done$← False 18:     For each $sensor$ 19:          $coordinates$ 20:               ← updated using $action$ 21:          $coordinates$ 22:               ← clipped to remain within map bounds 23:          $coverage\_map$ 24:               ← updated with $new$ $sensor$ $coordinates$ 25:     Compute reward: 26:          $total\_coverage$ 27:               ← sum of all covered cells in coverage map 28:          $overlap\_penalty$ 29:              ← calculated for overlapping sensors 30:          $boundary\_penalty$ 31:              ← calculated for sensors out of bounds 32:          $reward$ 33:               ←$total\_coverage$ - 34:                 $(overlap\_penalty+boundary\_penalty)$ 35:     Return: 36:          $observation$ 37:               ←$sensor\_coordinates$, $coverage\_map$ 38:          $reward$←$total\_coverage$ - $penalties$ 39:          $done$← True 40:          $info$← auxiliary information 41: Function _update_coverage($coord$): 42:     For each cell within the $sensor$ $radius$ 43:          cell ← marked as covered in $coverage\_map$ 44: Function _compute_reward($new\_coords$): 45:     $total\_coverage$ 46:               ← calculated from coverage map 47:     $overlap\_penalty$ 48:               ← computed for overlapping sensors 49:     $boundary\_penalty$ 50:               ← computed for out-of-bound sensors 51:     $reward$ 52:               ←$total\_coverage$ − 53:                 ($overlap\_penalty$ + $boundary\_penalty$) 54:     Return $reward$

Algorithm 4 RL and Deployment Phase with RL

Class RL:     Initialize:       Define policy network:          $Input$ $layer$ $size$               ← observation space dimension          $Hidden$ $layers$               ← a sequence of fully connected layers                    with $ReLU$ activation          $Output$ $layer$ $size$← action space dimension          $Output$ $activation$←$Softmax$       Define value network:          $Input$ $layer$ $size$← observation space dimension          $Hidden$ $layers$               ← a sequence of fully connected layers                       with $ReLU$ activation          $Output$ $layer$ $size$← estimated state value          $Output$ $activation$←$Linear$       $PPO$ $model$← initialized with                           $policy$ and $value$ $networks$       $learning\_rate$← set       $gamma$← set Function train($environment$):     $model$← trained on the provided $environment$     Return $trained\_model$ Function DeploymentPhaseWithRL($environment$,                   $trained\_model$, $initial\_coordinates$):     $initial\_sensor\_coordinates$← set in $environment$     $predicted\_actions$←$trained\_model$.predict()     $sensor\_coordinates$               ← updated based on $predicted\_actions$     Visualize $sensor$ placements and $coverage$     Return $final$ $coverage$ $map$, $sensor$ $positions$

4. Experimental Evaluations

This section evaluates the performance of the proposed movement and deployment stages through detailed numerical experiments. Experimental analysis evaluates the effectiveness of the proposed method by focusing on indicators such as Find Ratio, coverage overlap area size, and learning time, which is the percentage of accurate movement to a specific area. The experimental base environment is configured to be simulated in an environment tailored to the system setting of the Proposed Schemes section, and both stages use parameters that match each algorithm. The Movement Phase evaluates the node’s ability to reach its target location in duplicate with minimal energy consumption, while the Deployment Phase focuses on optimally deploying sensors to maximize coverage and minimize redundancy. To ensure robustness and statistical validity, results are derived based on experiments.

4.1. Movement Phase Evaluation

The Movement Phase is a step in evaluating the process of moving the sensor to the target area in an experimental environment. This step analyzes the sensor’s ability to efficiently reach the target area in its initial position. Therefore, the MLP model is used to determine the direction of movement of each sensor. MLP predicts the optimal direction of movement based on the current coordinate (x, y) of the sensor in the 100 × 100 simulation environment, and selects the direction to maximize the chance of successful arrival in the target area as represented in Figure 5.

In this experiment, we analyzed the performance of the Movement Phase in various initial sensor deployments and target area settings. Key performance indicators are learning time, distance traveled, and Find Ratio, which means the percentage of sensors that reached the target area. We also evaluate whether the proposed loss function was applied properly in the Movement Phase. Identify the efficiency of the proposed migration strategy and MLP-based approach by minimizing losses and optimizing sensor movement. Comprehensively analyze these metrics to verify that the Movement Phase has effectively achieved its intended goals.

4.1.1. Loss

The convergence of the loss function has a direct correlation with the performance of model learning, indicating that the model is gradually learning the optimal pattern. The Movement Phase learning combined Classification Loss with Distance-Based Loss to guide you to learn how to efficiently move to the target area.

For Figure 6, the total loss showed a high value in the early stages of learning, but gradually decreased and showed a tendency to converge stably as learning progressed. This means that the model is gradually learning the best path to the target area. In particular, Classification Loss played an important role in increasing predictive accuracy in the direction of travel, and Distance Loss helped sensors reduce distance from the target area. The combination of these loss functions complements the model to allow simultaneous learning of the optimization of travel paths and access to target areas. Loss convergence results show that the model returns an efficient navigation path by gradually reducing uncertainty during the movement.

4.1.2. Find Ratio and Train Time

Find Ratio shows the percentage of sensors that have successfully accessed the target area within 3000 steps based on a total of 1000 coordinate generations. The performance metrics in Figure 7 are calculated as the average over 100 independent simulation runs for each setting. This is used as an important indicator of how much MLP has generalized learning in the Movement Phase. A higher Find Ratio means that the model has learned to move accurately from initial locations to the target area.

We set the data to 100, 200, 300, 400, and 500 pieces, respectively, and analyzed the performance changes depending on the number of data points. The number of epochs gradually increased from 100 to 500, and the learning rate was set to 0.001. Analysis of the model’s performance in these simulation environments has shown that Find Ratio tends to increase as the number of data points and epochs increases. In particular, with 500 pieces of data, Find Ratio has increased rapidly since the beginning of the epoch, showing high performance. This shows that the more data is used for learning, the better MLP can learn different patterns and generalize the path to the target area. Figure 7 presents the measurements of Find Ratio and train time according to different epochs. For example, when learning with 500 pieces of data, since epoch 200, the Find Ratio remained close to 100%, showing excellent performance. Because the coordinates used in the learning were randomly extracted, the performance might not be constant when the deflected coordinates were extracted. You can see this in a graph, but in some cases, a small number of data (e.g., 400) showed a higher Find Ratio than using more data (e.g., 500). This indicates that random coordinate extraction can overfit a particular pattern or that some significant paths were not learned. Also, even if the initial value of Find Ratio is low, we could see how the Find Ratio gradually improves as the epoch increases. This shows that the MLP model has learned more accurately the direction of gradually moving to the target area through the learning process. However, it should be taken into account that the rate of improvement and final value of the Find Ratio may vary depending on the number of data points and the diversity of initial coordinates. In summary, through Find Ratio analysis, we can check that the performance of the MLP model in the Movement Phase was greatly influenced by the number of data and the learning epoch. Therefore, it is important to maintain a balance between the composition of learning data and the coordinate extraction process, which can further improve the generalization performance of the MLP model.

4.1.3. Traveled Distance

In order to evaluate the performance of the Movement Phase, Manhattan Distance was introduced as a comparison between how to measure the distance of the Movement Phase and how to evaluate it. All reported values represent the average results obtained from 100 independent simulation runs under the same conditions. First, the distance of the Movement Phase is calculated by measuring the total number of movements before the sensor reaches the target area along the direction predicted by the model. This is used as an important indicator of whether the model moves to the target area efficiently. Manhattan Distance, which was used for comparison, is the shortest distance calculation method that reflects the assumption that only four directions (top, bottom, left, right) can be moved. That is, it is a simple and effective criterion for evaluating realistic movement constraints, calculated as the sum of horizontal and vertical travel distances between two points.

As shown in

Table 1. Comparison of movement distances during the Movement Phase with Manhattan Distances for the 5 paths in the Figure 5a.

Path Number	Distance of Movement Phase	Distance of Manhattan
P1	86	86
P2	105	103
P3	35	35
P4	59	58
P5	53	50

,

Table 2. Comparison of movement distances during the Movement Phase with Manhattan Distances for the 5 paths in the Figure 5b.

Path Number	Distance of Movement Phase	Distance of Manhattan
P1	50	50
P2	38	36
P3	40	36
P4	50	49
P5	61	56

and

Table 3. Comparison of movement distances during the Movement Phase with Manhattan Distances for the 5 paths in the Figure 5c.

Path Number	Distance of Movement Phase	Distance of Manhattan
P1	77	74
P2	112	106
P3	45	42
P4	39	33
P5	25	25

, the distance traveled by the route obtained in the Movement Phase showed a difference of up to 5 percent compared with the shortest path in Manhattan under the same conditions. This indicates that the model has a very efficient path-finding capability.

While the proposed MLP-based approach does not explicitly guarantee the absolute shortest path as calculated by graph-based algorithms, it is designed to prioritize computational efficiency and scalability. Conventional graph-based methods often require exhaustive searches or iterative path recalculations, which can substantially increase computational overhead and hinder real-time applicability in dynamic environments. In contrast, the MLP model generates movement decisions with constant time complexity O(1) after training, enabling rapid inference regardless of the size of the environment. Although the measured travel distances may deviate by up to 5% from the corresponding Manhattan Distances, the proposed model consistently maintains a high Find Ratio. This observation indicates that such minor deviations in path length do not compromise the model’s ability to reach the target area effectively. Consequently, the MLP-based approach achieves a practical balance between path efficiency, decision-making speed, and adaptability, making it suitable for real-world deployment scenarios.

4.2. Deployment Phase Evaluation

The Deployment Phase evaluates the optimal placement of sensors that have successfully reached the target area during the Movement Phase. This phase focuses on maximizing the coverage of the 10 × 10 m target area, where the x and y axes represent the coordinate positions within this area, while minimizing the overlap between the sensor’s sensing ranges as shown in Figure 8. By achieving this balance, the Deployment Phase ensures efficient sensor deployment and optimal resource utilization.

The performance of the Deployment Phase was analyzed under various sensor deployments and target area configurations. The key performance indicators used were coverage ratio and overlap ratio. The coverage ratio measures the proportion of the target area covered by the sensors, while the overlap ratio quantifies unnecessary coverage caused by overlapping sensing ranges between sensors. These metrics provide critical insights into the effectiveness of the proposed deployment strategy. We also evaluate whether the reinforcement learning-based optimization algorithm, specifically Proximal Policy Optimization (PPO), was successfully applied during the Deployment Phase. The PPO algorithm incorporates a loss function that penalizes overlap to minimize redundancy while simultaneously optimizing for uncovered regions to ensure maximum coverage. By analyzing these metrics comprehensively, it can be verified that the Deployment Phase effectively achieved its objectives of maximizing coverage and minimizing overlaps. For example, the sensing redundancy rate was determined by combining the set target area size and the number of sensors. This confirms that the proposed algorithm has effectively achieved the goal of minimizing redundancy for coverage maximization in the Deployment Phase.

4.2.1. Loss

The Deployment Phase focuses on optimizing sensor placement to maximize coverage of the target area while minimizing overlap between sensor sensing ranges. The reward function utilized during this phase incorporates three critical components: coverage, overlap penalty, and boundary penalty. The coverage term measures the proportion of the target area effectively covered by the sensors. The overlap penalty penalizes overlapping regions between sensor sensing ranges, ensuring efficient utilization of sensors and minimizing redundancy. Lastly, the boundary penalty discourages sensor placements outside the allowable area, guiding sensors to remain within the target area boundaries.

As illustrated in Figure 9, the total loss during the Deployment Phase initially starts at a high value due to significant overlap and suboptimal placements. However, as the optimization progresses, the loss decreases substantially, showcasing a stable convergence pattern. This indicates that the proposed PPO-based reinforcement learning algorithm effectively learns to balance maximizing coverage and minimizing overlap. The loss convergence highlights the successful application of the overlap penalty and boundary penalty in the reward function. These penalties ensure that the sensors are optimally distributed across the target area, covering as much area as possible while avoiding unnecessary overlap. This demonstrates that the Deployment Phase achieves its objective of efficient sensor placement. Then, the stable convergence of the loss function underscores the reliability and effectiveness of the Deployment Phase in optimizing sensor placement. By achieving a balance between coverage maximization and overlap minimization, the Deployment Phase demonstrates its capability to support robust and efficient sensor networks in the Target Area.

4.2.2. Overlap Minimization

In the Deployment Phase, minimizing overlap between sensor coverage areas is essential for maximizing total coverage within a limited target area. By reducing overlap, the sensing regions are distributed more efficiently, avoiding redundancy and improving resource utilization. To evaluate the effectiveness of the Deployment Phase, the degree of overlap between sensors is measured. The overlap is calculated using the following Equation:

$$\begin{array}{c}\mathrm{Overlap}\mathrm{Area}=\pi r^2\times \left(1-{\displaystyle \frac{d}{2r}}\right)\end{array}$$

(12)

where r is the sensing radius, and d is the Euclidean distance between two sensor centers. If d is smaller than $2r$, the sensors overlap, and this formula estimates the overlap area. By summing the overlap areas for all sensor pairs, the total overlap in the deployment is quantified.

To validate the proposed approach, it is compared with a baseline method using K-Means Clustering. K-Means clustering is commonly used to partition data points into evenly distributed groups, ensuring that sensors are positioned uniformly across the area. However, it does not explicitly address overlap minimization, which can result in suboptimal resource utilization. Now, we compare the proposed Deployment Phase with the K-Means clustering scheme to confirm that the region’s overlap rate is reduced, confirming that the maximum coverage region can be achieved.

Table 4. Comparison of overlap area during the Deployment Phase with K-Means clustering in the Figure 8.

Sensor Number	Deployment Phase Overlap Area	K-Means Clustering Overlap Area
3	0	0
5	3.61	3.30
7	10.29	9.41

presents a comparison of overlap areas during the Deployment Phase using the proposed reinforcement learning-based approach and the K-Means clustering algorithm. Each value in Table 4 represents the average result from 100 simulation runs for the corresponding sensor configuration. Overall, the results indicate that both methods perform similarly in minimizing overlap. For the case of 3 sensors, both approaches achieved an overlap area of 0, demonstrating their effectiveness in avoiding redundant coverage in low-density scenarios. As the number of sensors increased to 5 and 7, some overlap was observed in both methods, with the reinforcement learning-based approach yielding slightly higher values compared with K-Means clustering. However, the differences remain relatively minor, indicating that the proposed model is capable of achieving performance levels close to K-Means clustering. The results highlight the strength of the proposed Deployment Phase model in adapting to different sensor configurations. Unlike K-Means clustering, which distributes sensors evenly without considering specific constraints, the proposed model incorporates objectives such as maximizing coverage and minimizing penalties for overlapping and boundary violations. This adaptability ensures that the model can handle diverse deployment scenarios while achieving the desired outcomes.

5. Conclusions

This study proposed a novel sensor deployment framework designed to address the challenges of dynamic waterway environments. By integrating a Movement Phase using a Multi-Layer Perceptron (MLP) and a Deployment Phase optimized via reinforcement learning (RL), the framework successfully optimized sensor placement for enhanced coverage and minimized overlap. Through comprehensive evaluations, the effectiveness of the proposed framework was validated across various scenarios. The results demonstrate that the Movement Phase enabled sensors to efficiently navigate toward the designated target area, while the Deployment Phase ensured optimal arrangement to maximize coverage while reducing redundancy. Comparisons with the K-Means clustering algorithm further highlighted the adaptability and performance of the proposed approach, achieving comparable results while incorporating additional constraints for real-world applicability. Furthermore, although the current framework does not explicitly optimize for temporal freshness, future extensions could incorporate AoI-based evaluation to quantify how rapidly and reliably environmental updates are delivered. This would provide an additional performance dimension, particularly relevant in fast-changing aquatic environments where delayed information could lead to suboptimal responses.