Energy Efficient Sensor Scheduling with a Mobile Sink Node for the Target Tracking Application

Measurement losses adversely affect the performance of target tracking. The sensor network's life span depends on how efficiently the sensor nodes consume energy. In this paper, we focus on minimizing the total energy consumed by the sensor nodes whilst avoiding measurement losses. Since transmitting data over a long distance consumes a significant amount of energy, a mobile sink node collects the measurements and transmits them to the base station. We assume that the default transmission range of the activated sensor node is limited and it can be increased to maximum range only if the mobile sink node is out-side the default transmission range. Moreover, the active sensor node can be changed after a certain time period. The problem is to select an optimal sensor sequence which minimizes the total energy consumed by the sensor nodes. In this paper, we consider two different problems depend on the mobile sink node's path. First, we assume that the mobile sink node's position is known for the entire time horizon and use the dynamic programming technique to solve the problem. Second, the position of the sink node is varied over time according to a known Markov chain, and the problem is solved by stochastic dynamic programming. We also present sub-optimal methods to solve our problem. A numerical example is presented in order to discuss the proposed methods' performance


Introduction
In a sensor network, a large number of small, low cost sensing devices called sensor nodes are deployed and interconnected to gather information from the field of interest and transmit it to the base station. The gathered information may be used to perform various kinds of tasks, such as environmental monitoring, habit monitoring, intelligent building and military applications. Each sensor node in the sensor field is powered by a battery and consumes energy for various purposes such as sensing, on board signal processing and transmitting. Energy required for transmission is significantly more than other purposes and is a function of the distance between sender and receiver. In general, sensor nodes are stationary and unattended. It is not easy and in most cases impractical to replace or recharge the battery in hazardous environments [1,2]. Once the battery runs out, the sensor node becomes inoperable, increasing the coverage hole and decreasing the connectivity of the whole network [2,3], on which the sensor network's performance depends highly. Therefore, efficient use of the sensor node battery's energy is an important aspect of sensor networks.
The life span of a sensor network is determined by the time duration until the sensor network fails to function due to inadequate number of sensor nodes [2,4]. Many strategies have been proposed to prolong the life span of sensor networks [5,6,7,8]. Activating a set of sensor nodes and putting the rest of the sensor nodes in sleep mode is a technique widely used to prevent unwanted energy consumption. In [9], the low-energy adaptive clustering hierarchy (LEACH) was proposed to distribute the energy consumption evenly throughout the sensor nodes, doubling the life span of the network. The alternative strategies for long distance transmission, such as multi-hop transmission and using a mobile sink node to perform the transmission [10,11] also save significant energy. Data latency is experienced in multi-hop transmission, and the lower the latency the better the performance, especially in time-critical applications. Generally a mobile sink node has more battery energy, but navigating the mobile sink node is a challenging task. Many solutions have been proposed to solve this navigation problem [12,13].
In target tracking, continuous collection of information about the target improves the tracking quality. The loss of the target and the measurement losses seriously affect the tracking quality [14]. Moreover in [15], an information theoretic approach was proposed to estimate the number of targets and their states simultaneously, where the expected information from the targets are maximized by selecting appropriate sensor nodes. In this paper, we show that for a linear Gaussian dynamics, the measurement losses adversely affect the tracking quality. In literature, the sensor scheduling was mainly studied in two different ways: first, scheduling the sensor nodes to minimize the tracking error subject to limited sensor usage and communication costs. Second, scheduling the sensor nodes to minimize the cost of sensor usage and communication subject to constraints on tracking quality. Our previous work [16] focused on maximizing the tracking quality, subject to limited battery energy available at the sensor nodes and we used several techniques to solve this constrained problem. In [17], the energy constraint was relaxed using Lagrangian multipliers and the problem was solved using an approximate dynamic programming technique. Predicting the target and activating a set of sensor nodes closer to the target, rather than activating all the sensor nodes, conserves energy in the sensor network and energy efficient sensor scheduling algorithm was presented in [18]. The optimal sensor scheduling was considered as minimizing the usage of sensor resources (i.e., battery energy and number of sensor nodes) whilst keeping the tracking accuracy at a desired level [19,20,21].
In this study, we minimize the total energy consumed by the sensor nodes whilst avoiding the additional error in the estimation due to the measurement losses. A mobile sink node is used to collect the measurements and transmit them to the base station (BS). The BS schedules the sensor nodes to send the measurements to the mobile sink node without loss considering the minimal energy consumption. In many situations, multiple sensor nodes cannot be activated at the same time due to the limited bandwidth or avoid interferences between sensor nodes. For example sonar sensor nodes cannot be operated simultaneously in the same frequency band in order to avoid interferences [22] and, therefore, a single sensor node is allowed to activate at each time step. In [23], it was assumed that once a sensor node is activated it remains active for a certain time period and can only be changed to another sensor node if the active time period of a sensor node is elapsed. Moreover switching between different types of sensor nodes caused additional cost [24]. In this paper, we assume that a single sensor node can be activated at any time step and each sensor node has an active time period. Therefore, a sensor node cannot be activated during the active period of a previously activated sensor node. We use deterministic and stochastic dynamic programming to solve this problem optimally when the path of the mobile sink node is known fully and partially, respectively. Moreover, we propose sub-optimal methods for both cases, and discuss the performance of the proposed methods with respect to the end result as well as to the computational cost.
The remainder of this paper is organized as follows. Section II constitutes the overview of our problem and analyzes the effect of measurement losses on tracking quality for a linear Gaussian dynamics. In section III, the assumptions and constraints made for this study are presented to formulate the sensor management problem. We describe optimal and sub-optimal methods to solve the sensor management problem for fully and partially known mobile sink node's paths. In Section IV, a numerical example of our problem is presented to analyze the performance and limitations of the proposed methods. Finally, our conclusion and future work are presented in section V. Figure 1 shows a wireless sensor network which is located far form the base station. Several sensors are deployed to track a target which moves in the field. In each time step a known set of sensor nodes cover the target and the sensor measurements are transmitted to a mobile sink node, for example an unmanned aerial vehicle (UAV) is flying over the sensor field. The mobile sink node transmits the collected measurements directly to the base station. The energy required for a sensor node to transmit the measurement depends on the distance between the sensor node and the mobile sink node. The distance is illustrated in Figure 1(b). An activated sensor node far away from the mobile sink node consumes more energy to transmit the measurement, compared to one closer to the mobile sink node and therefore the energy is wasted. Alternatively we may follow two approaches when the activated sensor node is far away from the mobile sink node: first, activating another sensor node closer to the mobile sink node: Second, not transmitting the measurement over a long distance. Once a sensor node is activated it cannot be changed to another one for a certain time period and therefore the first approach is void. Since measurement losses degrade the tracking quality, the second approach is strictly avoided. Therefore, the base station should manage the sensor nodes to minimize the total waste of energy without degrading the tracking quality. The energy model of the sensor node and the importance of availability of measurements for target tracking are presented below.

Sensor Model
The base station (BS) schedules the sensor nodes to either sleep or active mode to perform the tasks efficiently. In order to satisfy the limited bandwidth, one sensor node (or generally a set of sensor nodes) is in active mode at any given instant. We assume that the position of the target belongs to a region denoted by Ω k at time step k and it is known a priori. Let {S 1 , S 2 , ..., S N k } denote the set of feasible sensor nodes for scheduling at time step k and sensor node S i is an element in the set {S 1 , S 2 , ..., S N k } only if its sensing range covers the region Ω k . Here, N k is the total number of feasible sensor nodes available for the scheduling at time step k. If a sensor node S i is activated at time step k, the BS cannot activate another sensor node S j for a period of t i time steps (ie, BS can only activate the S j sensor node at k + t i time step or later). Each sensor node consumes energy for sensing, signal processing and transmitting. The energy consumption of the sensor node during its active and sleep modes is assumed as follows: The power consumption by transmission is denoted by ψ t and ψ 1 represents the power required for sensing and signal processing. The power consumption due to its own timer during the sleep mode is denoted by ψ 2 . In active mode the sensor node consumes more energy than in sleep mode, due to particularly the transmission. In order to transmit r bits of data to a distance of d, the sensor node requires (α 1 + α 2 d 2 )r energy at each time step [25], where α 1 denotes the electronic energy required to transmit one bit of data and α 2 is a constant related to the radio energy. The transmission range of the sensor node S i is set at default range r i . However, it can be set at r max i (> r i ) only when the mobile sink node is not reachable by the sensor node with r i . The intuition behind this method is to prolong the life span of the sensor network by avoiding unnecessary long distance transmissions.
Moreover, the sensor nodes' measurements are linearly related to the state of the target and corrupted by white Gaussian noise. The measurement from the sensor node S i at time step k is given by: The column vector X k = χ kχk ξ kξk denotes the state of the target at time step k where χ k and ξ k represent its positions in the x and y directions,χ k andξ k represent its velocities in the x and y directions.
The observation matrix is denoted as H i . The measurement noise v i k for each sensor node is assumed to be independent of each other and v i k ∼ N (0, R i ). Here R i denotes the error covariance of the sensor node S i . In this study, we use homogeneous sensor nodes and therefore, the error covariance R i and the observation matrix H i are the same for all the sensor nodes and are known a priori.

Tracking Quality and Measurement availability
In this sub-section, we analyze the consequences of measurement losses in target tracking applications. Let X k represent the state of the target at time step k. Then the motion of the target can be modeled by a stochastic equation. In many applications, the stochastic equation is assumed linear [16,26], non linear [27] and Markov process [28,29]. In this study, we assume that the target evolves according to the linear stochastic equation impaired by white Gaussian noise and given by: where w k is the white Gaussian process noise with a known covariance Q and the intensity of the process noise is determined by the scalar quantity q. F denotes the system matrix which models the state kinematics of the target. For a nearly constant velocity model of the target, F and Q are given by [30]: The state of the target evolves after ∆t time steps. Since the state of the target and the measurements are assumed to have linear Gaussian dynamics, the Kalman filter is used to optimally estimate the state and calculate its error covariance [31]. If the initial state of the target X 0 is known with the error covariance P 0 , the estimated state of the target ξ k|k = E {X k } and its error covariance The predicted state and the predicted measurement at time step k − 1 are denoted as ξ k|k−1 and z k|k−1 respectively. The predicted error of the estimated state P k|k−1 and the Kalman gain K k at time steps k − 1 and k are given as: It can be inferred from (5) and (7) that the error covariance of the estimated state is influenced by the error covariance R of the sensor node. In [16], we studied the problem of sensor management to enhance the tracking quality, and showed that the tracking quality depends on the sequence of the activated sensor nodes where the error covariances of the sensor nodes are different. However, the tracking quality does not depend on the sequence of activated sensor nodes if the error covariances of the sensor nodes are the same and the measurements are available at each time step.  In state estimation, measurements may be absent for many reasons, such as that the target is not covered by any sensor nodes or the sensed measurement is lost. Considering the limited transmission range of each sensor node, it is possible that measurements may not be collected by the mobile sink node when it is outside the transmission range of the activated sensor node. Let E k and E lost k denote the RMSE (root mean square error) of the estimated state at time step k without and with measurement loss, respectively. Thus E k is the square root of the estimated error for availability of continuous measurements and E lost k is the square root of estimation error for availability of noncontinuous measurements. We can calculate E k and E lost k as follows:  Figure 2, and it can be seen that E k reaches to a steady state error, but not E lost k . Here, we used ∆t = 1s , q = 10, observation matrix as 4 × 4 identity matrix and the error covariance of the sensor nodes as:   The variation in RMSE and cumulative RMSE are illustrated in Figure 3. Even though the number of measurement losses are equal for case 2 and case 3, the cumulative RMSE are not the same. Furthermore, the number of measurement losses for case 1 is less than that of case 2 and case 3, but produces a higher cumulative RMSE than case 2 and case 3. This indicates that the time steps of measurement losses are more important than the number of measurement losses for target tracking. Moreover, it can be seen in Figures 2 and 3 that measurement losses creates unnecessary error in estimation resulting in degraded tracking quality. Therefore, we make sure that the measurements are not lost at any time step to avoid the additional error in estimation. We increase the transmission range of the sensor nodes to avoid measurement loss only if necessary as this has to be done with minimal energy consumption of the sensor nodes. The following section describes the sensor management strategies according to the mobile sink node positions.

Sensor Management with Mobile Sink Node
denote the position of the sensor node S i and the mobile sink node in 3-dimensional Cartesian space at time step k respectively. The assumptions and constraints made in this study are as follows: 1. The position of the target belongs to a known region Ω k at time step k and the sensor set {S 1 , S 2 , ..., S N k } which covers the region Ω k is known a priori 2. Only a single sensor node can be active mode at any time step. 3. A sensor node can not be activated during the active period t i of the previously activated sensor node S i . 4. The transmission range of the activated sensor node S i is set at r max is selected in such a way that the mobile sink node is always within the range. Thus, Our objective is to minimize the total energy consumed by the sensor nodes in order to send the measurements to the mobile sink node without loss. We define the cost function J T for the entire time horizon {1, 2, .., T} as follows: where u k denotes the sensor node in active mode at time step k and u k ∈ {S 1 , S 2 , ..., S N k }. The measurement-loss function with the default transmission range is denoted as δ M k u k and given by: This problem would be easily solved if the mobile sink node were always reachable by a single sensor node with a lower transmission range than the rest of the sensor nodes. However, it may be impossible to find such a sensor node in practical terms. Our aim is to find the optimal sequence of activated sensor nodes {u * 1 , u * 1 , ...u * T } to minimize the total accumulated cost J T from time 1 to T: Since J T is a function of the relative position of the mobile sink node, we solve this optimization problem for cases when the mobile sink node's position is known both fully and partially.

Position of the mobile sink node is known
Let {M 1 , M 2 , ..., M T } denote the sequence of the position of the mobile sink node. We assume that it is known a priori. The base station (BS) collects information about the sensor nodes, such as location and the energy availability. We solve this problem using a dynamic programming technique and rollout algorithm as follows.

a. Dynamic Programming Approach
We used deterministic backward dynamic programming (DP) to find the optimal sequence of the activated sensor nodes. DP is a recursive technique which divides the problem into a sequence of subproblems using principle of optimality. In this paper, state refers the position of the mobile sink node, stage refers the time step and decision refers the activated sensor node. Since the state M k is known at the kth stage, this problem has only a single state at each stage. The decision at time step k when the position of the mobile sink node is at M k is denoted as µ k (M k ) and the number of decisions are equal to the number of available sensor nodes.
If the decision at time step k is made to activate µ k (M k ) = u k (i.e, to activate the sensor node u k at time step k), then the period cost (please note that it is not the instantaneous cost but the period cost because once the sensor node has been activated it remains in active mode for t i time steps) from k to k + t i − 1 is given by: We can rewrite our objective function J T as: The DP proceeds backwards from stage T to stage 1 and the "cost-to-go" functions are defined as follows: where DP solves the sub-problems from k = T to 1 and stores the optimal value for J k (M k ) and µ * k (M k ). Once all the sub-problems have been solved, DP backtracks the optimal sensor nodes from the stored values. The optimal total cost is given by: The optimal sensor node to be activated at time step 1 and k are given by: where S j denotes the previously activated optimal sensor node. For this problem, the computational cost of DP only depends on the number of stages (total time horizon) and the number of admissible decisions (available feasible sensor nodes). Since the DP calculations are done off-line, it is assumed that at each time step set of available feasible sensor nodes are known a priori. In real situations, the available feasible number of sensor nodes can vary with time, for example sensor nodes may be deleted due to dead battery or malfunction. In this study, we assume that the set of available feasible sensor nodes at each time step are known and the sensor nodes work without malfunction.

b. Rollout Approach
The rollout algorithm (RA) is an approximate dynamic programming technique to overcome the curse of dimensionality in DP. The algorithm performs according to the approximate cost-to-go function given by a sub-optimal base heuristic policy. It produces a solution not worse than the solution given by the base heuristic. The computational cost of the RA depends on the problem size and the base heuristic method. In general, RA is computationally more tractable than DP. Since M k has a single value at each time step, known in our problem, the number of states at each stage is one and therefore, the computational cost of DP is not an issue. However, DP may not be used if the decision variables are varying with time. RA updates stages, states and decisions at each time step and performs the optimization. The major steps involved in RA for our problem are as follows: Step 1: Here, J 1+t i (M 1+t i ) is solved by the base heuristic method.
Step 4: If a sensor node S j is in active mode setũ * k = S j and go to Step 3 with replace k by k + 1, else go to Step 5.
Step 5: Calculate g k (S i ) + J k+t i (M k+t i ) for all ∀ S i ∈ {1, 2, .., N k } and J k+t i (M k+t i ) is solved by the base heuristic method.
Step 6: choose the sensor nodeũ * k which minimizes g k (S i ) + J k+t i (M k+t i ).
Step 7: If k > T go to Step 8 , else go to Step 3, replacing k with k + 1.
Step 8: The sub-optimal sensor node sequence is given by We use the one-step-look ahead (OSLA) [32] method as our base heuristic method for RA. Since OSLA algorithm optimizes only the cost occurred at the current time step, the results are sub-optimal and the computational cost required is comparatively very small. The OSLA algorithm produces the sub-optimal sensor sequence {ȗ * 1 ,ȗ * 2 , ...,ȗ * T } and given by: where S j denotes the previously activated optimal sensor node.

Position of the mobile sink node is partially known
In this section, we assume that the exact position of the mobile sink node M k is known only at time step k otherwise it is only partially known. In other words, M k is known ∀k ≤ c and partially known ∀k > c at current time step c. We assume that the BS immediately gets to know the current position of the mobile sink node at time step k.
The The Markov chain parameters A and π 1 are assumed to be known.

a. Stochastic Dynamic Programming Approach
Since the exact future position of the mobile sink node is unknown at the current time step, we cannot use the deterministic dynamic programming for this problem. We present the stochastic dynamic programming (SDP) technique to produce the optimal sequence of activated sensor nodes when the mobile sink node' future position is uncertain. Let g k (S i , M k ) denotes the expected period cost from k to k + t i − 1 when the position of the mobile sink node is in M k and the activated sensor node at kth time step is S i . The period cost g k (S i , M k ) is given by: The future position of mobile sink node M t is unknown ∀t > k at time step k and therefore, we take the expectation to calculate the period cost g k (S i , M k ).  For this problem M t can take any state in S-state Markov chain and therefore, SDP has S number of states at each stage. Figure 4 shows the state − stage diagram of the SDP for this sensor management problem. The SDP proceeds backwards from stage T to stage 1 and the "cost-to-go" functions are defined as follows: where SDP solves the sub-problems from k = T to 1 and stores the values for J k (M k ) and µ * k (M k ). The optimal expected total cost J T is given by: The optimal sensor node to be activated at time steps 1 and k depend on the position of the mobile sink node and is given by: where j denotes the previously activated optimal sensor node. These calculations are done at the BS off-line. At each time step, the BS updates the current position of the mobile sink node and activates the optimal sensor node using the look up table, where the SDP stores J k (M k ) and µ * k (M k ). The computational cost of SDP for this problem depends on the number of stages, number of states and number of admissible decisions. Furthermore, we assume that the number of available sensor nodes (admissible decisions) at each time step are known a priori and the sensor nodes work without malfunction.

b. One-step-look-ahead method
For a large problem (where the number of states, stages and decisions are larger), SDP may not solve the problem with-in an acceptable time period, due to its huge computational cost. Consider a scenario where the parameters required for the SDP are known just before the application (target started moving) and the problem is large enough that we may not have enough time to solve it before the application. Moreover, if the set of available sensor nodes varies with time, SDP cannot be used. We use the one-step-look-ahead (OSLA) algorithm as a sub-optimal method to solve this scheduling problem. The OSLA finds the best sensor node which minimizes only the energy consumption at the current time step for a known M k since M k is known at the current time step k. For k = 1, 2, ...T, the OSLA produces the sub-optimal sensor sequence as {ũ * 1 ,ũ * 2 , ...,ũ * T } where the sub-optimal sensor node at time step 1 and k are given by: Although OSLA algorithm produces a sub-optimal solution for the sensor management problem, it is suitable when the availability of the feasible sensor nodes are unknown a priori or when we need to solve the problem within a short time period.

Results and Discussions
In this section, we present a numerical example of single target tracking with noisy sensor nodes and the BS is located far away from the sensor field. A mobile sink node flies around the sensor field, collects the measurements from the sensor nodes and transmits them to the BS in order to prolong the life span of the sensor network. We simulate a single target moving in a 2-dimensional Cartesian space according to (3). The system matrix F and the system noise covariance Q are the same as in section 2.2.

Parameter Settings for the Sensor Network
In our simulation, we use only 3 sensor nodes to enhance the clarity and simplicity of the problem. Sensor nodes S 1 , S 2 and S 3 are deployed in a sensor field of 500 m × 500 m and for the experimental purpose, we assume that each sensor node covers the target such that the feasible sensor set has S 1 , S 2 and S 3 sensor nodes at each time step. A mobile sink node collects the measurements and immediately transmits to the BS. The properties of the sensor nodes are given in Table 1. It is assumed that the measurements are sent to the mobile sink node in a single-hop communication without any time delay. The observation matrix H is a unit matrix, and therefore the measurements are linearly related to the state of the target. The time interval between the measurements is considered as ∆t = 1s. We assumed the measurement data size as 1 MB and the data rate as 8 Mbps. Without loss of generality, we neglect the energy consumption ψ 1 and ψ 2 and only consider ψ tx . The parameters in ψ tx are set α 1 = 50 nJ/b and α 2 = 100 pJ/bm 2 . The maximum transmission range of r max i = 500m was chosen for all sensor nodes. The target starts at X 0 with the known initial error covariance P o and moves for T s.

Sensor management with known mobile sink node's path
In this section, we assume that the position of the mobile sink node is known at each time step and the mobile sink node flies horizontally at a height of 100 m above the ground. We plot the measurement-loss function δ M k i with the r i for the total time horizon of 100s, shown in Figure 5.   We compare the sub-optimal solutions obtained by the rollout algorithm (RA) and one-step-lookahead (OSLA) method with the optimal solution obtained by dynamic programming (DP). We use OSLA method as the base heuristic method for RA. We increase the total time horizon from T = 30s to 100s. The results for all three methods are shown in Table 2. The variation of total energy consumption with time is shown in Figure 6 for T = 100s. We can see from the results in Table 2 that RA always produces better results than the base heuristic method (OSLA). Since the positions of the mobile sink node are known a priori, the computational cost of DP is not large and therefore DP is best suited for this particular problem.

Sensor management with partially known mobile sink node's path
In this section, we assume that the position of the mobile sink node varies according to a known Markov chain and that the exact position of the mobile sink node is known only at the current time step but not before.  Figure 7 shows the probable positions of the mobile sink node and of the sensor nodes, and the default transmission ranges of the sensor nodes The initial probability vector of the mobile sink node's position is assumed as π 1 = [0.2 0.4 0.2 0.2]. We choose two different transition probabilities to analyze the performance of our methods as follows: The optimal expected total energy consumption of the sensor nodes with A 1 and A 2 for T = 100s are calculated off-line using stochastic dynamic programming (SDP), and are given by: Since the future positions of the mobile sink node are uncertain, the OSLA schedules the sensor nodes on-line. Therefore, we simulate the mobile sink node's path according to the known Markov parameters to obtain the results for OSLA. The results in Table 3 were averaged over 100 independently simulated paths for the mobile sink node using the known Markov parameters. The mean and standard deviation of the total energy consumption are shown in Table 3. The results in Table 3 obtained from SDP become equal to E {J T } for very many simulated paths of the mobile sink node. For example, we can see that J 100 = 141.27 with A 1 whereas E {J A 1 T } = 142.54. Figure 8 shows a simulated path for the mobile sink node using the known Markov parameters and the total energy consumption obtained by OSLA and SDP.  Since OSLA is a sub-optimal method, on average it consumes more energy than SDP. Even though the results are not promising, the computational cost of the OSLA is very low compared to that of SDP.
The number of probable positions (e a ) of the mobile sink node does not affect the computational cost of the OSLA whereas the computational cost of the SDP increases. In order to analyze the effect of the number of sensor nodes on the computational cost of the algorithms, we increase the sensor nodes from 3 to 350 whilst keeping a constant number of probable positions of the mobile sink node. It can be seen from Figure 9 that the computational cost of SDP increases as the number of sensor nodes increases, whereas no significant variation for OSLA was observed.

Conclusion
In this paper, we studied optimal and sub-optimal methods for scheduling the sensor nodes with a mobile sink node to minimize the total energy consumption for target tracking. An activated sensor node increases its transmission range to maximum only if the measurement could not reach the mobile sink node. Moreover, a sensor node cannot be activated while a previously activated sensor node remains active. We studied the sensor management problem when the path of the mobile sink node is known both fully and partially. We solved the problem optimally using deterministic and stochastic dynamic programming techniques and sub-optimally using rollout and one-step-look-ahead algorithms. The deterministic dynamic programming is best suited for the problem where the path of the mobile sink node is fully known. The limitation of dynamic programming technique is that it cannot be used to solve the problem when the available feasible number of sensor nodes are unknown a priori or varying with time. In such cases, rollout and one-step-look-ahead algorithms are useful. Furthermore, for a big problem, stochastic dynamic programming technique does not solve the problem within a feasible time period due to its high computational cost.

In page 5, last paragraph:
Original version where w k is the white Gaussian process noise with a known covariance Q and the intensity of the process noise is determined by the matrix B.

Corrected version
where w k is the white Gaussian process noise with a known covariance Q and the intensity of the process noise is determined by the scalar quantity q.
2. In page 15, Figure 5: y-axis texts are enlarged to make it more clearer.