Evaluating the FER of IEEE 802.15.4 Frames Between UAVs and Wireless Sensor Nodes ^†

Abstract

Evaluating the noise on the transmitted bits or frames and its effect on connectivity is essential in Wireless Sensor Networks when designing and deploying IoT networks. When the sensor nodes are positioned on the ground, the number of correctly received frames can fluctuate due to obstructions, interference, and the processing capacity of the nodes. Therefore, implementing strategies and systems to monitor and evaluate the rate of lost frames is required. This paper presents the implementation of a system designed to assess the rate of erroneous IEEE 802.15.4 frames within a communication setup that includes a ground control station, an unmanned aerial vehicle, and surface-based sensor nodes.

1. Introduction

The use of unmanned aerial vehicles (UAVs) in various applications has promoted their use in the Internet of Things (IoT), specifically in systems that collect data from sensor nodes located on the ground [1]. In the IoT world, the IEEE 802.15.4 protocol is widely used as a physical and link layer protocol and applied to technological solutions such as ZigBee [2], 6LoWPAN [3], WirelessHART [4], and ISA100.11a [5]. Implementing wireless solutions that utilize UAVs as part of the communications system necessitates an examination of the effects of noise and other factors on the loss of transmitted frames. Having reliable transmission systems is a challenge today. Tools that evaluate the reliability of the communication system, such as those used in transmissions employing the IEEE 802.15.4 protocol, are necessary to achieve this objective. The parameters that quantify the communications system’s reliability typically use the transmission error probability, expressed in terms of the Bit Error Rate (BER) or the Frame Error Rate (FER). Furthermore, the signal-to-noise ratio (SNR) is used to measure system performance based on the required level of BER and FER; both are functions of the coding and signal modulation scheme and frame size.

A Wireless Sensor Network (WSN) that incorporates a UAV as a transmitter–receiver element for collecting information from wireless nodes requires operating with an adequate FER due to the ground-to-air nature of the communication. The FER values with which the WSN will operate influence the optimal spectral and energy efficiency.

2. Related Work

Nowadays, creating hardware and software tools to validate the designs of communications systems between sensor nodes and UAVs is a hot topic. Specifically, using the FER to determine the reliability of wireless transmissions is essential. In [6], an analytical model is proposed to evaluate the FER for both single-hop transmissions between sensor nodes and two-hop transmissions via an intermediate relay. Model parameters follow IEEE 802.15.4 specifications for the 2.45 GHz and 868/915 MHz bands. In [7], the authors study the real-time mobile relay network based on autonomous underwater vehicles. The reliability analysis in the multihop network uses maximum values of the FER. Similarly, in [8], the FER is used to evaluate the validity of the proposed scheme in improving the reliability of the WLAN network. In [9], the FER is used to evaluate the effect of equalizing channels with multipath fading in underground mines in WSNs using ZigBee/IEEE 802.15.4. A similar approach is addressed in [10]. An improved adaptive coding and modulation scheme with a hybrid switching standard is proposed in [11] for free-space optics communications between UAVs and surface nodes. In [12], the authors use measured and simulated results to evaluate the FER in the data transmitted between vehicles. A relevant study [13] proposes a link-level routing algorithm using implicit acknowledgments to ensure end-to-end reliability in linear multi-hop WSNs. Validated with real hardware, the algorithm reduces energy consumption and latency while maintaining reliability and connectivity in large-scale deployments.

3. Proposed Solution

This paper proposes a system to evaluate the FER in wireless communications systems in real-world environments that use the IEEE 802.15.4 protocol. This work considers that, during transmissions by a UAV, sensor nodes on the surface can be in transmission, reception, frame processing, or low-energy consumption (sleep) mode. A sensor node cannot simultaneously receive and transmit frames. Therefore, it is necessary to determine when the nodes can receive frames, because if a node is in transmitter mode, it will lose the frames even if the reception signal is excellent. The same situation occurs when the node is processing the frame or in sleeping mode. For this reason, it is necessary to understand the factors that affect the operation mode of the node to achieve reliable communication systems that comply with the given FER conditions. The implemented system requires the design and implementation of the software for the ground control station (GCS), the UAV transmitter node in the air, and the receiver nodes on the ground.

3.1. Design Considerations

The proposed system is composed of (i) a GCS that consists of a controller node to coordinate the communication between the UAV and sensors and a computer to evaluate the FER, (ii) a transmitter node located in the UAV whose function is to collect data from several sensor nodes, and (iii) several sensor nodes that are located on the ground. Figure 1 presents the communications system for evaluating the FER in IEEE 802.15.4.

3.2. System Components

3.2.1. Ground Control Station

In the GCS, the controller node sends the selected configuration information on the computer to the transmitter node in the UAV. The computer sends the information to the controller node through the serial port. The information sent to the UAV includes the number of blocks of IEEE 802.15.4 frames to be transmitted to the receiving node, the time interval between blocks, the time interval between frames within each block, the frame size, and the frame number. The information is placed in the IEEE 802.15.4 frame payload for transmission. Additionally, the controller instructs the UAV on when to begin transmitting the frame blocks. The computer in the GCS features a graphical interface that enables users to enter the parameters by which the nodes will operate. The data obtained from the transmitter node in the UAV and the receiver nodes on the surface are processed to evaluate the FER. The FER is determined by comparing the number of frames transmitted with the number received. The information stored in the computer enables the comparison of the content of transmitted and received frames bit by bit.

3.2.2. Transmitter Node

With the information received from the controller node, the transmitter node located in the UAV creates frames to transmit them to the receiver nodes on the surface. The UAV sends the payload of each IEEE 802.15.4 frame, which includes information on the battery level, the transmission power level of the frame, and a sequence number, in addition to adding random data to fill the payload field with values indicated by the controller node. The node in the UAV sends a varying number of frames in each block for transmission. The parameters include the time between frames, the time between blocks of frames, and the size of the long frame (a vector of 100 positions) or short frame (a vector of 23 positions). This information is configured remotely by the controller. Additionally, the transmitting node stores the position of the UAV and the power level with which each frame was transmitted. These data are transferred to the computer after sending blocks of frames to the receiving nodes.

3.2.3. Wireless Sensor Nodes

Receive the frames sent by the UAV and store them with their respective signal power received. This information, position, and the node identifier are sent to the GCS.

3.3. Algorithms for the Operation of the Communications System

3.3.1. Algorithm for the GCS

Algorithm 1 manages the node’s state (receive or transmit) and processes data based on inputs received from the application or the transmitting node. The controller node performs the following three main tasks:

• Send its unique address (SRC_ADDR) to the application to know if the node is operating correctly, and the latter can send or receive data from the application.
• Send several frames to the transmitting node with each of the instructions (entered in the application) that the transmitting node must take into account to execute its main function of sending frames.
• Send the data from the transmitting node, previously stored, to the application.

3.3.2. Algorithm for the Transmitter Node

Algorithm 2 for the transmitter node alternates between data reception and transmission states based on the received input and the configured transmission instructions. Its main function is to transmit frames to the receiving nodes based on the instructions received previously. It is also responsible for generating and sending a sequence number to an Arduino UNO device (see Figure 1). This sequence number is unique and takes on another value each time this node sends a frame to the receiving nodes. On the other hand, Algorithm 3 describes the operation of an Arduino Uno that works in conjunction with the transmitter node. This device remains in the reception state, waiting for a sequence number to be received. Its primary function is to obtain the location of the transmitting node each time it receives a new sequence number.

3.3.3. Algorithm for the Wireless Sensor Nodes

Algorithm 4 aims at the storage of frames sent by the transmitting node for subsequent transmission to the application.

3.4. Implementation

A summary of the technical features of the wireless node and the UAV used in our proposal is presented in Table 1 and in [14], respectively.

Figure 2 shows the application that runs on the GCS, which indicates to the transmitting node the number of frames it must generate and then sends them to the receiving nodes. Additionally, the application enables the retrieval and storage of data corresponding to the receiving nodes on the micro SD card. The application works together with a node to communicate with the transmitting node. The application has been developed in PyCharm, version 2020.3.2, an integrated development environment that utilizes the Python programming language (version 3.9) and the QT Designer tool, version 5.11.1. For a detailed view of the algorithms developed and the application to evaluate the FER, readers can access the respective repositories [15].

Table 1. Features of the ATZB-256RFR2-XPRO node [16].

Features	Specifications
Communication	Half Duplex
Standard	IEEE 802.15.4—2006
Cyclic Redundancy Check	CRC 16
Acknowledgments	ACK
Frame Control	Transmission, Reception, and Retransmission
Medium Access Method	CSMA-CA
Data Rates	250 Kbps, 500 Kbps, 1 Mbps, 2 Mbps
Receiver Sensitivity	−100 dBm
Transmit Power	−17 dBm to 3.5 dBm
PLL Synthesizer	5 MHz Channel Spacing
Buffer	128 Bytes
Temperature Range	−40 °C to 125 °C
Power Consumption	1.8 V to 3.6 V (10.1 mA to 18.6 mA)
Applications	ZigBee, ISM, IEEE 802.15.4, RF4CE, SP100, WirelessHART, 6LoWPAN

Table 1. Features of the ATZB-256RFR2-XPRO node [16].

Features	Specifications
Communication	Half Duplex
Standard	IEEE 802.15.4—2006
Cyclic Redundancy Check	CRC 16
Acknowledgments	ACK
Frame Control	Transmission, Reception, and Retransmission
Medium Access Method	CSMA-CA
Data Rates	250 Kbps, 500 Kbps, 1 Mbps, 2 Mbps
Receiver Sensitivity	−100 dBm
Transmit Power	−17 dBm to 3.5 dBm
PLL Synthesizer	5 MHz Channel Spacing
Buffer	128 Bytes
Temperature Range	−40 °C to 125 °C
Power Consumption	1.8 V to 3.6 V (10.1 mA to 18.6 mA)
Applications	ZigBee, ISM, IEEE 802.15.4, RF4CE, SP100, WirelessHART, 6LoWPAN

The receiving nodes remain waiting for data until the transmitter finishes sending, and then return the data to the transmitter. The transmitter node stores the data returned by the receiver nodes in the memory card through the Arduino platform. Once the transmission is finished, all the receiving nodes must contain each of the frames they manage to receive in their memory. To recover the stored data, the UAV will be located near the receiving nodes to request the data they contain. The receiving nodes will respond to this request and send data through the wireless medium; one by one, the data will be extracted from all the nodes and then stored within the micro SD memory managed by the Arduino UNO to which the node is connected. A .CSV file is created for each node, as shown in Figure 2.

4. Tests and Results

The tests to evaluate the UAV-sensor nodes system and the software tool to analyze the FER were carried out on the main campus of the Escuela Politécnica Nacional (the university campus). Under this consideration and according to preliminary measurements, with the UAV horizontally positioned at the midpoint of the IoT node line, the FER value evaluated with the IEEE 802.15.4 standard is mainly determined by the following:

• Frame loss due to noise and interference on the channel, caused because the frequency band is shared with the IEEE 802.11 standard (APs deployed mainly in channels 1, 6, and 11, with RSSIs between −68 dBm and −55 dBm [15]).
• Even when there is no noise or interference, frames are lost because the sensor node cannot receive and process them simultaneously, as is done in the IEEE 802.11 standard. This feature sometimes prevents the frame from being received and stored. The time interval at which the node processes information varies and depends on each application. The proposed tool can adjust the frame sending frequency to determine the minimum interval between frames that the receiving node can receive and process. This function of varying the interval between frames transmitted by the node was successfully tested, obtaining different FER values.

To carry out the measurements (i.e., power received by nodes from UAV and received frames) and based on a previous study [16], we decided to locate the receiving nodes in a straight line with a separation of 15 m between each node. In addition, we decided to vary the height of the UAV, starting from 10 m above the ground and reaching a maximum height of 26 m. Before beginning the transmission, the transmitting node must wait to receive the controller’s instructions and calculate the number of frames to send. Frames are sent periodically every $tFR$ seconds until a maximum of $nFR$ frames is reached, as shown in Figure 3. To ensure the quality and stability of results, each point (i.e., sensor node and UAV height level) has been measured 50 times, considering a confidence interval of 95%. In each transmission, the UAV sends 100 frames; the number of frames received and stored depends on the distance of each receiving node. The frame loss depends on the separation distance between the UAV and the receiving nodes, with the node farthest from the UAV experiencing the highest losses. Figure 4b shows the received power by sensor nodes according to the distance and height of the UAV. The maximum distance of the receiver nodes depends on the UAV’s transmission power and the sensor nodes’ sensibility level; in our tests, the wireless node (Table 1) reached a maximum distance of 60 m.

Equation (1) allows the FER to be calculated based on the number of correctly received frames and the total number of expected frames. A frame is considered erroneous when the data encapsulated in the IEEE 802.15.4 frame fail to reach their destination due to various reasons, including errors in the header, errors in the payload, or received power levels below the node’s reception threshold, which define the frame as lost. It is an essential metric in analyzing and evaluating the performance of communication systems, especially in wireless environments where conditions can vary and signal quality can be inconsistent.

Table 2. FER of the receiving nodes with variable UAV height.

Receiver Node	UAV at 10 m	UAV at 14 m	UAV at 18 m	UAV at 22 m	UAV at 26 m
0x0003	0.2	4.6	7.2	9.8	10.6
0x0004	5.8	7.6	8.2	11.4	16.8
0x0005	16.8	18.4	18.8	19.4	20.4
0x0006	17.4	19.4	19.8	20.4	31.2
0x0007	36.8	38.4	39.8	43.8	45.2

summarizes the FER value in each measured point. In addition, the variation of received frames and the sensor nodes is depicted in Figure 4a. The values in Table 2 and Figure 4a allow for verifying the proposed solution’s correct operation. In a particular case, the values of Table 2 could be set to a certain threshold to assess the transmission quality.

5. Conclusions

The system presented in this paper has been designed to evaluate the FER in a communications system between a UAV and a sensor node that operates with IEEE 802.15.4. However, it can be used in other applications that utilize the IEEE 802.15.4 protocol by establishing specific values for transmitted and received power, as well as the threshold for FER values. The proposed system enables the measurement of transmission parameters, including received power and frames, within the UAV coverage (an ecosystem comprising a UAV and sensor nodes). Additionally, the proposed tool can be configured to ensure the minimum intervals for transmitting frame sequences, allowing the receiving node to store and process them successfully.

$$FER=\left({\displaystyle \frac{\mathrm{Frames}\mathrm{with}\mathrm{errors}}{\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{frames}}}\right)\times 100$$

(1)

Many proposals to evaluate the FER consider theoretical foundations or simulations in the design process. Furthermore, the existing hardware tools pose challenges when the transmitter node must be placed on the UAVs to perform the measurements. According to the measurements carried out, the evaluation of the FER in real-world environments can be utilized to determine the optimal location of wireless nodes and UAVs, ensuring reliable information transmission, eliminating the need for ACKs provided by IEEE 802.15.4, and reducing power consumption. In future work, we hope to analyze the transmission when the UAV is in motion. In future work, we will evaluate the UAV-WSN system in dynamic UAV scenarios and various environmental conditions, including urban areas, industrial zones, and open fields.