Next Article in Journal
Numerical Calculation Method of Target Damage Effectiveness Evaluation under Uncertain Information of Warhead Fragments
Previous Article in Journal
On the Dynamics in Decoupling Buffers in Mass Manufacturing Lines: A Stochastic Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cell-Free Massive MIMO with Energy-Efficient Downlink Operation in Industrial IoT

1
School of Information Science and Technology, Nantong University, Nantong 226019, China
2
CATARC Automotive Test Center (Ningbo) Co., Ltd., Ningbo 315100, China
*
Author to whom correspondence should be addressed.
Mathematics 2022, 10(10), 1687; https://doi.org/10.3390/math10101687
Submission received: 18 April 2022 / Revised: 11 May 2022 / Accepted: 12 May 2022 / Published: 14 May 2022

Abstract

:
Cell-free massive Multi-input Multi-output (MIMO) can offer higher spectral efficiency compared with cellular massive MIMO by providing services to users through the collaboration of distributed APs, and cell-free massive MIMO systems with distributed operations are attracting a great deal of industry attention due to their simplicity and ease of deployment. This paper aims to find an optimal solution for energy efficiency in the downlink operation in the Industrial Internet based on cell-free massive MIMO systems with distributed operations. A system model is proposed, and a theoretical analysis on energy efficiency is presented. The optimization problem of efficient downlink operation is formulated as a mixed-integer nonlinear programming (MINLP) problem, which is further decomposed into two sub-problems, i.e., maximizing the sum-rate of the downlink transmission and optimizing the total energy consumption. The two sub-problems are addressed via AP selection and power allocation, respectively. The simulation results demonstrate that our algorithms can significantly improve the energy efficiency with low computational complexity in comparison with traditional distributed cell-free massive MIMO. Even in the presence of pilot contamination, the proposed algorithms can still provide significant energy efficiency when a large number of IoTDs are connected.

1. Introduction

The Industrial Internet connects massive amounts of mobile digital devices, manufacturing machines, industrial equipment, etc. [1]. These devices, which are also referred to as Internet of Things Devices (IoTDs), include radio frequency identification (RFID) tags, zigbee/long range radio (LoRa)/narrowband Internet of Things (NB-IoT)-based sensors, etc, which constantly generate large amounts of data and signals for sensing, control, system maintenance and data analysis [2]. A more flexible and scalable communication system is required to meet the ever expanding connection demands of IoTDs to access industrial networks.
The cell-free massive Multi-input Multi-output (MIMO) system connects distributed access points (APs) through backhaul links. The central processing unit (CPU) as the core device is used for data processing and distribution and collaborating with APs to develop a user-centric network [3]. With the possibility of all APs serving all user devices simultaneously, higher reachable data rates and higher spectral efficiency can be achieved in comparison with conventional cellular networks, which makes cell-free massive MIMO a promising solution [4]. Depending on the cooperation in channel estimation and data processing between APs, cell-free massive MIMO systems can be categorized into two groups—namely, cell-free massive MIMO systems with centralized operations and cell-free massive MIMO systems with distributed operations [5].
Cell-free massive MIMO systems with centralized operations share instantaneous channel state information (CSI) between APs allowing for higher spectral efficiency than do distributed cell-free massive MIMO systems. However, it is highly resource consuming in practice. Cell-free massive MIMO systems with distributed operations share only statistical channel information between APs and are capable of performing channel estimation and precoding locally. Channel estimation and precoding vectors are not transmitted to the CPU via the backhaul network [6]. Hence, cell-free massive MIMO systems with distributed operations are more scalable, allowing for the expansion of the number of distributed APs driven by the increasing number of IoTDs in the Industrial Internet.
Energy efficiency is a key factor to be considered in the Industrial Internet [7]. One of the purposes to adopt the Industrial Internet of Things (IIoT) was to reduce resource consumption and carbon emissions of industrial systems. However, the IIoT systems, including a diversity of IoTDs with sensing, processing, communication, and computing tasks consume substantial amounts of energy, which directly affects the lifetime of the IIoT systems and lead to an increasing carbon footprint [8].
On the one hand, IIoT systems typically consist of low-power devices operated on batteries, which also constrains the continuous operations of the IIoT systems. Though optimizing the power consumption of IoTDs may effectively reduce energy consumption, it is necessary to improve the energy efficiency of the communication systems.
Being the backbone of the IIoT systems, cell-free massive MIMO systems can be the main source of energy consumption. Hence, there is still a need to balance the performance of cell-free massive MIMO systems with the energy efficiency of the Industrial Internet [9]. On the other hand, the limited pilot resources in cell-free massive MIMO systems lead to interference between users using the same pilot sequence. In particular, distributed systems only share statistical CSI, thereby, resulting in inter-user interference [10]. On the other hand, more transmit power needs to be consumed at the transmitter side to suppress the impact of interference on the system throughput and leading to lower energy efficiency.
Energy efficiency can be improved by increasing user rates through a reasonable pilot allocation or precoding schemes that minimize inter-user interference and pilot contamination [11]. Other methods commonly used to improve energy efficiency are to save power through effective power allocation algorithms [12]. Whereas a large number of APs are deployed in different geographical locations in cell-free massive MIMO systems with distributed operations, the spatial complexity makes it difficult to implement the above approaches directly, particularly when time complexity is required.
User-centric approaches were proposed to collaboratively cluster APs to effectively establish connections between distributed APs and IoTDs and to reduce inter-user interference and pilot contamination [13]. However, it is still challenging to jointly consider power allocation and AP selection to optimize energy efficiency and system throughput by reducing interference.
In this paper, we aim to address the energy efficiency problem of distributed cell-free systems in the context of the Industrial Internet where massive IoTDs are required to be served simultaneously. In practice, it is common that the number of APs outnumber the active end-devices in massive MIMO systems, and reuse of the uplink pilots is inevitable, leading to high interference.
The system under consideration adopts a user-centric AP selection approach, in which IoTDs can be served flexibly depending on the channel condition. A dynamic collaborative cluster of APs centered on IoTDs can be formed based on the channel conditions between APs and IoTDs [14]. When the channel between an IoTD and an AP is very strong, the IoTD can be served by that AP. When the channel is subject to severe interference, the IoTD will be assigned multiple APs to maintain a reasonable signal-to-interference-plus-noise ratio (SINR). We focus on reducing the downlink power consumption and decouple the problem into two sub-problems, i.e., AP selection and power allocation.
An integer programming is formulated for the optimal selection of APs, and two heuristic algorithms are proposed for power allocation.
The main contributions of this paper are three-fold and are summarized as follows:
  • We first provide theoretical analysis on cell-free massive MIMO systems with distributed operations in the context of the Industrial Internet. Based on the presented system model, we focus on the energy efficiency optimization problem of the downlink transmission by formulating an integer programming model for optimal AP selection.
  • We formulate the downlink energy efficiency optimization problem as the mixed-integer nonlinear programming (MINLP) problem and decompose it into two sub-problems, i.e., maximizing the sum-rate of the downlink transmission and optimizing the total energy consumption. The two sub-problems are addressed via AP selection and power allocation, respectively. At the end, sub-optimal solutions to the original problem are jointly obtained by AP selection and power allocation algorithms.
  • We perform extensive simulation and show that our algorithms can significantly improve the energy efficiency with low computational complexity in comparison with traditional distributed cell-free massive MIMO. Even in the presence of pilot contamination, the proposed algorithms can still provide significant energy efficiency when a large number of IoTDs are connected.
The rest of the paper is structured as follows. Section 2 presents related works. Section 3 presents the system model for cell-free massive MIMO in the Industrial Internet. Section 4 introduces the energy efficiency optimization model, and Section 5 presents the AP selection-based power allocation algorithms. Section 5 presents the numerical results, and finally, Section 6 concludes the work of this paper.

2. Related Works

The research on the energy efficiency of cell-free massive MIMO systems can be divided into energy efficiency optimization in the uplink phase and energy efficiency optimization in the downlink phase.

2.1. Uplink Pilot Allocation, Power Control and Combining Processing to Improve the Energy Efficiency

In the phase of uplink transmission, various methods to improve energy efficiency have been proposed, including pilot allocation, the user transmits power control, user scheduling and combining processing [15]. Algorithms were proposed to jointly allocate pilots and control of transmitting power of the users to improve the uplink energy efficiency of the system [16].
Since the maximum ratio combining weighting of distributed APs is done at the CPU, reserachers proposed that only the quantized weighted signals are sent back to the CPU, and a successive convex approximation based transmission power allocation algorithm was proposed to improve energy efficiency considering per-user power and backhaul capacity constraints as well as throughput requirement constraints [17]. Given the fact that the pilot resources are limited, a competitive dynamic pilot reuse algorithm was proposed to allow a pair of users to share a pilot sequence, which effectively increased the reachable sum-rate and energy efficiency [18].
As the competitive pilot allocation algorithms can only achieve optimal allocation locally, an algorithm was proposed to allocate pilots based on the Hungarian algorithm to minimize interference between users using the same pilot and was shown to be effective in improving the system throughput and fairness among users [19]. As a large number of distributed APs are deployed in a wide area, acquiring the CSI is extremely complex. A user-centric approach was proposed to enable a limited number of APs to collaborate in serving users through scheduling, thereby, reducing the backhaul overhead of shared CSI while increasing the reachable rate per user [3].
In addition, due to backhaul capacity constraints, research proposed that the distributed AP could multiply the received uplink signals by conjugating the estimated channels and then send a quantized version of this signal back to the CPU for weighted combining [20]. Given the fact that considerable radiated power interference will exist in systems with many IoTDs, a power control algorithm was proposed to improve the system energy efficiency by significantly reducing the radiated power with little impact on the SINR [9]. In addition, using low-resolution analog-to-digital converters (ADCs) at the AP can also improve the uplink energy efficiency of the system [21].

2.2. Downlink Precoding and Power Allocation to Improve Energy Efficiency

In the case of downlink of the cell-free massive MIMO systems, power amplifiers consume 50–80% of the total power [22]. Therefore, the transmitter power is an important factor when energy efficiency is considered. Most of the existing studies on the optimization of energy efficiency focus on the physical layer of the downlink through a reasonable power allocation scheme. To improve the energy efficiency of cell-free massive MIMO systems, it is crucial to obtain accurate downlink channel state information.
An energy-efficient dominant path selection algorithm was proposed in  [23], which selects only the gained information of some dominant paths to maximize energy efficiency and feeds it back to the base station. As a result, the base station can obtain relatively accurate path gain information and use it to improve energy efficiency. To solve the downlink power allocation problem in cell-free MIMO systems, a bisection-search-based algorithm was proposed in [24] to maximize energy efficiency with a guaranteed minimum data rate for the user.
An optimal power allocation scheme was proposed in [25] for cell-free massive MIMO operating at millimeter-wave frequencies to maximize global downlink energy efficiency. In [26], energy efficiency optimization of the downlinks was formulated as a mixed-integer second-order cone problem. However, the computational complexity is high, and the cost required to deploy it in the network to operate in real-time is prohibitive.
In [27], it was reported that, depending on the number or locations of active user devices in the network, dynamically switching on/off certain APs could significantly improve the energy efficiency. Furthermore, in [28], the authors proposed that APs can form dynamic collaborative clusters to reduce unnecessary connections, resulting in more efficient power allocation.
Commonly adopted methods to improve downlink system energy efficiency involve the control of the AP transmitting power and precoding of transmitting signals [29]. Precoding is widely used to suppress interference, and forced-zero precoding is used in cell-free massive MIMO systems to provide five-fold to ten-fold improvements in 95%-likely per-user throughput over small cell operations [30]. Maximum-ratio (MR) precoding has been considered for industrial IoT systems to improve energy efficiency due to its low computational complexity.
MR precoding has a spectral efficiency (SE) hardening effect when a large number of IoTDs are connected. As a result, the performance can be similar to minimum mean square error (MMSE) precoding, which is sufficient for cell-free Massive MIMO to support massive IoT connections [9].
Similar to the case of uplink, the spatial complexity associated with a large number of distributed AP deployments implements traditional downlink precoding schemes very complex [31]. To address this issue, user-centric distributed precoding schemes were proposed to reduce the implementation complexity while significantly improving the spectral efficiency [32].
Notation: Throughout this paper, boldface lowercase letters, a , denote column vectors; and boldface uppercase letters, A , denote matrices. The superscripts T , * and H of vectors and matrices denote transpose, conjugate and conjugate transpose, respectively. R denotes the set of all real numbers, R + denotes the set of all positive real numbers, C denotes the set of all complex numbers, C N × M denotes the set of complex-valued N × M matrices. We use diag ( a 1 , , a n ) for a diagonal matrix, and use I M for the M × M identity matrix. The multi-variate circularly symmetric complex Gaussian distribution with correlation matrix Φ is denoted N C ( 0 , Φ ) . The expected value of a is denoted as E { a } .

3. System Model

This paper considers a time-division duplex (TDD)-mode Industrial Internet with massive IoTDs randomly accessing the network. The system under study is modeled for the manufacturing scenario where the IoTDs are relatively static. That is, the IoTDs are most likely embedded inside or fixed on the surface of the equipment and machines in factories whose location is not frequently changed. Hence, we assume that the system is not heavily impacted by mobility.
The system model is shown in Figure 1, where L APs are uniformly deployed. Each AP is equipped with M antennas and connected to the CPU via a robust backhaul link. To prolong the lifetime of the devices, IoTDs are configured to go into hibernation when there is no data transmission. ( ϱ + 1 ) K IoTDs are randomly distributed, communicating with the APs. Where ϱ is the ratio of the number of hibernating devices to the number of active devices in the system. ϱ K is the number of hibernating devices, and K is the number of active IoTDs operating simultaneously.
Let us define a coherent block of length τ , as shown in Figure 2, and τ = τ r s + τ d l . τ r s is the time used to transmit the uplink pilot sequence, and τ d l is the time used for downlink data transmission. It should be noted that TDD mode has channel reciprocity in the coherent block. The IoTDs can estimate the current channel information by sending pilot sequences in the uplink and can decide the corresponding precoding scheme accordingly in the downlink based on the estimated information.

3.1. Channel Model

The channel modeling obeys i.i.d rayleigh correlated fading channels due to the spatial channel correlation between AP antennas. In each coherent block, the independent channel realization between the l-th AP and the k-th IoTDs is denoted as:
h k l N C 0 , Φ k l
where Φ k l = E h k l h k l H = E n = 1 N p a n a n H , Φ k l C M × M is the spatial correlation matrix characterizing the correlation between the M antennas of the AP, and N p is the number of paths. Furthermore, the antenna array response a n can be expressed as:
a n = g n 1 e j 2 π d H sin φ ¯ n e j 2 π d H ( M 1 ) sin φ ¯ n T
where g n C is the path gain, and d H R + is the MIMO array antenna spacing of the AP normalized by the signal wavelength. φ ¯ can be modeled as φ ¯ = φ + δ φ , where φ is the nominal angle between the AP antenna and the IoTDs’ antenna, and δ φ is the Gaussian-distributed random deviation of the nominal angle, e.g., δ φ N C 0 , σ φ 2 . The correlation of the rayleigh fading channel is related to σ φ 2 . The channel has a strong correlation for small values of this parameter, which is also known as standard angular deviation (ASD). t r ( Φ k l ) / M is the normalized trace of the spatial correlation matrix representing the path loss and shadowing.

3.2. Pilot Transmission and Channel Estimation

Suppose that the IoTDs in the cell free massive MIMO system share τ r s orthogonal pilot sequences ϕ 1 , , ϕ τ r s C τ r s × 1 , and ϕ t 2 = τ r s , t = 1 , , τ r s in the system the number of IoTDs K τ r s , we need to reuse the pilot sequences and thus lead to interference between IoTDs using the same pilot sequence. Then, the signal Y l p i l o t C N × τ r s received at AP l for the pilot sequence can be given as:
Y l p i l o t = i = 1 K p i h i l ϕ t i T + N l
where p i > 0 is the pilot sequence transmit power of i t h IoTD. N l C N × τ r s is the cyclic symmetric complex Gaussian noise with independent N C ( 0 , σ u l 2 ) , and σ u l 2 is the noise power of uplink. To extract the channel estimation information for the k-th IoTD using the pilot sequence ϕ t , we multiply the pilot signal Y l p i l o t received at the l-th AP by the normalized pilot sequence ϕ t / τ r s . Then, the procedure of processing the received signal by AP l using the orthogonal property between the pilot sequence can be expressed as:
y t k l p i l o t = i = 1 K p i τ r s h i l ϕ t i T ϕ t k * + 1 τ r s N l ϕ t k * = i P k p i τ r s h i l + n t k l
where n t k l N l ϕ t k * / τ r s N C 0 , σ u l 2 I M is the resulting noise, and P k { 1 , , K } is the subset of IoTDs using the same pilot sequence. Equation (4) can be easily applied to estimate h k l following the MMSE estimation method [33]:
h ^ k l = p k τ r s Φ k l i P k p k τ r s Φ i l + σ u l 2 I M 1 y t k l p i l o t ,
The estimation error h ˜ k l = h k l h ^ k l has correlation matrix C k l , which is given by
C k l = E h ˜ k l h ˜ k l H = Φ k l p k τ r s Φ k l i P k p k τ r s Φ i l + σ u l 2 I M 1 Φ k l ,
It is worth mentioning that due to the nature of Markov chains, the independence between h ^ k l and h ˜ k l leads to E [ h ^ k l h ˜ k l H ] = 0 .

3.3. Downlink Data Transmission

The reciprocity of the channels in TDD mode allows us to obtain h k l = h k l H . h k l estimated by the pilot can be used in the local precoding in the downlink transmission, including local minimum mean-square error (L-MMSE) precoding and MR precoding [6]. Distributed cell-free massive MIMO system offers the opportunity of using all APs to serve all IoTDs, which can achieve higher spectral efficiency than can cellular networks. To improve the energy efficiency of the system, we use a binary parameter s k l { 0 , 1 } and
s k l = 1 , if k -th IoTD select l -th AP , 0 , otherwise .
The CPU sends the encoded data { ς k , k 1 , , K } to the set of APs that serve the IoTD k, and the AP runs data precoding locally. On the AP side, the signal transmitted by the l-th AP is denoted as:
x l = k = 1 K s l ρ k l w l ς k
where w l = w 1 l , , w K l is the local precoding vector based on channel estimation, and S l is the set of IoTDs served by l-th AP. s l = s 1 l , , s K l is the scheduler vector of IoTDs served by l-th AP. ρ l = ρ 1 l , , ρ K l is the power allocation vector of IoTDs served by l-th AP. As a result, the signal received at k-th IoTD is:
y k dl = l = 1 L h k l H x l + n k = l = 1 L ρ k l h k l H s k l w k l ς k + i = 1 i k K l = 1 L ρ i l h k l H s i l w i l ς i + n k
In Equation (9), the first term on the right side represents the signal expected by k-th IoTDs, and the second term represents the sum of the interference caused by the signals sent by other APs to other IoTDs. n k represents the noise in the downlink channel. Using Equation (9), we can calculate the reachable rate of IoTD k as:
γ k = B τ d l τ log 2 ( 1 + DS k ( s k , ρ k ) MUI k ( s k , ρ k ) DS k ( s k , ρ k ) + σ d l 2 )
DS k ( s k , ρ k ) = l = 1 L E ρ k l h k l H s k l w k l 2
MUI k ( s k , ρ k ) = i = 1 K E l = 1 L ρ i l h k l H s i l w i l 2
where DS k ( s k , ρ k ) is the desired signal power received by the IoTDs k. ( MUI k ( s k , ρ k ) DS k ( s k , ρ k ) ) is the multiuser interference signal power received by the IoTDs k, σ d l is the additive noise of the downlink channel, B is the signal transmission bandwidth. The sum-rate of all IoTDs in the system can be calculated as:
R ( S , P ) = k = 1 K γ k
Due to a large number of APs deployed in cell-free massive MIMO systems, which increase the spatial complexity, traditional precoding methods are difficult to implement at a tolerable cost, so MR precoding and L-MMSE precoding methods are considered in this paper to reduce the implementation cost.
The local precoding vector of the k-th IoTD designed at the l-th AP can be expressed as: w k l = w ¯ k l E w ¯ k l 2 . Where, w ¯ k l C M × M is an arbitrarily scaled vector pointing to the direction of the local precoding vector. Note that the normalization makes E w k l 2 = 1 . Hence, the MR precoding and L-MMSE precoding used in this paper can be expressed as w ¯ k l MR and w ¯ k l L - MMSE , respectively.
w ¯ k l MR = s k l h ^ k l
w ¯ k l L - MMSE = p k i = 1 K p i h ^ i l h ^ i l H + C i l + σ ul 2 I N 1 s k l h ^ k l
It is noteworthy that in a distributed cell-free massive MIMO system, the instantaneous CSI is not shared between APs. Only data from scheduled IoTDs can be precoded, e.g., s l h k l ^ . Thus, L-MMSE will bring less channel gain compared to MMSE.

4. Downlink Energy-Efficiency Optimization

The energy consumption of a distributed cell-free massive MIMO system in the context of the Industrial Internet is mainly composed of fixed power consumption and effective transmitted power (ETP) as shown in Equation (16).
P Total = P FIX + ETP
The fixed energy consumption, denoted as P FIX , is the power to keep the CPU running and the APs working, which is closely related to the number of APs and the number of antennas per AP; the effective transmitted power is the power used for signal transmission during the operation of the power amplifiers of both APs and IoTDs, whereas the amplification efficiency of power amplifiers are imperfect due to impairment caused by the manufacturing process, aging of the devices, etc.
Moreover, thermal wastage can be generated in the process of the signal power amplification, which can impact the amplification efficiency. Without loss of generality, we assume that the amplification efficiency of the power amplifiers under consideration is imperfect to model energy efficiency in realistic manufacturing scenarios.
Let μ AP and μ IoTD denote the amplification efficiency of the amplifiers of the APs and IoTDs, respectively. Then, ETP is expressed as:
ETP = τ r s τ k = 1 K p k μ IoTD + τ d l τ M l = 1 L k = 1 K s k l ρ k l μ AP
Energy efficiency (EE) is defined as the ratio of the sum of users’ data rates in the system to the total energy consumption [34]. According to Equations (13) and (16), the EE of the system under consideration can be represented as:
EE = R ( S , P ) P Total ( S , P ) = k = 1 K B τ d l τ log 2 ( 1 + DS k ( s k , ρ k ) MUI k ( s k , ρ k ) DS k ( s k , ρ k ) + σ d l 2 ) P FIX + τ r s τ k = 1 K p k μ IoTD + τ d l τ M l = 1 L k = 1 K s k l ρ k l μ AP
where S = [ s 1 , , s K ] is the scheduling matrix and P = [ ρ 1 , , ρ K ] is the power allocation matrix, respectively. As illustrated in Equation (18), S and P are directly related to the sum rates of the devices and energy loss, which are considered as the key parameters in optimizing energy efficiency of the system.

4.1. Problem Formulation

The objective of the optimization problem is to maximize the energy efficiency as formulated in Equation (19):
P : max S , P EE ( S , P )
s . t . C 1 : k = 1 K s k l ρ k l P max l , k ,
C 2 : ρ k l 0 l , k ,
C 3 : l = 1 L s k l R k R min s k l = 1 , l , k ,
C 4 : k = 1 K s k l ξ τ r s ξ { 0 , 1 } , l ,
C1 and C2 are power-consumption constraints, and C3 defines the downlink transmission rate constraints to guarantee normal communication services for IoTDs. In Industrial Internet scenarios, all APs should be able to run in a low-power mode with a minimum transmit power to maintain all devices in operation. That is, the system can not be operated normally if the transmit power is more than this value, which is defined as the threshold, denoted as P max .
The power allocation constraint is defined in Equation (20), that is, the transmit power k = 1 K s k l ρ k l allocated at AP l is less than the transmit power threshold. The transmit power between AP l and the IoTD k being scheduled should be a non-zero positive value and the power allocated by the AP for the unscheduled IoTDs is set to zero. As shown in Equation (21), transmit power ( ρ k l > 0 ) is allocated when IoTD k is scheduled by AP l.
To maintain the system, the minimum downlink transmission rate has to be guaranteed for each activated IoTD. As a result, the data rate of the k-th IoTD should be greater than the minimum QoS, denoted as R min . As defined in Equation (22), s k l R k is the effective rate provided to IoTDs k by AP l. This constraint is defined to ensure the minimum downlink transmission rate guaranteed when IoTD k is scheduled by AP l.
C4 specifies the scheduling constraint, which denotes that there exists a limit on the number of IoTDs scheduled by an AP. The distributed cell-free massive MIMO share only Statistics CSI among APs. Without loss of generality, we assume that the number of schedulable IoTDs k = 1 K s k l per AP is less than the number of uplink pilots τ r s , and the scaling factor ξ ( 0 , 1 ) is set to explore the impact of the number of IoTDs that can be scheduled from the AP on energy efficiency. otherwise, IoTDs with weaker channel conditions sharing the pilot frequencies will suffer strong interference. Based on the assumption, the scheduling constraint is defined as in Equation (23).
The problem P in Equation (19) is a mixed- integer nonlinear programming (MINLP) problem, which cannot be solved in polynomial time. In order to obtain the optimal solution, we hereby decompose the optimization problem into two sub-problems, i.e., maximize the sum-rate of the downlink transmission ( R ( S , P ) ) and optimize the total energy consumption ( P Total ). The two sub-problems are addressed via AP selection and power allocation, respectively. In the end, sub-optimal solutions to the original problem P are obtained by jointly AP selection and power allocation algorithms.

4.2. AP Selection

In this section, we optimize the system downlink rate through optimal AP selection. We observe that the sum channel gain of the downlink affects the transmission loss of the signal in the channel. The maximization of channel gain allows the optimization of the system downlink rate.
Proposition 1. 
We assume that there is no pilot contamination in a cell-free system and that perfect channel information is known at each AP and each IoTD, i.e., h ^ k l = h k l , and then we have:
max R max l = 1 L k = 1 K β k l
that is, the problem of maximizing channel gain is equivalent to the problem of maximizing the rate.
Proof of Proposition 1. 
The proof is given in Appendix A.     □
All IoTDs are randomly distributed in a cell-free massive MIMO system. Each AP supports multiple IoTDs accesses, while each IoTD can also be served by multiple APs. Therefore, subject to the constraint of C4 as shown in Equation (23), we construct the AP-selection problem based on the criterion of sum channel gain maximization as shown below:
max l = 1 L k = 1 K β k l s k l , s . t . C 4
where the binary variable s k l is used to select the l-th AP service for the k-th IoTD as shown in Equation (7), and β k l is the channel gain between the k-th IoTD and the l-th AP.
To find an optimal solution for the AP-selection problem shown in (25), we propose a mechanism based on the well-known Kuhn–Munkres (KM) algorithm, which is usually used to achieve the maximum weights matching for two sets with the same number of set elements.
The problem is solved by introducing virtual APs and IoTDs, so that the KM algorithm can be applied to it. For example, if an AP serves 10 IoTDs simultaneously, the AP is extended into 10 virtual APs so that one-to-one mapping can be realized. The available resources of the AP are shared by all virtual APs. Similarly, IoTDs can be scaled to a collection of virtual IoTDs to realize one-to-one mapping.
Let us denote the set of virtual APs and the set of virtual IoTDs as L = { AP 1 , , AP L × ξ τ r s } and K = { IoTD 1 , , IoTD L × ξ τ r s } , respectively. The system is considered as a weighted bipartite graph G = ( V , E , W ) , where V can be divided into two disjoint sets L and K , each edge e ( k , l ) E connects with a vertex AP l L and a vertex IoTD k K . We assign weights to each edge by means of a known channel gain coefficient β k l at each AP, that is, the weight of an edge e ( k , l ) is ω ( k , l ) = β k l W .
A tag t is assigned to each vertex in the graph G, which will be used for the unique match later. An edge is considered as feasible when the tags of two corresponding vertices satisfy the following conditions t ( IoTD k ) + t ( AP l ) ω ( k , l ) ,   AP l L ,   IoTD k K . We can then determine the maximum value of the feasible vertex tag obtained if and only if:
t ( IoTD k ) = max ( ω ( k , l ) ) , for IoTD k K t ( AP l ) = 0 , for AP l L
Let G l denote the subgraph of G consisting of connected edges. We use S to store the binary variable s k l that traverses the connections between all IoTDs and all APs. A subgraph G l becomes an equivalent subgraph of G when the connected edges between APs and IoTDs satisfy the condition t ( IoTD k ) + t ( AP l v ) = ω ( k , l ) .
Theorem 1. 
If S * is a perfect match for G l , then S * is a maximum weight match for G [35].
Proof of Theorem 1. 
The proof is given in Appendix B.     □
The procedure steps utilizing the KM algorithm to determine the optimal AP selection subproblem are summarised as follows.
Step 1. Initialize a feasible tag t ( v ) with arbitrary vertices v V , select t ( v ) and G l from G to find arbitrary maximum matching S .
Step 2. If S is a perfect match for G l , output S * = S as the optimal solution to the AP selection subproblem. Otherwise, if there is an unmatched vertex u, put u in F. T represents a set where matched vertices will be placed.
Step 3. Let J l ( F ) denote the set that associates F with the vertices in G l . if J l ( F ) = T , update the tag by (28) (force J l ( P ) T ).
α t = min AP l F , IoTD k T { t ( IoTD k ) + t ( AP l ) ω ( k , l ) }
where new tag t ^ ( v ) is defined as:
t ^ ( v ) = t ( v ) α t , v s . S t ( v ) + α t , v s . T t ( v ) , otherwise
Step 4, if J l ( F ) T , select IoTD k J l ( F ) T :
  • if IoTD k does not match, take u IoTD k as the augmented path, update S and go back to step 2;
  • if IoTD k has matched, for example with AP i , extend the alternate path: F = F AP i , T = T IoTD k , and go back to step 3.
By iterating the KM algorithm above, we obtain a novel optimal matching solution S * = [ s 1 * , , s k * ] for G that represents the optimal solution to the AP selection subproblem. The integer variables of the objective function have been solved optimally for problem P as shown in Equation (29).
EE ( S * , P ) = k = 1 K B τ d l τ log 2 ( 1 + l = 1 L E ρ k l h k l H s k l * w k l 2 i = 1 K E l = 1 L ρ i l h k l H s i l * w i l 2 l = 1 L E ρ k l h k l H s k l * w k l 2 + σ d l 2 ) P FIX + τ r s τ k = 1 K p k μ IoTD + τ d l τ M l = 1 L k = 1 K s k l * ρ k l μ AP

4.3. Power Allocation

In the problem formulation stage, we decouple the MINLP problem P into two sub-problems, and then obtain the optimal downlink rate via optimizing the AP selection sub-problem based on the KM algorithm. After the optimal selection of APs, we aim to minimize the energy consumption of the system through a power allocation scheme as presented in this section.
After obtaining the optimal solution for the integer variables, the problem of minimizing the energy consumption of the system can be modeled as:
P 1 : min P P Total ( S * , P ) ,
s . t . C 1 : k = 1 K s k l * ρ k l P max l , k ,
C 2 : ρ k l 0 l , k ,
C 3 : l = 1 L s k l * R k R min s k l * = 1 , l , k ,
The constraint C 3 of P 1 is in non-convex fractional form, which is difficult to solve with a tolerable time complexity using common methods, such as the Lagrangian dual method. We hereby propose two heuristic algorithms for power allocation, i.e., KM-based AP selection and Full Power Allocation (KM-FPA) and KM-based AP selection and Partial Power Allocation (KM-FPA), respectively.

4.3.1. Full Power Allocation

In the case of full power allocation, an AP allocates power to IoTDs until the maximum transmission power is allocated. Based on the heuristic fractional power control algorithm widely used in the literature, the power allocated by the l-th access point to the k-th IoT device is defined as [18]:
ρ k l = P max β k l l = 1 L s k l * β k l , l , k .
where s k l * S * , l { 1 , , L } denotes the AP selected by the k-th IoTD. l = 1 L s k l * β k l is the effective channel gain of the set of APs serving the k-th user. When the element s k l * is 1, the channel gain between the l-th AP and the k-th IoTD is counted as the effective channel gain; otherwise, it is not counted. The sum effective channel gain of the set of APs serving the k-th IoTDs is defined as the power allocation coefficient. We assign downlink power to the AP based on the proportion of the effective channel gain between the l-th AP and the k-IoTD to the sum effective channel gain.
As shown in Algorithm 1, the method of full power allocation based on the results of AP selection ( KM-FPA) assigns more power to better channels, which can effectively increase the downlink rate of the IoTD. As a result, the energy efficiency can be improved versus conventional solutions. It should be noted that the full power allocation algorithm allocates all the power to the IoTDs at once, and the low computational complexity algorithm without iteration will provide better low latency performance; however, there is no guarantee that each IoTD can achieve a downlink rate more than the minimum QoS. As assigning higher power to one IoTD can provide a higher data rate, while also causing strong interference to other IoTDs.
Algorithm 1 KM-based AP selection and Full Power Allocation (KM-FPA)
Input: 
Channel gain β k l , maximum transmitting power P max of the AP
Output: 
EE * ( S * , P * )
  1:
Initialization: weights of the edges between APs and IoTDs is β k l , tags in APs sets t ( AP l ) = 0 , tags in IoTDs sets t ( IoTD k ) = arg max ( ω ( k , l ) ) .
  2:
calculate the perfect match S * by (25).
  3:
forl = 1:L do
  4:
      for k = 1:K do
  5:
          calculate ρ k l by Equation (34).
  6:
      end for
  7:
end for
  8:
calculate R * ( S * , P * ) by Equation (13).
  9:
calculate P Total * ( S * , P * ) by Equation (16).
10:
calculate EE * ( S * , P * ) = R * ( S * , P * ) P Total * ( S * , P * ) .

4.3.2. Partial Power Allocation

KM-based AP selection and partial power allocation are shown in Algorithm 2. Contrary to the case of full power allocation, Algorithm 2 allocates the power based on the minimum QoS requirement on the connection (It is defined that a connection is established between an AP and an IoTD when the AP serves the IoTD) between an AP and an IoTD. Without pre-selection of APs, there are L × K connections between APs and IoTDs in a distributed cell-free massive MIMO system. By the proposed AP selection mechanism, the number of connections can be reduced to L × ( K τ r s ) ,   ( τ r s < K ) .
We introduce a new parameter—namely, the initialized power allocation factor, which is defined as κ = K L × L × ( K τ r s ) L × K ,   κ ( 0 , 1 ) , where L × K is the number of connections, and K L is the ratio of the number of IoTDs to the number of APs. The power allocation coefficient between the l-th AP and the k-th IoTDs here is modified as:
ρ k l = κ P max β k l l = 1 L s k * β k l , l , k .
The algorithm iterates when the rate on a connection is less than R min and the allocated power is not greater than the maximum transmit power of the AP. A power scaling factor ϑ ( 0 , 1 ) is introduced to Equation (36) to update the power allocation coefficient ρ k l ϵ + 1 . The algorithm iterates by update Equations (36)–(38) and returns a solution or breaks when there is no solution and it exceeds the predefined iteration boundary.
ρ k l ϵ + 1 = κ ϑ P max β k l l = 1 L s k * β k l , l , k .
R ϵ + 1 = k = 1 K B τ d l τ log 2 ( 1 + DS k ( s k * , ρ k ) MUI k ( s k * , ρ k ) DS k ( s k * , ρ k ) + σ d l 2 )
P ϵ + 1 = P F I X + τ r s τ k = 1 K η k l μ IoTD + τ d l τ l = 1 L k = 1 K s k l * ρ k l ϵ + 1 μ AP
Algorithm 2 KM-based AP selection and Partial Power Allocation (KM-PPA)
Input: 
Channel gain β k l , maximum transmitting power P max of the AP, iteration thresholds ϵ 1 , power allocation initialization factor κ ( 0 , 1 ] , power scaling factor ϑ ( 0 , 1 ]
Output: 
EE * ( S , P )
  1:
Initialization: weights of the edges between APs and IoTDs is ω ( k , l ) = β k l , tags in APs sets t ( AP l ) = 0 , tags in IoTDs sets t ( IoTD k ) = arg max ( ω ( k , l ) ) .
  2:
calculate the perfect match S *  by (25).
  3:
calculate R = ( R 1 , , R k ) by Equation (13).
  4:
if ( R k = min ( R ) < R min ) & ( k = 1 K ρ k l < P max ) then
  5:
    while  ϵ < ϵ 1  do
  6:
         for l = 1:L do
  7:
               for k = 1:K do
  8:
                     update ρ k l ϵ + 1 by Equation (36).
  9:
               end for
10:
        end for
11:
        update R ϵ + 1 by Equation (37).
12:
        update P ϵ + 1 by Equation (38).
13:
         ϵ = ϵ + 1 .
14:
    end while
15:
     P Total * ( S * , P * ) = P Total ϵ .
16:
     R * ( S * , P * ) = R ϵ .
17:
end if
18:
calculate EE * ( S , P ) = R * ( S * , P * ) P Total * ( S * , P * ) .

4.3.3. Time Complexity Analysis

In the worst case, the time complexity of the KM algorithm is O ( L K 2 ) , and the expanded set and virtual elements are L × τ r s , so the time complexity is O ( ( L × τ r s ) 3 ) . In the case of full power allocation, there is no iteration required. Therefore, Algorithm 1 has a time complexity of O ( ( L × τ r s ) 3 ) . In the case of partial power allocation, the worst-case scenario is that the maximum number of iterations ϵ 1 , the total number of iterations is O ( ϵ 1 ) . K IoTDs need to be computed K times, and thus the iterative process has an algorithmic complexity of O ( ϵ 1 K ) . Therefore, the time complexity of Algorithm 2 is O ( ( L × τ r s ) 3 + ϵ 1 K ) .

5. Performance Evaluation

5.1. Simulation Parameters

This section presents the performance evaluation of the solutions obtained by jointly addressing two sub-problems, which are compared with a benchmark algorithm. The key parameters of the cell-free massive MIMO system setup are given in Table 1. The total coverage area is 1 × 1 km. A large network with L = 200 APs is simulated, where K [ 10 , 200 ] active IoTDs receiving interference from all directions.
We assume that the APs are deployed uniformly and randomly over the coverage area unless otherwise stated. The communication occurs over a bandwidth of 20 MHz, and the total received noise power of the APs and IoTDs is −94 dBm. The uplink transmit power per IoTD is 100 mW, while the maximum downlink transmit power P max per AP is 200 mW. This difference indicates that the APs are connected to the grid and therefore less power constrained. The power ρ k l allocated by access point l to IoT device k satisfies k = 1 K ρ k l 200 mW .
During the execution of the algorithm, the scheduling scaling factor ξ = 0.5 . ϑ = 0.95 is the power scaling factor used to iteratively update the power. Each coherence block consists of τ = 200 samples. In order to ensure the accuracy of the data and the robustness of the algorithm, all results are averaged over 1000 Monte Carlo simulations in this paper.
We follow the 3GPP Urban Microcell model that is defined in [36] in our simulations and calculate the large scale fading coefficient (channel gain) in dB:
Y k l [ dB ] = Y 10 α log 10 d k l 1 km + F k l
where d k l [km] is the three-dimensional distance between l-th AP and k-th IoTD, taking the height difference into account, where Y represents the median channel gain of the signal propagating over a reference distance of 1 km. The value of α determines how fast the signal fades with distance as it propagates through the spatial channel, with larger values indicating higher fading rates, also known as the path loss exponent. Moreover, F k l N 0 , 4 2 is the shadow fading. The shadowing terms from an AP to different IoTDs are correlated as [36]:
E F k l F i j = 4 2 2 δ k i / 9 m l = j 0 l j
where δ k i is the distance between IoTD k and IoTD i. The second line in Equation (40) illustrates the correlation of the shading term associated with two different APs, which is assumed to be zero.

5.2. Baseline Algorithm and Proposed Algorithms

We propose two KM-based AP selection and resource allocation algorithms to improve the energy efficiency of cell-free massive MIMO systems with distributed operations for the Industrial Internet. We choose the AP selection and power allocation schemes for a conventional cell-free network as a benchmark and then compare the performance of the two proposed algorithms with the baseline algorithm.
The above algorithms use the MMSE estimator to estimate the uplink channel and design the precoding scheme. To compare the impact of different precoding schemes on the energy efficiency of the system, this paper uses the well-known MR precoding scheme with low computational complexity and the L-MMSE precoding scheme, which can achieve optimal performance. Based on the foregoing, the differences between the baseline and proposed algorithms are summarised in Table 2.

5.2.1. Baseline Algorithm: Traditional Cell-Free Massive MIMO Systems with Distributed Operation (CF)

The baseline algorithm does not perform AP selection in that all APs serve all IoTDs. An equal power allocation (EPA) scheme is used at the APs—that is, all IoTDs are allocated the same downlink signal power by the APs.

5.2.2. Proposed Algorithm: AP Selection-Based Full Power Allocation Algorithm (KM-FPA)

We use the KM algorithm to obtain the optimal solution for AP selection and perform full power allocation (FPA) based on the result of AP selection, the algorithm is executed in the steps shown in Algorithm 1.

5.2.3. Proposed Algorithm: AP Selection-Based Partial Power Allocation Algorithm (KM-PPA)

The KM algorithm is used to obtain the optimal solution for the AP selection, and a partial power allocation (PPA) is performed based on the results of the AP selection. The algorithm is executed in the steps shown in Algorithm 2.
In this research, the proposed heuristic algorithm is based on the classic algorithm, namely, fractional power control, which is widely used in the literature to remove far and near effects in communication networks [37]. It has been shown to be prone to local optimality.
Hence, it is not suitable for solving multi-objective optimization problems. Performing a high-dimensional solution space search to solve a multi-objective optimization problem consumes a large number of computational resources, which is hard to resolve in a reasonable time. To address such a multi-objective optimization problem, a metaheuristic algorithm would be ideal due to the capability of global optimal point search [38], which will be considered in our future work.

5.3. Simulation Results and Discussion

5.3.1. Influence of the Number of Pilots on Energy Efficiency

In this section, we simulate and analyze the energy efficiency and the sum-rate performance of downlinks for a cell-free massive MIMO system with the CF algorithm versus a cell-free massive MIMO system with the KM-FPA and KM-PPA algorithms. In Figure 3, we set the number of APs L = 200, the number of AP antennas M = 1 and M = 4, the number of IoTDs with a single antenna K = 50, and the coherent block length τ = 200.
We observe that the KM-PA algorithm can provide higher energy efficiency when the number of pilots is smaller. Whether the AP is equipped with a single antenna or multiple antennas, the KM-PA algorithm shows improvement in the sum-rate of IoTDs and energy efficiency. The reason for this gain is that we limit the number of IoTDs that can be scheduled by the AP up to the number of pilots.
The gain provided by the KM-PA algorithm decreases as the number of pilots becomes higher but is still higher than that of the CF. As the number of pilots decreases, the cell-free massive MIMO systems with distributed operations trend towards the small-cell system. A small-cell system was also shown in [20] to have better spectral efficiency compared with the cell-free massive MIMO systems with the distributed operations.
In Table 3, for M = 1, the KM-PPA algorithm provides almost the same sum-rate as the KM-FPA algorithm with less power, thus, obtaining a higher energy efficiency, and the KM-PPA algorithm reduces the power allocated to the effective connections as the interference is reduced, hence, maintaining similar system performance to the full power allocation scheme. The sum-rate of IoTDs shows the same trend as the energy efficiency because the number of IoTDs is higher. Even if AP selection is performed, all the APs still need to be activated to serve the IoTDs.
In Figure 4, given the number of IoTDs K = 40, compared to the CF algorithm, the KM-PPA algorithm provides at most 90% energy efficiency gain in a single-antenna scenario and at most 4% energy efficiency gain in a multi-antenna scenario. Whereas, the KM-FPA algorithm provides at most 95% energy efficiency gain with L-MMSE precoding in a single-antenna scenario and at most 4% energy efficiency gain in a multi-antenna scenario.
As shown in Figure 4 and Table 3, the KM-PPA and KM-FPA algorithms outperform the CF algorithm in terms of energy efficiency when the pilots are highly reused. This is due to the fact that the level of interference in the network can be limited through AP selection in the KM-PPA and KM-FPA algorithms. In comparison, the CF algorithm does not perform AP selection. As a result, the devices can be connected to APs with weak channel conditions, leading to degrading performance.
Interestingly, the KM-PA algorithm provides higher energy efficiency in the case of pilot reuse, both for single-antenna cell-free massive MIMO systems and multi-antenna cell-free massive MIMO systems. Thus, even though in this paper we consider serving unmovable IoTDs, the KM-PA algorithm is equally suitable for high-speed mobile scenarios with fewer coherent block resources.

5.3.2. Impact of the Number of IoTDs on Energy Efficiency

Based on the conclusions of the previous subsection, we set the length of the pilot sequence available in the system as less than the number of IoTDs, i.e., τ r s < K , thus, IoTDs must reuse the pilot and cause interference between IoTDs using the same pilot. We set the number of APs L = 200, the number of AP antennas M = 1 and M = 4, the minimum rate of the IoTD R min = 0.1 Mbps and R min = 0.2 Mbps, and the coherent block length τ = 200.
In Figure 5, the KM-PA algorithm boosts energy efficiency under the L-MMSE precoding scheme more than under the MR precoding scheme when the number of IoTDs increases. In a distributed cell-free massive MIMO system with single antenna APs, the peak of the energy efficiency of the KM-PA algorithm occurs at K = 100, as shown in Figure 5, which indicates that the optimal number of served IoTDs for a distributed cell-free massive MIMO system with single-antenna APs, as measured by energy efficiency, is approximately half the number of APs.
In Figure 6, when the number of APs L = 200 and the number of IoTDs used K = 180, the minimum rate R min = 0.2 Mbps, where the KM-PPA algorithm guarantees the minimum rate for users and still has a higher gain on energy efficiency than CF and KM-PPA algorithms. The KM-PPA algorithm prioritizes the minimum rate of IoTDs greater than 0.2 Mbps, while the interference introduced by the iterations of power allocation causes and performance of sum-rate of IoTDs degradation as shown in the circled section in Figure 5.
The energy efficiency performance degradation of the KM-PPA algorithm using the MR precoding scheme occurs at M = 80 because the MR precoding scheme is less resistant to interference than the L-MMSE precoding scheme, The KM-PPA algorithm using the L-MMSE precoding scheme has performance loss when K = 120, compared to the KM-PPA algorithm with a minimum rate of 0.1 Mbps.
In Figure 7, the energy efficiency increases with the number of IoTDs because cell-free massive MIMO with multiple antenna APs can provide more antenna freedom and thus can serve more IoTDs than the number of APs. Comparing Figure 5 and Figure 7, the APs equipped with multiple antennas can interfere better all the time and thereby provide better energy efficiency performance.
The KM-FPA algorithm does not require iteration and can provide better low latency performance compared to KM-PPA. Considering industrial scenarios requiring low latency, the KM-PPA algorithm provides at least 39% energy efficiency gain under single-antenna APs and at least 9.7% energy efficiency gain under multi-antenna APs with low computational complexity. In industrial scenarios where the latency tolerance is high, the KM-PPA algorithm can provide energy efficiency gains of at least 87% under single antenna APs and at least 47% under multi-antenna APs.
In Figure 8, the KM-PPA algorithm tended to activate fewer APs to perform more efficient services for the IoTD as seen in Figure 4 and Figure 9 where the KM-PPA algorithm provided higher energy efficiency with less power than the KM-FPA algorithm. This conclusion is demonstrated from another perspective in Table 4.
When the AP is equipped with a single antenna, the KM-PPA algorithm with the L-MMSE precoding scheme loses less than 10% of the sum-rate gain compared to the KM-FPA algorithm, and the sum-rate gain loss is below 2% with the local MR precoding scheme. When the AP is equipped with multiple antennas, the KM-PPA algorithm with the L-MMSE precoding scheme loses within 1% of the sum-rate gain compared to the KM-FPA algorithm, as well as lower than 0.1% of the sum-rate gain under the local MR precoding scheme.
In Figure 8, the CF algorithm keeps all APs running at all times, whereas the KM-PPA and KM-FPA algorithms only select a small number of APs with strong channels to serve IoTDs when the number of IoTDs is small. As shown in Figure 9, compared with the CF algorithm, the KM-PPA and KM-FPA algorithms, maintains fewer APs running, thus, saving energy consumption. The KM-PPA algorithm is more energy-efficient compared to the KM-FPA algorithm; however, as the minimum downlink rate demand increases, the KM-FPA will consume more power to guarantee the rate of IoTD.

5.3.3. The Effects of the Number of Antennas on the Energy-Efficiency

Simulations of both the length of the pilot sequences and the number of IoTDs show that multiple antennas can improve the energy efficiency of the cell-free massive MIMO systems with distributed operations. APs with multiple antennas have greater immunity to interference and also increase the power consumption significantly. We have already discussed the minimum rate in the previous subsection; therefore, we will only focus on the effect of the number of antennas on energy efficiency, assuming a minimum rate of 0.1 Mbps in this subsection. In this subsubsection, we set the number of APs L = 100, the number of IoTDs K = 50, the minimum IoTDs rate R min = 0.1 Mbps , and the length of the pilot sequence is τ r s = 10 .
As shown in Figure 10, APs with more antennas can improve the downlink system performance gain; however, they also consume more energy to maintain the antennas operational. Hence, more antennas consume more energy, leading to lower energy efficiency. In our simulation, we have observed that EE is optimal when an AP is equipped with two antennas.
As the number of antennas at the AP increases, the downlink system performance gain is improving. As can be seen in the simulation, the downlink performance gain reaches a maximal value when the number of antennas per AP is two. When the number of antennas per AP is larger than two, the energy efficiency is degraded due to the fact that more antennas also consume more energy to maintain operations.
In Table 5, the KM-PA algorithm can provide higher energy efficiency than CF even when the AP is equipped with a larger number of antennas. Inter-user interference is greatly diminished when a large number of antennas are equipped with the APs of distributed cell-free massive MIMO; however, the pilot contamination cannot be eliminated unless the IoTDs using the same pilot have very different spatial correlation matrices from each other; then, the APs can separate their channels in the estimation phase [18].
The APs form a cluster of APs centered on the IoTDs under scheduling constraints collaborative clusters, which can suppress interference. Similarly, from the AP’s perspective, the AP can schedule at most one user per pilot, and the AP is more willing to serve IoTDs with strong channels with less power when matching with IoTDs, which means that IoTDs using the same pilot at the same time suffer less pilot contamination.
The simulation results demonstrate that deploying a large number of antennas at the AP to boost sum-rate is an uneconomical choice, even though, as shown in Table 5, the sum-rate of IoTDs increases with the number of antennas. However, a reasonable option would be to choose M = 2, The option of choosing M = 2 is guaranteed to eliminate some of the interference while maintaining higher energy efficiency than a single antenna.
By comparing Table 2 and Table 5, given the number of IoTDs K = 50, the proposed algorithms with MR precoding can improve the sum-rate performance by up to 36%, 10%, 8% and 9% compared to the CF algorithm with M = 1, M = 4, M = 8 and M = 16, respectively. Whereas the proposed algorithms with L-MMSE precoding can improve the sum-rate performance by up to 43%, 11%, 11% and 12% compared to the CF algorithm with M = 1, M = 4, M = 8 and M = 16, respectively. The above numerical results show that our algorithm is effective in improving the system energy efficiency and sum rate.

6. Conclusions

This paper presented an optimization problem of downlink energy efficiency optimization in the Industrial Internet based on a distributed cell-free massive MIMO system. The MINLP problem was formulated and decoupled into two sub-problems, i.e., maximizing the downlink sum-rate and minimizing the power consumption. The first problem is addressed by optimizing the AP selection by formulating and solving an integer programming model; based on the optimal AP solution, the sub-problem of minimizing the power consumption was addressed using two heuristic algorithms.
We conducted extensive simulations and observed that the proposed algorithms significantly improved the energy efficiency with low computational complexity in comparison with traditional distributed cell-free massive MIMO. In the presence of pilot contamination, the proposed AP selection-based power allocation algorithms could still provide significant energy efficiency when a large number of IoTDs were connected, making it suitable for Industrial Internet scenarios where low latency and massive access are required.
In our future work, we will attempt to solve the energy efficiency optimization problem with metaheuristic algorithms. Another interesting research topic in our future work would be energy-efficiency optimization in the scenario where antenna selection is also considered.

Author Contributions

Conceptualization, T.Z. and X.C.; methodology, T.Z.; software, T.Z.; validation, T.Z. and X.C.; formal analysis, T.Z.; investigation, T.Z.; resources, X.C.; data curation, T.Z.; writing—original draft preparation, T.Z. and X.C.; writing—review and editing, T.Z., X.C., Q.S., Q.H. and M.X.; visualization, T.Z.; supervision, X.C.; project administration, X.C.; funding acquisition, X.C. All authors have read and agreed to the published version of the manuscript.

Funding

The research was funded by State Key Laboratory of Advanced Optical Communication Systems and Networks, China (Grant No. 2021GZKF006).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A. Proof of Proposition 1

If the channels h k l , l { 1 , , L } are known at IoTD k, we can rewrite the downlink date rate at IoTD k in (10) as:
γ k = B τ d l τ log 2 ( 1 + DS k ( s k , ρ k ) MUI k ( s k , ρ k ) + σ d l 2 )
where the desire signal power DS k is
DS k ( s k , ρ k ) = l = 1 L ρ k l h k l H s k l w k l 2
and the multi-user interference power MUI k is
MUI k ( s k , ρ k ) = i = 1 i k K l = 1 L ρ i l h k l H s i l w i l 2
by substituting w k l = h k l into Equations (A2) and (A3), respectively, and assuming that the number of antennas M = 1, we obtain:
DS k = l = 1 L ρ k l s k l β k l 2
MUI k = i = 1 , i k K l = 1 L ρ i l s i l β i l
where ρ k l and β k l are the power allocation coefficient and the channel gain coefficient of l-th AP to the k-th IoTD, respectively. In calculating the SINR for the k-th IoT device, we treat the channel gain between the other IoTDs and each AP as a constant. Then, we have an expression for the SINR of IoTD k with channel gain β k l as the variable:
SINR k = l = 1 L ρ k l s k l β k l 2 i = 1 , i k K l = 1 L ρ i l s i l β i l + σ d l 2 .
Conducting a derivative on Equation (A6) yields:
SINR k β k l = 2 l = 1 L ρ k l s k l β k l · i = 1 , i k K l = 1 L ρ i l s i l β i l + σ d l 2 0 · 0 i = 1 , i k K l = 1 L ρ i l s i l β i l + σ d l 2 2 > 0 .
From the above equation, SINR k is a monotonically increasing function of the channel gain β k l > 0 , l ( 1 , , L ) . Furthermore, the downlink rate γ k is a monotonically increasing function of SINR k ( log 2 ( 1 + x ) is a monotonically increasing function of x for x ( 0 , inf ) ). Thus, as shown in Appendix A, the problem of maximizing the downlink rate is equivalent to the problem of maximizing the channel gain.

Appendix B. Proof of Theorem 1

We show that there is no perfect matching with weight greater than S * . Suppose that S is an arbitrary perfect matching of a graph G and that x and y are arbitrary vertices from two disjoint sets. Then,
ω ( S ) = x , y S ω ( x , y ) x , y S { t ( x ) + t ( y ) } = x t ( x ) + y t ( y ) = x , y S * { t ( x ) + t ( y ) } = x , y S * ω ( x , y ) = ω ( S * )
thus, S * is a maximal perfect matching, and the Kuhn–Munkres algorithm is guaranteed to achieve a global optimum.

References

  1. Lee, B.M.; Yang, H. Energy-Efficient Operation of Massive MIMO in Industrial Internet-of-Things Networks. IEEE Internet Things J. 2021, 8, 7252–7269. [Google Scholar] [CrossRef]
  2. Gkonis, P.K.; Trakadas, P.T.; Kaklamani, D.I. A comprehensive study on simulation techniques for 5G networks: State of the art results, analysis, and future challenges. Electronics 2021, 9, 468. [Google Scholar] [CrossRef] [Green Version]
  3. Buzzi, S.; D’Andrea, C. Cell-free massive MIMO: User-centric approach. IEEE Wirel. Commun. Lett. 2017, 6, 706–709. [Google Scholar] [CrossRef]
  4. Han, T.; Zhao, D. Energy efficiency of user-centric, cell-free massive MIMO-OFDM with instantaneous CSI. Entropy 2022, 24, 234. [Google Scholar] [CrossRef]
  5. Ammar, H.A.; Adve, R.; Shahbazpanahi, S.; Boudreau, G.; Srinivas, K.V. Distributed resource allocation optimization for user-centric cell-free MIMO networks. IEEE Trans. Wireless Commun. 2021, 21, 3099–3115. [Google Scholar] [CrossRef]
  6. Antonioli, R.P.; Braga, I.M.; Fodor, G.; Silva, Y.C.B.; Almeida, A.L.F.D.; Freitas, W.C. On the energy efficiency of cell-free systems with limited fronthauls: Is coherent transmission always the best alternative? IEEE Trans. Wirel. Commun. 2022. [Google Scholar] [CrossRef]
  7. Su, Z.; Feng, W.; Tang, J.; Chen, Z.; Fu, Y.; Zhao, N.; Wong, K.-K. Energy efficiency optimization for D2D communications underlaying UAV-assisted industrial iot networks with SWIPT. IEEE Internet Things J. 2022. [Google Scholar] [CrossRef]
  8. Mao, W.; Zhao, Z.; Chang, Z.; Min, G.; Gao, W. Energy-efficient industrial internet of things: Overview and open issues. IEEE Trans. Ind. Inf. 2021, 17, 7225–7237. [Google Scholar] [CrossRef]
  9. Lee, B.M. Cell-free massive MIMO for massive low power internet of things networks. IEEE Internet Things J. 2021, 9, 6520–6535. [Google Scholar] [CrossRef]
  10. Almamori, A.; Mohan, S. Estimation of channel state information (CSI) in cell-free massive MIMO based on time of arrival (ToA). Wirel. Personal Commun. 2020, 114, 1825–1831. [Google Scholar] [CrossRef]
  11. Miretti, L.; Björnson, E.; Gesbert, D. Team MMSE precoding with applications to cell-free massive MIMO. IEEE Trans. Wirel. Commun. 2022. [Google Scholar] [CrossRef]
  12. Liang, F.; Shen, C.; Yu, W.; Wu, F. Towards optimal power control via ensembling deep neural networks. IEEE Trans. Commun. 2019, 68, 1760–1776. [Google Scholar] [CrossRef] [Green Version]
  13. Garcia, I.D.; Kusashima, N.; Sakaguchi, K.; Araki, K. Dynamic cooperation set clustering on base station cooperation cellular networks. In Proceedings of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Helsinki, Finland, 13–16 September 2010; pp. 2127–2132. [Google Scholar]
  14. Bjornson, E.; Jalden, N.; Bengtsson, M.; Ottersten, B. Optimality properties, distributed strategies, and measurement-based evaluation of coordinated multicell OFDMA transmission. IEEE Trans. Signal Process. 2011, 59, 6086–6101. [Google Scholar] [CrossRef] [Green Version]
  15. Ammar, H.A.; Adve, R.; Shahbazpanahi, S.; Boudreau, G.; Srinivas, K.V. User-centric cell-free massive MIMO networks: A survey of opportunities, challenges and solutions. arXiv 2021, arXiv:2104.14589. [Google Scholar] [CrossRef]
  16. Nguyen, T.K.; Nguyen, T.H. Performance of assigning pilot sequences in cell free massive MIMO under SINR constraints. In Proceedings of the IEEE seventh International Conference on Communications and Electronics (ICCE), Hue City, Vietnam, 18–20 July 2018; pp. 121–126. [Google Scholar]
  17. Bashar, M.; Cumanan, K.; Burr, A.G.; Ngo, H.Q.; Larsson, E.G.; Xiao, P. On the energy efficiency of limited-backhaul cell-free massive MIMO. In Proceedings of the IEEE International Conference on Communications (ICC), Shanghai, China, 20–24 May 2019; pp. 1–7. [Google Scholar]
  18. Sabbagh, R.; Pan, C.; Wang, J. Pilot allocation and sum-rate analysis in cell-free massive MIMO systems. In Proceedings of the IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 20–24 May 2018; pp. 1–6. [Google Scholar]
  19. Buzzi, S.; D’Andrea, C.; Fresia, M.; Zhang, Y.-P.; Feng, S. Pilot assignment in cell-free massive MIMO based on the Hungarian algorithm. IEEE Wirel. Commun. Lett. 2020, 10, 34–37. [Google Scholar] [CrossRef]
  20. Björnson, E.; Sanguinetti, L. Making cell-free massive MIMO competitive with MMSE processing and centralized implementation. IEEE Trans. Wirel. Commun. 2019, 19, 77–90. [Google Scholar] [CrossRef] [Green Version]
  21. Hu, X.; Zhong, C.; Chen, X.; Xu, W.; Lin, H.; Zhang, Z. Cell-free massive MIMO systems with low resolution ADCs. IEEE Trans. Commun. 2019, 67, 6844–6857. [Google Scholar] [CrossRef]
  22. Lee, B.M. Energy efficient selected mapping schemes based on antenna grouping for industrial massive MIMO-OFDM antenna systems. IEEE Trans. Ind. Inf. 2018, 14, 4804–4814. [Google Scholar] [CrossRef]
  23. Kim, S.; Shim, B. Energy-efficient cell-free systems using finite rate feedback. In Proceedings of the IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), Online, 18 November–16 December 2020; pp. 1–5. [Google Scholar]
  24. Hosseini, S.S.; Champagne, B.; Chang, X.-W. A green downlink power allocation scheme for cell-free massive MIMO systems. IEEE Access 2020, 9, 6498–6512. [Google Scholar] [CrossRef]
  25. Alonzo, M.; Buzzi, S.; Zappone, A.; D’Elia, C. Energy-efficient power control in cell-free and user-centric massive MIMO at millimeter wave. IEEE Trans. Green Commun. Netw. 2019, 3, 651–663. [Google Scholar] [CrossRef] [Green Version]
  26. Van Chien, T.; Björnson, E.; Larsson, E.G. Joint power allocation and load balancing optimization for energy-efficient cell-free massive MIMO networks. IEEE Trans. Wirel. Commun. 2020, 19, 6798–6812. [Google Scholar] [CrossRef]
  27. Femenias, G.; Lassoued, N.; Riera-Palou, F. AP switch ON/OFF strategies for green cell-free massive MIMO networking. IEEE Access 2020, 8, 21788–21803. [Google Scholar] [CrossRef]
  28. Björnson, E.; Jorswieck, E. Optimal resource allocation in coordinated multi-cell systems. Found. Trends Commun. Inf. Theory 2013, 9, 113–381. [Google Scholar] [CrossRef]
  29. Elhoushy, S.; Ibrahim, M.; Hamouda, W. Cell-free massive MIMO: A survey. IEEE Commun. Surv. Tutor. 2020, 24, 492–523. [Google Scholar] [CrossRef]
  30. Nayebi, E.; Ashikhmin, A.; Marzetta, T.L.; Yang, H.; Rao, B.D. Precoding and power optimization in cell-free massive MIMO systems. IEEE Trans. Wirel. Commun. 2017, 16, 4445–4459. [Google Scholar] [CrossRef]
  31. Yan, H.; Ashikhmin, A.; Yang, H. A scalable and energy efficient IoT system supported by cell-free massive MIMO. IEEE Internet Things J. 2021, 8, 14705–14718. [Google Scholar] [CrossRef]
  32. Interdonato, G.; Karlsson, M.; Björnson, E.; Larsson, E.G. Local partial zero-forcing precoding for cell-free massive MIMO. IEEE Trans. Wirel. Commun. 2020, 19, 4758–4774. [Google Scholar] [CrossRef] [Green Version]
  33. Nayebi, E.; Ashikhmin, A.; Marzetta, T.L.; Rao, B.D. Performance of cell-free massive MIMO systems with MMSE and LSFD receivers. In Proceedings of the IEEE 50th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 6–9 November 2016; pp. 203–207. [Google Scholar]
  34. Ngo, H.Q.; Tran, L.-N.; Duong, T.Q.; Matthaiou, M.; Larsson, E.G. On the total energy efficiency of cell-free massive MIMO. IEEE Trans. Green Commun. Netw. 2018, 2, 25–39. [Google Scholar] [CrossRef] [Green Version]
  35. Zhu, H.; Zhou, M.; Alkins, R. Group role assignment via a Kuhn–Munkres algorithm-based solution. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2011, 42, 739–750. [Google Scholar] [CrossRef]
  36. 3GPP. Evolved Universal Terrestrial Radio Access (E-UTRA); Further Advancements for E-UTRA Physical Layer Aspects. 3rd Generation Partnership Project (3GPP), Technical Specification (TS) 36.814, 03 2017. Version 9.2.0. Available online: https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=2493 (accessed on 20 November 2021).
  37. Riera-Palou, F.; Femenias, G.; Caus, M.; Shaat, M.; Pérez-Neira, A.I. Scalable cell-free massive MIMO networks with LEO satellite support. IEEE Access 2022, 10, 37557–37571. [Google Scholar] [CrossRef]
  38. Valencia-Ponce, M.A.; Tlelo-Cuautle, E.; de la Fraga, L.G. On the sizing of CMOS operational amplifiers by applying many-objective optimization algorithms. Electronics 2021, 10, 3148. [Google Scholar] [CrossRef]
Figure 1. Distributed cell-free massive MIMO system in the Industrial Internet.
Figure 1. Distributed cell-free massive MIMO system in the Industrial Internet.
Mathematics 10 01687 g001
Figure 2. Coherent block model.
Figure 2. Coherent block model.
Mathematics 10 01687 g002
Figure 3. The sum energy efficiency versus the number of pilots, access point (AP) with single antenna.
Figure 3. The sum energy efficiency versus the number of pilots, access point (AP) with single antenna.
Mathematics 10 01687 g003
Figure 4. The sum energy efficiency versus the number of pilots; APs equipped with multiple antennas.
Figure 4. The sum energy efficiency versus the number of pilots; APs equipped with multiple antennas.
Mathematics 10 01687 g004
Figure 5. The sum energy efficiency versus the number of IoT devices (IoTDs), access points (APs) equipped with single antenna.
Figure 5. The sum energy efficiency versus the number of IoT devices (IoTDs), access points (APs) equipped with single antenna.
Mathematics 10 01687 g005
Figure 6. Cumulative distribution function of data rates for IoTDs.
Figure 6. Cumulative distribution function of data rates for IoTDs.
Mathematics 10 01687 g006
Figure 7. The sum energy efficiency versus the number of IoTDs; APs equipped with multiple antennas.
Figure 7. The sum energy efficiency versus the number of IoTDs; APs equipped with multiple antennas.
Mathematics 10 01687 g007
Figure 8. The number of IoTDs versus the number of activated APs.
Figure 8. The number of IoTDs versus the number of activated APs.
Mathematics 10 01687 g008
Figure 9. The power consumption versus the number of IoTDs.
Figure 9. The power consumption versus the number of IoTDs.
Mathematics 10 01687 g009
Figure 10. The sum energy efficiency versus the number of antennas.
Figure 10. The sum energy efficiency versus the number of antennas.
Mathematics 10 01687 g010
Table 1. Simulation parameter configuration.
Table 1. Simulation parameter configuration.
ParameterValue
Network area 1 km 2
Bandwidth B 20 MHz
Receiver noise power σ ul 2 94 dBm
Uplink transmit power p 100 mW
Maximum downlink transmit power P max 200 mW
Samples per coherence block τ 200
Channel gain at 1 km , Y 140.6 dB
Pathloss exponent α 3.67
Hibernation rate ϱ 5
Scheduled scaling factor ξ 0.5
power scaling factor ϑ 0.95
Height difference between AP and UE 10 m
array antenna spacing d H 0.5
Angular standard deviation σ φ 10
nominal angle φ 30
Table 2. The baseline algorithm and the proposed algorithms.
Table 2. The baseline algorithm and the proposed algorithms.
AlgorithmEstimatorPrecoding SchemeAP SelectionPower Allocation
Baseline: CFMMSEMRFull selectionEPA
Baseline: CFMMSEL-MMSEFull selectionEPA
Proposed: KM-FPAMMSEMRKM AlgorithmFPA
Proposed: KM-FPAMMSEL-MMSEKM AlgorithmFPA
Proposed: KM-PPAMMSEMRKM AlgorithmPPA
Proposed: KM-PPAMMSEL-MMSEKM AlgorithmPPA
Table 3. The sum rates of IoT devices (IoTDs) with different numbers of pilots.
Table 3. The sum rates of IoT devices (IoTDs) with different numbers of pilots.
Number of Pilots (L = 200, K [ 20 , 100 ] , τ = 200, R min = 0.1 Mbps)
Algorithms20406080100
(M = 1) MR-CF(Mbps)60.5959.7454.3746.4238.94
(M = 4) MR-CF(Mbps)105.88102.0291.1172.6465.33
(M = 1) MR-KM-PPA MR-CF 119.75%107.48%103.44%103.29%103.19%
(M = 4) MR-KM-PPA MR-CF 113.33%106.73%104.61%103.05%102.91%
(M = 1) MR-KM-FPA MR-CF 120.11%107.81%103.76%103.60%103.51%
(M = 4) MR-KM-FPA MR-CF 113.48%106.89%104.76%103.20%103.07%
(M = 1) L-MMSE-CF(Mbps)93.3292.9385.2772.8660.77
(M = 1) L-MMSE-CF(Mbps)226.41220.42192.61167.69140.33
(M = 1) L-MMSE-KM-PPA L-MMSE-CF 207.75%197.61%190.12%167.90%163.25%
(M = 4) L-MMSE-KM-PPA L-MMSE-CF 114.41%106.61%104.32%102.55%102.46%
(M = 1) L-MMSE-KM-FPA L-MMSE-CF 234.47%207.14%195.14%170.49%164.53%
(M = 4) L-MMSE-KM-FPA L-MMSE-CF 114.89%107.08%104.78%103.00%102.90%
Table 4. The sum rates of IoTDs with different numbers of IoTDs.
Table 4. The sum rates of IoTDs with different numbers of IoTDs.
Number of IoTDs (L = 200, τ = 200, R min = 0.1 Mbps)
Algorithms4080120160200
(M = 1) MR-CF(Mbps)36.0247.0568.8779.5485.49
(M = 4) MR-CF(Mbps)91.81145.77181.94205.72220.21
(M = 1) MR-KM-PPA MR-CF 138.22%140.21%135.58%134.58%134.22%
(M = 4) MR-KM-PPA MR-CF 110.96%110.08%110.28%110.68%111.03%
(M = 1) MR-KM-FPA MR-CF 139.04%140.73%135.81%134.71%134.32%
(M = 4) MR-KM-FPA MR-CF 110.98%110.11%110.32%110.72%111.07%
(M = 1) L-MMSE-CF(Mbps)54.2369.95102.47114.53120.40
(M = 4) L-MMSE-CF(Mbps)216.46345.74423.56471.17498.32
(M = 1) L-MMSE-KM-PPA L-MMSE-CF 167.90%163.25%146.10%141.48%139.74%
(M = 4) L-MMSE-KM-PPA L-MMSE-CF 116.12%111.51%110.12%109.66%109.60%
(M = 1) L-MMSE-KM-FPA L-MMSE-CF 170.49%164.53%146.47%141.70%139.89%
(M = 4) L-MMSE-KM-FPA L-MMSE-CF 116.33%111.68%110.27%109.79%109.72%
Table 5. The sum rate of IoTDs; APs equipped with different numbers of antennas (L = 100 and K = 50).
Table 5. The sum rate of IoTDs; APs equipped with different numbers of antennas (L = 100 and K = 50).
Number of Antennas of AP
Algorithms14816
MR-CF(Mbps)36.8783.98111.84145.07
MR-KM-PPA MR-CF 136.12%110.18%108.29%109.86%
MR-KM-FPA MR-CF 136.33%110.29%108.37%109.94%
L-MMSE-CF(Mbps)58.40201.50281.37344.17
L-MMSE-KM-PPA L-MMSE-CF 143.19%111.08%110.65%112.27%
L-MMSE-KM-FPA L-MMSE-CF 143.51%111.57%111.24%112.96%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chen, X.; Zhao, T.; Sun, Q.; Hu, Q.; Xu, M. Cell-Free Massive MIMO with Energy-Efficient Downlink Operation in Industrial IoT. Mathematics 2022, 10, 1687. https://doi.org/10.3390/math10101687

AMA Style

Chen X, Zhao T, Sun Q, Hu Q, Xu M. Cell-Free Massive MIMO with Energy-Efficient Downlink Operation in Industrial IoT. Mathematics. 2022; 10(10):1687. https://doi.org/10.3390/math10101687

Chicago/Turabian Style

Chen, Xiaomin, Taotao Zhao, Qiang Sun, Qiaosheng Hu, and Miaomiao Xu. 2022. "Cell-Free Massive MIMO with Energy-Efficient Downlink Operation in Industrial IoT" Mathematics 10, no. 10: 1687. https://doi.org/10.3390/math10101687

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop