In this section, we give the design of 7-QAM constellation. In addition, the theoretical analysis of its performance is carried out by calculating its PAPR, deriving its exact expression for the SEP over the AWGN channel and analyzing the non-linear distortion introduced to transmitted signals.

#### 3.1. Design of 7-QAM Constellation

In order to reduce the influence of non-linear distortions on the signal points and simultaneously obtain better anti-noise performance, the new constellation is designed based on the hexagonal grid signal sets of seven signals given in [

14]. In addition, the configuration of new constellation is 1 + 6. Specifically, as shown in

Figure 2a, one signal point is on the origin point, and the other six signal points are evenly arranged on the outer circle with the phase difference of

$\pi /3$. The vector expression of 7-QAM signals can be written as

where

${R}_{1}$ is the radius of the 7-QAM constellation.

The constellation diagrams of the conventional 8-PSK and 8-QAM are also demonstrated in

Figure 2. The signal points are equiprobable, so that, for the normalization of power, the radiuses of these constellations are

${R}_{1}\approx 1.0801$,

${R}_{2}=1$,

${R}_{3}\approx 0.6501$ and

${{R}_{3}}^{\prime}\approx 1.2559$.

Table 1 shows the comparison for parameters of the three constellations presented in

Figure 2, and the minimum phase difference

${\theta}_{\mathrm{min}}$ of 7-QAM is the value except the signal point ‘0’. It can be known from

Table 1 that the minimum Euclidean distance

${d}_{\mathrm{min}}$ of the 7-QAM constellation is 3 dB higher than that of the 8-PSK constellation and is 1.4 dB higher than that of 8-QAM constellation, which reflects the fact that the anti-noise performance of 7-QAM is much better than that of 8-PSK and 8-QAM.

More serious non-linear distortion would be introduced as applying the constellation with higher PAPR to the satellite communication system. For the practical consideration, the PAPR of 7-QAM constellation is calculated. The PAPR of the constellation can be calculated by

where

${E}_{\mathrm{s}}$ is the average energy per symbol of the constellation and

${E}_{\mathrm{max}}$ is the maximum energy of the modulated signal. Specifically, in the case that half of the minimum distance between adjacent symbols is

d, the average energy per symbol of the 7-QAM constellation, denoted as

${E}_{7-\mathrm{QAM}}$, can be calculated as

where

${S}_{k}$ represents the

k-th signal point of the 7-QAM constellation,

$\mathrm{Re}\left({S}_{k}\right)$ and

$\mathrm{Im}\left({S}_{k}\right)$ mean the real and imaginary values of the signal point

${S}_{k}$, respectively. In addition, the maximum energy of 7-QAM signal is

$4{d}^{2}$. Thereby, the PAPR of the 7-QAM constellation is

$1.1667$. With the same method, the PAPRs of the 8-PSK and 8-QAM constellations are also calculated for comparison, and the calculation results are presented in

Table 2.

It can be known that the PAPR of 7-QAM constellation is 1.31 dB lower than that of 8-QAM constellation, and is 0.67 dB higher than that of 8-PSK constellation.

#### 3.2. SEP Expression of 7-QAM Constellation

The formula for calculating the SEP of a constellation helps to analyze its anti-noise performance. In this part, we derive the exact closed-form expression for the SEP of the new constellation over the AWGN channel for further theoretical analysis to its error performance.

The decision region for each signal point of the 7-QAM constellation is shown in

Figure 3, and DL1, DL2 (Q axis), and DL3 are decision boundaries. Obviously, according to the decision region, the signal points could be divided into two classes, interior point and points on the outer circle.

As shown in

Figure 3, the correct decision region for signal point ‘0’ is the regular hexagon (ABCDEF). It is decomposed into five regions, which can be expressed as (ABW) + (AFW) + (BCEF) + (DCS) + (DES). In addition, the probabilities of correct decision for the four right triangle (RT) regions are equal.

In order to express the probability of correct decision for the RT region (AFW) in terms of the Gaussian Q-function

$Q\left(x\right)$, which is the probability that the random variable will obtain a value larger than

x and is defined as

$Q\left(x\right)=\frac{1}{\sqrt{2\pi}}{\int}_{x}^{\infty}{e}^{-{t}^{2}/2}dt$, the RT region with right angle edge lengths of

${l}_{1}$ and

${l}_{2}$ can be divided into an infinite number of rectangles, as shown in

Figure 4a.

There are

${2}^{n-1}$ rectangles with length of

${2}^{-n}{l}_{1}$ and width of

${2}^{-n}{l}_{2}$ in the RT region, and each rectangle is denoted by

where

$n=1,2,\dots $ and

$k=0,1,2,\dots ,{2}^{n-1}-1$. If the transimitted signal is with a coordinate of

$\left({S}_{\mathrm{I}},{S}_{\mathrm{Q}}\right)$, which is determined by using two right-angled sides of the RT region as coordinate axes, the probability that the received signal falls in the region

${R}_{n}^{k}$ can be written as

which is obtained from [

27]. The probability that the received signal falls in the RT region can be obtained through summing all the probabilities

${P}_{n}^{k}$, namely

The probability that the received signal falls in the residual area of the RT region after division

n times decreases exponentially with the increase of

n, so Equation (

10) can be made arbitrarily accurate [

27].

The distance between points ‘F’ and ‘W’ is

d and that between points ‘A’ and ‘W’ is

$\frac{\sqrt{3}d}{3}$, where

d is half of the minimum distance between two adjacent signal points. The probability of correct decision for the (AFW) region could be written as

In addition, according to the reference [

28], the probability of correct decision for the (BCEF) region can be calculated by

Combining Equations (

11) and (

12), and denoting the probability of correct decision for signal point ‘0’ using

${P}_{0}$, it could be written as

As shown in

Figure 3, the correct decision region for signal point ‘1’ is (JFEK).

Figure 4b shows the schematic diagram of the division for the (JFEK) region. Thus, the (JFEK) region can be expressed as (JUTL)+(RVEK)+(LTR)+(FUT)+(FEV). It can be known from

Figure 4b that

$\overline{\mathrm{VE}}=\overline{\mathrm{UT}}=d$ and

$\overline{\mathrm{VT}}=\frac{\sqrt{3}d}{3}$. The probabilities of correct decision for the (JUTL) region and the (RVEK) region can be written as

The angle of

$\angle \mathrm{LTR}$ is

$\pi /3$. Thus, the probability of correct decision for the (LTR) region is

For the (FUT) region and the (FEV) region, there are

$\overline{\mathrm{VF}}=\overline{\mathrm{UF}}=\frac{\sqrt{3}d}{3}$. Thus, the probabilities of correct decision for (FUT) and (FEV) regions are

Combining Equations (

14) to (

18), and denoting the correct decision probability for signal point ‘1’ using

${P}_{1}$, it could be expressed as

Finally, combining Equations (

13) and (

19), denoting the exact average SEP for the 7-QAM constellation using

${P}_{7-\mathrm{QAM}}$, it can be expressed as

Substituting the average symbol signal-to-noise ratio (SNR) into Equation (

20), which is defined as

${\gamma}_{s}={E}_{s}/{N}_{0}=12{d}^{2}/\left(7{\sigma}^{2}\right)$, the exact expression for the average SEP of 7-QAM over the AWGN channel is

#### 3.3. Analysis of Constellation Distortion

If the modulated signals are sensitive to the non-linear characteristic of the HPA, the resulting non-linear distortion would reduce the overall performance gain expected with the application of higher modulation orders. In order to analyze the sensitivity of 7-QAM signals to the non-linear effects, the distorted constellation diagrams of different modulation methods over the non-linear satellite channel model are presented in

Figure 5. In addition, the characteristic parameters in the Saleh model, presented in Equations (

3) and (

4), for the non-linear conversion module are set as

${\alpha}_{\mathrm{a}}=1.9638$,

${\beta}_{\mathrm{a}}=0.9945$,

${\alpha}_{\mathrm{p}}=2.5293$ and

${\beta}_{\mathrm{p}}=2.8168$ in these simulations [

32].

In

Figure 5, the arrow points from the original constellation point to its corresponding distorted signal point through the non-linear channel model. Over the non-linear satellite channel model, the 8-QAM signals shown in

Figure 5c have serious non-linear distortion in both amplitude and phase. The 8-PSK modulated signals shown in

Figure 5b are changed a bit in amplitude but are shifted much in phase. The signals mapped to the signal point on the origin of 7-QAM constellation remain constant in both phase and amplitude, and the other 7-QAM signals are compressed a little more than the 8-PSK signals and also have much distortion in phase. Because of the larger

${\theta}_{\mathrm{min}}$, the 7-QAM signals are less sensitive to the AM/PM distortion than both the 8-PSK signals and the 8-QAM signals.

In summary, although the 8-PSK signals are robust to the non-linear AM/AM conversion, they are poor at the anti-noise performance and the tolerance of the AM/PM distortion. The 7-QAM signals have much better anti-noise performance and are mildly influenced by the AM/AM conversion and the AM/PM conversion, so that the 7-QAM signals may have better error performance over the non-linear channel model than the 8-PSK signals and 8-QAM signals.