# Design Method for a Wideband Non-Uniformly Spaced Linear Array Using the Modified Reinforcement Learning Algorithm

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Antenna Array Model

## 3. Cost Function

#### 3.1. Cost Function for Narrowband NUSLAs

#### 3.2. Cost Function for Wideband NUSLAs

## 4. Optimization Algorithm Based on Reinforcement Learning

Algorithm 1 An algorithm for optimizing NUSLA using MORELA. |

Initialization StepInitialize parameters ($T,K,\alpha ,\beta ,\gamma $) Generate randomly K parameter vectors (${\mathbf{d}}_{k}^{\left(0\right)}$ and ${\mathbf{w}}_{k}^{\left(0\right)}$) using Equation (9) for$k=1,\cdots ,K$Calculate cost function (8) for ${\mathbf{d}}_{k}^{\left(0\right)}$ and ${\mathbf{w}}_{k}^{\left(0\right)}$ end forStore the best performing parameters (${\mathbf{d}}_{best}^{\left(0\right)}$ and ${\mathbf{w}}_{best}^{\left(0\right)}$) Update Stepfor$t=1,\cdots ,T$Generate additional K parameter vectors (${\mathbf{d}}_{K+k}^{\left(t\right)}$ and ${\mathbf{w}}_{K+k}^{\left(t\right)}$) using Equation (11) for$k=1,\cdots ,K$Calculate the cost function (8) for original (${\mathbf{d}}_{k}^{\left(t\right)}$ and ${\mathbf{w}}_{k}^{\left(t\right)}$) and sub-environment (${\mathbf{d}}_{K+k}^{\left(t\right)}$ and ${\mathbf{w}}_{K+k}^{\left(t\right)}$) end forRemove the worst K parameter vectors Store the best performing parameters as (${\mathbf{d}}_{best}^{(t+1)}$ and ${\mathbf{w}}_{best}^{(t+1)}$) Calculate the K reward vectors using Equation (13) Update the survived K parameter vectors using Equation (12) end for |

## 5. Simulation Results

#### 5.1. Wideband Symmetric NUSLA

#### 5.2. Wideband Asymmetric NUSLA

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Designed Beam Patterns

**Figure A1.**The beam pattern of symmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=0\xb0$.

**Figure A2.**The beam pattern of symmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=30\xb0$.

**Figure A3.**The beam pattern of symmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=60\xb0$.

**Figure A4.**The beam pattern of asymmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=0\xb0$.

**Figure A5.**The beam pattern of symmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=30\xb0$.

**Figure A6.**The beam pattern of asymmetric NUSLA using (

**a**) MORELA, (

**b**) QPSO, (

**c**) FA, (

**d**) SSA, and (

**e**) MALO in case for ${\theta}_{0}=60\xb0$.

## References

- Volakis, J.L. Antenna Engineering Handbook; McGraw-Hill Education: New York, NY, USA, 2007. [Google Scholar]
- Stutzman, W.L.; Thiele, G.A. Antenna Theory and Design; John Wiley & Sons: Hoboken, NJ, USA, 2012. [Google Scholar]
- Balanis, C. A Antenna Theory: Analysis and Design; John Wiley & Sons: Hoboken, NJ, USA, 2015. [Google Scholar]
- Benesty, J.; Cohen, I.; Chen, J. Fundamentals of Signal Enhancement and Array Signal Processing; John Wiley & Sons: Hoboken, NJ, USA, 2017. [Google Scholar]
- Ridwan, M.; Abdo, M.; Jorswieck, E. Design of Non-Uniform Antenna Arrays Using Genetic Algorithm. In Proceedings of the 13th International Conference on Advanced Communication Technology (ICACT2011), Seoul, Korea, 13–16 February 2011; pp. 422–427. [Google Scholar]
- Basu, B.; Mahanti, G. Fire Fly and Artificial Bees Colony Algorithm for Synthesis of Scanned and Broadside Linear Array Antenna. Prog. Electromagn. Res. B
**2011**, 32, 169–190. [Google Scholar] [CrossRef] [Green Version] - Ram, G.; Mandal, D.; Kar, R.; Ghoshal, S.P. Optimized Hyper Beamforming of Receiving Linear Antenna Arrays Using Firefly Algorithm. Int. J. Microw. Wirel. Technol.
**2014**, 6, 181–194. [Google Scholar] [CrossRef] - Bai, H. Design of Non-Uniform Linear Array via Linear Programming and Particle Swarm Optimization and Studies on Phased Array Calibration. Masters Thesis, University of Massachusetts Amherst, Amherst, MA, USA, 2014. [Google Scholar]
- Sri, K.B.; Rao, N.V. Optimization of SLL of Linear Array Antennas using Enhanced Firefly Algorithm. Int. J. Eng. Res. Appl.
**2020**, 10, 19–23. [Google Scholar] - Li, S.; Yang, X.; Ning, L.; Long, T.; Sarkar, T.K. Broadband Constant Beamwidth Beamforming for Suppressing Mainlobe and Sidelobe Interferences. In Proceedings of the 2017 IEEE Radar Conference (RadarConf), Seattle, WA, USA, 8–12 May 2017; pp. 1041–1045. [Google Scholar]
- Liu, M.; Zou, L.; Wang, X. Practical Beamforming Technologies for Wideband Digital Array Radar. Prog. Electromagn. Res. Lett.
**2019**, 86, 145–151. [Google Scholar] [CrossRef] [Green Version] - Feng, Y.; Li, J.Y.; Zhang, L.K.; Yu, X.J.; Qi, Y.X.; Li, D.; Zhou, S.G. A Broadband Wide-Angle Scanning Linear Array Antenna with Suppressed Mutual Coupling for 5G Sub-6G Applications. IEEE Antennas Wirel. Propag. Lett.
**2022**, 21, 366–370. [Google Scholar] [CrossRef] - Wang, B.H.; Hui, H.T.; Leong, M.S. Optimal Wideband Beamforming for Uniform Linear Arrays Based on Frequency-Domain MISO System Identification. IEEE Trans. Antennas Propag.
**2010**, 58, 2580–2587. [Google Scholar] [CrossRef] - Zhang, W.; Su, T. Reference Beam Pattern Design for Frequency Invariant Beamforming Based on Fast Fourier Transform. Sensors
**2016**, 16, 1554. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Long, T.; Cohen, I.; Berdugo, B.; Yang, Y.; Chen, J. Window-Based Constant Beamwidth Beamformer. Sensors
**2019**, 19, 2091. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Xu, L.; Li, R.; Chen, X.; Wei, F.; Shi, X. Wideband Frequency Invariant Array Synthesis Based on Matrix Singular Value Decomposition. Electronics
**2021**, 10, 2039. [Google Scholar] [CrossRef] - Murino, V.; Trucco, A.; Regazzoni, C.S. Synthesis of Unequally Spaced Arrays by Simulated Annealing. IEEE Trans. Signal Process.
**1996**, 44, 119–123. [Google Scholar] [CrossRef] [Green Version] - Gangwar, V.S.; Singh, A.K.; Singh, S.P. Side Lobe Level Suppression in Randomly Spaced Linear Array Using Genetic Algorithm. In Proceedings of the 2015 IEEE MTT-S International Microwave and RF Conference (IMaRC), India, Hyderabad, 10–12 December 2015; pp. 381–384. [Google Scholar]
- Zaman, M.A.; Abdul Matin, M. Nonuniformly Spaced Linear Antenna Array Design Using Firefly Algorithm. Int. J. Microw. Sci. Technol.
**2012**, 2012, 256759. [Google Scholar] [CrossRef] [Green Version] - Miranda, A.V.; Ashwin, P.; Sharan, P.; Gangwar, V.S.; Singh, A.K.; Singh, S.P. An Efficient Synthesis of Unequally Spaced Antenna Array with Electronic Scan Capability Utilizing Particle Swarm Optimization. In Proceedings of the 2017 IEEE MTT-S International Microwave and RF Conference (IMaRC), India, Ahmedabad, 11–13 December 2017; pp. 255–258. [Google Scholar]
- Luo, Z.; Liu, F.; Zou, Z.; Guo, S.; Shen, T. Optimum design of both linear and planar sparse arrays with sidelobe level reduction using salp swarm algorithm. J. Electromagn. Waves Appl.
**2021**, 35, 690–704. [Google Scholar] [CrossRef] - Liu, Y.; Zhang, L.; Zhu, C.; Liu, Q.H. Synthesis of Nonuniformly Spaced Linear Arrays With Frequency-Invariant Patterns by the Generalized Matrix Pencil Methods. IEEE Trans. Antennas Propag.
**2015**, 63, 1614–1625. [Google Scholar] [CrossRef] - Gu, P.; Wang, G.; Fan, Z.; Chen, R. Efficient Unitary Matrix Pencil Method for Synthesising Wideband Frequency Patterns of Sparse Linear Arrays. IET Microwaves Antennas Propag.
**2018**, 12, 1871–1876. [Google Scholar] [CrossRef] - Van Luyen, T.; Giang, T.V.B. Null-Steering Beamformer Using Bat Algorithm. Appl. Comput. Electromagn. Soc. J.
**2018**, 33, 23–29. [Google Scholar] - BouDaher, E.; Hoorfar, A. Comparison of Nature-Inspired Techniques in Design Optimization of Non-Uniformly Spaced Arrays in The Presence of Mutual Coupling. Digit. Signal Process.
**2020**, 105, 1–19. [Google Scholar] [CrossRef] - Pradhan, H.; Mangaraj, B.B.; Behera, S.K. Chebyshev-Based Array for Beam Steering and Null Positioning Using Modified Ant Lion Optimization. Int. J. Microw. Wirel. Technol.
**2022**, 14, 143–157. [Google Scholar] [CrossRef] - Patidar, H.; Mahanti, G.K.; Muralidharan, R. Quantum Particle Swarm Optimization for Synthesis of Non-Uniformly Spaced Linear Arrays with Broadband Frequency Invariant Pattern. J. Microwaves Optoelectron. Electromagn. Appl.
**2017**, 16, 602–614. [Google Scholar] [CrossRef] [Green Version] - Ozan, C.; Baskan, O.; Haldenbilen, S. A Novel Approach Based on Reinforcement Learning for Finding Global Optimum. Open J. Optim.
**2017**, 6, 65–84. [Google Scholar] [CrossRef] [Green Version]

**Figure 4.**The HPBW performance of symmetric NUSLA in case for (

**a**) ${\theta}_{0}=0\xb0$, (

**b**) ${\theta}_{0}=30\xb0$, and (

**c**) ${\theta}_{0}=60\xb0$.

**Figure 5.**The SLL performance of symmetric NUSLA in case for (

**a**) ${\theta}_{0}=0\xb0$, (

**b**) ${\theta}_{0}=30\xb0$, and (

**c**) ${\theta}_{0}=60\xb0$.

**Figure 6.**The HPBW performance of symmetric NUSLA in the case for (

**a**) ${\theta}_{0}=0\xb0$, (

**b**) ${\theta}_{0}=30\xb0$, and (

**c**) ${\theta}_{0}=60\xb0$.

**Figure 7.**The SLL performance of symmetric NUSLA in the case for (

**a**) ${\theta}_{0}=0\xb0$, (

**b**) ${\theta}_{0}=30\xb0$, and (

**c**) ${\theta}_{0}=60\xb0$.

NUSLA Parameter | Value |
---|---|

Minimum/maximum frequency $({f}_{min}/{f}_{max})$ | 0.5/1 GHz |

The number of frequencies $\left({N}_{f}\right)$ | 11 |

Minimum/maximum spacing $({d}_{min}/{d}_{max})$ | 0.5/1.5 ${\lambda}_{min}$ |

Minimum/maximum amplitude $({a}_{min}/{a}_{max})$ | 0.1/1 |

Minimum/maximum phase $({\varphi}_{min}/{\varphi}_{max})$ | 0/$\pi $ |

Propagation speed $\left(c\right)$ | $3\times {10}^{8}$ m/s |

MORELA Parameter | Value |
---|---|

The number of maximum learning episode $\left(T\right)$ | 1000 |

Size of the sub-environment $\left(K\right)$ | 20 |

Learning rate $\left(\alpha \right)$ | 0.8 |

Discounting factor $\left(\gamma \right)$ | 0.2 |

Search space reducing factor $\left(\beta \right)$ | 0.99 |

Max HPBW (deg) | Min HPBW (deg) | PSLL (dB) | Time (s) | ||
---|---|---|---|---|---|

${\theta}_{0}=0\xb0$ | 6.16 | 3.08 | −20.15 | ||

MORELA | ${\theta}_{0}=30\xb0$ | 9.74 | 4.86 | −19.99 | 37.95 |

${\theta}_{0}=60\xb0$ | 22.20 | 10.52 | −18.42 | ||

${\theta}_{0}=0\xb0$ | 6.34 | 3.16 | −20.04 | ||

QPSO | ${\theta}_{0}=30\xb0$ | 9.82 | 4.90 | −19.99 | 20.07 * |

${\theta}_{0}=60\xb0$ | 16.55 | 8.05 | −0.15 | ||

${\theta}_{0}=0\xb0$ | 6.32 | 3.16 | −20.05 | ||

FA | ${\theta}_{0}=30\xb0$ | 10.43 | 5.21 | −19.98 | 194.98 |

${\theta}_{0}=60\xb0$ | 22.97 | 10.84 | −17.90 | ||

${\theta}_{0}=0\xb0$ | 6.48 | 3.24 | −20.00 | ||

SSA | ${\theta}_{0}=30\xb0$ | 11.50 | 5.74 | −14.67 | 18.87 |

${\theta}_{0}=60\xb0$ | 21.18 | 10.10 | −13.47 | ||

${\theta}_{0}=0\xb0$ | 10.38 | 5.18 | −18.63 | ||

MALO | ${\theta}_{0}=30\xb0$ | 12.32 | 6.14 | −15.18 | 47.50 |

${\theta}_{0}=60\xb0$ | 22.97 | 10.84 | −14.19 |

Max HPBW (deg) | Min HPBW (deg) | PSLL (dB) | Time (s) | ||
---|---|---|---|---|---|

${\theta}_{0}=0\xb0$ | 6.00 | 3.00 | −20.04 | ||

MORELA | ${\theta}_{0}=30\xb0$ | 9.91 | 4.96 | −20.01 | 117.56 |

${\theta}_{0}=60\xb0$ | 21.78 | 10.46 | −17.08 | ||

${\theta}_{0}=0\xb0$ | 6.22 | 3.10 | −20.03 | ||

QPSO | ${\theta}_{0}=30\xb0$ | 10.11 | 5.04 | −18.34 | 53.40 |

${\theta}_{0}=60\xb0$ | 14.42 | 7.13 | −0.15 | ||

${\theta}_{0}=0\xb0$ | 6.54 | 3.28 | −19.97 | ||

FA | ${\theta}_{0}=30\xb0$ | 10.84 | 5.42 | −18.32 | 520.75 |

${\theta}_{0}=60\xb0$ | 22.18 | 10.55 | −12.87 | ||

${\theta}_{0}=0\xb0$ | 6.58 | 3.28 | −16.30 | ||

SSA | ${\theta}_{0}=30\xb0$ | 11.03 | 5.51 | −18.08 | 52.44 |

${\theta}_{0}=60\xb0$ | 21.10 | 10.07 | −13.26 | ||

${\theta}_{0}=0\xb0$ | 9.96 | 4.98 | −14.47 | ||

MALO | ${\theta}_{0}=30\xb0$ | 11.14 | 5.55 | −12.11 | 136.30 |

${\theta}_{0}=60\xb0$ | 21.06 | 10.1 | −10.06 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kang, S.; Kim, S.; Park, C.; Chung, W.
Design Method for a Wideband Non-Uniformly Spaced Linear Array Using the Modified Reinforcement Learning Algorithm. *Sensors* **2022**, *22*, 5456.
https://doi.org/10.3390/s22145456

**AMA Style**

Kang S, Kim S, Park C, Chung W.
Design Method for a Wideband Non-Uniformly Spaced Linear Array Using the Modified Reinforcement Learning Algorithm. *Sensors*. 2022; 22(14):5456.
https://doi.org/10.3390/s22145456

**Chicago/Turabian Style**

Kang, Seyoung, Seonkyo Kim, Cheolsun Park, and Wonzoo Chung.
2022. "Design Method for a Wideband Non-Uniformly Spaced Linear Array Using the Modified Reinforcement Learning Algorithm" *Sensors* 22, no. 14: 5456.
https://doi.org/10.3390/s22145456