Next Article in Journal
A Novel Approach for Denoising Magnetic Flux Leakage Signals of Steel Wire Ropes via Synchrosqueezing Wavelet Transform and Dynamic Time–Frequency Masking
Previous Article in Journal
Functionalized Agave Bagasse Hydrochar for Reactive Orange 84 Removal: Synthesis, Characterization, and ANN–GA Optimization
Previous Article in Special Issue
Physics-Constrained Graph Attention Networks for Distribution System State Estimation Under Sparse and Noisy Measurements
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Processes
Volume 14
Issue 1
10.3390/pr14010011
processes-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
This is an early access version, the complete PDF, HTML, and XML versions will be available soon.
Article

Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning

by
Zhe Wang
1,
Yuxin Dai
1,
Wenxin Yang
1,
Yunzhang Yang
1,
Zhiqi Zhang
1,
Yahan Hu
1,
Jianquan Liao
2,* and
Tianchi Wu
2
1
State Grid Shaanxi Electric Power Company Limited Research Institute, Xi’an 710000, China
2
College of Electrical Engineering, Sichuan University Chengdu 610065, China
*
Author to whom correspondence should be addressed.
Processes 2026, 14(1), 11; https://doi.org/10.3390/pr14010011
Submission received: 19 November 2025 / Revised: 15 December 2025 / Accepted: 16 December 2025 / Published: 19 December 2025
(This article belongs to the Special Issue Advances in Optimal Operation of Modern Power Systems for Flexibility Enhancement)
Versions Notes

Abstract

Hybrid AC/DC transmission expansion planning must balance investment cost, supply reliability and AC/DC stability, which challenges conventional mathematical programming and heuristic methods. This paper proposes a multi-objective planning framework based on an improved multi-step backtracking α-Q(λ) reinforcement learning algorithm with eligibility traces and an adaptive learning factor. A tri-objective model minimises annual economic cost, expected power shortage and a comprehensive electrical index that combines electrical betweenness, commutation-failure margin and effective short-circuit ratio. The mixed-integer planning problem is reformulated as an interactive learning process, where the state encodes candidate line construction decisions, the action builds or cancels lines, and the eligibility-trace matrix is used to quantify line importance. Case studies on the Garver-6 system, the IEEE 24-bus reliability test system and a 500 kV regional hybrid AC/DC grid show that, compared with classical Q-learning, the proposed method yields lower annual cost, reduced expected power shortage and improved AC/DC stability; in the 500 kV system, the expected annual power shortage is reduced from 70,810 MWh to 28,320 MWh.
Keywords: transmission network planning; reinforcement learning; multi-step backtracking; multi-objective optimisation; eligibility trace; α-Q(λ) algorithm; AC/DC stability transmission network planning; reinforcement learning; multi-step backtracking; multi-objective optimisation; eligibility trace; α-Q(λ) algorithm; AC/DC stability

Share and Cite

MDPI and ACS Style

Wang, Z.; Dai, Y.; Yang, W.; Yang, Y.; Zhang, Z.; Hu, Y.; Liao, J.; Wu, T. Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning. Processes 2026, 14, 11. https://doi.org/10.3390/pr14010011

AMA Style

Wang Z, Dai Y, Yang W, Yang Y, Zhang Z, Hu Y, Liao J, Wu T. Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning. Processes. 2026; 14(1):11. https://doi.org/10.3390/pr14010011

Chicago/Turabian Style

Wang, Zhe, Yuxin Dai, Wenxin Yang, Yunzhang Yang, Zhiqi Zhang, Yahan Hu, Jianquan Liao, and Tianchi Wu. 2026. "Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning" Processes 14, no. 1: 11. https://doi.org/10.3390/pr14010011

APA Style

Wang, Z., Dai, Y., Yang, W., Yang, Y., Zhang, Z., Hu, Y., Liao, J., & Wu, T. (2026). Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning. Processes, 14(1), 11. https://doi.org/10.3390/pr14010011

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop