Digitalization Processes in Distribution Grids: A Comprehensive Review of Strategies and Challenges

Abstract

This systematic review meticulously explores the transformative impact of digital technologies on the grid planning, grid operations, and energy market dynamics of power distribution grids. Utilizing a robust methodological framework, over 54,000 scholarly articles were analyzed to investigate the integration and effects of artificial intelligence, machine learning, optimization, the Internet of Things, and advanced metering infrastructure within these key subsections. The literature was categorized to show how these technologies contribute specifically to grid planning, operation, and market mechanisms. It was found that digitalization significantly enhances grid planning through improved forecasting accuracy and robust infrastructure design. In operations, these technologies enable real-time management and advanced fault detection, thereby enhancing reliability and operational efficiency. Moreover, in the market domain, they support more efficient energy trading and help in achieving regulatory compliance, thus fostering transparent and competitive markets. However, challenges such as data complexity and system integration are identified as critical hurdles that must be overcome to fully harness the potential of smart grid technologies. This review not only highlights the comprehensive benefits but also maps out the interdependencies among the planning, operation, and market strategies, underlining the critical role of digital technologies in advancing sustainable and resilient energy systems.

1. Introduction

The shift towards digitalizing distribution power grids marks a pivotal step in modernizing energy infrastructure, emphasizing decentralization and renewable energy adoption. As distribution grids encounter increased variability and the integration of distributed energy resources (DERs), the necessity for dynamic grid management escalates. Digitalization serves as a transformative force, revolutionizing grid planning, operations, and market interactions.

The current infrastructure, supported by technologies like supervisory control and data acquisition (SCADA) systems and intelligent electronic devices (IEDs) in medium voltage (MV) networks, provides a solid base for adopting more advanced digital solutions by distribution system operators (DSOs). However, the challenge lies in handling the vast, complex data generated by smart grid (SGs), which stretches the limits of traditional data processing methods.

Artificial intelligence (AI) offers, for instance, breakthroughs in load forecasting, grid stability, fault detection, and security, as well as enhancing smart grid reliability and resilience. The potential of AI underscores the importance of further exploration and implementation in developing fully optimized SG systems [1].

This transformation is underscored by multiple studies [2,3,4,5] and is driven by advancements in the Internet of Things (IoT), artificial intelligence (AI), edge computing, advanced metering infrastructure (AMI), and blockchain.

Moreover, the development of electric digital twins and service-oriented architectures marks a shift towards more dynamic, automated, and resilient energy systems [2,3]. This indicates a future where energy systems are adaptable to changing demands and environmental challenges [2,4,5].

The successful digital transformation of the power sector extends beyond technology to strategic coordination [6]. As demonstrated by companies like Jibei Electric Power Company, this transformation needs to align digital efforts with broader organizational goals, enhancing power transmission, distribution capabilities, and internal digitalization [6].

Recent scholarly works offer insights into the complex nature of SG technologies. They illustrate the impact of digitalization and renewable energy integration on power systems and consumers [5,7,8,9,10,11,12]. From decentralizing renewable energy generation to leveraging broadband over power line infrastructure for unified data exchange, these technologies balance supply and demand and enhance grid intelligence.

This systematic literature review explores the expansive terrain of digitalization in power distribution grids, categorizing our analysis into three primary domains: grid planning, grid operations, and the energy market. Utilizing a robust search methodology, we identified a substantial corpus of literature from IEEE Xplore, encompassing both conference papers and journal articles, totaling 13,700 articles in the planning domain, 31,859 in operations, and 9098 in the energy market from 2014 to January 2024.

To ensure transparency and reproducibility, this section details the procedure followed in our study using Python version 3.10.13 scripting to manage and analyze data related to digital technologies in power distribution grids.

The process begins with the initialization and importing of essential libraries, including os for file operations, pandas for data manipulation, NumPy for numerical operations, and matplotlib.pyplot and seaborn for data visualization.

Next, the directory containing the CSV files is defined, with the scholarly articles obtained from an advanced IEEE Xplore search and listing all CSV files in this directory to process them systematically. Then, we initialize empty lists to store dataframes categorized by their relevance to planning, operations, and market.

Each CSV file is read and, based on the category inferred from the file name, the data are appended to the respective list (planning, operation, or market). Once the files are read and categorized, the dataframes in each category are concatenated into single dataframes (one for each category). Duplicates are then removed to ensure data integrity.

A list of relevant keywords related to digital technologies is defined and the occurrences of these keywords in the articles’ titles, abstracts, keywords, and IEEE terms is counted for each category. Following this, the percentage of articles containing each keyword was calculated to understand the proportional emphasis on different technologies.

To visually represent the keyword counts and percentages, we plotted bar charts across the three categories: planning, operation, and market. Additionally, a report was generated to identify and count any duplicate entries in the ‘Document Title’ column for each category.

Insights were derived based on the data analysis, and a detailed discussion on the challenges and potential solutions related to data complexity and system integration was provided. Finally, the procedure concludes after deriving insights and generating the necessary visualizations and reports.

Our analysis employed a two-layered approach, enhanced by Python scripting for precision and efficiency. Initially, we categorized articles based on keywords like ‘power distribution grid’ and ‘distribution network’, supplemented with domain-specific terms: ‘planning’, ‘operation’, and ‘market’. To ensure the integrity and uniqueness of our dataset, we rigorously removed duplicate papers that appeared across categories. Following the keyword analysis, we examined the temporal distribution of these articles to capture the evolving focus on digitalization over the past decade. This investigation highlights an increasing trend for research publications on this subject, indicative of the escalating integration and significance of digital technologies in this field. Figure 1 illustrates the annual number of publications in this area from 2014 through to 2023, revealing that research interest has more than doubled over the past decade.

The second layer of analysis involved a deeper dive into keywords representing various aspects of digitalization such as ‘Artificial Intelligence’, ‘Machine Learning’, and ‘Optimization’. This phase was critical for assessing how digital technologies are specifically addressed within the literature. The keywords were meticulously selected based on their relevance to digitalization, as determined by an extensive review of the literature and practical experience, ensuring that they closely align with the core aspects of digital transformation in power systems.

Figure 2 reveals the prominence of ‘Optimization’ across all categories in our study, with 5870 mentions in planning and peaking with 9711 in operation, highlighting its crucial role in enhancing grid efficiency and management. ‘Smart Grid’ technologies also feature prominently, with nearly 1959 mentions in planning and 5294 in operation, reflecting the integral role of SGs in advancing grid intelligence. In the market category, ‘Optimization’ and ‘Smart Grid’ are significant again, accompanied by a notable focus on ‘Blockchain’ (617 mentions), underscoring a shift towards secure, efficient energy transactions. These data illustrate the diverse yet focused digitalization efforts within the power distribution grid, showcasing a strong trend towards integrating advanced technologies for improved grid operations and market interactions.

Figure 3 depicts the percentage distribution of these keywords within each category, providing a relative measure of emphasis. Notably, ‘Optimization’ featured prominently, constituting 27.69% of the planning category and an even more significant 22.35% of the operation category, highlighting its critical role in enhancing efficiency and decision-making processes. In the market category, ‘Optimization’ also stood out at 23.32% of the total number of articles, demonstrating its pervasive influence across all sectors. ‘Smart Grid’ technologies followed closely in the market category, with 17.71%, underscoring their growing impact on market-driven energy innovations. It is crucial to note that these percentages are relative to the total number of articles within each respective category and do not sum to 100% due to the common occurrence of articles mentioning multiple keywords.

In the secondary analysis phase, we assessed the multidisciplinary nature of the articles by calculating the percentage within each category that contained more than one specified digitalization-related keyword. This analysis revealed significant thematic overlap: 38.35% in planning, 35.47% in operation, and 46.10% in market. These percentages highlight a strong trend for integrating multiple digital technologies within power distribution grid research, underscoring the complexity and interconnectedness of this field.

Furthermore, a significance threshold of 2% (red dashed line) for keyword percentages was set to focus on the most impactful terms within each category. Keywords surpassing this threshold were further analyzed in the context of the existing literature to evaluate their prominence and applications, as detailed in the references.

Table 1. Significant Key Digital Technologies Exceeding a 2% Impact Threshold in the Power Distribution Literature.

Keywords	References
	Operation/Planning	Market
Artificial Intelligence	[1,4,5,8,13,14,15,16,17,18,19,20]	[21,22]
Automation	[2,23,24,25,26,27,28,29,30,31,32,33,34]	[35,36]
Big Data	[4,5,26,37,38]	[39]
Blockchain	[4,6]	[21,22,35,40]
Cloud Computing	[41]	[42]
Clustering	[9,13,30,43,44,45,46,47,48,49]	-
Deep Learning	[13,15,17,46,48,50,51,52]	-
Demand Response	[53,54,55,56,57,58,59,60,61,62,63,64]	[21,42,65,66,67,68,69,70,71,72,73,74]
Forecasting	[1,13,14,15,16,26,37,46,47,48,50,51,52,75,76,77,78,79,80,81,82]	[70,83]
Game Theory	[63]	-
Genetic Algorithm	[84,85,86]	-
Internet of Things	[4,19,87,88,89,90]	[22,42,70]
Linear Programming	[49,62,91,92,93,94,95]	[96]
Machine Learning	[2,13,15,16,18,26,51,52,77,81,97,98]	[21,22]
Neural Network	[13,14,15,46,76,77,78]	-
Optimization	[1,6,41,47,53,58,60,62,63,77,78,79,85,91,94,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126]	[21,68,96,127,128,129,130,131]
P2P	-	[40,132,133]
Smart Grid	[1,4,10,12,24,25,26,29,33,34,57,60,90,98,134,135,136,137,138,139,140,141]	[21,73,142,143]
Software	[2,16,26,27,58,59,87,94,144,145,146]	[42,83,127,147]
Virtual Power Plant	-	[35,96,127]

collates specific references from this research, offering a detailed exploration of the role of each significant technology within the respective sectors. This approach not only highlights the most influential technologies but also guides future research directions and practical implementations by correlating our findings with documented advancements and applications in the power distribution domain.

This literature review systematically explores digitalization in power distribution grids across three primary sections: grid planning, grid operations, and energy market implications. Each section is subdivided based on an extensive review of the literature and industry insights, highlighting areas like grid expansion and integration challenges. This structured analytical approach dissects digital technologies’ impacts across these pivotal areas and also aims to provide a detailed exploration of how digitalization reshapes each aspect of power distribution systems. As we progress into the body of the review, we will link these technologies to their real-world applications, providing a comprehensive understanding of digital advancements in distribution power systems.

2. Grid Planning

The planning section examines digitalization’s impact on power grid planning, emphasizing the integration of AI, IoT, and big data analytics. It highlights how these technologies transform traditional planning, focusing on enhanced load forecasting accuracy, optimized network design, and improved reliability and efficiency. Figure 4 illustrates the planning section’s structure and its interconnected themes and main points.

2.1. Load and Generation Forecasting in Planning

2.1.1. Load Forecasting

Digital technologies have revolutionized load forecasting (LF) in power system planning by integrating renewable sources and evolving consumption patterns. AI and ML innovations have enhanced LF accuracy, as evidenced by the lower error metrics and setting new precision benchmarks [75]. Hybrid applications of AI and deep learning (DL) can improve power distribution forecasting, demonstrating the potential of diverse computational techniques for better reliability and efficiency [50]. This statement highlights the challenges of integrating advanced forecasting technologies into existing power grids and managing the large volumes of data they produce. It emphasizes the need for a balance between embracing innovation and ensuring practical, reliable grid operation. This balance is crucial to maintain efficiency and effectiveness in power distribution as we adopt new technologies [76]. Future research should focus on refining LF techniques for dynamic power systems to enhance renewable integration and address energy policy implications [13].

2.1.2. Generation Forecasting: Enhancing Predictive Accuracy Generation

Forecasting, particularly for renewable sources like solar and wind, employs advanced modeling techniques to predict energy output. Techniques such as physics-constrained long short-term memory (LSTM) models leverage domain knowledge and historical data to forecast photovoltaic power generation with greater precision [14]. Hybrid models that combine statistical approaches with machine learning, like combining wavelet transforms with support vector machines, further refine forecasting accuracy, especially when handling the variability and intermittency of renewable sources [51,77,78]. The integration of real-time data and continuous model refinement are critical for optimizing the performance of these forecasting systems [79].

2.1.3. Comprehensive Approaches to Forecasting

Clustering-based techniques streamline load forecasting by grouping similar load patterns, which aids in managing the variability from different energy sources and consumer behaviors. Techniques like K-means, hierarchical clustering, combined locally linear embedding (LLE), principal component analysis (PCA), and multi-layer perceptrons (MLPs), enhance accuracy, integrate renewable energy sources (RESs), aid demand-side management (DSM), and bolster SG functions [15,43,44,45,46,80]. Time series load forecasting (TSLF) methods utilize historical data, enhanced by advanced algorithms such as ARIMA and neural networks, to predict future demand. These techniques are pivotal for incorporating renewable energy sources effectively within the grid, supporting demand-side management and facilitating SG functionalities [47,48,52,81].

By advancing both load and generation forecasting techniques, the energy sector can achieve higher efficiency and sustainability, ensuring a reliable and balanced power system capable of supporting diverse energy landscapes.

2.2. Network Design and Expansion Planning

Network design and expansion planning utilizes advanced models and algorithms to adapt the electrical grid for future demands and integrate new technologies. Strategies such as mixed-integer linear programming (MILP), decentralized optimization, and geographic information system (GIS)-based methods are essential for efficient planning.

Enhancements in network reliability and the integration of renewable energy sources (RESs) utilize MILP and GIS to optimize substation configurations and infrastructure [99,100]. The challenges of integrating EVs are addressed using a mixed-integer non-linear programming (MINLP) model, which efficiently manages increasing EV demands [91].

Multistage expansion planning incorporates reliability constraints within an MILP framework to balance cost and reliability [92]. The optimization of energy storage systems (ESS) in network expansions focuses on performance and cost minimization [148]. Decentralized stochastic planning merges with cost-efficient strategies for integrating distributed generation (DG) and ESS, employing advanced methods like second-order cone programming (SOCP) and semidefinite programming (SDP) [53].

Network expansion strategies for demand response (DR) and renewable DG integration use analytical methods to ensure cost effectiveness [101,144]. Urban network optimization, including feeder routes and substation placements, employs programming, GIS, and particle swarm optimization (PSO) to reduce costs and losses [102,149].

2.3. Reliability and Resilience Planning

Reliability and resilience are central to network design and expansion planning, aiming for steady performance and quick recovery from disruptions under normal conditions and natural disasters. Modern planning employs GIS, MILP, Monte Carlo simulations, and ML for a resilient infrastructure. SG integration, predictive maintenance, smart technology, and cybersecurity can enhance reliability and resilience. Reference [150] illustrates how to use these technologies to maintain stability and facilitate recovery.

2.3.1. Role of SGs to Enhance Reliability and Resilience

SGs are pivotal in enhancing power distribution and enabling resilient microgrid development. The optimization of active distribution networks (ADNs) [103] significantly boosts grid reliability and resilience, highlighting the efficiency and cost effectiveness of DERs and adaptive load management. Strategies for optimizing DERs and power lines [104] aim to reduce the costs associated with power interruptions, contributing to stronger microgrid resilience. Additionally, integrating resilience estimation and cost–benefit analysis into microgrid planning [151] showcases the role of SGs in optimizing economic and operational performance in electricity distribution.

2.3.2. Role of Improving Predictive Maintenance to Enhance Reliability and Resilience

Integrating AI, ML, and GIS into predictive maintenance significantly enhances the reliability and resilience of power systems. For resilience against natural disasters, models that assess vulnerability rates aid in planning and reconfiguring networks [152]. Advances in fault detection come from hybrid AI models that combine convolutional neural networks (CNNs) and RNNs, offering quicker and more accurate diagnostics in active distribution networks (ADNs) [16]. Deep reinforcement learning (DRL) is employed to fine-tune resilience strategies, such as upgrading infrastructure to withstand hurricanes [153]. Additionally, tackling the impacts of heatwaves involves logistic regression and reliability models aimed at strategic network renovations, which is exemplified by Milan’s grid [154].

2.3.3. Role of Smart Networks to Enhance Reliability and Resilience

Advanced control technologies in smart distribution networks significantly shorten outage times and improve grid resilience, enabling quick responses to disruptions. This automation ensures reliable power distribution and reduces the impact of interruptions. Smart distribution systems (SDSs) enhance grid reliability and resilience, leveraging technologies like smart meters and remote-controlled switches for sophisticated outage management and fault location, isolation, and service restoration (FLISR) techniques [23].

2.3.4. Cybersecurity Strategies for Enhancing Power System Resilience

Cybersecurity is crucial for digitalized power systems, especially as smart technology and distributed energy resources (DERs) increase system vulnerability. Comprehensive measures including technological solutions, strategic planning, and regulatory compliance are necessary to mitigate these risks [155]. Strategies to combat false data injection (FDI) attacks and the application of AI and ML enhance anomaly detection and system optimization [17,134]. The holistic resilience cycle (HRC) promotes an integrated approach to cyber-physical security, spanning prevention to recovery to comprehensively strengthen grid defenses [156]. Securing information and communication technology (ICT) components like SCADA systems and DERs is vital and requires rigorous vulnerability assessments and adherence to standards, such as those from the National Institute of Standards and Technology (NIST) [157,158]. The challenges of DER integration highlight the need for sophisticated modeling and risk assessment tools for effective defense strategies [54].

2.4. Regulatory and Compliance Planning

Digital technologies significantly influence grid operations; they align with regulatory, safety, and environmental guidelines while improving efficiency and supporting sustainable management. The mixed-integer bilevel linear program (MIBLP) model [159] addresses distribution system planning within regulatory constraints, highlighting the impact of regulatory policies on utility operations and facilitating the adoption of new technologies.

The integration of DG brings to light the need for revised regulatory frameworks to accommodate its growth [160,161], with the strategic considerations for regulatory compliance and environmental and safety impacts discussed in [105]. This reveals the intricate relationship between DG strategies and regulatory standards, which is essential for the evolving power distribution scene. Further discussions in [55,162] emphasize the necessity for policies promoting network flexibility to manage DG investments. Digital technologies stand out for their role in boosting system efficiency and ensuring the adaptability required to meet ongoing and emerging challenges.

2.5. Economic, Financial, and Asset Management Planning

The economic and financial aspects are pivotal for the sustainable and equitable progression of grid planning, highlighting the need to balance financial health with fair cost allocation among stakeholders through cost–benefit analysis and strategic investments.

2.5.1. Economic and Financial Planning

The financial sustainability of grids emphasizes equitable cost sharing, using cost–benefit approaches and strategic investments. Digital simulations that merge performance with economic factors [56] utilize tools like sequential Monte Carlo simulation and three-phase power flow to balance service quality with economic efficiency. A multi-level planning strategy within clustered microgrids in Ref. [106] focuses on optimizing energy coordination using advanced optimization and DR for efficient energy transaction management. The role of decentralized energy storage is explored in [163], which highlights its importance in enhancing system reliability and cost efficiency, particularly with RES variability. Additionally, an incentive-based model for smart distribution systems in [164] employs MILP to manage uncertainties in system demand and distributed generation output, showcasing how digital tools can support economic strategies in power distribution.

2.5.2. Asset Management Planning

In SG power distribution, effective asset management is key to addressing aging infrastructure, fluctuating load demands, and the integration of renewables. Digitalization increases complexity, so requires sophisticated management tactics and flexible operational frameworks [18,165]. Predictive maintenance, enhanced by failure modes and effects analysis (FMEA) and applications like fuzzy logic, improves system reliability by identifying potential failures and facilitating preventive measures [18,24]. However, the growing complexity of SGs, particularly regarding the intertwined nature of cyber and physical systems, is challenging for traditional asset management methods. Future strategies will likely leverage data analytics for improved decision making [166]. Previous research has emphasized the importance of real-time data in managing aging assets and integrating renewable energy sources, underscoring strategies for efficient and reliable power provision [57].

As the sector develops, the role of digitalization in asset management, supported by FMEA and machine learning enhancements, is vital for maintaining system reliability and operational efficiency. Upcoming asset management approaches in power distribution will increasingly rely on advanced analytics and emerging technologies to meet the sector’s dynamic requirements.

2.6. Sustainability and Environmental Planning

This section is an in-depth exploration of integrating RESs into grid planning, emphasizing a commitment to sustainability and environmental stewardship. It begins by highlighting the alignment with broader sustainability objectives and compliance with environmental regulations, showcasing a shift towards cleaner energy and a nuanced understanding of renewable integration’s effects on grid dynamics.

Various studies [93,167] have focused on incorporating carbon emission trading and DR in planning, underscoring the importance of environmental considerations in energy systems.

The narrative deepens when considering [107], which stresses the need for coordination between district developers and utilities, particularly in sustainable, smart districts. This study explores the challenges and opportunities in managing energy across multiple scales, emphasizing the role of digital tools in optimizing energy distribution and handling complexities in urban multi-energy systems. Additionally, Ref. [108] introduces a multi-criteria optimization strategy that accounts for economic, environmental, and social factors. This approach highlights the benefits of decentralized generation in reducing CO2 emissions and increasing self-sufficiency, solidifying the role of sustainability in energy planning.

2.6.1. Renewable Energy Integration

Integrating renewable energy into power distribution is essential for advancing sustainability and efficiency in grid operations, offering both economic benefits and posing challenges. A model in [49] illustrates the cost reductions associated with increased DG penetration, emphasizing economic incentives. Studies [109,110] aim to optimize investment and operational expenses, minimizing integration challenges through sophisticated mathematical models.

The necessity of robust modeling to manage the variability of wind and solar power is underscored in [84] and involves capacity credit assessments of RESs and stochastic methods for precise planning. Addressing the variability of wind and solar energy requires advanced stochastic and probabilistic modeling to maintain grid stability and efficiency.

The transition to a renewable-focused energy system faces hurdles like RES intermittency. Solutions like energy storage and demand management become crucial [168], introducing algorithms for the strategic integration of RESs and battery energy storage systems (BESSs) based on the genetic algorithm (GA) to enhance grid reliability.

Digitalization plays a vital role in renewable integration, with studies highlighting AI’s and big data’s impact on optimizing grid planning and operations [19], modernization strategies for handling renewable influx [111], and analyses for economic and environmental sustainability [169].

Renewable integration, alongside predictive maintenance and analytics, prepares distribution systems for the changing energy landscape. Despite initial investment and intermittency challenges, the benefits include sustainability, a reduced environmental footprint, and potential cost savings, highlighting the need for strategic planning and digital technology deployment for a resilient and sustainable power system.

2.6.2. Battery Energy Storage Planning

BESSs are key to transitioning towards sustainable power systems, enhancing renewable energy use, and reducing CO₂ emissions. They address the intermittent nature of solar and wind power, ensuring a reliable energy supply, and are positioned as a cornerstone of the green power revolution.

Study [112] highlights the role of BESSs in mitigating renewable energy intermittency and demand unpredictability, improving network performance and voltage regulation and minimizing power losses. This positions BESSs as strategic solutions for managing renewable energy variations, having benefits for both the economic and environmental aspects of power systems.

The research in [113] introduces a two-stage optimization for integrating RESs and BESSs within distribution networks, aiming to reduce investment costs and improve voltage profiles and demonstrating the importance of coordinated planning and digital tools.

Further discussions in [170] explore BESSs’ contributions to generation capacity expansion and decarbonization, while [94] presents a planning method for a distributed BESS that addresses uncertainties in load forecasts and renewable outputs, enhancing power system efficiency. Additionally, Ref. [171] examines BESSs’ role in grid stability in the global south and [114] investigates the simultaneous optimization of DG and BESSs for balanced reliability, cost efficiency, and sustainability in network planning.

BESS integration marks a significant step in enhancing power system sustainability and efficiency, stabilizing the grid, and optimizing network performance. Strategic planning and advanced optimization make BESS investments more effective, significantly aiding voltage regulation and decarbonization efforts. Despite initial cost and regulatory challenges, BESSs are vital for a resilient and sustainable energy future.

2.6.3. Electric Vehicle Planning

EVs are pivotal in steering modern power distribution systems towards reduced carbon emissions, supporting global sustainability initiatives. Their integration, particularly through vehicle-to-grid (V2G) systems, enhances grid technologies, bolstering stability but also introducing new challenges due to their increasing prevalence.

Optimizing the placement of EV charging stations is crucial, with study [114] employing PSO to determine the optimal locations, leading to decreased power losses and better voltage profiles and highlighting the significant influence of infrastructure placement on grid efficiency.

The advancements in optimization techniques for dynamic EV charging demand prediction and voltage stability are explored in [85,115,116], showcasing the sophistication of current methods in managing demand, ensuring system stability, and maintaining cost efficiency and emphasizing digital technologies’ importance in EV infrastructure planning.

Further discussions in [86,117,172] delve into optimal operational planning and the broader impacts of EV integration on distribution systems, providing comprehensive insights into EV infrastructure development and operational strategies.

This narrative underscores EVs’ critical role in transforming power distribution, spotlighting advanced planning and digital innovations in enhancing EV infrastructure’s efficiency and sustainability. It presents a thorough synthesis of research on EV integration, from optimizing infrastructure to addressing grid stability and cost challenges, indicating the need for sophisticated forecasting models and regulatory adjustments to support EV integration’s expanding demands.

2.6.4. Energy Community Planning

The transition to sustainable, localized energy generation and consumption is increasingly powered by grassroots energy communities, blending democratic participation with renewable energy efforts, which is significantly aided by digital platforms for energy sharing and management.

The research in [173] explores smart energy community development, spotlighting the role of smart technologies like blockchain in optimizing community-based energy systems. Another study [135], discusses the planning of a sustainable district in Sweden and examines diverse stakeholder views on citizen energy communities (CECs) in decentralized energy solutions through a sociotechnical perspective, highlighting the potential for decentralized energy models.

The critical function of ICT in supporting energy communities is the focus of [174], which illustrates how digital tools aid energy management and market participation, emphasizing ICT’s role in facilitating a sustainable and decentralized energy transition.

Additionally, Ref. [175] presents a digital platform with a one-stop-shop (OSS) architecture aimed at enhancing community energy projects by simplifying stakeholder interactions and project management, further advancing the energy transition agenda.

Together, these discussions spotlight the shift towards community-driven, decentralized energy systems, stressing the merger of digital and renewable technologies for improved energy management and distribution. Future directions will involve enhancing digital sharing platforms and community engagement strategies, navigating regulatory landscapes to support the global shift towards more sustainable decentralized energy systems.

2.7. SG Integration in Planning

The transition to smarter power systems is driven by advancements in digital technologies, reshaping the planning of distribution systems. Key research highlights the use of AI and ML, including artificial neural networks (ANNs), to enhance tasks like voltage monitoring, generation output forecasting, and grid failure mitigation across various operational domains [25].

2.7.1. Economic Impact in SG Implementation

SG technology demonstrates economic benefits through DSM, energy efficiency, and DG integration. For example, in Oman, SGs have reduced the peak energy demand, showcasing the value of SG maturity models in strategic planning [136]. Additionally, they provide innovative solutions for forecasting and enhancing operational reliability, which has significantly improved the economic performance and system reliability in Boao Town, Hainan [26].

2.7.2. Technoeconomic Analysis in SGs

Technoeconomic analysis in SG planning balances technical feasibility with economic viability, optimizing grid and storage expansion investments to reduce costs, especially in medium- and low-voltage grids [58].

2.7.3. Smart Network Impact and Planning Approaches in SGs

As smart distribution networks evolve, novel planning models are required to accommodate bi-directional flows and interconnected operations. Various reviews have highlight a shift towards integrated multi-objective optimization models in SG planning [118,137]. The integration of EVs and new technologies necessitates sustainable operational strategies that harmonize renewable integration with grid reliability, with technologies like digital twins and agent-based simulation playing pivotal roles [138].

Furthermore, the integration of DERs into grid planning is aimed at optimizing operations and utilizing network capacity efficiently, impacting service reliability [139]. Overall, SG development focuses on challenges and opportunities including interoperability, network communications, and renewable energy integration, which is crucial for achieving efficiency and environmental sustainability while managing cybersecurity challenges in DR and grid operations.

3. Grid Operations

The operation of modern distribution grids, crucial for reliable and efficient electricity delivery, has evolved with the integration of decentralized sources and RESs, transforming them into dynamic networks. Key to this transformation are digital tools like smart sensors, the IoT, AI, and big data analytics, which enable real-time monitoring, predictive maintenance, and improved decision making. These technologies enhance grid reliability, efficiency, and cybersecurity while facilitating renewable integration and boosting customer engagement. Thus, digital tools are vital for achieving a resilient, sustainable, and customer-centric energy future. The grid operation subsections and key points are shown in Figure 5.

3.1. Grid Monitoring and Control in Operation

Effective grid monitoring and control are crucial for the operational efficiency of modern distribution grids. Utilizing advanced digital tools like smart sensors, the IoT, and real-time data analytics enables grid operators to achieve precise control and real-time oversight. This integration ensures a reliable electricity supply, an optimal response to fluctuating demand, efficient resource management, the maintaining of grid stability, and addressing challenges in the evolving energy landscape.

3.1.1. Real-Time Monitoring and Digital Platform Monitoring

Real-Time Monitoring

Modern distribution systems enhance operational efficiency through effective real-time grid monitoring by utilizing digital technologies. One method uses low-voltage measurements for medium-voltage waveform analysis to improve monitoring accuracy and enhance fault detection and grid stability in the presence of DERs [176].

Further innovations include the development of AC/DC switching technology and digital platforms that integrate intelligent power equipment into the power IoT ecosystem, enhancing features like real-time temperature detection [87]. The IoT’s versatility extends to systems aimed at preventing food spoilage through real-time anomaly detection [88] and web-based tools for accessible power quality monitoring [177].

Edge computing is also employed to boost grid operation efficiency and reliability [145], whereas continuous power quality monitoring in ADNs emphasizes quick fault detection [178].

These advancements underscore the importance of real-time monitoring in improving smart grids’ reliability and security, addressing challenges like data complexity and cybersecurity and fostering a more efficient energy landscape.

Advanced Digital Monitoring in Power Distribution: SCADA and IDMS Platforms

Advanced digital monitoring technologies like SCADA (supervisory control and data acquisition) and integrated distribution management systems (IDMSs) are crucial for the evolution of power distribution systems, driving improvements in grid efficiency and reliability. SCADA systems facilitate real-time data collection, remote control, and network communications, which are essential for the operation of modern grids.

The SCADA-based approach reported in [179] enhances real-time voltage stability monitoring, streamlining grids for better renewable energy integration. Studies on the health of SCADA infrastructure in power transmission [146,180] utilize SNMP for proactive management and early fault detection.

IDMSs, which integrate outage management and smart meter data, modernize grid management systems. A case study involving Unareti, the DSO for Milano and Brescia, showcases the retrofitting of substation monitoring with middleware to improve sensor integration, highlighting challenges such as data exchange latency.

These advancements emphasize the vital impact of SCADA and IDMSs on modern grid management, addressing operational efficiencies and the challenges of integrating new technologies and marking significant steps towards a resilient and sustainable power infrastructure.

3.1.2. Grid Control

With the increasing integration of DERs and digital advancements, grid control is becoming more crucial in power distribution networks. The concept of grid-edge control is reported upon in [181], it proposes integrating traditional grid systems with the autonomous control of DERs to effectively address integration challenges.

This approach encompasses various control architectures and operational layers, including hierarchical coordinated control strategies for PV inverters, as detailed in [119]. Such multi-layer mechanisms enhance voltage quality and reactive power management, illustrating the importance of SGs and digital tools like smart meters and vehicle-to-grid systems in grid control and operation and improving data acquisition and connectivity. Furthermore, Ref. [119] stresses the role of digital metering, communication technologies, and cloud computing in real-time grid monitoring and management. An optimized power control strategy for grid-connected PV inverters is introduced in [120], showcasing efficient inverter design, while [182] explores real-time electrical energy monitoring in distribution networks using modern metering devices and automated controls.

These discussions highlight the critical role of advanced control strategies and digital technologies in enhancing grid efficiency, reliability, and renewable integration, propelling power distribution towards more sophisticated, reliable, and efficient solutions.

3.2. DR and Load Management in Operation

In the modern power system landscape, DR and load management are pivotal in making electricity consumption more adaptive and efficient. DR dynamically adjusts electricity use in response to grid conditions and pricing, helping balance supply and demand, enhancing grid reliability, and offering potential economic benefits. Load management, on the other hand, optimizes energy use for system stability and efficiency, which is crucial for managing peak demands and renewable energy integration. Together, these strategies are vital for achieving a sustainable, resilient energy framework.

3.2.1. Demand Response

DR strategies, enhanced by digital advancements, are increasingly recognized for their ability to improve grid efficiency, manage peak loads, and reduce carbon emissions. Smart meters and specialized software are crucial for addressing challenges such as reverse power flow from distributed generators, which was highlighted by a successful DR initiative by an Italian DSO [59,60].

The review in [59] evaluates residential DR as a cost-effective alternative to traditional network upgrades, discussing activation strategies, challenges, and the future potential of residential demand responsiveness. Another study [61], examines the impact of bidirectional digital communication on DR management (DRM) programs, aiming to reduce operational costs and emissions through optimized DR programs.

Further research, reported in [62], introduces risk-based planning tools to address uncertainties brought by new technologies like DERs, using a robust linear programming model for a DR that showcases the potential economic benefits. Additionally, a game theoretic model [63] explores competition among demand response aggregators (DRAs) to stabilize the grid amidst renewable energy fluctuations.

Despite the benefits of DRs, challenges such as consumer hesitancy and the need for enhanced stakeholder collaboration persist. Future research will focus on evaluating the sector-specific efficacy of DR and the impact of advanced communication on market efficiency, particularly in developing countries. This underscores the need for strategic engagement, technical advancements, and pricing strategies to make DR economically viable, highlighting the critical role of digital tools in enhancing grid performance and sustainability.

3.2.2. Load Management

The evolution of digital tools has significantly enhanced load management in distribution power systems, as evidenced by recent studies. A power distribution network management system (DNMS), leveraging the Spring Boot + SSM framework and LSTM-based load forecasting with big data technology, offers comprehensive power system analysis for improved control and risk management, excelling in urban distribution complexities [37].

A cloud-based intelligent power management system is explored in [41] that utilizes analytics for balancing control signals and integrating demand power management with renewable energy, resulting in decreased consumption and costs through cloud optimization.

A toolbox designed for incorporating RESs into distribution networks [121] employs heuristic optimization to simulate energy storage and optimize power output, boosting operational efficiency. Additionally, a load power and energy management system [27] utilizes Proteus Visual Design software and Arduino Mega 2560 for advanced metering and load control that is suitable for various applications.

These studies highlight the transformative impact of digital advancements on power distribution load management, emphasizing the importance of innovative tools in system management, control, and optimization, particularly with renewable integration.

3.2.3. Customer Participation in DR and Its Operational Challenges

Customer participation in DR contributes to grid stability, especially with the increasing integration of RESs. It offers a two-fold benefit: helping to balance the electricity supply and demand and providing economic incentives to consumers. The effectiveness of these tools in promoting customer participation in DR is evident in their ability to manage reverse power flow, as shown in the real-life DR campaign discussed in [59].

Challenges in customer DR participation include engagement and awareness issues [60], technological integration complexities, unpredictability of response [63], data security concerns, and ensuring equitable access. Overcoming these necessitates collaboration among utilities, policymakers, and technology providers, with the development of risk-based tools [62] and innovative engagement strategies being crucial. Addressing these challenges is essential for leveraging DR’s full potential for a sustainable grid.

3.3. Renewable Energy Integration in Operation

As power systems globally transition towards RESs, the role of digital technologies is becoming increasingly vital in addressing the operational complexities this shift entails. This section explores the challenges and solutions related to the high penetration of renewables, emphasizing how digital tools and advanced technologies are key to effective integration and management.

3.3.1. Operational Challenges Posed by High Penetration of Renewables

The integration of renewables, as discussed in [122], introduces challenges such as variability, low inertia, and power quality issues, whereas the solutions involve various technologies and strategies, including advanced control systems, optimization techniques, and energy storage solutions. Digital technologies, particularly advanced data analytics and predictive modeling, play a crucial role in forecasting and managing the variable nature of RESs. Additionally, Ref. [183] highlights the need for sophisticated climate modeling tools to anticipate and mitigate the effects of climatic variability on renewable production. Methodologies from the reviewed studies include statistical analyses, power spectrum density plots, and correlation assessments of climate variables and renewable energy production.

3.3.2. Management of Grid Stability and Power Quality with Intermittent Renewable Sources

Addressing grid stability and power quality amidst fluctuating renewable energy sources is critical. Study [184] illustrates the utility of ISS-theory-based digital simulation tools for evaluating grid stability across diverse renewable penetration rates. The case study in [28] demonstrates how digital control systems in energy storage that offer synthetic inertia and fault current support can fortify grid stability in renewable-dense networks. Additionally, Ref. [123] showcases the integration of advanced computational methods with digital simulation tools and adaptive rat-swarm optimization (ARSO) for optimizing static VAR compensators (SVC) and power system stabilizers (PSSs). This synergy between traditional digital tools and innovative optimization strategies is vital for ensuring a stable, high-quality power supply in renewable-rich grids, paving the way for a resilient energy future.

3.3.3. Advanced Control Systems for Managing Renewable Sources

Study [185] emphasizes the necessity for dynamic, adaptive control systems for the renewable energy shift, presenting a complex dynamic optimization model as a multi-stage decision framework. This model integrates solar energy and storage capacities, fossil fuel use, and electricity consumption, aiming to maximize utility within carbon budget and technological constraints. It addresses solar variability with probabilistic functions and analyzes the impact of cloud coverage on solar generation, offering insights into optimizing solar energy systems amidst environmental fluctuations.

3.4. Outage Management and Reliability in Operation

In the evolving landscape of power distribution systems, the integration of specific digital tools such as smart meters, remote sensors, advanced data analytics, and automated control systems is playing a pivotal role in enhancing operational efficiency and reliability. This section delves into the recent advancements in outage management and reliability, focusing on strategies for the quick detection and restoration of power outages and the application of automated switchgear and reclosers. The referenced studies illustrate the transformative impact of these specific digital technologies in optimizing power distribution system operations.

3.4.1. Strategies for Quick Detection and Restoration of Power Outages

The shift in power distribution system operations increasingly focuses on rapid outage detection and restoration by leveraging data-driven technologies. Smart meters and remote fault indicators are pivotal for precise fault localization [95], utilizing digital tools for real-time data analysis. Distribution automation plays a key role in service restoration [29], which is crucial for quick network re-energization.

Studies [186,187] highlight the importance of fault diagnosis and prognosis through data analytics, offering methods for pinpointing faults in networks with DGs. Further research in [30,188] discusses advanced data analytics for fault diagnosis and a digital approach for locating faults in DG-integrated networks, respectively. Together, these studies underline the transformative impact of data-driven practices and automation in improving outage detection and recovery in power distribution systems.

3.4.2. Automated Switchgear and Reclosers for Improving Outage Response

The deployment of automated switchgear and reclosers plays a crucial role in enhancing outage management within distribution networks. The research in [31] underscores a digital strategy utilizing coordinated recloser–fusesaver systems, notably improving outage management by minimizing the frequency, duration, and customer impact of outages.

Further analysis in [32] explores a model assessing the impact of automated fault location, isolation, and service restoration on system resilience, particularly under disaster conditions, leveraging digital simulations for performance metrics. Study [124] investigates the strategic allocation of protective devices, including digital reclosers, adopting an MILP methodology to boost reliability in networks with DGs.

Additionally, Ref. [189] highlights the importance of strategically placing auto-reclosers to improve system reliability and efficiency, which is especially pertinent in network remodeling, using network modeling and simulation for optimal placement decisions.

Together, these studies illustrate the significant benefits of incorporating automated switchgear and reclosers in power distribution networks, underscoring a shift towards more resilient and efficient outage management.

3.5. Asset Management and Maintenance in Operation

The rapid digital evolution in power distribution necessitates integrating digital tools for asset management and maintenance, addressing system complexity and demands for efficiency, reliability, and sustainability. This section examines digitalization’s crucial role in transforming asset management strategies, focusing on predictive maintenance, AI-driven asset health monitoring, and strategic maintenance planning to reduce disruptions. Highlighting research and practical examples, it shows the impact of digital methodologies on power system management, from ML to advanced sensors, demonstrating how digital innovations are tackling current challenges and future demands.

3.5.1. Predictive Maintenance Strategies for Grid Infrastructure

The advancement of predictive maintenance in power grid infrastructure utilizes advanced data analytics, sensor technologies, and ML to pre-empt potential failures and refine maintenance schedules. Study [190] explores statistical, ML, and AI methods for time series data analysis, highlighting their effectiveness in pre-emptive maintenance strategies to boost grid performance and cost efficiency. Simultaneously, Ref. [97] examines advanced sensor technologies for monitoring switchgear, focusing on thermal, mechanical, and discharge detections integrated with ML for accurate maintenance predictions.

Additionally, Ref. [98] tackles the challenges posed by DERs and rapid power electronics, promoting real-time predictive maintenance and failure prediction using cyber physical systems (CPSs) and digital twin models, which is essential for modern power distribution complexities. Study [191] addresses the surge in EV charging demand, suggesting a deep reinforcement learning approach for timely distribution transformer replacements to meet operational demands and maintain grid reliability amid changing energy landscapes.

These discussions underline the pivotal role of predictive maintenance, driven by digital innovations, in ensuring grid reliability and efficiency.

3.5.2. Analytics and AI in Asset Health Monitoring

The study in [89] underlines the significance of utilizing low-cost, non-intrusive sensors combined with AI for asset health monitoring. These technologies provide continuous monitoring and real-time analysis that is essential for predictive maintenance. The integration of AI algorithms can process large datasets from sensors, facilitating early detection and proactive maintenance strategies. This approach not only ensures system reliability but also significantly reduces maintenance costs by avoiding unplanned outages and expensive repairs.

3.5.3. Planning of Maintenance Schedules to Minimize Operational Disruptions

In the study in [192], the importance of SG infrastructure and online monitoring devices in maintenance planning is emphasized. These tools provide crucial real-time insights for optimizing maintenance schedules. The review suggests that the integration of predictive analytics and advanced optimization algorithms enables utilities to make informed decisions, thereby effectively managing assets and optimizing maintenance activities.

Ref. [193] discusses the use of comprehensive digital tools, including remote monitoring systems, analytical software, and predictive algorithms. These tools enhance the effectiveness of asset management plans, enabling the efficient tracking and monitoring of assets. The paper highlights how real-time automation technologies and big data processing with predictive and prescriptive analytics play crucial roles in the decision-making processes related to asset operation and maintenance.

3.6. Voltage and Reactive Power Control in Operation

In modern power distribution systems, managing voltage and reactive power is essential for system stability, efficiency, and economic operations. Digital tools for voltage and reactive power control are at the forefront of enhancing these aspects. A comprehensive review in [20] discusses various control algorithms, including evolutionary, physical, and swarm algorithms, and highlights the significance of intelligent data analysis and sophisticated optimization tools in the renewable energy era.

3.6.1. Methods for Maintaining Optimal Voltage Levels in the Grid

For maintaining optimal voltage levels, Ref. [125] explores the integration of on-load tap changers (OLTC), voltage regulators (VR), capacitor banks (CB), and DGs through a mixed integer second-order cone programming (MISOCP) model with model predictive control (MPC). This method aims to minimize operational losses and sustain optimal voltage, utilizing linear programming and machine learning for better stability and response.

Furthermore, Ref. [194] details a centralized control strategy for ADNs that merges proportional integral and corrective control via MPC. This strategy benefits from real-time data and forecasts to enhance system adjustments, demonstrating MPC’s role in digital grid management to address voltage and quality issues proactively, thus ensuring stability and efficiency.

3.6.2. Management of Reactive Power to Improve Efficiency and Reduce Losses

Managing reactive power is crucial for operational efficiency in power systems. Study [195] introduces a coordinated volt-var controller (VVC) that employs a sensitivity matrix-based method, enhancing voltage and reactive power management. This approach leverages digital tools for sensitivity analysis and control optimization, simplifying complex calculations and improving decision making. The research emphasizes the ability of digital solutions to reduce computational complexity in traditional volt-var control, supporting real-time applications in smart distribution networks.

Additionally, a digital approach to voltage regulation in ADNs using a distributed control system with predictive analytics to anticipate renewable outputs and load shifts that enables proactive voltage control has been established. It integrates OLTCs and a distributed algorithm for coordinating reactive power from DERs by applying the ADMM optimization method. This underscores digitalization’s critical role in bolstering grid stability and efficiency, particularly in systems with high renewable energy penetration [126].

3.6.3. Automated Voltage Control Systems for Efficient Operation

Automated voltage control within low-voltage distribution networks can leverage advanced digital techniques. It features a control algorithm embedded in a voltage source converter (VSC) for effective regulation, utilizing both reactive and minimal active power at the point of common coupling (PCC). This approach addresses traditional inefficiencies, with a digital signal controller (DSC) underscoring the pivotal role of digital technology in optimizing power flow and enhancing voltage stability and quality in the complex terrain of low-voltage networks [196].

The integration of digital strategies, including MPC, volt-var controllers (VVCs), OLTCs, and DGs, signifies substantial progress in voltage and reactive power management. These developments indicate a transition from conventional to dynamic, real-time control mechanisms that is supported by SG technologies. This evolution focuses on maintaining grid stability and facilitating efficient RES integration.

3.7. Metering, Billing, and Customer Service in Operation

The integration of digital technologies, especially smart meters, is reshaping metering, billing, and customer service in distribution power systems, marking a shift towards more efficient, reliable, and customer-oriented operations. Study [82] underlines the intricate analysis enabled by smart meter data, from load forecasting to detecting non-technical losses (NTLs), emphasizing the role of smart meters in enhancing power network operations. Additionally, Ref. [64] delves into direct load control features within smart meters, improving demand management and customer engagement through remote connectivity services.

The research in [197] introduces an IoT-based framework for real-time power quality measurement, advancing metering technology by providing immediate insights into power metrics. Study [198] explores advanced metering infrastructures leveraging real-time data for identifying technical losses and preventing unauthorized energy consumption, showcasing the vast potential of digital tools for optimizing energy distribution.

Furthermore, Ref. [38] discusses the transformative impact of big data analytics on SGs, linking operational enhancements in metering and billing to improvements in customer service. This connection highlights the crucial role of digital innovations in driving more streamlined, responsive, and sustainable power distribution systems.

The adoption of digital tools across metering, billing, and customer interactions signals a significant move towards modernizing power grids, enhancing functionality from smart metering to IoT-based monitoring and big data analytics. These advancements collectively drive operational excellence and consumer satisfaction. Despite challenges like privacy and cybersecurity, ongoing research is dedicated to addressing these issues, with the goal of boosting consumer engagement and refining grid optimization for an automated, customer-centric energy landscape.

3.8. Cybersecurity and Grid Security in Operation

Modern power grids, especially those transitioning to SG technologies, face significant cybersecurity challenges. The integration of information and communications technologies, as highlighted in [140], has transformed electricity distribution networks, necessitating advanced security mechanisms for reliable operation. Similarly, Ref. [90] emphasizes the vulnerabilities introduced by digital evolution, particularly the susceptibility to cyberattacks like false data injection and distributed denial of service (DDoS).

3.8.1. Strategies for Protecting Grid Control Systems from Cyber Threats

The importance of a comprehensive approach for safeguarding grid control systems is apparent, which can be achieved by merging traditional security practices like data encryption with sophisticated defense-in-depth strategies, as suggested in [140]. Furthermore, Ref. [33] highlights the effectiveness of lightweight authentication in SGs, aimed at minimizing the computational demands while preserving robust security. The integration of advanced authentication protocols, the use of hash functions, and symmetric cryptography are proposed as efficient methods to bolster SG communication security, ensuring high operational performance without sacrificing security integrity.

3.8.2. Implications of Data Breaches and Security Incidents on Grid Operations

Data breaches and security incidents have severe implications for grid operations. As noted in [199], cyberattacks can lead to overloaded transformers and equipment damage, impacting the lifecycle of the equipment and the reliability of the grid. Ref. [34] elaborates on this, explaining how cyberattacks can disrupt normal operations and lead to system blackouts.

3.8.3. Digital Tools and Operation Approaches

Studies have emphasized the critical role of digital tools, including intrusion detection systems [141] and ML alongside digital twin technologies [200], in enhancing cybersecurity for power grids. The integration of advanced ML and AI methods is crucial for detecting and mitigating dynamic cyber threats, which advocates for a blend of new technologies with traditional security measures to ensure the resilience of SG operations against evolving cyber risks.

4. Energy Market

The market analysis section discusses how digitalization is revolutionizing power distribution markets through innovative technologies and business models. It highlights the role of digital tools in enhancing market efficiency, transparency, and competitiveness and considers the implications of integrating renewable energy sources, DR initiatives, and changes in regulatory frameworks. The challenges and opportunities these integrations present are explored, underlining the necessity for markets to adapt to digital transformation. The section concludes by visualizing the subsections and key points in Figure 6.

4.1. Structures and Regulations in the Electricity Market

The electricity market is rapidly evolving due to digitalization, which affects market structures and regulatory frameworks. This shift necessitates re-evaluation, as technologies like smart grids and the IoT reshape how electricity is managed and consumed.

4.1.1. Impact of Digitalization on Market Models and Distribution Utilities

Digitalization transforms electricity distribution, integrating renewable sources and prosumers and altering DSO–consumer dynamics [142]. The associated challenges include resistance to new technologies and uncertainties about the impact of smart meters. The study in [39] highlights the necessity for the electric power sector to continually adapt by using digital tools to navigate market reforms and balance stakeholder interests.

4.1.2. Digital Tools and Regulatory Influence

Digital tools enhance market operations and consumer relations, with studies like [42] supporting analytical approaches for more dynamic markets. Meanwhile, Ref. [161] discusses managing DERs’ challenges through digital technologies, stressing the need for regulatory adaptations to keep pace with technological advancements and support integration while addressing risks.

Digitalization enhances efficiency, sustainability, and customer interaction in the electricity market but challenges the existing structures and regulations. A shift towards dynamic, integrated management frameworks is crucial to handle these complexities [201]. This includes adjusting market models and updating regulations to fully embrace digital technologies, addressing specific issues like data privacy and cybersecurity for a successful transition.

4.2. DR in the Energy Market

DR is increasingly vital in modern power systems as it offers a dynamic approach to managing electricity demand. Enhanced by digital advancements like AMI and bidirectional communication, DR has seen substantial improvements in its effectiveness. The regulatory environment, crucial for DR adoption, varies across regions, with study [65] analyzing Europe’s scene and identifying obstacles and enablers for DR.

4.2.1. Mechanisms and Technologies in DR

This section examines DR mechanisms, emphasizing the essential role of digital technologies in SGs. Digital tools enable strategic energy consumption decisions and leverage optimization methods such as convex optimization, game theory, and dynamic programming to enhance load management. Study [60] highlights the significance of bidirectional digital communication and distributed energy management systems for sustainability and cost reduction. Ref. [66] discusses integrating DR within capacity remuneration mechanisms, advocating for differentiated payments tailored to DR technologies’ roles and emphasizing digital tools in DR activation and operation. It critiques traditional mechanisms while praising digital solutions for improving market responsiveness and DR’s economic viability. Furthermore, Ref. [21] explores the integration of AI and ML in DR, discussing their role in user engagement, price optimization, and device control, and reviews applications including blockchain to underscore its substantial influence on DR efficiency and SG management.

4.2.2. Market Integration and Economic Models of DR

The integration of DR into electricity markets involves sophisticated economic models and digital technologies to optimize operations and incentives. Study [67] introduces a distributed DR market clearing algorithm using cloud computing for computational efficiency, showcasing the ability of digital tools to handle market complexities. It also discusses a data-driven market strategy to coordinate dispatch between virtual power plants (VPPs) and DR aggregators, utilizing digital data processing to boost market operation efficiency. This approach employs a two-stage framework with noisy inverse optimization to accurately estimate customer load response, facilitating economically viable and efficient DR operations [68]. Additionally, Ref. [21] highlights AI’s role in managing aggregator clustering problems and optimizing pricing and scheduling via multi-agent systems, which significantly improves economic and operational performance by predicting market trends. Future DR strategies may incorporate advanced ML models and game theory to navigate energy supply and demand challenges, emphasizing the need for addressing data privacy, system integration, and scalability to adapt to evolving energy and market demands.

This streamlined version focuses on categorizing DR’s impact through digital tools and economic models, maintaining fluency and cohesion while highlighting the transformative influence of AI and ML on DR strategies and market integration.

4.3. Distributed Energy Resource Integration in the Energy Market

This introduction sets the stage for understanding DERs, emphasizing their diversity, as DERs include solar panels, wind turbines, and battery systems. Ref. [127] is crucial here. It highlights the transformative role of DERs in modern energy systems and not just their technological diversity. This section underscores the transformative role of digitalization in enhancing the capabilities of DERs for real-time control and optimization of energy networks.

4.3.1. Market Dynamics of DERs

Focusing on the market dynamics of DERs, this section delves into the role of VPPs in aggregating DERs for effective market participation. Ref. [35] is pivotal in illustrating how blockchain technology facilitates this aggregation and reshapes market structures beyond energy trading and management. It explores how blockchain can automate and digitalize the control of DERs. Additionally, Ref. [96] provides a foundational reference for future VPP optimization strategies.

4.3.2. The Role of Digital Tools in DER Integration

This section underscores the significance of digital tools, especially blockchain, in integrating DERs into power systems. Ref. [40] showcases blockchain’s potential to enhance the security, transparency, and efficiency of energy transactions within microgrids, presenting a decentralized market model and a case study on its advantages for energy management and community self-sufficiency through battery storage.

Furthermore, Ref. [83] discusses the integration of DERs using the transactive energy models facilitated by blockchain. This approach addresses the challenges of the intermittency of renewable energy sources and the need for a supply–demand balance. The paper highlights the importance of intelligent agents in maintaining equilibrium in networks with high DER penetration, showcasing the essential role of digital technologies in modern energy management and market integration.

4.3.3. Impact of DER Integration on Energy Markets

This section delves into how the integration of DERs impacts power markets, with Ref. [69] focusing on the transformation in consumer roles and market dynamics. It examines the adoption of solar panels, micro-cogeneration units, EVs, and distributed storage, illustrating how consumers are becoming active energy providers that affect both the distribution network and wholesale market. The paper also touches on the cybersecurity and resilience issues related to DER integration, advocating for policy reforms, comprehensive pricing strategies, and reduced market participation barriers.

Additionally, Ref. [202] highlights the necessity for market-based solutions to foster DER participation, emphasizing the significance of digital tools in adapting market frameworks. It addresses the growing inclusion of renewables, consumer empowerment, and the emergence of new entities like EVs within the market. The discussion extends to the potential of DERs in delivering flexibility services, challenging the traditional reliance on large power plants, and the anticipated demand for such services, suggesting the establishment of local energy markets to facilitate DER integration into the broader electricity markets.

Finally, addressing the challenges of integrating DERs, such as voltage optimization and overcoming regulatory obstacles, digital solutions emerge as key facilitators. Study [128] focuses on employing PV inverters for voltage control and leveraging optimal power flow tools for enhancing network operations, showcasing effective methods for integrating PV systems into distribution networks.

Ref. [70] introduces consumer digital twins (CDTs) for refining energy pricing within local energy markets (LEMs), utilizing the Internet of Things (IoT) to enable flexible DR and advocating for consumer-centric market models. This strategy enhances market efficiency by aligning with renewable variability and consumer preferences. Additionally, Ref. [70] delves into the development of DR, which is propelled by DERs and SG innovations, examining how digital technologies, including the Internet of Energy and IoT, are shaping DR strategies and their prospective roles in energy markets.

4.4. Customer Engagement and Retail Markets

Study [71] evaluates Spain’s retail electricity market’s digitalization, identifying gaps in digital adoption among electricity retailers, particularly in online services and billing, suggesting the need for a push for stronger digital engagement.

4.4.1. Digitalization as an Enabler for Sector Decarbonization and Decentralization

Digitalization is driving the power sector towards decarbonization and decentralization [203] by enhancing data usage and system efficiency, promoting competitive markets, and empowering consumers, while acknowledging new regulatory challenges.

4.4.2. Proposal for a Distribution-Level Retail Electricity Market

Ref. [204] suggests a new retail electricity market model managed by DSOs, incorporating DERs and employing advanced algorithms for real-time pricing and DER optimization, indicating the potential for lower electricity rates and improved market efficiency and consumer participation.

4.4.3. Digital Customer Relationship Management Systems for Customer Engagement

Advancements in CRM systems, highlighted in [72], show a shift towards engagement-focused strategies, emphasizing the importance of social CRM (SCRM) for deepening customer relationships and personalizing experiences in the electricity sector.

4.4.4. Digital Platforms and Tools for Better Energy Management

The transition to decentralized energy systems accentuates the role of digital platforms in improving energy management and consumer engagement. Studies [36,205] discuss integrating smart technologies and the FEEdBACk project, underscoring the importance of advanced technologies and social science methods in promoting energy-efficient behaviors.

These studies focus on the pivotal role of digitalization in enhancing customer engagement, market efficiency, and the integration of renewable energy sources and reflect on the power sector’s evolution towards more sustainable and consumer-centric operations.

4.5. Grid Services and Ancillary Markets

This section delves into digitalization’s impact on electricity markets, focusing on the integration of DRESs into ancillary services like frequency regulation and voltage control. The increasing presence of DRESs challenges traditional grid management practices, driving the need for new market mechanisms and models to ensure grid resilience and efficiency.

4.5.1. Market for Ancillary Services like Frequency Regulation and Voltage Control

The shift towards decentralized power systems, fueled by DRESs, is transforming ancillary services in distribution networks. This necessitates novel market tools for energy storage integration and SG advancements [73] to tackle renewables’ intermittency through state-of-the-art energy storage and grid management solutions. The associated challenges include regulatory barriers and the need for substantial investment in smarter grids capable of handling a larger share of renewable energy.

The emergence of DG introduces competitive dynamics in ancillary service markets, with studies [129,206,207] highlighting the need for flexible energy resources and future market designs. These include developing pricing mechanisms for ancillary and reactive power services and optimizing such services at the distribution level.

Research has identified barriers to integrating novel ancillary services at the distribution grid level, such as inertial response and voltage regulation [73]. The transition poses stability and security challenges and requires effective market tools for service procurement and addressing technical, regulatory, and financial challenges.

The future directions will require dynamic and flexible market designs that support the quick dynamics of renewable resources and should aim to enhance system reliability and efficiency through innovative regulations and market strategies.

4.5.2. Economic Models and Pricing for These Services

Integrating DERs into grids demands innovative pricing and economic models for the ancillary services that are essential in renewable-heavy systems [207]. An optimization model for distribution-level ancillary service pricing is proposed in [129], alongside P2P market designs for grid support and prosumer incentives, particularly with BESSs [132,208]. These models aim to enhance grid operation and accommodate multi-market dynamics at the distribution level, paving the way for efficient and sustainable energy management.

4.5.3. Software for Managing Participation in Ancillary Markets

The transition to renewable energy emphasizes the need for software that manages flexible resources in ancillary service markets (ASMs). The research in [209] explores integrating DRESs into ASMs, focusing on overcoming grid challenges through P2P market designs [132]. It stresses the importance of legislative adaptability, the technical readiness of DERs, and advanced control strategies for effective ASM participation. This approach underscores software’s critical role in optimizing DER integration for grid stability and efficiency.

4.6. Reimagining Pricing Strategies in the Energy Market

The evolution of the electricity market demands innovative pricing strategies and tariff designs driven by digitalization and the integration of renewables. This section reviews how digital tools and artificial intelligence are reshaping power distribution pricing, focusing on the impact on consumer behavior, ensuring cost recovery, maintaining fairness, and utilizing AI for dynamic pricing models. It aims to navigate the complexities of market pricing, balancing efficiency, fairness, and innovation.

4.6.1. Impact of Pricing Mechanisms on Consumer Dynamics

Recent studies have highlighted the significant influence of pricing mechanisms on consumer behavior and the overall energy ecosystem. The research in [133] explores these effects, whereas Ref. [210] examines the optimal pricing designs for varying consumer demographics, emphasizing customized strategies to boost engagement and efficient energy use. The role of digital technologies like smart meters and comparison tools in promoting consumer interaction with time-based tariffs and DR is discussed in [36], enhancing pricing transparency and consumer understanding. Additionally, Ref. [143] proposes a dual-stage pricing model optimizing social welfare and utility company benefits, illustrating the shift towards socioeconomically mindful pricing strategies.

4.6.2. Navigating Cost Recovery and Fair Pricing

As digitalization intersects with evolving pricing strategies, the focus on sustainable and equitable tariff designs intensifies. The research in [211] introduces a self-sustaining dynamic tariff model that uses real-time data to reflect demand changes, ensuring financial stability while accommodating consumption trends. Concurrently, Ref. [147] explores DR strategies that integrate renewable energy, advancing towards environmentally friendly energy solutions.

Highlighting the role of pricing in the transition to renewable-based grids, Ref. [133] examines its influence on consumption and production patterns, indicating pricing’s pivotal role in steering the market towards sustainable energy practices. Additionally, the exploration of innovative tariff tools in Ref. [211] tackles the efficiency and fairness balance, suggesting that digital advancements offer promising avenues to address these challenges.

These discussions emphasize the essential contribution of digital tools to shaping cost-effective, fair pricing strategies and tariff designs that align with the technological progress and evolving consumer expectations in the energy sector.

4.6.3. Leveraging AI for Dynamic Pricing in Energy Markets

AI and analytics are pivotal in crafting dynamic pricing models for the energy sector, bringing efficiency and adaptability to the forefront. Research has highlighted AI’s application in setting retail electricity prices, enhancing DR mechanisms, and optimizing energy storage operations.

A notable study [74], introduces a smart pricing model for EV charging, addressing both static and dynamic considerations to accommodate the rising presence of EVs. Similarly, Ref. [130] proposes a flexible pricing strategy for electricity retailers, factoring in user demand and renewable energy contributions, reflecting the market’s growing complexity and the shift towards more responsive models.

These insights showcase the significant potential of AI and analytics in refining power system operations and pricing strategies, pointing towards a future where energy systems are not only smarter but also more sustainable and user oriented.

4.7. Regulatory Compliance and Market Reporting

The digital transformation of distribution power systems brings significant benefits but also introduces new challenges, especially in terms of cybersecurity. The research in [158] delves into these emerging vulnerabilities, with a focus on SCADA systems and DERs, which are vital for grid integrity. It outlines the complexity of cyber threats and suggests comprehensive strategies to bolster security measures, ensuring the resilience of critical grid components against potential cyberattacks.

4.7.1. Cybersecurity in Market-Based Congestion Management

The research in [131] explores how cyberattacks can exploit market-based congestion management, particularly targeting vulnerable aggregators. By manipulating load profiles, attackers could induce congestion and inflate consumer costs. The study suggests securing these aggregators to safeguard demand-side management against cyber threats.

4.7.2. Impact of Cyberattacks in Integrated T&D Power Systems

Study [212] evaluates the impacts of cyberattacks on integrated transmission and distribution systems, focusing on systems with DER integration. Utilizing the OCTAVE Allegro method, it assesses risks and identifies attack vectors, underscoring the need for enhanced resilience against data integrity and control signal attacks.

4.7.3. AI and ML in Enhancing Cybersecurity

The research in [22] underscores AI’s and ML’s roles in strengthening cybersecurity within power systems. It points out the dynamic nature of cyber threats and the importance of continuous monitoring while also demonstrating AI’s capability for pre-empting and mitigating cyber risks.

These studies collectively emphasize the critical need for robust cybersecurity measures in the face of digital transformation in power systems. With increased complexity and connectivity, integrating traditional and advanced tools, including AI and ML, is vital for ensuring the reliability and efficiency of power distribution in a digitally evolving energy market.

5. Conclusions

This systematic review has delineated the critical role of emerging digital technologies in transforming power distribution grids in grid planning, grid operations, and the energy market. The integration of AI, the IoT, optimization, AMI, and blockchain was identified as pivotal in enhancing the operational efficiency, reliability, and security of these systems. Through the analysis of over 54,000 scholarly articles, our review demonstrates how these technologies not only optimize individual aspects of grid management but also collectively contribute to the overall resilience and sustainability of energy systems.

Significantly, the findings from this review underscore the necessity for a strategic approach to digitalization that embraces the complexity of modern power grids and addresses their multifaceted challenges. Data complexity and system integration are among the primary challenges identified. Data complexity arises from the vast volume, variety, and velocity of data generated by digital technologies, necessitating advanced big data analytics, machine learning algorithms, and robust data integration platforms. System integration challenges stem from ensuring interoperability among various digital technologies and legacy systems, which can be addressed through standard protocols, middleware solutions, and robust cybersecurity measures.

Future research should focus on overcoming the challenges of data complexity and system integration highlighted by this review. Additionally, further exploration into the interoperability of different technologies and their combined impact on grid performance will be crucial. As digital technologies continue to evolve, so must our strategies for their implementation; this will ensure that the digital transformation of power distribution grids effectively meets the demands of the next generation of energy consumers.

Finally, the journey to the digitalization of power distribution grids is complex and requires continuous innovation and strategic foresight. The insights provided by this extensive literature review contribute to a deeper understanding of this dynamic field and lay the groundwork for future advancements that will further enable the efficient, reliable, and secure operation of power systems worldwide.

In conclusion, as the sector moves towards an increasingly digital future, the integration of these technologies promises to enhance the adaptability and intelligence of grid systems, thereby supporting the broader goals of energy sustainability and economic efficiency. By addressing the challenges and harnessing the potential of digital technologies, we can build a more resilient, efficient, and sustainable energy infrastructure.