Recent Trends in Proxy Model Development for Well Placement Optimization Employing Machine Learning Techniques

Abstract

Well placement optimization refers to the identification of optimal locations for wells (producers and injectors) to maximize net present value (NPV) and oil recovery. It is a complex challenge in all phases of production (primary, secondary and tertiary) of a reservoir. Reservoir simulation is primarily used to solve this intricate task by analyzing numerous scenarios with varied well locations to determine the optimum location that maximizes the targeted objective functions (e.g., NPV and oil recovery). Proxy models are a computationally less expensive alternative to traditional reservoir simulation techniques since they approximate complex simulations with simpler models. Previous review papers have focused on analyzing various optimization algorithms and techniques for well placement. This article explores various types of proxy models that are the most suitable for well placement optimization due their discrete and nonlinear natures and focuses on recent advances in the area. Proxy models in this article are sub-divided into two primary classes, namely data-driven models and reduced order models (ROMs). The data-driven models include statistical- and machine learning (ML)-based approximations of nonlinear problems. The second class, i.e., a ROM, uses proper orthogonal decomposition (POD) methods to reduce the dimensionality of the problem. This paper introduces various subcategories within these two proxy model classes and presents the successful applications from the well placement optimization literature. Finally, the potential of integrating a data-driven approach with ROM techniques to develop more computationally efficient proxy models for well placement optimization is also discussed. This article is intended to serve as a comprehensive review of the latest proxy model techniques for the well placement optimization problem. In conclusion, while proxy models have their own challenges, their ability to significantly reduce the complexity of the well placement optimization process for huge reservoir simulation areas makes them extremely appealing. With active research and development occurring in this area, proxy models are poised to play an increasingly central role in oil and gas well placement optimization.

1. Introduction

Efficient reservoir management plays a pivotal role in the success of field development. The concept involves use of financial, technological, and human resources to maximize profit from a reservoir by optimizing recovery while minimizing capital investments and operating expenses [1]. Many issues in reservoir management are solved by optimization procedures. In the field development phase, some of the crucial decisions are determining the optimum locations for new wells, the type of wells, and the drilling schedule within a defined operational and economic scope. Well placement optimization is one of the most important challenges in efficient reservoir management that employs reservoir simulation as a tool for deriving an optimum solution. All optimization methods employed to address this problem are incorporated into reservoir simulations. This optimization objective is achieved by conducting various scenarios with different well locations in multiple simulation runs. The example of one such exhaustive simulation runs is shown in Figure 1 highlighting the map of cumulative production for single well placement. In a traditional optimization framework, the reservoir simulation is coupled with optimization algorithms to optimize the well locations for maximizing oil recovery and net present value. The problem becomes increasingly difficult to solve in terms of computational complexity and cost as different aspects of the geological uncertainty of the reservoir are considered in the decision-making process. The optimization task becomes cost effective if the computational time can be reduced [2]. Proxy models have the potential to reduce the computational efforts in the optimization problem while capturing the crux of geological uncertainties in the simulation model.

In the past, researchers have published reviews on well placement optimization [3] focusing mainly on general workflow and optimization techniques. With advances in computational capabilities and hardware, the application of deep learning-based methods is gaining significant traction. This paper examines applications of proxy models in the context of well placement optimization while focusing on recent improvements in the machine learning space.

Typically, well placement optimization is solved using an integration of reservoir simulation models and optimization algorithms. The process starts with building a high-fidelity reservoir model based on geological, petrophysical and fluid properties. Building and running these high-fidelity simulations require lab data as well as field measurements, such as reservoir pressure, production and injection rates, fluid contacts and PVT properties. The overall process of building the high-fidelity simulations is time-consuming and computationally expensive.

For well placement optimization, multiple parameters, such as the location, number and type of the wells (producer, injector, horizontal or vertical), as well as their operating conditions, exponentially increase the number of simulations required to cover all possible scenarios. Geological uncertainties add another layer of complexity to this problem by accounting for multiple geological realizations. Running these simulations, especially for large and complex reservoirs, can take hours or even days to complete, making it computationally expensive to explore a wide range of design options.

In order to enhance the efficiency of the process, optimization algorithms such as gradient-based methods and gradient-free methods are used to guide the search for optimal well placement. Nevertheless, with the aid of these methods, there is still a challenge posed by the necessity of conducting multiple high-quality reservoir simulations repeatedly, which proves to be a major limitation. Reducing the expenses of these simulations is crucial for making well location optimization viable in situations where real-time or near real-time decisions are required.

In recent years, advancements in machine learning and artificial intelligence have revolutionized the field of reservoir management. A notable breakthrough involves leveraging deep learning-based surrogate models to mimic the outcomes of complex reservoir simulations accurately. These surrogate models are trained on data generated from high-fidelity simulations and then used to predict reservoir performance across a range of well placement scenarios with far less computational effort. Methods such as convolutional neural networks (CNNs) [4,5] and recurrent neural networks (RNNs) [6,7] have been used to understand both the spatial and temporal aspects of reservoir behavior effectively. These techniques can be combined with optimization algorithms to efficiently navigate the solution space while taking uncertainties into account. This approach enables optimization to be carried out with a shorter simulation time.

1.1. Well Placement Optimization Problem Formulation

The well placement optimization problem can be formulated as an optimization problem with the primary goal of maximizing specified objective functions (e.g., oil recovery and net present value (NPV)). Refs. [8,9] have provided a detailed well placement optimization problem statement in the context of the infill drilling of production wells. That strategy is also applicable for injection wells and a combination of production and injection wells. The problem can be stated in the broader context of well placement as follows:

Given:

• Geological information (porosity, permeability, dimensions);
• PVT data (formation volume factors and fluid properties such as viscosity, density, compressibility);
• Existing wells and their types (producers/injectors, vertical/horizontal, gas/water injectors) and locations;
• Well control parameters (maximum injection rates for injectors, minimum bottom-hole pressures for producers);
• Operational constraints (well spacing, geological limitations—e.g., faults, barriers)

Economic data (drilling budget and costs, injection costs, discount rate, oil revenue forecasts, etc.).

Objective:

• Maximize the objective function (NPV, oil recovery, CO₂ storage potential).

Solution:

• Optimal location of the new production/injection wells;
• The commonly used objective function is the net present value for the optimization problem. Ref. [3] provided a simple mathematical formulation of NPV for a two-phase flow reservoir model as:

$$NPV={\sum }_{i=1}^T{\displaystyle \frac{Q_o{P}_o+Q_w{P}_w-OPEX}{{(1+D)}^i}}-CAPEX$$

where NPV is the net present value and Q_o and Q_w are cumulative oil and water production, respectively. P_o and P_w are oil price and water production cost, respectively. OPEX is the operational expenditure. CAPEX is the capital expenditure. D is the discount rate, and T is the number of years passed since the production started.

1.2. Well Placement Optimization Workflow

The main objective of well placement optimization is to find the optimal location for the well in such a way that it maximizes the targeted objective function, which is normally net present value (NPV) or the total oil recovery. Ref. [10] described the workflow of well placement optimization with various elements of the optimization process. Figure 2 shows the general workflow for well placement optimization. Researchers have made significant contributions to enhancing various aspects of the workflow over the years, including the development of proxy models and the implementation of diverse optimization algorithms such as gradient-based and gradient-free techniques.

In early stages, the primary focus of researchers was on evaluating different optimization algorithms as the main approach to addressing the issue. Various gradient-based algorithms [11,12] and gradient-free algorithms [13,14,15,16] were explored to solve this problem. It has been shown that gradient-free optimization problems are the most effective techniques for solving the well placement optimization problem [3]. Amongst the gradient-free optimization algorithms, particle swarm optimization [13] and genetic algorithms [16] have become the most commonly used optimization techniques.

The process of well placement optimization requires multiple simulations to find reasonable solutions. Proxy models can aptly represent key simulation output parameters and are attractive tools to be used as an efficient substitute for full reservoir simulation [17]. They have become an essential and key component of the well placement optimization process in recent years. This review paper discusses the development and use of proxy models for well placement optimization (WPO). Previous studies [3,10,18] have examined various optimization algorithms for WPO.

2. Summary of Proxy Models Development

Proxy models, also known as surrogate models or metamodels, play a significant role in reservoir simulation, providing efficient alternatives to full order numerical simulations, which can be computationally demanding and time-consuming. These proxy models are a good approximation of the behavior of complex reservoir systems, enabling faster evaluations of various scenarios and facilitating quicker decision-making processes in the reservoir management.

Many authors have categorized proxy models based on applications/objectives, and approximation strategy [19]. Based on their development approach, many authors and researchers have categorized these models in a different manner. Ref. [20] divided proxy models into two major categories: traditional proxy models (TPMs) and smart proxy models (SPMs). Traditional proxy models include multi-fidelity models and reduced order models, whereas smart proxy models (SPMs) include machine learning and pattern recognition techniques. Ref. [20] categorized the proxy models into four groups: the statistics-based approach, reduced physics approach, reduced order modeling approach and artificial intelligence (AI)-based techniques. Ref. [19] categorized the proxy models into four classes: multi-fidelity models, reduced order models, TPMs and SPMs. Each of these groups are defined based on different development procedures. In this study, we categorize proxy models for well placement optimization into two main groups, as outlined by [2]: data-driven models and reduced order models. Data-driven models include statistical- and machine learning-based techniques. Reduced order models include techniques that mainly involve simplification of problems and are not purely data-driven. Figure 3 shows an overview of the classification of proxy models based on [2].

The following sections will explain data-driven and reduced order models through detailed mathematical problem formulations. Various machine learning algorithms such as artificial neural networks, gradient-boosting techniques and autoencoders will be discussed along with the reduced order models.

3. Applications of Proxy Models for Well Placement Optimization

3.1. Data-Driven Proxy Models

In recent years, use of data driven methods to develop proxy models for reservoir simulation has gained significant attention. These models are mainly developed in artificial intelligence frameworks. These methods are classified as hybrid methods which combine proxy models with stochastic optimization algorithms to solve well placement optimization issues [3].

Data-driven models are divided into two main classes: (1) mathematics/statistics-based and (2) machine learning (ML-based) [2].

3.1.1. Mathematical/Statistical-Based Models

Mathematics/statistics-based methods commonly include response surface model (RSM) and kriging methods [2]. An RSM is a statistical approach that uses experimental design and understands the response variables within a certain range of predictor variables. Ref. [21] developed an RSM for hydraulically fractured horizontal wells with six uncertain parameters to fit a response surface, with NPV as the objective function. Ref. [22] employed a polynomial approximation-based response surface method to optimize well placement, emphasizing its limitations due to the curse of dimensionality and nonlinearity.

Another well-known proxy method in mathematical/statistical-based techniques is kriging. Kriging acts as an interpolation method that is based on the Gaussian process, which is governed by prior mean and covariance [2,23]. Even though kriging is used for the interpolation of geological properties at unsampled locations, it has also been used to approximate the dynamic output generated by reservoir simulations [24]. Several studies have applied hybrid approaches that combine kriging with other optimization techniques. Ref. [25] developed an integrated approach of combining kriging with finite difference gradient to reduce function evaluations. Similarly, Refs. [14,15] used hybrid methods incorporating kriging with genetic algorithms and other methods, demonstrating significant reductions in the number of required simulations. Ref. [16] employed Universal Trace Kriging to optimize well locations, showing improved production outcomes.

One of the recent applications of a kriging-based proxy model was developed by [26]. A new workflow combining a capacitance–resistance model and kriging into a single tool was developed. The new tool was capable of giving fast and reliable enough forecasts for new wells and, thus, assisting in making a decision about well positioning. The method uses a primary connectivity coefficient between producers and injectors and universal kriging to determine new coefficients for a capacitance–resistance model (CRM). The tool was used for a SPE10 model and a real field case for X field, located in Siberia, Russia. It was concluded that the new proposed well location, surrounded by several production and injection wells, had better performance prediction due to accurate interpolation.

Table S1, from the Supplementary Materials, shows the summary of the application of kriging and RSM-based proxy models for well placement optimization.

3.1.2. Machine Learning Proxy Models

Machine learning is a growing discipline in the well placement optimization domain. It includes development of various algorithms that enable computers to perform specific tasks without being explicitly programmed. Machine learning algorithms analyze and learn from the data and identify patterns and trends to make predictions or decisions.

There are four main types of machine learning techniques: supervised learning, unsupervised learning, semi-supervised learning and reinforcement learning as described in [27] and shown in Figure 4.

Most proxy model applications for reservoir simulation are categorized as supervised learning. A range of supervised machine learning algorithms, including support vector machines (SVMs), gradient-boosting techniques (XGBoosts) and support vector regression have been used to develop the proxy models for reservoir simulation applications [28,29]. While these data-driven machine learning models have shown significant promise for well placement optimization, they do not incorporate the underlying physical principles and theories governing subsurface flow processes [30].

The nonlinear and discrete nature of reservoir simulation makes neural network-based techniques the most suitable techniques for subsurface flow proxy models. Recent advances in the research of deep learning have led to exploration of various advanced neural network-based algorithms such as convolutional neural networks, recurrent neural networks, and autoencoders and their variants. To incorporate the fundamental physical principles of governing fluid flow in subsurface environment, researchers have used physics-informed neural networks to develop proxy models for subsurface flow processes [31,32].

Neural Networks

Neural networks are one of the most common machine learning algorithms that have gained significant interest in artificial intelligence research and applications. Ref. [33] provided the introduction and applications of neural networks for proxy model development. Neural networks consist of neurons which transmit information from one to another through weights and layers. The weights represent the strength of the information being carried while the layers represent the direction in which the information is flowing. The network learns to recognize patterns, make decisions or predict outcomes from the provided inputs. Typically, input data for neural network-based proxy models include reservoir properties, time series data, well completions and locations.

Recently, many variants of neural networks, such as convolutional neural network (CNNs) [30,34], recurrent neural networks (RNNs) [7] and feedforward neural networks (FNNs) [34,35] have been used for proxy model development as a substitute for reservoir simulation. These proxy models were coupled with various optimization algorithms to optimize production performance.

Neural networks have been used widely for proxy model development in well placement optimization in addition to well control optimization. The typical input and output data for well placement optimization problems are shown in Figure 5 and Figure 6. Ref. [36] combined neural networks with a covariance matrix adaptation evolution strategy for proxy model development to optimize well placement. Furthermore, Ref. [37] focused on optimization methodology for well placement using artificial neural network-based proxy models for CO₂ storage. Figure 5 shows typical ANN structure for WPO using productivity potential [38]. Table 1 shows the summary of applications of neural network-based proxy models.

The choice of neural networks is motivated by the need to handle the high-dimensionality and complex nonlinear relationships present in a typical reservoir simulation problem. Traditional proxy models, such as geostatistical- and kriging-based methods, often face challenges in dealing with large reservoirs and multiple decision variables [25]. Ref. [16] highlighted one of the major drawbacks of Universal Trace Kriging-based proxies as being their poor proxy predictive quality in regions with nonlinear responses, such as areas with sealing faults or well interchanging positions. The study focusing on a response surface proxy for fractured wells [21] mentioned the challenges of simulating the interaction between hydraulic fractures and geological heterogeneity.

Neural networks, including autoencoders, convolutional neural networks and physics-informed neural networks, provide robust solutions to these challenges. Autoencoders are capable of efficiently reducing high-dimensional complex data into lower-dimensional latent space, enabling faster predictions [5]. Convolutional neural networks (CNNs) capture complex spatial data, making them particularly well suited for capturing geological features, such as sealing faults. Physics-informed neural networks [5] incorporate governing physical equations directly into training process, ensuring that the model respects the physical constraints of the reservoir and predicts the nonlinear behaviors in areas with complex geological features.

Exploring convolutional neural networks is a promising research area, as the concept of CNNs is similar to the concept of a convolutional autoencoder, which is discussed in the subsequent section. A CNN is class of deep neural network (DNN) which is commonly used for image classification. It operates on 2D data through convolutional layers and pooling layers [4]. The convolutional layer consists of learnable filters which identify local features and generate outputs as feature maps.

CNN has been routinely applied in classification problems [39]. Ref. [40] used a multi-modal CNN for optimal infill well productivity estimation. Ref. [4] developed a proxy model based on streamlined time of flight (TOF) maps and a CNN, marking the first application of CNN for multi-well placement optimization. Figure 6 shows the input and output data in a CNN developed by [4]. Inputs consist of TOF maps and well locations and outputs consist of NPV.

Table 1. Summary of applications neural network-based proxy models for well placement optimization.

Literature	Method	Objective	Findings
[37]	Artificial neural network	Optimization of CO2 injection and brine production well placement in geological CO2 storage using artificial neural networks to reduce the simulation runs	ANN was coupled with genetic algorithm to optimize the well locations. Total number of runs were reduced by 80.7% from 622 to 120.
[5]	V-Net NN	V-Net NN with GA for well placement optimization	Physics-guided V-Net with skip connections, 3D convolutional filters, and a residual learning structure to handle 3D parameter fields results in 30 times faster processing.
[41]	Artificial neural network	Application of artifical neural networks trained in time-dependent manner to optimize well placement	Efficient dynamic, time dependent proxy with genetic algorithm comparable with commercial reservoir simulation
[30]	Convolutional neural network	Combination of theory guided convolutional neural network with genetic algorithm	Theory-guided neural network framework achieved better accuracy compared to purely data-driven models, even with limited training data. Time was also reduced from 142,595 s (simulation) to 133 s (proxy).
[42]	SimProxy	Simproxy to integrate reservoir and surface behavior to reduce computational cost in well placement optimization	Multilayer perceptron (MLP) was used to develop the NN-based proxy. Training samples obtained with principal component analysis (PCA) and Latin hypercube sampling (LHS) showed best results. Some drawbacks as the number of wells grows.
[36]	Artificial neural network	Combination of ANN with a covariance matrix adaptation evolution strategy (CMA-ES)	The ANN provides the average NPV and standard deviation of the NPV of an ensemble of geological realization for a given well configuration
[43]	LSTM	Well placement and well control optimization with multiple development objectives using LSTM surrogate model	Computational time was reduced by 82% and 95% in the 2D and 3D models, respectively. However, geological uncertainty was not considered.
[44]	Graph Neural Surrogate Model (GNSM)	Optimize well placement and well control using GNSM	Demonstrated high accuracy with relative errors of 1–2% for pressure and saturation. The model provided 5–7% median errors for well rates prediction. Longer training time (30 h) is one of the major limitations. The model was only designed for a 2D unstructured reservoir model.

Table 1. Summary of applications neural network-based proxy models for well placement optimization.

Literature	Method	Objective	Findings
[37]	Artificial neural network	Optimization of CO2 injection and brine production well placement in geological CO2 storage using artificial neural networks to reduce the simulation runs	ANN was coupled with genetic algorithm to optimize the well locations. Total number of runs were reduced by 80.7% from 622 to 120.
[5]	V-Net NN	V-Net NN with GA for well placement optimization	Physics-guided V-Net with skip connections, 3D convolutional filters, and a residual learning structure to handle 3D parameter fields results in 30 times faster processing.
[41]	Artificial neural network	Application of artifical neural networks trained in time-dependent manner to optimize well placement	Efficient dynamic, time dependent proxy with genetic algorithm comparable with commercial reservoir simulation
[30]	Convolutional neural network	Combination of theory guided convolutional neural network with genetic algorithm	Theory-guided neural network framework achieved better accuracy compared to purely data-driven models, even with limited training data. Time was also reduced from 142,595 s (simulation) to 133 s (proxy).
[42]	SimProxy	Simproxy to integrate reservoir and surface behavior to reduce computational cost in well placement optimization	Multilayer perceptron (MLP) was used to develop the NN-based proxy. Training samples obtained with principal component analysis (PCA) and Latin hypercube sampling (LHS) showed best results. Some drawbacks as the number of wells grows.
[36]	Artificial neural network	Combination of ANN with a covariance matrix adaptation evolution strategy (CMA-ES)	The ANN provides the average NPV and standard deviation of the NPV of an ensemble of geological realization for a given well configuration
[43]	LSTM	Well placement and well control optimization with multiple development objectives using LSTM surrogate model	Computational time was reduced by 82% and 95% in the 2D and 3D models, respectively. However, geological uncertainty was not considered.
[44]	Graph Neural Surrogate Model (GNSM)	Optimize well placement and well control using GNSM	Demonstrated high accuracy with relative errors of 1–2% for pressure and saturation. The model provided 5–7% median errors for well rates prediction. Longer training time (30 h) is one of the major limitations. The model was only designed for a 2D unstructured reservoir model.

Application of neural networks as proxy models have shown promising results by significantly reducing the number of simulation runs needed. Ref. [37] demonstrated a reduction in the number of numerical simulation runs by 81%. Many surrogate models showed high accuracy in comparison with traditional simulations. Ref. [5] achieved high accuracy, with an average R2 of 0.988 for pressure prediction and 0.998 for production rates, while providing computational efficiency improvements of up to 30 times faster than traditional simulations. Ref. [44] demonstrated high accuracy with relative errors for pressure and saturation in the range of 1–2%, as well as a speedup factor of 36 compared to traditional simulation.

The performance of the neural network-based proxy models highly depends on the quality of input data. Quality of prediction was observed to be declining when applied to new geological models not covered in the training dataset [44]. Also, ANN-based surrogate models are specific to the reservoir and well configurations. For well placement optimization, new training datasets would be required for different well configurations. Ref. [44] highlighted a major challenge where the Graph Neural Surrogate Model required 30 h of training using an Nvidia A100 graphic processer.

Table S1 from the Supplemental Materials highlights additional neural network-based models that are used for well placement optimization.

Autoencoders (AEs)

An autoencoder (AE) is a specific type of neural network framework which is mainly used to encode the input into a compressed and meaningful representation, also known as latent space, and then decode it back such that the reconstructed input is as similar as possible to the original one [45]. There are various types of autoencoders: regularized AEs, sparse AEs, denoising AEs, contractive AEs and variational AE. Ref. [45] described these types in further detail. Autoencoders are widely used for dimensionality reduction in fluid mechanics [46]. Autoencoders are developed using neural networks and consist of encoders, latent space (bottleneck) and decoders. An example of a basic AE is shown in Figure 7.

Recently, the application of AEs in proxy model development has gained significant interest [5,47,48,49]. Refs. [48,49] developed an encoder–decoder-based proxy metamodel as a substitute for reservoir simulation. An autoencoder was used to develop low-dimensional data in latent space, and a neural differential operator was used for learning about dynamical systems in latent space. Ref. [47] developed a transfer learning framework using denoising autoencoder for well placement optimization. Ref. [50] developed a recurrent neural network, also known as long short-term memory (LSTM), arranged in an encoder–decoder architecture that used well placement schedules as input. Another notable application is V-Net architecture for WPO developed by [5]. The V-net architecture in this study is a modification of an encoder–decoder network with an important feature of skip connection that keeps certain information from the encoding of the original images in the decoding process.

Overall, AE-based proxy models are used for mainly dimensionality reduction. This reduced dimensionality helps models to learn the dynamical systems with the most important captured features.

3.2. Reduced Order Models

Reduced order models (ROMs) are the second major type of proxy models that are used in production optimization. These models involve the application of proper orthogonal decomposition (POD) to obtain solutions, or ‘snapshots’ generated during one or more training simulations [51]. These solutions are represented as linear combinations of a relatively small number of the basis functions which are columns of the POD basis matrix. Ref. [51] described the flow chart for reduced order water flooding optimization and is shown in Figure 8. POD-based reduced order models have been used in [52,53,54,55,56,57,58,59].

The application of POD-based reduced order models for the well placement optimization problem is very limited. Based on the literature review conducted, reduced order model application for the well placement optimization problem was only mentioned in one instance [60]. Additionally, parametric model order reduction (PMOR) was developed, which generates reduced order models that characterize the system for different well locations.

4. Future Work: Challenges and Directions

Various proxy model techniques discussed in this paper have shown positive outcomes for optimizing the well placement problem. Most methods have shown reductions in the computational time and simulation runs to find the optimal solution for well placement by combining proxy models with gradient-free optimization algorithms such as PSO and GA. Future research directions may include the application of these strategies to more complex real-case studies, multi-well placement and geological uncertainty.

Although researchers have investigated robust optimization techniques incorporating geological uncertainties [61,62,63], application of the proxy models for well placement with geological uncertainty still remains limited. When geological uncertainty is represented by multiple ensemble realizations, the overall computational complexity might become intractable when each objective function evaluation requires the simulation of the whole ensemble of models [63]. Use of artificial neural networks can solve this challenge by acting as the decision-maker for well locations with changing inputs.

The review also indicates that the well placement problem has been addressed more often by using data-driven proxy models than reduced order models. Previously, reduced order models were commonly used for well control optimization problems with gradient-based optimization methods. The ROM workflows were not meant for well placement optimization, which involves changing well configuration. However, various deep learning techniques from computer vision and image recognition can be used to transform high-dimensional problems, such as WPO, into lower-dimension sub-space problems to generate training dataset for neural networks. This approach is comparable to the offline training processes of ROMs which generate snapshots of pressure and saturation maps. Ref. [5] have previously implemented an encoding–decoding-based V-Net algorithm to generate feature maps with a low dimensionality. These encoder–decoder-based methods can also be combined with neural operators-based algorithms, such as a Fourier neural operator (FNO), for learning about latent space dynamics [64].

Recently, incorporating spatial and temporal data into a proxy model with V-Net architecture and physics-guided neural networks showed promising results in terms of well placement optimization [5]. This study focused on using V-Net architecture with skip connections, 3D convolutional filters, and residual learning structures to handle 3D spatial data. To predict the dynamics of the reservoir system, a physics-guided neural network with governing equations was developed (PgV-Net). A reservoir model with 30 × 62 × 30 cells was used with a varying porosity of up to 0.5 and a permeability reaching 20,000 millidarcies. The architecture improved computational efficiency by up to 30 times compared to traditional numerical simulation. The average coefficient of determination (R²) for pressure and production rate prediction reached 0.988 and 0.998, respectively. Three optimization cases were used to verify the results. Horizontal well location optimization using PgV-Net was 12 times faster than a numerical simulator while achieving near-identical results. For 3D well arrangement optimization with vertical wells, the model was 20 times faster, showing a higher NPV than simpler horizontal-only cases. For simultaneous optimization of well number and 3D arrangement, the efficiency improved by up to 30 times compared to traditional simulations. Even though the study showed that incorporating physics into machine learning models can greatly enhance the accuracy and efficiency, future work could focus on addressing the challenges of handling more complex multiphase flows, and optimizing both injection and production wells. The study assumed known porosity and permeability fields, which is often not the case in complex real-world applications. To take geological uncertainty into consideration, constructing a stochastic surrogate model with uncertainty quantification is necessary. Furthermore, the advancement into newly developed neural network architectures for dimensionality reduction, feature extraction, as well as architectures with applications for learning about dynamical systems, can be combined effectively to develop highly accurate and efficient surrogate models. Parallel computing strategies can be implemented to construct such deeper convolutional neural networks with denser filters that can solve the complex problems involving highly complex physical processes.

5. Conclusions

In this article, an in-depth assessment of the applications of various proxy models is performed and summarized for the well placement optimization problem, with a focus being placed on data-driven methods. Two significant types of proxy models highlighted in the paper, i.e., data-driven models and reduced order models, have shown significant progress in recent years.

• Data-driven models, such as machine learning-based techniques, transform complex nonlinear reservoir simulation problems into simpler linear representation and provide quick and efficient approximations of the objective functions. Current ML-based models have shown promising results by predicting objective functions for various well placement scenarios, featuring above 90% accuracy in many cases.
• Data-driven models do not incorporate the underlying physical principles and theories governing subsurface flow processes, which limits their performance in terms of fully capturing complex reservoir behavior.
• Recently, neural network-based proxy models have gained significant importance and have shown potential for future use in well placement optimization problems due to their capabilities of capturing the nonlinearity and complexities involved in the WPO problem.
• Reduced order models use proper orthogonal decomposition (POD) to reduce dimensionality, capturing solutions in a lower-dimensional sub-space, and, while effective in continuous problems like well control optimization, they are less applicable to the well placement optimization problem due to its highly nonlinear and discrete nature.
• Possible future trends in proxy model development for well placement include physics-informed neural networks (PINNs) for incorporating physical principles, recurrent neural networks (RNNs) for capturing temporal dynamics and Fourier Neural Operators (FNO) for dynamical system learning.