A Deep Neural Network Based on ResNet for Predicting Solutions of Poisson–Boltzmann Equation

Abstract

The Poisson–Boltzmann equation (PBE) arises in various disciplines including biophysics, electrochemistry, and colloid chemistry, leading to the need for efficient and accurate simulations of PBE. However, most of the finite difference/element methods developed so far are rather complicated to implement. In this study, we develop a ResNet-based artificial neural network (ANN) to predict solutions of PBE. Our networks are robust with respect to the locations of charges and shapes of solvent–solute interfaces. To generate train and test sets, we have solved PBE using immersed finite element method (IFEM) proposed in (Kwon, I.; Kwak, D. Y. Discontinuous bubble immersed finite element method for Poisson–Boltzmann equation. Communications in Computational Physics 2019, 25, pp. 928–946). Once the proposed ANNs are trained, one can predict solutions of PBE in almost real time by a simple substitution of information of charges/interfaces into the networks. Thus, our algorithms can be used effectively in various biomolecular simulations including ion-channeling simulations and calculations of diffusion-controlled enzyme reaction rate. The performance of the ANN is reported in the result section. The comparison between IFEM-generated solutions and network-generated solutions shows that root mean squared error are below $5·{10}^{-7}$. Additionally, blow-ups of electrostatic potentials near the singular charge region and abrupt decreases near the interfaces are represented in a reasonable way.

1. Introduction

Poisson–Boltzmann (PB) theory has been used effectively in many disciplines including biophysics, electrochemistry, and colloid chemistry [1,2,3,4,5,6]. For example, biomolecular interactions and dynamics of electrons in semiconductors or plasma can be modeled via the PB equation (PBE). Additionally, the electrostatic potential and energy of molecular tRNA in an ionic solution can be described via PBE. Therefore, developing numerical methods for predicting solutions of PBE is of interest in many fields. Although there are many FEM/FDM based algorithms for PBE (see [1,4,7,8,9,10,11,12,13] and the references therein), one observes that algorithms are rather complicated due to the nature of the governing equation. First, the coefficients (for example dielectric parameter) of partial differential equations (PDEs) change abruptly across the solute–solvent interface. Second, a Dirac-delta type singularity arises at the right-hand side of PBE which should be handled properly via some regularization process. People usually use the idea of subtracting Green’s function to remove the singular terms [7,8,9]. Third, PBE is a nonlinear-type equation leading to the need for Newton–Krylov type iterations as in [9,14]. Besides the difficulties in developing FEM/FDM-based numerical algorithms for PBE, another bottleneck of FEM/FDM-based approaches is that a large time cost is required to produce a numerical solution. Since normalization for PBE requires multiple Newton iterations, the algebraic system ($Ax=b$) should be solved multiple times, which makes the real-time simulation of PBE solutions almost impossible. Therefore, there arises a need for alternative approaches to solve PBE in a more user-friendly and efficient way.

One of the alternative approaches can be made via machine learning (ML) or deep learning (DL) whose communities have developed tremendous advances in various fields. One of the advantages of ML or DL is that once the model is trained with some datasets, predictions can be made in almost real time. In particular, efficiency, wide applicability, and flexibility in building architectures are some of the advantages of the artificial neural networks (ANNs), whose applications were proven to be effective in various fields including image recognition [15,16,17], natural language processing [18,19,20], audio signal processing [21,22,23], etc. Additionally, the applicabilities of ANN to obtaining solutions of PDEs are gaining attention recently. As far as the source terms, geometrical information and solutions of governing equations can be represented in multi-channel images, one can develop ANNs for PDEs by defining certain optimization functions properly. For examples, one can find ANNs for Poisson-type PDEs [24,25], ODE-type PDEs [26,27], peridynamics equations [28,29], fluid dynamics problems [30,31], etc.

In this work, we propose a new way of predicting solutions of PBE based on ANN. Unlike previous FEM/FDM-based numerical algorithms, our ANN algorithms can predict solutions of PBE in almost real time once the networks are trained. Thus, our ANN can be used effectively in various biomolecular simulations including ion-channeling simulations [13] and calculations of diffusion-controlled enzyme reaction rate [32]. Since generating datasets directly affects the performance of a neural network, one must carefully choose the numerical methods to generate solutions of PBE. We adopt the immersed finite element method (IFEM [33,34,35,36,37]) based numerical methods introduced in [9] for the following reasons: (1) the numerical solutions by [9] converge to real solutions in an optimal convergence rate ($L^2$-errors are $\mathcal{O}\left(h^2\right)$ where h is a mesh size), (2) since IFEMs are implemented on a uniform grid, structures of the obtained numerical solutions are simple, leading to small post-processing necessary to put into neural networks, (3) algebraic equations ($Ax=b$) generated by the IFEM have simple structures and can be solved quickly [38,39]. To develop robust neural networks regardless of the charge locations and shapes of solute–solvent interfaces, we consider various combinations of interface/charge locations in generating IFEM solutions. However, the relatively large number of parameters were required in order for the neural networks to predict solutions of PBE with varying conditions. Since the so-called "gradient-vanishing" phenomenon occurs when the number of hidden layers increases without proper care, we adopted ResNet-type [17] networks. In [17], He et al. introduced Res-blocks whose main feature is an appearance of skip connection which prevents the gradient-loss at a backpropagation process. Using Res-Blocks repeatedly in our version of ResNet-type neural networks, we were able to predict PBE solutions accurately; the least square differences of ANN-generated solutions and IFEM-generated solutions are below $5·{10}^{-7}$.

The rest of the paper is organized as follows. In the next section, we describe the overall process of the methods including a generation of IFEM-solutions and a proposal of the ResNet-based neural networks for PBE. In Section 3, the performance of neural networks is reported. The conclusion and discussion follow in Section 4.

2. Methods

In this section, we describe the whole process of building neural networks to predict solutions of PBE. The governing equation is described in Section 2.1 and IFEM based methods to solve PBE are introduced in Section 2.2. In Section 2.3, generations of datasets are described. Finally, ResNet-based ANNs for PBE are proposed in Section 2.4.

2.1. Poisson–Boltzmann Equation and Regularization Process

We consider the PBE on the domain $\Omega ={\Omega }^{+}\cup {\Omega }^{-}\subset {\mathbb{R}}^2$ where ${\Omega }^{+}$ is the solvent region with dielectric constant ${ϵ}^{+}>0$ and ${\Omega }^{-}$ is the molecular region with dielectric constant ${ϵ}^{-}>0$. The molecular surface, denoted by $\Gamma$, is assumed to be $C^2$ continuous and $\partial \Omega$ denotes the boundary of the entire domain. The governing equation is the following:

$$\begin{array}{}\mathrm{(1a)}& -\nabla ·(ϵ\nabla u)+{\kappa }^2\sinh \left(u\right)& ={\rho }^f,{\rho }^f=\sum _{i=1}^N{q}_i\delta (x-x_i)\mathrm{(1b)}& \hfill {\left[u\right]}_{\Gamma }& =0\mathrm{on}\Gamma ,\hfill \mathrm{(1c)}& \hfill {\left[ϵ\frac{\partial u}{\partial {\mathbf{n}}_{\Gamma }}\right]}_{\Gamma }& =0\mathrm{on}\Gamma ,\hfill \mathrm{(1d)}& \hfill u& =0\mathrm{on}\partial \Omega ,\hfill \end{array}$$

where u is an electrostatic potential, ${\rho }^f$ is the singular charge distribution, and $\kappa$ is the related parameter in ${\Omega }^{+}$, and is equal to 0 in ${\Omega }^{-}$. In Equation (1a), $x_i$ is the location of charge, $q_i$ is amount of charge, and $\delta \left(x\right)$ is the Dirac delta function. ${\mathbf{n}}_{\Gamma }$ is a unit normal vector to $\Gamma$ and bracket ${[·]}_{\Gamma }$ denotes the jump across the interface, i.e., ${\left[u\right]}_{\Gamma }=u{{|}_{{\Omega }^{-}}-u|}_{{\Omega }^{+}}$.

Since PBE has a singularity, it is not easy to solve by conventional numerical methods. The singular charge distribution of the PBE makes its solution discontinuous. It is well known [8] that the solution of PBE does not belong to $H^1(\Omega )$. Hence the standard FEM cannot be applied. So we shall express the original solution u as sum of a regular solution $u^r$ and a singular potential $u^s$ which takes care of singularities [7].

$$\begin{array}{cc}\hfill & u=u^r+u^s\hfill \end{array}$$

(2)

Then, the original equation is converted into another equation for the regular solution $u^r$. There are many choices of $u^s$. We choose it as follows:

$$\begin{array}{cc}\hfill & u^s{=G|}_{{\Omega }^{-}}\mathrm{in}{\Omega }^{-}\hfill \\ \hfill & u^s=0\mathrm{in}{\Omega }^{+},\hfill \end{array}$$

where G is defined by $G=-{\sum }_{i=1}^N\frac{q_i}{2\pi ϵ}\log |\mathbf{x}-{\mathbf{x}}_{\mathrm{i}}|$ in ${\mathbb{R}}^2$ satisfying $-{ϵ}^{-}▵G={\rho }^f$. Using this choice of $u^s$ and (2), we obtain the following equation for the regular potential $u^r$ from (1).

$$\begin{array}{}\mathrm{(3a)}& \hfill -\nabla ·(ϵ\nabla u^r)+{\kappa }^2\sinh \left(u^r\right)& =0\mathrm{in}\Omega ,\hfill \mathrm{(3b)}& \hfill {}^r{]}_{\Gamma }& =J_1\mathrm{on}\Gamma ,\hfill \mathrm{(3c)}& \hfill {\left[ϵ\frac{\partial u^r}{\partial {\mathbf{n}}_{\Gamma }}\right]}_{\Gamma }& =J_2\mathrm{on}\Gamma ,\hfill \mathrm{(3d)}& \hfill u^r& =0\mathrm{on}\partial \Omega ,\hfill \end{array}$$

where $J_1=-u^s$ and $J_2=-{ϵ}^{-}\frac{\partial u^s}{\partial {\mathbf{n}}_{\Gamma }}$. Here, we have exploited the fact that $\kappa$ is equal to 0 in ${\Omega }^{-}$ and $u^s$ vanishes in ${\Omega }^{+}$.

2.2. Immersed Finite Element Method for Poisson–Boltzmann Equation

In this subsection, we briefly explain the IFEM based approaches to solve PBE introduced in [9].

Let ${\mathcal{T}}_h$ be a uniform triangulations of $\Omega$ which do not necessarily have to align with the interface. For example, we divide $\Omega$ by axi-parallel lines and then subdivide the subrectangles by diagonals. The set of all triangle elements divided by the interface is denoted by ${\mathcal{T}}_h^{*}$. We define $S_{\Gamma }^h$ to be the union of elements in ${\mathcal{T}}_h^{*}$.

The main idea used in [9] to effectively reduce the singularity in PBE is to decompose $u^r$ as two parts. First, we define a discontinuous bubble $u^{*}$ which has support on $S_{\Gamma }^h$ and satisfies the jump conditions,

$$\begin{array}{}\mathrm{(4a)}& \hfill {\left[\overline{u}\right]}_{\Gamma }& =J_1\mathrm{on}\Gamma ,\hfill \mathrm{(4b)}& \hfill {\left[ϵ\frac{\partial \overline{u}}{\partial {\mathbf{n}}_{\Gamma }}\right]}_{\Gamma }& =J_2\mathrm{on}\Gamma .\hfill \end{array}$$

In [9], discontinuous bubble is constructed as piecewise linear polynomials on each T on ${\mathcal{T}}_h^{*}$. A detailed explanation and implementations can be found in [9].

Thus, substracting discontinuous bubble from $u^r$, we obtain a new equation for $\overline{u}=u^r-u^{*}$:

$$\begin{array}{}\mathrm{(5a)}& \hfill -\nabla ·(ϵ\nabla \overline{u})+{\kappa }^2\sinh (\overline{u}+u^{*})& =\nabla ·(ϵ\nabla u^{*})\mathrm{in}\Omega \backslash \Gamma ,\hfill \mathrm{(5b)}& \hfill \overline{u}{]}_{\Gamma }& =0\mathrm{on}\Gamma ,\hfill \mathrm{(5c)}& \hfill {\left[ϵ\frac{\partial \overline{u}}{\partial {\mathbf{n}}_{\Gamma }}\right]}_{\Gamma }& =0\mathrm{on}\Gamma ,\hfill \mathrm{(5d)}& \hfill \overline{u}& =0\mathrm{on}\partial \Omega .\hfill \end{array}$$

The trial and test space for $\overline{u}$ is $P_1$ conforming based IFEM (${\widehat{S}}_h(\Omega )$). By solving following system $\overline{u}$ is obtained: Find $\overline{u}\in {\widehat{S}}_h(\Omega )$, satisfying

$$\begin{array}{cc}\hfill & \sum _{T\in {\mathcal{T}}_h}\left({\int }_Tϵ\nabla \overline{u}·\nabla \varphi d\mathbf{x}+{\int }_T{\kappa }^2\sinh (\overline{u}+u^{*})\varphi \mathrm{d}\mathbf{x}\right)+\sum _{e\in {\mathcal{E}}_h}\left(-{\int }_e{\{ϵ\nabla \overline{u}·{\mathbf{n}}_e\}}_e{\left[\varphi \right]}_{e}ds\right.\hfill \\ \hfill & \left.-{\int }_e{\{ϵ\nabla \varphi ·{\mathbf{n}}_e\}}_e{\left[\overline{u}\right]}_{e}ds+{\int }_e\frac{\sigma }{\left|e\right|}{\left[\overline{u}\right]}_e{\left[\varphi \right]}_{e}ds\right)={\int }_{\Gamma }{J}_2\varphi ds-\sum _{T\in {\mathcal{T}}_h}{\int }_Tϵ\nabla u^{*}·\nabla \varphi d\mathbf{x}\hfill \\ \hfill & -\sum _{e\in {\mathcal{E}}_h}\left({\int }_e{\{ϵ\nabla u^{*}·{\mathbf{n}}_e\}}_e{\left[\varphi \right]}_{e}ds+{\int }_e{\{ϵ\nabla \varphi ·{\mathbf{n}}_e\}}_e{\left[u^{*}\right]}_{e}ds+{\int }_e\frac{\sigma }{\left|e\right|}{\left[u^{*}\right]}_e{\left[\varphi \right]}_{e}ds\right).\hfill \end{array}$$

(6)

for all $v_h\in {\widehat{S}}_h(\Omega )$.

Finally, the entire algorithm to obtain the numerical solution $u_h$ of PBE is following (Algorithm 1):

Algorithm 1: IFEM.

(1) Construct $u_h^{*}$ which satisfies (4). (2) Solve (6) for $\overline{u}$ (3) Compute $u_h=\overline{u_h}+u_h^{*}+u^s$.

In [9], they verified the convergency property of Algorithm 1 numerically.

Proposition 1.

Suppose u is the solution of (1) and $u_h$ is a numerical solution obtained by Algorithm 1. There exists a constant $C>0$ such that

$$\begin{array}{c}\hfill \sum _{T\in {\mathcal{T}}_h}\left(\right||u-u_h{\left|\right|}_{0,T}+h\left|\right|u-u_h{\left|\right|}_{1,T})\le C{h}^2{\left|\right|u\left|\right|}_{{\tilde{H}}^2(\Omega )}.\end{array}$$

2.3. Formatting of Datasets and Data Preprocessing

In this subsection, we describe the generation process of the datasets which will be used to train neural networks. One of our goals is to develop a robust neural network regardless of the interface shapes and locations of charges. Thus, we generate various interface shapes and different locations of charges. Figure 1 shows some typical examples of such interface/charge locations. Once $u_h$ is obtained via Algorithm 1, we regularize the solutions as below since $u_h$ blows-up at the charge locations:

$$\begin{array}{cc}\hfill u_h\left(\mathbf{x}\right)& =L,\mathrm{if}\mathbf{x}\mathrm{is}\mathrm{the}\mathrm{charge}\mathrm{location},\hfill \\ \hfill u_h\left(\mathbf{x}\right)& =u_h\left(\mathbf{x}\right),\mathrm{otherwise},\hfill \end{array}$$

where L is some fixed number to replace ∞ values. In this work, we set $L=3$.

We construct datasets in the form of $(X_i,Y_i)$’s, $(i=1,\dots ,nSamples)$ which will be used to train neural network. Figure 2 shows the summarized process of generating typical samples $(X_i,Y_i)$. The variable $X_i$’s are three-dimensional data of size $(n,n,2)$ which contain the domain/charge information and $Y_i$’s are three-dimensional data of size $(n,n,1)$ which represent electro-potentials obtained by Algorithm 1. If the meshgrid ${\mathcal{T}}_h$ used in Section 2.2 has $m\times m$ nodes, we will set $n=m$. The first channel of $X_i$ contains the location of charges; $X(k,\ell ,1)=1$ if charge is located on $(k,\ell )$-th nodes of meshgrids and $X(k,\ell ,1)=0$ otherwise. The second channel of $X_i$’s describe the shape of the solvent–solute interface as follows

$$\begin{array}{cc}\hfill X(k,\ell ,1)=1,& \mathrm{if}(k,\ell )\text{-}\mathrm{th}\mathrm{nodes}\mathrm{are}\mathrm{located}\mathrm{in}{\Omega }^{-},\hfill \\ \hfill X(k,\ell ,1)=0,& \mathrm{Otherwise}.\hfill \end{array}$$

For $Y_i$, the point-wise values of $u_h$ on the nodal points of ${\mathcal{T}}_h$ are subtracted to $Y_i$, i.e., $Y_i(k,\ell )=u_h(x_k,y_{\ell })$. Finally, we normalize the $Y_i$ as below so that the $Y_i$’s values lie between $[0,1]$:

$$\begin{array}{c}\hfill Y_i=\frac{Y_i-min}{max-min}.\end{array}$$

Here, $max$ and $min$ are global maximum and minimum of the $Y_i$ before normalization which values are 3 and 0 respectively.

2.4. ResNet Based Artificial Neural Network for Poisson–Boltzmann Equation

In this subsection, we propose an ANN that predicts electro-potential solutions of PBE. In order to create an ANN that is robust to various combinations of charge and interface positions, we shall need a deep network with a large number of parameters. However, it is known that the so-called "gradient-vanishing" phenomenon occurs in deep neural networks without some proper care. This is because as the number of hidden layers increases, the correction terms transferred via the back-propagations tend to vanish. One of the works suggested to overcome the difficulty was proposed by He et al. [17], in which the ResNet was introduced. In [17], the concept of skip-connection is introduced, whose role is to ensure the correction term is transferred to the front layers without vanishing in the back-propagation process.

Let us introduce our version of ResNet for the solutions of PBE equations following the way of [17]. First, the concept of Res-Blocks shall be needed which plays an important role in ResNet. Let us denote convolution mapping with n by n kernel with m-number of filters as $CONV(kern=n,gen=m,stride=1)$. One residual block is described as

$$\begin{array}{c}\hfill Y=F(\mathbf{X};W)+\mathbf{X}.\end{array}$$

Here, $F(\mathbf{X};W)=(CONV·\sigma ·CONV)\left(\mathbf{X}\right)$ where $\sigma$ is a RELU activation function. In particular, we denote the residual block as RES-BLOCK$(kern=n,gen=m)$ (Figure 3).

We are in a position to define our version of ResNet, ResNet($\ell ,m)$, whose brief illustration is given in Figure 4. First, we apply a convolutional layer with ℓ number of filters. Next, the samples are downsized with a convolutional layer with $strides=2$ but with increasing filter numbers. Then, residual blocks are applied repeatedly m times. Then, the sizes of the resulting filter maps are upsized via a transposed convolutional layer with $strides=2$. Finally, the output is obtained by applying a sigmoid function.

To train ResNet($\ell ,m)$ which can predicts solutions of PBE, we define a least square type objective function was employed as below

$$\begin{array}{c}\hfill Obj\left(\theta \right)=\frac{1}{n}\sum _{i=1}^n{(Y_i-{\widehat{Y}}_i)}^2,\end{array}$$

where Y are IFEM-generated solutions and $\widehat{Y}$ are ResNet-generated solutions. The ADAM optimizer was used [40] to train the networks.

3. Results

We consider a domain $\Omega ={[-1,1]}^2$ and we consider a uniform triangulation ${\mathcal{T}}_h$ constructed by right triangles whose size $h=2^{-6}$, resulting in $128\times 128$ nodes. Additionally, we consider an interface which is a sine-type perturbed circle,

$$\begin{array}{c}\hfill r=r_0(1+a_{0}sin(\theta -{\theta }_0)+a_{1}sin\left(2(\theta -{\theta }_1)\right)+a_{2}sin\left(3(\theta -{\theta }_2)\right)\end{array}$$

Here, a sign of r determines the subdomain, i.e. when $r<0$ it means solute domain; when $r>0$ it means solvent domain. The parameters are chosen as in [9,41] whose values are

$$\begin{array}{c}\hfill {ϵ}^{-}\approx 2,{ϵ}^{+}\approx 80,\\ \hfill {\kappa }^2\approx 8.847.\end{array}$$

We generate the 12,000 number of PBE samples via Algorithm 1. To generate samples, we used intel CPU Core(TM) i9-10940X. Here, 1000 samples were assigned in 12 CPUs (in a parallel way), thus using multiple CPUs at the same time we have generated datasets within 21,584 seconds. First, 8000 samples were assigned as train sets, next, 1000 samples were assigned as the validation set, the remaining 3000 samples were assigned as test samples. For all tests, the learning rate of the ADAM optimizer was set to ${10}^{-4}$ and the epoch number was set to 200.

In order to find suitable parameters in the ResNet($\ell ,m)$, we conducted various experiments with various combination of $(\ell ,m)$. The number of total layers, kernels, and neurons for each case are summarized in

Table 1. The number of total layers, kernels, neurons appearing in ResNet$(\ell ,m)$ as ℓ and m change. A size of input ($\mathbf{X}$) is $(128,128,2)$ and a size of output ($\widehat{\mathbf{Y}}$) is $(128,128,1)$ for each network.

ResNet($\mathit{\ell },\mathit{m}$)	# Layers	# Kernels	# Parameters
(16, 10)	28	2913	3,147,921
(16, 20)	48	5473	6,099,601
(32, 10)	28	5825	12,581,153
(32, 20)	48	10,945	24,382,753
(64, 10)	28	11,649	50,303,553
(64, 20)	48	21,889	97,499,713

. Additionally, the change of data structure as input $\mathbf{X}$ passes through the CONV and RES-BLOCK layers is described in Figure 4. Here, all training and evaluation of the networks were conducted on a single NVIDIA RTX 3090 GPU. The results are reported in

Table 2. Performance of ResNet($\ell ,m$) with various combinations of $(\ell ,m$) in terms of total number of parameters, CPU time, RMSE, and MAE.

ResNet($\mathit{\ell },\mathit{m}$)	# Parameters	CPU time (s)	RMSE	MAE
(16, 10)	3,147,921	2090	4.71 × 10−7	2.25 × 10−4
(16, 20)	6,099,601	2876	4.87 × 10−7	2.20 × 10−4
(32, 10)	12,581,153	3779	3.85 × 10−7	2.20 × 10−4
(32, 20)	24,382,753	5569	3.66 × 10−7	2.01 × 10−4
(64, 10)	50,303,553	8565	3.45 × 10−7	2.13 × 10−4
(64, 20)	97,499,713	13,264	3.81 × 10−7	2.07 × 10−4

in terms of the CPU time taken to train the network and the evaluation of networks on the test sets. For the evaluation, the usual root mean squared error (RMSE) and mean absolute error (MAE) were used whose definitions are

$$\begin{array}{cc}\hfill \mathrm{RMSE}& =\sqrt{\frac{1}{n}\sum _{i=1}^n{|Y_i-{\widehat{Y}}_i|}^2},\hfill \\ \hfill \mathrm{MAE}& =\frac{1}{n}\sum _{i=1}^n|Y_i-{\widehat{Y}}_i|.\hfill \end{array}$$

One observes that as the numbers ℓ and m increase, both the number of parameters and CPU time increase. However, among various networks in Table 1, ResNet(64, 10) has the lowest errors, whose evolutions of the RMSE and MAE as increasing epochs are described at Figure 5. For all tests, RMSEs are below $5·{10}^{-7}$ and MAEs are below $3.3·{10}^{-4}$ showing that the ResNet-generated solutions converge to Algorithm 1-generated solutions via the training process of the neural network. One observes that both RMSE and MAE decrease as epochs increase for both the train set and validation set. However, we see that RMSE and MAE decrease in small amounts after epoch number>190, which is the reason why the target epoch number was set to 200.

Let us compare the shapes of ResNet-generated solutions and IFEM-generated solutions in Figure 6 where first column contains the information of domain/charge information, second and third columns contain the ResNet (${\widehat{Y}}_i$) and IFEM-solutions ($Y_i$) respectively, and fourth column shows the difference between solutions ($|Y_i-{\widehat{Y}}_i|$). We see that both $Y_i$ and $Y_i$ reflect the effects of the singular charges, i.e., the electro-potential blowups near the charges. Additionally, the phenomena of abrupt decreases of potential near the solute–solvent interfaces are described well by ResNet. Thus, we may conclude that ResNet represents the effects of charges and interfaces on the electro-potentials. As for the differences between solutions, $\max |Y_i-{\widehat{Y}}_i|$ are below 0.01 for all cases, which is only 0.33 % compared to the maximum of IFEM solutions. On the other hand, we see that the errors tend to be large near the locations of charges. This issue might be handled in future works.

4. Discussion and Conclusions

Predicting solutions of PBE can be used effectively in many disciplinaries including biophysics, electrochemistry, and colloid chemistry [1,2,3,4,5,6]. However, FEM/FDM-based numerical methods developed so far are rather complicated to implement which makes the real-time simulation of PBE solutions almost impossible. In this study, we develop a ResNet-based ANN to predict solutions of PBE, which we name ResNet$(\ell ,m)$ where ℓ and m are parameters determining the depth and complexity of the network. One of the main advantages of our algorithms is that one can predict solutions of PBE in almost real time once the networks are trained. Additionally, our networks are robust with respect to the locations of charges and interfaces. Thus, our model can be used effectively in various biomolecular simulations including ion-channeling simulations [13] and calculations of diffusion-controlled enzyme reaction rate [32]. To generate train and test sets, we have solved PBE using Algorithm 1 proposed in [9]. The results show that RMSE errors between ResNet generated solutions and IFEM generated solutions are below $5·{10}^{-7}$; thus one may conclude that ResNet are trained enough. Additionally, blow-ups of electrostatic potentials near the singular charge and abrupt decreases near the interfaces are represented reasonably by the predictions of ResNet. Therefore, we have proposed a convenient and efficient way of predicting solutions of PBE by substituting the charge/interface information to ResNet. On the other hand, the differences of ResNet-solutions and IFEM-solutions are found to be relatively high near the charges. The issue of reducing or smoothing the errors near the charges is left as future work. Additionally, we will extend our results in three-dimensional cases. Finally, let us discuss the possible applications of the proposed ResNet. First, our methods can be used effectively in simulations of biological processes based on ion-channeling such as ionic movements through transmembrane channels. Moreover, we can employ our methods in the study of diffusion-controlled enzyme reactions such as analysis of the effects of the substrate–enzyme interactions on the concentration variation of biomolecules or reagents [32].