Equivariant Neural Networks and Differential Invariants Theory for Solving Partial Differential Equations ^†

Abstract

This paper discusses the use of Equivariant Neural Networks (ENN) for solving Partial Differential Equations by exploiting their underlying symmetry groups. We first show that Group-Convolutionnal Neural Networks can be used to generalize Physics-Informed Neural Networks and then consider the use of ENN to approximate differential invariants of a given symmetry group, hence allowing to build symmetry-preserving Finite Difference methods without the need to formally derivate corresponding numerical invariantizations. The benefit of our approach is illustrated on the 2D heat equation through the instantiation of an SE(2) symmetry-preserving discretization.

1. Introduction

Numerically solving Partial Differential Equations (PDEs) is of paramount importance for a wide range of applications such as physics, crowd theory, epidemiology and quantitative finance. Conventional methods such as Finite Element or Finite Difference methods have the main advantage of being easy to implement but are highly time consuming. With the rise of Deep Learning in the past decade, new approximate methods based on Physics-Informed Neural Networks (PINN) have been developed [1,2,3] and allow to significantly improve simulation capacities [4,5].

Nonetheless, usual PDEs typically exhibit symmetries [6,7], and it is therefore natural to expect numerical solving schemes to comply with those. For Hamiltonian systems, symplectic integrators [8,9,10,11] have been introduced and have also recently been combined with machine learning techniques for the sake of efficiency [12]. For more general PDEs, symmetry-preserving Finite Difference schemes have been proposed [13,14], with the underlying theory being consolidated in [15]. Practical applications showing improvements with respect to the conventional approach have been presented in [16,17]. However, the formal derivation of the required numerical invariantization of the differential operators becomes more and more challenging as the number of variables increases, hence limiting the applicability of these methods and motivating the need for alternative approaches.

There are mainly two ways to imprint Deep Learning algorithms with symmetries. The first one, recently explored in [18] for PDEs solving, generalizes the data augmentation techniques widely used for image processing tasks and aims at learning symmetries directly from the data. The second one aims at directly encoding the symmetries within the learning algorithms by leveraging the emerging field of Geometric Deep Learning [19,20]. In this context, Equivariant Neural Networks (ENN), initially introduced in [21], have been shown very efficient, leveraging generalized convolution operators such as Steerable Convolution or $G-$convolution [22,23,24,25], and therefore providing equivariance to a wide range of symmetry groups. These equivariance mechanisms are very appealing, as proving theoretical guarantees with respect to the algorithms response to inputs variations and have been shown more efficient than data augmentation techniques from both theoretical [26] and empirical standpoints [27] in several contexts. Yet, these architectures cannot be applied directly to PDEs solving as one would for a conventional PINN and, at the time of writing, only [28] proposes to use steerable convolution to solve PDEs, but it is limited to special cases of symmetries.

Contributions

In this paper, we present two innovative ways of using ENN to solve PDEs while exploiting the associated symmetries. By anchoring in [29], we first show that Group-Convolutionnal Neural Networks can be used to generalize the PINN architecture to encode generic symmetries. By leveraging differential invariant theory [6], we then propose using ENN to approximate differential invariants of a given symmetry group, hence allowing to build symmetry-preserving Finite Difference methods without the need to formally derivate corresponding numerical invariantizations. A key advantage of this approach is that it allows solving any other PDE with the same symmetry group without any retraining. Finally, we illustrate the interest of our approach on the 2D Heat Equation and show in particular that a set of fundamental differential invariants of the roto-translation group SE(2) can be efficiently approximated by ENN for arbitrary functions by training on simple bivariate polynomials evaluations, allowing to easily build SE(2) symmetry-preserving discretization schemes.

2. PDEs and Symmetries

2.1. Systems of PDEs

We are interested in the following in solving systems of PDEs involving one time variable t, p independent space variables $\left(x_1,\dots ,x_p\right)=x\in \mathcal{X}$ and q dependent variables $\left(u_1,\dots ,u_q\right)=u\in \mathcal{U}$, for which a solution is of the form $u=f(t,x)$, with $u^j=f^j(t,x)$ for $j=1,\dots ,q$ in terms of components. In the following, we denote by $\mathcal{X}={\mathbb{R}}^p$, with coordinates $\left(x_1,\dots ,x_p\right),$ the space of the independent variables, and by $\mathcal{U}={\mathbb{R}}^q$, with coordinates u, that of the dependent variable.

We call $n-$order jet space $J^{\left(n\right)}$ the Cartesian product between the space of the independent variables $\mathcal{X}$ and enough copies of the space of the dependent variables $\mathcal{U}$ to include coordinates for each partial derivative of order less or equal than n

$$J^{\left(n\right)}=\mathcal{X}\times \underset{\left(\genfrac{}{}{0pt}{}{p+n}{n}\right)}{\underbrace{\mathcal{U}\times \dots \times \mathcal{U}}}$$

(1)

In the above definition, the binomial coefficient $\left(\genfrac{}{}{0pt}{}{p+n}{n}\right)$ corresponds to the number of partial derivatives (assumed to be smooth enough) with order less than or equal to n. A function $f:\mathcal{X}\to \mathcal{U}$ represented as $u=f\left(x\right)$ can naturally be prolonged to a function $u^{\left(n\right)}=f^{\left(n\right)}\left(x\right)$ from $\mathcal{X}$ to $J^{\left(n\right)}$ by evaluating f and the corresponding partial derivatives, so that $u^{\left(n\right)}=\left\{{\partial }_x^{\alpha }u,\left|\alpha \right|\le n\right\}$ with ${\partial }_x^{\alpha }u$ is the spatial cross-derivative corresponding to the multi-index $\alpha =\left(i_1,\dots ,i_p\right)\in {\mathbb{N}}^p$. According to this formalism, a PDEs system can be then written as

$$\Delta \left(t,x,u^{\left(n\right)}\right)=0$$

(2)

where $\Delta$ is an operator from ${\mathbb{R}}^{+}\times J^{\left(n\right)}$ to ${\mathbb{R}}^q$.

2.2. Symmetry Group and Differential Invariants

We consider a Lie group G of dimension m acting as $g.\left(x,u\right)$ on a sub-manifold $\mathcal{M}\subseteq \mathcal{X}\times \mathcal{U}$, with its Lie algebra $\mathfrak{g}$ generated by the vector fields ${\zeta }_1,\dots ,{\zeta }_m$. We can define the transform of a function $u=f(t,x)$ under the action of G by identifying f with its graph ${\Gamma }_f$ and by defining $g.f=f_g$, where the function $f_g$ is the function associated with the transformed graph $g.{\Gamma }_f={\Gamma }_{f_g}$.

A symmetry group G of a PDEs system is a group G such that if f is a solution, then its transform $f_g$ by the group action is also a solution. We then denote by ${pr}^{\left(n\right)}G$ the prolongation of the group action of G to $J^{\left(n\right)}$ for which a prolonged transform $g^{\left(n\right)}$, for $g\in G$, sends the graph ${\Gamma }_{f^{\left(n\right)}}$ onto ${\Gamma }_{{\left(g.f\right)}^{\left(n\right)}}$ and by ${pr}^{\left(n\right)}{\zeta }_1,\dots ,{pr}^{\left(n\right)}{\zeta }_m$ the corresponding prolonged vector fields. The algebraic invariants $I_G$ of the prolonged group action ${pr}^{\left(n\right)}G$ are called the differential invariants of order n of the group G and can be obtained by leveraging the infinitesimal invariance criteria ${pr}^{\left(n\right)}{\zeta }_i{I}_G=0$. A complete set of independent differential invariants of order n in the sense of Theorem 2.17 of [6] is generically denoted by $\partial {\varphi }_{u,n}^G=\left\{\partial {\varphi }_{u,n}^{G,1},\dots ,\partial {\varphi }_{u,n}^{G,k}\right\}$ in the sequel and is related to the symmetry group of PDEs systems as illustrated in Section 4.2.

3. Equivariant Neural Networks

To incorporate the symmetry information of PDEs into the neural network solver, it is of capital importance to introduce equivariance into neural networks. Multiple approaches have been studied these past years. Those approaches can be separated into two categories: G-CNN and Steerable CNN. The first method is the one we chose to work with and to generalize. The second one can be explored in [23,28,30,31,32].

3.1. G-CNN

The idea behind a G-CNN is to perform the convolution over the group G one wants equivariance with. These kind of convolution layers were first introduced by Cohen and Welling in [33] for discrete groups. There has been some important work on generalizing this approach to other groups [29,34,35,36]. Let us first start with some reminders about the group-based convolution operator and its properties.

Definition 1 

(Group Convolution). Let G be a compact Group and $V_1,V_2$ two vector spaces. Let $K:G\to (V_1,V_2)$ be a kernel, $f:G\to V_1$ be a feature function and μ the Haar measure on G. We define the group convolution for any $s\in G$ by

$$(K\ast f)\left(s\right)={\int }_{G}K\left(r^{-1}s\right)f\left(r\right)d\mu \left(r\right).$$

Proposition 1. 

If the actions of G on $V_1^G$ and $V_2^G$ has regular representations, then the group convolution defined in Figure 1 is G-equivariant.

Figure 1. Action of G with various representations. (a) Regular representation $\rho \left(g\right)=id$ on scalars. (b) Regular representation $\rho \left(g\right)=id$ on vectors. (c) Non-regular representation on vectors.

As illustrated in Figure 1b, regular representations only allow to describe limited group actions. Indeed, this group convolution does not revoke the constraint on the kernel (see [36]) if one wants equivariance to all kinds of actions and not only ones with regular representations.

3.2. A New Convolution

Definition 2 

(Representative Group Convolution). Let G be a compact Group and $V_1,V_2$ two vector spaces. Let $K:G\to \left(V_1,V_2\right)$ be a kernel, $f:G\to V_1$ be a feature function and μ the Haar measure on G. If ${\rho }_1:G\to \left(V_1\right)$ and ${\rho }_2:G\to \left(V_2\right)$ are the linear representations of the action of G on $V_1$ and $V_2$, respectively, we define the representative group convolution for any $s\in G$ by

$$(K⊛f)\left(s\right)={\int }_G{\rho }_2\left(r\right)K\left(r^{-1}s\right){\rho }_1\left(r^{-1}\right)f\left(r\right)d\mu \left(r\right)$$

(3)

Remark 1. 

In what follows, we keep the same definitions for G, K, V, ρ, μ and f.

Theorem 1. 

With the same hypothesis stated in Definition 2, let V denote either $V_1$ or $V_2$ and ρ either ${\rho }_1$ or ${\rho }_2$. If G acts on $V^G$ by

$$\rho \left(g\right)f\left(g^{-1}r\right)\forall g,r\in G\mathrm{and}f:G\to V,$$

then the representative group convolution is G-equivariant.

This new convolution layer is thus very powerful to create an estimator to an equivariant function, because it is itself equivariant by construction. However, we cannot compose it with other non-equivariant operations without breaking the equivariance for the whole network. Therefore, alone, a convolution layer is not that powerful, but a chain of multiple convolution layers is much more interesting.

Lemma 1. 

Any composition of G-equivariant functions is still G-equivariant.

Multiple representative-group convolution layers can be composed to obtain a G-equivariant network. Note that the action of G on the output of the ith layer must match the action of G on the input of the $(i+1)$th layer.

In Table 1, there is a chain of representations ${\rho }_0\to \dots \to {\rho }_L$, but what really matters is only the first one (${\rho }_0$) and the last one (${\rho }_L$). Indeed, by Lemma 1, the whole network is equivariant to G’s action with ${\rho }_0$ on the input and ${\rho }_L$ on the output. Thus, we have full choice over the other representations ${\rho }_{\ell }$ for $1\le \ell \le L-1$.

Table 1. Example of an Equivariant Neural Network.

input layer:	${\mathcal{N}}^0=\mathit{f}\in {\mathit{V}}_0^{\mathit{G}}$,
convolution layers:	${\mathcal{N}}^{\ell }=K^{\ell }⊛{\mathcal{N}}^{\ell -1}\in V_{\ell }^G$,	with ${\rho }_{\ell -1}$, ${\rho }_{\ell }$, for $1\le \ell \le L$.

Remark 2. 

One can still use non-equivariant functions between two hidden layers of the network, as long as these functions are point-wise and that the representations chosen for theses hidden layers are regulars. This covers the main usual architectures for convolutional neural networks.

Lifting the Coordinate Space

The Representative Group Convolution cannot be used right away since the convolution is constructed to be performed on G and not on the input data space (denoted $\mathcal{X}$ in the sequel). The problem is obviated by lifting the coordinates from $\mathcal{X}$ to G. One can find more details of this method in [29].

Definition 3 

(Lifting). Let $Q=\mathcal{X}/G$ be the set of orbits of G. If u is a mapping from $\mathcal{X}$ to V, we set $u^{\uparrow }:G\times Q\to V$ as the lifted version of u, defined by:

$$\begin{array}{ccccc}{u}^{\uparrow }& :& G\times Q& ⟶& V\\ & & \left(r,\overline{o_q}\right)& ⟼& u(r·o_q)\end{array}$$

An element $x\in \mathcal{X}$ is then lifted to a tuple $\left(r_x,\overline{o_q}\right)$.

Definition 4 

(Lifted Action). If G acts on $\mathcal{X}\times V$, then it has an extended action on the lifted space $\left(G\times \mathcal{X}/G\right)\times V$. If $\left(\left(r,q\right),u^{\uparrow }\left(r,q\right)\right)$ is a lifted element, and $g\in G$ is acting by:

$$\begin{array}{cc}\hfill g·\left(\left(r,q\right),u^{\uparrow }\left(r,q\right)\right)& =g·\left(r·o_q,u\left(r·o_q\right)\right)\hfill \\ \hfill & =\left(r·o_q,\rho \left(g\right)u\left(g^{-1}r·o_q\right)\right)\hfill \\ \hfill & =\left(\left(r,q\right),\rho \left(g\right)u^{\uparrow }\left(g^{-1}r,q\right)\right).\hfill \end{array}$$

4. Solving of PDEs with ENN

We discuss in this section two ways of using ENN for solving PDEs, starting with the use of G-CNN to generalize the PINN concept and then by building symmetry-preserving Finite Difference schemes by using the ENN as differential invariant approximators.

4.1. Equivariant PINN

The idea behind PINNs is somewhat straightforward. Let us consider the PDE defined in Section 2.1 but with added boundary conditions on a set B, which gives:

$$\left(E\right):\left\{\begin{array}{cc}\Delta \left(t,x,u^{\left(n\right)}\right)=0\hfill & \forall t,x\in {\mathbb{R}}^{+}\times \mathcal{X}\hfill \\ u\left(x\right)=u_b\left(x\right)\hfill & \forall x\in B\hfill \end{array}\right.$$

Now, we directly estimate a solution u of $\left(E\right)$ at $t_0+dt$ with ${\mathcal{N}}_{\theta }$ an Equivariant Neural Network (ENN) parameterized by $\theta$, taking as input the initial profile of the solution, being u at time $t_0$. This ENN is equivariant to the symmetry group of $\left(E\right)$.

In order to train the ENN ${\mathcal{N}}_{\theta }$ to approximate the solution of the PDE, we introduce the following optimization problem $\left(P\right)$:

$$\left(P\right):{\theta }^{\ast }=arg\underset{\theta }{min}\left\{\mathcal{L}\left(\theta ,\mathcal{T}\right)\right\}.$$

Additionally, the following loss function $\mathcal{L}$:

$$\mathcal{L}\left(\theta ,\mathcal{T}\right)=w_f{\mathcal{L}}_f\left(\theta ,{\mathcal{T}}_f\right)+w_b{\mathcal{L}}_b\left(\theta ,{\mathcal{T}}_b\right)$$

(4)

with

$$\begin{array}{cc}\hfill {\mathcal{L}}_f\left(\theta ,{\mathcal{T}}_f\right)& =\frac{1}{|{\mathcal{T}}_f|}\sum _{x\in {\mathcal{T}}_f}{∥\Delta \left(t,x,{\widehat{u}}^{\left(n\right)}\right)∥}_2^2\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\mathcal{L}}_b\left(\theta ,{\mathcal{T}}_b\right)& =\frac{1}{|{\mathcal{T}}_b|}\sum _{x\in {\mathcal{T}}_b}{∥\mathcal{B}\left({\widehat{u}}_{\theta },x\right)∥}_2^2\hfill \end{array}$$

(6)

with boundary conditions $\mathcal{B}(u_b,x)=0$ on B. Additionally, ${\mathcal{T}}_f$ and ${\mathcal{T}}_b$ are two training sets of randomly distributed points, with ${\mathcal{T}}_b\subset B$ and ${\mathcal{T}}_f\subset \mathcal{X}$.

Remark 3. 

A similar approach has been used by Wang et al. in [28] but with steerable neural networks. They used a U-Net and a ResNet architecture, common for these types of tasks [37], which they made equivariant in order to predict the PDEs’ results. To tackle the constraints on the kernels, they manually design some transformations to make their neural networks equivariant to 4 different actions. Our result is meant to be more general than this case-by-case design. We bypass the kernel’s constraints and have a fully equivariant network to any given group.

4.2. Symmetry-Preserving Finite Difference

We restrict ourselves in the following to PDEs systems of order n with a linear dependency with respect to the time differentials. According to the introduced formalism, we only consider systems of the form

$$\sum _{i=1}^{k_t}{a}_i{\partial }_t^{i}u=\Delta \left(t,x,u^{\left(n\right)}\right)$$

(7)

where $k_t\in \mathbb{N}$ and $\Delta$ is an operator from ${\mathbb{R}}^{+}\times J^{\left(n\right)}$ to ${\mathbb{R}}^q$. The above form covers most of the PDEs encountered in physics, ranging from the heat and wave equations to the Navier Stokes, Shrödinger and Maxwell equations. Assuming that the above PDEs system is regular enough, it admits G as symmetry if and only if the operator $\Delta$ can be expressed as a function of a complete set of differential invariants, i.e., if and only if (2) can can be re-written as

$$\sum _{i=1}^{k_t}{a}_i{\partial }_t^{i}u=F\left(\partial {\varphi }_{u,n}^{G,1},\dots ,\partial {\varphi }_{u,n}^{G,k}\right)$$

(8)

with $F:{\mathbb{R}}^{qk}\to {\mathbb{R}}^q$.

We propose in the following to approximate the differential invariants by neural networks equivariant to the corresponding group action. More precisely, let us consider a discretisation $x^{\left(1\right)},\dots ,x^{\left(n_x\right)}$ (resp. $t^{\left(1\right)},\dots ,t^{\left(n_t\right)})$ of the input space $\mathcal{X}$ (resp. time interval ${\mathbb{R}}^{+})$ and denote $f^{(i,j)}=f(t^{\left(i\right)},x^{\left(j\right)})$ for any $f:{\mathbb{R}}^{+}\times {\mathbb{R}}^p\to {\mathbb{R}}^q$. For a given differential invariant $\partial {\varphi }_f^G$, we are then interested in approximating the vector ${\left(\partial {\varphi }_f^{G,\left(i,j\right)}\right)}_{j=1}^{n_x}$ by the output of an equivariant neural network ${\mathcal{N}}_G$, taking as input the tensor ${\left(f^{\left(i\right)\left(j\right)}\right)}_{j=1}^{n_x}$, so that we have for the jth component

$${\mathcal{N}}_G{\left({\left(f^{\left(i\right)(\ell )}\right)}_{\ell =1}^{n_x}\right)}^j\approx \partial {\varphi }_f^{G,\left(i,j\right)}=\partial {\varphi }_f^G(t^{\left(i\right)},x^{\left(j\right)})$$

(9)

In the following, we propose to train ${\mathcal{N}}_G$ on multivariate polynomial functions in the space variables x of degree d, but other choices could be envisioned depending on the considered problem.

The use of equivariant neural networks is motivated here by the fact that the operator $f\to \partial {\varphi }_f^G$ that we are approximating is equivariant. Indeed, for a function $f:{\mathbb{R}}^{+}\times {\mathbb{R}}^p\to {\mathbb{R}}^q$, $\partial {\varphi }_f^G$ is a function from ${\mathbb{R}}^{+}\times {\mathbb{R}}^p$ to ${\mathbb{R}}^q$, and we can therefore consider its transform $g.\partial {\varphi }_f^G$ according to the action of G by considering its transformed graph, as defined in Section 2.2. As the differential invariant is an algebraic invariant of the prolonged group action ${pr}^{\left(n\right)}G$, it is possible to write

$$g.\partial {\varphi }_f^G=\partial {\varphi }_{g.f}^G$$

(10)

meaning that differential invariant operators are equivariant with respect to the associated group action, as illustrated on Figure 2 for the case of $SE\left(2\right)$.

Figure 2. From left to right and top to bottom: the initial function u, its rotated version $\tilde{u}$, the rotated version of the $SE\left(2\right)$ differential invariant $u_x^2+u_y^2$ (see Section 4.3) and the differential invariant of the rotated function. As expected, computing the differential invariant from u and applying the rotation (bottom left) gives the same results as computing the differential invariant from the rotated function (bottom right).

We now come back to the PDE systems (2) and detail the numerical scheme that we propose here for its integration. The idea is to first train an ENN ${\mathcal{N}}_G^i$ for approximating each of the k differential invariants $\partial {\varphi }_{.,n}^{G,j}$ which are involved and then to integrate by using an explicit scheme in which the differential invariants are replaced by their approximation with the ENN, leading to

$$\sum _{\ell =1}^{k_t}{a}_{\ell }{\partial }_t^{\ell }u(t^{\left(i\right)},x^{\left(j\right)})\approx F\left({\mathcal{N}}_G^1{\left({\left(u^{(i,\ell )}\right)}_{\ell =1}^{n_x}\right)}^j,\dots ,{\mathcal{N}}_G^k{\left({\left(u^{(i,\ell )}\right)}_{\ell =1}^{n_x}\right)}^j\right)$$

(11)

4.3. Numerical Experiments

4.3.1. Approximating $SE\left(2\right)$ Differential Invariants

Here, we considered the case of $\mathrm{SE}\left(2\right)$ for which a generating set of second-order differential invariants is given by

$$\partial {\varphi }_{u,2}^{\mathrm{SE}\left(2\right)}=\left\{\begin{array}{c}u,u_x^2+u_y^2,u_{xx}+u_{yy},u_x^2{u}_{xx}+2u_x{u}_y{u}_{xy}+u_y^2{u}_{yy},u_{xx}^2+2u_{xy}^2+u_{yy}^2\end{array}\right\}$$

(12)

We trained in the following 2 Neural Networks, namely one conventional Convolutional Neural Network ${\mathcal{N}}^{{\mathbb{R}}^2}$ with ${\mathbb{R}}^2-$equivariant layers and one $SE\left(2\right)-$ENN ${\mathcal{N}}^{SE\left(2\right)}$, both of them built to have roughly the same number of parameters (≈$2.2\times {10}^6$). The training set consists in $29\times 29$ evaluations of 2D-polynomials in $\mathbb{R}[X,Y]$ with degrees up to 10 and generated from random coefficients drawn uniformly in [−1, 1]. Polynomials evaluations were performed on the discrete grid ${\left(i/29,j/29\right)}_{i,j=-14,\dots ,14}$. An example of prediction with the trained ${\mathcal{N}}^{SE\left(2\right)}$, together with the corresponding theoretical value, is given in Figure 3.

Figure 3. The $SE\left(2\right)$ differential invariant $u_x^2+u_y^2$ computed for the function u depicted in Figure 2 with an SE(2)-CNN (left) and its theoretical value (right).

4.3.2. Solving the 2D Heat Equation

Here, we consider the 2D heat equation $u_t=u_{xx}+u_{yy}$ defined on a square domain [−a,a]×[−b,b], with the boundary condition $u(t,\pm a,b)=0$ and $u(t,\pm a,-b)=100$ and the initial condition $u_{t=0}=f$, for $f:{\mathbb{R}}^2\to \mathbb{R}$ an arbitrary function. Below, we give the results that we obtained by using an FD scheme relying on the approximation of the 2D-Laplacian, as described in Section 4.2, i.e., by computing the solution according to the following update rule:

$$u^{n+1}=u^n+{\delta }_t\times {\mathcal{N}}_{SE\left(2\right)}^{\Delta }\left(u^n\right)$$

(13)

where $u^n=u(n\times {\delta }_t,.)$. We ran ${10}^5$ steps of the simulation for ${\delta }_t\approx {10}^{-7}$ using the two trained architectures considered in Section 4.3.1, namely ${\mathcal{N}}^{{\mathbb{R}}^2}$ and ${\mathcal{N}}^{SE\left(2\right)}$, and compared the obtained heat profiles with that of the ground truth. The boundary condition was taken into account by overriding the predicted outputs by the conventional second-order derivative approximation for the corner cases. The obtained results are depicted in Figure 4, where we can, in particular, observe the high benefit of preserving the SE(2) symmetry during the numerical integration.

Figure 4. Comparison of the theoretical heat profile of the 2D heat equation with a top ${100}^{\circ }$ boundary condition (see Section 4.3.2) with those obtained through simulation with two symmetry-preserving FD schemes (see Section 4.2) by leveraging ${\mathbb{R}}^2$ (middle) and SE(2) (right) equivariant neural networks.

5. Conclusions and Further Work

We presented two innovative ways of using ENN to solve PDEs while exploiting the associated symmetries. We first showed that G-CNN can be used to generalize the PINN architecture to encode generic symmetries and then proposed using ENN to approximate differential invariants of a given symmetry group, hence allowing to build symmetry-preserving Finite Difference methods. Our approach is illustrated on the 2D Heat Equation for which we, in particular, showed that a set fundamental differential invariants of SE(2) can be efficiently approximated by ENN for arbitrary functions by training on simple bivariate polynomials evaluations, allowing to easily build SE(2) symmetry-preserving discretization schemes.

Additional work will include performing proper benchmarking between the two approaches and more conventional numerical schemes for PDE integration. More complex PDEs with richer symmetry groups such as Maxwell equations could be considered in this context.