Physics-Informed Neural Networks for Modal Wave Field Predictions in 3D Room Acoustics

Abstract

The generalization of Physics-Informed Neural Networks (PINNs) used to solve the inhomogeneous Helmholtz equation in a simplified three-dimensional room is investigated. PINNs are appealing since they can efficiently integrate a partial differential equation and experimental data by minimizing a loss function. However, a previous study experienced limitations in acoustics regarding the source term. A challenging but realistic excitation case is a confined (e.g., single-point) excitation area, yielding a smooth spatial wave field periodically with the wavelength. Compared to studies using smooth (unrealistic) sound excitation, the network’s generalization capabilities regarding a realistic sound excitation are addressed. Different methods like hyperparameter optimization, adaptive refinement, Fourier feature engineering, and locally adaptive activation functions with slope recovery are tested to tailor the PINN’s accuracy to an experimentally validated finite element analysis reference solution computed with openCFS. The hyperparameter study and optimization are conducted regarding the network depth and width, the learning rate, the used activation functions, and the deep learning backends (PyTorch 2.5.1, TensorFlow 2.18.0 1, TensorFlow 2.18.0 2, JAX 0.4.39). A modified (feature-engineered) PINN architecture was designed using input feature engineering to include the dispersion relation of the wave in the neural network. For smoothly (unrealistic) distributed sources, it was shown that the standard PINNs and the feature-engineered PINN converge to the analytic solution, with a relative error of 0.28% and $2\times {10}^{-4}$%, respectively. The locally adaptive activation functions with the slope lead to a relative error of 0.086% with a source sharpness of $s=1$ m. Similar relative errors were obtained for the case $s=0.2$ m using adaptive refinement. The feature-engineered PINN significantly outperformed the results of previous studies regarding accuracy. Furthermore, the trainable parameters were reduced to a fraction by Bayesian hyperparameter optimization (around 5%), and likewise, the training time (around 3%) was reduced compared to the standard PINN formulation. By narrowing this excitation towards a single point, the convergence rate and minimum errors obtained of all presented network architectures increased. The feature-engineered architecture yielded a one order of magnitude lower accuracy of 0.20% compared to 0.019% of the standard PINN formulation with a source sharpness of $s=1$ m. It outperformed the finite element analysis and the standard PINN in terms time needed to obtain the solution, needing 15 min and 30 s on an AMD Ryzen 7 Pro 8840HS CPU (AMD, Santa Clara, CA, USA) for the FEM, compared to about 20 min (standard PINN) and just under a minute of the feature-engineered PINN, both trained on a Tesla T4 GPU (NVIDIA, Santa Clara, CA, USA).

1. Introduction

Today, there are persisting challenges in room acoustics simulations, with errors mainly arising from uncertain input parameters of used building material, resulting in a high uncertainty in the used boundary conditions [1,2]. In particular, at low-frequency waves, the material parameters are off [3,4], hindering more accurate simulation results. These uncertain simulation input data underline the importance of using algorithms that allow for the on-the-fly minimization of the model discrepancy with reality. As a selected algorithm, physics-informed neural networks (PINNs) can predict the wave field and infer the material coefficients or boundary condition parameters with the use of data [3]. However, the computation of the time-harmonic acoustic wave field in large 3D geometries represents a challenge for current wave-based numerical and machine-learned methods [4,5]. The solutions to the Helmholtz equation in 3D are essential for understanding low-frequency room acoustic scenarios. The Finite Element Methods (FEM) analysis provides accurate results [4], but the extension to inverse parameters estimation is tedious compared to PINNs. PINNs show promising capabilities for that purpose, such that the interest in tailoring these algorithms to the accuracy of FEM has become an interesting topic.

So far, PINNs have been used to solve the Helmholtz equation [6,7,8] only in 1D and 2D domains for acoustic simulations [3,9,10,11,12,13,14,15], ultra-sound application [16], electromagnetic waveguides [17,18], and aeroacoustics [19]. It was already observed in [11] that the training deteriorates with increasing wave number due to the frequency-principle [20,21] and deteriorates when changing from the Dirichlet boundary condition to a Neumann boundary condition. A weak point regarding the acoustics predictions of previous studies using PINNs is that the solution of the inhomogeneous Helmholtz equation is similar to the widespread exciting source of the partial differential equation. A cosine function in space describes the wave field, and a scaled and unrealistic cosine function models the source covering the whole room. However, typical sources in acoustics are confined [22,23], resulting in a substantial increase in the prediction discrepancies from a 2.5% to 92% relative error [5]. These confined sources excite a spatial periodic wave field. This characteristic is known as the dispersion relation of the acoustic wave. Compared to previous studies focusing on unrealistic smooth excitation [9,11], we will analyze the network’s generalization capabilities towards realistic excitation scenarios caused by a (loudspeaker-like) confined source. A structured hyperparameter study provides intuition about the network key parameter changes (concerning the results of [11]), the activation functions, and various backends, including PyTorch [24], TensorFlow 1 and 2 [25], and JAX [26]. All results are compared to results from a validated FEM analysis using openCFS [4]. Rerunning the training of the PINN reported in [5] on a Tesla T4 GPU with Tensorflow 1, the training time of the 3D PINN took about 84 min compared to a solver time of the FEM analysis of 16 min on a standard notebook CPU. In [11], the randomly distributed point-wise $l^2$ error measure was used as a loss metric. It is highly questionable whether a random distribution of evaluation points computes a reliable error measure for assessing numerical convergence. The accuracy is evaluated in a best-practice manner by comparing the results on a structured evaluation grid with the FEM results and computing an $L^2$ error norm. As a follow-up to the hyperparameter study, additional data points are added by residual-based adaptive refinement. In addition to the standard neural network architectures, the locally adaptive activation functions with slope recovery [27] and the multi-scale Fourier feature network [28] are investigated for potentially improved neural network performance. In addition to these structures, a modified architecture was proposed using feature engineering to include the wave’s dispersion relation in the neural network. It shows how the input features transform the FEM reference solution into a monotonic line that can be more easily predicted by the neural network. For this feature-engineered network structure, a hyperparameter optimization using Gaussian processes-based Bayesian optimization yielded a substantial reduction in the trainable parameters (around 5% compared to the standard PINN formulation) and the training time (around 60 s compared to 20 min of the standard PINN formulation) while it slightly improved the prediction accuracy.

This article starts with Section 2, introducing the numerical examples of 3D room-like geometry, the differential equation with sound–hard wall boundary conditions, and a forcing source including the narrowing of the source by a sharpness parameter s. Section 3 presents the theoretical background of a standard PINN, the parameter space of the experimental hyperparameter analysis, the multi-scale Fourier feature network with randomly sampled Fourier features [28], and the locally adaptive activation functions with slope recovery [27]. Section 4 presents the investigated hypotheses. This is followed by the results section that presents the details of the investigations and the comparison of the FEM analysis by numerical error analysis. Based on the results, the conclusions of the multi-scale Fourier feature network led to the feature-engineered network architecture using the acoustic dispersion relation for feature engineering. Section 6 summarizes the significance and concludes the findings regarding the hypotheses.

2. Room Acoustic Application

A room’s linear acoustic wave field can be described by solving the inhomogeneous Helmholtz equation with suitable boundary conditions [29]. We present the application example as it will be essential to understand the details of the used PINN variants (see Section 3).

2.1. Case Study Setup

In particular, a simplified room-like 3D domain $\Omega ={[0,1]}^3$ ${\mathrm{m}}^3$ is considered. This room setup presents the lowest geometrical complexity while retaining the wall reflections interacting with the distributed and confined source regions. The inhomogeneous Helmholtz equation as the governing equation with homogeneous Neumann boundary conditions on $\partial \Omega ={\Gamma }_N$ reads as

$$\begin{array}{cc}\hfill (\Delta +k^2)p(\omega ,\mathit{x})& =g(\omega ,\mathit{x})\mathit{x}\in \Omega \hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \nabla p\left(\mathit{x}\right)·\mathit{n}& =0,\mathit{x}\in {\Gamma }_N.\hfill \end{array}$$

(2)

In this equation, $p\in \mathbb{C}$ is the time-harmonic acoustic pressure, $\mathit{x}=(x,y,z)$ is the space coordinate, $k=\omega /c=2\pi /\lambda$ is the wave number, $\lambda$ the wavelength, $\omega$ the angular frequency, $c=1$ the speed of sound, g the forcing term, $\Delta =\nabla ·\nabla$ is the Laplacian operator, and ∇ the gradient. Homogeneous Neumann boundary conditions model the room walls. The forcing consists of

$$g(x,y,z)=-2k^{2}cos\left(kx\right)cos\left(ky\right)cos\left(kz\right){\mathrm{e}}^{-{\displaystyle \frac{\left|\right|\mathit{x}-{\mathit{x}}_0{\left|\right|}_2^2}{2s^2}}},$$

(3)

with a source sharpness parameter s, the Euclidean distance $\left|\right|\mathit{x}-{\mathit{x}}_0{\left|\right|}_2$, and a source location ${\mathit{x}}_0=(0.5,0.5,0.5)$ m. In the case of $s\to 0$, the source leads to a point source confined in the source location ${\mathit{x}}_0$. As a follow-up to the previous 3D room acoustic examples [9,10,11,12,13,14,15], the source is modeled more realistically by a point source in the center of the room modeled for sufficiently small s.

2.2. Analytic Solution

For $s\to \infty$, the forcing can be simplified to

$$g(x,y,z)=-2k^{2}cos\left(kx\right)cos\left(ky\right)cos\left(kz\right).$$

(4)

This source leads to an analytic solution of

$$p(x,y,z)=cos\left(kx\right)cos\left(ky\right)cos\left(kz\right)$$

(5)

and it satisfies the homogeneous Neumann boundary condition $\nabla p(x,y,z)·\mathit{n}=0$. In the current case, the solution is only real-valued $p\in \mathbb{R}$; however, in the general case, the solution is complex $p=\Re \left(p\right)+i\Im \left(p\right)$ and may be treated by one network with two output neurons or two independent networks with one output each, providing the independent solutions parts $\Re \left(p\right)$ and $\Im \left(p\right)$. In the case of a finite s, the physical system is in resonance, and slight damping is included to make the system of equations well-posed $c^2=c_0^2+i{c}_1^2$, with $c_1^2=0.04$. In this case, the pressure $p\in \mathbb{C}$ is a complex function.

2.3. Spatial Frequency Content

The spatial frequency $f_{\mathrm{x}}=1/{\lambda }_{\mathrm{x}}$ of the solution is directly connected to the excited source by the linearity of the Helmholtz equation and is provided by the value of ${\lambda }_{\mathrm{x}}=\lambda$. For cases with $s>\lambda$, the solution wavelength is primarily determined by $\lambda$. However, in the case of $s<\lambda$, the wavelength content of the Gaussian envelope becomes important, which is determined by the Fourier transform of it

$$\mathcal{F}\left[{\mathrm{e}}^{-{(x/s)}^2}\right]\left(f_{\mathrm{x}}\right)=\sqrt{2\pi }s{\mathrm{e}}^{-2{\left(\pi s{f}_{\mathrm{x}}\right)}^2}.$$

(6)

The frequency content governed by the Gaussian $f_{\mathrm{x}}$ is sufficiently low for values of $f_{\mathrm{x}}$ being smaller than two standard deviations. In our case, this condition reads $f_{\mathrm{x}}<1/\left(\pi s\right)$ or more specifically $\lambda >\pi s$.

2.4. Validated FEM Reference Simulation

An experimentally validated FEM setup [22] is used to computed the reference solution with openCFS [30]. The mesh approximates a wavelength $\lambda$ per 50 degrees of freedom $N_{dof}/\lambda$, satisfying the conditions of [31]. The mesh consists of 1.06 million elements. For quadratic polynomial basis functions, the mesh will provide accurate results for $\lambda >0.06125$ m and $s>0.06125/\pi$ m [32]. A simulation run on a standard notebook CPU (AMD Ryzen 7 Pro 8840HS with 3.3 GHz (AMD, Santa Clara, CA, USA) ) takes 15 min and 30 s. The mesh study conducted led to the relative errors

$$e_{\mathrm{rel}}=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^{|{\mathcal{T}}_{\mathrm{test}}|}{\left(p_{i,\mathrm{FEM}}-p_{i,\mathrm{FEMx}}\right)}^2}{{\sum }_{i=1}^{|{\mathcal{T}}_{\mathrm{test}}|}{\left(p_{i,\mathrm{FEM}}\right)}^2}}},$$

(7)

where $|{\mathcal{T}}_{\mathrm{test}}|$ is the number of evaluation points, $p_{i,\mathrm{FEM}}$ is the FEM reference pressure of the $N_{dof}/\lambda =50$ mesh, and $p_{i,\mathrm{FEM}}$ is the pressure result from the coarser mesh reported in

Table 1. Convergence of the finite element results for varying source sharpness parameters s. For $s\to \infty$, the errors are compared to the analytic case. In the other two cases, the error is compared to the $N_{dof}/\lambda =50$ case.

${\mathit{N}}_{\mathit{dof}}/\mathit{\lambda }$	${\mathit{e}}_{\mathbf{rel}}$ in % $\mathit{s}\mathbf{\to }\mathbf{\infty }$	$\mathit{s}=\mathbf{1}$	$\mathit{s}=\mathbf{0.2}$	CPU Time
20	$1.3\times {10}^{-2}$	1.35	1.69	5 s
25	$4.5\times {10}^{-3}$	0.73	1.18	15 s
30	$1.8\times {10}^{-3}$	0.34	0.66	33 s
35	$8.0\times {10}^{-4}$	0.35	0.46	1 min 11 s
40	$3.5\times {10}^{-4}$	0.43	0.33	3 min 43 s
45	$1.5\times {10}^{-4}$	0.22	0.20	5 min 12 s
50	$5.0\times {10}^{-5}$	-	-	15 min 30 s

.

3. Physics-Informed Neural Networks for Wave-Based Room Acoustics

PINNs combine neural networks and partial differential equations (PDEs) using automatic differentiation and a modified loss function [33]. Figure 1 shows an L-layer-feed-forward neural network [34,35] PINN that can solve the Helmholtz Equation (2) with Neumann boundary conditions.

The following description gives the mathematical definitions of the individual blocks of the schematic in Figure 1. The prediction behavior of an L-layer-feed-forward neural network is defined by an operator from the input $\mathit{x}$ (spatial locations) to the estimated output (pressure of the acoustic wave field) $\widehat{p}\left(\mathit{x}\right)$. Let $\mathcal{N}\left(\mathit{x}\right):{\mathbb{R}}^{d_{\mathrm{in}}}\to {\mathbb{R}}^{d_{\mathrm{out}}}$ be an L-layer-feed-forward neural network which predicts $\widehat{p}\left(\mathit{x}\right)$, with $N_{\ell }$ neurons in the ℓ-th layer. The argument of the nonlinear activation function $\sigma$ is a tensor product of a weight matrix and inputs, and a bias vector is added

$$\sigma \left({\mathit{W}}^{\ell }\sigma \left({\mathcal{N}}^{\ell -1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^{\ell }\right).$$

(8)

The weight matrix in the ℓ-th layer is denoted by ${\mathit{W}}^{\ell }\in {\mathbb{R}}^{N_{\ell }\times N_{\ell -1}}$ and the bias vector by ${\mathit{b}}^{\ell }\in {\mathbb{R}}^{N_{\ell }}$, respectively. The feed-forward neural network is recursively defined by

$$\begin{array}{cc}\hfill \mathrm{Input}\mathrm{layer}:& {\mathcal{N}}^0\left(\mathbf{x}\right)=\mathbf{x}\in {\mathbb{R}}^{d_{\mathrm{in}}}\hfill \\ \hfill \mathrm{Hidden}\mathrm{layers}:& \sigma \left({\mathcal{N}}^{\ell }\left(\mathbf{x}\right)\right)=\sigma \left({\mathit{W}}^{\ell }\sigma \left({\mathcal{N}}^{\ell -1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^{\ell }\right)\in {\mathbb{R}}^{N_{\ell }},\mathrm{for}1\le \ell \le L-1,\hfill \\ \hfill \mathrm{Output}\mathrm{layer}:& {\mathcal{N}}^L\left(\mathbf{x}\right)={\mathit{W}}^L\sigma \left({\mathcal{N}}^{L-1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^L\in {\mathbb{R}}^{d_{\mathrm{out}}}.\hfill \end{array}$$

(9)

In the 3D case, the coordinates of one point $\mathit{x}=(x,y,z)$ are provided as the input at the input layer. Defined by the neural network, the output prediction at the output layer L is calculated by

$$\begin{array}{cc}\hfill \widehat{p}\left(\mathit{x}\right)=& {\mathit{W}}^L\sigma \left({\mathcal{N}}^{L-1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^L={\mathit{W}}^L\sigma \left({\mathit{W}}^{L-1}\sigma \left({\mathcal{N}}^{L-2}\left(\mathbf{x}\right)\right)+{\mathit{b}}^{L-1}\right)+{\mathit{b}}^L=\cdots \hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill =& {\mathcal{N}}^L\circ \sigma \circ {\mathcal{N}}^{L-1}\circ \sigma \circ \cdots \circ \sigma \circ {\mathcal{N}}^1\circ \mathit{x}=\mathcal{N}(\mathit{\theta },\mathit{x}),\hfill \end{array}$$

(11)

being a series of consecutive transformation functions of ${\mathcal{N}}^{\ell }$ and sigma. The transformation is determined by the nonlinear operation $\mathcal{N}(\mathit{\theta },\mathit{x})$. Its arguments are the trainable parameters $\mathit{\theta }$ of all weights and biases $\mathit{\theta }={\{{\mathit{W}}^{\ell },{\mathit{b}}^{\ell }\}}_{1\le \ell \le L}$ and the network input $\mathit{x}$.

During training, $\mathit{\theta }$ will be learned by unlabeled training data ${\mathcal{T}}_{\mathrm{train}}={\left\{{\mathit{x}}_i\right\}}_{i=1}^{|{\mathcal{T}}_{\mathrm{train}}|}$. The neural network operator $\widehat{p}=\mathcal{N}(\mathit{\theta },\mathit{x})$ approximates a continuous function mapping $h\left(\mathit{x}\right)=p$, regarding the minimal error described by the loss function $\mathcal{L}(\mathit{\theta },{\mathcal{T}}_{\mathrm{train}})$. The loss will be minimized by

$$\widehat{\mathit{\theta }}=\underset{\mathit{\theta }\in {\mathbb{R}}^{\mid \theta \mid }}{argmin}\left(\mathcal{L}(\mathit{\theta },{\mathcal{T}}_{\mathrm{train}})\right),$$

(12)

such that the weights and biases $\mathit{\theta }$ are optimized to the converged ones $\widehat{\mathit{\theta }}$. Within the study, we used a mini-batch variant of the gradient-based optimization algorithm Adam [36] and the quasi-Newton method LBFGS [37]. The weights and biases are adapted by the gradient of the loss function ${\nabla }_{\mathit{\theta }}\mathcal{L}$ to decrease it iteratively, using backpropagation [38] and automatic differentiation [39].

The residual of the Helmholtz equation $r_{\mathrm{PDE}}$ and the respective mean-squared error loss function term is defined by

$$\begin{array}{cc}\hfill r_{\mathrm{PDE}}\left(\mathit{x}\right)& =\Delta \widehat{p}\left(\mathit{x}\right)+k^2\widehat{p}\left(\mathit{x}\right)-g\left(\mathit{x}\right)\forall \mathit{x}\in D_{\mathrm{PDE}}\hfill \\ \hfill {\mathcal{L}}_{\mathrm{PDE}}(\mathit{\theta },{\mathcal{T}}_f)& ={\displaystyle \frac{1}{|{\mathcal{T}}_{\mathrm{train},\Omega }|}}\sum _{i=1}^{|{\mathcal{T}}_{\mathrm{train},\Omega }|}{\Vert r_{\mathrm{PDE}}\left(\mathit{x}\right)\Vert }^2\hfill \end{array}$$

(13)

with $\parallel ·\parallel$ being the $l_2$-norm. The loss function incorporates the partial differential equation into the training process. It is evaluated at randomly sampled space points $\mathit{x}$ within the computational domain $\Omega$, forming the dataset ${\mathcal{T}}_{\mathrm{train}}$. It consists of two subsets ${\mathcal{T}}_{\mathrm{train}}={\mathcal{T}}_{\mathrm{train},\Omega }\cup {\mathcal{T}}_{\mathrm{train},\Gamma }$. The points in the domain $\Omega$ form the subset ${\mathcal{T}}_{\mathrm{train},\Omega }={\left\{{\mathit{x}}_i\right\}}_{i=1}^{|{\mathcal{T}}_{\mathrm{train},\Omega }|}$ with a total number of $|{\mathcal{T}}_{\mathrm{train},\Omega }|$ samples. This set is used to evaluate the PDE residual. The second subset collects the points on the boundary $\Gamma$, ${\mathcal{T}}_{\mathrm{train},\Gamma }={\left\{{\mathit{x}}_i\right\}}_{i=1}^{|{\mathcal{T}}_{\mathrm{train},\Gamma }|}$ with a total number of $|{\mathcal{T}}_{\mathrm{train},\Gamma }|$ samples. This set is used to evaluate the Neuman boundary condition residual. The Neumann boundary condition residual $r_{\mathrm{NBC}}$ and the respective mean-squared error loss function term is given by

$$\begin{array}{cc}\hfill r_{\mathrm{NBC}}& =\nabla p\left(\mathit{x}\right)·\mathit{n}\forall \mathit{x}\in D_{\mathrm{NBC}}\hfill \\ \hfill {\mathcal{L}}_{\mathrm{NBC}}(\mathit{\theta },{\mathcal{T}}_{b2})& ={\displaystyle \frac{1}{|{\mathcal{T}}_{\mathrm{train},\Gamma }|}}\sum _{i=1}^{|{\mathcal{T}}_{\mathrm{train},\Gamma }|}{\Vert r_{\mathrm{NBC}}\left(\mathit{x}\right)\Vert }^2.\hfill \end{array}$$

(14)

The total loss function can be described by a weighted sum of individual error terms

$$\mathcal{L}=w_{\mathrm{PDE}}{\mathcal{L}}_{\mathrm{PDE}}+w_{\mathrm{NBC}}{\mathcal{L}}_{\mathrm{NBC}},$$

(15)

where $w_{\mathrm{PDE}}$ and $w_{\mathrm{NBC}}$ are weights.

3.1. Initial Setup [5]

The wavelength is selected to be $\lambda =1/2$ m. The PINN is setup using DeepXDE 1.12.2 (Python 3.10, Cuda 12.2) and trained on a Nvidia Tesla T4 GPU (NVIDIA, Santa Clara, CA, USA). There are 10 random collocation points per wavelength along each direction for training and 30 used for validation are defined. This amounts to $n_{\mathrm{train}}=20$ data points per direction and a total $|{\mathcal{T}}_{\mathrm{train},\Omega }|=n_{\mathrm{train}}^3=8000$ points inside the domain. On the surface, a total number of $|{\mathcal{T}}_{\mathrm{train},\Gamma }|6n_{\mathrm{train}}^2=2400$ is given. A fully connected neural network consists of $L=4$ layers and a layer width of $W=180$ neurons. A sinus activation function is used with Glorot uniform bias and weights initialization [40]. The loss term is defined in Equation (15), with $w_{\mathrm{NBC}}=5$, and $w_{\mathrm{PDE}}=1$. The wave field pressure inside the room on a $z=0.5$ m plane is illustrated in Figure 2. It shows that the real part of the pressure spatially varies between −1 Pa and +1 Pa, and one can see the wavelength. Figure 2a shows the exact solution of the acoustic pressure ($s\to \infty )$. Figure 2b shows a visualization of the predicted acoustic pressure by the PINN. The network is trained over 30,000 iterations by the ADAM optimizer with a learning rate of $\eta =0.001$, resulting in a validation loss of 0.1248. After 200,000 iterations by the ADAM optimizer, the validation loss decreased further to 0.0952 and the relative error $e_{\mathrm{rel}}$ to 0.36% compared to the analytic solution. Figure 2c shows the result after finetuning the results from Figure 2b with the L-BFGS optimizer for 2000 iterations. It brings down the test loss to 0.0609 (see Figure 2c) and $e_{\mathrm{rel}}$ to 0.35%. Visually, there are no differences between these plots, and the method seems to be promising.

3.2. Hyperparameter Study

To further tune the network’s prediction accuracy, an explorative hyperparameter study was performed using the following parameter space ${\theta }_{\mathrm{NN}}$

$${\theta }_{\mathrm{NN}}=\{L-1,W,w_{\mathrm{NBC}},\sigma ,\mathcal{B}\},$$

(16)

with

$$\begin{array}{cc}\hfill L-1=& \{1,2,3,4,5\}\hfill \\ \hfill W=& \{20,40,60,120,180,300\}\hfill \\ \hfill w_{\mathrm{NBC}}=& \{500,50,5,1,1/5,1/50\}\hfill \\ \hfill \sigma =& \left\{\sin ,\mathrm{ELU},\mathrm{GELU},\mathrm{ReLU},\mathrm{SELU},\mathrm{sigmoid},\mathrm{SiLU},\mathrm{swish},\tan \mathrm{h}\right\}\hfill \\ \hfill \mathcal{B}=& \{\mathrm{PyTorch}\mathrm{2.5.1}\left(\mathrm{P}\right),\mathrm{TensorFlow}\mathrm{2.18.0}1\left(\mathrm{T}1\right),\hfill \\ \hfill & \mathrm{TensorFlow}\mathrm{2.18.0}2\left(\mathrm{T}2\right),\mathrm{JAX}\mathrm{0.4.39}\left(\mathrm{J}\right)\}.\hfill \end{array}$$

Regarding the long simulation time reported in [5,11] and the difficulty in interpreting the outcome of [11], we first decided to run several hyperparameters by a manual line search approach. As the initial value of the unvaried hyperparameters, the reference setup from Figure 2b was used. The investigation procedure started with varying the network depth, followed by the network width, the residual weight, the activation function, and finally, the backends. In each step, the best model characteristics were used for the follow-up line search in the next hyperparameter dimension. The best model was selected based on the relative error compared to the validated FEM solution presented in Section 2.4.

3.3. Locally Adaptive Activation Functions

The locally adaptive activation function (LAAF) with slope recovery [27] is a method designed to improve the training speed and performance of physics-informed neural networks. It introduces a scalable parameter, a so-called activation slope $n{a}_i^{\ell }$ (for layer ℓ and neuron i), to the activation function optimized during training (see Figure 3). The locally adaptive activation function neural network (at the neuron level) is defined as

$$\begin{array}{cc}\hfill \mathrm{Input}\mathrm{layer}:& {\mathcal{N}}^0\left(\mathbf{x}\right)=\mathbf{x}\in {\mathbb{R}}^{d_{\mathrm{in}}}\hfill \\ \hfill \mathrm{Hidden}\mathrm{layers}:& 1\le \ell \le L-1,\hfill \\ & \sigma \left(n{a}^{\ell }{\mathcal{N}}^{\ell }\left(\mathbf{x}\right)\right)=\sigma \left(n{a}^{\ell }({\mathit{W}}^{\ell }\sigma \left(n{a}^{\ell -1}{\mathcal{N}}^{\ell -1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^{\ell })\right)\in {\mathbb{R}}^{N_{\ell }},\hfill \\ \hfill \mathrm{Output}\mathrm{layer}:& {\mathcal{N}}^L\left(\mathbf{x}\right)={\mathit{W}}^L\sigma \left(n{a}^{L-1}{\mathcal{N}}^{L-1}\left(\mathbf{x}\right)\right)+{\mathit{b}}^L\in {\mathbb{R}}^{d_{\mathrm{out}}};\hfill \end{array}$$

(17)

The output prediction at the output layer L is calculated by

$$\widehat{p}={\mathcal{N}}^L\circ \sigma \circ n{a}^{L-1}{\mathcal{N}}^{L-1}\circ \sigma \circ \cdots \circ \sigma \circ n{a}^1{\mathcal{N}}^1\circ \mathit{x},$$

(18)

a series of transformation functions. The slope recovery term

$$S\left(a\right)={\displaystyle \frac{L-1}{{\sum }_{\ell =1}^{L-1}exp\left(\frac{{\sum }_{i=1}^{N_{\ell }}{a}_i^{\ell }}{N_{\ell }}\right)}}$$

(19)

encourages the network to increase the activation slope, leading to faster training. The loss function is extended to include the slope recovery term by

$$\mathcal{L}=w_{\mathrm{PDE}}{\mathcal{L}}_{\mathrm{PDE}}+w_{\mathrm{NBC}}{\mathcal{L}}_{\mathrm{NBC}}+w_{\mathrm{a}}S\left(a\right).$$

(20)

The analysis in [27] shows that LAAFs effectively multiply the gradients by conditioning matrices, accelerating convergence without explicitly calculating these matrices. Experiments demonstrated that LAAF works for deep neural networks in the application of standardized datasets MNIST [41], Fashion-MNIST [42], KMNIST [43], Semeion [44], SVHN [45], and CIFAR [46]. Compared to fixed and global adaptive activations, neuron-wise LAAF exhibited faster training speeds when approximating a discontinuous function and, in combination with PINNs, the inverse problem of identifying the material coefficient of Poisson’s equation and Burgers’ equation in 2D. LAAF contributes to the overall challenge of reducing the number of iterations needed for convergence compared to the fixed activation function. This feature was tested with the backend TensorFlow 1 in DeepXDE and $n=10$ for the reliable hyperparameters with a network depth of $L-1=2$, each layer consisting of 180 neurons; $w_{\mathrm{NBC}}=5$, the n of the DeepXDE implementation of layer-wise (a uniform $a_i^{\ell }=a^{\ell }$ for all neurons in the layer) LAAF is selected to be 2. The parameters selected were a good compromise for the standard PINN.

3.4. Multi-Scale Fourier Feature Networks

The multi-scale Fourier feature architecture (see Figure 4) was proposed in [28] to address the spectral bias and low convergence rates observed in conventional PINN models [47]. The spectral bias expresses that the target function is first learned along the eigendirections connected to the larger eigenvalues of the neural tangent kernel and gradually the remaining components of smaller eigenvalues. Translated to conventional, fully connected neural networks, the eigenvalues are lowered by the increasing frequency of the corresponding eigenfunctions. As a result, the convergence rate of high-frequency components of $p\left(x\right)$ is low [48,49]. This learning tendency is a significant challenge when dealing with multi-scale problems like localized acoustic sources featuring space-dependent high-frequency solution features. The challenge is addressed by embedding multiple Fourier features into the network to learn features across scales and to mitigate the spectral bias behavior. Firstly, the input coordinates are transformed using multiple Fourier feature mappings. The forward pass of M Fourier features with $m=1,\dots ,M$ is defined by

$$\begin{array}{cc}\hfill \mathrm{Input}\mathrm{layer}:& {\mathcal{N}}^0\left(\mathbf{x}\right)=\mathbf{x}\in {\mathbb{R}}^{d_{\mathrm{in}}}\hfill \\ \hfill \mathrm{Fourier}\mathrm{layer}:& {\gamma }^{\left(m\right)}\left(\mathbf{x}\right)=\left[\begin{array}{c}cos\left(2\pi B^{\left(m\right)}x\right)\\ sin\left(2\pi B^{\left(m\right)}x\right)\end{array}\right]\in {\mathbb{R}}^{2d_{\mathrm{in}}\times M}\hfill \\ \hfill \mathrm{Firstst}\mathrm{hidden}\mathrm{layer}:& \sigma \left({\mathcal{N}}^{1,\left(m\right)}\left(\mathbf{x}\right)\right)=\sigma \left({\mathit{W}}^1{\gamma }^{\left(m\right)}\left(\mathbf{x}\right)+{\mathit{b}}^1\right)\in {\mathbb{R}}^{N_1},\hfill \\ \hfill \mathrm{Hidden}\mathrm{layers}:& \sigma \left({\mathcal{N}}^{\ell ,\left(m\right)}\left(\mathbf{x}\right)\right)=\sigma \left({\mathit{W}}^{\ell }\sigma \left({\mathcal{N}}^{\ell -1,\left(m\right)}\left(\mathbf{x}\right)\right)+{\mathit{b}}^{\ell }\right)\in {\mathbb{R}}^{N_{\ell }},\mathrm{for}2\le \ell \le L-1,\hfill \\ \hfill \mathrm{Output}\mathrm{layer}:& \widehat{p}\left(\mathbf{x}\right)={\mathit{W}}^L·\left[\sigma \left({\mathcal{N}}^{L,\left(1\right)}\right),\sigma \left({\mathcal{N}}^{L,\left(2\right)}\right),\dots ,\sigma \left({\mathcal{N}}^{L,\left(M\right)}\right)\right]+{\mathit{b}}^L\in {\mathbb{R}}^{d_{\mathrm{out}}};\hfill \end{array}$$

(21)

where $B^{\left(m\right)}$ represents the preselected matrices with entries randomly sampled from a Gaussian distribution $N(0,{\widehat{s}}_m^2)$. The ${\widehat{s}}_m$ values are chosen because they are additional hyperparameters that may be optimized, and typical values should be in the range of the expected frequency (e.g., $k=1/\lambda$) [28]. Overall, no additional training parameters $\theta$ are introduced by this architecture compared to a standard fully connected neural network.

3.5. Validation and Testing Procedure

The validation and testing procedure is based on the standard convergence evaluation criteria for solving the partial differential in scientific computing [50]. The final physical validation (testing) of the PINN simulation was performed with pyCFS-data 0.0.7 [51] using the relative $l^2$-error measure on an evenly spaced evaluation sample ${\mathcal{T}}_{\mathrm{test}}={\left\{{\mathit{x}}_i\right\}}_{i=1}^{|{\mathcal{T}}_{\mathrm{test}}|}$

$$e_{\mathrm{rel}}=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^{|{\mathcal{T}}_{\mathrm{test}}|}{\left(p_{i,\mathrm{FEM}}-p_{i,\mathrm{PINN}}\right)}^2}{{\sum }_{i=1}^{|{\mathcal{T}}_{\mathrm{test}}|}{\left(p_{i,\mathrm{FEM}}\right)}^2}}},$$

(22)

where $|{\mathcal{T}}_{\mathrm{test}}|$ is the number of evaluation points, $p_{i,\mathrm{FEM}}$ is the FEM reference pressure result, and $p_{i,\mathrm{PINN}}$ is the pressure result from the PINN. This error measure on a structured evaluation point sample is proportional to the $L^2$ error. The evenly spaced evaluation points coincide with the finite vertices of the FEM elements in the mesh.

4. Hypotheses

The core part of this article is the investigation of the influence on the prediction accuracy through narrowing the spread of the excitation source by decreasing the sharpness parameter s. In this preliminary section, the generalization capabilities of the standard neural network for confined sources are investigated. The source function $g\left(\mathit{x}\right)$ is modeled more by a gradually confined source in the center of the room modeled by Equation (3) and illustrated in Figure 5a. The sharpness of the source can be tuned by the parameter s in the exponential part of Equation (3). In Figure 5a, we see a range of s-parameters ranging from a distributed source for $s\to \infty$ to a relatively confined source $s=0.02$ m. When sharpening the excitation $g\left(\mathit{x}\right)$, the neural network must model additional spatial modes and feature a multi-scale problem. As pointed out [28] and indicated by the modal description of a point source [52], it may be difficult for a standard fully-connected neural network to produce accurate results of the acoustic pressure. Figure 5b shows the results provided by the FEM analysis on the selected center line $y=z=0.5$ m. The FEM analysis results show that independently of the source function, the acoustic pressure is primarily dominated by the wave number connected to the time-harmonic oscillation of $0.5$ m.

The maximum pressure amplitude is $max\left(p\right)=\{1.01,1.02,1.11,0.903,0.505,0.123\}$ Pa for $s=\{1,0.5,0.2,0.1,0.05$, $0.02\}$ m, respectively. The lowest level of $s=0.02$ was selected to respect the frequency considerations in Section 2.4 for accurate FEM solutions of the source concerning the FEM mesh. The standard network with hyperparameters ${\theta }_{\mathrm{NN}}=\{L-1=3,W=180,w_{\mathrm{NBC}}=5$, $sin,$ PyTorch} (motivated by [5,28]), and after 200,000 iterations of training with a learning rate of ${10}^{-3}$ previously resulted in large relative errors [5]. The recalculation of the PINNs with respect to the converged FEM solution presented in Table 1 shows that the standard network with hyperparameters ${\theta }_{\mathrm{NN}}=\{L-1=3,W=180$, $w_{\mathrm{NBC}}=5$, $sin,$ PyTorch} and after 200,000 iterations of training with a learning rate of ${10}^{-3}$ resulted in an error, with an increase in the relative error as the source sharpened $s\to 0$ (see

Table 2. Summary of the relative errors of the standard PINN for various source sharpness parameters s compared to the FEM reference solution.

s in m	0.2	0.1	0.05	0.02
$e_{\mathrm{rel}}$ in %	4.52	5.72	7.05	no convergence

).

Therefore, the following hypotheses (H1–H4) were formed to investigate this behavior and improve prediction accuracy.

H1. 

Tuning the hyperparameters positively influences the generalization and prediction accuracy.

H2. 

A layer-wise local adaptive activation function positively influences the generalization and prediction accuracy.

H3. 

Adding data points to the training dataset by loss-residual-based adaptive refinement positively influences the generalization and prediction accuracy.

H4. 

A multi-scale Fourier feature network positively influences the generalization and prediction accuracy.

H1–H4 will be tested by the FEM reference solution.

5. Results

The recent hyperparameter study in [11] presented a framework for automatically analyzing a set of hyperparameters in a Gaussian processes-based Bayesian optimization. Both a 2D and a 3D scenario were studied with a distributed force $s\to \infty$, with a very detailed description of the 2D case (Dirichlet problem), which in turn is not directly translatable to the 3D case (Neumann problem). The representativeness of the study shows limited training runs for the expensive training runs. The hyperparameter optimization study for the 3D case found an optimal configuration with the learning rate ${10}^{-4}$, a width of 207, a depth of 10, and a sine activation function and a boundary weight term of 1; the second best had a learning rate close to ${10}^{-3}$, a width of 292, a depth of 3, and a sine activation function and a boundary weight term of 19.1. The diversity of these two nearly optimal solutions expresses the challenge and high sensitivity of optimizing the network structure for this application. It was reported that for increased wave numbers, the performance of the PINNs was low in the 2D and 3D cases. Furthermore, by uniformly increasing the density of collocation points, the performance of the network prediction capability improved, suggesting that adaptive refinement may lead to further improvements. From the results of [11], a structured hyperparameter exploration was conducted for the general 3D case with a finite source sharpness parameter s to evaluate the generalization regarding the excitation.

5.1. Explorative Hyperparameter Study for $s=$ 1 m

Compared to the previous studies conducted for $s\to \infty$, the study investigates the networks’ capability of calculating the results for a slightly distorted field based on $s=1$ m. In this case, the amplitudes of the solution are marginally higher due to reflections at the sound–hard walls than for the simple example $s\to \infty$.

A relative error less than the relative $l^2$ deviations $e_{\mathrm{rel}}<3.14\%$ from the case $s\to \infty$ indicates that the network has learned a meaningful description of the field. The number of layers for a configuration improved the accuracy, as it can be seen in

Table 3. Error $e_{\mathrm{rel}}$ in percentages for the number of layers and PDE/BC balancing weight hyperparameter of ${\theta }_{\mathrm{NN}}$ compared to FEM results. Yellow-marked entries describe the best-performing parameter set in the direction of variation in the line search, and orange indicates the overall best value. For hyperparameters described as 5 + 50, 5 was chosen for the first step of the training sequence, with 90,000 Adams in this example, and then switched to 50 when continuing with the training by 15,000 iterations using LBFGS. The abbreviation "nc" stands for not converged.

$\mathit{L}-1$	W	${\mathit{w}}_{\mathbf{NBC}}$	$\mathit{\sigma }$	$\mathcal{B}$	Training Sequence	${\mathit{e}}_{\mathbf{rel}}$ in %
1	180	5	sin	P	90,000 Adams	nc
2	180	5	sin	P	90,000 Adams	0.77
3	180	5	sin	P	90,000 Adams	0.45
4	180	5	sin	P	90,000 Adams	0.14
5	180	5	sin	P	90,000 Adams	0.07
1	180	5	sin	P	90,000 Adams + 15,000 LBFGS	nc
2	180	5	sin	P	90,000 Adams + 15,000 LBFGS	0.61
3	180	5	sin	P	90,000 Adams + 15,000 LBFGS	0.42
4	180	5	sin	P	90,000 Adams + 15,000 LBFGS	0.13
5	180	5	sin	P	90,000 Adams + 15,000 LBFGS	0.07
1	180	5 + 50	sin	P	90,000 Adams + 15,000 LBFGS	nc
2	180	5 + 50	sin	P	90,000 Adams + 15,000 LBFGS	0.28
3	180	5 + 50	sin	P	90,000 Adams + 15,000 LBFGS	0.42
4	180	5 + 50	sin	P	90,000 Adams + 15,000 LBFGS	0.13
5	180	5 + 50	sin	P	90,000 Adams + 15,000 LBFGS	0.07
2	180	0.02	sin	P	90,000 Adams	24.15
2	180	0.2	sin	P	90,000 Adams	9.20
2	180	1	sin	P	90,000 Adams	7.19
2	180	5	sin	P	90,000 Adams	0.77
2	180	50	sin	P	90,000 Adams	0.70
2	180	500	sin	P	90,000 Adams	0.019
3	180	5	sin	P	90,000 Adams	0.45
3	180	50	sin	P	90,000 Adams	0.39
3	180	500	sin	P	90,000 Adams	0.041

and for the first cohort (separated by rules in the table). As a training speed accuracy compromise, the networks with $L-1=2$ and $L-1=3$ were selected for further investigations. This trend was observable for all hyperparameter tests provided in Table 3. In the second and third cohorts, a combination of switching the optimizer during the training was tested, and the effect of this on performance was examined. Interestingly, when using $w_{\mathrm{NBC}}=50$, the PDE and the BC loss were initially of the same level and caused an excellent prediction when combining it with the LBFGS optimizer to further iterate on the network trainable parameters (orange row in Table 3). In the final two cohorts, the boundary condition weights were tested. It was found that a relative weight increase from 5 to 500 improves the accuracy. As indicated by [53], an increasing relative boundary condition weight can positively influence the optimization by a more regular loss landscape.

In a subsequent study, the network width was tested, as was its influence on the accuracy (

Table 4. Error $e_{\mathrm{rel}}$ in percentages for the variable layer width as a hyperparameter compared to FEM results. Yellow-marked entries describe the best-performing parameter set in the direction of variation in the line search, and orange indicates the overall best value. The abbreviation “nc” stands for not converged.

$\mathit{L}-1$	W	${\mathit{w}}_{\mathbf{NBC}}$	$\mathit{\sigma }$	$\mathcal{B}$	Training Sequence	${\mathit{e}}_{\mathbf{rel}}$ in %
2	20	5	sin	P	90,000 Adams	6.78
2	40	5	sin	P	90,000 Adams	0.48
2	60	5	sin	P	90,000 Adams	3.49
2	120	5	sin	P	90,000 Adams	2.34
2	180	5	sin	P	90,000 Adams	0.77
2	240	5	sin	P	90,000 Adams	3.90
2	300	5	sin	P	90,000 Adams	5.57
3	20	5	sin	P	90,000 Adams	11.36
3	40	5	sin	P	90,000 Adams	2.35
3	60	5	sin	P	90,000 Adams	1.79
3	120	5	sin	P	90,000 Adams	1.32
3	180	5	sin	P	90,000 Adams	0.45
3	240	5	sin	P	90,000 Adams	0.18
3	300	5	sin	P	90,000 Adams	3.01
2	20	50	sin	P	90,000 Adams	nc
2	40	50	sin	P	90,000 Adams	0.76
2	60	50	sin	P	90,000 Adams	1.39
2	120	50	sin	P	90,000 Adams	1.81
2	180	50	sin	P	90,000 Adams	0.70
3	20	50	sin	P	90,000 Adams	5.44
3	40	50	sin	P	90,000 Adams	1.32
3	60	50	sin	P	90,000 Adams	2.02
3	120	50	sin	P	90,000 Adams	0.057
3	180	50	sin	P	90,000 Adams	2.37

). As expected, narrow networks performed comparably poor. In some occasions for a layer width of 40 neurons, no convergence to the FEM results was obtained. In conclusion, the layer width between 120 and 240 gave acceptable results for this case. Compared to the network’s depth, the width shows less sensitivity in terms of accuracy for a neuron range of 120–240 neurons. The accuracy was in the bounds for all tested backends (

Table 5. Error $e_{\mathrm{rel}}$ in percentages for the backend as a hyperparameter variation compared to FEM results. Yellow-marked entries describe the best-performing parameter set of the backend.

$\mathit{L}-1$	W	${\mathit{w}}_{\mathbf{NBC}}$	$\mathit{\sigma }$	$\mathcal{B}$	Training Sequence	${\mathit{e}}_{\mathbf{rel}}$ in %	Training Time in s
2	120	50	sin	P(A100)	90,000 Adams	1.59	591
2	120	50	sin	P	90,000 Adams	1.81	1502
3	120	50	sin	P	90,000 Adams	0.057	2304
2	120	50	sin	T1	90,000 Adams	0.70	765
3	120	50	sin	T1	90,000 Adams	0.13	1268
2	120	50	sin	T2	90,000 Adams	0.55	825
3	120	50	sin	T2	90,000 Adams	0.39	1263
2	120	50	sin	J	90,000 Adams	0.42	1168
3	120	50	sin	J	90,000 Adams	0.34	2046

), with a somewhat superior performance of TensorFlow 1 leading to an error below 0.12% and the highest training speed of about 20 min (being in the range of the FEM simulation). Regarding the accuracy, the PINN using JAX as a backend led to a low relative ${\mathrm{L}}^2$ error of 0.34%. This low relative error is comparable to the refined FEM simulation. Finally, the PINN was trained on an alternative GPU (NVIDIA A100-PCIE-40GB (NVIDIA, Santa Clara, CA, USA)) with a newer tensor core technology, reducing the training time by a factor of more than 2.5. The execution times are in the range reported previously [11].

Following up on the layer number and width study (

Table 6. Error $e_{\mathrm{rel}}$ in percentages for the activation function as a hyperparameter variation compared to FEM results. Yellow-marked entries describe the best-performing parameter set in the direction of variation in the line search.

$\mathit{L}-1$	W	${\mathit{w}}_{\mathbf{NBC}}$	$\mathit{\sigma }$	$\mathcal{B}$	Training Sequence	${\mathit{e}}_{\mathbf{rel}}$ in %
2	180	5	sin	P	90,000 Adams	0.77
2	180	5	ELU	P	90,000 Adams	nc
2	180	5	GELU	P	90,000 Adams	1.56
2	180	5	ReLU	P	90,000 Adams	nc
2	180	5	SELU	P	90,000 Adams	nc
2	180	5	sigmoid	P	90,000 Adams	nc
2	180	5	SiLU	P	90,000 Adams	9.77
2	180	5	swish	P	90,000 Adams	6.09
2	180	5	tanh	P	90,000 Adams	2.53

), the activation functions were tested $\sigma =\{$sin, ELU, GELU, ReLU, SELU, sigmoid, SiLU, swish, and tanh}. The solution oscillates around zero, yielding positive and negative values. The functions ELU, ReLU, SELU, and sigmoid performed poorly potentially due to their predominantly positive branch. It was confirmed in the tests that none of the networks using those activation functions reached the converged results with an error of less than 50%. The sinus function performed best; the GELU, swish, tanh, and SiLU activation reached similar errors, which were already quite large. In conclusion to H1, the hyperparameter optimization improved the accuracy from initially 0.45% to 0.019%.

Motivated by the improved convergence capabilities, the layer-wise LAAF was tested subsequently. The layer-wise LAAF is a promising approach with additional degrees of freedom and theoretically quick adjustments in learning the function approximation. For the layer-wise LAAF, only a few additional trainable parameters were added to the network (in total parameters at an amount of the number of hidden layers). Overall, the performance of the $n=1$ LAAF was superior compared to other n parameters tested (see

Table 7. Error $e_{\mathrm{rel}}$ in percentages for the layer-wise locally adaptive activation function network compared to FEM results. Yellow-marked entries describe the best-performing parameter set in the direction of variation in the line search.

$\mathit{L}-1$	W	${\mathit{w}}_{\mathbf{NBC}}$	$\mathit{\sigma }$	$\mathcal{B}$	Training Sequence	${\mathit{e}}_{\mathbf{rel}}$ in %
2	180	5	sin	P	90,000 Adams	0.77
2	180	5	LAAF-n 1	T1	90,000 Adams	0.086
2	180	5	LAAF-n 2	T1	90,000 Adams	0.370
2	180	5	LAAF-n 4	T1	90,000 Adams	0.120
2	180	5	LAAF-n 8	T1	90,000 Adams	0.096

). The accuracy was comparable to the one of the finite element method.

In conclusion to this experimental hyperparameter study (H1), we found that a layer number of 2 or 3 will be a reasonable choice for the standard network. The layer width between 120 and 240 is preferable, with the sin activation function in conjunction with switching the training from initial Adams iterations of about 100,000 with a boundary weight of 5 and then switching to L-BFGS using a boundary weight of 50. The layer-wise LAAF PINN structure enhanced the accuracy, and we were able to hold H2. Overall, the accuracy compared to the FEM seems good for $s=1$ m. In the next step, we investigate the potential of adaptive refinement (H3) in terms of the network accuracy and if further data are needed for this application example to improve $s=1$ m and the cases of $s=\{0.2,0.1,0.05,0.02\}$ m, respectively.

5.2. Adaptive Refinement of Training Set

The pre-trained standard network architecture with hyperparameters ${\theta }_{\mathrm{NN}}=\{L-1=3$, $W=180,w_{\mathrm{NBC}}=5,sin,$ PyTorch}, and a learning rate of ${10}^{-3}$, after 200,000 iterations, is used for the adaptive fine-tuning of the model. The idea that adaptive refinement may improve the results emerged by looking at Figure 6a and the overlaid black dots, which seemed to be distributed nicely but were relatively rare compared to a FEM mesh. The adaptive refinement procedure works in the following steps: in the first data refinement step, $j=1,\cdots ,10$, the domain was sampled uniformly for a new dataset ${\mathcal{T}}_{\mathrm{samp},\Omega ,j}$. At these data points, the residual of the loss (15) was evaluated, and the arbitrary choice of 150 data points with the most significant residual were added to ${\mathcal{T}}_{\mathrm{train},\Omega ,j}={\mathcal{T}}_{\mathrm{train},\Omega }\cup ({\mathcal{T}}_{\mathrm{samp},\Omega ,j}\mid {max}_{i=1\cdots 150}\left(\mathcal{L}\left({\mathit{x}}_i\right)\right)),\forall {\mathit{x}}_i\in {\mathcal{T}}_{\mathrm{samp},\Omega ,n}$. After the uniform refinement step, another 25 data refinement steps $j=11,\cdots ,35$ with random data point sampling were added to the training data points. In each refinement cycle, 1000 iterations of the optimizer were carried out, with early stopping activated.

The relative error results (see Equation (22)) for the case $s\to \infty$ present a reduction in the error from 0.35% to 0.33% in the first refinement stage and then another reduction to 0.28% after the whole procedure of adaptive refinement. This reduction of 0.07% points is an improvement of the network’s prediction accuracy by 20%. Figure 6a illustrates the pressure field predicted by the PINN in the evaluation plane ($z=0.5$ m) with a colormap representing the pressure magnitude. Overlaid black dots highlight the training points close to the evaluation plane. Overlaid red dots highlight the training points that the adaptive refinement procedure added and indicate regions where the training error was high before the refinement. Figure 6b shows the pressure field predicted by the PINN compared to the analytic solution on the evaluation line ($y=z=0.5$ m). Notably, the graphs do coincide qualitatively.

In the case of $s=1$ m, the relative error results in (22) were reduced from 0.45% to 0.21% in the first refinement stage and then further decrease to 0.18% after the whole procedure of adaptive refinement. Overall, the improvement was 60% with an improvement of 53.3% after the first refinement stage.

Figure 7 summarizes the adaptive refinement procedure and the added data points to the training set. Notably, more training points were added in the central areas of the plot compared to the case of $s\to \infty$. Interestingly, in Figure 7b, the pressure results of the PINN are in good agreement with the FEM.

Since the relative errors were significant for the cases of $s<1$ m [5] (error of nearly 100%), we expect improvements by using the slightly damped material model for the reference FEM case to obtain non-resonant and hence a bounded solution (see Table 1). For this benchmark, the case $s=0.2$ m is discussed, featuring an error of nearly 4.52%. We expect further improvement to the results from the adaptive refinement procedure. In the adaptive refinement step, the error was reduced to 0.76%, being an acceptable amount of error compared to the FEM results (Table 1).

Due to the adaptive refinement, the point distribution showed a remarkably uniform behavior, and the loss is significant in a large area (see Figure 8). Furthermore, a strong cluster of points close to the source location at $x=y=z=0.5$ m was visible and expected. Overall, the FEM result was captured by the neural network solution, which is illustrated in Figure 8. The findings suggest that the network’s capabilities to solve the forward problem in 3D room acoustics is feasible. In summary to H3, adaptive refinement improves the PINN prediction’s accuracy while the overall training time increases significantly with the number of adaptive refinement iteration cycles.

5.3. Multi-Scale Fourier Feature Networks

The prior knowledge of spatial scales in the system is strongly driven by the behavior of the wave and the emissions coinciding with the wavelength $\lambda$. Furthermore, it is expected that based on the sharpening of the force, the spatial frequency content gets more dispersed over the wave number and can lead to approximation difficulties using a modal field [52]. This was formally shown in Section 2.2. We selected an architecture with the hyperparameters ${\theta }_{\mathrm{NN}}=\{L-1=2,W=80,w_{\mathrm{NBC}}=5,sin,$ TensorFlow 1} and a learning rate of ${10}^{-3}$, and after 200,000 iterations, the errors were evaluated. The Fourier features were selected from a normal distribution with a standard deviation of ${\widehat{s}}_1=0.5$ m and ${\widehat{s}}_1=1.0$ m, which is in the range of the wavelength based on the selected wave number. In the case of $s\to \infty$, the relative error (22) with 1.4% was in the range of the standard feed-forward network with the same number of trainable parameters. This was also observed for the case $s=1$ m with an error of 1.93%. We observed no convergence for all the repeated test runs for $s<1$ m, which was unexpected. In a second test, the activation function of the network neurons was altered to tanh to avoid the possible poor performance of a combination of Fourier feature inputs and sinus activation functions. Similarly, the relative error measure was 1.4%, 2.3%, and no convergence for $s\to \infty$, for $s=1$ m, and $s=0.2$ m, respectively. The simulation time was evenly distributed for all tests, around 2500 s.

Overall, the multi-scale Fourier feature networks performed similarly to the standard model, and the means of H4 did not improve prediction accuracy. A possible explanation is based on the exact linearization of the input–output relation due to the wave’s dispersion relation [29]. In the precise form, the cosine and sine transformations for the s-dependent FEM results are illustrated in Figure 9.

The cosine transformation in Figure 9a analytically fulfills the boundary condition of sound–hard walls and yields approximately linear slopes. These linear and nearly linear behaviors can be predicted easily by the neural network. However, for a slightly shifted wavelength (by 10%), one can identify that the cosine and sine both transform into ellipses, making it comparably hard to solve the wave equation and estimate the solution. As soon as there is a deviation in the wave number, inherently present in the multi-scale network by the random draw of the Fourier features, it is fundamentally hard for the network architecture to find the solution of the wave equation. These facts motivate us to form hypothesis 5 (H5) based on the given angular frequency $\omega$, wave number k, and wavelength $\lambda$.

H5. 

Deterministic Fourier features as the input [54,55] that satisfy the dispersion relation of the wave will improve the prediction accuracy and the performance of the PINN.

5.4. Input Feature Generation

The ordinary input $\mathit{x}$ and additionally the transformed input are fed into the neural network. We select $\lambda =0.5$ m so that the underlying application’s acoustic wave number frequency relation is respected. The transform is motivated by Green’s function of the waves in a box. It can be modeled by a complete orthonormal solution basis as a product of solutions for each Cartesian coordinate

$$\begin{array}{c}\hfill \psi \left(\mathit{x}\right)=\prod _{x_i\in x,y,z}cos\left({\displaystyle \frac{2\pi }{{\lambda }_i}}{x}_i\right).\end{array}$$

(23)

It is important to note that an additive ansatz would fail and not reduce the complexity of the PINN problem. We selected an architecture with hyperparameters ${\theta }_{\mathrm{NN}}=\{L-1=2,W=10,w_{\mathrm{NBC}}=5,sin,$ TensorFlow 1} and a learning rate of ${10}^{-3}$, and, after 20,000 iterations, the errors were evaluated. In doing so, the following errors were reached: $2\times {10}^{-4}$%, 0.20%, 17%, and 45%, $s\to \infty$, $s=1$ m, $s=0.2$ m, and $s=0.02$ m, respectively. This significantly improves the other network structures by ingraining expert knowledge. For $s\to \infty$, the errors are in the range of the FEM method, and training took a few seconds, which is even faster than the FEM solution. The execution time is in the range reported in [11] for the 2D simulations without fixed boundary conditions (Quadro RTX 8000 Nvidia (Santa Clara, CA, USA)). We obtained the convergence of the amplitudes in the case of $s\to \infty$ and $s=1$ m. Discrepancies in the amplitudes and shapes for the cases of $s=0.2$ m and $s=0.02$ m are given. In Figure 10, the pressure prediction of the PINN compared to the analytic or FEM results is illustrated for $s\to \infty$, $s=1$ m, $s=0.2$ m, and $s=0.02$ m. In the case of $s=1$ m, the pressure agrees well with the FEM results. For $s=0.2$ m, additional features with wave numbers of $0.5k_0$, $1.5k_0$, $2k_0$, and $2.5k_0$ are necessary to obtain convergence as illustrated in Figure 10c. The graph of the pressure starts to deviate at the domain’s boundaries from the FEM solution. In the case of $s=0.02$ m, the pressure signal does not adapt to the reference FEM solution.

As a result of the fast training speed (below a minute and 20 times faster than the FEM simulation) in the feature-engineered neural network architecture, an automated hyperparameter optimization using a Gaussian processes-based Bayesian optimization [11] was performed in scikit-learn 1.5.2. Five hundred automated hyperparameter model iterations were performed. The learning rate, the network depth, the network width, the activation function, and the boundary condition loss weight were investigated. In doing so for $s=1$ m, the relative error was reduced from 0.2% to 0.1938%. In contrast to the claimed strong improvements by automated hyperparameter optimization in the loss reduction [11], only a slight improvement in the accuracy of the model was obtained (see

Table 8. Summary of the relative errors and network parameters of the automated hyperparameter optimization with $s=1$.

	Overall Best	Best Sin	Best Sigmoid	Fastest
$e_{\mathrm{rel}}$ in %	0.1938	0.195	0.21	0.204
Training time in s	621	404	89	50
Learning rate	0.0009	0.00077	0.025	0.001
$L-1$	9	3	10	2
W	80	80	12	35
$\sigma$	tanh	sin	sigmoid	sin
$w_{\mathrm{NBC}}$	1	0.1	4.7	5

). The fastest model took 50 s to train and had an error of 0.204%. Regarding hypothesis H5, it was confirmed that the feature-engineered network outperformed the other architectures presented in that article for $s>1$.

6. Conclusions

This study demonstrates the feasibility of using PINNs to model 3D sound fields in a room with sound–hard walls governed by the inhomogeneous Helmholtz equation. The prediction accuracy was assessed by a validated reference FEM simulation in openCFS, which took, on average, 15 min and 30 s to compute on a standard notebook CPU. Three different network architectures and feature engineering were used to improve the pressure field prediction. The most significant improvement was reached by ingraining the characteristics of the Green’s function based on modal expansion via feature engineering as additional input to the PINN. An explorative hyperparameter study provides intuition about the behavior of the standard PINN concerning prediction accuracy. The network depth and width, activation functions, training routine, fine-tuning with different optimizers, backends, and learning rate adjustment improved the prediction capability for a simple, fully connected neural network architecture from an error of 0.45% to 0.019% ($s=1$ m). The backend TensorFlow 1 on the Tesla T4 GPU with Cuda 12.2 and Python 3.10 had the best performance. The layer-wise local adaptive activation function network improved the prediction accuracy of the tested case. However, the neuron-wise algorithm, introducing more additional degrees of freedom, might further improve the capabilities.

Residual-based adaptive refinement improved the prediction accuracy. The refinement workflow connected both structured and randomly unstructured refinement. In each stage, 150 data points were added, adding about 1.5% additional data points. The convergence to the accurate solution reached relative errors as low as 0.28% for distributed sources from the initial 0.35%. In the case of $s=1$ m, the relative error improved by 60%, using adaptive refinement. In the case of $s=0.2$ m, the relative error was reduced drastically by adaptive refinement being similar than the error computed by the mesh study of the FEM results.

Using random Fourier features in the multi-scale Fourier feature network did not yield a higher accuracy. Studying the transformation behavior of the input by random Fourier features revealed visibly that the transformation does not reduce the complexity of the prediction. These visualizations led to the idea that using dispersion relation-based Fourier features may significantly enhance the prediction capabilities and the generalization of the neural network architecture concerning alternative source distributions. This feature-engineered architecture substantially reduces the number of trainable parameters that reach 5% and the training time on the GPU to less than a minute. This fast prediction outperforms the FEM analysis. Also, the accuracy is improved by three orders of magnitude for $s\to \infty$, (${10}^{-3}\%$). However, this is still two orders of magnitude less accurate than the pressure result of the FEM solution (${10}^{-5}\%$). This work underscores the potential of PINNs as memory-efficient alternatives to traditional numerical methods for low-frequency room acoustic simulations. For the case of $s=1$ m, the errors were reduced from initially 0.20% to 0.19% by a Gaussian process-based Bayesian hyperparameter optimization.

This article presented significant progress regarding the generalization of PINNs for confined sources. Challenges regarding the explainability of the network learning capabilities [53] and the generalization to multi-source and different room geometries remain. These challenges will be subject to a follow-up study. Transitional behavior for very localized sources will be a focus of upcoming work, including explainability.