Efficient Representations of Spatially Variant Point Spread Functions with Butterfly Transforms in Bayesian Imaging Algorithms ^†

Abstract

Bayesian imaging algorithms are becoming increasingly important in, e.g., astronomy, medicine and biology. Given that many of these algorithms compute iterative solutions to high-dimensional inverse problems, the efficiency and accuracy of the instrument response representation are of high importance for the imaging process. For this reason, point spread functions, which make up a large fraction of the response functions of telescopes and microscopes, are usually assumed to be spatially invariant in a given field of view and can thus be represented by a convolution. For many instruments, this assumption does not hold and degrades the accuracy of the instrument representation. Here, we discuss the application of butterfly transforms, which are linear neural network structures whose sizes scale subquadratically with the number of data points. Butterfly transforms are efficient by design, since they are inspired by the structure of the Cooley–Tukey Fast Fourier transform. In this work, we combine them in several ways into butterfly networks, compare the different architectures with respect to their performance and identify a representation that is suitable for the efficient respresentation of a synthetic spatially variant point spread function up to a $1\%$ error.

1. Introduction

Images of astronomical objects are the result of measurements by physical instruments and intricate postprocessing. In this procedure, instrument responses play an important role as they build the connection between the signal, i.e., the quantity of interest and the observables. Unfortunately, instrument responses are often non-trivial and hard to model in a simple and numerically efficient form. Examples for such instruments are the X-ray Observatories eROSITA [1] and Chandra [2]. Both are challenging to compute due to their inhomogeneous behavior in terms of space and energy. In order to efficiently make statistical field inferences, for example, by using NIFTy [3,4,5], a Python software package for Numerical Information Field Theory [6,7,8,9], these responses need to be fast and differentiable. One promising candidate for the efficient representation of instrument responses are butterfly transforms, a linear neural network structure inspired by the structure of the Fast Fourier Transform (FFT) algorithm, whose size scales with $\mathcal{O}(NlogN)$.

In many cases, the measurement equation for some data d, taken with an instrument response R of the signal s assuming additive noise n, can be formulated as $d=R\left(s\right)+n$. Regarding photographic instruments this response R is a linear map that can be separated into two operations D and O. Here, D is describing the measurement process of the detector, while O represents the optical properties of the instrument. The latter is also referred to as point spread function (PSF). Since computers are used for the analysis of the experiments performed, the continuous signal space is approximated by a discrete pixelation and thus all operators can be represented as matrices.

If O can be approximated by a circulant matrix, a matrix consisting of cyclic permutations of the same row vector a, its matrix multiplication with any vector simplifies to a discrete convolution with a, meaning that it is spatially invariant and homogeneous, respectively. In many physically relevant cases, this homogeneity can approximately be assumed for a given observed area of the instrument. Additionally, the convolution theorem states that a convolution corresponds to a point-wise multiplication in Fourier space. Consequently, convolutional responses can be represented in an efficient way, due to the fact that one only has to store one N-entry vector instead of a $N^2$ matrix, as well as due to the efficiency one gains by replacing a discrete Fourier transformation by the Fast Fourier Transformation (FFT).

Often this assumption only holds up to a certain degree and in a limited field of view. For spatially variant PSFs and thus non-circulant responses efficient representations are urgently needed. In their paper about learning fast linear transforms algorithms [10], Dao et al. proposed a way to learn fast linear transformation algorithms using so-called butterfly factorizations, which are closely related to the butterfly transforms introduced in this paper. They were able to learn several fast linear transformations, e.g., FFT, discrete sine transform, etc., and showed that their approach can be applied as an efficient replacement of generic matrices in machine learning pipelines. We propose using butterfly transforms to represent spatially variant PSFs in order to build likelihoods for instruments such as eROSITA, Chandra, and many more.

In this paper, we present a way to parameterize butterfly transforms, combine them into networks and compare different network architectures in terms of their efficiency and accuracy. Section 2 describes how butterfly transformations are parameterized and how they are inspired by the structure of the Cooley–Tukey–FFT algorithm. Section 3 gives a short introduction to information field theory and Section 4 describes different designs of likelihoods. In Section 5, we define a metric in order to compare different butterfly network architectures with respect to their capability to represent the synthetic response defined in Section 6. The final results can be found in Section 7.

2. Methods

2.1. Fast Fourier Transformation

Due to the convolution theorem, Fourier transformation is one of the key elements of convolutional processes and thus the algorithm of FFT is highly relevant for the representation of instrument responses on regular grids. The main idea of the FFT is to split the sum in the discrete Fourier transform (DFT) into two sums, over even and odd indices [11]. By using the mathematical properties of the N-th primitive root ${\omega }_N=e^{\frac{-2\pi i}{N}}$, it can be shown that

$${\widehat{f}}_k=\frac{1}{\sqrt{2}}{\widehat{f}}_k^{\mathrm{even}}+\frac{1}{\sqrt{2}}{\omega }_N^k{\widehat{f}}_k^{\mathrm{odd}}\mathrm{and}{\widehat{f}}_{k+\frac{N}{2}}=\frac{1}{\sqrt{2}}{\widehat{f}}_k^{\mathrm{even}}-\frac{1}{\sqrt{2}}{\omega }_N^k{\widehat{f}}_k^{\mathrm{odd}}.$$

(1)

This means that an N-sized Fourier transform can be separated into two $N/2$-sized Fourier transforms along the even and odd indices [12]. The components ${\widehat{f}}_k^{\mathrm{even}}$ and ${\widehat{f}}_k^{\mathrm{odd}}$ can then be used to calculate $f_k$ and $f_{k+\frac{N}{2}}$. Putting together the relations in Equation (1) yields.

$$\left(\begin{array}{c}{\widehat{f}}_k\\ {\widehat{f}}_{k+N/2}\end{array}\right)=\frac{1}{\sqrt{2}}\left(\begin{array}{cc}1& 1\\ 1& -1\end{array}\right)\left(\begin{array}{c}{\widehat{f}}_k^{\mathrm{even}}\\ {\omega }_N^k{\widehat{f}}_k^{\mathrm{odd}}\end{array}\right).$$

(2)

The two smaller Fourier transforms can be separated in the same way, resulting in a divide and conquer algorithm. Assuming that the initial value of N is a power of 2, this splitting can be applied ${log}_2\left(N\right)$ times. Inspired by machine learning language, these iterations are called layers in the following. With N additions in each of these layers, the total computational complexity is about $\mathcal{O}\left(N{log}_{2}N\right)$. Comparing this to a regular DFT with its computational complexity of $\mathcal{O}\left(N^2\right)$ (N components with N summands ) the amount of saved time in the FFT algorithm is significant.

2.2. Butterfly Transform and Convolution

The data-flow diagram visualizing the algorithm of Equation (2) is often called a butterfly diagram, due to its appearance (see Figure 1). Therefore, the abstraction of the FFT algorithm, resulting in a similar data-flow diagram, is called butterfly transform in the following. As the butterfly diagrams always connect to two components, most of the descriptions used in the following concerning their parameterization are 2-dimensional, to keep the notation simple.

In order to generalize the FFT while preserving its efficient structure, we decompose the operations in Equation (2) into a diagonal operator $\Phi$ and a mixing operator $\Theta$ as given in the following.

$$\Phi =\left(\begin{array}{cc}1& 0\\ 0& {\omega }_N\end{array}\right),\Theta =\frac{1}{\sqrt{2}}\left(\begin{array}{cc}1& 1\\ 1& -1\end{array}\right),\mathrm{and}\mathrm{thus}\left(\begin{array}{c}{\widehat{f}}_k\\ {\widehat{f}}_{k+N/2}\end{array}\right)=\Theta \Phi \left(\begin{array}{c}{\widehat{f}}_k^{\mathrm{even}}\\ {\widehat{f}}_k^{\mathrm{odd}}\end{array}\right).$$

(3)

For each component, we introduce free parameters that control how the operation deviates from an ordinary FFT. A general representation of $\Theta$ is obtained by parameterizing it by the sine and cosine of an angle $\theta$,

$${\Theta }_{\theta }=\left(\begin{array}{cc}cos\theta & sin\theta \\ sin\theta & -cos\theta \end{array}\right).$$

(4)

To preserve the generality of the transformation within one layer, the $\theta$s for different connected pairs, denoted by the index k in Equation (3), are independent. That means that for an N-size transformation there are $N/2$$\theta$s, in each layer, regulating the interaction between two connected data points. Considering this parameterization, we get the $\Theta$ from Equation (3), i.e., the one for an FFT, by inserting $\theta =\frac{\pi }{4}$. The operator $\Phi$ and an additional operator $\Gamma$ are parameterized as

$${\Phi }_{\varphi }=\left(\begin{array}{cc}{e}^{i{\varphi }_1}& 0\\ 0& e^{i{\varphi }_2}\end{array}\right),{\varphi }_j\in \mathbb{R}\mathrm{and}{\Gamma }_{\gamma }=\left(\begin{array}{cc}{e}^{{\gamma }_1}& 0\\ 0& e^{{\gamma }_2}\end{array}\right),{\gamma }_j\in \mathbb{R}.$$

(5)

This parameterization of $\Phi$ makes it possible to recover the correct phases for an FFT, but also to change them in an arbitrary way. The combination of $\Theta$ and $\Phi$ is sufficient to represent an entire FFT transform. To obtain an even more general transformation, the diagonal operator $\Gamma$ is introduced, which accounts for the real-valued amplitudes. This leads to a loss of unitarity for ${\gamma }_1,{\gamma }_2\ne 0$ in the combined transformation of $\Gamma$, $\Phi$ and $\Theta$.

Now, we can build a generic butterfly-structured transformation B, using the layered structure of an FFT as a guiding example. The subscript of the operators refers to the layer in the FFT algorithm and thus implies that the correct components are connected.

$$B={\Gamma }_0{\Phi }_0{\Theta }_0\cdots {\Gamma }_j{\Phi }_j{\Theta }_j\cdots {\Gamma }_{n-1}{\Phi }_{n-1}{\Theta }_{n-1}.$$

(6)

Given this butterfly transformation and the structure of a convolution operation, based on the convolutional theorem, a butterfly convolution-like operator O can be formulated as

$$O=B^{†}\Lambda B.$$

(7)

In this equation the $\Lambda$ operator corresponds to the Fourier transformed PSF. Usually physically reasonable PSFs are real valued in position space and thus complex-valued in harmonic space. Therefore, the $\Lambda$ operator is defined as a diagonal operator, with complex values,

$${\Lambda }_{\lambda }=\left(\begin{array}{cc}{e}^{{\lambda }_1}& 0\\ 0& e^{{\lambda }_2}\end{array}\right),{\lambda }_j\in \mathbb{C}.$$

(8)

$B^{†}$, in Equation (7), denotes the adjoint of B. For some experiments, the parameters of B and $B^{†}$ were strictly coupled, called mirrored architecture in the following. For others, the parameters were independent, denoted by different indices, resulting in a non-mirrored architecture:

$$O=B_1^{†}\Lambda B_2.$$

(9)

Since the butterfly structure is strongly related to the FFT, it would make sense to treat multidimensional butterfly transformations in the same way as multidimensional Fourier transformations. Therefore, butterfly transformations can be applied to each dimension separately. However, in this work the 2D application is slightly modified, in a way that the mixing operator $\Theta$ is applied to each axis separately (For the first axis all columns are transformed with the same $\theta$s, whereas for the second axis all rows transformations share the same $\theta$s.), but after this axis-wise $\Theta$ transformation the operators $\Phi$ and $\Gamma$ are applied as diagonal operators. Another approach is to reduce the number of dimensions to one (in this case, as we are dealing with images, from 2D to 1D) and just perform one butterfly transform to this one dimension. For the case of 2-dimensional inputs the dimensionality reduction can be easily done by concatenating all the column vectors to one long vector, which will be called flattening from now on. These two different approaches differ in the number of layers needed by the butterfly algorithm as well as in the number of parameters per layer.

3. Information Field Theory

To reach a better understanding for the area of use for the efficient responses, a brief introduction to information field theory (IFT) [9] will be given. Information field theory is the application of information theory to physical fields. Probably the most important relation within information theory is Bayes’ Theorem,

$$\mathcal{P}\left(s\right|d)=\frac{\mathcal{P}(d,s)}{\mathcal{P}\left(d\right)}=\frac{\mathcal{P}\left(d\right|s\left)\mathcal{P}\right(s)}{\mathcal{P}\left(d\right)},$$

(10)

which connects a posterior with the likelihood, the prior, and the evidence. The likelihood can be computed from the noise statistic $\mathcal{P}\left(n\right|s)$ and the measurement equation, here in the form $\mathcal{P}\left(d\right|s,n)=\delta (d-R\left(s\right)-n)$. Thus, the likelihood is

$$\begin{array}{cc}\hfill \mathcal{P}\left(d\right|s)& =\int dn\mathcal{P}\left(d\right|s,n\left)\mathcal{P}\right(n\left|s\right)\hfill \\ \hfill & =\int dn\delta (d-R(s)-n)\mathcal{P}\left(n\right|s)=\mathcal{P}(n=d-R\left(s\right)\left|s\right).\hfill \end{array}$$

(11)

The prior $\mathcal{P}\left(s\right)$ is chosen with respect to the physical knowledge one has about the observed quantity or situation. The evidence $\mathcal{P}\left(d\right)=\int ds\mathcal{P}\left(d\right|s\left)\mathcal{P}\right(s)$ is needed for the proper normalization of the posterior $\mathcal{P}\left(s\right|d)$. The information Hamiltonian is defined as the negative logarithm of the probability, $\mathcal{H}(d,s)=-ln\left[\mathcal{P}\right(d,s\left)\right]$. Due to the properties of the logarithm and the product rule of probabilities the information Hamiltonian, $\mathcal{H}$ is an additive quantity $\mathcal{H}(d,s)=\mathcal{H}\left(d\right|s)+\mathcal{H}(s)$. Assuming Gaussian priors for signal, $\mathcal{G}(s,S)$, and noise, $\mathcal{G}(n,N)$, and using Equation (11) the Hamiltonians simplify to

$$\begin{array}{cc}\hfill \mathcal{H}\left(s\right)& =-ln\left[\mathcal{G}(s,S)\right]=-ln\left[\frac{1}{\sqrt{2\pi S}}exp(-\frac{1}{2}{s}^{†}{S}^{-1}s)\right]=\frac{1}{2}ln\left|2\pi S\right|+\frac{1}{2}{s}^{†}{S}^{-1}s,\hfill \\ \hfill \mathcal{H}\left(d\right|s)& =-ln\left[\mathcal{G}(n,N)\right]=\frac{1}{2}ln\left|2\pi N\right|+\frac{1}{2}{(d-R\left(s\right))}^{†}{N}^{-1}(d-R\left(s\right)).\hfill \end{array}$$

(12)

One way to find an estimate for the signal s is to maximize the probability $\mathcal{P}\left(s\right|d)$. This can be achieved by minimizing the joint Hamiltonian $\mathcal{H}(d,s)$, with respect to the signal s. This is the maximum a posteriori (MAP) approximation. There are also ways to find an estimate for a signal with uncertainty quantification like metric Gaussian variational inference (MGVI) [13] or geometric variation inference (geoVI) [14]. As a minimization algorithm, Newton-CG [15] was used throughout all experiments. If the measurement process follows Poisson statistics, which is the case for realistic photographic measurements, a Poissonian likelihood model has to be used instead of a Gaussian likelihood model.

4. Parallel and Serial Likelihoods

Models for inference processes in NIFTy are built in a forward way, as so-called generative models. This means that a model of the physical signal is created first, followed by the instrument response, and finally by synthetic data. Applying the IFT formalism, described in Section 3, to a generative model with a butterfly convolution operator as a response yields a likelihood with dependencies on the signal s and the response parameters $\theta$, $\varphi$, $\gamma$, and $\lambda$.

In addition to being able to use butterfly convolution operators with mirrored, non-mirrored, flat, and 2D configurations, they can be combined into a network built in parallel or in series. For the case of n multiple butterfly convolution operators in series, the response operator in Equation (12) is

$$R(s,\theta ,\varphi ,\gamma ,\lambda )=O_1\cdots O_{n}s,$$

(13)

while in the case of n butterfly convolution operators applied in a parallel architecture

$$R(s,\theta ,\varphi ,\gamma ,\lambda )=(O_1+\cdots +O_n)s.$$

(14)

Before using a butterfly network in an imaging application, the response R needs to be trained on signal-data pairs of the instrument. Using these signal-data pairs, the joint Hamiltonian $\mathcal{H}(d,s,\theta ,\varphi ,\gamma ,\lambda )$ is minimized with respect to the response parameters $\theta$, $\varphi$, $\gamma$, and $\lambda$, resulting in a MAP approximation of the instrument. The initial values for these parameters, $\tilde{\theta },\tilde{\varphi },\tilde{\gamma }$, and $\tilde{\lambda }$, are chosen such that all $O_i$ correspond to a convolution with a delta peak ($\tilde{\theta }=\pi /4$, $\tilde{\gamma }=0$, $\tilde{\lambda }=1$, and $\tilde{\varphi }$ according to the needed phases, see Section 2.1). The prior distribution of the parameters is assumed to be Gaussian, with means at the initial values and unit variance. The final goal of the minimization is to obtain an efficient digital twin of the real physical instrument.

Once a butterfly response is trained, it can be used for imaging with the corresponding instrument. For this, the response parameters are fixed to the inferred values $\underline{\theta },\underline{\varphi },\underline{\gamma }$, and $\underline{\lambda }$, resulting in a response operator, which is linear in the signal s. The selection of a suitable generative model for s depends on the observation of interest. In order to obtain an estimate for the physical signal s, the inference algorithms MGVI or geoVI can be used.

5. Evaluation of the Response Approximation

Before using a trained butterfly response in an inference algorithm, it must be certified that the mapping done by the response representation is sufficiently accurate. Therefore, we compare the action caused by a signal, here a point source at position z, $s\left(x\right)=\delta (x-z)$, of the to-be-learned or simulated response with the butterfly response by their absolute difference. This will be called response approximation error

$$E\left(s\right)=\mathrm{abs}\left[R_{\mathrm{sim}.}\left(s\right)-R_{\mathrm{but}.}\left(s\right)\right].$$

(15)

To keep the evaluation simple, unit brightness point sources at all signal domain locations $z\in \Omega$ are considered. In order to quantify the total error with respect to all mapping errors, we calculate the 2-norm (${\parallel s\parallel }_2=\sqrt{{\sum }_{x\in \Omega }{\left|s_x\right|}^2}$) of the 4D matrix ${\mathbf{E}}_z=E\left[\delta (x-z)\right]$, containing the error images for all posible z-values and normalize it by dividing with the 2-norm of the matrix ${\mathbf{R}}_z=R_{\mathrm{sim}.}\left[\delta (x-z)\right]$, containing all true simulated responses resulting in the the total error $\widehat{\zeta }$:

$$\widehat{\zeta }=\frac{{\parallel \mathbf{E}\parallel }_2}{\parallel {\mathbf{R}}_{\mathrm{sim}.}{\parallel }_2}.$$

(16)

6. Synthetic Response

In order to investigate whether and to which degree butterfly networks are capable of approximating spatially variant PSFs, they were trained to approximate a synthetic response. This synthetic response can be regarded as the convolution of the signal s, which is a point source located at the position z, $s\left(x\right)=\delta (x-z)$, with a rotational symmetric PSF with a position dependent shape,

$${\left(Rs\right)}^y={\int }_{\Omega }\mathrm{PSF}(y-x,x)s\left(x\right)dx.$$

(17)

For the PSF a zero centered Gaussian was chosen,

$$\mathrm{PSF}(x,z)=\mathcal{G}(\rho ,{\sigma }^2)=\frac{1}{\sqrt{2\pi {\sigma }^2}}exp\left(-\frac{{\rho }^2}{2{\sigma }^2\left(z\right)}\right),$$

(18)

with $\rho ={\parallel x\parallel }_2$, where x is the coordinate vector of the image plane and ${\parallel x\parallel }_2=\sqrt{x_1^2+x_2^2}$ is its length. The dependence on the position z of the point source is encoded in the variance ${\sigma }^2\left(z\right)$ of the Gaussian. To keep this spatial dependency simple, only the distance from the center of the image c to the point source z, $r={\parallel c-z\parallel }_2$, influences the shape of the PSF. As this absolute value depends on the image resolution, r will be normalized by the maximal distance within the image, $\widehat{r}=r/r_{\max }$, to get a relative measure for the distance being in the interval $[0,1]$. As indicated, the variance ${\sigma }^2$ is a function of this relative distance $\widehat{r}$ between the point source at z and the image center c,

$${\sigma }^2\left(\widehat{r}\right)=a·{\widehat{r}}^2+ϵ.$$

(19)

The two parameters are set to $a=0.01$ and $ϵ={10}^{-5}$. Following the Equation (19), larger distances $\widehat{r}$ lead to larger values of the variance ${\sigma }^2$. This means that point sources with smaller values of $\widehat{r}$ are convolved with a sharper Gaussian, while point sources being at far distance from the center are convolved with broader Gaussians (see Figure 2). This results in an spatially variant PSF, which can be used to examine the expressiveness of the butterfly architecture.

7. Results

In search of a butterfly network capable of representing spatially variant point spread functions, various architectures were compared, in terms of their ability to represent the synthetic response, differing in their number of butterfly convolution operators (BCOs), mirrored (mr) or non-mirrored (nmr) architecture, flat or 2D network design, serial or parallel built likelihood (see

Table 1. Parameters and results for all seven network architectures. The density is here defined as the ratio of the number of parameters and the number of entries in a full matrix representation (16${}^4$ = 65,536). A lower density indicates a higher efficiency of the representation.

Network Name	${\mathbf{Net}}_1$	${\mathbf{Net}}_2$	${\mathbf{Net}}_3$	${\mathbf{Net}}_4$	${\mathbf{Net}}_5$	${\mathbf{Net}}_6$	${\mathbf{Net}}_7$
# BCOs	1	2	3	3	3	3	3
architecture	mr	mr	mr	nmr	mr	nmr	nmr
design	flat	flat	flat	flat	2D	2D	flat
likelihood	serial	serial	serial	serial	serial	serial	parallel
$\widehat{\zeta }$ in %	$7.96$	$3.14$	$2.00$	$1.04$	$2.45$	$1.50$	$6.86$
# parameters	5632	$\mathrm{11,264}$	$\mathrm{16,896}$	$\mathrm{32,256}$	7872	$\mathrm{14,208}$	$\mathrm{32,256}$
Density in %	$8.59$	$17.19$	$25.78$	$49.22$	$12.01$	$21.68$	$49.22$

). All of these networks were trained to approximate the synthetic response described in Section 6 until the optimization was sufficiently converged (300 Newton steps). As training data, a set of all possible PSFs within the given pixelation of $16\times 16$ was used. The signals were fixed to be point sources with brightness values 40 at the corresponding positions and the noise covariance N was set to be diagonal with entries of ${10}^{-6}$. In order to get a better understanding of the influence of some of these properties on the total approximation behaviour, the networks are regarded separately and with respect to their final total approximation error $\widehat{\zeta }$ in Table 1.

The comparison of the $\widehat{\zeta }$ value of ${\mathrm{Net}}_1$, ${\mathrm{Net}}_2$, and ${\mathrm{Net}}_3$ with 1, 2, and 3 BCOs, but otherwise the same properties, shows that a higher number of BCOs lowers the total error and thus increases the approximation capability. The second property of interest is the kind of architecture used, mirrored or non-mirrored. Therefore the $\widehat{\zeta }$ value of ${\mathrm{Net}}_3$, with its mirrored architecture, is compared to the one of ${\mathrm{Net}}_4$, with its non-mirrored architecture, while their other properties are equivalent. This shows that the non-mirrored architecture is performing better than the mirrored one. The same conclusion can be drawn by comparing $\widehat{\zeta }$ of ${\mathrm{Net}}_5$ and ${\mathrm{Net}}_6$, which also only differ in their state of mirroring. In a similar way the flattened and the 2D application can be examined. Since ${\mathrm{Net}}_3$ and ${\mathrm{Net}}_5$ only differ in this property, their error values suggest that the flat application is superior to the 2D application with respect to reconstruction capability. This is confirmed by regarding the error of ${\mathrm{Net}}_4$ and ${\mathrm{Net}}_6$, which are in a similar relationship.

Since more BCOs, flattening, and a non-mirrored architecture increase the number of parameters and thus lead to more degrees of freedom, it is assumed that these architectures are more flexible and can approximate the true response in a better way.

For the overall efficiency of the various networks it is not only important to approximate the synthetic response in a optimal way, but also to keep the number of parameters, and thus the network density (Network density is here defined as the ratio of network parameters and number of entries in a full matrix representation.), as low as possible (see Figure 3). In the examined cases, sparser architectures tend to perform worse in comparison to architectures with more parameters. Overall ${\mathrm{Net}}_4$ approximates the synthetic reponse best with an $1\%$ error. ${\mathrm{Net}}_6$, however, has only $44\%$ of the parameters of ${\mathrm{Net}}_4$ and is therefore less dense. This goes hand in hand with a slightly increased approximation error by an absolute value of $0.46\%$ (see Table 1). In the end, the number of parameters of butterfly networks still scales with $\mathcal{O}(NlogN)$. This means that they become less dense with increasing resolution.

8. Discussion

The need for efficient response representations in imaging led to the development of the models presented in this work, which were inspired by earlier research on butterfly matrices [10]. The efficient structure of butterfly matrices, inherited of Fast Fourier Transforms (FFT), results in a subquadratic algorithm scaling with $\mathcal{O}(NlogN)$ that is capable of representing an expensively simulated synthetic response up to $1\%$ error. To this end, ${\mathrm{Net}}_4$, a butterfly convolutional network with three butterfly convolution operators (BCOs) in series, non-mirrored architecture, and flat application is used, which is differentiable and thus suitable for the application as a response in generative models for measurement data. In order to improve the computational performance regarding support of GPUs and parallelization, more advanced machine learning platforms such as TensorFlow [16] or PyTorch [17] could be considered. After sufficient training, the corresponding butterfly network can be used to perform high-fidelity imaging using information field theory. Additonally, other fields of application with a connection to slightly inhomogeneous processes are imaginable. All in all, the method to represent instrument response functions introduced in this work is promising to improve imaging with complex photographic instruments and thus should be considered in further research.