Parallel Computation for Inversion Algorithm of 2D ZTEM

Abstract

ZTEM refers to the Z-axis tipper electromagnetic method. The ZTEM method is an airborne magnetotelluric sounding method based on the difference in rocks’ resistivity using the native electromagnetic field. The method is effective in exploring large-scale structures when the ground is fluctuant. The paper introduces the inversion algorithm of 2D ZTEM named the conjugate gradient method. This method, which avoids solving the Jacobi matrix, is very effective but not effective enough when the model is divided into a big grid. This study can perform further computation using parallel computation and then receive the processed data. We compare the results of the serial algorithm with the result of the parallel algorithm, which proves that the parallel algorithm is correct. When the number of processes is between three and six, the speedup ratio is between 1.74 and 3.19. It improves the effectiveness of the parallel algorithm of 2D ZTEM.

1. Introduction

The ZTEM (Z-axis tipper electromagnetic method) method is a native electromagnetic field exploration method. The ZTEM method is developed based on the MT. The abbreviation of magnetotelluric is MT. We can collect magnetic data in a helicopter using the ZTEM method. The method has special advantages in mineral deposit exploration and environmental exploration [1,2,3]. We need to spend a great deal of time collecting MT data on the ground when the MT method is applied to explore the area. If we can collect data in a helicopter, we can save more time and money. It is very difficult to collect data on the electric field, but tipper data are available from a helicopter. The tipper is the ratio of the vertical magnetic field and horizontal magnetic field.

With the development of this instrument, MT data processing technology has matured. The ZTEM method became possible. Domestic and foreign researchers have conducted research on the ZTEM inversion. Xu Zhibo performed research on 2D ZTEM inversion [4], Li ziqiang performed research on 3D ZTEM [5], Gu performed research on a parallel algorithm of 3D MT inversion [6], Holtham conducted research on 3D ZTEM inversion [7], and Sasaki conducted research on 3D ZTEM [8]. The research on the parallel algorithm of 2D ZTEM inversion is limited, so it is essential to perform research on the parallel algorithm of 2D ZTEM inversion. Two-dimensional conjugate gradient inversion is an effective method. The model is divided into m columns and n rows. When the number is large, more computation time is required. The model is divided into 400 × 400 in this paper, and the run time is 9804 s. We need to develop a parallel algorithm to solve the problem. First, we introduce the parallel algorithm.

The rest of this paper is organized as follows. Section 2 introduces 2D modeling and inversion. Section 3 introduces the MPI technology and how to realize the parallel algorithm. Section 4 shows the results and analysis of the parallel algorithm. Section 5 summarizes this paper and proposes an outlook for future work.

2. 2D ZTEM Forward Modeling and Inversion

2.1. ZTEM Forward Modeling

According to the convention of the 2D ZTEM problem, the earth is regarded as a Half space conductor ($z\ge 0$) and the insulated air layer is above the earth. The electromagnetic field source is above the ground (z = −j). The ZTEM vertical field is collected in different positions using the MT method, and the horizontal field is the same as in the MT method.

(1)

Math-physical model

The Maxwell equation differential form is expressed as:

$$\nabla \times E=\mathrm{i}\omega {\mu }_{0}H$$

(1)

$$\nabla \times H=\sigma E$$

(2)

Usually, a Descartes rectangular coordinate system is used. The origin of the coordinate is on the ground, the z-axis is straight down, and the y-axis is horizontal. E is the electric field, H is the magnetic field, ${\mu }_0$ is the magnetic conductivity, $\sigma$ is the conductivity, and $\omega$ is the angular frequency. There are two polarization models in Maxwell equations.

We set the height of the helicopter as h, and the positive z-axis is down to the earth in the coordinate system. The tipper T is as follows:

H_z = T_zxH_x + T_zyH_y

(3)

Because the H_z is only in the TE mode, Equation (3) is simplified as:

H_z = T_zyH_y

(4)

(2)

The finite differential method is used to solve the 2D ZTEM forward model problem

We use the finite differential method to determine the 2D ZTEM forward modeling problem. The main work of the forward problem is to solve the partial differential equation. We set the boundary condition of the equation, and we can obtain the result of the problem.

$$Kv=s$$

(5)

K is the coefficient matrix. v is the magnetic field. s is the right vector of the equation. We solve the equation and obtain the magnetic field. Finally, we can obtain the tipper.

2.2. 2D Inversion Modeling

The forward model is as follows:

$$d=F(m)+e$$

(6)

d is the data vector, m is the model vector, e is the error vector, and F is the forward function [9,10,11].

The data vector $d={\left[d^1,d^2,\cdots ,d^N\right]}^T$, $d^i$ is the tipper in an observation point. The model vector is $m={\left[m^1,m^2,\cdots ,m^M\right]}^T$. The number of the grid is M, and $m^j$ is the resistivity of the unit.

The objective function is as follows:

$$\psi (m)={(d-F(m))}^T{V}^{-1}(d-F(m))+\lambda m^T{L}^{T}Lm$$

(7)

$\lambda$ is the regularization parameter; V is the variance of error vector e; the matrix L is the Laplace operator. The first item is the data-fitting difference, and the second item is the model-smoothing item [12,13,14].

Because a great deal of time is spent computing the Jacobi matrix, in order to save run time, the non-linear conjugate gradient inversion algorithm avoids computing the Jacobi matrix. It computes the product of the Jacobi matrix (or the Jacobi transpose matrix) and a vector x. We need to perform two pseudo-forwards and then obtain $A^T{V}^{-1}e$ and Ap. Figure 1 shows the flow chart of inversion.

This requires a great deal of time to compute the conjugate of the object function $g=-2A^T{V}^{-1}e+2\lambda L^{T}Lm$ and $f=Ap$. When we obtain the value of $A^T{V}^{-1}e$ and $Ap$ [15], we can obtain the step length $\alpha$ and update the model $m=m+\alpha p$.

3. Parallel Algorithm

3.1. MPI Technology

We can realize data casting, sending, receiving, and synchronizing. The MPI supports several data types [16], including complex. The MPI program is one program, but every process can perform different computations. Every process has an ID, and we can allocate tasks to every process by ID.

The variable’s name is the same, but the values of the variables in different processes can be different. If the processes want to share the data, they need to communicate with each other.

3.2. Parallel Algorithm of ZTEM Inversion

The algorithm needs to perform the calculation many times. A calculation has three parts, namely, solving the equation forward (obtaining the value of T), the first pseudo-forward solution to the equation of inversion (obtaining the value of Aty), and the second pseudo-forward solution to the equation of inversion (obtaining the value of Ap). We analyzed the algorithms of forward and inversion and found that solving the equation is the main work and costs a large proportion of the running time. As with six frequencies, when the frequency is different, we can obtain the k, and k is also different. K is in the equation kv = s. In order to obtain the new m, we need to solve the equation 18 times in an iteration process. When there are three processes in the algorithm, every process needs to solve the equation six times. The process needs to solve the equation 18 times in the serial algorithm. The process in the parallel program needs to do less work than the serial program, so the parallel program needs less time to execute the inversion algorithm than the serial program. Therefore, the parallel efficiency is higher [17].

The six frequencies are freq(1)…freq(6) 400, 200, 100, 50, 20, and 10 Hz. When the frequency is higher than other frequencies, solving the equation needs less time. Two kinds of processes exist in the algorithm. One is the main process and the other is the subprocess. The 0 process is the main process. It owns the global data, distributes the work, sends the data, receives the data, and writes the result files. The subprocesses receive data from the 0 process, perform the work, and send the result to the 0 process. When there are three processes in the parallel program, the 0 process is the main process and the first and second processes are the subprocesses. The 0 process also needs to perform the work in the algorithm. We will introduce the main procedure.

(1)

MPI_INIT(), In the MPI environment. The main process reads the file, obtains the frequencies, models the information, and observes the data. MPI_Bcast() is the main process, which broadcasts the data to the other processes. The beginning resistivity of the object model is set as the background resistivity.

(2)

In

Table 1. The allocated frequencies.

Process ID	The Frequency
0	freq(1), freq(2)
1	freq(3), freq(4)
2	freq(5), freq(6)

, the allocated frequencies and process ID information are shown. Every process needs to solve the equation of the relevant frequencies separately. We obtain the apparent Tipper of the observed point after solving the forward equation. We obtain the $A^T{V}^{-1}e$ (the symbol is named aty_mpi in the program). In MPI_recv(), the main process obtains the moderestm_all and aty_all of other processes. Aty_all has six vectors of different frequencyies. Aty is the sum of six vectors. Mode_restm_all has six modrestm_mpi, and the sequence is 1, 2, 3, 4, 5, and 6. Then, we obtain the modrestm to continue computing.

In MPI_Bcast(), the main process broadcasts the data, including m and Aty, to the other processes.

(3)

After computing the value of object function $\psi$, the main process can obtain the rms. If rms is less than 1.5, we proceed to step 4; otherwise, we compute the second pseudo-forward equation. Every process needs to perform computing separately. Process can obtain the Ap_mpi. MPI_recv(), and the main process receives Ap_mpi from the other processes. The process puts it together and obtains the value of Ap_all. Ap_all has six Ap_mpi, of which the order is 1, 2, 3, 4, 5, and 6. We obtain the Ap. In MPI_Bcast(), the main process broadcasts Ap to the subprocesses, and we obtain the value of step length $\alpha$. We can obtain the new model m = m + αp, go to step 2.

(4)

The main process writes the m data in the result file. In MPI_FINALIZE(), we finish the parallel environment. The flow chart of the parallel computing of 2D inversion is shown in Figure 2.

3.3. Computing Environment

The program is developed in the environment as shown in

Table 2. Development environment.

OS:	Win10	Develop language:	fortran
CPU:	intel core i7 2.9 GHz support 8 processes	Compiler:	visual studio 2019
Memory:	16 GB	Parallel environment:	oneAPI HPC

.

4. Results

4.1. Two Low Resistivity Targets Model

The depth of the object rock is 350 m from the ground, and both of the objects are 300 m × 350 m. The width is 300 m, and the height is 350 m. The background resistivity is 100 $\mathsf{\Omega }·\mathrm{m}$. The object rock’s resistivity is 10 $\mathsf{\Omega }·\mathrm{m}$. The distance between the two objects is 900 m. $H_z$ data are collected in the air above the horizon. The height is 100 m. $H_y$ data are collected on the ground. The size of the model’s grid is 400 × 400, and the observed points are from point 1 to point 70 on the ground. The six frequencies range from 400 Hz to 10 Hz. We obtain the forward result including the tipper’s real part and imaginary part from the parallel program.

In Figure 3, the horizontal axis is in the point position, and we choose points 12, 14, 16…68, and 70 to draw the figure. When the frequency is 50 Hz, we set the result of the real part’s computation to o and set the result of the real part’s parallel computation to +; the result is the same. We set the result of the imaginary part’s computation to $\nabla$, and set the result of the imaginary part’s parallel computation to *; the results are the same.

Figure 4 shows the inversion figure of parallel computation.

We set 5% of the forward data as the random error. We add the random error to the forward data and obtain the synthetic data. We take the synthetic data as the collected data and perform inversion computation. When the rms is less than 1.5, we stop the inversion and obtain the inversion result. The color bar value indicates the resistivity. The background resistivity is 100 $\mathsf{\Omega }·\mathrm{m}$. The object rocks’ resistivity is 10 $\mathsf{\Omega }·\mathrm{m}$. We can determine that the program’s result shows two object rocks‘ positions correctly, and the value of resistivity is approximately 10 $\mathsf{\Omega }·\mathrm{m}$ in the figure.

4.2. Proving the Validity of the Program

We analyze the serial program and find that the program calculates the equation of different frequencies separately, so the parallel algorithm distributes the work to every process and receives a result from every process. The parallel calculation result is the same as the result of the serial algorithm. The study compares the forward result of the two algorithms in Figure 3, and the data are the same. Figure 4 is the inversion result. We can observe that the experiment results show the object rocks’ positions, sizes, and resistivity values correctly, so it proves the validity of the algorithm.

4.3. Parallel Efficiency

In order to evaluate the efficiency of the parallel program for different nodes, we calculate parallel speedup and parallel efficiency. Parallel speedup is the time of serial program/the time of parallel for N processes; parallel efficiency is the parallel speedup/the number of processes N.

Table 3. The statistics of the time for the algorithm.

The Mode of Program	The Amount of Processes	The Amount of the Frequency Distributed for Process	The Running Time of Program (s)	Parallel Speedup	Parallel Efficiency
Serial program	1	6	9804	empty	empty
Parallel	3	2, 2, 2	5632	1.74	58%
program	6	1, 1, 1, 1, 1, 1	3069	3.19	53.2%

shows the statistics of the time for the algorithm of 2D ZTEM forward modeling and inversion.

We can observe that the efficiency of the three processes is higher than the efficiency of the six processes in Table 2. The speedup changes and the efficiency decreases when the number of processes in the experiment is six. This is because the higher the number of processes, the more time exchanging data the processes need. Every process needs to perform computation. The computation of higher frequencies needs less time and the computation of lower frequencies needs more time. The process that performs high-frequency computation needs to wait for the process that performs the low-frequency computation. When the number of processes is six, the efficiency is lower than when the number is three. We will perform further study on the parallel algorithm.

5. Conclusions and Future Work

The computation of 2D ZTEM requires a great deal of time, so determining how to save the running time is key to this problem. We found that the parallel method can solve the problem. This study designed the parallel algorithm for 2D ZTEM forward and inversion in windows os. The experiment proved that the algorithm is correct and efficient. We will perform further study for 3D ZTEM forward and inversion based on the study.

Future work will mainly include two aspects: (1) We will optimize the algorithm with openMP and MPI mixed programming by performing an experiment, recording the running time, and calculating the efficiency of the mixed programming. Furthermore, (2) We will design the parallel algorithm for 3D ZTEM inversion.