Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations—II: Illustrative Application to Heat and Energy Transfer in the Nordheim–Fuchs Phenomenological Model for Reactor Safety

Abstract

This work presents an illustrative application of the newly developed “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE)” methodology to determine most efficiently the exact expressions of the first- and second-order sensitivities of NODE decoder responses to the neural net’s underlying parameters (weights and initial conditions). The application of the 2nd-FASAM-NODE methodology will be illustrated using the Nordheim–Fuchs phenomenological model for reactor safety, which describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted. The representative model responses that will be analyzed in this work include the model’s time-dependent total energy released, neutron flux, temperature and thermal conductivity. The 2nd-FASAM-NODE methodology yields the exact expressions of the first-order sensitivities of these decoder responses with respect to the underlying uncertain model parameters and initial conditions, requiring just a single large-scale computation per response. Furthermore, the 2nd-FASAM-NODE methodology yields the exact expressions of the second-order sensitivities of a model response requiring as few large-scale computations as there are features/functions of model parameters, thereby demonstrating its unsurpassed efficiency for performing sensitivity analysis of NODE nets.

1. Introduction

The mathematical/theoretical framework underlying the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE)” has been presented in the accompanying Part 1 [1]. In this work, the application of the newly developed 2nd-FASAM-NODE methodology to determine most efficiently the exact expressions of second-order sensitivities of decoder responses to the neural net’s underlying parameters (weights and initial conditions) will be illustrated using the Nordheim–Fuchs phenomenological model for reactor safety [2,3]. This phenomenological model describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted, either intentionally or by accident. The Nordheim–Fuchs model responses that will be analyzed in this work include the model’s state functions/variables (namely: the time-dependent total energy released per cm³, the reactor’s time-dependent temperature and the reactor’s time-dependent neutron flux), all of which are subject to uncertainties stemming from the model’s uncertain parameters. The application of the newly developed 2nd-FASAM-NODE methodology will also be illustrated by considering the reactor’s time-dependent thermal conductivity, which is a representative model response involving decoder weights that are also subject to uncertainties.

This work is structured as follows: Section 2 illustrates the application of the 2nd-FASAM-NODE methodology [1] to compute most efficiently the exact expressions of the first-order sensitivities of the model state functions and thermal conductivity. It is shown that the computation of the first-order sensitivities requires just a single large-scale computation per response. It is also shown that the number of large-scale computations can be reduced by identifying subsystems within the NODE structure which could be solved independently of each other. For example, the Nordheim–Fuchs NODE model can be decoupled into three equations, one for each of the dependent variables, which can be solved independently of each other. It is shown that the number of large-scale computations for determining the first-order (and, subsequently, the second-order) sensitivities is greatly reduced when the re-structured NODE equations can be solved independently of each other.

Section 3 presents illustrative applications of the 2nd-FASAM-NODE methodology to compute the second-order sensitivities of the Nordheim–Fuchs model’s dependent/state variables with respect to the underlying parameters. The computation of the second-order sensitivities by applying the 2nd-FASAM-NODE to the original NODE structure is presented in Section 3.1. The computation of the second-order sensitivities by applying the 2nd-FASAM-NODE to the decoupled NODE structure is presented in Section 3.2, illustrating the significant computational advantages gained by using decoupled subsystems, whenever possible.

Section 4 presents the application of the 2nd-FASAM-NODE methodology to compute second-order sensitivities of the thermal conductivity in the Nordheim–Fuchs model, which is a representative model response involving decoder weights subject to uncertainties. These second-order sensitivities are computed in Section 4.1 by applying the 2nd-FASAM-NODE methodology to the original NODE structure, and are subsequently computed alternatively, in Section 4.2, by applying the 2nd-FASAM-NODE methodology to the decoupled NODE structure. It is shown that using a decoupled structure, whenever possible, is even more advantageous for the computation of second-order sensitivities than for the computation of the first-order ones. The discussion presented in Section 5 concludes this work, highlighting the salient features and unparalleled capabilities of the 2nd-FASAM-NODE methodology for computing first- and second-order sensitivities of decoder response to uncertain NODE parameters and initial conditions.

2. Illustrative Application of the First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (1st-FASAM-NODE) to the Nordheim–Fuchs Reactor Dynamics/Safety Model

Section 2.1 presents the Nordheim–Fuchs phenomenological model formulated in NODE format. Section 2.2 presents a representative application of the 1st-FASAM-NODE to compute most efficiently the exact expressions of the first-order sensitivities of model state functions, while Section 2.3 presents a representative application of the 1st-FASAM-NODE to the exact expressions of first-order sensitivities of a typical response involving decoder weights. Section 2.4 compares the alternative ways of applying the 1st-FASAM-NODE to minimize the number of computations for evaluating the exact expressions of first-order sensitivities of model responses to model parameters by possibly identifying subsystems, within the NODE structure, that contain as few dependent variables as possible, and which could be solved independently of each other. For example, the Nordheim–Fuchs model can be decoupled into three equations, one equation for each of the dependent variables, which can be solved independently of each other.

2.1. NODE Modeling of the Nordheim–Fuchs Reactor Dynamics/Safety Phenomenological Model

The mathematical modeling of a NODE net was introduced in [4]. This modeling was generalized in [1] by introducing the concept of “features of primary model parameters/weights”, so that the generalized representation of a NODE network that comprises such “features” is provided by the following system of “augmented” equations:

$${\displaystyle \frac{dh\left(t\right)}{dt}}=f\left[h\left(t\right);F\left(\theta \right);t\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t>0,$$

(1)

$$h\left(t_0\right)=h_e\left(x,w\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}t=t_0,$$

(2)

$$r\left(t_f\right)=h_d\left[h\left(t_f\right);\phi \right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}at\hspace{0.17em}\hspace{0.17em}t=t_f,$$

(3)

where:

(i)

the $TH$-dimensional vector-valued function $h\left(t\right)\triangleq {\left[h_1\left(t\right),\mathrm{\dots },h_{TH}\left(t\right)\right]}^{†}$ represents the hidden/latent neural networks; in this work, all vectors are considered to be column vectors, and transposition is denoted by using the dagger “$†$” superscript;

(ii)

the $TH$-dimensional vector-valued nonlinear function $f\left[h\left(t\right);F\left(\theta \right);t\right]\triangleq {\left[f_1\left(h;F\left(\theta \right);t\right),\mathrm{\dots },f_{TH}\left(h;F\left(\theta \right);t\right)\right]}^{†}$ models the dynamics of the latent neurons;

(iii)

the components of the vector $\theta \triangleq {\left[{\theta }_1,\mathrm{\dots },{\theta }_{TW}\right]}^{†}$ represent learnable scalar adjustable weights, which are considered to be the “primary model parameters”; where $TW$ denotes the total number of adjustable weights in all of the latent neural nets;

(iv)

the components of the vector-valued function $F\left(\theta \right)\triangleq {\left[F_1\left(\theta \right),\mathrm{\dots },F_{TF}\left(\theta \right)\right]}^{†}$ represent the “feature” functions of the respective weights, which the quantity $TF\le TW$ denotes the “total number of feature/functions of the primary model parameters” comprised in the NODE;

(v)

the $TH$-dimensional vector-valued function $h_e\left(x,w\right)\triangleq {\left\{h_1^e\left(x,w\right),\mathrm{\dots },h_{TH}^e\left(x,w\right)\right\}}^{†}$ represents the “encoder” which is characterized by “inputs” $x\triangleq {\left[x_1,\mathrm{\dots },x_{TI}\right]}^{†}$ and “learnable” scalar adjustable weights $w\triangleq {\left[w_1,\mathrm{\dots },w_{TEW}\right]}^{†}$, where $TI$ denotes the total number of “inputs” and $TEW$ denotes the total number of “learnable encoder weights” that define the encoder;

(vi)

the $TR$-dimensional vector-valued function $r\left(t_f\right)\triangleq {\left\{r_1\left[h\left(t_f\right);\phi \right],\mathrm{\dots },r_{TR}\left[h\left(t_f\right);\phi \right]\right\}}^{†}=h_d\left[h\left(t_f\right);\phi \right]$ represents the vector of “system responses”; (vii) the vector-valued function $h_d\left[h\left(t_f\right);\phi \right]\triangleq {\left\{h_1^d\left[h\left(t_f\right);\phi \right],\mathrm{\dots },h_{TR}^d\left[h\left(t_f\right);\phi \right]\right\}}^{†}$ represents the “decoder” with learnable scalar adjustable weights, which are represented by the components of the vector $\phi \triangleq {\left[{\phi }_1,\mathrm{\dots },{\phi }_{TD}\right]}^{†}$, where $TD$ denotes the total number of adjustable weights that characterize the decoder.

The Nordheim–Fuchs phenomenological model comprises the following balance/conservation equations:

• The time-dependent neutron balance (point kinetics) equation for the neutron flux $\psi \left(t\right)$:

  $${\displaystyle \frac{d\psi \left(t\right)}{dt}}={\displaystyle \frac{\rho \left(t\right)-1}{l_p}}\psi \left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t>0,$$

  (4)

  $$\psi \left(0\right)={\psi }_0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t=0,$$

  (5)

  where $l_p$ denotes the prompt-neutron lifetime, $\rho \left(t\right)$ denotes the reactor’s multiplication factor and ${\psi }_0$ denotes the initial (i.e., extant flux) prior to initiating the transient at time $t=0$.
• The energy production equation:

  $$\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}E\left(t\right)=\gamma {\Sigma }_f{\displaystyle \underset{0}{\overset{t}{\int }}\psi \left(x\right)\hspace{0.17em}}dx,$$

  (6)

  where $\gamma$ denotes the recoverable energy per fission; ${\Sigma }_f\triangleq {\sigma }_f{N}_f$ denotes the reactor’s effective macroscopic fission cross section, where ${\sigma }_f$ denotes the reactor’s equivalent microscopic fission cross section while $N_f$ denotes the reactor’s equivalent atomic number density.
• The energy conservation equation:

  $$c_p\left[T\left(t\right)-T_0\right]=E\left(t\right),$$

  (7)

  where $E\left(t\right)$ denotes the total energy released (per cm³) at time $t$ in the reactor after the onset of reactivity change; $c_p$ denotes the specific heat (per cm³) of the reactor.
• The reactivity–temperature feedback equation: $\rho \left(t\right)={\rho }_0-{\alpha }_T{\rho }_0\left[T\left(t\right)-T_0\right]$, where ${\rho }_0\triangleq \rho \left(0\right)\ge 1$ denotes the changed multiplication factor following the reactivity insertion at $t=0$; ${\alpha }_T$ denotes the magnitude of the negative temperature coefficient; $T\left(t\right)$ denotes the reactor’s temperature; $T_0$ denotes the reactor’s initial temperature at time $t=0$. This work will consider the special case of a “prompt critical transient” which occurs when ${\rho }_0=1$ and the reactor becomes prompt critical after the reactivity insertion. In this particular case, the reactivity–temperature feedback equation takes on the following particular form:

  $$\rho \left(t\right)=1-{\alpha }_T\left[T\left(t\right)-T_0\right].$$

  (8)

Equations (4)–(8) can be transformed into the following system of nonlinear differential equations written in NODE format:

$${\displaystyle \frac{d\psi \left(t\right)}{dt}}=-{\displaystyle \frac{{\alpha }_T}{l_p{c}_p}}E\left(t\right)\psi \left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\psi \left(0\right)={\psi }_0,\hspace{0.17em}\hspace{0.17em}$$

(9)

$${\displaystyle \frac{dE\left(t\right)}{dt}}=\gamma {\sigma }_f{N}_f\psi \left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}E\left(0\right)=0,$$

(10)

$${\displaystyle \frac{dT\left(t\right)}{dt}}={\displaystyle \frac{\gamma {\sigma }_f{N}_f}{c_p}}\psi \left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}T\left(0\right)=T_0.$$

(11)

As detailed in [5], the Nordheim–Fuchs model described by Equations (9)–(11) can be solved analytically to obtain the following exact closed-form expression for the state functions $\psi \left(t\right)$, $E\left(t\right)$ and $T\left(t\right)$, which can be used for subsequent verification of all expressions of sensitivities:

$$E\left(t\right)=K_1\left(\alpha \right)\tanh \left[t{K}_2\left(\alpha \right)\right];$$

(12)

$$\psi \left(t\right)={\psi }_0\left\{1-{\tanh }^2\left[t{K}_2\left(\alpha \right)\right]\right\}={\displaystyle \frac{{\psi }_0}{{\cosh }^2\left[t{K}_2\left(\alpha \right)\right]}};$$

(13)

$$T\left(t\right)=T_0+{\displaystyle \frac{K_1\left(\alpha \right)}{c_p}}\tanh \left[t{K}_2\left(\alpha \right)\right];$$

(14)

where:

$$K_1\left(\alpha \right)\triangleq {\left[{\displaystyle \frac{2{\psi }_0\gamma {\sigma }_f{N}_f{l}_p{c}_p}{{\alpha }_T}}\right]}^{1/2};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{K}_2\left(\alpha \right)\triangleq {\left[{\displaystyle \frac{{\alpha }_T{\psi }_0\gamma {\sigma }_f{N}_f}{2l_p{c}_p}}\right]}^{1/2}.$$

(15)

Comparing the structure of the Nordheim–Fuchs model, cf. Equations (9)–(11), to the generic structure of a NODE defined by Equations (1)–(3) indicates the following correspondences for the weights/parameters of the hidden/latent units:

$$\begin{array}{l}\theta \triangleq {\left[{\theta }_1,\mathrm{\dots },{\theta }_{TW}\right]}^{†}\triangleq {\left({\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f\right)}^{†};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}TW=6;\\ x\triangleq {\left[x_1,x_2,x_3\right]}^{†}\triangleq {\left({\psi }_0,0,T_0\right)}^{†};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}TI=3\hspace{0.17em}.\end{array}$$

(16)

The precise values of the components of the vectors $\theta$ and $x$ remain unknown even after having trained the NODE, since the actual values of the parameters underlying the Nordheim–Fuchs model are experimentally-measured and are thus subject to uncertainties. However, the optimal parameter values obtained after having trained the NODE become the known nominal values of these parameters, and are considered to be exactly reproducible by the “trained” NODE; these nominal values will be denoted using a superscript “zero”, as follows:

$${\theta }^0\triangleq {\left[{\theta }_1^0,\mathrm{\dots },{\theta }_6^0\right]}^{†}\triangleq {\left({\alpha }_T^0,l_p^0,c_p^0,{\gamma }^0,{\sigma }_f^0,N_f^0\right)}^{†}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{x}^0\triangleq {\left[x_1^0,x_2^0,x_3^0\right]}^{†}\triangleq {\left[{\psi }_0^0,0;T_0^0\right]}^{†}.$$

(17)

It is apparent from the structure of Equations (9)–(11) that although there are several possibilities for choosing the components of the vector-valued function $F\left(\theta \right)\triangleq {\left[F_1\left(\theta \right),\mathrm{\dots },F_{TF}\left(\theta \right)\right]}^{†}$, it is advantageous to define these components so as to minimize their number while incorporating all of the primary model parameters. A choice that satisfies these considerations is as follows:

$$F_1\left(\theta \right)\triangleq -{\displaystyle \frac{{\alpha }_T}{l_p}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{F}_2\left(\theta \right)\triangleq \gamma {\sigma }_f{N}_f;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{F}_3\left(\theta \right)\triangleq {\displaystyle \frac{1}{c_p}}.$$

(18)

Using the definitions provided in Equation (18), Equations (9)–(11) can be re-written in the following NODE format:

$${\displaystyle \frac{d{h}_1\left(t\right)}{dt}}=F_1\left(\theta \right)F_3\left(\theta \right)h_1\left(t\right)h_2\left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{h}_1\left(0\right)={\psi }_0;$$

(19)

$${\displaystyle \frac{d{h}_2\left(t\right)}{dt}}=F_2\left(\theta \right)h_1\left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{h}_2\left(0\right)=0\hspace{0.17em};$$

(20)

$${\displaystyle \frac{d{h}_3\left(t\right)}{dt}}=F_2\left(\theta \right)F_3\left(\theta \right)h_1\left(t\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{h}_3\left(0\right)=T_0\hspace{0.17em}.$$

(21)

where:

$$h\left(t\right)\triangleq {\left[h_1\left(t\right),\mathrm{\dots },h_{TH}\left(t\right)\right]}^{†}\triangleq {\left[\psi \left(t\right),E\left(t\right),T\left(t\right)\right]}^{†};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}TH=3.$$

(22)

Since the exact values of the model parameters are unknown, the exact values of the functions $h\left(t\right)\triangleq {\left[h_1\left(t\right),h_2\left(t\right),h_3\left(t\right)\right]}^{†}\triangleq {\left[\psi \left(t\right),E\left(t\right),T\left(t\right)\right]}^{†}$ are also unknown. However, the nominal values $h^0\left(t\right)\triangleq {\left[h_1^0\left(t\right),h_2^0\left(t\right),h_3^0\left(t\right)\right]}^{†}\triangleq {\left[{\psi }^0\left(t\right),E^0\left(t\right),T^0\left(t\right)\right]}^{†}$ of these quantities (which are denoted using the superscript “zero”), are known after having solved Equations (19)–(21) at the nominal values $\left({\theta }^0,\hspace{0.17em}\hspace{0.17em}{x}^0\right)$.

The typical results of interest (called “model response”) for the Nordheim–Fuchs model are the values of the state functions at a “final-time” instance, denoted as $t=\tau$, after the initiation at $t=0$ of the prompt critical power transient. The most important response determined using the Nordheim–Fuchs model is the total energy (per cm³) released at the “final-time” instance $t=\tau$, which can be represented in “decoder” form as follows:

$$r\left(h\right)=E\left(\tau \right)={\displaystyle {\int }_0^{\tau }{h}_2\left(t\right)\delta \left(t-\tau \right)dt},$$

(23)

where $\delta \left(t-\tau \right)$ denotes the Dirac-delta functional. The neutron flux $\psi \left(\tau \right)$ and the reactor’s temperature $T\left(\tau \right)$ at a “final-time” instance $t=\tau$ can be represented by similar integrals.

A representative decoder response that also involves “weights” is provided by the temperature-dependent thermal conductivity, denoted as $K(T;\phi )$, of the conceptual reactor’s material. As a specific example, the material’s conductivity is considered to depend quadratically on the reactor’s temperature; the corresponding decoder response can be represented by the following expression:

$$K(T;\phi )\triangleq {\displaystyle {\int }_0^{\tau }{h}_4^d\left[h\left(t\right);\phi \right]}\hspace{0.17em}\delta \left(t-\tau \right)dt;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{h}_4^d\left[h\left(t\right);\phi \right]\triangleq {\phi }_0+{\phi }_{1}T\left(t\right)\hspace{0.17em}+{\phi }_2{T}^2\left(t\right),$$

(24)

where the scalar-valued components of $\phi \triangleq {\left[{\phi }_0,\hspace{0.17em}\hspace{0.17em}{\phi }_1,\hspace{0.17em}\hspace{0.17em}{\phi }_2\right]}^{†}$ are experimentally determined quantities and are thus subject to uncertainties. Of course, the temperature $T\left(\tau \right)$ is also subject to uncertainties since it is the solution of Equations (19)–(21), which themselves involve uncertain parameters.

2.2. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of the First-Order Sensitivities of Model State Functions to Uncertain Parameters

The application of the 1st-FASAM-NODE methodology will be illustrated by considering the total energy (per cm³) released at the “final-time” instance $t=\tau$, defined in Equation (23), as the NODE response. The total first-order sensitivity of this response is provided by the first-order G-differential of Equation (23), which is obtained, by definition, as follows:

$$\delta E\left(\tau \right)={\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[h_2^0\left(t\right)+\epsilon \delta h_2\left(t\right)\right]}\hspace{0.17em}\delta \left(t-\tau \right)dt\right\}}_{\epsilon =0}={\displaystyle {\int }_0^{\tau }\delta h_2\left(t\right)\delta \left(t-\tau \right)dt},$$

(25)

where the variation $\delta h_2\left(t\right)$ is the solution of the system obtained by taking the first-order Gateaux (G) variation of the original system of Equations (19)–(21), which yields:

$${\displaystyle \frac{d}{dt}}\delta h_1\left(t\right)=[\left(\delta F_1\right)F_3+F_1\left(\delta F_3\right)]h_1\left(t\right)h_2\left(t\right)+F_1{F}_3\left[\delta h_1\left(t\right)h_2\left(t\right)+h_1\left(t\right)\delta h_2\left(t\right)\right],$$

(26)

$${\displaystyle \frac{d}{dt}}\delta h_2\left(t\right)=\left(\delta F_2\right)h_1\left(t\right)+F_2\delta h_1\left(t\right),$$

(27)

$${\displaystyle \frac{d}{dt}}\delta h_3\left(t\right)=[\left(\delta F_2\right)F_3+F_2\left(\delta F_3\right)]h_1\left(t\right)+\left(F_2{F}_3\right)\delta h_1\left(t\right),$$

(28)

$$\delta h_1\left(0\right)=\delta {\psi }_0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\delta h_2\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\delta h_3\left(0\right)=\delta T_0\hspace{0.17em}.$$

(29)

The system defined by Equations (26)–(29) is called the “First-Level Variational Sensitivity System (1st-LVSS)”and can be conveniently written in matrix/vector form as follows:

$${\displaystyle \frac{d{v}^{\left(1\right)}\left(t\right)}{dt}}=J{v}^{\left(1\right)}\left(t\right)+{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\delta F,$$

(30)

$$v^{\left(1\right)}\left(0\right)\triangleq {\left[\delta h_1\left(0\right),\delta h_2\left(0\right),\delta h_3\left(0\right)\right]}^{†}={\left[\delta {\psi }_0,0,\hspace{0.17em}\hspace{0.17em}\delta T_0\hspace{0.17em}\right]}^{†},$$

(31)

where the following definitions were used:

$$J\triangleq {\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }h}}\triangleq \left(\begin{array}{ccc}{F}_1{F}_3{h}_2\left(t\right)& F_1{F}_3{h}_1\left(t\right)& 0\\ F_2& 0& 0\\ F_2{F}_3& 0& 0\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{v}^{\left(1\right)}\left(t\right)\triangleq \left(\begin{array}{c}\delta h_1\left(t\right)\\ \delta h_2\left(t\right)\\ \delta h_3\left(t\right)\end{array}\right);$$

(32)

$${\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\triangleq \left(\begin{array}{ccc}{F}_3{h}_1\left(t\right)h_2\left(t\right)& 0& F_1{h}_1\left(t\right)h_2\left(t\right)\\ 0& h_1\left(t\right)& 0\\ 0& F_3{h}_1\left(t\right)& F_2{h}_1\left(t\right)\end{array}\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\delta F\triangleq \left(\begin{array}{c}\delta F_1\\ \delta F_2\\ \delta F_3\end{array}\right)\hspace{0.17em}.$$

(33)

It is evident that the 1st-LVSS, defined by Equations (30) and (31), would need to be solved repeatedly in order to compute the first-level variational function $v^{\left(1\right)}\left(t\right)\triangleq {\left[\delta h_1\left(t\right),\delta h_2\left(t\right),\delta h_3\left(t\right)\right]}^{†}\triangleq {\left[\delta \psi \left(t\right),\delta E\left(t\right),\delta T\left(t\right)\right]}^{†}$ for every possible variations $\delta \theta \triangleq {\left[\delta {\theta }_1,\mathrm{\dots },\delta {\theta }_{TW}\right]}^{†}\triangleq {\left(\delta {\alpha }_T,\delta l_p,\delta c_p,\delta \gamma ,\delta {\sigma }_f,\delta N_f\right)}^{†}$ in the model parameters and variations $v^{\left(1\right)}\left(0\right)\triangleq {\left[\delta h_1\left(0\right),\delta h_2\left(0\right),\delta h_3\left(0\right)\right]}^{†}={\left[\delta {\psi }_0,0,\hspace{0.17em}\hspace{0.17em}\delta T_0\hspace{0.17em}\right]}^{†}$ in the “encoder” initial conditions. This computationally expensive path can be avoided by applying the concepts of the 1st- FASAM-NODE, as follows:

• Consider that the first-level variational function $v^{\left(1\right)}\left(t\right)\triangleq {\left[\delta h_1\left(t\right),\delta h_2\left(t\right),\delta h_3\left(t\right)\right]}^{†}\in {\mathrm{H}}_1\left({\Omega }_t\right)$, $\hspace{0.17em}{\Omega }_t\triangleq \left(0,\tau \right)$, is an element in a Hilbert space denoted as ${\mathrm{H}}_1\left({\Omega }_t\right)$ which comprises vector elements of the form ${\chi }^{(1)}\left(t\right)\triangleq {\left[{\chi }_1^{\left(1\right)}\left(t\right),{\chi }_2^{\left(1\right)}\left(t\right),{\chi }_3^{\left(1\right)}\left(t\right)\right]}^{†}$ and ${\eta }^{(1)}\left(t\right)\triangleq {\left[{\eta }_1^{\left(1\right)}\left(t\right),{\eta }_2^{\left(1\right)}\left(t\right),{\eta }_3^{\left(1\right)}\left(t\right)\right]}^{†}$, being endowed with an inner product ${〈{\chi }^{(1)}\left(t\right),{\eta }^{(1)}\left(t\right)〉}_1$ defined as follows:

  $${〈{\chi }^{(1)}\left(t\right),{\eta }^{(1)}\left(t\right)〉}_1\triangleq {\displaystyle {\int }_{t_0}^{\tau }{\left[{\chi }^{(1)}\left(t\right)\right]}^{†}{\eta }^{(1)}\left(t\right)dt}=\hspace{0.17em}{\displaystyle \sum _{i=1}^3{\displaystyle {\int }_0^{\tau }{\chi }_i^{\left(1\right)}\left(t\right){\eta }_i^{\left(1\right)}\left(t\right)dt}}.$$

  (34)
• Use Equation (34) to form the inner product of Equation (30) with a yet undefined function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_1\left({\Omega }_t\right)$ to obtain the following relation:

  $${\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}{\displaystyle \frac{d{v}^{\left(1\right)}\left(t\right)}{dt}}dt-{\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}J{v}^{\left(1\right)}\left(t\right)dt={\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\delta Fdt.$$

  (35)
• Integrating by parts the term on the left side of Equation (35) and rearranging the terms inside the integrals leads to the following relation:

  $$\begin{array}{l}{\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}{\displaystyle \frac{d{v}^{\left(1\right)}\left(t\right)}{dt}}dt-{\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}J{v}^{\left(1\right)}\left(t\right)dt\\ ={\left[a^{(1)}\left(\tau \right)\right]}^{†}{v}^{\left(1\right)}\left(\tau \right)-{\left[a^{(1)}\left(0\right)\right]}^{†}{v}^{\left(1\right)}\left(0\right)+{\displaystyle {\int }_0^{t_f}{\left[v^{\left(1\right)}\left(t\right)\right]}^{†}{A}^{\left(1\right)}{a}^{(1)}\left(t\right)dt}\hspace{0.17em},\end{array}$$

  (36)

  where:

  $$A^{\left(1\right)}{a}^{(1)}\left(t\right)\triangleq -{\displaystyle \frac{d{a}^{(1)}\left(t\right)}{dt}}-J^{†}{a}^{(1)}\left(t\right),$$

  (37)

  with

  $$J^{†}\triangleq {\left[{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }h}}\right]}^{†}=\left(\begin{array}{ccc}{F}_1{F}_3{h}_2\left(t\right)& F_2& F_2{F}_3\\ F_1{F}_3{h}_1\left(t\right)& 0& 0\\ 0& 0& 0\end{array}\right).$$

  (38)
• The definition of the function $a^{(1)}\left(t\right)$ is now completed by requiring that (i) the G-differential $\delta E\left(\tau \right)$ defined in Equation (25) be represented by the integral term on the right side of Equation (36) and (ii) the appearance of the unknown values of the components of $v^{\left(1\right)}\left(\tau \right)$ be eliminated from appearing in Equation (36). These requirements are satisfied by requiring the function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_1\left({\Omega }_t\right)$ to be the solution of the following “First-Level Adjoint Sensitivity System (1st-LASS)”:

  $$A^{\left(1\right)}{a}^{(1)}\left(t\right)\triangleq -{\displaystyle \frac{d{a}^{(1)}\left(t\right)}{dt}}-J^{†}{a}^{(1)}\left(t\right)={\left[0,\delta \left(t-\tau \right),0\right]}^{†};$$

  (39)

  $$a^{(1)}\left(\tau \right)\triangleq {\left[a_1^{(1)}\left(\tau \right),a_2^{(1)}\left(\tau \right),a_3^{(1)}\left(\tau \right)\right]}^{†}={\left[0,0,0\right]}^{†}.$$

  (40)

In component form, Equation (39) comprises the following relations written in NODE format:

$${\displaystyle \frac{d{a}_1^{(1)}\left(t\right)}{dt}}=-F_1{F}_3{h}_2\left(t\right)a_1^{(1)}\left(t\right)-F_2{a}_2^{(1)}\left(t\right)-F_2{F}_3{a}_3^{(1)}\left(t\right);$$

(41)

$${\displaystyle \frac{d{a}_2^{(1)}\left(t\right)}{dt}}=-F_1{F}_3{h}_1\left(t\right)a_1^{(1)}\left(t\right)-\delta \left(t-\tau \right);$$

(42)

$${\displaystyle \frac{d{a}_3^{(1)}\left(t\right)}{dt}}=0.$$

(43)

It is important to note that the 1st-LASS is linear in the first-level adjoint sensitivity function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}$. Furthermore, the 1st-LASS is independent of any parameter variations, so it needs to be solved only once. In particular, Equations (40)–(43) imply that $a_3^{(1)}\left(t\right)\equiv 0$. The actual/explicit expressions of the components $a_1^{(1)}\left(t\right)$ and $a_2^{(1)}\left(t\right)$ of $a^{(1)}\left(t\right)$ are unimportant for the purpose of the present illustrative/conceptual discussion.

5.

Using Equations (25) and (35) in Equation (36) yields the following expression for the first G-differential $\delta E\left(\tau \right)$:

$$\begin{array}{l}\delta E\left(\tau \right)=a_1^{(1)}\left(0\right)\delta {\psi }_0+a_3^{(1)}\left(0\right)\delta T_0\hspace{0.17em}+{\displaystyle {\int }_0^{\tau }{\left[a^{(1)}\left(t\right)\right]}^{†}}{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\delta Fdt\\ \triangleq {\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}\delta {\psi }_0+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{T}_0}}\delta T_0+{\displaystyle \sum _{i=1}^3\frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_i}\delta F_i}\hspace{0.17em}.\end{array}$$

(44)

Expanding the integral term on the right side of Equation (44) while recalling that $a_3^{(1)}\left(t\right)\equiv 0$ yields the following expressions for the first-order sensitivities of the response $E\left(\tau \right)$:

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}=a_1^{(1)}\left(0\right)={\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)\delta \left(t\right)dt};$$

(45)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{T}_0}}=a_3^{(1)}\left(0\right)=0;$$

(46)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_1}}=F_3\left(\theta \right){\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt};$$

(47)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_2}}={\displaystyle {\int }_0^{\tau }{a}_2^{(1)}\left(t\right)h_1\left(t\right)dt};$$

(48)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_3}}=F_1\left(\theta \right){\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}.$$

(49)

The expressions of the first-order sensitivities obtained in Equations (45)–(49) are to be evaluated at the nominal values of the respective parameters and functions. The numerical values of these first-order sensitivities can be computed after having solved the 1st-LASS to obtain the components of the first-level adjoint sensitivity function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}$.

The sensitivities of the response $E\left(\tau \right)$ with respect to the primary model parameters are obtained by using Equations (45)–(49) in conjunction with the following “chain-rule” type relations:

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\theta }_i}}={\displaystyle \sum _{j=1}^3\frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_j}\frac{\mathsf{\partial }{F}_j\left(\theta \right)}{\mathsf{\partial }{\theta }_i}}\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,TW=6.$$

(50)

Of course, the first-order sensitivities of $E\left(\tau \right)$ with respect to the primary model parameters $\theta \triangleq {\left[{\theta }_1,\mathrm{\dots },{\theta }_{TW}\right]}^{†}\triangleq {\left({\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f\right)}^{†}$ could have been computed directly, ab initio, using a single large-scale computation to solve the corresponding 1st-LASS, as presented in [5]. The advantage of using “features/functions of parameters” is minimal for computing first-order sensitivities but becomes increasingly important for computing second- and higher-order sensitivities. This fact will be demonstrated in Section 3 and Section 4, when computing second-order sensitivities of model responses to parameters.

It is important to note that for any other model response (“decoder”), only the source-terms on the right side of Equation (39) will be different. The left side of the 1st-LASS defined by Equation (39) and the “final-time” conditions in Equation (40) will remain unchanged. Therefore, if several responses are of interest, it would be computationally convenient if the inverse of the operator $A^{\left(1\right)}\left(h;\theta \right)$ could be stored in order to be used repeatedly, for computing all of the first-level adjoint sensitivity functions that would correspond to the responses of interest. Furthermore, the expressions on the right side of Equations (45)–(49) will remain formally the same for computing the first-order sensitivities of any other response. Of course, the numerical values of the components of the first-level adjoint sensitivity function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}$ will be replaced by the numerical values of the first-level adjoint sensitivity function that will correspond to the response under consideration. Thus, although the respective first-level adjoint sensitivity functions differ according to the response under consideration, the quadrature schemes needed to evaluate the integrals defining the respective sensitivities remain unchanged. Therefore, the same numerical procedures and/or neural nets can be used for computing the respective integrals that define the first-order sensitivities, while using the appropriate/corresponding values for the first-level adjoint sensitivity functions. If the response depends on parameters/weights, additional sensitivities will arise from the respective “direct-effect term”.

2.3. Representative Application of the 1st-FASAM-NODE to Compute Most Efficiently the Exact Expressions of First-Order Sensitivities of a Typical Response Involving Decoder Weights

The application of the 1st-FASAM-NODE methodology to compute efficiently and exactly the first-order sensitivities of a representative NODE decoder response involving decoder weights will be illustrated in this Subsection by considering the thermal conductivity response defined in Equation (24). The first-order sensitivities of this response are obtained by applying the definition of the G-differential to Equation (24), which yields the following expression:

$$\begin{array}{l}\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\triangleq \left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[\left({\phi }_0^0+\epsilon \delta {\phi }_0\right)+\left({\phi }_1^0+\epsilon \delta {\phi }_1\right)\left(T^0+\epsilon \delta T\right)\right.}\right.\\ {\left.\hspace{0.17em}+\left.\left({\phi }_2^0+\epsilon \delta {\phi }_2\right){\left(T^0+\epsilon \delta T\right)}^2\right]\delta \left(t-\tau \right)dt\right\}}_{\epsilon =0}\\ ={\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{dir}+{\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{ind}.\end{array}$$

(51)

In Equation (51), the direct-effect term ${\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{dir}$ comprises variations $\delta \phi \triangleq {\left[\delta {\phi }_0,\hspace{0.17em}\delta {\phi }_1,\hspace{0.17em}\delta {\phi }_2\right]}^{†}$ in the weights defining the decoder and is defined as follows:

$${\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{dir}\triangleq {\displaystyle {\int }_0^{\tau }\left[\delta {\phi }_0+\delta {\phi }_1{T}^0\left(t\right)+\delta {\phi }_2{\left(T^0\right)}^2\right]\hspace{0.17em}}\delta \left(t-\tau \right)dt,$$

(52)

while the indirect-effect term ${\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{ind}$ comprises the variations $\delta T\left(t\right)$ and is defined as follows:

$${\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{ind}\triangleq {\displaystyle {\int }_0^{\tau }\left[{\phi }_1^0+2{\phi }_2^0{T}^0\left(t\right)\right]}\hspace{0.17em}\delta T\left(t\right)\delta \left(t-\tau \right)dt.$$

(53)

The direct-effect term defined in Equation (52) yields the following first-order sensitivities with respect to the decoder’s weights, which can be computed immediately at the nominal value of the reactor temperature:

$${\displaystyle \frac{\mathsf{\partial }K\left[T\left(\tau \right)\right]}{\mathsf{\partial }{\phi }_0}}={\displaystyle {\int }_0^{\tau }\delta \left(t-\tau \right)dt\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=1};$$

(54)

$${\displaystyle \frac{\mathsf{\partial }K\left[T\left(\tau \right)\right]}{\mathsf{\partial }{\phi }_1}}\hspace{0.17em}={\displaystyle {\int }_0^{\tau }T\left(t\right)\delta \left(t-\tau \right)dt\hspace{0.17em}=}\hspace{0.17em}T\left(\tau \right);$$

(55)

$${\displaystyle \frac{\mathsf{\partial }K\left[T\left(\tau \right)\right]}{\mathsf{\partial }{\phi }_2}}={\displaystyle {\int }_0^{\tau }{T}^2\left(t\right)\hspace{0.17em}}\delta \left(t-\tau \right)dt=T^2\left(\tau \right).$$

(56)

The sensitivities stemming from the indirect-effect term can be obtained by applying the principles underlying the 1st-FASAM-NODE methodology by following the same steps as outlined in items (1)–(5) in Section 2.2, to obtain the following expression:

$$\begin{array}{l}{\left\{\delta K(T^0;{\phi }^0;\delta T;\delta \phi )\right\}}_{ind}=b_1^{(1)}\left(0\right)\delta {\psi }_0+b_3^{(1)}\left(0\right)\delta T_0\hspace{0.17em}+{\displaystyle {\int }_0^{\tau }{\left[b^{(1)}\left(t\right)\right]}^{†}}{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\delta Fdt\\ \triangleq {\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}\delta {\psi }_0+{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{T}_0}}\delta T_0+{\displaystyle \sum _{i=1}^3\frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_i}\delta F_i}\hspace{0.17em}.\end{array}$$

(57)

where the first-level adjoint sensitivity function $b^{(1)}\left(t\right)\triangleq {\left[b_1^{(1)}\left(t\right),b_2^{(1)}\left(t\right),b_3^{(1)}\left(t\right)\right]}^{†}$ is the solution of the following First-Level Adjoint Sensitivity System (1st-LASS):

$$A^{\left(1\right)}{b}^{(1)}\left(t\right)\triangleq -{\displaystyle \frac{d{b}^{(1)}\left(t\right)}{dt}}-J^{†}{b}^{(1)}\left(t\right)={\left\{0,0,\left[{\phi }_1+2{\phi }_{2}T\left(t\right)\right]\delta \left(t-\tau \right)\right\}}^{†};$$

(58)

$$b^{(1)}\left(\tau \right)\triangleq {\left[b_1^{(1)}\left(\tau \right),b_2^{(1)}\left(\tau \right),b_3^{(1)}\left(\tau \right)\right]}^{†}={\left[0,0,0\right]}^{†}.$$

(59)

The 1st-LASS defined by Equations (58) and (59) is solved at the nominal values for all parameters and original functions, but the superscript “zero” which would have denoted this fact has been omitted in order to simplify the notation. Expanding the vector-matrix product in the integral on the right side of Equation (57) and identifying the expressions that multiply the respective variations in the feature functions and initial conditions leads to the following expressions for the sensitivities of the response $K\left(\tau \right)$:

$${\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}=b_1^{(1)}\left(0\right)={\displaystyle {\int }_0^{\tau }{b}_1^{(1)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(60)

$${\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{T}_0}}=b_3^{(1)}\left(0\right)={\displaystyle {\int }_0^{\tau }{b}_3^{(1)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(61)

$${\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_1}}=F_3\left(\theta \right){\displaystyle {\int }_0^{\tau }{h}_1\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)dt};$$

(62)

$${\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_2}}={\displaystyle {\int }_0^{\tau }\left[b_2^{(1)}\left(t\right)+F_3\left(\theta \right)b_3^{(1)}\left(t\right)\right]h_1\left(t\right)dt};$$

(63)

$${\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_3}}={\displaystyle {\int }_0^{\tau }\left[F_1\left(\theta \right)b_1^{(1)}\left(t\right)h_2\left(t\right)+F_2\left(\theta \right)b_3^{(1)}\left(t\right)\right]h_1\left(t\right)dt}.$$

(64)

As indicated in Equations (60)–(64), there are five first-order sensitivities of the response $K\left(\tau \right)$ with respect to the two non-zero initial conditions and the three components of the feature function. The first-order sensitivities with respect to the six primary parameters ${\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f$ can be obtained by using Equations (60)–(64) in conjunction with chain-rule relations similar to those presented in Equation (50).

2.4. Discussion: Minimizing the Number of Computations for Evaluating the Exact Expressions of First-Order Sensitivities of Model Responses to Model Parameters

It is evident that the 1st-FASAM-NODE is the most efficient methodology for computing first-order sensitivities whenever it is possible to identify features/functions of the model parameters in the underlying NODE equations. Additional efficiency can be attained if the original system of NODE equations could be decoupled into subsystems that contain as few dependent variables as possible, which could then be solved independently of each other. For the Nordheim–Fuchs model, for example, Equations (9)–(11) can be decoupled into three equations, one equation for each of the dependent variables, which can be solved independently of each other. For example, the following equation has been obtained in [5] for the function $E\left(t\right)$ in NODE format:

$${\displaystyle \frac{dE\left(t\right)}{dt}}=-{\displaystyle \frac{{\alpha }_T}{2l_p{c}_p}}{E}^2\left(t\right)+{\psi }_0\gamma {\sigma }_f{N}_f,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}E\left(0\right)=0.$$

(65)

The “features/functions of parameters” in Equation (65) can be conveniently chosen as follows:

$$\begin{array}{l}{\mathrm{\Phi }}_1\left(p\right)\triangleq {\displaystyle \frac{{\alpha }_T}{2l_p{c}_p}};\hspace{0.17em}\hspace{0.17em}{\mathrm{\Phi }}_2\left(p\right)\triangleq {\psi }_0\gamma {\sigma }_f{N}_f;\hspace{0.17em}\hspace{0.17em}\mathrm{\Phi }\left(p\right)\triangleq {[{\mathrm{\Phi }}_1\left(p\right),{\mathrm{\Phi }}_2\left(p\right)]}^{†};\\ p\triangleq {\left[p_1,\mathrm{\dots },p_7\right]}^{†}\triangleq {\left[{\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f,{\psi }_0\right]}^{†}.\end{array}$$

(66)

In terms of the “feature function” $\mathrm{\Phi }\left(p\right)\triangleq {\left[{\mathrm{\Phi }}_1\left(p\right),{\mathrm{\Phi }}_2\left(p\right)\right]}^{†}$, Equation (65) can alternatively be written as follows:

$${\displaystyle \frac{dE\left(t\right)}{dt}}=-{\mathrm{\Phi }}_1\left(p\right)E^2\left(t\right)+{\mathrm{\Phi }}_2\left(p\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}E\left(0\right)=0.$$

(67)

Of course, a specific NODE model would need to be constructed to represent Equation (67).

In terms of the feature function $\mathrm{\Phi }\left(p\right)\triangleq {\left[{\mathrm{\Phi }}_1\left(p\right),{\mathrm{\Phi }}_2\left(p\right)\right]}^{†}$, the solution of Equation (67) has the following form:

$$E\left(t\right)={\left[{\displaystyle \frac{{\mathrm{\Phi }}_2\left(p\right)}{{\mathrm{\Phi }}_1\left(p\right)}}\right]}^{1/2}\tanh \left[tG\left(p\right)\right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}G\left(p\right)\triangleq \sqrt{{\mathrm{\Phi }}_1\left(p\right){\mathrm{\Phi }}_2\left(p\right)}.$$

(68)

The First-Level Variational Sensitivity System (1st-LVSS) for the variational function $\delta E\left(t\right)$ is obtained by applying the definition of the first-order G-differential to Equation (67), which yields the following 1st-LVSS:

$$\left[{\displaystyle \frac{d}{dt}}+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta E\left(t\right)=-\delta {\mathrm{\Phi }}_1{E}^2\left(t\right)+\delta {\mathrm{\Phi }}_2,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t>0,$$

(69)

$$\delta E\left(0\right)=0,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t=0.$$

(70)

The 1st-LVSS represented by Equation (69) is to be solved at the nominal values for the parameters and the state function $E\left(t\right)$ but the superscript “0” (which indicates “nominal values”) has been omitted to simplify the notation. Numerically, the 1st-LVSS would need to be solved anew for the various variations $\delta F_1$, $\delta F_2$, in the components of the feature function $\mathbf{\Phi }\left(p\right)$. This need for repeatedly solving the 1st-LVSS can be avoided by constructing the corresponding First-Level Adjoint Sensitivity System (1st-LASS). The Hilbert space appropriate for the construction of the 1st-LASS corresponding to Equation (69) will be denoted as ${\mathrm{H}}_{1,E}\left({\Omega }_t\right)$, where the subscript “1” indicates “first-level”, the subscript “E” indicates that this Hilbert space is constructed exclusively for the “energy variable” and where $\hspace{0.17em}{\Omega }_t\triangleq \left(0,\tau \right)$. The inner product appropriate for $H_{1,E}\left({\Omega }_t\right)$ between two functions ${\omega }_1\left(t\right)\in {\mathrm{H}}_{1,E}\left({\Omega }_t\right)$ and ${\omega }_2\left(t\right)\in {\mathrm{H}}_{1,E}\left({\Omega }_t\right)$ is denoted as ${〈{\omega }_1\left(t\right),{\omega }_2\left(t\right)〉}_{1,E}$ and is defined as follows:

$${〈{\omega }_1\left(t\right),{\omega }_2\left(t\right)〉}_{1,E}=\hspace{0.17em}{\displaystyle {\int }_0^{\tau }{\omega }_1\left(t\right){\omega }_2\left(t\right)dt}.$$

(71)

Using Equation (71), the 1st-LASS is constructed by applying the same sequence of steps as those leading to Equations (39)–(44), to obtain the following expression for the G-differential $\delta E\left(\tau \right)$ of the response $E\left(\tau \right)$:

$$\delta E\left(\tau \right)=-\left(\delta {\mathrm{\Phi }}_1\right){\displaystyle {\int }_0^{\tau }{e}^{\left(1\right)}\left(t\right)}{E}^2\left(t\right)dt+\left(\delta {\mathrm{\Phi }}_2\right){\displaystyle {\int }_0^{\tau }{e}^{\left(1\right)}\left(t\right)dt},$$

(72)

where the first-level adjoint sensitivity function $e^{(1)}\left(t\right)$ is the solution of the following First-Level Adjoint Sensitivity System (1st-LASS):

$$\left[-{\displaystyle \frac{d}{dt}}+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]e^{(1)}\left(t\right)=\delta \left(t-\tau \right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t>0,$$

(73)

$$e^{(1)}\left(\tau \right)=0,\hspace{1em}t=\tau .$$

(74)

The 1st-LASS represented by Equations (73) and (74) is independent of variations in the feature functions (or parameters) so it would need to be solved only once, numerically. In the present case, the 1st-LASS can be solved analytically to obtain the following closed-form expression for the first-level adjoint sensitivity function $e^{(1)}\left(t\right)$:

$$e^{(1)}\left(t\right)=H\left(\tau -t\right){\left\{{\displaystyle \frac{\cosh \left[tG\left(p\right)\right]}{\cosh \left[\tau G\left(p\right)\right]}}\right\}}^2,$$

(75)

where $H\left(t-\tau \right)$ denotes the Heaviside functional. It follows from Equations (72), (75) and (68) that the two sensitivities of the response $E\left(\tau \right)$ with respect to the two components of the feature function $\mathrm{\Phi }\triangleq {\left({\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}^{†}$ have the following expressions:

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}=-{\displaystyle {\int }_0^{\tau }{e}^{(1)}\left(t\right)E^2\left(t\right)\hspace{0.17em}}dt={\displaystyle \frac{1}{2}}{\left[{\displaystyle \frac{{\mathrm{\Phi }}_2\left(p\right)}{{\mathrm{\Phi }}_1\left(p\right)}}\right]}^{1/2}\left\{{\displaystyle \frac{\tau }{{\cosh }^2\left[\tau G\left(p\right)\right]}}-{\displaystyle \frac{\tanh \left[\tau G\left(p\right)\right]}{G\left(p\right)}}\right\};$$

(76)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}={\displaystyle {\int }_0^{\tau }{e}^{(1)}\left(t\right)dt}={\displaystyle \frac{1}{2G\left(p\right)}}\tanh \left[\tau G\left(p\right)\right]+{\displaystyle \frac{\tau }{2{\cosh }^2\left[\tau G\left(p\right)\right]}}.$$

(77)

The above expressions are to be evaluated at the nominal parameter values but the superscript “zero” has been omitted, for simplicity. The expressions obtained in Equations (76) and (77) can be verified by differentiating the expression provided in Equation (68), evaluated at a user-chosen time $t=\tau$ within the interval $0<\tau <\infty$.

The sensitivities of the response $E\left(\tau \right)$ with respect to the model parameters are obtained by using the following general relationship:

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau ;{\mathrm{\Phi }}_1;{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{p}_i}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1\left(p\right)}{\mathsf{\partial }{p}_i}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2\left(p\right)}{\mathsf{\partial }{p}_i}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,7.$$

(78)

The explicit expressions for the specific sensitivities of the response $E\left(\tau \right)$ with respect to the parameters underlying the feature functions are obtained using Equation (78) in conjunction with Equations (76) and (77) while recalling the definitions of the feature functions ${\mathrm{\Phi }}_1\left(p\right)$ and ${\mathrm{\Phi }}_2\left(p\right)$ defined in Equation (66). The expressions of these first-order sensitivities are as follows:

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\alpha }_T}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{\alpha }_T}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{\alpha }_T}}={\displaystyle \frac{1}{2l_p{c}_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}};$$

(79)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{l}_p}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{l}_p}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{l}_p}}=-{\displaystyle \frac{{\alpha }_T}{2{\left(l_p\right)}^2{c}_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}};$$

(80)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{c}_p}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{c}_p}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{c}_p}}=-{\displaystyle \frac{{\alpha }_T}{2{\left(c_p\right)}^2{l}_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}};$$

(81)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\sigma }_f}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{\sigma }_f}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{\sigma }_f}}={\psi }_0\gamma N_f{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}};$$

(82)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\sigma }_f}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{\sigma }_f}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{\sigma }_f}}={\psi }_0\gamma N_f{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}};$$

(83)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{N}_f}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1}{\mathsf{\partial }{N}_f}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{N}_f}}={\psi }_0\gamma {\sigma }_f{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}.$$

(84)

$${\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}={\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{\psi }_0}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2}{\mathsf{\partial }{\psi }_0}}=\gamma {\sigma }_f{N}_f{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}.$$

(85)

Notably, the application of the 1st-FASAM-NODE requires just one “large-scale” computation to solve the 1st-LASS, cf. Equations (73) and (74), which is a single NODE, to obtain the first-level adjoint function $e^{(1)}\left(t\right)$, which is a scalar-valued function. However, solving the 1st-LASS, comprising Equations (73) and (74), requires the construction of a separate, albeit simpler, NODE. The first-level adjoint function $e^{(1)}\left(t\right)$ is subsequently used for obtaining the two sensitivities of the response $E\left(\tau \right)$ with respect to the two components ${\mathrm{\Phi }}_1\left(p\right)$ and ${\mathrm{\Phi }}_2\left(p\right)$ of the feature function $\mathbf{\Phi }\left(p\right)\triangleq {\left({\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}^{†}$, which requires the evaluation of two integrals using quadrature formulas. Subsequently, all of the response sensitivities with respect to the model’s primary parameters are obtained analytically by using the chain rule to differentiate the components of the feature function with respect to the underlying model parameters and initial conditions.

On the other hand, the direct computation of the sensitivities of the response with respect to the model parameters and initial conditions, the original NODE can be used to solve (backward in time) the 1st-LASS, which comprises a system of three coupled NODEs for obtaining the first-level adjoint function $a^{(1)}\left(t\right)\triangleq {\left[a_1^{(1)}\left(t\right),a_2^{(1)}\left(t\right),a_3^{(1)}\left(t\right)\right]}^{†}$, which is a vector-valued function comprising three components. The respective vector-valued first-level adjoint function is subsequently used in performing six (rather than two, if the 1st-FASAM is used) integrals (quadrature) for obtaining the six sensitivities of the respective response with respect to the six model parameters.

After having obtained the first-order sensitivities of $E(\tau )$, the first-order sensitivities of the temperature $T(\tau )$ at a time instance $t=\tau$ can be obtained exactly and most effectively by using the expressions provided in Equations (79)–(85) in conjunction with the following relation obtained by differentiating the expression in Equation (7) with respect to the parameters $p_i$, $i=1,\mathrm{\dots },7$:

$${\displaystyle \frac{\mathsf{\partial }T\left(\tau \right)}{\mathsf{\partial }{p}_i}}={\displaystyle \frac{\mathsf{\partial }\left(1/c_p\right)}{\mathsf{\partial }{p}_i}}E\left(\tau \right)+{\displaystyle \frac{1}{c_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}.$$

(86)

Using Equations (9) and (65) yields the following relation:

$$\psi \left(t\right)={\psi }_0-{\displaystyle \frac{{\alpha }_T}{2l_p{c}_p\gamma {\sigma }_f{N}_f}}{E}^2\left(t\right).$$

(87)

The first-order sensitivities of the neutron flux $\psi (\tau )$ at a time instance $t=\tau$ can be obtained exactly and most effectively by using the first-order sensitivities of $E(\tau )$, in conjunction with the following relation obtained by differentiating Equation (87) with respect to the parameters $p_i$, $i=1,\mathrm{\dots },7$:

$${\displaystyle \frac{\mathsf{\partial }\psi \left(\tau \right)}{\mathsf{\partial }{\psi }_0}}=1;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{\mathsf{\partial }\psi \left(\tau \right)}{\mathsf{\partial }{p}_i}}=-E^2\left(\tau \right){\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_i}}\left({\displaystyle \frac{{\alpha }_T}{2l_p{c}_p\gamma {\sigma }_f{N}_f}}\right)-{\displaystyle \frac{{\alpha }_{T}E\left(\tau \right)}{l_p{c}_p\gamma {\sigma }_f{N}_f}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}.$$

(88)

After having obtained the first-order sensitivities of $T(\tau )$, the first-order sensitivities of the thermal conductivity $K(\tau )$ can be obtained exactly, analytically and most efficiently by differentiating Equation (24) with respect to the parameters $p_i$, $i=1,\mathrm{\dots },7$. This procedure yields the following expressions:

$$\begin{array}{l}{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\phi }_0}}=1;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\phi }_1}}=T\left(\tau \right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\phi }_2}}=T^2\left(\tau \right);\\ {\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{p}_i}}={\phi }_0+\left[{\phi }_1+2{\phi }_{2}T\left(\tau \right)\right]{\displaystyle \frac{\mathsf{\partial }T\left(\tau \right)}{\mathsf{\partial }{p}_i}}\\ ={\phi }_0+\left[{\phi }_1+2{\phi }_{2}T\left(\tau \right)\right]\left[{\displaystyle \frac{\mathsf{\partial }\left(1/c_p\right)}{\mathsf{\partial }{p}_i}}E\left(\tau \right)+{\displaystyle \frac{1}{c_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}\right].\end{array}$$

(89)

3. Illustrative Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of the Nordheim–Fuchs Model’s Dependent/State Variables with Respect to the Underlying Parameters

The computation of the second-order sensitivities of a dependent/state variable will be illustrated by using the response $E\left(\tau \right)$ as a paradigm example. The second-order sensitivities of this response are computed by using the fundamental definition of being “the first-order sensitivities of the first-order sensitivities”. In Section 3.1, the second-order sensitivities of $E(\tau )$ will be obtained from the NODE equations derived in Section 2.2, which led to the expressions presented in Equations (45)–(49). Alternatively, in Section 3.2, the second-order sensitivities of $E(\tau )$ will be obtained most efficiently from the first-order sensitivities with respect to the feature functions derived in Equations (76) and (77).

3.1. Computation of Second-Order Sensitivities of $E(\tau )$ Using the Coupled NODE Equations

The representative computation of second-order sensitivities of $E(\tau )$ can be illustrated by considering the expression obtained in Equation (49). The G-differential of this expression is obtained, by definition, as follows:

$$\begin{array}{l}\delta \left[{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_3}}\right]=\left\{{\displaystyle \frac{d}{d\epsilon }}\left(F_1^0+\epsilon \delta F_1\right)\right.{\displaystyle {\int }_0^{\tau }\left[a_1^{(1,0)}\left(t\right)+\epsilon \delta a_1^{(1)}\left(t\right)\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]}\\ {\left.\times \left[h_2^0\left(t\right)+\epsilon \delta h_2\left(t\right)\right]dt\right\}}_{\epsilon =0}=\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{dir}+\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{ind},\end{array}$$

(90)

where the direct-effect term $\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{dir}$ comprises parameter variations and is defined as follows:

$$\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{dir}\triangleq \left(\delta F_1\right){\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)h_2\left(t\right)}\hspace{0.17em}{h}_1\left(t\right)\hspace{0.17em}dt,$$

(91)

while the indirect-effect term $\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{ind}$ comprises variations in the state functions and is defined as follows:

$$\begin{array}{l}\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{ind}\triangleq F_1{\displaystyle {\int }_0^{\tau }\left\{\left[\delta a_1^{(1)}\left(t\right)h_2\left(t\right)h_1\left(t\right)\right]+\left[\delta h_2\left(t\right)a_1^{(1)}\left(t\right)h_1\left(t\right)\right]\right.}\\ \left.\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\left[\delta h_1\left(t\right)a_1^{(1)}\left(t\right)h_2\left(t\right)\right]\right\}\hspace{0.17em}dt\hspace{0.17em}.\end{array}$$

(92)

The expressions of the direct-effect and indirect-effect terms obtained in Equations (91) and (92) are to be evaluated at the respective nominal values of parameters and functions but the superscript “zero”, which has been used in Equation (90) to indicate this fact, has been omitted in order to simplify the notation.

The direct-effect term can be evaluated numerically at this time since all of the functions in Equation (91) are available. The indirect-effect term can be evaluated only after having determined the variational functions $\delta a^{\left(1\right)}\left(t\right)\triangleq {\left[\delta a_1^{(1)}\left(t\right),\delta a_2^{(1)}\left(t\right),\delta a_3^{(1)}\left(t\right)\right]}^{†}$ and $v^{\left(1\right)}\left(t\right)\triangleq {\left[\delta h_1\left(t\right),\delta h_2\left(t\right),\delta h_3\left(t\right)\right]}^{†}\triangleq {\left[\delta \psi \left(t\right),\delta E\left(t\right),\delta T\left(t\right)\right]}^{†}$. The variational function $v^{\left(1\right)}\left(t\right)$ is the solution of the 1st-LVSS defined by Equations (30) and (31). The variational function $\delta a^{\left(1\right)}\left(t\right)$ is the solution of the system of equations obtained by G-differentiating the 1st-LASS defined by Equations (39) and (40), which yields the following relations:

$$\begin{array}{l}{\displaystyle \frac{d}{dt}}\delta a_1^{(1)}\left(t\right)=-\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-\delta a_2^{(1)}\left(t\right)F_2-\delta a_3^{(1)}\left(t\right)F_2{F}_3-\delta h_2\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\\ -\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)a_1^{(1)}\left(t\right)-\left(\delta F_2\right)a_2^{(1)}\left(t\right)-\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]a_3^{(1)}\left(t\right)\hspace{0.17em};\end{array}$$

(93)

$$\begin{array}{l}{\displaystyle \frac{d}{dt}}\delta a_2^{(1)}\left(t\right)=-\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)-\delta h_1\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}-\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)a_1^{(1)}\left(t\right),\end{array}$$

(94)

$${\displaystyle \frac{d}{dt}}\delta a_3^{(1)}\left(t\right)=0.$$

(95)

$$\delta a^{\left(1\right)}\left(\tau \right)\triangleq {\left[\delta a_1^{(1)}\left(\tau \right),\delta a_2^{(1)}\left(\tau \right),\delta a_3^{(1)}\left(\tau \right)\right]}^{†}={\left[0,0,0\right]}^{†}$$

(96)

As indicated by Equations (93)–(96), the components of $\delta a^{\left(1\right)}\left(t\right)$ are connected to the components of $v^{\left(1\right)}\left(t\right)$. Therefore, the function $v^{\left(2\right)}\left(t\right)\triangleq {\left[v^{\left(1\right)}\left(t\right),\delta a^{(1)}\left(t\right)\right]}^{†}$ will be the solution of the so-called “Second-Level Variational Sensitivity System” (2nd-LVSS) obtained by concatenating Equations (93)–(96) with Equations (26)–(29). The function $v^{\left(2\right)}\left(t\right)\triangleq {\left[v^{\left(1\right)}\left(t\right),\delta a^{(1)}\left(t\right)\right]}^{†}$ will be called the “second-level variational function”. The superscript “(2)” indicates “second-level”.

It is impractical to solve the 2nd-LVSS to compute each second-level variational function $v^{\left(2\right)}\left(t\right)\triangleq {\left[v^{\left(1\right)}\left(t\right),\delta a^{(1)}\left(t\right)\right]}^{†}$ that would correspond to every component of the variations $\delta F$ and $\delta \phi$. The need for computing $v^{\left(2\right)}\left(t\right)$ can be circumvented by applying the principles of the 2nd-FASAM-NODE methodology [1], which comprises the following sequence of steps:

• Consider that $v^{\left(2\right)}\left(t\right)\triangleq {\left[v^{\left(1\right)}\left(t\right),\delta a^{(1)}\left(t\right)\right]}^{†}$ is an element in a Hilbert space denoted as ${\mathrm{H}}_2\left({\Omega }_t\right)$, ${\Omega }_t\triangleq \left[0,\tau \right]$, comprising as elements two-block vectors having the following structure: ${\chi }^{\left(2\right)}\triangleq {\left[{\chi }_1^{\left(2\right)}\left(t\right),{\chi }_2^{\left(2\right)}\left(t\right)\right]}^{†}$, with ${\chi }_1^{\left(2\right)}\left(t\right)\triangleq {\left[{\chi }_{1,1}^{\left(2\right)}\left(t\right),{\chi }_{1,2}^{\left(2\right)}\left(t\right),{\chi }_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and ${\chi }_2^{\left(2\right)}\left(t\right)\triangleq {\left[{\chi }_{2,1}^{\left(2\right)}\left(t\right),{\chi }_{2,2}^{\left(2\right)}\left(t\right),{\chi }_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$. The Hilbert space ${\mathrm{H}}_2\left({\Omega }_t\right)$ is considered to be endowed with an inner product denoted as ${〈{\chi }^{\left(2\right)},{\eta }^{\left(2\right)}〉}_2$ and defined as follows:

  $${〈{\chi }^{\left(2\right)},{\eta }^{\left(2\right)}〉}_2\triangleq {\displaystyle \sum _{i=1}^3{\displaystyle {\int }_0^{\tau }{\chi }_{1,i}^{\left(2\right)}\left(t\right){\eta }_{1,i}^{\left(2\right)}\left(t\right)dt}}+{\displaystyle \sum _{i=1}^3{\displaystyle {\int }_0^{\tau }{\chi }_{2,i}^{\left(2\right)}\left(t\right){\eta }_{2,i}^{\left(2\right)}\left(t\right)dt}}.$$

  (97)
• Use the definition of the inner product provided in Equation (97) to form the inner product of Equations (26)–(28) and (93)–(95) with a vector $a^{\left(2\right)}\triangleq {\left[a_1^{\left(2\right)}\left(t\right),a_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, where $a_1^{\left(2\right)}\left(t\right)\triangleq {\left[a_{1,1}^{\left(2\right)}\left(t\right),a_{1,2}^{\left(2\right)}\left(t\right),a_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and $a_2^{\left(2\right)}\left(t\right)\triangleq {\left[a_{2,1}^{\left(2\right)}\left(t\right),a_{2,2}^{\left(2\right)}\left(t\right),a_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$, to obtain the following relationship:

  $$\begin{array}{l}{\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)\frac{d}{dt}\delta h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3\left[\delta h_1\left(t\right)h_2\left(t\right)+h_1\left(t\right)\delta h_2\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{a}_{1,2}^{\left(2\right)}\frac{d}{dt}\delta h_2\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{a}_{1,2}^{\left(2\right)}{F}_2\delta h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}\frac{d}{dt}\delta h_3\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)\delta h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\frac{d}{dt}\delta a_1^{(1)}\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\left[\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+\delta a_2^{(1)}\left(t\right)F_2+\delta a_3^{(1)}\left(t\right)F_2{F}_3+\delta h_2\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\right]}\hspace{0.17em}dt\\ {\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}\frac{d}{dt}\delta a_2^{(1)}\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}\left[\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+\delta h_1\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{a}_{2,3}^{\left(2\right)}\frac{d}{dt}\delta a_3^{(1)}\left(t\right)dt}=q^{\left(2\right)},\end{array}$$

  (98)

  where:

  $$\begin{array}{l}{q}^{\left(2\right)}\triangleq {\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)\left[\left(\delta F_1\right)F_3+F_1\left(\delta F_3\right)\right]h_1\left(t\right)h_2\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{a}_{1,2}^{\left(2\right)}\left(\delta F_2\right)h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}\left[\left(\delta F_2\right)F_3+F_2\left(\delta F_3\right)\right]h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\left\{\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)a_1^{(1)}\left(t\right)+\left(\delta F_2\right)a_2^{(1)}\left(t\right)\right.}\\ \left.+\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]a_3^{(1)}\left(t\right)\right\}dt-{\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)a_1^{(1)}\left(t\right)dt}.\end{array}$$

  (99)
• Integrating the left side of Equation (98) by parts over the independent variable $t$ yields the following relation:

  $$\begin{array}{l}{\displaystyle {\int }_0^{\tau }\left\{-\delta h_1\left(t\right)\frac{d{a}_{1,1}^{\left(2\right)}\left(t\right)}{dt}-a_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3\left[\delta h_1\left(t\right)h_2\left(t\right)+h_1\left(t\right)\delta h_2\left(t\right)\right]\right\}dt}\\ +a_{1,1}^{\left(2\right)}\left(\tau \right)\delta h_1\left(\tau \right)-a_{1,1}^{\left(2\right)}\left(0\right)\delta h_1\left(0\right)+a_{1,2}^{\left(2\right)}\left(\tau \right)\delta h_2\left(\tau \right)-a_{1,2}^{\left(2\right)}\left(0\right)\delta h_2\left(0\right)\\ -{\displaystyle {\int }_0^{\tau }\left\{\delta h_2\left(t\right)\frac{d{a}_{1,2}^{\left(2\right)}}{dt}+a_{1,2}^{\left(2\right)}{F}_2\delta h_1\left(t\right)\right\}dt}+a_{1,3}^{\left(2\right)}\left(\tau \right)\delta h_3\left(\tau \right)-a_{1,3}^{\left(2\right)}\left(0\right)\delta h_3\left(0\right)\\ -{\displaystyle {\int }_0^{\tau }\left\{\delta h_3\left(t\right)\frac{d{a}_{1,3}^{\left(2\right)}}{dt}+a_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)\delta h_1\left(t\right)\right\}dt}+a_{2,1}^{\left(2\right)}\left(\tau \right)\delta a_1^{(1)}\left(\tau \right)-a_{2,1}^{\left(2\right)}\left(0\right)\delta a_1^{(1)}\left(0\right)\\ {\displaystyle {\int }_0^{\tau }\left\{-\delta a_1^{(1)}\left(t\right)\frac{d{a}_{2,1}^{\left(2\right)}}{dt}+a_{2,1}^{\left(2\right)}\right.\left[\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+\delta a_2^{(1)}\left(t\right)F_2+\delta a_3^{(1)}\left(t\right)F_2{F}_3\right.}\\ \left.\left.+\delta h_2\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\right]dt\right\}+a_{2,2}^{\left(2\right)}\left(\tau \right)\delta a_2^{(1)}\left(\tau \right)-a_{2,2}^{\left(2\right)}\left(0\right)\delta a_2^{(1)}\left(0\right)\\ +{\displaystyle {\int }_0^{\tau }\left\{-\delta a_2^{(1)}\left(t\right)\frac{d{a}_{2,2}^{\left(2\right)}}{dt}+a_{2,2}^{\left(2\right)}\left[\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+\delta h_1\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\right]\right\}dt}\\ +a_{2,3}^{\left(2\right)}\left(\tau \right)\delta a_3^{(1)}\left(\tau \right)-a_{2,3}^{\left(2\right)}\left(0\right)\delta a_3^{(1)}\left(0\right)-{\displaystyle {\int }_0^{\tau }\delta a_3^{(1)}\left(t\right)\frac{d{a}_{2,3}^{\left(2\right)}}{dt}dt}=q^{\left(2\right)},\end{array}$$

  (100)
• The unknown terms on the left side of Equation (100) are eliminated by imposing the following conditions:

  $$a_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

  (101)

  $$a_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_{2,3}^{\left(2\right)}\left(0\right)=0.$$

  (102)
• Using the conditions given in Equations (29), (96), (101) and (102) on the right side of Equation (100) and rearranging the remaining terms yields the following relation:

  $$\begin{array}{l}-a_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0-a_{1,3}^{\left(2\right)}\left(0\right)\delta T_0\\ +{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)\left[-\frac{d{a}_{1,1}^{\left(2\right)}\left(t\right)}{dt}-a_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-a_{1,2}^{\left(2\right)}{F}_2-a_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)+a_{2,2}^{\left(2\right)}{F}_1{F}_3{a}_1^{(1)}\left(t\right)\right]}\\ +{\displaystyle {\int }_0^{\tau }\delta h_2\left(t\right)\left[-\frac{d{a}_{1,2}^{\left(2\right)}}{dt}-a_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{a}_1^{(1)}\left(t\right)\right]dt}-{\displaystyle {\int }_0^{\tau }\delta h_3\left(t\right)\frac{d{a}_{1,3}^{\left(2\right)}}{dt}dt}\\ +{\displaystyle {\int }_0^{\tau }\delta a_1^{(1)}\left(t\right)\left[-\frac{d{a}_{2,1}^{\left(2\right)}}{dt}+a_{2,1}^{\left(2\right)}{F}_1{F}_3{h}_2\left(t\right)+a_{2,2}^{\left(2\right)}{F}_1{F}_3{h}_1\left(t\right)\right]}\\ +{\displaystyle {\int }_0^{\tau }\delta a_2^{(1)}\left(t\right)\left[-\frac{d{a}_{2,2}^{\left(2\right)}}{dt}+a_{2,1}^{\left(2\right)}{F}_2\right]dt}+{\displaystyle {\int }_0^{\tau }\delta a_3^{(1)}\left(t\right)\left[-\frac{d{a}_{2,3}^{\left(2\right)}}{dt}+a_{2,1}^{\left(2\right)}{F}_2{F}_3\right]dt}=q^{\left(2\right)},\end{array}$$

  (103)
• The integral terms on the left side of Equation (103) are now required to represent the “indirect-effect” term defined in Equation (92), which is achieved by imposing the following requirements on the components of the second-level adjoint sensitivity function $a^{\left(2\right)}\triangleq {\left[a_1^{\left(2\right)}\left(t\right),a_2^{\left(2\right)}\left(t\right)\right]}^{†}$:

  $$\begin{array}{l}-{\displaystyle \frac{d{a}_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-a_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-a_{1,2}^{\left(2\right)}\left(t\right)F_2-a_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+a_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{a}_1^{(1)}\left(t\right)\\ \hspace{0.17em}=F_1{a}_1^{(1)}\left(t\right)h_2\left(t\right);\end{array}$$

  (104)

  $$-{\displaystyle \frac{d{a}_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-a_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{a}_1^{(1)}\left(t\right)=F_1{a}_1^{(1)}\left(t\right)h_1\left(t\right);$$

  (105)

  $${\displaystyle \frac{d{a}_{1,3}^{\left(2\right)}\left(t\right)}{dt}}=0;$$

  (106)

  $$-{\displaystyle \frac{d{a}_{2,1}^{\left(2\right)}\left(t\right)}{dt}}+a_{2,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+a_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)=F_1{h}_2\left(t\right)h_1\left(t\right);$$

  (107)

  $$-{\displaystyle \frac{d{a}_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+a_{2,1}^{\left(2\right)}\left(t\right)F_2=0;$$

  (108)

  $$-{\displaystyle \frac{d{a}_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+a_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=0.$$

  (109)

The system of equations comprising Equations (101), (102), (104)–(109) constitutes the “Second-Level Adjoint Sensitivity System (2nd-LASS)” for the second-level adjoint sensitivity function $a^{\left(2\right)}\triangleq {\left[a_1^{\left(2\right)}\left(t\right),a_2^{\left(2\right)}\left(t\right)\right]}^{†}$. Evidently, the 2nd-LASS is linear in $a^{\left(2\right)}\triangleq {\left[a_1^{\left(2\right)}\left(t\right),a_2^{\left(2\right)}\left(t\right)\right]}^{†}$ and is independent of parameter variations. Notably, this system of equations does not need to be solved simultaneously, but can be solved sequentially, by first solving Equations (107)–(109) subject to the initial conditions given in Equation (102) to determine the function $a_2^{\left(2\right)}\left(t\right)$, and subsequently using the function $a_2^{\left(2\right)}\left(t\right)$ in Equations (104)–(106) to solve them subject to the “final-time” condition given in Equation (101) to obtain the function $a_1^{\left(2\right)}\left(t\right)$. The 2nd-LASS is to be solved using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity.

7.

Using the results obtained in Equations (104)–(109) in Equation (103) yields the following alternative expression for the “indirect-effect” term $\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{ind}$ defined in Equation (92):

$$\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{ind}=q^{\left(2\right)}+a_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0+a_{1,3}^{\left(2\right)}\left(0\right)\delta T_0.$$

(110)

Notably, the expression on the right side of Equation (110) no longer involves the second-level variational sensitivity function $v^{\left(1\right)}\left(t\right)$ but involves the second-level adjoint sensitivity function $a^{\left(2\right)}\triangleq {\left[a_1^{\left(2\right)}\left(t\right),a_2^{\left(2\right)}\left(t\right)\right]}^{†}$.

8.

Using in Equation (110) the definition of the function $q^{\left(2\right)}$ provided in Equation (99) and adding the resulting expression for the indirect-effect term $\delta {\left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]}_{dir}$ to the expression for the direct-effect term $\delta {\left(\mathsf{\partial }{r}_n/\mathsf{\partial }{F}_i\right)}_{dir}$ provided in Equation (91) yields the following expression for the total first-order G-differential $\delta \left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]$:

$$\begin{array}{l}\delta \left[\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3\right]=\left(\delta F_1\right){\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)h_2\left(t\right)}\hspace{0.17em}{h}_1\left(t\right)\hspace{0.17em}dt+a_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0+a_{1,3}^{\left(2\right)}\left(0\right)\delta T_0\\ +{\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)\left[\left(\delta F_1\right)F_3+F_1\left(\delta F_3\right)\right]h_1\left(t\right)h_2\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{a}_{1,2}^{\left(2\right)}\left(\delta F_2\right)h_1\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}\left[\left(\delta F_2\right)F_3+F_2\left(\delta F_3\right)\right]h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\left\{\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)a_1^{(1)}\left(t\right)\right.}\hspace{0.17em}\\ \left.+\left(\delta F_2\right)a_2^{(1)}\left(t\right)+\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]a_3^{(1)}\left(t\right)\right\}dt\\ -{\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)a_1^{(1)}\left(t\right)dt}.\end{array}$$

(111)

The expression shown in Equation (111) is to be evaluated at the nominal values of all functions and parameters/weights but the superscript “zero” (which has been used to indicate this fact) has been omitted for notational simplicity.

Identifying the expressions that multiply the individual variations in the quantities $\delta {\psi }_0$, $\delta T_0$ and $\delta F_i$, $i=1,2,3$, yields the following expressions for the second-order sensitivities that stem from the first-order sensitivity $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{F}_3}}=a_{1,1}^{\left(2\right)}\left(0\right);\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{F}_3}}=a_{1,3}^{\left(2\right)}\left(0\right)\hspace{0.17em};$$

(112)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{F}_3}}={\displaystyle {\int }_0^{\tau }{a}_1^{(1)}\left(t\right)h_2\left(t\right)}\hspace{0.17em}{h}_1\left(t\right)\hspace{0.17em}dt+F_3{\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}\\ -F_3{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}{h}_2\left(t\right)a_1^{(1)}\left(t\right)}\hspace{0.17em}-F_3{\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}{h}_1\left(t\right)a_1^{(1)}\left(t\right)dt};\end{array}$$

(113)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{F}_3}}={\displaystyle {\int }_0^{\tau }{a}_{1,2}^{\left(2\right)}{h}_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}{h}_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\left[a_2^{(1)}\left(t\right)+F_3{a}_3^{(1)}\left(t\right)\right]\hspace{0.17em}}dt\hspace{0.17em};$$

(114)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{F}_3}}=F_1{\displaystyle {\int }_0^{\tau }{a}_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}+F_2{\displaystyle {\int }_0^{\tau }{a}_{1,3}^{\left(2\right)}{h}_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{a}_{2,1}^{\left(2\right)}\left[F_1{h}_2\left(t\right)a_1^{(1)}\left(t\right)+F_2{a}_3^{(1)}\left(t\right)\right]}dt-F_1{\displaystyle {\int }_0^{\tau }{a}_{2,2}^{\left(2\right)}{h}_1\left(t\right)a_1^{(1)}\left(t\right)dt}.\end{array}$$

(115)

The second-order sensitivities that stem from the first-order sensitivities $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_2$ are obtained from the G-differentials of the expressions provided in Equations (45), (47) and (48), respectively, and by subsequently following the same procedure as was used in this Section for determining the sensitivities stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_3$, which led to the results presented in Equations (112)–(115). The 2nd-LASS that will thus be obtained for the second-level adjoint sensitivity functions that correspond to the indirect-effect terms stemming from the G-differentials of $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\psi }_0$, $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_2$, respectively, will have the same operators on their left sides as the left side of the 2nd-LASS obtained in Equation (104)–(109). Furthermore, the second-level adjoint sensitivity functions that correspond to the indirect-effect terms stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\psi }_0$,$\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_2$ will satisfy the same initial-time and final-time conditions as shown in Equations (101) and (102). On the other hand, the source terms on the right sides of the 2nd-LASS obtained in Equations (104)–(109) will correspond to the “indirect-effect” terms stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\psi }_0$, $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_2$. Therefore, these source terms will all differ from one another and will also differ from the source terms on the right sides of Equations (104)–(109).

To avoid repetition, the detailed derivation of the second-order sensitivities stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\psi }_0$,$\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{F}_2$ will be omitted, but it is clear that their derivation will entail the construction and subsequent solving of three distinct Second-Level Adjoint Sensitivity Systems. Conceptually, as was shown in the general mathematical formalism underlying the 2nd-FASAM-NODE [1], the computation of the second-order sensitivities entails the construction and solution of as many 2nd-LASS as there are non-zero first-order sensitivities of the response under consideration with respect to the components of the feature functions and encoder’s initial conditions. The second-order sensitivities with respect to the NODE parameters are obtained from the sensitivities with respect to the components of the feature functions by using the “chain rule” of differentiating Equation (50), as shown below:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\theta }_j\mathsf{\partial }{\theta }_i}}={\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{\theta }_j}}\left[{\displaystyle \sum _{k=1}^3\frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{F}_k}\frac{\mathsf{\partial }{F}_k\left(\theta \right)}{\mathsf{\partial }{\theta }_i}}\right]\hspace{0.17em};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}j,i=1,\dots ,TW=6.$$

(116)

Of course, the second-order sensitivities of the response $E\left(\tau \right)$ can be computed directly with respect to the parameters $\theta \triangleq {\left[{\theta }_1,\mathrm{\dots },{\theta }_{TW}\right]}^{†}\triangleq {\left({\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f\right)}^{†}$, $TW=6$, as demonstrated in [5]. In this case, a total of six (as opposed to three) distinct Second-Level Adjoint Sensitivity Systems would have been needed [5], each of these six 2nd-LASSs corresponding to the G-differential of each of the six first-order sensitivities that were determined in Equations (79)–(84).

It is important to note that the second-order mixed sensitivities will be determined twice, obtaining two distinct expressions involving two distinct second-level adjoint functions, for each mixed second-order mixed sensitivity. This fact has been demonstrated in detail in [5] for the second-order mixed sensitivities of $E\left(\tau \right)$ with respect to the primary model parameters and will also be demonstrated in Section 3.2 and Section 4. Determining the mixed second-order sensitivities twice, using distinct expressions and second-level adjoint sensitivity functions provides a stringent test for verifying the accuracy of the computations of the respective adjoint functions.

3.2. Computation of Second-Order Sensitivities of $E(\tau )$ Using the Corresponding Single NODE Equation

It is expected that the most effective procedure for computing second-order sensitivities of a NODE response would be to use as few NODE equations as possible. For the illustrative Nordheim–Fuchs model under consideration, it is expected that the most efficient procedure would be to obtain the second-order sensitivities stemming from the first-order sensitivities $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1$ and $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2$, which were determined in Equations (76) and (77), respectively.

3.2.1. Computation of Second-Order Sensitivities of $E(\tau )$ Stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1$

The second-order sensitivities which stem from the first-order sensitivity $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1$ defined in Equation (76) will be obtained by determining the first-order G-differential of $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1$, which is obtained, by definition, as follows:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}\right\}=-{\left\{{\displaystyle \frac{d}{d\epsilon }}\left[{\displaystyle {\int }_0^{\tau }\left(e^{\left(1\right)}+\epsilon \delta e^{\left(1\right)}\right){\left[E\left(t\right)+\epsilon \delta E\left(t\right)\right]}^{2}dt}\right]\right\}}_{\epsilon =0}\\ =-{\displaystyle {\int }_0^{\tau }\left[2e^{\left(1\right)}\left(t\right)E\left(t\right)\delta E\left(t\right)+\delta e^{\left(1\right)}\left(t\right)E^2\left(t\right)\right]dt}\\ ={\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1}}\delta {\mathrm{\Phi }}_1+{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_1}}\delta {\mathrm{\Phi }}_2.\end{array}$$

(117)

The variational function $\delta e^{\left(1\right)}\left(t\right)$ is the solution of the system of equations obtained by G-differentiating the 1st-LASS defined in Equations (73) and (74). Performing the G-differentiation of this 1st-LASS yields the following equations, which are to be evaluated at the nominal values of the respective parameters and functions:

$$\left[-{\displaystyle \frac{d}{dt}}+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta e^{(1)}\left(t\right)+2{\mathrm{\Phi }}_1{e}^{(1)}\left(t\right)\left[\delta E\left(t\right)\right]=-2\left(\delta {\mathrm{\Phi }}_1\right)e^{(1)}\left(t\right)E\left(t\right),\hspace{0.17em}\hspace{0.17em}0<t<\tau ,$$

(118)

$$\delta e^{(1)}\left(\tau \right)=0,\hspace{1em}t=\tau .$$

(119)

Concatenating Equations (118) and (119) with the 1st-LVSS for $\delta E\left(t\right)$ defined in Equations (69) and (70) yields the following Second-Level Variational Sensitivity System (2nd-LVSS) for the second-level variational function $v^{\left(2\right)}\left(2;t\right)\hspace{0.17em}\triangleq {\left[\delta E\left(t\right),\delta e^{\left(1\right)}\left(t\right)\right]}^{†}$, where the superscript “2”denotes “second-level”:

$$\left(\begin{array}{cc}d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)& 0\\ 2{\mathrm{\Phi }}_1{e}^{(1)}\left(t\right)& -d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\end{array}\right)\left(\begin{array}{c}\delta E\left(t\right)\\ \delta e^{\left(1\right)}\left(t\right)\end{array}\right)=\left(\begin{array}{c}-\left(\delta {\mathrm{\Phi }}_1\right)E^2\left(t\right)+\left(\delta {\mathrm{\Phi }}_2\right)\\ -2\left(\delta {\mathrm{\Phi }}_1\right)e^{(1)}\left(t\right)E\left(t\right)\end{array}\right);$$

(120)

$$\left(\begin{array}{c}\delta E\left(0\right)\\ \delta e^{(1)}\left(\tau \right)\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right).$$

(121)

The need for solving repeatedly the 2nd-LVSS defined by Equations (120) and (121) to obtain the value of the second-level variational function $v^{\left(2\right)}\left(2;t\right)\hspace{0.17em}\triangleq {\left[\delta E\left(t\right),\delta e^{\left(1\right)}\left(t\right)\right]}^{†}$ for every possible parameter variation is circumvented by deriving an alternative expression for the first-order G-differential $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\right\}$ defined in Equation (117), in which the variational function $v^{\left(2\right)}\left(2;t\right)\hspace{0.17em}\triangleq {\left[\delta E\left(t\right),\delta e^{\left(1\right)}\left(t\right)\right]}^{†}$ will be replaced by a two-component second-level adjoint function that will be denoted as $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ and which will be independent of parameter variations. The notation used for the two-component vector $e_1^{(2)}\left(t\right)$ is as follows: the superscript “2” indicates “second-level” while the subscript “1” indicates that this second-level adjoint sensitivity function will correspond to the first-order derivative $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1$ with respect to the “first” feature function, ${\mathrm{\Phi }}_1$. Subsequently, a second-level adjoint sensitivity function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$ will be used for determining the second-order sensitivities stemming from the first-order derivative $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2$ with respect to the “second” feature function, ${\mathrm{\Phi }}_2$.

The second-level adjoint function $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ will be the solution of a Second-Level Adjoint Sensitivity System (2nd-LASS) to be constructed by applying the principles underlying the 2nd-FASAM-NODE methodology [1]. This 2nd-LASS is constructed in a Hilbert space denoted as ${\mathrm{H}}_{2,E}\left({\Omega }_t\right)$, ${\Omega }_t\triangleq \left[0,\tau \right]$, where the subscript “2” indicates “second-level” and which comprises as elements two-component vectors of the same form as $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$. The Hilbert space ${\mathrm{H}}_{2,E}\left({\Omega }_t\right)$ is considered to be endowed with an inner product, denoted as ${〈{\chi }^{\left(2\right)},{\eta }^{\left(2\right)}〉}_{2,E}$, of two vectors ${\chi }^{\left(2\right)}\left(t\right)\triangleq {\left[{\chi }_1^{\left(2\right)}\left(t\right),{\chi }_2^{\left(2\right)}\left(t\right)\right]}^{†}$ and ${\eta }^{\left(2\right)}\left(t\right)\triangleq {\left[{\eta }_1^{\left(2\right)}\left(t\right),{\eta }_2^{\left(2\right)}\left(t\right)\right]}^{†}$, and is defined as follows:

$${〈{\chi }^{\left(2\right)},{\eta }^{\left(2\right)}〉}_{2,E}\triangleq {\displaystyle \sum _{i=1}^2{\displaystyle {\int }_0^{\tau }{\chi }_i^{\left(2\right)}\left(t\right){\eta }_i^{\left(2\right)}\left(t\right)dt}}.$$

(122)

Using the definition provided in Equation (122), form the inner product of $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ with Equation (120) to obtain the following relation:

$$\begin{array}{l}{\displaystyle {\int }_0^{\tau }{e}_{11}^{(2)}\left(t\right)\left[d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta E\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{e}_{12}^{(2)}\left(t\right)\left[2{\mathrm{\Phi }}_1{\omega }^{(1)}\left(t\right)\delta E\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{e}_{12}^{(2)}\left(t\right)\left[-d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta e^{\left(1\right)}\left(t\right)dt}=s_1^{\left(2\right)}\left(t\right),\end{array}$$

(123)

where:

$$s_1^{\left(2\right)}\left(t\right)\triangleq {\displaystyle {\int }_0^{\tau }{e}_{11}^{(2)}\left(t\right)\left[-\left(\delta {\mathrm{\Phi }}_1\right)E^2\left(t\right)+\left(\delta {\mathrm{\Phi }}_2\right)\right]dt}-2\left(\delta {\mathrm{\Phi }}_1\right){\displaystyle {\int }_0^{\tau }{e}_{12}^{(2)}\left(t\right)e^{(1)}\left(t\right)E\left(t\right)dt}.$$

(124)

Integrating by parts the terms involving $d/dt$ on the left side of Equation (123) and rearranging the resulting terms yields the following relation:

$$\begin{array}{l}{e}_{11}^{(2)}\left(\tau \right)\delta E\left(\tau \right)-e_{11}^{(2)}\left(0\right)\delta E\left(0\right)-e_{12}^{(2)}\left(\tau \right)\delta e^{\left(1\right)}\left(\tau \right)+e_{12}^{(2)}\left(0\right)\delta e^{\left(1\right)}\left(0\right)\\ +{\displaystyle {\int }_0^{\tau }\delta E\left(t\right)\left\{\left[-d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]e_{11}^{(2)}\left(t\right)+2{\mathrm{\Phi }}_1{e}^{(1)}\left(t\right)e_{12}^{(2)}\left(t\right)\right\}}dt\\ +{\displaystyle {\int }_0^{\tau }\delta e^{\left(1\right)}\left(t\right)\left[d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]e_{12}^{(2)}\left(t\right)dt}=s_1^{\left(2\right)}\left(t\right)\hspace{0.17em}.\end{array}$$

(125)

The integral terms on the left side of Equation (125) are required to represent the G-differential $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\right\}$ defined in Equation (117), and the unknown terms on the left side of Equation (125) are eliminated by requiring the function $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ to satisfy the following Second-Level Adjoint System (2nd-LASS):

$$\left(\begin{array}{cc}-d/dt+2{\mathrm{\Phi }}_1\left(p\right)E\left(t\right)& 2{\mathrm{\Phi }}_1\left(p\right)e^{(1)}\left(t\right)\\ 0& d/dt+2{\mathrm{\Phi }}_1\left(p\right)E\left(t\right)\end{array}\right)\left(\begin{array}{c}{e}_{11}^{(2)}\left(t\right)\\ e_{12}^{(2)}\left(t\right)\end{array}\right)=\left(\begin{array}{c}-2e^{\left(1\right)}\left(t\right)E\left(t\right)\\ -E^2\left(t\right)\end{array}\right)\hspace{0.17em};$$

(126)

$$\left(\begin{array}{c}{e}_{11}^{(2)}\left(\tau \right)\\ e_{12}^{(2)}\left(0\right)\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)\hspace{0.17em}.$$

(127)

The above 2nd-LASS is to be solved at the nominal values for the first-level adjoint function $e^{\left(1\right)}\left(t\right)$ and for the parameters $p\triangleq {\left[{\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f,{\psi }_0\right]}^{†}$. Notably, the 2nd-LASS is independent of variations in the feature functions (and/or parameter variations), so it needs to be solved just once to obtain the second-level adjoint function $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$. Furthermore, the 2nd-LASS is an upper-triangular system, so the respective equations need not solved simultaneously, but can be solved sequentially, first for the component $e_{12}^{(2)}\left(t\right)$ and subsequently for the component $e_{11}^{(2)}\left(t\right)$. The explicit expressions of the components of $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ are not essential for following the subsequent conceptual derivations.

Using in Equation (125) the 2nd-LASS defined by Equations (126) and (127) together with the conditions provided in Equation (121) yields the following expression for the variation $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\right\}$ in terms of the second-level adjoint function $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}\right\}={\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1}}\delta {\mathrm{\Phi }}_1+{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_1}}\delta {\mathrm{\Phi }}_2\\ ={\displaystyle {\int }_0^{\tau }{e}_{11}^{(2)}\left(t\right)\left[-\left(\delta {\mathrm{\Phi }}_1\right)E^2\left(t\right)+\left(\delta {\mathrm{\Phi }}_2\right)\right]dt}-2\left(\delta {\mathrm{\Phi }}_1\right){\displaystyle {\int }_0^{\tau }{e}_{12}^{(2)}\left(t\right)e^{(1)}\left(t\right)E\left(t\right)dt}.\end{array}$$

(128)

It follows from Equation (128) that the second-order sensitivities of the response $E\left(\tau \right)$ with respect to the feature functions ${\mathrm{\Phi }}_1$ and ${\mathrm{\Phi }}_2$ have the following expressions:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1}}=-{\displaystyle {\int }_0^{\tau }\left[e_{11}^{(2)}\left(t\right)E\left(t\right)+2e_{12}^{(2)}\left(t\right)e^{(1)}\left(t\right)\right]}\hspace{0.17em}E\left(t\right)dt;$$

(129)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_1}}={\displaystyle {\int }_0^{\tau }{e}_{11}^{(2)}\left(t\right)dt}.$$

(130)

3.2.2. Computation of Second-Order Sensitivities of $E(\tau )$ Stemming from $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2$

The second-order sensitivities which stem from the first-order sensitivity $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2$ are obtained from the first-order G-differential $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\right\}$ of the expression provided in Equation (77), which yields, by definition, the following expression:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}\right\}={\left\{{\displaystyle \frac{d}{d\epsilon }}\left[{\displaystyle {\int }_0^{\tau }\left[e^{\left(1\right)}\left(t\right)+\epsilon \delta e^{\left(1\right)}\left(t\right)\right]dt}\right]\right\}}_{\epsilon =0}\\ ={\displaystyle {\int }_0^{\tau }\delta e^{\left(1\right)}\left(t\right)dt}\triangleq {\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_{22}\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2}}\delta {\mathrm{\Phi }}_1+{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2}}\delta {\mathrm{\Phi }}_2.\end{array}$$

(131)

The variational function $\delta e^{\left(1\right)}\left(t\right)$ is the solution of Equations (118) and (119). Notably, the right side of Equation (131) depends only on the variational function $\delta e^{\left(1\right)}\left(t\right)$, but does not depend directly on the variational function $\delta E\left(t\right)$. Nevertheless, since the variational function $\delta e^{\left(1\right)}\left(t\right)$ is related to the variational function $\delta E\left(t\right)$ through Equations (118) and (119), the second-level adjoint function that will be constructed in order to eliminate the appearance of $\delta e^{\left(1\right)}\left(t\right)$ on the right side of Equation (131) will be the solution of a 2nd-LASS which will correspond to the 2nd-LVSS defined by Equations (120) and (121). The construction of the 2nd-LASS that will be used to eliminate the appearance of the variational function $\delta e^{\left(1\right)}\left(t\right)$ from Equation (131) is based on the principles underlying the 2nd-FASAM-NODE [1], using the same Hilbert Space, ${\mathrm{H}}_{2,E}\left({\Omega }_t\right)$, with the inner product defined in Equation (122), as was used for obtaining the results shown in Equations (129) and (130).

The inner product defined in Equation (122) is now used to construct the Second-Level Adjoint Sensitivity System (2nd-LASS) for the second-level adjoint function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$ by following the same sequence of steps as was used to obtain the expressions shown in Equations (129) and (130), but using the expression provided in Equation(131) to determine the right side (“source”) for the 2nd-LASS to be used for determining the second-order sensitivities stemming from $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\right\}$. Thus, using the definition provided in Equation (122), form the inner product of $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$ with Equation (120) to obtain the following relation:

$$\begin{array}{l}{\displaystyle {\int }_0^{\tau }{e}_{21}^{(2)}\left(t\right)\left[d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta E\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{e}_{22}^{(2)}\left(t\right)\left[2{\mathrm{\Phi }}_1{\omega }^{(1)}\left(t\right)\delta E\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{e}_{22}^{(2)}\left(t\right)\left[-d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]\delta e^{\left(1\right)}\left(t\right)dt}=s_2^{\left(2\right)}\left(t\right),\end{array}$$

(132)

where:

$$s_2^{\left(2\right)}\left(t\right)\triangleq {\displaystyle {\int }_0^{\tau }{e}_{21}^{(2)}\left(t\right)\left[-\left(\delta {\mathrm{\Phi }}_1\right)E^2\left(t\right)+\left(\delta {\mathrm{\Phi }}_2\right)\right]dt}-2\left(\delta {\mathrm{\Phi }}_1\right){\displaystyle {\int }_0^{\tau }{e}_{22}^{(2)}\left(t\right)e^{(1)}\left(t\right)E\left(t\right)dt}.$$

(133)

Integrating by parts the terms involving $d/dt$ on the left side of Equation (132) and rearranging the resulting terms yields the following relation:

$$\begin{array}{l}{e}_{21}^{(2)}\left(\tau \right)\delta E\left(\tau \right)-e_{21}^{(2)}\left(0\right)\delta E\left(0\right)-e_{22}^{(2)}\left(\tau \right)\delta e^{\left(1\right)}\left(\tau \right)+e_{22}^{(2)}\left(0\right)\delta e^{\left(1\right)}\left(0\right)\\ +{\displaystyle {\int }_0^{\tau }\delta E\left(t\right)\left\{\left[-d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]e_{21}^{(2)}\left(t\right)+2{\mathrm{\Phi }}_1{e}^{(1)}\left(t\right)e_{22}^{(2)}\left(t\right)\right\}}dt\\ +{\displaystyle {\int }_0^{\tau }\delta e^{\left(1\right)}\left(t\right)\left[d/dt+2{\mathrm{\Phi }}_{1}E\left(t\right)\right]e_{22}^{(2)}\left(t\right)dt}=s_2^{\left(2\right)}\left(t\right)\hspace{0.17em}.\end{array}$$

(134)

The integral terms on the left side of Equation (134) are required to represent the G-differential $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\right\}$ defined in Equation (131), and the unknown terms on the left side of Equation (134) are eliminated by requiring the function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$ to satisfy the following Second-Level Adjoint System (2nd-LASS)

$$\left(\begin{array}{cc}-d/dt+2{\mathrm{\Phi }}_1\left(p\right)E\left(t\right)& 2{\mathrm{\Phi }}_1\left(p\right)e^{(1)}\left(t\right)\\ 0& d/dt+2{\mathrm{\Phi }}_1\left(p\right)E\left(t\right)\end{array}\right)\left(\begin{array}{c}{e}_{21}^{(2)}\left(t\right)\\ e_{22}^{(2)}\left(t\right)\end{array}\right)=\left(\begin{array}{c}0\\ 1\end{array}\right)\hspace{0.17em};$$

(135)

$$\left(\begin{array}{c}{e}_{21}^{(2)}\left(\tau \right)\\ e_{22}^{(2)}\left(0\right)\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)\hspace{0.17em}.$$

(136)

The above 2nd-LASS is to be solved at the nominal values for the first-level adjoint function $e^{\left(1\right)}\left(t\right)$ and for the parameters $p\triangleq {\left[{\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f,{\psi }_0\right]}^{†}$. Since the 2nd-LASS is independent of variations in the feature functions (and/or parameter variations), it needs to be solved just once to obtain the second-level adjoint function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$. Furthermore, the 2nd-LASS is an upper-triangular system, so the respective equations need not be solved simultaneously, but can be solved sequentially, first for the component $e_{22}^{(2)}\left(t\right)$ and subsequently for the component $e_{21}^{(2)}\left(t\right)$.

Using in Equation (134) the 2nd-LASS defined by Equations (135) and (136) together with the conditions provided in Equation (121) yields the following expression for the variation $\delta \left\{\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\right\}$ in terms of the second-level adjoint function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}\right\}={\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2}}\delta {\mathrm{\Phi }}_1+{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1,{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2}}\delta {\mathrm{\Phi }}_2\\ ={\displaystyle {\int }_0^{\tau }{e}_{21}^{(2)}\left(t\right)\left[-\left(\delta {\mathrm{\Phi }}_1\right)E^2\left(t\right)+\left(\delta {\mathrm{\Phi }}_2\right)\right]dt}-2\left(\delta {\mathrm{\Phi }}_1\right){\displaystyle {\int }_0^{\tau }{e}_{22}^{(2)}\left(t\right)e^{(1)}\left(t\right)E\left(t\right)dt}.\end{array}$$

(137)

From Equation (137), we can infer the following:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2}}=-{\displaystyle {\int }_0^{\tau }\left[e_{21}^{(2)}\left(t\right)E\left(t\right)+2e_{22}^{(2)}\left(t\right)e^{(1)}\left(t\right)\right]}\hspace{0.17em}E\left(t\right)dt\hspace{0.17em};$$

(138)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2}}={\displaystyle {\int }_0^{\tau }{e}_{21}^{(2)}\left(t\right)dt}.$$

(139)

It is important to note that the second-order mixed partial derivative ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2={\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_1$ can be obtained from either Equation (138) or Equation (130). The equivalence between these expressions provides a stringent verification of the accuracy of solving (on the one hand) the 2nd-LASS comprising Equations (126) and (127) to obtain $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ and solving (on the other hand) the 2nd-LASS comprising Equations (135) and (136) to obtain $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$.

Notably, only two “large-scale computations” are necessary for solving the two distinct 2nd-LASS for obtaining the second-level adjoint functions $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$ and $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$ involved in the computation of the three distinct second-order response sensitivities, i.e., ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1$, ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2$ and ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2$, with respect to the two feature functions ${\mathrm{\Phi }}_1$ and ${\mathrm{\Phi }}_2$. The second-order sensitivities of the response $E\left(\tau \right)$ with respect to the primary model parameters are obtained by using the total differential of the expression provided in Equation (78) in conjunction with the parameter dependencies of the functions ${\mathrm{\Phi }}_1\left(p\right)$ and ${\mathrm{\Phi }}_2\left(p\right)$ defined in Equation (66), and with the expressions obtained in Equations (129), (130), (138) and (139).

In summary, computing all of the second-order sensitivities of the response $E\left(\tau \right)$ with respect to the feature functions and initial conditions using the original NODE equations requires as many large-scale computations as there are component features (three components) functions and initial conditions (two initial conditions), for a total of five “large-scale” computations to solve the respective 2nd-LASS systems, for a total of 5 × 6 = 30 differential equations (since each of the respective 2nd-LASS comprises six differential equations). In contradistinction, computing all of the second-order sensitivities of the response $E\left(\tau \right)$ with respect to the primary model parameters using the modified NODE equation for the single dependent variable $E\left(t\right)$ requires two “large-scale” computations to solve the 2nd-LASS represented by Equation (126) and, respectively, Equation (135), each involving two-coupled differential equations, amounting to a total of four differential equations, followed by inexpensive computations using the “chain rule” of differentiation. Hence, it is very advantageous to extract equations involving as few dependent variables as possible from the original NODE system (like the single equation for $E\left(t\right)$ in the illustrative example in this subsection), even if this procedure entails modifications to the original NODE system.

After having obtained the second-order sensitivities ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1$, ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2$ and ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2$, the exact expressions of the second-order sensitivities ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$ of the response $E\left(\tau \right)$ with respect to the components of the vector of primary parameters $p\triangleq {\left[p_1,\mathrm{\dots },p_7\right]}^{†}\triangleq {\left[{\alpha }_T,l_p,c_p,\gamma ,{\sigma }_f,N_f,{\psi }_0\right]}^{†}$ can be determined analytically by applying the chain rule of differentiation to the expression of the first-order sensitivities $\mathsf{\partial }E\left(\tau \right)/\mathsf{\partial }{p}_i$, $i=1,\mathrm{\dots },7$, shown in Equation (78), to obtain the following relation:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau ;{\mathrm{\Phi }}_1;{\mathrm{\Phi }}_2\right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}}={\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_j}}\left[{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_1}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_1\left(p\right)}{\mathsf{\partial }{p}_i}}+{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{\mathrm{\Phi }}_2}}{\displaystyle \frac{\mathsf{\partial }{\mathrm{\Phi }}_2\left(p\right)}{\mathsf{\partial }{p}_i}}\right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\mathrm{\dots },7.$$

(140)

After having obtained the second-order sensitivities ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$, the second-order sensitivities ${\mathsf{\partial }}^{2}T\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$ of the temperature response $T\left(\tau \right)$ can be determined analytically by applying the chain rule of differentiation to the expression of the first-order sensitivities $\mathsf{\partial }T\left(\tau \right)/\mathsf{\partial }{p}_i$, $i=1,\mathrm{\dots },7$, shown in Equation (86), to obtain the following relation:

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}T\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}}={\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_j}}\left[{\displaystyle \frac{\mathsf{\partial }\left(1/c_p\right)}{\mathsf{\partial }{p}_i}}E\left(\tau \right)+{\displaystyle \frac{1}{c_p}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}\right]={\displaystyle \frac{{\mathsf{\partial }}^2\left(1/c_p\right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}}E\left(\tau \right)\\ +{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_j}}{\displaystyle \frac{\mathsf{\partial }\left(1/c_p\right)}{\mathsf{\partial }{p}_i}}+{\displaystyle \frac{\mathsf{\partial }\left(1/c_p\right)}{\mathsf{\partial }{p}_j}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}+{\displaystyle \frac{1}{c_p}}{\displaystyle \frac{{\mathsf{\partial }}^{2}E\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\mathrm{\dots },7.\end{array}$$

(141)

After having obtained the second-order sensitivities ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$, the second-order sensitivities ${\mathsf{\partial }}^2\psi \left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$ of the neutron flux response $\psi \left(\tau \right)$ can be determined analytically by applying the chain rule of differentiation to the expression of the first-order sensitivities $\mathsf{\partial }\psi \left(\tau \right)/\mathsf{\partial }{p}_i$, $i=1,\mathrm{\dots },7$, shown in Equation (88), to obtain the following relations:

$${\displaystyle \frac{{\mathsf{\partial }}^2\psi \left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{\psi }_0}}=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}j=1,\mathrm{\dots },7;$$

(142)

$${\displaystyle \frac{{\mathsf{\partial }}^2\psi \left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}}=-{\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_j}}\left[E^2\left(\tau \right){\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_i}}\left({\displaystyle \frac{{\alpha }_T}{2l_p{c}_p\gamma {\sigma }_f{N}_f}}\right)+{\displaystyle \frac{{\alpha }_{T}E\left(\tau \right)}{l_p{c}_p\gamma {\sigma }_f{N}_f}}{\displaystyle \frac{\mathsf{\partial }E\left(\tau \right)}{\mathsf{\partial }{p}_i}}\right];\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\mathrm{\dots },7.$$

(143)

4. Application of the 2nd-FASAM-NODE Methodology to Compute Second-Order Sensitivities of an Illustrative Nordheim–Fuchs Model Response Involving Decoder Weights

This Section presents the application of the 2nd-FASAM-NODE methodology to compute second-order sensitivities of the thermal conductivity in the Nordheim–Fuchs model, which is a representative model response involving decoder weights subject to uncertainties. Section 4.1 presents the determination of the second-order sensitivities by applying the 2nd-FASAM-NODE methodology to the original NODE structure. Section 4.2 presents the alternative computation of these second-order sensitivities by applying the 2nd-FASAM-NODE methodology to the decoupled NODE structure. It will be shown that using a decoupled structure, whenever possible, is even more advantageous for the computation of second-order sensitivities than for the computation of the first-order ones.

4.1. Second-Order Sensitivities of the Thermal Conductivity Response $K\left(\tau \right)$ Computed Using the Coupled NODE Equations

The application of the 2nd-FASAM-NODE methodology to a Nordheim–Fuchs model response involving decoder weights will be illustrated by considering the thermal conductivity response $K\left(\tau \right)$. The first-order sensitivities of $K\left(\tau \right)$ were obtained in Section 2.3, as follows:

(i)

The first-order sensitivities of $K\left(\tau \right)$ with respect to the decoder weights arose from the “direct-effect term” defined in Equation (52), and were obtained in Equations (54)–(56). The computation of the second-order sensitivities stemming from these first-order sensitivities will be illustrated in Section 4.1.1.

(ii)

The first-order sensitivities of $K\left(\tau \right)$ with respect to the feature functions were obtained in Equations (62)–(64). The second-order sensitivities stemming from these first-order sensitivities will be illustrated in Section 4.1.2.

(iii)

The first-order sensitivities of $K\left(\tau \right)$ with respect to the initial conditions were obtained in Equations (60) and (61). The second-order sensitivities stemming from these first-order sensitivities will be illustrated in Section 4.1.3.

4.1.1. Second-Order Sensitivities Stemming from the First-Order Sensitivities of $K\left(\tau \right)$ with Respect to the Decoder Weights ${\phi }_0$, ${\phi }_1$ and ${\phi }_2$

A.

Second-order sensitivities stemming from $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_0$

It is evident from the expression of $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_0$ obtained in Equation (54) that all of the second-order sensitivities which stem from this first-order sensitivity are identically zero, i.e., ${\mathsf{\partial }}^{2}K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_i\mathsf{\partial }{\phi }_0\equiv 0$, for $i=1,2,3$; ${\mathsf{\partial }}^{2}K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_i\mathsf{\partial }{\phi }_0\equiv 0$, for $i=1,2,3$; a, for $i=1,2,3$.

B.

Second-order sensitivities stemming from $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1$

The second-order sensitivities which stem from the first-order sensitivity $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1$ are determined from the G-differential of the expression provided in Equation (55), which is obtained, by definition, as follows:

$$\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left[T\left(\tau \right)\right]}{\mathsf{\partial }{\phi }_1}}\right\}\hspace{0.17em}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[T^0\left(t\right)+\epsilon \delta T\left(t\right)\right]\delta \left(t-\tau \right)dt}\right\}}_{\epsilon =0}={\displaystyle {\int }_0^{\tau }\delta T\left(t\right)\delta \left(t-\tau \right)dt}.$$

(144)

As indicated by Equation (144), the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1\right\}$ depends only on the variation $\delta T\left(t\right)$. It therefore follows that $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1\right\}$ can be evaluated by following the procedure outlined in Section 2.3, to obtain the following expression:

$$\begin{array}{l}\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1\right\}=b_1^{(2)}\left(0\right)\delta {\psi }_0+b_3^{(2)}\left(0\right)\delta T_0\hspace{0.17em}+{\displaystyle {\int }_0^{\tau }{\left[b^{(2)}\left(t\right)\right]}^{†}}{\displaystyle \frac{\mathsf{\partial }f\left(h;F\right)}{\mathsf{\partial }F}}\delta Fdt\\ \triangleq {\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{\phi }_1}}\delta {\psi }_0+{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{\phi }_1}}\delta T_0+{\displaystyle \sum _{i=1}^3\frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_i\mathsf{\partial }{\phi }_1}\delta F_i}\hspace{0.17em},\end{array}$$

(145)

where the second-level adjoint sensitivity function $b^{(2)}\left(t\right)\triangleq {\left[b_1^{(2)}\left(t\right),b_2^{(2)}\left(t\right),b_3^{(2)}\left(t\right)\right]}^{†}$ is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS), which is to be solved at the nominal values for all parameters and original functions:

$$-{\displaystyle \frac{d{b}^{(2)}\left(t\right)}{dt}}-J^{†}{b}^{(2)}\left(t\right)={\left(0,0,1\right)}^{†};$$

(146)

$$b^{(2)}\left(\tau \right)\triangleq {\left[b_1^{(2)}\left(\tau \right),b_2^{(2)}\left(\tau \right),b_3^{(2)}\left(\tau \right)\right]}^{†}={\left(0,0,0\right)}^{†}.$$

(147)

It follows from Equation (145) that the respective second-order sensitivities have the following expressions:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{\phi }_1}}=b_1^{(2)}\left(0\right)={\displaystyle {\int }_0^{\tau }{b}_1^{(2)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(148)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{\phi }_1}}=b_3^{(2)}\left(0\right)={\displaystyle {\int }_0^{\tau }{b}_3^{(2)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(149)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{\phi }_1}}=F_3\left(\theta \right){\displaystyle {\int }_0^{\tau }{h}_1\left(t\right)h_2\left(t\right)b_1^{(2)}\left(t\right)dt};$$

(150)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{\phi }_1}}={\displaystyle {\int }_0^{\tau }\left[b_2^{(2)}\left(t\right)+F_3\left(\theta \right)b_3^{(2)}\left(t\right)\right]h_1\left(t\right)dt};$$

(151)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{\phi }_1}}={\displaystyle {\int }_0^{\tau }\left[F_1\left(\theta \right)b_1^{(2)}\left(t\right)h_2\left(t\right)+F_2\left(\theta \right)b_3^{(2)}\left(t\right)\right]h_1\left(t\right)dt}.$$

(152)

C.

Second-order sensitivities stemming from $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_2$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_2$ are determined from the G-differential of the expression provided in Equation (56), which is obtained, by definition, as follows:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left[T\left(\tau \right)\right]}{\mathsf{\partial }{\phi }_2}}\right\}\hspace{0.17em}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }{\left[T^0\left(t\right)+\epsilon \delta T\left(t\right)\right]}^2\delta \left(t-\tau \right)dt}\right\}}_{\epsilon =0}\\ =2{\displaystyle {\int }_0^{\tau }{T}^0\left(t\right)\delta T\left(t\right)\delta \left(t-\tau \right)dt}\hspace{0.17em}.\end{array}$$

(153)

The second-order sensitivities stemming from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_2\right\}$ obtained in Equation (153) are derived by following the same procedure as outlined in Section 2.3, which was also used above to obtain the second-order sensitivities stemming from $\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\phi }_1$. The final expressions of these second-order sensitivities are as follows:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{\phi }_2}}=c_1^{(2)}\left(0\right)={\displaystyle {\int }_0^{\tau }{c}_1^{(2)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(154)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{\phi }_2}}=c_3^{(2)}\left(0\right)={\displaystyle {\int }_0^{\tau }{c}_3^{(2)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt};$$

(155)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{\phi }_2}}=F_3\left(\theta \right){\displaystyle {\int }_0^{\tau }{h}_1\left(t\right)h_2\left(t\right)c_1^{(2)}\left(t\right)dt};$$

(156)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{\phi }_2}}={\displaystyle {\int }_0^{\tau }\left[b_2^{(2)}\left(t\right)+F_3\left(\theta \right)c_3^{(2)}\left(t\right)\right]h_1\left(t\right)dt};$$

(157)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{\phi }_2}}={\displaystyle {\int }_0^{\tau }\left[F_1\left(\theta \right)c_1^{(2)}\left(t\right)h_2\left(t\right)+F_2\left(\theta \right)c_3^{(2)}\left(t\right)\right]h_1\left(t\right)dt};$$

(158)

where the second-order adjoint sensitivity function $c^{(2)}\left(t\right)\triangleq {\left[c_1^{(2)}\left(t\right),c_2^{(2)}\left(t\right),c_3^{(2)}\left(t\right)\right]}^{†}$ is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS):

$$-{\displaystyle \frac{d{c}^{(2)}\left(t\right)}{dt}}-J^{†}{c}^{(2)}\left(t\right)={\left[0,0,2T\left(t\right)\right]}^{†};$$

(159)

$$c^{(2)}\left(\tau \right)\triangleq {\left[c_1^{(2)}\left(\tau \right),c_2^{(2)}\left(\tau \right),c_3^{(2)}\left(\tau \right)\right]}^{†}={\left(0,0,0\right)}^{†}.$$

(160)

4.1.2. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Feature Functions $F_1$, $F_2$ and $F_3$

A.

Second-order sensitivities stemming from $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1$ are obtained from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_1\right\}$, which is obtained by using Equation (62), as follows:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_1}}\right\}\triangleq \left\{{\displaystyle \frac{d}{d\epsilon }}\right.{\displaystyle {\int }_0^{\tau }\left[F_3^0+\epsilon \delta F_3\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]}\\ {\left.\times \left[h_2^0\left(t\right)+\epsilon \delta h_2\left(t\right)\right]\left[b_1^{(1,0)}\left(t\right)+\epsilon \delta b_1^{(1)}\left(t\right)\right]dt\right\}}_{\epsilon =0}=\left(\delta F_3\right){\displaystyle {\int }_0^{\tau }{h}_1^0\left(t\right)h_2^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}\\ +F_3^0{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)h_2^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}+F_3^0{\displaystyle {\int }_0^{\tau }\delta h_2\left(t\right)h_1^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}\\ +F_3^0{\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)h_1^0\left(t\right)h_2^0\left(t\right)dt}=\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{dir}+\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind},\end{array}$$

(161)

where the superscript “zero” indicates, as before, that the respective quantities are to be evaluated at their nominal values. The following definitions were used for the “direct-effect” and, respectively, “indirect-effect” terms defined in Equation (161):

$$\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{dir}\triangleq \left(\delta F_3\right){\displaystyle {\int }_0^{\tau }{h}_1^0\left(t\right)h_2^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}$$

(162)

$$\begin{array}{l}\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}\triangleq F_3^0{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)h_2^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}+F_3^0{\displaystyle {\int }_0^{\tau }\delta h_2\left(t\right)h_1^0\left(t\right)b_1^{(1,0)}\left(t\right)dt}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+F_3^0{\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)h_1^0\left(t\right)h_2^0\left(t\right)dt}.\end{array}$$

(163)

The variational function $\delta b^{\left(1\right)}\left(t\right)\triangleq {\left[\delta b_1^{(1)}\left(t\right),\delta b_2^{(1)}\left(t\right),\delta b_3^{(1)}\left(t\right)\right]}^{†}$ is the solution of the G-differentiated First-Level Adjoint Sensitivity System defined by Equations (58) and (59), which is obtained by applying the definition of the G-differential to these equations in order to obtain the following relations:

$$\begin{array}{l}{\displaystyle \frac{d}{dt}}\delta b_1^{(1)}\left(t\right)=-\delta b_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-\delta b_2^{(1)}\left(t\right)F_2-\delta b_3^{(1)}\left(t\right)F_2{F}_3-\delta h_2\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\\ -\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)b_1^{(1)}\left(t\right)-\left(\delta F_2\right)b_2^{(1)}\left(t\right)-\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]b_3^{(1)}\left(t\right)\hspace{0.17em};\end{array}$$

(164)

$$\begin{array}{l}{\displaystyle \frac{d}{dt}}\delta b_2^{(1)}\left(t\right)=-\delta b_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)-\delta h_1\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}-\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)b_1^{(1)}\left(t\right),\end{array}$$

(165)

$${\displaystyle \frac{d}{dt}}\delta b_3^{(1)}\left(t\right)=-\left[\delta {\phi }_1+2\delta {\phi }_2{h}_3\left(t\right)+2{\phi }_2\delta h_3\left(t\right)\right]\delta \left(t-\tau \right).$$

(166)

$$\delta b^{\left(1\right)}\left(\tau \right)\triangleq {\left[\delta b_1^{(1)}\left(\tau \right),\delta b_2^{(1)}\left(\tau \right),\delta b_3^{(1)}\left(\tau \right)\right]}^{†}={\left[0,0,0\right]}^{†}$$

(167)

The indirect-effect term defined by Equation (163) is evaluated by following the same procedural steps as described in Section 3.1, which comprises the following sequence of steps:

• Use the definition of the inner product provided in Equation (97) to form the inner product of Equations (26)–(28) and (164)–(166) with the second-level adjoint sensitivity functions $u^{\left(2\right)}\triangleq {\left[u_1^{\left(2\right)}\left(t\right),u_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, $u_1^{\left(2\right)}\left(t\right)\triangleq {\left[u_{1,1}^{\left(2\right)}\left(t\right),u_{1,2}^{\left(2\right)}\left(t\right),u_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and $u_2^{\left(2\right)}\left(t\right)\triangleq {\left[u_{2,1}^{\left(2\right)}\left(t\right),u_{2,2}^{\left(2\right)}\left(t\right),u_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ to obtain the following relationship:

  $$\begin{array}{l}{\displaystyle {\int }_0^{\tau }{u}_{1,1}^{\left(2\right)}\left(t\right)\frac{d}{dt}\delta h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{u}_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3\left[\delta h_1\left(t\right)h_2\left(t\right)+h_1\left(t\right)\delta h_2\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{u}_{1,2}^{\left(2\right)}\frac{d}{dt}\delta h_2\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{u}_{1,2}^{\left(2\right)}{F}_2\delta h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\frac{d}{dt}\delta h_3\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)\delta h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\frac{d}{dt}\delta b_1^{(1)}\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\left[\delta b_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+\delta b_2^{(1)}\left(t\right)F_2+\delta b_3^{(1)}\left(t\right)F_2{F}_3+\delta h_2\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\right]}\hspace{0.17em}dt\\ {\displaystyle {\int }_0^{\tau }{u}_{2,2}^{\left(2\right)}\frac{d}{dt}\delta b_2^{(1)}\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{u}_{2,2}^{\left(2\right)}\left[\delta b_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+\delta h_1\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }{u}_{2,3}^{\left(2\right)}\frac{d}{dt}\delta b_3^{(1)}\left(t\right)dt}+2{\phi }_2{\displaystyle {\int }_0^{\tau }\delta h_3\left(t\right)u_{2,3}^{\left(2\right)}\delta \left(t-\tau \right)dt}=r^{\left(2\right)},\end{array}$$

  (168)

  $$\begin{array}{l}{r}^{\left(2\right)}\triangleq {\displaystyle {\int }_0^{\tau }{u}_{1,1}^{\left(2\right)}\left(t\right)\left[\left(\delta F_1\right)F_3+F_1\left(\delta F_3\right)\right]h_1\left(t\right)h_2\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{u}_{1,2}^{\left(2\right)}\left(\delta F_2\right)h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\left[\left(\delta F_2\right)F_3+F_2\left(\delta F_3\right)\right]h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\left\{\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)b_1^{(1)}\left(t\right)+\left(\delta F_2\right)b_2^{(1)}\left(t\right)\right.}\\ \left.+\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]b_3^{(1)}\left(t\right)\right\}dt-{\displaystyle {\int }_0^{\tau }{u}_{2,2}^{\left(2\right)}\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)b_1^{(1)}\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{u}_{2,3}^{\left(2\right)}\left[\delta {\phi }_1+2\delta {\phi }_2{h}_3\left(t\right)\right]\delta \left(t-\tau \right)dt}.\end{array}$$

  (169)
• Integrating the left side of Equation (168) by parts over the independent variable $t$ yields the following relation:

  $$\begin{array}{l}{\displaystyle {\int }_0^{\tau }\left\{-\delta h_1\left(t\right)\frac{d{u}_{1,1}^{\left(2\right)}\left(t\right)}{dt}-u_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3\left[\delta h_1\left(t\right)h_2\left(t\right)+h_1\left(t\right)\delta h_2\left(t\right)\right]\right\}dt}\\ +u_{1,1}^{\left(2\right)}\left(\tau \right)\delta h_1\left(\tau \right)-u_{1,1}^{\left(2\right)}\left(0\right)\delta h_1\left(0\right)+u_{1,2}^{\left(2\right)}\left(\tau \right)\delta h_2\left(\tau \right)-u_{1,2}^{\left(2\right)}\left(0\right)\delta h_2\left(0\right)\\ -{\displaystyle {\int }_0^{\tau }\left\{\delta h_2\left(t\right)\frac{d{u}_{1,2}^{\left(2\right)}}{dt}+u_{1,2}^{\left(2\right)}{F}_2\delta h_1\left(t\right)\right\}dt}+u_{1,3}^{\left(2\right)}\left(\tau \right)\delta h_3\left(\tau \right)-u_{1,3}^{\left(2\right)}\left(0\right)\delta h_3\left(0\right)\\ -{\displaystyle {\int }_0^{\tau }\left\{\delta h_3\left(t\right)\frac{d{u}_{1,3}^{\left(2\right)}}{dt}+u_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)\delta h_1\left(t\right)\right\}dt}+u_{2,1}^{\left(2\right)}\left(\tau \right)\delta b_1^{(1)}\left(\tau \right)-u_{2,1}^{\left(2\right)}\left(0\right)\delta b_1^{(1)}\left(0\right)\\ +{\displaystyle {\int }_0^{\tau }\left\{-\delta b_1^{(1)}\left(t\right)\frac{d{u}_{2,1}^{\left(2\right)}}{dt}+u_{2,1}^{\left(2\right)}\right.\left[\delta a_1^{(1)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+\delta b_2^{(1)}\left(t\right)F_2+\delta b_3^{(1)}\left(t\right)F_2{F}_3\right.}\\ \left.\left.+\delta h_2\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\right]dt\right\}+u_{2,2}^{\left(2\right)}\left(\tau \right)\delta b_2^{(1)}\left(\tau \right)-u_{2,2}^{\left(2\right)}\left(0\right)\delta b_2^{(1)}\left(0\right)\\ +{\displaystyle {\int }_0^{\tau }\left\{-\delta b_2^{(1)}\left(t\right)\frac{d{u}_{2,2}^{\left(2\right)}}{dt}+u_{2,2}^{\left(2\right)}\left[\delta b_1^{(1)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+\delta h_1\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\right]\right\}dt}\\ +u_{2,3}^{\left(2\right)}\left(\tau \right)\delta b_3^{(1)}\left(\tau \right)-u_{2,3}^{\left(2\right)}\left(0\right)\delta b_3^{(1)}\left(0\right)-{\displaystyle {\int }_0^{\tau }\delta b_3^{(1)}\left(t\right)\frac{d{u}_{2,3}^{\left(2\right)}}{dt}dt}\\ +2{\phi }_2{\displaystyle {\int }_0^{\tau }\delta h_3\left(t\right)u_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=r^{\left(2\right)},\end{array}$$

  (170)
• The unknown terms on the left side of Equation (170) are eliminated by imposing the following conditions:

  $$u_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

  (171)

  $$u_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{u}_{2,3}^{\left(2\right)}\left(0\right)=0.$$

  (172)
• Using the conditions given in Equations (29), (167), (171) and (172) on the right side of Equation (170) and rearranging the remaining terms yields the following relation:

  $$\begin{array}{l}-u_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0-u_{1,3}^{\left(2\right)}\left(0\right)\delta T_0\\ +{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)\left[-\frac{d{u}_{1,1}^{\left(2\right)}\left(t\right)}{dt}-u_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-u_{1,2}^{\left(2\right)}{F}_2-u_{1,3}^{\left(2\right)}\left(F_2{F}_3\right)+u_{2,2}^{\left(2\right)}{F}_{1}F{b}_1^{(1)}\left(t\right)\right]}\\ +{\displaystyle {\int }_0^{\tau }\delta h_2\left(t\right)\left[-\frac{d{u}_{1,2}^{\left(2\right)}}{dt}-u_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{u}_1^{(1)}\left(t\right)\right]dt}\\ -{\displaystyle {\int }_0^{\tau }\delta h_3\left(t\right)\left[\frac{d{u}_{1,3}^{\left(2\right)}}{dt}+2{\phi }_2{u}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)\right]dt}\\ +{\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)\left[-\frac{d{u}_{2,1}^{\left(2\right)}}{dt}+u_{2,1}^{\left(2\right)}{F}_1{F}_3{h}_2\left(t\right)+u_{2,2}^{\left(2\right)}{F}_1{F}_3{h}_1\left(t\right)\right]}\\ +{\displaystyle {\int }_0^{\tau }\delta b_2^{(1)}\left(t\right)\left[-\frac{d{u}_{2,2}^{\left(2\right)}}{dt}+u_{2,1}^{\left(2\right)}{F}_2\right]dt}+{\displaystyle {\int }_0^{\tau }\delta b_3^{(1)}\left(t\right)\left[-\frac{d{u}_{2,3}^{\left(2\right)}}{dt}+u_{2,1}^{\left(2\right)}{F}_2{F}_3\right]dt}=r^{\left(2\right)},\end{array}$$

  (173)
• The integral terms on the left side of Equation (173) are now required to represent the “indirect-effect” term defined in Equation (163), which is achieved by imposing the following requirements on the components of the second-level adjoint sensitivity function $u^{\left(2\right)}\triangleq {\left[u_1^{\left(2\right)}\left(t\right),u_2^{\left(2\right)}\left(t\right)\right]}^{†}$:

  $$\begin{array}{l}-{\displaystyle \frac{d{u}_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-u_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-u_{1,2}^{\left(2\right)}\left(t\right)F_2-u_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+u_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\\ \hspace{0.17em}=F_3{b}_1^{(1)}\left(t\right)h_2\left(t\right);\end{array}$$

  (174)

  $$-{\displaystyle \frac{d{u}_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-u_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{u}_1^{(1)}\left(t\right)=F_3{b}_1^{(1)}\left(t\right)h_1\left(t\right);$$

  (175)

  $${\displaystyle \frac{d{u}_{1,3}^{\left(2\right)}}{dt}}+2{\phi }_2{u}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)=0;$$

  (176)

  $${\displaystyle \frac{d{u}_{1,3}^{\left(2\right)}}{dt}}+2{\phi }_2{u}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)=0;$$

  (177)

  $$-{\displaystyle \frac{d{u}_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+u_{2,1}^{\left(2\right)}\left(t\right)F_2=0;$$

  (178)

  $$-{\displaystyle \frac{d{u}_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+u_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=0.$$

  (179)

The system of equations comprising Equations (171), (172), (174)–(179) constitutes the “Second-Level Adjoint Sensitivity System (2nd-LASS)” for the second-level adjoint sensitivity function $u^{\left(2\right)}\triangleq {\left[u_1^{\left(2\right)}\left(t\right),u_2^{\left(2\right)}\left(t\right)\right]}^{†}$. Evidently, the 2nd-LASS is linear in $u^{\left(2\right)}\triangleq {\left[u_1^{\left(2\right)}\left(t\right),u_2^{\left(2\right)}\left(t\right)\right]}^{†}$ and is independent of parameter variations. Notably, this system of equations does not need to be solved simultaneously, but can be solved sequentially, by first solving Equations (177)–(179) subject to the initial conditions given in Equation (172) to determine the function $u_2^{\left(2\right)}\left(t\right)$, and subsequently using the function $u_2^{\left(2\right)}\left(t\right)$ in Equations (174)–(176) to solve these equations subject to the “final-time” condition given in Equation (171), to obtain the function $u_1^{\left(2\right)}\left(t\right)$. The 2nd-LASS is to be solved using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity.

6.

Using Equations (174)–(179) in Equation (173) yields the following alternative expression, in terms of $u^{\left(2\right)}\triangleq {\left[u_1^{\left(2\right)}\left(t\right),u_2^{\left(2\right)}\left(t\right)\right]}^{†}$, for the “indirect-effect” term $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}$ defined in Equation (163):

$$\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}=r^{\left(2\right)}+u_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0+u_{1,3}^{\left(2\right)}\left(0\right)\delta T_0.$$

(180)

7.

Using in Equation (180) the definition of the function $r^{\left(2\right)}$ provided in Equation (169) and adding the resulting expression for the indirect-effect term $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}$ to the expression for the direct-effect term $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{dir}$ provided in Equation (162) yields the following expression for the total first-order G-differential $\delta \left[\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right]$:

$$\begin{array}{l}\delta \left[\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right]\triangleq {\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{F}_1}}\delta {\psi }_0+{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{F}_1}}\delta T_0+{\displaystyle \sum _{i=1}^3\frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_i\mathsf{\partial }{F}_1}\delta F_i}\hspace{0.17em},\\ =\left(\delta F_3\right){\displaystyle {\int }_0^{\tau }{h}_1\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)dt}+u_{1,1}^{\left(2\right)}\left(0\right)\delta {\psi }_0+u_{1,3}^{\left(2\right)}\left(0\right)\delta T_0\\ +{\displaystyle {\int }_0^{\tau }{u}_{1,1}^{\left(2\right)}\left(t\right)\left[\left(\delta F_1\right)F_3+F_1\left(\delta F_3\right)\right]h_1\left(t\right)h_2\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{u}_{1,2}^{\left(2\right)}\left(\delta F_2\right)h_1\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\left[\left(\delta F_2\right)F_3+F_2\left(\delta F_3\right)\right]h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\left\{\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_2\left(t\right)b_1^{(1)}\left(t\right)+\left(\delta F_2\right)b_2^{(1)}\left(t\right)\right.}\\ \left.+\left[\left(\delta F_2\right)F_3+\left(\delta F_3\right)F_2\right]b_3^{(1)}\left(t\right)\right\}dt-{\displaystyle {\int }_0^{\tau }{u}_{2,2}^{\left(2\right)}\left[\left(\delta F_1\right)F_3+\left(\delta F_3\right)F_1\right]h_1\left(t\right)b_1^{(1)}\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{u}_{2,3}^{\left(2\right)}\left[\delta {\phi }_1+2\delta {\phi }_2{h}_3\left(t\right)\right]\delta \left(t-\tau \right)dt}.\end{array}$$

(181)

The expression shown in Equation (181) is to be evaluated at the nominal values of all functions and parameters/weights but the superscript “zero” (which has been used to indicate this fact) has been omitted for notational simplicity.

Identifying in Equation (181) the quantities that multiply the respective variations in the initial conditions and feature functions yields the following expressions for the second-order sensitivities stemming from $\delta \left[\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right]$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{F}_1}}=u_{1,1}^{\left(2\right)}\left(0\right);$$

(182)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{F}_1}}=u_{1,3}^{\left(2\right)}\left(0\right);$$

(183)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{F}_1}}=F_3{\displaystyle {\int }_0^{\tau }\left[u_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)-u_{2,1}^{\left(2\right)}\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)-u_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)\right]dt};$$

(184)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{F}_1}}={\displaystyle {\int }_0^{\tau }{u}_{1,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\left[b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt;$$

(185)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{F}_1}}={\displaystyle {\int }_0^{\tau }{h}_1\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)dt}+F_1{\displaystyle {\int }_0^{\tau }{u}_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}\\ +F_2{\displaystyle {\int }_0^{\tau }{u}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{u}_{2,1}^{\left(2\right)}\left(t\right)\left[F_1{h}_2\left(t\right)b_1^{(1)}\left(t\right)+F_2{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt\\ -F_1{\displaystyle {\int }_0^{\tau }{u}_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)dt}.\end{array}.$$

(186)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_1\mathsf{\partial }{F}_1}}=-{\displaystyle {\int }_0^{\tau }{u}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=-u_{2,3}^{\left(2\right)}\left(\tau \right);$$

(187)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_2\mathsf{\partial }{F}_1}}=-2{\displaystyle {\int }_0^{\tau }{u}_{2,3}^{\left(2\right)}\left(t\right)h_3\left(t\right)\delta \left(t-\tau \right)dt}=-2u_{2,3}^{\left(2\right)}\left(\tau \right)h_3\left(\tau \right).$$

(188)

The expressions of the second-order sensitivities represented by Equations (182)–(188) are also to be evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity.

B.

Second-order sensitivities stemming from $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2$ are obtained from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_2\right\}$, which is in turn obtained by using Equation (63), as follows:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_2}}\right\}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[b_2^{(1,0)}\left(t\right)+\epsilon \delta b_2^{(1)}\left(t\right)\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]dt}\right\}}_{\epsilon =0}\\ +{\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[F_3^0+\epsilon \delta F_3\right]\left[b_3^{(1,0)}\left(t\right)+\epsilon \delta b_3^{(1)}\left(t\right)\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]dt}\right\}}_{\epsilon =0}\\ ={\displaystyle {\int }_0^{\tau }\delta b_2^{(1)}\left(t\right)h_1^0\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)b_2^{(1,0)}\left(t\right)dt}+\left(\delta F_3\right){\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)h_1^0\left(t\right)dt}\\ +F_3^0{\displaystyle {\int }_0^{\tau }\delta b_3^{(1)}\left(t\right)h_1^0\left(t\right)dt}+F_3^0{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)b_3^{(1,0)}\left(t\right)dt}\\ =\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{dir}+\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{ind},\end{array}$$

(189)

where the superscript “zero” indicates, as before, that the respective quantities are to be evaluated at their nominal values. The following definitions were used for the “direct-effect” and, respectively, “indirect-effect” terms defined in Equation (189):

$$\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{dir}\triangleq \left(\delta F_3\right){\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)h_1^0\left(t\right)dt},$$

(190)

$$\begin{array}{l}\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{ind}\triangleq {\displaystyle {\int }_0^{\tau }\delta b_2^{(1)}\left(t\right)h_1^0\left(t\right)dt}+{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)b_2^{(1,0)}\left(t\right)dt}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+F_3^0{\displaystyle {\int }_0^{\tau }\delta b_3^{(1)}\left(t\right)h_1^0\left(t\right)dt}+F_3^0{\displaystyle {\int }_0^{\tau }\delta h_1\left(t\right)b_3^{(1,0)}\left(t\right)dt}\hspace{0.17em}.\end{array}$$

(191)

The indirect-effect term $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{ind}$, defined by Equation (191), is evaluated by following the same procedural steps as used above for determining the second-order sensitivities stemming from $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}$. The following expressions are ultimately obtained for the second-order sensitivities stemming from $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_2\right\}$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{F}_2}}=w_{1,1}^{\left(2\right)}\left(0\right);$$

(192)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{F}_2}}=w_{1,3}^{\left(2\right)}\left(0\right);$$

(193)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{F}_2}}=F_3{\displaystyle {\int }_0^{\tau }\left[w_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)-w_{2,1}^{\left(2\right)}\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)-w_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)\right]dt};$$

(194)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{F}_2}}={\displaystyle {\int }_0^{\tau }{w}_{1,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{w}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{w}_{2,1}^{\left(2\right)}\left[b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt;$$

(195)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{F}_2}}={\displaystyle {\int }_0^{\tau }{b}_3^{(1)}\left(t\right)h_1\left(t\right)dt}+F_1{\displaystyle {\int }_0^{\tau }{w}_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}\\ +F_2{\displaystyle {\int }_0^{\tau }{w}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{w}_{2,1}^{\left(2\right)}\left(t\right)\left[F_1{h}_2\left(t\right)b_1^{(1)}\left(t\right)+F_2{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt\\ -F_1{\displaystyle {\int }_0^{\tau }{w}_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)dt}.\end{array}.$$

(196)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_1\mathsf{\partial }{F}_2}}=-{\displaystyle {\int }_0^{\tau }{w}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=-w_{2,3}^{\left(2\right)}\left(\tau \right);$$

(197)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_2\mathsf{\partial }{F}_2}}=-2{\displaystyle {\int }_0^{\tau }{w}_{2,3}^{\left(2\right)}\left(t\right)h_3\left(t\right)\delta \left(t-\tau \right)dt}=-2w_{2,3}^{\left(2\right)}\left(\tau \right)h_3\left(\tau \right).$$

(198)

The second-level adjoint sensitivity function $w^{\left(2\right)}\triangleq {\left[w_1^{\left(2\right)}\left(t\right),w_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, where $w_1^{\left(2\right)}\left(t\right)\triangleq {\left[w_{1,1}^{\left(2\right)}\left(t\right),w_{1,2}^{\left(2\right)}\left(t\right),w_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and $w_2^{\left(2\right)}\left(t\right)\triangleq {\left[w_{2,1}^{\left(2\right)}\left(t\right),w_{2,2}^{\left(2\right)}\left(t\right),w_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$, which appears in the expression provided in Equations (192)–(198), is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS):

$$\begin{array}{l}-{\displaystyle \frac{d{w}_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-w_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-w_{1,2}^{\left(2\right)}\left(t\right)F_2-w_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+w_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\hspace{0.17em};\end{array}$$

(199)

$$-{\displaystyle \frac{d{w}_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-w_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{b}_1^{(1)}\left(t\right)=0;$$

(200)

$${\displaystyle \frac{d{w}_{1,3}^{\left(2\right)}\left(t\right)}{dt}}+2{\phi }_2{w}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)=0;$$

(201)

$$-{\displaystyle \frac{d{w}_{2,1}^{\left(2\right)}\left(t\right)}{dt}}+w_{2,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+w_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)=0;$$

(202)

$$-{\displaystyle \frac{d{w}_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+w_{2,1}^{\left(2\right)}\left(t\right)F_2=h_1\left(t\right);$$

(203)

$$-{\displaystyle \frac{d{w}_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+w_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=F_3{h}_1\left(t\right);$$

(204)

$$w_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{w}_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{w}_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

(205)

$$w_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{w}_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{w}_{2,3}^{\left(2\right)}\left(0\right)=0.$$

(206)

The 2nd-LASS represented by Equations (199)–(206) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. Furthermore, the expressions of the second-order sensitivities represented by Equations (192)–(198) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity.

C.

Second-order sensitivities stemming from $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3$ are obtained from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_3\right\}$ of $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3$, which is obtained using Equation (64) as follows:

$$\begin{array}{l}\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{F}_3}}\right\}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[F_2^0+\epsilon \delta F_2\right]\left[b_3^{(1,0)}\left(t\right)+\epsilon \delta b_3^{(1)}\left(t\right)\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]dt}\right\}}_{\epsilon =0}\\ +{\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[F_1^0+\epsilon \delta F_1\right]\left[b_1^{(1,0)}\left(t\right)+\epsilon \delta b_1^{(1)}\left(t\right)\right]\left[h_2^0\left(t\right)+\epsilon \delta h_2\left(t\right)\right]\left[h_1^0\left(t\right)+\epsilon \delta h_1\left(t\right)\right]dt}\right\}}_{\epsilon =0}\\ =\left(\delta F_2\right){\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)h_1^0\left(t\right)dt}+F_2^0{\displaystyle {\int }_0^{\tau }{h}_1^0\left(t\right)\delta b_3^{(1)}\left(t\right)dt}+F_2^0{\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)\delta h_1\left(t\right)dt}\\ +\left(\delta F_1\right){\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)h_2^0\left(t\right)h_1^0\left(t\right)dt}+F_1^0{\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)h_2^0\left(t\right)h_1^0\left(t\right)dt}\\ +F_1^0{\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)\delta h_2\left(t\right)h_1^0\left(t\right)dt}+F_1^0{\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)h_2^0\left(t\right)\delta h_1\left(t\right)dt}\\ =\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3\right\}}_{dir}+\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3\right\}}_{ind}.\end{array}$$

(207)

The superscript “zero” in Equation (207) indicates, as before, that the respective quantities are to be evaluated at their nominal values. The following definitions were used for the “direct-effect” and, respectively, “indirect-effect” terms in Equation (207):

$$\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3\right\}}_{dir}\triangleq \left(\delta F_1\right){\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)h_1^0\left(t\right)h_2^0\left(t\right)dt}+\left(\delta F_2\right){\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)h_1^0\left(t\right)dt},$$

(208)

$$\begin{array}{l}\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_3\right\}}_{ind}\triangleq F_1^0{\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)h_2^0\left(t\right)h_1^0\left(t\right)dt}+F_1^0{\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)\delta h_2\left(t\right)h_1^0\left(t\right)dt}\\ +F_1^0{\displaystyle {\int }_0^{\tau }{b}_1^{(1,0)}\left(t\right)h_2^0\left(t\right)\delta h_1\left(t\right)dt}+F_2^0{\displaystyle {\int }_0^{\tau }{h}_1^0\left(t\right)\delta b_3^{(1)}\left(t\right)dt}+F_2^0{\displaystyle {\int }_0^{\tau }{b}_3^{(1,0)}\left(t\right)\delta h_1\left(t\right)dt}\hspace{0.17em}.\end{array}$$

(209)

The indirect-effect term defined by Equation (209) is evaluated by following the same procedural steps as previously described when determining the second-order sensitivities stemming from $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_1\right\}}_{ind}$ and from $\delta {\left\{\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{F}_2\right\}}_{ind}$. The following expressions are ultimately obtained for the second-order sensitivities stemming from $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{F}_3\right\}$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{F}_3}}=z_{1,1}^{\left(2\right)}\left(0\right);$$

(210)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{F}_3}}=z_{1,3}^{\left(2\right)}\left(0\right);$$

(211)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{F}_3}}={\displaystyle {\int }_0^{\tau }{b}_1^{(1)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}\\ +F_3{\displaystyle {\int }_0^{\tau }\left[z_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)-z_{2,1}^{\left(2\right)}\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)-z_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)\right]dt}\hspace{0.17em};\end{array}$$

(212)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{F}_3}}={\displaystyle {\int }_0^{\tau }{b}_3^{(1)}\left(t\right)h_1\left(t\right)dt}\\ +{\displaystyle {\int }_0^{\tau }{z}_{1,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{z}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{z}_{2,1}^{\left(2\right)}\left[b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt\end{array}$$

(213)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{F}_3}}=F_1{\displaystyle {\int }_0^{\tau }{z}_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}+F_2{\displaystyle {\int }_0^{\tau }{z}_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{z}_{2,1}^{\left(2\right)}\left(t\right)\left[F_1{h}_2\left(t\right)b_1^{(1)}\left(t\right)+F_2{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt-F_1{\displaystyle {\int }_0^{\tau }{z}_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)dt}.\end{array}$$

(214)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_1\mathsf{\partial }{F}_3}}=-{\displaystyle {\int }_0^{\tau }{z}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=-z_{2,3}^{\left(2\right)}\left(\tau \right);$$

(215)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_2\mathsf{\partial }{F}_3}}=-2{\displaystyle {\int }_0^{\tau }{z}_{2,3}^{\left(2\right)}\left(t\right)h_3\left(t\right)\delta \left(t-\tau \right)dt}=-2z_{2,3}^{\left(2\right)}\left(\tau \right)h_3\left(\tau \right).$$

(216)

The second-level adjoint sensitivity function $z^{\left(2\right)}\triangleq {\left[z_1^{\left(2\right)}\left(t\right),z_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, where $z_1^{\left(2\right)}\left(t\right)\triangleq {\left[z_{1,1}^{\left(2\right)}\left(t\right),z_{1,2}^{\left(2\right)}\left(t\right),z_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and $z_2^{\left(2\right)}\left(t\right)\triangleq {\left[z_{2,1}^{\left(2\right)}\left(t\right),z_{2,2}^{\left(2\right)}\left(t\right),z_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$, which appears in the expressions provided in Equations (210)–(216), is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS):

$$\begin{array}{l}-{\displaystyle \frac{d{z}_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-z_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-z_{1,2}^{\left(2\right)}\left(t\right)F_2-z_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+z_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)\\ =F_2{b}_3^{(1)}\left(t\right)\hspace{0.17em};\end{array}$$

(217)

$$-{\displaystyle \frac{d{z}_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-z_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{b}_1^{(1)}\left(t\right)=F_1{b}_1^{(1)}\left(t\right)h_1\left(t\right);$$

(218)

$${\displaystyle \frac{d{z}_{1,3}^{\left(2\right)}\left(t\right)}{dt}}+2{\phi }_2{z}_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)=0;$$

(219)

$$-{\displaystyle \frac{d{z}_{2,1}^{\left(2\right)}\left(t\right)}{dt}}+z_{2,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+z_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)=F_1{h}_2\left(t\right)h_1\left(t\right);$$

(220)

$$-{\displaystyle \frac{d{z}_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+z_{2,1}^{\left(2\right)}\left(t\right)F_2=0;$$

(221)

$$-{\displaystyle \frac{d{z}_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+z_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=F_2{h}_1\left(t\right);$$

(222)

$$z_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{z}_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{z}_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

(223)

$$z_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{z}_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{z}_{2,3}^{\left(2\right)}\left(0\right)=0.$$

(224)

The 2nd-LASS represented by Equations (217)–(224) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. Furthermore, the expressions of the second-order sensitivities represented by Equations (210)–(216) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity.

4.1.3. Second-Order Sensitivities Stemming from the First-Order Sensitivities with Respect to the Initial Conditions ${\psi }_0$ and $T_0$

A.

Second-order sensitivities stemming from $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{\psi }_0$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{\psi }_0$ are determined from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\psi }_0\right\}$ of $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{\psi }_0$, which is obtained using Equation (60) as follows:

$$\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{\psi }_0}}\right\}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[b_1^{(1,0)}\left(t\right)+\epsilon \delta b_1^{(1)}\left(t\right)\right]\delta \left(t\right)dt}\right\}}_{\epsilon =0}={\displaystyle {\int }_0^{\tau }\delta b_1^{(1)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt}.$$

(225)

The second-order sensitivities stemming from the expression of $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\psi }_0\right\}$ obtained in Equation (225) are determined by following the same procedural steps as described in Section 4.1.2. The following expressions are ultimately obtained for the second-order sensitivities stemming from $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{\psi }_0\right\}$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{\psi }_0}}={\xi }_{1,1}^{\left(2\right)}\left(0\right);$$

(226)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{\psi }_0}}={\xi }_{1,3}^{\left(2\right)}\left(0\right);$$

(227)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{\psi }_0}}=F_3{\displaystyle {\int }_0^{\tau }\left[{\xi }_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)-{\xi }_{2,1}^{\left(2\right)}\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)-{\xi }_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)\right]dt}\hspace{0.17em};$$

(228)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{\psi }_0}}={\displaystyle {\int }_0^{\tau }{\xi }_{1,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{\xi }_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{\xi }_{2,1}^{\left(2\right)}\left[b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt;$$

(229)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{\psi }_0}}=F_1{\displaystyle {\int }_0^{\tau }{\xi }_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}+F_2{\displaystyle {\int }_0^{\tau }{\xi }_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{\xi }_{2,1}^{\left(2\right)}\left(t\right)\left[F_1{h}_2\left(t\right)b_1^{(1)}\left(t\right)+F_2{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt-F_1{\displaystyle {\int }_0^{\tau }{\xi }_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)dt}.\end{array}$$

(230)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_1\mathsf{\partial }{\psi }_0}}=-{\displaystyle {\int }_0^{\tau }{\xi }_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=-{\xi }_{2,3}^{\left(2\right)}\left(\tau \right);$$

(231)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_2\mathsf{\partial }{\psi }_0}}=-2{\displaystyle {\int }_0^{\tau }{\xi }_{2,3}^{\left(2\right)}\left(t\right)h_3\left(t\right)\delta \left(t-\tau \right)dt}=-2{\xi }_{2,3}^{\left(2\right)}\left(\tau \right)h_3\left(\tau \right).$$

(232)

The second-level adjoint sensitivity function ${\xi }^{\left(2\right)}\triangleq {\left[{\xi }_1^{\left(2\right)}\left(t\right),{\xi }_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, where ${\xi }_1^{\left(2\right)}\left(t\right)\triangleq {\left[{\xi }_{1,1}^{\left(2\right)}\left(t\right),{\xi }_{1,2}^{\left(2\right)}\left(t\right),{\xi }_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and ${\xi }_2^{\left(2\right)}\left(t\right)\triangleq {\left[{\xi }_{2,1}^{\left(2\right)}\left(t\right),{\xi }_{2,2}^{\left(2\right)}\left(t\right),{\xi }_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$, which appears in the expressions provided in Equations (226)–(230), is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS):

$$-{\displaystyle \frac{d{\xi }_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-{\xi }_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-{\xi }_{1,2}^{\left(2\right)}\left(t\right)F_2-{\xi }_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+{\xi }_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)=0;$$

(233)

$$-{\displaystyle \frac{d{\xi }_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-{\xi }_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{b}_1^{(1)}\left(t\right)=0;$$

(234)

$${\displaystyle \frac{d{\xi }_{1,3}^{\left(2\right)}\left(t\right)}{dt}}=0;$$

(235)

$$-{\displaystyle \frac{d{\xi }_{2,1}^{\left(2\right)}\left(t\right)}{dt}}+{\xi }_{2,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+{\xi }_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)=\delta \left(t\right);$$

(236)

$$-{\displaystyle \frac{d{\xi }_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+{\xi }_{2,1}^{\left(2\right)}\left(t\right)F_2=0;$$

(237)

$$-{\displaystyle \frac{d{\xi }_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+{\xi }_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=0;$$

(238)

$${\xi }_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\xi }_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\xi }_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

(239)

$${\xi }_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\xi }_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\xi }_{2,3}^{\left(2\right)}\left(0\right)=0.$$

(240)

The 2nd-LASS represented by Equations (233)–(240) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. The expressions of the second-order sensitivities represented by Equations (226)–(230) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity. In particular, Equations (235) and (239) imply that ${\xi }_{1,3}^{\left(2\right)}\left(t\right)\equiv 0$, so that the general expressions shown in Equations (227), (229) and (230) simplify accordingly.

B.

Second-order sensitivities stemming from $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{T}_0$

The second-order sensitivities stemming from the first-order sensitivity $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{T}_0$ are obtained from the G-differential $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{T}_0\right\}$ of $\mathsf{\partial }K\left(\tau \right)/\mathsf{\partial }{T}_0$, which is obtained using Equation (61) as follows:

$$\delta \left\{{\displaystyle \frac{\mathsf{\partial }K\left(\tau \right)}{\mathsf{\partial }{T}_0}}\right\}\triangleq {\left\{{\displaystyle \frac{d}{d\epsilon }}{\displaystyle {\int }_0^{\tau }\left[b_3^{(1,0)}\left(t\right)+\epsilon \delta b_3^{(1)}\left(t\right)\right]\delta \left(t\right)dt}\right\}}_{\epsilon =0}={\displaystyle {\int }_0^{\tau }\delta b_3^{(1)}\left(t\right)\hspace{0.17em}\delta \left(t\right)dt}.$$

(241)

The second-order sensitivities stemming from the expression of $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{T}_0\right\}$ obtained in Equation (225) are determined by following the same procedural steps as described in Section 4.1.2. The following expressions are ultimately obtained for the second-order sensitivities stemming from $\delta \left\{\mathsf{\partial }K\left[T\left(\tau \right)\right]/\mathsf{\partial }{T}_0\right\}$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\psi }_0\mathsf{\partial }{T}_0}}={\zeta }_{1,1}^{\left(2\right)}\left(0\right);$$

(242)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{T}_0\mathsf{\partial }{T}_0}}={\zeta }_{1,3}^{\left(2\right)}\left(0\right);$$

(243)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_1\mathsf{\partial }{T}_0}}=F_3{\displaystyle {\int }_0^{\tau }\left[{\zeta }_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)-{\zeta }_{2,1}^{\left(2\right)}\left(t\right)h_2\left(t\right)b_1^{(1)}\left(t\right)-{\zeta }_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)\right]dt}\hspace{0.17em};$$

(244)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_2\mathsf{\partial }{T}_0}}={\displaystyle {\int }_0^{\tau }{\zeta }_{1,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}+F_3{\displaystyle {\int }_0^{\tau }{\zeta }_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}-{\displaystyle {\int }_0^{\tau }{\zeta }_{2,1}^{\left(2\right)}\left[b_2^{(1)}\left(t\right)+F_3{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt;$$

(245)

$$\begin{array}{l}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{F}_3\mathsf{\partial }{T}_0}}=F_1{\displaystyle {\int }_0^{\tau }{\zeta }_{1,1}^{\left(2\right)}\left(t\right)h_1\left(t\right)h_2\left(t\right)dt}+F_2{\displaystyle {\int }_0^{\tau }{\zeta }_{1,3}^{\left(2\right)}\left(t\right)h_1\left(t\right)dt}\\ -{\displaystyle {\int }_0^{\tau }{\zeta }_{2,1}^{\left(2\right)}\left(t\right)\left[F_1{h}_2\left(t\right)b_1^{(1)}\left(t\right)+F_2{b}_3^{(1)}\left(t\right)\right]}\hspace{0.17em}dt-F_1{\displaystyle {\int }_0^{\tau }{\zeta }_{2,2}^{\left(2\right)}\left(t\right)h_1\left(t\right)b_1^{(1)}\left(t\right)dt}.\end{array}$$

(246)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_1\mathsf{\partial }{T}_0}}=-{\displaystyle {\int }_0^{\tau }{\zeta }_{2,3}^{\left(2\right)}\left(t\right)\delta \left(t-\tau \right)dt}=-{\zeta }_{2,3}^{\left(2\right)}\left(\tau \right);$$

(247)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{\phi }_2\mathsf{\partial }{T}_0}}=-2{\displaystyle {\int }_0^{\tau }{\zeta }_{2,3}^{\left(2\right)}\left(t\right)h_3\left(t\right)\delta \left(t-\tau \right)dt}=-2{\zeta }_{2,3}^{\left(2\right)}\left(\tau \right)h_3\left(\tau \right).$$

(248)

The second-level adjoint sensitivity function ${\zeta }^{\left(2\right)}\triangleq {\left[{\zeta }_1^{\left(2\right)}\left(t\right),{\zeta }_2^{\left(2\right)}\left(t\right)\right]}^{†}\in {\mathrm{H}}_2\left({\Omega }_t\right)$, where ${\zeta }_1^{\left(2\right)}\left(t\right)\triangleq {\left[{\zeta }_{1,1}^{\left(2\right)}\left(t\right),{\zeta }_{1,2}^{\left(2\right)}\left(t\right),{\zeta }_{1,3}^{\left(2\right)}\left(t\right)\right]}^{†}$ and ${\zeta }_2^{\left(2\right)}\left(t\right)\triangleq {\left[{\zeta }_{2,1}^{\left(2\right)}\left(t\right),{\zeta }_{2,2}^{\left(2\right)}\left(t\right),{\zeta }_{2,3}^{\left(2\right)}\left(t\right)\right]}^{†}$, which appears in the expressions provided in Equations (226)–(230), is the solution of the following Second-Level Adjoint Sensitivity System (2nd-LASS):

$$-{\displaystyle \frac{d{\zeta }_{1,1}^{\left(2\right)}\left(t\right)}{dt}}-{\zeta }_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)-{\zeta }_{1,2}^{\left(2\right)}\left(t\right)F_2-{\zeta }_{1,3}^{\left(2\right)}\left(t\right)F_2{F}_3+{\zeta }_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{b}_1^{(1)}\left(t\right)=0;$$

(249)

$$-{\displaystyle \frac{d{\zeta }_{1,2}^{\left(2\right)}\left(t\right)}{dt}}-{\zeta }_{1,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)+F_1{F}_3{b}_1^{(1)}\left(t\right)=0;$$

(250)

$${\displaystyle \frac{d{\zeta }_{1,3}^{\left(2\right)}\left(t\right)}{dt}}=0;$$

(251)

$$-{\displaystyle \frac{d{\zeta }_{2,1}^{\left(2\right)}\left(t\right)}{dt}}+{\zeta }_{2,1}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_2\left(t\right)+{\zeta }_{2,2}^{\left(2\right)}\left(t\right)F_1{F}_3{h}_1\left(t\right)=0;$$

(252)

$$-{\displaystyle \frac{d{\zeta }_{2,2}^{\left(2\right)}\left(t\right)}{dt}}+{\zeta }_{2,1}^{\left(2\right)}\left(t\right)F_2=0;$$

(253)

$$-{\displaystyle \frac{d{\zeta }_{2,3}^{\left(2\right)}\left(t\right)}{dt}}+{\zeta }_{2,1}^{\left(2\right)}\left(t\right)F_2{F}_3=\delta \left(t\right);$$

(254)

$${\zeta }_{1,1}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\zeta }_{1,2}^{\left(2\right)}\left(\tau \right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\zeta }_{1,3}^{\left(2\right)}\left(\tau \right)=0;$$

(255)

$${\zeta }_{2,1}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\zeta }_{2,2}^{\left(2\right)}\left(0\right)=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\zeta }_{2,3}^{\left(2\right)}\left(0\right)=0.$$

(256)

The 2nd-LASS represented by Equations (233)–(240) is to be solved using the nominal values of all required functions and parameters/weights, but the superscript “zero” has been omitted for notational simplicity. The expressions of the second-order sensitivities represented by Equations (226)–(230) are also evaluated using the nominal values of all functions and parameters/weights, but the superscript “zero” has been omitted here, as well, for notational simplicity. In particular, Equations (235) and (239) imply that ${\xi }_{1,3}^{\left(2\right)}\left(t\right)\equiv 0$, so that the general expressions shown in Equations (227), (229) and (230) simplify accordingly.

4.2. Second-Order Sensitivities of the Thermal Conductivity Response $K\left(\tau \right)$ Computed Using the Corresponding Single NODE Equation

Alternatively, the second-order sensitivities of the response $K\left(\tau \right)$ can be computed by differentiating analytically the single-NODE expressions which were obtained in Equation (89) for the first-order sensitivities of $K\left(\tau \right)$. This procedure yields the following expressions for the second-order sensitivities ${\mathsf{\partial }}^{2}K\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$:

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{\phi }_0}}=0;\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{\phi }_1}}={\displaystyle \frac{\mathsf{\partial }T\left(\tau \right)}{\mathsf{\partial }{p}_j}};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{\phi }_2}}=2T\left(\tau \right){\displaystyle \frac{\mathsf{\partial }T\left(\tau \right)}{\mathsf{\partial }{p}_j}};j=1,\mathrm{\dots },7$$

(257)

$${\displaystyle \frac{{\mathsf{\partial }}^{2}K\left(\tau \right)}{\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i}}={\displaystyle \frac{\mathsf{\partial }{\phi }_0}{\mathsf{\partial }{p}_j}}+{\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }{p}_j}}\left\{\left[{\phi }_1+2{\phi }_{2}T\left(\tau \right)\right]{\displaystyle \frac{\mathsf{\partial }T\left(\tau \right)}{\mathsf{\partial }{p}_i}}\right\};\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i,j=1,\mathrm{\dots },7.$$

(258)

For evaluating the expressions provided in Equations (257) and (258), the following expressions, previously obtained, are to be used: (i) the expressions for the first-order sensitivities $\mathsf{\partial }T\left(\tau \right)/\mathsf{\partial }{p}_i$, $i=1,\mathrm{\dots },7$, provided in Equation (86); and (ii) the expressions for the second-order sensitivities ${\mathsf{\partial }}^{2}T\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$, provided in Equation (141).

4.3. Computing the Second-Order Sensitivities of the Thermal Conductivity Response $K\left(\tau \right)$: Using the Coupled NODE Versus the Single NODE Equations

Computing all of the second-order sensitivities ${\mathsf{\partial }}^{2}K\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$ using the coupled NODE equations requires solving 39 differential equations (the total number of differential equations included in eight 2nd-LASSs), as follows:

(i)

3 × 3 = 9 differential equations for obtaining the second-order sensitivities of $K\left(\tau \right)$ with respect to the decoder weights ${\phi }_0$, ${\phi }_1$ and ${\phi }_2$.

(ii)

3 × 6 = 18 differential equations for obtaining the second-order sensitivities of $K\left(\tau \right)$ with respect to the feature functions $F_1$, $F_2$ and $F_3$.

(iii)

2 × 6 = 12 differential equations for obtaining the second-order sensitivities of $K\left(\tau \right)$ with respect to the initial conditions ${\psi }_0$ and $T_0$

Notably, the same operator appears on the left side of each of the eight 2nd-LASSs, i.e., all of these 2nd-LASSs have the following operator form: $A{y}_m=q_m$, $m=1,\mathrm{\dots },8$. Only the source terms, symbolically represented by $q_m$, which appear on the right sides of these eight 2nd-LASSs, differ from each other. Therefore, if the inverse operator $A^{-1}$ could be stored, then the respective coupled differential equations could be easily solved to obtain $y_m=A^{-1}{q}_m$. Storing the operator $A^{-1}$ may be feasible for small problems/systems but becomes impractical for large systems.

The mixed sensitivities ${\mathsf{\partial }}^{2}K\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$ are computed twice, using distinct second-order adjoint functions, which provides a stringent test for verifying the accuracy of the computations needed for solving the 2nd-LASS involved in determining the respective second-order adjoint functions.

On the other hand, computing all of the second-order sensitivities ${\mathsf{\partial }}^{2}K\left(\tau \right)/\mathsf{\partial }{p}_j\mathsf{\partial }{p}_i$, $i,j=1,\mathrm{\dots },7$ using the single-NODE equations requires solving just 2 × 2 = 4 differential equations, namely:

(i)

solving Equations (126) and (127) for obtaining the second-level adjoint sensitivity function $e_1^{(2)}\left(t\right)\triangleq {\left[e_{11}^{(2)}\left(t\right),e_{12}^{(2)}\left(t\right)\right]}^{†}$, which is required for computing ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_1$ and ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_1\mathsf{\partial }{\mathrm{\Phi }}_2$; and

(ii)

solving Equations (135) and (136) for obtaining the second-level adjoint sensitivity function $e_2^{(2)}\left(t\right)\triangleq {\left[e_{21}^{(2)}\left(t\right),e_{22}^{(2)}\left(t\right)\right]}^{†}$, which is required for computing ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_1$ and ${\mathsf{\partial }}^{2}E\left(\tau \right)/\mathsf{\partial }{\mathrm{\Phi }}_2\mathsf{\partial }{\mathrm{\Phi }}_2$.

The above considerations clearly highlight the advantages of decoupling the original NODE system whenever possible, to obtain and solve equations for single dependent variables, such as $E\left(t\right)$, rather than solving the entire system of coupled equations simultaneously.

5. Discussion and Conclusions

A new sensitivity analysis methodology, called the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations” and abbreviated as “2nd-FASAM-NODE” has been developed in [1]. This work has illustrated the application of the 2nd-FASAM-NODE to a paradigm benchmark model, called the Nordheim–Fuchs phenomenological model for reactor safety [2,3]. This phenomenological model describes a short-time self-limiting power transient in a nuclear reactor system having a negative temperature coefficient in which a large amount of reactivity is suddenly inserted, either intentionally or by accident. The Nordheim–Fuchs model responses analyzed in this work included the model’s state functions/variables (namely: the time-dependent total energy released per cm³, the reactor’s time-dependent temperature and the reactor’s time-dependent neutron flux), and the reactor’s time-dependent thermal conductivity, which is a representative model response involving decoder weights. All of the parameters underlying the Nordheim–Fuchs phenomenological model are subject to uncertainties.

“Large-scale” computations are those needed to solve systems of equations (algebraic, differential, integral) such as those underlying the original model and the adjoint sensitivity systems of various levels (1st-LASS, 2nd-LASS, etc.). By comparison, the evaluations of integrals, such as those expressing then various sensitivities, by means of quadrature formulas, are “small-scale” computations. For large-scale systems involving many parameters, the conventional (e.g., “statistical” or “finite-difference”) methods are impractical for computing response sensitivities higher than first-order. The 2nd-FASAM-NODE methodology provides the most efficient means of computing the exact expressions of the second-order sensitivities of a model-decoder response with respect to the underlying model parameters and initial conditions, requiring at most as many large-scale computations as there are features/functions of model parameters. Importantly, the mixed second-order sensitivities are computed twice, using distinct adjoint functions, thus providing an intrinsic mechanism for verifying the accuracy of the respective first- and second-level adjoint functions. Ongoing research aims at developing similarly efficient methodologies for performing high-order sensitivity analysis of integral and integro-differential equation neural networks.