A Symbolic Algorithm for Checking the Identifiability of a Time-Series Model

Abstract

Several authors have attempted to compute the asymptotic Fisher information matrix for a univariate or multivariate time-series model to check for its identifiability. This has the form of a contour integral of a matrix of rational functions. A recent paper has proposed a short Wolfram Mathematica notebook for VARMAX models that makes use of symbolic integration. It cannot be used in open-source symbolic computation software like GNU Octave and GNU Maxima. It was based on symbolic integration but the integrand lacked symmetry characteristics in the appearance of polynomial roots smaller or greater than 1 in modulus. A more symmetric form of the integrand is proposed for VARMA models that first allows a simpler approach to symbolic integration. Second, the computation of the integral through Cauchy residues is also possible. Third, an old numerical algorithm by Söderström is used symbolically. These three approaches are investigated and compared on a pair of examples, not only for the Wolfram Language in Mathematica but also for GNU Octave and GNU Maxima. As a consequence, there are now sufficient conditions for exact model identifiability with fast procedures.

1. Introduction

The asymptotic Fisher information matrix (AFIM) plays an important role in statistical model estimation since inverting it provides the asymptotic covariance matrix of the maximum likelihood estimators. Model identifiability is of major importance since the parameters in an unidentifiable model cannot be estimated correctly. In particular, in univariate time-series modeling, autoregressive moving average (ARMA) models are not identifiable if there is at least one common root to the autoregressive (AR) and the moving-average (MA) polynomials. The situation is more complex in multivariate time-series modeling, for vector ARMA (VARMA) models [1], but the absence of a common eigenvalue to the AR and MA matrix polynomials is a sufficient condition for identifiability. There are also time-series models with exogenous variables, like transfer function models, single-input single-output (SISO) models, ARMAX, and VARMAX models. For the identifiability of VARMAX models, see [2] or [3].

Many authors have proposed methods or computational algorithms for obtaining the Fisher information matrix for various univariate or multivariate time-series models that are assumed to be completely specified. Here are a few major references:

• For ARMA models: [4,5,6,7,8];
• For seasonal ARMA models (with 4 polynomials): [9,10,11];
• For ARMAX models (with 3 polynomials): [12,13];
• For SISO models (with 6 polynomials) or seasonal SISO models (with 12 polynomials): [14,15,16];
• For VARMA models: [17,18,19,20];
• For VARMAX models: [21,22,23,24,25,26].

Most of these algorithms are long and complex and practically all make use of floating-point calculations; hence, they do not deliver exact results. Several of them, like [7,22], or [25], are about a so-called exact form of the information matrix that is not considered here since I focus on the AFIM, as defined in Section 2.

Ref. [26] has proposed a short Wolfram Mathematica notebook that gives the AFIM of any ARMA, ARMAX, VARMA, or VARMAX model. Written in the Wolfram Language, it is based on formulas generalizing those in [27]. Moreover, because it makes use of a symbolic integration of matrices, it is exact, provided that all coefficients are expressed either as fractions or symbols, not as decimal numbers. The inconvenience is that it is very slow. We also have noticed that it is not possible to port the algorithm to open-source symbolic computation software packages like GNU Octave and GNU Maxima because symbolic integration gives a wrong result therein. In this paper, I did not consider R but it should be possible, as is discussed in the conclusions.

The purpose of this article is, therefore, to propose alternative procedures for symbolic computation software packages that are faster and more reliable. For that reason, I first change the original algorithm and make it more symmetric by separating polynomials with roots inside and outside the unit circle of the complex plane or Gauss plane, also called the Argand diagram, e.g., [28], because it is essential for several of the procedures. For Wolfram Mathematica, I benefit from a recently introduced procedure for computing sums of Cauchy residues. It is sometimes faster but ultimately based on solving polynomial equations, as will be discussed later. In GNU Maxima, there is a less advanced procedure for Cauchy residues. Finally, I write for GNU Octave a symbolic procedure based on an old algorithm by Söderström [29], aimed at the numerical contour integration of rational functions. Henceforth, I speak of Mathematica, Maxima, and Octave, respectively, to mean the Wolfram Languages used in Mathematica, GNU Maxima, and GNU Octave.

In Section 2, I summarize the method in [26] in the case of an ARMA or VARMA model. A review of VARMA models is available [30]. I also mention ARMAX or VARMAX models for completeness but, since these models simply require an additional integration, it will not be necessary to detail the changes. In Section 3, I present the new algorithms for Mathematica, Maxima, and Octave. I will compare the results in Section 4 and conclude the article in Section 5. Supplementary Materials are indicated as S1, S2, etc.

2. Materials and Methods

In the whole paper, $I_n$ is the identity matrix of order $n^T$; and * denote transposition and complex conjugate, respectively; ⊗ is the Kronecker product; and vec is the column vector obtained by stacking the columns of a matrix.

Let $\{y_t;t=1,\dots ,T\}$ be an n-dimensional time series, the observed output, and $\{x_t;t=1,\dots ,T\}$ be an m-dimensional time series, the observed input. Let $\{{\epsilon }_t;t\in \mathbb{Z}\}$ be an n-dimensional sequence of independent and identically distributed random vectors with mean 0 and a positive definite covariance matrix $\Sigma$, the unobserved errors. Let $A_j,B_j\in {\mathbb{R}}^{n\times n}$, $C_j$$\in {\mathbb{R}}^{n\times m}$ be the parameter matrices, with  $A_0\equiv B_0\equiv I_n$. A vector autoregressive-moving-average model with an explanatory variable (VARMAX) is defined by the equation

$${\displaystyle \sum _{j=0}^p}{A}_j{y}_{t-j}={\displaystyle \sum _{j=0}^{r-1}}{C}_j{x}_{t-j}+{\displaystyle \sum _{j=0}^q}{B}_j{\epsilon }_{t-j},t\in \mathbb{Z}.$$

(1)

When $n=m=1$, we have an ARMAX model. When $m=0$, there is no explanatory variable and we have a VARMA model—or an ARMA model if $n=1$ also.

The $\ell \times 1$ parameter vector is $\vartheta =\mathrm{vec}(A,B,C)$, where $A=\mathrm{vec}(A_1,A_2,\dots ,A_p)$, $B=\mathrm{vec}(B_1,B_2,\dots ,B_q)$, $C=\mathrm{vec}(C_0,C_1,C_2,\dots ,C_{r-1})$. Let the matrix polynomials $A\left(z\right)={\sum }_{j=0}^p{A}_j{z}^j$, $B\left(z\right)={\sum }_{j=0}^q{B}_j{z}^j$, and $C\left(z\right)={\sum }_{j=0}^{r-1}{C}_j{z}^j$. Here, z denotes a complex variable. The equations $\det \left(A\right(z\left)\right)=0$ and $\det \left(B\right(z\left)\right)=0$ should have their roots, the eigenvalues of the matrix polynomials, outside the unit circle. Let ${\epsilon }_t\left(\vartheta \right)$ be the residual of (1) at time t obtained recursively. We consider the Gaussian likelihood, i.e., we proceed as if ${\epsilon }_t$ were normally distributed—hence, as if the process was Gaussian. This is known as working with a Gaussian quasi-likelihood.

Remark 1.

The Gaussian quasi-likelihood estimators are consistent and asymptotically normal under rather general conditions (including the existence of second-order moments) but they are not necessarily efficient [31]. It is well-known that the least-squares estimation method (also called the conditional estimation method) is an approximation of the Gaussian quasi-likelihood method, the difference being in the treatment of the first observations. The AFIM I consider is thus based on a Gaussian assumption but it does not prevent ${\epsilon }_t$ from having another distribution. For instance, if we generate univariate time series using another distribution, like a logistic or a Laplace (also called double-exponential) distribution, but we use the Gaussian likelihood or the least-squares estimation methods, the standard errors obtained using the AFIM will be the same and will be correct (meaning that they will equate the standard deviation of the estimates).

The asymptotic Fisher information matrix (AFIM) at $\vartheta$ is defined by

$$\mathcal{F}\left(\vartheta \right)=\mathbb{E}\left\{{\left({\displaystyle \frac{\partial {\epsilon }_t\left(\vartheta \right)}{\partial {\vartheta }^T}}\right)}^T{\Sigma }^{-1}\left({\displaystyle \frac{\partial {\epsilon }_t\left(\vartheta \right)}{\partial {\vartheta }^T}}\right)\right\}.$$

(2)

Generalizing [27] (with $m=n$, itself a huge improvement w.r.t. [32]) like in [26] and then particularizing to the case where $m=0$, using the notations $u_k\left(z\right)={(1,z,z^2,\cdots ,z^{k-1})}^T$, for $k\ge 1$, and 

$$\begin{array}{ccc}\hfill \mathcal{G}\left(z\right)& =& \left(\begin{array}{c}{u}_p\left(z\right)\otimes A^{-1}\left(z\right)(-B\left(z\right))\\ u_q\left(z\right)\otimes I_n\end{array}\right),\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill \sigma \left(z\right)& =& B^{-1T}\left(z\right){\Sigma }^{-1}{B}^{-1}\left(z^{-1}\right),\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill \mathcal{P}\left(z\right)& =& \mathcal{G}\left(z\right)\Sigma {\mathcal{G}}^{*}\left(z\right),\hfill \end{array}$$

(5)

then the AFIM can be computed as

$$\mathcal{F}\left(\vartheta \right)={\displaystyle \frac{1}{2\pi \mathrm{i}}}{\oint }_{\left|z\right|=1}(\mathcal{P}\left(z\right)\otimes \sigma \left(z\right)){\displaystyle \frac{\mathrm{d}z}{z}}.$$

(6)

Here, $\mathrm{i}$ is the imaginary unit and the symbol ∮ represents a contour integral around the unit circle of the complex plane, i.e., $\left|z\right|=1$, traversed counterclockwise, e.g., [28]. Making the substitution $z=\exp \left(\mathrm{i}f\right)$, where the angle f varies between 0 and $2\pi$, the contour integral can be replaced by a trigonometric integral since $\exp \left(\mathrm{i}f\right)=sin\left(f\right)+\mathrm{i}cos\left(f\right)$. This is seen in Example 1.

If $m>0$, a second integral is added to the right-hand side but the treatment is similar to what is performed on the shown integral, so I omit this, take $m=0$, and stay with ARMA (when $n=1$) and VARMA (when $n>1$) models in this paper. See [26] for more details and illustrations on ARMAX and VARMAX models.

Example 1.

If $n=1$, $p=1$, $q=0$, we have the autoregressive model of order 1, or AR(1) model, defined by

$$y_t+A{y}_{t-1}={\epsilon }_t,t\in \mathbb{Z},$$

(7)

where $\left|A\right|<1$. The AFIM is equal to ${(1-A^2)}^{-1}$. Note that it can be obtained as the integral

$$\begin{array}{ccc}& & {\displaystyle \frac{1}{2\pi \mathrm{i}}}{\oint }_{\left|z\right|=1}{\displaystyle \frac{1}{(1+Az)(1+A{z}^{-1})}}{\displaystyle \frac{\mathrm{d}z}{z}}\hfill \\ & =& {\displaystyle \frac{1}{2\pi }}{\int }_0^{2\pi }{\displaystyle \frac{1}{(1+A\exp (\mathrm{i}f\left)\right)(1+A\exp (-\mathrm{i}f\left)\right)}}\mathrm{d}f.\hfill \end{array}$$

The Fisher information for a series of length T is T times the AFIM: $T{(1-A^2)}^{-1}$, so the standard error of the (maximum likelihood) estimator of A is

$${\displaystyle \frac{1}{\sqrt{T}}}\sqrt{1-A^2}.$$

(8)

There is no problem of identifiability for the AR(1) model in Example 1, contrary to the ARMA(1, 1) model in the following example.

Example 2.

The ARMA(1,1) model corresponds to the selection $n=1$, $p=1$, $q=1$. It is defined by the equation

$$y_t+A{y}_{t-1}={\epsilon }_t+B{\epsilon }_{t-1}$$

(9)

Then, the Fisher information for a series of length T is

$$T\left(\begin{array}{cc}{(1-A^2)}^{-1}& -{(1-AB)}^{-1}\\ -{(1-AB)}^{-1}& {(1-B^2)}^{-1}\end{array}\right),$$

(10)

so that it becomes singular if (and only if) $A=B$. Indeed, using a lag operator L such that $L{y}_t=y_{t-1}$, the model equation becomes $(1+AL)y_t=(1+BL){\epsilon }_t$ and it simplifies to $y_t={\epsilon }_t$ if and only if $A=B$. In the latter case, there is thus an identifiability problem and it is not possible to estimate either A or B. As suggested by a reviewer, if the data-generating process is such that $A=B$ and a series is generated using it, the estimation algorithm will not converge well, and the standard errors of the estimates of A and B will often be so large that the estimates are not significant, pointing out clearly that at least one of the parameters should be deleted.

In the Introduction, I wrote that [26] (Figure 1) has proposed a short Mathematica notebook that gives the AFIM of any ARMA, ARMAX, VARMA, or VARMAX model. We gave it there for a VARMAX model. For a VARMA model, it is still shorter (see Figure 1).

As mentioned in the conclusions of [26] (p. 12),

The open-source packages Octave and Maxima should be able to do the job since their symbolic integration procedure can work on matrices of functions, but they do not work presently. More developments would be needed to circumvent their failing integration.

• I tried with more recent versions without much success. Therefore, I had no other choice than to search for alternative approaches.

Meanwhile, it appeared that Mathematica version 13.3 has a new experimental procedure called ResidueSum for evaluating the integral of a rational function, based on Cauchy’s residue Theorem. The open-source symbolic computation software Maxima 5.47.0 offers a function called residue to evaluate a Cauchy residue provided that the poles within the unit circle are specified. The open-source scientific programming language Octave 8.1 is largely compatible with Mathworks Matlab and possesses a function residue but is numerical.

In several papers [8,9], we have used an algorithm [29] from Söderström in a numerical way. That algorithm is aimed at computing a circular integral of a rational function along the unit circle of the complex plane. That rational function has the form $\mathcal{B}\left(z\right)/\left\{\mathcal{A}\right(z\left)\mathcal{C}\right(z\left)\right\}$, where $\mathcal{A}\left(z\right)$ and $\mathcal{C}\left(z\right)$ are polynomials having their roots strictly inside and strictly outside, respectively, of the unit circle, and $\mathcal{B}\left(z\right)$ is a polynomial. The algorithm proceeds by using recurrences where the degrees of the polynomials are reduced one by one in turn until the integrand becomes a constant. Consequently, no integration is performed; computations are only performed on the coefficients of the polynomials.

3. Results

As a consequence of the arguments in Section 2, this article presents different solutions to compute the Fisher information matrix of ARMA and VARMA models for Mathematica, Maxima, and Octave. I first propose a symmetric alternative to the algorithm in [26] that is necessary for most of the approaches. Then, I use Cauchy’s residues for the three packages Mathematica, Maxima, and Octave. Finally, to circumvent the fact that Octave’s residue is numerical, I use a symbolic version of Söderström’s algorithm.

I illustrate the different methods in two examples.

Example 3.

I take the ARMA(2,2) model extracted from the ARMAX model used in [26] and first in [12]. It is based on the model defined by the equation $y_t-1.5y_{t-1}+0.7y_{t-2}={\epsilon }_t-{\epsilon }_{t-1}+0.2{\epsilon }_{t-2}$, so $A\left(z\right)=1-1.5z+0.7z^2$ and $B\left(z\right)=1-z+0.2z^2$. I take $\Sigma =1$. The model has $\ell =4$ parameters.

Example 4.

I take the VARMA(1,1) model treated by [33] extracted from the VARMAX model used in [18]. It is based on the model equation $y_t+A_1{y}_{t-1}={\epsilon }_t+B_1{\epsilon }_{t-1}$, with 

$$A_1=\left(\begin{array}{cc}0.6& 0.2\\ 0.4& -0.6\end{array}\right),B_1=\left(\begin{array}{cc}0.5& -0.76\\ 0.25& -0.5\end{array}\right).$$

I take $\Sigma =I_2$. There are $\ell =8$ parameters.

3.1. An Alternative Integral Representation

The representation of the Fisher information matrix presented in (3)–(6) is nice but has the inconvenience of having polynomials in z and $z^{-1}$ at several places. This is not convenient, neither for the use of Söderström’s algorithm in [29] nor for computing residues. Thus, I investigated an alternative integral representation of (6) that would be more symmetric in z and $z^{-1}$. Instead of integrating the matrix $\left(\mathcal{P}\right(z)\otimes \sigma (z\left)\right)/z$ with $\mathcal{P}\left(z\right)=\mathcal{G}\left(z\right)\Sigma {\mathcal{G}}^{*}\left(z\right)$ and $\sigma \left(z\right)=B^{-1T}\left(z\right){\Sigma }^{-1}{B}^{-1}\left(z^{-1}\right)$, I compute ${\Sigma }^{1/2}$, the positive definite square root of $\Sigma$, define $\mathcal{H}\left(z\right)=\left(\mathcal{G}\left(z\right){\Sigma }^{1/2}\right)\otimes \left(B^{-1T}\left(z\right){\Sigma }^{-1/2}\right)$, and exploit the identity

$$(\mathcal{G}\left(z\right)\Sigma {\mathcal{G}}^{*}\left(z\right))\otimes B^{-1T}\left(z\right){\Sigma }^{-1}{B}^{-1}\left(z^{-1}\right)=\mathcal{H}\left(z\right){\mathcal{H}}^T\left(z^{-1}\right).$$

Hence, (6) can be equivalently obtained by

$$\mathcal{F}\left(\vartheta \right)={\displaystyle \frac{1}{2\pi \mathrm{i}}}{\oint }_{\left|z\right|=1}\left(\mathcal{H}\left(z\right){\mathcal{H}}^T\left(z^{-1}\right)\right){\displaystyle \frac{\mathrm{d}z}{z}},$$

(11)

where $\mathcal{H}\left(z\right)$ is a block matrix with entries being rational functions in z while ${\mathcal{H}}^T\left(z^{-1}\right)$ is a block matrix with entries being rational functions in $z^{-1}$. Moreover, by construction, the entries of $\mathcal{H}\left(z\right)$ have a denominator with roots outside the unit circle whereas those of ${\mathcal{H}}^T\left(z^{-1}\right)$ are inside the unit circle. Consequently, we are able to compute Cauchy’s residues or to use that alternative representation for the algorithm of Söderström (1984) for a contour integral of a rational function.

To illustrate that alternative representation, let us consider a univariate model, so $n=1$. Then, $\mathcal{H}\left(z\right)$ is an $\ell \times 1$ vector and ${\mathcal{H}}^T\left(z^{-1}\right)$ is a $1\times \ell$ vector. We perform a simplification of the rational functions and isolate the numerators and denominators. Then, the entry $(j_1,j_2)$, $j_1,j_2=1,\dots ,\ell$ of the product has as numerator $\mathrm{num}\left({\mathcal{H}}_{j1}\left(z\right){\mathcal{H}}_{j2}^T\left(z^{-1}\right)\right)/z$ and as denominator $\mathrm{den}\left({\mathcal{H}}_{j1}\left(z\right)\right)\mathrm{den}\left({\mathcal{H}}_{j2}^T\left(z^{-1}\right)\right))$. Now, we can use either the residues at the roots of $\mathrm{den}\left({\mathcal{H}}_{j2}^T\left(z^{-1}\right)\right)$ or Söderström’s algorithm with $\mathcal{A}\left(z\right)=\mathrm{den}\left({\mathcal{H}}_{j2}^T\left(z^{-1}\right)\right)$, $\mathcal{B}\left(z\right)=\mathrm{num}\left({\mathcal{H}}_{j1}\left(z\right){\mathcal{H}}_{j2}^T\left(z^{-1}\right)\right)/z$, and $\mathcal{C}\left(z\right)=\mathrm{den}\left({\mathcal{H}}_{j1}\left(z\right)\right)$.

3.2. Methods Using Cauchy’s Residues

It is well known that a contour integral of a meromorphic function can be evaluated as a sum of residues by using Cauchy’s residue theorem. We can use it to evaluate the integral in (6) or in (11). The sum includes only the residues that correspond to a pole inside the contour. Since version 13.3, Mathematica has a new experimental function ResidueSum for evaluating such an integral, based on Cauchy’s residue theorem.

A double loop over the rows and columns of the information matrix is needed because, presently, ResidueSum is unable to work on a matrix. The advantage is, however, that the double loop around ResidueSum can take care of the symmetry property of the information matrix, contrary to Integrate, as supposed (although I do not know the function’s internals). That double loop wis also necessary with the other packages. It appears that the code using ResidueSum is much faster than Integrate (see below). There is, however, a serious restriction on the size of the model, i.e., p, q, and r. For instance, $p=4$, $q=4$, and $r=1$ or $p=2$, $q=2$, and $r=2$ would work but not, in general, $p=5$, $q=0$, and $r=1$ or $p=0$, $q=3$, and $r=2$ because it would involve polynomials of degrees 5 or more for which there is no algebraic solution using radicals, by Abel–Ruffini theorem. This restriction is, of course, valid for the other packages using residues.

Both Octave and Maxima have a function residue to compute these residues. Maxima’s residue function is symbolic (provided that the poles can be computed symbolically). Note that the computation of these poles in Maxima should be carried out before calling the function, although this is not required by Mathematica. This implies using the solve function and managing the poles for their use in the call to residue.

Contrary to Maxima, Octave’s residue function is numerical, not symbolic. For Maxima and Octave, some care is needed in the case of multiplicity because only the terms corresponding to the first pole need to be used. This is performed for Octave in a function called residuesum.m, available in Supplementary Material S1.

3.3. An Algorithm for Integrating Rational Functions Around the Unit Circle

Ref. [29] is a working paper that describes an algorithm integrating rational functions around the unit circle. It is said that the algorithm is inspired by [34], whose aim was to evaluate an integral of a rational function along the whole imaginary axis. The working paper also contains an implementation: a Fortran function called CINT. In several papers [8,9], that algorithm was used in a numerical way. Here, I propose a symbolic variation programmed in Matlab/Octave.

Let $\mathcal{A}\left(z\right)$, $\mathcal{B}\left(z\right)$, and $\mathcal{C}\left(z\right)$ be polynomials with respective degrees $n_a$, $n_b$, and $n_c$. We suppose the following:

• $\mathcal{A}\left(z\right)$ has all its zeros strictly inside the unit circle; and
• $\mathcal{C}\left(z\right)$ has all its zeros strictly outside the unit circle;

whereas there is no restriction on $\mathcal{B}\left(z\right)$. We need to compute the contour integral

$${\oint }_{\left|z\right|=1}{\displaystyle \frac{\mathcal{B}\left(z\right)}{\mathcal{A}\left(z\right)\mathcal{C}\left(z\right)}}{\displaystyle \frac{\mathrm{d}z}{z}}.$$

The algorithm has a finite number of steps with linearly transformed polynomials while keeping the integral’s value. At each step, one of the polynomials has a smaller degree, i.e., with decreasing $n_a$, $n_b$, and $n_c$. At the end, $n_a=0$ or $n_c=0$, and the calculation is elementary.

Since the symbolic Matlab/Octave program and its procedures are too long to be included in these pages, they are available as Supplementary Materials. This is carried out for Octave in a function called circintrat.m, available as Supplementary Material S2. Note that the original numerical Fortran code is still there but presented as comments.

3.4. A New Algorithm for Mathematica

In [26], we used the symbolic integration in (6) not as a circular integral but after a change of variable $z\mapsto \exp \left(\mathrm{i}f\right)$, i.e., using a definite trigonometric integral over $[0,2\pi ]$. We showed a figure with the most important part of the Mathematica notebook. Figure 2, part (a), shows the part of the notebook using my alternative representation of Section 3.1 with the computation of the contour integral directly, without a change of variable.

Moreover, the new ResidueSum function of Mathematica version 13.3 for evaluating the integral of a rational function, based on Cauchy’s residue theorem, is also added in Figure 2, part (b). The whole notebooks are available as Supplementary Materials S3 and S4 for Examples 3 and 4, respectively, with their output in Supplementary Materials S5 and S6, respectively. There was no reason to adapt the algorithm in [29] to Mathematica since we already have several computationally efficient solutions.

Let us now consider the application in the two examples.

Example 5.

For the univariate ARMA(2,2) model with four parameters, with each of the three methods, the one in [26], the variant in Section 3.1, or the experimental ResidueSum function, we have the same results:

 or 

The determinant of the exact AFIM is given by the fraction $6103515625/1277343936$. The model is therefore identifiable.

Example 6.

For the VARMA(1,1) model with eight parameters, using either of the three methods, we obtain the same matrix of rational numbers shown in Figure 3. The matrix of numerical values is also shown. The smallest eigenvalue of the exact AFIM is $0.00670618$; hence, the model is identifiable. Note that, like in [26] (Example 1 and Figure 5), it would be possible to have a very small number of symbolic parameters (such as a or b) instead of fractions (like $0.6=6/10$) for the coefficients, but the resulting output would be expressions using these literal parameters.

3.5. An Algorithm for Maxima

As explained in Section 2, although Maxima is, in principle, able to symbolically evaluate an integral of a matrix, it failed to provide the correct results. Hopefully, I found a residue function, but it is essential to describe the condition on the roots. The function does not work on a matrix, so we have to perform a double loop on the rows and columns (taking into account the symmetry of the matrix, of course) and symbolically evaluate the polynomials involved in each entry at the numerator and at the denominator. That required using the alternative integral representation in Section 3.1 because the asymmetric original representation used in [26] makes it difficult to separate the roots with respect to the unit circle. For the two examples, the results obtained with Maxima are exactly the same as with Mathematica. I have used wxMaxima 23.05.1. I discuss the practical aspects and the timings of the examples in Section 4. The whole Maxima workspaces are available as Supplementary Materials S7 and S8 for Examples 3 and 4, respectively, with their output in Supplementary Materials S9 and S10, respectively.

3.6. An Algorithm for Octave

Octave is a scientific programming language that is largely compatible with Matlab but it is open-source. Like Matlab, it contains a symbolic computation toolbox. It is based on Python SymPy. As for Maxima, Octave is, in principle, able to symbolically evaluate an integral of a matrix but it failed to provide the correct results.

Like Maxima, there is also a residue function but it is numerical, not symbolic, and, hence, unable to provide the same exact results as Mathematica and Maxima. To have an exact expression for the Fisher information matrix, I considered Söderström’s algorithm (1984) described in Section 3.3 and ported to the Matlab/Octave language. As for Maxima, it is not possible to implement either residuesum.m or circintrat.m straightforwardly on matrices: we have to perform a double or a triple loop on the rows and the columns and a summation index in order to prepare the polynomials to be used. Here, the alternative integral representation of Section 3.1 is again essential. The whole scripts are available as Supplementary Materials S11 and S12 for Examples 3 and 4, respectively, with their output in Supplementary Materials S13 and S14, respectively.

In the two examples, the results are the same as for Mathematica, under the form of fractions for circintrat.m or decimal numbers for residuesum.m. I have used Octave 8.1. Note that there is a pending problem for treating the examples in version 9.2. I again discuss the practical aspects and the timings in Section 4.

3.7. Application on Examples 3 and 4

The difference between Mathematica, Maxima, and Octave is in the computation time of the different methods. This is discussed in Section 4.

4. Discussion

In Section 3.4, Section 3.5 and Section 3.6, I have proposed algorithms for the computation of the Fisher information matrix of univariate ARMA or multivariate VARMA time-series models.

These algorithms are all based on the new integral representation in Section 3.1 and one of them makes use of Söderström’s algorithm in [29] described in Section 3.3.

I have used the following in Mathematica:

• The method proposed in [26];
• The variant proposed in Section 3.3;
• The method based on the experimental ResidueSum.

• We have seen in Section 3.3 that the three methods give the same exact results.

I have used in Maxima the method based on residue, which has the disadvantage of requiring the preliminary determination of the poles of the rational functions being involved. I could have considered porting circintrat.m to Maxima but this would require programming in LISP.

In the case of Octave, I have used the following two methods:

• The symbolic method using circintrat.m based on Söderström’s algorithm that gives the same exact results as Mathematica;
• The numerical method based on a function residuesum.m that gives approximately the same numerical results, at least in the examples.

• There is no doubt that the latter method with the function residuesum.m in Octave can lead to problems in near-singular cases.

The computation time of the different methods in Mathematica, Maxima, and Octave is quite variable. I give some orders of magnitude for the different methods and packages in

Table 1. Computation time for running Example 3.

Software	Integral	New Integral	ResidueSum	Residue	Söderström
Mathematica	$0.367$	$0.172$	$0.151$	not used	NA
Maxima	wrong (0)		NA	$0.011$	NA
Octave	wrong (0)		NA	$5.187$ *	$11.10$

(*) Numerical computation!

for Example 3 and

Table 2. Computation time for running Example 4.

Software	Integral	New Integral	ResidueSum	Residue	Söderström
Mathematica	$0.602$	$5.682$	$4.628$	not used	NA
Maxima	wrong ($)		NA	$1.795$	NA
Octave	wrong (0)		NA	$50.426$ *	$151.09$

(*) Numerical computation! ($) Except for a few elements.

for Example 4. They are given in seconds but, of course, they depend on the processor (an x64-based processor 11th Gen Intel Core i7-1165G7 clocked at 2.80 GHz, with 16 GB of installed RAM, and under Windows 11 version 23H2) and the charge. I took averages over 10,000 runs for the Maxima fast Example 3, 100 runs for Mathematica trigonometric integrals and the two slow methods with Octave, and 1000 runs for all the other methods. For Mathematica, I had to slightly change the model for each repetition because, otherwise, the results after the first run would be taken from the internal cache. For Example 3, the clear winner is residue in Maxima. For Example 4, the method of [26] in Mathematica was the best one, followed by residue in Maxima. Among the two methods used with Octave, the exact method using circintrat.m takes only about twice the time as the approximate method based on residuesum.m. They are much slower than the other methods. For Maxima, the reservations discussed previously about the need to determine the poles make it uneasy. Moreover, even for slightly more complex models, the impossibility of obtaining the poles symbolically will not allow us to use the methods based on Cauchy’s residues exactly, including ResidueSum in Mathematica. An alternative that is currently slow in Octave but does not require solving polynomial equations is provided by Söderström’s algorithm in [29].

5. Conclusions

To summarize, I have shown that a slight change in the presentation of the integrand (by making it more symmetric) in the integration procedure in the Wolfram Mathematica notebook proposed by [26] has allowed the development of two new methods and to address two other software packages, GNU Maxima and GNU Octave. Using Cauchy residues can lead to an improvement and is the solution for GNU Maxima, where the integration procedure does not work well presently. I have also tried a second open-source program with GNU Octave, where integration fails and the residue function is numeric. I have nevertheless obtained the correct results with a symbolic implementation of Söderström’s algorithm [29], although it is much slower than with the other solutions. I have mentioned in Section 4 the limitations of the methods based on residues.

As I have said in Section 2, there would be no problem for handling linearly and possibly lagged explanatory or exogenous variables, e.g., ARMAX or VARMAX models, since that would add a second matrix integral for which the methods discussed in this paper can be used. Consequently, there are now several exact procedures to check the identifiability of the ARMA, ARMAX, VARMA, and VARMAX models.

On the contrary, working with multiplicative seasonal ARMA or VARMA models, like in [10,11], is not so immediate but should be possible.

A reviewer has questioned the need for normality and why I did not consider the R language.

I have made clear that normality is related to the likelihood function. This means that the exact AFIM is obtained if parameter estimation is performed by maximizing the Gaussian quasi-likelihood or by least-squares, whatever the true distribution of the model errors.

If we know the distribution of ${\epsilon }_t$, another likelihood can be considered that should provide another AFIM, but there are relatively few popular alternatives except for count time series, e.g., [35,36,37,38,39], with a few exceptions like [40] or [41]. When we use the Gaussian likelihood instead of the appropriate likelihood, the information matrix is multiplied by a known factor, under very general conditions. As mentioned by [42] or [43], that factor is equal to ${\pi }^2/9$ for the logistic distribution or 2 for the Laplace distribution. This means that it is possible, in the latter case and with the correct likelihood, to decrease the asymptotic standard errors of the parameter estimators by a factor of $\sqrt{2}$; see also [16].

Alternatively, it is possible to use adaptive estimation methods like in [42] or R-estimation methods that are based on ranks. R-estimation is relatively easy for univariate time series like in [44] but much more difficult in the multivariate case, e.g., [45] or [46] and the references cited therein.

As for the R language, it is true that I did not consider R as a computer algebra system (CAS). After a search, I found at least two R packages on CRAN that offer exact computation and would be worth being considered: Caracas and Ryacas, which can be used for symbolic computations within R. Caracas offers an interface to SymPy, whereas Ryacas offers an interface to Yacas. Time constraints did not allow me to add them to the present study but I intend to consider them in the future. An alternative would be to use Python directly with SymPy. SymPy and Yacas offer matrices, polynomials, and integration, although not contour integration, apparently. I doubt that the program will be as short as the Mathematica code in Figure 1 and Figure 2. There is no reason why the Söderström approach would not work after reprogramming in the R language.