A New Algorithm for Computing Disjoint Orthogonal Components in the Parallel Factor Analysis Model with Simulations and Applications to Real-World Data

Abstract

In this paper, we extend the use of disjoint orthogonal components to three-way table analysis with the parallel factor analysis model. Traditional methods, such as scaling, orthogonality constraints, non-negativity constraints, and sparse techniques, do not guarantee that interpretable loading matrices are obtained in this model. We propose a novel heuristic algorithm that allows simple structure loading matrices to be obtained by calculating disjoint orthogonal components. This algorithm is also an alternative approach for solving the well-known degeneracy problem. We carry out computational experiments by utilizing simulated and real-world data to illustrate the benefits of the proposed algorithm.

1. Introduction, Notations, and Objectives

1.1. Introduction and Bibliographical Review

A third-order tensor (from now on, a three-way table for the applications in multivariate statistics) is an $I\times J\times K$ array $\underline{\mathit{X}}$ that stores observations on I individuals for J variables in K situations. For each way, we have a mode (set of entities), where the individuals are at the first mode, the variables in the second mode, and the situations in the third mode.

A popular multivariate statistical method is the principal component analysis (PCA), which allows for classification of variables or individuals [1,2]. When performing PCA on three-way tables, the aim is to reduce the dimension of at least one of the modes. A popular model for analyzing a three-way table is the parallel factor analysis, abbreviated as “Parafac”, where the three modes are reduced to R components, with $R<I,J,K$. This trilinear model has its origins in the research carried out by Hitchcock in 1927 [3,4]. In 1970, Harshman [5] and, Carroll and Chang [6], based on Hitchcock’s ideas, proposed the Parafac model, as it is known today. A method for determining the number of components in the Parafac model can be seen in [7].

It is important that the loading matrices $\mathit{A}$ (first mode), $\mathit{B}$ (second mode) and $\mathit{C}$ (third mode) in the Parafac model are easily interpretable. Scaling techniques, orthogonality constraints, and non-negativity constraints, are generally used to obtain simple structure loading matrices. It is known that the rows of a loading matrix are the original entities, while the columns are latent entities or components. For the loading matrix to be interpretable, in terms of determining the original entities that are represented by a particular component, the above techniques do not guarantee that a simple structure can be achieved in the matrix. In practice, note that some original entities in the loading matrices have similar loads on two or more components.

In the Parafac model, the components do not need to be orthogonal. However, the orthogonality constraint on at least one of the loading matrices allows the possibility of obtaining an interpretable solution [8,9]. The degeneracy problem is well known in the Parafac model [10]. This problem occurs when the inputs of the loading matrices diverge (matrix inputs are unstable), despite an apparent convergence in the fit of the model (as the algorithm iterates, the fit tends to stabilize at a value). A solution obtained in this way is called a degenerate solution and should not be interpreted. The degeneracy problem arises because the three-way data do not fit with the Parafac model, in other words, the model is not appropriate for the data. For a more detailed discussion of the degeneracy problem, see [11]. The degeneracy problem can be detected when two of the components have a high inverse correlation. For this reason, the component correlation matrix $\mathsf{\Psi }$ is usually calculated. If at least one of the inputs of $\mathsf{\Psi }$ is a value close to −1, it is suspected that the solution found is degenerate [12]. To avoid the degeneracy problem, orthogonality constraints are usually incorporated into the model in at least one of the loading matrices. Degeneracy can also be avoided by adding non-negativity constraints [13,14]. Imposing non-negativity constraints is also a technique used to obtain interpretable loading matrices. The use of an R package called ThreeWay that provides computational support for the Parafac model is illustrated in [15].

Sparse techniques consist of locating some zeros in the inputs of a matrix to obtain simple structure loading matrices. However, these techniques decrease the fit quality [16,17,18]. In a recent research [19], algorithms are proposed to build sparse loading matrices. Other authors [20,21] have used clusters in the Parafac model to obtain a better interpretation of the results. In all the above proposals, constraints are imposed on one mode. In general, the known techniques do not guarantee that a simple structure can be obtained for the loading matrices. However, the disjoint approach that we propose in this work for the Parafac model guarantees that a loading matrix is interpretable when computing disjoint orthogonal components. This is because each original entity of the matrix will belong to a unique component. Additionally, each component must represent at least one original entity.

In two-way tables, disjoint orthogonal components have proven to be a successful technique for obtaining simple structure loading matrices [22,23,24]. Calculation of disjoint orthogonal components for two-way tables by using an algorithm that is based on particle swarm optimization was proposed in [25]. A heuristic algorithm for calculating disjoint orthogonal components in the Tucker3 model is derived in [26], which is the first known algorithm that computes disjoint orthogonal components for three-way tables.

1.2. Acronyms, Notations, and Symbols

Next,

Table 1. Acronyms, notations, and symbols employed in the present document.

Acronym / Notation / Symbol	Definition
PCA	Principal component analysis.
Parafac	Parallel factor analysis.
$\underline{\mathit{X}}$	Tensor or three-way table (three dimensional).
$\mathit{X}$	Data matrix (two dimensional).
$I,J,K$	Number of entities in the first, second and third mode of the tensor, respectively.
R	Number of components in the Parafac model.
$i,j,k,r$	Indices.
$x_{ijk}$	Tensor element at position $ijk$, with $x_{ijk}\in \underline{\mathit{X}}$.
${\widehat{x}}_{ijk}$	Approximation of the element $x_{ijk}$.
${\epsilon }_{ijk}$	Error when approximating the element $x_{ijk}$.
$\mathit{A}$, $\mathit{B}$, $\mathit{C}$	Loading matrices of the first, second and third mode of the Parafac model, respectively.
$a_{ir}$	Element of loading matrix $\mathit{A}$ at position $ir$, with $a_{ir}\in \mathit{A}$.
$b_{jr}$	Element of loading matrix $\mathit{B}$ at position $jr$, with $b_{jr}\in \mathit{B}$.
$c_{kr}$	Element of loading matrix $\mathit{C}$ at position $kr$, with $c_{kr}\in \mathit{C}$.
${\mathit{X}}_{\mathit{A}}$, ${\mathit{X}}_{\mathit{B}}$, ${\mathit{X}}_{\mathit{C}}$	Frontal, horizontal and vertical slices matrices from $\underline{\mathit{X}}$, respectively.
${\mathit{E}}_{\mathit{A}}$, ${\mathit{E}}_{\mathit{B}}$, ${\mathit{E}}_{\mathit{C}}$	Error matrices when approximating matrices ${\mathit{X}}_{\mathit{A}}$, ${\mathit{X}}_{\mathit{B}}$ and ${\mathit{X}}_{\mathit{C}}$, respectively.
$\mathsf{\Psi }$	Component correlation matrix.
${\mathit{I}}_J$	The $J\times J$ identity matrix.
∘	Hadamard product.
⊙	Khatri–Rao product.
${\mathit{X}}^{\top }$	The transpose of matrix $\mathit{X}$.
${\mathit{X}}^{+}$	The Moore–Penrose generalized inverse of matrix $\mathit{X}$.
$\parallel \mathit{X}\parallel$	The Frobenius norm of matrix $\mathit{X}$.
$\mathit{B}\leftarrow \mathrm{vs}(\mathit{X},R,)$Tol	The DisjointPCA algorithm with tolerance Tol is applied to the data matrix $\mathit{X}$, and the
	disjoint orthogonal loading matrix $\mathit{B}$ with R components is obtained as a result using
	the Vichi-Saporta (vc) algorithm [22].

presents the acronyms, notations and math symbology considered in this paper to facilitate its reading.

1.3. Objectives and Description of Sections

This work is the result of an effort to incorporate disjoint orthogonal components in the Parafac model. Then, the objective of this paper is to propose a heuristic algorithm that allows the disjoint orthogonal components of the Parafac model to be computed. In addition, we present a procedure that provides a recommendation for the use of this algorithm, named here as DisjointParafacALS, because this is based on the alternating least squares (ALS) method. The DisjointParafacALS algorithm solves the degeneracy problem and eases the interpretation of loading matrices. We illustrate the algorithm with simulated and real-world data to show its potential applications.

The rest of the paper is organized as follows. Section 2 defines a disjoint orthogonal matrix and presents the mathematical model that must be optimized to calculate the disjoint orthogonal components in the Parafac model. In Section 3, we introduce the DisjointParafacALS algorithm. In Section 4, the computational aspects are presented while Section 5 reports the numerical experiments based on simulated and real-world data. Finally, some conclusions, other potential applications, and ideas for further research are provided in Section 6.

2. The Parafac Model and the Disjoint Approach

2.1. The Parafac Model

This model for an $I\times J\times K$ three-way table, $\underline{\mathit{X}}$ namely, has elements defined as

$$x_{ijk}={\widehat{x}}_{ijk}+{\epsilon }_{ijk},i=1,\dots ,I,j=1,\dots ,J,k=1,\dots ,K,$$

(1)

where ${\widehat{x}}_{ijk}$ is the approximation of the element $x_{ijk}\in \underline{\mathit{X}}$ and ${\epsilon }_{ijk}$ is the error term that contains all the variability not explained by the model. In (1), note that ${\widehat{x}}_{ijk}={\sum }_{r=1}^R{a}_{ir}{b}_{jr}{c}_{kr}$, where $a_{ir}$, $b_{jr}$, and $c_{kr}$ are the elements of the $I\times R$, $J\times R$, and $K\times R$ matrices, $\mathit{A}=\left(a_{ir}\right)$, $\mathit{B}=\left(b_{jr}\right)$, and $\mathit{C}=\left(c_{kr}\right)$ namely, respectively, which are called loading matrices. The integer R is such that $R<I$, $R<J$ and $R<K$, with R representing the number of components required for each mode.

The Parafac model can be represented by matrix equations based on all modes stated as ${\mathit{X}}_{\mathit{A}}=\mathit{A}{(\mathit{C}\odot \mathit{B})}^{\top }+{\mathit{E}}_{\mathit{A}}$, ${\mathit{X}}_{\mathit{B}}=\mathit{B}{(\mathit{C}\odot \mathit{A})}^{\top }+{\mathit{E}}_{\mathit{B}}$, and ${\mathit{X}}_{\mathit{C}}=\mathit{C}{(\mathit{B}\odot \mathit{A})}^{\top }+{\mathit{E}}_{\mathit{C}}$, where ⊙ is the Khatri–Rao product. Furthermore, the matrices ${\mathit{X}}_{\mathit{A}}$, ${\mathit{X}}_{\mathit{B}}$, and ${\mathit{X}}_{\mathit{C}}$ are the frontal, horizontal and vertical slices matrices from the tensor $\underline{\mathit{X}}$, respectively, whereas ${\mathit{E}}_{\mathit{A}}$, ${\mathit{E}}_{\mathit{B}}$ and ${\mathit{E}}_{\mathit{C}}$ are the corresponding error matrices [27].

The three loading matrices stated above may be estimated using some algorithms, with the ParafacALS algorithm being the most popular [28]. This algorithm calculates the three loading matrices by fixing two of them and finding the third matrix. This calculation is performed iteratively either until (i) the fit of the model do not differ significantly in two consecutive iterations (for example, $|{\mathtt{Fit}}_{\ell }-{\mathtt{Fit}}_{\ell -1}|<$Tol, with the tolerance Tol often being ${10}^{-6}$, a value that the package ThreeWay uses by default), or (ii) if a maximum number of iterations of the ALS algorithm, ALSMaxIter namely, is attained (for example, ALSMaxIter = 10,000). The aforementioned two conditions, (i) and (ii), are called the ParafacALS stopping criteria.

2.2. The ParafacALS Algorithm

This approach is summarized in Algorithm 1 with its outputs being the model fit, loading matrices $\mathit{A}$, $\mathit{B}$, $\mathit{C}$, and component correlation $R\times R$ matrix $\mathsf{\Psi }$. In Algorithm 1, the symbol ∘ is the Hadamard product [29], ⊙ is the Khatri–Rao product, and ${}^{+}$ is the Moore–Penrose generalized inverse of a matrix. For initialization of matrices $\mathit{B}$ and $\mathit{C}$, and to compute matrices ${\mathit{X}}_{\mathit{A}}$, ${\mathit{X}}_{\mathit{B}}$, and ${\mathit{X}}_{\mathit{C}}$, see details in [9].

Algorithm 1: ParafacALS approach

2.3. A Disjoint Approach for the Parafac Model

Next, the disjoint approach for the Parafac model is considered. The definition of a disjoint orthogonal matrix is as follows:

Definition of disjoint orthogonal matrix.

Let $\mathit{X}=\left(x_{ij}\right)$ be an $I\times J$ matrix, where $I>J$. Then, $\mathit{X}$ is disjoint if and only if, for any i, there exists a unique j such that $x_{ij}\ne 0$, and for any j, there exists i such that $x_{ij}\ne 0$. If $\mathit{X}$ also satisfies ${\mathit{X}}^{\top }\mathit{X}={\mathit{I}}_J$, where ${\mathit{I}}_J$ is the $J\times J$ identity matrix, then $\mathit{X}$ is a disjoint orthogonal matrix.

If a loading matrix of the Parafac model satisfies the definition of a disjoint orthogonal matrix, its interpretation is simplified. Suppose, for example, that the entities in the second mode of a data tensor are the following courses ($J=8$): grammar, math, physics, psychology, literature, history, chemistry and sports. Suppose also that the loading matrix $\mathit{B}$ is the one observed in

Table 2. Loading matrix $\mathit{B}$ of the example of courses.

Course	COMP1	COMP2	COMP3
Grammar	0	0.42683051	0
Math	0.58009486	0	0
Physics	0.56675657	0	0
Psychology	0	0.52692660	0
Literature	0	0.51315878	0
History	0	0.52614840	0
Chemistry	0.58504440	0	0
Sports	0	0	1

with $R=3$ components. Note that component COMP1 groups the courses of math, physics and chemistry. We can then affirm that component COMP1 is the “science component”. Component COMP2 represents the courses of grammar, psychology, literature and history. We can call component COMP2 as the “humanity component”. Component COMP3 only has the sports course. If there were other courses like gymnastics, crossfit and yoga, surely component COMP3 would catch them. Then, component COMP3 would group courses related to physical activities. By imposing the constraint on the Parafac model that one of the loading matrices is disjoint orthogonal, a clear interpretation of the components can be obtained. The algorithm proposed in this work allows us to compute disjoint orthogonal loading matrices like those that have been observed in the previous illustrative example. See Section 5 for other examples of disjoint orthogonal matrices.

Below, we present the mathematical model of an optimization problem that must be solved when one of the three loading matrices $\mathit{A}$, $\mathit{B}$ or $\mathit{C}$ is required to be disjoint orthogonal. The number of decision variables of this problem is $\left(I+J+K\right)R$. Suppose that $\mathit{B}$ is required to be a disjoint orthogonal matrix. Thus, the model is described as

$$\begin{array}{ccc}& & \underset{\begin{array}{c}\mathit{A},\mathit{B},\mathit{C}\end{array}}{min}\parallel {\mathit{X}}_{\mathit{A}}-\underset{{\widehat{\mathit{X}}}_{\mathit{A}}}{\underbrace{\mathit{A}{(\mathit{C}\odot \mathit{B})}^{\top }}}{\parallel }^2,\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill \mathrm{subject}\mathrm{to}:& & {\mathit{B}}^{\top }\mathit{B}={\mathit{I}}_R,\mathrm{with}\mathit{B}\mathrm{being}\mathrm{a}\mathrm{disjoint}\mathrm{matrix}.\hfill \end{array}$$

(3)

In (2), note that $\parallel ·\parallel$ is the Frobenius norm. The full constraint stated in (3) is needed for the matrix $\mathit{B}$ to be disjoint orthogonal. In the previous mathematical model, the objective function that is defined in (2) must be minimized, but in practice, the fit is calculated according to step 11 of Algorithm 1. The optimization problem can be solved with the DisjointParafacALS algorithm that allows for a dimensional reduction of R components in its three modes. The focus of this work is on the case where one of the three loading matrices is constrained to be a disjoint orthogonal matrix.

In a two-way table, a disjoint orthogonal loading matrix $\mathit{B}$ can be obtained by the following formulation:

$$\begin{array}{ccc}\hfill & & \underset{\begin{array}{c}\mathit{A},\mathit{B}\end{array}}{min}{\parallel \mathit{X}-\mathit{A}{\mathit{B}}^{\top }\parallel }^2\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill \mathrm{subject}\mathrm{to}:& & {\mathit{B}}^{\top }\mathit{B}={\mathit{I}}_R,\mathrm{with}\mathit{B}\mathrm{being}\mathrm{a}\mathrm{disjoint}\mathrm{matrix}.\hfill \end{array}$$

(5)

where $\mathit{X}$ is the $I\times J$ data matrix with I individuals and J variables, $\mathit{A}$ is the $I\times R$ score matrix, and $\mathit{B}$ is the $J\times R$ loading matrix. Observe that $R<J$ is the required number of components in the mode of variables and the full constraint (5) is needed for the loading matrix $\mathit{B}$ to be disjoint orthogonal.

In this paper, we use a greedy algorithm, known as the DisjointPCA algorithm, that was proposed in [22], in order to find a solution to the minimization problem defined in (4). The DisjointPCA algorithm plays a fundamental role in the operation of the DisjointParafacALS algorithm.

The notation $\mathit{B}\leftarrow \mathrm{vs}\left(\mathit{X},R,\right)$Tol means that the DisjointPCA algorithm is applied to the data matrix $\mathit{X}$ with a tolerance Tol, and the disjoint orthogonal loading matrix $\mathit{B}$ with R components is obtained as a result using the Vichi-Saporta (vc) algorithm [22]. Here, as mentioned, Tol is a tolerance parameter that represents the maximum distance allowed in the fit of the model for two consecutive iterations of the DisjointPCA algorithm. The above notation is used to explain how the DisjointParafacALS algorithm works. For more details about the DisjointPCA algorithm, see [23,25]. The DisjointPCA algorithm is also used by the DisjointTuckerALS algorithm in [26] to compute disjoint orthogonal components in the Tucker3 model.

3. The DisjointParafacALS Algorithm

3.1. Definitions

This novel algorithm allows us to compute the disjoint orthogonal components in the Parafac model. The algorithm has been designed to compute a single disjoint orthogonal loading matrix. The operation of the DisjointParafacALS algorithm is explained below. Before executing the algorithm, we recommend carrying out the corresponding preprocessing of the data. The DisjointParafacALS algorithm has two stages and its input parameters are:

• $\underline{\mathit{X}}$: Three-way table of data;
• R: Number of components in modes A, B, and C;
• ALSMaxIter: Maximum number of iterations of the ALS algorithm; and
• Tol: Maximum distance allowed in the fit of the model for two consecutive iterations.

3.2. Stages of the Algorithm

• [Stage 1] Computation of a disjoint orthogonal loading matrix with the DisjointPCA algorithm: The first stage is computing the disjoint loading matrix. In order to obtain R disjoint orthogonal components in the loading matrices $\mathit{A}$, $\mathit{B}$, or $\mathit{C}$, the DisjointPCA algorithm is applied to the matrices ${\mathit{X}}_{\mathit{A}}^{\top }$, ${\mathit{X}}_{\mathit{B}}^{\top }$, or ${\mathit{X}}_{\mathit{C}}^{\top }$, respectively. If $\mathit{A}$, $\mathit{B}$, and $\mathit{C}$ are required to be disjoint orthogonal, then we have that: $\mathit{A}\leftarrow \mathrm{vs}\left({\mathit{X}}_{\mathit{A}}^{\top },R,\right)$Tol; $\mathit{B}\leftarrow \mathrm{vs}\left({\mathit{X}}_{\mathit{B}}^{\top },R,\right)$Tol; and $\mathit{C}\leftarrow \mathrm{vs}\left({\mathit{X}}_{\mathit{C}}^{\top },R,\mathtt{Tol}\right)$, respectively.
• [Stage 2] Computation of non-disjoint orthogonal loading matrices: The second stage is computing the non-disjoint loading matrices. For instance, if it is required that the matrix $\mathit{B}$ has disjoint orthogonal components, the ALS algorithm must be applied to compute loading matrices $\mathit{A}$ and $\mathit{C}$ ($\mathit{B}$ matrix is fixed) as happens in the ParafacALS algorithm, initializing $\mathit{A}$ or $\mathit{C}$; see Figure 1. The other two cases can be developed from the illustration based on $\mathit{B}$ in this figure. The DisjointParafacALS algorithm finishes by outputing the matrices $\mathit{A}$, $\mathit{B}$, and $\mathit{C}$, and the fit of the model.

3.3. Algorithm and Flowchart

We summarize our proposal in Algorithm 2 to explain how the DisjointParafacALS algorithm works when performing a component analysis for three-way tables with the Parafac model. Computing the disjoint orthogonal components in Step 6 of Algorithm 2 simultaneously in two or three loading matrices is an approach that the analyst must choose. However, we do not recommend this approach due to the significant loss of fit that we have observed in the different computational experiments carried out. We suggest computing a single disjoint orthogonal loading matrix.

Note that the DisjointParafacALS algorithm can be utilized regardless of whether the degeneracy problem is present or not. This algorithm can be applied directly to a three-way table in order to interpret the loading matrices, with the corresponding loss of fit in the model; see Figure 2.

Algorithm 2: Procedure for using the DisjointParafacALS algorithm

Step 1  Collect the data in an $I\times J\times K$ three-way table $\underline{\mathit{X}}$, where I is the number of individuals, J is the number of variables, and K is the number of situations or times. Step 2  Preprocess $\underline{\mathit{X}}$ according to the analyst’s criterion. Step 3  Fit $\underline{\mathit{X}}$ with an unconstrained Parafac model based on R components. Step 4  Compute the fit of the model with no restrictions by using, for example, Algorithm 1. If degeneracy is suspected when analyzing the total number of iterations and the component correlation matrix $\mathsf{\Psi }$, continue to Step 5; otherwise, go to Step 8. Step 5  Impose orthogonality (or non-negativity) constraints to at least one of the loading matrices and compute loading matrices $\mathit{A}$, $\mathit{B}$, and $\mathit{C}$ (apply the scaling technique if necessary to maintain the fit and improve the interpretation). Step 6  Enforce the disjoint and orthogonality constraints on a single loading matrix and compute the loading matrices $\mathit{A}$, $\mathit{B}$, and $\mathit{C}$ by using the DisjointParafacALS algorithm (that is, compute the disjoint orthogonal components for one loading matrix). Step 7  Compare the results obtained in Step 5 and Step 6, and go to Step 10. Step 8  Calculate the loading matrices $\mathit{A}$, $\mathit{B}$, and $\mathit{C}$ without constraints by using, for example, Algorithm 1 (apply the scaling technique if necessary). Step 9  Do Step 6 and compare the results with those obtained in Step 8. Step 10  Analyze the results and conclusions.

4. Computational Aspects

4.1. Characteristics of Hardware and Software

Table 3. Hardware and software used in computational experiments.

Element	Characterist
Operating System	Windows 10 for 64 bits
RAM	8 Gigabytes
Processor	Intel Core i7-4510U 2-2.60 GHZ
Development tool—IDE—	Microsoft Visual Studio Express
Programming language	C#.NET
Statistical software	R
Data entry and presentation of results	Microsoft Excel
Communication between C#.NET and Excel	COM+
Communication between C#.NET and R	R.NET
Random number generation	runif function of the stats package
Singular value decomposition (SVD)	irlba function of the irlba package
Moore–Penrose generalized inverse of a matrix	ginv function of the MASS package

shows the details of the software and hardware used in the implementation of the algorithms and in the computational experiments carried out.

We must mention that the DisjointParafacALS algorithm requires more processing time (runtime) than the ParafacALS algorithm, even if the degeneracy problem is present. This is explained by the fact that calculating a disjoint orthogonal loading matrix consumes significant computational resources, which leads to longer processing time. The DisjointPCA, ParafacALS, and DisjointParafacALS algorithms that are presented in this paper were implemented mainly in C#.NET. For the particular case of the generation of random numbers, the SVD decomposition and the calculation of the generalized inverse of a matrix, the R statistical software was used.

4.2. Experimental Settings for Simulated Data

We designed and implemented a simulation algorithm to randomly build a three-way table with a disjoint latent structure of R components and its three modes, according to the Parafac model. Then, the DisjointParafacALS algorithm should be able to detect such a structure.

The loading matrices $\mathit{A}$, $\mathit{B}$ and $\mathit{C}$ used in the numerical experiments related to the simulated data were generated as in [26]. In order to complete the creation of the random three-way table $\underline{\mathit{X}}$, the matrix equation was defined as

$${\mathit{X}}_{\mathit{A}}=\mathit{A}{\left(\mathit{C}\odot \mathit{B}\right)}^{\top }.$$

(6)

The $I\times JK$ matrix ${\mathit{X}}_{\mathit{A}}$ defined in (6) had the frontal slices of $\underline{\mathit{X}}$. The algorithm that buildt the $I\times J\times K$ three-way random table $\underline{\mathit{X}}$ had to implement an application $\mathbf{\phi }$ stated as

$$\begin{array}{ccc}\hfill \mathbf{\phi }:{\mathbb{N}}^4\times {\mathbb{N}}^R\times {\mathbb{N}}^R\times {\mathbb{N}}^R& \to & {\mathrm{T}}_{I\times J\times K}\hfill \\ \hfill \left(\left\{I,J,K,R\right\},{\left\{{\alpha }_r\right\}}_{r=1}^R,{\left\{{\beta }_r\right\}}_{r=1}^R,{\left\{{\gamma }_r\right\}}_{r=1}^R\right)& \mapsto & \mathbf{\phi }\left(\left\{I,J,K,R\right\},{\left\{{\alpha }_r\right\}}_{r=1}^R,{\left\{{\beta }_r\right\}}_{r=1}^R,{\left\{{\gamma }_r\right\}}_{r=1}^R\right).\hfill \end{array}$$

(7)

This was subject to: $R<I$, $R<J$, and $R<K$. Note that ${\mathrm{T}}_{I\times J\times K}$ is the set of all three-way tables with real entries. Additionally, it must satisfy that

$$I=\sum _{r=1}^R{\alpha }_r,J=\sum _{r=1}^R{\beta }_r,K=\sum _{r=1}^R{\gamma }_r,$$

(8)

where ${\alpha }_r$ is the number of original individuals in the rth latent individual; ${\beta }_r$ is the number of original variables in the rth latent variable; and ${\gamma }_r$ is the number of original locations in the rth latent location. The output $\mathbf{\phi }$ defined in (7) is the $I\times J\times K$ three-way table $\underline{\mathit{X}}$ that has a disjoint latent structure to be detected by the DisjointParafacALS algorithm.

4.3. Experimental Application with Simulated Data

We conducted computational experiments to evaluate the performance and benefits of the DisjointParafacALS algorithm. The first experiment corresponded to data simulated by an algorithm that was exclusively designed to generate three-way data with a disjoint structure according to the Parafac model. The other experiments are presented in the following section and related to (i) real data of evaluation of TV programs [30]; (ii) real data of perception of parents’ behavior by their children and by the parents themselves [31]; and (iii) real data from the study of tongue movements when speakers pronounce vowels in English [32].

Here, we show how the DisjointParafacALS algorithm works by using the application $\mathbf{\phi }$ to generate the $15\times 20\times 10$ three-way random table $\underline{\mathit{X}}$. Specifically, consider the instance given by $\mathbf{\phi }\left(\left\{15,20,10,3\right\},\left\{5,6,4\right\},\left\{7,8,5\right\},\left\{2,5,3\right\}\right),$ according to the definition of $\mathbf{\phi }$ given in (7), with $R=3$. Furthermore, constraints stated in (8) were satisfied. The other parameter setting was given by $\mathtt{ALSMaxIter}=$ 10,000 and Tol $={10}^{-6}$.

Table 4. Component correlation matrix $\mathsf{\Psi }$ for the simulated data.

	COMP1	COMP2	COMP3
COMP1	1	0.019363	0.024297
COMP2	0.019363	1	0.010313
COMP2	0.024297	0.010313	1

shows the component correlation matrix $\mathsf{\Psi }$ from where we can conclude, based on the output, that the degeneracy problem was not present in the data.

The DisjointParafacALS algorithm was executed in three different situations. There are four execution scenarios. In the first scenario, the ParafacALS algorithm gave a fit of $99.31\%$. In the second scenario, in which only the loading matrix $\mathit{A}$ was disjoint orthogonal, a fit of $95.70\%$ was obtained. In the third scenario, in which only the loading matrix $\mathit{B}$ was disjoint orthogonal, a fit of $93.71\%$ was obtained. In the last and fourth scenario, in which only the loading matrix $\mathit{C}$ was disjoint orthogonal, a fit of $95.76\%$ was obtained. These computational experiments are summarized in

Table 5. Disjoint orthogonal components, its fit (in %) and runtime (in minutes) for the indicated matrix with simulated data.

Disjoint Orthogonal Components	Fit (in %)	Runtime (in Minutes)
None (ParafacALS)	99.31	0.0007
$\mathit{A}$ (DisjointParafacALS)	95.70	0.1736
$\mathit{B}$ (DisjointParafacALS)	93.71	0.1862
$\mathit{C}$ (DisjointParafacALS)	95.76	0.1325

.

Table 6. Loading matrix $\mathit{A}$ with the DisjointParafacALS algorithm for the simulated data.

Subject (individual)	COMP1	COMP2	COMP3
$S_1$	−0.39987415	0	0
$S_2$	−0.38886439	0	0
$S_3$	−0.46369775	0	0
$S_4$	−0.47750129	0	0
$S_5$	−0.49584480	0	0
$S_6$	0	0.45327357	0
$S_7$	0	0.40688744	0
$S_8$	0	0.37939130	0
$S_9$	0	0.34345457	0
$S_{10}$	0	0.41482141	0
$S_{11}$	0	0.44159947	0
$S_{12}$	0	0	−0.54088342
$S_{13}$	0	0	−0.41894439
$S_{14}$	0	0	−0.53459196
$S_{15}$	0	0	−0.49612716

reports the loading matrix $\mathit{A}$ for the second scenario. Observe that the DisjointParafacALS algorithm was able to detect the disjoint structure in the first mode.

Table 7. Loading matrix $\mathit{B}$ with the DisjointParafacALS algorithm for the simulated data.

Variable	COMP1	COMP2	COMP3
$V_1$	−0.40287714	0	0
$V_2$	−0.38117182	0	0
$V_3$	−0.38769074	0	0
$V_4$	−0.34328899	0	0
$V_5$	−0.31499905	0	0
$V_6$	−0.34929550	0	0
$V_7$	−0.45057171	0	0
$V_8$	0	−0.39457824	0
$V_9$	0	−0.33906833	0
$V_{10}$	0	−0.37809945	0
$V_{11}$	0	−0.30556330	0
$V_{12}$	0	−0.32902385	0
$V_{13}$	0	−0.32439850	0
$V_{14}$	0	−0.33306471	0
$V_{15}$	0	−0.41059636	0
$V_{16}$	0	0	−0.46109260
$V_{17}$	0	0	−0.43821079
$V_{18}$	0	0	−0.41637465
$V_{19}$	0	0	−0.38723790
$V_{20}$	0	0	−0.52157826

presents the loading matrix $\mathit{B}$ for the third scenario. Note that indeed the DisjointParafacALS algorithm was able to assess the disjoint structure in the second mode.

Table 8. Loading matrix $\mathit{C}$ with the DisjointParafacALS algorithm for the simulated data.

Time	COMP1	COMP2	COMP3
$T_1$	−0.72171067	0	0
$T_2$	−0.69219484	0	0
$T_3$	0	−0.43505908	0
$T_4$	0	−0.39485340	0
$T_5$	0	−0.49233526	0
$T_6$	0	−0.42701150	0
$T_7$	0	−0.47966818	0
$T_8$	0	0	−0.59150295
$T_9$	0	0	−0.53266172
$T_{10}$	0	0	−0.60530633

shows the loading matrix $\mathit{C}$ for the fourth scenario. Notice that the DisjointParafacALS algorithm was able to identify the disjoint structure in the third mode.

As observed in Table 6, Table 7 and Table 8, the disjoint structure facilitated the interpretation of the loading matrices, but with some loss of fit. In a fifth scenario, the DisjointParafacALS algorithm was executed with the loading matrices $\mathit{A}$ and $\mathit{B}$ set to be disjoint orthogonal as a constraint (illustration of a computational experiment with two disjoint orthogonal matrices). A fit of $87.41\%$ was obtained, that is, with two disjoint orthogonal matrices, we obtain a high loss of fit. In the rows of Table 6, Table 7 and Table 8 are the original entities, and in the columns are the latent entities.

We recommend running the DisjointParafacALS algorithm with a single disjoint orthogonal matrix as shown in Table 5, and where appropriate, running the DisjointParafacALS algorithm with two disjoint orthogonal matrices.

5. Applications with Real-World Data

5.1. Applying the DisjointParafacALS Algorithm to TV Data

We considered the well-known “TV data” set, where 30 university students classify 15 American programs using 16 bipolar scales. As noted in [15], the first mode corresponded to students, the second mode corresponded to bipolar scales and the third mode corresponded to TV programs. The full data set can be downloaded from the R package named ThreeWay.

A three component dimensional reduction with the Parafac model was chosen as in [30]. As part of the preprocessing, the three-way table was centered first with respect to scales (second mode) and then centered with respect to TV programs (third mode). Thus, normalization was applied with respect to the bipolar scales.

In the reported computational experiments, the following values were used: $\mathtt{ALSMaxIter}$ = 10,000 and Tol $={10}^{-6}$. When executing the ParafacALS algorithm with these data, the degeneracy problem was presented. After more than 7000 iterations, we obtained a fit of $47.93\%$ and the component correlation matrix stated in

Table 9. Component correlation matrix $\mathsf{\Psi }$ for TV data.

	COMP1	COMP2	COMP3
COMP1	1	0.05652381	−0.97495543
COMP2	0.05652381	1	−0.11506567
COMP3	−0.97495543	−0.11506567	1

. In this table, the correlation between the components one and three was approximately $-1$, a result that suggests the presence of degeneracy. Furthermore, a high number of iterations (7231 iterations) was required to reach the stopping criterion of the ParafacALS algorithm. To solve the degeneracy problem, in [10], it is recommended to impose orthogonality constraints on at least one of the loading matrices. By imposing the orthogonality constraint on the loading matrix $\mathit{B}$, a fit of $47.27\%$ was obtained in 239 iterations. The component correlation matrix $\mathsf{\Psi }$ obtained for $\mathit{B}$ is reported in

Table 10. Component correlation matrix $\mathsf{\Psi }$ with the orthogonal matrix $\mathit{B}$ for TV data.

	COMP1	COMP2	COMP3
COMP1	1	0	0
COMP2	0	1	0
COMP3	0	0	1

.

Similarly to [15], in order to aid the interpretation, in this last execution, we set the columns of the loading matrices $\mathit{B}$ and $\mathit{C}$ to be unit vectors. Such a constraint was compensated in matrix $\mathit{A}$. Then, the DisjointParafacALS algorithm was executed in two different scenarios. In the first of them, only the loading matrix $\mathit{B}$ had disjoint orthogonal components, but in the second scenario, only the loading matrix $\mathit{C}$ had disjoint orthogonal components. The fit obtained and the processing time in all executions is summarized in

Table 11. Comparison of fit and runtime for TV data.

Scenario	Fit (in %)	Runtime (in Minutes)
ParafacALS - Degeneracy	47.93	0.4258
ParafacALS with Orthogonal $\mathit{B}$	47.27	0.0182
DisjointParafacALS with Disjoint Orthogonal $\mathit{B}$	41.12	0.5664
DisjointParafacLAS with Disjoint Orthogonal $\mathit{C}$	41.21	0.5456

.

Note that having disjoint orthogonal components affected the fit, but the interpretation increased, as we will illustrate later. As stated in [30], the three components could be interpreted as “humor”, “sensitivity” and “violence”. It should be noted that a negative value in loading matrix $\mathit{B}$ was interpreted as belonging to the left side of the bipolar scale.

Table 12. Orthogonal loading matrix $\mathit{B}$ with the ParafacALS algorithm for TV data.

Bipolar Scale	Humor	Sensitivity	Violence
(1) Thrilling-Boring	−0.09	0.29	−0.12
(2) Intelligent-Idiotic	0.30	0.17	0.12
(3) Erotic-Not Erotic	−0.25	−0.06	−0.27
(4) Sensitive-Insensitive	0.03	−0.43	0.12
(5) Interesting-Uninteresting	0.00	0.37	0.26
(6) Fast-Slow	−0.08	0.19	−0.29
(7) Intell. Stimulating-Intell. Dull	0.27	0.20	0.25
(8) Violent-Peaceful	0.08	0.04	−0.68
(9) Caring-Callous	0.04	−0.47	0.02
(10) Satirical-Not Satirical	−0.46	0.09	0.18
(11) Informative-Uninformative	0.31	0.20	0.19
(12) Touching-Leave Me Cold	−0.07	−0.31	0.15
(13) Deep-Shallow	0.21	−0.14	0.16
(14) Tasteful-Crude	0.19	−0.31	0.02
(15) Real-Fantasy	0.38	0.06	−0.06
(16) Funny-Not Funny	−0.46	0.06	0.30

shows the orthogonal loading matrix $\mathit{B}$ that was obtained using the ParafacALS algorithm.

Table 13. Disjoint orthogonal matrix $\mathit{B}$ with the DisjointParafacALS algorithm for TV data.

Bipolar Scale	Humor	Sensitivity	Violence
(1) Thrilling-Boring	0	0.40	0
(2) Intelligent-Idiotic	0.39	0	0
(3) Erotic-Not Erotic	−0.35	0	0
(4) Sensitive-Insensitive	0	−0.50	0
(5) Interesting-Uninteresting	0	0.37	0
(6) Fast-Slow	0	0	−0.52
(7) Intell. Stimulating-Intell. Dull	0.39	0	0
(8) Violent-Peaceful	0	0	−0.47
(9) Caring-Callous	0	−0.52	0
(10) Satirical-Not Satirical	−0.39	0	0
(11) Informative-Uninformative	0.40	0	0
(12) Touching-Leave Me Cold	0	−0.41	0
(13) Deep-Shallow	0	0	0.51
(14) Tasteful-Crude	0	0	0.50
(15) Real-Fantasy	0.36	0	0
(16) Funny-Not Funny	−0.38	0	0

reports the disjoint orthogonal matrix $\mathit{B}$ that was obtained with the DisjointParafacALS algorithm. Loadings with absolute values greater than or equal to $0.25$ that we observe in Table 12 and Table 13 facilitate comparison. Except for minor differences, the interpretation was the same.

Table 14. Loading matrix $\mathit{C}$ with the ParafacALS algorithm for TV data.

TV Program	Humor	Sensitivity	Violence
(1) Mash	0.14	0.03	−0.11
(2) Charlie’s angels	0.21	0.13	0.54
(3) All in the family	0.20	0.04	−0.18
(4) 60 min	−0.35	−0.19	−0.18
(5) The tonight show	0.16	−0.23	−0.19
(6) Let’s make a deal	0.19	0.00	0.25
(7) The Waltons	−0.13	0.55	−0.23
(8) Saturday night live	0.35	−0.31	0.11
(9) News	−0.39	−0.27	0.00
(10) Kojak	0.09	−0.04	0.37
(11) Mork and Mindy	0.38	0.12	−0.10
(12) Jacques Cousteau	−0.40	−0.10	−0.29
(13) Football	−0.07	−0.32	0.39
(14) Little house on the prairie	−0.06	0.54	−0.18
(15) Wild kingdom	−0.31	0.04	−0.21

presents the loading matrix $\mathit{C}$ that was obtained using the ParafacALS algorithm.

Table 15. Disjoint orthogonal matrix $\mathit{C}$ with the DisjointParafacALS algorithm for TV data.

TV Program	Humor	Sensitivity	Violence
(1) Mash	0.17	0	0
(2) Charlie’s angels	0	0	0.53
(3) All in the family	0.28	0	0
(4) 60 min	−0.51	0	0
(5) The tonight show	0	−0.24	0
(6) Let’s make a deal	0	0	0.36
(7) The Waltons	0	0.60	0
(8) Saturday night live	0	−0.44	0
(9) News	−0.63	0	0
(10) Kojak	0	0	0.25
(11) Mork and Mindy	0.49	0	0
(12) Jacques Cousteau	0	0	−0.58
(13) Football	0	−0.31	0
(14) Little house on the prairie	0	0.55	0
(15) Wild kingdom	0	0	−0.44

reports the disjoint orthogonal matrix $\mathit{C}$ that was reached with the DisjointParafacALS algorithm. Loadings with absolute values greater than or equal to $0.25$ reported in Table 14 and Table 15 facilitate comparison. Except for minor differences, the interpretation was the same.

5.2. Applying the DisjointParafacALS Algorithm to Kojima Data

A Parafac model was applied in [31]. The three-way table $\underline{\mathit{X}}$ was a $153\times 18\times 4$ tensor with 153 Japanese girls in the first mode, 18 behavior scales in the second mode, and four judgements or conditions in the third mode. The full data set can be downloaded from http://three-mode.leidenuniv.nl (accessed on 9 July 2021).

With these data, we studied the perception of parents’ behavior by their children and by the parents themselves. To do so, the reactions of parents and children were received in parallel versions of the same questionnaire. The four conditions of the third mode were: FF (the father evaluates his own behavior), MM (the mother evaluates her own behavior), DF (the daughter evaluates the father’s behavior) and DM (the daughter evaluates the mother’s behavior).

The 18 scales were grouped into four types: Parental support or acceptance (PSA), behavioral and psychological control (BPC), rejection (R), and lax discipline (LD). As in [31], $R=3$ components were chosen. Furthermore, as part of the data preprocessing, a centering was performed in the first mode and then a normalization in the second mode. In the reported computational experiments, the following values were used: $\mathtt{ALSMaxIter}=$ 10,000 and Tol $={10}^{-6}$. When executing the ParafacALS algorithm with these data, the degeneracy problem was once again presented. After more than $5000$ iterations, we reached a fit of $38.82\%$ and the component correlation matrix $\mathsf{\Psi }$ is reported in in

Table 16. Component correlation matrix $\mathsf{\Psi }$ with $R=3$ components for Kojima data.

	COMP1	COMP2	COMP3
COMP1	1	−0.20159703	−0.93725207
COMP2	−0.20159703	1	0.06099119
COMP3	−0.93725207	0.06099119	1

.

Table 16 tells us that the degeneracy problem could be present again. Components 1 and 3 had a high negative correlation (value close to $-1$). In [31], the orthogonality condition of matrix $\mathit{A}$ was imposed. We proceed to assume that matrix $\mathit{B}$ must be disjoint orthogonal to improve the interpretation in the second mode and the DisjointParafacALS algorithm was used. We obtained that the component correlation matrix $\mathsf{\Psi }$ was the $3\times 3$ identity matrix. The DisjointParafacALS algorithm was executed in two different scenarios. In the first of these, only the loading matrix $\mathit{B}$ had disjoint orthogonal components with $R=3$ components, and in the second scenario, only the loading matrix $\mathit{B}$ had disjoint orthogonal components with $R=4$ components. Additionally, the ParafacALS algorithm was executed with $R=4$ components and the presence of the degeneration problem was also observed.

Table 17. Component correlation matrix $\mathsf{\Psi }$ with $R=4$ components for Kojima data.

	COMP1	COMP2	COMP3	COMP4
COMP1	1	0.09060317	−0.17620287	0.02597586
COMP2	0.09060317	1	−0.93490587	0.06082605
COMP3	−0.17620287	−0.93490587	1	−0.14096143
COMP4	0.02597586	0.06082605	−0.14096143	1

reports that components 2 and 3 had a high negative correlation. The DisjointParafacALS algorithm was used to solve the degeneracy problem with $R=4$ components. Then, we obtained that the component correlation matrix $\mathsf{\Psi }$ was the $4\times 4$ identity matrix. The fit obtained and the processing time in all executions are summarized in

Table 18. Comparison of fit and runtime for Kojima data.

Scenario	Fit (in %)	Runtime (in Min)
ParafacALS ($R=3$) - Degeneracy	38.82	0.0753
ParafacALS ($R=3$) with orthogonal $\mathit{A}$	38.01	0.0084
DisjointParafacALS ($R=3$) with disjoint orthogonal $\mathit{B}$	32.28	0.2396
ParafacALS ($R=4$) - Degeneracy	45.95	0.0492
DisjointParafacALS ($R=4$) with disjoint orthogonal $\mathit{B}$	36.69	0.4473

.

We proceed to the analysis of the loading matrix for the scales. In

Table 19. Matrix $\mathit{B}$ obtained by the orthogonal matrix $\mathit{A}$ condition for Kojima data.

Scale	Scale Type	COMP1	COMP2	COMP3
(1) Acceptance	PSA	0.64	0.27	0.24
(2) Child centredness	PSA	0.54	0.32	0.27
(3) Possessiveness	BPC	0.06	0.47	0.38
(4) Rejection	R	−0.53	0.36	0.24
(5) Control	BPC	0.02	0.32	0.56
(6) Enforcement	BPC	−0.18	0.34	0.49
(7) Positive involvement	PSA	0.54	0.4	0.3
(8) Intrusiveness	BPC	0.16	0.37	0.49
(9) Control through guilt	R	−0.34	0.36	0.35
(10) Hostile control	BPC	−0.17	0.34	0.61
(11) Inconsistent discipline	LD	−0.3	0.43	0.14
(12) Nonenforcement	LD	0.28	−0.22	0.01
(13) Acceptance individuation	PSA	0.64	0.29	0.17
(14) Lax discipline	LD	0.37	0	0.04
(15) Instilling persistent anxiety	BPC	−0.11	0.41	0.49
(16) Hostile detachment	R	−0.58	0.24	0.16
(17) Withdrawal of relations	R	−0.45	0.35	0.23
(18) Extreme autonomy	LD	0.32	−0.2	0.02

, we see the three components calculated in [31].

Table 20. Disjoint orthogonal matrix $\mathit{B}$ using $R=3$ components and obtained with the DisjointParafacALS algorithm for Kojima data.

Scale	Scale Type	COMP1	COMP2	COMP3
(1) Acceptance	PSA	0.49	0	0
(2) Child centredness	PSA	0.46	0	0
(3) Possessiveness	BPC	0	0.26	0
(4) Rejection	R	0	0.35	0
(5) Control	BPC	0	0.29	0
(6) Enforcement	BPC	0	0.33	0
(7) Positive involvement	PSA	0.47	0	0
(8) Intrusiveness	BPC	0.33	0	0
(9) Control through guilt	R	0	0.34	0
(10) Hostile control	BPC	0	0.36	0
(11) Inconsistent discipline	LD	0	0.26	0
(12) Nonenforcement	LD	0	0	0.57
(13) Acceptance individuation	PSA	0.46	0	0
(14) Lax discipline	LD	0	0	0.59
(15) Instilling persistent anxiety	BPC	0	0.34	0
(16) Hostile detachment	R	0	0.28	0
(17) Withdrawal of relations	R	0	0.33	0
(18) Extreme autonomy	LD	0	0	0.57

shows the three disjoint components calculated with the DisjointParafacALS algorithm. In Table 19, it is observed that the component 1 mainly groups the “parental support or acceptance” scales with high positive loadings and the “rejection” scales with high negative loadings. The component 2 mainly groups both, the “behavioral and psychological control” and “rejection” scales with high positive loadings, the second one with lower values. In the component 3, we observe some “behavioral and psychological control” scales with high positive loadings. The “lax discipline” scales are not related with any particular component.

When analyzing the three components, note that component 1 represents the “parental support or acceptance” scales. The component 2 groups “behavioral and psychological control” and “rejection”. The component 3 is represents “lax discipline”.

Table 21. Disjoint orthogonal matrix $\mathit{B}$ using $R=4$ components and obtained with the DisjointParafacALS algorithm for Kojima data.

Scale	Scale Type	COMP1	COMP2	COMP3	COMP4
(1) Acceptance	PSA	0.52	0	0	0
(2) Child centredness	PSA	0.49	0	0	0
(3) Possessiveness	BPC	0	0.34	0	0
(4) Rejection	R	0	0	0.55	0
(5) Control	BPC	0	0.39	0	0
(6) Enforcement	BPC	0	0.37	0	0
(7) Positive involvement	PSA	0.50	0	0	0
(8) Intrusiveness	BPC	0	0.35	0	0
(9) Control through guilt	R	0	0.35	0	0
(10) Hostile control	BPC	0	0.43	0	0
(11) Inconsistent discipline	LD	0	0	0.45	0
(12) Nonenforcement	LD	0	0	0	0.57
(13) Acceptance individuation	PSA	0.50	0	0	0
(14) Lax discipline	LD	0	0	0	0.59
(15) Instilling persistent anxiety	BPC	0	0.41	0	0
(16) Hostile detachment	R	0	0	0.50	0
(17) Withdrawal of relations	R	0	0	0.49	0
(18) Extreme autonomy	LD	0	0	0	0.57

reports what happens if four disjoint components are computed with the DisjointParafacALS algorithm. The components 1, 2, 3 and 4 group “parental support or acceptance”, “behavioral and psychological control”, “rejection" and “lax discipline” scales, respectively.

5.3. Applying the DisjointParafacALS Algorithm to Tongue Data

In [32], a study was carried out with the Parafac model about the position of certain points of the tongue while some speakers pronounced different vowels in English. These data were stored in the $10\times 13\times 5$ three-way table $\underline{\mathit{X}}$. The first mode corresponded to 10 vowels in English, the second mode corresponded to 13 predefined points of the tongue and the third mode corresponded to five speakers. To generate the data, x-rays were taken on five people while they pronounced the 10 vowels. The full data set is in [32].

Two components were chosen ($R=2$) and a centering was carried out in the first mode. In the reported computational experiments, the following values were used: $\mathtt{ALSMaxIter}=$ 10,000 and Tol $={10}^{-6}$.

Table 22. Comparison of fit and runtime for tongue data.

Disjoint Orthogonal Components	Fit (in %)	Runtime (in Min)
None (ParafacALS)	92.62	0.0016
$\mathit{A}$ (DisjointParafacALS)	86.62	0.0913
$\mathit{B}$ (DisjointParafacALS)	85.23	0.1432
$\mathit{C}$ (DisjointParafacALS)	85.31	0.0765

presents the fit obtained and the execution time in four scenarios. In the first one, no disjoint orthogonal components were computed. From the component correlation matrix observed in

Table 23. Component correlation matrix $\mathsf{\Psi }$ with $R=2$ components for tongue data.

	COMP1	COMP2
COMP1	1	0.06890887
COMP2	0.06890887	1

, the presence of the degeneracy problem is not suspected now. In the other three scenarios, disjoint orthogonal components were calculated for the loading matrices $\mathit{A}$, $\mathit{B}$ and $\mathit{C}$, respectively. In addition, in the last three scenarios, the component correlation matrix $\mathsf{\Psi }$ is the $2\times 2$ identity matrix.

From

Table 24. Loading matrix $\mathit{A}$ with the ParafacALS algorithm for tongue data.

Vowel	COMP1	COMP2
1	1.52	0.69
2	0.97	0.33
3	1.04	0.42
4	0.77	−0.17
5	0.33	−0.53
6	−0.31	−2.04
7	−1.07	−1.24
8	−1.38	0.26
9	−1.05	0.58
10	−0.83	1.69

, observe that vowels 1, 2, 3 and 4 are better represented in the first component with positive loadings. Furthermore, vowels 8 and 9 were also better represented in the first component but with negative loadings. In addition, vowels 5 and 6 were better represented in the second component with negative loadings and vowel 10 was better represented in the second component with positive loading. When interpreting the disjoint components of

Table 25. Disjoint orthogonal matrix $\mathit{A}$ with the DisjointParafacALS algorithm for tongue data.

Vowel	COMP1	COMP2
1	0.55	0
2	0.33	0
3	0.37	0
4	0.19	0
5	0	−0.12
6	0	−0.91
7	−0.51	0
8	−0.35	0
9	−0.20	0
10	0	0.40

, we conclude the same as in [32]. Vowel 7 had loadings less than $-1$ in both components in Table 24, but the DisjointParafacALS algorithm placed vowel 7 in the first component with negative loading.

Table 26. Loading matrix $\mathit{B}$ with the ParafacALS algorithm for tongue data.

Tongue Point	COMP1	COMP2
4	−0.60	−1.19
5	−0.88	−1.09
6	−1.05	−0.95
7	−1.16	−0.59
8	−1.14	−0.01
9	−0.95	0.46
10	−0.55	0.93
11	0.14	1.24
12	0.87	1.42
13	1.26	1.37
14	1.41	1.22
15	1.32	0.93
16	0.91	0.49

shows the components calculated in [32]. The first component was mainly related to large movements of the points of the tongue located in the front of the mouth, while the second component was mainly associated with larger movements of the points of the tongue located in the back of the mouth. In the analysis with disjoint components (see

Table 27. Disjoint orthogonal matrix $\mathit{B}$ with the DisjointParafacALS algorithm for tongue data.

Tongue Point	COMP1	COMP2
4	−0.25	0
5	−0.30	0
6	−0.31	0
7	−0.29	0
8	0	−0.66
9	0	−0.61
10	0	−0.43
11	0.18	0
12	0.34	0
13	0.40	0
14	0.41	0
15	0.36	0
16	0.23	0

), note that the tongue points 11 to 16, which were the closest to the mouth, were represented in the first component with positive loadings. Tongue points 4 to 7, which were the points that are further from the mouth, were in the first component with negative loadings. Furthermore, the tongue points 8, 9 and 10 located in the back of the mouth were represented in the second component with negative loadings.

As a reference, keep in mind that the tongue point 4 was located near the epiglottis (in the throat behind the tongue) and the other tongue points were located upwards in the direction of the mouth ending at point 16 which was located near the lips.

In

Table 28. Loading matrix $\mathit{C}$ with the ParafacALS algorithm for tongue data.

Speaker	COMP1	COMP2
1	−0.43	−0.27
2	−0.70	−0.36
3	−0.37	−0.37
4	−0.34	−0.49
5	−0.47	−0.31

, the speakers 1, 2 and 5 have negative loadings and are better represented in the first component. Furthermore, speaker 4 also has negative loading and is better represented in the second component, just as we see in

Table 29. Disjoint orthogonal matrix $\mathit{C}$ with the DisjointParafacALS algorithm for tongue data.

Speaker	COMP1	COMP2
1	−0.46	0
2	−0.72	0
3	0	−0.66
4	0	−0.75
5	−0.52	0

where we have the disjoint components. Speaker 3 has similar values in the two components of Table 28, but the DisjointParafacALS algorithm places speaker 3 in the second component with negative loading.

6. Conclusions, Other Potential Applications, and Future Work

One of the most popular models for component analysis in three-way tables is the Parafac model. However, a big problem with the existing techniques in the Parafac model is that interpretable loading matrices are not always obtained.

In the present study, a novel heuristic algorithm for computing disjoint orthogonal components in a three-way table with the Parafac model was proposed. This algorithm mixes the ParafacALS and disjoint PCA algorithms. The computational experiments reported that the main benefit of the algorithm is the easier interpretation of the loading matrices. In addition, the algorithm permited us to analyze a three-way table when the degeneracy problem exists. Such experiments indicated that the algorithm identifies and catches disjoint structures in a three-way table with the Parafac model. These experiments based on simulated and real data sets enabled us to demonstrate the good performance and potential applications of the new algorithm. This algorithm may be a helpful multivariate tool for practitioners, applied statisticians, and data scientists.

A limitation of our algorithm is that it does not guarantee an optimal solution. When additional constraints to those present in the original problem are absent, the feasible solutions state the global optimum. However, if the constraints of the DisjointParafacALS algorithm are incorporated, the feasible solutions are compressed, reaching a solution close to the global optimum. Therefore, the fit for the solution given by the algorithm is less than the fit of the ParafacALS algorithm. Nevertheless, the built-in constraints permit us to add zeros in the positions of the variables with low contribution in a component of the loading matrix, allowing the interpretation of the components more clearly. The new algorithm takes longer than the ParafacALS algorithm, even when the degeneracy problem is present in the data, so there is a trade-off between interpretation and speed.

In order to motivate practitioners, the new algorithm can be applied to:

(i)

Marketing and sales, for example in the selection of an appropriate commercial spoken for a given brand [8,21].

(ii)

The Parafac model also has applications in chemistry (chromatography, fluorescence spectroscopy, food industry, and second-order calibration) [33,34,35].

(iii)

There are also applications in environmental sciences related to water [36].

(iv)

The Parafac model is also used in psychometry [11,31].

Some open problems that arose from this study and that authors are analyzing are the following. We think that the disjoint method may be employed together with other existing methods to improve its performance, as in the multidimensional scaling of three-way data. Also, bootstrapping for the loading matrices of the Parafac model may be used. In addition, there exists a wide field of usages in statistical learning. Computing disjoint orthogonal components in higher order tensors is also a challenge.