Fast DCT-VIII Algorithms for Short-Length Input Sequences

Abstract

Discrete cosine transforms (DCTs) are widely used in intelligent electronic systems for storing, processing, and transmitting data. Their popularity stems, on the one hand, from their unique properties and, on the other hand, from the availability of fast algorithms that minimize the computational and hardware complexity of their implementation. Until recently, the Type VIII DCT had been one of the least studied variants, with virtually no publications addressing fast algorithms for its implementation. However, this situation has changed, making the development of efficient implementation methods for this transform a timely and important research problem. In this paper, several algorithmic solutions for implementing the Type VIII DCT are proposed. A set of Type VIII DCT algorithms for small lengths N = 3, 4, 5, 6, 7 is presented. The effectiveness of the proposed solutions is due to the possibility of successful factorization of small-sized DCT-VIII matrices, leading to a reduction in the computational complexity of implementing transforms of this type. Compared with direct matrix–vector computation, the proposed algorithms achieve an approximate 53% reduction in the number of multiplications, at the cost of an increase of about 21% in the number of additions. This work continues a series of previously published studies aimed at creating a library of small-sized algorithms for discrete trigonometric transforms.

1. Introduction

The discrete cosine transform (DCT) has long been widely used to address a variety of data processing tasks [1,2,3,4,5]. It has been applied, for example, to the computation of three-dimensional DCTs for high-definition television encoding [6] and to multicarrier modulators as an alternative to DFT-based systems for orthogonal frequency-division multiplexing and discrete multitone modulation [7]. Moreover, the DCT is used in the encoding and decoding of transformer neural networks [8], instance segmentation [9], image compression [10,11], and video coding [12,13,14]. It is also employed in real-time applications such as audio denoising [15,16], data filtering [17], and feature extraction [18].

The widespread use of the DCT stems from the theoretical result that, in terms of energy compaction, the conventional DCT closely approximates the optimal, signal-dependent Karhunen-Loève transform. This approximation holds when the underlying signal can be modeled as a first-order stationary Markov process. However, natural signals and images often exhibit complex structures and dynamics that do not strictly satisfy this first-order Markov assumption [19].

For this reason, the literature distinguishes eight types of the DCT, namely DCT types I-VIII [1,20,21]. In this classification, the conventional DCT corresponds to the type-II DCT (DCT-II). Recently, a set of DCTs and discrete sine transforms (DSTs) has been incorporated into the Versatile Video Coding (VVC) standard [3,5,13,22,23]. In particular, the Adaptive Multiple Transform (AMT) scheme was introduced to encode residual signals in inter-coded blocks. Depending on the coding mode, the encoder selects, for each block, a transform pair from a predefined pool of DCT/DST candidates [4,23]. The newly introduced transforms include the DCT-VIII and the type-VII DST (DST-VII) for multiple block sizes. Extended AMT proposals also considered the type-V DCT and type-I DST; however, these were not adopted due to negligible rate-distortion gains and higher computational complexity compared with DCT-II, DST-VII, and DCT-VIII. Consequently, for inter-predicted blocks, a fixed transform set consisting of DCT-II, DST-VII, and DCT-VIII is used. Since AMT evaluates two candidates along both horizontal and vertical directions, it supports 15 valid transform pairs per block, which significantly increases encoder complexity [4,23].

Consequently, numerous studies have focused on the development of fast algorithms for computing DCTs and DSTs. Such algorithms reduce the number of multiplications and additions compared with direct two-dimensional matrix multiplication, which requires $\mathcal{O}\left(N^2\right)$ arithmetic operations. However, most existing papers concentrate on DCT-II implementations [20,24,25], while relatively few studies investigate the relationships among DST-VII, DCT-VIII, and other trigonometric transforms or propose low-complexity algorithms for these transforms [4,26,27,28].

This article focuses on the development of fast algorithms for the DCT-VIII. As discussed in the literature [3,5,13,22,23], the DCT-VIII has a wide range of applications in image and video coding; however, its computational cost remains a significant challenge. The following section briefly reviews existing low-complexity algorithms for the DCT-VIII.

1.1. State of the Art of the Problem

Existing fast DCT-VIII algorithms have been developed either in the spectral domain, based on fast DFT/DCT/DST algorithms [4,26,27,28], or by exploiting repeated entries and structural properties of DCT-VIII matrices [12,29,30,31].

The first strategy was adopted in [4], where the DCT-VIII was decomposed into a preprocessing matrix, a DST-VII core, and a post-processing matrix by exploiting the linear relationship between the DCT-VIII and DST-VII. Fast computation was further achieved by leveraging the relationship between the DST-VII and the DFT. To efficiently compute DFTs for multiple DCT-VIII block sizes, prime-factor and Winograd algorithms were employed for $N=4,8,16,$ and 32. The resulting integer DCT-VIII kernels were approximated using norm scaling and bit shifts to ensure compatibility with quantization at each stage of video coding. As a result, the proposed integer DCT-VIII algorithms significantly reduced overall computational complexity, with only minor losses in Bjøntegaard Delta Bitrate (BD-rate). In particular, the numbers of additions and multiplications were reduced by 38% and 80.3%, respectively.

Based on the relationship between the DCT-VIII and DST-VII [1,26], fast DST-VII algorithms derived from DFT methods [28] can be applied to DCT-VIII computation. For instance, in [27], the DCT-VIII was expressed in terms of DST-VII using simple operations such as permutation and sign inversion. This shows that existing DST-VII algorithms can be reused for the DCT-VIII, eliminating the need to develop separate fast algorithms for each transform. Accordingly, [27] employed Winograd factorization of the DFT matrix to factorize the DCT-VIII matrices for $N=4$ and $N=8$.

The main advantage of the first strategy for fast DCT-VIII algorithms is increased codec flexibility, which is crucial for advanced video coding schemes such as AMT. The relationships between transforms enable the implementation of multiple odd-type transforms simultaneously, simplifying codec design.

However, the first strategy supports the development of fast DCT-VIII algorithms only for limited transform sizes, such as $N=4,8,16,$ and 32. While reorderings and sign inversions are computationally inexpensive, the underlying base transform (DFT or DST-VII) still needs to be implemented efficiently. For integer transforms, additional issues may arise related to approximation accuracy and quantization.

The second strategy exploits repeated elements in the rows or columns of the transform matrix [12,29]. These repetitions allow multiple multiplications by coefficients of the same absolute value within a basis row to be combined into a single multiplication. For instance, [12,29] identified three structural patterns within the DST-VII matrix columns suitable for fast algorithm design. The patterns are non-overlapping, meaning that each column follows exactly one pattern:

• A pattern consisting of several groups with a fixed number of elements, regardless of sign changes, where an identity exists among the sums within each group (e.g., the sum of two elements equals another element).
• A pattern in which a subset of column elements is repeated across other columns through permutation.
• A pattern in which a column contains only a single unique value, ignoring sign changes and zeros.

It has been shown that the proposed algorithms for N = 16, 32, and 64 in the state-of-the-art codec can provide average overall decoding time savings of 7% and 5% under All Intra and Random Access configurations, respectively.

In [30], the linearity of DCT-II, DST-VII, and DCT-VIII was exploited to reduce decoder complexity. Leveraging this property, the inverse transform can be accelerated by dividing a block into subblocks containing a single non-zero coefficient and summing their individual inverse transforms, rather than performing a full inverse transform on the entire block. The authors compared this linearity-based approach with the standard fast inverse transforms used in VVC (VTM-8.2) to determine when to switch methods. A precomputed threshold was applied: if the number of non-zero coefficients was below the threshold, the linearity-based method was used; otherwise, the conventional inverse transform was applied. Decoding time savings reached 4% in All-Intra and approximately 10% in Random Access compared to the standard VVC inverse transform.

Thus, the arithmetic complexity of fast DCT-VIII algorithms based on the second strategy largely depends on the structure of the transform matrix. In particular, it is affected by the repetition and placement of the matrix entries. The relative performance gains vary with matrix size. For very large transforms, the advantage becomes less pronounced compared to direct matrix-vector multiplication.

1.2. The Main Contributions of the Paper

To overcome the limitations of existing strategies for designing fast DCT-VIII algorithms, we propose using the structural approach [32,33] to reduce the computational complexity of this transform. Over time, we have refined this approach and successfully applied it to develop efficient algorithms for discrete trigonometric transforms [34]. Unlike other methods, the structural approach does not rely on specific analytical properties of the transform. Instead, it depends solely on the structure of the transform matrix, particularly the repetition and arrangement of its elements. This makes the structural approach applicable to a wide variety of transform matrices.

According to the structural approach, the transform matrix is first preprocessed by permuting selected rows and columns and changing the signs of certain entries. Submatrices of the resulting matrix are then matched to templates defined in [32,33], which represent matrix patterns with known factorizations, such as butterfly-type or circular-convolution-type patterns. The factorizations of these submatrices are subsequently combined to produce a factorization of the original transform matrix. This factorization is then used to construct the corresponding fast transform algorithm, which is represented by a data flow graph.

Unlike the first strategy, which relies on relationships between DCT-VIII and other trigonometric transforms, the structural approach reduces the number of arithmetic operations required for the direct matrix-vector product. Moreover, it can be applied to matrices of any size, not just those that are powers of two.

Compared to the second strategy, the structural approach generalizes and formalizes the ideas presented in [12,29]. The first pattern corresponds to a butterfly-type module, the second pattern represents a circular convolution, and the third pattern corresponds to a fan-shaped fragment of a data flow graph representing the sum of several identical entries. By using the structural approach, it is possible to formalize the rules and patterns from [12,29,30,31] and to develop fast DCT-VIII algorithms for arbitrary input lengths N.

The novelty of this research lies in its departure from conventional approaches to DCT-VIII algorithm design. Existing methods have predominantly relied on spectral and algebraic properties of the transform matrix, such as representations based on the discrete Fourier transform or explicit algebraic relationships among matrix elements. Consequently, prior algorithm designs were driven by analytical expressions describing the spectral characteristics of the transform matrix.

In contrast, this paper proposes, for the first time, the construction of fast DCT-VIII algorithms based on structural pattern recognition. Specifically, structural patterns of submatrices within the DCT-VIII transform matrix are identified and exploited, following the framework introduced in the foundational studies [32,33]. This approach emphasizes the direct recognition and factorization of recurring structural configurations, rather than relying on spectral or algebraic interpretations of the transform.

The structured approach applied in this paper for the construction of fast DCT-VIII algorithms offers the following practical benefits:

• Using the matrix factorizations and data flow graphs for submatrix structures introduced in [32,33], it is easy to construct factorizations and data flow graphs for fast DCT-VIII algorithms with different values of N.
• The structural patterns defined in [32,33] enable the construction of fast DCT-VIII algorithms applicable to arbitrary values of N, including small transform sizes, without restricting the transform matrix to lengths that are powers of two.
• Subgraphs corresponding to the data flow graphs of individual DCT-VIII submatrices can be reused as building blocks when constructing the data flow graph of an N-point fast DCT-VIII algorithm.

The primary contributions of this research are as follows:

• For selected matrix sizes, we have developed factorizations of DCT-VIII matrices into sparse and diagonal matrices for input lengths from 3 to 7. These factorizations were obtained using two techniques: comparison with patterns from [32] and identification of circular-convolution structures [33].
• The correctness of these factorizations has been verified both mathematically and through implementation in MATLAB R2023b.
• Based on the DCT-VIII matrix factorizations, we have designed efficient algorithms for this transform using data flow graphs. A unique feature of these graphs is that each path from an input vertex to an output vertex contains only one multiplication, which reduces both processing time and resource usage.

The remainder of the paper is organized as follows. Section 2 presents the mathematical background of the DCT-VIII. Section 3 derives factorizations of DCT-VIII matrices and introduces fast DCT-VIII algorithms using data flow graphs. Section 4 provides a comparative analysis of the arithmetic operations required by the proposed algorithms versus direct matrix-vector multiplication. Section 5 compares the structure of the proposed algorithms with existing methods developed using the same strategy. Finally, Section 6 concludes the paper with a comprehensive summary of the research.

2. Preliminary Background

2.1. Obtaining a Matrix with the DCT-VIII Coefficients

The DCT-VIII that we used in our solutions can be calculated as follows [1]:

$$y_k={\displaystyle \frac{2}{\sqrt{2N+1}}}\sum _{n=0}^{N-1}{x}_{n}cos\left({\displaystyle \frac{\left(k+0.5\right)\left(n+0.5\right)2\pi }{2N+1}}\right),$$

(1)

where

• k$=0,1,\dots ,N-1$,
• ${\mathit{y}}_k$ is the DCT-VIII transform coefficients,
• ${\mathit{x}}_n$ is input data, and
• N is a number of signal samples.

In matrix notation, the DCT-VIII can be represented as follows:

$${\mathbf{Y}}_{N\times 1}={\mathbf{C}}_N{\mathbf{X}}_{N\times 1},$$

(2)

where

$${\mathbf{Y}}_{N\times 1}={\left[y_0,y_1,\dots ,y_{N-1}\right]}^{\mathrm{T}},{\mathbf{X}}_{N\times 1}={\left[x_0,x_1,\dots ,x_{N-1}\right]}^{\mathrm{T}},$$

$$c_{k,n}={\displaystyle \frac{2}{\sqrt{2N+1}}}cos\left({\displaystyle \frac{\left(k+0.5\right)\left(n+0.5\right)2\pi }{2N+1}}\right),\mathrm{where}k,n=0,\dots ,N-1.$$

The DCT-VIII in matrix notation is as follows:

$$\left[\begin{array}{c}{y}_0\\ y_1\\ ⋮\\ y_{N-1}\end{array}\right]=\left[\begin{array}{cccc}{c}_{0,0}& c_{0,1}& \dots & c_{0,N-1}\\ c_{1,0}& c_{1,1}& \dots & c_{1,N-1}\\ ⋮& ⋮& \ddots & ⋮\\ c_{N-1,0}& c_{N-1,1}& \dots & c_{N-1,N-1}\end{array}\right]\left[\begin{array}{c}{x}_0\\ x_1\\ ⋮\\ x_{N-1}\end{array}\right].$$

The notations employed in this work are summarized in

Table 1. The notations used in this research.

Notation	Meaning
⊕	direct sum of two matrices
⊗	Kronecker product of two matrices
${\mathbf{H}}_2$	$2\times 2$ Hadamard matrix
${\mathbf{I}}_N$	identity matrix of order N
${\mathbf{P}}_N$	permutation matrix
$s_m^{\left(N\right)}$	multiplier
${\mathbf{D}}_N$	diagonal matrix of size $N\times N$
${\mathbf{W}}_{M\times N}$ and ${\mathbf{W}}_N$	M × N and N × N matrices describing pre-additions and post-additions, respectively
	An empty cell in a matrix indicates a zero entry

.

2.2. The Method of Creating Data Flow Graphs

In this work, we use a method for drawing data flow graphs. In this subsection, we explain how these graphs are constructed and how to interpret them for clarity.

Values of an input vertex ${\mathit{x}}_n$ are placed on the left side of the graph, while output values ${\mathit{y}}_n$ appear on the right. Solid lines indicate data flow, and dashed lines indicate data flow with a sign change. Rectangular blocks labeled ${\mathbf{H}}_2$ represent Hadamard matrices, and circles represent multipliers.

When drawing a graph from a specific expression, the process is reversed. The last matrix in the expression is drawn first from the left, while the first matrix is drawn first from the right.

2.3. Common Matrices for Different Solutions

For clarity, we present below the matrices that may be involved in several of the solutions:

$${\mathbf{T}}_{2\times 3}^{\left(3\right)}=\left[\begin{array}{ccc}0& 1& 1\\ 1& 0& 1\end{array}\right],{\mathbf{T}}_{2\times 3}^{\left(4\right)}=\left[\begin{array}{ccc}1& 0& 1\\ 0& 1& 1\end{array}\right],{\mathbf{T}}_{3\times 2}^{\left(3\right)}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right],$$

$${\mathbf{P}}_3^{\left({\pi }_3^{\left(0\right)}\right)}=\left[\begin{array}{ccc}& & 1\\ & 1& \\ 1& \end{array}\right],{\mathbf{H}}_2=\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right],$$

$${\mathbf{W}}_3^{\left(0\right)}=\left[\begin{array}{ccc}1& 1& 1\\ 1& & -1\\ & 1& -1\end{array}\right],{\mathbf{A}}_{4\times 3}=\left[\begin{array}{ccc}1& & \\ & 1& \\ & & 1\\ & 1& 1\end{array}\right],$$

$${\mathbf{A}}_{3\times 4}=\left[\begin{array}{cccc}1& & & \\ & 1& & -1\\ & & 1& -1\end{array}\right],{\mathbf{W}}_3^{\left(1\right)}=\left[\begin{array}{ccc}1& 1& \\ 1& -1& -1\\ 1& & 1\end{array}\right].$$

2.4. The Steps for Constructing Fast DCT-VIII Algorithms

Let us outline the procedure for constructing fast DCT-VIII algorithms based on the structural approach [32,33].

Step 1. Permutation of rows and/or columns of the initial transform matrix. At this stage, the appropriate permutation matrices are constructed.

Step 2. Extraction of identical matrix entries. The permuted matrix is represented as the sum of two matrices. The first matrix contains entries that are identical up to a sign change and is excluded from further processing to reduce the number of arithmetic operations. The second matrix contains the remaining entries of the permuted matrix.

Step 3. Identification and factorization of structural matrix patterns. Submatrices matching predefined structural patterns are extracted from the second matrix obtained in Step 2. Each extracted submatrix is factorized according to the corresponding pattern factorizations described in [32].

Step 4. Factorization of the initial transform matrix. If necessary, Steps 1–3 are recursively applied to the extracted submatrices. Using the resulting submatrix factorizations, the factorization of the original DCT-VIII matrix is constructed, including the permutation matrices obtained in Step 1.

Step 5. Enhancement of the resulting factorization. The final factorization of the transform matrix is refined to further reduce the number of addition operations.

3. The Fast DCT-VIII Algorithms

3.1. Fast Algorithm for the 3-Point DCT-VIII

Below, we present the matrix-vector form of the calculation of the DCT-VIII for N = 3:

$${\mathbf{Y}}_{3\times 1}={\mathbf{C}}_3{\mathbf{X}}_{3\times 1},$$

(3)

where

$${\mathbf{Y}}_{3\times 1}={\left[y_0,y_1,y_2\right]}^{\mathrm{T}},{\mathbf{X}}_{3\times 1}={\left[x_0,x_1,x_2\right]}^{\mathrm{T}},$$

$$\begin{array}{cc}\hfill {\mathbf{C}}_3=\left[\begin{array}{ccc}{a}_3& b_3& c_3\\ b_3& -c_3& -a_3\\ c_3& -a_3& b_3\end{array}\right],& \begin{array}{c}{a}_3=0.7370,\hfill \\ b_3=0.5910,\hfill \\ c_3=0.3280.\hfill \end{array}\hfill \end{array}$$

Now, we need to define the ${\pi }_3^{\left(0\right)}$ permutation as below:

$${\pi }_3^{\left(0\right)}=\left(\begin{array}{ccc}1& 2& 3\\ 3& 2& 1\end{array}\right).$$

We then apply ${\pi }_3^{\left(0\right)}$ to the matrix ${\mathbf{C}}_3$ in columns. Additionally, we multiply the third row and third column by −1:

$${\mathbf{C}}_3^{\left(\mathrm{P}\right)}={\mathbf{P}}_3^{\left(\mathrm{S}\right)}{\mathbf{C}}_3{\mathbf{P}}_3^{\left(\mathrm{S}\right)}{\mathbf{P}}_3^{\left({\pi }_3^{\left(0\right)}\right)},\mathrm{where}{\mathbf{P}}_3^{\left(\mathrm{S}\right)}=\left[\begin{array}{ccc}1& & \\ & 1& \\ & & -1\end{array}\right].$$

The matrix ${\mathbf{C}}_3^{\left(\mathrm{P}\right)}$ has the following structure:

$${\mathbf{C}}_3^{\left(\mathrm{P}\right)}=\left[\begin{array}{ccc}-c_3& b_3& a_3\\ a_3& -c_3& b_3\\ b_3& a_3& -c_3\end{array}\right].$$

This structure corresponding to a circular convolution [33]. We use this property and derive the final expression of the DCT-VIII algorithm for N = 3:

$${\mathbf{Y}}_{3\times 1}={\mathbf{P}}_3^{\left(\mathrm{S}\right)}{\mathbf{W}}_3^{\left(1\right)}{\mathbf{A}}_{3\times 4}{\mathbf{D}}_4{\mathbf{A}}_{4\times 3}{\mathbf{W}}_3^{\left(0\right)}{\mathbf{P}}_3{\mathbf{X}}_{3\times 1},$$

(4)

where

$${\mathbf{P}}_3={\left({\mathbf{P}}_3^{\left(\mathrm{S}\right)}{\mathbf{P}}_3^{\left({\pi }_3^{\left(0\right)}\right)}\right)}^{\mathrm{T}},{\mathbf{D}}_4=\mathrm{diag}\left(s_0^{\left(3\right)},s_1^{\left(3\right)},s_2^{\left(3\right)},s_3^{\left(3\right)}\right),s_0^{\left(3\right)}={\displaystyle \frac{\left(-c_3+a_3+b_3\right)}{3}},$$

$$s_1^{\left(3\right)}=-c_3-b_3,s_2^{\left(3\right)}=a_3-b_3,s_3^{\left(3\right)}={\displaystyle \frac{\left(-c_3+a_3-2b_3\right)}{3}}.$$

Figure 1 presents the data flow graph of the fast DCT-VIII algorithm for N = 3. Direct matrix-vector product requires 9 multiplications and 6 additions. Our solution requires 4 multiplications and 11 additions. The pseudocode for the DCT-VIII for $N=3$ is provided in

Table A1. The pseudocode for the developed fast 3-point DCT-VIII algorithm with variable reuse.

Step 1	Step 2	Step 3	Step 4
$y_0=-x_2+x_1+x_0$	$y_1=y_1{s}_1^{\left(3\right)}$	$p_2=y_0{s}_0^{\left(3\right)}$	$y_0=p_2+y_1$
$y_1=-x_2-x_0$	$y_2=y_2{s}_2^{\left(3\right)}$	$y_1=y_1-p_1$	$y_1=p_2-y_1-y_2$
$y_2=x_1-x_0$	$p_1=p_0{s}_3^{\left(3\right)}$	$y_2=y_2-p_1$	$y_2=-\left(p_2+y_2\right)$
$p_0=y_1+y_2$

.

Figure 2 illustrates the data flow graph of the DCT-VIII for an input size of 3, including labels for the individual nodes.

Table 2. Step-by-step verification of the computations associated with selected nodes in the DCT-VIII data flow graph for $N=3$.

Step 1	Step 2	Step 3	Step 4
$a_0=-x_2+x_1+x_0$	$c_0=a_1{s}_1^{\left(3\right)}$	$d_0=a_0{s}_0^{\left(3\right)}$	$e_0=d_0+d_1$
$a_1=-x_2-x_0$	$c_1=a_2{s}_2^{\left(3\right)}$	$d_1=c_0-c_2$	$e_1=d_0-d_1-d_2$
$a_2=x_1-x_0$	$c_2=b_0{s}_3^{\left(3\right)}$	$d_2=c_1-c_2$	$e_2=d_0+d_2$
$b_0=a_1+a_2$

provides the corresponding calculations, enabling step-by-step verification of the correctness of the derivation.

3.2. Fast Algorithm for the 4-Point DCT-VIII

Below, we present the matrix-vector form of the calculation of the DCT-VIII for N = 4:

$${\mathbf{Y}}_{4\times 1}={\mathbf{C}}_4{\mathbf{X}}_{4\times 1},$$

(5)

where

$${\mathbf{Y}}_{4\times 1}={\left[y_0,y_1,y_2,y_3\right]}^{\mathrm{T}},{\mathbf{X}}_{4\times 1}={\left[x_0,x_1,x_2,x_3\right]}^{\mathrm{T}},$$

$$\begin{array}{ccc}\hfill {\mathbf{C}}_4=\left[\begin{array}{cccc}{a}_4& b_4& c_4& d_4\\ b_4& 0& -b_4& -b_4\\ c_4& -b_4& -d_4& a_4\\ d_4& -b_4& a_4& -c_4\end{array}\right],& \begin{array}{c}{a}_4=0.6565,\hfill \\ \\ b_4=0.5774,\hfill \end{array}\hfill & \hfill \begin{array}{c}{c}_4=0.4285,\hfill \\ \\ d_4=0.2280.\hfill \end{array}\end{array}$$

Now, we need to define the ${\pi }_4^{\left(0\right)}$ permutation as below:

$${\pi }_4^{\left(0\right)}=\left(\begin{array}{cccc}1& 2& 3& 4\\ 1& 4& 3& 2\end{array}\right).$$

We then apply ${\pi }_4^{\left(0\right)}$ to the matrix ${\mathbf{C}}_4$ in both rows and columns:

$${\mathbf{C}}_4^{\left(\mathrm{P}\right)}={\mathbf{P}}_4^{\left({\pi }_4^{\left(0\right)}\right)}{\mathbf{C}}_4{\mathbf{P}}_4^{\left({\pi }_4^{\left(0\right)}\right)},$$

where

$${\mathbf{P}}_4^{\left({\pi }_4^{\left(0\right)}\right)}=\left[\begin{array}{cccc}1& & & \\ & & & 1\\ & & 1& \\ & 1& & \end{array}\right].$$

After this operation, the matrix ${\mathbf{C}}_4^{\left(\mathrm{P}\right)}$ has the following structure:

$${\mathbf{C}}_4^{\left(\mathrm{P}\right)}=\left[\begin{array}{cccc}{a}_4& d_4& c_4& b_4\\ d_4& -c_4& a_4& -b_4\\ c_4& a_4& -d_4& -b_4\\ b_4& -b_4& -b_4& 0\end{array}\right].$$

Let us write the matrix ${\mathbf{C}}_4^{\left(\mathrm{P}\right)}$ as the sum of two matrices:

$${\mathbf{C}}_4^{\left(\mathrm{P}\right)}={\mathbf{C}}_4^{\left(\mathrm{A}\right)}+{\mathbf{C}}_4^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{C}}_4^{\left(\mathrm{A}\right)}=\left[\begin{array}{cccc}{a}_4& d_4& c_4& 0\\ d_4& -c_4& a_4& 0\\ c_4& a_4& -d_4& 0\\ 0& 0& 0& 0\end{array}\right],{\mathbf{C}}_4^{\left(\mathrm{B}\right)}=\left[\begin{array}{cccc}0& 0& 0& b_4\\ 0& 0& 0& -b_4\\ 0& 0& 0& -b_4\\ b_4& -b_4& -b_4& 0\end{array}\right].$$

The matrix ${\mathbf{C}}_4^{\left(\mathrm{A}\right)}$, after removing the fourth row and the fourth column, takes the form:

$${\mathbf{C}}_3=\left[\begin{array}{ccc}{a}_4& d_4& c_4\\ d_4& -c_4& a_4\\ c_4& a_4& -d_4\end{array}\right].$$

We now need to apply the ${\pi }_3^{\left(0\right)}$ permutation to the matrix ${\mathbf{C}}_3$ and multiply the first row and the second and third columns by −1. These operations are described below:

$${\mathbf{C}}_3^{\left(\mathrm{P}\right)}={\mathbf{P}}_3^{\left({\pi }_3^{\left(0\right)}\right)}{\mathbf{P}}_3^{\left(\mathrm{R}\right)}{\mathbf{C}}_3{\mathbf{P}}_3^{\left(\mathrm{C}\right)},$$

where

$${\mathbf{P}}_3^{\left(\mathrm{C}\right)}=\left[\begin{array}{ccc}1& & \\ & -1& \\ & & -1\end{array}\right],{\mathbf{P}}_3^{\left(\mathrm{R}\right)}=\left[\begin{array}{ccc}-1& & \\ & 1& \\ & & 1\end{array}\right].$$

The matrix ${\mathbf{C}}_3^{\left(\mathrm{P}\right)}$ has the following structure:

$${\mathbf{C}}_3^{\left(\mathrm{P}\right)}=\left[\begin{array}{ccc}{c}_4& -a_4& d_4\\ d_4& c_4& -a_4\\ -a_4& d_4& c_4\end{array}\right].$$

This structure corresponding to a circular convolution [33]. Below, we present the final expression for the four-point DCT-VIII based on the factorization of the matrix ${\mathbf{C}}_4^{\left(\mathrm{B}\right)}$:

$${\mathbf{Y}}_{4\times 1}={\mathbf{P}}_4^{\left({\pi }_4^{\left(0\right)}\right)}{\mathbf{W}}_{4\times 5}^{\left(1\right)}{\mathbf{P}}_5^{\left(\mathrm{R}\right)}{\mathbf{W}}_{5\times 4}^{\left(1\right)}{\mathbf{W}}_{4\times 5}^{\left(0\right)}{\mathbf{D}}_5{\mathbf{W}}_{5\times 4}^{\left(0\right)}{\mathbf{W}}_4{\mathbf{P}}_4^{\left(\mathrm{C}\right)}{\mathbf{P}}_4^{\left({\pi }_4^{\left(0\right)}\right)}{\mathbf{X}}_{4\times 1},$$

(6)

where

$${\mathbf{P}}_4^{\left(\mathrm{C}\right)}=\left[\begin{array}{cc}{\mathbf{P}}_3^{\left(\mathrm{C}\right)}& \\ & 1\end{array}\right],{\mathbf{W}}_4=\left[\begin{array}{cccc}1& & -1& \\ & 1& -1& \\ & & & 1\\ 1& 1& 1& \end{array}\right],{\mathbf{W}}_{5\times 4}^{\left(0\right)}=\left[\begin{array}{cccc}1& & & \\ & 1& & \\ 1& 1& & \\ & & 1& \\ & & & 1\end{array}\right],$$

$${\mathbf{D}}_5=\mathrm{diag}\left(s_0^{\left(4\right)},s_1^{\left(4\right)},\dots ,s_4^{\left(4\right)}\right),s_0^{\left(4\right)}=c_4+a_4,s_1^{\left(4\right)}=d_4+a_4,s_2^{\left(4\right)}={\displaystyle \frac{c_4+d_4+2a_4}{3}},$$

$$s_3^{\left(4\right)}=s_4^{\left(4\right)}=b_4,{\mathbf{W}}_{4\times 5}^{\left(0\right)}=\left[\begin{array}{ccccc}1& & -1& & \\ & 1& -1& & \\ & & & 1& \\ & & & & 1\end{array}\right],{\mathbf{W}}_{5\times 4}^{\left(1\right)}=\left[\begin{array}{cccc}1& & & \\ -1& -1& & \\ & 1& & \\ & & 1& \\ & & & 1\end{array}\right],$$

$${\mathbf{P}}_5^{\left(\mathrm{R}\right)}=\left[\begin{array}{cc}{\left({\mathbf{P}}_3^{\left({\pi }_3^{\left(0\right)}\right)}{\mathbf{P}}_3^{\left(\mathrm{R}\right)}\right)}^{\left(\mathrm{T}\right)}& \\ & {\mathbf{I}}_2\end{array}\right],{\mathbf{W}}_{4\times 5}^{\left(1\right)}=\left[\begin{array}{ccccc}1& & & 1& \\ & 1& & -1& \\ & & 1& -1& \\ & & & & 1\end{array}\right].$$

Figure 3 presents the data flow graph of the fast DCT-VIII algorithm for N = 4. Direct matrix-vector product requires 15 multiplications and 11 additions. Our solution requires 5 multiplications and 11 additions. The first value $\left({\displaystyle \frac{c_4+d_4-a_4}{3}}\right)$ of the circular convolution matrix equals 0, so it is not specified in Equation (6) or in the data flow graph. The pseudocode for the DCT-VIII for $N=4$ is provided in

Table A2. The pseudocode for the developed fast 4-point DCT-VII algorithm with variable reuse.

Step 1	Step 2	Step 3
$p=x_0+x_2$	$p=p{s}_0^{\left(4\right)}$	$x_2=x_2-x_3$
$x_0=\left(x_0-x_3-x_2\right)s_4^{\left(4\right)}$	$x_2=x_2{s}_1^{\left(4\right)}$	$x_3=-x_2-p-x_1$
$x_2=x_2-x_3$	$x_1=x_1{s}_3^{\left(4\right)}$	$x_2=x_1-x_2$
$x_3=\left(p+x_2\right)s_2^{\left(4\right)}$	$p=p-x_3$	$x_1=p-x_1$

.

3.3. Fast Algorithm for the 5-Point DCT-VIII

Below, we present the matrix-vector form of the calculation of the DCT-VIII for N = 5:

$${\mathbf{Y}}_{5\times 1}={\mathbf{C}}_5{\mathbf{X}}_{5\times 1},$$

(7)

where

$${\mathbf{Y}}_{5\times 1}={\left[y_0,y_1,y_2,y_3,y_4\right]}^{\mathrm{T}},{\mathbf{X}}_{5\times 1}={\left[x_0,x_1,x_2,x_3,x_4\right]}^{\mathrm{T}},$$

$$\begin{array}{ccc}\hfill {\mathbf{C}}_5=\left[\begin{array}{ccccc}{a}_5& b_5& c_5& d_5& e_5\\ b_5& e_5& -d_5& -a_5& -c_5\\ c_5& -d_5& -b_5& e_5& a_5\\ d_5& -a_5& e_5& c_5& -b_5\\ e_5& -c_5& a_5& -b_5& d_5\end{array}\right],& \begin{array}{c}{a}_5=0.5969,\hfill \\ \\ b_5=0.5485,\hfill \\ \\ c_5=0.4557,\hfill \end{array}\hfill & \hfill \begin{array}{c}{d}_5=0.3260,\hfill \\ \\ e_5=0.1699.\hfill \end{array}\end{array}$$

Now, we need to define the ${\pi }_5^{\left(0\right)}$ permutation as below:

$${\pi }_5^{\left(0\right)}=\left(\begin{array}{ccccc}1& 2& 3& 4& 5\\ 5& 2& 4& 3& 1\end{array}\right).$$

Let us apply the ${\pi }_5^{\left(0\right)}$ permutation to the matrix ${\mathbf{C}}_5$ and multiply the third row and the third column by −1. These operations are described below:

$${\mathbf{C}}_5^{\left(\mathrm{P}\right)}={\mathbf{P}}_5^{\left(\mathrm{S}\right)}{\mathbf{P}}_5^{\left({\pi }_5^{\left(0\right)}\right)}{\mathbf{C}}_5{\mathbf{P}}_5^{\left(\mathrm{S}\right)},$$

where

$${\mathbf{P}}_5^{\left(\mathrm{S}\right)}=\left[\begin{array}{ccccc}1& & & & \\ & 1& & & \\ & & -1& & \\ & & & 1& \\ & & & & 1\end{array}\right],{\mathbf{P}}_5^{\left({\pi }_5^{\left(0\right)}\right)}=\left[\begin{array}{ccccc}& & & & 1\\ & 1& & & \\ & & & 1& \\ & & 1& & \\ 1& & & & \end{array}\right].$$

After these operations, the matrix ${\mathbf{C}}_5^{\left(\mathrm{P}\right)}$ looks as follows:

$${\mathbf{C}}_5^{\left(\mathrm{P}\right)}=\left[\begin{array}{ccccc}{e}_5& -c_5& -a_5& -b_5& d_5\\ b_5& e_5& d_5& -a_5& -c_5\\ -d_5& a_5& e_5& -c_5& b_5\\ c_5& -d_5& b_5& e_5& a_5\\ a_5& b_5& -c_5& d_5& e_5\end{array}\right].$$

Now, we split the matrix ${\mathbf{C}}_5^{\left(\mathrm{P}\right)}$ into two matrices, which we consider separately, and then combine the factorizations of these two matrices:

$${\mathbf{C}}_5^{\left(\mathrm{P}\right)}={\mathbf{C}}_5^{\left(\mathrm{A}\right)}+{\mathbf{C}}_5^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{C}}_5^{\left(\mathrm{A}\right)}=\left[\begin{array}{ccccc}{e}_5& -c_5& -a_5& -b_5& 0\\ b_5& e_5& d_5& -a_5& 0\\ -d_5& a_5& e_5& -c_5& 0\\ c_5& -d_5& b_5& e_5& 0\\ 0& 0& 0& 0& 0\end{array}\right],{\mathbf{C}}_5^{\left(\mathrm{B}\right)}=\left[\begin{array}{ccccc}0& 0& 0& 0& d_5\\ 0& 0& 0& 0& -c_5\\ 0& 0& 0& 0& b_5\\ 0& 0& 0& 0& a_5\\ a_5& b_5& -c_5& d_5& e_5\end{array}\right].$$

If we omit the zero rows and columns from the matrix ${\mathbf{C}}_5^{\left(\mathrm{A}\right)}$, we get the following 4-by-4 matrix:

$${\mathbf{C}}_4=\left[\begin{array}{cccc}{e}_5& -c_5& -a_5& -b_5\\ b_5& e_5& d_5& -a_5\\ -d_5& a_5& e_5& -c_5\\ c_5& -d_5& b_5& e_5\end{array}\right].$$

The matrix ${\mathbf{C}}_4$ has the following structure:

$${\mathbf{C}}_4=\left[\begin{array}{cc}{\mathbf{A}}_2& {\mathbf{B}}_2\\ {\mathbf{C}}_2& {\mathbf{A}}_2\end{array}\right],$$

where

$${\mathbf{A}}_2=\left[\begin{array}{cc}{e}_5& -c_5\\ b_5& e_5\end{array}\right],{\mathbf{B}}_2=\left[\begin{array}{cc}-a_5& -b_5\\ d_5& -a_5\end{array}\right],{\mathbf{C}}_2=\left[\begin{array}{cc}-d_5& a_5\\ c_5& -d_5\end{array}\right].$$

Then, matrix ${\mathbf{C}}_4$ can be described by the following expression:

$${\mathbf{C}}_4={\mathbf{T}}_{4\times 6}{\mathbf{D}}_6{\mathbf{T}}_{6\times 4},$$

where

$${\mathbf{T}}_{6\times 4}={\mathbf{T}}_{3\times 2}^{\left(3\right)}\otimes {\mathbf{I}}_2,{\mathbf{D}}_6=\left[\begin{array}{ccc}{\mathbf{E}}_2& & \\ & {\mathbf{F}}_2& \\ & & {\mathbf{A}}_2\end{array}\right],{\mathbf{T}}_{4\times 6}={\mathbf{T}}_{2\times 3}^{\left(3\right)}\otimes {\mathbf{I}}_2,$$

$${\mathbf{E}}_2={\mathbf{C}}_2-{\mathbf{A}}_2=\left[\begin{array}{cc}-d_5-e_5& a_5+c_5\\ c_5-b_5& -d_5-e_5\end{array}\right]=\left[\begin{array}{cc}-0.4959& 1.0526\\ -0.0928& -0.4959\end{array}\right],$$

$${\mathbf{F}}_2={\mathbf{B}}_2-{\mathbf{A}}_2=\left[\begin{array}{cc}-a_5-e_5& -b_5+c_5\\ d_5-b_5& -a_5-e_5\end{array}\right]=\left[\begin{array}{cc}-0.7668& -0.0928\\ -0.2225& -0.7668\end{array}\right].$$

Next, we consider three matrices: ${\mathbf{E}}_2$, ${\mathbf{F}}_2$, and ${\mathbf{A}}_2$. They are similar to the pattern [32]: $\left[\begin{array}{cc}a& b\\ c& a\end{array}\right]$. As a result, we obtain the following factorization of the ${\mathbf{C}}_4$ matrix:

$${\mathbf{C}}_4={\mathbf{T}}_{4\times 6}{\mathbf{T}}_{6\times 9}{\mathbf{D}}_9{\mathbf{T}}_{9\times 6}{\mathbf{T}}_{6\times 4},$$

where

$${\mathbf{T}}_{9\times 6}=\left[\begin{array}{ccc}{\mathbf{T}}_{3\times 2}^{\left(3\right)}& & \\ & {\mathbf{T}}_{3\times 2}^{\left(3\right)}& \\ & & {\mathbf{T}}_{3\times 2}^{\left(3\right)}\end{array}\right],{\mathbf{D}}_9=\mathrm{diag}\left(s_0^{\left(5\right)},s_1^{\left(5\right)},\dots ,s_8^{\left(5\right)}\right),$$

$$s_0^{\left(5\right)}=c_5-b_5+d_5+e_5,s_1^{\left(5\right)}=a_5+c_5+d_5+e_5,s_2^{\left(5\right)}=-d_5-e_5,$$

$$s_3^{\left(5\right)}=d_5-b_5+a_5+e_5,s_4^{\left(5\right)}=-b_5+c_5+a_5+e_5,s_5^{\left(5\right)}=-a_5-e_5,$$

$$s_6^{\left(5\right)}=b_5-e_5,s_7^{\left(5\right)}=-c_5-e_5,s_8^{\left(5\right)}=e_5,{\mathbf{T}}_{6\times 9}=\left[\begin{array}{ccc}{\mathbf{T}}_{2\times 3}^{\left(3\right)}& & \\ & {\mathbf{T}}_{2\times 3}^{\left(3\right)}& \\ & & {\mathbf{T}}_{2\times 3}^{\left(3\right)}\end{array}\right].$$

We can now formulate the final matrix-vector procedure for computing DCT-VIII with reduced computational complexity for N = 5, remembering to include the matrix ${\mathbf{C}}_5^{\left(\mathrm{B}\right)}$:

$${\mathbf{Y}}_{5\times 1}={\mathbf{P}}_5{\mathbf{W}}_{5\times 13}{\mathbf{T}}_{13\times 15}{\mathbf{T}}_{15\times 18}{\mathbf{D}}_{18}{\mathbf{T}}_{18\times 15}{\mathbf{T}}_{15\times 13}{\mathbf{W}}_{13\times 5}{\mathbf{P}}_5^{\left(\mathrm{S}\right)}{\mathbf{X}}_{5\times 1},$$

(8)

where

$$\begin{array}{cc}\hfill {\mathbf{W}}_{13\times 5}=\left[\begin{array}{ccccc}1& & & & \\ & 1& & & \\ & & 1& & \\ & & & 1& \\ & & & & 1\\ 1& & & & \\ & 1& & & \\ & & 1& & \\ & & & 1& \\ & & & & 1\\ & & & & 1\\ & & & & 1\\ & & & & 1\end{array}\right],& \begin{array}{c}{\mathbf{T}}_{15\times 13}=\left[\begin{array}{cc}{\mathbf{T}}_{6\times 4}& \\ & {\mathbf{I}}_9\end{array}\right],{\mathbf{T}}_{18\times 15}=\left[\begin{array}{cc}{\mathbf{T}}_{9\times 6}& \\ & {\mathbf{I}}_9\end{array}\right],\hfill \\ \\ {\mathbf{D}}_{18}=\mathrm{diag}\left(s_0^{\left(5\right)},s_1^{\left(5\right)},\dots ,s_{17}^{\left(5\right)}\right),s_9^{\left(5\right)}=e_5,\hfill \\ \\ s_{10}^{\left(5\right)}=s_{17}^{\left(5\right)}=a_5,s_{11}^{\left(5\right)}=s_{16}^{\left(5\right)}=b_5,s_{12}^{\left(5\right)}=s_{15}^{\left(5\right)}=-c_5,\hfill \\ \\ s_{13}^{\left(5\right)}=s_{14}^{\left(5\right)}=d_5,{\mathbf{T}}_{15\times 18}=\left[\begin{array}{cc}{\mathbf{T}}_{6\times 9}& \\ & {\mathbf{I}}_9\end{array}\right],\hfill \\ \\ {\mathbf{T}}_{13\times 15}=\left[\begin{array}{cc}{\mathbf{T}}_{4\times 6}& \\ & {\mathbf{I}}_9\end{array}\right],{\mathbf{P}}_5={\left({\mathbf{P}}_5^{\left(\mathrm{S}\right)}{\mathbf{P}}_5^{\left({\pi }_5^{\left(0\right)}\right)}\right)}^{\mathrm{T}},\hfill \end{array}\hfill \end{array}$$

$${\mathbf{W}}_{5\times 13}=\left[\begin{array}{ccccccccccccc}1& & & & & & & & & 1& & & \\ & 1& & & & & & & & & 1& & \\ & & 1& & & & & & & & & 1& \\ & & & 1& & & & & & & & & 1\\ & & & & 1& 1& 1& 1& 1& & & & \end{array}\right].$$

Figure 4 presents the data flow graph of the fast DCT-VIII algorithm for N = 5. Direct matrix-vector product requires 25 multiplications and 20 additions. Our solution requires 18 multiplications and 23 additions. The pseudocode for the DCT-VIII for $N=5$ is provided in

Table A3. The pseudocode for the developed fast 5-point DCT-VII algorithm with variable reuse.

Step 1	Step 2	Step 3
$p_0=x_0-x_2$	$y_3=x_1{s}_1^{\left(5\right)}+y_4$	$y_4=y_4+y_0+x_4{s}_{14}^{\left(5\right)}$
$p_1=x_1+x_3$	$y_2=x_0{s}_0^{\left(5\right)}+y_4$	$y_1=y_1+p_2+x_4{s}_{15}^{\left(5\right)}$
$y_4=\left(x_0+x_1\right)s_2^{\left(5\right)}$	$y_4=x_3{s}_4^{\left(5\right)}+y_0$	$y_3=-\left(y_3+y_0+x_4{s}_{16}^{\left(5\right)}\right)$
$y_0=\left(x_3-x_2\right)s_5^{\left(5\right)}$	$y_1=-x_2{s}_3^{\left(5\right)}+y_0$	$y_2=y_2+p_2+x_4{s}_{17}^{\left(5\right)}$
$p_2=\left(p_0+p_1\right)s_8^{\left(5\right)}$	$y_0=p_1{s}_7^{\left(5\right)}+p_2$	$y_0=x_4{s}_9^{\left(5\right)}+x_0{s}_{10}^{\left(5\right)}+x_1{s}_{11}^{\left(5\right)}-x_2{s}_{12}^{\left(5\right)}+x_3{s}_{13}^{\left(5\right)}$
	$p_2=p_0{s}_6^{\left(5\right)}+p_2$

.

3.4. Fast Algorithm for the 6-Point DCT-VIII

Below, we present the matrix-vector form of the calculation of the DCT-VIII for N = 6:

$${\mathbf{Y}}_{6\times 1}={\mathbf{C}}_6{\mathbf{X}}_{6\times 1},$$

(9)

where

$${\mathbf{Y}}_{6\times 1}={\left[y_0,y_1,y_2,y_3,y_4,y_5\right]}^{\mathrm{T}},{\mathbf{X}}_{6\times 1}={\left[x_0,x_1,x_2,x_3,x_4,x_5\right]}^{\mathrm{T}},$$

$$\begin{array}{ccc}\hfill {\mathbf{C}}_6=\left[\begin{array}{cccccc}{a}_6& b_6& c_6& d_6& e_6& f_6\\ b_6& e_6& -f_6& -c_6& -a_6& -d_6\\ c_6& -f_6& -a_6& -e_6& d_6& b_6\\ d_6& -c_6& -e_6& b_6& f_6& -a_6\\ e_6& -a_6& d_6& f_6& -b_6& c_6\\ f_6& -d_6& b_6& -a_6& c_6& -e_6\end{array}\right],& \begin{array}{c}{a}_6=0.5507,\hfill \\ \\ b_6=0.5187,\hfill \\ \\ c_6=0.4565,\hfill \end{array}\hfill & \hfill \begin{array}{c}{d}_6=0.3678,\hfill \\ \\ e_6=0.2578,\hfill \\ \\ f_6=0.1327.\hfill \end{array}\end{array}$$

Let us alter the sign of the third column, and the sixth and the second rows. Next, we need to define the ${\pi }_6^{\left(0\right)}$ and ${\pi }_6^{\left(1\right)}$ permutations as below:

$${\pi }_6^{\left(0\right)}=\left(\begin{array}{cccccc}1& 2& 3& 4& 5& 6\\ 1& 2& 4& 3& 6& 5\end{array}\right),{\pi }_6^{\left(1\right)}=\left(\begin{array}{cccccc}1& 2& 3& 4& 5& 6\\ 3& 5& 6& 1& 4& 2\end{array}\right).$$

We then apply ${\pi }_6^{\left(0\right)}$ to the matrix ${\mathbf{C}}_6$ in columns and permutation ${\pi }_6^{\left(1\right)}$ in rows:

$${\mathbf{C}}_6^{\left(\mathrm{P}\right)}={\mathbf{P}}_6^{\left({\pi }_6^{\left(1\right)}\right)}{\mathbf{C}}_6{\mathbf{P}}_6^{\left({\pi }_6^{\left(0\right)}\right)},$$

where

$${\mathbf{P}}_6^{\left({\pi }_6^{\left(0\right)}\right)}=\left[\begin{array}{cccccc}1& & & & & \\ & 1& & & & \\ & & & -1& & \\ & & 1& & & \\ & & & & & 1\\ & & & & 1& \end{array}\right],{\mathbf{P}}_6^{\left({\pi }_6^{\left(1\right)}\right)}=\left[\begin{array}{cccccc}& & & 1& & \\ & & & & & -1\\ 1& & & & & \\ & & & & 1& \\ & 1& & & & \\ & & -1& & & \end{array}\right].$$

After these operations, the matrix ${\mathbf{C}}_6^{\left(\mathrm{P}\right)}$ looks as follows:

$${\mathbf{C}}_6^{\left(\mathrm{P}\right)}=\left[\begin{array}{cccccc}{d}_6& -c_6& b_6& e_6& -a_6& f_6\\ -f_6& d_6& a_6& b_6& e_6& -c_6\\ a_6& b_6& d_6& -c_6& f_6& e_6\\ e_6& -a_6& f_6& -d_6& c_6& -b_6\\ b_6& e_6& -c_6& f_6& -d_6& -a_6\\ -c_6& f_6& e_6& -a_6& -b_6& -d_6\end{array}\right]=\left[\begin{array}{cc}{\mathbf{A}}_3& {\mathbf{B}}_3\\ {\mathbf{B}}_3& -{\mathbf{A}}_3\end{array}\right],$$

where

$${\mathbf{A}}_3=\left[\begin{array}{ccc}{d}_6& -c_6& b_6\\ -f_6& d_6& a_6\\ a_6& b_6& d_6\end{array}\right],{\mathbf{B}}_3=\left[\begin{array}{ccc}{e}_6& -a_6& f_6\\ b_6& e_6& -c_6\\ -c_6& f_6& e_6\end{array}\right].$$

The above structure allows us to reduce the number of multiplications operations, so we derive the expression for the first stage of DCT-VIII factorization for N = 6:

$${\mathbf{Y}}_{6\times 1}={\mathbf{P}}_6^{\left(1\right)}{\mathbf{T}}_{6\times 9}{\mathbf{D}}_9{\mathbf{T}}_{9\times 6}{\mathbf{P}}_6^{\left(0\right)}{\mathbf{X}}_{6\times 1},$$

where

$$\begin{array}{c}\hfill \begin{array}{c}{\mathbf{P}}_6^{\left(0\right)}={\left({\mathbf{P}}_6^{\left({\pi }_6^{\left(0\right)}\right)}\right)}^{\mathrm{T}},\hfill \\ {\mathbf{P}}_6^{\left(1\right)}={\left({\mathbf{P}}_6^{\left({\pi }_6^{\left(1\right)}\right)}\right)}^{\mathrm{T}},\hfill \end{array}\begin{array}{c}{\mathbf{T}}_{9\times 6}={\mathbf{T}}_{3\times 2}^{\left(3\right)}\otimes {\mathbf{I}}_3,\hfill \\ \\ {\mathbf{T}}_{6\times 9}={\mathbf{T}}_{2\times 3}^{\left(4\right)}\otimes {\mathbf{I}}_3,\hfill \end{array}{\mathbf{D}}_9=\left[\begin{array}{ccc}{\mathbf{E}}_3& & \\ & {\mathbf{F}}_3& \\ & & {\mathbf{B}}_3\end{array}\right],\begin{array}{c}{\mathbf{E}}_3={\mathbf{A}}_3-{\mathbf{B}}_3,\hfill \\ \\ {\mathbf{F}}_3=-\left({\mathbf{A}}_3+{\mathbf{B}}_3\right),\hfill \end{array}\end{array}$$

$${\mathbf{E}}_3=\left[\begin{array}{ccc}{d}_6-e_6& -c_6+a_6& b_6-f_6\\ -f_6-b_6& d_6-e_6& a_6+c_6\\ a_6+c_6& b_6-f_6& d_6-e_6\end{array}\right]=\left[\begin{array}{ccc}0.1101& 0.0941& 0.3859\\ -0.6514& 0.1101& 1.0072\\ 1.0072& 0.3859& 0.1101\end{array}\right],$$

$${\mathbf{F}}_3=\left[\begin{array}{ccc}-d_6-e_6& c_6+a_6& -b_6-f_6\\ f_6-b_6& -d_6-e_6& -a_6+c_6\\ -a_6+c_6& -b_6-f_6& -d_6-e_6\end{array}\right]=\left[\begin{array}{ccc}-0.6256& 1.0072& -0.6514\\ -0.3859& -0.6256& -0.0941\\ -0.0941& -0.6514& -0.6256\end{array}\right].$$

Now, we split the matrix ${\mathbf{E}}_3$ into two matrices, which we consider separately, and then combine the factorizations of these two matrices:

$${\mathbf{E}}_3={\mathbf{E}}_3^{\left(\mathrm{A}\right)}+{\mathbf{E}}_3^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{E}}_3^{\left(\mathrm{A}\right)}=\left[\begin{array}{ccc}{d}_6-e_6& a_6+c_6& b_6-f_6\\ b_6-f_6& d_6-e_6& a_6+c_6\\ a_6+c_6& b_6-f_6& d_6-e_6\end{array}\right],{\mathbf{E}}_3^{\left(\mathrm{B}\right)}=\left[\begin{array}{ccc}0& 2c_6& 0\\ 2b_6& 0& 0\\ 0& 0& 0\end{array}\right].$$

The matrix ${\mathbf{E}}_3^{\left(\mathrm{A}\right)}$ has the structure of a circular convolution matrix, so we present the expression for factorization of the matrix ${\mathbf{E}}_3$:

$${\mathbf{E}}_3={\mathbf{W}}_{3\times 5}{\mathbf{W}}_5^{\left(1\right)}{\mathbf{W}}_{5\times 6}{\mathbf{D}}_6^{\left({\mathbf{E}}_3\right)}{\mathbf{W}}_{6\times 5}{\mathbf{W}}_5^{\left(0\right)}{\mathbf{W}}_{5\times 3},$$

where

$${\mathbf{W}}_{5\times 3}=\left[\begin{array}{ccc}1& & \\ & 1& \\ & & 1\\ -1& & \\ & -1& \end{array}\right],{\mathbf{W}}_5^{\left(0\right)}=\left[\begin{array}{cc}{\mathbf{W}}_3^{\left(0\right)}& \\ & {\mathbf{I}}_2\end{array}\right],{\mathbf{W}}_{6\times 5}=\left[\begin{array}{cc}{\mathbf{A}}_{4\times 3}& \\ & {\mathbf{I}}_2\end{array}\right],$$

$${\mathbf{D}}_6^{\left({\mathbf{E}}_3\right)}=\mathrm{diag}\left(s_0^{\left(6\right)},s_1^{\left(6\right)},\dots ,s_5^{\left(6\right)}\right),s_0^{\left(6\right)}={\displaystyle \frac{d_6-e_6+b_6-f_6+a_6+c_6}{3}},$$

$$s_1^{\left(6\right)}=d_6-e_6-a_6-c_6,s_2^{\left(6\right)}=b_6-f_6-a_6-c_6,s_3^{\left(6\right)}={\displaystyle \frac{d_6-e_6+b_6-f_6-2\left(a_6+c_6\right)}{3}},$$

$$s_4^{\left(6\right)}=2b_6,s_5^{\left(6\right)}=2c_6,{\mathbf{W}}_{5\times 6}=\left[\begin{array}{cc}{\mathbf{A}}_{3\times 4}& \\ & {\mathbf{I}}_2\end{array}\right],{\mathbf{W}}_5^{\left(1\right)}=\left[\begin{array}{cc}{\mathbf{W}}_3^{\left(1\right)}& \\ & {\mathbf{I}}_2\end{array}\right],$$

$${\mathbf{W}}_{3\times 5}=\left[\begin{array}{ccccc}1& & & & 1\\ & 1& & 1& \\ & & 1& & \end{array}\right].$$

Next, we split the matrix ${\mathbf{F}}_3$ into two matrices, which we consider separately, and then combine the factorizations of these two matrices:

$${\mathbf{F}}_3={\mathbf{F}}_3^{\left(\mathrm{A}\right)}+{\mathbf{F}}_3^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{F}}_3^{\left(\mathrm{A}\right)}=\left[\begin{array}{ccc}-d_6-e_6& c_6-a_6& -b_6-f_6\\ -b_6-f_6& -d_6-e_6& c_6-a_6\\ c_6-a_6& -b_6-f_6& -d_6-e_6\end{array}\right],{\mathbf{F}}_3^{\left(\mathrm{B}\right)}=\left[\begin{array}{ccc}0& -2a_6& 0\\ -2f_6& 0& 0\\ 0& 0& 0\end{array}\right].$$

The matrix ${\mathbf{F}}_3^{\left(\mathrm{A}\right)}$ has the structure of a circular convolution matrix, so we present the expression for factorization of the matrix ${\mathbf{F}}_3$:

$${\mathbf{F}}_3={\mathbf{W}}_{3\times 5}{\mathbf{W}}_5^{\left(1\right)}{\mathbf{W}}_{5\times 6}{\mathbf{D}}_6^{\left({\mathbf{F}}_3\right)}{\mathbf{W}}_{6\times 5}{\mathbf{W}}_5^{\left(0\right)}{\mathbf{W}}_{5\times 3},$$

where

$${\mathbf{D}}_6^{\left({\mathbf{F}}_3\right)}=\mathrm{diag}\left(s_6^{\left(6\right)},s_7^{\left(6\right)},\dots ,s_{11}^{\left(6\right)}\right),s_6^{\left(6\right)}={\displaystyle \frac{-d_6-e_6-b_6-f_6+c_6-a_6}{3}},$$

$$s_7^{\left(6\right)}=-d_6-e_6-c_6+a_6,s_8^{\left(6\right)}=-b_6-f_6-c_6+a_6,$$

$$s_9^{\left(6\right)}={\displaystyle \frac{-d_6-e_6-b_6-f_6-2\left(c_6-a_6\right)}{3}},s_{10}^{\left(6\right)}=-2f_6,s_{11}^{\left(6\right)}=-2a_6.$$

Further, we split the matrix ${\mathbf{B}}_3$ into two matrices, which we consider separately, and then combine the factorizations of these two matrices:

$${\mathbf{B}}_3={\mathbf{B}}_3^{\left(\mathrm{A}\right)}+{\mathbf{B}}_3^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{B}}_3^{\left(\mathrm{A}\right)}=\left[\begin{array}{ccc}{e}_6& -c_6& f_6\\ f_6& e_6& -c_6\\ -c_6& f_6& e_6\end{array}\right],{\mathbf{B}}_3^{\left(\mathrm{B}\right)}=\left[\begin{array}{ccc}0& a_6-c_6& 0\\ f_6-b_6& 0& 0\\ 0& 0& 0\end{array}\right].$$

The matrix ${\mathbf{B}}_3^{\left(\mathrm{A}\right)}$ has the structure of a circular convolution matrix, so we present the expression for factorization of the matrix ${\mathbf{B}}_3$:

$${\mathbf{B}}_3={\mathbf{W}}_{3\times 5}{\mathbf{W}}_5^{\left(1\right)}{\mathbf{W}}_{5\times 6}{\mathbf{D}}_6^{\left({\mathbf{B}}_3\right)}{\mathbf{W}}_{6\times 5}{\mathbf{W}}_5^{\left(0\right)}{\mathbf{W}}_{5\times 3},$$

where

$${\mathbf{D}}_6^{\left({\mathbf{B}}_3\right)}=\mathrm{diag}\left(s_{12}^{\left(6\right)},s_{13}^{\left(6\right)},\dots ,s_{17}^{\left(6\right)}\right),s_{12}^{\left(6\right)}={\displaystyle \frac{e_6+f_6-c_6}{3}},s_{13}^{\left(6\right)}=e_6+c_6,s_{14}^{\left(6\right)}=f_6+c_6,$$

$$s_{15}^{\left(6\right)}={\displaystyle \frac{e_6+f_6+2c_6}{3}},s_{16}^{\left(6\right)}=f_6-b_6,s_{17}^{\left(6\right)}=a_6-c_6.$$

We can now formulate the final matrix-vector procedure for computing DCT-VIII with reduced computational complexity for N = 6:

$${\mathbf{Y}}_{6\times 1}={\mathbf{P}}_6^{\left(1\right)}{\mathbf{T}}_{6\times 9}{\mathbf{W}}_{9\times 15}{\mathbf{W}}_{15}^{\left(1\right)}{\mathbf{W}}_{15\times 18}{\mathbf{D}}_{18}{\mathbf{W}}_{18\times 15}{\mathbf{W}}_{15}^{\left(0\right)}{\mathbf{W}}_{15\times 9}{\mathbf{T}}_{9\times 6}{\mathbf{P}}_6^{\left(0\right)}{\mathbf{X}}_{6\times 1},$$

(10)

where

$${\mathbf{W}}_{15\times 9}=\left[\begin{array}{ccc}{\mathbf{W}}_{5\times 3}& & \\ & {\mathbf{W}}_{5\times 3}& \\ & & {\mathbf{W}}_{5\times 3}\end{array}\right],{\mathbf{W}}_{15}^{\left(0\right)}=\left[\begin{array}{ccc}{\mathbf{W}}_5^{\left(0\right)}& & \\ & {\mathbf{W}}_5^{\left(0\right)}& \\ & & {\mathbf{W}}_5^{\left(0\right)}\end{array}\right],$$

$${\mathbf{W}}_{18\times 15}=\left[\begin{array}{ccc}{\mathbf{W}}_{6\times 5}& & \\ & {\mathbf{W}}_{6\times 5}& \\ & & {\mathbf{W}}_{6\times 5}\end{array}\right],{\mathbf{D}}_{18}=\left[\begin{array}{ccc}{\mathbf{D}}_6^{\left({\mathbf{E}}_3\right)}& & \\ & {\mathbf{D}}_6^{\left({\mathbf{F}}_3\right)}& \\ & & {\mathbf{D}}_6^{\left({\mathbf{B}}_3\right)}\end{array}\right],$$

$${\mathbf{W}}_{15\times 18}=\left[\begin{array}{ccc}{\mathbf{W}}_{5\times 6}& & \\ & {\mathbf{W}}_{5\times 6}& \\ & & {\mathbf{W}}_{5\times 6}\end{array}\right],{\mathbf{W}}_{15}^{\left(1\right)}=\left[\begin{array}{ccc}{\mathbf{W}}_5^{\left(1\right)}& & \\ & {\mathbf{W}}_5^{\left(1\right)}& \\ & & {\mathbf{W}}_5^{\left(1\right)}\end{array}\right],$$

$${\mathbf{W}}_{9\times 15}=\left[\begin{array}{ccc}{\mathbf{W}}_{3\times 5}& & \\ & {\mathbf{W}}_{3\times 5}& \\ & & {\mathbf{W}}_{3\times 5}\end{array}\right].$$

Figure 5 presents the data flow graph of the fast DCT-VIII algorithm for N = 6. Direct matrix-vector product requires 36 multiplications and 30 additions. Our solution requires 18 multiplications and 48 additions. The pseudocode for the DCT-VIII for $N=6$ is provided in

Table A4. The pseudocode for the developed fast 6-point DCT-VII algorithm with variable reuse.

Step 1	Step 2	Step 3
$x_0=x_0-x_2$	$y_1=x_2{s}_{10}^{\left(6\right)}$	$p_2=\left(y_0+y_2+p_5\right)s_{12}^{\left(6\right)}$
$y_2=x_1+x_5$	$y_4=-x_5{s}_{11}^{\left(6\right)}$	$p_1=y_0-p_5$
$p_5=x_3+x_4$	$p_6=\left(-x_2+x_5+x_4\right)s_6^{\left(6\right)}$	$p_4=y_2-p_5$
$y_3=-x_1{s}_5^{\left(6\right)}$	$p_7=-x_2-x_4$	$p_5=\left(p_1+p_4\right)s_{15}^{\left(6\right)}$
$p_0=\left(x_0+x_1+x_3\right)s_0^{\left(6\right)}$	$p_8=x_5-x_4$	$p_1=p_1{s}_{13}^{\left(6\right)}$
$x_1=x_1-x_3$	$x_4=\left(p_7+p_8\right)s_9^{\left(6\right)}$	$p_4=p_4{s}_{14}^{\left(6\right)}$
$x_3=x_0-x_3$	$p_7=p_7{s}_7^{\left(6\right)}$
$p_3=\left(x_3+x_1\right)s_3^{\left(6\right)}$	$p_8=p_8{s}_8^{\left(6\right)}$
$x_3=x_3{s}_1^{\left(6\right)}$
$x_1=x_1{s}_2^{\left(6\right)}$
Step 4	Step 5	Step 6
$x_3=x_3-p_3$	$p_0=p_6+p_7$	$p_4=-y_0{s}_{16}^{\left(6\right)}+p_7$
$x_1=x_1-p_3$	$x_3=p_6-p_7-p_8$	$p_1=-y_2{s}_{17}^{\left(6\right)}+p_6$
$x_2=p_0+x_3$	$x_1=p_6+p_8$	$x_0=-\left(-x_0{s}_4^{\left(6\right)}+x_5+p_4\right)$
$x_5=p_0-x_3-x_1$	$p_1=p_1-p_5$	$y_3=y_3+x_2+p_1$
$p_3=p_0+x_1$	$p_4=p_4-p_5$	$y_1=y_1+x_3+p_4$
$p_7=p_7-x_4$	$p_6=p_2+p_1$	$y_4=y_4+p_0+p_1$
$p_8=p_8-x_4$	$p_7=p_2-p_1-p_4$	$y_0=p_8+p_3$
	$p_8=p_2+p_4$	$y_2=-p_8-x_1$

.

3.5. Fast Algorithm for the 7-Point DCT-VIII

Below, we present the matrix-vector form of the calculation of the DCT-VIII for N = 7:

$${\mathbf{Y}}_{7\times 1}={\mathbf{C}}_7{\mathbf{X}}_{7\times 1},$$

(11)

where

$${\mathbf{Y}}_{7\times 1}={\left[y_0,y_1,y_2,y_3,y_4,y_5,y_6\right]}^{\mathrm{T}},{\mathbf{X}}_{7\times 1}={\left[x_0,x_1,x_2,x_3,x_4,x_5,x_6\right]}^{\mathrm{T}},$$

$$\begin{array}{ccc}\hfill {\mathbf{C}}_7=\left[\begin{array}{ccccccc}{a}_7& b_7& c_7& d_7& e_7& f_7& g_7\\ b_7& e_7& 0& -e_7& -b_7& -b_7& -e_7\\ c_7& 0& -c_7& -c_7& 0& c_7& c_7\\ d_7& -e_7& -c_7& f_7& b_7& -g_7& -a_7\\ e_7& -b_7& 0& b_7& -e_7& -e_7& b_7\\ f_7& -b_7& c_7& -g_7& -e_7& a_7& -d_7\\ g_7& -e_7& c_7& -a_7& b_7& -d_7& f_7\end{array}\right],& \begin{array}{c}{a}_7=0.5136,\hfill \\ \\ b_7=0.4911,\hfill \\ \\ c_7=0.4472,\hfill \\ \\ d_7=0.3838,\hfill \end{array}\hfill & \hfill \begin{array}{c}{e}_7=0.3035,\hfill \\ \\ f_7=0.2100,\hfill \\ \\ g_7=0.1074.\hfill \end{array}\end{array}$$

Let us change the signs of the entries in the third column and the third row of the matrix ${\mathbf{C}}_7$. Next, we need to define the ${\pi }_7^{\left(0\right)}$ and ${\pi }_7^{\left(1\right)}$ permutations as below:

$${\pi }_7^{\left(0\right)}=\left(\begin{array}{ccccccc}1& 2& 3& 4& 5& 6& 7\\ 1& 6& 7& 3& 5& 2& 4\end{array}\right),{\pi }_7^{\left(1\right)}=\left(\begin{array}{ccccccc}1& 2& 3& 4& 5& 6& 7\\ 2& 6& 7& 3& 5& 1& 4\end{array}\right).$$

We then apply the permutation ${\pi }_7^{\left(0\right)}$ to the matrix ${\mathbf{C}}_7$ in columns and the permutation ${\pi }_7^{\left(1\right)}$ in rows:

$${\mathbf{C}}_7^{\left(\mathrm{P}\right)}={\mathbf{P}}_7^{\left({\pi }_7^{\left(1\right)}\right)}{\mathbf{C}}_7{\left({\mathbf{P}}_7^{\left({\pi }_7^{\left(0\right)}\right)}\right)}^{\left(\mathrm{T}\right)},$$

where

$${\mathbf{P}}_7^{\left({\pi }_7^{\left(0\right)}\right)}=\left[\begin{array}{ccccccc}1& & & & & & \\ & & & & & 1& \\ & & & -1& & & \\ & & & & & & 1\\ & & & & 1& & \\ & 1& & & & & \\ & & 1& & & & \end{array}\right],{\mathbf{P}}_7^{\left({\pi }_7^{\left(1\right)}\right)}=\left[\begin{array}{ccccccc}& & & & & 1& \\ 1& & & & & & \\ & & & -1& & & \\ & & & & & & 1\\ & & & & 1& & \\ & 1& & & & & \\ & & 1& & & & \end{array}\right].$$

After these operations, the matrix ${\mathbf{C}}_7^{\left(\mathrm{P}\right)}$ looks as follows:

$${\mathbf{C}}_7^{\left(\mathbf{P}\right)}=\left[\begin{array}{ccccccc}{f}_7& a_7& g_7& -d_7& -e_7& -b_7& c_7\\ a_7& f_7& -d_7& g_7& e_7& b_7& c_7\\ -d_7& g_7& f_7& a_7& -b_7& e_7& c_7\\ g_7& -d_7& a_7& f_7& b_7& -e_7& c_7\\ e_7& -e_7& -b_7& b_7& -e_7& -b_7& 0\\ b_7& -b_7& e_7& -e_7& -b_7& e_7& 0\\ c_7& c_7& c_7& c_7& 0& 0& -c_7\end{array}\right].$$

Now, we split the matrix ${\mathbf{C}}_7^{\left(\mathrm{P}\right)}$ into two matrices, which we consider separately, and then combine:

$${\mathbf{C}}_7^{\left(\mathrm{P}\right)}={\mathbf{C}}_7^{\left(\mathrm{A}\right)}+{\mathbf{C}}_7^{\left(\mathrm{B}\right)},$$

where

$${\mathbf{C}}_7^{\left(\mathrm{A}\right)}=\left[\begin{array}{ccccccc}{f}_7& a_7& g_7& -d_7& 0& 0& 0\\ a_7& f_7& -d_7& g_7& 0& 0& 0\\ -d_7& g_7& f_7& a_7& 0& 0& 0\\ g_7& -d_7& a_7& f_7& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0\end{array}\right],$$

$${\mathbf{C}}_7^{\left(\mathrm{B}\right)}=\left[\begin{array}{ccccccc}0& 0& 0& 0& -e_7& -b_7& c_7\\ 0& 0& 0& 0& e_7& b_7& c_7\\ 0& 0& 0& 0& -b_7& e_7& c_7\\ 0& 0& 0& 0& b_7& -e_7& c_7\\ e_7& -e_7& -b_7& b_7& -e_7& -b_7& 0\\ b_7& -b_7& e_7& -e_7& -b_7& e_7& 0\\ c_7& c_7& c_7& c_7& 0& 0& -c_7\end{array}\right].$$

If we omit the zero rows and columns from the matrix ${\mathbf{C}}_7^{\left(\mathrm{A}\right)}$, we obtain the following matrix of the fourth order:

$${\mathbf{C}}_4=\left[\begin{array}{cccc}{f}_7& a_7& g_7& -d_7\\ a_7& f_7& -d_7& g_7\\ -d_7& g_7& f_7& a_7\\ g_7& -d_7& a_7& f_7\end{array}\right].$$

The matrix ${\mathbf{C}}_4$ has the following structure:

$${\mathbf{C}}_4=\left[\begin{array}{cc}{\mathbf{A}}_2& {\mathbf{B}}_2\\ {\mathbf{C}}_2& {\mathbf{A}}_2\end{array}\right],$$

where

$${\mathbf{A}}_2=\left[\begin{array}{cc}{f}_7& a_7\\ a_7& f_7\end{array}\right],{\mathbf{B}}_2=\left[\begin{array}{cc}{g}_7& -d_7\\ -d_7& g_7\end{array}\right],{\mathbf{C}}_2=\left[\begin{array}{cc}-d_7& g_7\\ g_7& -d_7\end{array}\right].$$

Matrix ${\mathbf{C}}_4$ after the first stage of factorization can be described by the following expression:

$${\mathbf{C}}_4={\mathbf{T}}_{4\times 6}{\mathbf{D}}_6^{\left(0\right)}{\mathbf{T}}_{6\times 4},$$

where

$${\mathbf{T}}_{6\times 4}={\mathbf{T}}_{3\times 2}^{\left(3\right)}\otimes {\mathbf{I}}_2,{\mathbf{D}}_6^{\left(0\right)}=\left[\begin{array}{ccc}{\mathbf{E}}_2& & \\ & {\mathbf{F}}_2& \\ & & {\mathbf{A}}_2\end{array}\right],{\mathbf{T}}_{4\times 6}={\mathbf{T}}_{2\times 3}^{\left(3\right)}\otimes {\mathbf{I}}_2,$$

$${\mathbf{E}}_2={\mathbf{C}}_2-{\mathbf{A}}_2=\left[\begin{array}{cc}-d_7-f_7& g_7-a_7\\ g_7-a_7& -d_7-f_7\end{array}\right]=\left[\begin{array}{cc}-0.5938& -0.4062\\ -0.4062& -0.5938\end{array}\right],$$

$${\mathbf{F}}_2={\mathbf{B}}_2-{\mathbf{A}}_2=\left[\begin{array}{cc}{g}_7-f_7& -d_7-a_7\\ -d_7-a_7& g_7-f_7\end{array}\right]=\left[\begin{array}{cc}-0.1027& -0.8973\\ -0.8973& -0.1027\end{array}\right].$$

The matrices ${\mathbf{E}}_2$, ${\mathbf{F}}_2$, and ${\mathbf{A}}_2$ are similar to the pattern $\left[\begin{array}{cc}a& b\\ b& a\end{array}\right]$. Then, we have the following expression after the second stage of factorization of the ${\mathbf{C}}_4$ matrix:

$${\mathbf{C}}_4={\mathbf{T}}_{4\times 6}{\mathbf{W}}_6{\mathbf{D}}_6^{\left(1\right)}{\mathbf{W}}_6{\mathbf{T}}_{6\times 4},$$

where

$${\mathbf{W}}_6=\left[\begin{array}{ccc}{\mathbf{H}}_2& & \\ & {\mathbf{H}}_2& \\ & & {\mathbf{H}}_2\end{array}\right],{\mathbf{D}}_6^{\left(1\right)}=\mathrm{diag}\left(s_0^{\left(7\right)},s_1^{\left(7\right)},\dots ,s_5^{\left(7\right)}\right),$$

$$s_0^{\left(7\right)}={\displaystyle \frac{-d_7-f_7+g_7-a_7}{2}},s_1^{\left(7\right)}={\displaystyle \frac{-d_7-f_7-g_7+a_7}{2}},s_2^{\left(7\right)}={\displaystyle \frac{g_7-f_7-d_7-a_7}{2}},$$

$$s_3^{\left(7\right)}={\displaystyle \frac{g_7-f_7+d_7+a_7}{2}},s_4^{\left(7\right)}={\displaystyle \frac{f_7+a_7}{2}},s_5^{\left(7\right)}={\displaystyle \frac{f_7-a_7}{2}}.$$

The final DCT-VIII expression for N = 7 including the matrix ${\mathbf{C}}_7^{\left(\mathrm{B}\right)}$ is as follows:

$${\mathbf{Y}}_{7\times 1}={\left({\mathbf{P}}_7^{\left({\pi }_7^{\left(1\right)}\right)}\right)}^{\mathrm{T}}{\mathbf{W}}_{7\times 13}{\mathbf{W}}_{13\times 16}{\mathbf{W}}_{16}{\mathbf{D}}_{16}{\mathbf{W}}_{16\times 9}{\mathbf{W}}_9{\mathbf{W}}_{9\times 7}{\mathbf{P}}_7^{\left({\pi }_7^{\left(0\right)}\right)}{\mathbf{X}}_{7\times 1},$$

(12)

where

$$\begin{array}{c}\hfill {\mathbf{W}}_{9\times 7}=\left[\begin{array}{ccccccc}1& & & & & & \\ & 1& & & & & \\ & & 1& & & & \\ & & & 1& & & \\ 1& & 1& & & & \\ & 1& & 1& & & \\ & & & & 1& & \\ & & & & & 1& \\ & & & & & & 1\end{array}\right],{\mathbf{W}}_9=\left[\begin{array}{cc}{\mathbf{W}}_6& \\ & {\mathbf{I}}_3\end{array}\right],\end{array}$$

$$\begin{array}{cc}\hfill {\mathbf{W}}_{16\times 9}=\left[\begin{array}{ccccccccc}1& & & & & & & & \\ & 1& & & & & & & \\ & & 1& & & & & & \\ & & & 1& & & & & \\ & & & & 1& & & & \\ & & & & & 1& & & \\ & & & & & & 1& & \\ & 1& & & & & & & \\ & & & & & & 1& & \\ & & & -1& & & & & \\ & & & & & & & 1& \\ & & & 1& & & & & \\ & & & & & & & 1& \\ & 1& & & & & -1& & \\ & & & & & & & & 1\\ 1& & 1& & & & & & \end{array}\right],& \begin{array}{c}{\mathbf{W}}_{16}=\left[\begin{array}{cc}{\mathbf{W}}_6& \\ & {\mathbf{I}}_{10}\end{array}\right],\hfill \\ \\ {\mathbf{D}}_{16}=\mathrm{diag}\left(s_0^{\left(7\right)},s_1^{\left(7\right)},\dots ,s_{15}^{\left(7\right)}\right),\hfill \\ \\ s_6^{\left(7\right)}=s_7^{\left(7\right)}=s_{10}^{\left(7\right)}=s_{11}^{\left(7\right)}=e_7,\hfill \\ \\ s_8^{\left(7\right)}=s_9^{\left(7\right)}=s_{12}^{\left(7\right)}=s_{13}^{\left(7\right)}=b_7,\hfill \\ \\ s_{14}^{\left(7\right)}=s_{15}^{\left(7\right)}=c_7,\hfill \end{array}\hfill \end{array}$$

$$\begin{array}{c}\hfill {\mathbf{W}}_{13\times 16}=\left[\begin{array}{cccccccccccccccc}& & 1& & 1& & & & & & & & & & & \\ & & & 1& & 1& & & & & & & & & & \\ 1& & & & 1& & & & & & & & & & & \\ & 1& & & & 1& & & & & & & & & & \\ & & & & & & 1& & & & & & 1& & & \\ & & & & & & & 1& & & & & & & & \\ & & & & & & & & -1& & 1& & & & & \\ & & & & & & & & & 1& & & & & & \\ & & & & & & & & & & 1& & & & & \\ & & & & & & & & & & & 1& & & & \\ & & & & & & & & & & & & & 1& & \\ & & & & & & & & & & & & & & 1& \\ & & & & & & & & & & & & & & & 1\end{array}\right],\end{array}$$

$$\begin{array}{c}\hfill {\mathbf{W}}_{7\times 13}=\left[\begin{array}{ccccccccccccc}1& & & & -1& & & & & & & 1& \\ & 1& & & 1& & & & & & & 1& \\ & & 1& & & & 1& & & & & 1& \\ & & & 1& & & -1& & & & & 1& \\ & & & & -1& 1& & 1& & & & & \\ & & & & & & & & 1& 1& 1& & \\ & & & & & & & & & & & -1& 1\end{array}\right].\end{array}$$

Figure 6 presents the data flow graph of the fast DCT-VIII algorithm for N = 7. Direct matrix-vector product requires 45 multiplications and 38 additions. Our solution requires 16 multiplications and 34 additions. The pseudocode for the DCT-VIII for $N=7$ is provided in

Table A5. The pseudocode for the developed fast 7-point DCT-VII algorithm with variable reuse.

Step 1	Step 2
$p_7=x_0-x_5$,    $p_9=x_3+x_6$	$p_0=\left(x_0+x_5\right)s_0^{\left(7\right)}$,    $p_1=p_7{s}_1^{\left(7\right)}$
$p_{11}=-p_9{s}_{11}^{\left(7\right)}$	$p_2=\left(-x_3+x_6\right)s_2^{\left(7\right)}$,    $p_3=-p_9{s}_3^{\left(7\right)}$
$p_{13}=\left(p_7-x_4\right)s_{13}^{\left(7\right)}$	$p_4=p_6{s}_4^{\left(7\right)}$,    $p_6=x_4{s}_6^{\left(7\right)}$
$p_6=x_0-x_3$,    $p_8=x_5+x_6$	$p_7=p_7{s}_7^{\left(7\right)}$,    $p_8=x_4{s}_8^{\left(7\right)}$
$p_5=\left(p_6-p_8\right)s_5^{\left(7\right)}$	$p_9=p_9{s}_9^{\left(7\right)}$,    $p_{10}=x_1{s}_{10}^{\left(7\right)}$
$p_6=p_6+p_8$,    $p_{15}=p_6{s}_{15}^{\left(7\right)}$	$p_{12}=x_1{s}_{12}^{\left(7\right)}$,    $p_{14}=x_2{s}_{14}^{\left(7\right)}$
Step 3	Step 4	Step 5
$x_0=p_0+p_1$	$p_0=x_3+x_4$	$x_5=p_0-p_6+p_{14}$
$x_5=p_0-p_1$	$p_1=x_6+x_1$	$x_0=p_1+p_6+p_{14}$
$x_3=p_2+p_3$	$p_2=x_0+x_4$	$x_3=-\left(p_2-p_8+p_{14}\right)$
$x_6=p_2-p_3$	$p_3=x_5+x_1$	$x_6=p_3+p_8+p_{14}$
$x_4=p_4+p_5$	$p_6=p_6+p_{12}$	$x_4=-p_6+p_7+p_9$
$x_1=p_4-p_5$	$p_8=p_8-p_{10}$	$x_1=p_{10}+p_{11}+p_{13}$
		$x_2=-p_{14}+p_{15}$

.

4. Results

The obtained results were verified in two stages. First, the correctness of the DCT-VIII matrix factorizations presented in Section 3 was experimentally validated. Using MATLAB R2023b, the original DCT-VIII matrices were computed from expressions (3), (5), (7), (9), and (11). These matrices were then compared with the products of the corresponding factorized matrices derived from expressions (4), (6), (8), (10), and (12). For size N ranging from 3 to 7, the factorized matrices exactly matched the original matrices, thereby confirming the correctness of the proposed algorithms.

Second, the correctness of the software implementations of the proposed algorithms was verified, which in turn confirms the correctness of the constructed data flow graphs. The algorithms represented by the data flow graphs were implemented in software, and the corresponding implementations are presented as pseudocode in Appendix A.

For each proposed N-point DCT-VIII algorithm, 30 sequences of N random numbers were generated. Each sequence was applied as input to the corresponding algorithm, and the resulting output was compared with the result obtained from the direct matrix-vector multiplication for the same input. If the results coincided for all 30 test sequences, the algorithm was deemed correct. In this manner, the correctness of the pseudocode and the corresponding data flow graphs was confirmed for the developed DCT-VIII algorithms with N ranging from 3 to 7.

Next, the arithmetic complexity of the proposed DCT-VIII algorithms was evaluated and compared with that of direct matrix-vector multiplication. The number of multiplications was determined by counting the vertices labeled with factors (circles) in the data flow graphs, while additions were estimated by counting vertices where two edges converge. Zero entries were not considered. Overall, the proposed DCT-VIII algorithms achieve a reduction of approximately 53% in the number of multiplications and an increase of about 21% in the number of additions for input sizes from 3 to 7.

Table 3. Comparison of the direct method with the proposed solutions.

	Direct Method		Proposed Algorithms
N	Multiplications	Additions	Multiplications	Additions
3	9	6	4 (−55%)	11 (+83%)
4	15	11	5 (−66%)	11 (0%)
5	25	20	18 (−28%)	23 (+15%)
6	36	30	18 (−50%)	48 (+60%)
7	45	38	16 (−64%)	34 (−10%)

summarizes the number of arithmetic operations required by the proposed DCT-VIII algorithms. Percentage differences relative to the direct matrix-vector product are indicated in parentheses. A plus sign denotes an increase in the number of operations, whereas a minus sign indicates a reduction.

In

Table 4. Comparison of the proposed solutions with the DCT-VIII algorithms from [4,27].

	Algorithms Based on FFT [4]		Proposed Algorithms		Masera et al. [27]
N	Mults.	Adds.	Mults.	Adds.	Mults.	Adds.
3	11	32	4	11	-	-
4	14	39	5	11	5	11
5	26	88	18	23	-	-
6	27	99	18	48	-	-
7	22	90	16	34	-	-

, we present the number of arithmetic operations for DCT-VIII algorithms based on the fast Fourier transform (FFT). Detailed explanations of how the number of operations for these algorithms is calculated are provided in Appendix B.

Our analysis shows that, for N = 4, the proposed DCT-VIII algorithm using the structural approach requires the same number of multiplications and additions compared to the DCT-VIII algorithm from [27], which is based on the relationship between DST-VII and DCT-VIII. Moreover, the proposed solutions require a significantly lower number of the arithmetic operations than the DCT-VIII algorithms based on the FFT [4]. To the best of our knowledge, no other algorithms are available in the literature for direct comparison. We also consider it inappropriate to compare the proposed fast DCT-VIII algorithm with related integer-transform algorithms due to their differing mathematical foundations.

5. Discussion

Let us analyze the obtained algorithms. In the literature, DCT-VIII algorithms are typically presented only for input sequence lengths that are powers of two. This restriction arises because fast algorithms for this transform were developed primarily for video coding. However, other applications of DCT-VIII are also known. For example, in [17], it was used for fractional-pixel motion compensation in high-efficiency video coding, which requires fast algorithms for input sequences whose lengths are not powers of two.

We now examine the stages of the DCT-VIII matrix factorization based on the structural approach, which was used to construct the proposed algorithms (

Table 5. The structural patterns of the proposed and existing fast DCT-VIII algorithms.

	Stage 1 of the Factorization		Stage 2 of the Factorization
N	Proposed DCT-VIII Algorithms	Existing DCT-VIII Algorithms [12,29]	Proposed DCT-VIII Algorithms	Existing DCT-VIII Algorithms [12,29]
3	Circular convolution	Pattern 2	-	-
4	Circular convolution	Pattern 2	Fan-shaped structure	Pattern 3
5	Four submatrices 2 × 2	Pattern 1	Direct multiplication of submatrix on the input vector	-
6	Four submatrices 3 × 3	Pattern 1	Circular convolution	Pattern 2
7	Fan-shaped structure	Pattern 3	Four submatrices 2 × 2	Pattern 1

).

For $N=3$, the DCT-VIII matrix is fully characterized by the circular-convolution structure. Applying a factorization of this structure enabled the factorization of the 3-point DCT-VIII matrix. Using the terminology of [12,29], only the second pattern listed in Section 1.1 was applied for $N=3$. Specifically, this pattern involves a subset of column elements that is repeated in other columns with permutation.

For $N=4$, two patterns were identified in the transform matrix. The first pattern is a circular convolution applied to a $3\times 3$ submatrix. In the second pattern, the column and row of the original matrix contain the same values up to a sign. In the data flow graph, this pattern corresponds to a fan-shaped structure, where several edges leave a single node toward different outputs or enter a single node from multiple inputs. This structure matches the third pattern from Section 1.1 and allows a reduction in the number of multiplications by performing preliminary additions of the inputs, followed by a single multiplication.

For $N=5$, a structure consisting of four $2\times 2$ submatrices was extracted from the DCT-VIII matrix, and each submatrix was subsequently factorized. This corresponds to the first pattern described in [12,29], specifically a pattern consisting of several groups with a fixed number of elements, regardless of sign changes. For the entries of the DCT-VIII matrix not included in this structure, a direct multiplication of the input vector by the corresponding transform matrix entries was used. Thus, in this case, the structural approach was applied jointly with direct matrix-vector multiplication.

For $N=6$, a two-level hierarchical structure was identified in the DCT-VIII matrix. The upper level consists of a decomposition into $3\times 3$ submatrices, while the lower level represents the structure of these submatrices in the form of circular convolutions. In this case, patterns 1 and 2 from Section 1.1 and from [12,29] are successively identified.

For $N=7$, two patterns were identified in the matrix. Unlike the case of $N=6$, the extracted structure is non-hierarchical. A $4\times 4$ matrix was extracted from the original transform matrix and represented by four $2\times 2$ submatrices. This corresponds to pattern 1 according to the terminology of [12,29]. The remaining entries of the original DCT-VIII matrix for $N=7$ correspond to pattern 3 from Section 1.1, as they include several groups with identical elements, regardless of sign changes.

Identifying such structures enabled a significant reduction in the number of multiplications required to implement the DCT-VIII for small N compared to direct matrix-vector multiplication. However, the number of additions increases due to the preliminary summation of matrix entries that are multiplied by the same factor.

Another limitation of the structural approach is that it is better suited for constructing fast algorithms for short data sequences. As the sequence length increases, identifying the structure of the transform matrices becomes more difficult. Moreover, as discussed earlier, the efficiency of the proposed fast algorithms strongly depends on the structural properties of these matrices.

6. Conclusions

The fast DCT-VIII algorithms presented in this paper were constructed using the structural approach described in [32,33]. This approach derives fast algorithms from the structural properties of transform matrices for various input lengths. In this context, the structure of a transform matrix refers to properties such as symmetries in certain submatrices, recursion of matrix patterns, factorization of matrix patterns, and algebraic relationships between matrix entries.

The presented fast DCT-VIII algorithms were developed for short input sequences, specifically for lengths from 3 to 7. The derivation of the fast DCT-VIII algorithm for $N=8$ has already been addressed in several studies [27,31]. The proposed algorithms reduce computational complexity by significantly decreasing the number of multiplications compared to direct matrix-vector computation. On average, they achieve a reduction of approximately 53% in the number of multiplications and an increase of about 21% in the number of additions for input sizes from 3 to 7.

The algorithms are represented using data flow graphs. A key advantage of the proposed designs is that the input-output path in each graph contains only a single multiplication. It is well known that when more than one multiplication lies on the input-output path, the operand format doubles with each multiplication, introducing additional data-processing challenges. The proposed algorithms completely avoid this issue.

Implementing the constructed fast DCT-VIII algorithms ensures numerical stability, as the structural approach generally preserves the orthogonality of the transform and maintains good conditioning.

Additionally, the fast DCT-VIII algorithms for short input sequences can be reused as building blocks for other DCT and DST types due to cross-relations between transforms. For example, a DST-VII can be converted into a DCT-VIII through permutation and input reversal.

Although practical codecs commonly employ large transform sizes (e.g., 16, 32, and 64) [35,36], algorithms for such large transforms are typically constructed from smaller kernels using well-established nesting techniques [37,38]. Consequently, optimizing the computation of small-scale transforms remains critically important. Moreover, for this class of transforms, algorithmic computational complexity increases rapidly with transform size. To mitigate this issue, one can synthesize new large-scale orthogonal transforms from small-scale discrete transforms used as kernels. As a result, the local characteristics of the original transforms are preserved while achieving lower computational complexity than conventional large-transform implementations.