Low-Complexity Loeffler DCT Approximations for Image and Video Coding

Abstract

This paper introduced a matrix parametrization method based on the Loeffler discrete cosine transform (DCT) algorithm. As a result, a new class of 8-point DCT approximations was proposed, capable of unifying the mathematical formalism of several 8-point DCT approximations archived in the literature. Pareto-efficient DCT approximations are obtained through multicriteria optimization, where computational complexity, proximity, and coding performance are considered. Efficient approximations and their scaled 16- and 32-point versions are embedded into image and video encoders, including a JPEG-like codec and H.264/AVC and H.265/HEVC standards. Results are compared to the unmodified standard codecs. Efficient approximations are mapped and implemented on a Xilinx VLX240T FPGA and evaluated for area, speed, and power consumption.

1. Introduction

Discrete time transforms have a major role in signal-processing theory and application. In particular, tools such as the discrete Haar, Hadamard, and discrete Fourier transforms, and several discrete trigonometrical transforms [1,2] have contributed to various image-processing techniques [3,4,5,6]. Among such transformations, the discrete cosine transform (DCT) of type II is widely regarded as a pivotal tool for image compression, coding, and analysis [2,7,8]. This is because the DCT closely approximates the Karhunen–Loève transform (KLT) which can optimally decorrelate highly correlated stationary Markov-I signals [2].

Indeed, the recent literature reveals a significant number of works linked to DCT computation. Some noteworthy topics are: (i) cosine–sine decomposition to compute the 8-point DCT [9]; (ii) low-complexity-pruned 8-point DCT approximations for image encoding [10]; (iii) improved 8-point approximate DCT for image and video compression requiring only 14 additions [11]; (iv) HEVC multisize DCT hardware with constant throughput, supporting heterogeneous coding unities [12]; (v) approximation of feature pyramids in the DCT domain and its application to pedestrian detection [13]; (vi) performance analysis of DCT and discrete wavelet transform (DWT) audio watermarking based on singular value decomposition [14]; (vii) adaptive approximated DCT architectures for HEVC [15]; (viii) improved Canny edge detection algorithm based on DCT [16]; and (ix) DCT-inspired feature transform for image retrieval and reconstruction [17]. In fact, several current image- and video-coding schemes are based on the DCT [18], such as JPEG [19], MPEG-1 [20], H.264 [21], and HEVC [22]. In particular, the H.264 and HEVC codecs employ low-complexity discrete transforms based on the 8-point DCT. The 8-point DCT has also been applied to dedicated image-compression systems implemented in large web servers with promising results [23]. As a consequence, several algorithms for the 8-point DCT have been proposed, such as: Lee DCT factorization [24], Arai DCT scheme [25], Feig–Winograd algorithm [26], and the Loeffler DCT algorithm [27]. Among these methods, the Loeffler DCT algorithm [27] has the distinction of achieving the theoretical lower bound for DCT multiplicative complexity [28,29].

Because the computational complexity lower bounds of the DCT have been achieved [28], the research community resorted to approximation techniques to further reduce the cost of DCT calculation. Although not capable of providing exact computation, approximate transforms can furnish very close computational results at significantly smaller computational cost. Early approximations for the DCT were introduced by Haweel [30]. Since then, several DCT approximations have been proposed. In Reference [31], Lengwehasatit and Ortega introduced a scalable approximate DCT that can be regarded as a benchmark approximation [5,31,32,33,34,35,36,37,38,39,40]. Aiming at image coding for data compression, a series of approximations have been proposed by Bouguezel-Ahmad-Swamy (BAS) [5,34,35,36,37,41]. Such approximations offer very low complexity and good coding performance [2,39].

The methods for deriving DCT approximation include: (i) application of simple functions, such as signum, rounding-off, truncation, ceil, and floor, to approximate the elements of the exact DCT matrix [30,39]; (ii) scaling and rounding-off [2,32,39,42,43,44,45,46,47]; (iii) brute-force computation over reduced search space [38,39]; (iv) inspection [33,34,35,41]; (v) single-variable matrix parametrization of existing approximations [37]; (vi) pruning techniques [48]; and (vii) derivations based on other low-complexity matrices [5]. The above-mentioned methods are capable of supplying single or very few approximations. In fact, a systematized approach for obtaining a large number of matrix approximations and a unifying scheme is lacking.

The goal of this paper is two-fold. First we aim at unifying the matrix formalism of several 8-point DCT approximations archived in the literature. For that, we consider the 8-point Loeffler algorithm as a general structure equipped with a parametrization of the multiplicands. This approach allows the definition of a matrix subspace where a large number of approximations could be derived. Second, we propose an optimization problem over the introduced matrix subspace in order to discriminate the best approximations according to several well-known figures of merit. This discrimination is important from the application point of view. It allows the user to select the transform that fits best to their application in terms of balancing performance and complexity. The optimally found approximations are subject to mathematical assessment and embedding into image- and video-encoding schemes, including the H.264/AVC and the H.265/HEVC standards. Third, we introduce hardware architecture based on optimally found approximations realized in field programmable gate array (FPGA). Although there are several subsystems in a video and image codec, this work is solely concentrated on the discrete transform subsystem.

The paper unfolds as follows. Section 2 introduces a novel DCT parametrization based on the Loeffler DCT algorithm. We provide the mathematical background and matrix properties, such as invertibility, orthogonality, and orthonormalization, are examined. Section 3 reviews the criteria employed for identifying and assessing DCT approximations, such as proximity and coding measures, and computational complexity. In Section 4, we propose a multicriteria-optimization problem aiming at deriving optimal approximation subject to Pareto efficiency. Obtained transforms are sought to be comprehensively assessed and compared with state-of-the-art competitors. Section 5 reports the results of embedding the obtained transforms into a JPEG-like encoder, as well as in H.264/AVC and H.265/HEVC video standards. In Section 6, an FPGA hardware implementation of the optimal transformations is detailed, and the usual FPGA implementation metrics are reported. Section 7 presents our final remarks.

2. DCT Parametrization and Matrix Space

2.1. DCT Matrix Factorization

The type II DCT is defined according to the following linear transformation matrix [2,8]:

$$\begin{array}{cc}\hfill {\mathbf{C}}_{\mathrm{DCT}}& =\frac{1}{2}\left[\begin{array}{cccccccc}{c}_4& c_4& c_4& c_4& c_4& c_4& c_4& c_4\\ c_1& c_3& c_5& c_7& -c_7& -c_5& -c_3& -c_1\\ c_2& c_6& -c_6& -c_2& -c_2& -c_6& c_6& c_2\\ c_3& -c_7& -c_1& -c_5& c_5& c_1& c_7& -c_3\\ c_4& -c_4& -c_4& c_4& c_4& -c_4& -c_4& c_4\\ c_5& -c_1& c_7& c_3& -c_3& -c_7& c_1& -c_5\\ c_6& -c_2& c_2& -c_6& -c_6& c_2& -c_2& c_6\\ c_7& -c_5& c_3& -c_1& c_1& -c_3& c_5& -c_7\end{array}\right],\hfill \end{array}$$

(1)

where $c_k=\cos (k\pi /16)$, $k=1,2,\dots ,7$. Because several entries of ${\mathbf{C}}_{\mathrm{DCT}}$ are not rational, they are often truncated/rounded and represented in floating-point arithmetic [7,49], which requires demanding computational costs when compared with fixed-point schemes [5,7,33,50].

Fast algorithms can minimize the number of arithmetic operations required for the DCT computation [2,7]. A number of fast algorithms have been proposed for the 8-point DCT [24,25,26]. The vast majority of DCT algorithms consist of the following factorization [2]:

$$\begin{array}{cc}\hfill {\mathbf{C}}_{\mathrm{DCT}}& =\mathbf{P}·\mathbf{M}·\mathbf{A},\hfill \end{array}$$

(2)

where $\mathbf{A}$ is the additive matrix that represents a set of butterfly operations, $\mathbf{M}$ is a multiplicative matrix, and $\mathbf{P}$ is a permutation matrix that simply rearranges the output components to natural order. Matrix $\mathbf{A}$ is often fixed [2] and given by:

$$\begin{array}{cc}\hfill \mathbf{A}& =\left[\begin{array}{cccccccc}1& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}1\\ 0& 1& 0& 0& 0& 0& 1& 0\\ 0& 0& 1& 0& 0& 1& 0& 0\\ 0& 0& 0& 1& 1& 0& 0& 0\\ 0& 0& 0& 1& -1& 0& 0& 0\\ 0& 0& 1& 0& 0& -1& 0& 0\\ 0& 1& 0& 0& 0& 0& -1& 0\\ 1& 0& 0& 0& 0& 0& 0& -1\end{array}\right].\hfill \end{array}$$

(3)

Multiplicative matrix $\mathbf{M}$ can be further factorized. Since matrix factorization is not unique, each fast algorithm is linked to a particular factorization of $\mathbf{M}$. Finally, the permutation matrix is given below:

$$\begin{array}{cc}\hfill \mathbf{P}& =\left[\begin{array}{cccccccc}1& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 1\\ 0& 0& 1& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 1& 0& 0\\ 0& 1& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 0& 1& 0& 0& 0\end{array}\right].\hfill \end{array}$$

(4)

Among the DCT fast algorithms, the Loeffler DCT achieves the theoretical lower bound of the multiplicative complexity for 8-point DCT, which consists of 11 multiplications [27]. In this work, multiplications by irrational quantities are sought to be substituted with trivial multipliers, representable by simple bit-shifting operations (Section 2.2 and Section 3.2). Thus, we expect that approximations based on Loeffler DCT could generate low-complexity approximations. Therefore, the Loeffler DCT algorithm was separated as the starting point to devise new DCT approximations.

The Loeffler DCT employs a scaled DCT with the following transformation matrix:

$$\begin{array}{c}\hfill {\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}=2\sqrt{2}·{\mathbf{C}}_{\mathrm{DCT}}.\end{array}$$

(5)

Such scaling eliminates one multiplicand because $2\sqrt{2}·c_4=1$. Therefore, we can write the following expression:

$$\begin{array}{c}\hfill {\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}=\mathbf{P}·{\mathbf{M}}^{\prime }·\mathbf{A},\end{array}$$

(6)

where

$$\begin{array}{cc}\hfill {\mathbf{M}}^{\prime }=& 2\sqrt{2}·\mathbf{M}\hfill \\ \hfill =& \left[\begin{array}{cccccccc}1& 1& 1& 1& 0& 0& 0& 0\\ 1& -1& -1& 1& 0& 0& 0& 0\\ \sqrt{2}{c}_2& \sqrt{2}{c}_6& -\sqrt{2}{c}_6& -\sqrt{2}{c}_2& 0& 0& 0& 0\\ \sqrt{2}{c}_6& -\sqrt{2}{c}_2& \sqrt{2}{c}_2& -\sqrt{2}{c}_6& 0& 0& 0& 0\\ 0& 0& 0& 0& -\sqrt{2}{c}_1& \sqrt{2}{c}_3& -\sqrt{2}{c}_5& \sqrt{2}{c}_7\\ 0& 0& 0& 0& -\sqrt{2}{c}_5& -\sqrt{2}{c}_1& -\sqrt{2}{c}_7& \sqrt{2}{c}_3\\ 0& 0& 0& 0& \sqrt{2}{c}_3& \sqrt{2}{c}_7& -\sqrt{2}{c}_1& \sqrt{2}{c}_5\\ 0& 0& 0& 0& \sqrt{2}{c}_7& \sqrt{2}{c}_5& \sqrt{2}{c}_3& \sqrt{2}{c}_1\end{array}\right].\hfill \end{array}$$

(7)

Matrix ${\mathbf{M}}^{\prime }$ carries all multiplications by irrational quantities required by Loeffler fast algorithm. It can be further decomposed as:

$$\begin{array}{cc}\hfill {\mathbf{M}}^{\prime }=& \mathbf{B}·\mathbf{C}·\mathbf{D},\hfill \end{array}$$

(8)

where

$$\begin{array}{cc}\hfill \mathbf{B}=& \left[\begin{array}{cccccccc}1& \phantom{-}0& 0& 1& 0& 0& \phantom{-}0& \phantom{-}0\\ 0& 1& 1& 0& 0& 0& 0& 0\\ 0& 1& -1& 0& 0& 0& 0& 0\\ 1& 0& 0& -1& 0& 0& 0& 0\\ 0& 0& 0& 0& c_3& 0& 0& c_5\\ 0& 0& 0& 0& 0& c_1& c_7& 0\\ 0& 0& 0& 0& 0& -c_7& c_1& 0\\ 0& 0& 0& 0& -c_5& 0& 0& c_3\end{array}\right],\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \mathbf{C}=& \left[\begin{array}{cccccccc}1& 1& 0& 0& 0& 0& 0& 0\\ 1& -1& 0& 0& 0& 0& 0& 0\\ 0& 0& \sqrt{2}{c}_6& \sqrt{2}{c}_2& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0\\ 0& 0& -\sqrt{2}{c}_2& \sqrt{2}{c}_6& 0& 0& 0& 0\\ 0& 0& 0& 0& 1& 0& 1& 0\\ 0& 0& 0& 0& 0& -1& 0& 1\\ 0& 0& 0& 0& 1& 0& -1& 0\\ 0& 0& 0& 0& 0& 1& 0& 1\end{array}\right],\hfill \end{array}$$

(10)

and

$$\begin{array}{cc}\hfill \mathbf{D}=& \left[\begin{array}{cccccccc}1& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0& \phantom{-}0\\ 0& 1& 0& 0& 0& 0& 0& 0\\ 0& 0& 1& 0& 0& 0& 0& 0\\ 0& 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 0& -1& 0& 0& 1\\ 0& 0& 0& 0& 0& \sqrt{2}& 0& 0\\ 0& 0& 0& 0& 1& 0& \sqrt{2}& 0\\ 0& 0& 0& 0& 1& 0& 0& 1\end{array}\right].\hfill \end{array}$$

(11)

In order to achieve the minimum multiplicative complexity, the Loeffler fast algorithm uses fast rotation for the rotation blocks in matrix $\mathbf{B}$ and $\mathbf{C}$ [2,50]. Since each of the three fast rotations requires three multiplications, and we have the two additional multiplications on matrix $\mathbf{D}$, the Loeffler fast algorithm requires a total of 11 multiplications.

2.2. Loeffler DCT Parametrization

DCT factorization suggests matrix parametrization. In fact, replacing multiplicands $\sqrt{2}·c_i$, $i\in \{1,2,3,5,6,7\}$ in Matrix (8) by parameters ${\alpha }_k$, $k=1,2,\dots ,6$, respectively, yields the following parametric matrix:

$$\begin{array}{cc}\hfill {\mathbf{M}}_{\mathit{\alpha }}& =\left[\begin{array}{cccccccc}1& 1& 1& 1& 0& 0& 0& 0\\ 1& -1& -1& 1& 0& 0& 0& 0\\ {\alpha }_2& {\alpha }_5& -{\alpha }_5& -{\alpha }_2& 0& 0& 0& 0\\ {\alpha }_5& -{\alpha }_2& {\alpha }_2& -{\alpha }_5& 0& 0& 0& 0\\ 0& 0& 0& 0& -{\alpha }_1& {\alpha }_3& -{\alpha }_4& {\alpha }_6\\ 0& 0& 0& 0& -{\alpha }_4& -{\alpha }_1& -{\alpha }_6& {\alpha }_3\\ 0& 0& 0& 0& {\alpha }_3& {\alpha }_6& -{\alpha }_1& {\alpha }_4\\ 0& 0& 0& 0& {\alpha }_6& {\alpha }_4& {\alpha }_3& {\alpha }_1\end{array}\right],\hfill \end{array}$$

(12)

where subscript $\mathit{\alpha }={\left[\begin{array}{cccc}{\alpha }_1& {\alpha }_2& \cdots & {\alpha }_6\end{array}\right]}^{\top }$ denotes a real-valued parameter vector. Mathematically, the following mapping is introduced:

$$\begin{array}{cc}\hfill f:{\mathbb{R}}^6& ⟶\mathcal{M}\left(8\right)\hfill \\ \hfill \mathit{\alpha }& ⟼{\mathbf{T}}_{\mathit{\alpha }}=\mathbf{P}·{\mathbf{M}}_{\mathit{\alpha }}·\mathbf{A},\hfill \end{array}$$

(13)

where $\mathcal{M}\left(8\right)$ represents the space of $8\times 8$ matrices over the real numbers [51]. Mapping $f(·)$ results in image set $\mathcal{C}\left(8\right)\subset \mathcal{M}\left(8\right)$ [51] that contains 8 × 8 matrices with the DCT matrix symmetry. In particular, for ${\mathit{\alpha }}_0=\sqrt{2}·{\left[\begin{array}{cccccc}{c}_1& c_2& c_3& c_5& c_6& c_7\end{array}\right]}^{\top }$, we have that $f\left({\mathit{\alpha }}_0\right)={\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}$. Other examples are ${\mathit{\alpha }}_1={\left[\begin{array}{ccccccc}1& 1& 1& 1& 1& 1& 1\end{array}\right]}^{\top }$ and ${\mathit{\alpha }}_2=1/2·{\left[\begin{array}{ccccccc}1& 2& 1& 1& 1& 1& 2\end{array}\right]}^{\top }$, which result in the following matrices:

$$\begin{array}{c}\hfill {\mathbf{T}}_{{\mathit{\alpha }}_1}=\left[\begin{array}{cccccccc}1& 1& 1& 1& 1& 1& 1& 1\\ 1& 1& 1& 1& -1& -1& -1& -1\\ 1& 1& -1& -1& -1& -1& 1& 1\\ 1& -1& -1& -1& 1& 1& 1& -1\\ 1& -1& -1& 1& 1& -1& -1& 1\\ 1& -1& 1& 1& -1& -1& 1& -1\\ 1& -1& 1& -1& -1& 1& -1& 1\\ 1& -1& 1& -1& 1& -1& 1& -1\end{array}\right]\mathrm{and}{\mathbf{T}}_{{\mathit{\alpha }}_2}=\frac{1}{2}\left[\begin{array}{cccccccc}2& 2& 2& 2& 2& 2& 2& 2\\ 1& 1& 1& 2& -2& -1& -1& -1\\ 2& 1& -1& -2& -2& -1& 1& 2\\ 1& -2& -1& -1& 1& 1& 2& -1\\ 2& -2& -2& 2& 2& -2& -2& 2\\ 1& -1& 2& 1& -1& -2& 1& -1\\ 1& -2& 2& -1& -1& 2& -2& 1\\ 2& -1& 1& -1& 1& -1& 1& -2\end{array}\right],\end{array}$$

(14)

respectively. Although the above matrices have low complexity, they may not necessarily lead to a good transform matrix in terms of mathematical properties and coding capability.

Hereafter, we adopt the following notation:

$$\begin{array}{cc}\hfill {\mathbf{M}}_{\mathit{\alpha }}& =\left[\begin{array}{cc}{\mathbf{E}}_{\mathit{\alpha }}& {\mathbf{0}}_4\\ {\mathbf{0}}_4& {\mathbf{O}}_{\mathit{\alpha }}\end{array}\right],\hfill \end{array}$$

(15)

where ${\mathbf{0}}_4$ is the the $4\times 4$ null matrix, and

$$\begin{array}{cc}\hfill {\mathbf{E}}_{\mathit{\alpha }}& =\left[\begin{array}{cccc}1& 1& 1& 1\\ 1& -1& -1& 1\\ {\alpha }_2& {\alpha }_5& -{\alpha }_5& -{\alpha }_2\\ {\alpha }_5& -{\alpha }_2& {\alpha }_2& -{\alpha }_5\end{array}\right]\mathrm{and}{\mathbf{O}}_{\mathit{\alpha }}=\left[\begin{array}{cccc}-{\alpha }_1& {\alpha }_3& -{\alpha }_4& {\alpha }_6\\ -{\alpha }_4& -{\alpha }_1& -{\alpha }_6& {\alpha }_3\\ {\alpha }_3& {\alpha }_6& -{\alpha }_1& {\alpha }_4\\ {\alpha }_6& {\alpha }_4& {\alpha }_3& {\alpha }_1\end{array}\right].\hfill \end{array}$$

(16)

Submatrices ${\mathbf{E}}_{\mathit{\alpha }}$ and ${\mathbf{O}}_{\mathit{\alpha }}$ compute the even and odd index DCT components, respectively.

2.3. Matrix Inversion

The inverse of ${\mathbf{T}}_{\alpha }$ is directly given by:

$$\begin{array}{c}\hfill {\mathbf{T}}_{\alpha }^{-1}={\mathbf{A}}^{-1}·{\mathbf{M}}_{\mathit{\alpha }}^{-1}·{\mathbf{P}}^{-1}.\end{array}$$

(17)

Because ${\mathbf{A}}^{-1}=\frac{1}{2}{\mathbf{A}}^{\top }=\frac{1}{2}\mathbf{A}$ and ${\mathbf{P}}^{-1}={\mathbf{P}}^{\top }$ are well-defined, nonsingular matrices, we need only check the invertibility of ${\mathbf{M}}_{\mathit{\alpha }}$ [52]. By means of symbolic computation, we obtain:

$$\begin{array}{c}\hfill {\mathbf{M}}_{\mathit{\alpha }}^{-1}=\frac{1}{4}·{\left({\mathbf{M}}_{{\mathit{\alpha }}^{\mathbf{\prime }}}\right)}^{\top },\end{array}$$

(18)

where ${\mathit{\alpha }}^{\prime }={\left[\begin{array}{cccccc}{\alpha }_1^{\prime }& {\alpha }_2^{\prime }& {\alpha }_3^{\prime }& {\alpha }_4^{\prime }& {\alpha }_5^{\prime }& {\alpha }_6^{\prime }\end{array}\right]}^{\top }$ with coefficients equals to:

$$\begin{array}{cc}\hfill {\alpha }_1^{\prime }& =-4({\alpha }_1^3+2{\alpha }_1{\alpha }_3{\alpha }_4+{\alpha }_1{\alpha }_6^2+{\alpha }_3^2{\alpha }_6-{\alpha }_4^2{\alpha }_6)/\det \left({\mathbf{O}}_{\mathit{\alpha }}\right),\hfill \\ \hfill {\alpha }_2^{\prime }& =-16{\alpha }_2/\det \left({\mathbf{E}}_{\mathit{\alpha }}\right),\hfill \\ \hfill {\alpha }_3^{\prime }& =4(-{\alpha }_1^2{\alpha }_4-2{\alpha }_1{\alpha }_3{\alpha }_6-{\alpha }_3^3-{\alpha }_3{\alpha }_4^2+{\alpha }_4{\alpha }_6^2)/\det \left({\mathbf{O}}_{\mathit{\alpha }}\right),\hfill \\ \hfill {\alpha }_4^{\prime }& =-4({\alpha }_1^2{\alpha }_3-2{\alpha }_1{\alpha }_4{\alpha }_6+{\alpha }_3^2{\alpha }_4-{\alpha }_3{\alpha }_6^2+{\alpha }_4^3)/\det \left({\mathbf{O}}_{\mathit{\alpha }}\right),\hfill \\ \hfill {\alpha }_5^{\prime }& =-16{\alpha }_5/\det \left({\mathbf{E}}_{\mathit{\alpha }}\right),\hfill \\ \hfill {\alpha }_6^{\prime }& =4(-{\alpha }_1^2{\alpha }_6-{\alpha }_1{\alpha }_3^2+{\alpha }_1{\alpha }_4^2+2{\alpha }_3{\alpha }_4{\alpha }_6-{\alpha }_6^3)/\det \left({\mathbf{O}}_{\mathit{\alpha }}\right),\hfill \end{array}$$

(19)

where $\det (·)$ returns the determinant.

Note that the expression in Matrix (18) implies that the inverse of the matrix ${\mathbf{M}}_{\mathit{\alpha }}$ is a matrix with the same structure, whose coefficients are a function of the parameter vector $\mathit{\alpha }$. For the matrix inversion to be well-defined, we must have: $\det \left({\mathbf{E}}_{\mathit{\alpha }}\right)·\det \left({\mathbf{O}}_{\mathit{\alpha }}\right)\ne 0$. By explicitly computing $\det \left({\mathbf{E}}_{\mathit{\alpha }}\right)$ and $\det \left({\mathbf{O}}_{\mathit{\alpha }}\right)$, we obtain the following condition for matrix inversion:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\alpha }_2^2+{\alpha }_5^2\ne 0,\hfill \\ \frac{{({\alpha }_1^2+{\alpha }_6^2)}^2-4{\alpha }_3{\alpha }_4({\alpha }_6^2-{\alpha }_1^2)}{{({\alpha }_3^2+{\alpha }_4^2)}^2-4{\alpha }_1{\alpha }_6({\alpha }_4^2-{\alpha }_3^2)}\ne -1.\hfill \end{array}\right.\end{array}$$

(20)

2.4. Orthogonality

In this paper, we adopt the following definitions. A matrix $\mathbf{T}$ is orthonormal if $\mathbf{T}·{\mathbf{T}}^{\top }$ is an identity matrix [53]. If product $\mathbf{T}·{\mathbf{T}}^{\top }$ is a diagonal matrix, $\mathbf{T}$ is said to be orthogonal. For ${\mathbf{T}}_{\mathit{\alpha }}$, we have that symbolic computation yields:

$$\begin{array}{cc}\hfill {\mathbf{T}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }}^{\top }& =\left[\begin{array}{cccccccc}8& 0& 0& 0& 0& 0& 0& 0\\ 0& 2s_1& 0& -2d& 0& 2d& 0& 0\\ 0& 0& 2s_0& 0& 0& 0& 0& 0\\ 0& -2d& 0& 2s_1& 0& 0& 0& 2d\\ 0& 0& 0& 0& 8& 0& 0& 0\\ 0& 2d& 0& 0& 0& 2s_1& 0& 2d\\ 0& 0& 0& 0& 0& 0& 2s_0& 0\\ 0& 0& 0& 2d& 0& 2d& 0& 2s_1\end{array}\right],\hfill \end{array}$$

(21)

where $s_0=2({\alpha }_2^2+{\alpha }_5^2)$, $s_1={\alpha }_1^2+{\alpha }_3^2+{\alpha }_4^2+{\alpha }_6^2$, and $d={\alpha }_1({\alpha }_4-{\alpha }_3)+{\alpha }_6({\alpha }_4+{\alpha }_3)$. Thus, if $d=0$, then the transform ${\mathbf{T}}_{\mathit{\alpha }}$ is orthogonal.

2.5. Near Orthogonality

Some important and well-known DCT approximations are nonorthogonal [30,34]. Nevertheless, such transformations are nearly orthogonal [39,54]. Let $\mathbf{A}$ be a square matrix. Deviation from orthogonality can be quantified according to the deviation from diagonality [39] of $\mathbf{A}·{\mathbf{A}}^{\top }$, which is given by the following expression:

$$\begin{array}{cc}\hfill \delta (\mathbf{A}·{\mathbf{A}}^{\top })& =1-\frac{\Vert \mathrm{diag}(\mathbf{A}·{\mathbf{A}}^{\top }){\Vert }_{\mathsf{F}}^2}{\Vert \mathbf{A}·{\mathbf{A}}^{\top }{\Vert }_{\mathsf{F}}^2},\hfill \end{array}$$

(22)

where $\mathrm{diag}(·)$ returns a diagonal matrix with the diagonal elements of its argument and ${\Vert ·\Vert }_{\mathsf{F}}$ denotes the Frobenius norm [2]. Therefore, considering ${\mathbf{T}}_{\mathit{\alpha }}$, we obtain:

$$\begin{array}{cc}\hfill \delta ({\mathbf{T}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }}^{\top })& =1-\frac{1}{1+\frac{32d^2}{128+8s_0^2+16s_1^2}}.\hfill \end{array}$$

(23)

Nonorthogonal transforms have been recognized as useful tools. The signed DCT (SDCT) is a particularly relevant DCT approximation [30] and its deviation from orthogonality is $0.20$. We adopt such deviation as a reference value to discriminate nearly orthogonal matrices. Thus, for ${\mathbf{T}}_{\mathit{\alpha }}$, we obtain the following criterion for near orthogonality:

$$\begin{array}{cc}\hfill 0<d^2& \le 1+\frac{s_0^2}{16}+\frac{s_1^2}{8}.\hfill \end{array}$$

(24)

2.6. Orthonormalization

Discrete transform approximations are often sought to be orthonormal. Orthogonal transformations can be orthonormalized as described in References [2,33,53]. Based on polar decomposition [55], orthonormal or nearly orthonormal matrix ${\widehat{\mathbf{C}}}_{\mathit{\alpha }}$ linked to ${\mathbf{T}}_{\mathit{\alpha }}$ is furnished by:

$$\begin{array}{c}\hfill {\widehat{\mathbf{C}}}_{\mathit{\alpha }}=\left\{\begin{array}{cc}{\mathbf{S}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }},\hfill & \mathrm{if}d=0,\hfill \\ {\tilde{\mathbf{S}}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }},\hfill & \mathrm{if}\left(24\right)\mathrm{holds}\mathrm{true},\hfill \end{array}\right.\end{array}$$

(25)

where ${\mathbf{S}}_{\mathit{\alpha }}=\sqrt[]{{\left({\mathbf{T}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }}^{\top }\right)}^{-1}}$, ${\tilde{\mathbf{S}}}_{\mathit{\alpha }}=\sqrt[]{\mathrm{diag}{\left({\mathbf{T}}_{\mathit{\alpha }}·{\mathbf{T}}_{\mathit{\alpha }}^{\top }\right)}^{-1}}$, and $\sqrt{·}$ is the matrix square root [2].

3. Assessment Criteria

In this section, we describe the selected figures of merit for assessing the performance and complexity of a given DCT approximation. We separated the following performance metrics: (i) total error energy [32,54]; (ii) mean square error (MSE) [2,49]; (iii) unified coding gain [2,54], and (iv) transform efficiency [2]. For computational complexity assessment, we adopted arithmetic operation counts as figures of merit.

3.1. Performance Metrics

3.1.1. Total Error Energy

Total error energy quantifies the error between matrices in a Euclidean distance way. This measure is given by References [32,54]:

$$\begin{array}{cc}\hfill ϵ\left({\widehat{\mathbf{C}}}_{\mathit{\alpha }}\right)& =\pi ·\Vert {\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}-{\widehat{\mathbf{C}}}_{\mathit{\alpha }}{\Vert }_{\mathsf{F}}^2.\hfill \end{array}$$

(26)

3.1.2. Mean Square Error

The MSE of given matrix approximation ${\widehat{\mathbf{C}}}_{\mathit{\alpha }}$ is furnished by:

$$\begin{array}{cc}\hfill \mathrm{MSE}\left({\widehat{\mathbf{C}}}_{\mathit{\alpha }}\right)=\frac{1}{8}& ·\mathrm{trace}\left(({\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}-{\widehat{\mathbf{C}}}_{\mathit{\alpha }})·{\mathbf{R}}_{\mathbf{xx}}\right.\hfill \\ & ·\left.{({\mathbf{C}}_{\mathrm{Loeffler}-\mathrm{DCT}}-{\widehat{\mathbf{C}}}_{\mathit{\alpha }})}^{\top }\right),\hfill \end{array}$$

(27)

where ${\mathbf{R}}_{\mathbf{xx}}$ represents the autocorrelation matrix of a Markov I stationary process with correlation coefficient $\rho$, and $\mathrm{trace}(·)$ returns the sum of main diagonal elements of its matrix argument. The $(i,j)$-th entry of ${\mathbf{R}}_{\mathbf{xx}}$ is given by ${\rho }^{|i-j|}$, $i,j=0,1,\dots ,7$ [1,2]. The correlation coefficient is assumed as equal to 0.95, which is representative for natural images [2].

3.1.3. Unified Transform Coding Gain

Unified transform coding gain provides a measure to quantify the compression capabilities of a given matrix [54]. It is a generalization of usual transform coding gain as in Reference [2]. Let ${\mathbf{g}}_k$ and ${\mathbf{h}}_k$ be the kth row of ${\widehat{\mathbf{C}}}_{\mathit{\alpha }}^{\top }$ and ${\mathbf{C}}_{\mathit{\alpha }}$, respectively. Then, the unified transform coding gain is given by Reference [56]:

$$\begin{array}{cc}\hfill C_g\left({\widehat{\mathbf{C}}}_{\mathit{\alpha }}\right)& =10·{\log }_{10}\left[\prod _{k=1}^8\frac{1}{{(A_k·B_k)}^{\frac{1}{8}}}\right]\left(\mathrm{dB}\right),\hfill \end{array}$$

(28)

where $A_k=\mathrm{sum}[({\mathbf{h}}_k·{\mathbf{h}}_k^{\top })\circ {\mathbf{R}}_{\mathbf{xx}}]$, $\mathrm{sum}(·)$ returns the sum of elements of its matrix argument, operator ∘ denotes the element-wise matrix product, $B_k={\Vert {\mathbf{g}}_k\Vert }_2^2$, and ${\Vert ·\Vert }_2$ is the usual vector norm.

3.1.4. Transform Efficiency

Another measure for assessing coding performance is transform efficiency [2]. Let matrix ${\mathbf{R}}_{\mathbf{XX}}={\widehat{\mathbf{C}}}_{\mathit{\alpha }}·{\mathbf{R}}_{\mathbf{xx}}·{\widehat{\mathbf{C}}}_{\mathit{\alpha }}^{\top }$ be the covariance matrix of transformed signal $\mathbf{X}$. The transform efficiency of ${\widehat{\mathbf{C}}}_{\mathit{\alpha }}$ is given by [2]:

$$\begin{array}{cc}\hfill \eta \left({\widehat{\mathbf{C}}}_{\mathit{\alpha }}\right)& =\frac{\mathrm{trace}\left(|{\mathbf{R}}_{\mathbf{XX}}|\right)}{\mathrm{sum}\left(|{\mathbf{R}}_{\mathbf{XX}}|\right)}.\hfill \end{array}$$

(29)

3.2. Computational Cost

Based on Loeffler DCT factorization, a fast algorithm for ${\mathbf{T}}_{\mathit{\alpha }}$ is obtained and its signal flow graph (SFG) is shown in Figure 1. Stage 1, 2, and 3 correspond to matrices $\mathbf{A}$, ${\mathbf{M}}_{\mathit{\alpha }}$, and $\mathbf{P}$, respectively. The computational cost of such an algorithm is closely linked to selected parameter values ${\alpha }_k$, $k=1,2,\dots ,6$. Because we aim at proposing multiplierless approximations, we restricted parameter values ${\alpha }_k$ to set $\mathcal{P}=\{0,\pm 1/2,\pm 1,\pm 2\}$, i.e., $\mathit{\alpha }\in {\mathcal{P}}^6$. The elements in $\mathcal{P}$ correspond to trivial multiplications that affect null multiplicative complexity. In fact, the elements in $\mathcal{P}$ represent only additions and minimal bit-shifting operations.

The number of additions and bit-shifts can be evaluated by inspecting the discussed algorithm (Figure 1). Thus, we obtain the following expressions for the addition and bit-shifting counts, respectively:

$$\begin{array}{cc}\hfill A\left(\mathit{\alpha }\right)=& 8+2·\max \left\{1,{\mathbf{1}}_{\mathcal{P}-\left\{0\right\}}\left({\alpha }_2\right)+{\mathbf{1}}_{\mathcal{P}-\left\{0\right\}}\left({\alpha }_5\right)\right\}\hfill \\ & +4·\max \left\{1,\sum _{k\in \{1,3,4,6\}}{\mathbf{1}}_{\mathcal{P}-\left\{0\right\}}\left({\alpha }_k\right)\right\},\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill S\left(\mathit{\alpha }\right)=& 2·[{\mathbf{1}}_{\{\pm \frac{1}{2},\pm 2\}}\left({\alpha }_2\right)+{\mathbf{1}}_{\{\pm \frac{1}{2},\pm 2\}}\left({\alpha }_5\right)]\hfill \\ & +4·\sum _{k\in \{1,3,4,6\}}{\mathbf{1}}_{\{\pm \frac{1}{2},\pm 2\}}\left({\alpha }_k\right),\hfill \end{array}$$

(31)

where ${\mathbf{1}}_A\left(x\right)=1$, if $x\in A$, and 0 otherwise.

4. Multicriteria Optimization and New Transforms

In this section, we introduce an optimization problem that aims at identifying optimal transformations derived from the proposed mapping (Matrix (13)). Considering the various performances and complexity metrics discussed in the previous section, we set up the following multicriteria optimization problem [57,58]:

$$\begin{array}{c}\hfill \underset{\mathit{\alpha }\in {\mathcal{P}}^6}{\min }\left(ϵ({\widehat{\mathbf{C}}}_{\mathit{\alpha }}),\mathrm{MSE}({\widehat{\mathbf{C}}}_{\mathit{\alpha }}),-C_g({\widehat{\mathbf{C}}}_{\mathit{\alpha }}),-\eta ({\widehat{\mathbf{C}}}_{\mathit{\alpha }}),A(\mathit{\alpha }),S(\mathit{\alpha })\right),\end{array}$$

(32)

subject to:

i

the existence of inverse transformation, according to the condition established in Matrix (20);

ii

the entries of the inverse matrix must be in $\mathcal{P}$; to ensure both forwarded and inverse low-complexity transformations;

iii

the property of orthogonality or near-orthogonality according to the criterion in Equation (24).

Quantities $C_g({\widehat{\mathbf{C}}}_{\mathit{\alpha }})$ and $\eta ({\widehat{\mathbf{C}}}_{\mathit{\alpha }})$ are in negative form to comply to the minimization requirement.

Being a multicriteria optimization problem, Problem (32) is based on objective function set $\mathcal{F}=\{ϵ(·),\mathrm{MSE}(·),-C_g(·),-\eta (·),A(·),S(·)\}$. The problem in analysis is discrete and finite since there is a countable number of values to the objective function. However, the nonlinear, discrete nature of the problem renders it unsuitable for analytical methods. Therefore, we employed exhaustive search methods to solve it. The discussed multicriteria problem requires the identification of the Pareto efficient solutions set [57], which is given by:

$$\begin{array}{cc}\hfill \{{\mathit{\alpha }}^{\mathbf{\ast }}\in {\mathcal{P}}^6:& \mathrm{there}\mathrm{is}\mathrm{no}\alpha \in {\mathcal{P}}^6,\mathrm{such}\mathrm{that}\hfill \\ & f\left(\mathit{\alpha }\right)\le f\left({\mathit{\alpha }}^{\mathbf{\ast }}\right)\mathrm{for}\mathrm{all}f\in \mathcal{F}\mathrm{and}\hfill \\ & f_0\left(\mathit{\alpha }\right)<f_0\left({\mathit{\alpha }}^{\mathbf{\ast }}\right)\mathrm{for}\mathrm{some}{f}_0\in \mathcal{F}\}.\hfill \end{array}$$

(33)

4.1. Efficient Solutions

The exhaustive search [57] returned six efficient parameter vectors, which are listed in

Table 1. Efficient solutions.

i	${\mathit{\alpha }}_{\mathit{i}}^{\mathbf{\ast }}$
1	${\left[110000\right]}^{\top }$
2	${\left[1100\frac{1}{2}0\right]}^{\top }$
3	${\left[111000\right]}^{\top }$
4	${\left[1111\frac{1}{2}0\right]}^{\top }$
5	${\left[120010\right]}^{\top }$
6	${\left[121110\right]}^{\top }$

. For ease of notation, we denote the low-complexity matrices and their associated approximations linked to efficient solutions according to: ${\mathbf{T}}_i\triangleq {\mathbf{T}}_{{\mathit{\alpha }}_i^{\ast }}$ and ${\widehat{\mathbf{C}}}_i\triangleq {\widehat{\mathbf{C}}}_{{\mathit{\alpha }}_i^{\ast }}$, respectively.

Table 2. Efficient Loeffler-based discrete cosine transform (DCT) approximations and the DCT.

Transform	$\mathit{ϵ}$	MSE	${\mathit{C}}_{\mathit{g}}$	$\mathit{\eta }$	$\mathit{A}\left(\mathit{\alpha }\right)$	$\mathit{S}\left(\mathit{\alpha }\right)$	Orthonormal?	Description
${\widehat{\mathbf{C}}}_1$	$8.66$	$0.059$	$7.33$	$80.90$	14	0	Yes	Proposed in Reference [33]
${\widehat{\mathbf{C}}}_2$	$7.73$	$0.056$	$7.54$	$81.99$	16	2	Yes	Proposed as ${\mathbf{T}}_{10}$ in Reference [54]
${\widehat{\mathbf{C}}}_3$	$1.44$	$0.007$	$8.30$	$89.77$	18	0	No	Proposed as ${\tilde{\mathbf{T}}}_1$ in Reference [39]
${\widehat{\mathbf{C}}}_4$	$0.87$	$0.006$	$8.39$	$88.70$	24	2	Yes	Proposed as ${\widehat{\mathbf{D}}}_1$ in Reference [31]
${\widehat{\mathbf{C}}}_5$	$7.73$	$0.056$	$7.54$	$81.99$	16	2	Yes	Equivalent to ${\widehat{\mathbf{C}}}_2$ and proposed as ${\mathbf{T}}_4$ in Reference [54]
${\widehat{\mathbf{C}}}_6$	$0.87$	$0.006$	$8.39$	$88.70$	24	2	Yes	Equivalent to ${\widehat{\mathbf{C}}}_4$ and proposed as ${\mathbf{T}}_9$ in Reference [54]
DCT	0	0	$8.85$	$93.99$	–	–	Yes	Exact DCT as described in Reference [2]

summarizes the performance metrics, arithmetic complexity, and orthonormality property of obtained matrices ${\widehat{\mathbf{C}}}_i$, $i=1,2,\dots ,6$. We included the DCT for reference as well. Note that all DCT approximations except those by ${\widehat{\mathbf{C}}}_3$ are orthonormal.

4.2. Comparison

Several DCT approximations are encompassed by the proposed matrix formalism. Such transformations include: the SDCT [30], the approximation based on round-off function proposed in Reference [38], and all the DCT approximations introduced in Reference [39]. For instance, the SDCT [30] is another particular transformation fully described by the proposed matrix mapping. In fact, the SDCT can be obtained by taking $f\left({\mathit{\alpha }}_1\right)$, where ${\mathit{\alpha }}_1={\left[\begin{array}{cccccc}1& 1& 1& 1& 1& 1\end{array}\right]}^{\top }$. Nevertheless, none of these approximations is part of the Pareto efficient solution set induced by the discussed multicriteria optimization problem [57]. Therefore, we compare the obtained efficient solutions with a variety of state-of-the-art 8-point DCT approximations that cannot be described by the proposed Loeffler-based formalism. We separated the Walsh–Hadamard transform (WHT) and the Bouguezel–Ahmad–Swamy (BAS) series of approximations labeled ${\mathrm{BAS}}_1$ [34], ${\mathrm{BAS}}_2$ [35], ${\mathrm{BAS}}_3$ [41], ${\mathrm{BAS}}_4$ [36], ${\mathrm{BAS}}_5$ [37] (for $a=1$), ${\mathrm{BAS}}_6$ [37] (for $a=0$), ${\mathrm{BAS}}_7$ [37] (for $a=1/2$), and ${\mathrm{BAS}}_8$ [5].

Table 3. Performance of the Bouguezel–Ahmad–Swamy (BAS) approximations, the Walsh–Hadamard transform (WHT), and the DCT.

Transform	Orthogonalizable	$\mathit{ϵ}$	MSE	${\mathit{C}}_{\mathit{g}}$	$\mathit{\eta }$	$\mathit{A}\left(\mathit{\alpha }\right)$	$\mathit{S}\left(\mathit{\alpha }\right)$
${\mathrm{BAS}}_1$ [34]	No	$4.19$	$0.019$	$6.27$	$83.17$	21	0
${\mathrm{BAS}}_2$ [35]	Yes	$5.93$	$0.024$	$8.12$	$86.86$	18	2
${\mathrm{BAS}}_3$ [41]	Yes	$6.85$	$0.028$	$7.91$	$85.38$	18	0
${\mathrm{BAS}}_4$ [36]	Yes	$4.09$	$0.021$	$8.33$	$88.22$	24	4
${\mathrm{BAS}}_5$ [37]	Yes	$26.86$	$0.071$	$7.91$	$85.38$	18	0
${\mathrm{BAS}}_6$ [37]	Yes	$26.86$	$0.071$	$7.91$	$85.64$	16	0
${\mathrm{BAS}}_7$ [37]	Yes	$26.40$	$0.068$	$8.12$	$86.86$	18	2
${\mathrm{BAS}}_8$ [5]	Yes	$35.06$	$0.102$	$7.95$	$85.31$	24	0
WHT [2]	Yes	$5.05$	$0.025$	$7.95$	$85.31$	24	0
DCT [2]	Yes	0	0	$8.85$	$93.99$	–	–

shows the performance measures for these transforms. For completeness, we also show the unified coding gain and the transform efficiency measures for the exact DCT [2].

Some approximations, such as the SDCT, were not explicitly included in our comparisons. Although they are in the set of matrices generated by Loeffler parametrization, they are not in the efficient solution set. Thus, we removed them from further analyses for not being an optimal solution.

In order to compare all the above-mentioned transformations, we aimed at identifying the Pareto frontiers [57] in two-dimensional plots considering the performance figures of the obtained efficient solution as well as the WHT and BAS approximations. Thus, we devised scatter plots considering the arithmetic complexity and performance measures. The resulting plots are shown in Figure 2. Orthogonal transform approximations are marked with circles, and nonorthogonal approximations with cross signs. The dashed curves represent the Pareto frontier [57] for each selected pair of the measures. Transformations located on the Pareto frontier are considered optimal, where the points are dominated by the frontier correspond to nonoptimal transformations. The bivariate plots in Figure 2a,b reveal that the obtained Loeffler-based DCT approximations are often situated at the optimality site prescribed by the Pareto frontier. The Loeffler approximations perform particularly well in terms of total error energy and the MSE, which capture the matrix proximity to the exact DCT matrix in a Euclidean sense. Such approximations are particularly suitable for problems that require computational proximity to the the exact transformation as in the case of detection and estimation problems [59,60]. Regarding coding performance, Figure 2c,d shows that transformations ${\widehat{\mathbf{C}}}_1$, ${\widehat{\mathbf{C}}}_3$, ${\widehat{\mathbf{C}}}_6$, and ${\mathrm{BAS}}_6$ are situated on the Pareto frontier, being optimal in this sense. These approximations are adequate for data compression and decorrelation [2].

5. Image and Video Experiments

5.1. Image Compression

We implemented the JPEG-like compression experiment described in References [30,34,37,41,30] and submitted the standard Elaine image to processing at a high compression rate. For quantitative assessment, we adopted the structural similarity (SSIM) index [61] and the peak signal-to-noise rate (PSNR) [8,18] measure. Figure 3 shows the reconstructed Elaine image according to the JPEG-like compression considering the following transformations: DCT, ${\widehat{\mathbf{C}}}_1$, ${\widehat{\mathbf{C}}}_2$, ${\widehat{\mathbf{C}}}_3$, ${\widehat{\mathbf{C}}}_4$, and ${\mathrm{BAS}}_6$. We employed fixed-rate compression and retained only five coefficients, which led to 92.1875% compression rate. Despite very low computational complexity, the approximations could furnish images with quality comparable to the results obtained from the exact DCT. In particular, approximations ${\widehat{\mathbf{C}}}_1$ and ${\widehat{\mathbf{C}}}_2$ offered good trade-off, since they required only 14–16 additions and were capable of providing competitive image quality at smaller hardware and power requirements (Section 6).

Figure 4 shows the PSNR and SSIM for different number of retained coefficients. Considering PSNR measurements, the difference between the measurements associated to the ${\mathrm{BAS}}_6$ and to the efficient approximation were less than ≈1 dB. Similar behavior was reported when SSIM measurements were considered.

5.2. Video Compression

5.2.1. Experiments with the H.264/AVC Standard

To assess the Loeffler-based approximations in the context of video coding, we embedded them in the x264 [62] software library for encoding video streams into the H.264/AVC standard [63]. The original 8-point transform employed in H.264/AVC is a DCT-based integer approximation given by Reference [64]:

$$\begin{array}{c}\hfill {\widehat{\mathbf{C}}}_{\mathrm{AVC}}=\frac{1}{8}·\left[\begin{array}{cccccccc}8& 8& 8& 8& 8& 8& 8& 8\\ 12& 10& 6& 3& -3& -6& -10& -12\\ 8& 4& -4& -8& -8& -4& 4& 8\\ 10& -3& -12& -6& 6& 12& 3& -10\\ 8& -8& -8& 8& 8& -8& -8& 8\\ 6& -12& 3& 10& -10& -3& 12& -6\\ 4& -8& 8& -4& -4& 8& -8& 4\\ 3& -6& 10& -12& 12& -10& 6& -3\end{array}\right].\end{array}$$

(34)

The fast algorithm for the above transformation requires 32 additions and 14 bit-shifting operations [64].

We encoded 11 common intermediate format (CIF) videos with 300 frames from a public video database [65] using the standard and modified codec. In our simulation, we employed default settings and the resulting video quality was assessed by means of the average PSNR of chrominance and luminance representation considering all reconstructed frames. Psychovisual optimization was also disabled in order to obtain valid PSNR values and the quantization step was unaltered.

We computed the Bjøntegaard delta rate (BD-Rate) and the Bjøntegaard delta PSNR (BD-PSNR) [66] for modified codec compared to the usual H.264 native DCT approximation. BD-Rate and BD-PSNR were automatically furnished by the x264 [62] software library. A comprehensive review of Bjøntegaard metrics and their specific mathematical formulations can be found in Reference [67]. For that, we followed the procedure specified in References [66,67,68]. We adopted the same testing point as determined in Reference [69] with fixed quantization parameter (QP) in $\{22,27,32,37\}$. Following References [67,68], we employed cubic spline interpolation between the testing points for better visualization.

Figure 5 shows the resulting average BD-Rate distortion curves for the selected videos, where H.264 represents the integer DCT used in the x264 software library shown in Matrix (34). We note the superior performance of ${\widehat{\mathbf{C}}}_3$ and ${\widehat{\mathbf{C}}}_4$ approximations compared to BAS${}_2$ and BAS${}_6$ transforms. This performance is more evident for small bitrates. As the bitrate increases, the quality performance of ${\widehat{\mathbf{C}}}_3$, ${\widehat{\mathbf{C}}}_4$, BAS${}_2$, and BAS${}_6$ approximates native DCT implementation.

Table 4. Average Bjøntegaard delta (BD)-Rate and BD-PSNR for the 11 common intermediate format (CIF) videos selected from Reference [65].

Transform	BD-Rate (%)	BD-PSNR (dB)
${\widehat{\mathbf{C}}}_1$	$29.98$	$-1.08$
${\widehat{\mathbf{C}}}_2$	$31.40$	$-1.13$
${\widehat{\mathbf{C}}}_3$	$7.48$	$-0.29$
${\widehat{\mathbf{C}}}_4$	$3.94$	$-0.15$
BAS${}_2$	$17.43$	$-0.68$
BAS${}_6$	$13.90$	$-0.54$

shows the BD-Rate and BD-PSNR measures for the 11 CIF video sequence selected from Reference [65].

Figure 6a–f displays the first encoded frame of the ‘Foreman’ video sequence at $\mathrm{QP}=27$ according to the modified coded based on the proposed efficient approximations and BAS approximations. For reference, Figure 6g shows the resulting frame obtained from the standard H.264.

Note that Loeffler approximations ${\widehat{\mathbf{C}}}_1$ and ${\widehat{\mathbf{C}}}_2$, again, show better performance in terms of BD-PSNR measures than BAS transforms BAS${}_2$ and BAS${}_6$. We attribute the difference in quality between ${\widehat{\mathbf{C}}}_3$ and ${\widehat{\mathbf{C}}}_4$ in relation to ${\widehat{\mathbf{C}}}_1$ and ${\widehat{\mathbf{C}}}_2$ to the associated complexity discrepancy as depicted in Table 2. The reduction in the number of additions and bit-shifting operations come at the expense of performance.

5.2.2. Experiments with the H.265/HEVC Standard

We also embedded Loeffler-based approximations into the HM-16.3 H.265/HEVC reference software [70]. The H.265/HEVC employs the following integer transform:

$$\begin{array}{c}\hfill {\widehat{\mathbf{C}}}_{\mathrm{HEVC}}=\frac{1}{64}·\left[\begin{array}{cccccccc}64& 64& 64& 64& 64& 64& 64& 64\\ 89& 75& 50& 18& -18& -50& -75& -89\\ 83& 36& -36& -83& -83& -36& 36& 83\\ 75& -18& -89& -50& 50& 89& 18& -75\\ 64& -64& -64& 64& 64& -64& -64& 64\\ 50& -89& 18& 75& -75& -18& 89& -50\\ 36& -83& 83& -36& -36& 83& -83& 36\\ 18& -50& 75& -89& 89& -75& 50& -18\end{array}\right],\end{array}$$

(35)

which demands 72 additions and 62 bit-shifting operations, when considering the factorization method based on odd and even decomposition described in Reference [2]. Besides a particular 8-point integer transformation, the HEVC standard also employs 4-, 16-, and 32-point transforms [22]. To obtain a modified version of the H.265/HEVC codec, we derived 16- and 32-point approximations based on the scalable recursive method proposed in Reference [71]. The 8-point Loeffler-based efficient solutions were employed as fundamental building blocks for scaling up. Since the 4-point DCT matrix only possesses integer coefficients, it was employed unaltered. The additive complexity of the scaled 16- and 32-point approximations is $2·A\left(\mathit{\alpha }\right)+16$ and $4·A\left(\mathit{\alpha }\right)+64$, respectively [71]. For the bit-shifting count, we have $2·S\left(\mathit{\alpha }\right)$ and $4·S\left(\mathit{\alpha }\right)$, respectively.

In order to perform image-quality assessment, we encoded the first 100 frames of one standard sequence of each A to F video classes defined in the Common Test Conditions and Software Reference Configurations (CTCSRC) document [72]. The selected videos and their attributes are made explicit in the first two columns of

Table 5. Bjøntegaard metrics of the modified HEVC reference software for tested video sequences.

Class	Name	Resolution	BD-PSNR (dB) and BD-Rate (%)
		and Frame Rate	${\widehat{\mathbf{C}}}_1$	${\widehat{\mathbf{C}}}_2$	${\widehat{\mathbf{C}}}_3$	${\widehat{\mathbf{C}}}_4$	BAS${}_2$	BAS${}_6$
A	’PeopleOnStreet’	2560 × 1600	0.54 dB	0.53 dB	0.53 dB	0.34 dB	0.47 dB	0.50 dB
		30 fps	−9.75%	−9.61%	−9.49%	−6.30%	−8.47%	−8.99%
B	’BasketballDrive’	1920 × 1080	0.33 dB	0.32 dB	0.45 dB	0.20 dB	0.25 dB	0.27 dB
		50 fps	−11.56%	−11.28%	−15.52%	−7.24%	−8.90%	−9.50%
C	’RaceHorses’	832 × 480	0.67 dB	0.66 dB	0.83	0.52 dB	0.61 dB	0.61 dB
		30 fps	−8.06%	−7.98%	−9.70%	−6.36%	−7.38%	−7.38%
D	’BlowingBubbles’	416 × 240	0.26 dB	0.25 dB	0.26 dB	0.15 dB	0.20 dB	0.22 dB
		50 fps	−4.35%	−4.25%	−4.46%	−2.59%	−3.48%	−3.74%
E	’KristenAndSara’	1280 × 720	0.47 dB	0.46 dB	0.47 dB	0.30 dB	0.39 dB	0.41 dB
		60 fps	−8.97%	−8.78%	−9.00%	−5.93%	−7.49%	−7.90%
F	’BasketbalDrillText’	832 × 480	0.20 dB	0.20 dB	0.19 dB	0.12 dB	0.16 dB	0.17 dB
		50 fps	−3.82%	−3.79%	−3.63%	−2.26%	−3.06%	−3.28%

. Classes A and B present high-resolution content for broadcasting and video on demand services; Classes C and D are lower-resolution contents for the same applications; Class E reflects typical examples of video-conferencing data; and Class F provides computer-generated image sequences [73]. All encoding parameters, including QP values, were set according to the CTCSRC document for the Main profile and All-Intra (AI), Random Access (RA), Low Delay B (LD-B), and Low Delay P (LD-P) configurations. In the AI configuration, each frame from the input video sequence is independently coded. RA and LD (B and P) coding modes differ mainly by order of coding and outputting. In the former, the coding order and output order of the pictures (frames) may differ, whereas equal order is required for the latter (Reference [74], p. 94). RA configuration relates to broadcasting and streaming use, whereas LD (B and P) represent conventional applications.

It is worth mentioning that Class A test sequences are not used for LDB and LDP coding tests, whereas Class E videos are not included in the RA test cases because of the nature of the content they represent (Reference [74], p. 93).

As figures of merit, we computed the BD-Rate and BD-PSNR for the modified versions of the codec. Figure 7, Figure 8, Figure 9 and Figure 10 depict the rate-distortion curves and Table 5 presents the Bjøntegaard metrics for the considered sets of 8- to 32-point approximations for modes AI, RA, LDB, and LDP. BD-Rate and BD-PSNR are also automatically furnished by HM-16.3 H.265/HEVC reference software [70]. We employed the cubic spline interpolation based on four testing points on Figure 7, Figure 8, Figure 9 and Figure 10, as specified in References [67,68]. The curves denoted by HEVC represents the native integer approximations for the 8-, 16-, and 32-point DCT in the HM-16.3 H.265/HEVC software [70]. These four testing points were determined by the specification for HEVC as in Reference [69] with fixed $\mathrm{QP}\in \{22,27,32,37\}$. One can note that there is no significant image-quality degradation when considering approximate transforms. Furthermore, in all the cases, it can be seen that ${\widehat{\mathbf{C}}}_4$ performs better with a degradation of no more than $0.52$ dB.

The Loeffler-based approximations could present competitive image quality while possessing low computational complexity.

Table 6. Computational complexity for N-point transforms employed in the H.265/High Efficiency Video Coding (HEVC) experiments.

N	Transform	Add.	Mult.	Shifts
8	H.265/HEVC [75]	28	22	0
	${\mathbf{T}}_1$	14	0	0
	${\mathbf{T}}_3$	18	0	0
	${\mathbf{T}}_5$	16	0	2
	${\mathbf{T}}_6$	24	0	2
16	H.265/HEVC [75]	100	86	0
	${\mathbf{T}}_1$	44	0	0
	${\mathbf{T}}_3$	52	0	0
	${\mathbf{T}}_5$	48	0	4
	${\mathbf{T}}_6$	64	0	4
32	H.265/HEVC [75]	372	342	0
	${\mathbf{T}}_1$	120	0	0
	${\mathbf{T}}_3$	136	0	0
	${\mathbf{T}}_5$	128	0	8
	${\mathbf{T}}_6$	160	0	8

shows the arithmetic costs of both the original and modified extended versions, and the H.265/HEVC native transform. As a qualitative result, Figure 11 depicts the first frame of the ‘BasketballDrive’ video sequence resulting from the unmodified H.265/HEVC [70] and from the modified codec according to the discussed approximations and their scaled versions at $\mathrm{QP}=32$. We also display the related frames according to the BAS${}_2$ and BAS${}_6$ transforms.

6. FPGA Implementation

To compare the hardware-resource consumption of the discussed approximations, they were initially modeled and tested in Matlab Simulink and then physically realized on FPGA. The FPGA used was a Xilinx Virtex-6 XC6VLX240T installed on a Xilinx ML605 prototyping board. FPGA realization was tested with 100,000 random 8-point input test vectors using hardware cosimulation. Test vectors were generated from within the Matlab environment and routed to the physical FPGA device using a JTAG-based hardware cosimulation. Then, measured data from the FPGA was routed back to Matlab memory space.

We separated the ${\mathrm{BAS}}_6$ approximation for comparison with efficient approximations because its performance metrics lay on the Pareto frontier of the plots in Figure 2. The associated FPGA implementations were evaluated for hardware complexity and real-time performance using metrics such as configurable logic blocks (CLB) and flip-flop (FF) count, critical path delay ($T_{\mathrm{cpd}}$) in ns, and maximum operating frequency ($F_{\max }$) in MHz. Values were obtained from the Xilinx FPGA synthesis and place-route tools by accessing the xflow.results report file. In addition, static ($Q_p$) and dynamic power ($D_p$ in $\mathrm{mW}/\mathrm{GHz}$) consumption were estimated using the Xilinx XPower Analyzer. We also reported area-time complexity ($AT$) and area-time-squared complexity ($A{T}^2$). Circuit area (A) was estimated using the CLB count as a metric, and time was derived from $T_{\mathrm{cpd}}$.

Table 7. Hardware-resource consumption and power consumption using a Xilinx Virtex-6 XC6VLX240T 1FFG1156 device.

Approximation	CLB	FF	${\mathit{T}}_{\mathbf{cpd}}$$\left(\mathbf{ns}\right)$	${\mathit{F}}_{\mathbf{max}}$$\left(\mathbf{MHz}\right)$	${\mathit{D}}_{\mathit{p}}$$(\mathbf{mW}/\mathbf{GHz})$	${\mathit{Q}}_{\mathit{p}}$$\left(\mathbf{W}\right)$	$\mathit{AT}$	${\mathit{AT}}^2$
${\mathbf{T}}_1$	108	368	2.783	359	2.93	3.455	300.56	836.47
${\mathbf{T}}_3$	114	386	2.805	357	3.56	3.462	319.77	896.95
${\mathbf{T}}_5$	125	442	2.280	439	3.65	3.472	285	649.8
${\mathbf{T}}_6$	185	626	2.699	371	4.18	3.470	499.32	1347.65
${\mathrm{BAS}}_6$ [5]	116	316	2.740	365	4.05	3.468	317.84	870.88

lists the FPGA hardware resource and power consumption for each algorithm.

Considering the circuit complexity of the discussed approximations and ${\mathrm{BAS}}_6$ [5], as measured from the CBL count for the FPGA synthesis report, it can be seen from Table 7 that ${\mathbf{T}}_1$ is the smallest option in terms of circuit area. When considering maximum speed, matrix ${\mathbf{T}}_5$ showed the best performance on the Vertex-6 XC6VLX240T device. Alternatively, if we consider the normalized dynamic power consumption, the best performance was again measured from ${\mathbf{T}}_1$.

7. Conclusions and Final Remarks

In this paper, we introduced a mathematical framework for the design of 8-point DCT approximations. The Loeffler algorithm was parameterized and a class of matrices was derived. Matrices with good properties, such as low-complexity, invertibility, orthogonality or near-orthogonality, and closeness to the exact DCT, were separated according to a multicriteria optimization problem aiming at Pareto efficiency. The DCT approximations in this class were assessed, and the optimal transformations were separated.

The obtained transforms were assessed in terms of computational complexity, proximity, and coding measures. The derived efficient solutions constitute DCT approximations capable of good properties when compared to existing DCT approximations. At the same time, approximation requires extremely low computation costs: only additions and bit-shifting operations are required for their evaluation.

We demonstrated that the proposed method is a unifying structure for several approximations scattered in the literature, including the well-known approximation by Lengwehasatit–Ortega [31], and they share the same matrix factorization scheme. Additionally, because all discussed approximations have a common matrix expansion, the SFG of their associated fast algorithms are identical except for the parameter values. Thus, the resulting structure paves the way to performance-selectable transforms according to the choice of parameters.

Moreover, approximations were assessed and compared in terms of image and video coding. For images, a JPEG-like encoding simulation was considered, and for videos, approximations were embedded in the H.264/AVC and H.265/HEVC video-coding standards. Approximations exhibited good coding performance compared to exact DCT-based JPEG compression and the obtained frame video quality was very close to the results shown by the H.264/AVC and H.265/HEVC standards.

Extensions for larger blocklengths can be achieved by considering scalable approaches such as the one suggested in Reference [71]. Alternatively, one could employ direct parameterization of the 16-point Loeffler DCT algorithm described in Reference [27].