An Analytic Transform Kernel Derivation Method for Video Codecs

Abstract

In the standardization of versatile video coding (VVC), discrete cosine transform (DCT)-2, discrete sine transform (DST)-7, and DCT-8 are regarded as the primary transform kernels. However, DST-4 and DCT-4 can also be considered as the transform kernels instead of using DST-7 and DCT-8 owing to their effectiveness in smaller resolution test sequences. To implement these different block size transform kernels, a considerable amount of memory has to be allocated. Moreover, memory consumption to store different block size transform kernels is regarded as a major issue in video coding standardization. To address this problem, a common sparse unified matrix concept is introduced in this study, where any block size transform kernel matrix can be obtained after some mathematical operations. The proposed common sparse unified matrix saves approximately 80% of the static memory by storing only a few transform kernel elements for DCT-2, DST-7, and DCT-8. Full-required transform kernels are derived using the stored transform kernels and generated unit-element matrices and a permutation matrix. The static memory required is only for 1648 elements instead of 8180 elements, each with 8-bit precision. The defined common sparse unified matrix is composed of two parts: a unified DST-3 matrix and a grouped DST-7 matrix. The unified DST-3 matrix is used to derive different points of DCT-2 transform kernels, and the grouped DST-7 matrix is used to derive different points of DST-7 and DCT-8 transform kernels. The new technique of grouping concept is introduced, which shows the relationship between different rows of DST-7 transform kernels with various block sizes. The proposed grouping concept supports the fast algorithm of DST-7 by implementing the proposed method of the “one group one feature” principle. The simulation was conducted using the VTM-3.0 reference software under common test conditions. The simulation result of the all intra (AI) configuration is Y = 0.00%, U = −0.02%, V = 0.00% with an encoding time of 100%, and a decoding time of 100%. Similarly, the simulation results of random access (RA) configuration are Y = −0.01%, U = 0.09%, V = 0.06%, and the encoding and decoding times are 101% and 100%, respectively. The simulation result of the low delay B (LDB) configuration is Y = 0.01%, U = 0.08%, and V = −0.27%, for encoding and decoding times of 101% and 100%, respectively.

1. Introduction

Various standardization works for video compression have been performed, including H.261, H.262, H.263 [1], H.264/advanced video coding [2], and H.265/ high-efficiency video coding (HEVC) [3]. Recently, the standardization process of versatile video coding (VVC) [4], which aims to provide a significant improvement in compression performance over the existing HEVC standard and aid the deployment of higher quality video services and emerging applications, such as 360° omnidirectional immersive multimedia and high-dynamic-range video, has been standardized. In the standardization of VVC, discrete cosine transform-2 (DCT-2), discrete sine transform-7 (DST-7), and DCT-8 are regarded as three vital primary transform kernels [5,6,7,8,9], which can also be termed as the multiple transform set (MTS) [10,11]. The benefits of MTS are usually to enlarge transform sets when necessary, that is, reduced distortion in the R-D trade-off as complex residuals exploit the multiple transforms and reduce transform sets when necessary. This means reduced bitrate in the R-D trade-off by avoiding wasting bits signaling the transforms when necessary [12]. Similarly, DST-7 approximates the optimal transform better than DCT-2 along the prediction direction, that is, for intra prediction residuals [11].

Along with these three transform kernels, DST-4 and DCT-4 can also be regarded as the replacement of DST-7 and DCT-8, respectively, because they exhibit similar kernel behavior and show better coding efficiency for smaller resolution sequences [13].

The used transform kernel elements are derived based on the following mathematical equations.

DCT-2

$${\left[C_2\right]}_{n,k}=\sqrt{\frac{2}{N}}{\epsilon }_{k}cos\left(\frac{\pi \left(2n+1\right)k}{2N}\right)$$

(1)

where ${\epsilon }_k=\left\{\begin{array}{c}\frac{1}{\surd 2} k=0\\ 1 otherwise\end{array}\right.$

DCT-4

$${\left[C_4\right]}_{n,k}=\sqrt{\frac{2}{N}}cos\left(\frac{\pi \left(2n+1\right)\left(2k+1\right)}{4N}\right)$$

(2)

DCT-8

$${\left[C_8\right]}_{n,k}=\frac{2}{\sqrt{2\left(N-1\right)}}cos\left(\frac{\pi \left(2n+1\right)\left(2k+1\right)}{4N-2}\right)$$

(3)

DST-4

$${\left[S_8\right]}_{n,k}=\sqrt{\frac{2}{2N}}{\epsilon }_{k}sin\left(\frac{\pi \left(2n+1\right)\left(k+1\right)}{2N}\right)$$

(4)

where, ${\epsilon }_k=\left\{\begin{array}{c}\frac{1}{\surd 2} k=N-1\\ 1 otherwise\end{array}\right.$

DST-7

$${\left[S_7\right]}_{n,k}=\frac{2}{\sqrt{2N+1}}sin\left(\frac{\pi \left(2k+1\right)\left(n+1\right)}{2N+1}\right)$$

(5)

where N represents the block size.

Several design aspects of HEVC transform coding are inherited in the VVC codec. In addition to the conventional DCT-2, alternate transform types like DST-7 and DCT-8 are also adopted in the VVC, which is generally referred to as primary transform because it is first applied to the predicted residual pixels and named as contrast meaning of the secondary transform [14] in VVC. The size of DCT-2 ranges from 4-point to 64-point, whereas that of DST-7 and DCT-8 ranges from 4-point to 32-point. The kernels elements defined in VVC are composed of 8-bit signed integer values. The additional integer transform kernels defined in VVC are derived by scaling the floating-point transform kernel with 64 $\sqrt{N}$, where N represents the transform size [14]. For the alignment of the worst-case multiplications per coefficients with HEVC, for 64-point DCT-2 and 32-point DST-7/DCT-8, only the first 32 and 16 low-frequency coefficients are kept, and the higher frequency coefficients are zeroed out respectively, which is also considered in the last coefficient position coding and the coefficient group scanning [15].

All the kernels elements are stored in the reference software VTM-3.0 in an 8-bit representation. A single 64-point DCT-2 kernel matrix is newly defined in VCC, and other sizes of DCT-2 are kept unchanged with HEVC. Other individual sets of 4-, 8-, 16-, and 32-point DST-7 and DCT-8 kernel matrices are also stored in the reference software [16], adding together to the storage of 8180 elements which is a prime concern for the memory. Unlike the conventional video codec such as HEVC, the partial butterfly structure for the transform of DST-7 and DCT-8 is avoided in the transform process to be compatible with the virtual pipeline process and low delay in hardware design, which is thoroughly investigated and agreed in the standardization works of VVC. Instead, the fast transform method is designed to support the dual implementation of matrix multiplication and butterfly operation [17]. Thus, all kernel elements for matrix multiplication operation should be stored in the codec devices or software. However, the number of multiplication and addition is significantly reduced even though matrix operations are performed with matrix multiplications by utilizing several features of DST-7 and DCT-8 kernels [17].

Table 1. MTS transform kernels based on prediction and block sizes in VTM-3.0.

	Prediction	Prediction Tools	Block Size Checks	Transform Kernels
MTS	Intra	ISP [18] (intra sub-partition)	4 × 4 up to 16 × 16	DCT-2, DST-7
		Normal intra-prediction mode	4 × 4 up to N × 32/32 × N	DCT-2, DST-7
	Inter	SBT [19] (sub-block transform)	4 × 4 up to N × 32/32 × N	DST-7, DCT-8 depends on SBT position

shows the use of MTS transform kernels based on the prediction modes and block sizes in VVC.

Table 1 shows the use of multiple transform set (MTS) for intra and inter prediction. In the intra prediction, various prediction tools have been introduced in VVC, which are intra-sub-partition (ISP) and normal prediction. For ISP, the combinations of DCT-2 and DST-7 transform kernels are used as the horizontal and vertical transform types with a transform block size of 4 × 4 up to 16 × 16. For the normal intra-prediction modes, the combinations of DCT-2 and DST-7 transform kernel sets are used, based on the prediction mode and comparison of width and height of the block sizes, ranging from 4 to 16 pixels in accordance with the spec text [15]. The 16-point or 32-point DST-7 is applied to non-zero coefficients transform blocks in either the horizontal or vertical direction. For inter prediction, using sub-block transform (SBT) [19], as a prediction tool, the combinations of DST-7 and DCT-8 are applied on the partitioned transform unit block for a block size up to 32 pixels, depending on the cu_sbt_horizontal_flag and cu_sbt_vertical_flag [15] in VTM-3.0.

Despite significant coding gain improvement owing to the usage of MTS for different block sizes and kernels, memory storage of kernel elements is regarded as one of the crucial issues for the standardization of VVC. In MTS, DST-4 and DCT-4 can be used as a replacement for the DST-7 and DCT-8 transform kernels, respectively, owing to their relatively easier kernel derivation by sub-sampling of even and odd rows of higher-order DCT-2 transform kernels [20].

DST-4/DCT-4 provides significant improvement for lower resolution test sequences, whereas the use of DST-7 and DCT-8 provides significant results for all resolution test sequences; however, the derivation of different points DST-7 and DCT-8 transform kernels cannot be achieved by subsampling the larger block size DCT-2 transform kernels. Hence, separate memory must be allocated for the storage of these transform kernels, which results in the memory issue in VVC. The total number of elements for which memory has to be allocated is 8180 elements with 8-bit precision. To address this scenario, various proposals have been presented in VVC.

In [21], a 64 × 64 compound orthonormal transform (COT) matrix is introduced, which comprises two aspects: (1) for 4-point and 8-point DST-4/DCT-4 transform kernels replacing DST-7/DCT-8 transform kernels, and (2) 16-point and 32-point DST-7/DCT-8 transform kernels embedded into 64-point DCT-2. It is implemented using a single 64 × 64 matrix, which has a total of 8180 elements and provides all three types of transform kernels. In [21], 2-, 4-, 8-, 16-, and 32-point DCT-2 can be extracted from the even rows 64-point DCT-2, whereas 4-, 8-point DST-4/DCT-4 and 16- and 32-point DST-7/DCT-8 can be extracted from the odd rows 64-point DCT-2. The metrics of the operation counts and minimum bit-precision remain the same as VTM-3.0. With this approach of using a single 64 × 64 matrix, 33% of transform storage, i.e., 2.7 kilobytes of memory, can be saved compared to VTM-3.0. Although the proposed method provides good gains, it fails to provide the precise 64-point DCT-2 because some of the kernel values of DST-7/DCT-8 are embedded in some rows of 64-point DCT-2. Owing to the mismatch of the proposed 64-point DCT-2 transform kernel with the original DCT-2 transform kernel, coding performance improvement is not significant when only the DCT-2 transform kernel is used. Conversely, the fast computation algorithm [22] for the 64-point DCT-2 transform cannot be used. Similarly, the number of transform kernel matrices is also increased to 5, that is DCT-2, DST-7, DCT-8, DST-4, and DCT-4, even though it was aimed to use only three transform kernels.

In [23], a 64-point unified matrix was proposed. It is implemented using the 64 × 64 unified matrix with a total of 8180 elements and provides all three types of transform with all block sizes from 4 × 4 to 64 × 64 by sampling a subset of coefficients and basis vectors of larger transforms. This method is designed with an 8-bit representation. Using simple calculations from the proposed matrix, different kernel types with different sizes were derived. This method uses 131,072 bits amount of kernel elements which account for 16 kilobytes of memory for the total bits amount and is 19% lesser than the total memory of bits used in VTM-3.0. Experimental results show that no gain is observed for all intra (AI) configurations for higher resolution test sequences, that is, class A1 and A2, but they comprise some gain for lower resolution test sequences, that is, classes B, C, D, and E. For random access (RA) and low delay B (LDB) configurations, the overall result shows no gain. The losses are because of the mismatch between the original and derived DST-7 and DCT-8 transform kernels.

Similarly, in [24], the adjustment stage concept for memory reduction was introduced. The adjustment stages are defined as sparse block-band orthogonal matrices and are similar to filters with small numbers of taps. It is proposed to approximate different sizes and types of cosine and sine transform such as DCT-5, DCT-8, DST-1, and DST-7 by applying different adjustment matrices (stages). The major drawback of this method is that the number of multiplications increases by using these adjustment stages, and the normalized values also differ for different transform kernels, which results in a mismatch with the original transform kernels. A transform adjustment filter (TAF) was introduced in [25], which is similar to the adjustment stage in [24]. A sparse matrix is used as a preprocessor to the partial butterfly DCT-2 algorithm. It approximates the DCT-8 and DST-7 with an adjustment stage followed by DCT-2. The DCT-8/DST-7 kernels are approximated by the combination of a block band matrix and a DCT-2 kernel. The size-16, size-32, and size-64 adjustment matrices require the overall 356 8–bit coefficients. Also, the adjustment stage is efficiently implemented using the SIMD instructions. Symmetries are applied for the 32-point and 64-point adjustment matrix having the storage requirement of 93 and 189 8-bit coefficients, respectively, whereas, for the 16-point adjustment matrix, 74 coefficients are required. All the adjustment matrix coefficients require 8-bit storage. This method reduces the complexity of the DST-7/DCT-8 transform and reduces the memory usage by storing only the different point DCT-2 transform kernels. Although the number of TAFs is reduced compared to [24], the total number of multiplications increases, i.e., total multiplication required is 160, 482, and 1468 for 16-, 32- and 64-point, respectively. This is because additional multiplication with the TAF has to be performed, and the normalization values of TAF are also not identical, thus resulting in the mismatch of the derived transform kernels with the original transform kernels.

In this study, an analytical derivation of the transform kernel using a common sparse unified matrix is introduced. The proposed method reduces the total amount of memory for the MTS kernels by storing only 1648 elements instead of 8180 elements with 8-bit precision. The transform kernels used in this study were DCT-2, DST-7, and DCT-8. Using the proposed method, DST-4 and DCT-4 transform kernels can also be achieved without affecting the memory. Similarly, the proposed method supports the implementation of the fast algorithm of DCT-2 [22] and DST-7 [17]. Figure 1 shows the proposed common sparse unified matrix, composed of two parts: Unified DST-3 matrix (U) and grouped DST-7 matrix. The U matrix is composed of different block sizes of DST-4 and is used to derive different block sizes of DCT-2 transform kernels. It stores 1368 elements with 8-bit precision. Similarly, grouped DST-7 is used to derive different block sizes of DST-7 transform kernels. It comprises P, Q, R, and S matrices, which are different selected rows from 32-point, 16-point, 8-point, and 4-point DST-7 transform kernels. It stores 280 elements with 8-bit precision. Hence, the total memory allocation must be appointed for only 1648 elements for the overall common sparse unified matrix.

The remainder of this paper is organized as follows. In Section 2, the proposed MTS kernel derivation method is described. In Section 3, the experimental results are provided, and the concluding statements are provided in Section 4.

2. Proposed Common Sparse Unified Matrix

Figure 1 shows the proposed common sparse unified matrix. It comprises two parts: a unified DST-3 matrix (U) and a grouped DST-7 matrix. The unified DST-3 matrix (U) is used to derive the different sizes of DCT-2 transform kernels, whereas the grouped DST-7 matrix is used to derive the different sizes of DST-7 and DCT-8 transform kernels, where DCT-2, DST-7, and DCT-8 are the transform kernels of MTS in VVC. The U matrix includes 32-, 16-, 4-, and 2-point DST-4 transform kernels and a 2-point DST-3 transform kernel. Similarly, the grouped DST-7 matrix comprises rows selected from different point DST-7 transform kernels, that is, 6 $\times$ 32 block size P, 4 $\times$ 16 block size Q, 2 $\times$ 8 block size R, and 2 $\times$ 4 block size S matrices are the row elements selected from 32, 16, 8, and 4-point DST-7 transform kernels, respectively.

Note that the overall size of the proposed common sparse unified matrix is 1648 bytes (integer, 8 bit-precision). The original size of the transform kernel is 8180 bytes (integer, 8-bit precision) for storing different block sizes of DCT-2, DST-7, and DCT-8.

2.1. Unified DST-3 Matrix (U)

The proposed unified DST-3 matrix (U) was used to derive DCT-2 transform kernels of different sizes. It stores only 1368 elements with 8-bit precision for deriving any block size of the DCT-2 transform kernel, whereas the current VVC allocates memory for 5460 elements with 8-bit precision for deriving different block sizes of the DCT-2 transform kernels. The proposed basic structure of the unified DST-3 matrix is shown in Figure 2.

The matrix U comprises 32, 16, 8, 4, and 2-point DST-4 and a 2-point DST-3 matrix. The remaining elements of U were set to zero. A relationship exists between DST-3 and DCT-2 [26], which can be expressed as Equation (6).

$$C_2=F\times S_3\times S$$

(6)

where $C_2$ is DCT-2, $S_3$ is DST-3, and F and S are flipping and sign change matrices, respectively. The elements in F and S are defined as

$$F_{m,n}=\left\{\begin{array}{cc}1,& \mathrm{if}n=N-1-m,\\ 0,& \mathrm{otherwise},\end{array}\right.$$

(7)

$$S_{m,n}=\left\{\begin{array}{cc}{\left(-1\right)}^m,& \mathrm{if}n=m,\\ 0,& \mathrm{otherwise}.\end{array}\right.$$

(8)

where m and n are matrix indices, and N is the size of the matrix.

To retrieve any block size of the DCT-2 kernel, different block sizes of the DST-3 kernel are required, as in (6). As shown in Figure 2, a 2-point DST-3 transform kernel matrix exists at the right bottom part of U. Using (6) and the 2-point DST-3 transform kernel, a 2-point DCT-2 transform kernel can be derived. Similarly, different block sizes of DST-3 transform kernels are required to derive other sizes of DCT-2 transform kernels. To obtain different block sizes of DST-3 transform kernels from U, the predefined 64-point unit-element matrices, A, B, C, D, and E, are multiplied by U.

Figure 3 shows the DST-3 derivation mechanism from unified DST-3 matrix (U). The unit-element matrices A, B, C, D, and E are the matrices with values of −1, 0, or 1, and their elements are defined as Equations (9)–(13), respectively.

$$A_{m,n}=\left\{\begin{array}{c} 1 , if \left(m=n,\forall 0<\left(m,n\right)<60\right) \left|\right|\\ \left(m=n,\forall 60<\left(m,n\right)<62\right) \left|\right|\\ \left(\begin{array}{c}m+n=123 \forall (61<m<64) and\\ (60<n<62)\end{array}\right)||\\ \left(\begin{array}{c}i-j=-2, \forall (60<m<62) and\\ (61<n<64)\end{array}\right)\\ -1, if(m+n=125,\forall 61<\left(m,n\right)<64\\ 0, elsewhere\end{array}\right.$$

(9)

$$B_{m,n}=\left\{\begin{array}{c} 1 , if \left(m=n,\forall 0<\left(m,n\right)<56\right) \left|\right|\\ \left(m=n,\forall 55<\left(m,n\right)<60\right)\left|\right|\\ \left(\begin{array}{c}m+n=119 ,\forall (59<m<64) and\\ ( 55<n<60)\end{array}\right)||\\ \left(\begin{array}{c}i-j=-4, \forall (55<m<60) and\\ (59<n<60)\end{array}\right)\\ -1, if(m+n=123,\forall 59<\left(m,n\right)<64\\ 0, elsewhere\end{array}\right.$$

(10)

$$C_{m,n}=\left\{\begin{array}{c} 1 , if \left(m=n,\forall 0<\left(m,n\right)<48\right) \left|\right|\\ \left(m=n,\forall 47<\left(m,n\right)<56\right)\left|\right|\\ \left(\begin{array}{c}m+n=111 ,\forall \left(55<m<64\right) and\\ \left(47<n<56\right)\end{array}\right)||\\ \left(\begin{array}{c}i-j=-8, \forall \left(47<m<56\right) and \\ \left(55<n<64\right)\end{array}\right)\\ -1, if(m+n=119,\forall 55<\left(m,n\right)<64\\ 0, elsewhere\end{array}\right.$$

(11)

$$D_{m,n}=\left\{\begin{array}{c} 1 , if \left(m=n,\forall 0<\left(m,n\right)<32\right) \left|\right|\\ \left(m=n,\forall 31<\left(m,n\right)<48\right)\left|\right|\\ \left(\begin{array}{c}m+n=95 ,\forall (47<m<64) and\\ ( 31<n<48)\end{array}\right)||\\ \left(\begin{array}{c}i-j=-16, \forall (31<m<48) and \\ (47<n<64)\end{array}\right)\\ -1, if(m+n=111,\forall 47<\left(m,n\right)<64\\ 0, elsewhere\end{array}\right.$$

(12)

$$E_{m,n}=\left\{\begin{array}{c}1 , if \left(m=n,\forall 0<\left(m,n\right)<32\right) \left|\right|\\ \left(\begin{array}{c}m+n=63 ,\forall (31<m<64) and\\ ( 0<n<32)\end{array}\right)||\\ \left(\begin{array}{c}i-j=-32, \forall (0<m<32) and \\ (31<n<64)\end{array}\right)\\ -1, if(m+n=95,\forall 31<\left(m,n\right)<64\\ 0, elsewhere\end{array}\right.$$

(13)

The relation between U and the unit-element matrices can be expressed as Equation (14).

$$S_{3, 64}=U\times A\times B\times C\times D\times E$$

(14)

where $S_{3,64}$ is the 64-point DST-3 transform kernel, U is the proposed unified DST-3 matrix, and A, B, C, D, and E are the 64-point unit-element matrices. Using U and the unit-element matrices, DST-3 transform kernels with different sizes can be obtained, which is explained using Equations (15)–(19)

$$S_{3,4}=\mathrm{T}{\left[U_{64}\times A_{64}\right]}_4$$

(15)

$$S_{3,8}=\mathrm{T}{\left[U_{64}\times M_{64}\right]}_8$$

(16)

$$S_{3,16}=\mathrm{T}{\left[U_{64}\times N_{64}\right]}_{16}$$

(17)

$$S_{3,32}=\mathrm{T}{\left[U_{64}\times O_{64}\right]}_{32}$$

(18)

$$S_{3,64}=U_{64}\times P_{64}$$

(19)

where $S_{3,N}$ is an N-point DST-3 transform kernel, $\mathrm{T}{[]}_N$ represents a function that takes an N × N block of the right-bottom part of the matrix, and M, N, O, and P are defined using unit elements (−1, 0, and 1) as in Equation (20).

$$\begin{gathered} M=A\times B \\ N=A\times B\times C \\ O=A\times B\times C\times D \\ P=A\times B\times C\times D\times E \end{gathered}$$

(20)

For example, to derive the 16-point DCT-2 kernel, a 16-point DST-3 is required, as in Equation (6). First, U is multiplied by the A, B, and C unit-element matrices as in (17) in sequential order, thus leading to 32-point DST-4, 16-point DST-4, and 16-point DST-3 transform kernel matrices. Using the resulting matrices, the 16-point DST-3 matrix at the bottom right of the generated matrix is obtained. Finally, the 16-point DCT-2 transform kernel can be obtained using (6) and a 16-point DST-3. Figure 4 shows the overall process of generation of the 16-point DST-3 transform kernel.

Figure 5 shows the proposed derivation of the N-point DCT-2 transform kernel. First, the value of N is equal to 2. If the condition is satisfied, $S_{3,2}$ is obtained by the selection of a 2 × 2 block of the right-bottom part of the U matrix; otherwise, $S_{3,N}$ is achieved by the selection of the N × N block of the bottom right of the matrix obtained after the multiplication of U, and $E_{64}$. $E_{64}$ represents the unit-element matrices A, B, C, D, and E. $E_{64,N}^{}$ can be obtained by multiplying the unit-element matrices A, B, C, D, and E, depending on N. For example, $E_{64,N}^{}$ is calculated as $A\times B\times C\times D$, $A\times B\times C$, $A\times B,$ and A for N = 32, N = 16, N = 8, and N = 4, respectively. Finally, the N-point DCT-2 transform kernel was obtained using (6). Figure 5 shows an illustration of the proposed DCT-2 transform kernel derivation.

2.2. Proposed Grouped DST-7 Matrix

To derive the full DST-7 and DCT-8 matrices with less memory, a grouped DST-7 matrix is proposed. The proposed grouped DST-7 matrix is based on the principle of storing only a few selected rows of different block-sized DST-7 transform kernels. The rows selected from the original kernel matrix are used to generate the group of elements of the respective block size DST-7 transform kernel with the help of the permutation matrix (G) comprising −1, 0, or 1. The matrix G is multiplied by the selected rows, resulting in a full block-sized DST-7 transform kernel. The matrices P, Q, R, and S in Figure 1 indicate the rows for the derivation of 32, 16, 8, and 4-point DST-7 transform kernels, respectively, and they are only stored for DST-7 and DCT-8 kernel derivation, leading to a significant reduction in memory storage. The grouped DST-7 matrix is defined as a 14 × 64 matrix of the lower part, as shown in Figure 1, and the total memory allocation is required for only 280 elements with 8-bit precision, which significantly reduces the memory sizes of the current DST-7 in VVC with 1360 elements each with 8-bit precision.

To derive the full-size 32-point DST-7 transform kernel, only six-row vectors from the 32-point DST-7 are selected, that is, P = $\left[S_{7,0}{S}_{7,1}{S}_{7,2}{S}_{7,3}{S}_{7,5}{S}_{7,6}\right]$^T and can be base vectors for each group of DST-7 matrix in the proposed method. Similarly, for the derivation of full-size 16, 8, and 4-point DST-7 transform kernels, Q with four rows, R with two rows, and S with two rows, that is, Q = $\left[S_{7,0}{S}_{7,1}{S}_{7,2}{S}_{7,5}\right]$^T, R = $\left[S_{7,0}{S}_{7,1}\right]$^T, and S = $\left[S_{7,0}{S}_{7,1}\right]$^T, are selected from 16, 8, and 4-point DST-7, respectively and are regarded as the base vectors. Note that $S_{7, i}$ represents the i-th row of DST-7. Using a few selected rows and G, the full-size DST-7 transform kernel can be derived by retrieving the row vectors of each group. Figure 6 shows an illustration of kernel derivation using a grouped 32-point DST-7 matrix. As shown in Figure 6, the row vectors of P in the grouped DST-7 matrix are used to derive the row vectors of the DST-7 kernel in each group.

Figure 7 shows the details of the derivation process of the kernel for each group. The 0-th group is taken as an example, and the other groups follow the same process using different row vectors of matrix P.

In Figure 7, ${[S_{7,\left(i,j\right)}]}_N$ denotes the j-th row vector of the i-th group for the N-point DST-7. In the proposed method, matrix P comprises only the 0-th row vectors of the i-th group of the N-point DST-7. First, the 0-th row vector ${\left[S_{7,\left(i,0\right)}\right]}_N$ from P is obtained from the i-th group of DST-7. Subsequently, the absolute values of $e_l$ and $e_f$, which are the last (the rightmost) and first (the leftmost) elements in the row vector, are compared. If these two absolute values are identical, the row vector can be one row of the DST-7 kernel with the same index as stored in P. Otherwise, the row vector is multiplied by G and a comparison of $e_l$ and $e_f$ in the resulting row vector is performed as in the previous step. If the first element in each row is a negative value, then the sign of every element in each row is altered. It should be noted that G is designed such that the absolute value of the first element in the previous row equals the absolute value of the last element of the next row vector after multiplying with G. This process is repeated until the end of the last row vector of the current group.

Figure 8 shows a 16-point DST-7 transform kernel derived using Figure 7, and the elements in each row vectors are perfectly matched with the definition of DST-7 in (5). Similarly, 32-, 8-, and 4-point DST-7 can be derived using P, R, S, and G matrices, which are defined according to the size, which will be discussed in the next section.

For the derivation of the full-size 16-point DST-7 transform kernel, only four rows from the 16-point DST-7 are selected, that is, Q = $\left[S_{7,\left(0,0\right)}{S}_{7,\left(1,1\right)}{S}_{7,\left(2,2\right)}{S}_{7,\left(5,5\right)}\right]$^T and can be used as base vectors for each group of DST-7 matrix in the proposed method.

2.3. Permutation Matrix (G)

The permutation matrix G plays a crucial role in the derivation of the DST-7 transform kernel using the proposed grouping algorithm, as mentioned in the previous section. The process of deriving G in the proposed method is described in Figure 9, which shows the first group of 16-point DST-7 transform kernels represented with floating-point elements, where $S_{7,\left(0,0\right)}$ is only stored in matrix Q in the common sparse unified matrix shown in Figure 1.

The elements between the consecutive rows in the group are observed with specific patterns that appear based on the characteristics of DST-7 kernels. Figure 10 shows the patterns of the elements in consecutive rows. As shown in Figure 10, elements in the odd column (1st, 3rd,…) of the 0th row (${\left[S_{7,\left(0,0\right)}\right]}_{32}$) are repeated in the left-half column in the next row. The elements in the right-half column are copied from the elements of even columns (0th, 2nd,…) in the previous row vector in reverse order.

Thus, this relation can be described in Equation (21).

$${\left[S_{\left.7,(i,1\right)}\right]}_N=[{({[S_{\left.7,(i,1\right)}]}_{N/2})}_L,{({[S_{\left.7,(i,1\right)}]}_{N/2})}_R]$$

(21)

Based on Figure 7, (21) is the next row vector, composed of two parts, ${({[S_{\left.7,(i,1\right)}]}_{N/2})}_L$ and ${({[S_{\left.7,(i,1\right)}]}_{N/2})}_R$. ${({[S_{\left.7,(i,1\right)}]}_{N/2})}_L$ and ${({[S_{\left.7,(i,1\right)}]}_{N/2})}_R$ represent the left and right halves of ${\left[S_{\left.7,(i,1\right)}\right]}_N$.

${({[S_{\left.7,(i,1\right)}]}_{N/2})}_L$ is derived using (22), where elements in the odd column (1st, 3rd, …) of the 0th row (previous row) are selected. ${({[S_{\left.7,(i,1\right)}]}_{N/2})}_R$ is derived using (23), where elements in the even column (0th, 2nd, …) of the 0th row (previous row) are selected and substituted in a reverse way. Note that ${[S_{7,\left(i,j\right),k}]}_{N/2}$ denotes the element in the k-th column and j-th row vector of the i-th group for N-point DST-7.

$${({[S_{\left.7,(i,1\right)}]}_{N/2})}_L={[S_{\left.7,(i,0\right),2n-1}]}_{N/2}$$

(22)

$${({[S_{\left.7,(i,1\right)}]}_{N/2})}_R={[S_{\left.7,(i,0\right),2n-2}]}_{N/2}$$

(23)

where n is the integer value, N/2 < n > 0.

Hence, a change of one element in the row vectors in a group may affect the other elements of the other vector of the same group. Figure 10 shows an example of patterns observed between two consecutive row vectors of the 16-point DST-7 transform kernel. From the patterns, G comprising −1, 0, and 1 can be derived. Figure 11 shows the G matrix based on the pattern in Figure 10 for deriving the DST-7 transform kernel.

As shown in Figure 11, a smaller G can be embedded into a larger G with identical patterns of element values. Figure 12 shows an algorithm for obtaining the matrix. Even if the memory size of G in Figure 11 accounts for 256 bytes, there can be a case to derive G on the fly using the algorithm shown in Figure 12, not using static memory to save G.

As shown in Figure 12, the algorithm comprises two parts for the left column (S0) and right column part (S1) of G. First, variables used for the algorithm are initialized and the size (N) of the kernel matrix to derive is set. Then, variables a and b are initially set to 1 and 0, which will be used as initial values of the left column (S0) part of G. $i_r$ and $i_c$ represent the row and column indices of G, respectively. The index value of $i_r$ was initialized to 0. First, the condition is checked if the $i_r$ index value is less than the size N. If this is true, then the process is divided into two columns, the left column (S0) and right column (S1); then, the $i_c$ index value is initialized to 0 else, the process terminates.

Thus, the $i_c$ index value is checked if it is less than half of the value of N. If the condition defined is satisfied, then $i_r$ and $i_c$ corresponding to the respective a and b variables are checked. Variables a and b are set only for the S0 of the G matrix. The value of the permutation matrix at the respective location ${\left[{\mathrm{G}}_N\right]}_{i_r,i_c}$ is set to 0 if the checked condition is false, $i_c$ is incremented by 1, and looped back to the condition, where the $i_c$ index value is checked for whether it is less than half of N. If the checked condition holds true (T), $c_1$ is checked for whether it is true (T) or false (F). If $c_1$ is false, then the value ${\left[{\mathrm{G}}_N\right]}_{i_r,i_c}$ is set to −1, and $c_1$ is set to true. Otherwise, the value of ${\left[G_N\right]}_{i_r,i_c}$ is set to 1, and $c_1$ is set to false. Subsequently, the value of a is incremented by 2, and the value of b is incremented by 1. After setting the values of a and b, $i_c$ is incremented by 1 and finally looped back to the condition where the $i_c$ index value is checked whether it is less than half of the value of N.

For the right column (S1) part of G, variables m and n are used, whose initial values are set to 0 and N−1, respectively. If the condition, where $i_c$ index value is checked to be less than half of N is false, then the value of $i_c$ is set as half of N. Further, the $i_c$ index is checked to determine whether it is less than N. If the condition is false (F), then the $i_r$ index value is incremented by 1 and looped back to the condition, where $i_r$ is less than N or is not checked. However, if the condition exists (T), then the $i_r$ index equals m and the $i_c$ index, which equals n, is checked. If it is false, the value of ${\left[G_N\right]}_{i_r,i_c}$ is set to 0. Otherwise, the $c_2$ condition is further checked for whether it is false. If $c_2$ is false, the value ${\left[G_N\right]}_{i_r,i_c}$ is set to 1, and $c_2$ is set as false; thus, the $i_c$ index is incremented by 1 and looped back to the condition where $i_c$ is checked to determine whether it is less than N. Otherwise, the value ${\left[G_N\right]}_{i_r,i_c}$ is set as −1 and $c_2$ is set as true, the value of m is incremented by 2, and that of n is decremented by 1. Finally, the value of the $i_c$ index is incremented by 1 and looped back to the condition where the $i_c$ index is checked to determine whether it is less than N.

An example of the splitting of S0 and S1 in the G matrix for N = 8 is shown in Figure 13.

2.4. Integer Kernels with VVC Standard

Generally, the floating-point elements in the kernels are scaled up to integer elements to avoid a mismatch between the encoder and decoder in most video coding standards. Transform and quantization using resulting integer kernel matrices are implemented with integer arithmetic, which benefits the complexity of implementations and is one of the primary concerns in the standardization of VVC. Despite the benefits of integer kernels, several disadvantages exist. Orthogonality in the kernel matrix can be lost when scaled up to integer values and the magnitudes of the matrix elements should be as small as possible under the consideration of dynamic ranges for the process of transform and quantization [26]. Nevertheless, it is known that the kernel similarity between the derived kernel and original kernels is more important than orthogonality when quantization is applied to transform the coefficients. Thus, it is unnecessary to maintain kernel orthogonality at the expense of kernel similarity. By reflecting the requirements of transform kernels for video codec, integer kernels for every transform block are proposed and finally adopted [15]. In this section, we show how our proposed kernel derivation method is applied for transformation with no deviation, which means the kernels derived by the proposed method are perfectly matched with the standard kernels.

Figure 14 shows a 32-point integer DST-7 transform kernel adopted in VTM-3.0 [27] as the standard. In Figure 14, each row vector is shown in the order of the group. The element values in the row vectors were identical to those of the standard kernel.

As described in Figure 7 of Section 2.2, the absolute value of $e_f$ in the current row should be identical to the absolute value of $e_l$ of the next row, which is obtained by multiplying the current row vector by G. The multiplication can be repeated until $e_l$ of the base (first) row is equal to $e_f$ of the row multiplied by G. For the derivation of all sizes of DST-7, the multiplications are repeated until the end of the last vectors of the group, except for a few cases for 32-point DST-7, which are observed in the 3rd group for 32-point DST-7. Figure 15 shows the case shown in the 3rd group for the 32-point DST-7.

As shown in Figure 15, ${\left[S_{7,\left(3,18\right)}\right]}_{32}$ is obtained after the multiplication of ${\left[S_{7,\left(3,25\right)}\right]}_{32}$ and G. Because $e_l$ of ${\left[S_{7,\left(3,3\right)}\right]}_{32}$ equals $e_f$ of ${\left[S_{7,\left(3,18\right)}\right]}_{32}$, the algorithm for the current group is complete, as shown in Figure 7. Thus, ${\left[S_{7,\left(3,4\right)}\right]}_{32}$, ${\left[S_{7,\left(3,23\right)}\right]}_{32}$ and ${\left[S_{7,\left(3,14\right)}\right]}_{32}$ for the full-size DST-7 kernel cannot be obtained using the proposed algorithm. This only happens when the element values are scaled up to make the element values of the kernel integer. To address this problem for integer kernels of DST-7, especially for the 32-point kernel, two approaches are proposed in our method. The first method is to tune $e_l$ of ${\left[S_{7,\left(3,3\right)}\right]}_{32}$ from 88 to 89 without considering the sign value, and by tuning the value, the case of stopping the algorithm in ${\left[S_{7,\left(3,18\right)}\right]}_{32}$ obtained after the multiplication of ${\left[S_{7,\left(3,25\right)}\right]}_{32}$ and G can be removed, resulting in a full-size 32-point DST-7 transform kernel. Figure 16a shows this procedure. Similarly, the second method is to tune the 3rd element of the value of ${\left[S_{7,\left(3,3\right)}\right]}_{32}$ from 88 to 89 without considering the sign value, as shown in Figure 16b. By tuning the values, the related values in the other vectors are changed by the proposed algorithm, accordingly.

Finally, once all sizes of DST-7 kernels are derived, DCT-7 and all DCT-8 transform kernels can be easily derived using relation Equation (24) as

$$C_8=S\times S_7\times F$$

(24)

where $C_8$ and $S_7$ are the DCT-8 and DST-7 transform kernels, respectively, and S and F are the sign-changing and flipping matrices, defined in (7) and (8), respectively.

2.5. Application of the Proposed Method to the DST-7/DCT-8 Fast Algorithm in VTM

In Equation (23), a fast DST-7/DCT-8 algorithm with dual support was introduced and adopted in VTM-3.0. The fast algorithm is based on three different features for integer N-point DST-7/DCT-8 transform kernels shown in Figure 17, which can be defined as Figure 17.

Where {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p} are the N-point unique integer numbers for the 16-point DST-7 kernel. Assume that the output vector $y ={\left[{\mathit{y}}_0,{\mathit{y}}_1,\dots ,{\mathit{y}}_{N-1}\right]}^{\mathit{T}}$ is obtained by multiplying the input vector $x ={\left[{\mathit{x}}_0,{\mathit{x}}_1,\dots ,{\mathit{x}}_{N-1}\right]}^{\mathit{T}}$ with the transform kernel C, that is, ($y=xC$).

Feature #1:

It is observed that the sum of several (2 or 3) numbers is equal to the sum of several (1 or 2) numbers. The elements l, m, n, o, and p are obtained by (a + j), (b + i), (c + h), (d + g), and (e + f), respectively.

To calculate y₀, instead of performing vector-by-vector multiplication that requires 16 multiplications, the following alternative implementation is computed as shown in Equation (25):

y₀ = a·(x₀ + x₁₁) + b·(x₁ + x₁₂) + … + j·(x₉ + x₁₁) + k·x₁₀

(25)

which requires 10 multiplications.

Feature #2:

This is based on the feature that some elements in a row vector of DST-7 are symmetrically mirrored with each other. For example, the second vector {c, f, i, l, o, o, l, i, f, c, o, $-$ c, $-$f, $-$i, $-$l, $-$o} in Figure xx with only c, f, i, l, and o elements.

To calculate y₁, instead of performing vector-by-vector multiplication that requires 16 multiplications, the following alternative implementation is computed, as shown in Equation (26):

y₁ = c·(x₀ + x₉ − x₁₁) + f·(x₁ + x₈ − x₁₂) + i·(x₂ + x₇ − x₁₃) + l·(x₃ + x₆ − x₁₄) + o·(x₄ + x₅ − x₁₅)

(26)

which requires five multiplications.

Feature #3:

This is based on the basic vectors that contain very few numbers (1 or 2) of distinct values without considering sign values {k, k, 0, −k, −k, 0, k, k, 0, −k, −k, 0, k, k, 0, −k}. To calculate y₅, instead of performing vector-by-vector multiplication that requires 16 multiplications, the following alternative implementation is computed as shown in Equation (27):

y₅ = k·(x₀ + x₁− … −x₁₅)

(27)

which requires one multiplication.

The major drawback of the fast algorithm [28] for DST-7 transform in VTM3.0 is that one feature out of the mentioned three features has to be applied in each row. However, once a feature is selected for the basic vector such as ${\left[S_{7,\left(0,0\right)}\right]}_{16}$, ${\left[S_{7,\left(1,1\right)}\right]}_{16}$, ${\left[S_{7,\left(2,2\right)}\right]}_{16}$ and ${\left[S_{7,\left(5,5\right)}\right]}_{16}$, the same feature is applied to vectors belonging to the same group. Hence, the proposed method follows “one group one feature” and removes the matching of a single feature among the three features in each row.

For example, in the first row, a vector of DST-7, Feature #1 is applied, whereas in the second-row vector, Feature #2 is applied. Similarly, for the other rows, a single feature out of three features was applied. Initially, before applying the feature in each row vector, we were unaware of the feature to be applied in the respective row vector. Consequently, we need to check if the respective row vector follows Feature #1, #2, or #3 such that the single matched feature is applied. For a larger matrix, the number of row vectors is large; therefore, for each row, we have to manually check if each row follows Feature #1, #2, or #3, which is time-consuming.

Based on the proposed grouping method, only for the base row vector, a feature out of three features is selected manually, and the row vectors that depend on the base row, that is, each grouped row vector, follow the same feature that is applied to the based row vector. Figure 18 shows the application of the proposed 16-point grouped DST-7 transform to the fast algorithm in VVC. Feature 1 can be applied to the 0th and 2nd groups. The elements in the l, m, n, o, and p columns are obtained by the summation of the column elements such as (a + j), (b + i), (c + h), (d + g), and (e + f), respectively, of the 0th and 2nd groups. Feature #2 is applied to the 1-st group where column elements in f, g, h, i, and j are mirrored values of column elements e, d, c, b, and a, respectively. Similarly, elements in l, m, n, o, and p are mirrored in the column elements j, i, h, g, and f with flipped signed values, respectively. Similarly, Feature #3 is applied to ${\left[S_{7,\left(5,5\right)}\right]}_{16}$ as it contains only a single distinct element, that is, 77. This is possible with the help of the proposed grouping method.

3. Experimental Results and Analysis

To verify the proposed method for the transform kernel derivation for VVC, the similarities between the proposed kernels and the original kernels in VTM-3.0 [27] were investigated by plotting the kernels in a graphical representation. Figure 19 shows the comparison of the transform kernels. It can be seen that the derived transform kernels are almost perfectly matched with the original transform kernels. As described in Section 3, only a few integer elements in the 32-point DST-7 kernel are tuned from the original kernels to obtain full transform kernels. Thus, it can be inferred that the proposed transform kernels are more likely to have identical coding gains.

Experiment for verification of the proposed method were performed on the VVC reference software, VTM-3.0 [27], using CTC conditions [29]. The PC for simulation was Centos 7 OS with an Intel^® Xeon^® Silver 4114 CPU @ 2.20 GHz processor with 200 cores and 156 GB of RAM. Parallel processing is turned off to precisely measure the encoding and decoding complexity in the execution time. The coding performance was measured in BDBR (%) [30].

Table 2. Simulation results.

Test Sequences		AI (All Intra)			RA (Random Access)			LDB (Low Delay B)
		Y	U	V	Y	U	V	Y	U	V
Class A1 (4K)	Tango2	−0.01	−0.31	0.08	0.04	0.14	0.01	-	-	-
	FoodMarket4	0.01	−0.01	0.04	0.00	0.11	0.17	-	-	-
	Campfire	−0.01	0.08	−0.10	0.00	−0.04	0.01	-	-	-
Class A2 (4K)	CatRobot1	−0.01	−0.01	0.01	−0.04	0.10	0.17	-	-	-
	DaylightRoad2	0.00	−0.10	0.02	−0.04	0.19	0.07	-	-	-
	ParkRunning3	0.01	0.00	−0.03	−0.03	−0.06	0.01	-	-	-
Class B (1080p)	MarketPlace	0.02	−0.13	0.00	−0.02	0.28	−0.13	0.05	−0.04	−0.35
	RitualDance	0.00	0.07	0.10	−0.01	−0.02	0.00	0.03	−0.11	−0.27
	Cactus	0.01	0.01	0.05	−0.03	−0.04	0.14	−0.10	−0.45	−0.06
	BasketballDrive	−0.01	−0.02	0.07	0.05	−0.01	−0.14	0.01	−0.16	−0.13
	BQTerrace	0.00	0.15	0.09	0.01	0.69	0.17	−0.05	−0.68	−1.28
Class C (WVGA)	BasketballDrill	0.01	−0.18	−0.12	−0.01	0.13	0.15	0.04	0.55	0.10
	BQMall	−0.01	0.15	−0.06	−0.02	−0.12	0.18	0.16	0.61	0.11
	PartyScene	0.00	−0.04	−0.10	0.02	0.18	−0.18	0.04	−0.22	−0.08
	RaceHorses	0.01	−0.09	0.00	0.00	−0.15	0.24	−0.01	−0.10	0.28
Class D (WQVGA)	BasketballPass	0.01	−0.17	−0.08	−0.03	−0.09	−0.17	−0.09	0.36	−0.43
	BQSquare	0.00	0.23	0.28	−0.07	−0.95	−0.24	−0.02	0.55	0.59
	BlowingBubbles	0.00	−0.03	0.03	−0.01	0.01	0.30	0.00	−0.43	−0.83
	RaceHorses	0.00	−0.03	0.18	0.08	0.23	−0.47	−0.02	−0.16	−0.17
Class E (720p)	FourPeople	0.00	−0.05	−0.01	-	-	-	0.17	−0.77	−0.30
	Johnny	−0.02	0.18	−0.07	-	-	-	−0.22	2.81	−0.22
	KristenAndSara	0.02	−0.02	0.08	-	-	-	0.00	−0.43	−1.00
Overall		0.00	−0.02	0.00	−0.01	0.09	0.06	0.01	0.08	−0.27
Unit: %

shows the experimental results of the proposed method. As shown in Table 2, the coding performances of the proposed method are almost identical to the original VTM3.0 for all configurations. This is because the similarity of the proposed transform kernel with the original transform kernel is almost perfect.

Table 3. Comparison of the overall simulation results for the derivation of transform kernels with the proposed method (AI).

	COT [18]					Unified Matrix [20]					AMT [21]					TAF [22]					Proposed Method
	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
Class A1	−0.007	−0.014	0.092	101	100	0.200	0.162	0.299	101	100	0.061	0.184	0.187	101	97	0.101	0.115	0.125	95	84	0.000	−0.080	0.000	100	100
Class A2	−0.050	−0.051	−0.012	102	100	0.152	0.089	0.085	101	100	0.063	0.027	0.105	102	97	0.139	0.076	0.030	97	88	0.000	−0.041	0.001	100	100
Class B	−0.136	−0.131	−0.125	102	98	−0.024	−0.147	−0.140	101	99	0.031	0.093	0.135	102	98	0.075	0.029	0.097	96	88	0.001	0.021	0.059	100	100
Class C	−0.289	−0.214	−0.310	101	99	−0.304	−0.354	−0.415	102	100	0.009	−0.005	0.113	105	105	0.015	0.065	−0.032	97	93	0.000	−0.040	−0.071	100	100
Class D	−0.314	−0.324	−0.307	104	104	−0.349	−0.356	−0.474	103	102	0.007	0.052	−0.009	107	109	0.042	−0.114	0.085	97	96	0.000	0.000	0.100	102	100
Class E	−0.335	−0.277	−0.132	98	98	−0.214	−0.350	−0.289	100	98	0.025	−0.063	0.107	109	106	0.063	0.044	0.110	96	89	0.001	0.041	0.001	100	100
Overall	−0.167	−0.141	−0.112	101	99	−0.058	−0.136	−0.115	101	99	0.033	0.054	0.107	104	101	0.075	0.060	0.064	96	88	0.000	−0.020	0.001	100	100
Unit: %

,

Table 4. Comparison of the overall simulation results for the derivation of transform kernels with the proposed method (RA).

Seq.	COT [18]					Unified Matrix [20]					AMT [21]					TAF [22]					Proposed Method
	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
Class A1	0.002	0.021	0.048	101	99	0.175	0.259	0.324	101	101	0.032	0.059	0.139	101	99	0.090	0.106	0.079	99	98	0.011	0.070	0.060	101	100
Class A2	−0.021	0.168	0.093	99	99	0.140	0.190	0.311	99	99	0.040	0.118	0.215	100	100	0.055	0.195	0.239	99	98	−0.040	0.080	0.080	101	100
Class B	−0.069	0.060	0.012	101	99	0.067	0.172	−0.104	100	100	0.048	−0.128	0.052	101	99	0.047	0.248	0.117	99	98	0.002	0.178	0.011	100	100
Class C	−0.111	−0.046	0.037	101	100	−0.069	−0.043	0.049	98	96	0.000	0.041	0.027	99	98	0.056	−0.124	0.041	99	100	0.001	0.010	0.100	101	100
Class D	−0.136	−0.438	−0.341	101	101	−0.145	−0.445	−0.259	98	96	0.055	0.256	0.238	100	100	−0.017	−0.447	−0.511	99	100	−0.006	−0.201	−0.149	101	100
Overall	−0.057	0.045	0.042	101	99	0.067	0.136	0.105	99	99	0.035	0.060	0.130	100	99	0.060	0.110	0.114	99	99	−0.010	0.086	0.061	101	100
Unit: %

and

Table 5. Comparison of the overall simulation results for the derivation of transform kernels with the proposed method (LDB).

Seq.	COT [18]					Unified Matrix [20]					TAF [22]					Proposed Method
	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
Class B	−0.015	−0.263	0.078	100	100	0.168	0.016	0.326	98	96	0.048	−0.169	0.118	100	101	−0.010	−0.291	−0.420	100	100
Class C	0.015	0.437	0.174	99	96	0.148	0.341	0.388	98	98	0.013	−0.009	0.214	100	100	0.060	0.211	0.100	101	100
Class D	0.016	0.202	0.078	104	102	0.057	0.054	−0.170	98	97	−0.035	0.206	−0.171	100	100	−0.030	0.081	−0.210	101	100
Class E	0.098	1.063	0.158	102	98	0.186	0.812	−0.355	96	88	0.019	1.000	0.162	100	99	−0.020	0.540	−0.510	101	100
Overall	0.023	0.302	0.130	100	98	0.166	0.323	0.177	97	95	0.030	0.177	0.161	100	100	0.000	0.079	−0.270	101	100
Unit: %

show a comparison of the results of the proposed method for the AI, RA, and LDB configurations [31], respectively.

Although COT [21] provides better coding gain, it fails to provide a 64-point DCT-2 transform kernel, as some rows of 64-point DCT-2 transform kernels are removed and replaced with selected row values of DST-7/DCT-8 transform kernels [21].

In addition, [21] used five transform kernels, that is, DCT-2, DST-7, DCT-8, DST-4, and DCT-4. Similarly, the unified matrix [23] presented excellent coding gain; however, the higher resolution class sequences, Classes A1 and A2 have higher coding loss. This results from the deviation of the proposed mathematically derived transform kernels with the original transform kernels. In [24,25], owing to the mismatch in the proposed transform kernels with original transform kernels, coding losses can be seen. In our proposed method, no coding loss is observed in higher-resolution class sequences. The overall result showed a minor gain of 0.02% in the U-chroma. This is because the proposed transform kernels were in good agreement with the original transform kernels, as shown in Figure 19.

Table 4 shows the overall experimental results of RA using the VTM-3.0 reference software under the CTC conditions. Coding loss can be seen in [23,24,25] but [21] showed less significant luma gain. Similarly, the proposed method showed no significant loss compared to the other methods.

Table 5 shows the overall experimental results of LDB using the VTM-3.0 reference software under the CTC conditions. [21,23,25] showed higher coding loss, whereas the proposed method showed a 0.27% coding gain in V-chroma and negligible loss in U-chroma.

4. Conclusions

This study proposed an analytical kernel derivation method for the VVC video codec. Memory consumption to store different block size transform kernels is regarded as a major issue in video coding standardization. To address this scenario, the proposed method attempts to save static memory by storing only fewer elements of the transform kernels. Full-required transform kernels are derived using the stored transform kernels and generated unit-element matrices and a permutation matrix. In the proposed method, all the different block size transform kernels are generated dynamically during runtime. The static memory required is only for 1648 elements instead of 8180 elements, each with 8-bit precision, which means that approximately 80% of the static memory can be saved. The fast algorithm of DCT-2 and DST-7 can be implemented in the proposed method as the proposed signal closely overlaps with the original transform kernels, that is, DCT-2, DCT-8, and DST-7. Similarly, the proposed grouping concept of the DST-7 transform kernel shows the effect of tuning on one-row elements on the remaining rows that depend on the tuned row element using the permutation matrix.