Detection of Transcoding from H.264/AVC to HEVC Based on CU and PU Partition Types

Abstract

High Efficiency Video Coding (HEVC) is a worldwide popular video coding standard due to its high coding efficiency. To make profits, forgers prefer to transcode videos from previous standards such as H.264/AVC to HEVC. To deal with this issue, an efficient method is proposed to expose such transcoded HEVC videos based on coding unit (CU) and prediction unit (PU) partition types. CU and PU partitioning are two unique syntactic units of HEVC that can reflect a video’s compression history. In this paper, CU and PU partition types of I pictures and P pictures are firstly extracted. Then, their mean frequencies are calculated and concatenated as distinguishing features, which are further sent to a support vector machine (SVM) for classification. Experimental results show that the proposed method can identify transcoded HEVC videos with high accuracy and has strong robustness against frame-deletion and shifted Group of Pictures (GOP) structure attacks.

1. Introduction

Owing to the rapid development of mobile web and video capture devices, people can shoot, transmit, and watch videos at any time and any place. Every minute, 500 h worth of video content are uploaded to the video sharing website YouTube [1]. However, by using increasingly user-friendly video editing software, forgers can change the original video contents at will, such as inserting/ deleting particular pictures or objects, splicing different video sequences, which may lead to judicial misjudgment or fake news. Therefore, it is an essential thing to authenticate the credibility of a video. As we know, falsified videos must have encountered a re-encoding process because forgers have to decompress one video before tampering and recompress it after tampering [2], so re-encoding detection can be used as the first step when verifying the authentication of one given video, and it has become one of the most important video forensics techniques [3].

In the literature, numerous re-encoding detection methods have been proposed for videos recompressed with the same codec as the original one. Discrete cosine transform (DCT) coefficients [4] and block artifacts [5] were utilized to detect re-encoded MPEG videos. Reference [6] was proposed for MPEG and H.264/AVC (AVC for short) re-encoding. After the release of the latest video coding standard, High Efficiency Video Coding (HEVC), a number of researchers turned to the field of its re-encoding detection [7,8,9,10,11,12]. However, not many algorithms have been reported to detect transcoding, i.e., re-encoding with different codecs. In fact, transcoding is a common operation of re-encoding. However, the transcoding process would cover up the video’s original compression information, and users can only get the re-encoding compression information.

In Reference [13], the proposed algorithm can detect AVC videos transcoded from MPEG videos and estimate the original Group of Pictures (GOP) size. In recent years, HEVC has got growing support from the industry and has begun to replace AVC. In such circumstances, a forger may re-encode low quality AVC videos to disguise as HEVC videos, and then upload them to a video-sharing website to gain popularity or even profits. Therefore, it is of great importance to reliably detect HEVC videos transcoded from AVC videos. For short, we call these kinds of videos “AVC/HEVC videos” in this paper.

For detecting AVC/HEVC videos, Costanzo et al. [14] analyzed the frequencies of motion prediction modes over all B pictures and utilized the critical point where the distance between the given video and its recompression version began to increase to identify AVC/HEVC videos. However, it is only suitable for videos encoded with constant Quantization Parameter (QP) mode, and it needs to recompress the videos a number of times, which leads to high computational complexity. Bian et al. [15] combined the frequency of PU partition types in I pictures and P pictures to detect AVC/HEVC videos, and the results confirmed that PU partition types could be effective in identifying AVC/HEVC videos. However, it did not consider the characteristic of CU partition types. Like Reference [15], Reference [12], which was proposed for detecting videos recompressed with the same codec as the original one, also only explored PU partition types and did not consider the CU partition types. Furthermore, PU partition types were only extracted from P pictures in Reference [12]. However, CU is the parent partition of PU in the HEVC coding standard. Compared with PU, which can better capture the subtle differences of Coding Tree Unit (CTU) contents, CU usually reflects the complexity of CTU contents, and would also be influenced by the transcoding process. Stimulated by this, we will comprehensively analyze the partition difference between singly compressed HEVC videos and AVC/HEVC videos in terms of both CU types and PU types in this paper.

The proposed method consists of two stages. In the first stage, the CU and PU partition types of I pictures and the first P pictures in each GOP are extracted, and the corresponding mean frequencies are calculated and concatenated. The support vector machine (SVM) classification is carried out in the second stage. The experimental results show that compared with the state-of-the-art work, the proposed method has higher classification accuracy in identifying AVC/HEVC videos, and stronger robustness against frame-deletion and shifted GOP structure attacks.

The rest of the paper is organized as follows. Section 2 analyzes the CU and PU partition types in I and P pictures. Section 3 gives the theoretical analysis and examples of the difference between AVC/HEVC videos and singly compressed HEVC videos. Classification features are described in Section 4. Experimental results are discussed in Section 5, and Section 6 summarizes the paper.

2. CU and PU Partition Types in I and P Pictures of HEVC

HEVC was jointly released by ITU-T [16] and ISO/IEC [17] in 2013 and has been continually improved [18] and supported by hardware producers. It employs the same hybrid coding framework as AVC, including intra/inter prediction and 2D transform coding. However, HEVC has brought in various new techniques to increase compression efficiency. One major technique is the introduction of CU, PU, and transform unit (TU). CU is the basic coding unit with flexible size, and PU and TU are the basic units for intra/inter prediction and transform/quantization, respectively.

The Coding Tree Unit (CTU) is defined as the basic processing unit in HEVC. The first step of HEVC encoding process is to split each coding picture into non-overlapped CTU in which the size is determined by the user. Then, each CTU is divided into multiple CUs in a quadtree structure. Figure 1 displays an example of a quadtree partition of one CTU from a coding picture. The red square denotes one CTU with size a of 64×64, and the digit in the CTU represents the coding order of each CU. We can see that the red CTU in Figure 1a is divided into 16 CUs with the largest size of 32×32 and the smallest size of 8×8. The CUs can be grouped into four types according to their sizes. I picture and P picture adopt the same CTU partition rule; thus, they have the same CU types. In order to demonstrate clearly, we list the four CU types and their corresponding indexes in

Table 1. Indexes of CU types in I picture and P picture.

Index	1	2	3	4
CU Type	8×8	16×16	32×32	64×64

.

When implementing a CTU quadtree partition, the encoder will traverse all the PU partition modes and determine the optimal one for each CU according to rate distortion function. As shown in Figure 2, there are two kinds of prediction modes: intra prediction and inter prediction. In intra prediction mode, two PU partition modes are stipulated. One 2N×2N (N = 4, 8, 16, 32) CU can either be coded as one single PU or divided into four PUs with equal size (N×N). In inter prediction mode, the CU can be split into symmetric or asymmetric PUs. Both symmetric and asymmetric PU consist of four partition modes.

Figure 3 illustrates the PU partition of the CTU marked red in Figure 1. The CU without the dotted line means no PU subdivision, i.e., the PU size equals the CU size, while the CU with the dotted line means that the CU is subdivided into two PUs. For instance, the 4th CU and 15th CU are encoded as one single PU, while asymmetric and symmetric PU partitions are taken by the 1st and 6th CU, respectively. Similar to CUs, the PUs can be categorized into different types depending on the sizes of PUs. During the encoding process of I pictures, only the intra prediction mode is allowed, while both the intra prediction mode and inter prediction mode are supported in the P picture. Thus, there are a total of 4 PU types in the I picture, and 25 PU types in the P picture.

Table 2. Indexes of PU types in the I picture.

Index	1	2	3	4	5
PU Type	4×4	8×8	16×16	32×32	64×64

and

Table 3. Indexes of PU types in the P picture.

Index	1	2	3	4	5
PU Type	4×4	8×8	4×8	8×4	16×16
Index	6	7	8	9	10
PU Type	16×8	8×16	16×12	16×4	12×16
Index	11	12	13	14	15
PU Type	4×16	32×32	32×16	16×32	32×24
Index	16	17	18	19	20
PU Type	32×8	24×32	8×32	64×64	64×32
Index	21	22	23	24	25
PU Type	32×64	64×48	64×16	48×64	16×64

list the correspondence between PU types and PU indexes for the I picture and P picture, respectively.

CU and PU partitions are closely related to image contents. Generally, for regions with simple contents, HEVC tends to select coarse CUs and PUs, i.e., CUs and PUs with large sizes. On the contrary, CUs and PUs with small sizes are often selected for regions with complex contents to adapt to the variable shape of the target. The transcoding process will introduce an irreversible change of the video contents and make the reconstructed video different from the original one. Therefore, we suppose the CU and PU partition types will be different between AVC/HEVC videos and singly compressed HEVC videos. In the subsequent section, we will construct a theoretical model to analyze the factors leading to the differences.

3. CU and PU Partition Types Analysis in Singly Compressed HEVC Videos and AVC/HEVC Videos

In the AVC and HEVC encoding process, quantization and reconstruction are basic operations, but they will introduce irreversible quantization errors and reconstruction errors, which make the decoded video different from the original one. The change of video content will further affect the CU and PU partition types and makes them different between AVC/HEVC videos and singly compressed HEVC videos. We will illustrate the difference elaborately in this section.

3.1. Theoretical Analysis

Figure 4 describes the simplified block diagram of AVC/HEVC transcoding. YUV video is a kind of uncompressed video and often used as test example of video encoders. Given a YUV video V, the first step is to encode V into the AVC bit stream $H_1$ with bitrate $r_1$. Then $H_1$ will be decoded to YUV video $\widehat{\mathit{V}}$ and recompressed into the HEVC bit stream ${\mathit{H}}_2$ with bitrate $r_2$. Please note two points here. One is that the reconstruction module in the encoding process is equivalent to the decoding process; thus we directly use the reconstruction module to represent the decoder in Figure 4 to save space. The other one is that only HEVC encoding is implemented for a singly compressed HEVC video. That is to say, for a singly compressed HEVC video, the input video is the uncompressed YUV video V, not its decoded version $\widehat{\mathit{V}}$.

From Figure 4, we can see that CU and PU partition types of each picture are determined by the content of the picture and the number of bits allocated to it by the rate control module. Here, please note that in this paper, a picture contains only one slice. Though CU and PU partition types in I pictures are different from P pictures, the partition strategy is similar. Therefore, we take the CU types in P pictures as an example to analyze the difference between AVC/HEVC videos and singly compressed HEVC videos.

Now let’s consider the AVC/HEVC transcoding process. Assume one uncompressed video sequence V consists of N P pictures and is expressed as Equation (1), where $I_n$ denotes the nth P picture of V, then the bit stream ${\mathit{H}}_1$ can be obtained by implementing the prediction, transform, quantization, and entropy coding process.

$$V=\left\{I_n|n=1,2,\cdots ,N\right\}$$

(1)

In the AVC encoding process, a rate control process is implemented. Assume the bit rate for V is r, let $u(\cdot )$ denote the rate control process, then, the number of bits allocated to the nth P picture can be represented as $b_n^{(r)}=u\left(n;\mathit{V};r\right)$. After that, the quantization step $Q_p$ for the picture will be determined according to $b_n^{(r)}$. In addition, AVC standard adopts macroblock as the basic coding unit and does not introduce the concept of CU; hence, CU types do not exist in the AVC encoding process.

The decoding process is the inverse process of encoding. Let $C_n$ stand for the prediction signal of $I_n$, $\mathrm{DCT}(\cdot )$ and $\mathrm{IDCT}(\cdot )$ represent discrete cosine transform (DCT) and inverse DCT, respectively, then the decoded video sequence $\widehat{\mathit{V}}$ can be obtained by Equations (2) and (3), where ${\widehat{\mathit{I}}}_n$ means the decoded version of ${\mathit{I}}_n$, $[\cdot ]$ represents the rounding operator,$\mathit{E}({\mathit{I}}_n)$ and $\mathit{E}({\mathit{C}}_n)$ denote the irreversible quantization error and reconstruction error of ${\mathit{I}}_n$ and ${\mathit{C}}_n$, respectively. The quantization error means the error introduced in the quantization process. The reconstruction error means the rounding error and truncation error generated in the reconstruction process.

$$\begin{array}{ll}{\widehat{\mathit{I}}}_n& =\mathrm{IDCT}([\mathrm{DCT}({\mathit{I}}_n-{\mathit{C}}_n)/Q_p]\times Q_p)+{\mathit{C}}_n\\ & \approx \mathrm{IDCT}([\mathrm{DCT}({\mathit{I}}_n)/Q_p]\times Q_p)-\mathrm{IDCT}([\mathrm{DCT}({\mathit{C}}_n)/Q_p]\times Q_p)+{\mathit{C}}_n\\ & ={\mathit{I}}_n+\mathit{E}({\mathit{I}}_n)-\mathit{E}({\mathit{C}}_n)\end{array}$$

(2)

$$\widehat{\mathit{V}}=\left\{{\widehat{\mathit{I}}}_n\right|n=1,2,\dots ,N\}$$

(3)

In the process of transcoding $\widehat{\mathit{V}}$ to the HEVC bit stream ${\mathit{H}}_2$, the number of bits are allocated to the nth P picture according to Equation (4). Here we use $u^{\prime }(\cdot )$ rather than $u(\cdot )$ to represent the rate control process because the bits allocation function adopted in HEVC is different from the AVC standard.

$$b_n^{(r_1,r_2)}=u^{\prime }(n;\widehat{\mathit{V}};r_2)$$

(4)

Knowing the number of bits allocated to the nth P picture, the type of the kth CU in the nth P picture $PC{U}_{n,k}^{(r_1,r_2)}$ can be written as Equation (5), where $\omega (\cdot )$ stands for the CU partition process, and ${\widehat{\mathit{C}}}_n$ denotes the corresponding prediction signal of ${\widehat{\mathit{I}}}_n$.

$$PC{U}_{n,k}^{(r_1,r_2)}=\omega ({\widehat{\mathit{I}}}_n;b_n^{(r_1,r_2)};{\widehat{\mathit{C}}}_n)$$

(5)

For a singly compressed HEVC video with bitrate $r_2$, the number of bits allocated to the nth P picture, and the CU partition type will be determined by Equations (6) and (7).

$$b_n^{(r_2)}=u^{\prime }(n;\mathit{V};r_2)$$

(6)

$$PC{U}_{n,k}^{(r_2)} =\omega ({\mathit{I}}_n;b_n^{(r_2)};C_n)$$

(7)

Eventually, we can get the difference of CU partition types between the AVC/HEVC video and the singly compressed HEVC video according to Equation (8).

$$\begin{array}{ll}PC{U}_{n,k}^{(r_1,r_2)}-PC{U}_{n,k}^{(r_2)}& =\omega ({\widehat{I}}_n;b_n^{(r_1,r_2)};{\widehat{C}}_n)-\omega (I_n;b_n^{(r_2)};C_n)\\ & =\omega ({\widehat{I}}_n;u^{\prime }(n;\widehat{V};r_2);{\widehat{C}}_n)-\omega (I_n;u^{\prime }(n;V;r_2);C_n)\\ & =\omega ({\widehat{I}}_n;u^{\prime }(n;\left\{{\widehat{I}}_n|n=1,2,\cdots ,N\right\};r_2);{\widehat{C}}_n)-\omega (I_n;u^{\prime }(n;\left\{I_n|n=1,2,\cdots ,N\right\};r_2);C_n)\end{array}$$

(8)

As shown in Equation (8), we can see that $PC{U}_{n,k}^{(r_1,r_2)}$ would be different from $PC{U}_{n,k}^{(r_2)}$. There are two factors that lead to the difference. One is the difference between ${\widehat{I}}_n$ and $I_n$. Equation (2) states that the irreversible quantization error and reconstruction error would make ${\widehat{I}}_n$ different from $I_n$. Here, please note that the quantization error is closely related to $Q_p$. The bigger the $Q_p$, the bigger the quantization error. However, $Q_p$ is determined by the number of bits allocated to the picture $b_n^{(r)}$, which makes $Q_p$ indirectly decided by the rate control process. Because the rate control process in AVC is quite different from HEVC, the selection of $Q_p$ in AVC would be different from that in HEVC, which further enlarges the difference between ${\widehat{I}}_n$ and $I_n$. The other factor that causes $PC{U}_{n,k}^{(r_1,r_2)}$ to be different from $PC{U}_{n,k}^{(r_2)}$ is the difference between ${\widehat{C}}_n$ and $C_n$. ${\widehat{C}}_n$ and $C_n$ are the prediction signals of ${\widehat{I}}_n$ and $I_n$, respectively, which are calculated according to the reconstruction pictures encoded before ${\widehat{I}}_n$ and $I_n$. Since there also exists quantization error and reconstruction error in the reconstruction pictures, ${\widehat{C}}_n$ would be different from $C_n$.

Because of the CU types in the I pictures, the PU types in I pictures and P pictures would also be inevitably affected by the quantization error, reconstruction error, and bits allocation method; the theoretical analyses of them are similar to the CU types in P pictures. Therefore, it can be concluded that the CU types in I pictures and P pictures and the PU types in I pictures and P pictures would be different between singly compressed HEVC videos and AVC/HEVC videos.

3.2. Illustrative Examples

In this subsection, we will exhibit the CU and PU partition types of singly compressed HEVC videos and AVC/HEVC videos to demonstrate the difference between them. YUV sequence “sign_irene” is selected as the testing video. For singly compressed HEVC video, this YUV sequence is directly encoded with an HEVC standard at bitrate 300 Kbps, while for a transcoded HEVC video, this YUV sequence is encoded with an AVC standard at bitrate 200 Kbps, 300 Kbps, and 400 Kbps followed by an HEVC standard at bitrate 300 Kbps, respectively. Figure 5 presents the CU and PU partition of the first P picture in the fourth GOP, where solid lines and dotted lines indicate the CU boundaries and PU boundaries, respectively. Figure 5a shows the CU and PU partition of the singly compressed HEVC video. Figure 5b–d show the CU and PU partition of HEVC videos transcoded from AVC videos with bitrates 200 Kbps, 300 Kbps, and 400 Kbps, respectively.

It can be observed that the CU and PU partition in AVC/HEVC transcoded picture is much different from singly compressed HEVC picture, even though they have the same visual content. This phenomenon verifies the analysis depicted in Section 3.1, and thus, CU and PU partition types can be exploited as footprints for AVC/HEVC video detection. Furthermore, looking at the block marked by the red boundaries in Figure 5a,b, they have the same CU partitions, but their PU partitions differ. CU usually reflects the complexity of the CTU contents, and PU can better capture the subtle differences in content. That is to say, CU types and PU types are complementary when encoding one video. The CU types and PU types in the I pictures have a similar phenomenon. Therefore, CU types and PU types can be used as complementary features, and we can merge them to detect AVC/HEVC videos.

In order to illustrate the characteristics of CU and PU partitions more clearly, the numbers of each CU and PU partition type in Figure 5 are exhibited in Figure 6 and Figure 7. We can see that the number of each CU and PU partition type in the AVC/HEVC video is much different from the singly compressed HEVC video. For example, for the 8×8 PU partition type, the numbers of them in 200–300 Kbps and 300–300 Kbps AVC/HEVC videos are nearly 25 and 20 smaller than that in the singly compressed HEVC video, respectively, while the number of it in the 400–300 Kbps AVC/HEVC video is much bigger than that in singly compressed HEVC video. Based on the above theoretic analysis and examples, it is reasonable to take both CU and PU types in the I pictures and P pictures as classification features and use them to identify AVC/HEVC videos with the SVM classification method. The specific feature extraction method and the classification method will be explicated in Section 4.

4. Proposed Method for Detecting HEVC Videos Transcoded from AVC

From the above analysis, we know that the numbers of CU partition types and PU partition types in both I pictures and P pictures are different between singly compressed HEVC videos and AVC/HEVC videos. In order to describe this characteristic and make it more universal, we consider utilizing the mean frequencies of CU partition types from I pictures (I-CU for short), CU partition types from P pictures (P-CU for short), PU partition types from I pictures (I-PU for short), and PU partition types from P pictures (P-PU for short) as classification features. Please note that we utilize the first P picture of each GOP to extract features for P pictures in this paper. The reason is that the characteristics of each P picture in one GOP are similar. In this section, we will firstly describe the feature extraction method in detail and then present the flow of the proposed method.

4.1. Feature Extraction

In the proposed method, we use $s_i (i=1,2,3,4)$ to represent the ith mean frequency set of PU and CU partition types. Specifically, $s_1$ means the 4-dimensional mean frequency set of I-CU partition types, $s_2$ states the 4-dimensional mean frequency set of P-CU partition types, $s_3$ and $s_4$ represent the 5-dimensional mean frequency set of I-PU partition types and 25-dimensional mean frequency set of P-PU partition types, respectively. Each mean frequency set can be given by Equation (9), where $f_{ij}$ represents the mean frequency of the jth partition type of the ith mean frequency set in the whole video, ${(x_{ij})}_k$ means the number of the jth partition type of the ith mean frequency set in the kth I picture or P picture, N denotes the total number of I pictures or P pictures in one video, $M_i$ represents the dimension of the ith mean frequency set, so $M_i$ equals to 4, 4, 5, and 25 when i is 1, 2, 3 and 4, respectively.

$$s_i=\left\{f_{ij}|f_{ij}={\displaystyle {\sum }_{k=1}^N{(x_{ij})}_k/N},j=1,2,\dots ,M_i\right\}$$

(9)

$$S=\left\{s_i|i=1,2,3,4\right\}$$

(10)

According to Section 3.2, PU partition types and CU partition types are complementary features, and we can fuse them to enhance their expressiveness. Therefore, we concatenate the mean frequency set of I-CU, P-CU, I-PU, and P-PU, and get the final combination feature set S, as shown in Equation (10). From the above analysis, S is a 38-dimensional feature set that describes the distribution characteristics of the CU and PU partition.

In Figure 8, a line chart is utilized to show the difference between the classification feature set S of the singly compressed HEVC video and that of the AVC/HEVC video. For display purposes, the mean frequencies of each element in S are plotted in logarithmic form. The red line represents one singly compressed HEVC video, which is directly encoded with an HEVC standard at bitrate 300 Kbps, while the green line, blue line, and black line represent its corresponding AVC/HEVC versions, which are encoded with an AVC standard at bitrate 200 Kbps, 300 Kbps, and 400 Kbps followed by HEVC standard at bitrate 300 Kbps, respectively. The abscissa indicates the index of each element in the classification feature set S, that is to say, one index denotes one kind of PU or CU partition type. The ordinate indicates the corresponding mean frequency of each index, i.e., the mean frequency of each CU or PU partition type. From Figure 8, we can see that the mean frequencies of the singly compressed HEVC video and its corresponding AVC/HEVC versions differ greatly at most indexes, which indicate that the mean frequencies of most CU or PU partition types of the singly compressed HEVC video are greatly different from those of the AVC/HEVC video, and further indicate the effectiveness of the classification feature set. However, it is difficult to separate them by intuitive linear classification. Thus, we bring in SVM, which is quite suitable for solving the classification problem of nonlinear high dimension pattern recognition and small samples, and the detailed processes are provided in the next subsection.

4.2. The Flow of the Proposed Method

For one given test video, the flow of identifying whether it is an AVC/HEVC video is illustrated in Figure 9. Firstly, CU and PU partition types of I pictures and the first P pictures in each GOP are extracted. Then classification feature set S is calculated according to Equations (9) and (10) and sent to SVM. Finally, SVM will decide the category of the given video by using the classification model, which is well trained in the SVM training process beforehand. In the SVM training process, singly compressed HEVC videos samples and AVC/HEVC videos samples are required. The classification feature set S is calculated for both of them and then sent to SVM to get the optimal classification model.

5. Experimental Results

In this section, 29 widely known standard YUV sequences are used, including one low-resolution video set of 18 CIF format videos with a resolution of 352×288, and one high-resolution video set of 11 1080p videos with the resolution of 1920×1080. To increase the size of the video sets, each video is segmented into non-overlapped video clips with a length of 100 frames. Finally, a total of 43 CIF video clips and 32 1080p video clips are generated and used for the subsequent training and testing.

HM [19] with encoder_lowdelay_P_main configuration file and JM [20] with encoder_main configuration file are used to implement the HEVC and AVC encoding and decoding process, respectively. The frame rate, period of the I picture, and GOP structure are set as 30, 4, and IPPP, respectively.

A singly compressed HEVC video is generated by using HM with bitrate $r_2$. In the generation procedure of AVC/HEVC transcoded videos, YUV sequences are firstly encoded with JM at bitrate $r_1$, and then they are decoded with JM and recompressed with HM at bitrate $r_2$. To guarantee the visual quality of compressed videos, the bitrates $r_1$ and $r_2$ used for the CIF video set are selected from {200, 300, 400, 500} (Kbps). $r_1$ and $r_2$ for the 1080p video set are selected from {3, 4, 5} (Mbps).

LIBSVM, proposed by Chang et al. [21], is a popular kind of SVM and suitable for non-linear classification. Therefore, LIBSVM with PolySVC kernel is adopted as a classifier in this paper to distinguish AVC/HEVC videos from firstly compressed ones. A total of 35 and 27 videos are randomly selected as training samples for the CIF video set and 1080p video set, respectively, while the rest videos are used for testing.

As shown in Equation (11), Accuracy Rate (AR) is used to evaluate the performance of the proposed method, where TNR means true negative rate and is defined as the rate that firstly compressed videos are labeled as firstly compressed ones, and TPR means true positive rate and is defined as the rate that transcoded videos are labeled as transcoded ones. The training and testing process is repeated 100 times, and the mean value of AR is treated as the final classification accuracy.

$$\mathrm{AR}=\left(\mathrm{TNR}+\mathrm{TPR}\right)/2$$

(11)

5.1. Accuracy of Detecting AVC/HEVC Videos

We tested the effectiveness of the proposed method in CIF and 1080p video sets, and the results are shown in

Table 4. Classification accuracy in the CIF video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{kbps}\right)$	200	300	400	500
200	0.939	0.991	0.987	0.988
300	0.919	0.991	0.968	0.988
400	0.904	0.976	0.983	0.980
500	0.926	0.953	0.974	0.986

and

Table 5. Classification accuracy in the 1080p video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Mbps}\right)$	2	3	4
2	0.997	0.99	0.993
3	0.991	0.989	0.992
4	0.985	0.983	0.989

. We can see that, whether $r_1$ is smaller than $r_2$ or not, the classification accuracy is higher than 0.9 with a minimum value of 0.904 and a maximum value of 0.997 in the two video sets. The classifying results have verified the theoretical analysis in Section 3 and indicated that the proposed method could identify AVC/HEVC videos very well.

To the best of our knowledge, Reference [16] is the most recent literature, which is published in 2019, and performs best in detecting AVC/HEVC videos. Therefore, in this paper, a comparative analysis with Reference [16] is conducted in the low-resolution CIF video set and high-resolution 1080p video set. Furthermore, the same SVM training and testing process as implemented in the proposed method is adopted for comparison, and the results are reported in

Table 6. Comparison results of the proposed method and Reference [16] in the CIF video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Kbps}\right)$	200		300		400		500
	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed
200	0.911	0.939	0.963	0.991	0.982	0.987	0.983	0.988
300	0.876	0.919	0.978	0.991	0.976	0.968	0.986	0.988
400	0.889	0.904	0.950	0.976	0.979	0.983	0.977	0.980
500	0.876	0.926	0.941	0.953	0.969	0.974	0.983	0.986

and

Table 7. Comparison results of the proposed method and Reference [16] in the 1080p video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Mbps}\right)$	3		4		5
	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed
3	0.984	0.997	0.978	0.99	0.986	0.993
4	0.98	0.991	0.98	0.989	0.98	0.992
5	0.974	0.985	0.977	0.983	0.978	0.989

.

Observing Table 6 and Table 7, it can be seen that the classification accuracy of the proposed method is higher than Bian’s method in almost all cases except the entry (300, 400) in Table 6. When $r_1 =500\mathrm{Kbps},r_2 =200\mathrm{Kbps}$, the accuracy of the proposed method is even 5% higher than Reference [16]. The reason is that only PU types were adopted as the classification features in Bian’s method, and they did not consider the characteristic of CU types and the complementary relationship between PU types and CU types, while the relationship between them was explored and more comprehensive features based on them were extracted in the proposed method. For the entry (300, 400) in Table 6, the classification accuracy of Bian’s method is 0.8 percent higher than the proposed method, which may be caused by the special bitrate and video contents. CU partition types and PU partition types are decided by the video contents and the allocated bitrate. When the video contents are fixed, the CU partition types of singly compressed HEVC videos would be similar to AVC/HEVC videos at some special bitrate. Figure 10 shows the CU partition of one picture in the singly compressed HEVC video at bitrate 400 kbps and its corresponding transcoding version at bitrate 300 kbps for AVC encoding and 400 kbps for HEVC encoding. We can see that the CU partition types are identical at the special bitrate. This would make the features extracted from I-CU and P-CU ineffective and cause redundancy of the feature set S, which further leads to the decrease of SVM classification accuracy.

5.2. Robustness against Frame-Deletion

Frame-deletion is a common forgery type for digital videos. Hence, we tested the robustness of it against the proposed method and Reference [16]. The frame-deleted video is generated as follows. Firstly, the YUV video sequence in the 1080p video set is encoded and decoded with AVC codec at bitrate $r_1$. Then, 30 frames are randomly deleted for each video, and finally, the frame-deleted videos are encoded with HEVC at bitrate $r_2$.

Table 8. Classification accuracy of the proposed method and Reference [16] when deleting 30 frames in the 1080p video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Mbps}\right)$	3		4		5
	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed
3	0.984	0.994	0.976	0.991	0.98	0.994
4	0.981	0.994	0.979	0.99	0.98	0.99
5	0.971	0.987	0.987	0.99	0.975	0.982

exhibits the classification accuracy.

We can see that the classification accuracy is above 0.97 for both the proposed method and Reference [16], which indicates that the proposed method and Reference [16] are effective for detecting frame-deleted AVC/HEVC videos. This is caused by the difference between ${\widehat{I}}_n$ and $I_n$, which is one major factor that causes the difference of CU and PU partition types between AVC/HEVC videos and singly compressed HEVC videos according to the analysis in Section 3. Moreover, frame-deletion operation deletes several frames and further boosts the difference between ${\widehat{I}}_n$ and $I_n$.

On the other hand, it can be observed that the classification accuracy of the proposed method performs better than Reference [16] with the average improvement being about 1.1%, which implies again that CU types are effective features and the utilization of the complementary relationship between PU types and CU types can improve the performance of detecting AVC/HEVC videos.

5.3. Robustness against Shifted GOP Structure

Shifted GOP structure means the GOP size adopted in recompression is different from that adopted in the first compression, which is also a common forgery type for digital videos. Therefore, we tested the robustness of the proposed method and Reference [16] against the shifted GOP structure in the 1080p video set. In the former AVC encoding, the GOP size and GOP structure are set as four and IPPP, respectively. In the latter HEVC encoding, the GOP size and GOP structure should be changed. In this section, the tests are carried out in two cases. One case is that the GOP size used in HEVC encoding is an integral multiple of that in AVC encoding. The other case is the opposite of the former case. In this section, therefore, for the first case, the GOP size and GOP structure are set as eight and IPPPPPPP, respectively, and we call the corresponding video set the GOP4-GOP8 video set for abbreviation. The GOP size and GOP structure used for the second case are six and IPBBPB, respectively, and the corresponding video set is called theGOP4-GOP6 video set. The classification accuracies of the proposed method and Reference [16] are listed in

Table 9. Classification accuracy of the proposed method and Reference [16] in the 1080p GOP4-GOP8 video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Mbps}\right)$	3		4		5
	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed
3	0.991	0.997	0.997	1	0.999	1
4	0.989	0.995	0.991	1	0.999	1
5	0.982	0.987	0.992	1	0.998	1

and

Table 10. Classification accuracy of the proposed method and Reference [16] in the 1080p GOP4-GOP6 video set.

${\mathit{r}}_1\backslash {\mathit{r}}_2\left(\mathbf{Mbps}\right)$	3		4		5
	Ref [16]	Proposed	Ref [16]	Proposed	Ref [16]	Proposed
3	0.991	0.995	0.985	0.989	0.985	0.987
4	0.985	0.988	0.982	0.988	0.985	0.989
5	0.987	0.997	0.99	0.994	0.986	0.99

.

We observed that, whether or not the GOP size used in HEVC encoding is an integral multiple of that in AVC encoding, both the proposed method and Reference [16] performs well under the shifted GOP structure with a classification accuracy above 0.98. Additionally, the proposed method performs better than Reference [16], with the classification accuracy being one in some cases. It indicates that the proposed method is also effective under the shifted GOP structure case.

6. Conclusions

In this paper, an effective method is proposed to detect HEVC videos transcoded from AVC videos. Both PU partition types and CU partition types are analyzed, and the conclusion that they can be used as complementary features is obtained. Based on the analysis, mean frequencies of I-CU, I-PU, P-CU, and P-PU can be calculated and form the final 38-dimensional feature set, which is then sent to SVM for classification. The experimental results verify that the proposed method performs well to detect transcoded HEVC videos and has strong robustness against frame-deletion and shifted GOP structure attack. In future work, we will look for other valid features to improve the classification accuracy and even estimate the codec parameters adopted in the first compression.