Motion Estimation in HEVC/H.265: Metaheuristic Approach to Improve the Efficiency ^†

Abstract

An ever increasing use of digital video applications such as video telephony, broadcast and the storage of high and ultra-high definition videos has steered the development of video coding standards. The state of the art video coding standard is High Efficiency Video Coding (HEVC) or otherwise known as H.265. It promises to be 50 percent more efficient than the previous video coding standard H.264. Ultimately, H.265 provides significant improvement in compression at the expense of computational complexity. HEVC encoder is very complex and 50 percent of the encoding consists of Motion Estimation (ME). It uses a Test Zone (TZ) fast search algorithm for its motion estimation, which compares a block of pixels with a few selected blocks in the search region of a referenced frame. However, the encoding time is not suitable to meet the needs of real time video applications. So, there is a requirement to improve the search algorithm and to provide comparable results to TZ search to save a substantial amount of time. In our paper, we aim to study the effects of a meta-heuristic algorithm on motion estimation. One such suitable algorithm for this task is the Firefly Algorithm (FA). FA is inspired by the social behavior of fireflies and is generally used to solve optimization problems. Our results show that implementing FA for ME saves a considerable amount of time with a comparable encoding efficiency.

1. Introduction

High Efficiency Video Coding (HEVC) is a joint venture of telecommunication industries (ITU-Ts) and technology industries (ISO/IEC MPEG), working together under the Joint Collaborative Team on Video Coding (JCT-VC). In ITU-T recommendations, it is referred to as H.265 and in ISO/IEC it is referred to as MPEG-H part 2. The motivation behind developing HEVC was to provide better coding efficiency in high resolution and ultra-high resolution videos. This is accomplished in HEVC by using a computationally complex encoding process, with many options to increase efficiency. One of the improvements in HEVC is its increased range of block sizes while partitioning a frame into blocks, i.e., from 4 × 4 to 64 × 64. The two most important processes in the block-based video coding standards are Motion Estimation and Motion Compensation. Motion estimation is process of finding location of the best match for a block of pixels in the referenced frame by making a motion vector (MV) and Compensation is to generate the compensated frames by using the motion vectors. HEVC has two modes for its motion estimation, i.e., Full search and Fast search. In full search, a block is compared with every block in the search region of the referenced frame. Whereas, in fast search, a block of pixels is compared with a few selected blocks in the referenced frame. The selection of blocks depends on the fast search algorithm being used [1].

Section 2 provides an overview of HEVC and its encoding process than in Section 3 fast search motion estimation algorithms are explained, Firefly is explained in Section 4 whereas the detailed results and analysis are described in Section 5. Finally, the paper is concluded in Section 6.

2. HEVC

In recent years, an increase in the resolution of the videos such as 4K or 8K UHD, has led to the need to develop a video coding standard with much more coding efficiency than H.264. The key feature of HEVC is that it offers 50% better bitrate saving than the previous H.264, and it is much more effective for HD and UHD resolutions. In many ways HEVC is an extension of the concepts used in H.264; however, it is a state of the art video coding standard that has many more options for encoding than H.264. Its encoder is far more complex than that of H.264 and allows it to surpass the coding efficiency of any previous encoder [2,3].

Inter-Picture Prediction

Inter-picture prediction or motion estimation helps to identify the best match for a block of pixels from the referenced frame, and a motion vector is made to point to the location of the best match. Inter-picture prediction removes the temporal redundancy in the video sequence by only encoding the difference in the current block and its best match in the referenced frame. Motion vector is associated with every block and specifies the location of best match in the referenced frame [4,5,6].

3. Fast Search Motion Estimation Algorithms

Many search algorithms have been developed for the fast search of best matching block within a search region. The aim of these fast search motion estimation algorithms is to reduce the number of search points and achieve efficiency comparable to Full search. Additionally, the computational complexity is considerably reduced by using limited search points. Many block-based search algorithms have been introduced, including Three Step Search (TSS), Two Dimensional Logarithmic Search (TDL), Four Step Search (FSS), Diamond Search, Hexagon Search and Test Zone Search (TZ). TZ search is used in HEVC [7,8,9].

4. Firefly Algorithm

Firefly can be used for motion estimation by appropriately establishing the parameters and cost function. In the case of motion estimation, we can use the Sum of Absolute Difference (SAD) as the cost function and a block of pixels can act as our firefly. An appropriate number of fireflies should be placed randomly in the search window of the referenced frame, and their intensities can be calculated according to the cost function. As ME uses the SAD function, the firefly with the minimum SAD should have maximum intensity. Additionally, the number of iterations is controllable and can be adjusted to provide comparable results to TZ search while saving considerable time [10,11,12].

5. Results & Analysis

Test sequences of different resolutions were selected. Each of these sequences have different camera and object movements to establish the dependability of results for different scenarios. In order to test the performance of FA for motion estimation and compare this with the TZ search algorithm, both the algorithms were implemented in a MATLAB 2016 programming environment and executed on an Ubuntu operating system running on Intel Core i7, CPU @ 4.00 GHz and 32 GB RAM.

5.1. CIF Resolution Sequence

The CIF resolution selected was the ‘football.yuv’ sequence, in which football players have a very random motion and the camera pans and tilts to capture the moving players. Results for ‘football.yuv’ sequence are shown in

Table 1. Football Sequence.

Algorithm	Iteration	ME Time (s)	SAD × 106	Time Ratio	SAD% Difference
TZ search	-	25.513	1.9059	-	-
FA	1	4.123	2.0298	6.1843	6.5045
FA	3	16.46	2.1589	3.4344	1.8974

.

The TZ search algorithm required 25.513 s for ME whereas a single iteration of FA computed the same in just 4.123 s, resulting in a speed-up factor of 6.18 times; however, the quality of estimation is compromised with 6.5% more SAD values for FA as compared to TZ search. This difference in SAD values is reduced by increasing the number of iterations of FA, which in effect reduces the speed-up factor. Still, with three iterations of FA, the speed up factor is 3.43 times and the percentage SAD difference reduces to 1.89%, which is comparable to the TZ search in terms of quality of estimation.

5.2. 576. p Resolution Sequence

The 576p resolution sequence is ‘ice.yuv’, for which the camera pans to capture people skating on ice. Results for this sequence are presented in

Table 2. Ice Sequence.

Algorithm	Iteration	ME Time (s)	SAD × 106	Time Ratio	SAD% Difference
TZ search	-	50.277	5.3056	-	-
FA	1	29.533	5.2948	1.7024	−0.2048

. For ‘ice.yuv’, the estimation time calculated using TZ search algorithm was 50.27667 s with SAD value of 5.3056 × 10⁶. Using a single iteration of FA the average time taken for motion estimation is 29.533 s and SAD value is 5.295 × 10⁶. So, a single iteration of the FA algorithm speeds up the process of finding MVs by 1.7 times, with a decrease in the SAD values of 0.205%, as compared to TZ search algorithm.

5.3. 720. p Resolution Sequence

The raw video sequence of 720p resolution selected for testing was ‘Ducks.yuv’. In this sequence, a duck sitting on water takes off, with a medium amount of motion, recorded using a static camera. Results for this sequence are presented in

Table 3. Ducks Sequence.

Algorithm	Iteration	ME Time (s)	SAD × 106	Time Ratio	SAD% Difference
TZ search	-	86.617	2.221	-	-
FA	1	56.838	2.211	1.5239	−0.4427

. Using a single iteration, the FA algorithm speeds up the process of finding MVs by 1.5239 times with an average decrease in the SAD values of 0.443% as compared to the TZ search algorithm.

6. Conclusions

It is evident from the results that the Firefly algorithm can be used for motion estimation in HEVC. The input parameters can be adjusted to obtain the best possible results. The Firefly algorithm reduces encoding time compared to the TZ search algorithm. A single iteration of FA can speed up the process of estimation by up to 6 times. Even for the worst cases, the FA saves times by a factor of 1.47, which is equivalent to a reduction in time of almost 50%. Based on these results, FA shows great potential for use in motion estimation in HEVC.