A Rank-Ordered Marginal Filter for Deinterlacing

This paper proposes a new interpolation filter for deinterlacing, which is achieved by enhancing the edge preserving ability of the conventional edge-based line average methods. This filter consists of three steps: pre-processing step, fuzzy metric-based weight assignation step, and rank-ordered marginal filter step. The proposed method is able to interpolate the missing lines without introducing annoying articles. Simulation results show that the images filtered with the proposed algorithm restrain less annoying pixels than the ones acquired by other methods.


Introduction
Interlaced scanning has been advanced from the early days of TV and still adopted for SDTV and 1080i HDTV broadcast standards [1]. However, nearly all late model flat panel displays (LCD, PDP, etc.) use progressive scanning formats. For these display devices, an entering interlaced video signal has to be transformed to a progressive one, and thus a scanning format conversion that gives compatibility between various video formats is required [2]. The super-resolution (SR) is a class of techniques that enhance the resolution of an imaging system [3][4][5][6][7][8]. The deinterlacing only considers vertical direction, while SR considers both of vertical and horizontal directions. Thus, the intra-field deinterlacing is a special case of SR.
Many deinterlacing methods have been proposed, including spatial methods [9][10][11][12][13] and motion-based methods [14]. Although motion-based methods yield better subjective quality than spatial methods, they require reliable motion models and the estimated trajectories must be sufficiently proper, which generally causes excessive computational complexity. On the other hand, spatial methods have lower computational complexity since they only demand the current frame, making them more suitable for real-time applications. Therefore, in this paper, we focus on the spatial method.
Among spatial approaches, deinterlacing based on edge direction is the most outstanding and broadly adopted method. These methods calculate edge information first and then decide edge direction to utilize appropriate pixels for interpolation. Thus the edge information calculation and edge direction decision are the key steps. However, conventional methods have yielded poor performance when edge direction is not credible.
To shorten this issue, we propose a deinterlacing algorithm using rank-ordered fuzzy metric approach to reduce artifacts in deinterlaced images. In our approach, the missing lines are calculated by weight obtained using fuzzy metric (FM) from the existing neighbor pixels. The local FM infers the weight of the edge information. Thus, we deinterlace the interlaced signal without calculating edge directions as the traditional approaches do. After that, the rank-ordered differences statistic introduced in [15] is accommodated to the fuzzy context utilizing the introduced FM.
The paper is arranged as follows. Section 2 introduces FM used in the weight assignation step. After that, the proposed filtering technique is described. Section 3 shows simulation results including performance comparison and computational complexity. Finally, conclusions are drawn in Section 4.

Fuzzy Metric for Weight Assignment
A stationary FM, on a set S, is a fuzzy set of S × S satisfying the following conditions for all p, q, r ∈ S [15]: Rule 1 : FM S (p, q, t) > 0; Rule 2 : FM S (p, q, t) = 1 if and only if p = q; Rule 3 : FM S (p, q, t) = FM S (q, p, t); where Rule is a FM of rule number , * is a continuous t-norm, FM S (p, q, t) stands for the degree of nearness between p and q according to Rule 2 , FM S (p, q, t) is close to 0 when p is far from q. Let S be the set {0, 1, . . . , 255}, then, the function F M : S × S → [0, 1] given by where b is a small positive value for preventing max(p, q) = 0 singularity. As the difference between the components p and q become bigger, the value of FM S falls quickly. Thus, we assume FM S (p, q) is the fuzzy distance between the image components p and q. Clearly, FM is F-founded and it meets for all p, q ∈ S, I max is maximum pixel intensity, and I max = 255 in this paper.

Deinterlacing Implementation
The proposed filter consists of three steps: (1) pre-processing step, (2) FM-based weight assignation step, and (3) rank-ordered marginal filter step. To begin with, we conduct interpolation with three missing pixels at location (-1, 0), (0, 0), and (1, 0), with vertical six-tap filters. After that, we evaluate FM degree using the introduced FM equation. The obtained FM degree is used for assigning weights. Finally, the missing pixel is calculated using the rank-ordered marginal filtering (ROMF) scheme.
Let us assume that I is an image and I (c,r) is the pixel intensity at a position of (c, r), c is column number and r is raw number, and I (0,0) is the centered missing pixel to be processed. We denote W as a filtering window centered on the pixel under processing of size N × N, N = 3, 5, 7, . . . , which contains n = N 2 pixels. The pixels in W are symbolized as I (c,r) , and c, r = −1, 0, 1 for N = 3 case.
The first step of the ROMF method is vertical six-tap filter (STF). This fixed coefficient six-tap Wiener filter is widely used to estimate the sub-pixels in video codec, such as MPEG-4, H.264/AVC, and some deinterlacing methods [16]. The coefficients of this filter can be different such as , I (0,1) and I (1,1) are employed to deinterlace the missing center pixel at (0,0). In this paper, we take FM S as the distance function (note, however, that any other function such as Euclidean distance could be used). Therefore, the distance between two pixels I STF (0,0) and I (c,r) is symbolized as FM S (I STF (0,0) , I (c,r) ). We denote W the set of neighbors of I STF (0,0) , that is, W = W − I STF (0,0) . The second step is to calculate eight FM using FM S (I STF (0,0) , I (c,r) ) where I (c,r) ∈ W . The proposed deinterlacing solves the problem by looking for the most robust I (c,r) pixel. To compute ROMF, the distance FM S (I STF (0,0) , I (c,r) ) are rearranged in an ascending order so that a group of non-negative real values χ m , where fixed a positive integer m ≤ n − 1, are obtained. Note that χ m is not always different: Generally speaking, χ j is the j th smallest FM S (I STF (0,0) , I (c,r) ) value, and its associated I (c,r) is denoted as I χ j . Finally, the proposed ROMF calculates the missing pixel I ROMF (0,0) : where χ j is assumed to be a weight factor. It can be observed from Equations (1) and (4) that, when I χ j and I STF (0,0) have similar values, the weight factor χ j becomes large. On the other hand, when the difference between I χ j and I STF (0,0) are large, χ j becomes smaller. Thus, the missing line is deinterlaced based on the similarity among their neighbor pixels, I χ j , with allocated weights based on the FM.

Simulation Results
To evaluate the performance of the proposed algorithm, we present the simulation results in this section. We considered twenty images and video sequences as the dataset, which are shown in Table 1. The ten images starting with "A" to "G" are the test images, and the others (images starting with "K" to "Z") are the training images.  We conducted simulation using MATLAB with an Intel(R) Core(TM) i5 CPU M460 @ 2.53 GHz processor. We compared the proposed method with MELA [9], LABI [10], FEPD [11], MCAD [12] and LSMD [13] methods. Note that the designed filter parameters a and b and the number of considered neighbor pixels m play crucial roles, making it important to set them appropriately. One assumption is that, as we mentioned in Section 2, parameter b is a small positive value for avoiding max(p, q) = 0 singularity. Thus we gave b = 1, which is the smallest intensity step. Figure 2 shows the average MSE    The PSNR metric in decibels (dB) was selected to evaluate the performance. Table 2 shows the comparison results of the PSNR performance of the proposed method to the benchmarks. After the experiments, it is obvious that the proposed method outperforms other methods by 0.959 (MELA), 1.199 (LABI), 2.414 (FEPD), 1.541 (MCAD), and 1.377 (LSMD) dB in terms of average PSNR. For Akiyo and Bus image, MELA showed a better PSNR performance of 0.107 dB and 0.035 dB. However, the proposed method showed the best PSNR performance for the other images. Table 3 shows the CPU time per image. As we can see, the proposed method has more complexity than MELA. However, the proposed technique reduces the average CPU time up to 93.74%, 96.89%, 95.57%, and 79.52% when compared with LABI, FEPD, MCAD, and LSMD, respectively. The Barbara image in Figure 4 has many low and high-angle directions that the previous methods may miss. Figure 4(a-g) shows poor performance because only a limited number of edge directions were utilized, which does not compensate for inaccurate edge information. Figure 4(h,i) shows better results than the other conventional methods. However, the diagonal edge reconstruction is not sufficient. The proposed method, however, performs well for this case as shown in Figure 4(j). Figure 5 shows the results for the Boat image. The result for this image also shows that the proposed method is superior to other methods.

Conclusions
This paper presented an effective spatial deinterlacing method, which is achieved by improving the edge preserving ability of the conventional edge-based line average method. This filter consists of three steps: pre-determined six-tap filter based pre-processing step, FM-based weight assignation step, and rank-ordered marginal filter step. The experimental results indicated that ROMF has achieved these two goals and has promising performance subjectively and objectively. Meanwhile, ROMF has merits of low complexity for real-time application.