Next Article in Journal
Link-Quality Measurement and Reporting in Wireless Sensor Networks
Previous Article in Journal
A Novel Cell-Based Hybrid Acoustic Wave Biosensor with Impedimetric Sensing Capabilities

Sensors 2013, 13(3), 3056-3065; doi:10.3390/s130303056

Article
A Rank-Ordered Marginal Filter for Deinterlacing
Gwanggil Jeon 1, Marco Anisetti 2 and Seok Hoon Kang 1,*
1
Department of Embedded Systems Engineering, Incheon National University, 12-1 Songdo-dong, Yeonsu-gu, Incheon 406-772, Korea; E-Mail: gjeon@incheon.ac.kr
2
Dipartimento di Informatica, Universita degli Studi di Milano, via Bramante 65 - 26013 Crema (CR), Italy; E-Mail: marco.anisetti@unimi.it
*
Author to whom correspondence should be addressed; E-Mail: hana@incheon.ac.kr; Tel.: +82-32-835-8422; Fax: +82-32-835-0782.
Received: 5 January 2013; in revised form: 13 February 2013 / Accepted: 16 February 2013 /
Published: 4 March 2013

Abstract

: This paper proposes a new interpolation filter for deinterlacing, which is achieved by enhancing the edge preserving ability of the conventional edge-based line average methods. This filter consists of three steps: pre-processing step, fuzzy metric-based weight assignation step, and rank-ordered marginal filter step. The proposed method is able to interpolate the missing lines without introducing annoying articles. Simulation results show that the images filtered with the proposed algorithm restrain less annoying pixels than the ones acquired by other methods.
Keywords:
rank-ordered filter; adaptive marginal filter; deinterlacing; format conversion

1. Introduction

Interlaced scanning has been advanced from the early days of TV and still adopted for SDTV and 1080i HDTV broadcast standards [1]. However, nearly all late model flat panel displays (LCD, PDP, etc.) use progressive scanning formats. For these display devices, an entering interlaced video signal has to be transformed to a progressive one, and thus a scanning format conversion that gives compatibility between various video formats is required [2]. The super-resolution (SR) is a class of techniques that enhance the resolution of an imaging system [38]. The deinterlacing only considers vertical direction, while SR considers both of vertical and horizontal directions. Thus, the intra-field deinterlacing is a special case of SR.

Many deinterlacing methods have been proposed, including spatial methods [913] and motion-based methods [14]. Although motion-based methods yield better subjective quality than spatial methods, they require reliable motion models and the estimated trajectories must be sufficiently proper, which generally causes excessive computational complexity On the other hand, spatial methods have lower computational complexity since they only demand the current frame, making them more suitable for real-time applications. Therefore, in this paper, we focus on the spatial method.

Among spatial approaches, deinterlacing based on edge direction is the most outstanding and broadly adopted method. These methods calculate edge information first and then decide edge direction to utilize appropriate pixels for interpolation. Thus the edge information calculation and edge direction decision are the key steps. However, conventional methods have yielded poor performance when edge direction is not credible.

To shorten this issue, we propose a deinterlacing algorithm using rank-ordered fuzzy metric approach to reduce artifacts in deinterlaced images. In our approach, the missing lines are calculated by weight obtained using fuzzy metric (FM) from the existing neighbor pixels. The local FM infers the weight of the edge information. Thus, we deinterlace the interlaced signal without calculating edge directions as the traditional approaches do. After that, the rank-ordered differences statistic introduced in [15] is accommodated to the fuzzy context utilizing the introduced FM.

The paper is arranged as follows. Section 2 introduces FM used in the weight assignation step. After that, the proposed filtering technique is described. Section 3 shows simulation results including performance comparison and computational complexity. Finally, conclusions are drawn in Section 4.

2. Proposed Method

2.1. Fuzzy Metric for Weight Assignment

A stationary FM, on a set S, is a fuzzy set of S × S satisfying the following conditions for all p,q,rS[15]:

  • Rule1: FMs(p,q,t) > 0;

  • Rule2: FMs(p, q, t) = 1 if and only if p = q;

  • Rule3: FMs(p,q,t) = FMs(q,p,t);

  • Rule4: FMs(p, q, t) ≥ FMs(q, r, u) * FMs(p, r,t + u);

where Rule# is a FM of rule number #, * is a continuous t-norm, FMs(p, q, t) stands for the degree of nearness between p and q according to Rule2, FMs(p, q, t) is close to 0 when p is far from q. Let S be the set {0,1,…, 255}, then, the function FM : S × S → [0,1] given by
F M S ( p , q ) = ( min ( p , q ) + b max ( p , q ) + b ) a
where b is a small positive value for preventing max(p, q) = 0 singularity. As the difference between the components p and q become bigger, the value of FMs falls quickly. Thus, we assume FMs(p, q) is the fuzzy distance between the image components p and q. Clearly, FM is F-founded and it meets
0 b I max + b F M S ( p , q ) 1
for all p,qS, Imax is maximum pixel intensity, and Imax = 255 in this paper.

2.2. Deinterlacing Implementation

The proposed filter consists of three steps: (1) pre-processing step, (2) FM-based weight assignation step, and (3) rank-ordered marginal filter step. To begin with, we conduct interpolation with three missing pixels at location (−1, 0), (0, 0), and (1, 0), with vertical six-tap filters. After that, we evaluate FM degree using the introduced FM equation. The obtained FM degree is used for assigning weights. Finally, the missing pixel is calculated using the rank-ordered marginal filtering (ROMF) scheme.

Let us assume that I is an image and I(c,r) is the pixel intensity at a position of (c, r), c is column number and r is raw number, and I(0,0) is the centered missing pixel to be processed. We denote W as a filtering window centered on the pixel under processing of size N×N,N = 3,5,7,…, which contains n = N2 pixels. The pixels in W are symbolized as I(c,r), and c, r = −1, 0,1 for N = 3 case.

The first step of the ROMF method is vertical six-tap filter (STF). This fixed coefficient six-tap Wiener filter is widely used to estimate the sub-pixels in video codec, such as MPEG-4, H.264/AVC, and some deinterlacing methods [16]. The coefficients of this filter can be different such as h = [1, −5, 20, 20, −5, 1]/32 or h = [3, −17, 78, 78, −17, 3]/128. In this paper, we chose the previous one for our system under the assumption that h can calculate missing lines in the sub-pixel position properly. The missing pixels at (c, 0) position, c = −1,0,1, are estimated using the adjacent pixels at (c, −5), (c, −3), (c, −1), (c, 1), (c, 3), and (c,5), and we denote them as I(c,−5), I(c,−3), I(c,−1), I(c,1), I(c,3), and I(c,5), respectively. To interpolate the pixel more precisely, we must adapt the filter to accommodate the new interpolation condition. Now, three pixels in the missing line I ( 1 , 0 ) STF, I ( 0 , 0 ) STF and I ( 1 , 0 ) STF are approximately deinterlaced applying Equation (3); however, they are not the same with the original missing pixel. Figure 1 shows the pixel positions with filter coefficients.

I ( c , 0 ) STF = h ( 1 ) I ( c , 5 ) + h ( 2 ) I ( c , 3 ) + h ( 3 ) I ( c , 1 ) + h ( 4 ) I ( c , 1 ) + h ( 5 ) I ( c , 3 ) + h ( 6 ) I ( c , 5 )

For the ROMF, eight neighboring pixels, I(−1,−1), I(0,−1), I(1,−1), I ( 1 , 0 ) STF, I ( 1 , 0 ) STF, I(−1,1), I(0,1) and I(1,1) are employed to deinterlace the missing center pixel at (0,0). In this paper, we take FMS as the distance function (note, however, that any other function such as Euclidean distance could be used). Therefore, the distance between two pixels I ( 0 , 0 ) STF and I(c,r) is symbolized as FMS ( I ( 0 , 0 ) STF,I(c,r)). We denote the set of neighbors of I ( 0 , 0 ) STF, that is, W ¯ = W I ( 0 , 0 ) STF.

The second step is to calculate eight FM using FMs( I ( 0 , 0 ) STF, I(c,r)) where I(c,r). The proposed deinterlacing solves the problem by looking for the most robust I(c,r) pixel. To compute ROMF, the distance FMS( I ( 0 , 0 ) STF,I(c,r)) are rearranged in an ascending order so that a group of non-negative real values χm, where fixed a positive integer m ≤ n − 1, are obtained. Note that χm is not always different: χ1χ2 ≤ … ≤ χm ≤ … χn−1. Generally speaking, χj is the jth smallest FMS( I ( 0 , 0 ) STF,I(c,r)) value, and its associated I(c,r) is denoted as Iχj. Finally, the proposed ROMF calculates the missing pixel I ( 0 , 0 ) ROMF:

I ( 0 , 0 ) ROMF = 1 2 I ( 0 , 0 ) STF + 1 2 j = 1 m χ j I χ j j = 1 m χ j
where χj is assumed to be a weight factor. It can be observed from Equations (1) and (4) that, when Iχj and I ( 0 , 0 ) STF have similar values, the weight factor χj becomes large. On the other hand, when the difference between Iχj and I ( 0 , 0 ) STF are large, χj becomes smaller. Thus, the missing line is deinterlaced based on the similarity among their neighbor pixels, Iχj, with allocated weights based on the FM.

3. Simulation Results

To evaluate the performance of the proposed algorithm, we present the simulation results in this section. We considered twenty images and video sequences as the dataset, which are shown in Table 1. The ten images starting with “A” to “G” are the test images, and the others (images starting with “K” to “Z”) are the training images.

We conducted simulation using MATLAB with an Intel(R) Core(TM) i5 CPU M460 @ 2.53 GHz processor. We compared the proposed method with MELA [9], LABI [10], FEPD [11], MCAD [12] and LSMD [13] methods. Note that the designed filter parameters a and b and the number of considered neighbor pixels m play crucial roles, making it important to set them appropriately. One assumption is that, as we mentioned in Section 2, parameter b is a small positive value for avoiding max(p,q) = 0 singularity. Thus we gave b=1, which is the smallest intensity step. Figure 2 shows the average MSE performance of the proposed method according to various m values under the condition of b = 1 and 1 ≤ a ≤ 15. From Figure 2, m = 3 is determined to give the least MSE. Another parameter a = 10 is determined under the condition of b = 1 and m = 3, as shown in Figure 3.

The PSNR metric in decibels (dB) was selected to evaluate the performance. Table 2 shows the comparison results of the PSNR performance of the proposed method to the benchmarks. After the experiments, it is obvious that the proposed method outperforms other methods by 0.959 (MELA), 1.199 (LABI), 2.414 (FEPD), 1.541 (MCAD), and 1.377 (LSMD) dB in terms of average PSNR. For Akiyo and Bus image, MELA showed a better PSNR performance of 0.107 dB and 0.035 dB. However, the proposed method showed the best PSNR performance for the other images.

Table 3 shows the CPU time per image. As we can see, the proposed method has more complexity than MELA. However, the proposed technique reduces the average CPU time up to 93.74%, 96.89%, 95.57%, and 79.52% when compared with LABI, FEPD, MCAD, and LSMD, respectively.

The Barbara image in Figure 4 has many low and high-angle directions that the previous methods may miss. Figure 4(a–g) shows poor performance because only a limited number of edge directions were utilized, which does not compensate for inaccurate edge information. Figure 4(h,i) shows better results than the other conventional methods. However, the diagonal edge reconstruction is not sufficient. The proposed method, however, performs well for this case as shown in Figure 4(j). Figure 5 shows the results for the Boat image. The result for this image also shows that the proposed method is superior to other methods.

4. Conclusions

This paper presented an effective spatial deinterlacing method, which is achieved by improving the edge preserving ability of the conventional edge-based line average method. This filter consists of three steps: pre-determined six-tap filter based pre-processing step, FM-based weight assignation step, and rank-ordered marginal filter step. The experimental results indicated that ROMF has achieved these two goals and has promising performance subjectively and objectively. Meanwhile, ROMF has merits of low complexity for real-time application.

This work was supported by the Incheon National University Research Grant in 2012.

References

  1. Jack, K. Video Demystified—A Handbook for the Digital Engineer, 4th ed.; Elsevier: Oxford, UK, 2005. [Google Scholar]
  2. Bellars, E.B.; Haan, G.D. Deinterlacing: A Key Technology for Scan Rate Conversion; Elsevier: Oxford, UK, 2000. [Google Scholar]
  3. Wu, J.; Paul, A.; Xing, Y.; Fang, Y.; Jeong, J.; Jiao, L.; Shi, G. Morphological dilation image coding with context weights prediction. Signal Process. Image Commun. 2010, 25, 717–728. [Google Scholar]
  4. Wu, W.; Liu, Z. Learning-based super resolution using kernel partial least squares. Image Vis. Comput. 2011, 29, 394–406. [Google Scholar]
  5. Wu, J.; Huang, J.; Jeon, G.; Jeong, J.; Jiao, L.C. An adaptive autoregressive de-interlacing method. Opt. Eng. 2011, 50. [Google Scholar] [CrossRef]
  6. Wu, J.; Liu, Z.; Gueaieb, W.; He, X. Single-image super-resolution based on Markov random field and contourlet transform. J. Electron. Image 2011, 20. [Google Scholar] [CrossRef]
  7. Wu, J.; Li, T.; Hsieh, T.-J.; Chang, Y.-L.; Huang, B. Digital signal processor-based 3D wavelet error-resilient lossless compression of high-resolution spectrometer data. J. Appl. Remote Sens. 2011, 5. [Google Scholar] [CrossRef]
  8. Wu, J.; Liu, Z.; Krys, D. Improving laser image resolution for pitting corrosion measurement using markov random field method. Autom. Constr. 2012, 21, 172–183. [Google Scholar]
  9. Kim, W.; Jin, S.; Jeong, J. Novel intra deinterlacing algorithm using content adaptive interpolation. IEEE Trans. Cons. Electron. 2007, 53, 1036–1043. [Google Scholar]
  10. Lee, D.-H. A new edge-based intra-field interpolation method for deinterlacing using locally adaptive-thresholded binary image. IEEE Trans. Cons. Electton. 2008, 54, 110–115. [Google Scholar]
  11. Yang, S.; Kim, D.; Jeong, J. Fine edge-preserving deinterlacing algorithm for progressive display. IEEE Trans. Cons. Elect 2009, 55, 1654–1662. [Google Scholar]
  12. Park, S.J.; Jeon, G.; Jeong, J. Covariance-Based Adaptive Deinterlacing Method Using Edge Map. Proceedings of the IEEE IPTA2010, Paris, France, July 2010; pp. 166–171.
  13. Park, S.J.; Jeong, J. Local surface model-based deinterlacing algorithm. Opt. Eng. 2011, 50. [Google Scholar] [CrossRef]
  14. Jeon, G.; Anisetti, M.; Kim, D.; Bellandi, V.; Damiani, E.; Jeong, J. Fuzzy rough sets hybrid scheme for motion and scene complexity adaptive deinterlacing. Image Vis. Comput. 2009, 27, 425–436. [Google Scholar]
  15. George, A.; Romaguera, S. Characterizing completable fuzzy metric spaces. Fuzzy Set. Syst. 2004, 144, 411–420. [Google Scholar]
  16. Hong, S.M.; Park, S.-J.; Jeong, J. Deinterlacing algorithm using fixed directional interpolation filter and adaptive distance weighting scheme. SPIE Opt. Eng. 2011, 50. [Google Scholar] [CrossRef]
Sensors 13 03056f1 1024
Figure 1. The pixel positions with filter coefficients.

Click here to enlarge figure

Figure 1. The pixel positions with filter coefficients.
Sensors 13 03056f1 1024
Sensors 13 03056f2 1024
Figure 2. Performance of the proposed method in terms of MSE as a value of m under the condition of b = 1, for 1 ≤ a ≤ 15.

Click here to enlarge figure

Figure 2. Performance of the proposed method in terms of MSE as a value of m under the condition of b = 1, for 1 ≤ a ≤ 15.
Sensors 13 03056f2 1024
Sensors 13 03056f3 1024
Figure 3. Performance of the proposed method in terms of MSE as a value of a under the condition of b = 1 and m = 3.

Click here to enlarge figure

Figure 3. Performance of the proposed method in terms of MSE as a value of a under the condition of b = 1 and m = 3.
Sensors 13 03056f3 1024
Sensors 13 03056f4 1024
Figure 4. Comparison of subjective qualities in Barbara image: (a) original Barbara; (b) MELA; (c) LABI; (d) FEPD; (e) MCAD; (f) LSMD; and (g) ROMF.

Click here to enlarge figure

Figure 4. Comparison of subjective qualities in Barbara image: (a) original Barbara; (b) MELA; (c) LABI; (d) FEPD; (e) MCAD; (f) LSMD; and (g) ROMF.
Sensors 13 03056f4 1024
Sensors 13 03056f5 1024
Figure 5. Comparison of subjective qualities in Boat image: (a) original Boat; (b) MELA; (c) LABI; (d) FEPD; (e) MCAD; (f) LSMD; and (g) ROMF.

Click here to enlarge figure

Figure 5. Comparison of subjective qualities in Boat image: (a) original Boat; (b) MELA; (c) LABI; (d) FEPD; (e) MCAD; (f) LSMD; and (g) ROMF.
Sensors 13 03056f5 1024
Table Table 1. Test and training sets classified by alphabetical order.

Click here to display table

Table 1. Test and training sets classified by alphabetical order.
Test images (I) and video (V) sequences (images starting with “A” to “G”):
Airplane (I), Akiyo (V), Barbara (I), Bluesky (V), Boat (I),
Bus (V), City (V), Finger (I), Football (V), Girl (I)
Training images (I) and video (V) sequences (images start with “K” to “Z”):
Kimono (V), Lena (I), Man (I), Milkdrop (I), Mobile (V),
News (V), Peppers (I), Raven (V), Toys (V), Zelda (I)
Table Table 2. Comparison of the average PSNR for 10 test images and video sequences among different deinterlacing methods.

Click here to display table

Table 2. Comparison of the average PSNR for 10 test images and video sequences among different deinterlacing methods.
MELALABIFEPDMCADLSMDROMFRanking
airplane35.08835.34534.38535.08535.66036.0841
akiyo40.20538.84137.25539.72638.14940.0982
barbara32.01831.93028.87925.92929.41433.5621
bluesky37.90037.79837.51038.10739.37339.5471
boat35.18635.27733.07435.34233.76236.0341
bus28.65428.21728.10428.26228.09528.6192
city31.46031.49731.25831.52731.65631.7261
finger31.32331.36230.67931.81032.08532.9461
football35.05734.47533.30835.03434.76335.7911
girl41.79341.53539.67642.03841.54543.8611
avg.34.86834.62833.41334.28634.45035.8271
Table Table 3. Comparison of the average CPU time for 10 test images and video sequences among different deinterlacing methods.

Click here to display table

Table 3. Comparison of the average CPU time for 10 test images and video sequences among different deinterlacing methods.
MELALABIFEPDMCADLSMDROMFRanking
airplane0.54714.69828.49122.5474.2861.2314
akiyo0.2075.69410.9477.8611.8380.5964
barbara0.49014.15429.52720.6094.4090.7874
bluesky3.076123.768228.331159.36934.8956.0674
boat0.47013.49028.72720.1914.0761.1074
bus0.1645.40610.9198.8462.3040.7034
city1.35848.484105.55373.75216.5022.5904
finger0.39712.34929.00520.3154.0361.0574
football0.2035.05712.6767.7551.6680.7944
girl0.43312.30530.80119.8864.0701.0654
avg.0.73425.5451.49836.1137.8091.5994
Sensors EISSN 1424-8220 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert