Tracking-Based Denoising: A Trilateral Filter-Based Denoiser for Real-World Surveillance Video in Extreme Low-Light Conditions ^†

Video denoising in extremely low-light surveillance scenarios is a challenging task in computer vision, as it suffers from harsh noise and insufficient signal to reconstruct fine details. The denoising algorithm for these scenarios encounters challenges such as the lack of ground truth, and the noise distribution in the real world is far more complex than in a normal scene. Consequently, recent state-of-the-art (SOTA) methods like VRT and Turtle for video denoising perform poorly in this low-light environment. Additionally, some methods rely on raw video data, which is difficult to obtain from surveillance systems. In this paper, a denoising method is proposed based on the trilateral filter, which aims to denoise real-world low-light surveillance videos. Our trilateral filter is a weighted filter, allocating reasonable weights to different inputs to produce an appropriate output. Our idea is inspired by an experimental finding: noise on stationary objects can be easily suppressed by averaging adjacent frames. This led us to believe that if we can track moving objects accurately and filter along their trajectories, the noise may be effectively removed. Our proposed method involves four main steps. First, coarse motion vectors are obtained by bilateral search. Second, an amplitude-phase filter is used to judge and correct erroneous vectors. Third, these vectors are refined by a full search in a small area for greater accuracy. Finally, the trilateral filter is applied along the trajectory to denoise the noisy frame. Extensive experiments have demonstrated that our method achieves superior performance in terms of visual effects and quantitative tests.