Consider a scene represented as a vector X of length N. The CI camera observes the scene and generates a measurement vector Y of length M. In a noise free scenario, each of the M elements in the measurement Y represents a projection of the scene X onto the basis vectors comprising the projection matrix Φ. In matrix vector form, this set of linear equations can be expressed as:

or:

where the dimensions of the projection matrix Φ are M × N, and each row of Φ represents a sampling of the underlying image signal. If image signals are sparse, such signals can be expressed by a set of coefficients θºR^N in some orthonormal basis ψ ∈ R^N×N:

In many cases, the basis ψ = [ψ₁ψ₂ … ψ_n] can be chosen so that only K ≪ N coefficients have significant magnitude. The image signal can be called K-sparse. The key principle of CS is that, with slightly more than K well-chosen measurements, a K-sparse signal can be recovered by multiplying it by a random projection matrix Φ_M×N. Here M is significantly smaller than N but larger than K. Substituting Equation (3) into Equation (2) we observe that:

CS addresses the problem of solving for X when the measurements are much smaller than original image signals. This is generally an ill-posed problem, because there are an infinite number of candidate solutions for X. Nevertheless, the CS theory provides a set of conditions that, if X is sparse or compressible in a basis ψ, and Φ in conjunction with ψ satisfies a technical condition called the Restricted Isometry Property (RIP):

Candes and Tao [4,5] show that the signal X can be exactly recovered from few measurements by solving a l₂ – l₁ minimization problem:

Here the regularization parameter λ > 0 helps to overcome the ill-posed problem, and the l₁ penalty term drives small components of θ to zero and helps promote sparse solutions. In fact, the RIP constrained condition of Equation (5) suggests that the energy contained in the projected image Y is close to the energy contained in the original image X.

Compared with conventional camera architectures, the CI camera is specifically designed to exploit the CS framework for imaging. For example, the single pixel camera designed by Rice University differs fundamentally from a conventional camera [6]. A programmed digital micro-mirror device is used to perform linear projections of an image onto a single optical photodiode. In this type of optical architecture, the system cycles sequentially through the rows of the projection matrix Φ to determine the measurement elements one at a time. Any arbitrary pattern of values in the domain [0,1] can be easily used by reprogramming the control software. However, as the measurement elements of y are measured sequentially, dynamic imaging is inherently time consuming. Considering the dynamic scene imaging problem, researchers have proposed some other optical CI systems. Rather than measuring a sequence of a scene image to a single pixel, they make a parallel measurement of the original scene image onto a small set of pixels. For example, the Duke University group describes the design of coded aperture masks for super resolution image reconstruction from a single, low-resolution, noisy observation image [7,8]. This architecture is simple and highly suitable for optical CS imaging because all measurements are collected at one time. More recently, based on their prior work, Harmany et al.[9] proposed a coded aperture keyed exposure sensing paradigm to realize spatio-temporal compressive sensing imaging. However, how to make the random coded aperture practically remains a key problem that needs to be solved. Fergus et al. reported a compact CI camera that uses a random lens [10]. This approach can achieve an ultra-thin optical system design and can be applied to numerous practical applications. However obtaining the sensing matrix from these random lenses is difficult. Shi et al.[11] proposed a compressive optical imaging system based on spherical aberration. Spherical aberration is an optical phenomenon attributed to the intrinsic refraction property of a spherical lens. The larger the curvature of the lens surface, the more serious the aberration will be. The optical structure of this architecture only needs a lens with significant spherical aberration. Although the research on this method is being undertaken, the method by which to design and to manufacture this special lens may be not easy. In [12,13], Neifeld et al. proposed an adaptive feature-specific imaging system for face recognition tasks.

In summary, all the aforementioned compressive sampling strategies satisfy the following features: each element x_i in the source image contributes to all compressed measurements {y₁y₂ … y_m} and each compressed measurement y_i is a linear combination of all source elements {x₁x₂ … x_n}. The coding of a particular pixel y_i is relatively uncorrelated with that of its neighbors.

In surveillance systems, background subtraction is commonly used for segmenting out objects of interest in a scene. However background subtraction techniques may require complicated density estimates for each pixel, which become burdensome in the case of a high-resolution image. In fact, performing background subtraction on compressed images, such as MPEG images, is not novel. In [14], the authors performed background subtraction on a MPEG-compressed video by using the DC-DCT coefficients of image frames. Toreyin et al.[15] similarly used this technique on wavelet representation. However, our technique focuses on CS imaging data, not on compressed video files. Moreover for motion tracking algorithms, Kalman filter, particle filter and mean shift methods are often used for tracking motion targets. However higher data dimensionality may be detrimental to the real time performance of tracking, which will lead to greater computational complexity when performing the density and background model estimations.

Compared with the information that is ultimately of use, researchers have begun to consider whether such a large amount of image data is substantially necessary. New motion target detection and tracking strategies need to be developed. With the emergence of CS theory, researchers have begun to engage in motion detection and tracking algorithms by using CS data. For example, [16] describes a method to directly recover background subtracted images by using the CS theory. A single Gaussian distribution background model is employed and a compressive single-pixel camera is used to obtain the compressive sampling images. However the researchers need to recover the original image to update the background model and a single-pixel camera is used to obtain compressive images, which is time consuming and unsuitable for dynamic scenes imaging. In [17], compressive measurements of a surveillance video sequence are decomposed into a low rank matrix and a sparse matrix. The low rank matrix represents the background model, and the sparse components are utilized to identify the moving objects. The augmented Lagrangian alternating direction method is employed to solve the low rank and the sparse matrix simultaneously. However this algorithm requires a video sequence to identify the moving targets, which cannot be used in real time applications. In [18], authors propose a signal tracking algorithm the use compressive observations. The signal being tracked is assumed to be sparse and with slow changes. Compressive measurements are obtained by projecting the known signal x_i onto a matrix Φ_i, which retains only the columns of Φ with indices that lie in x_i. A Kalman filter in the compressive domain is utilized to estimate signal changes. This algorithm is only suitable for stationary or slowly-moving objects in surveillance scenarios. Wang et al. [19] developed a compressive particle filtering algorithm for moving targets tracking with compressive measurements to avoid image reconstruction procedures. Recently, Mei et al. [20] proposed a robust l₁ tracker. Each motion target is expressed as a sparse representation of multiple pre-established templates. The l₁ tracker demonstrates promising robustness compared with a number of existing trackers. However computational complexity hinders its real time applications.

For motion detection algorithms background images are generally assumed to be temporally stationary, whereas moving objects or foreground objects change over time. Suppose that x_b and x_t are real background and test images in the scene and x_d is a difference image or a foreground image. Given that the foreground image is composed by those pixels which only differ from background images. Therefore the foreground image is always smaller than the background image, and can be considered as a sparse signal in a special transformation domain. Suppose that we obtain compressive measurements y_b of training background images x_b and y_t the compressed measurements of current images, the compressive measurements of the foreground image y_d can be expressed as:

where n_t is an additional Gaussian noise of y_t, n_b and n_d are the noises of y_b and y_d respectively. By solving a l₂ – l₁ minimization problem [4–5]:

The foreground image x_d can be exactly recovered. In Equation (10), ψ can be the wavelet basis which is always used as the sparse basis. Although detecting moving objects in the compressive domain can be easily achieved by using a background subtraction algorithm and recovering the foreground image in the real world space with l₂ – l₁ minimization, reconstructing the foreground image frame by frame is time consuming. Can we detect the moving object directly in the compressive domain without recovering the foreground image? If the answer is positive, it will dramatically reduce the computational cost and energy consumption of surveillance systems.

The Gaussian background model is often used to segment the foreground and background region in conventional motion detection algorithms. Each pixel (x, y) over a time series t = 1,2……T is modeled by a Gaussian distribution I(x, y) ∼ N(u, σ²I). σ²I is the covariance matrix of the Gaussian model, and N is a Gaussian probability density function. According to the Gaussian theorem, if M₁, M₂ are two independent Gaussian random variables, with means μ₁, μ₂ and standard deviations σ₁, σ₂, then their linear combination will also be Gaussian distributed a M 1 + b M 2 ~ N ( a μ 1 + b μ 2 , a 2 σ 1 2 + b 2 σ 2 2 ). Therefore it is reasonable to assume each compressive measurement with a Gaussians distribution N ( y i , σ i 2 I ). Here the mean value is y_i = Φ_ix. When the scene changes to include an object that was not part of the background model, theoretically every compressive pixel value y_i, i = 1,2……m will be against the existing Gaussian distributions. In order to handle image acquisition noise and illumination changes, we use a mixture Gaussian distribution [24,25] to model the background of compressive images and a simple threshold test to declare motion targets.

Using K Gaussian distributions, the probability density function of each compressive measurement at time t can be expressed as:

where w_i,j,t, μ_i,j,t and Σ_i,j,t are the estimates of the weight, mean value, and covariance matrix of the j th Gaussian distribution of the i th pixel at time t in the mixture model respectively. The j th Gaussian probability density function p(y_i,t, μ_i,j,t, Σ_i,j,t) is defined as:

when a compressive measurement belongs to one Gaussian distribution, its weight parameter w_i,j,t will be large and the standard deviation σ_i,j,t will be small, which indicates that the measurement belongs to a distribution with high certainty. In this paper, the background model parameters w_i,j,t, μ_i,j,t and Σ_i,j,t are estimated by using EM algorithm [26].

With static background and lighting, only additional Gaussian noise is incurred in the sampling process, the density of background image could be described by a Gaussian distribution centered at the mean pixel value. However most surveillance videos involve lighting changes, shadows, slow moving objects and objects introduced to or removed from the scene. It is very necessary to update the background model continuously. Otherwise, errors in the background accumulate over time and finally trigger unwanted detections.

To update the background, the background parameter of pixel y_i,t+1 at time instant t + 1 can be estimated by using following equations:

where α is the leaning rate and the parameter ρ = N(y_t+1,μ_j,Σ_j) If the pixel y_i,t+1 matches one of the K distributions and is declared as the foreground, then that matched distribution is updated as defined above. Otherwise, the distribution with the smallest weight is discarded, and initialized to this pixel's value.

As described in [27], at time t the K distributions of the background model are ordered in descending order based on w j , t σ j , t. This ordering supposes that a background pixel corresponds to a high weight with a weak variance due to the fact that the background is more static and the background pixel value is practically constant. The first B Gaussian distributions which exceed a certain threshold T are considered a background distribution:

The other distributions are considered to represent a foreground distribution. At time t + 1, if a pixel matches a Gaussian distribution of any B distribution, this pixel will be identified as “background”, otherwise the pixel is classified as “foreground”. If no match is found with any of the K Gaussians, the pixel is also classified as “foreground”. We declare that there is a new object when the result of Equation (17) is above a threshold.

The l₁ tracker proposed by the authors in [20] is a promising motion target tracking algorithm, which can handle occlusions, corruption, and lighting changes issues. Their algorithm is based on a particle filter framework and each tracking target x^T ∈ ℝ^d is sparsely represented in a feature dictionary A ∈ ℝ^d×(Nt+2d) spanned by target template sets T ∈ ℝ^d×Nt and noises templates sets [I −I] as:

They use particle filter to estimate the posterior distribution p ( s t | x t T ). The state variable s_t is modeled by affine transformation parameters of a target object at time t, and the observation x_t is the corresponding object cropped from images by using s_t as parameters. Let S = {s¹, s², …, sⁿ} be the n state candidates and X^T = {x^T1, x^T2, …x^Tn} be the corresponding target candidates at time t. The target candidate is estimated by finding the smallest projection errors:

An l₁ optimization algorithm is used to solve the sparse coefficient c as follows:

A template update scheme is subsequently employed to reduce the drift. The main problem of the l₁ tracker is the extremely high dimensionality of its feature dictionary space, which leads to a heavy computation burden. Inspired by their outstanding work, we aim to accelerate their tracking algorithm and discuss its application in CI systems. According to Equation (18), in the context of CS the corresponding compressive measurements y^T of x^T can be represented by:

where Φ′ ∈ ℝ^m×d is a projection matrix. Obviously, the sparse coefficient c in Equation (21) can also be recovered with high probability by using TV optimization algorithm [28], OMP algorithm [29], gradient projection algorithms [30], LARS algorithm [31], and other l₁ – l₂ algorithms:

The feature dictionary A in Equation (18) is substituted by a sparse projection dictionary D = Φ′ A, which can be considered as a compressive measurement of original feature dictionary A. As [20] does, the sparse feature dictionary D should also be updated to avoid drift. Clearly, the dimension of dictionary D ∈ ℝ^m×(Nt+2d) (m ≪ d) is reduced by using the random projection matrix Φ′. This will significantly speeds up the process of solving Equation (22).

After observing Equation (21), we have a intuitive idea, whether the compressive measurements y^T can be found in a CI system. Suppose that the motion target x^T has been detected through our motion detection algorithm and then reconstructed and labeled (see Figure 2), then we can utilize a projection matrix Φ_T to obtain compressive measurements image y^T. Here Φ_T is a projection matrix by only keeping those columns of Φ whose indices lie in x^T. For our CI system, the projection matrix Φ can be accurately identified by an optical calibration method. Therefore, given the location index of motion targets, the projection matrix Φ_T can be acquired. However, with the movement of target x^T, the projection matrix Φ_T changes as well. In order to simplify our tracking algorithm, the projection matrix Φ′ used in Equation (21) is fixed. The compressive dictionary D can be constructed with these compressive target templates. Figure 3 illustrates our motion detection and tracking framework that uses CS sampling images.

Romberg has proven that the random Toeplitz or Gaussian matrix is incoherent with any orthonormal basis ψ with high probability [32]. In [33], a random binary matrix is also proven to be suitable for a projection matrix. Therefore in our experiments, random Gaussian, Toeplitz and binary matrixes are all utilized for phase coded masks. The CAVIAR database provided by INRIA Labs at Grenoble [34] is utilized as original image sequences. In an outdoor sequence, each frame has a size of 288 × 384 with dynamic range [0,255] and motion objects have been generated manually. Figure 4 shows three different phase coded masks we used in our simulation experiments. The corresponding compressive image using random Gaussian phase mask via Matlab simulation is shown in Figure 5.

A total variation (TV) optimization algorithm is used to reconstruct the original image from compressive measurements [28]. The reconstruction is performed using several measurement rates ranging from 50% to 5% and with random Gaussian, Toeplitz and binary phase coded masks, respectively. In our experiments, the signal-to-noise ratio (SNR) is applied to evaluate reconstruction performance. Figure 6 shows the reconstruction results with a random Gaussian phase mask.

From Figure 6(a), we can see that the measurement rate can reduce to 20% without sacrificing performance. While a further decreasing measurement rate, the performance is gradually reduced. With rates as low as 5%, the background and test images are not recovered accurately. Figure 6(b) shows the reconstruction results of foreground y_d. We can clearly find in Figure 6(b) that the sparser foreground can be recovered correctly from y_d with rates as low as 5%. These simulation results can be explained by the following assumptions: when the sizes of moving objects are smaller than the original image sizes, we can assume that the sparsity of the motion image K_d is smaller than K_b and K_t. According to the CS theory, the number of compressive measurements necessary to reconstruct original image can be given by Klog(N/k). Therefore, if K_d < K_b ≈ K_t, the number of compressive measurements will be smaller than the background and test images.

Table 1 compares the reconstruction results by using different phase coded masks. Here, the sampling rate decreased from 100% to 5%, the same TVAL recovery algorithm is utilized to reconstruct the original image, and the SNR is taken as the average of 10 tests. According to Table 1, the reconstruction algorithm that employs random Gaussian and Toeplitz masks achieves superior recoverey performances than a random binary mask.

As presented earlier, we utilize a mixture Gaussian distribution to model the background. The foreground detection algorithm described in Section 4.3 is used to declare motion objects in compressive sampling space. The motion detection algorithms that use random binary, Gaussian, and Toeplitz phase masks are denoted by RB, RG, and RT respectively in this paper. Figure 7 shows the energy curves computed by using Equation (17) for three different phase mask systems with sampling rates of 10%, 50% and 70% in a 64 × 64 CI block (which included a motion target). Comparing random Gaussian, Toeplitz and binary projections, the energy value collected of compressive measurements is ordered as E_binary > E_gaussian >E_toeplitz. With the decrease of the sampling rate, the energy values computed by using different phase coded masks all reduced gradually. The CS image is declared to include motion targets by using following equation:

where thershold = log(E_bu + Cσ), E_y is the energy computed by using Equation (17), and E_bμ is the mean energy of the background CS image. σ is the standard variance of E_μ and C is a constant.

We employ the Area Under Curve (AUC) metrics to evaluate the performance of our motion detection algorithm. Table 2 shows that the AUC values are affected by the constant C. The motion detection performance is the best with constant C = 8. Meanwhile the motion detection performance of RB is slightly better than that of RG and RT. The reconstruction performance of RG and RT is better than RB (see Table 1). This observation can be explained by the CS theory. In [32], researchers have proven that random Gaussian and random Toeplitz is incoherent with almost all sparse basis Ψ and thus can recover compressive signals with high possibility. While the binary matrix we used in our experiments are 0–1 matrices, which has been shown that 0,1-matrices require more than O (k log (n/k)) rows to satisfy the RIP [35]. Therefore when the sparsity of the original image is fixed, we need more compressive measurements to recover original signals by using a random binary mask.