The two main problems in motion analysis in image sequences are the correspondence and the aperture problem. The correspondence problem, well exposed by Duda and Hart [34], is related to the relation velocity-sampling rate, and defines two broad research lines. The first one consists in studying two consecutive images in a static manner and then analyzing how some significant pixels have moved between both frames. The second line consists in locally studying each pixel and its neighborhood along time. The aperture problem, also broadly treated [35–40] is related to the task of associating the apparent movement in the environment of a concrete pixel with the real movement of the element to which this pixel belongs. The complexity of the problem increases in three-dimensional scenes [41].

Models based on local motion detection face the correspondence problem considering that a pixel in time t + Δt is close to the same pixel in time instant t. These models are usually based on gradient analysis or local correlation. Some gradient analysis models calculate the velocity using the spatial-temporal derivative of the brightness in a pixel and its immediate environment. Among this type of models we can highlight the direction selectivity model of Marr and Ullman [37], which obtains the direction of motion but not the velocity. Lawton's motion direction prediction model [42] calculates the direction of the velocity from the gradient. Fennema and Thompson [35] calculate the velocity using the gradient, but they impose restrictions on velocity and gray level. The most extended model of this family is the optical flow, proposed by Horn and Schunck [36], which calculates the apparent velocity of each pixel using the spatial and temporal gradient of the brightness in each pixel. This model imposes the uniformity constraint, and the non-existence of spatial discontinuities in the shapes.

Correlation based models [43, 44] are usually based on correlating the brightness of a pixel and its closer neighbors along time. Some of them are the relational selectivity model of Reichardt and Hassenstein [43] or the direction selectivity model of Barlow and Levick [44], which calculate the direction of velocity by comparing the input value with the previous one and with the neighbors. Another group of this type of models is based on spatio-temporal energy [45, 46]. In Heeger's model [46] image sequences are represented as a three-dimensional space, two spatial and one temporal, which calculates the velocity by means of three-dimensional filters. The model of human vision of Watson and Ahumada [47] is correlational but uses biologically inspired tools.

There are also models based on the uniformity restriction. These impose that the moving objects velocity fields vary uniformly, since objects usually have uniform surfaces. They analyze local velocity fields to obtain information about the real velocity of the objects. Some examples are the visual motion measurement model of Hildreth [39], the neural networks primary vision model of Koch, Marroquin and Yuille [48], and, the model of computational theory for the perception of the coherent visual motion of Yuille and Grzywacz [49].

The method proposed, based on the effect called permanency [50], is performed on those sensor pixels where motion is detected during a time interval [t − Δt, t], where Δt is the maximum time between the total discharge and the saturation associated to each pixel of the input image sensor. The concept of permanency, associated to pixel (i, j) is related to the time elapsed with no variation in the image input signal I(i, j; t) on this pixel. The variable associated to the permanency concept is defined as the accumulative computation charge. This is the main difference of our method compared to other motion analysis methods related to the optical flow In other approaches, the analysis is only performed on image pixels where motion has taken place in the present time t.

The AC approach is neurally inspired. Usually the time evolution of the neuron membrane potential is modeled by a first order differential equation known as the “leaky integrator model”. A different way of modeling time evolution of membrane potential is to consider the membrane as a local working memory in which neither the triggering conditions nor the way in which the potential tries to return to its input-free equilibrium value, needs to be restricted to thresholds and exponential increases and decays. This type of working memory is characterized by the possibility of controlling its charge and discharge dynamics in terms of:

The presence of specific spatio-temporal features with values over a certain threshold.

The upper part of Figure 1 shows the AC model's block diagram. The lower part of Figure 1 illustrates the temporal evolution of the state of the charge in an AC working memory in front of a particular one-dimensional stimuli sequence. From [9, 10] we reformulate the equations of the AC method as formulated for the motion detection task. Firstly, Equation (1) covers the need to segment each input image I into a preset group of gray level bands (N).

This formula assigns pixel (i, j) to gray level band k. Then, the accumulated charge value related to motion detection at each input image pixel is obtained, as shown in formula (2): (2) Q k ( i , j ; t ) = { min if I k ( i , j ; t ) = 0 max if ( I k ( i , j ; t ) = 1 ) AND ( I k ( i , j ; t − Δ t ) = 0 ) max [ Q k ( i , j ; t − Δ t ) − δ Q , min ] if ( I k ( i , j ; t ) = 1 ) AND ( I k ( i , j ; t − Δ t ) = 1 )

The charge value at pixel (i, j) is discharged down to min when no motion information is available, is saturated to max when motion is detected at t, and, is decremented by a value δQ when motion goes on being detected in consecutive intervals t and t − Δt.

Let us suppose, without loss of generality, that it is enough to distinguish eight levels of accumulated charge (N = 8) and, consequently, that we can use as a model of the control underlying the inferential scheme that describes the data flow corresponding to the calculation of this subtask an 8 states automaton (S₀, S₁, …, S₇), where S₀ corresponds to min and S₇ to max. Let us also suppose that discharge (δQ = 1) takes the values corresponding to the descent of one state.

Now, the aim is to detect the temporal and local (pixel to pixel) contrasts of pairs of consecutive binarised images at gray level k. The subtask firstly gets as input data the values of the 256 gray level input pixels and generates N = 8 binary images, I_k(i, j; t). The output space has a FIFO memory structure with two levels, one for the current value and another one for the previous instant value. Thus, for N bands, there are 2N = 16 binary values for each input pixel; at each band there is the current value I_k(i, j; t) and the previous value I_k(i, j; t − Δt), such that Equation (1) turns into: (3) I k ( i , j ; t ) = { 1 , if I ( i , j ; t ) ∈ [ 32 ⋅ k , 32 ⋅ ( k + 1 ) − 1 ] 0 , otherwisewhere k = 0, 1, …, 7, is the band index. Thus, we are in front of a vector quantization (scalar quantization) algorithm generally called multilevel thresholding. As well as segmentation in two gray level bands is a usual thing, here we are in front of a refinement to the segmentation in N = 8 gray level bands. Thus, multilevel thresholding is a process that segments a gray-level image into several distinct regions. Figure 2 shows the state transition diagram corresponding to the different inputs and outputs. The following situations can be observed:

I_k(i, j; t − Δt) = {0, 1}, I_k(i, j; t) = 0

In this case the calculation element (i, j) is not able to detect any contrast with respect to the input of a moving object in that band (I_k(i, j; t) = 0). It may have detected it (or not) in the previous interval (I_k(i, j; t − Δt) = 1, I_k(i, j; t) = 0). In any case, the element passes to state S₀, the state of complete discharge, independently of which was the initial state.

The presented scheme suffers from low performance when a pixel is in the border of two bands. In this situation, a pixel with a mean value in the border of two bands and some noise that makes the pixel change from one band to another close band, activates the stimuli sequence and, consequently, motion is detected when there is no real motion in the scene.

However the scheme can be slightly modified to overcome this problem. Indeed, the previous scheme can be modified to take into account a hysteresis cycle defined through I k + t h and I k − t h.

In this case the accumulated charge for band k is now rewritten as: (6) Q k ( i , j ; t ) = { min if ( I k ( i , j ; t ) = 0 ) AND ( I k ( i , j ; t − Δ t ) = 0 ) max if ( I k ( i , j ; t ) = 1 ) AND ( I k ( i , j ; t − Δ t ) = 0 ) AND ( I k + t h ( i , j ; t − Δ t ) = 0 ) AND ( I k − t h ( i , j ; t − Δ t ) = 0 ) max [ Q k ( i , j ; t − Δ t ) − δ Q , min ] if ( I k ( i , j ; t ) = 1 ) AND ( I k ( i , j ; t − Δ t ) = 0 ) AND ( I k + t h ( i , j ; t − Δ t ) = 1 ) OR ( I k − t h ( i , j ; t − Δ t ) = 1 ) max if ( I k ( i , j , t ) = 0 ) AND ( I k ( i , j , t − Δ t ) = 1 ) AND ( I k + t h ( i , j ; t ) = 0 ) AND ( I k − t h ( i , j ; t ) = 0 ) max [ Q k ( i , j ; t − Δ t ) − δ Q , min ] if ( I k ( i , j ; t ) = 0 ) AND ( I k ( i , j ; t − Δ t ) = 1 ) AND ( I k + t h ( i , j , t ) = 1 ) OR ( I k − t h ( i , j ; t ) = 1 ) max [ Q k ( i , j ; t − Δ t ) − δ Q , min ] if ( I k ( i , j ; t ) = 1 ) AND ( I k ( i , j ; t − Δ t ) = 1 )where the parameter th selects the hysteresis cycle and allows variations in the interval [v + th, v − th] not to be considered as motion. th must be selected according to the noise of the images.

We have used an infrared surveillance sequence captured by our research team, where different persons appear and disappear in the scene. Figure 6 shows the result of the AC detection modules dedicated to one of the eight infrared grey level bands.

Notice that motion not detected in one band is detected in another one. Notice that the background motion is mainly obtained at bands 2 and 3, whereas the foreground is obtained at bands 4 to 7. Bands 1 and 7 do not offer much information, neither on foreground nor on background motion. A deeper insight into the figure show some interesting results. A gross conclusion is that band 4 mostly gets the contours of the foreground moving elements (people, in this case), whereas bands 5 and 6 show the main parts of the moving bodies. This is why, in this particular case, it seems reasonable to sum up bands 5 and 6 to obtain moving people in infrared imagery. Now, Figure 7 shows the efficiency of the combination of the AC modules corresponding to bands 5 and 6 for segmenting moving people in the sequence.

In this case study, we have used a data set containing 1109 frames captured in an office room. Figure 8a offers one input image number of the sequence. Here, for the purpose of testing the proposal applied to color images, we are interested in tracking a range of colors in the RGB (reg-green-blue) color model. This range has to cover in this case a red t-shirt dressed by a young woman. This could be a typical example of tracking suspicious people in the visual surveillance domain.