Single-Pixel Imaging with Origami Pattern Construction

Single-pixel compressive imaging can recover images from fewer measurements, offering many benefits especially for the imaging modalities where array detection is unavailable. However, the widely used random projections fail to explore internal relations between coding patterns and image reconstruction. Here, we propose a single-pixel imaging method based on a deterministic origami pattern construction that can lead to a more accurate pattern ordering sequence and better imaging quality. It can decrease the sampling ratio, closer to the upper bounds. The experimental realization of this approach is a big step forward towards practical applications.

Imaging is extremely important for acquiring the light field information of the target [1].Single-pixel imaging [2][3][4][5], as one of popular imaging technologies, can obtain images by using only a point detector without spatial resolution, thus it has extraordinary application prospects, especially at non-visible wavelengths.Except for point-by-point scanning, single-pixel imaging has also developed some imaging techniques based on Fourier [6]or Hadamard transforms [7][8][9] (i.e., complete orthogonal basis modulation, full sampling) and correlated/ghost imaging (GI) [10][11][12][13][14] (i.e., random modulation, oversampling).In recent ten years, with the rapid development of information theory, single-pixel imaging [15] based on compressed sensing [16,17] has come into being.This approach obtains images of sparse/compressive targets at very low sampling rates, breaking the Nyquist-Shannon sampling limit and bringing dawn of subsampling.However, compressive imaging sacrifices the measurement time and the computational complexity for acquiring spatial resolution, and generally uses random modulation.As a consequence, its sampling rate is still higher than 30% by a large probability, which has become the bottleneck of compressive imaging for realizing real-time practical applications, especially in large pixel-scale imaging.Recent studies have demonstrated that not all measurements contribute to the reconstructions of computational imaging, resulting in a technology named cor-respondence imaging (CI) [18][19][20][21][22], where only the measurements well above or below the mean of detected values are used for reconstruction.Now that there exists such a phenomenon, we are thinking, why not just construct a universal deterministic (instead of random) measurement matrix where the modulated patterns with the most contributions to the reconstruction are preferentially displayed.In particular, an earlier research work [23] reported a ghost imaging method based on "Russian Dolls" (RD) ordering of the Hadamard basis, which provided us with inspiration.
In this Letter, we propose an origami model to construct deterministic patterns, decreasing the sampling ratio to 0.5%, which is ∼ 2 orders of magnitude lower than that of traditional compressive imaging.The row vectors stretched from each constructed pattern are orthogonal to each other, suitable for largepixel-scale reconstruction.Therefore, it opens a door to the super subsampling applications for compressive video [24,25].Now let us begin with the definition of differential ghost imaging (DGI) [26], one method that can greatly improve the quality of conventional GI.The bucket/single-pixel signal can be defined as S S R S R I R (x, y) .A high image quality can be guaranteed by DGI, but at the cost of oversampling.
Later, another method called CI [18][19][20][21][22] was reported, in which a positive (or negative) ghost image can be retrieved by only averaging some small fractions of the reference subset I + R (or It can be seen that CI just need to calculate addition instead of second-order correlation, so CI takes less computation time than DGI.Although only a small number of patterns are involved in the calculation, the number of measurements is not decreased, and the conditional averaging of the reference patterns is made after oversampling.But it still can be concluded that not all the measured bucket/single-pixel intensities are necessary for reconstruction.Since single-pixel compressive imaging shares the same mathematical model with ghost imaging, this conclusion can be also applicable to compressive imaging.Obviously, the random patterns are not the best choices to minimize the number of measurements, because such a pattern has a high probability of transmitting or reflecting light at half of all pixels.It is really difficult for us to determine which pattern has a higher reconstruction contribution or produces a higher bucket intensity.However, we find that, if we encode on the spatial light modulator (SLM) with a pattern consisting of all-one, which can be treated as a connected domain, the detected intensity value will be the highest.If the pattern is divided equally into two connected domains with the white-black pixels 50%-50%, then the detected value will be the second highest with a high probability.On this basis, the problem can be partially deduced to the contributions of the piece number of connected domains (CDs) of patterns to the image quality.Here we propose a novel origami pattern construction method, which makes full use of symmetric reverse folding (reverse the values on the corresponding pixels at the symmetric positions) and the axial symmetry of the rescaled pattern.The steps of our origami pattern construction are as follows.
Step 1: Assume that the pixel-sizes of patterns are all p × q = n square matrices, with the same size as the object, where p, q (p = q) are all even numbers.Suppose there are n patterns in total, we divide them into n/4 groups, each of which with four patterns.The group number is denoted by i = 1, 2, 3, • • • , n/4.Note, as an initial setting, the first pattern P 1,1 in the first group is a matrix with all ones.Then the second and the third patterns in this group P 1,2 , P 1,3 are obtained by inversely folding the first one P 1,1 in this group up and down, left and right, respectively.That is, remain the upper (or the left) half unchanged while the lower (or right) half is axisymmetric with the upper (or left) half but with its values in the lower (or right) half taking the opposite.By this means, we realize black (-1) and white (+1) inverting.The fourth pattern P 1,4 in this group is formed by performing both up-down and left-right reverse folding operations (the sequence of operations is interchangeable) on the first pattern P 1,1 .Similarly, for all the subsequent groups, the rest three patterns in the ith group, P i,2 , P i,3 and P i,4 will be generated from P i,1 following the same processes.
Step 2: The first pattern P i,1 in the ith group (also the 4(i − 1) + 1th pattern in the complete sequence) is built on the basis of the ith pattern in the current complete sequence.Take P 2,1 in the second group for example, we firstly scale both horizontal and vertical pixel dimensions of the second pattern in the complete sequence (i.e., P 1,2 ) to their 1  2 size, and place the scaled one on the upper left 1/4 part of a p × q square matrix with all zeros.Secondly, we perform the up and down, left and right axial symmetry about the midline lines on the vertical and horizontal axes of the matrix.After that, the pattern P 2,1 (also the 5th pattern in complete sequence) is generated.Then, repeat Step 1 to acquire the second to fourth patterns in the second group, P 2,2 , P 2,3 and P 2,4 (also the 6th to 8th patterns in complete sequence).By analogy, we can create all n/4 groups.Fig. 1(a) gives a good illustration of Steps 1 and 2, and Fig. 1(b) shows the result obtained after Step 2.
Step 3: Adjust the pattern sequence to ensure that the number of CDs for each pattern in the ith group is in an incremental order.Here a CD is defined as an up-down-left-right connected area consisting of equal values (see the upper right corner of Fig. 2).The neighborhoods on both sides of the axis may cause the partition of CDs, which is worth of investigating.Let's first look at the pattern P i,1 whose pixel values on both sides of the symmetry axis are the same, its contribution to the recovery is the largest in the ith group.Since the pattern P i,4 needs double symmetric reverse folding, its number of CDs is largest (with lowest contribution) in the ith group.The patterns P i,2 and P i,3 are produced by only once symmetric reverse folding, whether to first perform up-down or left-right operation is approximately equivalent, thus they are particularly worth considering.As shown in Fig. 2, the patterns P i,2 and P i,3 with a group indication (ID) (note: not the absolute subscript) i = 3, 9 ∼ 12, 15 need to be order reversed for the case of n = 64, while those with a group ID i = 3, 9 ∼ 12, 15, 33 ∼ 48, 51, 57 ∼ 60, 63 require adjustments for the case of n = 256.This rule also exists in higher-order cases.For this, we set fourfold as a level.From top to bottom, the total n/4 groups (as a whole) can be catalogued into four parts: only the second part stays still, the groups in the third part all need to be adjusted, while the groups in the first and fourth parts also need to be adjusted but with the same group IDs, depending on the recursive layer.Then repeat the operation for the first part (as a new whole) until the number of groups in each new quarter part equals to 1.In the last layer, only the third part needs to be adjusted.Then, switch the order of the patterns P i,2 and P i,3 in these found groups.If we perform left-right and up-down reverse folding to generate P i,2 and P i,3 in Step 1, respectively, then the ID positions should be adjusted accordingly in Step 3.After the above three steps, we will get the final pattern sequence.Fig. 3 gives an example of the final pattern sequence for n = 64.In order to deeply analyze the advantages of our method, we compare it with the RD method.It is interesting to find that these two approaches have similar numbers of CDs in low-order part (especially for the first 16 patterns, they are the same).Since their subsequent results are both derived from these first 16 patterns, it is advisable to set each 16 patterns as a comparison unit.Fig. 4 shows the differences in their high-order parts (the last 64 patterns in 256-order sequence which also comprises the largest division of the RD method).As shown in Fig. 4(a), the RD ordering becomes very rough in this part, and has a lot of pattern pairs with the same number of CDs in each comparison unit.Because the maximum partition length of RD is n/4 while its minimum segmentation length is 4, which will inevitably incur too much uncertainty.Although the internal order exchange in these pattern pairs does not affect the RD sorting too much, it does bring hundreds of millions possibilities of finding the optimal pattern sequence to get the best imaging quality.While in our scheme, each four patterns are treated as a group, i.e., the partition length is always 4. As shown in Fig. 4(b), there are only four red marks in our method for the highorder part, with only 2 4 uncertainty.It makes the reconstructed images more accurate.Thus our method can be regarded as a better strategy.Now, each pattern I R can be sequentially reshaped into a row vector of 1×n, and then n such row vectors constitute a full-rank square matrix, which happens to be an orthogonal matrix.It is quite suitable for forming the measurement matrix A ∈ R m×n of CS by just selecting the first m row vectors.Here we use a TVAL3 solver [27] to recover the image.For a fair comparison, we combine the RD ordering of the Hadamard basis with CS, instead of second-order correlation used in the original version [23].Here we introduce an unitless performance quantitative measure, the root mean square error (RMSE), which is defined as RMSE = It describes the square root of differences between the recovered image Ũ and the original image U o for all pixels.Generally, the smaller the RMSE value, the better the quality of the image recovered.Since it is a pixel-by-pixel calculation, it cannot provide a very accurate characterization of performance.Therefore, in addition to the RMSE curves of CS, Russian dolls compressed sensing (RDCS) and origami pattern compressed sensing (OPCS), we also present their recovered results under different sampling ratios from 0.5%-8.5% with a 2% stepping increase, as shown in Fig. 5.It is obvious that the performance of our method outperforms   those of CS and RDCS.Only 2.5% measurements are enough for OPCS to generate a nice reconstruction, and its limit sampling rate can reach 0.5%.
Our experimental realization is based on a conventional single-pixel camera, as shown in Fig. 6.The thermal light from a stabilized halogen tungsten lamp with a wavelength of 360∼2600 nm is collimated and attenuated via a beam expander and a neutral density filter (NDF), respectively.Here the NDF is used to attenuate the light to the ultra-weak light level.After that, the light passes through a negative 1951 USAF resolution test chart, and then it is imaged onto a digital micromirror device (DMD) via an imaging lens and a mirror.The applied DMD consists of 1024×768 micromirrors, each of which can be switched between two directions of ±12 • , corresponding to 1 and 0. Since the values of our patterns are either 1 or −1, we need to divide each generated pattern I R into a complementary matrix pair [28-30] ÎR = (I R +1)/2 and ǏR = 1 N ×N − ÎR , where I R = ÎR − ǏR , and 1 N ×N stands for an all 1 matrix.Then, these complementary matrices are sequentially encoded on the DMD.A counter-type Hamamatsu H10682-210 PMT is placed on one of the reflection orientations to accordingly make a differential measurement of two adjacent total photon counts.By using OPCS algorithm, we can acquire a very satisfactory image quality, as shown in the illustration of Fig. 6.
In conclusion, a single-pixel compressive imaging method with origami pattern construction is proposed and demonstrated experimentally with differential modulation.In our scheme, following the fluctuation characteristic of measurement values, the patterns with the most contributions to the recovery should be preferentially displayed.By symmetric reverse folding, axial symmetry and partial pattern order adjustment, the generated deterministic patterns ensure a better image quality but with less uncertainty of the pattern sequence, compared with traditional CS and RDCS.The sampling ratio can be reduced to 0.5%, even about 2 orders of magnitude lower than that of traditional CS, actually realizing super sub-Nyquist sampling.Therefore, our scheme can be used to greatly improve real-time performance of single-pixel compressive video applications, especially where array detectors are unavailable.
B = I B (x o , y o )T (x o , y o )dx o dy o , where I B (x o , y o ) and T (x o , y o ) represents the intensity and transmission function at the spatial position (x o , y o ) of the object x of p × q = N pixels, respectively.Similarly, the reference bucket signal can also be defined as S R = I R (x, y)dxdy, where I R (x, y) stands for the values at the spatial position (x, y) of the reference speckle patterns.By using a differential bucket signal S ∆ = S B − S B S R S R instead of S B in second-order correlation S B I R (x, y) , where • • • signifies the ensemble average, we can compute a differential ghost image U DGI from M N measurements via U DGI = S ∆ I R (x, y) = S B I R (x, y) − S B

Figure 2 :
Figure 2: Group numbers for cases of n = 64, 256.The patterns P i2 and P i3 with red marked group numbers need to be order exchanged.The subgraph in the upper right corner illustrates an example of the connected domains with a number of 5.

Figure 3 :
Figure 3: Numbers of CDs for a final pattern sequence of n = 64.

Figure 4 :
Figure 4: Comparison of the number of CDs between Russian dolls (RD) ordering of the Hadamard basis (a) and our method (b) in high-order part.The patterns with the same CDs are circled by red ellipses.

Figure 6 :
Figure 6: Schematic of the experimental setup.After being collimated and broadened, the light illuminates the object (a negative 1951 USAF resolution test chart of 3 inch×3 inch) and then it is imaged on the DMD.A PMT collects the differential photon counts via a focusing lens.The reconstructed image of 128 × 128 pixel-size is given in the lower right corner.