In this sub-section, we describe the HW/SW co-design methodology implemented in this study. The HW/SW co-design is a hybrid method aimed at increasing the flexibility of the implementation and improvement of the overall design process [16–20]. When a co-processor-based solution is employed in the HW/SW co-design architecture, the computational time can be drastically reduced. Two opposite alternatives can be considered when exploring the HW/SW co-design of a complex SP system. One of them is the use of standard components whose functionality can be defined by means of programming. The other one is the implementation of this functionality via a microelectronic circuit specifically tailored for that application. It is well known that the first alternative (the software alternative) provides solutions that present a great flexibility in spite of high area requirements and long execution times, while the second one (the hardware alternative) optimizes the size aspects and the operation speed but limits the flexibility of the solution. Halfway between both, hardware/software co-design techniques try to obtain an appropriate trade-off between the advantages and drawbacks of these two approaches.

The HW/SW co-design methodology encompasses the following general stages:

Algorithmic implementation (reference simulation in MATLAB and C++ platforms);

From the analysis of the HW/SW co-design methodology, one can deduce that the RS algorithm is first adapted in a co-design scheme applying HPEC techniques, and then, the selected computationally complex reconstructive operations are efficiently implemented in bit-level high-throughput accelerator architectures.

The algorithm of the selected RS-reconstructive operations, i.e., the b̂ = (FU) ⊙ (FU)⁺ can be represented by nested loops or FOR-loops programs. First, let us define from the reconstructive RS estimator of Equation (1), the n × m matrix F and the vector u of dimension m as follows: (5) y = Fuwhere y is an n-dimensional (n-D) output vector. The j-th element of y is computed as: (6) y j = ∑ i = 1 n F ji u i ,         j = 1 , ... , mwhere F_ji represents the corresponding element of F.

Next, the localization method converts the algorithm into an algorithmic representation with local and regular dependencies [27,28]. The following algorithm achieves locality via affine scheduling transformations as presented below: (7) input operations F [ i ,   j ] ← F i , j − 1 ∀ ( i ,   j )     |     1 ≤ i ≤ n ;     1 ≤ j ≤ m . u [ 0 ,   j ] ← u j ∀     j     |     1 ≤ j ≤ m . y [ i , 0 ] ← 0 ∀     i     |     1 ≤ i ≤ n .   computations   for ( i = 0 ; i < n ; i + + ) {     y [ i , 0 ] = 0 ;     for ( j = 0 ; j < m ; j + + ) {       u [ i ,   j ] = u [ i − 1 ,   j ] ;       y [ i ,   j ] = y [ i ,   j − 1 ] + F [ i ,   j ] ⋅ u [ i ,   j ] ;       }   } output operations y [ i ] ← y [ i , m ] ∀ ( i ,   j )     |     1 ≤ i ≤ n ;       1 ≤ j ≤ m . where the index space is I = { ( i ,   j ) T ∈ ℤ 2 | 1 ≤ i ≤ n ;     1 ≤ j ≤ m }

Once the algorithm is transformed into their localized representation (i.e., locally recursive form), one is ready to proceed with the tiling procedure in order to achieve realistic large-scale RS structures with the fixed-sized SA architecture.

The tiling technique is a well known loop transformation used to automatically create sub-block algorithms [26,29,30]. The advantage of this method is that, while computing within a block, there is a high degree of data locality, allowing better performance. The tiling procedure consist of dividing the large-scale index space defined by the loop structures into regular tiles (or blocks) of some real-scale size and shape RS complex operations, and then traversing the tiles to cover the whole index space [30]. The conventional tiling procedure combine two well-known transformations: loop permutation and strip-mining. The loop permutation is used to establish the order in which the iterations inside the tiles are traversed and the strip-mining transformation is used to partition one dimension of the index space into strips. It also decomposes a single loop into two nested loops; the outer loop steps between strips of consecutive indexes, and the inner loop traverses the indexes within a strip. Both transformations can be obtained using the theory of unimodular transformations and, to compute the exact bounds, the Fourier-Motzkin elimination algorithm [30] is applied.

Now, considering the locally recursive representation presented in Equation (7), the strip-mining transformation is applied to the outermost loop in order to perform the one-dimensional partition of the i-index algorithm. The resulting index partition is represented as follows: (8) for ( i = 0 ; i < n ; i + + ) for ( tile_i = 0 ; tile_i < n ; tile_i = tile_i + StSizei ) for ( j = 0 ; j < m ; j + + ) → strip − mining for ( i = tile_i ; i ≤ min [ tile_i + StSizei − 1 , n − 1   ] ; i + + ) Loop body for ( j = 0 ; j < m ; j + + ) Loop body     [ i ,   j ]where tile_i represents the i-index-tile loop, i and j are the inner element’s loops and StSize is the strip size.

The second step of the tiling procedure consists in implement the loop permutation transformation based on the Polytope model [30]. For the loop permutation, the following unimodular transformation T P = [ 0 1 1 0 ] was applied in order to permute the index-space of the locally recursive algorithm I = [i j]^T into the required new index-space I_P = [i’ j’]^T=[j i]^T. In this step, it is defined the Polytope model as a set of inequations such as ΓI_P ≤ H, where I_P = [i’ j’]^T represents the index-space after the i-strip-mining procedure, and the matrix Γ and vector H represents the boundaries of each FOR-loop of the algorithm presented above in Equation (8).

The source Polytope is described in a convex form by a set of half-spaces, where the intersection of all half-spaces corresponds to the Polytope and the target representation is presented as follows: (9) Γ   I ≤ H T P   I = I P }     ⇒   Γ   T P − 1   I P ≤ H Γ I P ≤ H   ⇒   [ − 1 0 1 0 1 0 0 − 1 0 1 ] ︸ Γ   [ 0 1 1 0 ] ︸ T P − 1   [ i ′ j ′ ] ︸ I P   ≤   [ − tile_i tile_i + StSize − 1 n − 1 0 m − 1 ] ︸ H .

The new loop bounds are derived with the Fourier-Motzkin elimination algorithm. The resulting reconstructive RS algorithm after the loop permutation transformation is now presented, in which the proper substitutions are integrated: (10) for ( tile_i = 0 ; tile_i < n ; tile_i = tile_i + StSizei ) { for ( j = 0 ; j < m ; j + + ) { for ( i = tile_i ; i ≤ min [ tile_i + StSizei − 1 , n − 1   ] ; i + + ) { Loop body     [ j ,   i   ] } }

The third step of the tiling procedure corresponds again to the strip-mining transformation procedure but in this case, the procedure is applied over the j-index. Furthermore, the resulting tiled algorithm after this strip-mining transformation is next represented. The final step consists in to employ again the loop permutation. This final transformation is required to order the inner loop for the final tiled algorithm represented as follows: (11) for ( tile_i = 0 ; tile_i < n ; tile_i = tile_i + StSizei ) { for ( tile_j = 0 ; tile_j < m ; tile_j = tile_j + StSizej ) { for ( i = tile_j ; i ≤ min [ tile_j + StSizej − 1 , m − 1 ] ; i + + ) { for ( j = tile_i ; j ≤ min [ tile_i + StSizei − 1 , n − 1 ] ; j + + ) {     Loop body     [ i ,   j ]     } }

From the analysis of the tiled parallel algorithm of Equation (11), one is ready to deduce the dual SSA core and the local control system architecture presented in Figure 2.

The space-time mapping procedure onto SAs is a technique that transforms an index-space representation into a space-time representation where each node of their iteration node is mapped to a certain PE and it is scheduled to a certain instance of time [27,28]. Recall that the SA is a space-time representation of the computational operations, in which the function description defines the behavior within a node, whereas the structural description specifies the interconnections (edges and delays) between the corresponding graph nodes [27]. In order to derive a SA architecture with a minimum possible number of nodes, we address a linear projection approach for processor assignment, i.e., the nodes of the structure array in a certain straight line are to be properly projected onto the corresponding PEs of the SA represented by the corresponding assignment projection vector d. Thus, we seek for a linear order reduction, in which the transformation T_m : G^N → Ĝ^N−1 maps the N-dimensional dependence graph (G^N) onto the (N−1)-dimensional SA (Ĝ^N−1), where N represents the dimension of their dependence graph (see proofs in [19,27] and details in [30]). Moldovan in [31], proved the mapping theory, as follows: (12) T m = [ Π Σ ] ,where Π is a (1 × N) − D vector (composed of the first row of T_m) which (in the partitioning terms [19,29]) determines the time scheduling, and the (N − 1) × N sub-matrix Σ in Equation (29) is composed of the rest rows of T_m that determine the space processor specified by the so-called projection vector d [19,31]. Next, such partitioning (12) yields the regular SA of (N – 1) – D specified by the mapping: (13) T m   Φ = K ,where K is composed of the new revised vector schedule (represented by the first row of the SA) and the inter-processor communications (represented by the rest rows of the SA), and the matrix Φ specifies the data dependencies of the parallel representation algorithm. For a more detailed explanation of the mapping theory, see [19,27,28]. Next, we define the following specifications for performing the mapping of the fixed-sized reconstructive RS operation, i.e., the y = Fu algorithm of Equation (1), onto each parallel SA core: Π = [1 1] specifies the vector schedule, d = [1 0] specifies the projection vector, and Σ = [0 1] specifies the corresponding space processor.

With these specifications, the transformation matrix becomes T = [ Π Σ ] = [ 1 1 0 1 ]. Next, we specify the dependence vectors of the locally recursive algorithm: Φ = [ Φ F Φ u Φ y ], where Φ F = [ 0 1 ], Φ u = [ 1 0 ] and Φ y = [ 0 1 ] represent the dependencies of the corresponding variables in the algorithm. These specifications result in the following SA dependencies: (14) T Φ = K   →     [ 1 1 0 1 ] ︸ T   [ 0 1 0 1 0 1 ] ︸ Φ = [ 1 1 1 1 0 1 ] ︸ K .

The number of PEs required by each coarse grain SA of the dual SSA core architecture is n, and the required computational time is 2n – 1 clock periods. In Figures 3 and 4, how the SA architecture is implemented at a coarse grain detail for the reconstructive processing of realistic large-scale RS scenes (e.g., 1K × 1K pixel size) by reusing the same fixed-sized SA architecture for each partitioned scene frame is conceptualized. At this stage, the scalability in terms of HW resources can be analyzed varying the number of PEs of the fixed-sized SA architecture.

Once the coarse grain SAs of the selected RS reconstructive algorithm have been defined, we are ready to conceptualize and implement the bit-level fixed-sized dual SSA core. The internal structure of each fixed-sized SA contains identical PEs linearly-connected also in a systolic fashion. The same SA-based design flow, implemented in the previous sub-sections (i.e., algorithmic implementation, tiling and mapping techniques), is again employed for the bit-level dual SSA core as an accelerator structure. Figure 5 illustrates the fixed-sized bit-level SSA architecture (i.e., at a fine grain detail) of each PE of the previously conceptualized RS reconstructive operation.

From the analysis of Figure 5, one can note the improvement achieved with this highly-pipelined architecture in terms of its hardware performance. The bit-level multiply accumulate (MAC) structure of each PE is described as follows: the architecture receives 32-bits operands and generates 64-bits product. The multiplexor in the figure performs the truncate function of the bit-level MAC operation implemented by the array of logic cells. Finally, the logic full-adder implements the rounded function for a better performance.

The SA for performing the bit-level MAC operation of each PE of the dual SSA core employs the following specifications in the transformation defined by Equation (12): Π = [1 2] specifies the vector schedule, d = [1 0] specifies the projection vector and Σ = [0 1] specifies the corresponding space processor. The dependence matrix of the MAC algorithm is specified by Φ = [ 1 0 − 1 0 1 1 ]. For the mapping-optimized projection vector d = [1 0], these specifications yield the following SA structure: (15) T Φ = K   →   [ 1 2 0 1 ]   [ 1 0 − 1 0 1 1 ] = [ 1 2 1 0 1 1 ] .

This bit-level SA-based MAC architecture requires an array of ρ bit-level multiply-accumulate operations with 3ρ − 2 clock periods. In this study, we consider 32 bits operands (i.e., ρ = 32). The high-performance analysis achieved with this dual SSA core architecture will be presented further on in Section 4.

The comparative performance analysis of the HW-level implementation of the dual SSA core architecture is presented in this sub-section. Such HW-level performance analysis is focused on define which is the best area-time tradeoff. The Xilinx XST tool of the Integrated Software Environment (ISE™) WebPACK™ was used for the synthesis of the proposed architecture. The clock frequency of 100 MHz is the selected timing constrain considered for the synthesis procedure. The following parameters were considered in the synthesis: the order of each fixed-sized SSA is m = 64 and the data-output sample word length of 32 bits. In Table 1, it is reported the synthesis results evaluated by different metrics that are indicative of the efficiency of the proposed reconfigurable architecture with respect to the selected FPGA-targets Xilinx Virtex-5 XC5VFX130T and Virtex-4 XC4VSX35.

From the analysis of Table 1, one can conclude that one of the relevant implementation results is related to the high-speed and high-throughput performance, in which the proposed architecture is able to run up to 920.93 MHz.

Next, the scalability analysis in terms of HW resources is presented for the relevant dual SSA core architecture in Figure 6.

In such analysis, the number of precision bits and the number of processing elements (PEs) are modified in order to estimate the HW resources. The results reveal the area resource utilization of the dual SSA-based architecture in the FPGA-target platform.

Other alternative implementations for RS applications that implement specialized HW architectures (i.e., SA-based) are presented in [16,17,32,33]. For example, in [32], a digital custom space-based FPGA co-processor for high-performance space computing is presented. However, in the design of such coprocessors do not consider an MPSoC scheme for a parallel real-time application. Another approach for high-speed computational implementation of reconstructive RS image processing based on the use of clusters of PCs was presented by Yang et al. in [33], in which the cluster NSPO Parallel TestBed for performing parallel radiometric and geometrical corrections of the large-scale 3,600 × 2,944-pixel RS images was implemented. The reconstructive image processing was conducted using a PC-Cluster composed by three PCs each one with a Pentium-III 550 MHz with 128 MB of RAM connected with 100 Mbps Fast-Ethernet LAN. The processing time achieved with such three-PC cluster was only 33.3 s (near-real time for conventional RS users), while the corresponding processing performed with one single processor required 84.65 s. Once more, the authors believe that this dual SSA core is unique and completely differs from other specialized HW architectures in recent RS applications.

In the next section, a concrete real-world Geospatial test application from multi-sensor array SAR systems scenario is presented. This test will evaluate the accuracy of the proposed dual SSA core that it is integrated in a MPSoC design via the HW/SW co-design. The reported results of such enhancement/reconstructive model will be also discussed further on in the next sub-section.

In this sub-section, we present an illustrative test case study related to the HW-implementation of the Weighted Constrain Least Square (WCLS) regularization technique for the enhancement/reconstruction of RS applications. This HW-implementation is based on the proposed here, dual SSA core in aggregation with a Microblaze embedded processor and the On Chip Peripheral Bus (OPB) for transferring the data to/from the embedded processor. In the HW design, we consider to use the precision of 32 bits fixed-point, 9-bit integer and 23-bits decimal for the implementation of all fixed-point operations in each SSA core. Once the HW/SW co-design methodology has been employed, we are ready to establish the verification statements to evaluate the accuracy of the MPSoC system.

In the HW implementation, a large scale (1K-by-1K) pixel format RS image borrowed from the real-world high-resolution terrain SAR was employed. The quantitative evaluation of the RS reconstruction performances was employed using the following quality metric defined by the improvement in the output signal-to-noise ratio (IOSNR) [34]. In this evaluation, the signal formation operator of all RS images is factorized along two axes in the image plane: the azimuth (horizontal axis, x) and the range (vertical axis, y). Following the common practically motivated technical considerations [3,4,15–20], we modeled the Gaussian shape in the SAR range PSF Ψ_r(y) in the range direction y, and the side-looking SAR azimuth PSF Ψ_a(x) in the cross-range direction x at the zero crossing level for the simulated SAR system with fractionally synthesized aperture.

The quantitative measures of the image enhancement/reconstruction performance gains achieved with the particular employed WCLS technique, evaluated via the IOSNR metric, are reported in Table 2 with two different real-world high-resolution scene images.

Next, the qualitative results are presented in Figures 7 and 8, with two different real-world high-resolution scenes. Figure 7(a,b) show the original test scene images. Figures 7(b) and 8(b) present the noised low-resolution (degraded) scene images formed with the conventional MSF algorithm. Figures 7(c) and 8(c) present the scene images reconstructed with the CLS-regularized algorithm. Figures 7(d) and 8(d) present the scene images reconstructed employing the WCLS-regularized algorithm.

From the analysis of the qualitative and quantitative implementation results reported in Figures 7 and 8, and Table 2, one may deduce that the dual SSA core was efficiently integrated in MPSoC embedded system via the HW/SW co-design method. Additionally, such WCLS implementation results over-perform the robust non-adaptive CLS in all simulated scenarios.

Finally, in Table 3, we report the processing times required for implementing the WCLS image reconstruction algorithms using the developed dual SSA core in a MPSoC embedded system.

In the first case in Table 3, the WCLS algorithm was implemented in C++ software in a personal computer (PC) running at 3 GHz with a AMD Athlon (tm) 64 dual-core processor and 2 GB of RAM memory. In the second case, the same WCLS-related algorithm was implemented using the proposed here efficient architecture approach with the specialized dual SSA core employed in the Xilinx FPGA Virtex-5 XC5VFX130T.

The implementation of this high-speed architecture helps to drastically reduce the overall processing time. Particularly, the proposed implementation of the WCLS algorithm with the proposed HW-specialized architecture takes only 0.81 s for the large-scale RS image reconstruction in contrast to 12.6 s required with the C++ reference implementation. Thus, the processing time of the proposed dual SSA core-oriented architecture is approximately 16 times less than the corresponding processing time achievable with the conventional C++ PC-based implementation.

In this regard, the emergence of specialized hardware devices such as the dual SSA core in FPGAs represents a new paradigm to develop real-time systems for remote sensing data processing. The increasing computational demands of remote sensing applications can now be benefit from these compact hardware components, taking advantage of the small size and relatively low cost of these units as compared to clusters or networks of computers. These aspects are of great importance in other areas like hyperspectral imaging for Earth observation and remote sensing missions that require an extremely large number of spectral bands and high spatial resolution.