8.3.1. Results of Classical Approach without Coordination
Table 2 contains the results of the
classical approach (columns “classical”) for the different instance sets as
N×
N-
with the dimension
of the 2D mesh of the NoC and the total computational utilization factor
(see
Section 8.1). The columns in
Table 2 represent bins corresponding to the number of successfully solved instances out of 5 independent synthesis runs.
We observe that for “3 × 3-50-classic”, the platform where the 9 tiles are arranged in a 3 × 3 mesh grid with a total computational utilization of 50% of the platform, the
classical synthesis is almost able to solve all except one instance of the 25 instances within at least one run of five runs per instance (based on the entry “1” in column “0/5” and column “classical”). However, there was no instance for which all five independent runs were solved. If the complexity of the synthesis problem is increased towards larger platforms and higher computational utilization, the
classical approach is almost not able to solve any more instances. While the work at hand also includes refinements that should contribute to the scalability of the
classical approach (which, best to our knowledge, are not considered in any other work on symbolic system synthesis), e.g., the Korst-constraint (cf. Equation (
2)) in the answer set solver and the portfolio solving approach in the background theory, the
classical approach is still not able to scale as desired.
For the runs of instances that reached the timeout, the bar chart in
Figure 6 shows the average runtime distribution between the answer set solver (blue) and the background theory solver (red). Regarding the number of couplings where the background theory provided a feedback to restrict the search space of the answer set solver, this number was between 2 and 4. In the bar chart we observe that for the unfinished runs in the
classical approach more than 90% of the time is spent on average in deciding whether a time-triggered schedule exists. Hence, a significant imbalance exists between the effort spend in the answer set solver and the background theory. A deeper evaluation also revealed that this time is almost exclusively spend in the scheduling stage where tiles are scheduled independently or the extraction of minimal unsatisfiable cores within tile schedules. This observation especially encouraged us to develop the load balancing scheme within the
coordinated synthesis approach as discussed before (see
Section 5).
8.3.2. Results of Coordinated Approach
Table 2 contains the number of solved runs for the different instances in the instance sets for the
coordinated approach (columns “coord”). First of all, we compare the entries of the
classical and the
coordinated approach where none of the instances of an instance set is solved (“0/5” column). We see that up to the 5 × 5-70 instances the
coordinated approach considerably more often is able to find at least one feasible synthesis out of five runs per instance than the
classical approach.
Table 2 also indicates the limitations of the coordinated approach. Starting with the 5 × 5-70 instance set, considerably more instance cannot be synthesized within five independent runs of an instance. Ultimately, for the hardest instances of 6 × 6-70, only one run for one instance was solved. Despite this, by looking at the obtained results as a whole, the
coordinated approach is still a considerable improvement regarding the applicability towards larger problem instances compared to the
classical approach.
To one essential part, the increase in applicability of the coordinated approach is due to the load balancing objective in the coordinated approach as we conjecture that deciding schedulability of not fully utilized tiles can be considered as an easier task. Hence, in the background theory, the tile scheduling stage of the hierarchical scheduling scheme consumes significantly less time than in the classical approach. Furthermore, also the other optimization statements contribute to the improvement of the coordinated approach as these statements allow to decide scheduling faster within the independent application scheduling stage in the hierarchical scheduling scheme.
However, adding the necessary auxiliary variables and the optimization of the coordinated approach to the answer set solver comes with a non-negligible cost in terms of additional variables and constraints. Looking at our instances, the additional number of variables, which are reported by the answer set solver, increased between for the smaller instance up to for the largest instances. Similarly, the number of additional constraints increased between for the smallest instances, respectively, for the largest instances. As one numerical example, for the 6 × 6-70 instances and the classical approach, the average number of variables and constraints and the standart deviation was 821,999 ± 16,971 variables and 3,848,766 ± 303,098 constraints. In contrast, for the coordinated approach these values increase on average to 1,154,760 ± 35,410 variables and 5,198,396 ± 326,980 constraints.
Often, having more variables and constraints in the answer set solver can increase the time to derive a solution of the binding and routing sub-problem. Additionally, once an initial solution is found, the answer set solver spends up to to optimize this solution (or prove that the found initial one is the optimal solution). While for all the 3 × 3 mesh grid instances in the classical approach (note that more detailed data of the solved instances for the classical approach is not shown for the sake of brevity.) the total average time spend in the answer set solver in successful runs is less then , for the same instances it is around 20 s in the coordinated approach.
Figure 7 summarizes the average total times spent in the respective parts of the answer set solver and the background theory solver within the
coordinated approach for the runs that finished successfully (note that the figure also contains the runtime for the enhancements with the DSH).
Regarding the number of couplings between logic solver and background theory, this number varies on average between 3 and 6. Note that the number of required couplings to find a feasible synthesis in all experiments of the work at hand is generally reduced. Here, the addition of the Korst-constraint (cf. Equation (
2)) in the answer set solver, missing in previous approaches [
26,
27], already ensures that all pairwise computational task combinations on a tile are schedulable and potential pairwise scheduling conflicts are not encountered in the background theory anymore.
The changed distribution of efforts in
Figure 7 between answer set solving and background theory solver compared to the
classical approach becomes more prominent if one computes the quotient of averaged runtimes of the time spent in the background theory divided by the time spent in the answer set solver. This quotient is between 70 and 600 for all instances with 3 × 3 mesh grids in the
classical approach, whereas the quotient is in the range 2–10 for all instances with any investigated mesh grid size in the
coordinated synthesis approach. Thus, w.r.t. to the share of the overall solving time in the
coordinated approach, considerable more effort is shifted to the answer set solver.
However, as already indicated earlier, starting with the 5 × 5-70 instances, also the
coordinated approach reaches its limits. The
Table 3,
Table 4 and
Table 5 capture three different categories of timeout runs of the
coordinated approach.
Here, the runs reaching the set timeout for a synthesis run can be divided into two categories:
Table 3 shows selected data of runs of different instance sets where a synthesis run is dominated by the average time the coordinated approach spends within the background theory. This is similar to what we saw in the results for the
classical approach but now we make this observation at considerable harder instance sets.
However, as a new observation,
Table 4 shows average times of synthesis runs where the runtime until the timeout is mainly dominated by the answer set solver. In these cases, the answer set solver could not find a new binding and routing candidate in the meanwhile restricted search space after a few couplings with the background theory solver.
The final and third category is that for larger instances the background theory allocates memory reaching the overall synthesis memory limit and thus the synthesis run is canceled (cf.
Table 5). Thereby, the memory limit is always reached in the last stage of the hierarchical scheduling scheme where the schedulability of independent app clusters is decided. However, in these cases there is always only one large application cluster resulting in scheduling problems spanning all hardware resources (PEs, NIs and routers) of the hardware platform. As a consequence, these scheduling problems are too memory intense to decide given our memory limit of 62 GB RAM.
All the results presented in
Table 3,
Table 4 and
Table 5 are not desired as they limit the scalability of the
coordinated synthesis approach, especially in the 6 × 6 mesh grid regime at higher utilization (cf.
Table 2). Thus, we enhanced the
coordinated synthesis approach by including domain-specific heuristics within the answer set solver. Again, the goal was to have “nicer” solutions proposed to the background theory which can be decided within reasonable time and without running into memory limitations. Furthermore, another goal was to reduce the occurring long solving times in the answer set solver as observed for some 6 × 6 mesh grid instances (cf.
Table 4).
8.3.3. Results of Coordinated Approach with DSH
Table 2 (columns “DSH”) shows the respective number of solved instances for the
coordinated+DSH approach. Furthermore,
Figure 7 includes the average runtime distribution between the answer set solver and the background theory for solved runs of instances.
By comparing the solved instances between the coordinated approach with the coordinated+DSH approach we make two main observations. First and most obvious, the scaling by using DSH is significant better which is evident by looking at the “0/5” bin column starting at the 5 × 5-70 instance set. Second, by looking at the instances where more runs of an instance where solved (“5/5” and “4/5” bin), noticeable more runs of instances were solved in the regime of harder instance sets, i.e., those sets with 70% platform utilization. Thus, by using our domain knowledge encoded in the DSH, we are able to make the coordinated synthesis approach even more scalable and more robust (since more runs of instances are solved).
Regarding the time to derive a feasible system synthesis, by comparing the bars in
Figure 7, the average time spend in the answer set solver and background theory solver is also always lower in the
coordinated+DSH approach, and therefore is the average total synthesis time (with the exception of the single data point (single solved run) for 6 × 6-70-coord in
Figure 7 which is neglected in our statement due to statistical relevance.)
Note that the answer set solver does not report any overhead in terms of additional variables or constraints if domain-specific heuristics are used within the coordinated approach. As described in [
17], the additional formulation of the domain-specific heuristics are only used if non-deterministic choices have to be performed during the solving process and have no immediate contribution to the underlying complexity to the problem that needs to be solved.
While we also observed that the
coordinated approach can spend a considerable amount of time within the answer set solver (see
Table 4), this issue does not occur anymore if we use domain-specific heuristics together with the coordinated approach. Furthermore, not one synthesis run reaches the memory limit anymore.
Overall, our experiments show that the combined coordinated+DSH approach enriched with DSH considerably increases the scalability of the solely coordinated approach. Additionally, the average synthesis times are reduced. However, its limitations become more noticeable especially for the hardest instance set where almost half the instances in the 6 × 6-70 set could not be solved.