DynDSE: Automated Multi-Objective Design Space Exploration for Context-Adaptive Wearable IoT Edge Devices

Abstract

We describe a simulation-based Design Space Exploration procedure (DynDSE) for wearable IoT edge devices that retrieve events from streaming sensor data using context-adaptive pattern recognition algorithms. We provide a formal characterisation of the design space, given a set of system functionalities, components and their parameters. An iterative search evaluates configurations according to a set of requirements in simulations with actual sensor data. The inherent trade-offs embedded in conflicting metrics are explored to find an optimal configuration given the application-specific conditions. Our metrics include retrieval performance, execution time, energy consumption, memory demand, and communication latency. We report a case study for the design of electromyographic-monitoring eyeglasses with applications in automatic dietary monitoring. The design space included two spotting algorithms, and two sampling algorithms, intended for real-time execution on three microcontrollers. DynDSE yielded configurations that balance retrieval performance and resource consumption with an F1 score above 80% at an energy consumption that was 70% below the default, non-optimised configuration. We expect that the DynDSE approach can be applied to find suitable wearable IoT system designs in a variety of sensor-based applications.

1. Introduction

Autonomous wearable IoT devices are being used for physiological and behavioural health-monitoring [1] and provide relevant health status information to their wearers [2,3]. Miniaturised electronics embedded in wearable accessories, garments, etc., provide the resources to retrieve pattern events from streaming sensor data and to interact with the wearer, which led to the concept of edge computing [4]. Edge computing aims to process data at the devices end, rather than the cloud to reduce network load and service response time. Furthermore, reducing communication bandwidth often lowers energy consumption, as well as privacy, and security concerns. For example, in medical IoT monitoring applications [5], a device may retrieve relevant events using embedded machine learning methods, thus sending only abstract event information to the cloud. Nevertheless, resource constrains are a key feature of IoT devices. A wearable IoT device typically consists of multiple sensors, a microcontroller ($\mu C$), which runs data processing algorithms, memory, and a radio module for data communication (Figure 1). Therefore, the optimisation of resource-constrained IoT edge devices has high priority in the design process [6].

Figure 1. DynDSE procedure for wearable IoT edge devices. $\mathcal{X}|\mathcal{E},\Omega$: design space; $\mathcal{X}|E,\Omega$: system configuration; $\mathit{\pi }$ benefit metrics set; $\mathit{\rho }$: cost metrics set; ${\mathit{z}}_{\pi }$: benefit requirement set; ${\mathit{z}}_{\rho }$: cost requirement set; $s_{i,k}$: data sample i from channel k.

The design of an IoT device can be interpreted as the selection of an optimal hardware and software configuration from the wide design space of possible options. Certain system configurations may be not compatible with specific system requirements, e.g., energy saving and retrieval performance.

Available $\mu C$s present differences in terms of resource consumption, execution time, and energy consumption. With the complex interplay of hardware and software components, it is a difficult task to quantify resource use and to manually identify optimal configurations that fulfil system requirements best. The size of the architectural design space often makes manual exploration of embedded systems unfeasible. Automated Design Space Exploration (DSE) [7,8] provides a computational framework to identify optimal configurations.

The design problem is exacerbated when the system does include functions that cannot be statically approximated, and by those with a dynamic effect on the resource-performance trade-off. Sampling strategies, e.g., context-adaptive sampling [9], are a basic dynamic function of wearable IoT devices. A context-adaptive sampling is a dynamic sampling strategy that tunes the sensor’s sampling rate based on a context measure, thus aiming at minimising energy consumption. The stochastic and variable nature of human behavioural patterns induces variability into context-adaptive sampling behaviour, which, in turn, drastically affects the resource-perfomance trade-off. Therefore, the main challenge for the design of a context-adaptive wearable IoT device lies in the identification of viable configurations that fulfil the system requirements under dynamically varying conditions. With DynDSE, we explicitly incorporate context-adaptive system behaviour in the design exploration and simulate systems with actual sensor data.

In this paper, we provide the following contributions:

• We present a simulation-based Design Space Exploration (DynDSE) procedure for wearable IoT devices that employ context-adaptive pattern recognition algorithms for event retrieval, see Figure 1. We provide a formal characterisation of the design space, given a set of system functionalities, components, and their parameters. An iterative search evaluates configurations according to system requirements. The inherent trade-off embedded in conflicting objectives are explored to find an optimal configuration.
• We perform a wearable IoT application evaluation to analyse the resource-performance trade-off considering static and dynamic design aspects through simulations with actual data of Electromyography (EMG)-monitoring eyeglasses in automated dietary monitoring.

In the present work, we formally introduce the DynDSE exploration framework in Section 3 and relevant optimisation metrics in Section 4. Subsequently, a comprehensive wearable IoT application case is presented and analysed in Section 5 and Section 6 to detail the potential of DynDSE.

2. Related Work

DSE frameworks are used for hardware/software co-design of heterogeneous multiprocessor and system-on-chip architectures [10], embedded systems [11], or Field Programmable Gate Array (FPGA) platforms [12]. In conventional DSE approaches, multiple metrics, e.g., energy consumption, memory demand, and cost, must be optimised concurrently according to some application requirements. The conflicting nature of objectives, which reflect many system characteristics, produces trade-offs inherent to the overall system performance. A decision on the most adequate system configuration needs to be taken according to a multi-objective optimisation process. The majority of DSE methods belongs to one of three analysis and evaluation categories, i.e., prototype-based, analytics-based, and simulation-based. The three categories differ in terms of a design time-modelling accuracy trade-off [8]. For example, the prototype-based evaluation provides adequate modelling accuracy, but requires development time with limited exploration capability. The analytics-based evaluation relies on analytical description of component interactions, which allows designers to explore larger design space portions in acceptable time. However, especially with complex architectures, the modelling accuracy of analytics-based evaluations is limited. The simulation-based evaluation is the most versatile approach, as time-modelling accuracy trade-offs of different designs can be achieved by tuning the simulation characteristics. For example, a lower abstraction simulation level, i.e., simulating digital signals between registers and combinational logic, yields higher accuracy but lowers simulation time for analysing software stacks. A higher abstraction simulation level, i.e., simulating system components at the cycle level, computes more efficient simulation, at the cost of averaging inter-cycle system state information. Moreover, simulation-based DSE enables dynamic profiling at run time, which allows the designer to quantify and optimise complex dynamic component interactions and workload.

In last two decades, many DSE approaches have been proposed to design wearable devices. For example, Bharatula et al. [13] proposed a design method to achieve a resource-performance trade-off for a highly-miniaturised multi-sensor context recognition system. Their evaluation showed that, through variations of system design space, the optimisation method was able to extend the battery lifetime by several hours. The same research group introduced multiple metrics to analyse the resource-performance trade-off of a wearable system, i.e., an accelerometer, a microphone, a light sensor, and a TI MSP430F1611 $\mu$C [14]. The authors presented an experimental validation in which a manual multi-objective optimisation was applied to find the best system configuration. In contrast to the works of Bharatula et al., we propose an analytical characterisation of the system architecture with the aim to automate the DSE modelling. Anliker et al. [15,16] presented an automatic design methodology based on abstract modules and task-dependent constraint sets. A multi-objective function incorporated recognition accuracy, energy consumption, and wearability, applied to classification of three modes of locomotion. In contrast, our simulation-based analysis is based on a realistic dataset collected in daily living. We evaluated the relevant effects of the free-living settings on the metrics. Beretta et al. [17] presented a model-based design optimisation to analytically characterise the energy consumption of a wearable sensor node. The authors described a multi-objective exploration algorithm to evaluate system configurations and relative trade-off. The method was application-driven with a fixed system architecture. Stäger et al. [18] took in account several system configuration options, including sensor types, sensor parameters, features, and classifiers. A case study was described related to detection of interactions with household appliances by means of a wrist worn microphone and accelerometer. Evidences were presented for an improvement in battery lifetime by a factor 2–4 with only little degradation in recognition performance.

The aforementioned DSE approaches focused on the evaluation and exploration of wearable device architectures under static workloads. However, todays wearable systems may adopt opportunistic sensing strategies to balance energy consumption and information acquired by a wearable or mobile systems. For example, Rault et al. [19] provided an analysis of techniques for energy consumption reduction in wearable sensors for healthcare applications. Opportunistic sensing strategies consider dynamic effects on the resource consumption trade-off, not considered in a static DSE. For example, an adaptive sampling scheme may reduce sampling rate according to a detected lower signal entropy. For instance, Mesin [20] proposed an adaptive sampling scheme based on sample prediction, where a non-uniform schedule increased the sampling rate only during bursts of physical activity. A multi-layer perceptron predicted subsequent samples and their uncertainties, triggering a measurement when the uncertainty of the prediction exceeded a threshold. In contrast, our approach employs a transparent state-based reactive model to estimate relevance of future samples. Scarabottolo et. al. [21] presented a dynamic sampling strategy for low-power embedded devices. The sampling rate tuning was based on the analysis of the signal’s spectral content. Rieger and Taylor [22] proposed a low-power analog system for real-time adaptive sampling rate tuning, proportional to the signal curvature. Different from our approach, a-priori knowledge was required. Moreover, in contrast to Rieger and Taylor’s [22] low-power analog system, we consider a pattern spotting problem to analyse performance that is frequently required in wearable IoT systems.

3. Exploration Framework

3.1. Design Space Representation

The design space configurations consist of set $\mathcal{X}$ of functionalities, realised by a set $\mathcal{E}$ of components, which are characterised by the set $\Omega$ of parameters. An example is provided at the end of this section.

Formally, the set of functionalities $\mathcal{X}$ is expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \mathcal{X}={\left\{{\Xi }_{\xi }\right\}}_{\xi \in \mathbf{\xi }},\hfill \\ \hfill & \mathbf{\xi }=\left\{\xi \right|\xi \in {\mathbb{N}}_1,\xi \le N_{\Xi }\},\hfill \end{array}\end{array}$$

(1)

where the functionality ${\Xi }_{\xi }$ is the element of the set $\mathcal{X}$, $\mathbf{\xi }$ is the index set to $\mathcal{X}$, and $N_{\Xi }$ is the number of elements of $\mathbf{\xi }$.

A functionality ${\Xi }_{\xi }$ is carried out by one or more components grouped in the set ${ϵ}_{\xi }$, indexed by the index set ${\mathit{q}}_{\xi }$, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {ϵ}_{\xi }={\left\{{ϵ}_{\xi ,q}\right\}}_{q\in {\mathit{q}}_{\xi }}\mathrm{and}\hfill \\ \hfill & {\mathit{q}}_{\xi }=\left\{q\right|q\in {\mathbb{N}}_1,q\le N_{\xi }\},\hfill \end{array}\end{array}$$

(2)

where the component ${ϵ}_{\xi ,q}$ is the element of the set ${ϵ}_{\xi }$, ${\mathit{q}}_{\xi }$ is the index set to ${ϵ}_{~}$, and $N_{\xi }$ is the number of components associated to the functionality $\xi$.

Overall, the design space consists of a collection $\mathcal{E}$ of system components sets, indexed by the collection $\mathit{Q}$ of index sets, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \mathcal{E}={\left\{{ϵ}_{\xi }\right\}}_{\xi \in \mathbf{\xi }}\mathrm{and}\hfill \\ \hfill & \mathit{Q}=\left\{{\mathit{q}}_{\xi }\right|\xi \in \mathbf{\xi }\}.\hfill \end{array}\end{array}$$

(3)

A component ${ϵ}_{\xi ,q}$ is characterised by one or more component parameters grouped in the set ${\mathit{\omega }}_{\xi ,q}$, indexed by the index set ${\mathit{w}}_{\xi ,q}$, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {\mathit{\omega }}_{\xi ,q}={\left\{{\omega }_{\xi ,q,w}\right\}}_{w\in {\mathit{w}}_{\xi ,q}}\mathrm{and}\hfill \\ \hfill & {\mathit{w}}_{\xi ,q}=\left\{w\right|w\in {\mathbb{N}}_1,w\le N_{\xi ,q}\},\hfill \end{array}\end{array}$$

(4)

where the component parameter ${\omega }_{\xi ,q,w}$ is the element of the set ${\mathbf{\omega }}_{\xi ,q}$, $w_{\xi ,q}$ is the index set to ${\omega }_{\xi ,q}$, and $N_{\xi ,q}$ is the number of component parameters associated to the functionality $\xi$ and the component q. Overall, the design space consists of a collection $\Omega$ of component parameters sets, indexed by the collection $\mathit{W}$ of index sets, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \Omega ={\left\{{\mathit{\omega }}_{\xi ,q}\right\}}_{(\xi ,q)\in {\bigcup }_{\xi \in \xi }(\xi \times {\mathit{q}}_{\xi })}\mathrm{and}\hfill \\ \hfill & \mathit{W}=\left\{{\mathit{w}}_{\xi ,q}\right|(\xi ,q)\in \bigcup _{\xi \in \mathbf{\xi }}(\xi \times {\mathit{q}}_{\xi })\}.\hfill \end{array}\end{array}$$

(5)

For example, a spotting algorithm ${\Xi }_1$ is represented by a component ${ϵ}_{1,1}$, e.g., a FFT-based algorithm, characterised by a component parameter ${\omega }_{1,1,1}$, e.g., the data frame size. Data sampling ${\Xi }_2$ is represented by a component ${ϵ}_{2,1}$, e.g., an uniform sampling. A processing unit ${\Xi }_3$ is represented by a component ${ϵ}_{3,2}$, e.g., a Texas Instrument $\mu C$, characterised by a component parameter ${\omega }_{3,2,1}$, e.g., the $\mu C$’s clock frequency. In a compact form, the design space is expressed as:

$$\mathcal{X}|\mathcal{E},\Omega .$$

(6)

3.2. Configuration Generation

The configuration generation selects a design candidate to be evaluated in the simulation, i.e., see Section 3.3. The configuration generation is composed by two main stages. In the first stage, for each functionality ${\Xi }_{\xi }$, a component set ${ϵ}_{\xi }^c$, indexed by the index set ${\mathit{q}}_{\xi }^c$, is selected as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {ϵ}_{\xi }^c={\left\{{ϵ}_{\xi ,q}\right\}}_{q\in {\mathit{q}}_{\xi }^c}\mathrm{and}\hfill \\ \hfill & {\mathit{q}}_{\xi }^c\subseteq {\mathit{q}}_{\xi },\hfill \end{array}\end{array}$$

(7)

where ${\mathit{q}}_{\xi }^c$ represents a subset of the the index set ${\mathit{q}}_{\xi }$. Overall, a configuration consists of a collection E of system components sets, indexed by the collection ${\mathit{Q}}^c$ of index sets, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & E={\left\{{\mathit{ϵ}}_{\xi }^c\right\}}_{\xi \in \mathbf{\xi }}\mathrm{and}\hfill \\ \hfill & {\mathit{Q}}^c=\left\{{\mathit{q}}_{\xi }^c\right|\xi \in \mathbf{\xi }\},\hfill \end{array}\end{array}$$

(8)

In the second stage, for each component ${ϵ}_{\xi ,q}\in {ϵ}_{\xi }^c$ a component parameters set ${\omega }_{\xi ,q}^c$, indexed by the index set ${\mathit{w}}_{\xi ,q}^c$, is selected as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {\omega }_{\xi ,q}^c={\left\{{\omega }_{\xi ,q,w}\right\}}_{w\in {\mathit{w}}_{\xi ,q}^c}\mathrm{and}\hfill \\ \hfill & {\mathit{w}}_{\xi ,q}^c\subseteq {\mathit{w}}_{\xi ,q},\hfill \end{array}\end{array}$$

(9)

where ${\mathit{w}}_{\xi ,q}^c$ represents a subset of the the index set ${\mathit{w}}_{\xi ,q}^c$. Overall, a system configuration consists of a collection $\Omega$ of system component parameters sets, indexed by the collection ${\mathit{W}}^c$ of index sets, expressed as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \Omega ={\left\{{\mathit{\omega }}_{\xi ,q}^c\right\}}_{(\xi ,q)\in {\bigcup }_{\xi \in \mathbf{\xi }}(\xi \times {\mathit{q}}_{\xi }^c)}\mathrm{and}\hfill \\ \hfill & {\mathit{W}}^c=\left\{{\mathit{w}}_{\xi ,q}^c\right|(\xi ,q)\in \bigcup _{\xi \in \mathbf{\xi }}(\xi \times {\mathit{q}}_{\xi }^c)\}.\hfill \end{array}\end{array}$$

(10)

In a compact form, a system configuration is expressed as:

$$\mathcal{X}|E,\Omega \subseteq \mathcal{X}|\mathcal{E},\Omega .$$

(11)

3.3. Configuration Evaluation

A metric estimates benefits $\pi$ or costs $\rho$ of a configuration. The configuration evaluation is based on two sets of metrics. The benefit metric set is defined as:

$$\begin{array}{cc}\hfill & \mathit{\pi }={\left\{{\pi }^p\left(\mathcal{X}\right|E,\Omega )\right\}}_{p\in \mathit{p}}\mathrm{with}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & \mathit{p}=\left\{p\right|p\in {\mathbb{N}}_1,p\le N_p\},\hfill \end{array}$$

(13)

where the element ${\pi }^p\left(\mathcal{X}\right|E,\Omega )$ is a benefit metric and $N_p$ is the number of benefit metrics. The benefit metric set $\mathit{\pi }$ is subjected to the benefit requirement set as:

$$\begin{array}{c}\hfill {\mathit{z}}_{\pi }={\left\{z_{\pi }^p\right\}}_{p\in \mathit{p}},\end{array}$$

(14)

Similarly, the cost metric set is defined as:

$$\begin{array}{cc}\hfill & \mathit{\rho }={\left\{{\rho }^r\left(\mathcal{X}\right|E,\Omega )\right\}}_{r\in \mathit{r}}\mathrm{with}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & \mathit{r}=\left\{r\right|r\in {\mathbb{N}}_1,r\le N_r\}.\hfill \end{array}$$

(16)

where the element ${\rho }^r\left(\mathcal{X}\right|E,\Omega )$ is a cost metric and $N_r$ is the number of cost metrics. The cost objective set $\mathit{\rho }$ is subjected to the cost requirement set as:

$$\begin{array}{c}\hfill {\mathit{z}}_{\rho }={\left\{z_{\rho }^r\right\}}_{r\in \mathit{r}}.\end{array}$$

(17)

Each ${\pi }^p\left(\mathcal{X}\right|E,\Omega )$ and ${\rho }^r\left(\mathcal{X}\right|E,\Omega )$ maps a configuration $\mathcal{X}|E,\Omega$ to the real space $\mathrm{I}\mathrm{R}$, i.e., $\mathcal{X}|E,\Omega \to \mathrm{I}\mathrm{R}$.

The overall optimisation process is formally described as follows. Given a design space $\mathcal{X}|\mathcal{E},\Omega$, the goal of DynDSE is to find the configuration $\mathcal{X}|E,\Omega$, which maximises benefits $\mathit{\pi }$ and minimises costs $\mathit{\rho }$, while respecting the respective set of requirements. The problem can be interpreted as a constrained multi-objective optimisation:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{\mathcal{X}|E,\Omega \subseteq \mathcal{X},\mathcal{E}|\Omega }{max}& \sum _p{\pi }^p\left(\mathcal{X}\right|E,\Omega )-\sum _r{\rho }^r\left(\mathcal{X}\right|E,\Omega )\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.z_{\rho }^r\le 0\forall r\in \mathit{r},\hfill \\ \hfill & z_{\pi }^p\ge 0\forall p\in \mathit{p}.\hfill \end{array}\end{array}$$

(18)

The optimisation provides a set of mutually conflicting solutions, which reflects the trade-offs in the design. To define optimality, one can usually exploit the concept of Pareto-dominance, i.e., a decision maker prefers a configuration to another if it is equal or better in all objectives and strictly better in at least one. As Künzli et al. [23] pointed out, several approaches exists to solve the multi-objective optimisation, e.g., exploration by hand [24], exhaustive search [25], or reduction to a single objective [26], as done in this work.

4. Optimisation Metrics

Table 1 lists the metrics introduced in this Section and their respective requirements.

Table 1. Our metrics and their relative system requirements. The table also indicates the elements which affect the system requirements. T denotes the runtime duration and m the data frame length.

							The Requirement Is Imposed by
	Objective	Abbreviation	Description		Requirement	Value	Task	Battery life	$\mathit{\mu }\mathit{C}$
$\pi \left(\mathcal{X}\right|E,\Omega )$	${\pi }^1\left(\mathcal{X}\right|E,\Omega )$	P	Precision		$z_{\pi }^1$	$\ge 70\%$,	X
	${\pi }^2\left(\mathcal{X}\right|E,\Omega )$	R	Recall		$z_{\pi }^2$	$\ge 80\%$	X
$\rho \left(\mathcal{X}\right|E,\Omega )$	${\rho }^1\left(\mathcal{X}\right|E,\Omega )$	ET	Ex. time in real-time		$z_{\rho }^1$	Real-time: $<m$	X		X
			Ex. time online			Online: $<T$	X		X
	${\rho }^2\left(\mathcal{X}\right|E,\Omega )$	EC	Energy Consumption		$z_{\rho }^2$	$\le 52.8\mathrm{mWh}$	X	X
	${\rho }^3\left(\mathcal{X}\right|E,\Omega )$	MD	Memory Demand		$z_{\rho }^3$	Variable (see Table 5)			X

4.1. Retrieval Performance Metric

The retrieval performance of an event retrieval algorithm is expressed through Precision-Recall metric [27], as follows:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{Precision}\mathrm{P}={\pi }^1\left(\mathcal{X}\right|E,\Omega ),\\ \hfill \mathrm{Recall}\mathrm{R}={\pi }^2\left(\mathcal{X}\right|E,\Omega ).\end{array}\end{array}$$

(19)

From the application perspective, the algorithm must be able to keep an adequate level of retrieval performance. The retrieval performance requirements is usually defined by expert knowledge and we refer to it as $z_{\pi }^1$ and $z_{\pi }^2$.

4.2. Execution Time Metric

A measure of computational complexity denotes the execution time of an algorithm, by pairing the algorithm module and the processing module. An algorithm’s abstraction is the decomposition of an algorithm in distinct stages, in which each stage is composed by one or more functions. Each function f is broken down by counting additions and subtractions (Add), multiplications (Mult), divisions (Div), square roots (Root), exponentials (Exp), and comparisons (Comp).

The number of machine cycles $n_f^{cyc}$ to compute a function f on a $\mu C$ is defined as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill n_f^{cyc}=& (n_{f,\mathrm{Add}}^{op}\times n_{\mathrm{Add}}^{cyc})+(n_{f,\mathrm{Mult}}^{op}\times n_{\mathrm{Mult}}^{cyc})+\hfill \\ \hfill & (n_{f,\mathrm{Div}}^{op}\times n_{\mathrm{Div}}^{cyc})+(n_{f,\mathrm{Root}}^{op}\times n_{\mathrm{Root}}^{cyc})+\hfill \\ \hfill & (n_{f,\mathrm{Exp}}^{op}\times n_{\mathrm{Exp}}^{cyc})+(n_{f,\mathrm{Comp}}^{op}\times n_{\mathrm{Comp}}^{cyc}),\hfill \end{array}\end{array}$$

(20)

where $n_{f,x}^{op}$ is the number of executions related to an arithmetical operation x to compute a function f and $n_x^{cyc}$ is the number of cycles to execute an arithmetical operation x on a $\mu C$. To estimate the execution time ${\mathrm{ET}}_f$ of a function f, the number of machine cycles $n_f^{cyc}$ is divided by the clock frequency $\nu$ of the $\mu C$, as follows:

$${\mathrm{ET}}_f=\frac{n_f^{cyc}}{\nu }.$$

(21)

The definition of the execution time ET depends on the runtime mode. When the runtime mode is real time, ET is computed as:

$$\mathrm{ET}={\rho }^1\left(\mathcal{X}\right|E,\Omega )=\sum _f{\mathrm{ET}}_f.$$

(22)

When the runtime mode is online, ET is computed as:

$$\mathrm{ET}=\sum _{fr}\sum _f{\mathrm{ET}}_f,$$

(23)

where $fr$ is a data frame process. We refer to the system requirements for ET as $z_{\rho }^1$.

4.3. Energy Consumption Metric

$\mu$C energy consumption: Most $\mu C$s support an active state and a stand-by state. The average energy consumption in active state ${\mathrm{EC}}_f^{\mu C}$, related to the computation of a function f, is proportional to the ${\mathrm{ET}}_f$ and defined as:

$${\mathrm{EC}}_f^{\mu C}={\mathrm{ET}}_f\times P_{\mathrm{act}}^{\mu C},$$

(24)

where ${\mathrm{EC}}_f^{\mu C}$ is expressed in Wh, $P_{\mathrm{act}}^{\mu C}$ is the power consumption of the $\mu C$ in active mode expressed in W, and ${\mathrm{ET}}_f$ is expressed in hours.

The average energy consumption in stand-by state ${\mathrm{EC}}_{\mathrm{stb}}^{\mu C}$ is modelled as:

$${\mathrm{EC}}_{\mathrm{stb}}^{\mu C}=T_{\mathrm{stb}}\times P_{\mathrm{stb}}^{\mu C},$$

(25)

where $T_{\mathrm{stb}}$ is the time period of inactivity expressed in hours, and $P_{\mathrm{stb}}^{\mu C}$ is the power consumption in stand-by mode expressed in W.

The $\mu$C energy consumption was calculated as:

$${\mathrm{EC}}^{\mu C}=\sum _{fr}\sum _f{\mathrm{EC}}_f^{\mu C}+{\mathrm{EC}}_{\mathrm{stb}}^{\mu C},$$

(26)

where ${\sum }_{fr}{\sum }_f{\mathrm{EC}}_f^{\mu C}$ denotes the active state energy.

Sensor energy consumption: The average instantaneous energy consumption ${\mathrm{EC}}_t^s$ for a single sensing component is computed by applying the following equations:

$$\begin{array}{cc}\hfill & I_t^s=I_{\mathrm{act}}^s\times D_t+I_{\mathrm{stb}}^s\times (1-D_t),\hfill \\ \hfill & {\mathrm{EC}}_t^s=I_t^s\times V\times t_r,\hfill \end{array}$$

(27)

where $I_{\mathrm{act}}^s$ is the sensor’s average current in active state, $I_{\mathrm{stb}}^s$ is the sensor’s average current in stand-by state, $D_t$ is the instantaneous duty cycle rate, ${\mathrm{EC}}_t^s$ is expressed in Wh, V is the voltage level of the sensing component, and $t_r$ is the temporal resolution expressed in hours.

The sensor energy consumption was calculated as:

$${\mathrm{EC}}^s=\sum _t{\mathrm{EC}}_t^s.$$

(28)

fFlash/Non-Volatile Memory Energy Consumption: To estimate the flash and programmable memory, we formulated an energy model inspired by Konstantakos et al. [28]. Writing energy consumption was determined by:

$${\mathrm{EC}}_w^m=\sum _b{I}_{\mathrm{write}}^m\times V\times t_{\mathrm{w}},$$

(29)

where b indicates a block, $I_{\mathrm{write}}^m$ is the average current consumption in writing mode, and write time $t_{\mathrm{w}}$ is the time required to write a memory block. The energy required to read a memory block was neglected. A static memory energy consumption term was computed as:

$${\mathrm{EC}}_s^m=I_{\mathrm{stb}}^m\times V\times (T-\sum _b{t}_{\mathrm{w}}),$$

(30)

where $I_{\mathrm{b}}^m$ is the stand-by state current value, and T is the total simulation time.

The memory energy consumption was calculated as:

$${\mathrm{EC}}^m={\mathrm{EC}}_w^m+{\mathrm{EC}}_s^m.$$

(31)

Radio transmission energy consumption measure: To estimate the energy consumption of the wireless communication, we relied on the energy model described by Prayati et al. [29]. The model considered the following three stages for transmission: (1) initialisation of transmission and transferring of the frame data from memory to the radio chip FIFO buffer, (2) back-off timeout, and (3) packet transmission via the wireless channel.

In order to calculate the energy consumption to transmit a packet, the following formula was applied:

$$E{C}_p^r=I_{\mathrm{trans}}\times V\times t_{\mathrm{trans}},$$

(32)

where $I_{\mathrm{trans}}$ is the transmission current, p indicating packets, and $t_{\mathrm{trans}}$ is the time requested to prepare and send a packet. As $E{C}_p^r$ is expressed in Wh, $t_{\mathrm{trans}}$ must be converted in hours. In this work we considered $I_{\mathrm{trans}}=21.7$ mA, when a transmission power threshold 0 dBm is chosen, and $t_{\mathrm{trans}}=16\mathrm{ms}$ is the time requested to prepare and send a packet size of 114 Bytes.

The radio transmission energy consumption was calculated as:

$${\mathrm{EC}}^r=\sum _{p}E{C}_p^r.$$

(33)

Total energy consumption: The energy consumption metric EC is computed as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \mathrm{EC}={\rho }^2\left(\mathcal{X}\right|E,\Omega )=\hfill \\ \hfill & {\mathrm{EC}}^{\mu C}+{\mathrm{EC}}^s+{\mathrm{EC}}^m+{\mathrm{EC}}^r.\hfill \end{array}\end{array}$$

(34)

The application context imposes an energy budget requirement to the behavioural monitoring. For example, while monitoring dietary behaviour, the wearable system should be able to work uninterruptedly for the entire day, in order to not miss relevant activities. The system requirement $z_{\rho }^2$ for EC indicates the energy required to deploy the application for the entire runtime. We defined $z_{\rho }^2$ as the quantity in mW calculated as:

$$z_{\rho }^2=\frac{\mathrm{BC}}{\mathrm{RT}}·\varphi ,$$

(35)

where BC is the battery capacity in mWh, RT is the required runtime of the application expressed in hours, e.g., 16 h, and the factor $0<\varphi \le 1$ considers the effect of external factors, which can affect the battery life. In this work we considered $\varphi =0.9$.

4.4. Memory Demand Metric

The memory demand MD is an upper bound of the memory required by the system to execute an event retrieval. MD is computed by considering four terms, i.e., the code memory $m_c$, the data memory $m_d$, the processing memory $m_f$, and the event memory $m_e$.

The memory demand is defined as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \mathrm{MD}={\rho }^3\left(\mathcal{X}\right|E,\Omega )=m_c+m_d+\sum _f{m}_f+m_e,\hfill \\ \hfill & \mathrm{with}{m}_f=m_{f,\mathrm{Int}}+m_{f,\mathrm{Float}},\hfill \\ \hfill & \mathrm{and}{m}_e=m_{e,\mathrm{Int}}+m_{e,\mathrm{Float}},\hfill \end{array}\end{array}$$

(36)

where $m_{f,\mathrm{Int}}$ and $m_{e,\mathrm{Int}}$ are the memory required to store integer values, and $m_{f,\mathrm{Float}}$ and $m_{e,\mathrm{Float}}$ are the memory required to store float values. We refer to the system requirement for MD as $z_{\rho }^3$, which represents the maximum amount of memory available to store information on a certain $\mu C$.

4.5. Communication Latency Metric

The communication latency is the time span between the event-related raw sensor data measurements and the delivery moment at the receiver of the retrieved event information. The communication latency metric CL is computed as follows:

$$\begin{array}{cc}\hfill & \mathrm{CL}={\rho }^4\left(\mathcal{X}\right|E,\Omega )=\sum _f{\mathrm{ET}}_f+\frac{m_e}{{\nu }_{tr}},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill & \mathrm{with}{\nu }_{tr}=\frac{\mathrm{MPS}}{connInterval},\hfill \end{array}$$

(38)

where MPS is the transmitter’s maximum payload size, e.g., 216 bits for Bluetooth Low Energy (BLE), $connInterval$ defines the time of connection events, i.e., ranges from 7.5 ms to 4.0 s with steps of 1.25 ms for BLE, and ${\nu }_{tr}$ is the transmission data rate. We refer to the system requirement for CL as $z_{\rho }^4$, which represents the communication latency tolerance for the application.

5. IoT Application Evaluation

5.1. Smart Eyeglasses to Monitor Eating in Free-Living

We implemented our DynDSE for the design optimisation of 3D-printed regular-looking eyeglasses, which accommodate processing electronics, EMG electrodes, antenna, and power supply. Smart eyeglasses are particularly suited for automated dietary monitoring to unobtrusively detect intake and eating events from activities around the head throughout everyday life, thus replacing classic food journaling and supporting disease management [30,31]. As typical wearable IoT devices, smart eyeglasses could process data locally and provide estimates to other body-worn devices, e.g. smartphones. Furthermore, the monitoring task is a typical example of wearable IoT applications in remote health assistance.

Two pairs of electrodes were symmetrically integrated at the eyeglasses frame, located around the temple ear bends. Contraction of temporalis muscles was monitored bilaterally resulting in two EMG signal channels.

The EMG sensor data stream was segmented into eating and non-eating periods by pattern spotting algorithms. The spotting algorithms were designed to extract features in continuous EMG sensor data and perform one-class classification to identify eating events (i.e., time span between the start and the end of an eating activity), see Section 5.2.1.

5.2. Design Space Representation

We considered $N_{\xi }=4$ system functionalities: ${\Xi }^1$, an algorithm for event retrieval, ${\Xi }^2$, a data sampling strategy, ${\Xi }^3$, a $\mu C$ and ${\Xi }^4$, a runtime mode.

Table 2 presents the design space considered in this work. Our design space included two spotting algorithms, i.e., ${ϵ}_{1,1}$ and ${ϵ}_{1,2}$, which were considered for execution on three $\mu$Cs, i.e., ${ϵ}_{3,1},{ϵ}_{3,2}$ and ${ϵ}_{3,3}$. Moreover, two data sampling strategies, i.e., ${ϵ}_{2,1}$ and ${ϵ}_{2,2}$, were considered, while applying two runtime modes, i.e., ${ϵ}_{4,1}$ and ${ϵ}_{4,2}$.

Table 2. Design space for the EMG-monitoring eyeglasses.

Functionality (${\mathbf{\Xi }}_{\mathit{\xi }}$)	Algorithm (${\mathbf{\Xi }}_1$)	Data Sampling (${\mathbf{\Xi }}_2$)	Microcontroller (${\mathbf{\Xi }}_3$)	Runtime Mode (${\mathbf{\Xi }}_4$)
Component (${ϵ}_{\xi ,q}$)	FFT-based (${ϵ}_{1,1}$)	Uniform (${ϵ}_{2,1}$)	PSoC1 M8C (${ϵ}_{3,1}$)	Real time (${ϵ}_{4,1}$)
	WPD-based (${ϵ}_{1,2}$)	Adaptive (${ϵ}_{2,2}$)	TI MSP430F1611 (${ϵ}_{3,2}$)	Online (${ϵ}_{4,2}$)
			ARM CortexM3 (${ϵ}_{3,3}$)
Parameter (${\omega }_{\xi ,q,w}$)	Data frame size m (${\omega }_{1,1,1}$)
	Data frame size m (${\omega }_{1,2,1}$)	Context measure’s upper bound ${\theta }_h$ (${\omega }_{2,2,1}$)
	Dim. feature space d (${\omega }_{1,2,2}$)

To deal with the requirements in this case study, we relaxed the optimisation problem of Equation (18) into one of maximisation as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{\mathcal{X}|E,\Omega \subseteq \mathcal{X}|\mathcal{E},\Omega }{max}& \sum _p{\pi }^p\left(\mathcal{X}\right|E,\Omega )\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.z_{\rho }^r\le 1\forall r\in \mathit{r},\hfill \\ \hfill & z_{\pi }^p\ge 1\forall p\in \mathit{p}.\hfill \end{array}\end{array}$$

(39)

The above constrained optimisation problem was solved by a grid search-based approach, evaluating a grid of possible configurations with an exhaustive search. At each iteration, a system configuration was generated and the sensor data processed through the simulation. The metrics were estimated and compared with the respective requirements.

5.2.1. Algorithm (${\Xi }_1$)

FFT-based spotting: The first pattern spotting method was introduced by Zhang and Amft [32] in order to identify eating moments. An online non-overlapping sliding window segmentation with length m expressed in seconds was used to extract features in continuous EMG data. A one-class classification was performed by a one-class SVM (oc-SVM). Details of the feature extraction and one-class classification can be found in [32]. The ocSVM was trained applying the leave-one-participant-out (LOPO) cross-validation strategy. Hyperparameter optimisation was performed using grid search approach.

WPD-based spotting: We designed the second pattern spotting method inspired by the classification task presented in [33]. An online non-overlapping sliding window segmentation with length m expressed in seconds was used. From each segmented frame, the maximum sample value was extracted and compared to a threshold experimentally found. When the sample value was lower than the threshold, the frame was not fed to the spotting pipeline. In the pre-processing module, the EMG signals were passed through a notch filter of 50 Hz to remove power line’s interferences, likely to occur in free-living, also de-trended by a digital high pass filter of 20 Hz and rectified. In the feature extraction module, the signal was passed through a Wavelet Packet Decomposition (WPD), to extract c features, i.e., the WPD coefficients, in the time-frequency domain. The depth level of the tree decomposition was kept constant, i.e., $l=2$. A principal component analysis (PCA) was subsequently used to reduce the number of features from c to d. After normalisation, the features were used as discriminant of the target class by using a ocSVM, with number of support vectors $v=1500$.

The ocSVM was trained applying the LOPO cross-validation strategy. Hyperparameter optimisation was performed using grid search approach.

Table 3 presents a breakdown of the spotting algorithms.

Table 3. Analytical breakdown of the two spotting algorithms. Each function f belongs to an algorithm stage $\kappa$. The variable m denotes the length of the window size. For the WPD-based feature extraction, the variable c denotes the number of WPD coefficients, l denotes the depth level of the WPD decomposition tree, and d denotes the dimensionality of the feature space after applying PCA. For spotting, the variable v is the number of support vectors. The operations for pre-processing, feature extraction, and spotting are calculated on each sliding window’s instance.

Algorithm	Stage $\left(\mathit{\kappa }\right)$	Function $\left(\mathit{f}\right)$	N. Arithmetical Operations							Memory Demand
			Add	Mult	Div	Root	Comp	Exp		Integers	Floats
Both	Pre-processing	Low-pass filtering	m	-	-	-	-	-		-	1
FFT-based	Feature extraction	Standard deviation	$3m-1$	m	2	1	-	-		1	-
		Fast Fourier trasform	$3m·lo{g}_{2}m$	$2m·lo{g}_{2}m$	-	-	-	-		-	k
		Maximum	-	-	-	-	$m-1$	-		1	-
		L2-norm	$d-1$	d	-	1	-	-		1	-
WPD-based	Feature extraction	WPD	${\sum }_i^l\frac{(m/2^i)-1}{2+1}·2^i$	${\sum }_i^l\frac{(m/2^i)+1}{4}·2^i$	-	-	-	-		-	$4+d$
		PCA	$d·(c-1)$	$d·c$	-	-	-	-		-	$c·d$
		L2-norm	$d-1$	d	-	1	-	-		1	-
Both	Spotting	Kernel SVM	$v+1$	$2v$	-	-	1	-		v	$v·(d+1)$
		+ Radial basis kernel	$v·(2d-1)$	$v·(d+1)$	-	-	1	v		-	1

5.2.2. Data Sampling (${\Xi }_2$)

A context-adaptive sampling algorithm needs two main components, i.e., a context measure and a response model, to adapt the sampling rate depending on an estimation of relevance of future samples. As a context measure, we employed a basic representation of EMG signal energy. A feedforward state-based model that alternates between attentive and sleep states was adopted as response model. Our sampling strategy was based on the n-shots measure paradigm: the sensor wakes up, takes n samples, and goes to sleep again. The energy content from the n samples was used to compute the context measure as:

$$\begin{array}{ccc}\hfill & e_k=\frac{{\sum }_{i=1}^n{s}_{i,k}}{n},\hfill & \hfill {\theta }_t=max\left(e_1,\dots ,e_k,\dots ,e_K\right),\end{array}$$

(40)

where $e_k$ is the signal energy for the $k^{th}$ channel, $s_{i,k}$ is the $i^{th}$ value sampled from the n-shots measurement, ${\theta }_t$ is the context measure, t is the time-step, K is the number of channels that connect to the same system. A linear mapping function converted ${\theta }_t$ to a candidate duty cycle rate $D_{t+1}^{*}$:

$$D_{t+1}^{*}=D_l+\left[\frac{D_h-D_l}{{\theta }_h-{\theta }_l}·{\theta }_t-{\theta }_l\right],$$

(41)

where $D_l$ is the minimum duty rate set to ${\theta }_l$, which was estimated from the signals noise. A maximum duty rate $D_h$ was set to ${\theta }_h$. We adjusted the model’s sensitivity by tuning ${\theta }_h$.

The behaviour of the response model is described by Equation (42).

The computation of the duty cycle rate for the next period was based on a comparison between the candidate duty cycle rate $D_{t+1}^{*}$ and the threshold value $D_{TH}$.

As the $D_{t+1}^{*}$ exceeds the threshold $D_{TH}$, the response model switches from inattentive state to attentive state. The attentive state is characterised by a monotonically increasing duty cycle rate. As the $D_{t+1}^{*}$ drops below the threshold $D_{TH}$ and the attention time $\tau$ expires, the response model switches back from attentive state to inattentive state. The duty cycle rate’s decision rules for the two states were computed as follows:

$$\begin{array}{cc}\hfill \left(1\right)& \mathrm{Inattentive}\mathrm{state}\hfill \\ \hfill & \mathrm{If}(D_{t+1}^{*}<D_{TH})\mathrm{and}\tau \mathrm{elapsed}):\hfill \\ \hfill & D_{t+1}=D_{t+1}^{*}\hfill \\ \hfill \left(2\right)& \mathrm{Attentive}\mathrm{state}\hfill \\ \hfill & \mathrm{If}(D_{t+1}^{*}>D_{TH}):\hfill \\ \hfill & D_{t+1}=\left\{\begin{array}{c}{D}_t,D_{t+1}^{*}\le D_t\hfill \\ D_{t+1}^{*},D_{t+1}^{*}>D_t.\hfill \end{array}\right.\hfill \end{array}$$

(42)

Table 4 presents a break down of the context-adaptive sampling algorithm. More details can be found in our previous work [34].

Table 4. Analytical breakdown of the context-adaptive sampling algorithm. The variable n denotes the number of samples taken during the n-shots measure, and g denotes the number of channels. The function is executed at any n-shot measure.

Function $\left(\mathit{f}\right)$	N. Arithmetical Operations						Memory Demand
	Add	Mult	Div	Root	Comp	Exp	Integers	Floats
Context measure	$2n·g$	-	2	-	$n-1$	-	-	1
Response output	4	1	1	-	-	-	-	4
Attention time	-	-	-	-	2	-	-	2

5.2.3. $\mu$C (${\Xi }_3$)

The processing module simulates the behaviour of an energy-efficient $\mu C$ while computing the algorithm’s functions at a certain clock frequency. We considered three commonly widely used $\mu C$s, i.e., PSoC1 M8C, TI MSP430F1611, and ARM CortexM3. A Li-Ion polymer battery provided energy to the system. Each $\mu C$ was provided with memory for local data processing, i.e., Flash and non-volatile memory. Table 5 shows the number of machine cycles $n_x^{cyc}$ related to the arithmetical operation x for different $\mu C$s.

Table 5. Number of machine cycles $n_x^{cyc}$ for the arithmetical operation x and memory specifications for the considered $\mu C$s.

		Number of Machine Cycles per Operation							Memory spec. $\left[\mathbf{kB}\right]$
		$n_{\mathrm{Add}}^{cyc}$	$n_{\mathrm{Mult}}^{cyc}$	$n_{\mathrm{Div}}^{cyc}$	$n_{\mathrm{Root}}^{cyc}$	$n_{\mathrm{Comp}}^{cyc}$	$n_{\mathrm{Exp}}^{cyc}$		ROM/ Flash	RAM
$\mu C$	PSoC1 M8C	544	560	912	1344	80	2672		32	2
	TI MSP430F1611	177	153	405	668	37	334		48.25	10
	ARM CortexM3	60	50	80	380	12	210		512	96

5.2.4. Runtime mode (${\Xi }_4$)

Two runtime modes were considered in this work, i.e., online and real time. In our application, real time mode implies that as soon as a data frame has been recorded, the output of the data processing must be available. With online data processing, the temporal requirement is more relaxed, as the output of the data processing must be available by the end of the runtime.

5.3. Configuration Evaluation

5.3.1. Sensor Dataset

Ten healthy volunteers (4 females, 6 males) aged between 20 and 30 years wore the EMG-monitoring eyeglasses for one day. The application data used in the simulation were sensor data collected at uniform sampling rate, i.e., 256 Hz. The eyeglasses were attached after getting up in the morning and kept on till bed time. When a risk of contamination with water existed, the participants were allowed to remove the eyeglasses. Participants manually logged the occurrence of eating events in a diet journal with a one minute resolution.

5.3.2. Multi-Objective Computation

Our framework is based on a simulation which reproduces the functionalities of a wearable IoT system. The sensor’s and $\mu C$’s behaviour can be emulated with a finite-state machine approach, e.g., Buschhoff et al. [35], in order to reproduce dependencies between the system components and between hardware and software.

We evaluated the algorithm’s retrieval performance considering the total number of samples of all eating events according to ground truth labels, the total number of samples of all retrieved eating events, and the sum of the number of samples correctly retrieved belonging to eating events segments.

The number of machine cycles for an instance of a oc-SVM for the WPD-based spotting algorithm, with hyper-parameters $(m=256,d=20)$, applying Equation (20), is computed as:

• Kernel SVM:
  $[v·60]+[2v·50]+[1·12]\approx 24\times {10}^4$ cycles, where $v=1500$, is the number of support vectors;
• Radial basis function:
  $[v(2d-1)·60]+[v(d-1)·50]+[1·12]+[v·210]\approx 525\times {10}^4$ cycles.

For simplicity, we assumed that all operations were performed using float data type.

Accordingly to Equation (21), the ${\mathrm{ET}}_f$ is computed from the number of machine cycles $n_f^{cyc}$, as inversely proportional to the clock frequency $\nu$ of a considered $\mu C$. Table 6 lists the clock frequencies of the three candidate $\mu C$s.

Table 6. Component electrical characteristics to estimate the EC and MD measures via simulation. All values were extracted from datasheets. The following symbology is used. $I_{\mathrm{act}}$: current consumption in active mode; $I_{\mathrm{stb}}$: current consumption in stand-by mode; $\mathrm{V}$: voltage; $\nu$: frequency; Res.: resolution; Cap.: capacity; $M_b$: data block’s size; $I_{\mathrm{write}}^m$: current consumption for memory writing; $t_{\mathrm{write}}$: time for writing a data block.

Functionality	Component	Electric Characteristics
								Memory Write Operation
		${\mathit{I}}_{\mathbf{act}}\left[\mathbf{mA}\right]$	${\mathit{I}}_{\mathbf{stb}}\left[\mathbf{mA}\right]$	V	$\mathit{\nu }\left[\mathbf{MHz}\right]$	Res.$\left[\mathbf{bit}\right]$	Cap.$\left[\mathbf{mWh}\right]$	${\mathit{M}}_{\mathit{b}}$[Byte]	${\mathit{I}}_{\mathbf{write}}^{\mathit{m}}\left[\mathsf{\mu }\mathit{A}\right]$	${\mathit{t}}_{\mathbf{write}}\left[\mathbf{ms}\right]$
$\mu C$ (${\Xi }^3$)	PSoC1 M8C (${ϵ}^{3,1}$)	8.0	0.025	3.3	24	8	-	64	619.5	1.5
	TI MSP430F1611 (${ϵ}^{3,2}$)	0.57	0.05	3	8	16	-	60	2300	23.0
	ARM CortexM3 (${ϵ}^{3,3}$)	7.0	0.55	3.3	48	32	-	256	500	3.28
Data Sampling (${\Xi }^2$)	EMG sensing	4.0	0.008	3.3	$256\times {10}^{-6}$	-	-
Power supply	Li-Ion polymer battery	-	-	3.7	-	-	925

Time ${\mathrm{ET}}_f$ to execute the oc-SVM on a ARM CortexM3 was $114.3\mathrm{ms}$.

Accordingly to Equation (24), the ${\mathrm{EC}}_f^{\mu C}$ is computed multiplying the ${\mathrm{ET}}_f$ and the $\mu C$ energy consumption $P_{\mathrm{act}}^{\mu C}$. The $\mu C$ switching behaviour is regulated by the time constraints given by the ET and the sensor data frequency. Table 6 lists the current consumptions $I_{\mathrm{act}}^{\mu C}$ in active state, $I_{\mathrm{stb}}^{\mu C}$ in stand-by state, and the voltage level V, of the three candidate $\mu C$s. For example, the energy consumption ${\mathrm{EC}}_f^{\mu C}$ to execute an instance of the oc-SVM on a ARM CortexM3 was $0.732\mathsf{\mu }\mathrm{Wh}$.

Figure 2a,b show the linear relation between ${\sum }_f{\mathrm{ET}}_f$ and ${\sum }_f{\mathrm{EC}}_f^{\mu C}$, for the individual spotting stage $\kappa$. A log scale for the y axes was defined in order to make the plot more readable. A tuning effect of the spotting parameters on the execution time and on the energy consumption, is evident. The higher complexity of a parameter combination, the more machine cycles $n_f^{cyc}$, execution time ${\mathrm{ET}}_f$, and energy consumption ${\mathrm{EC}}_f$.

Figure 2. (a) FFT-based spotting: Relation between execution time and energy consumption. Some of the spotting parameters were omitted for readability. The spotting parameter m represents the data frame size. (b) WPD-based spotting. Relation between execution time and energy consumption. The spotting parameter m represents the data frame’s size and d represent the reduced feature space’s dimension.

Table 5 and Table 6 list the memory specifications and the data resolution for the three candidate $\mu C$s. Each $\mu C$ had a RAM memory and a larger flash support, and their capacity represented a system constraint.

By switching runtime mode, i.e., real-time mode or online mode, $m_d$ was defined as follows. In real-time mode, the $m_d$ was defined as the amount of data memory for a window size m. In online mode, the $m_d$ was estimated as the peak of memory footprint required to process the data stream continuously without data loss, using a ring buffer.

Depending on their characteristics, the considered $\mu C$s had a certain latency that shaped the distribution of their memory footprint. For example, Figure 3 shows the distribution of the memory $m_d$ footprint related to a daylong processing. The BLE transmission’s specifications defined the maximum ${\nu }_{tr}$ as 305 kbps and the MPS as 215 bits [36]. Equation (38) defines the actual ${\nu }_{tr}$ by tuning the $connInterval$ parameter, which ranged from 7.5 ms to 4.0 s with steps of 1.25 ms. For example, a retrieved eating event was represented by two time stamps, which represent the start and end of an eating event. A time stamp can be represented as an unsigned short variable type in 2 bytes (i.e., 5 bits for day, 5 bits for hour, and 6 bits for minute). The communication latency CL to deliver an event retrieved by the FFT-based spotting (m = 13) on a ARM CortexM3 was $886\mathrm{ms}$, according to Equation (37).

Figure 3. Memory $m_d$ foot-print for the online data buffer while executing the WPD-based spotting. Each colour is related to a specific $\mu C$.

Considering the typical event frequencies in human daily behaviour and the negligible memory footprint for event information, communication latency was omitted from the following analyses.

5.3.3. Multi-Objective Visualisation

To visualise whether the metrics lie within the system requirements’ boundary conditions or not, we exploited radar plots. Radar plots support the representation of the trade-off across the design space. Each y-axes is related to an objective, i.e., P, R, EC, ET, and MD, and normalised for the respective requirements, yielding a feasibility region within the unitary axes. Thus, a radar plot point beyond the unitary axes indicates unacceptable configurations. The unitary axes represents an upper bound regarding the requirements.

5.3.4. Multi-Objective Analysis

For FFT-based, by tuning the parameter m, the precision’s variation ranged from $48.8\%$ for $m=1$, to $95.7\%$ for $m=33$. The recall R variation ranged from $62.9\%$ for $m=29$, to $99.2\%$ for $m=1$. The F1-score variation ranged from $63.3\%$ for $m=1$, to $85.2\%$ for $m=9$. For WPD-based, by tuning the parameters m and d the precision P variation ranged from $38.7\%$ for $m=0.25,d=50$, to $95.7\%$ for $m=0.5,d=15$. The recall R variation ranged from $52.7\%$ for $m=0.25,d=28$, to $86.2\%$ for $m=1,d=40$. The F1-score variation ranged from $49.1.0\%$ for $m=0.25,d=50$, to $85.9\%$ for $m=1,d=40$. The system requirements $z_{\pi }^1$ and $z_{\pi }^2$ are indicated as horizontal lines. It is evident that only a subset of spotting parameters respects the requirements.

Figure 4a,b show the results of the cross-validated FFT-based and WPD-based spotting algorithms in uniform sampling mode, when tuning their spotting parameters. Specifically, precision P, recall R, and F1-score, are reported for all spotting parameters combinations. The degree of sampling reduction was changed by tuning ${\theta }_h$ in order to find a balance between retrieval performance and resource consumption. Figure 5a,b show the trade-off between F1-score and sampling reduction for the FFT-based and WPD-based spotting algorithms. It is clear that we can keep the retrieval performance up over $80\%$ while performing a sampling reduction over $70\%$, in both methods. Comparing the two methods, the WPD-based spotting appears to be more robust against the down-sampling effect, presenting a lower degradation of retrieval performance for a given sampling reduction degree.

Figure 4. (a) FFT-based spotting. Average retrieval performance when varying the spotting parameters in uniform sampling mode. Bars were sorted by increasing retrieval performance. The spotting parameter m represents the data frame size. (b) WPD-based spotting. Average retrieval performance when varying the spotting parameters in uniform sampling mode. Bars were sorted by increasing retrieval performance. The spotting parameter m represents the data frame’s size and d represent the reduced feature space’s dimension.

Figure 5. (a) FFT-based spotting. Average retrieval performance vs. sampling reduction when varying ${\theta }_h$. Individual lines correspond to the spotting parameters, which respect P and R requirements in Table 1. The F1-score requirement is derived by the same P and R requirements. The used parameters for the context-adaptive sampling were: $D_h=1$, $D_l=0.1$, $D_{TH}=0.6$, $n=4$, $\tau =3\mathrm{s}$, ${\theta }_l=10\mathrm{mV}$. (b) WPD-based spotting. Average retrieval performance vs. sampling reduction when varying ${\theta }_h$. Individual lines correspond to the spotting parameters that respect P and R requirements in Table 1. The F1-score requirement is derived from the same P and R requirements. The used parameters for the context-adaptive sampling were: $D_h=1$, $D_l=0.1$, $D_{TH}=0.6$, $n=4$, $\tau =3\mathrm{s}$, ${\theta }_l=10\mathrm{mV}$.

From Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11, the trade-off across the metrics are shown for different system configurations. Figure 6 and Figure 7 show results for the FFT-based spotting in real-time for uniform and context-adaptive sampling. In uniform sampling, i.e., Figure 6, the largest requirement breach on any $\mu C$ was due to the energy consumption EC. In context-adaptive sampling, i.e., Figure 7, the reduction of the energy consumption EC determined a set of feasible configurations on the ARM Cortex M3. In both sampling modalities, the execution time ET fulfilled the real-time requirements on any $\mu C$, although the memory demand MD compromised the feasibility of the configurations on the PSoC1 M8C.

Figure 6. FFT-based spotting and uniform sampling. Resource-performance trade-off for real-time mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of objectives: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand.

Figure 7. FFT-based spotting and context-adaptive sampling. Resource-performance trade-off for real-time mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of metrics: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand. Context-adaptive sampling parameters are: $D_h=1$, $D_l=0.1$, $D_{TH}=0.6$, $n=4$, $\tau =3s$, ${\theta }_l=10\mathrm{mV}$, ${\theta }_h=180\mathrm{mV}$.

Figure 8. WPD-based spotting and uniform sampling. Resource-performance trade-off for real-time mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of metrics: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand.

Figure 9. WPD-based spotting and adaptive sampling. Resource-performance trade-off for real-time mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of metrics: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand. The used parameters are: $D_h=1$, $D_l=0.1$, $D_{TH}=0.6$, $n=4$, $\tau =3\mathrm{s}$, ${\theta }_l=10\mathrm{mV}$, ${\theta }_h=180\mathrm{mV}$.

Figure 10. WPD-based spotting and uniform sampling. Resource-performance trade-off for online mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of metrics: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand.

Figure 11. WPD-based spotting and context-adaptive sampling. Resource-performance trade-off for online mode, including different $\mu C$s: ARM CortexM3 (left), TI MSP430F1611 (center), PSoC1 M8C (right). List of metrics: P = precision, R = recall, EC = energy consumption, ET = execution time, MD = memory demand. The used parameters are: $D_h=1$, $D_l=0.1$, $D_{TH}=0.6$, $n=4$, $\tau =3\mathrm{s}$, ${\theta }_l=10\mathrm{mV}$, ${\theta }_h=180\mathrm{mV}$.

Figure 8 and Figure 9 show results for the WPD-based spotting in real-time for uniform and context-adaptive sampling. In uniform sampling, i.e., Figure 8, the largest requirement breaches on any $\mu C$ were due to the execution time ET and the energy consumption EC. In context-adaptive sampling, i.e., Figure 9, the reduction of the execution time ET determined a set of feasible configurations on the ARM Cortex M3. Overall, the memory demand MD was neglectable in all configurations, as only a data frame had to be stored.

Figure 10 and Figure 11 show results for the WPD-based spotting in online mode for uniform and context-adaptive sampling. The online mode required a more intensive memory use, due to the longer ring buffer, that in turn made the $z_{\rho }^3$ a more stringent requirement. In uniform sampling, i.e., Figure 10, the largest requirement breaches on any $\mu C$ were due to the energy consumption EC and the memory demand MD. In context-adaptive sampling, i.e., Figure 11, the reduction of the energy consumption EC determined a set of feasible configurations on the ARM Cortex M3.

Figure 12 depicts the energy consumption estimated from the best system configuration, related to the individual participants. For three participants, the boundary conditions are not respected, implying a reduced application’s runtime with respect to the system requirements.

Figure 12. Estimated energy consumption (EC) for the individual participants on the TI MSP430F1611. Left: FFT-based spotting with $m=13$. Right: WPD-based spotting with $(m=256,d=20)$.

6. Discussion

The simulation-based DynDSE presented here targets wearable IoT device design, which run time-variable recognition algorithms on the device. Processing a large volume of data locally and enabling local inference is key to a scalable IoT network. Furthermore, manual tuning of hardware and algorithms in a physical implementation is tedious. The DynDSE simulation-based approach can cover a wide configuration space to identify a balance between resources and event retrieval performance. Our case study demonstrated interactions and dependencies among hardware and algorithm components, and justified the need for co-designing and developing of associated functionalities. We chose two example retrieval algorithms to illustrate different effects on the optimisation result (cf. Figure 9, Figure 10, Figure 11 and Figure 12). FFT-based oc-SVM spotting was published before [32]. The WPD-based spotting showed performance improvements P and R, but also implications for execution time ET, thus further illustrating the design trade-off features of our methodology. The processing functionality (${\Xi }_3$) affected the execution time ET, due to the $\mu C$ type and speed, and the memory demand MD, due to its memory capacity. The $\mu C$’s energy consumption EC heavily depended on the algorithmic complexity, see Figure 12. Executing the WPD-based spotting, the $\mu C$’s energy consumption EC was comparable with the sensor’s energy consumption EC. As the computational complexity was shrunk by employing the FFT-based spotting, the energy consumption EC became neglectable. The algorithm (${\Xi }_1$) and data sampling (${\Xi }_2$) functionality were the most influencing configuration elements on the system’s metrics. Tuning the algorithm parameters affected precision P and recall R, and the required memory demand MD. Data sampling had the highest impact on the energy consumption EC and the tuning of the algorithm parameters had the lowest.

We found that our context-adaptive sampling strategy kept the performance of the spotting pipeline at a average F1-score over $80\%$ while reaching almost $70\%$ reduction in resource consumption.

DynDSE requirements in the application evaluation were set according to Table 1. While in our analysis, retrieval performances (P, R) larger than 80% were reached for optimised parameter settings, our requirements $z_{\pi }^1$ and $z_{\pi }^2$ followed literature recommendations [37] suggesting that even 70% in retrieval performance has relevant application value. Energy consumption requirement $z_{\rho }^2$ was set considering the capacity and size of standard lithium-ion batteries, and the application runtime, according to Equation (35). Execution time and memory demand requirements $z_{\rho }^1$ and $z_{\rho }^3$ were dictated by the algorithm and the $\mu C$ characteristics, respectively. While the retrieved system configurations and their performances appear relevant, a direct comparison to prior work is not feasible, due to the diversity in analysis goals, applications, and dataset characteristics. First, many investigations optimise for a fraction of the DynDSE metrics only, e.g., recognition performance. Second, sensor and algorithm choice span a wide value space for performance metrics. Our investigation aimed at defining a generalisable procedure, which provides trade-off indicators across a variety of design space options and could thus assist designers in taking decisions and investigate details depending on application relevance.

Figure 7 and Figure 8 show the analysis of the variance in the resource-perfomance trade-off. Under the same P and R, higher sampling reduction can be achieved, which corresponds to lower energy consumption EC. In context-adaptive sampling mode, the resource consumption is proportional to the event frequency and the duration of event patterns. Consequently, for one configuration, resource saving varies according to individual behaviour. Population-averaged models do not guarantee to fulfil the system requirements for every individual. In our case study, a homogeneous study group of university students was included, however the resource consumption estimation did not respect the boundary conditions for all participants, as shown in Figure 12.

Personalising models increases computational complexity and entails more complicated deployment. A reasonable approach is to take into account the heterogeneity of the population by defining subpopulations having similar behaviour and include a safety margin.

We derived approximate machine cycle numbers, which limit accuracy of execution time estimation. The exact number of cycles is highly dependent on the algorithm implementation and compiler. Thus, our analysis could be integrated with extended target-dependent hardware and machine instruction simulators. Another source of inaccuracy are the energy consumption measures, as we did not consider overhead of the electronic circuits. We considered for simplicity only floating point operations. Differentiating between integer and floating point operations would result in higher modelling accuracy. Also, differentiating the range of variables and datatypes may improve the modelling.

The metric set serves as mapping between the design space and the application specifications, whose definition largely depends on the application requirements. Therefore, a direct comparison of metric outcomes is limited. However, depending on the defined metric set, well-determined functional implications and system properties can be identified and compared.

For example, Bharatula et al. [14] defined four conflicting metrics and analysed the inherent trade-off on an activity recognition task: Flexibility, which included estimation of memory demand and $\mu$C’s operating frequency, electronic packaging, relative recognition performance, and energy consumption measures. The conflicting nature of the four metrics was highlighted as orthogonal, meaning that optimising all the four metrics at the same time is not feasible. Similar to our work, the authors embedded recognition algorithm characteristics in the trade-off analysis, namely classification accuracy. However, we included execution time ET to highlight dynamic design aspects that appear during runtime. The ET metric linked the algorithmic computational complexity with the hardware $\mu$C characteristics in time. Understanding of the system temporal constraints enables DynDSE to leverage dynamic system behaviour for context-adaptivity. Azariardi’s [38] DSE included ET and classification accuracy in the metric set but omitted energy consumption. The authors were able to estimate temporal constraints for SVM processing and investigated how the DSE solution matches with application requirements and free-living user. However, assumptions were needed to compensate for the missing energy consumption (EC) metric. The application of Beretta et al. [17] consisted of a wearable node transmitting sensor data using compressive sensing. Objectives were the node’s energy consumption, the percentage root-mean-square difference (PRD) to approximate the information loss due to compression, the communication delay and the packet error rate (PER) of the radio transmission. The solution space was compared with the one reported by Kumar et al. [39], which optimised only energy consumption and communication delay. Under the same energy consumption and communication delay solution, the PRD and PER were significantly higher. Moreover, Kumar et al. were able to discover only the 2.3% of the solution space with respect to Beretta’s work.

From the above comparison, it appears that the descriptive power of the trade-off analysis in DSE depends on a careful selection of the metrics. Neglecting metrics may result in misleading results. The same conclusion can be drawn with regard to our work. For example, consider Figure 9 and Figure 10: When omitting EC, it may seem that the choice of sampling mode does not affect the system behaviour. As the EC is included in our trade-off analysis, it becomes evident how the uniform sampling mode does not provide any feasible configuration. DSE frameworks that consider hardware-software co-design, in principle, achieve higher system performances as a consequence of the flexibility given by a finer model granularity. For example, Shoaib et al. [40] optimised the individual processing stages of a SVM pipeline by exploring hardware architectures based on custom instructions and coprocessor computations. The authors reported a reduction in energy consumption of almost three orders of magnitude compared to that of a low-power $\mu$C, as targeted by our work. The energy consumption metric was computed as the sum of several real measurements related to hardware components involved in the SVM processing stage. The design space solutions included specific hardware to run kernel-based classification in varying contexts. The optimisation potential of hardware-software co-design comes at the cost of an expensive design, which includes custom-made platforms, and design space and metrics definition that rely on hardware-specific knowledge.

Overall, dynamic system configurations have been rarely considered in DSE for wearable systems. The inclusion of data sampling strategies into the design space enabled us to adapt system designs to context. Moreover, memory demand has been infrequently included into the DSE objective set, although memory limitations are common in $\mu$Cs and represent a bottleneck for embedded recognition algorithm deployment, as evident from Figure 9 and Figure 10. We argue that memory demand should be considered in the design phase.

This work focuses on one typical wearable IoT application in order to derive a detailed analysis of the design space spanned by two retrieval algorithms, three $\mu C$s, and two sampling procedures introducing dynamic variations. Nevertheless, we kept the DynDSE design space formalism general, such that a wide variety of other components, system architectures, metrics, and IoT applications could be explored, including other hardware, data, and recognition algorithms. Thus, the DynDSE approach does not depend on the particular application considered nor does the method require modifications for other applications. Rather, we deem it essential to match the DynDSE approach with appropriate sensor data to drive the simulation.

For larger design spaces than the one considered here, DynDSE may require approximate rules. Nevertheless, the exhaustive search deployed here remains a suitable option for coarse design selections before investigating further design variables in subsequent, local explorations.

7. Conclusions and Future Work

We introduced a general methodology for multi-objective DynDSE applied to context-adaptive wearable IoT edge devices, which retrieve events from streaming sensor data using pattern recognition algorithms. We provided a formal characterisation of the configuration space given a set of system functionalities, components and their parameters. A constrained optimisation problem was formulated to identify an optimal system configuration according to application-dependent system requirements. The simulation can provide crucial information about the compatibility of system components. The method is particularly suitable to analyse design options at an early stage of the development process, to approximate key system design aspects, e.g., size of wireless battery powered devices, to confirm software and hardware choices under given design constraints, and to review designs under varying data patterns.

Further investigations may consider automated, on-demand resource distribution between functions of an embedded system that incorporates the DynDSE methodology. Dynamic resource management may result in wearable IoT systems that reconfigure themselves at runtime according to dynamic conditions. Furthermore, the increasing ubiquity and interconnection among wearable IoT devices rise concerns about security and privacy, as malicious interactions are more likely to happen. System security objectives could be incorporated into the dynamic optimisation to represent varying privacy concerns. Nevertheless, further research is needed to effectively quantify security and privacy concerns in metrics.