G-CTRNN: A Trainable Low-Power Continuous-Time Neural Network for Human Activity Recognition in Healthcare Applications

Featured Application

Real-time, on-device human activity recognition using low-power MEMS-based wearables for remote patient monitoring, fall detection and personalized healthcare in resource-constrained edge environments.

Abstract

Continuous-time Recurrent Neural Networks (CTRNNs) are well-suited for modeling temporal dynamics in low-power neuromorphic and analog computing systems, making them promising candidates for edge-based human activity recognition (HAR) in healthcare. However, training CTRNNs remains challenging due to their continuous-time nature and the need to respect physical hardware constraints. In this work, we propose G-CTRNN, a novel gradient-based training framework for analog-friendly CTRNNs designed for embedded healthcare applications. Our method extends Backpropagation Through Time (BPTT) to continuous domains using TensorFlow’s automatic differentiation, while enforcing constraints on time constants and synaptic weights to ensure hardware compatibility. We validate G-CTRNN on the WISDM human activity dataset, which simulates realistic wearable sensor data for healthcare monitoring. Compared to conventional RNNs, G-CTRNN achieves superior classification accuracy with fewer parameters and greater stability—enabling continuous, real-time HAR on low-power platforms such as MEMS computing networks. The proposed framework provides a pathway toward on-device AI for remote patient monitoring, elderly care, and personalized healthcare in resource-constrained environments.

1. Introduction and Literature Review

1.1. Introduction

Human Activity Recognition (HAR) has become a cornerstone technology in modern healthcare systems, enabling applications such as fall detection, chronic disease monitoring, rehabilitation tracking, and ambient assisted living [1,2]. With the increasing adoption of wearable sensors and edge AI, there is a growing need for machine learning models that can operate in real time and under strict energy and hardware constraints. In this context, Continuous-Time Recurrent Neural Networks (CTRNNs) offer an analog-friendly, biologically inspired framework for modeling temporal dynamics in continuous data streams [3,4], such as acceleration signals from wearable healthcare devices. Their formulation using differential equations allows direct mapping onto low-power embedded hardware like mixed-signal processors [5] and MEMS sensors [6,7,8]. However, a persistent barrier to their practical adoption is the difficulty of training such systems using gradient-based methods, especially when time constants and other physical parameters must remain bounded and interpretable [9].

Traditional training techniques for recurrent networks, such as Backpropagation Through Time (BPTT), are typically applied in discrete time and do not directly extend to physical systems governed by continuous dynamics. Moreover, existing approaches often fail to account for hardware constraints, including energy efficiency, device variability, and limited tunability of analog parameters. In this paper, we introduce G-CTRNN, a gradient-based training framework for analog-friendly CTRNNs, tailored specifically for low-power healthcare-focused HAR applications. Implemented in TensorFlow, our approach extends BPTT to continuous-time networks while enforcing parameter constraints that reflect real-world hardware requirements. This makes it uniquely suited for real-time HAR on low-power embedded systems such as MEMS sensors used in wearables. We validate G-CTRNN using the WISDM dataset, which contains smartphone-based activity data commonly used in healthcare applications. The results show that G-CTRNN provides superior accuracy and efficiency compared to traditional RNNs while significantly reducing the number of parameters and resource consumption.

1.2. Related Work

CTRNNs were initially introduced as continuous analogs to traditional RNNs, capable of modeling the temporal evolution of dynamical systems [2,3,4]. Funahashi and Nakamura [9] demonstrated that CTRNNs can approximate any dynamical system given enough neurons, laying a theoretical foundation for their use in temporal learning tasks. Beer [3] later extended these ideas by applying CTRNNs to biologically inspired agents in evolutionary robotics, highlighting their capacity to capture complex temporal behaviors.

Despite their potential, training CTRNNs remains a bottleneck. Traditional methods like genetic algorithms [10] and differential evolution [11] have been explored, particularly in low-data or hardware-constrained scenarios. While these approaches provide flexibility in parameter search, they suffer from scalability issues and slow convergence. More recently, advances in neural ordinary differential equations (Neural ODEs) have enabled differentiable modeling of continuous-time systems [12]. These models integrate deep learning with ODE solvers and backpropagation through continuous-time dynamics. Our approach builds on these ideas by incorporating automatic differentiation and constrained optimization of time constants within a CTRNN framework, aligning with recent efforts to make continuous-time models trainable and scalable [13].

Recent research has also explored advanced strategies for improving learning efficiency and human activity recognition. Orthogonal learning designs have been applied to enhance the performance of metaheuristic optimization algorithms [14], while fuzzy architecture search with classifier-assisted multi-objective optimization shows promise in tuning neural models [15]. In the context of HAR, attention-based temporal convolutional networks have improved sequential modeling performance [16], and cross-modal federated learning has enabled privacy-preserving activity recognition across devices [17]. These efforts highlight the ongoing push toward efficient, robust, and scalable AI models, which align with the goals of G-CTRNN for edge deployment in healthcare.

2. Continuous-Time Recurrent Neural Network (CTRNN)

This section aims to describe the continuous-time recurrent neural network mathematical model and its analogy to physical systems such as MEMS computing networks.

2.1. CTRNN Model

The CTRNN mathematical model consists of six preliminary components, as shown in Figure 1a and Equation (1). The differential part in Equation (1) makes them intricately intertwined to mimic the behavior of human brain neuron cells [12]:

$${\mathit{\tau }}_{\mathit{i}}{\displaystyle \frac{\mathit{d}{\mathit{y}}_{\mathit{i}}}{\mathit{d}\mathit{t}}}=-{\mathit{y}}_{\mathit{i}}+{\mathit{f}}_{\mathit{i}}({\mathit{B}}_{\mathit{i}}+{\sum }_{\mathit{j}=1}^{\mathit{n}}{\mathit{w}}_{\mathit{i}\mathit{j}}{.\mathit{y}}_{\mathit{j}})+{\mathit{I}}_{\mathit{i}}(\mathit{t})$$

(1)

where ${\mathit{\tau }}_{\mathit{i}}$ is the time constant of neuron i; ${\mathit{y}}_{\mathit{i}}$_i and ${\mathit{f}}_{\mathit{i}}$_i are the state and activation function of the ith neuron; and ${\mathit{w}}_{\mathit{i}\mathit{j}}$, ${\mathit{B}}_{\mathit{i}}$, ${\mathit{I}}_{\mathit{i}}\left(\mathit{t}\right)$ represent the weight vector, bias shift, and external input at time t of the ith neuron. The first part of the neuron is the continuous state of the current neuron, represented by ${\mathit{y}}_{\mathit{i}}$ This state changes over time and depends on the previous state and all the other five components. The second part is the activation or nonlinearity function, which is denoted as ${\mathit{f}}_{\mathit{i}}$, the Sigmoid function. Rectified Linear Units (ReLUs) and hyperbolic tangent functions (Tanhs) are all examples of widely used non-linear functions. The behavior of a biological neuron inspires the activation function to capture the nonlinearity of a complex pattern. As a group of neurons activate together, signal propagation occurs through them in a specific path. Different paths capture different patterns. The third part is the synaptic weight vector, represented as ${\mathit{w}}_{\mathit{i}\mathit{j}}$. It describes the strength of the connection between neurons j and i. The fourth part is the external input, ${\mathit{I}}_{\mathit{i}}\left(\mathit{t}\right)$, representing the continuous input signal from the sensors or other data sources. The fifth part is the time constant represented by ${\mathit{\tau }}_{\mathit{i}}$, which controls how quickly the current cell reacts to any external input; a high τ means the neuron needs more time to respond and change its status, while a low time constant means this neuron has a rapid response to any change. The last part is the neuron dynamics, represented by the state’s rate of change (differential part), represented by ${\displaystyle \frac{\mathit{d}{\mathit{y}}_{\mathit{i}}}{\mathit{d}\mathit{t}}}$.

2.2. MEMS Analogy

As stated before and shown in Figure 1b, the MEMS dynamics can be engineered to match the equation of a continuous-time recurrent neuron [6,7,8]. Thus, eventually, the MEMS can perform the CTRNN calculation without the need for a computer. In this mapping, the time constant τ in the CTRNN corresponds to the mechanical time constant of MEMS resonators, governed by the mass-to-damping coefficient ratio. Synaptic weights can be mapped to coupling stiffness values or transconductance gains in MEMS electro-mechanical systems, while biases relate to baseline actuation voltages or equilibrium positions. However, practical limitations must be considered. MEMS devices are affected by process variability, temperature-dependent drift, and mechanical noise, which may introduce deviations from ideal continuous-time behavior. Despite these challenges, recent work in analog computing shows promise in mitigating such effects through calibration and robust design strategies.

2.3. Time Constant Effect

To study the effect of the time constant on the CTRNN response, we discretize the CTRNN, as shown in Equation (2), using the Euler method.

$$y_{i+1}=[y_i+{\displaystyle \frac{1}{{\tau }_i}}\ast [-y_i+f_i(B_i+{\sum }_{j=1}^n{w}_{ij}{.y}_j)]+I_i(t)]$$

(2)

where h is the Euler democratization step size; τ_i is the time constant for the ith neuron. Equation (2) implies that when τ_i approaches a large value, the subsequent state y_i+1 equals the current state y_i (tiny change). Conversely, when τ_i approaches zero, the next state will be strongly influenced by input changes or the status of other neurons. It is crucial to select the τ value greater than h, as if h >> τ, Equation (2) indicates that the second term of the equation will be multiplied by a value greater than 1, causing system instability. This instability arises from amplifying the other neurons’ external input and output, leading to an unstable neuron state. Equation (2) recommends choosing h << τ to ensure the neurons and the model’s stability.

3. Model Development and Training

In this section, we discuss the human activity training data and the development of the G-CTRNN model to classify them. We also explain the training algorithms to learn and optimize the model parameters.

3.1. Human Activity Detection

We used human activity detection applications to verify our G-CTRNN model and to ensure its real-world applicability, test its temporal learning capabilities, validate its performance, and facilitate comparative analysis with other models in activity recognition tasks. Human activity detection has many field applications, including healthcare, security, robotics, and smart buildings [18,19]. Many technologies were utilized to implement human activity detection, like wearable, camera-based, infrared (IR), and Wi-Fi sensors [20]. Human activity detection enables intelligent, responsive environments that adapt to the residents’ needs and preferences. Recognizing human activities will also help save energy consumption, enhance security, improve indoor environmental quality, and provide personalized services. For example, ref. [21] employed human activity recognition to manage HVAC systems, lighting, heating, ventilation, and air conditioning by responding to occupancy and activity patterns, decreasing energy usage, and enhancing energy efficiency. Moreover, the work in [22] utilized human activity recognition, which identifies irregularities, like unauthorized access or atypical behavior, thereby bolstering security measures and thwarting potential intrusions.

In this work, we used the WISDM (Wireless Sensor Data Mining) dataset for training and testing G-CTRNN. The WISDM data contains six attributes extracted from smartphones for various groups of people while carrying those devices. The data contains six activities: jogging, walking, upstairs–downstairs, sitting, and standing, as shown in Figure 2. The data holds significant prominence in the literature due to its widespread utilization and well-organized structure [23]. However, while the WISDM dataset demonstrates the feasibility of applying G-CTRNN to sensor-based human activity recognition, we acknowledge its limitations in representing clinical scenarios. The dataset involves healthy subjects performing standard activities, which may not capture the complexity of gait abnormalities, post-surgical recovery movements, or neurological impairments. Future work will include validating G-CTRNN on more diverse and clinically representative datasets to assess its broader applicability in healthcare contexts.

3.2. Model Structure

As shown in Figure 3, the model consists of a set of consecutive connected layers. The first layer has sixteen recurrent cells. The first layer is fully connected to the second layer, which has ten cells. The final layer has six cells, one for each human activity. The model systematically traverses the dataset. We used well-known algorithms called sliding windows [24]. The algorithm employs a predetermined window size and constant sliding steps. The purpose is to group subsequent records into one entity. Based on the literature, we picked the window size to be fifty records, corresponding to approximately 2.5 s at WISDM’s 20 Hz sampling rate. While this fixed-window method is standard for WISDM, we acknowledge that different healthcare applications may require varying temporal resolutions from sub-second seizure detection to multi-minute rehabilitation tracking. Future work may consider adaptive windowing strategies to dynamically adjust window lengths based on signal characteristics or medical context.

To maintain data integrity, we only include instances where the dominating activity constitutes at least 70% of the observations in the training phase. Including this percentage will ensure a higher degree of activity purity. The data was divided into three distinct portions. Precisely 10% was allocated for testing, and 90% was left for training. From the training data, 10% was reserved as a validation dataset to evaluate the model’s performance while training, which will help the model avoid problems like overfitting, a scenario where the model performs well during training but poorly when applied to testing data.

3.3. Backpropagation and Backpropagation Through Time

The universal approximation theorem states that a neural network with sufficient neurons and appropriate activation functions can approximate any continuous function [25]. The neural network is a set of interconnected layers containing a vector of neurons. The vision of machine learning is to approximate desired functions by learning the best weights that map the input to the output. The main challenge is training the model and updating the weights to optimize the model’s mathematical function. The most popular technique for training is backpropagation. Typically, when data is fed into the neural network, it flows sequentially from the input layer to the hidden layer and finally to the output layer. The output is then compared to the ground truth (the true data). The loss is a function that mirrors the difference between our model expectation and the true data and is computed.

The backpropagation adjusts the weight of the whole network starting from the output layer and propagates back into the input layer to reduce the loss function (reduce the error). As the training goes back through each layer and reaches the recurrent layer with a self-loop, the recurrent layer should be unfolded [26]. Thus, the backpropagation method must go through every step to train the inner weights. To achieve this, the G-CTRNN uses the Backpropagation Through Time (BPTT) concept to adjust the internal weights and the time constant. In the continuous-time domain, BPTT unfolds the recurrent neural network over time by discretizing it into small time steps and computing gradients through each time step to update weights using the chain rule. We chose BPTT because it is the standard approach for training continuous-time recurrent models where hidden states depend on past temporal dynamics, which is central to G-CTRNN operation.

To train the G-CTRNN using the BPTT, it is necessary to solve the associated ordinary differential equations (ODEs). One commonly used numerical method for solving ODEs is the Euler method [27], derived from the Taylor series. The G-CTRNN Euler representation was implemented using the TensorFlow library 2.8. TensorFlow is esteemed for its modern design, reliability, and adaptability [28]. In this implementation, the TensorFlow engine converts the network to a static computation graph before training, allowing full GPU acceleration and parallelism compatibility [29]. Figure 4 shows the steps to implement the forward and backward passes in TensorFlow for the G-CTRNN. These passes are tightly coupled in TensorFlow, but we separated them in the figure to illustrate the implementation details more clearly. In these steps, the auto-differentiation engine in TensorFlow traces the computational graph over all unrolled time steps. Using the chain rule, gradients are computed for both synaptic weights and time constants. The built-in ADAM optimizer updates parameters, with modifications to accommodate time-constant tuning. The ADAM Optimizer employs adaptive learning rates to accelerate the convergence of the model. During the initial learning phase of the model, it commences with a random initialization [30]. The ADAM optimizer leverages a movement’s momentum to accelerate the progression consistently, leading to smooth convergences.

3.4. G-CTRNN Time Constant Training

Training the time constant in the G-CTRNN using the BPTT is challenging, as this parameter has a physical meaning and some restrictions, such as that it should be a positive number within a specific range. The regularization technique is one option to solve this problem by limiting the training time constant to account for this constraint. In this method, the parameter of interest, the time constant here, will be added to the loss function to force the training to minimize those parameters [31]. However, this method will not guarantee that the parameter (time constant) does not exceed a specific value. Alternatively, TensorFlow has a non-popular functionality to override the constraints of the parameters of each cell to have values from a specific distribution [32]. In this work, we adapted this method to limit the time constant value during the training. This constraint will use the MINMAX Norm, which will constrain the values of the weights of any layers to have the norm between a lower bound and an upper bound.

Finally, it is worth mentioning that our implementation adopts a single time constant per neuron to maintain architectural simplicity and facilitate hardware integration. While this approach effectively stabilizes training and ensures feasibility under physical constraints, modeling temporal complexity in some applications, such as healthcare, may benefit from extended architectures. Specifically, incorporating multiple time constants per neuron or introducing hierarchical time scales would allow simultaneous encoding of rapid signals (e.g., tremors) and slower trends (e.g., postural transitions). These architectural variations present a promising direction for future research to generalize G-CTRNN across multi-scale scenarios.

4. Result

4.1. Time-Constant Effects

This section presents the results of using the G-CTRNN for human activity classification. We built the G-CTRNN and compared it to the standard RNN. We also examine the importance of training the time constant parameter and its significant impact on the accuracy of the G-CTRN model. Toward this objective, we conducted different experiments. The initial investigation involved training the model by setting a time constant value equal to a constant value of one in all sixteen cells of the first layer, which is the standard practice in the literature. The time constant values were next initialized in the subsequent experiment using a uniform random distribution with a range of [0, 2]. Finally, for the first time, the time constant of each cell in G-CTRNN is trained using backpropagation through time. The models were trained for 50 epochs in all scenarios, and the G-CTRNN results were compared to the standard RNN.

The RNN model shows poor performance with inconsistent accuracy as the number of epochs increases despite all the models using the same optimizer and starting with the same learning rate. The G-CTRNN model with the time constant set to one shows an accuracy of 68%, and the model successfully converged smoothly throughout 50 epochs. The G-CTRNN model had a better learning opportunity when the time constant was randomly initialized (resulting in varying time constant values for each cell). This is because different time constant values imply different response speeds for each cell, essential for extracting knowledge from the sequence data. The accuracy in this scenario improved to 76%, as depicted in Figure 5. Finally, an accuracy of 81.9% was achieved when the G-CTRNN time constants were included in the training. In summary, Figure 5 shows that G-CTRNN exhibits smoother convergence and improved stability across epochs compared to traditional RNNs, which suffer from inconsistent learning and catastrophic forgetting. The ability to adjust time constants dynamically contributes to G-CTRNN’s robust and efficient convergence behavior. Finally,

Table 1. Different G-CTRNN configuration runs and results of ten independent runs.

G-CTRNN Model	Accuracy %
Static time constant equals one	68.2 ± 0.6
Random-initialization time constant	76.1 ± 2.1
Training the time constant	81.9 ± 1.4

summarizes our findings concerning the different time-constant scenarios. All models started with the same learning rate to ensure a fair comparison: 0.001, the default value for ADAM algorithms in Keras 2.8.

In addition to the time-constant variations, we conducted internal ablation studies to validate key design choices. Removing the MINMAX constraint on the time constant resulted in numerical instability and occasional divergence during training, confirming the necessity of enforcing physical bounds. Similarly, using standard discrete-time backpropagation instead of continuous-time BPTT led to significant performance degradation and failure to converge when training the time constant, underscoring the importance of using continuous-time gradient propagation for temporal consistency and parameter sensitivity.

Finally, it is worth noting that while our experimental comparison centers on traditional RNNs due to their structural similarity to the G-CTRNN, we note that other lightweight models such as Gated Recurrent Units (GRUs), Echo State Networks (ESNs), and Spiking Neural Networks (SNNs) have also been proposed for efficient temporal modeling. GRUs simplify gating mechanisms to reduce parameter count, ESNs offload learning to the readout layer but require careful reservoir tuning, and SNNs operate with biologically inspired spike timing but often demand event-based hardware. In contrast, G-CTRNN offers a compact continuous-time formulation amenable to analog or MEMS-based hardware, providing a balance between trainability and low-power implementation. Future work will expand experimental comparisons to include these models.

Full evaluation metrics, including confusion matrices, sensitivity, and specificity for each classified activity across all models, are provided in Appendix A for further reference.

4.2. A Parameter Size Comparison Between RNN and G-CTRNN

The G-CTRNN model is a sequence of layers. Each layer is a set of cells, and each cell has different parameters that control the behavior of those cells. This section compares the different G-CTRNN and RNN models regarding parameter count and accuracy (

Table 2. Comparison of G-CTRNN and RNN parameter sizes and accuracies of ten independent runs.

G-CTRNN Model (Recurrent Cells)	Network Parameters Size	Accuracy %
G-CTRNN (16)	572	81.9 ± 1.4
RNN (32)	1548	81.4 ± 1.2
G-CTRNN (4)	156	71.8 ± 1.1
RNN (16)	556	73.1 ± 0.9

). G-CTRNN reached an accuracy of about 82% with a total number of 572 parameters. The RNN reaches 73%, using almost the same number of parameters. To reach an accuracy close to the RNN accuracy, the G-CTRNN has only four neurons in the first layer. In this case, the G-CTRNN parameters are only 152, about 27% of the RNN parameters (less than one-third of the parameters!). The subsequent inquiry in the investigation pertains to determining the requisite number of parameters for the RNN to achieve an accuracy of 81.9%. RNN, with a parameter count of 1,548, achieved an accuracy of 81.4%, which falls short of the performance exhibited by the G-CTRNN with only 572 parameters. Therefore, the G-CTRNN, while having just 38% of the parameters, showed superior performance compared to the RNN.

4.3. Computational Overhead

Including continuous-time dynamics in G-CTRNN adds considerable computational load during training. Solving differential equations with Euler integration, combined with gradient-based optimization of the time constant under parameter constraints, lengthens the per-epoch training time. In our experiments, training G-CTRNN for 50 epochs on a machine equipped with an Intel Xeon CPU and 16 GB RAM took about 6.5 h, while a comparable RNN was trained in under 1 h (approximately 56 min) on the same setup. This reflects a 7× increase in training time due to the overhead of resolving continuous dynamics and enforcing parameter constraints. However, during inference, the target physical implementations—such as a MEMS-based or analog hardware system—will naturally solve the differential equations in real time through their intrinsic dynamics. Consequently, the computational burden at deployment shifts from software-based simulation to native physical processes, allowing for energy-efficient, real-time operation on edge devices.

5. Conclusions

In this paper, we introduced G-CTRNN, a novel gradient-based training framework for Continuous-Time Recurrent Neural Networks (CTRNNs), designed to enable efficient learning on hardware-constrained platforms such as MEMS-based or analog signal processors. Unlike conventional RNNs or heuristic-based CTRNNs, G-CTRNN leverages differentiable programming and constraint-aware optimization to train both synaptic weights and neuron time constants in a biologically inspired, real-time-compatible fashion.

We evaluated G-CTRNN on the WISDM dataset, a widely used benchmark for human activity recognition (HAR) from wearable sensor data. The results demonstrate that G-CTRNN outperforms traditional RNNs in terms of accuracy, convergence stability, and parameter efficiency—qualities that are particularly important for healthcare applications such as elderly monitoring, activity-based diagnosis, and remote rehabilitation support. G-CTRNN also shows strong promise for low-power operation, making it well-suited for edge deployment in wearable or mobile health systems.

These results suggest that G-CTRNN is a promising model for real-time embedded AI, assuming physical implementations (e.g., MEMS or analog hardware) that can passively solve the differential dynamics without iterative digital computation. While the current simulation uses Euler integration, which incurs computational overhead, such constraints must be mitigated in native hardware where time dynamics unfold naturally.

Future work will focus on hardware-software co-design for analog and MEMS implementation of G-CTRNN, robustness to sensor noise, and clinical validation in healthcare environments. Overall, this work contributes a significant step toward enabling real-time embedded AI in next-generation healthcare monitoring systems through the integration of machine learning, edge computing, and analog signal processing. Finally, while this work constrains model parameters to remain hardware-compatible, future research will incorporate direct optimization for energy consumption, fabrication variability, and real-world MEMS dynamics during training, enabling even tighter integration between model behavior and physical implementation.