4.1. General Approach
Recently, the advent of IoT technologies has redefined health care analytics. Abnormal measurements play a prominent role in healthcare analytics [
27,
28]. An IoT node transmits the data through a Bluetooth connection. At that moment, a doctor or a person responsible for the patient can observe the temperature in correlation with the activity and judge the health situation.
Wearable activity trackers can provide unique insights and contribute to the early detection of COVID-19 cases [
29] by monitoring patients using wearable devices. The ideal combination can be achieved with the wearable device and the usage of IoT for the health monitoring of people in isolation [
30]. This study provided an activity recognition tool using a 9-axis IMU, which allows the variation of signals to reflect the human activity. The approach focuses on distinguishing between regular activity, so the CDA is proved effective.
Using several temperature sensors allowed us to track the temperature fluctuations because fever is one of the key symptoms of COVID-19. The contact thermometer was configured appropriately to monitor the user’s body temperature with great fidelity. It is deemed worthy to mention that in the beginning of the monitoring, the sensor required a few seconds to calibrate. The standard body temperature threshold was automatically adapted depending on the physical exercise status. An infrared thermometer is an indispensable sensor that quickly finds the users’ temperature but lacks precision [
31]. This type of recognition allows the observer to keep track of the users’ temperature variations and find out if the temperature is due to physical exercise [
32].
On the contrary, the ambient sensor HTS221—which was embedded in the board—responsible for measuring room temperature and humidity is necessary to assure the quarantined person’s suitable living conditions and restrain COVID-19 [
33]. Therefore, in our design, there was a combination of three temperature sensors, each one checking the different types of temperature (touch/I.R./ambient) and humidity, as well as continuously checking the AmT (Ambient Temperature) and the AmH (Ambient temperature) surrounding the user. All the abovementioned data were shown to the user via the OLED display. Tracking of the ambient parameters is also important because if one is under supervision, it is mandatory to adjust the environment variables to halt the spread of the virus, in the particular values between 20–60% of humidity and 20–24 °C concerning the ambient temperature [
34].
By analyzing the data measurements between room conditions and body state, we could improve the monitoring capabilities of infected people without needing to access large databases and without large training datasets. Besides that, we then send warnings via the OLED display and the BLE communicate when the body temperature exceeds the allowed threshold. The importance of the proposed system is that it combines simple logic, a small size, and multiple functionalities. This system features upgradeability, utilizing machine learning models. It can be customized or progressed to a more complex and multipurpose system that uses the rest of the sensors embedded on the Arduino Nano 33 BLE sense board.
4.2. Benefits of the Approach
We developed a Continuous Displacement Algorithm (CDA) which computes the number of observations surpassing the physical activity threshold, inciting automatic update of the temperature threshold. The accelerometer inside the wearable device provided the acceleration value. The extremes were calculated based on the acceleration variation as the input of the CDA model. The average human body temperature ranges from 36.8 to 37.0 °C since each person’s body temperature is different with negligible fluctuations. However, during physical exercise, the temperature hovers around 37.5 °C [
35,
36]. Additionally, ambient temperature and humidity are critical to impede the virus diffusion. The ambient temperature (AmT) should be around 20–24 °C, and the Ambient Humidity (AmH) approximately 20–60%. In this project, a wearable device based on Arduino nano 33 BLE sense’s embedded ambient sensors, two other temperature sensors were added to this device to obtain reliable readings, namely, an MLX90614 infrared thermometer and an MCP9808 high-accuracy contact thermometer [
37].
The primary protocols used to transmit the information are BLE and were used as peripheral, serial transmission to an OLED display attached to the wearable device [
38]. The measurements were available to the user at any time. They could simply connect to a Bluetooth central application like LightBlue primarily used in this project for data decoding) or nRF Connect to access the available data [
39,
40]. Altogether, the system is based on thermometer and accelerometer readings. The objectives include the smooth combination of contact—and non-contact—ambient thermometers to update the temperature and acquire accurate readings. The overall purpose of this device is to provide warnings if the body temperature is above the permissible limit, which the CDA detects.
Besides the simplicity and accuracy, an additional benefit of this design is distant patient supervision. In future applications, it is highly upgradeable, and it can perform pulse oximetry which can assist in notifying supervising staff or patients of the help they need and acquire accurate information about their current condition. Furthermore, one of the most significant benefits is that the Arduino Nano BLE 33 Sense can implement machine learning models with TinyML or TensorFlow [
41,
42].
Overall, this paper is an alternative approach to the system described in [
43], using similar principles for acquiring and handling the data. It offers excellent future upgradeability while maintaining power efficiency—one of the critical features of the BLE 5.0 protocol.
4.3. Continuous Displacement Algorithm
To recognize human activity, it is mandatory to collect and process data provided by the IMU unit. In this case, the acceleration magnitude (Amag) was used to calculate the difference between its two-consecutive samples (Δαmag) with Amag, Amag-1, and compute the average between them, the previous and the current readings, for activity recognition.
where x, y, z are the three axes of the accelerometer.
Figure 3,
Figure 4 and
Figure 5 show the behavior of Amag and |ΔAmag| during three activity conditions:
The Acc signal acquisition was carried out for 1 min in real-time. The signal can only react to the apparent motion of the body. Under stillness, no extremes were found, and the accelerometer maintained a constant value without any extremes, so the |Δαmag| maintained a constant value without unnecessary interruptions. Hence, the |Δαmag| was almost 1 at all times. Regular activities can cause sudden variations, but it is easy to realize that the signal behavior is random with tiny |Δαmag| at low frequencies. Likewise, if the user performs physical exercises, the Δαmag frequently varies with high repetitive extremes due to numerous motions that cause and |Δαmag| fluctuations repetitively with similar values. To distinguish between normal activities and training, the CDA counts the number of extremes per a specified period (20 s) to decide whether the person is exercising. Additionally, pertinent literature indicates that the body’s temperature returns to standard levels approximately 20 min after exercise. Therefore, the timer is to count that period and adjust the temperature threshold accordingly at 37.5 °C, which is the maximum body temperature during exercising.
The CDA flowchart, in
Figure 6, presents a comprehensive overview of the implemented algorithm. We gathered accelerometer data using the Vector Equation and stored the data into an array of 100 elements each time, calculating the average between two vector elements each time. After the data were stored into another array, a threshold was generated, computing the total average of the 100 elements. Values above that threshold were counted as local extreme values. If the local extremes were above a set value acting as a threshold, corporal activity was detected and adjusted the points accordingly. The following equation calculates the average vector:
where:
a is the capacity of an array ranging from 0 to 99 and Average Vector[
a] is an array of 100 elements which calculates the avg of the vector[
a] which is calculated by the ΔAmag formula.
This process was repeated for 20 s each time. Suppose the number of spikes we found on that time interval is above the spike threshold. In this instance, the exercise’s Boolean variable is shifted from 0 to 1. Afterward, we adjust the temperature threshold from 36.8 to 37.5 °C which is the temperature threshold that comes into place if someone is exercising. This value is held for 20 min. The data acquisition is accomplished by using the device’s serial port to transmit data and a computer (in combination with a serial monitor) to receive data. In summary, if they are enough extremes in a specified period and if the corresponding extreme values are above the threshold, the flag is shifted from 0 to 1. Consequently, the temperature cut-off value is adjusted from 36.8 °C to 37.5 °C.
4.4. Deep Learning Analysis
Subsequently, we performed classifications and determined the condition of the user concerning their body temperature. Explicitly, we fed the measurements of the touch temperature sensors to identify an increase in the body temperature potentially caused by pathogenesis, thus signifying fever. The models employed for this task were unidimensional Convolutional Neural Networks. CNNs are a class of deep learning neural networks that display excellent capabilities in pattern recognition schemes. Their ability to extract regional features enables researchers to deal with complex tasks that are not possible with classic ANNs. Although, our signals stemmed from one device, and thus we did not possess a plethora of samples; we were prompted to select unidimensional CNNs as our classifier to provide proof of feasibility for our application in this task.
This selection also delineates our future directions regarding the simultaneous acquisition of multiple signals from interconnected devices in the context of family members wearing this device and obtaining feedback in real-time concerning their condition and relatives. In this vein, these networks present excellent scaling capabilities, thus rendering them great candidates to constitute the foundation for our analysis. Furthermore, these networks were also employed in our previous work [
44], where they demonstrated significant efficacy in classifying the condition of vessel equipment. Therefore, it is deemed necessary to use methodologies established in other domains and determine their transferability. The primary functionalities of CNNs are explicated in detail, as follows:
Feature extraction: First, the convolution is characterized as a linear element-wise multiplication as a small array called the kernel glides through the input tensor of the layer and convolves the value of each time step. Equation (3) delineates this operation.
The emerging output tensor was fed into a nonlinear activation function. Recently, the rectifier linear unit (ReLU) has started to gain popularity, as it has been evinced capable of yielding remarkable results while also solving gradient problems related to other popular activation functions of the past, such as the sigmoid and the tanh.
Classification: The output feature map of the last convolution layer was converted into a vector, which was input into a traditional Feed-Forward Neural Network (FFNN) carrying out the assignment of the samples to the output labels.
Loss: The main objective of the optimization procedure is the minimization of the loss function. The utilized loss function was concluded based on the nature of the problem at hand. Explicitly, for our application, the Categorical Cross Entropy was utilized in the present study. Afterward, we assumed a dataset with M labels and N observations, and the vector of ground truth measurements is denoted as
. Similarly, the predicted values are denoted as
. Accepting CCE as our loss function then gives:
Subsequently, the backpropagation succeeded the forward pass. The gradient, namely the partial derivative of the loss, was calculated concerning the model’s output. Through this computation, the model was capable of updating its learnable parameters, namely weights and biases, achieving convergence of the inferred values and actual measurements, hence augmenting the model’s accuracy. Mathematically, we can formulate the aforementioned procedure as follows:
where
p represents any learnable parameter and α denotes the learning rate of the optimizer of the model.
For our application, we had continuous time-series signals pertinent to the variables monitored by the watch’s sensors. We had available 2 h worth of data with a 1 Hz sampling frequency, accumulating to approximately 7200 data points, corresponding to 1 h (3600 data points) with the being user healthy and another hour with the user being febrile.
Table 2 summarizes the data acquisition for the N.N. classification task and the user’s physical condition while wearing the watch.
For the needs of our application, it was essential to parse the continuous-time series signals and create distinct segments that shall constitute the samples fed to the CNN. The dataset segmentation was carried out concerning a preconditioned timeframe, as illustrated in
Figure 7, where the segmentation of an arbitrary signal was divided into 10 sections. Explicitly, for our application, a 5-min segment horizon was employed. Additionally, it is worth noting that an overlap of 75% between adjacent time windows was applied to maximize the number of available samples fed into the neural network. The 5-min input window with the 75% overlap yielded 250 samples across the two possible conditions of the user’s health (healthy and patient). These two states were equivalently represented in the dataset, as derived from the equal amount samples, respectively.
Table 3 demonstrates the distribution of the samples to the training, validation, and test subsets for the neural net, respectively.