Abstract
Tiny machine learning (tinyML) involves the application of ML algorithms on resource-constrained devices such as microcontrollers. It is possible to improve tinyML performance by using a meta-learning approach. In this work, we proposed lightweight base models running on a microcontroller to predict air pollution and show how performance can be improved using a stacking ensemble meta-learning method. We used an air quality dataset for London. Deployed on a Raspberry Pi Pico microcontroller, the tinyML file sizes were 3012 bytes and 5076 bytes for the two base models we proposed. The stacked model could achieve RMSE improvements of up to 4.9% and 14.28% when predicting NO2 and PM2.5, respectively.
1. Introduction
Tiny machine learning is a rapidly growing field of machine learning [1], targeting resource constrained devices such as microcontrollers to optimise energy efficiency and latency [2]. This work aims to improve tinyML models running on a microcontroller using a meta-learning method for predicting hourly air pollutants. With a stacking ensemble architecture, the meta-learner learns from each base model and improves the final prediction. We designed lightweight models for base members and selected a simple least squares linear regression as a meta-learner.
2. Materials and Methods
Air quality data from five monitoring sites in the Greater London area between 1 July 2019 and 13 December 2021 were collected using the Openair framework [3]. These sites were London Bexley (BEX), London Westminster (HORS), London N. Kensington (KC1), London Eltham (LON6), and London Marylebone Road (MY1). Seven features were selected as inputs (NOx, NO2, NO, PM2.5, modelled wind speed, wind direction, and air temperature) to predict two hourly pollutants (NO2 and PM2.5). This work used 80% of the data for the training set and 20% for the test set. All features were normalised to the range of [0,1], and missing values were filled using a multivariate imputer function provided by scikit-learn.
Figure 1a shows the stacking ensemble concept. The stacking architecture consisted of two base models in Level-0 and a least squares linear regression as the meta-learner in Level-1. Figure 1b shows the proposed base models, consisting of dense layers as Base-1 and a combination of 2-D CNN and dense layers as Base-2. The rectified linear activation function (ReLU) was used in all the layers, except the output layers. In this work, we used the TensorFlow framework to create the ML models and deployed the lite versions to a Raspberry Pi Pico microcontroller without performing quantisation.
Figure 1.
(a) Stacking ensemble concept; (b) Base-1 and Base-2 model architectures.
3. Discussion
As reported in Table 1, the stacked model could reduce RMSE values obtained by the individual base models. The linear approximation by the meta-learner found the best way to combine the Level-0 members’ outputs by minimising the residual sum of squares between the input and target sets. Converting the ML models to lite versions could reduce the model size by about 83% and 77% for Base-1 and Base-2, respectively. The deployed tinyML model sizes were 3012 bytes for Base-1 and 5076 bytes for Base-2, considered lite enough even without performing any quantisation techniques. The lite version’s accuracy was not degraded compared to that of the original model.
Table 1.
RMSE values of the base and stacked models obtained from the test set.
Author Contributions
Conceptualisation, I.N.K.W.; methodology, I.N.K.W., S.A.F. and J.W.G.; software, I.N.K.W.; data collection, I.N.K.W.; data analysis I.N.K.W.; writing—original draft preparation, I.N.K.W.; writing—review and editing, I.N.K.W., S.A.F. and J.W.G.; and supervision, S.A.F. and J.W.G. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported in part by the Indonesia Endowment Fund for Education (LPDP), Ministry of Finance, Republic of Indonesia, under grant number Ref: S-1027/LPDP.4/2019.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The data presented in this study are available upon request from the corresponding author.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Shafique, M.; Theocharides, T.; Reddy, V.J.; Murmann, B. TinyML: Current Progress, Research Challenges, and Future Roadmap. In Proceedings of the 2021 58th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 5–9 December 2021; pp. 1303–1306. [Google Scholar]
- Giordano, M.; Baumann, N.; Crabolu, M.; Fischer, R.; Bellusci, G.; Magno, M. Design and Performance Evaluation of an Ultralow-Power Smart IoT Device with Embedded TinyML for Asset Activity Monitoring. IEEE Trans. Instrum. Meas. 2022, 71, 1–11. [Google Scholar] [CrossRef] [Scilit]
- Carslaw, D.C.; Ropkins, K. Openair—An R Package for Air Quality Data Analysis. Environ. Model. Softw. 2012, 27–28, 52–61. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).