A Flexible IoT Stream Processing Architecture Based on Microservices

Round 1

Reviewer 1 Report

This is a very thorough research study that explains a flexible IOT stream processing architecture based on microservices. I read the manuscript with great interest and believe its topic is important and relevant. The authors performed a careful and thorough review of the literature, as the section was very informative and substantial. Appropriate theoretical framework was applied. I found the methodological part to be well justified and reasonable for this type of analysis. Although the manuscript is overall well-written and structured, it might benefit from additional spell/language checking. However, I have some comments which I would like to be addressed before the acceptance of this paper.

Major comments

1. Change the heading title of section “5” Related Work and Conclusions to conclusion and add the relevant work in a new section called related work. In the conclusion section it’s unnecessary.
2. The section 2. The Microservices Architectural Style and Java OSGi and section 3. A Microservices Architecture for Adaptive Real-time IoT Stream Processing are too big. Divide these into smaller sub sections or explain their significance, issues, solutions etc.  
3. There are several interesting papers that look into real time data processing in terms of healthcare and E-commerce. The work of Alfian et al. and Ali, Farman et al. has some interesting implications that you could discuss in your introduction and how it relates to your work.
4. Alfian et al. "Customer behavior analysis using real-time data processing." Asia Pacific Journal of Marketing and Logistics (2019).
5. Ali, Farman, et al. "An intelligent healthcare monitoring framework using wearable sensors and social networking data." Future Generation Computer Systems 114 (2020): 23-43.
6. Alfian, Ganjar, Muhammad Syafrudin, Muhammad Fazal Ijaz, M. Alex Syaekhoni, Norma Latif Fitriyani, and Jongtae Rhee. "A Personalized Healthcare Monitoring System for Diabetic Patients by Utilizing BLE-Based Sensors and Real-Time Data Processing." Sensors 18, no. 7 (2018): 2183.

I believe that integrating such work would make the paper even more interesting.

I look forward to read the final version.

Author Response

Dear Reviewer,

 first of all thanks for your report.

We have modified the manuscript in order to put more emphasis on motivations, goals, and novel contributions in the introduction. Furthermore, we have restructed the different sections (introducing subsections and moving all preliminaries on technologies in a dedicated section), and added references suggested by one reviewer.

Concerning motivations and contributions (as remarked in the revised manuscript):

In our work we focused on the design principles and on a p.o.c. based on a particular microservices technology (Java OSGi) for a  flexible and adaptive solution for integrating real-time stream processing functionalities into IoT platforms. One of the main goal is to provide services that can dynamically be relocated to different endpoints (core and edge layers) of an IoT platform.

The innovative aspect of our solution, with respect to common IoT platforms, consists in limiting the expressive power for defining stream processing rules to a predefined set of templates, in favour of a much more flexible and dynamic deployment model. This is needed in order to provide an abstraction layer that can be mapped to different types of stream processing engines independently from the underlying IOT layer.

To achieve the desired flexibility, the proposal architecture has been designed following a microservices architectural pattern.
In this context:

-We introduce the notion of $\mu$-service, a stream processing unit that can indifferently be allocated on the edge- and core-level. 

-To implement the different roles of  $\mu$-services, we propose a Reference Architecture based on Proxy, Adapter and Data Processing  $\mu$-services for dealing with real-time stream processing in a very flexible way. 

-To abstract away from the underlying stream processing engine and IoT layer (edge/cloud) and provide interoperability, portability and compositionality  our service definition languages consists of the following ingredients:

--- a configuration language based on the JSON format, a widely adopted language for facilitating systems interoperability

---  a rule-based  query language with  filter operations that are common to the most used stream processing query languages  to ensure portability from engine to another.

--- connectors that can be used in order to pipeline  filters defined in our query language to provide compositionality when creating complex filters on top of simpler ones.

As described in the related work section, we believe that the above mentioned points are distinguished features w.r.t. approaches proposed in other existing solutions.

Concerning benchmarks, we believe that it is an important and interesting direction. However our paper is focused on the reference architecture details. For complete benchmarks it would be necessary to engineer our solution transforming it into a fullfledge component of Senseioty. This work requires additional work that is part of our collaboration with Flairbit.

Thanks again for your attention,
 The authors

Reviewer 2 Report

1. Results: Recommend to be Major revisions

This paper proposes an adaptive microservices architecture for an IoT platform able to integrate real-time stream processing functionalities in a dynamic and flexible way, with the goal of covering the different real-time processing requirements that exist among the different levels of an IoT application. A preliminary prototype has been implemented as proof of concept of the feasibility and benefits of the proposed architecture. The solution of the proposed approach meets the need to introduce different real-time stream processing technologies in IoT platforms, in order to offer streaming analytic functionalities on the different architectural levels of IoT applications.

It is with some merits for Information. However, it requires major revisions.

Firstly, the abstract is disorganized, it should be refined to precisely illustrate what authors have done in this paper within 200 words.

Secondly, for Section 1, authors should provide the comments of the cited papers after introducing each relevant work. What readers require is, by convinced literature review, to understand the clear thinking/consideration why the proposed approach can reach more convinced results. This is the very contribution from authors. In addition, authors also should provide more sufficient critical literature review to indicate the drawbacks of existed approaches, then, well define the main stream of research direction, how did those previous studies perform? Employ which methodologies? Which problem still requires to be solved? Why is the proposed approach suitable to be used to solve the critical problem? We need more convinced literature reviews to indicate clearly the state-of-the-art development. by the way, the contributions should also be refined to enhance the novelty.

For Section 2, authors should introduce their proposed research model/framework more effective, i.e., some essential brief explanation vis-à-vis the text with a total research flowchart or framework diagram for each proposed algorithm to indicate how these employed models are working to receive the experimental results. It is difficult to understand how the proposed approaches are working. For the employed data set, please provide more details illustration.

For Sections 3 and 4, authors should also conduct some statistical test to ensure the superiority of the proposed approach, i.e., how could authors ensure that their results are superior to others? Meanwhile, authors also have to provide some insight discussion of the results.

Author Response

Dear Reviewer,

 first of all thanks for your report.

We have modified the manuscript in order to put more emphasis on motivations, goals, and novel contributions in the introduction. Furthermore, we have restructured the different sections (introducing subsections and moving all preliminaries on technologies in a dedicated section), and added references as suggested in one report (they are related to possible application domains of this kind of platforms and possible extensions with dynamic services).

Concerning motivations and contributions (as remarked in the revised manuscript):

In our work we focused on the design principles and on a p.o.c. based on a particular microservices technology (Java OSGi) for a  flexible and adaptive solution for integrating real-time stream processing functionalities into IoT platforms. One of the main goal is to provide services that can dynamically be relocated to different endpoints (core and edge layers) of an IoT platform.

The innovative aspect of our solution, with respect to common IoT platforms, consists in limiting the expressive power for defining stream processing rules to a predefined set of templates, in favour of a much more flexible and dynamic deployment model. This is needed in order to provide an abstraction layer that can be mapped to different types of stream processing engines independently from the underlying IOT layer.

To achieve the desired flexibility, the proposal architecture has been designed following a microservices architectural pattern.
In this context:

-We introduce the notion of $\mu$-service, a stream processing unit that can indifferently be allocated on the edge- and core-level. 

-To implement the different roles of  $\mu$-services, we propose a Reference Architecture based on Proxy, Adapter and Data Processing  $\mu$-services for dealing with real-time stream processing in a very flexible way. 

-To abstract away from the underlying stream processing engine and IoT layer (edge/cloud) and provide interoperability, portability and compositionality  our service definition languages consists of the following ingredients:

--- a configuration language based on the JSON format, a widely adopted language for facilitating systems interoperability

---  a rule-based  query language with  filter operations that are common to the most used stream processing query languages  to ensure portability from engine to another.

--- connectors that can be used in order to pipeline  filters defined in our query language to provide compositionality when creating complex filters on top of simpler ones.

As described in the related work section, we believe that the above mentioned points are distinguished features w.r.t. approaches proposed in other existing solutions.

Concerning benchmarks, we believe that it is an important and interesting direction. However our paper is focused on the reference architecture details. For complete benchmarks it would be necessary to engineer our solution transforming it into a fullfledge component of Senseioty. This work requires additional work that is part of our collaboration with Flairbit.

Thanks again for your attention,
 The authors

Round 2

Reviewer 1 Report

All my comments are addressed; hence manuscript is accepted.

Reviewer 2 Report

Authors have completely addressed all my concerns.