Applying Machine Learning in Cloud Service Price Prediction: The Case of Amazon IaaS

Round 1

Reviewer 1 Report

RESUME:
Authors propose to apply machine CatBoost algorithm learning (ML) to predict the price for the Amazon EC2 reserved instance type. For that, authors fed the ML model with price historical data, from 2016 to 2022, and used standard metrics such as mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) to evaluate the model's accuracy. Based on the author's study, findings suggest that prediction based on data within more recent time periods can yield more accurate price projections.


NOTES:
- Authors explain why they decided to choose CatBoost algorithm, and evidenced its main characteristics
- The methodology followed is described in sufficient detail
- References list many of conference papers, and it is recommended to cite indexed journals instead
- When authors compare actual and predicted prices for the Memory Optimized, and mention "The discrepancy between predicted and actual prices could be attributable to a number of factors, such as the complexity of pricing patterns unique to this instance family or the presence of external factors not adequately accounted for by the model", it's important that authors further investigate the issue, to understand what the root causes or limitations of the model/algorithm are and what can be done to improve the solution
- Authors need emphasize why they decided to chose z-score to identify outliers
- Ability to predict the price of instances does not mean a user will pay less in the end; let's consider the case: based on the predicted price, a user decides to buy a set of instances in a certain region and take the upfront option; latter, the user can decide to sell their idle reserved instances; the moment of selling can be more or less beneficial according to the region (or any other factors);

- The weakest point of this study is that authors do not perform any comparison with (some) state-of-the alternatives. Since authors observed that old data seems to introduce noise in the module, negatively affecting the prediction accuracy, it would be interesting to know what is the impact of old data in some other ML algorithms. In this regarding, this introductory study needs further investigation. Nevertheless to provide answers to some questions/limitations found.

- Correct some typos (ex.: "these these remain inactive", "Training is carrying out", "resource prediction using was carried", "we not that in 2016", "MAPE on the other measures", etc.)

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article introduces a machine-learning approach based on the CatBoost algorithm to provide a price prediction model for the reserve instance market. The analysis is based on historical data from Amazon Web Services from 2016 to 2022. The idea is well presented, and the contributions are clear; however, I have the following comments.

-          Most of the considered related works are old studies; the authors are recommended to consider recent proposals.

-          The authors inserted many unnecessary citations. Subsection 2.2 should be reconsidered, and all unnecessary citations should be excluded.

-          A comparison table with the existing proposals and the proposed work should be introduced to present the novelty of the work.

-          What are the considered features in the ML algorithm? Please clarify.

-          What is the computation cost of the proposed model?

The English language is fine; the article requires quick proofreading.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

All comments have been addressed.

Minor editing is required.