Performance-Based Classification of Users in a Containerized Stock Trading Application Environment Under Load
Round 1
Reviewer 1 Report
Comments and Suggestions for Authors The study proposes a performance-based user classification framework, validates the feasibility of distinguishing behavior classes from system logs, and explores future optimizations using graph databases, deep learning, and large language models. However, the current version suffers from redundant expressions, with repetitive descriptions of experimental procedures in multiple sections. Moreover, it lacks graphic-text integrated analysis—the only illustration Fig. 1 is blurred due to low resolution. 1、All illustrations should be inserted in vector format. Fig. 1 suffers from low resolution, with blurred texts and data flows.2、The formats and typesettings of Tables 3 and 4 are too messy, and several columns of data are crowded in the front part of the tables.
3、The introduction needs deeper critical analysis. When comparing Docker performance studies, highlight their lack of user behavior classification, and strengthen empirical data citation on latency impacts in high-frequency trading scenarios.
4、The current experimental analysis lacks visual qualitative analysis. It is recommended to relocate relevant visual contents from the Appendix to the corresponding analysis sections in the main text, accompanied by interpretative descriptions.
Author Response
Thank you for this valuable remarks.
New content is marked in red in the article text.
------------------------------------------------
Comments 1:
However, the current version suffers from redundant expressions, with repetitive descriptions of experimental procedures in multiple sections.
Comments 2:
Moreover, it lacks graphic-text integrated analysis—the only illustration Fig. 1 is blurred due to low resolution.
Response 1 and 2:
We have included a diagram visualizing the user classes to improve understanding (appendix) - Added action diagrams for user classes. The web system transaction diagram was added.
------------------------------------------------
Comments 3:
All illustrations should be inserted in vector format. Fig. 1 suffers from low resolution, with blurred texts and data flows.
Response 3:
We added the figure in EPS format.
------------------------------------------------
Comments 4:
The formats and typesettings of Tables 3 and 4 are too messy, and several columns of data are crowded in the front part of the tables.
Response 4:
Table 3
For better readability, the third table has been divided into test groups that differ in specific parameters.
Table 4
Explanations of the parameters of the fourth table are included in:
\begin{itemize}
\item \textbf{Number of users of each class} - We denote by $A$ the count of ActiveUsers, $NA$ the count of ReadOnly (non-active) users, and $AZA$ the count of AnalyticalUsers (active with analysis). In some tests we varied the total number of users (with a fixed ratio of types), while in others we changed the mix of user types for a fixed total.
\item \textbf{Time between transactions ($T_{tr}$)} - This is the delay between successive trade-matching cycles executed by the Celery worker. Increasing this interval means trades are processed less frequently (allowing more orders to queue up). We experimented with different $T_{tr}$ values to see how batching trades versus real-time processing affects performance.
\item \textbf{Time between user requests ($T_{rq}$)} - This parameter controls the think-time or delay between consecutive actions by the same simulated user. A smaller $T_{rq}$ means users hammer the system with requests continuously, whereas a larger $T_{rq}$ gives each user some idle time between actions, emulating more human-like pacing.
\item \textbf{Number of transaction worker containers ($K_{tr}$)} - In some scenarios, we scaled the number of Celery worker processes dedicated to transaction processing. This tests the effect of horizontal scaling of the backend processing on overall throughput and latency.
\end{itemize}
------------------------------------------------
Comments 5:
The introduction needs deeper critical analysis. When comparing Docker performance studies, highlight their lack of user behavior classification, and strengthen empirical data citation on latency impacts in high-frequency trading scenarios.
Response 5:
Thank you for pointing this out. We agree with this comment, but in this case our emphasis on behaviour-based classification.
Our work deliberately models three distinct user classes whose interactions stress the system in fundamentally different ways. We can use Docker a a system platform or not. In other case, we will also have system logs from the native system and we will classify them.
The rate of growth of the number of users is irrelevant. It was selected experimentally so as not to prolong and complicate the application testing process. Performance studies were not the content of the conducted research work.
------------------------------------------------
Comments 6:
The current experimental analysis lacks visual qualitative analysis. It is recommended to relocate relevant visual contents from the Appendix to the corresponding analysis sections in the main text, accompanied by interpretative descriptions.
Response 6:
Information from the appendix C moved to the section "The Confusion Matrices" and expanded to include conclusions.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe scientific research is relevant and up-to-date. The manuscript can be published after corrections are made. I have the following questions and suggestions to the authors:
1. What are the structural parameters of the AI models? It is good to describe the parameters of each of the models - for RF, number of trees and depth, etc., for MLP - number of layers and number of neurons in each layer, activation function, etc.
2. How are selected these parameters to ensure optimal performance of the respective model?
3. Why were nonlinear models not tested, for example, LinearSVC was chosen, and an SVM with a nonlinear kernel (usually RBF) could be used, which has better results?
4. What metrics were used to evaluate the effectiveness of the models? Which parameter do the percentages in Table 5 indicate?
5. Assuming that Table 5 shows classification accuracy, do you think 79% is a good value? When optimizing the parameters of the models, the classification accuracy should be significantly improved for some of them. Moreover, there is a large dataset available where neural networks have the best parameters. For example, if the appropriate number of layers and neurons is chosen for MLP, the percentage should be around and above 90%. You can do additional experiments in this direction.
6. In Appendix C, the tables should be formatted according to the Electronics standard, respectively, in all appendices, Figures, Tables, etc. should be labeled starting with “A”—e.g., Figure A1, Figure A2, etc. (https://www.mdpi.com/files/word-templates/electronics-template.dot)
Author Response
Thank you for this valuable remarks.
New content is marked in red in the article text.
------------------------------------------------
Comments 1:
What are the structural parameters of the AI models? It is good to describe the parameters of each of the models - for RF, number of trees and depth, etc., for MLP - number of layers and number of neurons in each layer, activation function, etc.
Response 1:
To improve transparency and reproducibility we have added a new Tabel with parameters.
------------------------------------------------
Comments 2:
How are selected these parameters to ensure optimal performance of the respective model?
Response 2:
Thank you for highlighting the need to justify our choice of hyper-parameters. We agree with this comment. All classifiers should tune with a x-fold stratified cross-validation grid search.
The 10-fold stratified cross-validation grid search was applied to every classifier to avoid over-fitting, but we don't present this in article.
We are sending the results in the attachment.
We added this information to the article.
------------------------------------------------
Comments 3:
Why were nonlinear models not tested, for example, LinearSVC was chosen, and an SVM with a nonlinear kernel (usually RBF) could be used, which has better results?
Response 3:
The following models were tried in classification:
1) Xgb.
2) ExtraTreesClassifier.
3) DecisionTreeClassifier.
4) KNeighborsClassifier.
5) RandomForestClassifier.
6) LinearSVC
7) MLPClassifier.
8) ExtraTreeClassifier.
9) BernoulliNB.
10) GaussianNB.
11) NearestCentroid.
12) RadiusNeighborsClassifier.
13) LinearDiscriminantAnalysis.
14) QuadraticDiscriminantAnalysis.
Seven were selected for further analyses presented in the article.
We tested this algorithm at an early stage and it was not selected. We re-verified the results.
with a nonlinear kernel (RBF) was not taken into consideration. We check it now:
'SVC_RBF': SVC(
kernel='rbf',
C=10,
gamma='scale',
class_weight='balanced',
max_iter=10_000,
random_state=RANDOM_STATE
)
and for test22 we got much lower accuracy:
130_A+NA+AZA_30s_600ms_3
SVC_RBF time: 618.5235815048218
precision recall f1-score support
WebsiteActiveUser 0.36 0.58 0.44 24912
WebsiteActiveUserWtihMarketAnalize 0.36 0.29 0.32 32956
WebsiteReadOnlyUser 0.00 0.00 0.00 9849
accuracy 0.36 67717
macro avg 0.24 0.29 0.25 67717
weighted avg 0.30 0.36 0.32 67717
We think about other models but training and fine-tuning takes time.
------------------------------------------------
Comments 4:
What metrics were used to evaluate the effectiveness of the models? Which parameter do the percentages in Table 5 indicate?
Response 4:
The title of the table 5 is: "User class classification accuracy using selected algorithms." - values reported in Table 5 correspond to overall accuracy.
------------------------------------------------
Comments 5:
Assuming that Table 5 shows classification accuracy, do you think 79% is a good value? When optimizing the parameters of the models, the classification accuracy should be significantly improved for some of them. Moreover, there is a large dataset available where neural networks have the best parameters. For example, if the appropriate number of layers and neurons is chosen for MLP, the percentage should be around and above 90%. You can do additional experiments in this direction.
Response 5:
The efficiency of 79% is not the best, but one of the factors is also the speed of operation (description added to the related works section). A real-time tool with an appropriate action scenario (production pipeline) focused on the speed of operation and correctness of user class detection is planned. In subsequent studies, it is planned to adjust parameters and select other methods.
We acknowledge that 79 % leaves room for improvement. Usually, the better accuracy required a few more training time and increased inference latency. Because our target scenario is an on-line decision aid with strict timing constraints, we prefer the 79-80 % solutions that execute within 20 ms per request. We have added this discussion to the article.
------------------------------------------------
Comments 6:
In Appendix C, the tables should be formatted according to the Electronics standard, respectively, in all appendices, Figures, Tables, etc. should be labeled starting with “A”—e.g., Figure A1, Figure A2, etc. (https://www.mdpi.com/files/word-templates/electronics-template.dot)
Response 6:
Information from the appendix C moved to the section "The Confusion Matrices" and expanded to include conclusions.
Formatting has been adapted to the style of the journal.
Author Response File: 
Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for Authors - Author to expand the introduction and system overview section to clearly set the objective to the reader. Considering the 1st paragraph of the Introduction section, please clarify "citeCharytanowicz2023" , add relevant references or modify the content accordingly.- This paper is based on a software experiment on a containerized stock trading system. Please expand on how these findings translate into specific, actionable optimization steps for platform designers that would strengthen the practical contribution of the paper.
- Please provide a comprehensive and detailed explanation of the algorithm implemented in the proposed method as it pertains to the study.
- Section 4, experimental setup - please clarify the rationale behind the chosen ramp-up rate of 1 user per second, was this based on real-world usage patterns, or could alternative ramp-up profiles better emulate production load?
- Section 5, The Results and Analysis section effectively identifies key system bottlenecks, such as the strong correlation between high application processing time and high database query time, and clearly explains how resource utilization impacts performance under different load scenarios.
- Section 5.3, The machine learning user classification analysis is thorough, but would benefit from reporting additional metrics (e.g., precision, recall) and discussing the practical implications of classification accuracy for real-world system adaptation.
- The paper lacks visual aids. Please include high-resolution images and well-labeled diagrams that are directly relevant to the study. These visual elements should effectively support and clarify the concepts, methodologies, and results presented in the research.
- Please ensure that the paper is reviewed and modified in accordance with the publisher’s AI usage policy.
Author Response
Thank you for this valuable remarks.
New content is marked in red in the article text.
The text was re-read and the style was improved.
Comments 1:
Author to expand the introduction and system overview section to clearly set the objective to the reader.
Response 1:
Thank you for pointing this out. We agree with this comment. We have therefore added a concise statement of purpose at the end of the opening paragraph of the Introduction and inserted a bridging sentence in the first paragraph of Section 3 to connect the architectural description with our research goal.
------------------------------------------------
Comments 2:
Considering the 1st paragraph of the Introduction section, please clarify "citeCharytanowicz2023" , add relevant references or modify the content accordingly.
Response 2:
Mistake. We forgot backslash in latex (cite). Corrected.
Comments 3:
This paper is based on a software experiment on a containerized stock trading system. Please expand on how these findings translate into specific, actionable optimization steps for platform designers that would strengthen the practical contribution of the paper.
Response 3:
The efficiency of 79% is not the best, but one of the factors is also the speed of operation (description added to the related works section). A real-time tool with an appropriate action scenario (production pipeline) focused on the speed of operation and correctness of user class detection is planned. In subsequent studies, it is planned to adjust parameters and select other methods.
------------------------------------------------
Comments 4:
Please provide a comprehensive and detailed explanation of the algorithm implemented in the proposed method as it pertains to the study.
Response 4:
We have added a new algorithm that presents step-by-step pseudocode of the implemented classification pipeline.
------------------------------------------------
Comments 5:
Section 4, experimental setup - please clarify the rationale behind the chosen ramp-up rate of 1 user per second, was this based on real-world usage patterns, or could alternative ramp-up profiles better emulate production load?
Response 5:
The rate of growth of the number of users is irrelevant. It was selected experimentally so as not to prolong and complicate the application testing process. Performance studies were not the content of the conducted research work.
------------------------------------------------
Comments 6:
Section 5.3, The machine learning user classification analysis is thorough, but would benefit from reporting additional metrics (e.g., precision, recall) and discussing the practical implications of classification accuracy for real-world system adaptation.
Response 6:
Information from the appendix C moved to the section "The Confusion Matrices" and expanded to include conclusions.
------------------------------------------------
Comments 7:
- The paper lacks visual aids. Please include high-resolution images and well-labeled diagrams that are directly relevant to the study. These visual elements should effectively support and clarify the concepts, methodologies, and results presented in the research.
Response 7:
Added action diagrams for user classes. Added web system transaction diagram. Improved readability of figures.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors have addressed all my concerns. I have no more comments.
Author Response
Thank you!
Reviewer 2 Report
Comments and Suggestions for AuthorsI thank the authors for the changes to the manuscript and the responses. I accept the manuscript in its current form.
Author Response
Thank you!
Reviewer 3 Report
Comments and Suggestions for Authors- Thanks for incorporating the review feedback. The paper looks good now.
- Please avoid self citations.
- Please use good resolution diagrams for Figure 1 and 2
Author Response
Thank you for this remarks.
------------------------------------------------
Comment 1:
Please avoid self citations.
Answer 1:
We removed one autocitation (28. Borowiec, M.; Piszko, R.; Rak, T. Knowledge Extraction and Discovery about Web System Based on the Benchmark Application of Online Stock Trading System) that duplicated citation number 26.
------------------------------------------------
Comment 2:
Please use good resolution diagrams for Figure 1 and 2
Answer 2:
The drawings have been enlarged. All were created using the draw.io system (SVG or PNG).
