Exploring the Performance of Transparent 5G NTN Architectures Based on Operational Mega-Constellations

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper investigates the integration of satellite mega-constellations (specifically Starlink) into 5G Non-Terrestrial Networks (NTN). The study focuses on three transparent architectures where the satellite relays the backhaul, midhaul, or New Radio (NR) links. However, some concerns and problems that should be addressed. Below are my suggestions.

1. Optimize the RFSimulator tool to reduce traffic overload or explore alternative tools for NR signal transmission, and investigate real-time implementations to better align with actual network conditions.
2. Include multi-user scenarios to evaluate performance under higher loads, as the current study focused on a single UE.
3. Conduct comparative studies with regenerative payloads to validate latency reductions and performance improvements for NR links. Include throughput and reliability metrics alongside RTT and jitter for a comprehensive performance evaluation.

Author Response

Comment 1: Optimize the RFSimulator tool to reduce traffic overload or explore alternative tools for NR signal transmission, and investigate real-time implementations to better align with actual network conditions.

Response 1: Thank you very much for your comment. We applied traffic compression based on the ZSTD algorithm, as explained in Section 4.2.5, which significantly reduced the traffic overload. RFSimulator is a tool specially adapted to OAI and, therefore, it lacks compatibility and portability. This means that alternative tools are not suitable for OAI, as they should be carefully modified for OAI. We understand that the optimization of the radio simulator goes a step further in our current work, but we will take your comment into account for future studies.

Comment 2: Include multi-user scenarios to evaluate performance under higher loads, as the current study focused on a single UE.

Response 2: Thank you so much for the valuable suggestion. We have raised the idea of including multi-user scenarios, but we have considered that it will not make a notable difference in the paper contribution. The paper results show time metrics based on ping measurements. These results are only altered if the channel is overloaded and packet losses significantly increase. We think that this scenario is not part of the objective of this paper, as it would open numerous issues that should be well-addressed. Indeed, we are concerned that the inclusion of such a scenario may affect the readability and logical flow of the manuscript, creating confusion for readers.

Comment 3: Conduct comparative studies with regenerative payloads to validate latency reductions and performance improvements for NR links. Include throughput and reliability metrics alongside RTT and jitter for a comprehensive performance evaluation.

Response 3: We are grateful for the reviewer’s valuable input. Part of the value of our work is in utilizing real-world satellite communications. This implies that we had to opt for commercial solutions, such as Starlink, which do not contemplate allowing customers to access satellite payloads. Although it would be really interesting, we do not have resources to deploy software on board of real satellites, so the proposed study is unfortunately out of the scope of this work.

Reviewer 2 Report

Comments and Suggestions for Authors

The authors explore 5G NTN transparent architectures by studying their implementation with mega-constellations. The paper presents a methodology to deploy a testbed, based on open-source software, which combines the required 5G functionality with real mega-constellation connectivity. Overall, the ideas presented in the paper are clear and straightforward. However, there are a few additions to the paper that need to be considered to make the paper and its evaluation more engaging for readers.

Here are some of my concerns regarding the paper:

1. The paper would benefit from thorough proofreading and corrections to several sentences. The authors should double-check the entire document and carefully revise it, as some minor grammatical errors persist.
2. The authors state in lines 106-108: “To date, several proposals for mega-constellation projects have been formalized, but only two have reached commercialization: OneWeb by Eutelsat and Starlink by SpaceX.” In the evaluation, only Starlink was considered. The paper lacks a comparative analysis of the proposed method.
3. I would recommend including a comparative analysis of the performance of the proposed method when using Starlink versus Eutelsat. 

Author Response

Comment 1: The paper would benefit from thorough proofreading and corrections to several sentences. The authors should double-check the entire document and carefully revise it, as some minor grammatical errors persist.

Response 1: We thank the reviewers for their careful evaluation of our work. We have double-checked the manuscript and some errors have been corrected. Thank you again for your observation.

Comment 2: The authors state in lines 106-108: “To date, several proposals for mega-constellation projects have been formalized, but only two have reached commercialization: OneWeb by Eutelsat and Starlink by SpaceX.” In the evaluation, only Starlink was considered. The paper lacks a comparative analysis of the proposed method.

Response 2: We appreciate the reviewers’ efforts to improve our manuscript. We justify the selection of Starlink in page 5 lines 173-177, which is essentially due to a logistical issue. “Regarding the mega-constellation connectivity, the current market is predominantly dominated by Starlink, but also by OneWeb. These two initiatives are the only ones that offer commercial services. After reviewing the end-user service accessibility and subscription plans of both options, we opted for Starlink due to its end-user-centric business model, which facilitated the equipment acquisition and deployment.”

We also provide a technical comparison of Starlink and OneWeb in Table 1 according to their space configuration.

Comment 3: I would recommend including a comparative analysis of the performance of the proposed method when using Starlink versus Eutelsat. 

Response 3: We value the reviewer’s constructive recommendation. We consider that we should not give more importance than necessary to the commercial mega-constellations themselves so as not to affect the logical flow of the paper. We chose Starlink as a solution of mega-constellation, but we did not go further with an in-depth analysis. We considered it appropriate to conduct a performance characterization of Starlink to better understand the adopted technology that would impact on the paper results. We think that a detailed comparison of Starlink and OneWeb is out of the scope of this paper, since we have focused on studying the feasibility of mega-constellations in 5G NTN.

Reviewer 3 Report

Comments and Suggestions for Authors

The work proposes 5G NTN architectures  using satellite mega-constellations and uses starlink for backhaul, the work is interesting and a thorough explanation is done. However, it misses information which can give readers a better understanding, like 

1. The author mentioned gNB, how this is different from previous generations or can we employ previous eNB, if yes what impact will it have positive and negative both in handling users.
2. In the figures, where is the Uu interface, and what is Uu?
3. Which architecture is best backhaul, NR or Midhaul shown in Figure 11. 

Author Response

Comment 1: The author mentioned gNB, how this is different from previous generations or can we employ previous eNB, if yes what impact will it have positive and negative both in handling users.

Response 1: We are grateful for the reviewer’s valuable input. We only mention gNB because the focus of the paper is on 5G. Unlike eNB, gNB implements additional features of 5G, such as advanced techniques of network slicing, massive MIMO, or functional split configurations. Both gNB and eNB are suitable for satellites, enabling the implementation of NTN. Nevertheless, gNB should be better for handling users due to the inherent capabilities of New Radio and the advanced network features of 5G. Among the architectures proposed in this paper, only the relaying of the midhaul would not be addressable by an eNB, as it requires the configuration of functional split, which does not exist in 4G.

Comment 2: In the figures, where is the Uu interface, and what is Uu?

Response 2: Thank you for the thoughtful comment. We have clarified the concept of the Uu interface by adding the following sentence (page 2 line 68): “The Uu interface connects the UE to the gNB.”

Comment 3: Which architecture is best backhaul, NR or Midhaul shown in Figure 11.

Response 3: We sincerely thank the feedback. We open a discussion about the advantages and disadvantages of each architecture in page 18 lines 530-540, where we suggest that there is no one better. For readers with the same question, we have added the following sentences to the introductory paragraph of Section 6 in order to guide them towards the aforementioned idea (page 16 lines 462-464). “Likewise, Figure 11 illustrates the key elements within the deployment that influence the test outcomes (explained below). The following section opens a discussion about these outcomes.”

Reviewer 4 Report

Comments and Suggestions for Authors

Dear Authors,

please find attached some minor suggestions aimed at enhancing clarity and presentation.

Comments for author File: Comments.pdf

Author Response

Comment 1: Clarify the extent to which performance degradation in the NR architecture is due to RFSimulator, rather than the architecture itself.

Response 1: We are grateful for the feedback. We mentioned that the degradation is caused by RFSimulator in Section 6 (page 17 lines 501-518). However, we totally agree that we are not clarifying to what extent. We cannot quantify it numerically, but we can provide a detailed explanation of what it is supposed to be happening (as in the aforementioned page 17 lines 501-518). Despite this, we have included the following sentence (page 17 lines 507-509) in order to emphasize that RFSimulator is most likely causing most of that degradation. “It could be causing most of the degradation, resulting in RFSimulator being the bottleneck that makes the results obtained from the relay of NR contrast drastically with those of the rest of the architectures.”

Comment 2: Suggest or test alternative tools to RFSimulator for more accurate radio interface evaluation.

Response 2: Thank you very much for the comment. RFSimulator is a custom tool that has been specifically designed to operate in the OAI environment. This means that alternative tools would not fit into OAI, as they should be carefully adapted to the existing software. The same happens with projects equivalent to OAI, such as srsRAN that utilizes ZeroMQ to simulate the radio interface. Each project implements its own simulator, which is not compatible with external software.

Comment 3: Include scalability tests (e.g., multi-UE scenarios) to improve the applicability of the results.

Response 3: Thank you so much for the valuable recommendation. We have raised the idea of including multi-UE scenarios, but we have considered that it will not make a notable difference in the paper contribution. The paper results show time metrics based on ping measurements. These results are only altered if the channel is overloaded and packet losses significantly increase. We think that this scenario is not part of the objective of this paper, as it would open numerous issues that should be well-addressed. Indeed, we are concerned that the inclusion of such a scenario may affect the readability and logical flow of the manuscript, creating confusion for readers.

Comment 4: Add a brief discussion comparing potential advantages and limitations of regenerative architectures as a complementary approach.

Response 4: Thank you very much for this comment. We were concerned about making assertions that may be scientifically inaccurate. For this reason, we were extremely careful with the analysis of regenerative architectures (page 18 lines 559-568). However, we have expanded the discussion to provide a clear overall idea (page 19 lines 569-574). “Regenerative architectures offer potential advantages, including reduced latency between UE and gNB, and enhanced management of radio resources. These improvements can mitigate several constraints associated with transparent payloads, particularly in time-sensitive scenarios. Nonetheless, they also introduce limitations, such as increased satellite complexity. While regenerative architectures can complement transparent ones in some use cases, their adoption strongly depends on mission constraints and system capabilities.”

Comment 5: Provide standard deviations or error bars with measured metrics to enhance the statistical transparency of the results.

Response 5: We thank the reviewer for this helpful suggestion. We have added the standard deviation value to Table 4 and Table 5 (we have splitted the original Table 4 due to formatting reasons). Additionally, we have provided a brief text introduction of the tables content (page 16 lines 460-462). “These tables present the minimum, first quartile (Q1), median (Q2), third quartile (Q3), maximum, mean, and standard deviation values, providing a numerical summary equivalent to a boxplot.”