You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Electronics
Download PDF
  • Review
  • Open Access

30 March 2023

Mobile Broadband Adoption, Performance Measurements and Methodology: A Review

,
,
and
1
Department of Electrical Power Engineering, Durban University of Technology, Durban 4001, South Africa
2
Department of Electrical and Electronic Engineering, Akwa Ibom State University, Mkpat Enin 524106, Nigeria
3
Department of Electrical, Electronic and Computer Engineering, Cape Peninsula University of Technology, Bellville 7535, South Africa
4
Department of Industrial Engineering, Durban University of Technology, Durban 4001, South Africa
Electronics2023, 12(7), 1630;https://doi.org/10.3390/electronics12071630
This article belongs to the Section Microwave and Wireless Communications
Version Notes
Order Reprints

Abstract

Mobile broadband (MBB) penetration has deepened globally over the last twenty years. This is largely due to the adoption of smart devices, improved mobile communications network coverage, and the perpetual drive to develop ever faster mobile and wireless communication technologies. However, information on the quality of service (QoS) delivered by MBB operators to the end users remains an issue of concern. This has driven independent researchers and mobile communication industry regulators to develop methodologies for independent and unbiased evaluation of the QoS offered by MBB networks. This paper provides a detailed review of MBB adoption and penetration across several regions of the world. It also includes the existing methodologies for evaluating the performance of MBB systems as experienced by the end user. Specifically, methodologies such as the drive and walk tests, crowd-sourced mobile device-based methods and the software applications they employ, and the dedicated measurement testbeds are reviewed. Based on this, the challenges of adopting each of the methods are discussed in order to make a case for the development of more robust, partially autonomous and scalable MBB measurement platforms for the future.

1. Introduction

Access to mobile broadband (MBB) has caused a phenomenal increase in the number of Internet subscribers over the years []. This is because faster internet access through MBB allows for an increasing number of services that were hitherto rendered by physical contact to be provided virtually. This growth was recently accelerated with the emergence of the COVID-19 pandemic, where some countries witnessed up to a 13.3% uptake in Internet connectivity, while some other countries saw a marginal decline []. From data presented in [], the increase in Internet access is mostly driven by an expansion of MBB coverage and this has impacted positively the social and economic wellbeing of the global society []. Several studies including [,,,,,] have assessed this impact and reported that there is a direct correlation between access to broadband and the human development index in the countries and territories where such access is provided.
MBB is delivered to the end users mostly by terrestrial mobile network operators (MNOs), who are responsible for building adequate infrastructure for optimum quality of service (QoS) delivery. However, some of these MNOs do not deliver the speeds they advertise nor the theoretical speeds defined by the respective standards of the 3rd Generation Partnership Project (3GPP) []. Given this scenario and the importance of MBB networks, it is necessary for independent and unbiased performance analyses of the services of existing MNOs as experienced by the end users, to be conducted periodically. Such information would assist the MNOs to improve the capabilities of their MBB networks to be able to deliver service at an acceptable QoS. The evaluation of end user experience QoS is usually carried out through systematic end-to-end measurements using dedicated infrastructure and standard methodologies. In spite of numerous analyses (MBB) conducted over the years across different parts of the world, there is still a dearth of published literature that examines the infrastructure used and the methodologies adopted for such MBB performance assessments. The purpose of this paper is to present a current global MBB penetration and a review of existing infrastructure and methodologies developed around the world for MBB performance analysis. General challenges associated with MBB performance assessment are discussed, and the conclusion provides a direction for future research efforts. It is worth noting that the context of this paper is limited to measurement and evaluation of the performance of end-to-end MBB networks viewed from a user centric perspective. The balance of the paper examines the challenges associated with MBB measurements. The paper is structured as follows: Section 2 discusses the evolution of MBB. Section 3 examines the global penetration of MBB in order to make a case for the need for frequent assessment of MBB performance. Section 4 discusses QoS and customer quality of experience (QoE). Section 5 presents performance metrices, methodologies, testbeds and tools used in determining QoS and QoE for MBB performance evaluation. Section 6 reviews existing MBB performance measurement studies and projects, while Section 7 briefly discusses the challenges associated with MBB measurement. Section 8 is the conclusion.

2. Evolution of Mobile Broadband

Mobile broadband was birthed when the 2nd generation (2G) mobile communication systems evolved to 2.5G (3GPP Release 98 []) from the initial iteration deployed circa 1991 [,]. This was when the theoretical gross bit rate of the system exceeded the International Telecommunication Union (ITU) definition for broadband [], achieving a maximum rate of 384 kpbs in the downlink direction. Ever since, the 3rd, 4th and 5th generation (3G, 4G and 5G) mobile communication systems have progressively achieved faster speeds, as shown in Table 1.
Table 1. Summary of mobile broadband technologies.
Table 1 shows the technologies used by various generations of mobile communication technologies. For example, 2G systems include General Packet Radio Service (GPRS) and Enhanced Data Rate for GSM Evolution (EDGE) [,]. Other technologies considered as part of 2G include the CDMA-based IS-95 and the digital version of the Advanced Mobile Phone System (D-AMPS) [,]. These additional technologies set the motion for 2G-based Internet services in 2G by offering enhanced data rates, enabling wireless devices to access the Internet and delivering improved QoS for voice and data services [,].
Due to the demand for better features and services, and industry drive for improved spectral and energy efficiency, newer generation of mobile communication networks have become a necessity. In Table 1, these newer systems have been classified as 3G, 4G and 5G. 3G consist of technologies such as Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA) and Evolved HSPA (HSPA+). Please note that the standard for each technology is defined by the 3GPP and the ITU. 3G also uses CDMA in standardized family format as CDMA2000 and CDMA200 1xEv-DO (Evolution-Data Optimize) [,,,].
The 4G network was developed to deliver greater capacity for a faster and better MBB experience. It has the Long-Term Evolution (LTE), Advanced LTE (LTE-A) and Worldwide Interoperability for Microwave Access (WiMAX 2) as its main technologies along with others [,,,] listed in Table 1. The 5G mobile network, which has recently been deployed in many countries and territories offers enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable and low-latency communications (URLLC) [,,,]. 5G uses Orthogonal Frequency Division Multiple Access (OFDMA) and Non-Orthogonal Multiple Access (NOMA) as its core technology in addition to the technologies used by the previous generations [,,]. This is in line with the ITU requirements for the International Mobile Telecommunications for 2020 and beyond (IMT-2020) [,,,,].
A defining characteristic of eMBB, being one of the three core 5G features, is its theoretical potential to achieve a gross throughput of up to 20 Gbps as shown in Table 1. According to [], eMBB would be able to support data intensive applications such as ultra-high definition video streaming, and virtual and augmented reality applications.

3. Mobile Broadband Adoption

According to ITU [], the deployment and utilization of MBB have witnessed significant growth over the years. As summarized in Table 2, the number of active MBB subscriptions worldwide has grown by up to 99% when data from year 2015 and 2021 are compared. This growth is even more significant in least developed countries (LDCs), being up to 198%. For the country groupings in Table 2, the other acronyms LLDC and SIDS represent land-locked developing countries and small island developing states, respectively. The growth trend per 100 inhabitants is graphically illustrated in Figure 1a, where it can be observed that there is a huge gap between broadband penetration in developed countries and the rest of the world.
Table 2. Growth in active mobile broadband (3G and above) subscription in millions by development status.
Figure 1. Active mobile broadband subscription per 100 inhabitants: (a) by development status; (b) by region.
Table 3 uses regional groupings according to the ITU’s Telecommunication Development Bureau [] to show the growth in active MBB subscriptions. The table reveals that the largest growth, occurring between years 2015 and 2021 was 148% and this was in Africa. The Asia-Pacific region at 142% came second while the Commonwealth of Independent States (CIS) saw a 61% growth.
Table 3. Growth in active mobile broadband (3G and above) subscription in millions by region.
Figure 1b bar charts display the active MBB subscriptions per 100 inhabitants by region. It should be noted that even with the massive disruptions in broadband usage brought about by the COVID-19 pandemic [], Africa recorded the least number of subscriptions per 100 inhabitants. In numerical terms, the Asia-Pacific and the Americas region have the highest numbers (Table 3) but Europe still dwarfs these regions in terms of penetration (Figure 1b).
The Organization for Economic Co-operation and Development (OECD) also collates and presents yearly MBB data for its 38 member countries []. Published data show that the group has a total of over 1.7 billion active MBB subscriptions as of December 2021 (see Table 4). It also indicates a remarkable year-on-year penetration increase for year 2020 to 2021, averaged at 6.15, as shown in Table 4. Since MBB is mostly accessed through smart phones, a report by GSMA [] has provided an informative projection of figures regarding smart phone adoption for the years, 2021 to 2025. The report predicts a 9% increase in global adoption, and this is equivalent to 7.5 billion smart phone connections, which would mostly be driven by frontier markets in sub-Saharan Africa and the Asia-Pacific regions.
Table 4. Growth in active mobile broadband (3G and above) subscription in millions by region.
Consequently, this progressive MBB penetration and the proliferation of mobile devices across the globe make a case for a regular systematic end-to-end performance assessment of MBB networks. In doing so, the parameters used to characterize the MBB performance need to be understood.

4. Quality of Service and Quality of Experience

MBB services and the underlying terrestrial telecommunications networks are gradually being designed to consider the end-to-end performance needed by the user’s application. In measuring the quality of MBB services, QoS and QoE are the two predominant measures that are often adopted. However, in recent times, the mean opinion score MOS has also been given more attention []. This section reviews the definition of the two most widely adopted terms for end-to-end performance assessment of MBB networks, being the quality of service (QoS) and the quality of experience (QoE).
Quality of Service (QoS) is a term that is broadly used in the telecommunication world but increasingly gaining traction with regards to MBB. QoS is the overall quality of the applications experienced by the network users. According to the ITU [,], “Quality of Service (QoS) is the totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service”. Therefore, end-to-end latency, jitter, packet loss and throughput are key QoS parameters adopted to measure the performance of MBB networks []. From Figure 2, QoS has four points of view when adopted within the concept of service quality. The four points of view include QoS offered by provider, QoS achieved by provider, QoS perceived by subscriber, and the subscriber’s QoS requirements []. The subscriber’s QoS requirements state the level of quality that a specific service requires, which may be expressed in non-technical languages with the subscribers particularly concerned about the resulting end-to-end service quality and not the internal design of the network. QoS offered by the service provider accounts for the level of quality expected to be offered to the customer by the service provider, which maybe expressed in non-technical terms for customers to understand, and in technical terms for use within the business. QoS perceived by the subscribers expresses the level of quality experienced that the subscribers believe they have experienced, which is usually expressed in terms of degrees of satisfaction and not in technical terms. QoS achieved by the service provider accounts for the level of quality truly achieved and delivered to the subscribers, which is used by the industry and sometimes by regulators, for publication in the interests of subscribers [].
Figure 2. The four QoS viewpoints.
QoE involves the end-user in the overall quality assessment and satisfaction in terms of usability, accessibility and service integrity. ITU in [,] defines QoE as “the degree of delight or annoyance of the user of an application or service”. QoE is not only limited to technical network performance but also non-technical aspects that affect the perception and satisfaction of the user as highlighted in Table 5 [,]. Recent studies such as [] have also shown how QoE and QoS can depend on each other in the overall telecommunication network system (OTS). Four normalization techniques were used to predict the QoE parameters based on only the QoS indicators in the OTS.
Table 5. Technical and non-technical aspect of QoE.

5. Mobile Broadband Performance Evaluation Metrics, Methodologies and Tools

This section is concerned with reviewing the available evaluation metrics and the methodologies and tools used in assessing mobile broadband performance from a user-centric perspective.

5.1. Performance Evaluation Metrics

There are different metrics for measuring and assessing the performance of MBB network operators. The key metrics include throughput, latency, packet loss and jitter. Throughput is the actual amount of data that are successfully sent or received over a communication network or link. It is measured both in the uplink and downlink and presented in kbps, Mbps or even higher values such as Gbps [,].
Latency, also called delay, is the time it takes for a data packet to be transmitted from its source to its expected destination. Latency is measured in milliseconds and depends on the type of network access, protocol, packet loss as well as network configuration of the MNOs. Latency portrays how responsive a network is [,]. Jitter is related to latency. It is described as the difference or inconsistencies in latency between end-to-end packet flow. Jitter is measured in milliseconds [].
Packet loss is a QoS metric that describes the percentage of packets of data not reaching their destination after they have been sent across a network. Packet loss is mostly caused by network congestion and is expressed as a percentage [].
In analyzing different MBB networks and use cases, inferences should not be drawn in isolation based on individual results of the listed performance metrics. This is due to the complex relationship between the different performance metrics and the physical layer parameters such as distance between access points and user equipment, prevailing channel fading conditions, modulation and channel coding schemes employed, signal-to-interference and noise ratio, and the time of measurements. These effects individually and collectively influence the results of the performance evaluation.

5.2. Performance Evaluation Methodologies

To measure the MBB performance delivered to the end user, a systematic end-to-end approach is usually preferred []. In some cases, a dedicated testbed is developed while in other cases, simplified methods that use software and mobile applications are adopted. Regulatory agencies, operators or independent researchers can carry out drive tests and walk tests to identify coverage gaps and performance problems. The major disadvantages of these kinds of tests are high costs and poor scalability [,]. Another approach is to depend on the end user to initiate performance measurement by visiting a website or by running performance measurement applications preinstalled on their mobile device []. Even though this method is scalable, there can arise issues of privacy, and bias in measurements by participants. Additionally, this method often lacks vital context information and metadata that is essential in putting the MBB measurement result in the right context.
The challenges with the aforementioned methods birthed the adoption of testbed-based MBB measurement platforms. These testbeds are robust enough to allow for controlled, scalable measurements that can span over a long period of time. Whichever approach is adopted depends on what metrics are to be assessed and what data are required, as all the approaches have their advantages and disadvantages. This subsection therefore provides an overview of the methodologies and approaches that can be adopted for MBB performance assessment.

5.2.1. Drive Test

A drive test is usually carried out to evaluate and objectively compare the capacity, coverage and quality of service (QoS) provided by mobile networks []. Such tests involve mounting measurement equipment inside moving vehicles to systematically collect measurements []. Important features of drive testing are the need for consistency in the collection of measurement data and minimizing the variation in as many factors as possible that might influence the measurement results []. Additionally, performance impacting variables such as collection of samples that reflects consistent speed, time-of-day, application use or device configuration are controlled to improve the ability of researchers to draw inferences. Drive tests provide a common picture of the QoS of the mobile user over a given geographical area. There are fundamentally two main formats for performing drive tests.
The first is a user equipment-based testing where measurements are performed using a typical user-equipment such as smart phones. The second method engages specialized receivers for measurement and benchmarking [,]. There is no defined methodology or standard for conducting a drive test. However, Figure 3 shows a typical configuration of drive test []. Usually, an MBB testing application is installed in identical smartphones, each dedicated to a particular MNO as shown in Figure 4 []. The smartphones are mounted in a moving vehicle and programmed to take measurements while the vehicle is in motion and to evaluate several performance metrics relating to user experience. The data collected are stored in the mobile device, which are retrieved and analyzed in real time or at the end of the measurement campaign. The major disadvantage of this kind of MBB measurement is that it is expensive and may not be scalable.
Figure 3. A typical methodology for MBB drive test [].
Figure 4. Smartphones mounted in moving vehicles during drive test [].

5.2.2. Walk Test

The walk test is another type of MBB measurement that is setup in a similar way to drive testing. However, walk tests involve walking along typical locations such as subways, pedestrians, sports stadiums, airports and malls and taking network measurements. The test equipment for a walk test is affixed to the back of the test taker or placed in a carrying harness during measurement periods []. Figure 5 shows an example of such a carrying case with mounting prepared to firmly carry six mobile phones. The cell phones are programmed to take performance measurements for the different MNOs through an application installed in them, as the carrier walks around defined areas. Walk tests are more expensive than drive tests as the cost of collecting data per unit area is high. The test is also limited by the walk speed and distance covered by the carrier of the test device.
Figure 5. Bag pack with six mounted smartphones.

5.2.3. Crowdsourced Method

The crowdsourced software application-based approach to MBB performance evaluation is a large-scale measurement method that typically depends on voluntary participation by the end user [,]. Mobile apps are installed on the participant’s smartphones for MBB performance assessment. They are two basic approaches to this type of MBB performance measurement. The first is the app-based panel testing, where participants known as the panel of testers have the app installed and running on their smartphones. An autonomous agent then initiates the test and collects the data in the background without the knowledge of the user [,]. The Federal Communication Commission (FCC) in the US is one regulator that uses this panel-based approach. The second approach relies on users to voluntarily install the app and also, initiate the test.
The crowdsourced app-based approach is cost-effective, scalable and can cover a larger geographic area than other test methods. However, different MNOs cannot be tested at the same time, and relying on the user to initiate the test may not be very effective. Privacy is also an issue as the data collected by real participants must be encrypted to be completely anonymous. Additionally, important context information such as location, type of subscription and mode of connection is often missing in this type of measurement approach [,,,,,]. Industry regulators often adopt this approach for national scale measurement campaigns. Some of such notable measurements have been carried out by USA’s FCC [], the UK’s Office of Communications (OFCOM) [,,], the Canadian Radio-Television and Telecommunications Commission (CRTC) [] and the Australian Competition and Consumer Commission (ACCC) [].
Generally, it is important to observe that each methodology has its unique benefits and weaknesses as shown in Table 6. Hence, adopting more than one approach allows for cross-validating of measurement results. In recognizing this, regulators from many countries and private researchers are gradually employing multiple approaches for MBB network assessment. This is evident in a draft report released by the Body of European Regulators for Electronic Communications (BEREC) that summarizes responses received from its EU member countries about their MBB measurement activities []. A section of the report presented in Table 7 shows the diverse methodologies adopted within each country, with drive test being the most commonly used approach in twenty-seven of the thirty-four countries.
Table 6. Advantages and disadvantages of mobile broadband measurement techniques.
Table 7. Different methodologies adopted for BEREC countries.

5.3. Mobile Broadband Measurement Software Tools

Most MBB performance evaluation methodologies make use of software either installed as applications in smartphones, measurement nodes or accessed online. Speedtest by Ookla [] is one of the most popularly used applications developed for crowdsourced based MBB measurement. Speedtest maintains a free testing website where visitors can measure the performance of their network using any of the available public Ookla test servers around the world []. Each measurement taken with the Speedtest app is initiated by the user and this consists of upload and download throughput, jitter, packet loss and latency, which can be aggregated at the provider, state or country level []. Over 43 billion tests have been made using the Speedtest engine since its inception []. The test run by Ookla on its website is between a flash-based applet embedded in a web page and a server hosted on a web browser while the mobile version runs on smart phones []. Figure 6a shows the interface of the mobile application which is freely available in 17 languages on Google play store for both Android and iOS-based smartphones [].
Figure 6. Mobile applications during measurement: (a) Speedtest; (b) SamKnows.
SamKnows is also a well-known network performance company that develops MBB crowdsourced measurement applications as one of its products []. The SamKnows mobile application app supports MBB performance in six main categories: download speed, upload speed, latency, packet loss, web browsing and YouTube streaming MBB measurement. It can also record passive metrics such as signal strength of the connection and device manufacturer, type and model. However, for this app, the test can be conducted anonymously on its own at timed intervals [,]. SamKnows’ crowdsourced app is modular, allowing regulators to easily customize the app for quick MBB speed assessment. SamKnows also collaborate to develop custom-made apps for regulators such as the FCC speed test app []. Figure 6b shows the interface of the mobile application, which is supported by both Android and iOS-based smartphones.
Other mobile applications for MBB assessment include MBPerf [,], the Meteor app by OpenSignal, [], MobiPerf app [], RTR-NetTest [], Netradar and the SpeedSmart speedtest app [].

6. Review of Existing Mobile Broadband Performance Measurements

This section reviews existing user-centric mobile broadband performance measurements studies and testbed-based projects.

6.1. User-Centric Performance Evaluation Works

The research objective of most MBB studies is conducted to comparatively evaluate the MBB performance delivered by different MNOs. The research reported in [,] adopted a host and crowdsourced based approached using MBperf as the mobile application to measure the performance of 2G and 3G MBB networks, while [] used a simplified Raspberry Pi testbed for measurement of the performance of 3G and 4G MBB networks over an extended period. The results of this research carried out in Nigeria reveal variations in MBB speeds delivered by four major MNOs in the country. Similarly, studies reported in [,,] also used a panel-based crowdsourced approach for a comparative assessment of 3G and 4G MBB networks in Nepal, Pakistan and South Africa, respectively. They identified that the MBB speeds delivered to end users do not meet the values advertised by the MNOs. The behavior of these crowdsourced MBB measurement datasets can be analyzed using machine learning for more accurate estimations [,].
Apart from comparative analysis, other types of MBB performance evaluation have been carried out. For instance, [] used a panel-based crowdsourced method for performance assessment of MBB services offered by different Internet service providers during defined peak periods and off-peak periods in major Canadian metropolitan areas. They defined peak periods as the time between 7 pm and 11 pm from Monday to Friday and off-peak periods as any hours or days exclusive of peak periods. Additionally, [,,] used a dedicated testbed and drive tests approach to study the performance of different MBB networks under mobility, while [,] adopted the walk test methodology to perform coverage and capacity measurement and characterize the performance of MBB networks during peak periods and off-peak period.
Furthermore, when designing future technologies, MBB measurement can be valuable for benchmarking and planning network upgrades. The MBB performance measurement of the 4G networks reported in [,] are studies conducted to determine the baseline for 5G capabilities and assess the inefficiencies that should be addressed in the 5G network. Some of the points highlighted and the benchmarks estimated were considered in the 5G pilot MBB measurement reported in [,,,].

6.2. Testbed-Based Measurement Projects

The limitations posed by using the aforementioned methodologies have driven institutions and private researchers to develop more robust infrastructure for testbed-based experiments on MBB performance. Although some of these testbeds are expensive to build, they allow for a controlled and scalable measurement over a long period and thus, eliminate many limitations of the other methodologies. This section introduces some testbed-based MBB performance evaluation platforms and projects that already exist. It goes further to explain the network tools used for these testbeds.

6.2.1. The Nornet Edge (NNE) Platform

The Nornet Edge (NNE) platform is a testbed dedicated to the measurement and study of MBB networks and is presented in [,,]. Figure 7 shows the overview of the testbed for MBB experiments. Renowned as one of the largest infrastructures in the world for MBB measurements, the NNE has over 400 fully programmable and multi-homed nodes shown in Figure 8, placed at different locations in Norway. The NNE measurement nodes comprise custom-made single-board computers running a standard Linux operating system that allow 2–5 MNOs to be connected to it using MBB modems. The node is equipped with a Samsung S5PV210 Cortex A8 1 GHz processor with 512 MB RAM, 512 MB NAND flash memory and a 16 GB SD card for storage. Sets of servers form a central backend system for collection and storing data on the NNE platform. There is also an algorithm designed to manage the nodes and run measurements for a long time on a national scale. The platform allows for the collection of status information from the modems on mobile broadband cell ID, connection mode and signal strength.
Figure 7. The overall system architecture of the NNE platform [].
Figure 8. NNE node with 4 modems connected [].
Since the platform is able to simultaneously connect to multiple networks, it is possible to directly compare QoS metrics across different MNOs. The NNE platform is built for future compatibility with new systems as its designs makes it seamless to install new measurement applications to gather new or additional data. A website is also created for real-time viewing of the status of all NNE nodes, including the status of each MBB connection.
NNE is well suited for national scale measurements and experiments that require a large number of geographically distributed measurement nodes, simultaneous connections to multiple operators, information regarding the context in which measurements are taken, and continuous measurements that span long. One such experiment and research is reported in [].

6.2.2. The MONROE Platform

The MONROE testbed and its operation presented in [,,,] is the first open access European transnational hardware-based platform for independent, multihomed, large-scale experimentation in MBB measurements. Figure 9 shows the overview of the MONROE MBB performance evaluation platform. MONROE has a set of 150 nodes, both mobile and stationary, which are multihomed to 5 different MNOs with the aid of commercial grade subscriptions across numerous European countries. The MONROE MBB measurement node shown in Figure 9 is based on Debian GNU/Linux “stretch” distribution integrating two small programmable computers. The computers are made of PC engines APU2 board interfacing with three 3G/4G MC7455 miniPCI express modems using LTE CAT6 and one WiFi modem. Each of the nodes gathers metadata such as carrier, technology, signal strength, GPS location and sensor data from the different modems. MONROE runs its MBB experiments using Docker containers (lightweight virtualized environment) to provide agile reconfiguration. Only users who are authenticated can access resources on the platform through a web portal, and also have access to the MONROE scheduler to deploy experiments. After each experiment on the MONROE platform, the results are periodically transferred from the nodes to a repository at a back-end server, while the MONROE scheduler also sets data quotas to ensure fairness among users. Some of the vast experiments run with the MONROE testbed have been reported in [,,].
Figure 9. Overview of the MONROE platform.
Three vital features of MONROE make the platform unique. It allows measurements to be repeated and controlled for precise and scientifically verifiable results for both fixed and mobile scenarios, enables support for demanding applications such as web and video services and supports protocol and service innovation.

6.2.3. The Simplified Raspberry Pi Platform

A simplified testbed for MBB performance evaluation that follows the setup of the NNE albeit using easily sourced commercial-off-the-shelf (COTS) devices is presented in [,]. Figure 10 shows the overall system architecture with the Raspberry Pi forming the core of the remote MBB measurement node. The Raspberry Pi 4 with 64 quad-core Cortex-A72 processors and 2GB Low-Power Double Data Rate (LPDDRA) RAM on its board is used for the node. The testbed uses USB modems and retrofitted WiFi to connect up to 4 MNOs for 3G and 4G MBB networks, respectively. The Raspberry Pi nodes are configured with the 4-way 5V relay modules mounted and an executable script written in python to achieve multihoming for 3G and 4G MBB measurements. The node autonomously initiates the measurement at regular intervals and stores the information, which an authorized user can access remotely at the testbed core for evaluation. This simplified MBB testbed is not as sophisticated as NNE; however, it can measure the key MBB performance metrics over an extended period.
Figure 10. Overview of the Simplified Raspberry Pi platform [].
The aforementioned testbeds have been dedicated mostly to 3G and 4G MBB experiments, albeit allowing compatibility with future mobile communication networks like the recently deployed 5G network. To the best of our knowledge, there is no dedicated testbed to assess the QoS delivered to end users on the 5G MBB network from a user-centric perspective. However, as part of the 5G Public Private Partnership (5G-PPP) initiative, the EU funded 5GENESIS [] project has been developed as a flexible and open experimentation testbed for validating the end-to-end key performance indicators (KPIs) of 5G networks. The 5GENESIS architecture is designed to provide an integrated and open experimentation framework that facilitates interactions between the experimenters and the testing facilities. A detailed description of the experimentation suite is presented in [], while pilot 5G experiments using the testbed have been reported in [,].
Furthermore, there are other testbed federations such as Fed4FIRE+ [,] and 5TONIC [], developed to carry out experiments on numerous aspects of 4G and 5G. Fed4FIRE+ was the largest federation of internet testbeds in Europe consisting of 23 testbeds equipped with numerous user-friendly tools that enabled remote testing in different areas of interest. The Fed4FIRE+ project, which was a successor to the Fed4FIRE project, came to an end in June 2022 and its legacy will be taken by Scientific LargeScale Infrastructure for Computing/Communication Experimental Studies. (SLICES-RI) [,]. 5TONIC is an open research and innovation laboratory developed to create an open global environment for industry experts and members of academia to work together on specific projects that focus on 5G technologies []. Some studies that utilized the 5TONIC platform have been reported in [,].
Table 8 presents a summary of extensively reviewed user-centric MBB performance evaluation studies, highlighting the method adopted, the QoS metrics considered, the type of access network and a summary of each study. Table 9 compares the different MBB performance measurement methods already discussed.
Table 8. Summary of existing mobile broadband performance evaluation works.
Table 9. Comparison of the different MBB measurement methods.

7. Challenges of Mobile Broadband Measurements

Different MBB measurements have their unique challenges, hence the adoption of more than one measurement approach to allow opportunities for cross-validation of results. This section briefly discusses some of the general challenges that are associated with MBB measurement shown in Figure 11.
Figure 11. MBB measurements challenges.

7.1. Variety of Performance

There are infinite indices that can be used to characterize the performance of MBB during measurements. These indices include geographic locations and measurement orientations, the type of user equipment, and the context surrounding the user equipment. Small changes along the dimensions of their parameters most often result in a significant impact on the measured performance []. Therefore, the challenge of a proper definition of the context of measurements, collection and handling of the numerous data sets will arise.

7.2. Privacy and User Engagement

Inducing several participants across the MBB measurement ecosystem to share infrastructure or data provokes difficult challenges []. Privacy is one of such issue that places significant constraints on data collection due to the prevalence of single users or participants. Capturing important metadata like location related data from the user equipment during measurement poses a huge threat to individual privacy []. Sometimes, there are agreement terms with the participants to assure them of no privacy breach. However, such an agreement does not translate to the users not interfering with the measurement process due to differing sentiments on privacy. This could significantly impede the authenticity of data collection. Furthermore, recruiting participants and getting them engaged and motivated during the measurement process is challenging. If methods are not devised to tackle this, the participants could simply abandon their device or cite economic challenges []. Similarly, personnel operating dedicated measurement testbeds may interfere with the measurement processes if not properly guided.

7.3. Mobility

Conducting MBB measurement in mobility is challenging when considering the mobility of the end-user’s location in time and space and the mobility of all network resources. For instance, for a more accurate result, when conducting drive test and walk test, measurement samples are collected in as many locations as possible but with differing physical speeds of the devices. The mobility during such measurement can easily be dictated by natural or unforeseen occurrences such as existing vehicular or pedestrian traffic in the area. This will cause the exact model of geographic mobility being tested to vary, and achieving the exact details for the different locations would prove to be a challenging task []. Additionally, the need for shorter duration of tests, better accuracy, and analysis mechanism to handle performances during handover between base stations, roaming and wireless technologies at different speeds, is very important for how the performance data are aggregated and reported [,].

7.4. Lack of Methodological Transparency

It is observed that information which describes the overall structure of the measurements and reports on the various methodologies is usually available, whereas there is a dearth of details regarding the adopted measurement methodology and how the raw data are analyzed to produce the final results. This dearth of methodological transparency makes it difficult for researchers and others to critically analyze and distinguish between good and bad implementations of MBB measurement methodologies. The lack of transparency could translate to a lack of trust among researchers for future research in the evolving MBB measurement ecosystem [].

7.5. Need for Proactive Measurements

Standard objectives during MBB measurement are needed. For example, a measurement objective may be to seek broadband performance in locations where actual users have not yet attempted to use mobile devices. Another objective may be intentionally trying to find the locations and conditions under which mobile service expectations are not being met. The aim is usually to identify coverage/availability gaps or dead spots within already covered areas or to find the boundaries of the existing coverage zones []. However, identifying the scope and appropriate methodologies for such mobile broadband measurement objectives is challenging.

7.6. Sustainability and Management of Infrastructure

It is viable to develop methodologies and build appropriate infrastructure for MBB measurement. However, ensuring that the infrastructure is sustainable for a foreseeable time will present a continuous challenge that needs to be considered before deployments. Adequate resources required to cater for the sustainability of the measurement infrastructure should be available or sourced []. There is also a prevailing challenge with managing large testbeds, especially keeping the hardware up to date with emerging technologies []. In terms of QoS/QoE, the lack of well-defined scalable QoS and QoE indicators limit the proper estimation of various service compositions as a part of the overall telecommunication network.

8. Conclusions

This paper has used available data to evaluate the global importance and impact of MBB. It provides an understanding of end-to-end MBB performance assessment and presented existing methodologies, approaches and testbeds for evaluating the performance of MBB networks. The challenges associated with MBB measurement are highlighted.
However, the emergence of 5G and future technologies with higher gigabit data rates and the projected increase in simultaneously connected devices would translate to more data consumption. Such development may result in significant variations in peak periods and off-peak periods data consumption patterns. These possible challenges need to be identified and addressed. Therefore, it is important that future research be geared toward optimizing present MBB measurement methods and infrastructure to accommodate these imminent changes. Furthermore, it has been highlighted that MBB performance evaluation is complex for one evaluation method to be applicable in all situations. For example, the measurement of MBB performance under static and mobile conditions will require different methods. As a result, future research should consider the development of distributed MBB measurement platforms that are capable of supporting a range of MBB experiments. Intensive research in this direction would result in the development and deployment of more robust, semi-autonomous, interlinkable measurement platforms.

Author Contributions

Conceptualization, V.U. and U.E.; methodology, V.U., J.A. and U.E.; formal analysis, V.U. and J.A.; resources, V.U., U.E. and I.D.; writing—original draft preparation, V.U.; writing—review and editing, V.U., I.D., U.E. and J.A.; visualization, V.U.; supervision, I.D. and U.E.; All authors have read and agreed to the published version of the manuscript.

Funding

Article publishing fee was funded by The Research Directorate, Durban University of Technology, South Africa.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Acknowledgments

This project was supported by the Nigerian Communications Commission (NCC) and the Space Science Research Centre, Durban University of Technology.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Edquist, H. The economic impact of mobile broadband speed. Telecommun. Policy 2022, 46, 102351. [Google Scholar] [CrossRef]
  2. International Telecommunication Union. Measuring Digital Development Facts and Figures 2021; International Telecommunication Union: Geneva, Switzerland, 2021. [Google Scholar]
  3. Edquist, H.; Goodridge, P.; Haskel, J.; Li, X.; Lindquist, E. How important are mobile broadband networks for the global economic development? Inf. Econ. Policy 2018, 45, 16–29. [Google Scholar] [CrossRef]
  4. International Telecommunication Union. The Impact of Broadband on the Economy: Research to Date and Policy Issues, in Broadband Series; International Telecommunication Union: Geneva, Switzerland, 2012. [Google Scholar]
  5. Kongaut, C.; Bohlin, E. Impact of broadband speed on economic outputs: An empirical study of OECD countries. Econ. Bus. Rev. 2017, 3, 12–32. [Google Scholar] [CrossRef]
  6. Mayer, W.; Madden, G.; Wu, C. Broadband and economic growth: A reassessment. Inf. Technol. Dev. 2020, 26, 128–145. [Google Scholar] [CrossRef]
  7. Minges, M. Exploring the relationship between broadband and economic growth. In World Development Report 2016: Digital Dividends; World Bank: Washington, DC, USA, 2016. [Google Scholar]
  8. Ugboma, M.U. Bridging the digital divide: With special reference to Nigeria. Inf. Impact J. Inf. Knowl. Manag. 2012, 3, 1–2. [Google Scholar]
  9. 3gpp. Specifications Home. Available online: https://www.3gpp.org/specifications/specifications (accessed on 2 August 2022).
  10. Chitrapu, P.; Aghili, B. Evolution of GSM into the next generation wireless world. In Proceedings of the 2007 IEEE Long Island Systems, Applications and Technology Conference, Farmingdale, NY, USA, 4 May 2007; IEEE: New York, NY, USA, 2007; pp. 1–10. [Google Scholar]
  11. Pawar, D.S.; Deshpande, A. Evolution of Wireless Technology. Int. J. Comput. Sci. Mob. Comput. IJCSMC 2020, 9, 91–94. [Google Scholar]
  12. Venkatachalam, S.; McDowell, S.D. What is broadband? Where is rural? Gov. Inf. Q. 2003, 20, 151–166. [Google Scholar] [CrossRef]
  13. Molisch, A.F. GSM–Global System for Mobile Communications. In Wireless Communications; John Wiley & Sons: Hoboken, NJ, USA, 2012; Chapter 24; p. 587. [Google Scholar]
  14. Sauter, M. General Packet Radio Service (GPRS) and EDGE. In From GSM to LTE-Advanced Pro and 5G: An Introduction to Mobile Networks and Mobile Broadband; John Wiley & Sons: Hoboken, NJ, USA, 2017; Chapter 2; pp. 73–128. [Google Scholar]
  15. Schwartz, M. Second-generation, digital, wireless systems. In Mobile Wireless Communications; Cambridge University Press, The Edinburgh Building: Cambridge, UK, 2005; Volume 25, Chapter 8; pp. 199–245. [Google Scholar]
  16. Li, X.; Gani, A.; Salleh, R.; Zakaria, O. The Future of Mobile Wireless Communication Networks. In Proceedings of the 2009 International Conference on Communication Software and Networks, Chengdu, China, 27–28 February 2009; pp. 554–557. [Google Scholar] [CrossRef]
  17. Sauter, M. From GSM to LTE-Advanced Pro and 5G an Introduction to Mobile Networks and Mobile Broadband; John Wiley & Sons Ltd.: Hoboken, NJ, USA, 2017. [Google Scholar]
  18. Siwakota, Y.R. Measuring Performance of Mobile Broadband Netwrok under Moblity; Department of Informatic, University of Oslo: Oslo, Norway, 2014. [Google Scholar]
  19. Sauter, M. Universal Mobile Telecommunications System (UMTS) and High-Speed Packet Access (HSPA). In From GSM to LTE-Advanced Pro and 5G: An Introduction to Mobile Networks and Mobile Broadband; John Wiley & Sons: Hoboken, NJ, USA, 2017; Chapter 3; pp. 129–232. [Google Scholar]
  20. Shukla, S.; Khare, V.; Garg, S.; Sharma, P. Comparative Study of 1G, 2G, 3G and 4G. J. Eng. Comput. Appl. Sci 2013, 2, 55–63. [Google Scholar]
  21. Churi, J.R.; Surendran, T.S.; Tigdi, S.A.; Yewale, S. Evolution of networks (2G-5G). In Proceedings of the International Conference on Advances in Communication and Computing Technologies (ICACACT), Kochi, India, 9–11 August 2012; Volume 51, pp. 8–13. [Google Scholar]
  22. Molisch, A.F. WCDMA/UMTS. In Wireless Communications; John Wiley & Sons: Hoboken, NJ, USA, 2012; pp. 635–663. [Google Scholar]
  23. Sauter, M. Long Term Evolution (LTE) and LTE-Advanced. In From GSM to LTE-Advanced Pro and 5G: An Introduction to Mobile Networks and Mobile Broadband; John Wiley & Sons: Hoboken, NJ, USA, 2017; Chapter 4; pp. 235–325. [Google Scholar]
  24. Tabbane, S. 4G to 5G Networks and Standard Releases. Available online: https://www.itu.int/en/ITU-D/Regional-Presence/AsiaPacific/SiteAssets/Pages/Events/2019/ITU-ASP-CoE-Training-on-/3GPP_4G%20to%205G%20networks%20evolution%20and%20releases.pdf (accessed on 1 January 2022).
  25. Ezhilarasan, E.; Dinakaran, M. A Review on Mobile Technologies: 3G, 4G and 5G. In Proceedings of the 2017 Second International Conference on Recent Trends and Challenges in Computational Models (ICRTCCM), Tindivanam, India, 3–4 February 2017; pp. 369–373. [Google Scholar] [CrossRef]
  26. Arshad, Q.K.U.D.; Kashif, A.U.; Quershi, I.M. A Review on the Evolution of Cellular Technologies. In Proceedings of the 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan, 8–12 January 2019; pp. 989–993. [Google Scholar] [CrossRef]
  27. Arshad, Q.K.U.D.; Kashif, A.U.; Quershi, I.M. Mobility management solutions for 5G networks: Architecture and services. Comput. Netw. 2020, 169, 107082. [Google Scholar] [CrossRef]
  28. Ghafoor, U.; Ali, M.; Khan, H.Z.; Siddiqui, A.M.; Naeem, M. NOMA and future 5G & B5G wireless networks: A paradigm. J. Netw. Comput. Appl. 2022, 204, 103413. [Google Scholar] [CrossRef]
  29. Yang, C.; Liang, P.; Fu, L.; Cui, G.; Huang, F.; Teng, F.; Bangash, Y.A. Using 5G in smart cities: A systematic mapping study. Intell. Syst. Appl. 2022, 14, 200065. [Google Scholar] [CrossRef]
  30. Zhu, Z.; Chu, Z.; Li, X. Chapter 2-Three major operating scenarios of 5G: eMBB, mMTC, URLLC. In Intelligent Sensing and Communications for Internet of Everything; Zhu, Z., Chu, Z., Li, X., Eds.; Academic Press: Cambridge, MA, USA, 2022; pp. 15–76. [Google Scholar]
  31. International Telecommunication Union. Setting the Scene for 5G: Opportunities & Challenges; International Telecommunication Union: Geneva, Switzerland, 2018. [Google Scholar]
  32. Li, Z.; Uusitalo, M.A.; Shariatmadari, H.; Singh, B. 5G URLLC: Design challenges and system concepts. In Proceedings of the 2018 15th International Symposium on Wireless Communication Systems (ISWCS), Lisbon, Portugal, 28–31 August 2018; IEEE: New York, NY, USA, 2018; pp. 1–6. [Google Scholar]
  33. ITU-R. IMT Vision–Framework and Overall Objectives of the Future Development of IMT for 2020 and Beyond; Recommendation ITU: Geneva, Switzerland, 2015. [Google Scholar]
  34. Ateya, A.A.; Muthanna, A.; Makolkina, M.; Koucheryavy, A. Study of 5G services standardization: Specifications and requirements. In Proceedings of the 2018 10th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Moscow, Russia, 5–9 November 2018; IEEE: New York, NY, USA, 2018; pp. 1–6. [Google Scholar]
  35. GSMA. 5G Implementation Guidelines; CK Hutchison: Hongkong, China, March 2019. [Google Scholar]
  36. ITU. Statistics. Available online: https://www.itu.int/en/ITU-D/Statistics/Pages/stat/default.aspx (accessed on 20 July 2022).
  37. ITU. Economy Classification. Available online: http://www.itu.int/en/ITU-D/Statistics/Pages/definitions/regions.aspx (accessed on 21 July 2022).
  38. OECD. Broadband Portal. Available online: https://www.oecd.org/sti/broadband/broadband-statistics/ (accessed on 14 July 2022).
  39. GSMA. The Mobile Economy; GSMA: London, UK, 2022. [Google Scholar]
  40. El-Saleh, A.A.; Alhammadi, A.; Shayea, I.; Alsharif, N.; Alzahrani, N.M.; Khalaf, O.I.; Aldhyani, T.H.H. Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends. Sustainability 2022, 14, 829. Available online: https://www.mdpi.com/2071-1050/14/2/829 (accessed on 1 January 2022). [CrossRef]
  41. ITU. Recommendation -T P. 10/G. 100, Vocabulary for Performance, Quality of Service and Quality of Experience; ITU: Geneva, Switzerland, 2017. [Google Scholar]
  42. ITU-T. E.800-Series–Guidelines on Regulatory Aspects of QoS. 2021. Available online: https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14832&lang=en (accessed on 31 July 2022).
  43. Malisuwan, S.; Milindavanij, D.; Kaewphanuekrungsi, W. Kaewphanuekrungsi, Quality of service (QoS) and quality of experience (QoE) of the 4G LTE perspective. Int. J. Future Comput. Commun. 2016, 5, 158. [Google Scholar] [CrossRef]
  44. ITU-T Recommendation, G. 1000, Communications Quality of Service: A Framework and Definitions, 11/2001. Available online: https://www.itu.int/itu-t/recommendations/rec.aspx?rec=G.1000 (accessed on 31 July 2022).
  45. ITU-T. Quality of Experience (QoE) Requirements for Real-Time Multimedia Services over 5G Networks. 2022. Available online: https://www.itu.int/pub/T-TUT-QOS-2022-1 (accessed on 31 July 2022).
  46. Budiman, E.; Moeis, D.; Soekarta, R. Broadband quality of service experience measuring mobile networks from consumer perceived. In Proceedings of the 2017 3rd International Conference on Science in Information Technology (ICSITech), Bandung, Indonesia, 25–26 October 2017; IEEE: New York, NY, USA, 2017; pp. 423–428. [Google Scholar]
  47. Poryazov, S.A.; Saranova, E.T.; Andonov, V.S. Overall Model Normalization towards Adequate Prediction and Presentation of QoE in Overall Telecommunication Systems. In Proceedings of the 2019 14th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), Nis, Serbia, 20–22 October 2019; IEEE: New York, NY, USA, 2019; pp. 360–363. [Google Scholar]
  48. Feamster, N.; Livingood, J. Measuring internet speed: Current challenges and future recommendations. Commun. ACM 2020, 63, 72–80. [Google Scholar] [CrossRef]
  49. ITU. Quality of service guaranteed mechanisms and performance model for public packet telecommunication data networks. In Recommendation ITU-T Y.2617; ITU: Geneva, Switzerland, 2016. [Google Scholar]
  50. Umoh, V.B.; Ukommi, U.S.; Ekpe, U.M. A Comparative Study of User Experienced Mobile Broadband Performance. Niger. J. Technol. 2022, 41, 560–568. [Google Scholar] [CrossRef]
  51. Alay, O.; Lutu, A.; Garcia, R.; Peon-Quiros, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. Measuring and assessing mobile broadband networks with MONROE. In Proceedings of the 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), Coimbra, Portugal, 21–24 June 2016; pp. 1–3. [Google Scholar] [CrossRef]
  52. Kvalbein, A.; Baltrūnas, D.; Evensen, K.; Xiang, J.; Elmokashfi, A.; Ferlin-Oliveira, S. The Nornet Edge platform for mobile broadband measurements. Comput. Netw. 2014, 61, 88–101. [Google Scholar] [CrossRef]
  53. Bauer, S.; Lehr, W. Measuring Mobile Broadband Performance. In Proceedings of the 29th European regional Conference of the International Telecommunications Society (ITS): “Towards a Digital Future: Turning Technology into Markets”? Trento, Italy, 1–4 August 2018; International Telecommunications Society (ITS): Calgary, AL, Canada, 2018. [Google Scholar]
  54. Alay, Ö.; Lutu, A.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Evensen, K.; Hansen, A.; Alfredsson, S.; Karlsson, J.; Brunstrom, A.; et al. Experience: An open platform for experimentation with commercial mobile broadband networks. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, Snowbird, UT, USA, 16–20 October 2017; pp. 70–78. [Google Scholar]
  55. Akinlabi, A.A.; Dahunsi, F.M. Mobile broadband quality of service analysis using host-based performance measurements. Afr. J. Sci. Technol. Innov. Dev. 2021, 14, 1–19. [Google Scholar] [CrossRef]
  56. Midoglu, C.; Svoboda, P. Opportunities and challenges of using crowdsourced measurements for mobile network benchmarking a case study on RTR open data. In Proceedings of the 2016 SAI computing conference (SAI), London, UK, 13–15 July 2016; IEEE: New York, NY, USA, 2016; pp. 996–1005. [Google Scholar]
  57. Federal Communications Commission (FCC). Eleventh Measuring Broadband America Fixed Broadband Report. 2021. Available online: https://www.citizenscience.gov/assets/files/fcc-speed-test.pdf (accessed on 1 January 2022).
  58. OFCOM. Measuring 4G mobile broadband and voice performance. In Smartphone Cities; OFCOM: London, UK, 2016. [Google Scholar]
  59. OFCOM. Measuring mobile broadband and voice performance. In Smartphone Cities; OFCOM: London, UK, 2016. [Google Scholar]
  60. OFCOM. Measuring Mobile Broadband Performance in the UK 4G and 3G Network Performance; OFCOM: London, UK, 2014. [Google Scholar]
  61. The Canadian Radio-television and Telecommunications Commission (CRTC). In Measuring Broadband Canada; CRTC: Ottawa, ON, Canada, 2020.
  62. Australian Competition and Consumer Commission (ACCC). Measuring Broadband Australia. 2021. Available online: https://www.accc.gov.au/regulated-infrastructure/telecommunications-and-internet/telecommunications-monitoring/measuring-broadband-australia-program (accessed on 1 January 2022).
  63. BEREC. Common Position on Monitoring Mobile Coverage; BEREC: Rīga, Republic of Latvia, 2018. [Google Scholar]
  64. Bauer, S.; Clark, D.D.; Lehr, W. Understanding Broadband Speed Measurements. 2010. Available online: https://www.semanticscholar.org/paper/Understanding-Broadband-Speed-Measurements-Bauer-Clark/7157455aad24d88bdcaf1961977ec64911680e16 (accessed on 31 July 2022).
  65. SPEEDTEST. Available online: https://www.speedtest.net/about (accessed on 29 July 2022).
  66. Goel, U.; Wittie, M.P.; Claffy, K.C.; Le, A. Survey of end-to-end mobile network measurement testbeds, tools, and services. IEEE Commun. Surv. Tutor. 2015, 18, 105–123. [Google Scholar] [CrossRef]
  67. Speedtest. Speedtest Apps for Mobile. Available online: https://www.speedtest.net/apps/mobile (accessed on 31 July 2022).
  68. SamKnows. Cellular. Available online: https://www.samknows.com/products/cellular (accessed on 29 July 2022).
  69. Dahunsi, F.; Akinlabi, A. Measuring mobile broadband performance in Nigeria: 2G and 3G. Niger. J. Technol. 2019, 38, 422–436. [Google Scholar] [CrossRef]
  70. Folasade, D.; Ayokunle, A.; Olumide, O.; Jide, P. Performance Monitoring of Mobile Broadband in a Developing Country. In Proceedings of the IST-Africa 2019 Conference, Nairobi, Kenya, 8–10 May 2019. [Google Scholar]
  71. OPENSIGNAL. Available online: https://www.opensignal.com/apps (accessed on 31 July 2022).
  72. MobPerf. Available online: http://www.mobiperf.com/ (accessed on 31 July 2022).
  73. RTR. RTR-NetTest. Available online: https://www.netztest.at/en/ (accessed on 1 August 2022).
  74. SpeedSmart. Powerful Speed Test. Available online: https://speedsmart.net/app (accessed on 31 July 2022).
  75. Umoh, V.B.; Ukommi, U.S.; Ekpe, U.M. A Simplified Method for Extended Duration Measurements of Mobile Broadband Performance. In Proceedings of the 2022 IEEE Nigeria 4th International Conference on Disruptive Technologies for Sustaina-ble Development (NIGERCON), Lagos, Nigeria, 5–7 April 2022; pp. 1–5. [Google Scholar] [CrossRef]
  76. Karn, N.K.; Hongli, Z.; Shafiq, M. Measuring broadband internet performance in Nepal: A comparative study. Procedia Comput. Sci. 2017, 107, 64–69. [Google Scholar] [CrossRef]
  77. Awan, M.F.; Ahmad, T.; Qaisar, S.; Feamster, N.; Sundaresan, S. Measuring broadband access network performance in Pakistan: A comparative study. In Proceedings of the 2015 IEEE 40th Local Computer Networks Conference Workshops (LCN Workshops), Clearwater Beach, FL, USA, 26–29 October 2015; IEEE: New York, NY, USA, 2015; pp. 595–602. [Google Scholar]
  78. Chetty, M.; Sundaresan, S.; Muckaden, S.; Feamster, N.; Calandro, E. Measuring broadband performance in South Africa. In Proceedings of the 4th Annual Symposium on Computing for Development, Cape Town, South Africa, 6–7 December 2013; pp. 1–10. [Google Scholar]
  79. Kousias, K.; Alay, O.; Argyriou, A.; Lutu, A.; Riegler, M. Estimating downlink throughput from end-user measurements in mobile broadband networks. In Proceedings of the 2019 IEEE 20th International Symposium on A World of Wireless, Mobile and Multimedia Networks(WoWMoM), Washington, DC, USA, 10–12 June 2019; IEEE: New York, NY, USA, 2019; pp. 1–10. [Google Scholar]
  80. Kousias, K.; Midoglu, C.; Alay, O.; Lutu, A.; Argyriou, A.; Riegler, M. The same, only different: Contrasting mobile operator behavior from crowdsourced dataset. In Proceedings of the 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Montreal, QC, Canada, 8–13 October 2017; IEEE: New York, NY, USA, 2017; pp. 1–6. [Google Scholar]
  81. Samknows. Analysis of Broadband Performance in Canada March & April 2016; The Canadian Radio-television and Telecommunications Commission (CRTC): Ottawa, ON, Canada, 2016. [Google Scholar]
  82. Ahmad, S.; Musleh, S.; Nordin, R. The gap between expectation & reality: Long term evolution (LTE) & third generation (3G) network performance in campus with test mobile system (TEMS). In Proceedings of the 2015 9th Asia Modelling Symposium (AMS), Kuala Lumpur, Malaysia, 7–9 September 2015; IEEE: New York, NY, USA, 2015; pp. 164–168. [Google Scholar]
  83. Parichehreh, A.; Moosavi, R.; Ramachandra, P.; Alfredsson, S.; Brunstrom, A. LTE as a Road Toward 5G: QoS Analysis in Mobility Scenario Using the Monroe Platform. In Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakesh, Morocco, 15–18 April 2019; IEEE: New York, NY, USA, 2019; pp. 1–7. [Google Scholar]
  84. Shayea, I.; Ergen, M.; Azmi, M.H.; Nandi, D.; El-Salah, A.A.; Zahedi, A. Indoor network signal coverage of mobile telecommunication networks in West Malaysia: Selangor and Johor Bahru. In Proceedings of the 2017 IEEE 13th Malaysia International Conference on Communications (MICC), Johor Bahru, Malaysia, 28–30 November 2017; pp. 288–293. [Google Scholar] [CrossRef]
  85. Turniski, F.; Lackovic, S.; Pilinsky, S.Z.; Tekovic, A. Analysis of 3G and 4G download throughput in pedestrian zones. In Proceedings of the 2016 International Symposium ELMAR, Zadar, Croatia, 12–14 September 2016; IEEE: New York, NY, USA, 2016; pp. 9–12. [Google Scholar]
  86. Daengsi, T.; Wuttidittachotti, P. Quality of Service as a Baseline for 5G: A Recent Study of 4G Network Performance in Thailand. In Proceedings of the 2020 IEEE International Conference on Communication, Networks and Satellite (Comnetsat), Batam, Indonesia, 17–18 December 2020; IEEE: New York, NY, USA, 2020; pp. 395–399. [Google Scholar]
  87. Daengsi, T.; Ungkap, P.; Wuttidittachotti, P. A Study of 5G Network Performance: A Pilot Field Trial at the Main Skytrain Stations in Bangkok. In Proceedings of the 2021 International Conference on Artificial Intelligence and Computer Science Technology (ICAICST), Yogyakarta, Indonesia, 29–30 June 2021; IEEE: New York, NY, USA, 2021; pp. 191–195. [Google Scholar]
  88. Daengsi, T.; Ungkap, P.; Wuttidittachotti, P. 5G Network Performance: A Study using Stationary and Mobility Tests on Sukhumvit Line–BTS Skytrain in Bangkok. In Proceedings of the 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), Depok, Indonesia, 14–15 September 2021; IEEE: New York, NY, USA, 2021; pp. 447–450. [Google Scholar]
  89. Daengsi, T.; Ungkap, P.; Pornpongtechavanich, P.; Wuttidittachotti, P. QoS Measurement: A Comparative Study of Speeds and Latency for 5G Network Using Different Speed Test Applications for Mobile Phones. In Proceedings of the 2021 IEEE 7th International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), Virtual, 23–25 August 2021; pp. 206–210. [Google Scholar] [CrossRef]
  90. Narayanan, A.; Ramadan, E.; Carpenter, J.; Liu, Q.; Liu, Y.; Qian, F.; Zhang, Z.-L. A first look at commercial 5G performance on smartphones. Proc. Web Conf. 2020, 2020, 894–905. [Google Scholar]
  91. Gran, E.G.; Dreibholz, T.; Kvalbein, A. NorNet Core–A multi-homed research testbed. Comput. Netw. 2014, 61, 75–87. [Google Scholar] [CrossRef]
  92. Dreibholz, T.; Gran, E.G. Design and Implementation of the NORNET CORE Research Testbed for Multi-homed Systems. In Proceedings of the 2013 27th International Conference on Advanced Information Networking and Applications Workshops, Barcelona, Spain, 25–28 March 2013; pp. 1094–1100. [Google Scholar] [CrossRef]
  93. Alay, O.; Lutu, A.; García, R.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. MONROE, a distributed platform to measure and assess mobile broadband networks: Demo. In Proceedings of the Tenth ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation, and Characterization, New York, NY, USA, 3–7 October 2016. [Google Scholar] [CrossRef]
  94. Alay, O.; Lutu, A.; García, R.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. MONROE: Measuring Mobile Broadband Networks in Europe. 2017. Available online: https://dspace.networks.imdea.org/handle/20.500.12761/947 (accessed on 1 January 2022).
  95. Mancuso, V.; Quirós, M.P.; Midoglu, C.; Moulay, M.; Comite, V.; Lutu, A.; Alay, Ö.; Alfredsson, S.; Rajiullah, M.; Brunström, A.; et al. Results from running an experiment as a service platform for mobile broadband networks in Europe. Comput. Commun. 2019, 133, 89–101. [Google Scholar] [CrossRef]
  96. Midoglu, C.; Kousias, K.; Alay, Ö.; Lutu, A.; Argyriou, A.; Riegler, M.; Griwodz, C. Large scale speedtest experimentation in Mobile Broadband Networks. Comput. Netw. 2021, 184, 107629. [Google Scholar] [CrossRef]
  97. Midoglu, C.; Wimmer, L.; Lutu, A.; Alay, Ö.; Griwodz, C. MONROE-Nettest: A configurable tool for dissecting speed measurements in mobile broadband networks. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Honolulu, HI, USA, 15–19 April 2018; IEEE: New York, NY, USA, 2018; pp. 342–347. [Google Scholar]
  98. 5GENESIS. Available online: https://5genesis.eu/ (accessed on 24 October 2022).
  99. 5GENESIS. Deliverable D2.4: Final Report on Facility Design and Experimentation Planning. 2020. Available online: https://5genesis.eu/wp-content/uploads/2020/07/5GENESIS_D2.4_v1.0.pdf (accessed on 1 January 2022).
  100. Zayas, A.D.; Caso, G.; Alay, Ö.; Merino, P.; Brunstrom, A.; Tsolkas, D.; Koumaras, H. A modular experimentation methodology for 5G deployments: The 5GENESIS approach. Sensors 2020, 20, 6652. [Google Scholar] [CrossRef]
  101. Christopoulou, M.; Xilouris, G.; Sarlas, A.; Koumaras, H.; Kourtis, M.-A.; Anagnostopoulos, T. 5g experimentation: The experience of the athens 5genesis facility. In Proceedings of the 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), Bordeaux, France, 18–20 May 2021; IEEE: New York, NY, USA, 2021; pp. 783–787. [Google Scholar]
  102. FED4FIRE+. Available online: https://www.fed4fire.eu/ (accessed on 14 January 2023).
  103. Demeester, P.; Van Daele, P.; Wauters, T.; Hrasnica, H. Fed4FIRE–The Largest Federation of Testbeds in Europe. In Building the Future Internet through FIRE; River Publishers: Denmark, The Netherlands, 2022; pp. 87–109. [Google Scholar]
  104. 5TONIC. Available online: https://www.5tonic.org/ (accessed on 14 January 2023).
  105. Nogales, B.; Vidal, I.; Lopez, D.R.; Rodriguez, J.; Garcia-Reinoso, J.; Azcorra, A. Design and deployment of an open management and orchestration platform for multi-site nfv experimentation. IEEE Commun. Mag. 2019, 57, 20–27. [Google Scholar] [CrossRef]
  106. Azcorra, A. Advanced 5G trials with verticals in 5TONIC laboratory. In International Ericsson All Employees Meeting of R&D; Ericsson: Madrid, Spain, 2017. [Google Scholar]
  107. Khatouni, A.S.; Mellia, M.; Marsan, M.A.; Alfredsson, S.; Karlsson, J.; Brunstrom, A.; Alay, O.; Lutu, A.; Midoglu, C.; Mancuso, V. Speedtest-like measurements in 3g/4g networks: The monroe experience. In Proceedings of the 2017 29th International Teletraffic Congress (ITC 29), Genoa, Italy, 4–8 September 2017; IEEE: New York, NY, USA, 2017; pp. 169–177. [Google Scholar]
  108. Shayea, I.; Azmi, M.H.; Ergen, M.; El-Saleh, A.A.; Han, C.T.; Arsad, A.; Rahman, T.A.; Alhammadi, A.; Daradkeh, Y.I.; Nandi, D. Performance analysis of mobile broadband networks with 5g trends and beyond: Urban areas scope in Malaysia. IEEE Access 2021, 9, 90767–90794. [Google Scholar] [CrossRef]
  109. Rajiullah, M.; Khatouni, A.S.; Midoglu, C.; Alay, Ö.; Brunstrom, A.; Griwodz, C. Mobile network performance during the COVID-19 outbreak from a testbed perspective. In Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization, London, UK, 21 September 2020; pp. 110–117. [Google Scholar]
  110. Shayea, I.; Ergen, M.; Azmi, M.H.; Nandi, D.; El-Salah, A.A.; Zahedi, A. Performance Analysis of Mobile Broadband Networks With 5G Trends and Beyond: Rural Areas Scope in Malaysia. IEEE Access 2020, 8, 65211–65229. [Google Scholar] [CrossRef]
  111. Al Jahdhami, M.A.; El-Saleh, A.; Alhammadi, A.; Shayea, I. Performance Analysis of Mobile Broadband Networks in Ibra City, Oman. In Proceedings of the 2021 International Conference on Artificial Intelligence and Big Data Analytics, Xi’an, China, 27–29 October 2021; pp. 1–6. [Google Scholar] [CrossRef]
  112. Fresolone, F.; Kloibhofer, R.; Ralbovsky, A.; Farkas, P.; Rakus, M.; Palenik, T. Throughput and one-way latency measurements in a 3G/4G live-network hi-mobility uplink. In Proceedings of the 2016 39th International Conference on Telecommunications and Signal Processing (TSP), Vienna, Austria, 27–29 June 2016; IEEE: New York, NY, USA, 2016; pp. 44–49. [Google Scholar]
  113. Albaladejo, M.B.; Leith, D.J.; Manzoni, P. Measurement-based modelling of lte performance in dublin city. In Proceedings of the 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Valencia, Spain, 4–8 September 2016; IEEE: New York, NY, USA, 2016; pp. 1–6. [Google Scholar]
  114. Engiz, B.K.; Kurnaz, Ç. Comparison of Signal Strengths of 2G/3G/4G services on a University Campus. Int. J. Appl. Math. Electron. Comput. 2016, 4, 37–42. [Google Scholar] [CrossRef]
  115. Koprivica, L.Đ.M.; Nešković, N.; Nešković, A. Experimental performance analysis of THE 2G/3G/4G public mobile network. In Proceedings of the 2016 24th Telecommunications Forum (TELFOR), Belgrade, Serbia, 22–23 November 2016; pp. 1–4. [Google Scholar] [CrossRef]
  116. Skocir, P.; Katusic, D.; Novotni, I.; Bojic, I.; Jezic, G. Data rate fluctuations from user perspective in 4G mobile networks. In Proceedings of the 2014 22nd International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, 17–19 September 2014; IEEE: New York, NY, USA, 2014; p. 180185. [Google Scholar]
  117. Imoize, A.; Adegbite, O. Measurements-based performance analysis of a 4G LTE network in and around shopping malls and campus environments in Lagos Nigeria. Arid. Zone J. Eng. Technol. Environ. 2018, 14, 208. [Google Scholar]
  118. Nikravesh, A.; Choffnes, D.R.; Katz-Bassett, E.; Mao, Z.M.; Welsh, M. Mobile Network Performance from User Devices: A Longitudinal, Multidimensional Analysis. In Passive and Active Measurement; Cham, M.F., Kuzmanovic, A., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2014; pp. 12–22. [Google Scholar]
  119. Elmokashfi, A.; Kvalbein, A.; Xiang, J.; Evensen, K.R. Characterizing delays in Norwegian 3G networks. In International Conference on Passive and Active Network Measurement; Springer: Berlin/Heidelberg, Germany, 2012; pp. 136–146. [Google Scholar]
  120. Wang, X.; Xu, C.; Jin, W.; Zhao, G. A First Look at Cellular Network Latency in China. In Communications and Networking; Chen, Q., Meng, W., Zhao, L., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; pp. 339–348. [Google Scholar]
  121. Xu, Y.; Wang, Z.; Leong, W.K.; Leong, B. An End-to-End Measurement Study of Modern Cellular Data Networks. In Passive and Active Measurement; Faloutsos, M., Kuzmanovic, A., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2014; pp. 34–45. [Google Scholar]
  122. Schwind, A.; Midoglu, C.; Alay, Ö.; Griwodz, C.; Wamser, F. Dissecting the performance of YouTube video streaming in mobile networks. Int. J. Netw. Manag. 2020, 30, e2058. [Google Scholar] [CrossRef]
  123. Schwind, A.; Seufert, M.; Alay, Ö.; Casas, P.; Tran-Gia, P.; Wamser, F. Concept and implementation of video QoE measurements in a mobile broadband testbed. In Proceedings of the 2017 Network Traffic Measurement and Analysis Conference (TMA), Dublin, Ireland, 21–23 June 2017; IEEE: New York, NY, USA, 2017; pp. 1–6. [Google Scholar]
  124. Casas, P.; Seufert, M.; Wamser, F.; Gardlo, B.; Sackl, A.; Schatz, R. Next to you: Monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 2016, 13, 181–196. [Google Scholar] [CrossRef]
  125. Daengsi, T.; Chatchalermpun, S.; Praneetpolgrang, P.; Wuttidittachotti, P. A study of 4G network performance in Thailand referring to download speed. In Proceedings of the 2020 IEEE 10th Symposium on Computer Applications & Industrial Electronics (ISCAIE), Penang, Malaysia, 18–19 April 2020; IEEE: New York, NY, USA, 2020; pp. 160–163. [Google Scholar]
  126. Budiman, E.; Wicaksono, O. Measuring quality of service for mobile internet services. In Proceedings of the 2016 2nd International Conference on Science in Information Technology (ICSITech), Balikpapan, Indonesia, 26–27 October 2016; IEEE: New York, NY, USA, 2016; pp. 300–305. [Google Scholar]
  127. Soós, G.; Ficzere, D.; Varga, P.; Szalay, Z. Practical 5G KPI measurement results on a non-standalone architecture. In Proceedings of the Noms 2020–2020 IEEE/IFIP Network Operations and Management Symposium, Budapest, Hungary, 20–24 April 2020; IEEE: New York, NY, USA, 2020; pp. 1–5. [Google Scholar]
  128. Makino, I.; Wang, Z.; Terai, J.; Miki, N. Throughput and Delay Performance Measurements in Multi-Floor Building Employing Private LTE. IEEE Access 2022, 10, 24288–24301. [Google Scholar] [CrossRef]
  129. Narayanan, A.; Zhang, X.; Zhu, R.; Hassan, A.; Jin, S.; Zhu, X.; Zhang, X.; Rybkin, D.; Yang, Z.; Mao, Z.; et al. A variegated look at 5G in the wild: Performance, power, and QoE implications. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference, Virtual, 23–27 August 2021; pp. 610–625. [Google Scholar]
  130. Xu, D.; Zhou, A.; Zhang, X.; Wang, G.; Liu, X.; An, C.; Shi, Y.; Liu, L.; Ma, H. Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, New York, NY, USA, 10–14 August 2020; pp. 479–494. [Google Scholar]
  131. Pan, Y.; Li, R.; Xu, C. The first 5G-LTE comparative study in extreme mobility. Proc. ACM Meas. Anal. Comput. Syst. 2022, 6, 1–22. [Google Scholar]
  132. Fiandrino, C.; Martínez-Villanueva, D.J.; Widmer, J. Uncovering 5G Performance on Public Transit Systems with an App-based Measurement Study. In Proceedings of the ACM/IEEE International Conference on Modelling, Analysis and Simulation of Wireless and Mobile Systems, Montreal, QC, Canada, 24 October 2022; pp. 65–73. [Google Scholar]
  133. Rochman, M.I.; Sathya, V.; Nunez, N.; Fernandez, D.; Ghosh, M.; Ibrahim, A.S.; Payne, W. A comparison study of cellular deployments in Chicago and Miami using apps on smartphones. In Proceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization, New Orleans, LA, USA, 31 January–4 February 2022; pp. 61–68. [Google Scholar]
  134. Bauer, S.; Clark, D.; Lehr, W. Gigabit broadband measurement workshop report. SIGCOMM Comput. Commun. Rev. 2020, 50, 60–65. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Proton Radiation Effects of CMOS Image Sensors on Different Star Map Recognition Algorithms for Star Sensors
Previous
Human Activity Recognition Based on Two-Channel Residual–GRU–ECA Module with Two Types of Sensors
Next