Capitalizing on Cellular Technology—Opportunities and Challenges for Near Ground Weather Monitoring ^†

Abstract

The use of existing measurements from a commercial wireless communication system as virtual sensors for environmental monitoring has recently gained increasing attention. In particular, measurements of the signal level of commercial microwave links (CMLs) used in the backhaul communication network of cellular systems are considered as opportunistic sensors for precipitation monitoring. Research results have demonstrated the feasibility of the suggested technique for the estimating and mapping of rain, as well as for monitoring other-than-rain phenomena. However, further advancement toward implementation and commercial use are heavily dependent on multidisciplinary collaborations: Communication and network engineers are needed to enable access to the existing measurements; signal processing experts can utilize the different data for improving the accuracy and the tempo-spatial resolution of the estimates; atmospheric scientists are responsible for the physical modeling; hydrologists, meteorologists, and others can contribute to the end uses; economists can indicate the potential benefits; etc. In this paper I will review state-of-the-art results and the open challenges, demonstrating the benefit to the public good from utilizing the opportunistic-sensing approach. I will also analyze the various obstacles on the way there.

1. Introduction

The relation between the rain intensity $R$ (in mm/h) and the attenuation of a microwave wireless signal $A$ traveling in the atmosphere is relatively simple:

$$A=a{R}^{b}l$$

(1)

where $A$ is in $\mathrm{dB}$, $l$ is the path length (in km) of the link and $a,b$ are constants, depending on the frequency and the polarization of the signal, as well as on the drop size distribution (DSD) of the rain, which is considered as typical to an area. This relation is a simplified model of complex physical relations [1,2] which has empirically been found to be a good approximation for the rain-induced signal’s attenuation, for microwave frequencies and for links of length of about 0.5–20 km. Equation (1), which become linear ($b=1$) for a certain choice of signal parameters, first suggested in Reference [3]. This raised the idea to use microwave links (MLs) for rainfall measurements in the early 90s [4,5,6], and it was experimentally tested in a multinational European project [7]. However, as the installation of dedicated MLs is costly, in combination with their limited coverage and questionable accuracy, this idea has not spread. In 2006, Messer et al. [8] first demonstrated the idea of taking advantage of the existing, widely spread, cellular communication technology, and used the MLs, which are part of its backhaul network, for environmental monitoring. While the relation (1) still forms the basis of this idea, the use of commercial microwave links (CMLs) instead of dedicated MLs as in Reference [9], brings new opportunities, as well as challenges. The major opportunity is obvious: the availability of millions of potential virtual meteorological sensors almost everywhere on Earth, with no costs for installation, maintenance, or communication. Since 2006, interest in this technology has rapidly increased and many research groups around the world are contributing to it. However, the fact that it has not yet been commercialized is indicative of the challenges its implementation poses.

In this paper, I will review the most advanced CML technology and its future directions as regards becoming an operational environmental monitoring system.

2. Materials and Methods

The CML-based weather monitoring technology depends on the availability of materials: measurements of the received signal level (RSL) and the transmitted signal level (TSL) from the microwave backhaul network of a cellular communication system. In most countries, a cellular company owns the infrastructure, so the required measurements are owned by a private company. While the use of measurements of the transmitted/received signal levels is of no risk to either the communication services or to the privacy of the users, most cellular providers are reluctant to provide a third party access to their intra-network. On the other hand, researchers interested in CML technology have approached cellular companies and succeeded in receiving measurements. The following sections explore these protocols.

2.1. The Passive Approach

Manufacturers of the backhaul transmission networks have implemented tools in their systems which monitor and log the signal levels of all links in the network. The tool, known as the network management system (NMS), produces RSL and TSL indicators which are automatically logged by the network operators (i.e., the cellular providers). The passive approach relies on the use of the already existing NMS records as inputs for the CML weather monitoring technology. The major advantage of the passive approach is that it puts neither burdens nor risks on the cellular providers, so it is relatively simple to get them to share this data. However, as the NMS data is kept for network monitoring, and in particular, for monitoring the actual link budget, the RSL (and TSL) signals in the NMS go through a highly nonlinear process. Typically, only the minimum and the maximum RSL (and TSL) values, from the measurements taken over a window of 15 min, are stored. Moreover, these values are quantized at a 0.1–1 dB resolution. Furthermore, as they are mostly used for analysis, the NMS records are rarely available in real time. An example of a typical RSL time series from an NMS record is depicted in Figure 1.

2.2. The Active Approach

Modern microwave communication networks are remotely managed. That is, the network operators can access and inquire the status of the different CMLs remotely. Specifically, most CML hardware is connected to the provider’s intra-net, and uses the simple network management protocol (SNMP) to submit queries to the CMLs, and receive the requested information. The active approach is to use the SNMP to collect RSL measurements dedicated for weather monitoring. A recent publication details this methodology, and establishes a set of open-source tools which can be used to actively access the CMLs of main manufacturers, and receive the instantaneous RSL (and TSL) samples [9]. With this approach, RSLs (and TSLs) are available in real time as instantaneous samples, at sampling intervals that can be as small as 10 s. Note that to avoid unnecessary traffic load for the cellular provider, the active approach also requires adding a designated server for handling the RSL measurements. Also, in most cases, the RSL and the TSL samples collected by this approach still suffer from quantization, as the quantization process is a property (and a limitation) of the sampling hardware itself.

CML measurements collected by the active approach are most suitable for environmental monitoring, both because of the excellent temporal resolution and the lack of the highly nonlinear min/max processing of the NMS. Moreover, the availability of real time measurements is most attractive for applications such as now-casting and flood prediction. However, the active approach requires a high level of involvement from the cellular provider, including permission to a third party to cross its firewall. Most providers are reluctant to allow it, as they see it as a potential risk to their main business, i.e., communication.

Table 1. Available commercial microwave link (CML) measurements.

Characteristics	Passive	Active
Source of measurements	Existing records from network management systems (NMS)	Designated data collection system
Temporal resolution	Minutes-days Typical-15 min	Seconds-minutes Typical-10 s
Non-linear preprocessing	Typically min/max values over a given interval	Non
Quantization	Yes	Yes
Major advantage	Simple access, no risk for cellular operators	Real time
Major disadvantage	Not available in real time	Hard to get
Summary	Recommended for research purposes and for historic studies	Essential for real time applications

summarizes the two approaches.

2.3. Methods

If instantaneous RSL/TSL measurements are available (as in the Active approach), the total attenuation (in dB) of the CML’s signal can be extracted by subtracting one from the other, and it can be described by [10]:

$$A_T(t,l)=A_0(l)+A_P(t,l)+A_H(t,l)+A_C(t)+N(t)$$

(2)

where $t$ is the time index and, as in Equation (1), $l$ is the length of the link. $A_0$ is the frequency-dependent propagation loss which is constant over time. $A_P$ is the attenuation caused by precipitation, if existing. For rain, for example, $A_P=A$ of Equation (1). $A_H$ is the attenuation due to hydrometeors other than precipitation, e.g., fog or water vapor. This component changes slowly over time, while being small compared with $A_P$, if existing. $N(t)$ represents the measurements noise, and $A_C(t)$ is a component which represents the disruption of the signal by other objects, if existing. Note that Equation (2) is a simplified description of the heavily studied physics of the attenuation of a signal propagating in the atmosphere [11,12], which has a great effect on the performance of communication systems [13].

Depending on the application, the first stage is to isolate $A_P(t,l)$ (for precipitation monitoring), or $A_H(t,l)$ (for monitoring other-than-rain atmospheric phenomena). This is commonly done by using side information, or by taking advantage of the built-in diversity of CML technology, as there are usually multiple CMLs in a given area of different lengths and frequencies (e.g., Reference [14]). The next step is to estimate the parameter of interest (e.g., rain-rate) from the corresponding term, using the known relation between the signal attenuation and the phenomenon of interest. A flowchart of this possible process is presented in Figure 2.

Obviously, measurements collected by the passive approach are far from being ready to use for this analysis. The instantaneous total attenuation $A_T(t,l)$ required by Equation (2) cannot be extracted from the min/max indicators of the TSL/RSL. A systematic approach, for example, for extracting rain-rate estimates from extreme measurements provided using the passive approach is described in Reference [15] and is depicted in Figure 3.

3. Results

Since first introduced in 2006, research groups from different disciplines have started to study this technology, and dozens of peer-reviewed papers have been published. The references include many selected publications of studies which are CML measurement-based. In general, these papers can be divided into four groups:

• Papers in which the capabilities of CML technology for environmental monitoring have been demonstrated (see

  Table 2. Demonstrations of capabilities.

  Atmospheric Phenomenon	Reference
  Rainfall sensing	[8,16,17,18]
  Rainfall mapping	[8,19]
  Humidity sensing	[20]
  Fog sensing	[21,22,23]
  Precipitation classification	[24]
  Dew detection	[25]
  Wind estimation	[26]
  Air pollution detection	[27]

  ). Naturally, foremost potential is attributed to the near-ground rainfall monitoring capability. Several papers demonstrated the CML as a rainfall sensor, and many CMLs as a sensors network, capable of 2D rainfall mapping. Later, other papers have demonstrated the use of CMLs for monitoring other-than-rain phenomena, including humidity, fog, dew, snow and sleet, and even wind and air pollution (indirectly).
• The next step was to study the accuracy of CMLs as virtual rainfall sensors. Since cellular networks have been designed to operate optimally for efficient telecommunication service and not for measuring rain (or other atmospheric variables), its opportunistic use for rain monitoring is challenging, since the network must be taken as is.

  Table 3. Errors and uncertainties analysis.

  Sources of Errors	Reference
  General	[10,28,29,30,31]
  Dry/Wet	[32,33,34,35]
  Wet antenna	[36,37,38]
  Calibration	[39]
  Quantization bias	[40]
  Non-linear preprocessing	[15,41]
  Network topology	[42]

  presents a summary of the major contributions to an errors and uncertainties analysis. The analysis aims at quantifying the different sources’ errors and their effect on the resulting rain estimates. Generally speaking, the uncertainties can be put into two groups: one which is related to physical, atmospheric effects, e.g., wet antenna, which cause attenuation that may read as higher rain-intensity value in Equation (1) if not properly handled. The second group consists of errors caused by the opportunistic use of existing technology not aimed at atmospheric monitoring. This may include signal quantization and non-linear pre-processing (applied on the signal for efficient network management), as well as errors resulting from the non-optimal, given spatial spread of links and frequencies in the CML network, when being used for atmospheric monitoring.
• In

  Table 4. Algorithms and tools.

  Focus of the Algorithm	Reference
  Instantaneous rain mapping	[19,45,46,51,52,53,54]
  Dynamic rain mapping	[48,49,55]
  Heavy rain detection	[56]
  Merging with other measurements	[50,57,58]
  Rainfall tomography	[47,59]
  Accumulated precipitation	[60]
  Open software tools	[43,44]

  , a list of papers suggesting algorithms for rainfall monitoring is presented. As the main opportunity in CML technology is in near ground, bottom-up rain mapping, most algorithms are focused on this. The straightforward approach is to treat each CML as a local point measurement and to interpolate local measurements to a grid, using standard spatial interpolation techniques (e.g., inverse distance weighting IDW, Kriging, etc.). On the basis of this approach, open software tools were developed [43,44]. More advanced algorithms have been developed by signal processing experts, on which the tempo-spatial resolution of the rainfall maps, their accuracy and their coverage have been improved by exploiting the spatial spread of the CML measurements. Different authors used different approaches, such as: an iterative approach in which variability of rain along the links is exploited [19]; a compressed sensing approach [45,46]; a model based, parametric approach; a tomographic approach [47]; and dynamic mapping [48,49]. The main future challenge is to improve CML rainfall maps by merging with other types of measurements (mostly radar), where these exist (see Reference [50] for a review of this issue).
• Table 5. Applications and use.

  Application	Reference (Year)	Area/Comments
  Large scale rainfall estimation/mapping	[61,62]	Holland
  Rainfall measurements	[63,64]	Africa
  	[65]	Israel
  	[66,67]	Germany
  	[68]	Holland
  	[69]	Ecuador
  Flood prediction	[70]	Israel
  Disaster alarm	[71]
  Calibration of other sensors	[72,73,74,75,76,77]
  Hydrology	[78,79,80,81,82]	Urban drainage

  details a partial list of applications. In all papers in this table, actual CML measurements were employed and empirical results were presented and validated over time, in different climatological areas.

Lately, after more than a decade of expanding research, the proposed approach has finally gained the attention of the private sector. First, Ericsson initiated a pilot project [83], and in 2015 a startup company was established [84].

4. Conclusions

The CML environmental monitoring technology was introduced and has developed into academic research. By negotiating with local cellular providers, multidisciplinary research groups all over the world have received access to CML measurements in their countries, mostly for no cost, and are studying different aspects of this technology. Most of these groups are now collaborating and sharing experience, tools and knowledge in different ways, so a new scientific community has been built. While the achievements of this community are impressive, the road ahead is challenging.

4.1. The Commercialization Challenge

Proving the feasibility of CML technology and having an active scientific community are most important for the sustainability and for the future advancement of this emerging technology. However, the next step in the journey is the technology transfer for the public good. The established research is an important component, but is not sufficient. A necessary condition for CML technology to be used is to ensure access to measurements. Fortunately, the changes in the communication markets push cellular companies to look for new business, so they are now more open to explore the potential of creating revenues from CML technology. Another part in this equation is the market itself, in which measurements and (big) data of any kind become valuable assets. Multinational companies such as IBM™ and Google™ are now interested in weather, and CML technology is the best source for weather-related (big) data. Note, however, that sustainable access to the measurements is a key issue for the commercial use of CML technology, and a necessary pre-condition for its practical use.

4.2. Potential Use

The vast research reviewed in this paper indicates the great potential of CML technology in future environmental monitoring, once the availability of measurements is granted. Potential use can be divided between three main families:

• Covering blind spots. There are areas where almost no near-ground measurements are available. One such example includes country-wide areas in developing countries, such as Africa [60,63]. Other examples are local, and include specific challenging landscapes such as slopes and urban areas, where traditional ground weather stations are known to be less reliable. Even with the limited accuracy of CML technology, in cases where there is no alternative, its potential is extremely important.
• Improving monitoring accuracy. Even in areas where the coverage of conventional weather-monitoring facilities (e.g., gauges and radar) is good, the use of additional ground-level measurements can improve performance. The potential improvement highly depends on the topology of the network (e.g., its density) and on the temporal resolution of the available measurements.
• Improving models. Complex meteorological and hydrological models, used for forecasting, are continuously improved by comparing their predictions to actual measurements. CML technology offers a new dimension of data to be assimilated in such models.

4.3. Limitations

As discussed in Reference [53], cellular networks have been designed and dimensioned to operate optimally for efficient telecommunication service and not for measuring rain (or other atmospheric variables). The design of microwave links as designated sensors for the observation of the lower atmosphere would be very different. This raises an intriguing scientific challenge for the research community. Signal processing and machine learning algorithms are being developed to overcome limitations in the measurements. Moreover, the question of potential improvements in performance resulting from using CML measurements is an open, and important one. If it can be theoretically proven that the potential performance improvement is significant, then the motivation to use CML technology will be higher. Note, however, that performance can be defined in different ways, depending on the application. It can be the accuracy of measuring total rainfall in a given area and time slot, or accuracy of measuring instantaneous rain rate at a given point, or accuracy of the spatial, 2-D representation of the rain in a given time slot or period, etc. The analysis and the results will depend on the CML network as well as on the characteristics of the atmospheric phenomenon under study (e.g., the spottiness of the rain). In Reference [42], for example, the achievable spatial resolution of rain mapping was studied and has been characterized as a function of the sparsity of the rain, as well as of the statistical features of the CMLs in the area.

4.4. A Test Case for Opportunistic Sensing of the Environment

CML technology has recently been mentioned as one of the first Internet of Things (IoT) applications [85], which is now getting much attention. While the conditions for CML technology becoming useful seems to be right, and we may see it soon in products as well as in public-good services, it also serves as a pioneering example of the new trend of capitalizing on existing technology by utilizing it for non-intended, opportunistic use [86,87,88]. Opportunistic sensing is believed to be the future of environmental monitoring, being a sustainable source of (big) environmental data. The analyses provided in this paper for the case of CML technology can serve as a test case for this emerging trend of opportunistic sensing, demonstrating the need for open bi-directional communication channels within the world of academia, where innovative ideas are initiated and studied, and for contemporary industry, which serves as a source of opportunistic measurements as well as a platform for utilization of new ideas.

To conclude: The uniqueness of CML technology stems from its special situation, standing between science and technology, between academia and industry. Future development of this technology and its potential use in practice depend on business challenges as well as on science and technology. The focus of this paper is to review the most advanced developments in CML technology and to anticipate its future development, based on an analysis of the opportunities and challenges faced by researchers in this area over the years. Depending on the future evolution of CML technology, it will be important to provide a deep scientific critical review of this technology, including meticulous scientific background on the different sources to the signal’s attenuation, comparative analysis of the different algorithms, etc.