Design and Implementation of a Monitoring System Using Optical Camera Communication for a Smart Factory

Abstract

Wireless technologies that are based on radio frequencies are currently widely used, with numerous applications around the world. However, they pose some disadvantages to human health. High frequencies can have potentially harmful effects on children, hospital patients, and even healthy people if the signal power exceeds the permitted standard. Conversely, the use of visible light for data transmission is a trend that presents new options, including optical wireless communication, optical camera communication, and visible light communication. This paper proposes a modulation scheme based on on-off keying in the time domain, which is applied to a monitoring system using optical camera communication. This scheme has various compatible supports for the global-shutter camera and rolling-shutter camera, which are popular commercially available cameras. Furthermore, this scheme facilitates a low-cost monitoring system. By using small light-emitting diodes (LEDs) and controlling the exposure time in a single camera, the camera, as a receiver, can simultaneously detect signals from up to 10 sensor devices in different positions at a maximum distance of up to 50 m, with a low error rate.

1. Introduction

Currently, the Internet of Things (IoT) is widely applied for various purposes in different areas such as system-monitoring systems for smart factories, smart agriculture, smart homes, and healthcare [1]. The purpose of IoT is to connect physical devices with the virtual world, using internet protocol as the intermediary to communicate data [2]. This system includes computing devices, machines, animals, and people, all of which are assigned unique identifiers and the capacity to transmit data over the network automatically, without human interaction. However, current techniques that involve IoT are mostly based on radio frequency (RF) technology, such as ZigBee, Bluetooth, and WiFi. It is important to note that radio frequencies come with potentially harmful effects on human health [3], especially for children, old people, and hospital patients. Therefore, many researchers focus on finding new wireless technologies to replace RF technologies. The use of visible light for data transmission highlights a good alternative to replace RF technologies, including new viable candidates such as visible-light communication (VLC), optical wireless communication (OWC), and optical camera communication (OCC), whose advantages over RF communication are as follows:

• Visible-light wave bandwidth is more than 1000 times the RF bandwidth.
• Visible light waves do not negatively affect human health [4], while RF waves generate electromagnetic interference, which not only reduces the performance of the system but also affects human health.
• Visible light waves are more efficient and secure if the channel model achieves line-of-sight (LOS) transmission.

With such great potential, companies are extensively funding research on improving this technology. Standards for OWC technologies have been presented in IEEE 802.15 [5]. In [5], the uniqueness of OWC and its wide range of protocol complexity acceptance are also recognized. The purpose of the IEEE 802.15.7m Task Group (TG 7m) is to amend the previous standard, which was classified into four categories:

• Existing VLC modes available in the IEEE 802.157-2011 standard [6].
• OCC: baseband modulation techniques that use cameras or image sensors to receive signals from a light source.
• Light-emitting diode identification (LED-ID): communication techniques that use photodiodes at low speeds, with data rates below 1 Mbps at the physical layer.
• High-speed Light Fidelity (Li-Fi): high-rate photodiode modulation techniques with data rates exceeding 1 Mbps at the physical layer. Li-Fi technologies utilize optical orthogonal frequency division multiplexing modulation.

Because of improvements in manufacturing technology, LEDs have the potential to become next-generation lighting sources, owing to their advantages in terms of long life, power consumption, and good visibility. Their ability to switch on/off at a fast rate [7,8] facilitates high-rate transmission in OWC/OOC and VLC systems.

As mentioned in [9,10], RF technologies generate electromagnetic interference (EMI), whereas visible light technology has emerged as a novel EMI-free alternative that could replace RF technologies [11]. A conventional VLC system includes LEDs that transmit data based on fast-switching on/off status and photodiodes (PDs) on the receiver side, which receive power from LEDs and decode data [12]. In [13,14], the authors implement an ultra-high-speed PD modulation scheme, based on VLC, with extremely effective spectral properties. Multiple-input multiple-output (MIMO) technology, with an ultra-high-speed multichannel for the VLC system, is also presented in [15,16]. However, the disadvantage of a VLC or Li-Fi system is that it only applies to short transmission distances, so using PDs as detectors limits the VLC or Li-Fi system’s suitability to indoor environments only. Unlike VLC or Li-Fi communications that use PDs as receivers, the OCC technique uses an image sensor as the detector to receive data. The authors of [17,18] assert that OCC system performance depends on the type of image sensor. Two popular types of cameras, which are currently available commercially, are the global-shutter camera and rolling-shutter camera. OCC performance in the global-shutter camera is primarily decided by the camera frame rate, which determines whether the sampling rate satisfies the Nyquist sampling requirement. In the rolling-shutter camera, the sampling rate also depends on the camera frame rate and rolling rate. The camera lens can focus the light beam on the image sensor and thus enhance the Signal to Noise (SNR). PDs without a lens are unable to do this. Li-Fi researchers are currently designing a PD-detector system with a lens that can extend the communication distance, yet the maximum distance is only 10 m, as noted in [19].

Furthermore, as the camera lens can spatially separate light sources, our OCC system can implement MIMO. This makes the simultaneous connection with many light sources feasible for the camera, whereas the massive amount of links makes this infeasible for the PD-based system. In addition to its many advantages, OCC technology has been employed for many different applications. Using region-of-interest (RoI) signaling, a waveform for a vehicular system has been presented in [20]. Based on the rolling-shutter effect, in [21] the authors present camera on-off keying (OOK) modulation with a high data rate, but the rolling-shutter method has some drawbacks, including the limited transmission distance and high error rate. Two-dimensional orthogonal frequency division multiplexing (OFDM) for OCC, using the screen method, is discussed in [22], which demonstrates a high data rate but also reveals a disadvantage: the large size of the transmitter and short distance make it an expensive transmitter.

In [13,23], it propose a mobile environment monitoring system that uses color LEDs to transmit data. The use of color LEDs has some disadvantages compared with OOK, such as the short transmission distance and high error rate. In this paper, we discuss a monitoring system that uses the OCC technique, which is a combination of RoI signaling waveforms and MIMO-OOK modulation. This OCC system takes advantage of Closed-Circuit Television (CCTV) as its receiver because this system is easily deployable as an environment monitoring system. Furthermore, by using small LEDs, low-cost transmitters and power consumption reduction can be realized. Furthermore, our MIMO-OOK modulation proposal aims to increase the data rate for large numbers of sensors.

The remainder of this paper consists of four Sections. In Section 2, we survey OOK modulation and highlight the contributions of the scheme. Section 3 describes the system architecture for the monitoring system. Finally, Section 4 presents the results of implementations, which concludes the paper.

2. Our Contributions

Throughout this study, we applied MIMO-OOK modulation to an environment monitoring system. Multiple LED is proposed to increase the data rate in the case of nodes with a lot of payload from many sensors. This scheme has many advantages, which are outlined below:

• Multi-users detection support: the selection and initialization of links of interest among thousands of artificial light sources are obstacles to be eliminated. Rx can detect active light and initialize a connection within the allowed delay time. The conditional method for object detection is based on a conditional computer vision (CV), but this reveals some disadvantages. For instance, CV can detect exactly which objects seem to improve the OCC link setup, but this is complex and has a long processing time delay. As such, we need another technique that can facilitate quick detection and suits the OCC system. Moreover, RoI signaling techniques support the OCC system, as has already been presented in IEEE 802.15.7m. Kookmin University and Intel have contributed to this concept, during meetings of the IEEE 802.15.7m Task Group. A camera with an RoI mode is permitted to set up the communication link for better quality with a shorter time delay.
• Support for all types of commercially available camera: the method of using OOK modulation for the OCC system can be applied to both the rolling-shutter camera and global-shutter camera. RoI signaling can similarly be applied to both the rolling-shutter camera and global-shutter camera.
• Low power consumption and low cost: using small LEDs to transmit the environment parameters with a sleep mode significantly reduces power consumption, compared with RF technologies. Consequently, the module in the transmitter side can operate for long periods of time without the need for frequent battery replacement. Furthermore, the system’s design is very cost-efficient. The transmitter side has a Microcontroller (MCU), sensors, and small LEDs for the transmitter. The receiver side in this proposed scheme is compatible with all types of cameras, including CCTVs and webcams.
• Eliminate the noise light source: RoI technology helps to detect the position of light sources in the image. However, it is necessary to differentiate between the noise signal and the Tx signal. Therefore, after using the RoI technique to detect objects, the sampling signals from all objects are checked.

3. System Architecture

The OCC system’s main concept is to use optical signal intensity for modulation and demodulation, with the purpose of communication, since good modulation and demodulation can improve communication efficiency. In OCC systems, the encoding technology for communication is generally based on OOK, pulse position modulation (PPM), binary phase-shift keying (BPSK), and OFDM. The analysis and comparison of different modulation approaches reveal that the BPSK method can effectively decrease inter-symbol interference in wireless channels, but its transmission information ability and bandwidth utilization rate are both very low. As important modulation methods used in OCC systems, the PPM and OFDM methods have high transmission rates. However, they also have low anti-interference abilities and high data–error ratios to external light and are complex to implement. After analyzing and comparing compatibility support aspects for image sensors, we adopted an OOK method for our OCC system which is presented in Figure 1. This scheme is a viable option for a factory monitoring system as it is easy to implement, reduces power consumption, and is compatible with various types of cameras.

3.1. Line Coding

Run-length limited (RLL) coding is a line-coding technique that is used to send arbitrary data over a communications channel that has bandwidth limits. RLL codes are defined by four main parameters: m, n, d, and k. The first two, m and n, refer to the rate of the code, while the remaining two specify the minimum (d) and maximum (k) number of zeros between consecutive ones. Particular RLL codes are used in light communication to provide DC-balance since flicker mitigation is required in VLC (except in some OCC modes). With the proposed scheme, we recommend three types of line coding: Manchester code, 4B6B, and 8B10B, which also proposed in IEEE 802.15.7-2018 standard [6].

3.2. Add Start Frame

A preamble symbol is a signal applied in communications to synchronize and detect the start frame, which can be called a header. This is a special bit sequence, as the preamble length can affect the performance of the system. This is because the redundant part (that does not carry the desired data) is added to synchronize the start frame. Then, the preamble length needs to be carefully considered. The preamble should be configured so that it is suitable for the RLL code, which is used in the data packet, as given in

Table 1. Coding scheme and proposed preamble.

RLL Code	Proposed Preamble (SF)
Manchester	011100
4B6B	0011111000
8B10B	0000111111111100000

.

3.3. RoI Detection

Using RoI to detect objects has been the focus of research for many years. Almost all RoI algorithms use two approaches: object-based and feature-based [24]. The feature-based approach searches pixels that have significant optical features and then uses analysis and a target to make RoIs. It can collect almost all target pixels based on the similarity of optical features. As all pixels do not have a strong feature that is shown in optical pixels, the detection using feature-based methods is usually not sufficient for identifying the entire target. Conversely, the object-based method can detect RoIs better than the pixel-to-pixel feature-based method, as it utilizes information about the structure and shapes for detection. This includes many typical approaches, such as matched filter and template matching [24]. However, the method is limited by its many calculation requirements, and it is also unreliable and produces low-quality images.

In this work, RoI methods were used to detect LED objects in each image, which were captured by a camera. The detection performance may not be as good as Computer vision functions but RoI detection presented its compatible with OWC/OCC system due to the short time processing caused by its simple processes. RoI methods are deployed by only processing a subset of the pixel to detect LEDs in images. From that, the position of signals is detected and demodulated in the receiver side. The concept of RoI-signaling technique for OCC has recently been introduced into IEEE 802.15.7m task group [25].

4. Implementation

4.1. Pixel E_b/N₀ Computation and Noise Modeling

The pixel noise in CCD/CMOS cameras can be approximately modeled using Equation (1), according to [25].

$$n~N(0,\delta {(s)}^2),$$

(1)

where s is the pixel value, ${\delta }^2(s)=s.a.\alpha +\beta$, with $a$ being the mark and space amplitude, and $\alpha$, $\beta$ indicate the fit parameters obtained in the experiments. Here our system uses the model-fitting constants $\alpha =0.01529$, $\beta =0.1973$ as the mode-fitting constant values in our system, which were estimated experimentally in [25]. The pixel $E_b/N_0$ of the Rx camera can be estimated using Equation (2) [25], with the assumption that one symbol contains one bit.

$$Pixel\frac{E_b}{N_0}=\frac{E\left[s^2\right]}{E\left[n^2\right]}\approx \frac{a^2\times \Delta }{a\times \alpha \times \Delta +\beta },$$

(2)

where $E_b$ represents the bit energy, $N_0$ is the noise density, $s$ is the pixel value, $a$ mark and space amplitude, $\Delta =T_{\exp osure}/T_{bit}$ is the ratio of camera exposure time and bit interval, and $\alpha$, $\beta$ are the fit parameters.

As no camera parameters have any existing criterion, the evaluation should be performed for each camera. Figure 2 presents an example of a theoretical estimation of pixel SNR as a nonlinear curve, using Equation (2), with a rolling-shutter camera (10 kHz shutter speed).

4.2. Experimental SNR Measurement

SNR was measured to ensure that the SNR criterion of 10 dB required to achieve a BER of ${10}^{-4}$ in OOK modulation is satisfied. In the experiments, a small LED (5V DC-0.2 W) was used to measure SNR. Figure 3 shows the devices used in this measurement. A Point Grey rolling-shutter camera was set up with exposure times of 2000, 3000, and 4000 µs, and a distance of 20–50 m. A small LED (5 V- 0.2 W, 2 mm in diameter) was used in a model of the system.

Figure 4 demonstrates the SNR computation of the 20, 30, 40, and 50 m scenarios. The red line represents pixel amplitude when there is no dominant light signal from the LED, which is considered background noise. Furthermore, for the divergence of distance, the blue line illustrates pixel amplitude when there is a dominant light signal from the LED. From this, the SNR (in dB) can be expressed as:

$$SN{R}_{dB}=20\log \left(\frac{\sqrt{\frac{1}{n}{{\displaystyle \sum _{i=0}^{n-1}\left|A\_o{n}_{{}_i}\right|}}^2}}{\sqrt{\frac{1}{n}{{\displaystyle \sum _{i=0}^{n-1}\left|A\_of{f}_{{}_i}\right|}}^2}}\right),$$

(3)

where A_on denotes the measured pixel amplitude with the LED turned on, A_off is the measured pixel amplitude with the LED turned off, i is the i-th of measured samples and n is the number of measured samples.

The received pixel amplitude at the camera will be larger over a short communication distance, and vice versa. Another critical factor that affects the SNR calculation is shutter speed, otherwise known as the camera’s exposure time. Figure 5 illustrates the relationship between pixel SNR and pixel intensity for different exposure times (2000, 3000, and 4000 µs). The SNR was measured based on changes in exposure time and distance. With long distance, the received signal power is low, lead to the SNR measurement is small, and the BER performance increases. However, the SNR measurement can be controlled by changing the exposure time, which is shown in Figure 5. The image sensor operates as a low-pass filter; so, a greater exposure time will smoothen the high-tone signal. When the exposure time increases, the communication bandwidth decreases, thus reducing the total noise power spread over the bandwidth. This proves that if the exposure time is set longer, the SNR result is higher.

In summary, the LOS link always guarantees the minimum SNR of 10 dB that is required for a BER value of ${10}^{-4}$, which is lower than that with single-carrier modulation. The SNR can be increased to more than 34 dB by controlling the camera’s exposure time. However, the decrease in communication bandwidth must be carefully considered to guarantee system performance. Furthermore, higher exposure times increase the probability of fuzzy LED states. To mitigate the higher ratio of fuzzy states, OOK modulation must employ forward error correction or a smart decoding algorithm (such as an artificial intelligence decoder).

4.3. BER Estimation for Optical OOK Method

In this communication modulation process, the received electrical signal can be calculated as [26]

$$r(t)=I(t)+{\displaystyle \sum _{i=-\infty }^{+\infty }I(t)a_{i}g(t-i{T}_s)}+n(t),$$

(4)

where $a_i$ is the level of the i-th symbol and $a_i$∈ {−1,1}; the transmission probabilities of bit −1 and 1 are $P_0$ and $P_1$, respectively, g(t) is the rectangular pulse shape function and $T_s$ is the symbol time. When there is no turbulence, and only Additive white Gaussian noise (AWGN) is present, the BER can be written as: [27]

$$P_e=\frac{1}{2}erfc(\sqrt{\frac{E_b}{2{\sigma }_n^2}}).$$

(5)

As the transmitted signal corresponding to bit “0” is zero, the received signal for this bit will only have AWGN. However, both turbulence noise and AWGN are present for a bit “1.” Therefore, the signal currents for bits “0” and “1” can be written as:

$$r(t)=\left\{\begin{array}{l}n(t),a_i=-1\\ 2I(t)+n(t),a_i=1\end{array}\right..$$

(6)

Figure 6 shows the bit error probability of Optical-On Off Keying(O-OOK) modulation versus pixel E_b/N₀ based on the AWGN channel. The paragraph illustrates that O-OOK schemes require a pixel E_b/N₀ of at least 11 dB to achieve BER of ${10}^{-4}$. Comparing with the theoretical pixel E_b/N₀ calculation in Figure 2 and the result of experimental SNR as shown in Figure 6, the proposed scheme can be gained BER of ${10}^{-4}$ at distance of 10 m or more by controlling suitably the exposure time of the camera. However, as mentioned below, the SNR can be improved by changing a longer exposure time, however, it causes the reducing of the communication bandwidth. From that, the setting of exposure time needs to be considered carefully to achieve better performance.

4.4. Proposed Scheme

4.4.1. Data Frame Structure

Figure 7 illustrates the proposed data frame structure for the MIMO-OOK scheme. To access many users, we added the node ID part to each frame. Each user will be defined by a unique ID so that the receiver can categorize the signal from different users. Defining users by ID nodes helps the system accept dozens of users, up to hundreds of users. Furthermore, when module devices have many sensors, the amount of data is large. As such, we propose the use of multiple LEDs to increase the data rate. Additionally, we propose the addition of LED-ID to the data frame structure to support the receiver’s detection and collection of data, as shown in Figure 7.

4.4.2. Detect Signal

As previously discussed, we added a preamble into each packet to detect the start frame. This is a special bit sequence, whose definition is based on the RLL code that was used, and which both the transmitter and receiver know in advance. A preamble has two tasks. Firstly, the receiver can classify the signal light source and unexpected light sources (such as background light and noise light). As shown in Figure 8, the signal noises have intensity values changing chaotically while the intensity values of the expected signal are modulated based on On-Off-Keying modulation with discriminating intensity values represented for 0 level and 1 level in On-Off-Keying modulation, which is shown in Figure 9. By using the preamble part for each data frame, payload data is inputted between two preambles. By looking, we can easily and rapidly categorize the expected signal and noise signal. At the receiver side, we just need to detect the known preamble at all signals. If the received signal stream has the preamble part, it is the desired signal and vice versa. The preamble’s second task is to help the receiver identify the starting point of the transmission frame. After categorizing the expected noise, the signal is passed through the matched filter to detect the position start frame. The desired data is in the middle part, between two adjacent preambles. Then, we can decode and categorize data packets from any user, based on the LED-ID and Node ID, which is shown in Figure 7.

4.4.3. Minimum Distance Between Two LEDs

By using RoI to detect sources from LEDs, many users can be allocated to only one camera. However, the number of users that can be accepted in this system depends on the focal length, work distance, and distance between the two LEDs. Their relationship is presented in Figure 10.

The relationship between the focal length and work distance [28] is

$$\frac{d[m]}{D\_LEDs[m]}=\frac{f[mm]}{d\_LEDs[mm]},$$

(7)

where d is the work distance, f is the focal length of the camera, and D_LEDs and d_LEDs are the distance between two LEDs in reality and in the image, respectively. Equation (7) allows us to calculate the minimum distance between two LEDs if we have the focal length of the camera, sensor size, size of the picture, and work distance. Based on Nyquist’s theorem, the minimum distance between two LEDs, which RoI methods can detect, is 2 pixels. Then, the minimum distance between two LEDs is given as

$$D\_LEDs[m]=2.\frac{d[m]}{f[mm]}\times \frac{h\_sensor\_image[mm]}{N\_pixel\_image\_row},$$

(8)

where N_pixel_image_row is the number of pixel rows in the image, h_sensor_image is the height of the sensor image in millimeters, and d[m] is simply the theoretical distance since the work distance also depends on the light power. If the power of the transmitter is smaller than that of the noise lights, the system may experience interference. Consequently, we can design the desired number of users in accordance with the monitoring environment.

4.5. Implementation System Monitoring Using MIMO-OOK

The proposed system consists of multiple sensor modules, with light transmitters for data sensing at multiple locations, and a camera. Figure 11 illustrates the proposed system architecture. Temperature and humidity sensors are selected as the key parameters to be monitored in the environment monitoring system to ensure a healthy and comfortable environment for humans. The sensing data from the sensor are collected by the MCU. From the MCU, sensing data is modulated by OOK modulation. LEDs are used to transmit data from Tx to Rx through the ON/OFF status of the LEDs. On the receiver side, the camera captures the signal from the transmitter. Figure 12 and Figure 13 illustrate the scenario and the OOK waveform of the receiving camera.

Figure 14 shows the results of the experiment, which was conducted several times with different distances and exposure times. The use of a preamble not only helps detect the start frame of the OOK modulation but also distinguishes the signal and noise light sources from other lights. These results reveal that exposure time affects the performance of OOK modulation as well as OCC modulations. Subsequently, we can suitably adjust the exposure time to achieve optimal performance. Detecting objects using RoI signaling allows us to apply dozens of signals from many users. However, the trade-off between the number of users and work distance must be carefully considered. Figure 14 shows that with the same communication distance and noise condition, the Bit Error Rate can be changed if the exposure time changed, but as we mentioned above, the control of the exposure time needs to be carefully considered due to relationship of the exposure time and the communication bandwidth as well as the trade-off between the Signal to Noise and the exposure time. These results illustrate that at the communication distance of 10 m, the proposed scheme achieved a BER value of ${10}^{-4}$ which meets the performance requirement of Single-Carrier Modulation in the Light of Sight Communication system. These implement results illustrate that the performance of the proposed scheme has many advantages of the VLC system using color light which proposed in [11], in terms of both bit error rate and work distance. To mitigate the Bit Error Rate in the far distance, we can use Forward Error Correction to improve the performance. The parameters of MIMO-OOK scheme is shown in

Table 2. Characteristic parameters for MIMO-C-OOK scheme for implementation.

Transmitter Side
The number of LEDs	1	2	3
Clock rate	20 Hz
LED type	5V DC-0.2 W
Receiver Side
Camera type	PointGrey rolling shutter camera
Camera frame rate	60 fps
Throughput
Bit rate	20 bps	40 bps	60 bps

. We implement this scheme with temperature and humidity sensors at the distance of 20 m, which can be visually assessed by Video S1 (See the Supplementary Materials.

5. Conclusions

In this paper, we proposed a monitoring system that uses a MIMO-OOK scheme that is based on an OCC system. The proposed scheme uses a RoI method, which can detect dozens of users with high accuracy, making it compatible with almost all types of commercially available cameras, including the rolling-shutter camera and global-shutter camera. Furthermore, our advances in SNR measurements with different exposure times confirm that the performance of the system depends on the work distance and exposure time. Finally, we implemented the bit error rate of the proposed scheme using different distances to highlight the system’s effectiveness in comparison with similar systems.