Next Article in Journal
The Role of Dactylis Glomerata and Diesel Oil in the Formation of Microbiome and Soil Enzyme Activity
Next Article in Special Issue
LDPC-Coded CAP with Spatial Diversity for UVLC Systems over Generalized-Gamma Fading Channel
Previous Article in Journal
Measurements of Aerodynamic Interference of a Hybrid Aircraft with Multirotor Propulsion
Previous Article in Special Issue
Characterization and Performance of a Thermal Camera Communication System
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Optical Camera Communications for IoT–Rolling-Shutter Based MIMO Scheme with Grouped LED Array Transmitter

1
Faculty of Electrical Engineering, Czech Technical University in Prague, Technicka, 16627 Prague, Czech Republic
2
Institute for Technological Development and Innovation in Communications, Universidad de Las Palmas de Gran Canaria, 35001 Las Palmas, Spain
3
Optical Communications Research Group, Northumbria University, Newcastle-upon-Tyne NE1 7RU, UK
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(12), 3361; https://doi.org/10.3390/s20123361
Submission received: 27 April 2020 / Revised: 5 June 2020 / Accepted: 11 June 2020 / Published: 13 June 2020
(This article belongs to the Special Issue Visible Light Communication, Networking, and Sensing)

Abstract

:
In optical camera communications (OCC), the provision of both flicker-free illumination and high data rates are challenging issues, which can be addressed by utilizing the rolling-shutter (RS) property of the image sensors as the receiver (Rx). In this paper, we propose an RS-based multiple-input multiple-output OCC scheme for the Internet of things (IoT) application. A simplified design of multi-channel transmitter (Tx) using a 7.2 × 7.2 cm2 small 8 × 8 distributed light emitting diode (LED) array, based on grouping of LEDs, is proposed for flicker-free transmission. We carry out an experimental investigation of the indoor OCC system by employing a Raspberry Pi camera as the Rx, with RS capturing mode. Despite the small area of the display, flicker-free communication links within the range of 20–100 cm are established with data throughput of 960 to 120 bps sufficient for IoT. A method to extend link spans up to 1.8 m and the data throughput to 13.44 kbps using different configurations of multi-channel Tx is provided. The peak signal-to-noise ratio of ~14 and 16 dB and the rate of successfully received bits of 99.4 and 81% are measured for the shutter speeds of 200 and 800 µs for a link span of 1 m, respectively.

1. Introduction

The fifth-generation (5G) telecommunication standards have set the new platform for mobile wireless networks, rather than just extending the transmission capacity and reliability of the 4G network [1]. With the three main focuses on capacity enhancement, massive connectivity, and ultra-high reliability (i.e., low latency), 5G development stems largely from the increasing number of users and smart devices within the context of Internet of things (IoT)-based smart environments being connected to the cellular networks [2]. In future smart environments, such as homes, offices, cities, etc., there will be a growing need for the communications networks that can facilitate connectivities between a large number of devices or sensors and the end-users [3]. This will mean access to substantial transmission resources (i.e., bandwidth), and thus a paradigm shift in the way the wireless transmission resources are utilized effectively.
To address this paradigm, a number of technologies have been proposed, including millimeter-wave [4], massive multiple-input multiple-output (MIMO) [5], small cell [6], and optical wireless communications (OWC) [7,8], in order to meet the requirements of 5G-IoT. The popularity of the latter is due mainly to its inherent advantages of safety, security, low cost, and large transmission bandwidth, which are essential in IoT applications, particularly in indoor environments [8]. In indoor environments, the OWC technologies of visible light communications (VLC) and optical camera communications (OCC) have been widely considered and investigated [8,9]. While VLC can support major features of 5G in terms of the data rates, OCC can offer a promising solution. The VLC technology, which uses light-emitting diode (LED)-based lights as the transmitter (Tx), has a lower initial deployment cost. The cameras in smartphones (six billion front and rear cameras), and pre-existing camera-based infrastructures, such as traffic lights, security, surveillance, and vehicles can be adopted as the OCC receiver (Rx) effectively [10]. Reflecting on the OCC potential and its availability, the IEEE 802.15.7r1 task group has been established to develop a standard for OCC within OWCs [9]. Various OCC issues and considerations, such as image sensor architecture, synchronization, data rate, perspective distortion, flickering and dimming, MIMO, and diversity are studied in [11]. The use of high-rate Tx’s, such as LEDs, and low-rate Tx’s, such as liquid crystal displays and digital signage, in OCC schemes is also provided in [11]. Although OCC does not support high capacity link connectivities, due to the speed limit of cameras, it can be employed in numerous low data rate applications, such as indoor positioning, mobile robot navigation, vehicular communications, small identification information, and advertisements [8,9,10,11]. Despite lower data rates, OCC is simple to implement, with low path loss, additional imaging functionality, and spatial modulation capability, as compared with high rate VLC links [12].
However, in OCC, the data rate is limited by the frame-rate of the image sensors (ISs). The data rate can be increased by using higher frame rate cameras, which are very costly, and increasing the camera capture speed, which is defined as the physical parameter of the sensor (electronics) and the graphics processor speed in the hardware domain. Therefore, OCC is further extended to offer massive MIMO capabilities in order to increase the data rate, using LED and photodetector (PD) arrays in the form of multiple pixels in ISs for IoT applications, in both indoor and outdoor environments [13]. In addition, hybrid modulations schemes based on the intensity, color, spatial, phase, and frequency are also suggested in [12] as a solution to improve the transmission data rates in OCC.
The IEEE 802.15.7-2018 standard [14] defines new clauses for the physical layer (PHY) types V and VI for OCC links. These PHY layers are mainly intended for use in systems with diffused light sources and video displays with kbps data rates, as well as using complex modulation schemes of spatial two-phase shift keying (S2-PSK), dimmable spatial eight-PSK, undersampled frequency shift on-off keying (UPSOOK), variable transparent amplitude-shape-color, etc. However, the standard clauses on the PHYs V and VI are mainly on the demodulation schemes, and their real applications are still being revised.
In [13], a data rate of 126.72 kbps was achieved, using 192 data-carrying LEDs modulated using color intensity modulation (CIM) and a 330 frames per second (fps) global-shutter (GS) camera-based Rx, which is expensive and not commonly used, over a link span of 1.4 m. However, the Tx was set to the refresh rate (i.e., transmission frequency) of 82.5 Hz, which is still lower than the maximum allowed flickering time period of 5 ms (200 Hz) [14]. A red, green, and blue (RGB) LED-based rolling-shutter (RS) OCC (RS-OCC) utilizing a combination of UPSOOK, wavelength-division multiplexing, and MIMO offering improved space efficiency of 3 bits/Hz/LED was reported in [15]. In [16], a multilevel intensity modulation (IM) RS-based camera detection link with a data throughput of 10 kbps over a transmission range of 2 m was reported in [15,16]. Whereas, a beacon jointed packet reconstruction scheme for mobile-phone-based VLC with commercial white phosphor LEDs and a 60 fps RS camera achieved the net data rate of 10.3 kbps (172 bits/frame observed over a large LED surface) over a transmission distance of 20 cm [17]. In [18], COTS LEDs with raptor code (with linear time encoding and decoding, thus reduced computational complexity and decoding overhead) have been investigated in RS-OCC. Non-line-of-sight MIMO links using Luxeon LEDs based on diffused reflections and space and time division multiple access, as well as the equal gain combining technique, was reported in [19], which achieved flicker-free transmission up to a 10 m link span. However, as listed in the survey in [20], for indoor OCC based IoT applications, low data rates ranging from 15 to 896 bps for transmitting short messages in device-to-device communications over transmission spans of 25 cm to 1 m is more than sufficient.
Over the past few years, multiple neopixel boards have been used in electronic devices such as screen displays in home automation, advertising, televisions, human interfaces, etc. [21]. These devices can be used as part of the Tx to provide IoT based MIMO-OCC links in smart environments. High-throughput links such as visual MIMO systems for screen-camera communications link were proposed in [22], where the impact of non-linear channel equalization, non-binary channel coding, probabilistic shaping, and non-linear precoding for high-order modulation schemes and respective applications such as inter/intra vehicle communications, near field communication, and augmented reality were investigated. In [22], it was shown that the reliability and throughput of the optical communication links can be improved using various channel coding techniques based on nonbinary low-density parity-check codes [23], polar and turbo codes [24], and advanced modulation schemes, such as chaos and LoRa [25]. As previously mentioned, short-range and low data rates IoT links in indoor environments such as device-to-device communications with short messages, indoor positioning, navigation, small identification information, and communications through advertisements need to be considered [26]. However, the major challenge in implementing such systems is the requirement for flicker-free transmission at lower data rates (i.e., most cases in IoT applications). Therefore, in this paper, we propose a low data rate RS-based MIMO-OCC scheme with grouped LED Tx for indoor IoT environments, which is flicker-free. For this reason, we present a simplified design of MIMO-OCC grouped LED array-based Tx, which uses 64-neopixel LEDs distributed in an 8 × 8 array and a commercial, low-cost Raspberry Pi camera (RaspiCam) as the Rx. The Tx unit is divided into eight different groups, with eight LEDs per group in order to increase the data rate and achieve flicker-free transmission. At the Rx, RaspiCam, with a resolution of 1920 × 1080 pixels and a capture speed of 30 fps, captures the LED array in RS-mode at different shutter speeds (SS) and transmission links L. The novelty of this work is on the design of a simplified Tx for multiple channels transmission-based LED grouping with perfect synchronization, and the use of an RS camera for flicker-free transmission in short-range and low data rates IoT applications. The paper gives a detailed analysis on the quality matrix of the captured image in terms of the peak signal-to-noise ratio, and the success rate of received bit sequences with respect to the transmission span and the camera’s SS.
The remainder of the paper is organized as follows: Section 2 describes the proposed RS-based MIMO-OCC scheme using the grouped LED array, while Section 3 shows the experiment setup, followed by the discussion of results. Conclusions are drawn in Section 4.

2. Proposed RS-Based MIMO-OCC Scheme with Grouped LED Tx

2.1. MIMO-OCC Tx Characterization

A new, simple design of MIMO Tx unit is proposed in this paper, as illustrated in Figure 1. It is composed of a 64-neopixel array with an 8 × 8 small chip-LED, and a 1 cm-thick LED grouping grid, (see Figure 1a,b), which is attached over the Tx LED array. This LED grid is designed to divide a 64-neopixel chip-LED into eight different column-wise groups that are individual transmission channels with eight chip-LEDs per group to allow eight different data transmissions, using a single neopixel LED array. In addition, the LED grid (Figure 1c) is very effective in combating interference due to adjacent LEDs within the groups. The effect of interference with supporting analysis without using the grouping grid to capture the LED array with a GS-based camera Rx was initially reported by the authors in [27]. A 2 mm-thick opaline methacrylate LED diffuser, commonly used, is placed over the Tx [28]. The size of the LED array is 7.2 × 7.2 cm2. Since the size of each chip-LED (i.e., 5 × 5 mm2) is much smaller than the distance between the adjacent chip-LED (i.e., 9 mm), light from each LED is captured as a discrete image using an IS.
Since both optical and electrical characterization of neopixel RGB LEDs are yet to be investigated, we first provide a characterization of the proposed MIMO Tx in terms of its optical radiation pattern and output optical power–current–voltage (Llux-I-V) curves. The optical radiation pattern of the neopixel LEDs was measured to obtain its spatial intensity distribution for use in analyzing the coverage and signal distribution in VLC and OCC links [8]. The light intensity of LEDs defined in terms of the angle of irradiance θ is given by [8]:
I ( θ ) = m + 1 2 π I ( 0 ) cos m ( θ ) ,   θ = [ π 2 , π 2 ] ,
where I(0) is the center luminous intensity of an LED and m is Lambertian order given as [8]:
m = ln ( 2 ) ln [ cos ( θ 1 / 2 ) ] .
A lux meter was used to measure the angular dependence of the luminance of the LED (i.e., single and 8 × 8 horizontal and vertical array). As expected, the profiles for (i) a single LED represent a complete hemisphere close to Lambertian emitter with m of 1 (see Figure 2a); (ii) the LED array with no diffuser is broader with m of 0.74, due to the 9 mm spacing between the adjacent LED chips; and (iii) the LED array with the diffuser has m of 0.75 (see Figure 2b). Note that the measured radiation patterns can be further used to study the proposed MIMO-OCC links with mobility, and in a multi-user scenario. For example, rotation compensation schemes based on different Tx configurations [29] and wide receiver orientations [30] can be employed to ensure the operability of the proposed MIMO-OCC system in IoT environments.
The illuminance levels of the LEDs were measured using a Testo 545 lux meter. Each neopixel in the array draws up to 60 mA of current I to turn ON at the maximum brightness. In realistic environments, when the LED panel is to be used as a lamp, luminance can be efficiently controlled by varying switching power supply output current at the LED drive circuit. Therefore, the implementation of dimming techniques can be included as a part of the further extension of the proposed scheme [31,32]. However, increasing the power would induce crosstalk due to illumination from adjacent LEDs in the MIMO Tx [33].
In practical use, it is rare for all pixels to be turned ON at its maximum drawing current, due to the risk of overheating and damaging the LED panel. Therefore, it is recommended to drive each LED with ILED = 0.33I = 20 mA [21]. Thus, the drive current for the LED array is estimated using the rule of thumb, as given by [21]:
I LED - array = N pixels × I LED 1000 ,
where Npixels = 64 is the total number of neopixel LEDs. In this work, ILED-array is set to 1.28 A for measuring the Llux-I-V curves, which are depicted in Figure 3a,b for the MIMO-OCC Tx unit with, and without, the diffuser, respectively. Note: (i) the illumination levels are largely reduced, due to the use of the LED grid and the diffuser; (ii) linear Llux-I plots, which are highly desirable in IM VLC systems; and (iii) the neopixels used either as a single LED chip or in an array depict similar optical characteristics to those of commonly used RGB LEDs.

2.2. Rolling-Shutter Based MIMO Rx in OCC

The CMOS IS, along with the imaging lens, is composed of a large number of pixels, with each pixel acting as an independent PD. Unlike a conventional PD-based Rx, which cannot be used to separate the mixed signals, CMOS-based IS can capture lights coming from different directions, and project them onto different sections of the IS [13]. Therefore, spatial separation of incoming light signals and their intensities can be measured by obtaining the pixel value for each light source image in the received frame. Image processing can be applied to the received image frames to extract the data from pixelated images [13]. Therefore, CMOS IS can be used as the MIMO-OCC Rx without the need for extended hardware. In [34], the analysis of the MIMO-OCC Rx using Bayer-pattern filters, which can differentiate the incoming signals being transmitted from the MIMO Tx, was reported.
In an OCC-VLC system, the RS effect of a CMOS IS can be used to achieve flicker-free transmission and increased data rate [16,17,18,19]. In this mode, the camera sequentially integrates light on all pixels at the exposure time trow-exp, similar to the scanning function, as illustrated in Figure 4a. In RS, the sensor scans the entire image row-by-row (line-wise) and generates a sequential readout. This scan process is governed by the system clock and is limited by the sampling rate of the analog to digital converter module. In GS-based IS, all pixels are exposed to the light simultaneously, i.e., ON or OFF states of the LED in a single frame [35], as illustrated in Figure 4b; while in RS, each row of pixels is exposed to light at a given exposure time trow-exp, sequentially similar to the scanning function. In RS cameras, the readout time tread-out ensures that there is no overlapping of the rows of pixels, and allows multiple exposures in a single captured image. The latter enables multiple LED states to be achieved at the same time in a single frame, as each row is exposed once to the light. Therefore, the captured image of the switched LED is composed of a set of black and white stripes. The proposed MIMO-OCC scheme differs from the RS-based OCC links (see Figure 4c), as in [16,17,18,19]. In traditional RS-based OCC, only a single bit is captured within one exposure time trow-exp (see Figure 4a), in contrast to the proposed work, where 8-bit (1-bit per channel) are captured in a single row with time trow-exp, as depicted in Figure 4c. In addition, the proposed scheme can be used for flicker-free and high data rates transmission by allocating multiple bits per trow-exp. The widths and the number of strips depend on the data rate (i.e., modulation frequencies) and the camera-LED distance, respectively [16,18].
Note that, in the RS capturing mode, each row starts with a certain delay, which results in the row shift, trow-shift. The frame time is given as:
t frame N row × t row - shift + t row - exp ,
where Nrow is the pixel rows, which are based on the camera resolution. Note, trow-exp is the exposure time of the last row per frame (very small value).

2.3. System Overview of MIMO-OCC Using Proposed Multi-Channel Tx Design

Figure 5 illustrates the flow diagram for the proposed MIMO-OCC, using the 64-neopixel Tx unit and a RS-based RaspiCam as the Rx. The proposed scheme is an initial study to investigate the upper bounds of the system using the proposed Tx unit, therefore, we assume perfect synchronization and line-of-sight transmission. Neopixels are controlled in the Arduino software domain. First, Npixels are assigned to Nchips of neopixels, which are then grouped column-wise into eight Ngroups to form eight different transmission channels within one Tx unit. For data transmission, we have adopted a non-return-to-zero (NRZ) on-off keying (OOK) data format for IM of tri-color (band i, j, k) RGB channels (Pi, Pj and Pk) as Pi + Pj + Pk = 1 and Pi + Pj + Pk = 0, which is most commonly used in OCC. The data is generated in the Arduino unit and mapped to the LED addresses with the frequency, fs, given as:
f s = ( t chip ) 1 ,
where tchip is the 1-bit time per neopixel chip, and its minimum value is 2.5 ms, due to Arduino hardware limitation to ensure flicker-free transmission at fs of 400 Hz. The maximum number of visible bits per group in a single frame is given as:
N visible = t frame t chip .
Note that Nvisible will change with respect to the distance between the source and the camera, as well as camera resolution. Based on Nvisible, the data transmission rate is given as:
R d = N groups ( 1 t chip ) ,
where Ngroups is the number of data transmission channels in the Tx unit (see Figure 1b).
Table 1 shows the resolutions, frame rate and Nvisible of RaspiCam. For further analysis, we selected a 1920 × 1080 pixel resolution, which is the most commonly used in cameras.
To determine the upper bound of the system, we have selected NRZ-OOK data bit streams for IM of LED groups for transmission over the free space channel. Note that the same data is transmitted by all 8-LEDs per group, as shown in Figure 6.
At the Rx, a RaspiCam for a given SS, resolution, and frame rate is used for capturing the images (i.e., recording a video stream for 3 s) of the IM light sources over an L ranging from 20 to 100 cm for post-processing. Note that a smaller image containing the emitter’s signal information is transmitted, in order to speed up the processing time at the Rx. Therefore, the first step represents the detection of the region of interest (ROI) [36]. The obtained coordinates, which define boundaries of the ROI, are used for image cropping. Then, image processing is performed on the cropped images, which are then converted to the grayscale in order to retrieve the intensity profile. The threshold level is set based on the average of the received image intensity profile within the ROI. Following thresholding, binarization of the data frames is performed to convert the frame into vector transformation. This process is performed and applied to the remaining frames for decoding the transmitted data bit streams.

3. Experiment Results and Analysis

The experimental setup for investigating the proposed MIMO-OCC scheme is shown in Figure 7. The Tx unit is controlled using an Arduino Uno board, which is an open-source microcontroller board based on the ATmega328 [37]. The 64-bit long data stream (i.e., 8-bit per group, see Figure 6) is generated in the Arduino software domain and mapped to each LED address using the Arduino Uno board.
The key experimental parameters are listed in Table 2.
The camera used for capturing is the Raspberry Pi official camera (PiCamera V2), which is based on the Sony IMX219 sensor [38]. The RaspiCam is attached to the Raspberry touchscreen display to provide the easy interface and control over the camera capturing modes and settings. For the demonstration of the proposed study, experiments were performed for nine transmission distances and four different values of SS (see Table 2). The NRZ-OOK modulated signal was recorded in the form of a 3 s video stream (90 frames in total).
The current proof-of-concept experiments were performed under the ambient light, where we measured the light intensity using a Testo 545 lux meter. The measured light intensity of the Tx with a diffuser was 300 lux at a distance of 50 cm. We also measured reflected lights from walls to be approximately 3.5 lux (±0.5 lux), which is very small compared with the Tx’s illuminance; therefore, the ambient light influence on the integrity of data transmission is insignificant. The experiments were first conducted without a grid and a diffuser. To validate the scheme, a binary bit sequence of 1 and 0 was transmitted via all LEDs in the array. Figure 8 shows the original captured image frames with GS and RS modes, along with respective single row grayscale intensity profiles. Figure 8a shows the captured LED array using GS mode, along with its intensity profile at a maximum L of 20 cm. It can be seen that interference from the adjacent LEDs results in the blooming effect and, therefore, causes inter-cluster-interference. It can be seen that LED array captured using RS mode (see Figure 8b) at a minimum L of 5 cm, Nvisible is 4 at the surface of each LED. Moving farther away from the Tx, Nvisible is reduced to 3 at L of 7 cm (see Figure 8c). This is due to the reduced LED’s surface area of 5 × 5 mm2. Moreover, the RS rows also become saturated, which leads to inter-cluster-interference. Therefore, setting threshold levels become problematic, and hence lead to increased bit error rates (see Figure 8b,c). Therefore, further analysis was performed using the proposed LED array configuration shown in Figure 1.
Figure 9 shows the original captured image frames with grouping grid and diffuser, quantized intensity of the detected data, and the top view of intensity profiles for L of 20, 60, and 100 cm, and SS of 200, 600, and 800 µs. The received intensity distribution within the image frame is shown in the form of quantized intensity profiles of the captured original images. These intensity profiles play an important role in determining the higher and lower intensities representing 1 and 0 bits in the received image frames for further thresholding and demodulation [39]. The dotted yellow box in the original image frames is the ROI, which fills only the captured Tx within the full image frame. The clear and sharp distinction between data lines of the adjacent Ngroups can be seen at SS of 200 µs (see Figure 9a), while the lines get saturated for higher values of SS (see Figure 9b,c), which affects data demodulation (i.e., a higher number of bit error). Note that the camera’s SS can be used to combat the effect of ambient lights, where lower and higher SS results in lower and higher levels of light to pass through the camera lens, as seen in the captured images in Figure 9.
Based on the received bits in the image frames, the data throughput is given as:
Data   throughput = N groups × N visible ( fps 2 ) ,
Figure 10 illustrates the data throughput and Nvisible with respect to L. Note that Nvisible is the number of visible bits in each group. The maximum data throughput of 960 bps is observed at the minimum distance of 20 cm, where 8 full bits are visible in each group. The Tx illumination surface reduces with the increased L, thus resulting in reduced Nvisible and the data throughput. For L of 100 cm, only one full bit is visible, therefore the data throughput is reduced to 120 bps.
Table 3 shows the predicted data throughput based on the approximation of different Tx configurations, L, Ngroups, and Nvisible. It can be seen that increasing the number of LEDs leads to a larger surface area of the Tx, thus limiting higher values of Nvisible. For example, for a Tx using a 24 × 24 LED array, 24 Ngroups can be formed, which will increase the data throughput by up to 7.92 and 0.360 kbps for L of 60 and 160 cm, respectively; while for a Tx with a 32 × 32 LED array, 32 Ngroups can be formed, which will increase the data throughput by up to 13.44 and 0.480 kbps for L of 80 and 180 cm, respectively.
Since in OCC the data is captured in the form of a two-dimensional image, a conventional signal-to-noise ratio (SNR) measurement cannot fully reflect the quality of the link. Therefore, we have adopted peak signal-to-noise ratio (PSNR), which is widely used as a quality metric in image processing systems. To compute the PSNR, the mean squared error between the transmitted and received images is given by [39,40]:
MSE = m = 1 H n = 1 W [ I T x ( m , n ) I R x ( m , n ) ] 2 N column × N row ,
where ITx(m, n) and IRx(m, n) are the intensity levels within the ROI of transmitted and received images of size height (H) × width (W), and Ncolumn and Nrow are the number of columns and rows of the images, respectively. The PSNR is then given as:
PSNR = 10 log 10 ( R 2 MSE ) ,
where R is the maximum span of input data (e.g., in the current scheme, the input image has an 8-bit unsigned integer data type; therefore, R = 255).
Figure 11a shows the performance of the OCC link in terms of PSNR with respect to L for different values of SS. As shown, PSNR increases with SS and decreases with the link span. This is due to the fact that the images of captured Tx at higher SS are more saturated, compared with those captured at lower SS (see Figure 9). The PSNR values of ~14 and ~16 dB are measured for SS values of 200 and 800 µs, respectively, for L of 100 cm, increasing by 4 and 3 dB for L of 20 cm for the same SS values, respectively.

4. Conclusions

This paper demonstrated the experimental implementation of an RS acquisition-based camera capturing in MIMO-OCC, employing an LED array in an indoor static environment. The neopixel’s light source could be used either as a single LED chip or an array display with similar optical characteristics to those of commonly used RGB LEDs. We showed the transmission spans of 7 cm and 1 m using the Tx without, and with, the LED grid and a diffuser, respectively. The proposed system with the multi-channel LED array-based Tx offered the same performance at the SS of 200 and 400 µs over an L of 80 cm. However, at higher SS, the captured images were saturated, which resulted in increased PSNR and reduced percentage success of received bits. The maximum PSNR values of ~18 and ~19 dB and a 100% success rate of received bits were measured for the SS of 200 and 800 µs at the link span of 20 cm. A theoretical approximation (see Table 3) to further extend the L up to 1.8 m and the data throughput up to 13.44 kbps within the proposed MIMO-OCC using a grouped LED array by employing a Tx with a larger illuminating surface (large size) for practical indoor environments was provided. We conclude that the proposed Tx design can provide flicker-free transmission by employing multiple channels Tx with perfect inter-LED synchronization and an RS-based camera for use in short-range and low data rates indoor IoT applications, such as display-to-camera communications. The complexity of the proposed RS-based MIMO-OCC scheme will be further investigated for its implementation in practical scenarios, such as mobility, rotational support, and multiuser, based on ROI detection and bit or image pattern recognition within neural network algorithms.

Author Contributions

The contributions of the authors in this paper are the following: conceptualization: S.R.T. and V.M.; investigation: S.R.T. and V.M.; methodology: S.R.T.; project administration: S.Z., R.P.-J., and Z.G.; software: S.R.T.; validation: S.Z., R.P.-J., S.V., and Z.G. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the European Union’s Horizon 2020 Research and Innovation Programme, under the Marie Sklodowska-Curie grant agreement no 764461 (VisIoN).

Conflicts of Interest

The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Boccardi, F.; Heath, R.W.; Lozano, A.; Marzetta, T.L.; Popovski, P. Five disruptive technology directions for 5G. IEEE Commun. Mag. 2014, 52, 74–80. [Google Scholar] [CrossRef] [Green Version]
  2. Al-Fuqaha, A.; Guizani, M.; Mohammadi, M.; Aledhari, M.; Ayyash, M. Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Commun. Surv. Tutor. 2015, 17, 2347–2376. [Google Scholar] [CrossRef]
  3. Aksu, H.; Babun, L.; Conti, M.; Tolomei, G.; Uluagac, A.S. Advertising in the IoT Era: Vision and Challenges. IEEE Commun. Mag. 2018, 56, 138–144. [Google Scholar] [CrossRef] [Green Version]
  4. Tercero, M.; von Wrycza, P.; Amah, A.; Widmer, J.; Fresia, M.; Frascolla, V.; Vijay, A. 5G Sstems: The mmMAGIC Project Perspective on Use Cases and Challenges between 6–100 GHz. In Proceedings of the IEEE Wireless Communications and Networking Conference Workshops (WCNCW), Doha, Qatar, 29 August 2016; pp. 200–205. [Google Scholar]
  5. Kashima, T.; Qiu, J.; Shen, H.; Tang, C.; Tian, T.; Wang, X.; Kishiyama, Y. Large-scale massive MIMO field trial for 5G mobile communications system. In Proceedings of the 2016 International Symposium on Antennas and Propagation (ISAP), Okinawa, Japan, 24–28 September 2016; pp. 602–603. [Google Scholar]
  6. Ayyash, M.; Elgala, H.; Khreishah, A.; Jungnickel, V.; Little, T.; Shao, S.; Rahaim, M.; Schulz, D.; Hilt, J.; Freund, R. Coexistence of WiFi and LiFi toward 5G: Concepts, opportunities, and challenges. IEEE Commun. Mag. 2016, 54, 64–71. [Google Scholar] [CrossRef]
  7. Ghassemlooy, Z.; Popoola, W.; Rajbhandari, S. Visible Light Communications. In Optical Wireless Communications; Informa UK Limited: Colchester, UK, 2019; pp. 397–468. [Google Scholar]
  8. Ghassemlooy, Z.; Popoola, W.; Rajbhandari, S. Optical Wireless Communications; Informa UK Limited: Colchester, UK, 2019. [Google Scholar]
  9. Jang, M.J. IEEE 802.15 WPAN 15.7 Amendment-Optical Camera Communications Study Group (SG 7a). Available online: http://www.ieee802.org/15/pub/IEEE%20802_15%20WPAN%2015_7%20Revision1%20Task%20GroupOLD.htm (accessed on 18 November 2019).
  10. Nguyen, T.; Islam, A.; Hossan, T.; Chowdhury, M.Z. Current Status and Performance Analysis of Optical Camera Communication Technologies for 5G Networks. IEEE Access. 2017, 5, 4574–4594. [Google Scholar] [CrossRef]
  11. Le, N.T.; Hossain, M.A.; Jang, Y.M. A survey of design and implementation for optical camera communication. Signal Process. Image Commun. 2017, 53, 95–109. [Google Scholar] [CrossRef]
  12. Liu, W.; Xu, Z. Some practical constraints and solutions for optical camera communication. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 2020, 378, 20190191. [Google Scholar] [CrossRef] [Green Version]
  13. Huang, W.; Tian, P.; Xu, Z. Design, and implementation of a real-time CIM-MIMO optical camera communication system. Opt. Express 2016, 24, 24567. [Google Scholar] [CrossRef]
  14. IEEE Approved Draft Standard for Local and Metropolitan Area Networks—Part 15.7: Short-Range Optical Wireless Communications. In IEEE P802.15.7/D3a, pp. 1–428, 4 December 2018. Available online: https://standards.ieee.org/standard/802_15_7-2018.html (accessed on 23 April 2019).
  15. Luo, P.; Zhang, M.; Ghassemlooy, Z.; Le Le Minh, H.; Tsai, H.-M.; Tang, X.; Png, L.C.; Han, D. Experimental Demonstration of RGB LED-Based Optical Camera Communications. IEEE Photon. J. 2015, 7, 1–12. [Google Scholar] [CrossRef]
  16. Rachim, V.P.; Chung, W.-Y. Multilevel Intensity-Modulation for Rolling Shutter-Based Optical Camera Communication. IEEE Photon-Technol. Lett. 2018, 30, 903–906. [Google Scholar] [CrossRef]
  17. Wang, W.-C.; Chow, C.-W.; Chen, C.-W.; Hsieh, H.-C.; Chen, Y.-T. Beacon Jointed Packet Reconstruction Scheme for Mobile-Phone Based Visible Light Communications Using Rolling Shutter. IEEE Photon. J. 2017, 9, 1–6. [Google Scholar] [CrossRef]
  18. Yang, Y.; Hao, J.; Luo, J. CeilingTalk: Lightweight Indoor Broadcast Through LED-Camera Communication. IEEE Trans. Mob. Comput. 2017, 16, 3308–3319. [Google Scholar] [CrossRef]
  19. Hassan, N.B.; Ghassemlooy, Z.; Zvanovec, S.; Biagi, M.; Vegni, A.M.; Zhang, M.; Luo, P. Non-Line-of-Sight MIMO Space-Time Division Multiplexing Visible Light Optical Camera Communications. J. Light Technol. 2019, 37, 2409–2417. [Google Scholar] [CrossRef]
  20. Hasan, M.K.; Chowdhury, M.Z.; Shahjalal; Nguyen, V.T.; Jang, Y.M. Performance Analysis, and Improvement of Optical Camera Communication. Appl. Sci. 2018, 8, 2527. [Google Scholar] [CrossRef] [Green Version]
  21. Burgess, P. Adafruit Neopixel Uberguide, WS2812B Datasheet; Philips, The Netherlands, 2019. Available online: https://cdn-learn.adafruit.com/downloads/pdf/adafruit-neopixel-uberguide.pdf (accessed on 13 June 2020).
  22. Fujihashi, T.; Koike-Akino, T.; Orlik, P.; Watanabe, T. High-Throughput Visual MIMO Systems for Screen-Camera Communications. IEEE Trans. Mob. Comput. 2020, 1. [Google Scholar] [CrossRef]
  23. Jerkovits, T.; Liva, G.; I Amat, A.G. Improving the Decoding Threshold of Tailbiting Spatially Coupled LDPC Codes by Energy Shaping. IEEE Commun. Lett. 2018, 22, 660–663. [Google Scholar] [CrossRef] [Green Version]
  24. Fang, Y.; Chen, P.; Cai, G.; Lau, F.C.M.; Liew, S.-C.; Han, G. Outage-Limit-Approaching Channel Coding for Future Wireless Communications: Root-Protograph Low-Density Parity-Check Codes. IEEE Veh. Technol. Mag. 2019, 14, 85–93. [Google Scholar] [CrossRef]
  25. Elshabrawy, T.; Robert, J. Interleaved Chirp Spreading LoRa-Based Modulation. IEEE Internet Things J. 2019, 6, 3855–3863. [Google Scholar] [CrossRef]
  26. Shahjalal; Hasan, M.K.; Chowdhury, M.Z.; Jang, Y.M. Smartphone Camera-Based Optical Wireless Communication System: Requirements and Implementation Challenges. Electronics 2019, 8, 913. [Google Scholar] [CrossRef] [Green Version]
  27. Teli, S.R.; Zvanovec, S.; Ghassemlooy, Z. The first tests of smartphone camera exposure effect on optical camera communication links. In Proceedings of the 2019 15th International Conference on Telecommunications (ConTEL), Graz, Austria, 3–5 July 2019; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2019; pp. 1–6. [Google Scholar]
  28. Chua, J.B.Y. System and Method for Enhancing Light Emissions from Light Packages by Adjusting the index of Refraction at the Surface of the Encapsulation Material. U.S. Patent 8,089,083, 3 January 2012. [Google Scholar]
  29. Cahyadi, W.A.; Kim, Y.H.; Chung, Y.H.; Ahn, C.-J. Mobile Phone Camera-Based Indoor Visible Light Communications with Rotation Compensation. IEEE Photon. J. 2016, 8, 1–8. [Google Scholar] [CrossRef]
  30. Cahyadi, W.A.; Chung, Y.H. Wide receiver orientation using diffuse reflection in camera-based indoor visible light communication. Opt. Commun. 2019, 431, 19–28. [Google Scholar] [CrossRef]
  31. Gancarz, J.; Elgala, H.; Little, T.D. Impact of lighting requirements on VLC systems. IEEE Commun. Mag. 2013, 51, 34–41. [Google Scholar] [CrossRef]
  32. Zafar, F.; Karunatilaka, D.; Parthiban, R. Dimming schemes for visible light communication: The state of research. IEEE Wirel. Commun. 2015, 22, 29–35. [Google Scholar] [CrossRef]
  33. Pergoloni, S.; Biagi, M.; Cusani, R.; Scarano, G. Space-time multichannel adaptive filtering scheme for VLC color crosstalk equalization. Opt. Express 2018, 26, 19750–19761. [Google Scholar] [CrossRef] [PubMed]
  34. Ghassemlooy, Z.; Uysal, M.; Khalighi, M.A.; Ribeiro, V.; Moll, F.; Zvanovec, S.; Belmonte, A. An Overview of Optical Wireless Communications; Springer Science and Business Media LLC: Berlin/Heidelberg, Germany, 2016; pp. 1–23. [Google Scholar]
  35. Le, T.; Le, N.T.; Jang, Y.M.; Thithanhnhan, L.; Nam-Tuan, L.; Min, J.Y. Performance of rolling shutter and global shutter camera in optical camera communications. In Proceedings of the 2015 International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea, 28–30 October 2015; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2015; pp. 124–128. [Google Scholar]
  36. Thieu, M.D.; Pham, L.; Nguyen, T.; Chowdhury, M.Z. Optical-RoI-Signaling for Vehicular Communications. IEEE Access 2019, 7, 69873–69891. [Google Scholar] [CrossRef]
  37. Atmel Corporation. 8-bit Microcontroller with 4/8/16/32K Bytes in System Programmable Flash; Atmel Datasheet; Atmel Corporation: San Jose, CA, USA, 2009. [Google Scholar]
  38. Sony Corporation. IMX219PQHS-C Datasheet; 2014. Available online: https://datasheetspdf.com/pdf/1404029/Sony/IMX219PQH5-C/1 (accessed on 13 June 2020).
  39. Teli, S.R.; Zvanovec, S.; Ghassemlooy, Z. Performance evaluation of neural network assisted motion detection schemes implemented within indoor optical camera-based communications. Opt. Express 2019, 27, 24082–24092. [Google Scholar] [CrossRef]
  40. Huynh-Thu, Q.; Ghanbari, M. Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 2008, 44, 800. [Google Scholar] [CrossRef]
Figure 1. The transmitter (Tx) unit: (a) assembled unit with the grid and a diffuser, (b) an LED array configuration and (c) an Arduino Uno controller board and the LED panel.
Figure 1. The transmitter (Tx) unit: (a) assembled unit with the grid and a diffuser, (b) an LED array configuration and (c) an Arduino Uno controller board and the LED panel.
Sensors 20 03361 g001
Figure 2. Measured Tx radiation patterns of: (a) a single LED in an array and for full LED array; and (b) a full LED array with the grid and the diffuser.
Figure 2. Measured Tx radiation patterns of: (a) a single LED in an array and for full LED array; and (b) a full LED array with the grid and the diffuser.
Sensors 20 03361 g002
Figure 3. The Llux-I-V curves of the multiple-input multiple-output-optical camera communications (MIMO-OCC) Tx unit: (a) with the diffuser attached; and (b) without the diffuser.
Figure 3. The Llux-I-V curves of the multiple-input multiple-output-optical camera communications (MIMO-OCC) Tx unit: (a) with the diffuser attached; and (b) without the diffuser.
Sensors 20 03361 g003
Figure 4. The camera capturing modes in OCC: (a) traditional rolling-shutter (RS) mechanism, (b) global-shutter (GS); and (c) RS-based capturing of the proposed multi-channel Tx.
Figure 4. The camera capturing modes in OCC: (a) traditional rolling-shutter (RS) mechanism, (b) global-shutter (GS); and (c) RS-based capturing of the proposed multi-channel Tx.
Sensors 20 03361 g004
Figure 5. MIMO-OCC data processing flow diagram.
Figure 5. MIMO-OCC data processing flow diagram.
Sensors 20 03361 g005
Figure 6. Data bit streams for intensity modulation (IM) of LED groups.
Figure 6. Data bit streams for intensity modulation (IM) of LED groups.
Sensors 20 03361 g006
Figure 7. Experiment setup of the MIMO-OCC scheme.
Figure 7. Experiment setup of the MIMO-OCC scheme.
Sensors 20 03361 g007
Figure 8. MIMO-OCC performance analysis without grouping grid and diffuser: captured images for data detection and their grayscale intensity profiles for: (a) GS captured image for L = 20 cm; (b) RS captured images for L = 5 cm; and (c) RS captured images for L = 7.
Figure 8. MIMO-OCC performance analysis without grouping grid and diffuser: captured images for data detection and their grayscale intensity profiles for: (a) GS captured image for L = 20 cm; (b) RS captured images for L = 5 cm; and (c) RS captured images for L = 7.
Sensors 20 03361 g008
Figure 9. MIMO-OCC performance analysis with grouping grid and diffuser: the quantized intensity profiles of originally captured images for data detection at: (a) L = 20 cm and SS = 200 µs; (b) L = 60 cm and SS = 600 µs; and (c) L = 100 cm and SS = 800 µs.
Figure 9. MIMO-OCC performance analysis with grouping grid and diffuser: the quantized intensity profiles of originally captured images for data detection at: (a) L = 20 cm and SS = 200 µs; (b) L = 60 cm and SS = 600 µs; and (c) L = 100 cm and SS = 800 µs.
Sensors 20 03361 g009
Figure 10. MIMO-OCC performance analysis: the data throughput with respect to Nvisible bits.
Figure 10. MIMO-OCC performance analysis: the data throughput with respect to Nvisible bits.
Sensors 20 03361 g010
Figure 11. MIMO-OCC performance analysis: (a) the peak signal-to-noise ratio (PSNR); and (b) the percentage success of received bits with respect to L.
Figure 11. MIMO-OCC performance analysis: (a) the peak signal-to-noise ratio (PSNR); and (b) the percentage success of received bits with respect to L.
Sensors 20 03361 g011
Table 1. RaspiCam resolutions.
Table 1. RaspiCam resolutions.
Resolution (pixels)Frame Rate (fps)Nvisible (bits)
1920 × 1080308
3280 × 24641522
1640 × 12324010
1640 × 922408
1280 × 720906
640 × 4802004
Table 2. Key parameters of the experiment setup.
Table 2. Key parameters of the experiment setup.
ParameterValue
RaspiCam chip size5.09 mm (H) × 4.930 mm (W)
Diagonal: 4.60 mm
RaspiCam resolution1920 × 1080 pixels
Raspberry display size7 ” (diagonally)
Raspberry display resolution800 × 400 pixels
tchip2.5 ms
fs400 Hz
RaspiCam frame rate30 fps
Nrow1080 pixels
Ngroups8 LED groups with 8LED/group
tframe0.216 ms
SS200, 400, 600 and 800 µs
Rd3.2 kbps
L20–100 cm
Table 3. Theoretical (approximation) data throughput based on different Tx configurations.
Table 3. Theoretical (approximation) data throughput based on different Tx configurations.
Number of NeopixelsNgroupsL (cm)NvisibleData Throughput (kbps)
16 × 161640–14014 (max)–1 (min)3.36 (max)–0.240 (min)
24 × 242460–16022–17.92–0.360
32 × 323280–18028–113.44–0.480

Share and Cite

MDPI and ACS Style

Teli, S.R.; Matus, V.; Zvanovec, S.; Perez-Jimenez, R.; Vitek, S.; Ghassemlooy, Z. Optical Camera Communications for IoT–Rolling-Shutter Based MIMO Scheme with Grouped LED Array Transmitter. Sensors 2020, 20, 3361. https://doi.org/10.3390/s20123361

AMA Style

Teli SR, Matus V, Zvanovec S, Perez-Jimenez R, Vitek S, Ghassemlooy Z. Optical Camera Communications for IoT–Rolling-Shutter Based MIMO Scheme with Grouped LED Array Transmitter. Sensors. 2020; 20(12):3361. https://doi.org/10.3390/s20123361

Chicago/Turabian Style

Teli, Shivani Rajendra, Vicente Matus, Stanislav Zvanovec, Rafael Perez-Jimenez, Stanislav Vitek, and Zabih Ghassemlooy. 2020. "Optical Camera Communications for IoT–Rolling-Shutter Based MIMO Scheme with Grouped LED Array Transmitter" Sensors 20, no. 12: 3361. https://doi.org/10.3390/s20123361

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop