TickRS: A High-Speed Gapless Signal Sampling Method for Rolling-Shutter Optical Camera Communication

Abstract

Using the rolling-shutter mechanism to enhance the signal sampling frequency of Optical Camera Communication (OCC) is a low-cost solution, but its periodic sampling interruptions may cause signal loss, and existing solutions often compromise communication rate and distance. To address this, this paper proposes NoGap-RS, a no-gap sampling method, theoretically addressing the signal loss issue at longer distances from a perspective of CMOS exposure timing. Experiments show that NoGap-OOK, a OCC system based on NoGap-RS and On-Off key modulation, can achieve a communication rate of 6.41 Kbps at a distance of 3 m, with a BER of ${10}^{-5}$ under indoor artificial light. This paper further proposes TickRS, a time slot division method, innovatively addressing the overlap that occurs during consecutive-row exposures to further enhance communication rate. Experiments show that TickRS-CSK, a OCC system based on TickRS and Color-Shift Key, can achieve a communication rate of 20.09 Kbps at a distance of 3.6 m, with a BER of ${10}^{-2}$ under indoor natural light.

1. Introduction

With the development of mobile communications, the spectrum resource of traditional RF has become increasingly strained, prompting a shift in the research focus towards higher electromagnetic frequency domains [1,2]. The unlicensed, ultra-large-bandwidth Optical Wireless Communication (OWC) has become one of alternative solutions to the crisis. Optical Camera Communication (OCC), as an emerging research direction in this field, has been incorporated into the Visible Light Communication branch of the OWC standard: IEEE 802.15.7-2018 [3]. Recent studies have demonstrated the significant application potentials of OCC across multiple domains including underwater communication [4,5], vehicular networks [6], Internet of Things [7], and indoor positioning [8].

The OCC system equip LEDs as the emitter and various image sensors as the receiver, among which the Rolling-Shutter CMOS image sensor (RS-CMOS) stands out for its cost effectiveness, and the derived branch technology is known as Rolling-Shutter Optical Camera Communication (RS-OCC) [9]. RS-OCC takes the advantage of the rolling-shutter exposure mechanism, enabling the signal-sampling frequency at the receiver to substantially exceed the frame rate. Compared to the global-shutter CMOS image sensor, RS-CMOS offers both an economic advantage and a higher communication rate, with a theoretical throughput ranging from 100 bps to 100 Kbps [10,11]. The widespread equipment of RS-CMOS in mainstream mobile devices, including smartphones and tablets, provides a base of potential users to RS-OCC, underscoring the importance of continued research in this field.

However, RS-OCC faces a challenge of signal loss, manifested as periodic sampling interruptions. This issue stems from two factors: the first factor is the frame-to-frame time gap, preventing signal sampling during this gap. The second factor is a limited signal region. Experimental measurements revealed that the signal region illuminated by the LED covered only 7% of the CMOS pixel array under typical background light and at a communication distance of 3 m, and there was a symbol loss rate of 93.7%where.

Existing mitigation approaches employ various compensation strategies, such as data retransmission [9,12], error correction codes [13,14], and periodic matching [15]. Danakis et al. [9] pioneered the use of RS-CMOS to enhance OCC sampling frequency, employing a simple compensation method. This method sends each data packet twice to enable the receiver to reconstruct the complete packet. Rollinglight [13] is a low-rate yet robust method extending the bit period to exceed half of the frame period, then performing parity checking to recover lost bits. Nguyen et al. [14,16] proposed distributing the bits during each frame gap across different Hamming blocks, then performing error correction on each Hamming block to recover all lost bits. The basic idea of IoTorch [12] is similar to that of Danakis et al. [9], ensuring reliable transmission through retransmission. But IoTorch focuses on the periodicity of signal loss and designed a minimum repetition mechanism. These methods usually only consider the factor of the frame gap, and their communication distances are relatively short. In addition, these methods significantly increase the amount of redundant information and lower the net communication rate. To compensate for the rate, researchers have also implemented multiple channels employing Spatial Division Multiplexing by LED arrays [17,18], and employed advanced modulation schemes such as Color-Shift Keying (CSK) [19,20] and Orthogonal Frequency Division Multiplexing (OFDM) [21,22]. These improvements somewhat alleviate signal loss but inevitably increase the decoding latency of the receiver.

The paper focuses on CMOS exposure timing rather than the conventional method of the OCC data frame to address signal loss, and the proposed method requires the capability to precisely control the CMOS exposure timing. However, the parameter heterogeneity between OCC and CMOS presents a significant challenge. The paper establishes a relationship between the OCC sampling parameters and CMOS exposure parameters to identify the critical optimization parameters. Furthermore, the control of exposure timing involves operations at a hardware level. However, the existing experimental platform relies on Android or Windows, and their API demonstrates limited granularity in CMOS parameter control. To address this limitation, the paper implements a experimental platform through a open-source hardware and OS to achieve precise control. The following are the contributions of the paper:

• NoGap-RS proposes a gapless signal sampling method for high-rate, low-cost RS-OCC, which addresses the issue of signal loss from a theoretical perspective to extend the application scenarios of RS-OCC. It can work under background light and at a communication distance of 3 m, which is about 5 times better than the baseline [12].
• NoGap-RS is compatible with most modulation schemes, and when used with the On–Off Keying (OOK) modulation under SER of ${10}^{-5}$, it can achieve a communication rate of 6.41 Kbps, which is about 10 times better than the baseline [12].
• TickRS further proposes a time slot division method, addressing the overlap that occurs during consecutive-row exposures to further enhance the communication rate. TickRS-CSK, an OCC system based on TickRS and Color-Shift Key, can achieve a 48.52% improvement in bit rate compared to baseline systems, along with an extended communication range from 0.2 m to 3.6 m, representing a 3.4 m increase, while maintaining a bit error rate (BER) of 4.36%.

The following is the arrangement of sections in the paper. Section 2 briefly introduces the basic concepts of the RS-CMOS exposure and RS-OCC, and analyzes in detail two factors that lead to signal loss. Section 3 introduces NoGap-RS, a gapless signal sampling method, to address signal loss. Section 4 introduces TickRS, a time slot division method based on NoGap-RS, to improve the symbol rate. Section 5 designs TickRS-CSK, a complete RS-OCC process. Section 6 implements an experimental platform of NoGap-RS and TickRS-CSK to evaluate them.

2. Overview

2.1. RS-CMOS Exposure Concept

CMOS adopts a row-by-row exposure mechanism, and the exposure processes of all pixels within a row are fully synchronized [23]. Figure 1a shows the exposure parameters, where the exposure duration $T_{exp}$ of each row is the same, and there is a fixed readout time $T_{ro}$ after the exposure ends. Since all rows share a same readout circuit, there is a necessary time gap between the readout operations of adjacent rows [24], called the row blanking time $T_{hb}$. In addition, time overlaps occur during consecutive-row exposures and consecutive-frame exposures. As shown in Figure 1b, due to the overlap, five rows of pixels in both frames repeatedly capture the light in the light duration.

2.2. RS-OCC Concept

In RS-OCC, the sender encodes, frames, and modulates information into a series of luminance or chrominance signals, then sends them by rapidly adjusting the intensity of LED, with a transmission period $T_e$ that is much longer than $T_{exp}$ of CMOS. The receiver samples the LED signals, with each signal being sampled by multiple rows of pixels as shown in Figure 2a; the output image is visually represented as a series of stripes as shown in Figure 2b. Then the receiver decodes the stripes to reconstruct the original information through image processing or machine learning methods [21].

2.3. Problem Analysis

The primary factor of signal loss is the frame-to-frame time gap. The “Frame Gap” in Figure 3a represents the frame gap, during which the CMOS is in a non-exposure state, so it is evident that any signals sent during the gap will be lost. As OCC is asynchronous communication, the sender cannot foresee the timing of receiver’s frame gap; thus, signal loss caused by the frame gap is unavoidable. Theoretically, the length of the frame gap is positively correlated with the signal loss rate and is influenced by multiple parameters such as $T_{exp}$ and $fps$ of CMOS, which are interrelated in a complex manner. The conventional OCC implementation faces additional constraints, as commercial Android and Windows platforms restrict the application-layer access to low-level configuration interfaces of CMOS. This limitation introduces a substantial technical challenge in achieving precise temporal control for the frame gap.

Another key factor of signal loss is the insufficient area of the signal region. As shown in Figure 3b, RS-OCC works under strong background light at extended communication distances in practical application scenarios, so the LED of the sender illuminates only a restricted portion of the CMOS pixel array of receiver. Consequently, the receiver can reliably extract high signal-to-noise ratio signals solely from this illuminated region (manifested as alternating bright and dark stripes). The paper refers to the illuminated region as the signal region. As shown in Figure 3a, once the exposure of the last row in the signal region ends, the receiver will be unable to effectively sample signals until the signal region is subjected to exposure again in the next exposure cycle. Any signals sent during the wait will be lost, and this duration is represented by $T_{loss}$ in Figure 3a. The communication distance is positively correlated with the signal region area: the increased distance reduces the area of the signal region proportionally, thereby extending $T_{loss}$ and aggravating signal loss. This fundamental limitation explains why current RS-OCC implementations are distance-constrained, presenting a substantial technical barrier to achieve reliable long-distance communication.

3. NoGap-RS: A No-Gap Sampling Method for RS-OCC

The goal of NoGap-RS is to ensure that the consecutive-frame exposures in the signal region are sufficiently close in time, i.e., $T_{loss}<0$, to achieve gapless signal sampling to address signal loss. The approach of NoGap-RS is to precisely control $T_{loss}$ by adjusting various parameters of CMOS.

3.1. Read Collision Elimination

Before controlling $T_{loss}$, possible readout collisions should be eliminated. As shown in Figure 4, the readout collision refers to the readouts of two frames overlapping in time. Reducing $T_{loss}$ will significantly increase the probability of readout collisions, as CMOS needs to read out all rows under default settings, and the unnecessary non-signal region will occupy a large amount of the readout time. The method of eliminating readout collisions in NoGap-RS is to use exposure window, a configuration of CMOS, which strictly limits the exposure operation to a preset window region. NoGap-RS sets the exposure window to the signal region, thereby stopping the exposure and readout of the non-signal region, effectively eliminating readout collisions.

As shown in Figure 5, the exposure window is a rectangular region of a CMOS pixel array, and its position and size are determined by four parameters: $HS$ (horizontal start coordinate), $VS$ (vertical start coordinate), $HE$ (horizontal end coordinate), and $VE$ (vertical end coordinate). $(HS,VS)$ represents the coordinates of the first vertex of the window, while $(HE,VE)$ represents the coordinates of the diagonal vertex. Therefore, the number of rows of the window is

$$Y=VE-VS,$$

(1)

and the number of columns is

$$X=HE-HS.$$

(2)

Additionally, there is a minimum size constraint in the hardware on the exposure window, and the signal region will be smaller than the minimum exposure window under long-distance communication, resulting in some parts of the non-signal region participating in readout. In this case, $T_{loss}$ should be minimized as much as possible while avoiding readout collisions.

3.2. $T_{loss}$ Duration Control

This paragraph analyzes the various parameters of CMOS, with Figure 6 displaying the key parameters of exposure. $T_{ro}$ is the row readout duration, representing the time consumed to read all pixels in one row, and hence it is related to X, i.e.,

$$T_{ro}={\displaystyle \frac{X}{PCLK}},$$

(3)

where $PCLK$ is the readout clock frequency. $HTS$ is the row period, representing the time difference in readout between the previous and the next row, i.e.,

$$HTS=T_{ro}+T_{hb}.$$

(4)

$T_{vb}$ is the vertical blanking duration, representing the time difference between the last $HTS$ in the previous frame and the first $HTS$ in the next frame. $VTS$ is the frame period, representing the time difference in readout between the previous and the next frame, i.e.,

$$VTS=Y\ast HTS+T_{vb}.$$

(5)

$HTS$, $VTS$, X, Y, $T_{exp}$, and $PCLK$ are independent variables that can be set through CMOS registers.

NoGap-RS controls $T_{loss}$ by precisely setting the various parameters of CMOS. An original quantitative relationship between $T_{loss}$ and parameters can be obtained:

$$T_{loss}+T_{exp}=HTS+T_{vb},$$

(6)

convert to a relationship between $T_{loss}$ and the independent variables:

$$T_{loss}=(1-Y)·HTS+VTS-T_{exp}.$$

(7)

By using different parameter sets, $T_{loss}$ can be controlled.

To quantify the sampling situations of different $T_{loss}$, an overlap coefficient $\theta$ is defined as follows:

$$\theta =-{\displaystyle \frac{T_{loss}}{T_{exp}-HTS}},$$

(8)

where the denominator is the duration of the consecutive-row overlap. The following discusses the sampling situations of different $\theta$:

• When $\theta <0$, $T_{loss}>0$. It indicates the presence of signal loss.
• When $\theta =0$, $T_{loss}=0$. It satisfies the minimum requirement of the gapless sampling, where the exposure of the first row in the next frame starts precisely when the exposure of the last row in the previous frame ends.
• When $0<\theta <1$, $T_{loss}<0$. It indicates the presence of the time overlap during the consecutive-frame exposure.
• When $\theta =1$, $T_{loss}=T_{exp}-HTS$, and $T_{vb}=0$. At this point, the first $HTS$ in the next frame starts precisely when the last $HTS$ in the previous frame ends, giving the appearance that two frames merge into one frame, thus achieving transparent transmission as shown in Figure 7. But $\theta =1$ is the ideal situation, $T_{vb}$ has a minimum limitation in hardware, so $\theta$ can only approach 1.

In practice, making $\theta \ge 0.6$ can achieve approximately transparent transmission.

3.3. Theoretical Performance Analysis

This paragraph analyzes the theoretical sampling frequency $Fs$ of NoGap-RS. In RS-OCC, each exposure represents one sample, and hence

$$Fs=fps·Y,$$

(9)

converts to a relationship with the independent variables

$$Fs={\displaystyle \frac{Y}{VTS}}.$$

(10)

This paragraph analyzes the maximum modulation frequency f of simulating modulation in NoGap-RS. f is not directly related to $Fs$ because the sampling of RS-OCC partially overlaps in time, whereas traditional sampling should be independent. Therefore, f is directly related to $T_{exp}$:

$$2·T_{exp}\le {\displaystyle \frac{1}{f}}.$$

(11)

This paragraph analyzes the maximum symbol rate of digital modulation in NoGap-RS. Some samples contain the signals of multiple symbols and are considered invalid. In contrast, samples that contain the signal of a single symbol are regarded as valid. As shown in Figure 8, each symbol should be validly sampled at least twice to distinguish between the valid and the invalid samples. A relationship between the times of the valid sampling n and $T_e$ is as follows:

$$T_e=T_{exp}+n·HTS,n\ge 2.$$

(12)

So the symbol rate B is

$$B={\displaystyle \frac{1}{T_{exp}+n·HTS}},n\ge 2.$$

(13)

This paragraph qualitatively analyzes the impact of the different parameters on performance:

• According to Equations (11) and (13), $T_{exp}$ is negatively correlated with the communication rate. However, $T_{exp}$ should not be too small because the OCC sampling is essentially the process of cumulative exposure. If $T_{exp}$ is too small, the dynamic range of the signal is limited, thus reducing the ability of the receiver to distinguish between different light levels, which may lead to an increase in Symbol Error Rate (SER).
• According to Equation (13), the sampling count n is negatively correlated with B, but if n is too small, it complicates the receiver’s task of distinguishing sample validity.
• According to Equation (13), $HTS$ is negatively correlated with B, but if $HTS$ is too low, the ratio of Y to $fps$ becomes too low, which means a higher frequency of nearly the transparent transmissions, potentially leading to an increase in SER.

4. TickRS: A Time-Slot-Division Sampling Method Based on NoGap-RS

The goal of this section is to design a time slot division mechanism called TickRS to better utilize high-density pixel arrays in order to improve sampling speed. The approach involves precisely dividing each exposure into multiple Ticks based on the $T_{loss}$ control method proposed in Section 3, and utilizing the partial correlation of the exposure values of neighboring rows to calculate the exposure value within each Tick.

4.1. Tick Division

We divide each exposure into N Ticks, the duration $t_{Tick}$ of each Tick being the same as the time $t_{read}$ to read out a row of pixels. Obviously, the exposure duration $t_{exposure}$ should be an integer multiple of $t_{Tick}$, and the total number of Ticks per frame is $N+Y-1$, recorded as S, where Y is the number of rows of CMOS. Figure 9 is a simple example of TickRS for two consecutive frames, with $N=4$, $Y=5$, and $S=8$.

4.2. Tick Solution

Theoretically, it can be considered that the light signal in the same Tick has the same effect on all pixels being exposed, and the exposure of each pixel can be considered the sum of the light signals of all Ticks in the exposure period. Therefore, we have the following formula for the exposure value c of each pixel:

$$c=v_y+v_{y+1}+...+v_{y+N-1},$$

(14)

where y is the number of the row where the pixel is located, and $v_y$ is the light signal of the yth Tick. Taking Figure 9 as an example, for all rows of pixels, we have the following equation:

$$\left\{\begin{array}{c}\overline{c_1}=v_1+v_2+v_3+v_4+v_5\hfill \\ \overline{c_2}=v_2+v_3+v_4+v_5+v_6\hfill \\ \overline{c_3}=v_3+v_4+v_5+v_6+v_7,\hfill \\ \overline{c_4}=v_4+v_5+v_6+v_7+v_8\hfill \\ \overline{c_5}=v_5+v_6+v_7+v_8+v_9\hfill \end{array}\right.$$

(15)

where $\overline{c_1}$ is the statistical value of the exposure values of all pixels in the first row, which depends on the algorithm used to select the column pixels. And $v_1$ is the light signal of the first Tick of the frame. At the receiver, $\overline{c_1}\sim \overline{c_5}$ is a known value, and $v_1\sim v_9$ is a value to be calculated. Obviously, this equation has multiple solutions.

In order to make the equation have a unique solution, we define a constant light value $v_{constant}$ that is larger than the upper limit value $v_{limit}$ for the pixel exposure value. Then we let the LED continue to emit light for a certain time in each frame cycle so that the exposure value of at least one row of pixels in each frame of the CMOS reaches $v_{constant}$. When the exposure value of a row of pixels is $v_{constant}$, the light signal of all Ticks in the exposure period of the pixel is the known value $v_{constant}/N$ so that Equation (15) has a unique solution. It should be noted that $v_{limit}$ is the maximum value of valid information, and $v_{constant}$ is a value set to distinguish valid information from constant light signals. Neither of them is the full trap capacity of the pixel.

4.3. Constant Light Signal Duration

OCC can be considered asynchronous communication, and the sender does not know the frame timing of the receiver during operation. Therefore, the constant light signal of the LED emitter may occur partly in the previous frame of the receiver and partly in the next frame. At this time, the constant light signal must last for $2N+1$ Ticks to guarantee that the exposure value of a row of pixels in each frame is $v_{constant}$.

However, the constant light signal is not valid data. The length of the constant light signal is determined by N, so a too large N will cause a loss in the Tick efficiency p, which violates the original intention of TickRS. The specific quantitative relationship between N and p is as follows:

$$p={\displaystyle \frac{E}{S}}={\displaystyle \frac{Y-N-2}{N+Y-1}},$$

(16)

where E is the number of valid Ticks in a frame.

4.4. Sampling Rates and Performance Analysis

The number of samples per second is the number of frames per second multiplied by the number of valid Ticks per frame, so the sampling rate $Fs$ has the following equation:

$$Fs=fps·E,$$

(17)

where $fps$ is $PCLK$ minus the number of clock cycles consumed per frame, so $fps$ has the following equation:

$$fps={\displaystyle \frac{PCLK}{HTS·VTS}},$$

(18)

substituting in

$$Fs={\displaystyle \frac{PCLK}{HTS·(Y+N-1)}}·(Y-N-2).$$

(19)

Next, we explore the impact of $HTS$, Y, and N on $Fs$ and sampling delay $delay$. $delay$ is defined as the average time from each valid Tick in the frame to the end of the output of the last row of pixels in the frame. The $PCLK$ is fixed at 120 MHz. Figure 10a shows $Fs$ and $delay$ when $HTS$ = 1970. The horizontal axis is the number of rows Y in the exposure window. It can be seen from Figure 10a that the smaller the number of Ticks N, the larger the $Fs$. This is because when N increases, the Tick efficiency p decreases.

As Y increases, $Fs$ also increases, but $delay$ increases, so comprehensive consideration should be given. And when the signal region is very small, in order to minimize the blank period between frames, Y should also be as small as possible. In addition, if a serious deviation occurs within a Tick, it may cause an error in the entire frame, and an excessively large Y means that there are more valid Ticks in a frame, and SER will increase.

Figure 10b shows $Fs$ and $delay$ when Y = 480, with $HTS$ as the horizontal axis. It can be seen that as $HTS$ increases, $Fs$ decreases and $delay$ increases, so $HTS$ should not be too large. However, CMOS sampling is not instantaneous and requires a cumulative exposure process. Therefore, a too small $T_{tick}$ causes the optical signal in a period to be excessively dispersed into multiple consecutive Ticks, which does not help reduce SER, and even requires more calculations to recover information during demodulation, so $HTS$ should not be too small. In addition, an excessively large $HTS$ will also cause $T_{exp}$ to be too long, causing overexposure.

5. TickRS-CSK

This section constructs TickRS-CSK, a complete receiving process for RS-OCC, based on the CSK modulation and TickRS. CSK employs different colors as different symbols, and uses a constellation map based on color space to decide symbols. In OCC-based CSK, different LEDs can emit the visible light of three wavelengths, and CMOS receives and distinguishes them through a Color Filter Array (CFA), which separates pixels into three channels (usually red, green, and blue) based on the optical wavelength. CSK improves the spectral utilization of visible light in the OCC system. Figure 11 is the specific process of TickRS-CSK.

5.1. TickRS Under CFA

TickRS is a pixel-level method that focuses on the exposure timing of each row of pixels, which means that TickRS-CSK needs to directly receive signals from pixels from the three channels simultaneously. However, not every row contains pixels from all three channels. Take the following Bayer array as an example:$\left[\begin{array}{cc}B& G\\ G& R\end{array}\right]$. Each row contains pixels from two channels, resulting in unsynchronized exposure between different channels. Therefore, TickRS requires certain modifications to be applicable to CSK.

Obviously, the red and blue pixels are arranged in alternate rows, which means that the interval between the red and blue pixels starting to be exposed is two Ticks. According to the principle of TickRS, only the total light signal of two Ticks can be solved at this time, so the red and blue pixels can only be solved with two Ticks as a unit for TickRS; N is an even number. However, green pixels are present in every row, and the number is twice that of the red or blue pixels. The paper uses only the green pixels in odd rows, and also solves them with two Ticks as a unit, aligned with the unit of blue pixels as shown in Figure 12.

It should be noted that in order to ensure that all color channels can be sampled continuously, the frame interval is reduced by one Tick in quasi-overlap mode. Therefore, the total effective Ticks E in the frame and the frame rate $fps$ of TickRS under CSK modulation both change, so $Fs$ also changes:

$$F{s}_{CSK}={\displaystyle \frac{PCLK}{HTS·(Y+N-2)}}·({\displaystyle \frac{Y}{2}}-{\displaystyle \frac{N}{2}}-2).$$

(20)

After each color group is solved, the units of different colors are arranged in the timeline as shown in Figure 12.

5.2. Background Light Measurement

First, capture a frame with a preset exposure time. If the signal region pixels are full, change the preset parameters. Then, in order to fully utilize the ADC range to prevent overflow or loss of accuracy, determine the appropriate analog gain. Algorithm 1 calculates the analog gain, and also outputs the background light intensity $\mathbf{D}$, which is a three-dimensional RGB vector for the subsequent RGB correction.

Algorithm 1 Calculate $ISO$ and $\mathbf{D}$

Input: $list of ISO$, $limit of pixel$ and $T_{exp}$;

Output: $ISO$ and $\mathbf{D}$;

1: Turn on all LEDs;   2: $i=0$;   3: repeat   4:      $i=i+1$;   5:      $ISO=list of ISO\left[i\right]$;   6:      Capture signal region with $ISO$ and $T_{exp}$;   7:      Output raw $on pixels$;   8: until $max(on pixels)<0.8\ast limit ofpixel$   9: Turn off all LEDs; 10: Capture signal region with $ISO$ and $T_{exp}$; 11: Output raw $offpixels$; 12: $\mathbf{D}=\left(\begin{array}{ccc}\overline{offpixel{s}_R}& \overline{offpixel{s}_G}& \overline{offpixel{s}_B}\end{array}\right)$ 13: return $ISO$ and $\mathbf{D}$.

5.3. RGB Correction

What needs to be emphasized is the color difference problem, that is, the color received by the receiver is different from the color sent by the sender. There are three main reasons:

• White balance: CMOS usually responds more strongly to a certain frequency of light than other colors, usually green, and the raw data has not been white balanced, so it appears to be severely green.
• Inter-channel interference: The frequency response curves of the Bayer array for different colors of light overlap, that is, the same color of light may affect multiple pixels of different colors at the same time [25].
• Heterogeneity of CMOS and LED: The spectrum curve of LED and the frequency response curve of the CMOS Bayer array are not completely matched.

Therefore, it is necessary to correct the RGB value before CSK demodulation. The correction method proposed by [25] is applied to TickRS-CSK. The calculation method of the correction matrix $\mathbf{T}$ is different from the original method. Algorithm 2 is its algorithm flow, the matrix $\mathbf{D}$ comes from Algorithm 1, and $leve{l}_{pwm}$ is the inverse of the PWM resolution of the sender, that is, the upper limit value $v_{limit}$ of the signal.

After TickRS is solved, $\mathbf{D}$ is subtracted from the result and then corrected by Equation (21). $R_{rx}$ is the light signal of the red pixel unit of the receiver, which is the value before calibration; $r_{tx}$ is the corrected value of the unit, which is the estimated value of the original light signal at the sender:

$$\left(\begin{array}{ccc}{r}_{tx}& g_{tx}& b_{tx}\end{array}\right)=\left(\begin{array}{ccc}{R}_{rx}& G_{rx}& B_{rx}\end{array}\right)·{\mathbf{T}}^{-1}.$$

(21)

Algorithm 2 Calculate $\mathbf{T}$

Input: $ISO$, N, $\mathbf{D}$ and $leve{l}_{pwm}$;

Output: $\mathbf{T}$;

1: Turn on red LED;   2: Capture signal region with $ISO$;   3: Output raw $rpixels$;   4: ${\mathbf{H}}_{\mathbf{r}}=\left(\begin{array}{ccc}\overline{rpixel{s}_R}& \overline{rpixel{s}_G}& \overline{rpixel{s}_B}\end{array}\right)-\mathbf{D}$;   5: Turn off red LED;   6:     7: Turn on green LED brightest;   8: Capture signal region with $ISO$;   9: Output raw $gpixels$; 10: ${\mathbf{H}}_{\mathbf{b}}=\left(\begin{array}{ccc}\overline{gpixel{s}_R}& \overline{gpixel{s}_G}& \overline{gpixel{s}_B}\end{array}\right)-\mathbf{D}$; 11: Turn off green LED; 12:   13: Turn on blue LED brightest; 14: Capture signal region with $ISO$; 15: Output raw $bpixels$; 16: ${\mathbf{H}}_{\mathbf{b}}=\left(\begin{array}{ccc}\overline{bpixel{s}_R}& \overline{bpixel{s}_G}& \overline{bpixel{s}_B}\end{array}\right)-\mathbf{D}$; 17:   18: $\mathbf{T}={\left(\begin{array}{ccc}{{\mathbf{H}}_{\mathbf{r}}}^T& {{\mathbf{H}}_{\mathbf{g}}}^T& {{\mathbf{H}}_{\mathbf{b}}}^T\end{array}\right)}^T/leve{l}_{pwm}/N$; 19: return $\mathbf{T}$.

5.4. Symbol Synchronization

According to Nyquist theorem, the symbol rate $B={\displaystyle \frac{F{s}^{\prime }}{2}}$. In other words, the symbols of the emitter are likely to be distributed between two units of the receiver. In order to correctly recover the symbols, the symbol period should be at least twice $T_{unit}$. In addition, the receiver needs to recover the clock of the sender to determine which units represent a complete symbol [23]. Therefore, in this experiment, the sender inserts four groups of ‘01’ before the constant light signal of each color channel in the first frame at the beginning of communication, with turning off the LED as ‘0’ and $v_{constant}$ as ‘1’.

6. Experiments and Assessment

This section evaluates the performance of NoGap-RS method and TickRS-CSK process. Section 6.1 introduces the details of the hardware and software implementations of experiment platform. Section 6.2 describes the experiment conditions, parameter setups, and other details of the NoGap-RS experiment and TickRS-CSK experiment.

6.1. Implement

6.1.1. Receiver

To facilitate the software implementation and the performance evaluation, this system uses the open-source OrangePi 5 development board as the receiver hardware platform, with the operating system being Ubuntu 22.04 and the Linux kernel version 6.1.43 as shown in Figure 13. The camera is an accompanying module with a lens that has the field of view of 77.6° and a CMOS model OV13850. The OV13850 has 4224 × 3136 pixels, and it outputs the image in the “10-bit RGB RAW” format. The parameter such as $ISO$, $T_{exp}$ and $fps$ can be controlled via registers, and the programmable parameters are beneficial for the implementation of NoGap-RS.

The receiver implements the NoGap-RS method by modifying the Linux driver source code to configure the OV13850 registers. The specific implementation method involves adding multiple groups of parameter sets conforming to NoGap-RS standard to the driver. The modified driver is then loaded; this application can support NoGap-RS communication in different conditions by selecting different parameter sets and manually specifying the signal region as the exposure window, while retaining the ability to capture images to meet image processing needs such as signal region detection.

The receiver saves the frame stream in the format of 10-bit RGB RAW from the CMOS to files.Then the researcher uses Python 3.11 on a PC for file parsing, pixel extraction, row averaging, demodulation, SER and BER calculation steps, and so on. Row averaging is achieved by calculating the arithmetic mean of all pixels in the each row, resulting in a unique sampling value for the unit time period. The validity of this method is based on the synchrony of exposure for the pixels in the same row.

6.1.2. Sender

The sender consists of a control part and emitter. The role of the control part is to convert the data to symbols, then convert the symbols to waveforms of LED, thus driving the fixture to send a brightness signal. As shown in Figure 14a, the control circuit uses the ESP32-C3 development board, which has a 384 KB ROM and a dedicated cache of 16 KB, and supports dedicated GPIO, with a maximum GPIO speed of 80 MHz. The driving circuit uses the AOD4184 NMOS, with a response time of 35 ns.

As shown in Figure 14b, the system uses an LED fixture with a large luminous region as emitters, ensuring that the signal region meets the minimum exposure window during long-distance communication experiments. The fixture has built-in 10 RGB LEDs, and is equipped with a light diffuser module of 13 cm long and 6.5 cm wide. Each LED has a rated power of 150 mW. The sender controls all LEDs of the same color as a single emission source, ensuring that the brightness of the LEDs of the same color is strictly synchronized during communication. Therefore, the system implements wavelength division channels through three color emission sources.

To ensure the accuracy of the signal emission timing, the sender’s software architecture undergoes targeted optimization. A timer interrupt with a period of $T_e$ is used, and the interrupt service function is responsible for reading the next level of wavelength from the buffer and outputting it to the pin.

6.2. Experiment Setup

6.2.1. NoGap-OOK Setup

To separately assess the performance of the NoGap-RS method, the NoGap-RS method is integrated with OOK modulation to form a NoGap-OOK communication process. It adopts a basic modulation and intentionally avoids using any error correction coding so as to ensure that the measured SER truly reflects the performance of NoGap-RS itself. Due to OOK being the non-multi-channel modulation, only the green channel among three wave division channels of the experiment platform is used in the NoGap-OOK experiment. In addition, as OOK modulation only has two symbols, 0 and 1, the sender for NoGap-OOK experiment directly uses GPIO to output the high or low level. A direct channel between the output pin and the CPU is established through the dedicated GPIO functionality, effectively reducing the delay time of the level output.

This paragraph describes the conditions of the NoGap-OOK experiment. Firstly, the experiment sets up four communication distances, 0.6 m, 1.2 m, 2.4 m, and 3 m, to verify the communication distance robustness of NoGap-RS. Secondly, the experiment sets up three background lights—the outdoor natural light, the indoor natural light, and the indoor artificial light without the frequency flicker—to verify the background light robustness of NoGap-RS. Finally, the experiment sends 15 data samples (each 15 KB), covering various file types such as audio, images, video, executable programs, and text documents, to verify data robustness of NoGap-RS. The NoGap-OOK experiment is conducted under 3 background lights and 4 communication distances, totaling 12 groups.

All experiments also use the ’NoGao-OOK’ parameter set in

Table 1. The parameter sets of various methods.

	X	Y	$\mathit{HTS}$	$\mathit{VTS}$	${\mathit{T}}_{\mathit{exp}}$/µs	N	${\mathit{Fs}}_{\mathit{CSK}}$/KHz	${\mathit{delay}}_{\mathit{CSK}}$/ms	Maximum Distance/m	Communication Rate/Kbps
NoGap-OOK	152	216	160	224	89.3	/	/	/	3	6.41
IoTorch-OOK	/	/	/	/	200	/	/	/	0.4	0.5556
TickRS-CSK Mode1	1962	480	1970	502	394	24	27.42	4.53	0.6	41.14
TickRS-CSK Mode2	2602	320	2610	342	522	24	19.63	4.26	1.2	29.45
TickRS-CSK Mode3	3242	160	3250	182	650	24	13.39	3.14	3.6	20.09

, and the reason for the parameters is as follows: (1) To achieve the longer communication distance, the size of the exposure window (X and Y) should be slightly smaller. (2) To achieve the maximum rate, $HTS$ and $T_{exp}$ should be as low as possible. (3) To better assess the validity of sampling, the valid sampling count n is 50. (4) The final communication rate is determined by n, $HTS$, and $T_{exp}$.

In addition, the author uses the same hardware platform to implement the IoTorch-OOK system based on IoTorch [12] and OOK modulation to provide an intuitive comparison of NoGap-RS with the existing methods. As the IoTorch method requires that the total exposure time of each frame is greater than the interval duration, the IoTorch-OOK experiment is conducted under 0.4 m communication distance and three background lights, totaling 3 groups.

This paragraph describes the parameters of the IoTorch-OOK experiment. The signal region in the image has a height of 1801 rows, and $T_{exp}$ is set to 200 µs, meaning the total exposure duration of the signal region per frame is 72.2 ms, with an interval duration of 60.92 ms. In addition, the number of symbol samples is set to 10, meaning $T_e$ is set to 600 µs. According to the IoTorch [12] formula, the packet length is set to 10 symbols, which is 6 ms; the block size is set to 2, and the number of retransmissions is set to 3. Thus, the block cycle is 36 ms, and the net communication rate is approximately 555.6 bps.

6.2.2. TickRS-CSK Setup

This section describes the conditions of the TickRS-CSK experiment. Firstly, the experiment sets up four communication distances, 0.6 m, 1.2 m, 2.4 m, and 3.6 m, to verify the communication distance robustness of TickRS-CSK. Secondly, as shown in Figure 15, the experiment site is placed in the indoor natural light. The experiments are conducted at three different points in the day to introduce the background lights of varying intensities to verify the background light robustness of TickRS-CSK. Thirdly, the experiment sends 30 data samples (each 3 MB) to verify the data robustness of TickRS-CSK. Finally, the experiment sets up three modes to verify the communication rate robustness of TickRS-CSK, and the corresponding parameter sets are in Table 1. The NoGap-OOK experiment is conducted under 3 background lights, 3 modes and required communication distances, totaling 21 groups.

The reason for the parameters of the three modes is as follows: (1) To test the impact of different Y on the system, three modes adopt different Y. (2) To achieve different bit rates, three modes adopt different $HTS$, resulting in different $T_{exp}$. (3) To avoid overexposure and reading collisions, N is conservative. (4) The final communication rate is determined by N, Y, and $HTS$.

The considerations for parameters in different modes mainly concern the communication distance and communication rate. Mode1 has a larger Y and a smaller $HTS$, so it has the largest communication rate among the three modes and is suitable for short-distance communication and good channel quality. Mode3 has a smaller Y and a larger $HTS$, so it has the smallest communication rate among the three modes and is suitable for long-distance communication.

6.3. Experiment Results

6.3.1. NoGap-OOK Results and Comparison

There are 12 groups in the NoGap-OOK experiment, and the result of each group is averaged over 15 SER. Figure 16 compares the SER of 12 groups.

This paragraph analyzes the results of the NoGap-OOK experiment. (1) Regarding the background light robustness, under outdoor natural light, SER is the highest, reaching the order of ${10}^{-2}$; under indoor artificial light, SER is the lowest, reaching the order of ${10}^{-5}$. It indicates that different background lights have a noticeable impact on system performance, but SER for all groups remains within an acceptable range. (2) Regarding the communication distance robustness, SER varies by one to two orders of magnitude at different communication distances. It indicates that there is a positive correlation between communication distance and SER, and them all remain within an acceptable range. (3) Regarding data robustness, the upper and lower limits of SER for all groups fluctuate within one order of magnitude. It indicates that NoGap-RS is data independent. The experiment confirms that the NoGap-RS method has a certain degree of background light robustness, strong communication distance robustness, and significant data robustness.

There are 3 groups in the IoTorch-OOK experiment, and the results of each group are averaged over 15 SER. Figure 17 compares the SER of 6.41 Kbps NoGap-OOK at 0.6 m communication distance with SER of 555.6 bps IoTorch-OOK at 0.4 m communication distance.

This paragraph compares IoTorch-OOK with NoGap-OOK. The result show that the difference in SER is relatively limited, but the IoTorch-OOK SERs under three background lights are all lower than those of NoGap-OOK, which can be attributed to the longer exposure time of IoTorch-OOK, thus resulting in better distinction between the different symbols. In the indoor artificial light with relatively high signal-to-noise ratios, even a very short exposure time can ensure good symbol distinction, so the difference in SER is not significant in this case. In fact, NoGap-OOK can also achieve lower SER by increasing the exposure time by decreasing communication rate. Furthermore, the SER fluctuation in IoTorch-OOK is greater, which may be related to the fewer sampling times. In summary, NoGap-OOK achieves a longer communication distance and greater communication rate, with an increase of about 5 times and 10 times respectively, at the cost of a slight increase in SER.

6.3.2. TickRS-CSK Results and Comparison

There are 21 groups in the TickRS-CSK experiment, and the result of each group is averaged over 20 BER. Figure 18 compares the BER of 21 groups. The first three sub-figures are BER graphs under three background lights, and the fourth sub-figure compares the BER of various background lights under Mode3.

This paragraph analyzes the results of the TickRS-CSK experiment. (1) Regarding the communication rate robustness, the difference in BER is not significant for different modes, and BER of the slowest Mode3 is the lowest in all experiments. This is attributed to the fact that its Y and valid units in the frame are both minimal; if the entire frame suffers an error, then Mode3 will have fewer erroneous units. Additionally, its $HTS$ is the largest, meaning the symbol period was the longest, allowing for a greater distinction between different symbols. (2) Regarding the communication distance robustness, BER varies by one order of magnitude at different communication distances, and BER is inversely proportional to the distance. (3) Regarding the background light robustness, according to the fourth sub-image in Figure 18, the difference in BER is not significant for different light robustness, and BER is directly proportional to the background light intensity.

The overall BER of TickRS-CSK is within an acceptable range, but it reaches the order of ${10}^{-2}$, which is relatively high compared to NoGap-OOK. There are three possible reasons. Firstly, the TickRS method has intra-frame coupling, leading to a higher possibility of entire frame error. Secondly, TickRS-CSK uses PWM to emit the signal at the sender and requires software to change the PWM duty cycle, which may result in an emission timing delay. In contrast, NoGap-OOK uses dedicated GPIO at the sender to reduce emission timing delays. Thirdly, the RGB correction in Section 5.3 should be time-independent, but as shown in Figure 12, there is a temporal mismatch between the red pixel units and the others, resulting in inaccurate RGB correction.

This paragraph compares TickRS-CSK with other RS-OCC systems. The systems are introduced as follows: D-ELFC [22] uses RS-CMOS as a receiver and uses relatively advanced OFDM modulation. It has the characteristics of high speed, but it adopts the traditional sampling method. Like TickRS-CSK, SCSK [20], CSSK [26] and RS-CSK [27] also use RS-CMOS and CSK modulation, but they also adopt the traditional sampling method. Figure 19 compares the communication rate and BER of these systems, and the BER and distance data of other systems are obtained from previous research papers. The communication rate of TickRS-CSK is improved by 48.52% compared to D-ELFC, and the communication distance of TickRS-CSK is increased from 0.2 m of D-ELFC to at least 0.6 m. In summary, although the BER of TickRS-CSK is relatively high, it achieves a communication distance and communication rate that surpass currently advanced RS-OCC systems.

7. Conclusions and Discussion

The paper presented and elaborated on NoGap-RS, which effectively addresses the technical challenge of signal loss in the RS-OCC system by implementing the programmable control of sampling parameters, thereby significantly enhancing the upper limit of the RS-OCC’s communication rate. Theoretically, NoGap-RS can be combined with most modulation schemes to improve the communication rate of the RS-OCC system. In addition, the SoC, operating system, and CMOS interface protocol of the experimental platform are similar to most mobile devices, suggesting that NoGap-RS is compatible with most mobile phones to some extent, which expands the applicability of NoGap-RS.

In addition, the paper further explored the potential of NoGap-RS and developed TickRS, a high rate method based on time slot division. The TickRS-CSK achieved a BER of 4.36% at a rate of 20.09 Kbps and a distance of 3.6 m, while at a shorter distance of 0.6 m, the data rate of 41.14Kbps exceeded most conventional RS-OCC systems, demonstrating the potential of NoGap-RS and TickRS in high-rate OCC field. Future investigations will focus on extensive experimental validation across multiple CMOS platforms, outdoor light scenarios, and artificial light scenarios with power line interference.