# Detection of False Synchronization of Stereo Image Transmission Using a Convolutional Neural Network

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Stereo-Vision

## 3. Database

- the first-named MuchSlower, when the object in the right image was much further from the lens than the object in the left image,
- the second called Slower, when the object in the right image was further from the lens than the object in the left image,
- the third called Regular when the object in the right image was at the same distance from the lens as the object in the left image,
- the fourth named Faster when the object in the right image was slightly closer to the lens than the object in the left image,
- the fifth named MuchFaster, when the object in the right image was much closer to the lens than the object in the left image.

## 4. Convolutional Neural Network

## 5. Research

#### 5.1. Convolutional Neural Network Structure

#### 5.2. Input Data

#### 5.3. Learning Process

## 6. Results and Discussion

- $TP$–the sum of true positive results,
- $FP$–the sum of false-positive results,
- $TN$–the sum of true negative results,
- $FN$–the sum of false-negative results.

## 7. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Roldán, J.J.; Peña-Tapia, E.; Garcia-Aunon, P.; Del Cerro, J.; Barrientos, A. Bringing adaptive and immersive interfaces to real-world multi-robot scenarios: Application to surveillance and intervention in infrastructures. IEEE Access
**2019**, 7, 86319–86335. [Google Scholar] [CrossRef] - Musić, J.; Kružić, S.; Stančić, I.; Papić, V. Adaptive Fuzzy Mediation for Multimodal Control of Mobile Robots in Navigation-Based Tasks. Int. J. Comput. Intell. Syst.
**2019**, 12, 1197–1211. [Google Scholar] [CrossRef] - Walter, M.R.; Antone, M.; Chuangsuwanich, E.; Correa, A.; Davis, R.; Fletcher, L.; Frazzoli, E.; Friedman, Y.; Glass, J.; How, J.P.; et al. A Situationally Aware Voice-commandable Robotic Forklift Working Alongside People in Unstructured Outdoor Environments. J. Field Robot.
**2015**, 32, 590–628. [Google Scholar] [CrossRef] [Green Version] - Cadena, C.; Carlone, L.; Carrillo, H.; Latif, Y.; Scaramuzza, D.; Neira, J.; Reid, I.; Leonard, J.J. Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Trans. Robot.
**2016**, 32, 1309–1332. [Google Scholar] [CrossRef] [Green Version] - Beinschob, P.; Reinke, C. Graph SLAM based mapping for AGV localization in large-scale warehouses. In Proceedings of the 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3–5 September 2015; IEEE: Hoboken, NJ, USA, 2015; pp. 245–248. [Google Scholar] [CrossRef]
- Huang, B.; Zhao, J.; Liu, J. A Survey of Simultaneous Localization and Mapping. arXiv
**2019**, arXiv:1909.05214. [Google Scholar] - Chen, Y.; Wu, Y.; Xing, H. A complete solution for AGV SLAM integrated with navigation in modern warehouse environment. In Proceedings of the 2017 Chinese Automation Congress (CAC), Jinan, China, 20–22 October 2017; IEEE: Hoboken, NJ, USA, 2017; pp. 6418–6423. [Google Scholar] [CrossRef]
- Xue, B.; Cao, L.; Han, D.; Bai, X.; Zhou, F.; Jiang, Z. A DAISY descriptor based multi-view stereo method for large-scale scenes. J. Vis. Commun. Image Represent.
**2016**, 35, 15–24. [Google Scholar] [CrossRef] - Zhang, S. High-speed 3-D shape measurement with structured light methods: A review. Opt. Lasers Eng.
**2018**, 106, 119–131. [Google Scholar] [CrossRef] - Domínguez-Morales, M.; Domínguez-Morales, J.P.; Jiménez-Fernández, Á.; Linares-Barranco, A.; Jiménez-Moreno, G. Stereo Matching in Address-Event-Representation (AER) Bio-Inspired Binocular Systems in a Field-Programmable Gate Array (FPGA). Electronics
**2019**, 8, 410. [Google Scholar] [CrossRef] [Green Version] - Wang, F.; Lü, E.; Wang, Y.; Qiu, G.; Lu, H. Efficient Stereo Visual Simultaneous Localization and Mapping for an Autonomous Unmanned Forklift in an Unstructured Warehouse. Appl. Sci.
**2020**, 10, 698. [Google Scholar] [CrossRef] [Green Version] - Cruz-Santos, W.; Venegas-Andraca, S.E.; Lanzagorta, M. A QUBO Formulation of the Stereo Matching Problem for D-Wave Quantum Annealers. Entropy
**2018**, 20, 786. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Peng, J.; Xu, W.; Liang, B.; Wu, A.G. Pose Measurement and Motion Estimation of Space Non-cooperative Targets based on Laser Radar and Stereo-vision Fusion. IEEE Sens. J.
**2018**, 19, 3008–3019. [Google Scholar] [CrossRef] - Peng, J.; Xu, X.; Liang, B.; Wu, A. Virtual Stereo-vision Measurement of Non-cooperative Space Targets for a Dual-arm Space Robot. IEEE Trans. Instrum. Meas.
**2019**, 69, 1–13. [Google Scholar] [CrossRef] - Peng, J.; Xu, W.; Yuan, H. An Efficient Pose Measurement Method of a Space Non-Cooperative Target Based on Stereo Vision. IEEE Access
**2017**, 5, 22344–22362. [Google Scholar] [CrossRef] - Kulawik, J. The effect of edge operation on the detection of the reference element using the FREAK and SURF methods. In Monografia Naukowa “Mała Wielka Nauka”; Solarczyk, P., Ed.; Wydawnictwo Fundacji Promovendi: Łódź, Poland, 2017; Volume 1, pp. 26–41. [Google Scholar]
- Domínguez-Morales, M.J.; Jiménez-Fernández, Á.; Jiménez-Moreno, G.; Conde, C.; Cabello, E.; Linares-Barranco, A. Bio-Inspired Stereo Vision Calibration for Dynamic Vision Sensors. IEEE Access
**2019**, 7, 138415–138425. [Google Scholar] [CrossRef] - Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Choraś, M.; Giełczyk, A.; Demestichas, K.P.; Puchalski, D.; Kozik, R. Pattern Recognition Solutions for Fake News Detection. In Computer Information Systems and Industrial Management; Saeed, K., Homenda, W., Eds.; Springer: Berlin/Heidelberg, Germany, 2018; Volume 11127, pp. 130–139. [Google Scholar]
- Ksieniewicz, P.; Choraś, M.; Kozik, R.; Woźniak, M. Machine Learning Methods for Fake News Classification. In Intelligent Data Engineering and Automated Learning–IDEAL 2019; Lecture Notes in Computer Science; Yin, H., Camacho, D., Tino, P., Tallón-Ballesteros, A., Menezes, R., Allmendinger, R., Eds.; Springer: Berlin/Heidelberg, Germany, 2019; Volume 11872, pp. 332–339. [Google Scholar]
- Woźniak, M.; Połap, D. Adaptive neuro-heuristic hybrid model for fruit peel defects detection. Neural Netw.
**2018**, 98, 16–33. [Google Scholar] [CrossRef] [PubMed] - Ke, Q.; Zhang, J.; Wei, W.; Połap, D.; Woźniak, M.; Kośmider, L.; Damaševĭcius, R. A neuro-heuristic approach for recognition of lung diseases from X-ray images. Expert Syst. Appl.
**2019**, 126, 218–232. [Google Scholar] [CrossRef] - Woźniak, M.; Połap, D. Bio-inspired methods modeled for respiratory disease detection from medical images. Swarm Evol. Comput.
**2018**, 41, 69–96. [Google Scholar] [CrossRef] - Kubanek, M. A New Approach to Speech Recognition Using Convolutional Neural Networks. In Mathematical Modeling in Physics and Engineering (MMPE); Wydawnictwo Wydziału Zarządzania Politechniki Częstochowskiej: Czestochowa, Poland, 2018; pp. 24–30. [Google Scholar]
- Kubanek, M.; Bobulski, J.; Kulawik, J. A Method of Speech Coding for Speech Recognition Using a Convolutional Neural Network. Symmetry
**2019**, 11, 1185. [Google Scholar] [CrossRef] [Green Version] - Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning. Systemy Uczące Się; Wydawnictwo Naukowe PWN: Warsaw, Poland, 2018. [Google Scholar]
- Patterson, J.; Gibson, A. Deep Learning. Praktyczne Wprowadzenie; Wydawnictwo Helion: Gliwice, Poland, 2018. [Google Scholar]
- Chollet, F. Deep Learning. Praca z Językiem Python i Biblioteką Keras; Wydawnictwo Helion: Gliwice, Poland, 2019. [Google Scholar]
- Géron, A. Uczenie Maszynowe z Użyciem Scikit-Learn i TensorFlow. Wydanie II; Wydawnictwo Helion: Gliwice, Poland, 2020. [Google Scholar]
- © 1994–2020 The MathWorks, Inc. MATLAB Documentation; The MathWorks, Inc.: Natick, MA, USA, 2020; Available online: https://uk.mathworks.com/help/matlab/index.html (accessed on 1 December 2020).

**Figure 1.**An example of a pair of images. (

**a**) The image from the left camera. (

**b**) The image from the right camera.

**Figure 3.**Examples of image pairs for all five cases for the “orange” and “strawberry” objects. (

**a**) MuchSlower. (

**b**) Slower. (

**c**) Regular. (

**d**) Faster. (

**e**) MuchFaster.

**Figure 5.**The characteristics for the training progress and loss for Convolutional Neural Network (CNN). (

**a**) The characteristics for the training progress. (

**b**) The characteristics for the training loss.

**Figure 6.**Sample results of the proposed network for pairs of images from the Test set. (

**a**) MuchSlower 100.00%. (

**b**) MuchSlower 99.90%. (

**c**) Slower 100.00%. (

**d**) Slower 99.90%. (

**e**) Regular 99.20%. (

**f**) Regular 100.00%. (

**g**) Faster 99.80%. (

**h**) Faster 99.90%. (

**i**) MuchFaster 98.50%. (

**j**) MuchFaster 99.20%.

Event | Threshold [mm] |
---|---|

Much Slower | from −578 to −301 |

Slower | from −300 to −23 |

Regular | from −22 to 22 |

Faster | from 23 to 300 |

Much Faster | from 301 to 578 |

Event | Average Probability [%] |
---|---|

MuchSlower | 97.76 |

Slower | 97.70 |

Regular | 98.17 |

Faster | 95.61 |

MuchFaster | 97.99 |

**Table 3.**Appropriate metrics calculated for the obtained results of testing the correctness of the operation of the proposed network.

Event | Accuracy | Recall | Specificity | Precision |
---|---|---|---|---|

(A) [%] | (R) [%] | (S) [%] | (P) [%] | |

MuchSlower | 99.67 | 98.37 | 100.00 | 100.00 |

Slower | 99.67 | 100.00 | 99.59 | 98.40 |

Regular | 99.67 | 98.37 | 100.00 | 100.00 |

Faster | 98.86 | 95.93 | 99.59 | 98.33 |

MuchFaster | 99.19 | 100.00 | 98.98 | 96.09 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kulawik, J.; Kubanek, M.
Detection of False Synchronization of Stereo Image Transmission Using a Convolutional Neural Network. *Symmetry* **2021**, *13*, 78.
https://doi.org/10.3390/sym13010078

**AMA Style**

Kulawik J, Kubanek M.
Detection of False Synchronization of Stereo Image Transmission Using a Convolutional Neural Network. *Symmetry*. 2021; 13(1):78.
https://doi.org/10.3390/sym13010078

**Chicago/Turabian Style**

Kulawik, Joanna, and Mariusz Kubanek.
2021. "Detection of False Synchronization of Stereo Image Transmission Using a Convolutional Neural Network" *Symmetry* 13, no. 1: 78.
https://doi.org/10.3390/sym13010078