# Rotation Invariant Networks for Image Classification for HPC and Embedded Systems

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

#### 2.1. Continuous Filter Sampling

#### 2.2. Discrete Angular Sampling

## 3. Methodology

#### 3.1. Oriented Representation Mapping

**x**(Figure 2a). An oriented component is obtained by the product $\mathrm{x}\times {g}^{\phi}$ where $\phi $ is the orientation of an oriented edge detector g. Furthermore, we have a set of orientations such as $i=0,\dots ,N$, with $N\in {\mathbb{Z}}^{+}$ where N is the number of orientations in which we want to decompose the input image. This means that the set $[\mathrm{x}\times {g}^{{\phi}_{i}}]$ contains N oriented components (features) of the input image $\mathrm{x}$. Furthermore, we can define $d\phi =2\pi /N$ as the angular sampling magnitude in degrees for a filter with periodicity $2\pi $. Then the orientation of the filter in the position i position is ${\phi}_{i}=2\pi i/N$.

**Definition**

**1**

**.**Let Φ be a mapping $\mathsf{\Phi}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n\times N}$, and $\mathsf{\Phi}\left(\mathrm{x}\right)$ a feature representation of $\mathrm{x}$

#### Gabor Filters

#### 3.2. Feature Extraction

#### State-of-the-Art Architectures as Backbone (ResNet)

#### 3.3. Translating Predictor

#### 3.4. Memory-Footprint vs. Data Parallelism

## 4. Results

#### 4.1. MNIST

#### 4.2. CIFAR-10

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Gu, Y.; Wang, Y.; Li, Y. A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection. Appl. Sci.
**2019**, 9, 2110. [Google Scholar] [CrossRef][Green Version] - Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Westberg, S.; Sidike, P.; Nasrin, M.S.; Hasan, M.; Van Essen, B.C.; Awwal, A.A.; Asari, V.K. A state-of-the-art survey on deep learning theory and architectures. Electronics
**2019**, 8, 292. [Google Scholar] [CrossRef][Green Version] - Véstias, M.P. A survey of convolutional neural networks on edge with reconfigurable computing. Algorithms
**2019**, 12, 154. [Google Scholar] [CrossRef][Green Version] - O’Gara, S.; McGuinness, K. Comparing Data Augmentation Strategies for Deep Image Classification; IMVIP; Technological University Dublin: Dublin, Ireland, 2019. [Google Scholar]
- Zhang, C.; Patras, P.; Haddadi, H. Deep learning in mobile and wireless networking: A survey. IEEE Commun. Surv. Tutor.
**2019**, 21, 2224–2287. [Google Scholar] [CrossRef][Green Version] - Maiterth, M.; Wilde, T.; Lowenthal, D.; Rountree, B.; Schulz, M.; Eastep, J.; Kranzlmüller, D. Power aware high performance computing: Challenges and opportunities for application and system developers—Survey & tutorial. In Proceedings of the International Conference on High Performance Computing & Simulation (HPCS), Genoa, Italy, 17–21 July 2017; pp. 3–10. [Google Scholar]
- Rofouei, M.; Stathopoulos, T.; Ryffel, S.; Kaiser, W.; Sarrafzadeh, M. Energy-aware high performance computing with graphic processing units. In Proceedings of the Workshop on Power Aware Computing and System, San Diego, CA, USA, 7 December 2008. [Google Scholar]
- Zong, Z.; Ge, R.; Gu, Q. Marcher: A heterogeneous system supporting energy-aware high performance computing and big data analytics. Big Data Res.
**2017**, 8, 27–38. [Google Scholar] [CrossRef] - Shang, Y. Vulnerability of networks: Fractional percolation on random graphs. Phys. Rev. E
**2014**, 89, 012813. [Google Scholar] [CrossRef] [PubMed] - Wang, L.; Khan, S.U.; Chen, D.; KołOdziej, J.; Ranjan, R.; Xu, C.Z.; Zomaya, A. Energy-aware parallel task scheduling in a cluster. Future Gener. Comput. Syst.
**2013**, 29, 1661–1670. [Google Scholar] [CrossRef] - Zhang, C.; Wu, D.; Sun, J.; Sun, G.; Luo, G.; Cong, J. Energy-efficient CNN implementation on a deeply pipelined FPGA cluster. In Proceedings of the 2016 International Symposium on Low Power Electronics and Design, San Francisco, CA, USA, 19–21 August 2016; pp. 326–331. [Google Scholar]
- Gai, K.; Qin, X.; Zhu, L. An energy-aware high performance task allocation strategy in heterogeneous fog computing environments. IEEE Trans. Comput.
**2020**. [Google Scholar] [CrossRef] - Abdelouahab, K.; Pelcat, M.; Serot, J.; Berry, F. Accelerating CNN inference on FPGAs: A survey. arXiv
**2018**, arXiv:1806.01683. [Google Scholar] - Worrall, D.E.; Garbin, S.J.; Turmukhambetov, D.; Brostow, G.J. Harmonic deep: Networks translation and rotation equivariance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar] [CrossRef][Green Version]
- Cohen, T.S.; Geiger, M.; Koehler, J.; Welling, M. Spherical CNNs. ICLR. 2018. Available online: http://xxx.lanl.gov/abs/1801.10130 (accessed on 21 December 2020).
- Cohen, T.; Weiler, M.; Kicanaoglu, B.; Welling, M. Gauge equivariant convolutional networks and the icosahedral CNN. In Proceedings of the 36th International Conference on Machine Learning, (ICML 2019), Long Beach, CA, USA, 18–23 January 2019; pp. 1321–1330. [Google Scholar]
- Cohen, T.; Welling, M. Group equivariant convolutional networks. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 20–22 June 2016; pp. 2990–2999. [Google Scholar]
- Sifre, L.; Mallat, S. Rotation, scaling and deformation invariant scattering for texture discrimination. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Portland, Oregon, 23–28 June 2013; pp. 1233–1240. [Google Scholar] [CrossRef][Green Version]
- Marcos, D.; Volpi, M.; Komodakis, N.; Tuia, D. Rotation equivariant vector field networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; Volume 2017, pp. 5058–5067. [Google Scholar] [CrossRef][Green Version]
- Zhou, Y.; Ye, Q.; Qiu, Q.; Jiao, J. Oriented response networks. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017; Volume 2017, pp. 4961–4970. [Google Scholar] [CrossRef][Green Version]
- Shin, C.; Yun, J. Deep rotating kernel convolution neural network. In Proceedings of the 2019 Third IEEE International Conference on Robotic Computing (IRC), Naples, Italy, 25–27 February 2019; pp. 441–442. [Google Scholar]
- Gao, L.; Li, H.; Lu, Z.; Lin, G. Rotation-equivariant convolutional neural network ensembles in image processing. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, London, UK, 9–10 September 2019; Association for Computing Machinery: New York, NY, USA, 2019. [Google Scholar]
- Follmann, P.; Bottger, T. A rotationally-invariant convolution module by feature map back-rotation. In Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018, Lake Tahoe, NV, USA, 12–15 March 2018; Volume 2018, pp. 784–792. [Google Scholar] [CrossRef]
- Weiler, M.; Hamprecht, F.A.; Storath, M. Learning steerable filters for rotation equivariant CNNs. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 849–858. [Google Scholar] [CrossRef][Green Version]
- Luan, S.; Chen, C.; Zhang, B.; Han, J.; Liu, J. Gabor convolutional networks. IEEE Trans. Image Process.
**2018**, 27, 4357–4366. [Google Scholar] [CrossRef] [PubMed][Green Version] - Oyallon, E.; Mallat, S. Deep roto-translation scattering for object classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 2865–2873. [Google Scholar]
- Rodriguez Salas, R.; Dokladalova, E.; Dokladal, P. Rotation invariant CNN using scattering transform for image classification. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–25 September 2019; pp. 654–658. [Google Scholar] [CrossRef][Green Version]
- Shang, Y. Subgraph robustness of complex networks under attacks. IEEE Trans. Syst. Man Cybern. Syst.
**2017**, 49, 821–832. [Google Scholar] [CrossRef] - Mallat, S. A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way, 3rd ed.; Academic Press, Inc.: Cambridge, MA, USA, 2008. [Google Scholar]
- Rodriguez Salas, R.; Dokládal, P.; Dokladalova, E. RED-NN: Rotation-Equivariant Deep Neural Network for Classification and Prediction of Rotation. 2019. Available online: https://hal.archives-ouvertes.fr/hal-02170933 (accessed on 1 January 2021).
- Szeliski, R. Computer Vision: Algorithms and Applications; Springer Science & Business Media: Berlin, Germany, 2010. [Google Scholar]
- Petkov, N. Biologically motivated computationally intensive approaches to image pattern recognition. Future Gener. Comput. Syst.
**1995**, 11, 451–465. [Google Scholar] [CrossRef][Green Version] - Seto, K.; Nejatollahi, H.; An, J.; Kang, S.; Dutt, N. Small memory footprint neural network accelerators. In Proceedings of the 20th International Symposium on Quality Electronic Design (ISQED), Santa Clara, CA, USA, 6–7 March 2019; pp. 253–258. [Google Scholar] [CrossRef]

**Figure 1.**Rotational Invariant CNN architecture. Input is class number 4 rotated 90° clockwise. ($N=8$, $K=10$ convolutions in (

**c**,

**e**) scan each place with shared weights).

**Figure 2.**Oriented mapping. First, the image is decomposed by a set of N oriented components. Second, the result is re-oriented to the reference. This becomes a roto-translational feature stage.

**Figure 3.**Oriented representation mapping to obtain roto-translational feature space. Left side: Up-right oriented input (

**a**), and the oriented components (

**b**) generate a translating output product of the rotation compensation (

**c**). Right side: Rotated input (

**d**) and the oriented components (

**e**) with output translated by one space over axis N (

**f**).

**Figure 5.**The backbone feature extraction is applied to each filter of the oriented feature space. ResNet used as backbone example. Output is a translating feature space. (Image blocks with $N=8$).

**Figure 6.**Accuracy values forthe proposed architecture on CIFAR-10 (training: up-right/validation: rotated).

Method | Error Rate | # Parameters |
---|---|---|

ORN-8 (ORPooling) [20] | 16.67% | 397 k |

ORN-8 (ORAlign) [20] | 16.24% | 969 k |

RotInv Conv. (RP_RF_1) [23] | 19.85% | 130 k |

RotInv Conv. (RP_RF_1_32) [23] | 12.20% | 1 M |

RotDCF (60 degrees) [22] | 17.64% | 760 k |

Spherical CNN [15] | 6.00% | 68 k |

Icosahedral CNN [16] | 30.01% | n.c. |

RI-LBCNNs [21] | 25.77% | 390 k |

Covariant CNN [27] | 17.21% | 7 k |

RIN (Steerable + Convolutional) (this paper) | 2.05% | 42 k |

RIN (Gabor + Convolutional) (this paper) | 1.71% | 9 k |

Method | Error Rate | # Parameters |
---|---|---|

RotInv Conv. (RP_RF_1) [23] | 55.88% | 130 k |

ORN [20] | 59.31% | 382 k |

RP_1234 [17] | 62.55% | 130 k |

RIN (Gabor + Convolutional) (this paper) | 37.60% | 93 k |

RIN (Gabor + Convolutional) (this paper) | 28.69% | 238 k |

RIN (Gabor + Convolutional) (this paper) | 33.25% | 586 k |

RIN (Gabor + ResNet8) (this paper) | 27.68% | 243 k |

RIN (Gabor + ResNet14) (this paper) | 27.34% | 341 k |

RIN (Gabor + ResNet20) (this paper) | 21.10% | 438 k |

RIN (Gabor + ResNet26) (this paper) | 21.50% | 537 k |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rodriguez Salas, R.; Dokladal, P.; Dokladalova, E.
Rotation Invariant Networks for Image Classification for HPC and Embedded Systems. *Electronics* **2021**, *10*, 139.
https://doi.org/10.3390/electronics10020139

**AMA Style**

Rodriguez Salas R, Dokladal P, Dokladalova E.
Rotation Invariant Networks for Image Classification for HPC and Embedded Systems. *Electronics*. 2021; 10(2):139.
https://doi.org/10.3390/electronics10020139

**Chicago/Turabian Style**

Rodriguez Salas, Rosemberg, Petr Dokladal, and Eva Dokladalova.
2021. "Rotation Invariant Networks for Image Classification for HPC and Embedded Systems" *Electronics* 10, no. 2: 139.
https://doi.org/10.3390/electronics10020139