# Machine Learning Techniques for Calorimetry

## Abstract

**:**

## 1. Introduction

- 1.
- GNNs can be applied on the data from complex detector geometries.
- 2.
- They are easily applied to sparse data with variable input sizes.
- 3.
- GNNs can be applied on non-Euclidean data (unlike convolutional neural networks).
- 4.
- In GNNs, the information can flow between close-by nodes of the graph.

## 2. $\mathbf{e}/\mathbf{\gamma}$ Reconstruction

#### 2.1. ECAL

- The barrel with crystal size: $2.2\phantom{\rule{3.33333pt}{0ex}}\times \phantom{\rule{3.33333pt}{0ex}}2.2\phantom{\rule{3.33333pt}{0ex}}\times \phantom{\rule{3.33333pt}{0ex}}23\phantom{\rule{3.33333pt}{0ex}}\mathrm{cm}$, covering pseudorapidity $\left|\eta \right|<1.479$.
- The endcaps with crystal size: $2.9\phantom{\rule{3.33333pt}{0ex}}\times \phantom{\rule{3.33333pt}{0ex}}2.9\phantom{\rule{3.33333pt}{0ex}}\times \phantom{\rule{3.33333pt}{0ex}}22\phantom{\rule{3.33333pt}{0ex}}\mathrm{cm}$, covering pseudorapidity $1.479<\left|\eta \right|<3.0$.

#### 2.2. Reconstruction in the ECAL

- Energy lost before reaching the ECAL, and in detector gaps.
- Energy leakage out of the back of the ECAL.
- The use of finite energy thresholds to suppress noise in the detector electronics.
- Energy deposited by the multiple additional interactions, so-called pileup interactions.

## 3. SuperClustering

#### 3.1. DeepSC Model

- Dense layers are used to extract the vectors of the latent features.
- Graph Convolutional Network/Graph Highway Network (GHN) [14], where the information can be shared and aggregated between the close-by clusters. The two algorithms are very similar to each other, with GHN being more robust to over-smoothing during the training.

#### 3.2. Dataset Description

#### 3.3. Results

#### 3.3.1. Energy Resolution

#### 3.3.2. Particle Identification

## 4. Energy Regression

#### 4.1. The Dynamic Reduction Network

- 1.
- The position and energy coordinates of each RecHit are mapped into a high-dimensional latent space using a fully-connected neural network.
- 2.
- The message-passing process is performed to aggregate the information between the neighbors and learn the global information.
- 3.
- Additional human-engineered features are added to the learned features that were not encoded in the initial hit collection. In particular, two additional features that describe the amount of energy leakage at the back of the ECAL and the energy density from pileup events, are concatenated to the learned features.
- 4.
- The resulting set of high-level features is passed through another fully connected neural network to produce the regression output.

#### 4.2. Training

#### 4.3. Results

## 5. Conclusions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- The CMS Collaboration. The CMS experiment at the CERN LHC. J. Instrum.
**2008**, 3, S08004. [Google Scholar] - The CMS Collaboration. Measurements of Higgs boson production cross sections and couplings in the diphoton decay channel at $\sqrt{\mathit{s}}$= 13 TeV. J. High Energy Physic.
**2021**, 2021, 27. [Google Scholar] [CrossRef] - The CMS Collaboration. Search for nonresonant Higgs boson pair production in final state with two bottom quarks and two tau leptons in proton-proton collisions at $\sqrt{\mathit{s}}$ = 13 TeV. arXiv
**2021**, arXiv:2011.12373. [Google Scholar] - Stoye, M.; on Behalf of the CMS Collaboration. Deep learning in jet reconstruction at CMS. J. Phys. Conf. Ser.
**2018**, 1085, 042029. [Google Scholar] [CrossRef] - Pata, J.; Duarte, J.; Mokhtar, F.; Wulff, E.; Yoo, J.; Vlimant, J.R.; Pierini, M.; Girone, M. Machine Learning for Particle Flow Reconstruction at CMS. arXiv
**2022**, arXiv:2203.00330. Available online: https://cds.cern.ch/record/2802826 (accessed on 20 July 2022). - Shlomi, J.; Battaglia, P.; Vlimant, J.R. Graph Neural Networks in Particle Physics. Mach. Learn. Sci. Technol.
**2021**, 2, 021001. [Google Scholar] [CrossRef] - Qasim, S.R.; Kieseler, J.; Iiyama, Y.; Pierini, M. Learning representations of irregular particle-detector geometry with distance-weighted graph networks. Eur. Phys. J. C
**2019**, 79, 608. [Google Scholar] [CrossRef] - Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z.; Wang, L.; Li, C.; Sun, M. Graph Neural Networks: A Review of Methods and Applications. arXiv
**2018**, arXiv:1812.08434. [Google Scholar] [CrossRef] - The CMS Collaboration. The CMS Electromagnetic Calorimeter Project: Technical Design Report. 1997. Available online: http://cds.cern.ch/record/349375 (accessed on 20 July 2022).
- The CMS Collaboration. Electron and photon reconstruction and identification with the CMS experiment at the CERN LHC. J. Instrum.
**2021**, 16, P05014. [Google Scholar] [CrossRef] - Valsecchi, D. Deep Learning Techniques for Energy Clustering in the CMS ECAL. 2022. Available online: https://cds.cern.ch/record/2803235 (accessed on 20 July 2022).
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need. arXiv
**2017**, arXiv:1706.03762. [Google Scholar] - Niu, Z.; Zhong, G.; Yu, H. A Review on the Attention Mechanism of Deep Learning. 2021. Available online: https://www.sciencedirect.com/science/article/pii/S092523122100477X (accessed on 30 July 2022).
- Xin, X.; Karatzoglou, A.; Arapakis, I.; Jose, J.M. Graph Highway Networks. arXiv
**2020**, arXiv:2004.04635. [Google Scholar] - Khachatryan, V.; Sirunyan, A.M.; Tumasyan, A.; Adam, W.; Asilar, E.; Bergauer, T.; Brandstetter, J.; Brondolin, E.; Dragicevic, M.; Ero, J.; et al. Jet energy scale and resolution in the CMS experiment in pp collisions at 8 TeV. J. Instrum.
**2017**, 12, P02014. [Google Scholar] [CrossRef] - Zhuang, F.; Qi, Z.; Duan, K.; Xi, D.; Zhu, Y.; Zhu, H.; Xiong, H.; He, Q. A Comprehensive Survey on Transfer Learning. Proc. IEEE
**2021**, 109, 43–76. [Google Scholar] [CrossRef] - The CMS Collaboration. Particle-flow reconstruction and global event description with the CMS detector. J. Instrum.
**2017**, 12, P10003. [Google Scholar] [CrossRef] - Gray, L.; Klijnsma, T.; Ghosh, S. A dynamic reduction network for point clouds. arXiv
**2020**, arXiv:2003.08013. [Google Scholar] - Rothman, S. Calibrating Electrons and Photons in the CMS ECAL Using Graph Neural Networks. 2021. Available online: https://cds.cern.ch/record/2799575 (accessed on 20 July 2022).
- del Amo Sanchez, P.; Lees, J.P.; Poireau, V.; Prencipe, E.; Tisser, V.; Tico, J.G.; Grauges, E.; Martinelli, M.; Palano, A.; Pappagallo, M.; et al. Study of B→Xγ decays and determination of |Vtd/Vts|. Phys. Rev. D
**2010**, 82, 051101. [Google Scholar] [CrossRef]

**Figure 1.**DeepSC model architecture. The input to the network consists of selected features and rechits of the clusters that fall into a predefined geometrical window. Using dense layers, the latent features are extracted from the initial input; they are processed and combined together using different types of graph architectures: Graph Convolutional Network (GCN) and Graph Highway Network (GHN). Self-attention layers are used as well, to help the network with focusing on the most important features/inputs. The final outputs are the following: information on whether each of the clusters belongs to the SuperCluster (cluster classification), the type of the particle from which the SuperCluster originated (window classification), and energy correction (energy calibration factor).

**Figure 2.**Energy resolution for DeepSC and Mustache algorithms. The resolution of the reconstructed uncorrected energy divided by the true energy deposits vs. generated transverse energy of the particle (

**left**) and the number of simulated pileup interactions (

**right**) is presented. The bottom panels show the ratio of the energy resolutions quantifying the improvement of the DeepSC model over the Mustache algorithm.

**Figure 3.**Model score distributions for particle identification. The jet score (

**left**) represents the likelihood of the SuperCluster to originate from a photon. Clear discrimination between jet and photon samples is visible for this case. The electron score (

**right**) represents the likelihood of the SuperCluster to originate from an electron. It demonstrates that despite the absence of the tracker information, some discrimination can be achieved between photon and electron samples.

**Figure 4.**Flowchart of the operation of the Dynamic Reduction Network. A point cloud of rechits is mapped into a high-dimensional latent space using a fully-connected neural network, where it is then iteratively transformed and pooled using graph operations. This resulting high-level learned features are then concatenated with extra high-level information not available from the raw collection of rechits, and passed through another fully-connected neural network to obtain the regression output.

**Figure 5.**Dynamic Reduction Network (DRN) and Boosted Decision Tree performance in the ECAL barrel (

**left**) and endcaps (

**right**) as a function of generated transverse momentum. The DRN shows an improved resolution by >10%.

**Figure 6.**Di-photon invariant mass distributions of $\mathrm{H}\to \gamma \gamma $ events for both the Dynamic Reduction Network (DRN) and Boosted Decision Tree architectures in the ECAL barrel (

**left**) and endcaps (

**right**). The DRN shows an improved resolution by >5% in both detector regions.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Simkina, P., on behalf of the CMS Collaboration. Machine Learning Techniques for Calorimetry. *Instruments* **2022**, *6*, 47.
https://doi.org/10.3390/instruments6040047

**AMA Style**

Simkina P on behalf of the CMS Collaboration. Machine Learning Techniques for Calorimetry. *Instruments*. 2022; 6(4):47.
https://doi.org/10.3390/instruments6040047

**Chicago/Turabian Style**

Simkina, Polina on behalf of the CMS Collaboration. 2022. "Machine Learning Techniques for Calorimetry" *Instruments* 6, no. 4: 47.
https://doi.org/10.3390/instruments6040047