# A Novel Approach for Classification and Forecasting of Time Series in Particle Accelerators

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Dataset and Preprocessing

#### 2.2. Problem Formulation

#### 2.3. The RPCNN Model

#### 2.3.1. Preprocessing

Algorithm 1: Random feature selection |

#### 2.3.2. Implementation of the RPCNN Model

#### (a) RP generation

#### (b) Feature extraction

#### (c) Classification

#### 2.4. The RF Model

#### 2.5. Evaluation Metric

#### 2.5.1. Receiver Operating Curve

#### 2.5.2. Beam Time Saved

## 3. Results

#### 3.1. Model Performance in Terms of Evaluation Metrics

#### 3.2. Model Performance in a Simulated Live Setting

## 4. Discussion

#### 4.1. Achievements

#### 4.2. Limitations

#### 4.3. Outlook

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Interlock Statistics

**Figure A1.**Interlock occurrences per day from 18-9-2019 to 25-12-2019. Regular maintenance periods can be observed.

## Appendix B. Example Recurrence Plots

## Appendix C. Architecture of the RPCNN Model

## References

- Edelen, A.; Mayes, C.; Bowring, D.; Ratner, D.; Adelmann, A.; Ischebeck, R.; Snuverink, J.; Agapov, I.; Kammering, R.; Edelen, J.; et al. Opportunities in machine learning for particle accelerators. arXiv
**2018**, arXiv:1811.03172. [Google Scholar] - Edelen, A.L.; Biedron, S.; Chase, B.; Edstrom, D.; Milton, S.; Stabile, P. Neural networks for modeling and control of particle accelerators. IEEE Trans. Nucl. Sci.
**2016**, 63, 878–897. [Google Scholar] [CrossRef] [Green Version] - Arpaia, P.; Azzopardi, G.; Blanc, F.; Bregliozzi, G.; Buffat, X.; Coyle, L.; Fol, E.; Giordano, F.; Giovannozzi, M.; Pieloni, T.; et al. Machine learning for beam dynamics studies at the CERN Large Hadron Collider. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2020**, 985, 164652. [Google Scholar] [CrossRef] - Zhao, W.; Patil, I.; Han, B.; Yang, Y.; Xing, L.; Schüler, E. Beam data modeling of linear accelerators (linacs) through machine learning and its potential applications in fast and robust linac commissioning and quality assurance. Radiother. Oncol.
**2020**, 153, 122–129. [Google Scholar] [CrossRef] [PubMed] - Adelmann, A. On nonintrusive uncertainty quantification and surrogate model construction in particle accelerator modeling. SIAM/ASA J. Uncertain. Quantif.
**2019**, 7, 383–416. [Google Scholar] [CrossRef] - Kirschner, J.; Nonnenmacher, M.; Mutnỳ, M.; Krause, A.; Hiller, N.; Ischebeck, R.; Adelmann, A. Bayesian Optimisation for Fast and Safe Parameter Tuning of SwissFEL. In Proceedings of the 39th International Free-Electron Laser Conference, FEL2019, Hamburg, Germany, 26–30 August 2019; JACoW Publishing: Geneva, Switzerland, 2019; pp. 707–710. [Google Scholar]
- Fol, E.; Coello de Portugal, J.; Franchetti, G.; Tomás, R. Optics corrections using Machine Learning in the LHC. In Proceedings of the 2019 International Particle Accelerator Conference, Melbourne, VIC, Australia, 19–24 May 2019. [Google Scholar]
- Azzopardi, G.; Muscat, A.; Redaelli, S.; Salvachua, B.; Valentino, G. Operational Results of LHC Collimator Alignment Using Machine Learning. In Proceedings of the 10th International Particle Accelerator Conference (IPAC’19), Melbourne, VIC, Australia, 19–24 May 2019; JACoW Publishing: Geneva, Switzerland, 2019; pp. 1208–1211. [Google Scholar] [CrossRef]
- Leemann, S.; Liu, S.; Hexemer, A.; Marcus, M.; Melton, C.; Nishimura, H.; Sun, C. Demonstration of Machine Learning-Based Model-Independent Stabilization of Source Properties in Synchrotron Light Sources. Phys. Rev. Lett.
**2019**, 123, 194801. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Fol, E.; Coello de Portugal, J.; Franchetti, G.; Tomás, R. Application of Machine Learning to Beam Diagnostics. In Proceedings of the FEL’19, Hamburg, Germany, 26–30 August 2019; JACoW Publishing: Geneva, Switzerland, 2019; pp. 311–317. [Google Scholar] [CrossRef]
- Tilaro, F.; Bradu, B.; Gonzalez-Berges, M.; Roshchin, M.; Varela, F. Model Learning Algorithms for Anomaly Detection in CERN Control Systems. In Proceedings of the 16th International Conference on Accelerator and Large Experimental Control Systems (ICALEPCS’17), Barcelona, Spain, 8–13 October 2017; JACOW: Geneva, Switzerland, 2018; pp. 265–271. [Google Scholar]
- Piekarski, M.; Jaworek-Korjakowska, J.; Wawrzyniak, A.I.; Gorgon, M. Convolutional neural network architecture for beam instabilities identification in Synchrotron Radiation Systems as an anomaly detection problem. Measurement
**2020**, 165, 108116. [Google Scholar] [CrossRef] - Donon, Y.; Kupriyanov, A.; Kirsh, D.; Di Meglio, A.; Paringer, R.; Serafimovich, P.; Syomic, S. Anomaly Detection and Breakdown Prediction in RF Power Source Output: A Review of Approaches. In Proceedings of the NEC 2019 27th Symposium on Nuclear Electronics and Computing, Budva, Montenegro, 30 September–4 October 2019. [Google Scholar]
- Donon, Y.; Kupriyanov, A.; Kirsh, D.; Di Meglio, A.; Paringer, R.; Rytsarev, I.; Serafimovich, P.; Syomic, S. Extended anomaly detection and breakdown prediction in LINAC 4’s RF power source output. In Proceedings of the 2020 International Conference on Information Technology and Nanotechnology (ITNT), Samara, Russia, 26–29 May 2020; IEEE: New York, NY, USA, 2020; pp. 1–7. [Google Scholar]
- Rescic, M.; Seviour, R.; Blokland, W. Predicting particle accelerator failures using binary classifiers. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip.
**2020**, 955, 163240. [Google Scholar] [CrossRef] - Hashemian, H.M. State-of-the-art predictive maintenance techniques. IEEE Trans. Instrum. Meas.
**2010**, 60, 226–236. [Google Scholar] [CrossRef] - Selcuk, S. Predictive maintenance, its implementation and latest trends. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf.
**2017**, 231, 1670–1679. [Google Scholar] [CrossRef] - Scheffer, C.; Girdhar, P. Practical Machinery Vibration Analysis and Predictive Maintenance; Elsevier: Amsterdam, The Netherlands, 2004. [Google Scholar]
- Kovalev, D.; Shanin, I.; Stupnikov, S.; Zakharov, V. Data mining methods and techniques for fault detection and predictive maintenance in housing and utility infrastructure. In Proceedings of the 2018 International Conference on Engineering Technologies and Computer Science (EnT), Moscow, Russia, 20–21 March 2018; IEEE: New York, NY, USA, 2018; pp. 47–52. [Google Scholar]
- Reggiani, D.; Blau, B.; Dölling, R.; Duperrex, P.A.; Kiselev, D.; Talanov, V.; Welte, J.; Wohlmuther, M. Improving beam simulations as well as machine and target protection in the SINQ beam line at PSI-HIPA. J. Neutron Res.
**2020**, 22, 1–11. [Google Scholar] [CrossRef] - Dalesio, L.R.; Kozubal, A.; Kraimer, M. EPICS Architecture; Technical Report; Los Alamos National Lab.: Los Alamos, NM, USA, 1991. [Google Scholar]
- Lutz, H.; Anicic, D. Database driven control system configuration for the PSI proton accelerator facilities. In Proceedings of the ICALEPCS’11, Grenoble, France, 10–15 October 2011; JACoW Publishing: Geneva, Switzerland, 2011; pp. 289–291. [Google Scholar]
- Ebner, S. Data API for PSI SwissFEL DataBuffer and EPICS Archiver. Available online: https://github.com/paulscherrerinstitute/data_api_python (accessed on 10 March 2021).
- Chollet, F. Keras. Available online: https://keras.io (accessed on 10 March 2021).
- Eckmann, J.P.; Kamphorst, S.O.; Ruelle, D. Recurrence Plots of Dynamical Systems. EPL Europhys. Lett.
**1987**, 4, 973. [Google Scholar] [CrossRef] [Green Version] - Zbilut, J.P.; Koebbe, M.; Loeb, H.; Mayer-Kress, G. Use of recurrence plots in the analysis of heart beat intervals. In Proceedings of the Computers in Cardiology, Chicago, IL, USA, 23–26 September 1990; IEEE: New York, NY, USA, 1990; pp. 263–266. [Google Scholar]
- Webber, C.L., Jr.; Zbilut, J.P. Recurrence quantification analysis of nonlinear dynamical systems. Tutor. Contemp. Nonlinear Methods Behav. Sci.
**2005**, 94, 26–94. [Google Scholar] - Recurrence Plots and Cross Recurrence Plots. Available online: www.recurrence-plot.tk (accessed on 15 January 2021).
- Marwan, N.; Romano, M.C.; Thiel, M.; Kurths, J. Recurrence plots for the analysis of complex systems. Phys. Rep.
**2007**, 438, 237–329. [Google Scholar] [CrossRef] - Webber, C.L., Jr.; Marwan, N. Recurrence Quantification Analysis: Theory and Best Practices; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
- Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258. [Google Scholar]
- Ho, T.K. Random decision forests. In Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, QC, Canada, 14–16 August 1995; IEEE: New York, NY, USA, 1995; Volume 1, pp. 278–282. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res.
**2011**, 12, 2825–2830. [Google Scholar] - Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett.
**2006**, 27, 861–874. [Google Scholar] [CrossRef] - Grillenberger, J.; Humbel, J.M.; Mezger, A.; Seidel, M.; Tron, W. Status and Further Development of the PSI High Intensity Proton Facility. In Proceedings of the 20th International Conferenceon Cyclotrons and Their Applications (Cyclotrons’13), Vancouver, BC, Canada, 16–20 September 2013; pp. 37–39. [Google Scholar]

**Figure 3.**Distribution of the interlock events by type. “Other Types” denotes all interlock causes that are not prevalent enough to have a separate category. Please note that an interlock may be labeled as more than one type.

**Figure 5.**Definition of interlock windows (orange) and stable windows (green). 5 overlapping interlock windows are cut at 1 s before the interlocks, and non-overlapping stable windows are cut in between two interlocks with a 10-min buffer region. Each window has a length of 12.8 s (64 timesteps).

**Figure 8.**Examples of recurrence plots generated from the RPCNN model. From left to right: uncorrelated stochastic data, data starting to grow, and stochastic data with a linear trend. The top row shows the signals before the recurrence plot generation, and the bottom row shows the corresponding recurrence plots.

**Figure 9.**Illustration of section (a) RP generation of the RPCNN model (c.f. Figure 6).

**Figure 10.**Expected lost beam time without (

**top**) and with (

**bottom**) the proposed 10% current reduction. An interlock equivalently leads to 25 s of lost beam time, i.e., $-25$ s of beam time saved. With the current reduction, the interlock is expected to be avoided with a cost of six s of lost beam time, i.e., $-6$ s of beam time saved.

**Figure 11.**ROC curves of the best-performing models in terms of Beam time saved and AUC: (

**a**) RF; (

**b**) RPCNN. The mean and 95% confidence interval is calculated from re-sampling of the validation sets. (

**c**) Varying initialization of the same RPCNN model as in (

**b**). The mean and 95% confidence interval are calculated from the validation results of 25 model instances.

**Figure 12.**Screenshots from simulated live predictions over the validation data. The blue line is the prediction value from the model, with higher values indicating higher probability that interlocks are approaching. The orange line is the readout of the beam current channel “MHC1:IST:2” where a drop to zero indicates an interlock. The green line is the binary classification threshold, here taking $0.65$ for the RPCNN and $0.7$ for the RF. (

**a**,

**b**) show two examples of successful prediction of interlocks by RPCNN compared to the result of RF on the same time periods. The positive predictions closer to interlocks are regarded as TP marked by green circles, and the earliest times that the model output cross the threshold are enclosed in purple circles. (

**c**) shows the FPs from the RPCNN in red circles, compared to the result of RF. (

**d**) shows an example of FN from the RPCNN compared to the result of RF. A clear trend increasing with time is present in the output of RPCNN, shown by the red arrow.

**Figure 13.**Reverse cumulative histogram of the first instance the alarm threshold was crossed before an interlock event and the model prediction values remained above the threshold. From evaluation of the ROC curve, a threshold value of $0.65$ is chosen. Displayed are the results for the best-performing RPCNN model. Please note that the interlock event is at 0 s, i.e., time runs from right to left.

Channel | Section | Description (Unit) |
---|---|---|

AHA:IST:2 | PKANAL | Bending magnet ($\mathrm{A}$) |

CR1IN:IST:2 | RING | Voltage of Ring RF Cavity 1 ($\mathrm{kVp}$) |

MHC1:IST:2 | PKANAL | Beam current measurement at the ring exit ($\mu \mathrm{A}$) |

Type | Description |
---|---|

Electrostatic | Interlock related to electrostatic elements |

Transmission | Interlock related to transmission through target |

Losses | Interlock related to beam losses |

Other type | Interlock related to another type or unknown |

**Table 3.**Numbers of samples in the training and validation sets, before and after bootstrapping, where interlock samples are randomly drawn to have the same number as stable samples.

# Interlock Events | # Interlock Samples | # Stable Samples | # Interlock Samples after Bootstrapping | |
---|---|---|---|---|

Training set | 731 | 3655 | 176,046 | 176,046 |

Validation set | 163 | 815 | 44,110 | not applied |

Parameter | Value |
---|---|

Learning rate | 0.001 |

Batch size | 512 |

Optimizer | Adam |

Window size | 12.8 s |

Number of channels | 97 |

Std of the Gaussian noise | $0.3$ |

Dropout ratio | $0.1$ |

Parameter | Value |
---|---|

Number of estimators | 90 |

Maximum depth | 9 |

Number of channels | 376 |

Maximum leaf nodes | 70 |

Criterion | Gini |

**Table 6.**Detail of beam time saved ${T}_{s}$ according to different classification result and action for one sample.

${\mathit{T}}_{\mathit{s}}$ per TP(s) | ${\mathit{T}}_{\mathit{s}}$ per FN (s) | ${\mathit{T}}_{\mathit{s}}$ per FP(s) | ${\mathit{T}}_{\mathit{s}}$ per TN(s) | |
---|---|---|---|---|

Without current reduction | −25 | −25 | 0 | 0 |

With current reduction | −6 | −25 | −6 | 0 |

Incentive | 19 | 0 | −6 | 0 |

**Table 7.**Beam time saved (${T}_{s}$) and AUC results of the best-performing RF and RPCNN models over re-sampling of the validation sets.

Model | ${\mathit{T}}_{\mathit{s}}$ [s/interlock] | AUC |
---|---|---|

RF best | $0.7\pm 0.1$ | $0.65\pm 0.01$ |

RPCNN best | $0.5\pm 0.2$ | $0.71\pm 0.01$ |

RPCNN mean over initialization | $0.1\pm 0.1$ | $0.61\pm 0.04$ |

**Table 8.**Classification results in terms of True Positive (TP), False Positive (FP), True Negative (FN), False Negative (FN), True Positive Rate (TPR) and True Negative Rate (TNR) of the best-performing RF and RPCNN models. The thresholds for both models were obtained from the ROC curves so that the beam time saved is maximized, with $0.70$ for the RF and $0.65$ for the RPCNN.

Model | TP | FP | TN | FN | TPR or Recall | TNR or Specificity |
---|---|---|---|---|---|---|

RF best | 25 | 28 | $\mathrm{44,082}$ | 790 | $3.1\%$ | $99.9\%$ |

RPCNN best | 40 | 75 | $\mathrm{44,035}$ | 775 | $4.9\%$ | $99.8\%$ |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Li, S.; Zacharias, M.; Snuverink, J.; Coello de Portugal, J.; Perez-Cruz, F.; Reggiani, D.; Adelmann, A.
A Novel Approach for Classification and Forecasting of Time Series in Particle Accelerators. *Information* **2021**, *12*, 121.
https://doi.org/10.3390/info12030121

**AMA Style**

Li S, Zacharias M, Snuverink J, Coello de Portugal J, Perez-Cruz F, Reggiani D, Adelmann A.
A Novel Approach for Classification and Forecasting of Time Series in Particle Accelerators. *Information*. 2021; 12(3):121.
https://doi.org/10.3390/info12030121

**Chicago/Turabian Style**

Li, Sichen, Mélissa Zacharias, Jochem Snuverink, Jaime Coello de Portugal, Fernando Perez-Cruz, Davide Reggiani, and Andreas Adelmann.
2021. "A Novel Approach for Classification and Forecasting of Time Series in Particle Accelerators" *Information* 12, no. 3: 121.
https://doi.org/10.3390/info12030121