Programming Quantum Neural Networks on NISQ Systems: An Overview of Technologies and Methodologies

Noisy Intermediate-Scale Quantum (NISQ) systems and associated programming interfaces make it possible to explore and investigate the design and development of quantum computing techniques for Machine Learning (ML) applications. Among the most recent quantum ML approaches, Quantum Neural Networks (QNN) emerged as an important tool for data analysis. With the QNN advent, higher-level programming interfaces for QNN have been developed. In this paper, we survey the current state-of-the-art high-level programming approaches for QNN development. We discuss target architectures, critical QNN algorithmic components, such as the hybrid workflow of Quantum Annealers and Parametrized Quantum Circuits, QNN architectures, optimizers, gradient calculations, and applications. Finally, we overview the existing programming QNN frameworks, their software architecture, and associated quantum simulators.


Introduction
Quantum computing is emerging as a disruptive and promising approach to attacking computational and data analysis problems. Quantum computing relies on three essential quantum effects inaccessible directly by classical computing systems [1,2]: (i) calculation on a superposition of quantum states somehow reminiscent of parallel computing, (ii) entanglement to correlate different quantum states, and (iii) quantum tunneling. These three effects can be used to seek the so-called quantum advantage [3] over classical algorithms by, for instance, computing in a superposition or hopping between optimization landscapes via quantum tunneling. The first critical quantum computing applications with quantum advantage are in the area of cryptology and search algorithms with the most famous Shor's and Grover's algorithms. Today, researchers' attention started focusing on the possibility of developing quantum Machine Learning (ML) applications [4,5] for classical and quantum data, e.g., data encoded as a superposition of quantum states, resulting from quantum simulations or sensing.
The early quantum ML approaches rely on the so-called quantum Basic Linear Algebra Subprograms (qBLAS) primitives [4]. Examples of qBLAS routines are the Quantum Fourier Transform (QFT), Quantum Phase Estimation (QPE) for obtaining eigenstates and eigenphases, and the Harrow-Hassidim-Lloyd (HHL) algorithm for solving linear systems [6]. These qBLAS-based ML methods consist of classical ML approaches, such as the quantum Principal Component Analysis (PCA) [7], quantum regression with leastsquare fitting [8], quantum topological analysis [9], quantum Bayesian inference [10], and quantum Support Vector Machine (SVD) [11]. While these quantum ML methods exhibit a clear quantum advantage concerning corresponding classical algorithms, severe constraints, such as embedding classical data into quantum states, the need for quantum memory, technologies in use, such as QNN building blocks, optimizers, and automatic differentiation techniques.

Target NISQ Architectures for QNN
At a high level, we can divide the QNN target quantum computer architectures into two broad categories: 1. Quantum Annealers (QA). In this quantum computing approach, the loss function is expressed as the cost function of a QUBO (Quadratic Unconstrained Binary Optimization) problem, equivalent to the Hamiltonian of an Ising system [22]. Currently, the most established QA machines are from the Canadian D-Wave. Additional companies working on and researching the development of QA platforms are Fujitsu, with its Digital Annealer [23,24], Toshiba, with its Simulated Bifurcation Machine (SBM) [25], NEC (developing a QA processor using the so-called Lechner-Hauke-Zoller architecture [26]), and Qilimanjaro Quantum Tech, a spinoff of the Barcelona Supercomputing Center. 2. Universal Gate Quantum Computers. In this quantum computing model, the QNN loss function is expressed in terms of a measurement associated with a parametrized quantum circuit using universal quantum gates. Differently from QAs, universal quantum computers can solve problems beyond optimization tasks, formulated as the minimization of an Ising Hamiltonian. There are two formulations for the universal quantum gates that can be used to express the QNN loss function: (a) Discrete Qubit-Based Quantum Computing. Qubit-based architectures are the most established general-purpose quantum computing approach. They use the discrete formulation of a quantum state equivalent to a bit [27]. The qubit |φ is expressed as the combination (or a superposition) of the states |0 and |1 as |φ = φ 0 |0 + φ 1 |1 . We use a set of discrete complex-valued coefficients, such as φ 0 and φ 1 , whose modulus squared corresponds to the probability of measuring |0 and |1 in the qubit system measurement. Discrete-qubit QNNs rely on parametrizing discrete quantum gates, such as rotation and Pauli gates. Discrete qubit-based QNNs are generally considered a good match for classification tasks because of the discrete nature of the problem. Among the most famous hardware implementations (and associated software) in this category, there are IBM (Qiskit), Google (Cirq), Rigetti (Forest), and OriginQ (Qpanda) quantum computers. All these implementations use superconducting/transmon qubit technologies. Another prominent company is Pasqal, with a neutral atom quantum computer that can be used in analog and digital versions [28]. (b) Continuous Variable (CV) Quantum Computing. The CV quantum computing approach is the analog version of quantum computing [29], still using a QC gate formulation [30]. CV is based on the concept of qumode, the continuous analogous of the qubit. The qumode |ψ is expressed in the basis expansion of quantum states, as |ψ = ψ(x) |x , where x are the real-valued eigenvalues and |x are the eigenstates of thex quadrature,x |x = x |x . CV quantum computing and CV QNN use continuous quantum gates, such as displacement, squeeze, rotation, and Kerr gates, to express the quantum circuit operations. Because of the continuous approach, CV QPC is regarded as an excellent fit for QNN regression-like tasks. In addition, CV QNNs are a critical building block for developing quantum Physics Informed Neural Networks (PINN) using CV gates [31]. The most established technology to implement CV quantum gates is photonics. The Canadian Xanadu is among the most active and established companies developing photonics quantum chips. Among others, Xanadu is one of the leading companies for the development of QNN programming frameworks: Strawberry Fields (and, most importantly, its integration with a TensorFlow backend) and PennyLane are important examples of programming frameworks that allow for CV QNNs.

Quantum Neural Network Input Data
QNNs can operate on two kinds of data: 1. Classical Data. In this case, the training datasets consist of classical data, such as the pixel values of an image. When QNN uses classical data, then an encoding of the classical data into quantum states is required. The most used encoding techniques are amplitude, angle, basis, and Hamiltonian encodings [5,32]. The encoding often requires the usage of an additional QNN layer. 2. Quantum Data and Integration with Quantum Simulators. Quantum data are encoded as a superposition of quantum states, where each quantum state has an associated amplitude and a phase. Quantum data cannot be generated classically but might result from quantum sensing or quantum circuit running a quantum algorithm or quantum simulations. An example of code using quantum data is the TensorFlow Quantum Hello Many-Worlds code [33] (https://github.com/tensorflow/quantum/ blob/research/binary_classifier/binary_classifier.ipynb, accessed on 3 April 2023) that classifies two classes of quantum data points distributed in the Bloch sphere [27]. Classical NN cannot operate on quantum data, and QNN provides the only mean to process quantum data directly. If the QNN uses quantum data, then a special data loader or integration with quantum simulations programming frameworks, such as OpenFermion [34], and PySCF [35] are required. All the main QNN frameworks provide integration of quantum simulations as part of the same package or integration with OpenFermion and PySCF.

Quantum Neural Network Approaches
This section discusses the two main algorithmic strategies for developing QNN on QAs and universal gate-based quantum computers.

QNN with Quantum Annealers
Historically, the first approach to tackling QNN development relies on using QAs, specialized quantum computers, on solving optimization problems [36,37]. In essence, QAs provide the ground state of a Hamiltonian of an Ising system (used, for instance, in magnetism problems and energy-based ML methods). If we formulate the QNN loss function as an Ising model, then finding the quantum system ground state corresponds to finding the loss function minimum. In the case of QA-based QNNs, the loss function can be expressed as: where J i,j are the QNN weights, h i , the QNN biases, and s i the spins (encoded in the qubit) that can take only the values +1 and −1. The QAs minimize the loss function of Equation (1), returning the weights and biases. To run on the quantum computer, Equation (1) must be first formulated in an equivalent QUBO matrix format: L = X T QX with x i = (1 − s i )/2 (the so-called spin to binary relation). Then, the loss function must be mapped to the underlying QA hardware and network topology through a process called graph embedding [38,39]. In the case of D-Wave systems, the embedding is into a Chimera graph. The workflow to run a QNN on QAs is represented in Figure 1. The QNN loss function is first formulated as a QUBO problem and then embedded into the underlying quantum computer topology graph. These steps are performed on the classical computer. The QAs calculate the loss function minimum (equivalent to the ground energy state of Ising Hamiltonian) and associated QNN weights and biases. A resampling phase allows for loss function minimum sampling several times. Because QA-based QNNs use Ising Hamiltonian in their formulation, they can straightforwardly represent energy-based NNs [40], such as Hopfield networks [41], Boltzmann machines [42], Restricted Boltzmann Machines (RBM) [43], and used as a part of the Deep Belief Network (DBN) model [44].

Ising Model / QUBO
Formulate the Loss Function with the Ising Model / QUBO Matrix.

Embedding
Embed the loss function into the Chimera Graph.

Quantum Annealer
Transition to the ground state

Resample
Sample the solution (the minimum of the loss function and corresponding weights and biases) several times and obtain a distribution.

QNN with Parametrized Quantum Circuits
The second QNN class can use universal quantum computers instead of QAs and goes under the name of Parametrized Quantum Circuit (PQC) [45], or Variational Quantum Circuits (VQC) [46,47]. The basic fundamental PQC idea is to express the weights and biases of the neural network as parameters of an exemplar quantum circuit (also called the Ansatz) and adapt the parameters to minimize a loss or cost function using a classical optimizer, such as Stochastic Gradient Descent (SGD) [48] or Adam [49] optimizers. Figure 2 shows the typical workflow when running a PQC. The first step randomly initializes the QNN weights w and biases b. These are parameters characterizing a gate in the PQC. For instance, the angle of a rotation gate can be a QNN parameter, e.g., a QNN weight. Then for each training sample, we first encode the input data (an image, for instance) into a quantum state using an encoding layer; we then execute the measure of the PQC results with current w and b (this corresponds to apply a unitary circuit U(w, b) to the encoded sample |0 as in U(w, b) |0 = |ψ(w, b) ). The norm of the difference between the measurement and training sample label will provide the loss function. For instance, a loss function is calculated using the PQC measurement and label data (y |0 ): Finally, similarly to NN, we can use the back-propagation step to update the QNN parameters. The loss function value drives an optimization step to determine new updated parameter values (w and b) to minimize the loss function. We repeat this process for each training sample. An essential point about PQC loss functions is that they are not limited to QUBO problems such as QA but are more general. In fact, it is possible to solve Ising problems using PQC.
A significant research effort is made to address the so-called barren plateau problem [54] for the QPC optimization landscape: in several PQCs, the average value of the gradient tends to zero, and as the Hilbert dimension increases, the more states will lead to a flat optimization landscape. For this reason, the optimizer cannot converge to the minimum of the loss function. To address this issue, a few techniques are proposed, including an initialization technique to initialize randomly only a subset of the parameters [55], using a local instead of a global loss function [56], and data re-uploading [57].

Quantum Device
Classical Device Encoding

Quantum Neural Network Architectures
In the case of PQC, it is possible to build QNNs by combining different layers in a similar way to the classical NN. The most common kinds of QNN layers are: • Encoding/Embedding Layers. These layers are used to encode classical data into quantum Hilbert space. Basically, the encoding process is equivalent to a feature map that assigns data to quantum states [58,59]. Inner products of such data-encoding quantum states give rise to quantum kernels. These feature maps are used in QNNs as a way to perform nonlinear transformations, akin to activation functions in NN, on the input data. Common feature maps used in the QNNs are amplitude, angle, basis, and Hamiltonian encodings. Amplitude and angle encodings map classical data to the amplitudes and phases of a quantum state, respectively. Basis embedding encodes the binary feature vector into a basis state. Hamiltonian encoding associates a system's Hamiltonian with a matrix representing the data transformation. An example of Hamiltonian encoding is using a quantum circuit with single-qubit rotations to encode the input data. This encoding using multiple quantum rotation gates, for instance, allows us to express quantum models as Fourier-type sums [60]. In CV QNNs, the most used encoding is displacement embedding, which encodes features into the displacement of qumodes amplitudes or phases. Encoding layers are critical for developing QNN as the data-encoding strategy largely defines the QNN expressivity, e.g., the features QNN can represent [59,61]. Feature maps are critical building blocks for developing scientific quantum machine learning and Differentiable Quantum Circuit (DQC) [62][63][64]. • Variational Layers. These layers are the PQC building block and include trainable parameters (w and b) in the quantum circuit. These parameters are optimized during the QNN training. They typically consist of a series of single-and two-qubit gates, with associated gate parameters optimized during training. • Entangling Layers. An important subclass of variational layers is the entangling layers class that creates entangled quantum states. These layers comprise one-parameter single-qubit rotations on each qubit, followed by a CNOT gate chain. Basic entangling layers have a CNOT gate chain connecting every qubit with its neighbor. Strongly entangling layers feature a CNOT gate chain also connecting non-neighbor qubits [65]. Random entangling layers have single qubit rotations and CNOT gates, acting on randomly chosen qubits. Another entangling layer is the so-called 2-design, consisting of qubit rotations and Controlled-Z (CZ gate) entangling layers [56]. • Pooling Layers. Pooling layers reduce the quantum circuit size by typically grouping together several qubits and performing operations that reduce the quantum state dimensionality. The way to implement pooling layers is to measure a qubit subset of the qubits and then use the measurement to control the following operations. Pooling layers are an important component of quantum convolutional networks [66]. • Measurement Layers. Measurement layers are used to measure classical information (bit) from the superposition of quantum states in the QNN. Measurements layers typically are single-qubit measurements of the output qubits that provide classical values for the QNN output.
In addition, the basic CV QNN layer consists of displacement, squeezing gates, interferometers to mimic the linear transformation of a neural network, and a Kerr gate to introduce nonlinearity to mimic the neural network activation function [30]. Figure 3 shows a few simple QNN examples used to construct the full PQC.

Entangling Layer
Pooling Layer Measurement Layer Encoding/Embedding Layer Variational Layer Figure 3. Examples of common quantum layers used for constructing QNNs: an encoding/embedding layer using a circuit block S(x) as Hamiltonian encoding, a variational layer with a unitary gate U with four parameters (θ 1 , θ 2 , θ 3 and θ 4 ), a simple entangling layer with rotation operation (R) and CNOT gates operating on neighbor qubits, a pooling layer used for quantum convolutional networks, and finally a measurement layer.
How to compose QNN layers automatically into PQC for solving a specific problem and minimizing the noise impact on real quantum machines is an active research area and led to the development of the SuperCircuit [67] and Supernet [68].

Optimizers for Parametrized Quantum Circuits
A key technology for training the PQC is the optimizer that allows us to find the minimum or maximum of a multi-variable function, e.g., the loss function in our case. The optimizers can be divided into two broad categories: • Gradient-free Optimizers. Gradient-free optimization methods are techniques that do not require the calculation of the gradient for the back-propagation step [69], reducing the complexity of performing differentiation on a quantum circuit. For this reason, they were widely used in developing the first QNNs. This optimizer class includes the Nelder-Mead [70] and COBYLA algorithms [71]. These gradient-free optimizer methods are often provided within the QNN frameworks (e.g., they are readily available in Qiskit) or available via external packages, such as SciPy [72]. • Gradient-based Optimizers. Gradient-based optimizers require gradient calculation on the QNN. Compared to gradient-free optimizers, gradient-based optimizers provide advantages from convergence guarantees [73] and are the method of choice in modern QNNs. Examples of gradient-based optimizers are the deeplearning workhorse algorithms, such as the Stochastic Gradient Descent (SGD) and Adam. These optimizers are readily available in many QNN frameworks or are obtained from integrating QNN programming frameworks with TensorFlow/Keras and PyTorch. For instance, Quantum TensorFlow and Strawberry Fields can readily use TensorFlow 2 and Keras optimizers. Together with traditional ML optimizers, additional optimizers are used to reduce evaluation costs and address the problem of the barren plateau. For instance, a popular optimizer, robust to noise, is the Simultaneous Perturbation Stochastic Approximation (SPSA) [74], which is a stochastic method to approximate the loss function gradient. In this optimizer, the loss function is evaluated using perturbed parameter vectors: each component of the parameter vector is shifted by a random value. Another example is the doubly stochastic gradient descent method [73] that reduces the cost of evaluating the gradient at each iteration by evaluating only a random subset of the gradient components. Additionally, the Quantum Natural Gradient (QNGOptimizer) [75,76] improves the quality of our optimization landscape (affected by the barren plateau problem) by moving along the steepest direction in the Hilbert space instead of the parameter space.

Differentiation for Parametrized Quantum Circuits
When using classical gradient-based optimizers, the optimization step relies on calculating the gradients of the loss function in the optimization landscape. In classical NN, derivatives on the neural network are calculated using the automatic differentiation technique [77]. The fact that the loss function is defined as a quantum circuit constitutes a challenge for this formulation. Some differentiation approaches [78,79] for PQC on quantum hardware and simulators are possible: • Parameter Shift Rule/Quantum Automatic Differentiation. This differentiation technique allows calculating derivatives using the same PQC with a difference only in a shift of the argument [80,81]. The basic idea of this technique is to consider these quantum functions as Fourier series. The partial derivative of a function can then be formulated as a linear combination of them. An intuitive example of the parameter shift rule workings (https://pennylane.ai/qml/glossary/parameter_shift.html, accessed on 3 April 2023) is the calculation of sin(x) that is equivalent to a shifted formulation: 1/2 sin(x + π/2) − 1/2 sin(x − π/2). The same underlying algorithm can be reused to compute both sin(x) and its derivative at ±π/2. This works for many PQCs of interest, and the same PQC can be used to evaluate both the loss function and its gradient on a quantum computer. • Numerical Derivative. Numerical derivative methods are based on finite-different discretization. This differentiation calculation can run on a quantum computer as a black box as it requires PQC evaluations common at two separated points in the parameter w at a distance ∆: f (w) = ( f (w + ∆) − f (w))/∆ in a simple case of forward finite-difference. The challenge with this technique is the number of PQC evaluations that this method requires and the accuracy (given the dependency on ∆). • Adjoint Derivative (for quantum simulators). This differentiation method applies only to quantum computer simulators, as the method requires examining and modifying the full quantum state vector. This method works iteratively by applying the inverse (adjoint) gate [82] and has significantly lower memory usage and a similar runtime than the backprop. For this reason, this is the method of choice for HPC implementation of automatic differentiation on quantum computer simulators. • Quantum analytic descent (on classical computers). This method constructs a classical model approximating the optimization landscape in the minimum proximity by using a sum of multilinear trigonometric terms in each parameter so that the gradients can be easily calculated on a classical computer that is computationally convenient [83].

Quantum Neural Network Software Frameworks
This section briefly reviews existing and emerging QNN programming frameworks. We note that new programming environments are continuously developed as new approaches and quantum computer systems arise. The list we present strives to cover the most used programming approaches, but it is necessarily not exhaustive.

Amazon Braket SDK
Amazon offers its quantum cloud, called Amazon Braket. Unlike many other vendors, Amazon does not develop quantum hardware; instead, it provides services over third-party quantum hardware [94] using superconducting, trapped ion, neutral-atom, and photonics technologies. Current quantum hardware providers within Amazon Braket include IonQ, Oxford Quantum Circuits (OQC), QuEra, Rigetti, and Xanadu.
QNNs can be programmed using the Amazon Braket Python SDK that provides means of connecting quantum computers and simulators and the basic programming abstractions for PQC programming. While Amazon Braket SDK does not offer a dedicated library for QNNs, it is possible to develop a PQC from scratch using Braket gates and measurement features (https://aws.amazon.com/blogs/quantum-computing/aioi-usingquantum-machine-learning-with-amazon-braket-to-create-a-binary-classifier/, accessed on 3 April 2023). Braket does not provide optimizers; however, it is possible to use the SciPy optimizers, such as the second-order L-BFGS [95]. Amazon Braket also provides a set of local and on-demand quantum computer simulators. The on-demand simulators can use distributed HPC systems and execute elastic Amazon Web Services (AWS) runs. Braket SDK simulators include state-vector, density matrix, and tensor-networks simulators. An important aspect of Amazon Braket is that it provides access to several other QNN programming frameworks, such as PennyLane and Qiskit.

D-Wave Ocean
D-Wave provides a software framework called Ocean SDK to connect and run quantum optimization problems on the D-Wave QA machines. As mentioned previously, QAs must first have the problem cast to a QUBO formulation and then embedded into the underlying qubit topology (a Chimera graph in the case of the D-Wave machines). To convert the Ising problem to a QUBO problem, the pyQUBO library [96] is typically used. The method EmbeddingComposite embeds the QUBO to the Chimera graph of the physical QA in D-Wave. After the problem is embedded in the QUBO form, it can be run calling the method sample_qubo(...,num_sample=...) providing the number of samples. Different samplers are provided in D-wave are provided: quantum, hybrid, and classical solvers, including simulated annealing, tabu (a heuristic that employs local search), among the others. At high-level, the D-Wave Ocean framework consists of these different software components: • Problem Definition. This software layer provides tools for defining optimization problems that can be solved using quantum annealing. It includes tools for defining variables, constraints, and objective functions. Utilities. This component provides a set of utility functions that can be used to analyze the results of the quantum annealing runs, visualize the embeddings, and debug the models.
OpenJIJ (https://github.com/OpenJij/OpenJij, accessed on 3 April 2023) is an opensource library that simulates the QAs and can be used to experiment without the D-Wave computers.

Intel HQCL
Intel has developed a Software Development Kit (SDK) called Intel Quantum SDK [97]. Currently, the Intel Quantum SDK supports only PQC simulations; however, it is expected to support real quantum hardware in the future. In particular, Intel is investing in quantum dot-based quantum computers. Future Intel Quantum SDK releases will include a quantum dot qubit simulator and an Intel quantum dot qubit device. The Intel Quantum SDK allows writing PQC based on C++ and an LLVM-based compiler toolchain that optimizes the quantum runtime for executing hybrid quantum-classical workloads [98]. The Intel quantum computer simulator is called IQS, for Intel Quantum Simulator.
Regarding PQC implementations, Intel provides the Hybrid Quantum-Classical Library (HQCL), a high-level library to express Hybrid Quantum-Classical algorithms exploiting Intel Quantum SDK and run on the quantum computer simulator [99].

Microsoft Azure QDK
Microsoft Azure Quantum provides access to quantum computers from several vendors, including IonQ (trapped-ion Technology), Honeywell (trapped-ion technology), Quantum Circuits Inc. (superconducting qubit technology), Rigetti (superconducting qubit technology), and Pasqal (neutral atom technology). Microsoft Azure Quantum allows for submitting provider-specific formatted quantum circuits (for instance, in QASM or JSON format) to supported quantum computing targets via the Azure Quantum services.
Microsoft also provides the Quantum Development Kit (QDK) that replaces the LIQUi|> programming environment [100] with a new programming language, called Q#. The QDK offers a library specifically for ML in Q# (https://learn.microsoft.com/enus/azure/quantum/user-guide/libraries/, accessed on 3 April 2023).
The QDK includes a back-end circuit simulator and front-end support for the Q# language, integrated with Microsoft Visual Studio.

Nvidia CUDA Quantum
Nvidia, one of the leading GPU producers, recently developed a unified programming model called CUDA Quantum, designed explicitly for running heterogeneous workloads-as the one for PQC-with CPUs, GPUs, and QPUs working side by side (https://developer. nvidia.com/cuda-quantum, accessed on 3 April 2023). CUDA Quantum intends to support quantum hardware backends from different quantum computer partners, including Rigetti, Xanadu, and Pasqal to name a few. CUDA Quantum provides a C++-based programming model, and it is specifically designed to enable interoperable workflows with existing classical parallel programming models and compiler toolchains, such as Nvidia CUDA. Regarding quantum simulation technologies, Nvidia provides the cuQuantum Appliance and the cuQuantum SDK to accelerate HPC simulators with Nvidia GPUs.
Early experiments with CUDA Quantum include the development of benchmarking a GPU-accelerated hybrid QGAN [101] with a quantum generator and a classical discriminator [102].

OriginQ QPanda
QPanda is a software stack developed by the Chinese Origin Quantum that has launched a 6-Qubit and 2-Qubit superconducting quantum chip accessible via the cloud.
QPanda provides both C++ and Python interfaces. Regarding PQC development, QPanda exploits the quantum machine learning VQNet library [103,104]. QPanda also provides several noiseless and adjustable simulation backends.

PennyLane
PennyLane is a Python library designed explicitly for differentiable computing, focusing on QNNs and quantum simulations. PennyLane is developed by Xanadu and is one of the best existing tools for prototyping and designing new QNN methods and architectures. The PennyLane framework can be divided into the following software components: • Pennylane Templates. The software component provides higher-level building blocks for constructing QNNs. Templates are a library of ready-to-use templates of widely used PQC architectures. For instance, templates can be used to encode data into quantum states or to select pre-made QNN layers. • Gradients and Training. This software layer provides optimization tools to train the quantum circuits. It includes automatic differentiation libraries, such as libraries from NumPy [105], PyTorch [21], JAX [106], and TensorFlow [20], and integrates them into the quantum computing framework. • Quantum Operators and Measurements. This software layer provides different quantum operators, including quantum gates, noisy channels, state preparations, and measurements. As for the measurement, PennyLane supports results from quantum devices: observable expectation, its variance, single measurement samples, and computational basis state probabilities. • Quantum Circuit/Device The software component provides the interface between the software and the hardware. In PennyLane, calculations involving the execution of one or more quantum circuits are formulated as quantum node objects. The quantum nodes are used to express the quantum circuit, pin the computation to a specific device, and execute it. This software layer comprises PennyLane plugins for different quantum hardware devices and simulators. These plugins enable users to execute quantum circuits on different devices and return the measurement outcomes.
PennyLane provides several quantum computer simulators, including a state simulator of qubit-based quantum systems, Gaussian states (for operations on CV architectures), qubit-based quantum circuit architectures written in TensorFlow for automatic differentiation, and qubit-based quantum circuit architectures for automatic differentiation with the autograd library [107].

Qiskit Machine Learning
The IBM Qiskit programming framework is one of the most popular and established approaches for programming quantum computers, as the IBM quantum systems were among the first to become available to the general public on the cloud. Qiskit provides an API to connect to and run a quantum code on the IBM quantum computers and a range of abstractions for gate-based quantum computing. Most importantly, for PQC and QNN development, Qiskit provides a library called qiskit-machine-learning, specifically designed to develop QNNs. At a high level, the qiskit-machine-learning framework can be divided into different software components: • Data Preparation. This component is responsible for preprocessing the input data before it is used to train or test a quantum machine learning model. • Feature Maps. The feature maps layer defines the quantum circuits that map the input data onto a quantum state. It includes pre-built feature maps for common ML tasks. • Neural Networks. This component contains a programming interface for the QNNs (called NeuralNetwork) and two specific implementations (i) EstimatorQNN: this network is based on evaluating quantum mechanical observables, and (ii) SamplerQNN: a network based on the samples measuring a quantum circuit. These high-level classes provide methods for configuring the PQC, its initialization, and performing the forward and backward passes. • Classifiers and Regressors. To train and use Quantum Neural Networks, qiskitmachine-learning provides different learning algorithms such as the NeuralNet-workClassifier and NeuralNetworkRegressor. These take a QNN as input and then use it for classification or regression. Two convenience implementations are provided to allow an easy start: the Variational Quantum Classifier (VQC) and the Variational Quantum Regressor (VQR). • Qiskit. At the bottom of the qiskit-machine-learning software stack, there is Qiskit that provides quantum gate and circuits primitives (including parametrized gates), gradients, and optimizers.
In addition, qiskit-machine-learning provides a connector to PyTorch for implementing hybrid classical-quantum NNs, e.g., some nodes are classical, and some are quantum. This hybrid architecture is obtained by embedding a quantum layer in a classical PyTorch network. Regarding quantum computer simulators, the Qiskit Aer module provides different quantum computer simulator backends, including ideal and noisy state vectors, density matrix, and unitary simulation backends.

Rigetti Grove
The Rigetti Forest programming environment includes a quantum instruction language Quil, its Python interface, called pyQuil, and a library of quantum programs called Grove. Rigetti Grove is a collection of high-level primitives that can be used to develop QNNs. The Rigetti Forest also provides a quantum simulation environment called QVM (Quantum Virtual Machine).

Strawberry Fields
Strawberry Fields is a Python library designed to run quantum CV programs on quantum photonics hardware [108]. It is based on the language named Blackbird, and provides three different simulator backends: a simulator of Gaussian states, Fock states, and a Fock-basis backend written using the TensorFlow (that can provide automatic differentiation and optimizers). Regarding the PQC development, the TensorFlow backend is critical for optimizers and gradients from TensorFlow 2. Thanks to Strawberry Fields, it is possible to experiment and design a CV Quantum Neural Network, as discussed in the seminal paper on CV QNN [30].

TensorFlow Quantum
TensorFlow Quantum (TQ) is a Python library designed for ML workloads using quantum-classical QNN models [33]. TQ is developed by Google and leverages and unifies Google's Cirq within TensorFlow. While integrating quantum computing algorithms and gates designed in Cirq, TQ delivers additional quantum computing primitives in line with the TensorFlow API and high-performance quantum circuit simulators. The basic TQ software layers are: • Classical and Quantum Data. TFQ allows the processing of classical and quantum data (in the form of quantum circuits and operators). • Keras API. TQ integrates with the core TensorFlow and Keras [109], providing NN models and optimizers. In addition to Cirq, TQ also provides a high-performance C++ TQ-native (e.g., not relying on the Cirq simulators) quantum computer simulator for QNN called qsim.

Torch Quantum
Torch Quantum [67] is a PyTorch library designed explicitly for quantum machine learning and simulations at MIT. Torch Quantum leverages the main characteristics that made PyTorch popular and widespread in the data-science community: easy NN/PQC construction, dynamic computation graph for easier debugging, and gradient calculations via autograd. Torch Quantum can be easily deployed on real quantum devices such as IBM Quantum systems. Torch Quantum provides an HPC state vector simulator (with support for GPUs), and pulse simulation is planned to be implemented in the future.

Zapata Orquestra
Zapata offers a quantum computational platform, Orquestra, including a quantum SDK (for circuit, gate, and noise models) and an algorithm suite that comprises quantum ML, chemistry, cryptography, and error mitigation methods. Zapata developed a proprietary generative AI technique that exploits hybrid classical-quantum systems [110] and uses Quantum Circuit Born Machine (QCBM). Among the most important Orquestra features, there are the workflow manager and integration with deployment orchestration tools, such as Slurm and Ray, that allow for quantum-enabled workflows and execution on quantum and classical HPC resources. Orquestra supports different quantum computer backends, including IBM, D-Wave, IonQ systems, and the Qulacs quantum computer simulator.

Summary
To summarize the feature of the different QNN programming frameworks, we provide an overview of current QNN programming frameworks in Table 1, providing the target quantum architectures (possibly, also of future implementations), main programming languages, availability of quantum simulators, and distinctive features of the programming frameworks.

Conclusions
In this paper, we surveyed the current state-of-the-art high-level programming approaches for QNN development. We discussed target architectures, quantum data, critical QNN algorithmic components, such as the hybrid workflow of QA and PQC, optimizers, and techniques for performing gradient calculations on quantum computer hardware and simulators. We also presented existing programming QNN frameworks. The field of QNN methods and programming frameworks quickly evolves, and new techniques and methods will certainly develop to tackle current QNN limitations. Currently, one of the main QNN challenges is to address the problem of barren plateau in the optimization landscape.
Additional quantum computer architectures will become available for QNN developers and users in the future. An example is the PsiQuantum's photonics fusion-based quantum chip [112] or the Microsoft topological quantum computers [113]. Despite the potential Cambrian explosion of different quantum computer architectures, programming these new quantum systems will likely retain the existing quantum computing abstractions (gates, circuit, measurements, QNN layer, ...) and reuse existing programming approaches to ensure portability across different platforms, an important issue already in the HPC field. An example of a portable quantum programming framework is PennyLane, which allows for developing specific plugins to support different and possibly new QPU devices.
Following the existing development of machine learning frameworks, such as Tensor-Flow, it is likely that in the future, QNN frameworks will rely more and more on domainspecific languages and compiler technologies to provide an Intermediate Representation (IR) that can be translated to different quantum hardware (and simulator) backends. Compiler toolchains, such as LLVM and MLIR [114][115][116], are already in use by the Intel Quantum SDK [98], and CUDA Quantum. These technologies might have a prominent role in the future of programming QNN on a quantum computer.

Conflicts of Interest:
The author declares no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: