Off-The-Grid Variational Sparse Spike Recovery: Methods and Algorithms

Abstract

Gridless sparse spike reconstruction is a rather new research field with significant results for the super-resolution problem, where we want to retrieve fine-scale details from a noisy and filtered acquisition. To tackle this problem, we are interested in optimisation under some prior, typically the sparsity i.e., the source is composed of spikes. Following the seminal work on the generalised LASSO for measures called the Beurling-Lasso (BLASSO), we will give a review on the chief theoretical and numerical breakthrough of the off-the-grid inverse problem, as we illustrate its usefulness to the super-resolution problem in Single Molecule Localisation Microscopy (SMLM) through new reconstruction metrics and tests on synthetic and real SMLM data we performed for this review.

1. Introduction

In this paper, we propose to conduct a comprehensive review on the so-called off-the-grid variational methods to solve the sparse spike recovery problem. We will exhibit the main theoretical and numerical results in the literature, underlining the interest of these methods for various domains dealing with inverse problems. As part of this review and our former work on gridless methods, we developed an implementation of the more consistent numerical methods with a focus on efficiency and computation time. With this implementation, we were able to apply off-the-grid method to fluorescence microscopy super-resolution problem. The codes and the computed result are an addition to the off-the-grid literature, and constitute further evidence supporting the relevance of this domain in the inverse problem field.

Loosely speaking, inverse problems consist in the reconstruction of the causes from the consequences. The problem is generally ill-posed, meaning that existence, uniqueness, and stability of a solution(s) is (are) not guaranteed. A case arising in numerous fields such as image or signal processing, telecommunications, machine learning, super-resolution, etc. is the sparse spike problem. It consists of the reconstruction of spikes located on a domain $\mathcal{X}$ from an acquisition y, with the prior of sparsity on the cause; or in layman terms, the source is composed of a few spikes. This includes sources such as stars in astronomy, fractures in seismology, etc. A spike is typically modelled by a Dirac measure $a{\delta }_x$ with amplitude $a\in \mathbb{C}$ and position $x\in \mathcal{X}$. All the difficulty lies in the estimation of the number N of spikes, of their amplitudes ${\left(a_i\right)}_{i=1}^N$ and their positions ${\left(x_i\right)}_{i=1}^N$. Hence, the goal is to reconstruct the measure $m={\sum }_{i=1}^N{a}_i{\delta }_{x_i}$ only from a few number of observations y in a Hilbert space $\mathcal{H}$ (typically ${\mathrm{L}}^2\left(\mathcal{X}\right)$) linked to m through an operator $\Phi$ accounting for deterioration of the input (blur, downsizing by the sampling) such as $y\stackrel{\mathrm{def}.}{=}\Phi m+w$ where $w\in \mathcal{H}$ is an additive noise. The reconstruction of the spikes may be off-the-grid i.e., the positions ${\left(x_i\right)}_{i=1}^N$ are not constrained on a grid hence ${\left(x_i\right)}_{i=1}^N$ are not limited to a finite set of values: this allows interesting new mathematical insights and guarantees for the reconstruction, at the cost of some challenges for the numerical implementation. The general sparse spike problem is encountered in many situations, such as:

• compressed sensing domain [1], where one wants to recover a s-sparse vector $v\in {\mathbb{C}}^N$ from M measurements $A{v}_0$ where $A\in {\mathbb{C}}^{M\times N}$;
• machine learning, sketching mixtures, etc. For example, we desire to fit a probability distribution with respect to given data. The point is to estimate parameters $\left(a_i\right)\in {\mathbb{R}}^N$ and $\left(x_i\right)\in {\mathcal{X}}^N$ of a mixture ${\sum }_{i=1}^N{a}_i\phi \left(x_i\right)$ of N elementary distributions described by $\phi$. For instance, one wants to retrieve the means ${\mu }_i\in \mathbb{R}$ and standard deviations ${\sigma }_i\in {\mathbb{R}}^{+}$ of a Gaussian mixture, see [2] for more insights on this question:
• deep learning such as training neural networks with a single hidden layer [3];
• signal processing, for instance low rank tensor decomposition for Direction of Arrival estimation through sensor array (multiple sampling points);
• super-resolution, a rather central problem in image processing. Roughly speaking, it consists of the reconstruction of details from an altered input of signal/image. It includes a classic physical operator of acquisition such as Fourier measurements, Laplace transform or Gaussian convolution.

The latter item will be our case of interest in the sparse spike problem for this paper. All the difficulty stems from the degradation in the acquisition process, which entails in general two things: a deterioration by the system of acquisition, typically modelled by the Point Spread Function in imagery which acts as a low-pass filter sensor acquisition which results in sampling and pollution by noise of different types, characterised by densities such as Gaussian, Poisson, etc. To sum up, we want to reconstruct the correct number of spikes with correct amplitudes and positions in the continuous setting from a noisy and filtered discrete acquisition. It can be tackled from the theoretical point of view by either the variational approach or Prony’s method:

• Prony’s method and its variants such as MUSIC (MUltiple SIgnal Classification), ESPRIT (Estimation of Signal Parameters by Rotational Invariance Techniques) or Matrix Pencil, which recover the signal source from Fourier measurements in a noiseless 1D setting. It consists of the decomposition of the signal onto a basis of exponentials with different amplitudes, damping factors, frequencies, and phase angles to match the observed data. The results are compelling in the 1D noiseless case, and can be extended to a multivariate and noisy context, but still these methods lack of versatility since they cannot be sometimes extended to the context of interest. Thus, we will not consider this approach in this paper;
• variational approach which does not impose any particular structure on the acquisition operator, which can be adapted to any type of noise and does not need any prior on the number of point sources [4]. The key idea is to solve the inverse problem by finding among all possible signal sources the one minimising an objective function called the energy, formulated as a trade-off between a fidelity data term and a regularisation term, typically enforcing the sparsity prior here.

Then, there are two types of variational approaches: the discrete and the off-the-grid. In the discrete setting, one seeks to recover the spikes on a prescribed fine grid, typically with more points than the acquisition image. Indeed, we call coarse grid for the low-resolved acquisition, and fine grid for the finer (by a so-called super-resolution factor $q\in {\mathbb{N}}^{\ast }$) grid of the reconstruction. Thus, it consists of a finite dimensional problem, where the positions of the spikes must lie on a grid $\mathcal{G}$ of L points meshing the domain $\mathcal{X}$. This problem is a problem of sparse vectors reconstruction, and it can be tackled by enforcing sparsity through minimisation of the ${\ell }_1$ norm of the unknown vector. This is known as the LASSO [5] or the Basis-pursuit problem, defined as the variational problem with tuning parameter $\lambda >0$ controlling the trade-off between fidelity to the data and enforcement of the prior:

$$\begin{array}{c}\hfill \underset{a\in {\mathbb{R}}^L}{min}\underset{\mathrm{data}\mathrm{term}}{\underbrace{{\Vert y-{\Phi }_{L}a\Vert }_{\mathcal{H}}^2}}+\underset{\mathrm{sparsity}\mathrm{prior}}{\underbrace{\lambda {\Vert a\Vert }_1}}\end{array}$$

(LASSO)

where ${\Phi }_L:{\mathbb{R}}^L\to \mathcal{H}$ is the acquisition operator with a vector of size L as an input and $\mathcal{H}$ is a Hilbert space. A grid is useful to epitomise the concept of sparsity in the case of spikes: indeed, sparsity is just the fact that only a few points of the L grid have a non-zero value. Moreover, since a computer can only store array and vector quantities, it seems rather fair to work with finite dimensional problem, even for the theoretical analysis. However, how does one choose the discretisation? A grid with a step-size too small yields numerical instabilities [6] while choosing a step-size that is too large leads to round-off errors. Moreover, one would like to localise the spikes as precisely as possible without having to rely on a grid: a discretisation of positions would necessarily convey approximation on positions. The appropriate mathematical objects to get rid of these discretisation drawbacks is to represent a collection of spikes with Dirac measures, an element of the space of Radon measures $\mathcal{M}\left(\mathcal{X}\right)$. The operator of acquisition is now $\Phi :\mathcal{M}\left(\mathcal{X}\right)\to \mathcal{H}$, the sparsity is enforced by a norm on $\mathcal{M}\left(\mathcal{X}\right)$ called the TV-norm. This variational problem is called the BLASSO (for Beurling LASSO):

$$\begin{array}{c}\hfill \underset{m\in \mathcal{M}\left(\mathcal{X}\right)}{min}\underset{\mathrm{data}\mathrm{term}}{\underbrace{{\Vert y-\Phi m\Vert }_{\mathcal{H}}^2}}+\underset{\mathrm{sparsity}\mathrm{prior}}{\underbrace{\lambda {\Vert m\Vert }_{\mathrm{TV}}}}.\end{array}$$

(BLASSO)

In this latter setting, the spikes can move continuously on the domain $\mathcal{X}$: a comparison between the discrete and the off-the-grid reconstruction is given in Figure 1. The off-the-grid setting can be seen as the limit of the discrete case with a finer and finer grid [7].

This shift from the discrete domain to the continuous setting called off-the-grid or gridless leads to some crucial mathematical insights, in particular a sharp signal-dependent criterion for stable spikes recovery [8], the minimum separation distance (see the next section). Obviously, some difficulties arise also due to the infinite dimension and the lack of algebraic properties of the set of optimisation. The comparison between discrete and gridless settings may be summed up by:

• the discrete problem is tackled by LASSO, through the minimisation of a convex function defined on a fine grid i.e., a convenient finite dimension Hilbert ${\mathbb{R}}^L$ space. Due to the ${\ell }_1$ norm, there are some cases where the sparsity is not properly enforced: one can then replace the ${\ell }_1$ norm by the non-continuous pseudo-norm ${\ell }_0$, but this yields an NP-hard combinatory non-convex problem. There exists some continuous relaxation of ${\ell }_0$ such as CEL0 [9], but, due to the non-convex aspect, the problem is still hard from the theoretical and numerical point of view. Despite the lack of guarantees, there are numerous algorithms to compute the solution of LASSO or its ${\ell }_0$ relaxed variant;
• the off-the-grid problem is treated by BLASSO, a convex functional defined on $\mathcal{M}\left(\mathcal{X}\right)$. The convex property is handy from the theoretical point of views as it leads to some crucial insights on the existence/uniqueness/support estimation w.r.t. noise, at the cost of the set of optimisation, namely $\mathcal{M}\left(\mathcal{X}\right)$ a Banach (no Hilbertian structure so no straightforward proximal algorithm) infinite dimensional and non-reflexive space for the strong topology (convergence results are then essentially on the weak-* topology). Despite this lack of algebraic properties, one has currently a wide range of algorithms to tackle this problem, such as root-finding or greedy algorithms.

Gridless reconstruction can then be evaluated through suitable metrics, namely the Flat Metric based on optimal transport of measures. This metric assesses the quality of the reconstruction and can be applied straightforwardly to off-the-grid and even discrete reconstruction outputs.

In the following, we give a review on the key results in the variational off-the-grid domain. The paper is organised in three sections, namely:

• the variational analysis of the space $\mathcal{M}\left(\mathcal{X}\right)$, the properties and the guarantees of reconstruction concerning the sparse spike problem are now quite well-documented [8,10,11,12] and will be recalled in the theoretical Section 2;
• multiple strategies were considered to numerically tackle BLASSO, the more compelling will be presented and put into context in the numerical Section 3;
• interesting practical applications and new metrics have been considered for the gridless method, such as the SMLM super-resolution; these results are shown and discussed in Section 4.

At the end of each paragraph, a grey box (beginning either with ‘summary’ or ‘shorthand’) like this one will recall the main results highlighted in the section. Please refer to it for a quick summary.

2. A Theoretical Background for Gridless Spike Recovery

In the following, $\mathcal{X}$ denotes the ambient space where the positions of the spikes live. We suppose $\mathcal{X}$ is a subset of ${\mathbb{R}}^d$ such that its interior $\mathring{\mathcal{X}}$ is a submanifold of dimension $d\in {\mathbb{N}}^{\ast }$ [13]. This setting encompasses $\mathcal{X}={\mathbb{R}}^d$, the torus $\mathcal{X}={\mathbb{T}}^d\stackrel{\mathrm{def}.}{=}{\mathbb{R}}^d/{\mathbb{Z}}^d$, any compact with non-empty interior, etc. The reader is invited to take a look at the

Table A1. Main notations used in the review.

d	dimension of the ambient space
$\mathcal{X}$	ambient space of the spike positions e.g., the torus, ${[0,1]}^d$, ${\mathbb{R}}^d$, etc.
${\mathcal{C}}_0\left(\mathcal{X}\right)$	space of evanescent continuous functions
$\mathcal{M}\left(\mathcal{X}\right)$	space of signed Radon measures
${\mathcal{M}}^{+}\left(\mathcal{X}\right)$	space of non-negative Radon measures
${\mathcal{M}}_{\mathbb{C}}\left(\mathcal{X}\right)$	space of complex Radon measures
$\mathcal{H}$	Hilbert space, typically ${\mathrm{L}}^2\left(\mathcal{X}\right)$
m	Radon measure
$\left|m\right|\left(\mathcal{X}\right)$	TV norm of m
$\delta$	Dirac measure
$a,x$	respectively amplitudes and positions of the spike $a{\delta }_x$
N	number of Dirac measures in a discrete measure
$\lambda$	regularisation parameter in ($𝒫$λ(y))
$\Phi$	forward acquisition operator with kernel $\phi$ and adjoint ${\Phi }^{\ast }$
${\mathcal{F}}_n$	forward Fourier acquisition operator with n measurements
$p_{\lambda }$	solution of the dual problem ($𝒟$λ(y))
${\eta }_{\lambda }$	dual certificate of ($𝒫$λ(y))
$T_{\lambda }$	BLASSO ($𝒫$λ(y)) functional
$\mathsf{\Omega }$	Lifted space $\mathsf{\Omega }\stackrel{\mathrm{def}.}{=}{\mathbb{R}}^{+}\times \mathcal{X}$
J	BLASSO functional on ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$
R	data-term $R:\mathcal{M}\left(\mathcal{X}\right)\to \mathcal{H}$
$F_N$	‘discrete’ functional on ${\mathsf{\Omega }}^N$
F	functional on $\mathcal{M}\left(\mathsf{\Omega }\right)$
${\left({\mu }_t\right)}_t$, ${\left({\nu }_t\right)}_t$	Gradient flows respectively in ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ and ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$
$\alpha$, $\beta$	Cone metric/ Fisher–Rao–Wasserstein tuning parameters
${\mathcal{W}}_p$	Wasserstein distance of order p
${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$	space of probability measures with 2nd moment endowed with ${\mathcal{W}}_2$

to remind the notations.

2.1. What Is a Measure?

As we have stated in the section above, the Dirac measure is the proper object to describe a spike not constrained on a finite set of positions. This object is not a function, since one cannot exhibit any integrable equivalence class satisfying the properties of the Dirac (see below). Thus, one should considerate the notion of Radon measure, a formal extension of functions. From a distributional standpoint, it is a subset of the distribution space ${\mathcal{D}}^{\prime }\left(\mathcal{X}\right)$, namely the space of linear forms over the space of test functions $\mathcal{D}\left(\mathcal{X}\right)$ i.e., smooth functions (continuous derivatives of all orders) compactly supported. This functional approach consists in the definition of a measure as a linear form on some function space, namely:

Definition 1

(Evanescent continuous function on$\mathcal{X}$).We call ${\mathcal{C}}_0\left(\mathcal{X},\mathcal{Y}\right)$ the set of continuous functions with zero at infinity (or evanescent), namely all the continuous map $\psi :\mathcal{X}\to \mathcal{Y}$ such that:

$$\begin{array}{c}\hfill \forall \epsilon >0,\exists K\subset \mathcal{X}\mathrm{compact},\underset{x\in \mathcal{X}\backslash K}{sup}{\psi \left(x\right)}_{\mathcal{Y}}\le \epsilon .\end{array}$$

When $\mathcal{Y}=\mathbb{R}$, we will simply write ${\mathcal{C}}_0\left(\mathcal{X}\right)$. Since we dispose of a suitable test functions space, we need to precise the notion of duality at stake in this review.

Definition 2

(Topological dual space).If E is a topological vector space, we denote $E^{\ast }$ its topological dual i.e., the space of all continuous linear forms $\psi :E\to \mathbb{R}$. The pairing between an element $\varphi \in E$ and a map $\psi \in E^{\ast }$ is denoted by the bilinear mapping ${\langle \varphi ,\psi \rangle }_{E\times E^{\ast }}\stackrel{\mathrm{def}.}{=}\psi \left(\varphi \right)$ called the duality bracket.

This notion allows us to define the Radon measure through duality in the following definition.

Definition 3

(Set of Radon Measures).We denote $\mathcal{M}\left(\mathcal{X}\right)$ the set of real signed Radon measures on $\mathcal{X}$ of finite masses. It is the topological dual of ${\mathcal{C}}_0\left(\mathcal{X}\right)$ with supremum norm ${\Vert ·\Vert }_{\infty ,\mathcal{X}}$ by the Riesz–Markov representation theorem. It can also be defined as the topological dual of the space of continuous function $\mathcal{C}\left(\mathcal{X}\right)$ if $\mathcal{X}$ is compact [14]. Thus, a Radon measure m is a continuous linear form evaluated on functions $f\in {\mathcal{C}}_0\left(\mathcal{X}\right)$, with for $m\in \mathcal{M}\left(\mathcal{X}\right)$ the duality bracket denoted by ${〈f,m〉}_{{\mathcal{C}}_0\left(\mathcal{X}\right)\times \mathcal{M}\left(\mathcal{X}\right)}={\int }_{\mathcal{X}}f\mathrm{d}m$.

The term ‘signed’ refers to the generalisation of the concept of (positive) measure, by allowing the quantity ${〈f,m〉}_{{\mathcal{C}}_0\left(\mathcal{X}\right)\times \mathcal{M}\left(\mathcal{X}\right)}$ to be negative. We can define in the same way the space of real non-negative Radon measures ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$ dual of ${\mathcal{C}}_0\left(\mathcal{X},{\mathbb{R}}^{+}\right)$ and the space of complex Radon measures ${\mathcal{M}}_{\mathbb{C}}\left(\mathcal{X}\right)$ dual of ${\mathcal{C}}_0\left(\mathcal{X},\mathbb{C}\right)$. Classic examples of Radon measures are:

• the Lebesgue measure of dimension $d\in \mathbb{N}$;
• the Dirac measure ${\delta }_z$ centred in $z\in \mathcal{X}$, also called the $\delta$-peak. For all $f\in {\mathcal{C}}_0\left(\mathcal{X}\right)$, one has ${〈f,{\delta }_z〉}_{{\mathcal{C}}_0\left(\mathcal{X}\right)\times \mathcal{M}\left(\mathcal{X}\right)}=f\left(z\right)$;
• discrete measures $m_{a,x}\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_i{\delta }_{x_i}$ where $N\in \mathbb{N}$, a$\in {\mathbb{C}}^N$, x$\in {\mathcal{X}}^N$.

Since ${\mathcal{C}}_0\left(\mathcal{X}\right)$ is a Banach space, $\mathcal{M}\left(\mathcal{X}\right)$ is complete [11] by endowing it with its dual norm called the total variation (TV) norm, defined for $m\in \mathcal{M}\left(\mathcal{X}\right)$ by:

$$\begin{array}{c}\hfill \left|m\right|\left(\mathcal{X}\right)\stackrel{\mathrm{def}.}{=}sup\left({\int }_{\mathcal{X}}f\mathrm{d}m,f\in {\mathcal{C}}_0\left(\mathcal{X}\right),{\Vert f\Vert }_{\infty ,\mathcal{X}}\le 1\right).\end{array}$$

The TV norm of a measure is also called its mass. One can note that, in the case of a discrete measure defined as before $m_{a,x}\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_i{\delta }_{x_i}$, one has $|m_{a,x}|\left(\mathcal{X}\right)={\Vert a\Vert }_1$.

The interested reader might take a look at the Appendix B.1 for more details on some functional analysis notions and results.

Summary: we model a spike by a Dirac measure, an element of the Radon measure spaces $\mathcal{M}\left(\mathcal{X}\right)$. This space is defined by duality, it is endowed by the TV-norm and is complete. It is, however, infinite dimensional and non-reflexive (see Appendix B.1), this poses additional difficulties to be taken into account in the optimisation.

2.2. Observations

Let us introduce the space where the acquired data live. We will denote by $\mathcal{H}$ this Hilbert space; for the instance of images, $\mathcal{H}={\mathrm{L}}^2\left(\mathcal{X}\right)$. Let $m\in \mathcal{M}\left(\mathcal{X}\right)$ be the source measure, we call acquisition $y\in \mathcal{H}$ the result of the forward/acquisition map $\Phi :\mathcal{M}\left(\mathcal{X}\right)\to \mathcal{H}$ evaluated on m, with measurement kernel $\phi :\mathcal{X}\to \mathcal{H}$:

$$\begin{array}{c}\hfill y\stackrel{\mathrm{def}.}{=}\Phi m={\int }_{\mathcal{X}}\phi \left(x\right)\mathrm{d}m\left(x\right).\end{array}$$

(1)

The latter integral ought to not be confused with the duality bracket ${〈f,m〉}_{{\mathcal{C}}_0\left(\mathcal{X}\right)\times \mathcal{M}\left(\mathcal{X}\right)}$$={\int }_{\mathcal{X}}f\left(x\right)\mathrm{d}m\left(x\right)$ mentioned in Definition 3 above. Indeed, while $f\left(x\right)\in \mathbb{R}$ for $x\in \mathcal{X}$, we have $\phi \left(x\right)\in \mathcal{H}$: the integral in (1) is then a Böchner integral [15] i.e., the proper notion to deal with vector valued map. It is valid as long as $\phi$ is continuous and bounded [3,13].

Remark 1.

Measures are objects that generalise functions at the cost of losing some of their properties. Thus, one cannot define a product of measures (what would be the square of the Dirac?) and one ought to be aware of some caveats concerning the functions of measure: these functionals need to be at most (sub)linear in order to be well-defined [16].

In the following, we will impose $\phi \in {\mathcal{C}}^2(\mathcal{X},\mathcal{H})$. Let us also define the adjoint operator of $\Phi :\mathcal{M}\left(\mathcal{X}\right)\to \mathcal{H}$ in the weak-* topology, namely the map ${\Phi }^{\ast }:\mathcal{H}\to {\mathcal{C}}_0\left(\mathcal{X}\right)$. It is defined for all $x\in \mathcal{X}$ and $p\in \mathcal{H}$ by ${\Phi }^{\ast }\left(p\right)\left(x\right)={\langle p,\phi \left(x\right)\rangle }_{\mathcal{H}}$. The choice of $\phi$ and $\mathcal{H}$ depends on the physical process of acquisition; indeed, generic measurement kernels are:

• convolution kernel with typically $\mathcal{H}={\mathrm{L}}^2\left(\mathcal{X}\right)$ and $\forall x\in \mathcal{X},\phi \left(x\right)\stackrel{\mathrm{def}.}{=}(s\mapsto \tilde{\phi }(s-x))\in \mathcal{H}$, for the PSF $\tilde{\phi }\in {\mathcal{C}}_0^2\left({\mathbb{R}}^d\right)$. One has, for instance, the Gaussian kernel, centred in $c\in \mathcal{X}$ with spread $\sigma >0$, defined by $s\mapsto \tilde{\phi }(s-c)\stackrel{\mathrm{def}.}{=}1/\sqrt[d/2]{2\pi {\sigma }^2}{e}^{-{\Vert s-c\Vert }_2^2/2{\sigma }^2}$;
• Fourier kernel with cut-off frequency $f_{\mathrm{c}}\in \mathbb{N}$ and $\mathcal{H}={\mathbb{C}}^{2f_{\mathrm{c}}+1}$, for $x\in \mathcal{X}=\mathbb{T}$ in 1D:

  $$\begin{array}{c}\hfill \phi \left(x\right)={\left(e^{2\mathrm{i}\pi kx}\right)}_{\left|k\right|\le f_{\mathrm{c}}};\end{array}$$
• Laplace kernel [4] for non-negative weighting function $\xi \in \mathcal{C}\left(\mathcal{X}\right)$ specific to the physical acquisition process and $\mathcal{H}={\mathrm{L}}^2\left({\mathbb{R}}^{+}\right)$: $\forall x\in \mathcal{X}, \phi \left(x\right)\stackrel{\mathrm{def}.}{=}(s\mapsto \xi \left(x\right)e^{-sx})\in \mathcal{H}$.

These three kernels correspond to various physical context of imagery, hence they are encountered in multiple acquisition process, such as Nuclear Magnetic Resonance spectroscopy (Fourier), SMLM super-resolution (convolution), MA-TIRF (Laplace), etc.

We will now on use the following notation for the discrete forward map: let $x=(x_1,\cdots ,x_N)$ and $a\in {\mathbb{R}}^N$: ${\Phi }_x\left(a\right)\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_i\phi \left(x_i\right).$

Shorthand: an acquisition living in the Hilbert space $\mathcal{H}$ of a measure m is the quantity $\Phi m$. $\Phi$ is the forward operator, completely defined by a kernel $\phi$ specific to the physical context of imagery.

2.3. An Off-The-Grid Functional: The BLASSO

Let $m_{a_0,x_0}\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_{0,i}{\delta }_{x_{0,i}}$ be the source measure with amplitudes $a_0$$\in {\mathbb{R}}^N$ and positions $x_0$$\in {\mathcal{X}}^N$, the sparse spike problem is to recover this measure from the acquisition $y\stackrel{\mathrm{def}.}{=}\Phi m_{a_0,x_0}+w$, where $w\in \mathcal{H}$ is an additive noise, typically white Gaussian noise. To tackle this problem, we use the following convex functional [11,17] also called the BLASSO, which stands for Beurling-LASSO:

$$\begin{array}{c}\hfill \underset{m\in \mathcal{M}\left(\mathcal{X}\right)}{argmin}{T}_{\lambda }\left(m\right)\stackrel{\mathrm{def}.}{=}{\displaystyle \frac{1}{2}}{\Vert y-\Phi \left(m\right)\Vert }_{\mathcal{H}}^2+\lambda \left|m\right|\left(\mathcal{X}\right)\end{array}$$

($𝒫$_λ(y))

with regularisation parameter $\lambda >0$ which accounts for the trade-off between fidelity and sparsity of the reconstruction. The name BLASSO was coined in the work of [17,18] according to the link between the Generalised Minimal Extrapolation (GME) problem where one seeks to reconstruct a Radon measure from several observations on its Fourier coefficients, and the work [19] of the Norwegian mathematician Beurling, which coincides with GME in the case of a Fourier forward operator.

The BLASSO in a noiseless setting writes down:

$$\begin{array}{c}\hfill \underset{\Phi m=y_0}{argmin}\left|m\right|\left(\mathcal{X}\right)\mathrm{with}{y}_0=\Phi m_{a_0,x_0}.\end{array}$$

($𝒫$₀(y₀))

BLASSO is genuinely linked with its discrete counterpart the (LASSO) [8]: one can formally see BLASSO as the functional limit of LASSO on a finer and finer grid. If the LASSO problem exhibits existence and uniqueness of the solution, what can one say for its off-the-grid counterpart? First of all, let us observe that:

• $m\mapsto \left|m\right|\left(\mathcal{X}\right)$ is lower semi-continuous w.r.t. the weak-* convergence (see Appendix B.1 for more insights);
• $\Phi$ is continuous from the weak-* topology of $\mathcal{M}\left(\mathcal{X}\right)$ to the weak topology of $\mathcal{H}$.

Thus, one can establish the existence of solutions to ($𝒫$_λ(y)) thanks to convex analysis results, as proved in [11].

Summary: the sparse spike problem is tractable thanks to the convex functional on $\mathcal{M}\left(\mathcal{X}\right)$ called the BLASSO and denoted by ($𝒫$_λ(y)). With $m\in \mathcal{M}\left(\mathcal{X}\right)$ as an input, it consists of a data term comparing observed data versus $\Phi m$, and a regularisation accounting for sparsity prior through the TV-norm of m. Existence of solutions of the BLASSO is known and proved.

The difficulties now lie in the following questions:

(1)

what are the conditions to recover a sparse measure, within a certain noise regime? Is the minimum unique?

(2)

under which conditions can we retrieve exactly the number of spikes, the amplitude, and the positions; when do we have support stability?

(3)

how can we tackle numerically the infinite dimensional and non-reflexive nature of the space $\mathcal{M}\left(\mathcal{X}\right)$?

In order to address these points, we need to introduce some notions of convex analysis in the following subsection.

2.4. Dual Problems and Certificates

The BLASSO in Equation ($𝒫$_λ(y)) above is a minimisation problem with a convex functional. Then, we can apply Ekeland–Temam ([20] Remark 4.2) results and define a dual problem which writes down for $p\in \mathcal{H}$ (see Appendix B.2 for the proof):

$$\begin{array}{c}\hfill \underset{{\Vert {\varphi }^{\ast }p\Vert }_{\infty ,\mathcal{X}}\le 1}{argmax}{\langle y,p\rangle }_{\mathcal{H}}-{\displaystyle \frac{\lambda }{2}}{\Vert p\Vert }_{\mathcal{H}}^2\end{array}$$

($𝒟$_λ(y))

which can be recast as the projection onto a closed convex [11,18]:

$$\begin{array}{c}\hfill \underset{{\Vert {\varphi }^{\ast }p\Vert }_{\infty ,\mathcal{X}}\le 1}{argmax}{\Vert {\displaystyle \frac{y}{\lambda }}-p\Vert }_{\mathcal{H}}^2\end{array}$$

($𝒟$_λ’(y))

Fenchel’s duality between ($𝒫$_λ(y)) and ($𝒟$_λ(y)) is proved in [11]. Therefore, any solution $m_{\lambda }$ of ($𝒫$_λ(y)) is linked [8] to the unique solution $p_{\lambda }$ of ($𝒟$_λ(y)) by the extremality conditions:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\Phi }^{\ast }{p}_{\lambda }\in \partial |m_{\lambda }|\left(\mathcal{X}\right),\hfill \\ -p_{\lambda }={\displaystyle \frac{1}{\lambda }}(\Phi m_{\lambda }-y)\hfill \end{array}\right.\end{array}$$

(2)

where $\partial \left|·\right|\left(\mathcal{X}\right)$ is the sub-differential of the TV norm. Indeed, since the total variation is not differentiable (as the ${\ell }^1$ norm) but lower semi-continuous w.r.t. the weak-*topology, we use its sub-differential which for $m\in \mathcal{M}\left(\mathcal{X}\right)$ identifies to:

$$\begin{array}{c}\hfill \partial \left|m\right|\left(\mathcal{X}\right)=\left\{\eta \in {\mathcal{C}}_0\left(\mathcal{X}\right);{\Vert \eta \Vert }_{\infty ,\mathcal{X}}\le 1\mathrm{and}{\int }_{\mathcal{X}}\eta \mathrm{d}m=\left|m\right|\left(\mathcal{X}\right)\right\}.\end{array}$$

(3)

Elements of this subgradient are called certificate. Thanks to strong duality, one can define peculiar certificates called the dual certificates [10].

Definition 4.

We call ${\eta }_{\lambda }\stackrel{\mathrm{def}.}{=}{\Phi }^{\ast }{p}_{\lambda }$, where $p_{\lambda }$ satisfies (2), a dual certificate of $m_{\lambda }$.

It is a certificate since ${\Phi }^{\ast }{p}_{\lambda }\in \partial |m_{\lambda }|\left(\mathcal{X}\right)$, and it is called dual because it verifies the second extremality (2) condition: it is thus defined by the dual solution $p_{\lambda }$. Loosely speaking, a dual certificate ${\eta }_{\lambda }$ is associated with a measure $m_{\lambda }$, and it certifies that the measure $m_{\lambda }$ is a minimum of the BLASSO. For instance, if there exist solutions of ($𝒫$_λ(y)) of the form $m_{\lambda }\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_i{\delta }_{x_i}$, the support satisfies [8] for all $0\le i\le N$ : $|{\eta }_{\lambda }|\left(x_i\right)=1$.

In the same fashion, one has the link between a solution $m_0$ of the noiseless BLASSO ($𝒫$₀(y₀)) and its certificates ${\eta }_0$, which are not unique in general. Then, in the rest of the document, we will refer to ${\eta }_0$ as the minimal norm certificate i.e., the dual certificate ${\eta }_0$ with minimal supremum norm ${\Vert {\eta }_0\Vert }_{\infty ,\mathcal{X}}$. It is shown in [8] that this minimal norm certificate ${\eta }_0$ has important properties, since it somehow drives the stability of the recovered spike locations when the additive noise is small, in particular how close they are to the positions of the true measure $m_{a_0,x_0}$: see Definition 6 in the section below.

Summary: we defined the primal problem in the former section, thanks to convexity, we can define the dual problem of the BLASSO. A solution $m_{\lambda }$ of the BLASSO and a solution $p_{\lambda }$ of the dual problem are linked through extremality condition. The dual solution $p_{\lambda }$ defines the dual certificate, an element of the subgradient specified by ${\eta }_{\lambda }={\Phi }^{\ast }{p}_{\lambda }$: the dual certificate ${\eta }_{\lambda }$ certifies that $m_{\lambda }$ is a solution of the BLASSO. We can then establish more precise conditions on the uniqueness/support recovery.

2.5. Support Recovery Guarantees

We will address in this section the first two questions we have laid down, namely existence, uniqueness, and support recovery conditions. A classical tool to establish some recovery properties lies in the notion of the minimum separation distance.

Definition 5

(Minimum separation distance).The minimum separation distance is a characterisation of the support of the discrete measure $m_{a_0,x_0}$ by:

$$\begin{array}{c}\hfill \mathsf{\Delta }\left(m_{a_0,x_0}\right)\stackrel{\mathrm{def}.}{=}\underset{i\ne j}{min}\left|x_{0,i}-x_{0,j}\right|.\end{array}$$

The reconstruction condition is driven by this minimum separation distance, itself determined by the type of measure (complex, real, real non-negative) and the type of forward operator.

• if the operator is an acquisition of the Fourier spectrum within $[-f_{\mathrm{c}},f_{\mathrm{c}}]$ with frequency cut-off $f_{\mathrm{c}}$ for $\mathcal{X}={\mathbb{T}}^d$ the d-torus in the noiseless setting, it is necessary that $\mathsf{\Delta }\left(m_{a_0,x_0}\right)\gtrsim \frac{2}{f_{\mathrm{c}}}$ if the source measure is complex [10]. Upon a few conditions [21], one can weaken it to $\mathsf{\Delta }\left(m_{a_0,x_0}\right)\gtrsim \frac{1.26}{f_{\mathrm{c}}}$, and $\mathsf{\Delta }\left(m_{a_0,x_0}\right)\gtrsim \frac{1.87}{f_{\mathrm{c}}}$ if the source measure is real [10];
• regardless of the operator $\Phi$ [17,22], there is no condition on the separation for a real positive source measure in the noiseless setting; however, stability constant explodes when $\mathsf{\Delta }\left(m_{a_0,x_0}\right)\to 0$.

These results are important but do not provide a sharp characterisation of the recovery in the presence of noise; however, we expect to find noise in the images we deal with and therefore to be limited by this noise regime. To account for this effect, we need to add some conditions on the ground-truth measure; following the work of [8], we introduce:

Definition 6

(Non-degenerate source condition).The source $m_{a_0,x_0}$ verifies the NDSC (Non-Degenerate Source Condition) if:

• there exists $\eta \in \mathrm{Im}{\Phi }^{\ast }$ such that $\eta \in \partial |m_{a_0,x_0}|\left(\mathcal{X}\right)$;
• $\forall s\in \mathcal{X}\backslash {\cup }_{i=1}^N\left\{x_{0,i}\right\},\left|{\eta }_0\left(s\right)\right|<1$;
• $\forall i\in ⟦1,N⟧,$ the Hessian matrix ${\nabla }^2{\eta }_0\left(x_{0,i}\right)\in {\mathbb{R}}^{d\times d}$ is invertible.

The first condition amounts to assuming that $m_{a_0,x_0}$ is a solution to ($𝒫$₀(y₀)) and there exists a solution to its dual problem. If the two latter conditions are matched, we say that ${\eta }_0$ is not degenerate. This allows us to write the main result of [8], namely:

Theorem 1

(Noise robustness [8]).Let ${\Gamma }_{x_0}$ the $N\times N$ matrix defined by ${\Gamma }_{x_0}\stackrel{\mathrm{def}.}{=}{(\phi (·-x_{0,i}),{\phi }^{\prime }(·-x_{0,i}))}_{i=1}^N$. Assume that ${\Gamma }_{x_0}$ has full column rank and that $m_{a_0,x_0}$ verifies the NDSC. Then, there exists $\alpha >0,{\lambda }_0>0$ such that for all $0\le \lambda \le {\lambda }_0$ and w such that $\Vert w\Vert \le \alpha \lambda$; there exists N pairings $(a_{\lambda ,i},x_{\lambda ,i})$ such that $m_{\lambda }\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_{\lambda ,i}{\delta }_{x_{\lambda ,i}}$ is the unique solution of Ref. ($𝒫$_λ(y)) composed of exactly N spikes. In particular, for $\lambda =1/\alpha {\Vert w\Vert }_{\mathcal{H}}$, we have the control over the discrepancies:

$$\begin{array}{c}\hfill \forall i\in ⟦1,N⟧:\Vert x_{\lambda ,i}-x_{0,i}\Vert =𝒪\left({\Vert w\Vert }_{\mathcal{H}}\right)\mathrm{and}\left|a_{\lambda ,i}-a_{0,i}\right|=𝒪\left({\Vert w\Vert }_{\mathcal{H}}\right).\end{array}$$

Under the Non-Degenerate Source Condition, for $\lambda$ and ${\Vert w\Vert }_{\mathcal{H}}^2/\lambda$ small enough, one can reconstruct a measure with the same number of spikes as the ground-truth measure $m_{a_0,x_0}$. Furthermore, the reconstructed measure (weak-*)converges to the ground-truth measure when the noise level drops to 0. The authors of [8] also introduce the notion of vanishing derivatives precertificate. The ${\eta }_0$ certificate is indeed hard to compute from the dual problem of ($𝒫$₀(y₀)). Because of the constraint ${{\Vert \eta \Vert }_0}_{\infty ,\mathcal{X}}\le 1$, the precertificate allows for leveraging this computation by solving instead a linear system. The interested reader is advised to take a glance at this article among other ones [8,22] for these new concepts.

Shorthand: the minimum separation distance criterion is used to assess recovery possibilities in the noiseless setting. In a low regime of noise, a theorem states that the source measure $m_{a,x}$ composed of N spikes can be recovered through BLASSO, with a control over the discrepancies (amplitudes/positions) between the reconstructed and the source measures.

We were therefore able to establish some guarantees on the reconstruction of the source measures in the presence of noise. In the next section, we propose to address the third question and to discuss strategies to compute the numerical solution of the inverse problem; a difficult task requiring accounting for the difficulties of the optimisation space.

3. Numerical Strategies to Tackle the BLASSO

The BLASSO problem $𝒫$_λ(y) is an optimisation over the set of Radon measures, an infinite dimensional and non-reflexive space. We recall that it writes down:

$$\begin{array}{c}\hfill \underset{m\in \mathcal{M}\left(\mathcal{X}\right)}{argmin}{T}_{\lambda }\left(m\right)\stackrel{\mathrm{def}.}{=}{\displaystyle \frac{1}{2}}{\Vert y-\Phi \left(m\right)\Vert }_{\mathcal{H}}^2+\lambda \left|m\right|\left(\mathcal{X}\right).\end{array}$$

($𝒫$_λ(y))

A naive approach would be to enforce the measure m to be supported on a fine grid ${\left(p_i\right)}_i^L$ which is equivalent to solving the LASSO problem:

$$\begin{array}{c}\hfill \underset{a\in {\mathbb{R}}^L}{min}{\Vert y-{\Phi }_{L}a\Vert }_{\mathcal{H}}^2+{\Vert a\Vert }_1\end{array}$$

with the discrete operator ${\Phi }_{L}a\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^L{a}_i\phi \left(p_i\right)$ and $\phi$ the kernel of the forward operator. This approach conveys numerous cons: for instance, the solution of the LASSO, in small noise regime and when the step size tends to 0, contains pairs of spikes around the true one [6,7]. Furthermore, refining the step size leads to a worse conditioning of the forward operator, accounting for numerical difficulties. The following classes of algorithms better account for the infinite dimensional nature of $\mathcal{M}\left(\mathcal{X}\right)$. We present in detail the three methods with the most established results in the literature [13,18,23]. Before describing these methods, let us remark that there also exist some promising avenues, such as the projected gradient descent [24,25]. It relies on an over parametrised initialisation i.e., a discrete measure with numerous $\delta$-peaks compared to the ground-truth, then one applies a gradient descent on the amplitudes and positions of the over parametrised measure combined at each step with a projection on a set of positions constraints to enforce the separation of the spikes. This projection can be replaced by a ‘heuristic’ which boils down to the merging of $\delta$-peaks that are not enough separated [25].

3.1. Semi-Definite Recasting and Hierarchy

Semi-definite programming was one of the first schemes solving the BLASSO in the specific case of a Fourier acquisition on the 1D torus ${\mathbb{T}}^1$ [10,12,17,18]. Before explaining in layman terms the SDP scheme, let us first introduce and detail the relevant quantities for this section. Let $d=1$ be the dimension of the interior of $\mathcal{X}$, let us study the case where the forward operator denoted by ${\mathcal{F}}_n$ (and not $\Phi$ for this section) is a Fourier coefficient measurement up to some cut-off frequency $f_{\mathrm{c}}\in \mathbb{N}$, with $n=2f_{\mathrm{c}}+1$ the number of measurements. We have ${\mathcal{F}}_n:{\mathcal{M}}_{\mathbb{C}}\left(\mathcal{X}\right)\to {\mathbb{C}}^n$ and, for a discrete measure $m_{a,x}\stackrel{\mathrm{def}.}{=}{\sum }_{j=1}^N{a}_j{\delta }_{x_j}$, it writes down ${\mathcal{F}}_n\left(m_{a,x}\right)={\left({\sum }_j{a}_j{e}^{2\mathrm{i}\pi k{x}_j}\right)}_{\left|k\right|\le f_{\mathrm{c}}}$ and its adjoint operator ${\mathcal{F}}_n^{\ast }:{\mathbb{C}}^n\to {\mathcal{C}}_0\left(\mathcal{X},\mathbb{C}\right)$ is for $s\in \mathcal{X}$:

$$\begin{array}{c}\hfill \forall c\in {\mathbb{C}}^n,{\mathcal{F}}_n^{\ast }\left(c\right)\left(s\right)={〈c,{\left(e^{2\mathrm{i}\pi ks}\right)}_{\left|k\right|\le f_{\mathrm{c}}}〉}_{{\mathbb{C}}^n}=\sum _{\left|k\right|\le f_{\mathrm{c}}}{c}_{f_{\mathrm{c}}+k}{e}^{2\mathrm{i}\pi ks}.\end{array}$$

(4)

This method is based on semi-definite programming (SDP) for efficiently computing the minima of BLASSO. It stems from the Hilbert approach [26] when one globally decomposes the objective function into simple pieces, atoms. The solution of the dual problem of ($𝒫$_λ(y)), denoted here $\left({𝒟}_{\lambda }^{ℱ}\right(y\left)\right)$, is a polynomial p linked to a certificate by ${\mathcal{F}}_n^{\ast }p$: the idea then is the reconstruction of the dual certificate as a linear sum of trigonometric polynomials [18], which is enough to find the measure associated with this reconstructed certificate. This associated measure is a solution to the BLASSO. The dual problem, on the other hand, is tractable thanks to a semi-definite programming approach.

Since ${\mathcal{F}}_n^{\ast }p$ is a trigonometric polynomial for any $p\in {\mathbb{C}}^n$ by the definition above, one can recast the constraint ${\Vert {\mathcal{F}}_n^{\ast }p\Vert }_{\infty ,\mathcal{X}}\le 1$ (imposed by definition of a certificate, see Equation (3)) and rewrite it as the intersection of the cone of positive semi-definite matrices $\{A:A⪰0\}$ with an affine hyperplane [10,27]. Hence, the Fenchel dual problem of ($𝒫$_λ(y)) for the Fourier forward operator ${\mathcal{F}}_n$:

$$\begin{array}{}\hfill \left({𝒟}_{\lambda }^{ℱ}\right(y\left)\right)& \hfill \underset{p{\in }^n}{max}\mathrm{Re}\{y,p\}-{\displaystyle \frac{\lambda }{2}}{\Vert p\Vert }_{\mathscr{H}}^2\mathrm{constrained}\mathrm{by}{\Vert {\mathcal{F}}_n^{\ast }p\Vert }_{\infty ,\mathcal{X}}\le 1\end{array}$$

with Hermitian product $\left\{·,·\right\}$, has the equivalent formulation [27]

$$\begin{array}{}& \hfill \underset{p\in {\mathbb{C}}^n,Q\in {\mathbb{C}}^{n\times n}}{max}\mathrm{Re}\{y,p\}-{\displaystyle \frac{\lambda }{2}}{\Vert p\Vert }_{\mathcal{H}}^2\mathrm{constrained}\mathrm{by}\hfill \left({\stackrel{\sim }{𝒟}}_{\lambda }^{ℱ}\right(y\left)\right)& \hfill \left(\begin{array}{cc}Q& p\\ p^{\ast }& 1\end{array}\right)⪰0\mathrm{and}\sum _{k=1}^{n-j}{Q}_{k,k+j}={\delta }_{0,j}\mathrm{for}j\in ⟦1,n-1⟧\end{array}$$

with Q being a Hermitian matrix and p a vector of coefficients (accounting for the dual variable p), and ${\delta }_{0,j}$ the Kronecker delta equal to 1 if $j=0$ and 0, otherwise. The choice of regulariser $\lambda$ is crucial: if chosen to be too high, it will yield a solution with fewer spikes, if chosen too low, it will recover a solution with spurious spikes. This finite dimensional formulation can now be tackled with classic semi-definite programming solvers, as did the authors of [10] who proposed an algorithm of Interior Point Method, given in Algorithm 1. The first step reaches a solution p, allowing the definition of the certificate $p_{2n-2}\left(e^{2\mathrm{i}\pi t}\right)\stackrel{\mathrm{def}.}{=}1-{\left|{\mathcal{F}}_n^{\ast }p\right|}^2\left(t\right)$, where ${\mathcal{F}}_n^{\ast }$ is defined in Equation (4).

Algorithm 1. Interior Point Method applied to the BLASSO.

1 Solve $$\begin{array}{c}\hfill \underset{p\in {\mathbb{C}}^n,Q\in {\mathbb{C}}^{n\times n}}{max}\mathrm{Re}y,p-{\displaystyle \frac{\lambda }{2}}{\Vert p\Vert }_{\mathcal{H}}^2\end{array}$$  subject to $\left(\begin{array}{cc}Q& p\\ p^{\ast }& 1\end{array}\right)⪰0$ and ${\sum }_{i=1}^{n-j}{Q}_{i,i+j}={\delta }_{0,j}$ for $j=1,\cdots ,n-1.$ 2 Reconstruct the support $\widehat{X}$ of m by locating the roots of $p_{2n-2}$ on the unit circle (e.g., by computing the eigenvalues of its companion matrix). 3 Solve ${\sum }_{t\in \widehat{X}}{a}_t{e}^{-2\mathrm{i}\pi kt}=y_k$ to recover the amplitudes a.

One can note the link between the dual and the primal problem, i.e., that p, the solution of $\left({\stackrel{\sim }{𝒟}}_{\lambda }^{ℱ}\right(y\left)\right)$, entails the location of the spikes: as ${\mathcal{F}}_n^{\ast }p$ yields its extremal points on the support of m since it is the certificate of a discrete measure, note that $p_{2n-2}\left(e^{2\mathrm{i}\pi t}\right)=1-{\left|{\mathcal{F}}_n^{\ast }p\right|}^2\left(t\right)$ has all its roots on the unit circle, and these roots are the support of the target measure [10]. Thus, the strategy is to solve the dual problem and then to use a root-finding algorithm on the certificate ${\mathcal{F}}_n^{\ast }p$ associated with the dual solution, hence reconstructing the support of the measure then the measure (after a last amplitude recover step). We present an example of the reconstruction of three Dirac measures on the $1D$ torus ${\mathbb{T}}^1$ through the observed noisy data y and the roots of the polynomial $p_{2n-2}\left(e^{2\mathrm{i}\pi t}\right)$ in Figure 2.

This strategy is only suitable for $d=1$. For the multi-variate case, one needs to make use of a so-called Lasserre Hierarchy [28]. Consider the semi-definite relaxation of order m with $m>n=2f_{\mathrm{c}}+1$:

$$\begin{array}{c}\hfill \underset{p\in {\mathbb{C}}^n,Q\in {\mathbb{C}}^{n\times n}}{max}\mathrm{Re}y,p\mathrm{constrained}\mathrm{by}\\ \hfill \left\{\begin{array}{c}0⪯\left(\begin{array}{cc}Q& \tilde{p}\\ {\tilde{p}}^{\ast }& 1\end{array}\right) \mathrm{where} {\tilde{p}}_k=\left\{\begin{array}{c}{c}_k\mathrm{if}k\in {[-f_{\mathrm{c}},f_{\mathrm{c}}]}^d\hfill \\ 0\mathrm{otherwise}\hfill \end{array}\right.\hfill \\ \mathrm{Tr}\left({\mathsf{\Theta }}_{k}Q\right)={\delta }_{0,k}\mathrm{with}k\in ⟦-m,m⟧\hfill \end{array}\right.\end{array}$$

with ${\mathsf{\Theta }}_k={\theta }_{k_d}\otimes \cdots \otimes {\theta }_{k_1}$, where ${\theta }_{k_j}$ and the entries of $m\times m$ elementary Toeplitz matrix are 1 on its $k_j$-th diagonal and 0 elsewhere, and ⊗ the Kronecker product. In a nutshell, Lasserre’s hierarchies give a sequence of nested outer SDP approximations of the cone of moments of non-negative measure. This method has been successfully applied to super-resolution in [12]. Some reconstructions in the 1D setting with a Fourier kernel are given in Figure 3, and the interested reader may find a more in-depth tutorial in the Numerical Tours on ‘Sparse spikes measures’ joined with the code used to compute the following figure (https://nbviewer.jupyter.org/github/gpeyre/numerical-tours/blob/master/matlab/sparsity_8_sparsespikes_measures.ipynb, consulted on 30 November 2021).

These methods are proved to be asymptotically exact [12]. Nonetheless, it is not known if the algorithm has finite convergence in general: one does not know when to stop the hierarchy to obtain a solution of the BLASSO [3]. This stems from the fact that non-negative trigonometric polynomials in dimension $d>1$ are not necessarily sums of square. Moreover, these SDP based approaches are rather limited to a certain class of measurement map $\Phi$, typically the Fourier forward operator or at least filters with compact Fourier supports. With the two following classes of algorithms, one can better exploit the continuous setting and get rid of the discretisation drawback.

Summary (1st algorithm): the scheme boils down to the resolution of the dual problem, the reconstruction of the measure’s support thanks to the certificate associated with the dual solution, and finally the solving of a linear problem to yield the corresponding estimated amplitudes. This strategy can be extended to a multivariate context, but, still, it is quite restrictive on the forward operator and it does not have finite convergence in general.

3.2. Greedy Algorithm: The Conditional Gradient

The conditional gradient method, also called the Frank–Wolfe (FW) algorithm [29,30], aims at solving ${min}_{m\in C}f\left(m\right)$ for C a weakly compact convex set of a topological vector space and f a convex and differentiable function (the differential is then denoted by df). It relies on the iterative minimisation of a linearised version of f. Hence, the interest of this algorithm lies in the fact that it uses only the directional derivatives of f and that it does not require any Hilbertian structure, contrary to a classic proximal algorithm formulated in terms of Euclidean distance. We recall the definition of the conditional gradient in the pseudocode Algorithm 2 for the general problem of minimising f:

Algorithm 2. Frank–Wolfe.

One can make the following remarks:

• the compactness assumption on C ensures that the argmin in step 2 is non-empty;
• in line 7, we can replace $m^{k+1}$ by any element $\widehat{m}\in C$ such that $f\left(\widehat{m}\right)\le f\left(m^{k+1}\right)$ without changing the convergence properties of the algorithm;

There are, however, two problems that prevent us from applying straightforwardly this algorithm to BLASSO: $T_{\lambda }$ is not differentiable, and the optimisation set $\mathcal{M}\left(\mathcal{X}\right)$ is not bounded. It is thus necessary to perform an epigraphical lift [13,31] to reach a differentiable functional that shares the same minimum measures as $T_{\lambda }$:

$$\begin{array}{}\left({\stackrel{\sim }{𝒫}}_{\lambda }\right(y\left)\right)& \hfill \underset{(r,m)\in C}{min}{\tilde{T}}_{\lambda }(m,r)\stackrel{\mathrm{def}.}{=}{\displaystyle \frac{1}{2}}{\Vert y-\Phi \left(m\right)\Vert }_{\mathcal{H}}^2+\lambda r\end{array}$$

with the bounded set $C=\left\{(r,m)\in {\mathbb{R}}^{+}\times \mathcal{M}\left(\mathcal{X}\right);\left|m\right|\left(\mathcal{X}\right)\le r\le M\right\}$ and $M\stackrel{\mathrm{def}.}{=}\frac{{\Vert y\Vert }^2}{2\lambda }$. Even though C is not weakly compact, it is compact for the weak-* topology and the hypotheses for Algorithm 2 are still matched. The Frank–Wolfe algorithm is then well-defined for the energy ${\tilde{T}}_{\lambda }$, differentiable in the Fréchet sense on the Banach $\mathbb{R}\times \mathcal{M}\left(\mathcal{X}\right)$. Its differential writes down:

$$\begin{array}{c}\hfill \mathrm{d}{\tilde{T}}_{\lambda }:(r^{\prime },m^{\prime })\mapsto {\int }_X{\Phi }^{\ast }(\Phi m-y)\mathrm{d}{m}^{\prime }+\lambda r^{\prime }.\end{array}$$

Finally, one has that $m^{\ast }$ is a minimum of $T_{\lambda }$ iff $\left(\right|m^{\ast }|\left(\mathcal{X}\right),m^{\ast })$ minimises $\left({\stackrel{\sim }{𝒫}}_{\lambda }\right(y\left)\right)$, and $T_{\lambda }\left(m^{\ast }\right)={\tilde{T}}_{\lambda }\left(|m^{\ast }|\left(\mathcal{X}\right),m^{\ast }\right)$. In the rest of the document, we will omit the r-part, and we will refer to the quantity $\left(\right|m^{\ast }|\left(\mathcal{X}\right),m^{\ast })$ by only $m^{\ast }$.

We note before that the update $m^{k+1}$ in line 7 can be replaced by any value $\widehat{m}$ improving the objective function; this remark is rather interesting as it can drastically improve the convergence property of the algorithm [11,32]. Hence, an interesting improvement to the Frank–Wolfe algorithm relies in the change of the final update step by a non-convex optimisation on both the amplitudes and the positions of the reconstructed $\delta$-peaks in a simultaneous fashion. This modification is presented in Algorithm 3.

Algorithm 3.Sliding Frank-Wolfe.

This tweak yields a theoretical convergence to the unique solution of BLASSO in a finite number of iterations, empirically a N-step convergence. This version is called the Sliding Frank–Wolfe algorithm [13], as the spike positions are sliding on the continuous domain $\mathcal{X}$. The authors also proved in the same paper that the generated measure sequence $m^{\left[k\right]}$ converges towards the minimum for the weak-* topology.

A reconstruction by Sliding Frank-Wolfe for the same Fourier operator, ground-truth spikes, and acquisition as the latter section is plotted in Figure 4. Contrary to SDP in Figure 3, no spurious spike is reconstructed. As in the SDP method, the choice of regulariser $\lambda$ is crucial: if chosen too high, it will yield a solution with fewer spikes than needed; if set too low, it will recover a solution with spurious spikes. We set $\lambda =1$ for the 1D Fourier example as in the former SDP section.

The line 3 in Algorithm 3 is typically solved by a grid search, the convex step in line 5 can use a FISTA solver [33], and the non-convex step in line 6 can be tackled by a modified Broyden–Fletcher–Goldfarb–Shann method (L-BFGS-B) implementation [34]. Reconstructions in the 2D setting with a convolution kernel, similar to the SMLM conditions, are presented in Figure 5. Since luminosity is always a non-negative quantity, one can restrict [4] the SFW to build a positive measure of the cone ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$, by changing:

• the stopping condition to ${\eta }^{\left[k\right]}\left(x_{\ast }^{\left[k\right]}\right)\le 1$;
• the LASSO step is solved for $a\in {\mathbb{R}}_{+}^{N^{\left[k\right]+1}}$;
• the non-convex step is solved on ${\mathbb{R}}_{+}^{N^{\left[k\right]+1}}\times {\mathcal{X}}^{N^{\left[k\right]+1}}$.

Hence, this modified algorithm offers a good trade-off between precision and theoretical guarantees. However, it suffers from the high computation load for one iteration, making it slow to compute. The next section algorithm is a promising alternative with easier/cheaper iteration while still taking advantage of the continuous setting.

Shorthand (2nd algorithm): conditional gradient method is a greedy algorithm consisting in the iterative minimisation of a linearised version of the objective convex function. This algorithm can be applied to any forward operator without restriction on the space $\mathcal{X}$. Up to a modification (SFW), the Frank–Wolfe algorithm reaches a finite convergence, empirically a N-step convergence for a source measure with N spikes. The iterations, however, are computationally costly, yielding long computation time.

3.3. Optimal Transport Based Algorithm: The Particle Gradient Descent

All the following results are proven for a domain $\mathcal{X}$ with no boundaries, e.g., the d-dimensional torus ${\mathbb{T}}^d$. The case described in the former sections—$\mathcal{X}$ is any compact of ${\mathbb{R}}^d$—is included in this new setting, since any compact $\mathcal{X}$ can be periodised to yield a domain with no boundaries. The forward operator kernel $\phi :\mathcal{X}\to \mathcal{H}$ should also be differentiable in the Fréchet sense. The least squares term in BLASSO is denoted by the more general data term $R:\mathcal{H}\to {\mathbb{R}}^{+}$, the functional $T_{\lambda }$ of the BLASSO will now be restricted to ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$ and denoted J; its Fréchet differential at point $\nu \in {\mathcal{M}}^{+}\left(\mathcal{X}\right)$ is denoted $J_{\nu }^{\prime }$:

$$\begin{array}{cc}\hfill J\left(\nu \right)& ={\Vert y-\Phi \nu \Vert }_{\mathcal{H}}^2+\lambda \left|\nu \right|\left(\mathcal{X}\right),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill J_{\nu }^{\prime }\left(x\right)& ={〈\phi \nabla \left(x\right),R〉}_{\mathcal{H}}+\lambda \mathrm{for}\mathrm{all}x\in \mathcal{X}\hfill \end{array}$$

(6)

A comprehensive guide on its computation is given in Appendix B.4. In the following, we describe the setting for non-negative measures of ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$, but it can be extended in a straightforward fashion [23] to signed measures of $\mathcal{M}\left(\mathcal{X}\right)$ by performing the method on the positive then negative part of the signed measure (see Jordan decomposition in Appendix B.1). Figure 6 sums up the chief quantities and relations introduced in this section, the reader is advised to refer to it whenever he or she needs a global view on the optimal transport problem.

Sparse optimisation on measures through optimal transport [3,23] relies on the approximation of the ground-truth positive measure $m_{a_0,x_0}$ by a ‘system of $N\in {\mathbb{N}}^{\ast }$ particles’, i.e., an element of the space ${\mathsf{\Omega }}^N\stackrel{\mathrm{def}.}{=}{({\mathbb{R}}^{+}\times \mathcal{X})}^N$. The point is then to estimate the ground-truth measure by a gradient-based optimisation on the objective function:

$$\begin{array}{c}\hfill F_N((r_1,x_1),\cdots ,(r_N,x_N))\stackrel{\mathrm{def}.}{=}{\displaystyle \Vert }y-\frac{1}{N}\sum _{i=1}^N{r_i}^2\phi \left(x_i\right){\Vert }_{\mathcal{H}}^2+\frac{\lambda }{N}{r_i}^2\end{array}$$

(7)

where $(r_i,x_i)$ belongs to the lifted space $\mathsf{\Omega }\stackrel{\mathrm{def}.}{=}{\mathbb{R}}^{+}\times \mathcal{X}$ endowed with a metric. Hence, the hope is that the gradient descent on $F_N$ converges to the amplitudes and the positions of the ground-truth measure, despite the non-convexity of functional (7). The author of [23] proposes the definition of a suitable metric for the gradient of $F_N$, which enables separation of the variables in the gradient descent update. Let $\alpha ,\beta$ be two parameters such that $\alpha >0$ and $\beta >0$ and for any $(r,\theta )\in \mathsf{\Omega }$, we define the Riemannian inner product of $\mathsf{\Omega }$ called the cone metric endowing $\mathsf{\Omega }$ as defined by $\forall (\delta r_1,\delta r_2)\in {\mathbb{R}}_{+}^2$, $\forall (\delta {\theta }_1,\delta {\theta }_2)\in {\mathcal{X}}^2$:

$$\begin{array}{c}\hfill {\langle (\delta r_1,\delta {\theta }_1),(\delta r_2,\delta {\theta }_2)\rangle }_{(r,\theta )}\stackrel{\mathrm{def}.}{=}{\displaystyle \frac{\delta r_1\delta r_2}{\alpha }}+r^2{\displaystyle \frac{{〈\delta {\theta }_1,\delta {\theta }_2〉}_{\theta }}{\beta }}.\end{array}$$

We denote by ${〈·,·〉}_{\theta }$ the metric on the manifold $\mathcal{X}$ at the point $\theta$. For the gradient of the functional $F_N$ for all $i\in ⟦1,N⟧$ w.r.t.; the cone metric writes down [2,23]:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\nabla }_{r_i}{F}_N=2\alpha r_i{J}_{\nu }^{\prime }\left(x_i\right)=-2\alpha r_i\lambda ({\eta }_{\lambda }-1)\hfill \\ {\nabla }_{x_i}{F}_N=\beta \lambda \nabla J_{\nu }^{\prime }\left(x_i\right)=-\beta \lambda \nabla {\eta }_{\lambda }\hfill \end{array}\right.\mathrm{for}\nu \stackrel{\mathrm{def}.}{=}\sum _{i=1}^N{r_i}^2{\delta }_{x_i},{\eta }_{\lambda }\stackrel{\mathrm{def}.}{=}-J_{\nu }^{\prime }/\lambda .\end{array}$$

(8)

See Appendix B.4 for more details on this computation. We now present the theoretical results on the particle gradient descent, which corresponds to the blue dashed lines in Figure 6. The reader is invited to refer to this figure any time he needs to get a hold on the broader picture.

3.3.1. Theoretical Results

The main idea of these papers [3,23] boils down to the following observation: the minimisation of function (7) is a peculiar case of a more general problem, formulated in terms of measure of the lifted space $\mathsf{\Omega }$. The space is more precisely ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ subset of $\mathcal{M}\left(\mathsf{\Omega }\right)$, namely the space of probabilities with finite second moments endowed with the 2-Wasserstein metric i.e., the optimal transport distance: see Appendix B.5 for more details. Hence, the lift of the unknown $m\in {\mathcal{M}}^{+}\left(\mathcal{X}\right)$ to $\mu \in {\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ enables the removal of the asymmetry for discrete measures between position $x\in \mathcal{X}$ and amplitude $a\in {\mathbb{R}}^{+}$ by lifting $a{\delta }_x$ to ${\delta }_{(a,x)}$. The lifted functional now writes down for parameter $\lambda >0$:

$$\begin{array}{c}\hfill \forall \mu \in {\mathcal{P}}_2\left(\mathsf{\Omega }\right),F\left(\mu \right)\stackrel{\mathrm{def}.}{=}{\Vert y-\tilde{\Phi }\mu \Vert }_{\mathcal{H}}^2+\lambda \tilde{V}\left(\mu \right)\end{array}$$

(9)

where $\tilde{\Phi }\mu \stackrel{\mathrm{def}.}{=}{\int }_{\mathsf{\Omega }}\varphi (a,x)\mu (a,x)$ for $\varphi (a,x)\stackrel{\mathrm{def}.}{=}a\phi \left(x\right)$ and $\tilde{V}$ is the TV-norm on the spatial component of the measure $\mu$. The functional is non-convex, its Fréchet differential is denoted $F^{\prime }$, and for $u\in \mathsf{\Omega }$:

$$\begin{array}{c}\hfill F^{\prime }\left(\mu \right)\left(u\right)\stackrel{\mathrm{def}.}{=}{〈{\tilde{R}}^{\prime }\left(\mu \right),\varphi \left(u\right)〉}_{\mathcal{H}}+\lambda \end{array}$$

with ${\tilde{R}}^{\prime }\stackrel{\mathrm{def}.}{=}{\Vert y-{\int }_{\mathsf{\Omega }}\nabla \varphi (a,x)\mu (a,x)\Vert }_{\mathcal{H}}^2$. Then, a discrete measure ${\mu }_N\stackrel{\mathrm{def}.}{=}\frac{1}{N}{\sum }_i^N{\delta }_{a_i,x_i}$ of ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ can also be seen as an element of ${\mathsf{\Omega }}^N$ from the standpoint of its components $(a_i,x_i)$. It allows the authors of [3,23] to perform a precise characterisation of the source recovery conditions, through the measures and the tools of optimal transport such as gradient flow (see below).

Then, one may run a gradient descent on the amplitudes and positions $(a_i,x_i)\in {({\mathbb{R}}^{+}\times \mathcal{X})}^N$ of the measure ${\mu }_N$, in order to exploit the differentiability of the kernel $\phi$. Note that the measure ${\mu }_N$ is over-parametrized, i.e., its number of $\delta$-peaks is larger compared to the number of spikes of the ground-truth measure: thus, the particles, namely the $\delta$-peaks of the space $\mathsf{\Omega }$, are covering the domain $\mathcal{X}$ for their spatial part.as an example, where ${\mu }_N$ is plotted in red dots.

Before giving the main results, we need to clarify the generalised notion of gradient descent to measure function called the gradient flow [35,36] from optimal transport theory, the main ingredient in the particle gradient descent. Letting $F:{\mathbb{R}}^d\to \mathbb{R}$ be the objective function with certain regularity, a gradient flow describes the evolution of a curve $x\left(t\right)$ such that its starting point at $t=0$ is $x_0\in {\mathbb{R}}^d$, evolving by choosing at any time t in the direction that decreases the function F the most [36]:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{x}^{\prime }\left(t\right)=-\nabla F\left(x\left(t\right)\right)\mathrm{pour}t>0\hfill \\ x\left(0\right)=x_0.\hfill \end{array}\right.\end{array}$$

The interest of gradient flow is its extension to spaces X with no differentiable structure. In the differentiable case, one can consider the discretisation of the gradient flow i.e., the sequence defined for a step-size $\tau >0$, $k\in {\mathbb{N}}^{\ast }$:

$$\begin{array}{c}\hfill x_{k+1}^{\tau }\in \underset{x\in X}{argmin}F\left(x\right)+{\displaystyle \frac{|x-x_k^{\tau }{|}^2}{2\tau }}.\end{array}$$

It is the implicit Euler scheme for the equation ${\left(x^{\tau }\right)}^{\prime }=-\nabla F\left(x^{\tau }\right)$, or the weaker ${\left(x^{\tau }\right)}^{\prime }\in \partial F\left(x^{\tau }\right)$ if F is convex and non-smooth. The gradient flow is then the limit (under certain hypotheses) of the sequence ${\left(x_k^{\tau }\right)}_{k\ge 0}$ for $\tau \to 0$ for a starting point $x_0\in X$. Gradient flow can be extended to metric space: indeed, for a metric space $(X,d)$ and a map $F:X\to \mathbb{R}$ lower semi-continuous one can define the discretisation of gradient flow by the sequence

$$\begin{array}{c}\hfill x_{k+1}\in \underset{x\in X}{argmin}F\left(x\right)+{\displaystyle \frac{d{(x,x_k)}^2}{2\tau }}.\end{array}$$

(10)

In the case of the metric space of probability measures i.e., the measures with unitary mass, the limit $\tau \to 0$ of the scheme exists and converges to the unique gradient flow starting at $x_0$ element of the metric space. A typical case is the space of probabilities with finite second moments ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$, endowed with 2-Wasserstein metric, i.e., the optimal transport distance (see Appendix B.5): a gradient flow in this space ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ is a curve $t\mapsto {\mu }_t$ called a Wasserstein gradient flow starting at ${\mu }_0\in {\mathcal{P}}_2\left(\mathsf{\Omega }\right)$, for all $t\in {\mathbb{R}}^{+}$, one has ${\mu }_t\in {\mathcal{P}}_2\left(\mathsf{\Omega }\right)$, obeying the partial differential equation in the sense of distributions:

$$\begin{array}{c}\hfill {\partial }_t{\mu }_t=-div({\mu }_t\nabla F^{\prime }\left({\mu }_t\right)).\end{array}$$

(11)

Recall that $div\left(m\right)={\sum }_{i=1}^d\frac{\partial m}{\partial x_d}$ for all $m\in \mathcal{M}\left(\mathcal{X}\right)$, derivatives ought to be understood in the distributional sense. This equation ensures the conservation of the mass, namely, at each time $t>0$, one has $|{\mu }_t|\left(\mathsf{\Omega }\right)=|{\mu }_0|\left(\mathsf{\Omega }\right)$. Hence, despite the lack of differentiability structure of ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ which forbids straightforward application of a classical gradient-based algorithm, one can perform an optimisation on the space through gradient flow to reach a minimum of F by discretizing (11).

The interesting case of a gradient flow in ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ is the flow starting at ${\mu }_{N,0}\stackrel{\mathrm{def}.}{=}1/N{\sum }_{i=1}^N{\delta }_{(a_i^0,x_i^0)}$, uniquely defined by Equation (11), which writes down for all $t\in {\mathbb{R}}^{+}$: ${\mu }_{N,t}\stackrel{\mathrm{def}.}{=}1/N{\sum }_{i=1}^N{\delta }_{(a_i\left(t\right),x_i\left(t\right))}$, where $a_i:{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$ and $x_i:{\mathbb{R}}^{+}\to \mathcal{X}$ are continuous maps. This path ${\left({\mu }_{N,t}\right)}_{t\ge 0}$ is a Wasserstein gradient flow, and uses N Dirac measures over $\mathsf{\Omega }$ to optimise the objective function F in (9). When the number of particles N goes to infinity and if ${\mu }_{N,0}$ converges to some ${\mu }_0\in {\mathcal{P}}_2\left(\mathcal{X}\right)$, the gradient flow ${\left({\mu }_{N,t}\right)}_{t\ge 0}$ converges to the unique Wasserstein gradient flow of F starting from ${\mu }_0$, described by the time-dependent density ${\left({\mu }_t\right)}_{t\ge 0}$ valued in ${\mathcal{P}}_2\left(\mathcal{X}\right)$ obeying the latter partial differential Equation (11).

For these non-convex gradient flows, the authors of [3] give a consistent result for gradient based optimisation methods: under a certain hypothesis, the gradient flow ${\left({\mu }_{N,t}\right)}_{t\ge 0}$ converges to global minima in the over-parametrization limit i.e., for $N\to +\infty$. It relies on two important assumptions that prevent the optimisation from being blocked in non-optimal points:

• homogeneity (A function f between vector spaces is positively p-homogeneous if, for $\lambda >0$ and argument x, one has $f\left(\lambda x\right)={\lambda }^{d}f\left(x\right)$.) of $\varphi$ in order to select the correct magnitude for each feature, or at least partially 1-homogeneity (i.e., boundedness of $\phi$ in [3]);
• diversity in the initialisation of parameters, in order to explore all combinations of features. Too few or too close particles will not reach all source peaks and will only yield local minima.

We can then introduce the fundamental result for the many particle limits [3], the mean-field limits of gradient flows ${\left({\mu }_{N,t}\right)}_{t\ge 0}$, despite the lack of convexity of these gradient flows:

Theorem 2

(Global convergence—informal).If the initialisation ${\mu }_{N,0}$ is such that ${\mu }_0\stackrel{\mathrm{def}.}{=}{lim}_{N\to +\infty }{\mu }_{N,0}$ support separates (The support of a measure m is the complement of the largest open set on which m vanishes. In an ambient space $\mathcal{X}$, we say that a set C separates the sets A and B if any continuous path in $\mathcal{X}$ with endpoints in A and B intersects C.) $\{-\infty \}\times \mathcal{X}$ from $\{+\infty \}\times \mathcal{X}$ then the gradient flow ${\mu }_t$ weakly-* (see Appendix B.1) converges in ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$ to a global minimum of F, and we also have:

$$\begin{array}{cc}\hfill \underset{N,t\to \infty }{lim}F\left({\mu }_{N,t}\right)& =\underset{m\in {\mathcal{M}}^{+}\left(\mathcal{X}\right)}{min}J\left(m\right).\hfill \end{array}$$

Limits can be interchanged; the interested reader might take a look at [3] for precise statements and exact hypothesis (boundary conditions, ‘Sard-type’ regularity e.g., $\phi$ is d-times continuously differentiable, etc).

Since we have a convergence result, we can then investigate the numerical implementation. This optimisation problem is tractable thanks to the Conic Particle Gradient Descent algorithm [23] denoted CPGD: the proposed framework involves a slightly different gradient flow ${\left({\nu }_t\right)}_{t\ge 0}$ defined through a projection of ${\left({\mu }_t\right)}_{t\ge 0}$ onto ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$. This new gradient flow ${\left({\nu }_t\right)}_{t\ge 0}$ is defined for a specific metric in ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$, which is now a trade-off between Wasserstein and Fisher–Rao (also called Hellinger metric.) metric [23], it is then called a Wasserstein–Fisher–Rao gradient flow. Then, the Wasserstein–Fisher–Rao gradient flow starting at ${\nu }_{N,0}\stackrel{\mathrm{def}.}{=}{\sum }_{i=1}^N{a}_i^0{\delta }_{x_i^0}$ in ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$ writes down $t\mapsto {\nu }_{N,t}\stackrel{\mathrm{def}.}{=}\frac{1}{N}{\sum }_{i=1}^N{r_i\left(t\right)}^2{\delta }_{x_i\left(t\right)}$ in ${\mathcal{M}}^{+}\left(\mathcal{X}\right)$, rather than the Wasserstein flow $t\mapsto {\mu }_{N,t}\stackrel{\mathrm{def}.}{=}\frac{1}{N}{\sum }_{i=1}^N{\delta }_{r_i\left(t\right),x_i\left(t\right)}$ starting at ${\mu }_{N,0}$ in ${\mathcal{P}}_2\left(\mathsf{\Omega }\right)$. The partial differential equation of a Wasserstein–Fisher–Rao flow writes down:

$$\begin{array}{c}\hfill {\partial }_t{\nu }_t=-4\alpha {\nu }_t{T}_{\lambda }\left({\nu }_t\right)+\beta div({\nu }_t\nabla J_{\nu }^{\prime }\left({\nu }_t\right))\end{array}$$

(12)

for the two parameters $\alpha ,\beta >0$ arising from the cone metric, $\alpha$ tunes the Fisher–Rao metric weight, while $\beta$ tunes the Wasserstein metric one. All statements on convergence could be made alternatively on ${\mu }_t$ or ${\nu }_t$, and we have indeed the same theorem:

Theorem 3

(Global convergence—informal).If ${\nu }_0$ has full support (its support is the whole set $\mathcal{X}$) and ${\left({\nu }_t\right)}_{t\ge 0}$ converges for $t\to +\infty$, then the limit is a global minimum of J. If ${\nu }_{N,0}\underset{N\to +\infty }{\to }{\nu }_0$ in the weak-* sense, then:

$$\begin{array}{cc}\hfill \underset{N,t\to \infty }{lim}J\left({\nu }_{N,t}\right)& =\underset{m\in {\mathcal{M}}^{+}\left(\mathcal{X}\right)}{min}J\left(m\right).\hfill \end{array}$$

Summary (3rd algorithm theoretical aspects): we introduced the proposed solution of [3,23], namely approximating the source measure by a discrete non-convex objective function of amplitudes and positions. The analytical study of the discrete function is an uphill problem and could be tackled thanks to the recast of the problem in the space of measures. Then, we exhibited the theoretical framework on gradient flows, understood in the sense of generalisation of gradient descent in the space of measures. Eventually, we presented the convergence results of the gradient flow denoted ${\left({\nu }_t\right)}_t$ towards the minimum of the BLASSO, thus enabling results for the convergence. Gradient descent on the discrete objective approximates well the gradient flow dynamic and can then benefit from the convergence results exhibited before.

We now discuss the numerical results of the particle gradient descent. The reader is advised to take a look at Figure 6, more precisely at red and green ellipses, to get a grasp on the numerical part.

3.3.2. Numerical Results

We recall that a gradient flow ${\left({\nu }_{N,t}\right)}_{t\ge 0}$ starting at $\stackrel{\mathrm{def}.}{=}1/N{\sum }_{i=1}^N{\left(r_i^{\left(0\right)}\right)}^2{\delta }_{x_i^{\left(0\right)}}$ can be seen as a (time continuous) generalisation of gradient descent in the space of measures, allowing precise theoretical statements on the recovery conditions. To approach this gradient flow, we use the Conic Particle Gradient Descent algorithm [23] denoted CPGD: the point is to discretise the evolution of the gradient flow $t\mapsto {\nu }_{N,t}$ through a numerical scheme on (12). This consists of a gradient descent on the amplitudes r and positions x through the gradient of the functional $F_N$ in Equation (8), a strategy which approximates well the dynamic of the gradient flow [23].

This choice of gradient with the cone metric enables multiplicative updates in r and additive in x, the two updates being independent of each other. Then, the algorithm consists of a gradient descent with the definition of $r_i^{\prime }\left(t\right)$ and $x_i^{\prime }\left(t\right)$ according to [2,23]:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{r}_i^{\prime }\left(t\right)=-2\alpha r_i\lambda ({\eta }_{\lambda }\left(x_i\left(t\right)\right)-1)\hfill \\ x_i^{\prime }\left(t\right)=-\beta \lambda \nabla {\eta }_{\lambda }\left(x_i\left(t\right)\right)\hfill \end{array}\right.\end{array}$$

(13)

thanks to a gradient in Equation (8), for the mirror retraction (The notion of retraction compatible with cone structure is central: in the Riemann context, a retraction is a continuous mapping that maps a tangent vector to a point on the manifold. Formally, one could see it as a way to enforce the gradient evaluation to be mapped on the manifold. See [23] for other choices of compatible retractions and more insights on these notions.) and ${\eta }_{\lambda }=-J_{\nu }^{\prime }/\lambda$. The structure of the CPGD is presented in Algorithm 4. Note that the multiplicative updates in r yields an exponential of the certificate, and that the updates of the quantities $r,x$ are separated.

This algorithm has rather easy and cheap iterations: to reach an accuracy of $\epsilon$—i.e., a distance such as the ∞-Wasserstein distance between the source measure $m_{a_0,x_0}$ and the reconstructed measure $m^{\ast }$ is below $\epsilon$—the CPGD yields a typical complexity cost of $log\left({\epsilon }^{-1}\right)$ rather than ${\epsilon }^{-1/2}$ for convex program ([23] Theorem 4.2). A reconstruction from the latter 1D Fourier measurements is plotted in Figure 7, the reconstruction is obtained through two gradient flows, the former on the positive measures to recover the positive $\delta$-peaks of the ground-truth and the latter on the negative measures to recover the negative one: the merging of the two results gives the reconstructed $\delta$-peaks. The noiseless reconstruction (See our GitHub repository for our implementation: https://github.com/XeBasTeX, accessed on 30 November 2021) for 2D Gaussian convolution with the same setting as the Frank–Wolfe section is plotted in Figure 8. One can see that the spikes are well-recovered as some non-zero red and blue particles cluster around the three $\delta$-peaks.

Algorithm 4. Conic particle gradient descent algorithm.

Summary (3rd algorithm numerical aspects): the gradient flow ${\left({\nu }_t\right)}_t$ is computable by the Conic Particle Gradient Descent algorithm, consisting in an estimation through a gradient (w.r.t. cone metric) descent on both amplitudes and positions of an over-parametrised measure, namely a measure with a fixed number of $\delta$-peaks exceeding the source’s one. The iterations are cheaper than the SFW presented before, but the CPGD lacks guarantees in a low-noise regime.

To sum up all the pros and cons of these algorithms, we give

Table 1. Pros and cons for the different off-the-grid algorithm strategies, Semi-definite programming (SDP) vs. Sliding Frank-Wolfe (SFW) algorithm vs. Conic Particle Gradient Descent (CPGD).

Algorithm	Operator	Space $\mathcal{X}$	Convergence Rate	Computation Time	Tuning Parameters
SDP [10]	Fourier	Torus ${\mathbb{T}}^d$	Asymptotic	Mild	$\lambda$
SFW [11,13]	All	Any compact	Sublinear	Long	$\lambda$
CPGD [23]	All	Torus ${\mathbb{T}}^d$	$log\left(\epsilon \right)$	Quick	$\lambda ,\alpha ,\beta$

for a quick digest. Since the CPGD lacks guarantees on the global optimality of its output, the following section will use the conditional gradient and more precisely the Sliding Frank-Wolfe in order to tackle the SMLM super-resolution problem.

4. Applications and Results in SMLM Imaging

As an illustration of off-the-grid applications’ in this review, we propose to solve the super-resolution problem, aiming to retrieve biological structures at very small scales.

4.1. Metrics of Quality of Reconstruction

If one has access to the ground-truth i.e., the real position of the point sources, one is able to assess the quality of the reconstruction by:

• detection metrics, such as the Jaccard index;
• quality of reconstruction metrics, such as the ${\mathrm{L}}^2$ norm in the discrete case.

Detection metrics can be applied to the off-the-grid output in a straightforward manner. We will rather focus in this part on the ‘quality of reconstruction’ metric. Any of the former algorithms returns a list of Dirac measures, which can be compared with the ground-truth measure $m_{a_0,x_0}$. This comparison cannot be done with discrete tools, such as the ${\mathrm{L}}^2$ norm of the reconstructed acquisitions: we cannot compare an element of $\mathcal{M}\left(\mathcal{X}\right)$ with ${\mathrm{L}}^2\left(\mathcal{X}\right)$. Examining the ${\mathrm{L}}^2$ norm of the $x_i$ vector of reconstructed positions against the $x_{0,i}$ vector is not sufficient either: we need the same number of elements for x and $x_0$, we have to sort the vector of positions, and we have no guarantee that the matching of one position of x with another of $x_0$ is the right one.

Hence, a distance on the measure space is the good tool of comparison. We will use in the following the Wasserstein 1-distance ${\mathcal{W}}_1$ [37]: see Appendix B.5 for some recall on the useful definition and more insights on the optimal transport setting used in this section. The Wasserstein distance with measures of equal mass is defined (Actually, it is well-defined on the subset $X\stackrel{\mathrm{def}.}{=}\left\{m\in \mathcal{M}\left(\mathcal{X}\right),\left|m\right|\left(\mathcal{X}\right)\le y_{\mathcal{H}}^2/2\lambda \right\}$ because only the bounded subset of $\mathcal{M}\left(\mathcal{X}\right)$ are metrizable for the weak-* topology, so we have to restrain the set of measures to X in order to reach a Polish space i.e., the convenient framework for this OT-based metric, see Appendix B.5. Since all solutions of the BLASSO belong to X [8], we will keep this slight abuse of notation in the rest of the paper.) as:

Definition 7

(Balanced optimal transport).For $0\le p<+\infty$ and $m_1,m_2\in \mathcal{M}\left(\mathcal{X}\right)$ such that $|m_1|\left(\mathcal{X}\right)=|m_2|\left(\mathcal{X}\right)$, the p-Wasserstein distance is written:

$$\begin{array}{c}\hfill {\mathcal{W}}_p(m_1,m_2)\stackrel{\mathrm{def}.}{=}{\left(\underset{\gamma \in \Gamma (m_1,m_2)}{min}{\int }_{\mathcal{X}\times \mathcal{X}}{\left|u-v\right|}^p\mathrm{d}\gamma (u,v)\right)}^{1/p}.\end{array}$$

(14)

$\Gamma (m_1,m_2)$ is the set of transport plans between $m_1$ and $m_2$, one can take a look at Appendix B.5 for more insights on this notion. However, this notion is not sufficient for our application since the metric can only take measures of equal masses (i.e., equal TV-norm) as an input. In the case of a discrete measure, we recall that mass is simply the sum of the modulus of individual amplitudes: hence, in general, we cannot compare a source measure and a reconstructed measure with differing amplitudes. The classic solution is then to distribute the unit mass, divided by the number of spikes, uniformly over each $\delta$-peak of the discrete measure. Still, it would be way more convenient to incorporate the case of differing masses in the metric. The proper metric to compare two measures of different masses is called the Kantorovtich–Rubinstein norm also referred as the Flat Metric [37,38,39].

Definition 8

(Unbalanced optimal transport).Let us denote $m\in \mathcal{M}\left(\mathcal{X}\right)$ of finite first moment and $\tau >0$, and the following quantity is called Kantorovtich–Rubinstein norm:

$$\begin{array}{c}\hfill F_{\tau }\left(m\right)\stackrel{\mathrm{def}.}{=}\underset{f\in {\mathcal{C}}_{\mathrm{b}}\left(\mathcal{X}\right)}{sup}\left({\int }_{\mathcal{X}}f\mathrm{d}m,{\Vert f\Vert }_{\infty ,\mathcal{X}}\le \tau ,f\mathrm{Lipschitz},{\Vert f\Vert }_{\mathrm{Lip}}\le 1\right)\end{array}$$

where ${\Vert f\Vert }_{\mathrm{Lip}}$ is the Lipschitz constant of f. We then define the Flat Metric $d_{\tau }$ for $m_1,m_2\in \mathcal{M}\left(\mathcal{X}\right)$ of finite first moments:

$$\begin{array}{c}\hfill d_{\tau }(m_1,m_2)\stackrel{\mathrm{def}.}{=}{F}_{\tau }(m_1-m_2).\end{array}$$

The parameter $\tau$ is homogeneous to a distance, and it is understood in the optimal transport sense as the cost of creating/destroying a Dirac measure. The Flat Metric coincides with the 1-Wasserstein distance, for $m_1$, $m_2$ of equal masses, when $\tau \to +\infty$ [38]; it also coincides with the total variation norm of $m_1-m_2$ when $\tau \to 0$. Then, it may be seen as an interpolation between the total variation norm and the 1-Wasserstein norm. Moreover, when the number of $\delta$-peaks is correctly estimated, the Flat Metric stands for the mean error in terms of localisation and is similar to the RMSE [39]. Eventually, the Flat metric can be extended to discrete reconstruction i.e., images on a fine grid; this metric is then a method applicable to discrete reconstruction, namely images with a finer grid.

To sum up, there are two possibilities if one wants to compare the reconstructed measure and the ground-truth one:

• let the source measure be composed of N spikes, we set the amplitude of each $\delta$-peak at $1/N$. We apply the same procedure to the reconstructed (with differing or not number of spikes), hence dividing uniformly the unit mass over all the $\delta$-peaks of the considered measure. Therefore, the reconstructed luminosity is not considered as relevant and discarded: we can compute directly the 1-Wasserstein distance, since it is equal to the Flat Metric in this case;
• we want to account for the luminosity, and we use the Flat Metric to compare the reconstructed measure against the ground-truth one.

Summary: classic quality of reconstruction metrics such as the ${\mathrm{L}}^2\left(\mathcal{X}\right)$ norm cannot be straightforwardly applied to off-the-grid reconstruction. Instead, one could use optimal transport score such as the Flat Metric: it accounts for both amplitude and position reconstructions, while it can be easily extended to discrete reconstruction (images on a fine grid).

4.2. Results for an SMLM Stack

In super-resolution for biomedical imaging, one wants to retrieve some fine scale details to better study biological structures of interest. Indeed, the studied bodies are generally smaller than the Rayleigh limit at 200 nm, a length at which the phenomenon of light diffraction comes into play. This diffraction causes a blurring of the image, which can be described as a convolution of the image by the PSF mentioned above. Hence, we want to perform a deconvolution i.e., remove the blur of diffraction to get a super-resolved image. It is worth noticing that other imaging systems exist, for which the inverse problems to solve are a bit different from deconvolution: e.g., Nuclear Magnetic Resonance spectroscopy with Fourier measurements [40], MA-TIRF with Laplace [13].

In order to enhance spatial resolution over standard diffraction-limited microscopy techniques and allow imaging of biological structures below the Rayleigh criterion, one can use SMLM, which stands for ‘Single Molecule Localisation Microscopy’. It is a compelling technique in fluorescence microscopy to tackle the super-resolution problem [41]. It requires photoactivable fluorophores with, roughly speaking two states, for example ‘On’ and ‘Off’. These molecules are therefore only visible on the acquisitions in the ‘On’ case, and the idea is then to light up some molecules in the sample to make the acquisition and to be able to locate them precisely; the fluorescent molecules are bound to the biological structure and, since only a few molecules are emitting in one frame, the resulting image is rather sparse, which allows accurate localisation. This process is repeated until all the molecules have been lit and imaged. All the positions of the imaged molecules frame-by-frame can then be put together to form a super-resolved image that go below the diffraction barrier, ridden of the degradation by the process of acquisition (blur, noise, etc.). The quality of the image reconstruction is naturally limited by the number of acquisitions necessary to reconstruct the image, which implies a cost in time (precious insofar as the organism studied moves) and in physical memory and by the density of fluorophores lit at each stage. Indeed, there is a risk of overlap hindering the localisation of the molecules since the separation criterion is not matched.

Off-the-grid methods can be applied to any SMLM stack with only the knowledge of the forward operator, the acquisition system’s PSF in this case. In this review, a gridless method based on Sliding Frank–Wolfe is tested on an 2D SMLM acquisition stack from the 2013 EPFL Challenge (https://srm.epfl.ch/DatasetPage?name=MT0.N1.HD, accessed on 30 November 2021). For this purpose, we consider the first image of the stack, locate the source points, and store the coordinates of these points. Then, we move on to the second image, we locate the source points, and so on. Note that off-the-grid method with this variational approach is not the only method taking advantage of a continuous domain like the PSF-fitting such as DAOSTORM [42], etc.

Deconvolution is a first challenge to solve this inverse problem, but we must also take into account the noise. One has to deal with three main types of noise on these acquisitions:

• photon noise (also known as shot noise or quantum noise) is due to the quantum nature of light. It arises from the fact that fluorophores emit photons randomly, so that, between t and $t+1$ (exposure time), a variable number of photons have been emitted, and therefore a variable number of photons have been collected by the sensor. Thus, the amplitude of the electrical signal generated in the sensor (at each pixel) fluctuates according to a Poisson statistic;
• the dark current is a phenomenon due to the natural agitation of electrons. This natural agitation is sufficient to occasionally eject an electron from the valence band to the conduction band without any photoelectric effect. Additional charges are therefore created which interfere with the signal. The number of electrons generated by thermal agitation follows a Poisson distribution;
• amplification and readout noise. This noise is produced by the electronic circuit that amplifies and converts the electron packets into voltage. It is generally modelled by a Gaussian noise.

Thus, we have several noises that pollute each of the observed images. To deal with this ill-posed inverse problem, we use the results on BLASSO, with the least-squares term as the data-fitting term and the TV norm as the regulariser of the inverse problem. In the Bayesian approach, the least-square term is modelling the maximum of likelihood when the acquisition is polluted by Gaussian noise, hence our model is making the approximation of Gaussian noise. Measurements are discrete so at each image one has to deal with images with $N_1\times N_2$ pixels, each of them with size $(b_1,b_2)$. Let $(c_{i,1},c_{i,2})$ be the centre of the ith pixel, and we denote the ith camera pixels by

$$\begin{array}{c}\hfill {\mathsf{\Omega }}_i\stackrel{\mathrm{def}.}{=}(c_{i,1},c_{i,2})+\left[-\frac{b_1}{2N_1},\frac{b_1}{2N_1}\right]\times \left[-\frac{b_2}{2N_2},\frac{b_2}{2N_2}\right].\end{array}$$

We can then clarify the forward operator $\Phi :m\mapsto {\mathbb{R}}^{N_1{N}_2}$ which encapsulates the integration over camera pixels [13]; indeed, with the evaluation of the discrete Gaussian kernel $\phi$ with standard deviations $\sigma$, for $i\in \{1,\cdots ,N_1{N}_2\}$:

$$\begin{array}{c}\hfill {\left[\phi \left(x\right)\right]}_i\stackrel{\mathrm{def}.}{=}{\displaystyle \frac{1}{2\pi {\sigma }^2}}{\int }_{{\mathsf{\Omega }}_i}{e}^{-\left(\frac{{(x_1-s_1)}^2}{2{\sigma }^2}+\frac{{(x_2-s_2)}^2}{2{\sigma }^2}\right)}\mathrm{d}{s}_1{s}_2.\end{array}$$

In the SMLM data set, one has the PSF standard deviation $\sigma =149.39\mathrm{nm}$ and $N_1=N_2=100\mathrm{nm}$. The reconstruction is performed by our implementation of the Sliding Frank–Wolfe in python (See our GitHub repository for our PyTorch implementation: https://github.com/XeBasTeX, accessed on 30 November 2021) insofar as it is the more robust method available: indeed, it works with Gaussian kernel, and it has proven results in a noise regime, etc. The results are presented in Figure 9. The stack of 2500 images of $64\times 64$ is qualified as high density with high SNR: the number of activated fluorophores is quite important, and the noise is not negligible (see the EPFL Challenge page for more insights.).

A flat metric between the reconstructed measure $m_{a,x}$ and the ground-truth measure $m_{a_0,x_0}$ is then computed, and it reaches $d_{\tau }(m_{a,x},m_{a_0,x_0})=1.7\times {10}^{-2}$. The reconstruction is convincing and well captures the fine details of the biological structures, and one can clearly see the interweaving tubulins in the right part of the image.

Note that an interesting feature of the gridless reconstruction is that, once the Radon measure is computed, it is straightforward to plot it through any operator on a fine grid of one choice. Indeed, as one cannot represent a discrete measure m, we rather plot $\Phi m$, where $\Phi$ is the PSF with a slightly smaller variance, in order to clearly see the deconvolution. In all of our reconstructions, we convolve the reconstruction through the PSF with variance $\sigma /6$ and plot it on a grid 32 times finer. As a matter of comparison, discrete methods are performed for a fixed fine grid, and, if one wants a finer reconstruction, one has to recompute everything.

We finally test the off-the-grid reconstruction on a real data set of tubulins with high density molecules, provided by the 2013 IEEE ISBI SMLM challenge. In this stack of 500 frames of $128\times 128$ pixels, the FWHM (full width at half maximum) of the acquisition system is estimated at 351.8 $\mathrm{nm}$. We recall that the FWHM is the width of the Gaussian curve measured between those points on the y-axis, which are half the maximum amplitude, also note that it is linked to the variance $\sigma$ by $\mathrm{FWHM}=2\sqrt{2ln2}\times \sigma$. We compare the reconstruction of the off-the-grid method with the output of the Deep-STORM [43] algorithm, touted as the algorithm with the most visually compelling results. The reconstructions of the gridless method and the Deep-STORM algorithm are presented in Figure 10, where one can appreciate the reconstruction by off-the-grid on fine details. The reconstruction seems a small bit blurry compared to Deep-STORM, due to the plotting through a small spread Gaussian kernel. However, it is noteworthy that both comparisons perform well to retrieve the filaments, in particular in the enhancing yellow circles: the off-the-grid reconstruction seems to better preserve the structure compared to the Deep-STORM’s rough output. The quality of the reconstruction is notably interesting for off-the-grid reconstruction since it does not require any test sets to yield this reconstruction, contrary to Deep-STORM. The only data needed are the knowledge of (an estimation of) the forward operator, and the off-the-grid reconstruction can be then performed from any input without having to train the model on different types and levels of noise.

Shorthand: We tested an off-the-grid method on both SMLM synthetic and experimental data set. The gridless problem is tractable thanks to the Sliding Frank–Wolfe algorithm, and yields compelling results. The results are all the more interesting since there is only one parameter, handy to tune and robust w.r.t. noise. Thus, it can be easily adapted to any other dataset with a known acquisition operator.

5. Conclusions

We described in this review the off-the-grid variational settings for the sparse spike problem, through the definition of the space of signed measures $\mathcal{M}\left(\mathcal{X}\right)$ and the functional BLASSO defined over this set. Thanks to the trade-off between the convexity of the functional and the infinite dimensional, non-reflexive space of optimisation $\mathcal{M}\left(\mathcal{X}\right)$; the BLASSO can be defined to solve the sparse spike recovery problem. We review in this paper the theoretical guarantees to reach the correct minimum as the literature provides multiple results, in particular a sharp criterion for stable spike recovery under a low noise regime. Numerical methods to tackle the BLASSO problem were also discussed, with insights on the SDP approach, which is asymptotically exact but only suited for Fourier measurements, the Frank–Wolfe approach with known rate of convergence but a high computation load and the Conic Particle Gradient Descent with cheap iterations but lacks of guarantees. We were finally able to present the result of the off-the-grid approach with a Sliding Frank–Wolfe algorithm in the case of SMLM synthetic data and real data from the EPFL Challenge, and to illustrate the usefulness of these methods to recover fine-scale details.