Adversarial Robustness with Partial Isometry

Abstract

Despite their remarkable performance, deep learning models still lack robustness guarantees, particularly in the presence of adversarial examples. This significant vulnerability raises concerns about their trustworthiness and hinders their deployment in critical domains that require certified levels of robustness. In this paper, we introduce an information geometric framework to establish precise robustness criteria for $l_2$ white-box attacks in a multi-class classification setting. We endow the output space with the Fisher information metric and derive criteria on the input–output Jacobian to ensure robustness. We show that model robustness can be achieved by constraining the model to be partially isometric around the training points. We evaluate our approach using MNIST and CIFAR-10 datasets against adversarial attacks, revealing its substantial improvements over defensive distillation and Jacobian regularization for medium-sized perturbations and its superior robustness performance to adversarial training for large perturbations, all while maintaining the desired accuracy.

1. Introduction

One of the primary motivations for investigating machine learning robustness stems from the susceptibility of neural networks to adversarial attacks, wherein small perturbations in the input data can deceive the network into making the wrong decision. These adversarial attacks have been shown to be both ubiquitous and transferable [1,2,3]. Beyond posing a security threat, adversarial attacks underscore the glaring lack of robustness in machine learning models [4,5]. This deficiency in robustness is a critical challenge as it undermines trustworthiness in machine learning systems [6].

In this paper, we shed an information geometric perspective to adversarial robustness in machine learning models. We show that robustness can be achieved by encouraging the model to be isometric in the orthogonal space of the kernel of the pullback Fisher information metric (FIM). We subsequently formulate a regularization defense method for adversarial robustness. While our focus is on $l_2$ white-box attacks within multi-class classification tasks, the method’s applicability extends to more general settings, including unrestricted attacks and black-box attacks across various supervised learning tasks. The regularized model is evaluated on MNIST and CIFAR-10 datasets against projected gradient descent (PGD) $l_{\infty }$ attacks and AutoAttack [7] with $l_{\infty }$ and $l_2$ norms. Comparisons with the unregularized model, defensive distillation [8], Jacobian regularization [9], and Fisher information regularization [10] show significant improvement in robustness. Moreover, the regularized model is able to ensure robustness against larger perturbations compared to adversarial training.

The remainder of this paper is organized as follows. Section 2 introduces notations, notions of adversarial machine learning, and definitions related to geometry. Then, we derive a sufficient condition for adversarial robustness at a given sample point. Section 3 presents our method for approximating the robustness condition, which involves promoting model isometry in the orthogonal complement of the kernel of the pullback of the FIM. In Section 4, several experiments are presented to evaluate the proposed method. Section 5 discusses the results in the context of related work on adversarial defense. Finally, Section 6 concludes the paper and outlines potential extensions of this research. Appendix A provides the proof of the results stated in the main text.

2. Notations and Definitions

2.1. Notations

Let $d,c\in {\mathbb{N}}^{*}$ such that $d\ge c>1$. Let $m=c-1$. In the learning framework, d will be the dimension of the input space, while c will be the number of classes. The range of a matrix M is denoted as $rg\left(M\right)$. The rank of M is denoted as $rk\left(M\right)$. The Euclidean norm (i.e., $l_2$ norm) is denoted as $\parallel ·\parallel$. We use the notation ${\delta }_{ij}=1$ if $i=j$ and 0 otherwise. We denote the components of a vector v by $v^i\in \mathbb{R}$ with a superscript. Smooth means $C^{\infty }$.

2.2. Adversarial Machine Learning

An adversarial attack is any strategy aiming at deliberately altering the expected behavior of a model or extracting information from a model. In this work, we focus on attacks performed at inference time (i.e., after training), sometimes referred to as evasion attacks. The most well-known evasion attacks are gradient-based. Such gradient-based attacks all follow the same idea that we explain thereafter.

To reach good accuracy and generalization, a machine learning model f (with input x and parameter w) is typically trained by minimizing a loss function $L(y,f(x,w\left)\right)$ with respect to the parameters w of the model. In its simpler form, the loss function quantifies the error between the prediction of the model $f(x,w)$ and ground-truth y. Given a clean input $x_0$, an adversarial example $x_{*}$ can be crafted by maximizing the loss function $L(y,f(x,w\left)\right)$, starting from $x_0$ and using gradient ascent $x_{t+1}-x_t\propto {\nabla }_{x}L(y,f(x,w))$, where the gradient is computed with respect to the input x (and not the parameter w as during training). In order for $x_{*}$ to be an adversarial example, $x_0$ and $x_{*}$ must be close to each other according to some dissimilarity measure, typically a $l_p$ norm. An adversarial example $x_{*}$ is successful if the model f classifies $x_{*}$ differently from $x_0$. Some well-known gradient-based attacks include the fast gradient sign method [2] and projected gradient descent [3].

Adversarial attacks can be classified according to their threat model. White-box attacks assume that the adversary has perfect knowledge of the targeted model, including access to the training data, model architecture, and model parameters. Such an adversary can directly compute the gradient ${\nabla }_{x}L(y,f(x,w))$ of the targeted model and craft adversarial examples. More realistic threat models are classified as gray-box or black-box attacks, where some or all of the information is unknown to the adversary. In this work, we use both white-box attacks as well as simple gray-box attacks where the adversary can access the training data and model architecture, but not the model parameters. To craft such gray-box adversarial examples, another model is trained with the same data and architecture. Then, white-box attacks are performed on this model. Finally, the adversarial examples can be transferred to the targeted model.

Adversarial robustness aims to build models that classify both $x_{*}$ and $x_0$ with the same class while preserving sufficient accuracy for the clean examples $x_0$. Various defenses have been proposed to improve adversarial robustness. The most efficient defense is called adversarial training, which was first described in [2] and further developed in [3]. The idea behind adversarial training is to obtain the parameters $w_{*}$ of the trained model as:

$$w_{*}=\underset{w}{\mathrm{arg\; min}}\underset{ϵ\in \Delta \left(x\right)}{\max }L(y,f(x+ϵ,w)),$$

in place of the original parameters ${\mathrm{arg\; min}}_{w}L(y,f(x,w))$. The set $\Delta \left(x\right)$ is a set of allowed adversarial attacks for x, e.g., a $l_2$ ball with a given radius (or budget). In practice, adversarial training is performed by adding adversarial examples to the training set, thus providing a lower bound for ${\max }_{ϵ\in \Delta \left(x\right)}L(y,f(x+ϵ,w))$.

2.3. Geometrical Definitions

Consider a multi-class classification task. Let $\mathcal{X}\subseteq {\mathbb{R}}^d$ be the input domain, and let $\mathcal{Y}=\{1,\dots ,c\}\subset \mathbb{N}$ be the set of labels for the classification task. For example, in MNIST, we have $\mathcal{X}={[0,1]}^d$ (with $d=784$) and $c=10$. We assume that $\mathcal{X}$ is a d-dimensional embedded smooth connected submanifold of ${\mathbb{R}}^d$. Let $m=c-1$.

Definition 1

(Probability simplex). Define the probability simplex of dimension m by

$${\Delta }^m=\left\{\theta \in {\mathbb{R}}^m:\forall k\in \{1,\dots ,m\},{\theta }^k>0 \mathrm{and} \sum _{i=1}^m{\theta }^i<1\right\}.$$

(1)

${\Delta }^m$ is a smooth submanifold of ${\mathbb{R}}^c$ of dimension m. We can see $\theta =({\theta }^1,\dots ,{\theta }^m)$ as a coordinate system from ${\Delta }^m$ to ${\mathbb{R}}^m$. Then, let us define ${\theta }^c=1-{\sum }_{i=1}^m{\theta }^i$.

A machine learning model (e.g., a neural network) is often seen as assigning a label $y\in \mathcal{Y}$ to a given input $x\in \mathcal{X}$. Instead, in this work, we see a model as assigning the parameters of a random variable Y to a given input $x\in \mathcal{X}$. The random variable Y has a probability density function $p_{\theta }$ belonging to the family of c-dimensional categorical distributions $\mathcal{S}=\{p_{\theta }:\theta \in {\Delta }^m\}$.

$\mathcal{S}$ can be endowed with a differentiable structure by using $p_{\theta }\in \mathcal{S}\mapsto ({\theta }^1,\dots ,{\theta }^m)\in {\mathbb{R}}^m$ as a global coordinate system. Hence, $\mathcal{S}$ becomes a smooth manifold of dimension m (more details on this construction can be found in [11], Chapter 2). We can identify $p_{\theta }$ with $({\theta }^1,\dots ,{\theta }^m)$.

We see any machine learning model as a smooth map $f:\mathcal{X}\to {\Delta }^m$ that assigns to an input $x\in \mathcal{X}$, the parameters $\theta =f\left(x\right)\in {\Delta }^m$ of a c-dimensional categorical distribution $p_{\theta }\in \mathcal{S}$. In practice, a neural network produces a vector of logits $s\left(x\right)$. Then, these logits are transformed into the parameters $\theta$ with the softmax function: $\theta =\mathrm{softmax}\left(s\right(x\left)\right)$.

In order to study the sensitivity of the predicted $f\left(x\right)\in {\Delta }^m$ with respect to the input $x\in \mathcal{X}$, we need to be able to measure distances both in $\mathcal{X}$ and in ${\Delta }^m$. In order to measure distances on smooth manifolds, we need to equip each manifold with a Riemannian metric.

First, we consider ${\Delta }^m$. As described above, we see ${\Delta }^m$ as the family of categorical distributions. A natural Riemannian metric for ${\Delta }^m$ (i.e., a metric that reflects the statistical properties of ${\Delta }^m$) is the Fisher information metric (FIM).

Definition 2

(Fisher information metric). For each $\theta \in {\Delta }^m$, the Fisher information metric (FIM) g defines a symmetric positive-definite bilinear form $g_{\theta }$ over the tangent space $T_{\theta }{\Delta }^m$. In the standard coordinates of ${\mathbb{R}}^c$, for all $\theta \in {\Delta }^m$ and all tangent vectors $v,w\in T_{\theta }{\Delta }^m$, we have

$$g_{\theta }(v,w)=v^T{G}_{\theta }w,$$

(2)

where $G_{\theta }$ is the Fisher information matrix for parameter $\theta \in {\Delta }^m$, defined by

$$G_{\theta ,ij}=\frac{{\delta }_{ij}}{{\theta }^i}+\frac{1}{{\theta }^c}.$$

(3)

For any $\theta \in {\Delta }^m$, the matrix $G_{\theta }$ is symmetric positive-definite and non-singular (see Proposition 1.6.2 in [12]). The FIM induces a distance on ${\Delta }^m$, called the Fisher–Rao distance, denoted as $d({\theta }_1,{\theta }_2)$ for any ${\theta }_1,{\theta }_2\in {\Delta }^m$.

The FIM has two remarkable properties. First, it is the “infinitesimal distance” of the relative entropy, which is the loss function used to train a multi-class classification model. More precisely, if D is the relative entropy (also known as the Kullback–Leibler divergence) and if d is the Fisher–Rao distance, then given two distributions ${\theta }_1$ and ${\theta }_2$, we have (see Theorem 4.4.5 in [12]):

$$D\left({\theta }_1\left|\right|{\theta }_2\right)=\frac{1}{2}{d}^2\left({\theta }_1,{\theta }_2\right)+o\left(d^2\left({\theta }_1,{\theta }_2\right)\right).$$

The same result can be restated infinitesimally using the FIM g, as follows:

$$D\left(\theta \left|\right|\theta +d\theta \right)=\frac{1}{2}{g}_{\theta }\left(d\theta ,d\theta \right)+o\left(g_{\theta }\left(d\theta ,d\theta \right)\right),$$

(4)

where $d\theta$ is seen as a tangent vector of $T_{\theta }\mathcal{S}$.

The other remarkable property of the FIM is Chentsov’s theorem [13], claiming that the FIM is the unique Riemannian metric on ${\Delta }^m$, which is invariant under sufficient statistics (up to a multiplicative constant). Informally, the FIM is the only Riemannian metric that is statistically meaningful. In [14], Amari and Nagaoka state a more general result. Along with the FIM, they introduce a family of affine connections parameterized by a real parameter $\alpha$, called the $\alpha$-connections. Theorem 2.6 in [14] states that an affine connection is invariant under sufficient statistics if and only if it is an $\alpha$-connection for some $\alpha \in \mathbb{R}$. In other words, the $\alpha$-connections are the only affine connections that have a statistical meaning. While Equation (4) gives the second-order approximation of the relative entropy, an $\alpha$-connection can be seen as the third-order term in the Taylor approximation of some divergence [14]. More precisely, a given $\alpha$-connection can be canonically associated with a unique divergence (while the second-order term is always given by the FIM). If $\alpha =\pm 1$, the canonical divergences are the relative entropy and its dual (obtained by switching the arguments in $D\left({\theta }_2\right|\left|{\theta }_1\right)$). More generally, for $\alpha \ne 0$, the canonical divergence is not symmetric. The only canonical divergence that is symmetric is obtained for $\alpha =0$, and is precisely the square of the Fisher–Rao distance. Thus, the Fisher–Rao distance is the only statistically meaningful distance. This provides a motivation for using the Fisher–Rao distance to measure lengths in ${\Delta }^m$.

Now, we consider $\mathcal{X}$. Since we are studying adversarial robustness, we need a metric that formalizes the idea that two close data points must be “indistinguishable” from a human perspective (or any other relevant perspective). A natural choice is the Euclidean metric induced from ${\mathbb{R}}^d$ on $\mathcal{X}$.

Definition 3

(Euclidean metric). We consider the Euclidean space ${\mathbb{R}}^d$ endowed with the Euclidean metric $\overline{g}$. It is defined in the standard coordinates of ${\mathbb{R}}^d$ for all $x\in {\mathbb{R}}^d$ and for all tangent vectors $v,w\in T_x{\mathbb{R}}^d$ by

$${\overline{g}}_x(v,w)=v^{T}w,$$

(5)

thus, its matrix is the identity matrix of dimension d, denoted as $I_d$. The Euclidean metric induces a distance on ${\mathbb{R}}^d$ that we will denote with the $l_2$-norm: $\parallel x_1-x_2\parallel$ for any $x_1,x_2\in {\mathbb{R}}^d$.

From now on, we fix:

• A smooth map $f:(\mathcal{X},\overline{g})\to ({\Delta }^m,g)$. We denote by $f^i$ the i-th component of f in the standard coordinates of ${\mathbb{R}}^c$.
• A point $x\in \mathcal{X}$.
• A positive real number $ϵ>0$.

Define the Euclidean open ball centered at x with radius $ϵ$ by

$$\overline{b}(x,ϵ)=\left\{z\in {\mathbb{R}}^d:\parallel z-x\parallel <ϵ\right\}.$$

(6)

Definition 4.

Define the set (Figure 1):

$${\mathcal{A}}_x=\left\{\theta \in {\Delta }^m:\arg \underset{i}{\max }{\theta }^i=\arg \underset{i}{\max }{f}^i\left(x\right)\right\}.$$

(7)

For simplicity, assume that $f\left(x\right)$ is not on the “boundary” of ${\mathcal{A}}_x$, such that $\arg {\max }_i{f}^i\left(x\right)$ is well-defined.

Figure 1. $ϵ$-robustness at x is enforced if and only if $f\left(\overline{b}(x,ϵ)\right)\subseteq {\mathcal{A}}_x$.

The set ${\mathcal{A}}_x$ is the subset of distributions of ${\Delta }^m$ that have the same class as $f\left(x\right)$.

Definition 5

(Geodesic ball of the FIM). Let $\delta >0$ be the Fisher–Rao distance between $f\left(x\right)$ and ${\Delta }^m\setminus {\mathcal{A}}_x$ (Figure 2), i.e., the Fisher–Rao distance between $f\left(x\right)$ and the closest distribution of ${\Delta }^m$ with a different class.

Figure 2. $ϵ$-robustness at x is enforced if $\overline{b}(x,ϵ)\subseteq \tilde{b}(x,\delta )$.

Define the geodesic ball centered at $f\left(x\right)\in {\Delta }^m$ with radius δ by

$$b(f\left(x\right),\delta )=\left\{\theta \in {\Delta }^m:d(f\left(x\right),\theta )\le \delta \right\}.$$

(8)

In Section 3.3, we propose an efficient approximation of δ.

Definition 6

(Pullback metric). On $\mathcal{X}$, define the pullback metric $\tilde{g}$ of g by f. In the standard coordinates of ${\mathbb{R}}^d$, $\tilde{g}$ is defined for all tangent vectors $v,w\in T_x\mathcal{X}$ by

$${\tilde{g}}_x(v,w)=v^T{J}_x^T{G}_{f\left(x\right)}{J}_{x}w,$$

(9)

where $J_x$ is the Jacobian matrix of f at x (in the standard coordinates of ${\mathbb{R}}^d$ and ${\mathbb{R}}^c$). Define the matrix of ${\tilde{g}}_x$ in the standard coordinates of ${\mathbb{R}}^d$ by

$${\tilde{G}}_x=J_x^T{G}_{f\left(x\right)}{J}_x.$$

(10)

Definition 7

(Geodesic ball of the pullback metric). Let $\tilde{d}$ be the distance induced by the pullback metric $\tilde{g}$ on ${\mathbb{R}}^d$. We can define the geodesic ball centered at x with radius δ by

$$\tilde{b}(x,\delta )=\left\{z\in {\mathbb{R}}^d:\tilde{d}(x,z)\le \delta \right\}.$$

(11)

Note that the radius δ is the Fisher–Rao distance between $f\left(x\right)$ and ${\Delta }^m\setminus {\mathcal{A}}_x$ as defined in Definition 5.

2.4. Robustness Condition

Definition 8

(Robustness). We say that f is ϵ-robust at x if

$$\forall z\in {\mathbb{R}}^d,\parallel z-x\parallel <ϵ\Rightarrow f\left(z\right)\in {\mathcal{A}}_x.$$

(12)

Equivalently, we can write (Figure 1):

$$f\left(\overline{b}(x,ϵ)\right)\subseteq {\mathcal{A}}_x.$$

(13)

Proposition 1

(Sufficient condition for robustness). If $\overline{b}(x,ϵ)\subseteq \tilde{b}(x,\delta )$, then f is ϵ-robust at x (Figure 2).

Our goal is to start from Proposition 1 and make several assumptions in order to derive a condition that can be efficiently implemented.

Working with geodesic balls $\overline{b}(x,ϵ)$ and $\tilde{b}(x,\delta )$ is intractable, so our first assumption consists of using an “infinitesimal” condition by restating Proposition 1 in the tangent space $T_x\mathcal{X}$ instead of working directly on $\mathcal{X}$. In $T_x\mathcal{X}$, define the Euclidean ball of radius $ϵ$ by

$${\overline{\mathcal{B}}}_x(0,ϵ)=\left\{v\in T_x\mathcal{X}:{\overline{g}}_x(v,v)=v^{T}v\le {ϵ}^2\right\}.$$

(14)

Similarly, in $T_x\mathcal{X}$, define the ${\tilde{g}}_x$-ball of radius $\delta$ by

$${\tilde{\mathcal{B}}}_x(0,\delta )=\left\{v\in T_x\mathcal{X}:{\tilde{g}}_x(v,v)=v^T{\tilde{G}}_{x}v\le {\delta }^2\right\}.$$

(15)

Assumption 1.

We replace Proposition 1 by

$${\overline{\mathcal{B}}}_x(0,ϵ)\subseteq {\tilde{\mathcal{B}}}_x(0,\delta ).$$

(16)

Proposition 2.

Equation (16) is equivalent to

$$\forall v\in T_x\mathcal{X},{\tilde{g}}_x(v,v)\le \frac{{\delta }^2}{{ϵ}^2}{\overline{g}}_x(v,v).$$

(17)

Since $m<d$, the Jacobian matrix $J_x$ has a rank smaller or equal to m. Thus, since $G_{f\left(x\right)}$ has full rank, ${\tilde{G}}_x=J_x^T{G}_{f\left(x\right)}{J}_x$ has a rank of at most m (when $J_x$ has a rank of m).

Assumption 2.

The Jacobian matrix $J_x$ has a full rank equal to m.

Using Assumptions 1 and 2, the constant rank theorem ensures that for small enough $\delta$, f is $ϵ$-robust at x. However, contrary to Proposition 1, Assumption 1 does not offer any guarantee on the $ϵ$-robustness at x for arbitrary $\delta$.

3. Derivation of the Regularization Method

In this section, we derive a condition for robustness (Proposition 4), which can be implemented as a regularization method. Then, we provide two useful results for the practical implementation of this method: an explicit formula for the decomposition of the FIM as $G=P^{T}P$ (Section 3.2), and an easy-to-compute upper-bound of $\delta$, i.e., the Fisher–Rao distance between $f\left(x\right)$ and ${\Delta }^m\setminus {\mathcal{A}}_x$ (Section 3.3).

3.1. The Partial Isometry Condition

In order to simplify the notations, we replace

• $J_x$ with J, which is a full-rank $m\times d$ real matrix.
• $G_{f\left(x\right)}$ with G, which is an $m\times m$ symmetric positive definite real matrix.
• ${\tilde{G}}_x$ with $\tilde{G}$, which is a $d\times d$ symmetric positive-semidefinite real matrix.

We define $D={\left(\ker \left(\tilde{G}\right)\right)}^{\perp }$. We will use the two following facts.

Fact 1.

$$D=\mathrm{rg}\left(J^T\right)={\left(\ker \left(J\right)\right)}^{\perp }={\left(\ker \left(J^{T}GJ\right)\right)}^{\perp }$$

(18)

Fact 2.

$J^{T}GJ$ is symmetric positive semidefinite. Thus, by the spectral theorem, the eigenvectors associated with its nonzero eigenvalues are all in $D=\mathrm{rg}\left(J^T\right)$.

In particular, since $\mathrm{rk}\left(J\right)=m$, there exists an orthonormal basis of $T_x\mathcal{X}$, denoted as $\mathcal{B}=(e_1,\dots ,$$e_m,e_{m+1},\dots ,e_d)$, such that each $e_i$ is an eigenvector of $J^{T}GJ$, and such that $(e_1,\dots ,e_m)$ is a basis of $D=\mathrm{rg}\left(J^T\right)$ and $(e_{m+1},\dots ,e_d)$ is a basis of $\ker \left(J\right)$.

The set $D=\mathrm{rg}\left(J^T\right)$ is an m-dimensional subspace of $T_x\mathcal{X}$. ${\tilde{g}}_x$ does not define an inner product on $T_x\mathcal{X}$ because $\tilde{G}$ has a nontrivial kernel of dimension $d-m$. In particular, the set ${\tilde{\mathcal{B}}}_x(0,\delta )$ is not bounded, i.e., it is a cylinder rather than a ball. However, when restricted to D, ${\tilde{g}}_x{|}_D$ defines an inner product. We define the restriction of ${\tilde{\mathcal{B}}}_x(0,\delta )$ to D:

$${\tilde{\mathcal{B}}}_D(0,\delta )=\left\{v\in D:v^T\tilde{G}v\le \delta \right\},$$

(19)

and similarly, we define the restriction of ${\overline{\mathcal{B}}}_x(0,ϵ)$ to D:

$${\overline{\mathcal{B}}}_D(0,ϵ)=\left\{v\in D:v^{T}v\le {ϵ}^2\right\}.$$

(20)

Assume that f is such that Equation (16) holds (i.e., ${\overline{\mathcal{B}}}_x(0,ϵ)\subseteq {\tilde{\mathcal{B}}}_x(0,\delta )$). Moreover, assume that we are in the limit case defined as follows: for any perturbation size, we can find a smaller perturbation of f such that Equation (16) does not hold anymore. This limit case is equivalent to having ${\overline{\mathcal{B}}}_D(0,ϵ)={\tilde{\mathcal{B}}}_D(0,\delta )$. In this case, ${\tilde{\mathcal{B}}}_x(0,\delta )$ is the smallest possible ${\tilde{g}}_x$-ball (for the inclusion) such that Equation (16) holds. We noticed experimentally that enforcing this stronger criteria yields a larger robustifying effect. Thus, we make the following assumption:

Assumption 3.

We replace Equation (16) with

$${\overline{\mathcal{B}}}_D(0,ϵ)={\tilde{\mathcal{B}}}_D(0,\delta ).$$

(21)

Proposition 3.

Equation (21) is equivalent to

$$\forall v\in D,{\tilde{g}}_x(v,v)=\frac{{\delta }^2}{{ϵ}^2}{\overline{g}}_x(v,v).$$

(22)

We can rewrite Equation (22) in matrix form:

$$\forall v\in D,v^T\tilde{G}v=\frac{{\delta }^2}{{ϵ}^2}{v}^{T}v.$$

(23)

In Section 3.2, we show how to exploit the properties of the FIM to derive a closed-form expression for a matrix $P\in {\mathrm{GL}}_m\left(\mathbb{R}\right)$, such that $G=P^{T}P$. For now, we assume that we can easily access such a P and we are looking for a condition on P and J, which is equivalent to Equation (23).

Proposition 4.

The following statements are equivalent:

$$\begin{array}{cc}\hfill \left(i\right)& \forall u\in D,u^T{J}^{T}GJu=\frac{{\delta }^2}{{ϵ}^2}{u}^{T}u,\hfill \\ \hfill \left(ii\right)& PJ{J}^T{P}^T=\frac{{\delta }^2}{{ϵ}^2}{I}_m,\hfill \end{array}$$

where $I_m$ is the identity matrix of dimension $m\times m$.

Proposition 4 constrains the matrix $PJ$ to be a semi-orthogonal matrix (multiplied by a homothety matrix). A smooth map f between Riemannian manifolds $(\mathcal{X},\overline{g})$ and $({\Delta }^m,g)$ is said to be (locally) isometric if the pullback metric (denoted $f^{*}g$) coincides with $\overline{g}$, i.e., $f^{*}g=\overline{g}$. Such a map f locally preserves distances. In our case, $f^{*}g=\tilde{g}$ is not a metric (since its kernel is non-trivial); thus, f cannot be an isometry. However, Equation (22) ensures that f locally preserves distances along directions spanned by D. Hence, f becomes a partial isometry, at least in the neighborhood of the training points.

Under the Assumptions 1–3, Equation (ii) in Proposition 4 implies robustness as defined in Definition 8. In other words, Equation (ii) is a sufficient condition for robustness. However, there is no reason for a neural network to satisfy Equation (ii). This is why we define the following regularization term:

$$\alpha \left(x,ϵ,f\right)=\frac{1}{m^2}|||PJ{J}^T{P}^T-\frac{{\delta }^2}{{ϵ}^2}{I}_m|||,$$

(24)

where |||·||| is any matrix norm, such as the Frobenius norm or the spectral norm. We use the Frobenius norm in the experiments of Section 4. To compute $\alpha (x,ϵ,f)$, we only need to compute the Jacobian matrix J, which can be efficiently achieved with backpropagation. Finally, the loss function is:

$$L\left(y,x,ϵ,f\right)=l\left(y,f\left(x\right)\right)+\lambda \alpha \left(x,ϵ,f\right),$$

(25)

where l is the cross-entropy loss, and $\lambda >0$ is a hyperparameter controlling the strength of the regularization with respect to the cross-entropy loss. The regularization term $\alpha (x,ϵ,f)$ is minimized during training, such that the model is pushed to satisfy the sufficient condition of robustness.

3.2. Coordinate Change

In this subsection, we show how to compute the matrix P that was introduced in Proposition 4. To this end, we isometrically embed ${\Delta }^m$ into the Euclidean space ${\mathbb{R}}^c$ using the following inclusion map:

$$\begin{array}{cc}\hfill \mu :{\Delta }^m& ⟶{\mathbb{R}}^c\hfill \\ \hfill \left({\theta }^1,\dots ,{\theta }^m\right)& \mapsto 2\left(\sqrt{{\theta }^1},\dots ,\sqrt{{\theta }^m},\sqrt{1-\sum _{i=1}^m{\theta }^i}\right)\hfill \end{array}$$

We can easily see that $\mu$ is an embedding. If ${\mathcal{S}}^m\left(2\right)$ is the sphere of radius 2 centered at the origin in ${\mathbb{R}}^c$, then $\mu \left({\Delta }^m\right)$ is the subset of ${\mathcal{S}}^m\left(2\right)$, where all coordinates are strictly positive (using the standard coordinates of ${\mathbb{R}}^c$).

Proposition 5.

Let g be the Fisher information metric on ${\Delta }^m$ (Definition 2), and $\overline{g}$ be the Euclidean metric on ${\mathbb{R}}^c$. Then μ is an isometric embedding of $({\Delta }^m,g)$ into $({\mathbb{R}}^c,\overline{g})$.

Now, we use the stereographic projection to embed ${\Delta }^m$ into ${\mathbb{R}}^m$:

$$\begin{array}{cc}\hfill \tau :\mu \left({\Delta }^m\right)& ⟶{\mathbb{R}}^m\hfill \\ \hfill ({\mu }^1,\dots ,{\mu }^m,{\mu }^c)& \mapsto 2\left(\frac{{\mu }^1}{2-{\mu }^c},\dots ,\frac{{\mu }^m}{2-{\mu }^c}\right),\hfill \end{array}$$

with ${\mu }^c=2\sqrt{1-{\sum }_{i=1}^m{\theta }^i}$.

Proposition 6.

In the coordinates τ, the FIM is:

$$G_{\tau ,ij}=\frac{4}{{\left({1+\parallel \tau /2\parallel }^2\right)}^2}{\delta }_{ij}.$$

(26)

Let $\tilde{J}$ be the Jacobian matrix of $\tau \circ \mu :{\Delta }^m\to {\mathbb{R}}^m$ at $f\left(x\right)$. Then, we have:

$$G={\tilde{J}}^T{G}_{\tau }\tilde{J}=\frac{4}{{\left({1+\parallel \tau /2\parallel }^2\right)}^2}{\tilde{J}}^T\tilde{J}.$$

(27)

Thus, we can choose:

$$P=\frac{2}{{1+\parallel \tau /2\parallel }^2}\tilde{J}.$$

(28)

Write $f\left(x\right)=\theta =({\theta }^1,\dots ,{\theta }^m)$ and ${\theta }^c=1-{\sum }_{i=1}^m{\theta }^i$. For simplicity, write ${\tau }^i\left(\theta \right)={\tau }^i\left(\mu \left(\theta \right)\right)=2\sqrt{{\theta }^i}/(1-\sqrt{{\theta }^c})$ for $i=1,\dots ,m$. More explicitly, we have:

Proposition 7.

For $i,j=1,\dots ,m$:

$$P_{ij}=\frac{{\delta }_{ij}}{\sqrt{{\theta }^i}}-\frac{{\tau }^i\left(\theta \right)}{2\sqrt{{\theta }^c}}.$$

(29)

3.3. The Fisher–Rao Distance

In this subsection, we derive a simple upper-bound for $\delta$ (i.e., the Fisher–Rao distance between $f\left(x\right)$ and ${\Delta }^m\setminus {\mathcal{A}}_x$). In Proposition 5, we show that the probability simplex ${\Delta }^m$ endowed with the FIM can be isometrically embedded into the m-sphere of radius 2. Thus, the angle $\beta$ between two distributions of coordinates ${\theta }_1$ and ${\theta }_2$ in ${\Delta }^m$ with ${\mu }_1=\mu \left({\theta }_1\right)$ and ${\mu }_2=\mu \left({\theta }_2\right)$ is:

$$\cos \left(\beta \right)=\frac{1}{4}\sum _{i=1}^c{\mu }_1^i{\mu }_2^i=\sum _{i=1}^c\sqrt{{\theta }_1^i{\theta }_2^i}.$$

(30)

The Riemannian distance between these two points is the arc length on the sphere:

$$d({\theta }_1,{\theta }_2)=2\arccos \sum _{i=1}^c\sqrt{{\theta }_1^i{\theta }_2^i}.$$

(31)

In the regularization term defined in Equation (24), we replace $\delta$ with the following upper bound:

$$\delta =d\left(f\left(x\right),{\Delta }^m\setminus {\mathcal{A}}_x\right)\le d\left(f\left(x\right),O\right),$$

(32)

where $O=\frac{1}{c}(1,\dots ,1)$ is the center of the simplex ${\Delta }^m$. Thus,

$$\delta \le 2\arccos \sum _{i=1}^c\sqrt{\frac{f^i\left(x\right)}{c}}.$$

(33)

4. Experiments

The regularization method introduced in Section 3 is evaluated on MNIST and CIFAR-10 datasets. Our method uses the loss function introduced in Equation (25).

4.1. Experiments on MNIST Dataset

4.1.1. Experimental Setup

For the MNIST dataset, we implement a LeNet model with two convolutional layers of 32 and 64 channels, respectively, followed by one hidden layer with 128 neurons. The code is available here: https://github.com/lshigarrier/geometric_robustness.git (accessed on 1 December 2022). We train three models: one regularized model, one baseline unregularized model, and one model trained with adversarial training. All three models are trained with the Adam optimizer (${\beta }_1=0.9$ and ${\beta }_2=0.999$) for 30 epochs, with a batch size of 64, and a learning rate of ${10}^{-3}$. For the regularization term, we use a budget of $ϵ=5.6$, which is chosen to contain the $l_{\infty }$ ball of radius 0.2. The adversarial training is conducted with 10 iterations of PGD with a budget ${ϵ}_{adv}=0.2$ using $l_{\infty }$ norm. We found that $\lambda ={10}^{-6}$ yields the best performance in terms of robustness–accuracy trade-off; this value is small because we did not attempt to normalize the regularization term.

The models are trained on the 60,000 images of MNIST’s training set and then tested on 10,000 images of the test set. The baseline model achieves an accuracy of 98.9% (9893/10,000), the regularized model achieves an accuracy of 94.0% (9403/10,000), and the adversarially trained model achieves an accuracy of 98.8% (9883/10,000). Although the current implementation of the regularized model is almost six times slower to train than the baseline model, it may be possible to accelerate the training using, for example, the technique proposed by Shafahi et al. [15], or using another method to approximate the spectral norm of $\tilde{J}$. Even without relying on these acceleration techniques, the regularized model is still faster to train than the adversarially trained model.

4.1.2. Robustness to Adversarial Attacks

To measure the adversarial robustness of the models, we use the PGD attack with the $l_{\infty }$ norm, 40 iterations, and a step size of 0.01. The $l_{\infty }$ norm yields the hardest possible attack for our method, and corresponds more to the human notion of “indistinguishable images” than the $l_2$ norm. The attacks are performed on the test set, and only on images that are correctly classified by each model. The results are reported in Figure 3. The regularized model has a slightly lower accuracy than the baseline model for small perturbations, but the baseline model suffers a drop in accuracy above the attack level $ϵ=0.1$. Adversarial training achieves high accuracy for small- to medium-sized perturbations but the accuracy decreases sharply above $ϵ=0.3$. The regularized model remains robust even for large perturbations. The baseline model reaches 50% accuracy at $ϵ=0.2$ and the adversarially trained model at $ϵ=0.325$, while the regularized model reaches 50% accuracy at $ϵ=0.4$.

Figure 3. Accuracy of the baseline (dashed, blue), regularized (solid, green), and adversarially trained (dotted, red) models for various attack perturbations on the MNIST dataset. The perturbations are obtained with PGD using $l_{\infty }$ norm.

Table 1 provides more results against AutoAttack (AA) [7], which was designed to offer a more reliable evaluation of adversarial robustness. For a fair comparison, and in addition to a baseline model (BASE), we compare the partial isometry defense (ISO) with several other computationally efficient defenses: distillation (DIST) [8], Jacobian regularization (JAC) [9], which also relies on the Jacobian matrix of the network, and Fisher information regularization (FIR) [10], which also leverages information geometry. We also consider an adversarially trained (AT) model using PGD. ISO is the best defense that does not rely on adversarial training. In future work, ISO may be combined with AT to further boost performance. Note that ISO and JAC are more robust against $l_2$ attacks since they were designed to defend the model against such attacks. On the other hand, AT is more robust against $l_{\infty }$ attacks, because the adversarial training was conducted with the $l_{\infty }$ norm.

Table 1. Clean and robust accuracy on MNIST against AA, averaged over 10 runs. The number in parentheses is the attack strength.

Defense	BASE	ISO	DIST	JAC	FIR	AT
Clean	99.01	96.51	98.81	98.95	98.84	98.98
AA-$l_2$ (0.15)	35.70	43.38	35.35	38.74	1.68	73.34
AA-$l_{\infty }$ (1.5)	10.38	22.15	9.63	13.30	0.03	95.43

4.2. Experiments on CIFAR-10 Dataset

We consider a DenseNet121 model fine-tuned on CIFAR-10 using pre-trained weights for ImageNet. The code is available here: https://github.com/lshigarrier/iso_defense.git (accessed on 26 January 2023). As for the MNIST experiments, we compare the partial isometry defense with distillation (DIST), Jacobian regularization (JAC), and Fisher information regularization (FIR). Here, adversarial training (AT) relies on the fast gradient sign method (FGSM) attack [16]. All defenses are compared against PGD for various attack strengths. The results are presented in Table 2. The defenses are evaluated in a “gray-box” setting where the adversary can access the architecture and the data but not the weights. More precisely, the adversarial examples are crafted from the test set of CIFAR-10 using another unregularized DenseNet121 model. AT is the more robust method, but ISO achieves a robust accuracy 30% higher than the next best analogous method (FIR).

Table 2. Clean and robust accuracy on CIFAR-10 against PGD. The number in parentheses is the attack strength.

Defense	BASE	ISO	DIST	JAC	FIR	AT
Clean	92.93	76.86	84.96	86.17	89.98	80.78
PGD (4/255)	2.49	40.17	7.54	8.56	9.74	68.82
PGD (8/255)	0.47	39.68	3.35	3.66	4.05	66.61

One of our goals is to provide alternatives to adversarial training (AT). Apart from high computational costs, AT suffers from several limitations: it only robustifies against the chosen attack at the chosen budget and it does not offer a robustness guarantee. For example, under Gaussian noise, AT accuracy decreases faster than baseline accuracy (i.e., no defense). Achieving high robustness accuracy against specific attacks on a specific benchmark is insufficient and misleading to measure the true robustness of the evaluated model. Our method offers a new point of view that can be extended to certified defense methods in future works.

5. Discussion and Related Work

In 2019, Zhao et al. [17] proposed to use the Fisher information metric in the setting of adversarial attacks. They used the eigenvector associated with the largest eigenvalue of the pullback of the FIM as an attack direction. Following their work, Shen et al. [10] suggested a defense mechanism by suppressing the largest eigenvalue of the FIM. They upper-bounded the largest eigenvalue by the trace of the FIM. As in our work, they added a regularization term to encourage the model to have smaller eigenvalues. Moreover, they showed that their approach is equivalent to label smoothing [18]. In our framework, their method consists of expanding the geodesic ball $\tilde{b}(x,\delta )$ as much as possible. However, their approach does not guarantee that the constraint imposed on the model will not harm the accuracy more than necessary. In our framework, matrix $PJ$ (compared with $\delta /ϵ$) informs the model on the precise restriction that must be imposed to achieve adversarial robustness in the $l_2$ ball of radius $ϵ$.

Cisse et al. [19] introduced another adversarial defense called Parseval networks. To achieve adversarial robustness, the authors aim to control the Lipschitz constant of each layer of the model to be close to unity. This is achieved by constraining the weight matrix of each layer to be a Parseval tight frame, which is another name for semi-orthogonal matrix. Since the Jacobian matrix of the entire model with respect to the input is almost the product of the weight matrices, the Parseval network defense is similar to our proposed defense, albeit with completely different rationales. This suggests that geometric reasoning could successfully supplement the line of work on Lipschitz constants of neural networks, such as in [20].

Following another line of work, Hoffman et al. [9] advanced a Jacobian regularization to improve adversarial robustness. Their regularization consists of using the Frobenius norm of the input–output Jacobian matrix. To avoid computing the true Frobenius norm, they relied on random projections, which are shown to be both efficient and accurate. This method is similar to the method of Shen et al. [10] in the sense that it will also increase the radius of the geodesic ball. However, the Jacobian regularization does not take into account the geometry of the output space (i.e., the Fisher information metric) and assumes that the probability simplex ${\Delta }^m$ is Euclidean.

Although this study focuses on $l_2$ norm robustness, it must be pointed out that there are other “distinguishability” measures that can be used to study adversarial robustness, including all other $l_p$ norms. In particular, the $l_{\infty }$ norm is often considered to be the most natural choice when working with images. However, the $l_{\infty }$ norm is not induced by any inner product and, hence, there is no Riemannian metric that induces the $l_{\infty }$ norm. However, given an $l_{\infty }$ budget ${ϵ}_{\infty }$, we can choose an $l_2$ budget ${ϵ}_2=\sqrt{d}{ϵ}_{\infty }$, such that any attack in the ${ϵ}_{\infty }$ budget will also respect the ${ϵ}_2$ budget. When working on images, other dissimilarity measures are rotations, deformations, and color changes of the original image. Contrary to the $l_2$ or $l_{\infty }$ norms, these measures do not rely on a pixel-based coordinate system. However, it is possible to define unrestricted attacks based on these spatial dissimilarities, for example, in [21].

In this work, we derive the partial isometry regularization for a classification task. The method can be extended to regression tasks by considering the family of multivariate normal distributions as the output space. On the probability simplex ${\Delta }^m$, the FIM is a metric with constant positive curvature, while it has constant negative curvature on the manifold of multivariate normal distributions [22].

Finally, the precise quantification of the robustness condition presented in Equation (12) and Proposition 4 paves the way for the development of a certified defense [23] in this framework. By strongly enforcing Proposition 4 on a chosen proportion of the training set, it may be possible to maximize the accuracy under the constraint of a chosen robustness level, which offers another solution to the robustness–accuracy trade-off [24,25]. Certifiable defenses are a necessary step for the deployment of deep learning models in critical domains and missions, such as civil aviation, security, defense, and healthcare, where a certification may be required to ensure a sufficient level of trustworthiness.

6. Conclusions and Future Work

In this paper, we introduce an information geometric approach to the problem of adversarial robustness in machine learning models. The proposed defense consists of enforcing a partial isometry between the input space endowed with the Euclidean metric and the probability simplex endowed with the Fisher information metric. We subsequently derived a regularization term to achieve robustness during training. The proposed strategy is tested on the MNIST and CIFAR-10 datasets, and shows a considerable increase in robustness without harming the accuracy. Future works will evaluate the method on other benchmarks and real-world datasets. Several attack methods will also be considered in addition to PGD and AutoAttack. Although this work focuses on $l_2$ norm robustness, future work will consider other “distinguishability” measures.

Our work extends a recent, promising but understudied framework for adversarial robustness based on information geometric tools. The FIM has already been harnessed to develop attacks [17] and defenses [10,26] but a precise robustness analysis is yet to be proposed. Our work is a step toward the development of such an analysis, which might yield certified guarantees relying on these geometric tools. The study of adversarial robustness, which is non-local by definition and contrary to accuracy, should benefit greatly from a geometrical vision. However, the current literature on adversarial robustness is mainly concerned with the FIM and its spectrum (which are very local objects) without unfolding the full arsenal developed in information geometry. In our work, we demonstrate the usefulness of such an approach by developing a preliminary robustification method. Model robustification is a hard, unsolved yet vital problem to ensure the trustworthiness of deep learning tools in safety-critical applications. Our framework could be extended and applied to existing certification strategies, such as Lipschitz-based [27] or randomized smoothing [23], where statistical models naturally appear.