Bounded Variation Separates Weak and Strong Average Lipschitz

Abstract

We closely examine a recently introduced notion of average smoothness. The latter defined a weak and strong average-Lipschitz seminorm for real-valued functions on general metric spaces. Specializing to the standard metric on the real line, we compare these notions to bounded variation (BV) and discover that the weak notion is strictly weaker than BV while the strong notion is strictly stronger. Along the way, we discover that the weak average smooth class is also considerably larger in a certain combinatorial sense, which is made precise by the fat-shattering dimension.

1. Introduction

A function $f:[0,1]\to \mathbb{R}$ is L-Lipschitz if $|f\left(x\right)-f\left(x^{\prime }\right)|\le L|x-x^{\prime }|$ for all $x,x^{\prime }\in [0,1]$, and ${∥f∥}_{Lip}$ is the smallest L for which this holds. If f has an integrable derivative, its variation $V\left(f\right)$ is given by $V\left(f\right)={\int }_0^1\left|f^{\prime }\left(x\right)\right|\mathrm{d}x$ (the more general definition is given in (2)). Since $\left|f^{\prime }\left(x\right)\right|\le {∥f∥}_{Lip}$, we have the obvious relation $V\left(f\right)\le {∥f∥}_{Lip}$. No reverse inequality is possible: since for monotone f, we have $V\left(f\right)=\left|f\right(0)-f(1\left)\right|$ [1], a function whose value increases from 0 to $\epsilon$ with a sharp “jump” in the middle can have an arbitrarily large L and arbitrarily small V.

Motivated by questions in machine learning and statistics, Ashlagi et al. [2] introduced two notions of “average Lipschitz” in general metric probability spaces: a weak one and a strong one (follow-up works extended these results to average Hölder smoothness [3,4]). For the special case of the metric space $\Omega =[0,1]$ equipped with the standard metric $\rho (x,x^{\prime })=|x-x^{\prime }|$ and the uniform distribution U, their definitions are as follows. Both notions rely on the local slope of $f:[0,1]\to \mathbb{R}$ at a point x, which is defined (and denoted) as follows:

$$\begin{array}{ccc}\hfill {\Lambda }_f\left(x\right)& =& \underset{x^{\prime }\in [0,1]\setminus \left\{x\right\}}{sup}{\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}},x\in [0,1].\hfill \end{array}$$

(1)

The strong and weak average smoothness of f are defined, respectively, by

$$\begin{array}{l}{\parallel f\parallel }_{\mathrm{S}}=\mathbb{E}\left[{\Lambda }_f\left(X\right)\right],\\ {\parallel f\parallel }_{\mathrm{W}}=\mathbb{W}\left[{\Lambda }_f\left(X\right)\right]=\underset{t>0}{sup}tU\left(\left\{x\in \Omega :{\Lambda }_f\left(x\right)\ge t\right\}\right),\end{array}$$

where X is a random variable distributed according to U on $[0,1]$, $\mathbb{E}$ is the usual expectation, and $\mathbb{W}$ is the weak $L_1$ norm of the random variable Z:

$$\mathbb{W}\left[Z\right]=\underset{t>0}{sup}t\mathbb{P}\left(\right|Z|\ge t).$$

Both ${\parallel ·\parallel }_{\mathrm{S}}$ and ${\parallel ·\parallel }_{\mathrm{W}}$ satisfy the homogeneity axiom of seminorms (meaning that $∥\alpha f∥=\left|\alpha \right|·∥f∥$), and ${\parallel ·\parallel }_{\mathrm{S}}$ additionally satisfies the triangle inequality and hence is a true seminorm. The weak $L_1$ norm satisfies the weaker inequality $\mathbb{W}[X+Y]\le 2\left(\mathbb{W}\right[X]+\mathbb{W}[Y\left]\right)$ [5], which ${\parallel ·\parallel }_{\mathrm{W}}$ also inherits.

We now recall the definition of the variation of $f:[a,b]\to \mathbb{R}$:

$$\begin{array}{c}\hfill V_a^b\left(f\right)=\underset{a=x_0<x_1<x_2<\dots <x_n\le b}{sup}\sum _{i=1}^n|f\left(x_i\right)-f\left(x_{i-1}\right)|\end{array}$$

(2)

(when $a=0$ and $b=1$, we omit these), as well as the Lipschitz and bounded variation function classes:

$$\begin{array}{l}Lip=\left\{f:[0,1]\to \mathbb{R};{∥f∥}_{Lip}<\infty \right\},\\ BV=\left\{f:[0,1]\to \mathbb{R};V\left(f\right)<\infty \right\}.\end{array}$$

The discussion above implies the (well-known) strict containment

$$\begin{array}{c}\hfill Lip⊊BV.\end{array}$$

(3)

In addition, we define the strong and weak average smoothness classes

$$\begin{array}{l}{\overline{Lip}}^{\mathsf{s}}=\left\{f:[0,1]\to {\mathbb{R};\parallel f\parallel }_{\mathrm{S}}<\infty \right\},\\ {\overline{Lip}}^{\mathsf{w}}=\left\{f:[0,1]\to {\mathbb{R};\parallel f\parallel }_{\mathrm{W}}<\infty \right\}.\end{array}$$

By Markov’s inequality and the fact that the expectation is bounded by the supremum, we have

$${\parallel f\parallel }_{\mathrm{W}}\le {\parallel f\parallel }_{\mathrm{S}}\le \underset{x\in \Omega }{sup}{\Lambda }_f\left(x\right)={∥f∥}_{Lip}$$

whence

$$\begin{array}{c}\hfill Lip\subseteq {\overline{Lip}}^{\mathsf{s}}\subseteq {\overline{Lip}}^{\mathsf{w}};\end{array}$$

(4)

all of these containments were shown to be strict in [2]. The containments in (3) and (4) leave open the relation between BV and ${\overline{Lip}}^{\mathsf{s}},{\overline{Lip}}^{\mathsf{w}}$, which we resolve in this work:

Theorem 1. 

${\overline{Lip}}^{\mathsf{s}}⊊BV⊊{\overline{Lip}}^{\mathsf{w}}$.

We also provide a quantitative, finitary relation between these clases:

Theorem 2. 

For any $f:[0,1]\to \mathbb{R}$, we have ${\displaystyle \frac{1}{2}}{\parallel f\parallel }_{\mathrm{W}}\le V\left(f\right)\le {\parallel f\parallel }_{\mathrm{S}}.$

Finally, we recall the definition of the fat-shattering dimension, which is a combinatorial complexity measure of function classes of central importance in statistics, empirical processes, and machine learning [6,7]. Let F be a collection of functions mapping $[0,1]$ to $\mathbb{R}$. For $\gamma >0$, a set $S=\left\{x_1,\dots ,x_m\right\}\subset [0,1]$ is said to be $\gamma$-shattered by F if

$$\begin{array}{c}\hfill \underset{r\in {\mathbb{R}}^m}{sup}\underset{y\in {\left\{-1,1\right\}}^m}{min}\underset{f\in F}{sup}\underset{i\in \left[m\right]}{min}{y}_i(f\left(x_i\right)-r_i)\ge \gamma .\end{array}$$

(5)

The $\gamma$-fat-shattering dimension, denoted by ${fat}_{\gamma }\left(F\right)$, is the size of the largest $\gamma$-shattered set (possibly ∞). It is known [8] that for $F=\{f:[0,1]\to \mathbb{R}\mid V(f)\le L\}$, we have ${fat}_{\gamma }\left(F\right)=1+⌊{\displaystyle \frac{L}{2\gamma }}⌋$. This same bound holds for $F=\{f:[0,1]\to \mathbb{R}\mid Lipf\le L\}$.

Although the strong smoothness class has the same combinatorial complexity as the BV and Lipschitz classes, for weak average smoothness, this quantity turns out to be considerably greater:

Theorem 3. 

For $L>0$, let $F_{\mathrm{W}}=\left\{f:[0,1]\to {\mathbb{R};\parallel f\parallel }_{\mathrm{W}}\le L\right\}$ and $F_{\mathrm{S}}=\left\{f:[0,1]\to {\mathbb{R};\parallel f\parallel }_{\mathrm{S}}\le L\right\}$. Then,

1. 

${fat}_{\gamma }\left(F_{\mathrm{W}}\right)=\infty$ whenever $\gamma \le {\displaystyle \frac{L}{6}}$;

2. 

${fat}_{\gamma }\left(F_{\mathrm{S}}\right)=1+⌊{\displaystyle \frac{L}{2\gamma }}⌋$ for $\gamma >0$.

Notation.

We write $\left[n\right]:=\left\{1,\dots ,n\right\}$ and use $m(·)$ to denote the Lebesgue measure (length) of sets in $\mathbb{R}$.

2. Proofs

We begin with a variant of the standard covering lemma.

Lemma 1. 

For any sequence $s_1,\dots ,s_n$ of closed segments in $\mathbb{R}$, there is a subsequence indexed by $I\subseteq \left[n\right]$ such that for all distinct $i,j\in I$ we have $s_i\cap s_j=\varnothing$ and ${\displaystyle \sum _{i\in I}}m\left(s_i\right)\ge {\displaystyle \frac{1}{2}}m\left({\displaystyle \bigcup _{i=1}^n}{s}_i\right)$.

Proof. 

We proceed by induction on n. Let $G=\left(\right[n],E)$ denote the intersection graph of the $s_i$: the vertices correspond to the segments and $(i,j)\in E$ if $s_i\cap s_j\ne \varnothing$.

Suppose that G contains a cycle, and let $s_1=[a_1,b_1],\dots ,s_k=[a_k,b_k]$ be the segments in the cycle sorted by their right endpoint. Since $s_1\cap s_k\ne \varnothing$, we have $a_k\le b_1$. If $a_{k-1}\ge a_1$, then $s_{k-1}\subseteq s_1\cup s_k$. Otherwise, $a_{k-1}<a_1$ and $s_1\subseteq s_{k-1}$. Either way, we have found a segment that is completely covered by the other vertices of G. After removing it, we obtain $I\subseteq \left[n\right]$ of size $n-1$ with ${\displaystyle \bigcup _{i\in I}}{s}_i={\displaystyle \bigcup _{i=1}^n}{s}_i$, so applying the inductive hypothesis on the segments in I yields the desired result. If G does not contain a cycle, then $G=A\cup B$ is bipartite, where $A,B\subseteq \left[n\right]$ are disjoint and nonempty. Clearly, $m\left({\displaystyle \bigcup _{i=1}^n}{s}_i\right)\le m\left({\displaystyle \bigcup _{i\in A}}{s}_i\right)+m\left({\displaystyle \bigcup _{i\in B}}{s}_i\right)$, and thus $max\left(m\left({\displaystyle \bigcup _{i\in A}}{s}_i\right),m\left({\displaystyle \bigcup _{i\in B}}{s}_i\right)\right)\ge {\displaystyle \frac{1}{2}}m\left({\displaystyle \bigcup _{i=1}^n}{s}_i\right)$, so taking either $I=A$ or $I=B$ (which is possible since the segments inside each part are disjoint) yields the desired result. □

Next, we reduce the proof of Theorem 2 to the case of right-continuous monotone functions.

Lemma 2. 

If for every right-continuous monotone function $f:[0,1]\to \mathbb{R}$ we have ${\parallel f\parallel }_{\mathrm{W}}\le 2V\left(f\right)$, then the bound holds for all $f:[0,1]\to \mathbb{R}$. Furthermore, both inequalities are tight.

Proof. 

We begin by observing that we can restrict our attention to monotone functions, since $T_f\left(x\right)=V_f\left([0,x]\right)$ is monotone and has the same variation as f, but ${\Lambda }_{T_f}\left(x\right)={\displaystyle \underset{x^{\prime }\ne x}{sup}}{\displaystyle \frac{|T_f\left(x\right)-T_f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}\ge \underset{x^{\prime }\ne x}{sup}{\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}$, which means ${\parallel T_f\parallel }_{\mathrm{W}}\ge {\parallel f\parallel }_{\mathrm{W}}$.

Thus, the inequality ${\parallel T_f\parallel }_{\mathrm{W}}\le 2V\left(T_f\right)$ immediately implies that ${\parallel f\parallel }_{\mathrm{W}}\le {\parallel T_f\parallel }_{\mathrm{W}}\le 2V\left(T_f\right)=2V\left(f\right)$. If f is monotone, it can only have jump discontinuities. Let $I\subset [0,1]$ denote the set of right discontinuities of f. Note that since f is monotone, I is at most countable. Define the modified version of f to be

$$\begin{array}{c}\hfill \tilde{f}\left(x\right)=\left\{\begin{array}{cc}f\left(x\right)\hfill & x\notin I\hfill \\ \underset{\epsilon \to 0^{+}}{lim}f(x+\epsilon )\hfill & x\in I.\hfill \end{array}\right.\end{array}$$

(6)

Note that $\tilde{f}$ is monotone and right-continuous. It is not hard to see that if $0\notin I$, then $V\left(\tilde{f}\right)=V\left(f\right)$ and ${\Lambda }_{\tilde{f}}\left(x\right)={\Lambda }_f\left(x\right)$ for all $x\notin I$, which implies that ${\parallel f\parallel }_{\mathrm{W}}={\parallel \tilde{f}\parallel }_{\mathrm{W}}$ and allows us to restrict our discussion to right-continuous functions. If $0\in I$, then we can extend the domain of $\tilde{f}$ to $[-\epsilon ,1]$ for all $\epsilon >0$, where $\tilde{f}\left(x\right)=f\left(0\right)$ for all $x<0$. Denote the extended function by ${\tilde{f}}_{\epsilon }$, then since ${\parallel f\parallel }_{\mathrm{W}}={lim}_{\epsilon \to 0}{\parallel {\tilde{f}}_{\epsilon }\parallel }_{\mathrm{W}}$ and $V\left({\tilde{f}}_{\epsilon }\right)=V\left(f\right)$ for all $\epsilon >0$, we can conclude that

$${\parallel f\parallel }_{\mathrm{W}}=\underset{\epsilon \to 0}{lim}{\parallel {\tilde{f}}_{\epsilon }\parallel }_{\mathrm{W}}\le \underset{\epsilon \to 0}{lim}2V\left({\tilde{f}}_{\epsilon }\right)=2V\left(f\right).$$

□

2.1. Proof of Theorem 2

We first show that ${\parallel f\parallel }_{\mathrm{W}}\le 2V\left(f\right)$. We may assume without loss of generality that $V\left(f\right)<\infty$. We will use the notation $V_f\left([a,b]\right)$ for the variation of f when restricted to the segment $[a,b]$. Since f is of bounded variation, the function $T_f\left(x\right)=V_f\left([0,x]\right)$ is well defined for $x>0$. By Lemma 2, we may assume without loss of generality that f is right-continuous. Thus, $T_f:[0,1]\to \mathbb{R}$ is monotone and right-continuous and thus induces a Lebesgue–Stieltjes measure on $[0,1]$, which we denote by ${\mu }_f$. We now define the maximal function $M_f:[0,1]\to \mathbb{R}$ as follows:

$$\begin{array}{c}\hfill M_f\left(x\right)=\underset{r_1,r_2>0}{sup}{\displaystyle \frac{{\mu }_f\left([x-r_1,x+r_2]\right)}{r_1+r_2}}=\underset{r_1,r_2>0}{sup}{\displaystyle \frac{V_f\left([x-r_1,x+r_2]\right)}{r_1+r_2}},\end{array}$$

(7)

where the segments $[a,x],[x,b]$ are taken to be $[0,x],[x,1]$, respectively, whenever $a<0$ or $b>1$. A standard argument shows that $M_f^{-1}\left((t,\infty )\right)$ is open, whence $M_f$ is measurable.

We now observe that $M_f\ge {\Lambda }_f$ everywhere in $[0,1]$. Indeed, if $x^{\prime }>x$, then $M_f\left(x\right)\ge {\displaystyle \frac{V_f\left([x-\epsilon ,x^{\prime }]\right)}{\epsilon +(x^{\prime }-x)}}\ge {\displaystyle \frac{|f\left(x^{\prime }\right)-f\left(x\right)|}{x^{\prime }-x+\epsilon }}$ holds for $\epsilon >0$, and hence $M_f\left(x\right)\ge \underset{x^{\prime }>x}{sup}{\displaystyle \frac{|f\left(x^{\prime }\right)-f\left(x\right)|}{x^{\prime }-x}}$. The case of $x^{\prime }<x$ is completely analogous, whence $M_f\left(x\right)\ge \underset{x^{\prime }\ne x}{sup}{\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}={\Lambda }_f\left(x\right)$. For X uniformly distributed over $[0,1]$, we have $\mathbb{P}\left({\Lambda }_f\left(X\right)\ge t\right)\le \mathbb{P}\left(M_f\left(X\right)\ge t\right)$ and showing ${\parallel f\parallel }_{\mathrm{W}}\le 2V\left(f\right)$ reduces to bounding the latter probability by $2V\left(f\right)/t$.

We now closely follow the proof of Theorem 7.4 in [9] and bound $m\left(M_f^{-1}\left((t,\infty )\right)\right)$ by bounding $m\left(K\right)$ for arbitrary compact $K\subseteq M_f^{-1}\left((t,\infty )\right)$. For $x\in K\subseteq M_f^{-1}\left((t,\infty )\right)$, denote by $r_1\left(x\right),r_2\left(x\right)$ some lengths such that ${\displaystyle \frac{{\mu }_f\left([x-r_1\left(x\right),x+r_2\left(x\right)]\right)}{r_1\left(x\right)+r_2\left(x\right)}}\ge t$. Denote by $S_x$ the open interval $(x-r_1\left(x\right),x+r_2\left(x\right))$. Then, clearly, $K\subseteq {\displaystyle \bigcup _{x\in K}}{S}_x$. Since K is compact, a finite cover by intervals $S_{x_1},\dots ,S_{x_n}$ exists. By Lemma 1, there exists $I\subseteq \left[n\right]$ such that for all distinct $i,j\in I$, we have $S_{x_i}\cap S_{x_j}=\varnothing$ and ${\displaystyle \sum _{i\in I}}m\left(S_{x_i}\right)\ge {\displaystyle \frac{1}{2}}m\left({\displaystyle \bigcup _{i=1}^n}{S}_{x_i}\right)$. Finally, by the definition of the $S_x$’s, for each $i\in \left[n\right]$, it holds that $m\left(S_{x_i}\right)\le {\displaystyle \frac{{\mu }_f\left(S_{x_i}\right)}{t}}$. We can now write

$$\begin{array}{c}\hfill m\left(K\right)\le m\left({\displaystyle \bigcup _{i=1}^n}{S}_{x_i}\right)\le 2{\displaystyle \sum _{i\in I}}m\left(S_{x_i}\right)\le {\displaystyle \frac{2}{t}}{\displaystyle \sum _{i\in I}}{\mu }_f\left(S_{x_i}\right)\le {\displaystyle \frac{2}{t}}{\mu }_f\left([0,1]\right),\end{array}$$

(8)

where the last inequality holds since the intervals in I are disjoint. Since ${\mu }_f\left([0,1]\right)=V\left(f\right)$, it immediately follows that ${\parallel f\parallel }_{\mathrm{W}}\le 2V\left(f\right)$.

It remains to show that $V\left(f\right)\le {\parallel f\parallel }_{\mathrm{S}}$. Let us denote by $P_n$ the partition $0\le x_1<x_2<\dots <x_n\le 1$ of $[0,1]$, and let $V\left(P_n\right)={\displaystyle \sum _{i=1}^{n-1}}|f\left(x_{i+1}\right)-f\left(x_i\right)|$ denote the variation of f relative to $P_n$. It suffices to show that for any such partition $P_n$, we have ${\parallel f\parallel }_{\mathrm{S}}\ge V\left(P_n\right)$. Now

$$\begin{array}{c}\hfill {\parallel f\parallel }_{\mathrm{S}}=\mathbb{E}\left[{\Lambda }_f\left(X\right)\right]\ge {\displaystyle \sum _{i=1}^{n-1}}|x_{i+1}-x_i|\mathbb{E}\left[{\Lambda }_f\left(X\right)|X\in [x_i,x_{i+1}]\right].\end{array}$$

(9)

Note that for all $x\in [x_i,x_{i+1}]$, we have

$$\begin{array}{c}\hfill {\Lambda }_f\left(x\right)\ge max\left({\displaystyle \frac{|f\left(x\right)-f\left(x_i\right)|}{x-x_i}},{\displaystyle \frac{|f\left(x_{i+1}\right)-f\left(x\right)|}{x_{i+1}-x}}\right)\ge {\displaystyle \frac{|f\left(x_{i+1}\right)-f\left(x_i\right)|}{x_{i+1}-x_i}}.\end{array}$$

(10)

Applying this to (9) yields

$$\mathbb{E}\left[{\Lambda }_f\right]\ge {\displaystyle \sum _{i=1}^{n-1}}|x_{i+1}-x_i|{\displaystyle \frac{|f\left(x_{i+1}\right)-f\left(x_i\right)|}{x_{i+1}-x_i}}=V\left(P_n\right).$$

Finally, the tightness of the first claimed inequality is witnessed by the step function $f\left(x\right)=\mathbf{1}[x>1/2]$ and of the second inequality by $f\left(x\right)=x$. □

2.2. Proof of Theorem 1

The claimed containments are immediate from Theorem 2; only the separations remain to be shown. The first of these is obvious: the step function has bounded variation but infinite strong average Lipschitz [2] (Appendix I). We proceed with the second separation:

Lemma 3. 

There exists an $f:[0,1]\to [0,1]$ such that $V\left(f\right)=\infty$ but ${\parallel f\parallel }_{\mathrm{W}}\le 2$.

Proof. 

Let $f:[0,1]\to [0,1]$ be the piecewise linear function defined on $x_n={\displaystyle \frac{1}{n}}$, $n\ge 1$, by

$$f\left({\displaystyle \frac{1}{n}}\right)={\displaystyle \sum _{k=1}^n}{\displaystyle \frac{{(-1)}^{k+1}}{k}}$$

and extended to $[0,1]$ by linear interpolation.

Clearly, $V\left(f\right)={\displaystyle \sum _{n=1}^{\infty }}{\displaystyle \frac{1}{n}}=\infty$. To bound ${\parallel f\parallel }_{\mathrm{W}}$, note that any $x,x^{\prime }\in [0,1]$ witnessing ${\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}\ge t$ also verify $|x-x^{\prime }|\le {\displaystyle \frac{1}{t}}$. Let $I_n$ denote the interval $\left[{\displaystyle \frac{1}{n+1}},{\displaystyle \frac{1}{n}}\right]$. If ${\Lambda }_f\left(x\right)\ge t$, then there is an $x^{\prime }$ such that ${\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}\ge t$. Now, either x or $x^{\prime }$ lies in $I_n$ for $n\ge t$. If $x\in I_n$ with $n\ge t$, then $x\le {\displaystyle \frac{1}{t}}$. If, however, $x^{\prime }\in I_n$ for $n\ge t$, then $x^{\prime }\le {\displaystyle \frac{1}{t}}$ and since $|x-x^{\prime }|\le {\displaystyle \frac{1}{t}}$, we have $x\le {\displaystyle \frac{2}{t}}$. We conclude that ${\Lambda }_f\left(x\right)\ge t$ implies $x\le {\displaystyle \frac{2}{t}}$ and hence $\mathbb{P}\left({\Lambda }_f\left(x\right)\ge t\right)\le {\displaystyle \frac{2}{t}}$; this proves the claim. □

Remark.

Another function with this property is $xsin{\displaystyle \frac{1}{x}}$.

2.3. Proof of Theorem 3

2.3.1. Proof That ${fat}_{\gamma }\left(F_{\mathrm{W}}\right)=\infty$ Whenever $\gamma \le {\displaystyle \frac{L}{6}}$

Consider the partition of $[0,1]$ into segments $I_n=\left[x_{n+1},x_n\right]$ where $x_n=2^{-n}$. We define $f\left(x_n\right)={(-1)}^n\gamma$. This specifies f at all endpoints of $I_n$. For $x\in (x_{n+1},x_n)$, we define $f\left(x\right)={(-1)}^n{\displaystyle \frac{\gamma }{|I_n|}}(x-x_{n+1})+{(-1)}^n{\displaystyle \frac{\gamma }{|I_n|}}(x-x_n)$, i.e., f is piecewise linear with slope ${(-1)}^{n}4\gamma 2^n$ in $I_n$. Similarly to Lemma 3, if ${\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}\ge t$, then $|x-x^{\prime }|\le {\displaystyle \frac{2\gamma }{t}}$. Now, suppose ${\Lambda }_f\left(x\right)\ge t$, i.e., there exists $x^{\prime }$ with ${\displaystyle \frac{|f\left(x\right)-f\left(x^{\prime }\right)|}{|x-x^{\prime }|}}\ge t$. This implies that either x or $x^{\prime }$ lies in $I_n$ for some $n\ge log{\displaystyle \frac{t}{4\gamma }}$ (the slope of the line connecting x and $x^{\prime }$ lies between the slopes of the segments containing $x,x^{\prime }$). If $x\in I_n$ for some $n\ge log{\displaystyle \frac{t}{4\gamma }}$, then $x\le x_n\le {\displaystyle \frac{4\gamma }{t}}$. If, however, $x^{\prime }\in I_n$ for some $n\ge log{\displaystyle \frac{t}{4\gamma }}$ then $x^{\prime }\le {\displaystyle \frac{4\gamma }{t}}$ and since $|x-x^{\prime }|\le {\displaystyle \frac{2\gamma }{t}}$, we have $x\le {\displaystyle \frac{6\gamma }{t}}$. Since ${\Lambda }_f\left(x\right)\ge t$ implies $x\le {\displaystyle \frac{6\gamma }{t}}$, we can conclude that ${\parallel f\parallel }_{\mathrm{W}}\le 6\gamma$. An immediate corollary is that ${\overline{Lip}}_L^{\mathsf{w}}$$\gamma$-shatters the infinite set ${\left\{x_n\right\}}_{n=1}^{\infty }$ for $\gamma \le {\displaystyle \frac{L}{6}}$ — which is even stronger than having arbitrarily large $\gamma$-shattered sets. Note that this is close to tight, since for $\gamma >{\displaystyle \frac{L}{2}}$, we cannot $\gamma$-shatter even a set of two points. Suppose $f\left(x_1\right)>{\displaystyle \frac{L}{2}}$ and $f\left(x_2\right)<-{\displaystyle \frac{L}{2}}$, then for $x\in [x_1,x_2]$ we have ${\Lambda }_f\left(x\right)>{\displaystyle \frac{L}{|x_2-x_1|}}$, hence ${\parallel f\parallel }_{\mathrm{W}}>L$, which means $\{x_1,x_2\}$ is not $\gamma$-shattered by ${\overline{Lip}}_L^{\mathsf{w}}$. □

2.3.2. Proof That ${fat}_{\gamma }\left(F_{\mathrm{S}}\right)=1+⌊{\displaystyle \frac{L}{2\gamma }}⌋$ for $\gamma >0$

The upper bound follows immediately from $V\left(f\right)\le {∥f∥}_{Lip}$. For the lower bound, take a $2\gamma /L$ packing $\left\{x_1,\dots ,x_{1+⌊{\displaystyle \frac{L}{2\gamma }}⌋}\right\}$ of $[0,1]$. For labeling $y_i\in {\left\{-1,1\right\}}^n$, consider the linear interpolation of $\left(x_i,y_i\gamma \right)$, and observe that the interpolation f satisfies ${\Lambda }_f\left(x\right)\le {\displaystyle \frac{2\gamma }{2\gamma /L}}=L$ everywhere. □