The Various Definitions of Multiple Differentiability of a Function f: ℝⁿ→ ℝ

Abstract

Since the 17-th century the concepts of differentiability and multiple differentiability have become fundamental to mathematical analysis. By now we have the generally accepted definition of what a multiply differentiable function $f:{\mathbb{R}}^n\to \mathbb{R}$ is (in this paper we call it standard). This definition is sufficient to prove some of the key properties of a multiply differentiable function: the Generalized Young’s theorem (a theorem on the independence of partial derivatives of higher orders of the order of differentiation) and Taylor’s theorem with Peano remainder. Another definition of multiple differentiability, actually more general in the sense that it is suitable for the infinite-dimensional case, belongs to Fréchet. It turns out, that the standard definition and the Fréchet definition are equivalent for functions $f:{\mathbb{R}}^n\to \mathbb{R}$. In this paper we introduce a definition (which we call weak) of multiple differentiability of a function $f:{\mathbb{R}}^n\to \mathbb{R}$, which is not equivalent to the above-mentioned definitions and is in fact more general, but at the same time is sufficient enough to prove the Generalized Young’s and Taylor’s theorems.

1. Introduction

The main goal of this paper is to compare different definitions of multiple differentiability of a function $f:{\mathbb{R}}^n\to \mathbb{R}$ and to show that the classical theorems of real analysis such as the Generalized Young’s theorem (a theorem on the independence of partial derivatives of higher orders of the order of differentiation) and Taylor’s theorem with Peano remainder hold under the assumptions which are in fact weaker (in this paper we call these assumptions weak multiple differentiability) then multiple differentiability in Fréchet sense (which is now the generally accepted assumption for proving these theorems, see [1,2] for example). Although the definition we use can be found in some classical texts on real analysis (see [3] for example), the authors never deduced the above-mentioned theorems using only this definition, some additional assumptions (which in fact led to the multiple differentiability in Fréchet sense) on f were always required in order to prove them. In this paper we show that no additional assumptions are needed.

Throughout this paper we assume that $n\in \mathbb{N}$, ${\mathbf{x}}_0\in {\mathbb{R}}^n$, $\delta >0$, $O_{\delta }({\mathbf{x}}_0):=\{\mathbf{x}\in {\mathbb{R}}^n:$$\parallel \mathbf{x}-{\mathbf{x}}_0\parallel <\delta \}$, $f:O_{\delta }({\mathbf{x}}_0)\to \mathbb{R}$. First we introduce three definitions of multiple differentiability of a function f at some point ${\mathbf{x}}_0$. The following definition is the main subject of this paper and can be found, for example, in [3].

Definition 1 (weak).

Function f is said to be k times differentiable at ${\mathbf{x}}_0$ iff all of it’s partial derivatives of order $k-1$ are Fréchet differentiable at ${\mathbf{x}}_0$.

The next definition (see [4] for example) is the most popular and standard for modern analysis.

Definition 2 (standard).

Function f is said to be k times differentiable at ${\mathbf{x}}_0$ iff f is $k-1$ times differentiable in some neighborhood of ${\mathbf{x}}_0$ and all of it’s partial derivatives of order $k-1$ are Fréchet differentiable at ${\mathbf{x}}_0$.

The third definition follows directly from the Fréchet definition of differential as a linear operator (see [5,6,7]) and is suitable also for infinite-dimensional spaces.

Definition 3 (Fréchet).

Function f is said to be k times Fréchet differentiable at ${\mathbf{x}}_0$ iff it’s Fréchet differential of order $k-1$ is Fréchet differentiable at ${\mathbf{x}}_0$.

Note that a survey of various definitions of differentiability can be found in [8].

The rest of the paper is organised as follows. In Section 2 we first prove that the Definitions 2 and 3 are equivalent for the functions $f:{\mathbb{R}}^n\to \mathbb{R}$ (though this result is not new and in case $k=2$, the ideas of the proof can be found in [7], for example, here we give a rigorous proof for all $k\in \mathbb{N}$). Then we show that the Definition 1 is really weaker than the others by constructing an example of a function of two variables which satisfies the Definition 1, but doesn’t satisfy the Definitions 2 and 3 for $k=2$. In Section 3 we proceed to our main results and prove that the Generalized Young’s theorem and Taylor’s theorem with Peano remainder hold for functions which are weakly k times differentiable at some point ${\mathbf{x}}_0$ i.e., satisfy the Definition 1. Section 4 contains the conclusions and an outline of some further generalizations of the main results.

2. Comparison of the Various Definitions

By ${\partial }_i$ we denote the operator of partial differentiation with respect to the ith component. For all $\mathbf{h}\in {\mathbb{R}}^n$ we denote by ${\Delta }_{\mathbf{h}}$ the difference operator of the first order ${\Delta }_{\mathbf{h}}f(\mathbf{x}):=f(\mathbf{x}+\mathbf{h})-f(\mathbf{x})$. We also denote the partial derivatives and the differences of higher orders respectively: ${\partial }_{i_1\dots i_k}f(\mathbf{x}):={\partial }_{i_1}\circ \dots \circ {\partial }_{i_k}f(\mathbf{x})$, ${\Delta }_{{\mathbf{h}}_1\dots {\mathbf{h}}_k}f(\mathbf{x}):={\Delta }_{{\mathbf{h}}_1}\circ \dots \circ {\Delta }_{{\mathbf{h}}_k}f(\mathbf{x})$, where ${\mathbf{h}}_1,\dots ,{\mathbf{h}}_k\in {\mathbb{R}}^n$. Note that all the operators ${\partial }_i$, ${\Delta }_{\mathbf{h}}$ commute on their domains. By ${\mathbf{e}}_i$ we denote the ith vector of the natural basis in ${\mathbb{R}}^n$.

Theorem 1.

The Definitions 2 and 3 are equivalent.

Proof. 

In case $k=1$ these definitions obviously coincide. Lets assume that for some k the theorem is already proved and prove it for $k+1$. If f is $k+1$ times differentiable at ${\mathbf{x}}_0$ at least by one of the Definitions 2 or 3, then for all $\mathbf{x}\in O_{\delta }({\mathbf{x}}_0)$ we have a well-defined $d^{k}f(\mathbf{x})\in N_k:=\underset{k\mathrm{times}}{\underbrace{\mathcal{L}({\mathbb{R}}^n,\dots ,\mathcal{L}({\mathbb{R}}^n,}}\mathbb{R}\underset{k\mathrm{times}}{\underbrace{)\dots )}}$, where by $\mathcal{L}(H,N)$ we denote a normed space of bounded linear operators (equipped with the operator norm) acting from a normed space H to a normed space N. For all $B\in N_k$ we define a linear isomorphism ${\phi }_k:N_k\stackrel{\mathrm{Lin}}{\cong }{\mathbb{R}}^{n^k}$ by the formula ${\phi }_k(B):=\mathbf{a}\in {\mathbb{R}}^{n^k}$, where the components of $\mathbf{a}$ are defined by the equality $a^{(i_1-1)n^{k-1}+\dots +(i_{k-1}-1)n+i_k}:=B{\mathbf{e}}_{i_1}\dots {\mathbf{e}}_{i_k}$ for all $i_1,\dots ,i_k\in \overline{1,n}$. Its easy to see that ${\phi }_k$ is a linear operator. Its injectivity follows from the fact that if the values of operators $B_1$, $B_2\in N_k$ coincide on all the sets of basis vectors, then $B_1=B_2$. Surjectivity of the operators ${\phi }_k$ can be proved by induction. Let ${\phi }_{k-1}$ be surjective. We have to show that for each $\mathbf{a}\in {\mathbb{R}}^{n^k}$ there exists an element $B_{\mathbf{a}}\in N_k$ such that ${\phi }_k(B_{\mathbf{a}})=\mathbf{a}$. We represent $\mathbf{a}$ in the form $\mathbf{a}=({\tilde{\mathbf{a}}}_1,\dots ,{\tilde{\mathbf{a}}}_n)$, where ${\tilde{\mathbf{a}}}_j\in {\mathbb{R}}^{n^{k-1}}$ for all $j\in \overline{1,n}$. We define the operator $B_{\mathbf{a}}$ on the basis vectors (and therefore on the whole ${\mathbb{R}}^n$) by the rule $B_{\mathbf{a}}({\mathbf{e}}_j):={\tilde{B}}_{{\tilde{\mathbf{a}}}_j}$ for all $j\in \overline{1,n}$, where the operators ${\tilde{B}}_{{\tilde{\mathbf{a}}}_j}\in N_{k-1}$ satisfy the condition ${\phi }_{k-1}({\tilde{B}}_{{\tilde{\mathbf{a}}}_j})={\tilde{\mathbf{a}}}_j$ and exist by assumption of induction. Then ${\phi }_k{(B_{\mathbf{a}})}^{(i_1-1)n^{k-1}+\dots +(i_{k-1}-1)n+i_k}$ = $B_{\mathbf{a}}{\mathbf{e}}_{i_1}\dots {\mathbf{e}}_{i_k}$ = ${\tilde{B}}_{{\tilde{\mathbf{a}}}_{i_1}}{\mathbf{e}}_{i_2}\dots {\mathbf{e}}_{i_k}$ = ${\phi }_{k-1}{({\tilde{B}}_{{\tilde{\mathbf{a}}}_{i_1}})}^{(i_2-1)n^{k-2}+\dots +(i_{k-1}-1)n+i_k}$ = ${\tilde{a}}_{i_1}^{(i_2-1)n^{k-2}+\dots +(i_{k-1}-1)n+i_k}$ = $a^{(i_1-1)n^{k-1}+\dots +(i_{k-1}-1)n+i_k}$ for all $i_1,\dots ,i_k\in \overline{1,n}$, i.e., ${\phi }_k(B_{\mathbf{a}})=\mathbf{a}$. So we checked that ${\phi }_k$ is a linear isomorphism. For all $\mathbf{x}\in {\mathbb{R}}^{n^k}$ we define a norm $\parallel \mathbf{x}\parallel :=\parallel {\phi }^{-1}{(\mathbf{x})\parallel }_{N_k}$ and obtain an isometric isomorphism ${\phi }_k:N_k\stackrel{\mathrm{Norm}}{\cong }({\mathbb{R}}^{n^k},\parallel ·\parallel )$.

It is easy to check the equivalence of the differentiability of $d^{k}f$ at ${\mathbf{x}}_0$ to the differentiability of ${\phi }_k\circ d^{k}f$ (which takes values in the space $({\mathbb{R}}^{n^k},\parallel ·\parallel )$) at ${\mathbf{x}}_0$. However, all the norms are equivalent on ${\mathbb{R}}^{n^k}$, so (easy to check by definition) the differentiability of ${\phi }_k\circ d^{k}f$ in the space $({\mathbb{R}}^{n^k},\parallel ·\parallel )$ is equivalent to its differentiability in the space ${\mathbb{R}}^{n^k}$ equipped by the Euclidean norm, which, being considered componentwise, is equivalent to the differentiability of all the partial derivatives of f of the order k at ${\mathbf{x}}_0$ due to the equality ${\phi }_k{(d^{k}f(\mathbf{x}))}^{(i_1-1)n^{k-1}+\dots +(i_{k-1}-1)n+i_k}=d^{k}f(\mathbf{x}){\mathbf{e}}_{i_1}\dots {\mathbf{e}}_{i_k}={\partial }_{i_1}\dots {\partial }_{i_k}f(\mathbf{x})$, which holds for all $i_1,\dots ,i_k\in \overline{1,n}$. □

Theorem 2.

There exists a function $g:{\mathbb{R}}^2\to \mathbb{R}$ which satisfies the Definition 1, but does not satisfy the Definitions 2 and 3 for $k=2$.

Proof. 

We will give a nice explicit example, suggested by Pinelis. We denote $r:=\sqrt{x^2+y^2}$ and consider a function $u(x,y):=\left\{\begin{array}{c}{\displaystyle \frac{xy}{r}\in [-\frac{r}{2},\frac{r}{2}]\mathrm{for}r\ne 0;}\hfill \\ 0\mathrm{for}r=0.\hfill \end{array}\right.$

It is easy to check that u is continuous on ${\mathbb{R}}^2$, infinitely differentiable on ${\mathbb{R}}^2\backslash (0,0)$, not differentiable at $(0,0)$ and that $u_x(x,y):=\left\{\begin{array}{c}{\displaystyle \frac{y^3}{r^3}\in [-1,1]\mathrm{for}r\ne 0;}\hfill \\ 0\mathrm{for}r=0,\hfill \end{array}\right.$ $u_y(x,y):=\left\{\begin{array}{c}{\displaystyle \frac{x^3}{r^3}\in [-1,1]\mathrm{for}r\ne 0;}\hfill \\ 0\mathrm{for}r=0.\hfill \end{array}\right.$ Now we define a function

$${\displaystyle g(x,y):=\sum _{j=1}^{\infty }\frac{r^3}{j^2}u(x-\frac{1}{j},y).}$$

(1)

We see that the series

$${\displaystyle \sum _{j=1}^{\infty }[\frac{r^3}{j^2}u(x-\frac{1}{j},y){]}_x=\sum _{j=1}^{\infty }[\frac{r^3}{j^2}{u}_x(x-\frac{1}{j},y)+\frac{3rx}{j^2}u(x-\frac{1}{j},y)]=O(r^2)=o(r)}$$

as $(x,y)\to (0,0)$ and

$${\displaystyle \sum _{j=1}^{\infty }[\frac{r^3}{j^2}u(x-\frac{1}{j},y){]}_y=\sum _{j=1}^{\infty }[\frac{r^3}{j^2}{u}_y(x-\frac{1}{j},y)+\frac{3ry}{j^2}u(x-\frac{1}{j},y)]=O(r^2)=o(r),}$$

as well is the series (1) converge uniformly on every bounded set in ${\mathbb{R}}^2$ by the Weierstrass test (since for their terms $a_j(x,y)$ and for every bounded set $\Omega \subset {\mathbb{R}}^2$ there exists some $c=c(\Omega )>0$ such that ${\displaystyle |a_j(x,y)|<\frac{c}{j^2}}$ for all $(x,y)\in \Omega$), so the partial derivatives $g_x(x,y)=o(r)$ and $g_y(x,y)=o(r)$ exist on ${\mathbb{R}}^2$ and are continuous on ${\displaystyle {\mathbb{R}}^2\backslash \bigcup _{j\in \mathbb{N}}\{(\frac{1}{j},0)\}}$, which means that g is differentiable there. In addition, we have $g_x(0,0)=g_y(0,0)=0$. By the same reasoning for every $k\in \mathbb{N}$ the function ${\displaystyle g(x,y)-\frac{r^3}{k^2}u(x-\frac{1}{k},y)=\sum _{j\in \mathbb{N}\backslash \left\{k\right\}}\frac{r^3}{j^2}u(x-\frac{1}{j},y)}$ is differentiable at ${\displaystyle (\frac{1}{k},0)}$, while ${\displaystyle \frac{r^3}{k^2}u(x-\frac{1}{k},y)}$ is not, therefore g is not differentiable at every point of the set ${\displaystyle \bigcup _{j\in \mathbb{N}}\{(\frac{1}{j},0)\}}$ and hence g is not differentiable in any neighborhood of $(0,0)$ and doesn’t satisfy the Definition 2 for $k=2$ at $(0,0)$. Since $g_x(0,0)=0$, it is easy to check that $g_{xx}(0,0)=g_{xy}(0,0)=0$ and

$$\underset{(x,y)\to (0,0)}{lim}\frac{g_x(x,y)-g_x(0,0)-g_{xx}(0,0)x-g_{xy}(0,0)y}{r}=\underset{(x,y)\to (0,0)}{lim}\frac{o(r)}{r}=0,$$

so $g_x$ is differentiable at $(0,0)$. Similarly $g_y$ is, thus g is weakly twice differentiable at $(0,0)$ by the Definition 1. □

3. Main Results

Now we will show that for a weakly k times differentiable function (see the Definition 1) its partial derivatives of order k are independent of the order of differentiation, and even prove a more general result.

Theorem 3 (Generalized Young’s theorem).

Let $k\ge 2$, $i_1,\dots ,i_k\in \overline{1,n}$, σ be an arbitrary permutation of numbers $\{1,\dots ,k\}$. For all $\mathbf{x}\in O_{\delta }({\mathbf{x}}_0)$ let there exist ${\partial }_{i_2\dots i_k}f(\mathbf{x})$, ${\partial }_{i_{\sigma (2)}\dots i_{\sigma (k)}}f(\mathbf{x})$ and let the functions ${\partial }_{i_2\dots i_k}f$, ${\partial }_{i_{\sigma (2)}\dots i_{\sigma (k)}}f$ be differentiable at ${\mathbf{x}}_0$. Then ${\partial }_{i_1\dots i_k}f({\mathbf{x}}_0)={\partial }_{i_{\sigma (1)}\dots i_{\sigma (k)}}f({\mathbf{x}}_0)$.

Proof. 

We fix ${\displaystyle h\in (0,\frac{\delta }{k})}$. For all $p\in \overline{1,k}$ on an open ball $O_{\frac{k-p}{k}\delta }({\mathbf{x}}_0)$ there are well-defined functions ${\Delta }^{p}f:={\Delta }_{{\mathbf{h}}_p}\circ \dots \circ {\Delta }_{{\mathbf{h}}_1}f$, where ${\mathbf{h}}_m:=h{\mathbf{e}}_{i_m}$, $m\in \overline{1,k}$. Moreover, the function ${\Delta }^{k}f$ is well defined at ${\mathbf{x}}_0$.

For all $p\in \overline{1,k}$, $\mathbf{x}\in {\mathbb{R}}^n$ and every function g such that the partial derivative ${\partial }_{i_p}g(\mathbf{z})$ exists for all $\mathbf{z}\in O_{\frac{p}{k}\delta }(\mathbf{x})$, using the commutativity of the corresponding operators and the Lagrange theorem, we have

$${\Delta }^{p}g(\mathbf{x})={\Delta }_{{\mathbf{h}}_p}({\Delta }^{p-1}g)(\mathbf{x})={\partial }_{i_p}({\Delta }^{p-1}g)(\mathbf{x}+{\theta }_p{\mathbf{h}}_p)h={\Delta }^{p-1}({\partial }_{i_p}g)(\mathbf{x}+{\theta }_p{\mathbf{h}}_p)h,$$

(2)

where ${\theta }_p\in (0,1)$. Applying (2) successively for $p:=k,k-1,\dots ,2$; $g:=f,{\partial }_{i_k}f,\dots ,{\partial }_{i_2\dots i_k}f$; $\mathbf{x}:={\mathbf{x}}_0,{\mathbf{x}}_0+{\theta }_{k}h{\mathbf{e}}_{i_k},\dots ,{\mathbf{x}}_0+{\theta }_{k}h{\mathbf{e}}_{i_k}+\dots +{\theta }_{2}h{\mathbf{e}}_{i_2}$, by induction we get

$${\Delta }^{k}f({\mathbf{x}}_0)={\Delta }_{{\mathbf{h}}_1}({\partial }_{i_2\dots i_k}f)({\mathbf{x}}_0+\mathbf{z})h^{k-1},$$

(3)

where $\mathbf{z}:=h{\displaystyle \sum _{p=2}^k}{\theta }_p{\mathbf{e}}_{i_p}$ and ${\theta }_p\in (0,1)$ for $p\in \overline{2,k}$. Denote by A the derivative of the function ${\partial }_{i_2\dots i_k}f$ at ${\mathbf{x}}_0$. For all ${\displaystyle h\in (0,\frac{\delta }{k})}$ consider the function $\Phi (h):={\Delta }^{k}f({\mathbf{x}}_0)$. Using (3) we get

$$\begin{array}{c}\Phi (h)={\Delta }_{{\mathbf{h}}_1}({\partial }_{i_2\dots i_k}f)({\mathbf{x}}_0+\mathbf{z})h^{k-1}=[{\partial }_{i_2\dots i_k}f({\mathbf{x}}_0+\mathbf{z}+{\mathbf{h}}_1)-{\partial }_{i_2\dots i_k}f({\mathbf{x}}_0+\mathbf{z})]h^{k-1}=\\ =[{\partial }_{i_2\dots i_k}f({\mathbf{x}}_0)+A(\mathbf{z}+{\mathbf{h}}_1)+o(\parallel \mathbf{z}+{\mathbf{h}}_1\parallel )-({\partial }_{i_2\dots i_k}f({\mathbf{x}}_0)+A\mathbf{z}+o(\parallel \mathbf{z}\parallel ))]h^{k-1}=\\ =[hA{\mathbf{e}}_{i_1}+o(h)]h^{k-1}=[{\partial }_{i_1\dots i_k}f({\mathbf{x}}_0)+o(1)]h^k.\end{array}$$

Using the commutativity of difference operators and acting completely similarly to the procedure described above, we obtain

$$\Phi (h)={\Delta }_{{\mathbf{h}}_{\sigma (k)}}\circ \dots \circ {\Delta }_{{\mathbf{h}}_{\sigma (1)}}f({\mathbf{x}}_0)=[{\partial }_{i_{\sigma (1)}\dots i_{\sigma (k)}}f({\mathbf{x}}_0)+o(1)]h^k$$

for all ${\displaystyle h\in (0,\frac{\delta }{k})}$. Multiplying the identity

$$\Phi (h)=[{\partial }_{i_1\dots i_k}f({\mathbf{x}}_0)+o(1)]h^k=[{\partial }_{i_{\sigma (1)}\dots i_{\sigma (k)}}f({\mathbf{x}}_0)+o(1)]h^k$$

by ${\displaystyle \frac{1}{h^k}}$ and passing to the limit as $h\to 0+$, we obtain the assertion of the theorem. □

Now we recall Taylor’s formula

$$f({\mathbf{x}}_0+\mathbf{h})=f({\mathbf{x}}_0)+\sum _{m=1}^k\frac{d^{m}f({\mathbf{x}}_0,\mathbf{h})}{m!}+r_k({\mathbf{x}}_0,\mathbf{h}),$$

(4)

where $\mathbf{h}=(h_1,\dots ,h_n)$, $d^{m}f({\mathbf{x}}_0,\mathbf{h}):={(h_1{\partial }_1+\cdots +h_n{\partial }_n)}^{m}f({\mathbf{x}}_0)$. Using Theorem 3 we can now prove the following lemma.

Lemma 1.

Let $m\ge 1$ and f be weakly m times differentiable by the Definition 1 at ${\mathbf{x}}_0$, then for all $p\in \overline{1,n}$ and for all $\mathbf{h}\in {\mathbb{R}}^n$ there holds the equality ${\displaystyle \frac{\partial }{\partial h_p}{d}^{m}f({\mathbf{x}}_0,\mathbf{h})=m{d}^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h})}$.

Proof. 

In case $m=1$ the lemma is obviously true. Let it be true for $m-1$, then we have

$$\begin{array}{c}{\displaystyle \frac{\partial }{\partial h_p}{d}^{m}f({\mathbf{x}}_0,\mathbf{h})=\frac{\partial }{\partial h_p}\left[{(h_1{\partial }_1+\dots +h_n{\partial }_n)}^{m-1}(h_1{\partial }_1+\dots +h_n{\partial }_n)f\right]({\mathbf{x}}_0)=}\hfill \\ {\displaystyle =\frac{\partial }{\partial h_p}\sum _{j=1}^n{h}_j{d}^{m-1}({\partial }_{j}f)({\mathbf{x}}_0,\mathbf{h})=d^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h})+}\\ {\displaystyle +\sum _{j=1}^n{h}_j\frac{\partial }{\partial h_p}{d}^{m-1}({\partial }_{j}f)({\mathbf{x}}_0,\mathbf{h})=d^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h})+}\\ +(m-1)\sum _{j=1}^n{h}_j{d}^{m-2}({\partial }_p{\partial }_{j}f)({\mathbf{x}}_0,\mathbf{h})\stackrel{\mathrm{Th}.3}{=}{d}^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h})+\\ \hfill +(m-1)d^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h})=m{d}^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,\mathbf{h}).\end{array}$$

 □

Finally we are ready to prove Taylor’s theorem.

Theorem 4(Taylor’s theorem with Peano remainder).

Let $k\ge 1$ and f be weakly k times differentiable by the Definition 1 at ${\mathbf{x}}_0$. Then Formula (4) holds, where $r_k({\mathbf{x}}_0,\mathbf{h})={o(\parallel \mathbf{h}\parallel }^k)$ as $\mathbf{h}\to \mathbf{0}$.

Proof. 

For $k=1$ the theorem is trivial. Let $k\ge 2$ and the theorem be true for $k-1$. Consider the function $\phi (\mathbf{h}):=f({\mathbf{x}}_0+\mathbf{h})-P_k({\mathbf{x}}_0,\mathbf{h})$. Then $\phi (\mathbf{0})=0$ and $r_k({\mathbf{x}}_0,\mathbf{h})=\phi (\mathbf{h})=\phi (\mathbf{h})-\phi (\mathbf{0})$. By the theorem hypothesis there exists $\delta >0$ such that all the partial derivatives of f, and therefore of $\phi$, of order $k-1$ are defined in some ball $O_{\delta }({\mathbf{x}}_0)$. For all $\mathbf{h}\in O_{\delta }(\mathbf{0})$ consider the equality

$$r_k({\mathbf{x}}_0,\mathbf{h})=\phi (\mathbf{h})-\phi (\mathbf{0})=\sum _{p=1}^n[\phi (0,\dots ,0,h_p,\dots ,h_n)-\phi (0,\dots ,0,h_{p+1},\dots ,h_n)],$$

(5)

where all the arguments of $\phi$ belong to $O_{\delta }(\mathbf{0})$. Using the convexity of $O_{\delta }(\mathbf{0})$, we apply Lagrange theorem to the pth square bracket from the equality (5) and take into account that ${\partial }_{p}f$ satisfies the theorem hypothesis for $k-1$:

$$\begin{array}{c}\phi (0,\dots ,0,h_p,\dots ,h_n)-\phi (0,\dots ,0,h_{p+1},\dots ,h_n)={\partial }_p\phi ({\xi }_p)h_p\hfill \\ {\displaystyle =[{\partial }_{p}f({\mathbf{x}}_0+{\xi }_p)-\sum _{m=1}^k\frac{{\displaystyle \frac{\partial }{\partial h_p}(d^{m}f)({\mathbf{x}}_0,{\xi }_p)}}{m!}]h_p\stackrel{\mathrm{Lemma}1}{=}[{\partial }_{p}f({\mathbf{x}}_0+{\xi }_p)-\sum _{m=1}^k\frac{d^{m-1}({\partial }_{p}f)({\mathbf{x}}_0,{\xi }_p)}{(m-1)!}]h_p}\\ {\displaystyle =[{\partial }_{p}f({\mathbf{x}}_0+{\xi }_p)-{\partial }_{p}f({\mathbf{x}}_0)-\sum _{q=1}^{k-1}\frac{d^q({\partial }_{p}f)({\mathbf{x}}_0,{\xi }_p)}{q!}]h_p=o(\parallel {\xi }_p{\parallel }^{k-1})h_p}\\ \hfill {\displaystyle ={o(\parallel \mathbf{h}\parallel }^{k-1})h_p={o(\parallel \mathbf{h}\parallel }^k)\frac{h_p}{\parallel \mathbf{h}\parallel }={o(\parallel \mathbf{h}\parallel }^k),}\end{array}$$

where ${\xi }_p=(0,\dots ,0,{\theta }_p{h}_p,h_{p+1},\dots ,h_n)$, ${\theta }_p\in (0,1)$. Substituting the expressions obtained for $p=1,2,\dots ,n$ into the Formula (5), we have $r_k({\mathbf{x}}_0,\mathbf{h})={\displaystyle \sum _{p=1}^n}{o(\parallel \mathbf{h}\parallel }^k{)=o(\parallel \mathbf{h}\parallel }^k)$ as $\mathbf{h}\to \mathbf{0}$, which completes the proof of the theorem. □

4. Conclusions and Further Generalizations

Theorems 3 and 4 motivate us to use the Definition 1 for multiple differentiability in the case $f:{\mathbb{R}}^n\to \mathbb{R}$, which is simpler on paper and in fact more general than the classical Definitions 2 and 3 (see Theorem 2), since in most cases the validity of these theorems is exactly what is required of a multiply differentiable function. Furthermore the results of Section 2 and Section 3 can be generalized in the sense of [7] to the case $f:H_1\times \dots \times H_n\to N$, where $H_1,\dots ,H_n,N$ are arbitrary normed spaces.