#
Least-Squares Solution of Linear Differential Equations^{ †}

^{†}

## Abstract

**:**

## 1. Introduction

#### 1.1. Background on the Existing Approaches

**Collocation Techniques.**In these techniques, the solution components are approximated by piecewise polynomials on a mesh. The coefficients of the polynomials form the unknowns to be computed. The approximation to the solution must satisfy the constraint conditions and the DEs at collocation points in each mesh subinterval. We note that the placement of the collocation points is not arbitrary. A modified Newton-type method, known as quasi-linearization, is then used to solve the nonlinear equations for the polynomial coefficients. The mesh is then refined by trying to equidistribute the estimated error over the whole interval. An initial estimate of the solution across the mesh is also required. However, a common weakness of all these collocation techniques (low-order Taylor expansion models) is that they are not effective in enforcing algebraic constraints. Enforcing the DE constraints is the strength of the proposed method, as the DE constraints are embedded in the searched solution expressions. This means that the analytical solution obtained, even if completely wrong, perfectly satisfies all the DE constraints.**Spectral methods.**Spectral methods are used to numerically solve DEs by writing the solution as a sum of certain “basis functions” (e.g., Fourier series) and then choosing the coefficients in the sum in order to satisfy the DEs as well as possible (e.g., by least-squares). Additionally, in spectral methods, the constraints must then be enforced. This is done by replacing one (or more) of the least-squares equations with the constraint conditions. The proposed method proceeds in the reverse sequence. It first builds a special function, called a “constrained expression”, that has the boundary conditions embedded, no matter what the final estimation is. Then, it uses this expression to minimize the DE residuals by expressing the free function of the constrained expression, $g\left(t\right)$, as a sum of some basis functions (e.g., orthogonal polynomials). The resulting estimation always perfectly satisfies the boundary conditions as long as they are linear. To provide an example, a linear constraint involving four times (${t}_{1}$, ${t}_{2}$, ${t}_{3}$ and ${t}_{4}$) can be:$$\pi =7y\left({t}_{1}\right)-e\phantom{\rule{0.166667em}{0ex}}\dot{y}\left({t}_{1}\right)-2\phantom{\rule{0.166667em}{0ex}}y\left({t}_{2}\right)-5\phantom{\rule{0.166667em}{0ex}}\dot{y}\left({t}_{3}\right)+\sqrt{3}\phantom{\rule{0.166667em}{0ex}}\ddot{y}\left({t}_{3}\right)-y\left({t}_{4}\right)$$It is not clear how the spectral methods can enforce this kind of general linear constraint. Additionally, the spectral methods provide solutions approximating the constraints.

#### 1.2. The Idea Behind the Proposed Method

## 2. Least-Squares Solution of IVPs

#### 2.1. Accuracy Tests

#### 2.2. Second-Order DE Subject to $y\left({t}_{0}\right)={y}_{0}$ and $\ddot{y}\left({t}_{0}\right)={\ddot{y}}_{0}$

#### 2.3. Second-Order DE Subject to $\dot{y}\left({t}_{0}\right)={\dot{y}}_{0}$ and $\ddot{y}\left({t}_{0}\right)={\ddot{y}}_{0}$

## 3. Least-Squares Solutions of BVPs

#### 3.1. Numerical Accuracy Tests for a BVP with Known Solution

#### 3.2. Numerical Accuracy Tests for a BVP with Unknown Solution

#### 3.3. Numerical Accuracy Tests for a BVP with No Solution

#### 3.4. Numerical Accuracy Tests for a BVP with Infinite Solutions

#### 3.5. Constrained Expressions for BVP with Different Constraints

- (1)
- The first constraints considered are the function at the initial time, $y\left({t}_{0}\right)={y}_{0}$, and the first derivative at the final time, $\dot{y}\left({t}_{f}\right)={\dot{y}}_{f}$. For this case, the final constraint in terms of the new variable, $x\in [-1,+1]$, becomes ${y}_{f}^{\prime}={\dot{y}}_{f}/c$, and the constrained expression:$$y\left(x\right)=g\left(x\right)+({y}_{0}-{g}_{0})+(x+1)({y}_{f}^{\prime}-{g}_{f}^{\prime})$$$${c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}\phantom{\rule{0.166667em}{0ex}}{g}^{\u2033}+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}({g}^{\prime}-{g}_{f}^{\prime})+{f}_{0}[g-{g}_{0}-{g}_{f}^{\prime}(x+1)]=f-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\phantom{\rule{0.166667em}{0ex}}{y}_{f}^{\prime}-{f}_{0}[{y}_{0}+{y}_{f}^{\prime}(x+1)]$$
- (2)
- The second constraints are the function at the initial time, $y\left({t}_{0}\right)={y}_{0}$, and the second derivative at the final time, $\ddot{y}\left({t}_{f}\right)={\ddot{y}}_{f}$. For this case, the final constraint in terms of the new variable is ${y}_{f}^{\u2033}={\ddot{y}}_{f}/{c}^{2}$, and the constrained expression is:$$y\left(x\right)=g\left(x\right)-x\phantom{\rule{0.166667em}{0ex}}({y}_{0}-{g}_{0})+{\displaystyle \frac{{x}^{2}+x}{2}}\phantom{\rule{0.166667em}{0ex}}({y}_{f}^{\u2033}-{g}_{f}^{\u2033})$$Substituting this into Equation (5), we obtain:$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}({g}^{\u2033}-{g}_{f}^{\u2033})+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left({g}^{\prime}+{g}_{0}-{g}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{2x+1}{2}}\right)+{f}_{0}\left(g+{g}_{0}\phantom{\rule{0.166667em}{0ex}}x-{g}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{{x}^{2}+x}{2}}\right)\hfill \\ =f-{c}^{2}\phantom{\rule{0.166667em}{0ex}}{y}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{f}_{2}-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left(-{y}_{0}+{y}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{2x+1}{2}}\right)-{f}_{0}\left(-{y}_{0}\phantom{\rule{0.166667em}{0ex}}x+{y}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{{x}^{2}+x}{2}}\right)\hfill \end{array}$$Then, it follows as in the first case.
- (3)
- The third constraints are the first derivative at the initial time, $\dot{y}\left({t}_{0}\right)={\dot{y}}_{0}$, and the function at the final time, $y\left({t}_{f}\right)={y}_{f}$. For this case, the initial constraint in terms of the new variable is ${y}_{0}^{\prime}={\dot{y}}_{0}/c$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+({y}_{f}-{g}_{f})+(x-1)({y}_{0}^{\prime}-{g}_{0}^{\prime})$$$${c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}\phantom{\rule{0.166667em}{0ex}}{g}^{\u2033}+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}({g}^{\prime}-{g}_{0}^{\prime})+{f}_{0}[g-{g}_{f}-{g}_{0}^{\prime}(x-1)]=f-c\phantom{\rule{0.166667em}{0ex}}{y}_{0}^{\prime}\phantom{\rule{0.166667em}{0ex}}{f}_{1}-{f}_{0}[{y}_{f}+{y}_{0}^{\prime}(x-1)]$$
- (4)
- The fourth constraints are the first derivative at the initial and final times, $\dot{y}\left({t}_{0}\right)={\dot{y}}_{0}$ and $\dot{y}\left({t}_{f}\right)={\dot{y}}_{f}$. This special equation is known as Mathieu’s DE [9,10]. For this case, the constraints in terms of the new variable are ${y}_{0}^{\prime}={\dot{y}}_{0}/c$ and ${y}_{f}^{\prime}={\dot{y}}_{f}/c$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+{\displaystyle \frac{x}{2}}\left(1-{\displaystyle \frac{x}{2}}\right)({y}_{0}^{\prime}-{g}_{0}^{\prime})+{\displaystyle \frac{x}{2}}\left(1+{\displaystyle \frac{x}{2}}\right)({y}_{f}^{\prime}-{g}_{f}^{\prime})$$$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}\left({g}^{\u2033}+{\displaystyle \frac{{g}_{0}^{\prime}-{g}_{f}^{\prime}}{2}}\right)+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left[{g}^{\prime}-{\displaystyle \frac{(1-x){g}_{0}^{\prime}+(x+1){g}_{f}^{\prime}}{2}}\right]+{f}_{0}\left\{g-{\displaystyle \frac{x}{2}}\left[\left(1-{\displaystyle \frac{x}{2}}\right){g}_{0}^{\prime}+\left({\displaystyle \frac{x}{2}}+1\right){g}_{f}^{\prime}\right]\right\}\hfill \\ =f-{\displaystyle \frac{{c}^{2}}{2}}({y}_{f}^{\prime}-{y}_{0}^{\prime}){f}_{2}-{\displaystyle \frac{c}{2}}{f}_{1}\left[(1-x){y}_{0}^{\prime}+(x+1){y}_{f}^{\prime}\right]-{f}_{0}{\displaystyle \frac{x}{2}}\left[\left(1-{\displaystyle \frac{x}{2}}\right){y}_{0}^{\prime}+\left({\displaystyle \frac{x}{2}}+1\right){y}_{f}^{\prime}\right]\hfill \end{array}$$
- (5)
- The fifth constraints are the first derivative at the initial time, $\dot{y}\left({t}_{0}\right)={\dot{y}}_{0}$, and the second derivative at the final time, $\ddot{y}\left({t}_{f}\right)={\ddot{y}}_{f}$. For this case, the constraints in terms of the new variable are ${y}_{0}^{\prime}={\dot{y}}_{0}/c$ and ${y}_{f}^{\u2033}={\ddot{y}}_{f}/{c}^{2}$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+x\phantom{\rule{0.166667em}{0ex}}({y}_{0}^{\prime}-{g}_{0}^{\prime})+x\phantom{\rule{0.166667em}{0ex}}\left({\displaystyle \frac{x}{2}}+1\right)({y}_{f}^{\u2033}-{g}_{f}^{\u2033})$$$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}({g}^{\u2033}-{g}_{f}^{\u2033})+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}[{g}^{\prime}-{g}_{0}^{\prime}-{g}_{f}^{\u2033}(x+1)]+{f}_{0}\left[g-{g}_{0}^{\prime}\phantom{\rule{0.166667em}{0ex}}x-{g}_{f}^{\u2033}\left({\displaystyle \frac{x}{2}}+1\right)x\right]=\\ =f-{c}^{2}\phantom{\rule{0.166667em}{0ex}}{y}_{f}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{f}_{2}-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}[{y}_{0}^{\prime}+{y}_{f}^{\u2033}(x+1)]-{f}_{0}\left[x\phantom{\rule{0.166667em}{0ex}}{y}_{0}^{\prime}+\left({\displaystyle \frac{x}{2}}+1\right)x\phantom{\rule{0.166667em}{0ex}}{y}_{f}^{\u2033}\right]\end{array}$$
- (6)
- The sixth constraints are the second derivative at the initial time, $\ddot{y}\left({t}_{0}\right)={\ddot{y}}_{0}$, and the function at the final time, $y\left({t}_{f}\right)={y}_{f}$. For this case, the first constraint in terms of the new variable is ${y}_{0}^{\u2033}={\ddot{y}}_{0}/{c}^{2}$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+x\phantom{\rule{0.166667em}{0ex}}({y}_{f}-{g}_{f})+{\displaystyle \frac{x}{2}}\phantom{\rule{0.166667em}{0ex}}(x-1)({y}_{0}^{\u2033}-{g}_{0}^{\u2033})$$$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}({g}^{\u2033}-{g}_{0}^{\u2033})+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left({g}^{\prime}-{g}_{f}-{g}_{0}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{2x-1}{2}}\right)+{f}_{0}\left(g-{g}_{f}\phantom{\rule{0.166667em}{0ex}}x-{g}_{0}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{{x}^{2}-x}{2}}\right)\\ =f-{c}^{2}\phantom{\rule{0.166667em}{0ex}}{y}_{0}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{f}_{2}-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left({y}_{f}+{y}_{0}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{\displaystyle \frac{2x-1}{2}}\right)-{f}_{0}\left({y}_{f}+{y}_{0}^{\u2033}{\displaystyle \frac{{x}^{2}-x}{2}}\right)\end{array}$$
- (7)
- The seventh constraints are the second derivative at the initial time, $\ddot{y}\left({t}_{0}\right)={\ddot{y}}_{0}$, and the first derivative at the final time, $\dot{y}\left({t}_{f}\right)={\dot{y}}_{f}$. For this case, the constraints in terms of the new variable are ${y}_{0}^{\u2033}={\ddot{y}}_{0}/{c}^{2}$ and ${y}_{f}^{\prime}={\dot{y}}_{f}/c$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+x\phantom{\rule{0.166667em}{0ex}}({y}_{f}^{\prime}-{g}_{f}^{\prime})+{\displaystyle \frac{x}{2}}\phantom{\rule{0.166667em}{0ex}}(x-2)({y}_{0}^{\u2033}-{g}_{0}^{\u2033})$$$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}({g}^{\u2033}-{g}_{0}^{\u2033})+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}[{g}^{\prime}-{g}_{f}^{\prime}-{g}_{0}^{\u2033}(x-1)]+{f}_{0}\left[g-{g}_{f}^{\prime}\phantom{\rule{0.166667em}{0ex}}x-{g}_{0}^{\u2033}\left({\displaystyle \frac{{x}^{2}}{2}}-x\right)\right]=\\ =f-{c}^{2}{y}_{0}^{\u2033}\phantom{\rule{0.166667em}{0ex}}{f}_{2}-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left[{y}_{f}^{\prime}+{y}_{0}^{\u2033}(x-1)\right]-{f}_{0}\left[{y}_{f}^{\prime}\phantom{\rule{0.166667em}{0ex}}x+{y}_{0}^{\u2033}\left({\displaystyle \frac{{x}^{2}}{2}}-x\right)\right]\end{array}$$
- (8)
- The final (eighth) constraints are the second derivative at the initial and final times, $\ddot{y}\left({t}_{0}\right)={\ddot{y}}_{0}$ and $\ddot{y}\left({t}_{f}\right)={\ddot{y}}_{f}$. For this case, the constraints in terms of the new variable are ${y}_{0}^{\u2033}={\ddot{y}}_{0}/{c}^{2}$ and ${y}_{f}^{\u2033}={\ddot{y}}_{f}/{c}^{2}$, while the constrained equation:$$y\left(x\right)=g\left(x\right)+{\displaystyle \frac{{x}^{2}}{12}}\phantom{\rule{0.166667em}{0ex}}(3-x)({y}_{0}^{\u2033}-{g}_{0}^{\u2033})+{\displaystyle \frac{{x}^{2}}{12}}\phantom{\rule{0.166667em}{0ex}}(3+x)({y}_{f}^{\u2033}-{g}_{f}^{\u2033})$$$$\begin{array}{c}{c}^{2}\phantom{\rule{0.166667em}{0ex}}{f}_{2}\left[{g}^{\u2033}-{\displaystyle \frac{(1-x){g}_{0}^{\u2033}+(x+1){g}_{f}^{\u2033}}{2}}\right]+c\phantom{\rule{0.166667em}{0ex}}{f}_{1}\left[{g}^{\prime}-{\displaystyle \frac{(2x-{x}^{2}){g}_{0}^{\u2033}+({x}^{2}+2x){g}_{f}^{\u2033}}{4}}\right]\hfill \\ +{f}_{0}\left[g-{\displaystyle \frac{(3{x}^{2}-{x}^{3}){g}_{0}^{\u2033}+({x}^{3}+3{x}^{2}){g}_{f}^{\u2033}}{12}}\right]\hfill \\ =f-{\displaystyle \frac{{c}^{2}}{2}}{f}_{2}[{y}_{0}^{\u2033}(1-x)+{y}_{f}^{\u2033}(x+1)]-c\phantom{\rule{0.166667em}{0ex}}{f}_{1}{\displaystyle \frac{(2x-{x}^{2}){y}_{0}^{\u2033}+({x}^{2}+2x){y}_{f}^{\u2033}}{4}}-{f}_{0}{\displaystyle \frac{(3{x}^{2}-{x}^{3}){y}_{0}^{\u2033}+({x}^{3}+3{x}^{2}){y}_{f}^{\u2033}}{12}}\hfill \end{array}$$

## 4. MVP Example

## 5. Conclusions

- (1)
- Extension to weighted least-squares.
- (2)
- Error-bound estimates for the problems admitting solutions.
- (3)
- Nonuniform distribution of points and optimal distributions of points to increase accuracy in specific ranges of interest.
- (4)
- Comparisons between different function bases and identification of an optimal function base (if it exists).
- (5)
- Analysis using Fourier bases.
- (6)
- Accuracy analysis of number of basis functions versus points distribution.
- (7)
- Extension to nonlinear DEs.
- (8)
- Extension to partial DEs.
- (9)
- Extension to nonlinear constraints.

## Acknowledgments

## Conflicts of Interest

## Abbreviations

DE | Differential equation |

IVP | Initial value problem |

BVP | Boundary value problem |

MVP | Multi-point value problem |

## References

- Dormand, J.R.; Prince, P.J. A Family of Embedded Runge-Kutta Formulae. J. Comp. Appl. Math.
**1980**, 6, 19–26. [Google Scholar] [CrossRef] - Strang, G. Differential Equations and Linear Algebra; Wellesley-Cambridge Press: Wellesley, MA, USA, 2015; ISBN 9780980232790. [Google Scholar]
- Lin, Y.; Enszer, J.A.; Stadtherr, M.A. Enclosing all Solutions of Two-Point Boundary Value Problems for ODEs. Comput. Chem. Eng.
**2008**, 32, 1714–1725. [Google Scholar] [CrossRef] - Berry, M.M.; Healy, L.M. Implementation of Gauss-Jackson integration for orbit propagation. J. Astronaut. Sci.
**2004**, 52, 331–357. [Google Scholar] - Elgohary, T.A.; Junkins, J.L.; Atluri, S.N. An RBF-Collocation Algorithm for Orbit Propagation. In Proceedings of the 25th AAS/AIAA Space Flight Mechanics Meeting, Williamsburg, VA, USA, 11–15 January 2015. [Google Scholar]
- Junkins, J.L.; Bani Younes, A.; Woollands, R.; Bai, X. Picard Iteration, Chebyshev Polynomials, and Chebyshev Picard Methods: Application in Astrodynamics. J. Astronaut. Sci.
**2015**, 60, 623–653. [Google Scholar] [CrossRef] - Jorba, A.; Zou, M. A software package for the numerical integration of ODEs by means of high-order Taylor methods. Exp. Math.
**2005**, 14, 99–117. [Google Scholar] [CrossRef] - Mortari, D. The Theory of Connections. Part 1: Connecting Points. In Proceedings of the 27th AAS/AIAA Space Flight Mechanics Meeting Conference, San Antonio, TX, USA, 5–9 February 2017. [Google Scholar]
- Mathieu, E. Mémoire sur Le Mouvement Vibratoire d’une Membrane de Forme Elliptique. J. Math. Pures Appl.
**1868**, 13, 137–203. [Google Scholar] - Meixner, J.; Schäfke, F.W. Mathieusche Funktionen und Sphäroidfunktionen; Springer: Berlin, Germany, 1954. [Google Scholar]

**Table 1.**Simplest selection of $p\left(x\right)$ and $q\left(x\right)$ for various initial value problem and boundary value problem constraints.

Differential Equation Constraints | $\mathit{p}\left(\mathit{x}\right)$ | $\mathit{q}\left(\mathit{x}\right)$ |
---|---|---|

$[{y}_{0},{y}_{0}^{\prime}]$ or $[{y}_{0},{y}_{f}]$ or $[{y}_{0},{y}_{f}^{\prime}]$ or $[{y}_{0}^{\prime},{y}_{f}]$ | 1 | x |

$[{y}_{0},{y}_{0}^{\u2033}]$ or $[{y}_{0},{y}_{f}^{\u2033}]$ or $[{y}_{0}^{\u2033},{y}_{f}]$ | 1 | ${x}^{2}/2$ |

$[{y}_{0}^{\prime},{y}_{0}^{\u2033}]$ or $[{y}_{0}^{\prime},{y}_{f}^{\prime}]$ or $[{y}_{0}^{\prime},{y}_{f}^{\u2033}]$ or $[{y}_{0}^{\u2033},{y}_{f}^{\prime}]$ | x | ${x}^{2}/2$ |

$[{y}_{0}^{\u2033},{y}_{f}^{\u2033}]$ | ${x}^{2}/2$ | ${x}^{3}/3$ |

© 2017 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mortari, D. Least-Squares Solution of Linear Differential Equations. *Mathematics* **2017**, *5*, 48.
https://doi.org/10.3390/math5040048

**AMA Style**

Mortari D. Least-Squares Solution of Linear Differential Equations. *Mathematics*. 2017; 5(4):48.
https://doi.org/10.3390/math5040048

**Chicago/Turabian Style**

Mortari, Daniele. 2017. "Least-Squares Solution of Linear Differential Equations" *Mathematics* 5, no. 4: 48.
https://doi.org/10.3390/math5040048