Next Article in Journal
Multi-Label Classification with Optimal Thresholding for Multi-Composition Spectroscopic Analysis
Previous Article in Journal
Towards Image Classification with Machine Learning Methodologies for Smartphones
Open AccessArticle

Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections

1
Department of Aerospace Engineering, Texas A&M University, College Station, TX 77843, USA
2
Mathematics Department, Blinn College, Bryan, TX 77802, USA
*
Author to whom correspondence should be addressed.
Mach. Learn. Knowl. Extr. 2019, 1(4), 1058-1083; https://doi.org/10.3390/make1040060
Received: 21 August 2019 / Revised: 2 October 2019 / Accepted: 5 October 2019 / Published: 9 October 2019
Differential equations (DEs) are used as numerical models to describe physical phenomena throughout the field of engineering and science, including heat and fluid flow, structural bending, and systems dynamics. While there are many other techniques for finding approximate solutions to these equations, this paper looks to compare the application of the Theory of Functional Connections (TFC) with one based on least-squares support vector machines (LS-SVM). The TFC method uses a constrained expression, an expression that always satisfies the DE constraints, which transforms the process of solving a DE into solving an unconstrained optimization problem that is ultimately solved via least-squares (LS). In addition to individual analysis, the two methods are merged into a new methodology, called constrained SVMs (CSVM), by incorporating the LS-SVM method into the TFC framework to solve unconstrained problems. Numerical tests are conducted on four sample problems: One first order linear ordinary differential equation (ODE), one first order nonlinear ODE, one second order linear ODE, and one two-dimensional linear partial differential equation (PDE). Using the LS-SVM method as a benchmark, a speed comparison is made for all the problems by timing the training period, and an accuracy comparison is made using the maximum error and mean squared error on the training and test sets. In general, TFC is shown to be slightly faster (by an order of magnitude or less) and more accurate (by multiple orders of magnitude) than the LS-SVM and CSVM approaches. View Full-Text
Keywords: differential equation; numerical methods; support vector machines; function approximation; least-squares; theory of functional connections differential equation; numerical methods; support vector machines; function approximation; least-squares; theory of functional connections
Show Figures

Figure 1

MDPI and ACS Style

Leake, C.; Johnston, H.; Smith, L.; Mortari, D. Analytically Embedding Differential Equation Constraints into Least Squares Support Vector Machines Using the Theory of Functional Connections. Mach. Learn. Knowl. Extr. 2019, 1, 1058-1083.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

1
Back to TopTop