Why Do Big Data and Machine Learning Entail the Fractional Dynamics?
Abstract
:1. Fractional Calculus (FC) and Fractional-Order Thinking (FOT)
1.1. Complexity and Inverse Power Laws
1.2. Heavy-Tailed Distributions
1.2.1. Lévy Distribution
1.2.2. Mittag–Leffler PDF
1.2.3. Weibull Distribution
1.2.4. Cauchy Distribution
1.2.5. Pareto Distribution
1.2.6. The -Stable Distribution
1.3. Mixture Distributions
1.3.1. Gaussian Distribution
1.3.2. Laplace Distribution
1.4. IPL Tail-Index Analysis
2. Big Data, Variability and FC
2.1. Hurst Parameter, fGn, and fBm
2.2. Fractional Lower-Order Moments (FLOMs)
2.3. Fractional Autoregressive Integrated Moving Average (FARIMA) and Gegenbauer Autoregressive Moving Average (GARMA)
2.4. Continuous Time Random Walk (CTRW)
2.5. Unmanned Aerial Vehicles (UAVs) and Precision Agriculture
3. Optimal Machine Learning and Optimal Randomness
- What is the optimal way to optimize?
- What is the more optimal way to optimize?
- Can we demand “more optimal machine learning”, for example, deep learning with the minimum/smallest labeled data)?
3.1. Derivative-Free Methods
3.2. The Gradient-Based Methods
Nesterov Accelerated Gradient Descent (NAGD)
3.3. What Can the Control Community Offer to ML?
4. A Case Study of Machine Learning with Fractional Calculus: A Stochastic Configuration Network with Heavytailedness
4.1. Stochastic Configuration Network (SCN)
4.2. SCN with Heavy-Tailed PDFs
4.3. A Regression Model and Parameter Tuning
Performance Comparison among SCNs with Heavy-Tailed PDFs
4.4. MNIST Handwritten Digit Classification
Performance Comparison among SCNs on MNIST
5. Take-Home Messages and Looking into the Future: Fractional Calculus Is Physics Informed
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ACF | Auto-Correlation Function |
AI | Artificial Intelligence |
ARMA | Autoregression and Moving Average |
CLT | Classical Central Limit Theorem |
CS | Cuckoo Search |
CTRW | Continuous Time Random Work |
EOM | Equation of Motion |
fBm | Fractional Brownian Motion |
fGn | Fractional Gaussian Noise |
FARIMA | Fractional Autoregressive Integrated Moving Average |
FC | Fractional Calculus |
FIGARCH | Fractional Integral Generalized Autoregressive Conditional Heteroscedasticity |
FLOM | Fractional Lower-Order Moments |
FOCV | Fractional-Order Calculus of Variation |
FODA | Fractional-Order Data Analytics |
FOEL | Fractional-Order Euler–Lagrange |
FOT | Fractional-Order Thinking |
GARMA | Gegenbauer Autoregressive Moving Average |
GD | Gradient Descent |
GDM | Gradient Descent Momentum |
GEV | Generalized Extreme Value |
IMP | Internal Model Principle |
IPL | Inverse Power Law |
ISE | Integral Squared Error |
LGD | Long Range Dependence |
LTI | Linear Time Invariant |
MAD | Modeling, Analysis and Design |
ML | Machine Learning |
MLL | Mittag–Leffler Law |
MNIST | Modified National Institute of Standards and Technology Database |
NAGD | Nesterov Accelerated Gradient Descent |
NDVI | Normalized Difference Vegetation Index |
NILT | Numerical Inverse Laplace Transform |
NN | Neural Networks |
PA | Precision Agriculture |
Probability Density Function | |
PID | Proportional, Integral, Derivative |
PSO | Particle Swarm Optimization |
RBF | Randomized Radial Basis Function (RBF) Networks |
RGB | Red, Green, Blue |
RMSE | Root Mean Squared Error |
RVFL | Random Vector Functional Link |
RW-FNN | Feed-Forward Networks with Random Weights |
SCN | Stochastic Configuration Network |
SGD | Stochastic Gradient Descent |
SLFNNs | Single-Layer Feed-Forward Neural Networks |
UAVs | Unmanned Aerial Vehicles |
USDA | United States Department of Agriculture |
wGn | White Gaussian Noise |
Appendix A. SCN Codes
References
- Vinagre, B.M.; Chen, Y. Lecture notes on fractional calculus applications in automatic control and robotics. In Proceedings of the 41st IEEE CDC Tutorial Workshop, Las Vegas, NV, USA, 9 December 2002; pp. 1–310. [Google Scholar]
- Valério, D.; Machado, J.; Kiryakova, V. Some Pioneers of the Applications of Fractional Calculus. Fract. Calc. Appl. Anal. 2014, 17, 552–578. [Google Scholar] [CrossRef] [Green Version]
- Abel, N. Solution of a Couple of Problems by Means of Definite Integrals. Mag. Naturvidenskaberne 1823, 2, 2. [Google Scholar]
- Podlubny, I.; Magin, R.L.; Trymorush, I. Niels Henrik Abel and the Birth of Fractional Calculus. Fract. Calc. Appl. Anal. 2017, 20, 1068–1075. [Google Scholar] [CrossRef] [Green Version]
- Ross, B. The Development of Fractional Calculus 1695–1900. Hist. Math. 1977, 4, 75–89. [Google Scholar] [CrossRef] [Green Version]
- Tarasov, V.E. Fractional Dynamics: Applications of Fractional Calculus to Dynamics of Particles, Fields and Media; Springer Science & Business Media: Berlin, Germany, 2011. [Google Scholar]
- Klafter, J.; Lim, S.; Metzler, R. Fractional Dynamics: Recent Advances; World Scientific: Singapore, 2012. [Google Scholar]
- Pramukkul, P.; Svenkeson, A.; Grigolini, P.; Bologna, M.; West, B. Complexity and the Fractional Calculus. Adv. Math. Phys. 2013, 2013, 498789. [Google Scholar] [CrossRef] [Green Version]
- Chen, D.; Xue, D.; Chen, Y. More optimal image processing by fractional order differentiation and fractional order partial differential equations. In Proceedings of the International Symposium on Fractional PDEs, Newport, RI, USA, 3–5 June 2013. [Google Scholar]
- Chen, D.; Sun, S.; Zhang, C.; Chen, Y.; Xue, D. Fractional-order TV-L 2 Model for Image Denoising. Cent. Eur. J. Phys. 2013, 11, 1414–1422. [Google Scholar] [CrossRef] [Green Version]
- Yang, Q.; Chen, D.; Zhao, T.; Chen, Y. Fractional Calculus in Image Processing: A Review. Fract. Calc. Appl. Anal. 2016, 19, 1222–1249. [Google Scholar] [CrossRef] [Green Version]
- Seshadri, V.; West, B.J. Fractal dimensionality of Lévy processes. Proc. Natl. Acad. Sci. USA 1982, 79, 4501. [Google Scholar] [CrossRef] [Green Version]
- Metzler, R.; Klafter, J. The Random Walk’s Guide to Anomalous Diffusion: A Fractional Dynamics Approach. Phys. Rep. 2000, 339, 1–77. [Google Scholar] [CrossRef]
- Metzler, R.; Glöckle, W.G.; Nonnenmacher, T.F. Fractional Model Equation for Anomalous Diffusion. Phys. A Stat. Mech. Appl. 1994, 211, 13–24. [Google Scholar] [CrossRef]
- Sheng, H.; Chen, Y.; Qiu, T. Fractional Processes and Fractional-Order Signal Processing: Techniques and Applications; Springer Science & Business Media: Berlin, Germany, 2011. [Google Scholar]
- Mandelbrot, B.B.; Wallis, J.R. Robustness of the Rescaled Range R/S in the Measurement of Noncyclic Long Run Statistical Dependence. Water Resour. Res. 1969, 5, 967–988. [Google Scholar] [CrossRef]
- Geweke, J.; Porter-Hudak, S. The Estimation and Application of Long Memory Time Series Models. J. Time Ser. Anal. 1983, 4, 221–238. [Google Scholar] [CrossRef]
- Liu, K.; Chen, Y.; Zhang, X. An Evaluation of ARFIMA (Autoregressive Fractional Integral Moving Average) Programs. Axioms 2017, 6, 16. [Google Scholar] [CrossRef] [Green Version]
- Montroll, E.W.; Weiss, G.H. Random Walks on Lattices. II. J. Math. Phys. 1965, 6, 167–181. [Google Scholar] [CrossRef]
- Liakos, K.G.; Busato, P.; Moshou, D.; Pearson, S.; Bochtis, D. Machine Learning in Agriculture: A Review. Sensors 2018, 18, 2674. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nesterov, Y. A Method for Unconstrained Convex Minimization Problem with the Rate of Convergence O (1/k2). Doklady an Ussr 1983, 269, 543–547. [Google Scholar]
- Montroll, E.W.; West, B.J. On An Enriched Collection of Stochastic Processes. Fluct. Phenom. 1979, 66, 61. [Google Scholar]
- Francis, B.A.; Wonham, W.M. The Internal Model Principle of Control Theory. Automatica 1976, 12, 457–465. [Google Scholar] [CrossRef]
- Zadeh, L.A. Fuzzy Logic. Computer 1988, 21, 83–93. [Google Scholar] [CrossRef]
- Unser, M.; Blu, T. Fractional Splines and Wavelets. SIAM Rev. 2000, 42, 43–67. [Google Scholar] [CrossRef] [Green Version]
- Samoradnitsky, G. Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance; Routledge: Oxford, UK, 2017. [Google Scholar]
- Crovella, M.E.; Bestavros, A. Self-similarity in World Wide Web Traffic: Evidence and Possible Causes. IEEE/ACM Trans. Netw. 1997, 5, 835–846. [Google Scholar] [CrossRef] [Green Version]
- Burnecki, K.; Weron, A. Levy Stable Processes. From Stationary to Self-similar Dynamics and Back. An Application to Finance. Acta Phys. Pol. Ser. B 2004, 35, 1343–1358. [Google Scholar]
- Pesquet-Popescu, B.; Pesquet, J.C. Synthesis of Bidimensional α-stable Models with Long-range Dependence. Signal Process. 2002, 82, 1927–1940. [Google Scholar] [CrossRef]
- Hartley, T.T.; Lorenzo, C.F. Fractional-order System Identification Based on Continuous Order-distributions. Signal Process. 2003, 83, 2287–2300. [Google Scholar] [CrossRef]
- Wolpert, R.L.; Taqqu, M.S. Fractional Ornstein–Uhlenbeck Lévy Processes and the Telecom Process: Upstairs and Downstairs. Signal Process. 2005, 85, 1523–1545. [Google Scholar] [CrossRef]
- Bahg, G.; Evans, D.G.; Galdo, M.; Turner, B.M. Gaussian process linking functions for mind, brain, and behavior. Proc. Natl. Acad. Sci. USA 2020, 117, 29398–29406. [Google Scholar] [CrossRef]
- West, B.J.; Geneston, E.L.; Grigolini, P. Maximizing Information Exchange between Complex Networks. Phys. Rep. 2008, 468, 1–99. [Google Scholar] [CrossRef]
- West, B.J. Sir Isaac Newton Stranger in a Strange Land. Entropy 2020, 22, 1204. [Google Scholar] [CrossRef] [PubMed]
- Csete, M.; Doyle, J. Bow Ties, Metabolism and Disease. Trends Biotechnol. 2004, 22, 446–450. [Google Scholar] [CrossRef]
- Zhao, J.; Yu, H.; Luo, J.H.; Cao, Z.W.; Li, Y.X. Hierarchical Modularity of Nested Bow-ties in Metabolic Networks. BMC Bioinform. 2006, 7, 1–16. [Google Scholar] [CrossRef] [Green Version]
- Doyle, J. Universal Laws and Architectures. Available online: http://www.ieeecss-oll.org/lecture/universal-laws-and-architectures. (accessed on 2 February 2021).
- Doyle, J.C.; Csete, M. Architecture, Constraints, and Behavior. Proc. Natl. Acad. Sci. USA 2011, 108, 15624–15630. [Google Scholar] [CrossRef] [Green Version]
- Sheng, H.; Chen, Y.Q.; Qiu, T. Heavy-tailed Distribution and Local Long Memory in Time Series of Molecular Motion on the Cell Membrane. Fluct. Noise Lett. 2011, 10, 93–119. [Google Scholar] [CrossRef]
- Graves, T.; Gramacy, R.; Watkins, N.; Franzke, C. A Brief History of Long Memory: Hurst, Mandelbrot and the Road to ARFIMA, 1951–1980. Entropy 2017, 19, 437. [Google Scholar] [CrossRef] [Green Version]
- West, B.J.; Grigolini, P. Complex Webs: Anticipating the Improbable; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
- Barabási, A.L.; Albert, R. Emergence of Scaling in Random Networks. Science 1999, 286, 509–512. [Google Scholar] [CrossRef] [Green Version]
- Sun, W.; Li, Y.; Li, C.; Chen, Y. Convergence Speed of a Fractional Order Consensus Algorithm over Undirected Scale-free Networks. Asian J. Control 2011, 13, 936–946. [Google Scholar] [CrossRef]
- Li, M. Modeling Autocorrelation Functions of Long-range Dependent Teletraffic Series Based on Optimal Approximation in Hilbert Space—A Further Study. Appl. Math. Model. 2007, 31, 625–631. [Google Scholar] [CrossRef]
- Zhao, Z.; Guo, Q.; Li, C. A Fractional Model for the Allometric Scaling Laws. Open Appl. Math. J. 2008, 2, 26–30. [Google Scholar] [CrossRef]
- Sun, H.; Chen, Y.; Chen, W. Random-order Fractional Differential Equation Models. Signal Process. 2011, 91, 525–530. [Google Scholar] [CrossRef]
- Kello, C.T.; Brown, G.D.; Ferrer-i Cancho, R.; Holden, J.G.; Linkenkaer-Hansen, K.; Rhodes, T.; Van Orden, G.C. Scaling Laws in Cognitive Sciences. Trends Cogn. Sci. 2010, 14, 223–232. [Google Scholar] [CrossRef] [PubMed]
- Gorenflo, R.; Mainardi, F. Fractional Calculus and Stable Probability Distributions. Arch. Mech. 1998, 50, 377–388. [Google Scholar]
- Mainardi, F. The Fundamental Solutions for the Fractional Diffusion-wave Equation. Appl. Math. Lett. 1996, 9, 23–28. [Google Scholar] [CrossRef] [Green Version]
- Luchko, Y.; Mainardi, F.; Povstenko, Y. Propagation Speed of the Maximum of the Fundamental Solution to the Fractional Diffusion–wave Equation. Comput. Math. Appl. 2013, 66, 774–784. [Google Scholar] [CrossRef]
- Luchko, Y.; Mainardi, F. Some Properties of the Fundamental Solution to the Signalling Problem for the Fractional Diffusion-wave Equation. Open Phys. 2013, 11, 666–675. [Google Scholar] [CrossRef]
- Luchko, Y.; Mainardi, F. Cauchy and Signaling Problems for the Time-fractional Diffusion-wave Equation. J. Vib. Acoust. 2014, 136, 050904. [Google Scholar] [CrossRef] [Green Version]
- Li, Z.; Liu, L.; Dehghan, S.; Chen, Y.; Xue, D. A Review and Evaluation of Numerical Tools for Fractional Calculus and Fractional Order Controls. Int. J. Control 2017, 90, 1165–1181. [Google Scholar] [CrossRef] [Green Version]
- Asmussen, S. Steady-state Properties of GI/G/1. In Applied Probability and Queues; Springer: New York, NY, USA, 2003; pp. 266–301. [Google Scholar]
- Bernardi, M.; Petrella, L. Interconnected Risk Contributions: A Heavy-tail Approach to Analyze US Financial Sectors. J. Risk Financ. Manag. 2015, 8, 198–226. [Google Scholar] [CrossRef]
- Ahn, S.; Kim, J.H.; Ramaswami, V. A New Class of Models for Heavy Tailed Distributions in Finance and Insurance Risk. Insur. Math. Econ. 2012, 51, 43–52. [Google Scholar] [CrossRef]
- Resnick, S.I. Heavy-tail Phenomena: Probabilistic and Statistical Modeling; Springer Science & Business Media: Berlin, Germany, 2007. [Google Scholar]
- Rolski, T.; Schmidli, H.; Schmidt, V.; Teugels, J.L. Stochastic Processes for Insurance and Finance; John Wiley & Sons: Hoboken, NJ, USA, 2009; Volume 505. [Google Scholar]
- Foss, S.; Korshunov, D.; Zachary, S. An Introduction to Heavy-Tailed and Subexponential Distributions; Springer: Berlin, Germany, 2011; Volume 6. [Google Scholar]
- Niu, H.; Chen, Y.; Chen, Y. Fractional-order extreme learning machine with Mittag-Leffler distribution. In Proceedings of the ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Anaheim, CA, USA, 18–21 August 2019. [Google Scholar]
- Hariya, Y.; Kurihara, T.; Shindo, T.; Jin’no, K. Lévy flight PSO. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 25–28 May 2015. [Google Scholar]
- Yang, X.S. Nature-Inspired Metaheuristic Algorithms; Luniver Press: London, UK, 2010. [Google Scholar]
- Yang, X.S.; Deb, S. Engineering Optimisation by Cuckoo Search. Int. J. Math. Model. Numer. Optim. 2010, 1, 330–343. [Google Scholar] [CrossRef]
- Haubold, H.J.; Mathai, A.M.; Saxena, R.K. Mittag-Leffler Functions and Their Applications. J. Appl. Math. 2011, 2011, 298628. [Google Scholar] [CrossRef] [Green Version]
- Jayakumar, K. Mittag-Leffler Process. Math. Comput. Model. 2003, 37, 1427–1434. [Google Scholar] [CrossRef]
- Wei, J.; Chen, Y.; Yu, Y.; Chen, Y. Improving cuckoo search algorithm with Mittag-Leffler distribution. In Proceedings of the ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Anaheim, CA, USA, 18–21 August 2019. [Google Scholar]
- Rinne, H. The Weibull Distribution: A Handbook; CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar]
- Johnson, N.L.; Kotz, S.; Balakrishnan, N. Continuous Univariate Distributions; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 1995. [Google Scholar]
- Feller, W. An Introduction to Probability Theory and Its Application Vol II; John Wiley & Sons: Hoboken, NJ, USA, 1971. [Google Scholar]
- Liu, T.; Zhang, P.; Dai, W.S.; Xie, M. An Intermediate Distribution Between Gaussian and Cauchy Distributions. Phys. A Stat. Mech. Appl. 2012, 391, 5411–5421. [Google Scholar] [CrossRef] [Green Version]
- Bahat, D.; Rabinovitch, A.; Frid, V. Tensile Fracturing in Rocks; Springer: Berlin, Germany, 2005. [Google Scholar]
- Geerolf, F. A Theory of Pareto Distributions. Available online: https://fgeerolf.com/geerolf-pareto.pdf (accessed on 2 February 2021).
- Mandelbrot, B. The Pareto-Levy Law and the Distribution of Income. Int. Econ. Rev. 1960, 1, 79–106. [Google Scholar] [CrossRef] [Green Version]
- Levy, M.; Solomon, S. New Evidence for the Power-law Distribution of Wealth. Phys. A Stat. Mech. Appl. 1997, 242, 90–94. [Google Scholar] [CrossRef]
- Lu, J.; Ding, J. Mixed-Distribution-Based Robust Stochastic Configuration Networks for Prediction Interval Construction. IEEE Trans. Ind. Inform. 2019, 16, 5099–5109. [Google Scholar] [CrossRef]
- Spiegel, M.R.; Schiller, J.J.; Srinivasan, R. Probability and Statistics; McGraw-Hill: New York, NY, USA, 2013. [Google Scholar]
- Embrechts, P.; Klüppelberg, C.; Mikosch, T. Modelling Extremal Events: For Insurance and Finance; Springer Science & Business Media: Berlin, Germany, 2013; Volume 33. [Google Scholar]
- Novak, S.Y. Extreme Value Methods with Applications to Finance; CRC Press: Boca Raton, FL, USA, 2011. [Google Scholar]
- De Haan, L.; Ferreira, A. Extreme Value Theory: An Introduction; Springer Science & Business Media: Berlin, Germany, 2007. [Google Scholar]
- Bottou, L.; Bousquet, O. The Tradeoffs of Large Scale Learning. Adv. Neural Inf. Process. Syst. 2007, 20, 161–168. [Google Scholar]
- Bottou, L. Large-scale machine learning with stochastic gradient descent. In Proceedings of the COMPSTAT; Springer: Berlin, Germany, 2010; pp. 177–186. [Google Scholar]
- Simsekli, U.; Sagun, L.; Gurbuzbalaban, M. A tail-index analysis of stochastic gradient noise in deep neural networks. arXiv 2019, arXiv:1901.06053. [Google Scholar]
- Yanovsky, V.; Chechkin, A.; Schertzer, D.; Tur, A. Lévy Anomalous Diffusion and Fractional Fokker–Planck Equation. Phys. A Stat. Mech. Appl. 2000, 282, 13–34. [Google Scholar] [CrossRef] [Green Version]
- Viswanathan, G.M.; Afanasyev, V.; Buldyrev, S.; Murphy, E.; Prince, P.; Stanley, H.E. Lévy Flight Search Patterns of Wandering Albatrosses. Nature 1996, 381, 413–415. [Google Scholar] [CrossRef]
- Hilbert, M.; López, P. The World’s Technological Capacity to Store, Communicate, and Compute Information. Science 2011, 332, 60–65. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ward, J.S.; Barker, A. Undefined by data: A survey of big data definitions. arXiv 2013, arXiv:1309.5821. [Google Scholar]
- Reinsel, D.; Gantz, J.; Rydning, J. Data Age 2025: The Evolution of Data to Life-critical. Don’t Focus Big Data 2017, 2, 2–24. [Google Scholar]
- Firican, G. The 10 Vs of Big Data. 2017. Available online: https://tdwi.org/articles/2017/02/08/10-vs-of-big-data.aspx (accessed on 2 February 2021).
- Nakahira, Y.; Liu, Q.; Sejnowski, T.J.; Doyle, J.C. Diversity-enabled sweet spots in layered architectures and speed-accuracy trade-offs in sensorimotor control. arXiv 2019, arXiv:1909.08601. [Google Scholar]
- Arabas, J.; Opara, K. Population Diversity of Non-elitist Evolutionary Algorithms in the Exploration Phase. IEEE Trans. Evol. Comput. 2019, 24, 1050–1062. [Google Scholar] [CrossRef]
- Ko, M.; Stark, B.; Barbadillo, M.; Chen, Y. An Evaluation of Three Approaches Using Hurst Estimation to Differentiate Between Normal and Abnormal HRV. In Proceedings of the ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Boston, MA, USA, 2–5 August 2015. [Google Scholar]
- Li, N.; Cruz, J.; Chien, C.S.; Sojoudi, S.; Recht, B.; Stone, D.; Csete, M.; Bahmiller, D.; Doyle, J.C. Robust Efficiency and Actuator Saturation Explain Healthy Heart Rate Control and Variability. Proc. Natl. Acad. Sci. USA 2014, 111, E3476–E3485. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hutton, E.L. Xunzi: The Complete Text; Princeton University Press: Princeton, NJ, USA, 2014. [Google Scholar]
- Boyer, C.B. The History of the Calculus and Its Conceptual Development: (The Concepts of the Calculus); Courier Corporation: Chelmsford, MA, USA, 1959. [Google Scholar]
- Bardi, J.S. The Calculus Wars: Newton, Leibniz, and the Greatest Mathematical Clash of All Time; Hachette UK: Paris, France, 2009. [Google Scholar]
- Tanner, R.I.; Walters, K. Rheology: An Historical Perspective; Elsevier: Amsterdam, The Netherlands, 1998. [Google Scholar]
- Chen, Y.; Sun, R.; Zhou, A. An Improved Hurst Parameter Estimator Based on Fractional Fourier Transform. Telecommun. Syst. 2010, 43, 197–206. [Google Scholar] [CrossRef]
- Sheng, H.; Sun, H.; Chen, Y.; Qiu, T. Synthesis of Multifractional Gaussian Noises Based on Variable-order Fractional Operators. Signal Process. 2011, 91, 1645–1650. [Google Scholar] [CrossRef]
- Sun, R.; Chen, Y.; Zaveri, N.; Zhou, A. Local analysis of long range dependence based on fractional Fourier transform. In Proceedings of the IEEE Mountain Workshop on Adaptive and Learning Systems, Logan, UT, USA, 24–26 July 2006; pp. 13–18. [Google Scholar]
- Pipiras, V.; Taqqu, M.S. Long-Range Dependence and Self-Similarity; Cambridge University Press: Cambridge, UK, 2017; Volume 45. [Google Scholar]
- Samorodnitsky, G. Long Range Dependence. Available online: https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118445112.stat04569 (accessed on 2 February 2021).
- Gubner, J.A. Probability and Random Processes for Electrical and Computer Engineers; Cambridge University Press: Cambridge, UK, 2006. [Google Scholar]
- Clegg, R.G. A practical guide to measuring the Hurst parameter. arXiv 2006, arXiv:math/0610756. [Google Scholar]
- Decreusefond, L. Stochastic Analysis of the Fractional Brownian Motion. Potential Anal. 1999, 10, 177–214. [Google Scholar] [CrossRef]
- Koutsoyiannis, D. The Hurst Phenomenon and Fractional Gaussian Noise Made Easy. Hydrol. Sci. J. 2002, 47, 573–595. [Google Scholar] [CrossRef]
- Mandelbrot, B.B.; Van Ness, J.W. Fractional Brownian Motions, Fractional Noises and Applications. SIAM Rev. 1968, 10, 422–437. [Google Scholar] [CrossRef]
- Ortigueira, M.D.; Batista, A.G. On the Relation between the Fractional Brownian Motion and the Fractional Derivatives. Phys. Lett. A 2008, 372, 958–968. [Google Scholar] [CrossRef] [Green Version]
- Chen, Y.; Sun, R.; Zhou, A. An overview of fractional order signal processing (FOSP) techniques. In Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Las Vegas, NV, USA, 4–7 September 2007. [Google Scholar]
- Liu, K.; Domański, P.D.; Chen, Y. Control performance assessment with fractional lower order moments. In Proceedings of the 2020 7th International Conference on Control, Decision and Information Technologies (CoDIT), Prague, Czech Republic, 29 June–2 July 2020. [Google Scholar]
- Cottone, G.; Di Paola, M. On the Use of Fractional Calculus for the Probabilistic Characterization of Random Variables. Probabilistic Eng. Mech. 2009, 24, 321–330. [Google Scholar] [CrossRef] [Green Version]
- Cottone, G.; Di Paola, M.; Metzler, R. Fractional Calculus Approach to the Statistical Characterization of Random Variables and Vectors. Phys. A Stat. Mech. Appl. 2010, 389, 909–920. [Google Scholar] [CrossRef] [Green Version]
- Ma, X.; Nikias, C.L. Joint Estimation of Time Delay and Frequency Delay in Impulsive Noise Using Fractional Lower Order Statistics. IEEE Trans. Signal Process. 1996, 44, 2669–2687. [Google Scholar]
- RongHua, F. Modeling and Application of Theory Based on Time Series ARMA. Sci. Technol. Inf. 2012, 2012, 153. [Google Scholar]
- Shalalfeh, L.; Bogdan, P.; Jonckheere, E. Fractional Dynamics of PMU Data. IEEE Trans. Smart Grid 2020. [Google Scholar] [CrossRef]
- Harmantzis, F. Heavy network traffic modeling and simulation using stable FARIMA processes. In Proceedings of the 19th International Teletraffic Congress (ITC19), Beijing, China, 29 August–2 September 2005. [Google Scholar]
- Sheng, H.; Chen, Y. FARIMA with Stable Innovations Model of Great Salt Lake Elevation Time Series. Signal Process. 2011, 91, 553–561. [Google Scholar] [CrossRef]
- Li, Q.; Tricaud, C.; Sun, R.; Chen, Y. Great Salt Lake surface level forecasting using FIGARCH model. In Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Las Vegas, NV, USA, 4–7 September 2007; pp. 1361–1370. [Google Scholar]
- Brockwell, P.J.; Davis, R.A.; Fienberg, S.E. Time Series: Theory and Methods; Springer Science & Business Media: Berlin, Germany, 1991. [Google Scholar]
- Boutahar, M.; Dufrénot, G.; Péguin-Feissolle, A. A Simple Fractionally Integrated Model with a Time-varying Long Memory Parameter dt. Comput. Econ. 2008, 31, 225–241. [Google Scholar] [CrossRef]
- Gray, H.L.; Zhang, N.F.; Woodward, W.A. On Generalized Fractional Processes. J. Time Ser. Anal. 1989, 10, 233–257. [Google Scholar] [CrossRef]
- Woodward, W.A.; Cheng, Q.C.; Gray, H.L. A k-factor GARMA Long-memory Model. J. Time Ser. Anal. 1998, 19, 485–504. [Google Scholar] [CrossRef]
- West, B.J. Fractional Calculus View of Complexity: Tomorrow’s Science; CRC Press: Boca Raton, FL, USA, 2016. [Google Scholar]
- Zaslavsky, G.M.; Sagdeev, R.; Usikov, D.; Chernikov, A. Weak Chaos and Quasi-regular Patterns; Cambridge University Press: Cambridge, UK, 1992. [Google Scholar]
- Hilfer, R.; Anton, L. Fractional Master Equations and Fractal Time Random Walks. Phys. Rev. E 1995, 51, R848. [Google Scholar] [CrossRef] [Green Version]
- Gorenflo, R.; Mainardi, F.; Vivoli, A. Continuous-time Random Walk and Parametric Subordination in Fractional Diffusion. Chaos Solitons Fractals 2007, 34, 87–103. [Google Scholar] [CrossRef] [Green Version]
- Niu, H.; Hollenbeck, D.; Zhao, T.; Wang, D.; Chen, Y. Evapotranspiration Estimation with Small UAVs in Precision Agriculture. Sensors 2020, 20, 6427. [Google Scholar] [CrossRef]
- Díaz-Varela, R.; de la Rosa, R.; León, L.; Zarco-Tejada, P. High-resolution Airborne UAV Imagery to Assess Olive Tree Crown Parameters Using 3D Photo Reconstruction: Application in Breeding Trials. Remote Sens. 2015, 7, 4213–4232. [Google Scholar] [CrossRef] [Green Version]
- Gonzalez-Dugo, V.; Goldhamer, D.; Zarco-Tejada, P.J.; Fereres, E. Improving the Precision of Irrigation in a Pistachio Farm Using an Unmanned Airborne Thermal System. Irrig. Sci. 2015, 33, 43–52. [Google Scholar] [CrossRef]
- Swain, K.C.; Thomson, S.J.; Jayasuriya, H.P. Adoption of an Unmanned Helicopter for Low-altitude Remote Sensing to Estimate Yield and Total Biomass of a Rice Crop. Trans. ASABE 2010, 53, 21–27. [Google Scholar] [CrossRef] [Green Version]
- Zarco-Tejada, P.J.; González-Dugo, V.; Williams, L.; Suárez, L.; Berni, J.A.; Goldhamer, D.; Fereres, E. A PRI-based Water Stress Index Combining Structural and Chlorophyll Effects: Assessment Using Diurnal Narrow-band Airborne Imagery and the CWSI Thermal Index. Remote Sens. Environ. 2013, 138, 38–50. [Google Scholar] [CrossRef]
- Niu, H.; Zhao, T.; Wang, D.; Chen, Y. Estimating evapotranspiration with UAVs in agriculture: A review. In Proceedings of the ASABE Annual International Meeting, Boston, MA, USA, 7–10 July 2019. [Google Scholar]
- Niu, H.; Zhao, T.; Wang, D.; Chen, Y. A UAV resolution and waveband aware path planning for onion irrigation treatments inference. In Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 11–14 June 2019; pp. 808–812. [Google Scholar]
- Niu, H.; Wang, D.; Chen, Y. Estimating crop coefficients using linear and deep stochastic configuration networks models and UAV-based Normalized Difference Vegetation Index (NDVI). In Proceedings of the International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece, 1–4 September 2020. [Google Scholar]
- Niu, H.; Wang, D.; Chen, Y. Estimating actual crop evapotranspiration using deep stochastic configuration networks model and UAV-based crop coefficients in a pomegranate orchard. In Proceedings of the Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping V. International Society for Optics and Photonics, 27 April–8 May 2020. held online. [Google Scholar]
- Che, Y.; Wang, Q.; Xie, Z.; Zhou, L.; Li, S.; Hui, F.; Wang, X.; Li, B.; Ma, Y. Estimation of Maize Plant Height and Leaf Area Index Dynamic Using Unmanned Aerial Vehicle with Oblique and Nadir Photography. Ann. Bot. 2020, 126, 765–773. [Google Scholar] [CrossRef]
- Deng, R.; Jiang, Y.; Tao, M.; Huang, X.; Bangura, K.; Liu, C.; Lin, J.; Qi, L. Deep Learning-based Automatic Detection of Productive Tillers in Rice. Comput. Electron. Agric. 2020, 177, 105703. [Google Scholar] [CrossRef]
- Zhao, T.; Chen, Y.; Ray, A.; Doll, D. Quantifying almond water stress using unmanned aerial vehicles (UAVs): Correlation of stem water potential and higher order moments of non-normalized canopy distribution. In Proceedings of the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, OH, USA, 6–9 August 2017. [Google Scholar]
- Zhao, T.; Niu, H.; de la Rosa, E.; Doll, D.; Wang, D.; Chen, Y. Tree canopy differentiation using instance-aware semantic segmentation. In Proceedings of the 2018 ASABE Annual International Meeting, Detroit, MI, USA, 29 July–1 August 2018. [Google Scholar]
- Géron, A. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems; O’Reilly Media: Sevastopol, CA, USA, 2019. [Google Scholar]
- Mitchell, T.M. Machine Learning; McGraw-Hill: New York, NY, USA, 1997. [Google Scholar]
- Polyak, B.T. Some Methods of Speeding up the Convergence of Iteration Methods. USSR Comput. Math. Math. Phys. 1964, 4, 1–17. [Google Scholar] [CrossRef]
- Duchi, J.; Hazan, E.; Singer, Y. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. J. Mach. Learn. Res. 2011, 12, 2121–2159. [Google Scholar]
- Hinton, G.; Tieleman, T. Slide 29 in Lecture 6. 2012. Available online: http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf (accessed on 2 February 2021).
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Zeng, C.; Chen, Y. Optimal Random Search, Fractional Dynamics and Fractional Calculus. Fract. Calc. Appl. Anal. 2014, 17, 321–332. [Google Scholar] [CrossRef] [Green Version]
- Wei, J.; Yu, Y.; Wang, S. Parameter Estimation for Noisy Chaotic Systems Based on an Improved Particle Swarm Optimization Algorithm. J. Appl. Anal. Comput. 2015, 5, 232–242. [Google Scholar]
- Wei, J.; Yu, Y.; Cai, D. Identification of Uncertain Incommensurate Fractional-order Chaotic Systems Using an Improved Quantum-behaved Particle Swarm Optimization Algorithm. J. Comput. Nonlinear Dyn. 2018, 13, 051004. [Google Scholar] [CrossRef]
- Wei, J.; Chen, Y.; Yu, Y.; Chen, Y. Optimal Randomness in Swarm-based Search. Mathematics 2019, 7, 828. [Google Scholar] [CrossRef] [Green Version]
- Wei, J.; Yu, Y. A Novel Cuckoo Search Algorithm under Adaptive Parameter Control for Global Numerical Optimization. Soft Comput. 2019, 24, 4917–4940. [Google Scholar] [CrossRef]
- Wei, J.; Yu, Y. An adaptive cuckoo search algorithm with optional external archive for global numerical optimization. In Proceedings of the International Conference on Fractional Differentiation and its Applications (ICFDA), Amman, Jordan, 16–18 July 2018. [Google Scholar]
- Wilson, A.C.; Recht, B.; Jordan, M.I. A Lyapunov analysis of momentum methods in optimization. arXiv 2016, arXiv:1611.02635. [Google Scholar]
- Feynman, R.P. The Principle of Least Action in Quantum Mechanics. In Feynman’s Thesis—A New Approach to Quantum Theory; World Scientific: Singapore, 2005; pp. 1–69. [Google Scholar]
- Hamilton, S.W.R. On a General Method in Dynamics. Richard Taylor, 1834. Available online: http://www.kurims.kyoto-u.ac.jp/EMIS/classics/Hamilton/GenMeth.pdf (accessed on 2 February 2021).
- Hawking, S.W. The Path-integral Approach to Quantum Gravity. In General Relativity; World Scientific: Singapore, 1979. [Google Scholar]
- Kerrigan, E. What the Machine Should Learn about Models for Control. 2020. Available online: https://www.ifac2020.org/program/workshops/machine-learning-meets-model-based-control (accessed on 2 February 2021).
- Vinnicombe, G. Uncertainty and Feedback: H∞ Loop-Shaping and the ν-Gap Metric; World Scientific: Singapore, 2001. [Google Scholar]
- Viola, J.; Chen, Y.; Wang, J. Information-based model discrimination for digital twin behavioral matching. In Proceedings of the International Conference on Industrial Artificial Intelligence (IAI), Shenyang, China, 23–25 October 2020; pp. 1–6. [Google Scholar]
- Kashima, K.; Yamamoto, Y. System Theory for Numerical Analysis. Automatica 2007, 43, 1156–1164. [Google Scholar] [CrossRef]
- An, W.; Wang, H.; Sun, Q.; Xu, J.; Dai, Q.; Zhang, L. A PID controller approach for stochastic optimization of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 8522–8531. [Google Scholar]
- Fan, Y.; Koellermeier, J. Accelerating the Convergence of the Moment Method for the Boltzmann Equation Using Filters. J. Sci. Comput. 2020, 84, 1–28. [Google Scholar] [CrossRef]
- Kuhlman, K.L. Review of Inverse Laplace Transform Algorithms for Laplace-space Numerical Approaches. Numer. Algorithms 2013, 63, 339–355. [Google Scholar] [CrossRef] [Green Version]
- Xue, D.; Chen, Y. Solving Applied Mathematical Problems with MATLAB; CRC Press: Boca Raton, FL, USA, 2009. [Google Scholar]
- Wang, D.; Li, M. Stochastic Configuration Networks: Fundamentals and Algorithms. IEEE Trans. Cybern. 2017, 47, 3466–3479. [Google Scholar] [CrossRef] [Green Version]
- Bebis, G.; Georgiopoulos, M. Feed-forward Neural Networks. IEEE Potentials 1994, 13, 27–31. [Google Scholar] [CrossRef]
- Broomhead, D.; Lowe, D. Multivariable Functional Interpolation and Adaptive Networks. Complex Syst. 1988, 2, 321–355. [Google Scholar]
- Pao, Y.H.; Takefuji, Y. Functional-link Net Computing: Theory, System Architecture, and Functionalities. Computer 1992, 25, 76–79. [Google Scholar] [CrossRef]
- Wang, D.; Li, M. Deep stochastic configuration networks with universal approximation property. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–8. [Google Scholar]
- Li, M.; Wang, D. 2-D Stochastic Configuration Networks for Image Data Analytics. IEEE Trans. Cybern. 2019, 51, 359–372. [Google Scholar] [CrossRef] [PubMed]
- Huang, C.; Huang, Q.; Wang, D. Stochastic Configuration Networks Based Adaptive Storage Replica Management for Power Big Data Processing. IEEE Trans. Ind. Inf. 2019, 16, 373–383. [Google Scholar] [CrossRef]
- Scardapane, S.; Wang, D. Randomness in Neural Networks: An Overview. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 2017, 7, e1200. [Google Scholar] [CrossRef]
- Wei, J. Research on Swarm Intelligence Optimization Algorithms and Their Applications to Parameter Identification of Fractional-Order Systems. Ph.D. Thesis, Beijing Jiaotong University, Beijing, China, 2020. [Google Scholar]
- Chen, Y. Fundamental Principles for Fractional Order Gradient Methods. Ph.D. Thesis, University of Science and Technology of China, Hefei, China, 2020. [Google Scholar]
- Tyukin, I.Y.; Prokhorov, D.V. Feasibility of random basis function approximators for modeling and control. In Proceedings of the IEEE Control Applications, (CCA) & Intelligent Control, (ISIC), St. Petersburg, Russia, 8–10 July 2009. [Google Scholar]
- Nagaraj, S. Optimization and learning with nonlocal calculus. arXiv 2020, arXiv:2012.07013. [Google Scholar]
- Tarasov, V.E. Fractional Vector Calculus and Fractional Maxwell’s Equations. Ann. Phys. 2008, 323, 2756–2778. [Google Scholar] [CrossRef] [Green Version]
- Ortigueira, M.; Machado, J. On Fractional Vectorial Calculus. Bull. Pol. Acad. Sci. Tech. Sci. 2018, 66, 389–402. [Google Scholar]
- Feliu-Faba, J.; Fan, Y.; Ying, L. Meta-learning Pseudo-differential Operators with Deep Neural Networks. J. Comput. Phys. 2020, 408, 109309. [Google Scholar] [CrossRef] [Green Version]
- Hall, D.L. Dao De Jing: A Philosophical Translation; Ballantine Books: New York, NY, USA, 2010. [Google Scholar]
Characteristics | Description |
---|---|
1. Volume | Best known characteristic of big data; more than 90 percent of the whole data were created in the past couple of years. |
2. Velocity | The speed at which data are being generated. |
3. Variety | Processing structured, unstructured and semistructured data. |
4. Variability | Inconsistent speed of data loading, multitude of data dimensions, and number of inconsistencies. |
5. Veracity | Confidence or trust in the data. |
6. Validity | Refers to how accurate and correct the data are. |
7. Vulnerability | Security concerns, data breaches. |
8. Volatility | Design policy for data currency, availability, and rapid retrieval of information when required. |
9. Visualization | Develop new tools considering the complex relationships between the above properties. |
10. Value | The most important of the 10 Vs; substantial value must be found. |
Topics | Description |
---|---|
1. Climate variability | Changes in the components of the climate system and their interactions. |
2. Genetic variability | Measurements of the tendencies of individual genotypes between regions. |
3. Heart rate variability | Physiological phenomenon where the time interval between heart beats varies. |
4. Human variability | Measurements of the characteristics, physical or mental, of human beings. |
5. Spatial variability | Measurements at different spatial points exhibit different values. |
6. Statistical variability | A measure of dispersion in statistics. |
0.4 | 0.8 | 1.2 | 1.6 | 2.0 | 2.4 | |
---|---|---|---|---|---|---|
a | −0.6 | −0.2 | 0.2 | 0.6 | 1 | 1.4 |
b | 1.5 | 0.25 | −0.1667 | −0.3750 | −0.5 | −0.5833 |
0.4 | 0.8 | 1.2 | 1.6 | 2.0 | 2.4 | |
---|---|---|---|---|---|---|
a | 0.6439 | 0.5247 | −0.4097 | −0.5955 | −1.0364 | −1.4629 |
b | 0.0263 | 0.0649 | 0.0419 | −0.0398 | 0.0364 | 0.0880 |
c | 1.5439 | 0.5747 | −0.3763 | −0.3705 | −0.5364 | −0.6462 |
d | 0.0658 | 0.0812 | 0.0350 | −0.0408 | 0.0182 | 0.0367 |
0.3 | 0.5 | 0.7 | 0.9 | |
---|---|---|---|---|
1.8494 | 1.6899 | 1.5319 | 1.2284 | |
20 | 20 | 20 | 20 |
Properties | Values |
---|---|
Name: | “Stochastic Configuration Networks” |
Version: | “1.0 beta” |
L: | hidden node number |
W: | input weight matrix |
b: | hidden layer bias vector |
Beta: | output weight vector |
r: | regularization parameter |
tol: | tolerance |
Lambda: | random weight range |
L: | maximum number of hidden neurons |
T: | maximum times of random configurations |
nB: | number of node being added in one loop |
Models | Mean Hidden Node Number | RMSE |
---|---|---|
SCN | 75 ± 5 | 0.0025 |
SCN-Lévy | 70 ± 6 | 0.0010 |
SCN-Cauchy | 59 ± 3 | 0.0057 |
SCN-Weibull | 63 ± 4 | 0.0037 |
SCN-Mixture | 70 ± 5 | 0.0020 |
Models | Training Accuracy | Test Accuracy |
---|---|---|
SCN | 94.0 ± 1.9% | 91.2 ± 6.2% |
SCN-Lévy | 94.9 ± 0.8% | 91.7 ± 4.5% |
SCN-Cauchy | 95.4 ± 1.3% | 92.4 ± 5.5% |
SCN-Mixture | 94.7 ± 1.1% | 91.5 ± 5.3% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Niu, H.; Chen, Y.; West, B.J. Why Do Big Data and Machine Learning Entail the Fractional Dynamics? Entropy 2021, 23, 297. https://doi.org/10.3390/e23030297
Niu H, Chen Y, West BJ. Why Do Big Data and Machine Learning Entail the Fractional Dynamics? Entropy. 2021; 23(3):297. https://doi.org/10.3390/e23030297
Chicago/Turabian StyleNiu, Haoyu, YangQuan Chen, and Bruce J. West. 2021. "Why Do Big Data and Machine Learning Entail the Fractional Dynamics?" Entropy 23, no. 3: 297. https://doi.org/10.3390/e23030297