Fuzzy Model Parameter and Structure Optimization Using Analytic, Numerical and Heuristic Approaches
Abstract
:1. Introduction
- This work presents a method based on optimization, to obtain the parameters of the antecedent and consequent parts and the appropriate structure of the fuzzy system;
- This work provides a fast analytic strategy for finding the optimal parameters of the consequent part for each set of parameters of the antecedent part, which sidesteps the current method found in the literature, of searching through a long set of parameters for the antecedent and consequent part simultaneously;
- This work proposes a hill climbing heuristic strategy to determine the optimal structure. This strategy uses, as a fitness function, the RMSE found with the algorithm that optimizes the parameters of the antecedent part.
2. Literature Overview
3. Proposed Methodology
- Consequent coefficients determination;
- Antecedent parameter determination;
- Structural parameter determination.
3.1. Consequent Coefficients Determination
Algorithm 1 Obtaining the optimal consequent parameters C |
|
3.2. Antecedent Parameter Determination
Algorithm 2 Gradient descent |
|
3.3. Structural Parameter Determination
Algorithm 3 Hill climbing |
|
4. Case Studies
4.1. Case Study I
4.2. Case Study II
4.3. Case Study III
4.4. Case Study IV
5. Discussion of the Case Studies
6. Conclusions and Future Work
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Dote, Y.; Ovaska, S. Industrial applications of soft computing: A review. Proc. IEEE 2001, 89, 1243–1265. [Google Scholar] [CrossRef]
- Kindo, A.A.; Kaladzavi, G.; Malo, S.; Camara, G.; Tapsoba, T.M.Y.; Kolyang. Fuzzy logic approach for knowledge modeling in an Ontology: A review. In Proceedings of the 2020 IEEE 2nd International Conference on Smart Cities and Communities (SCCIC’2020), Ouagadougou, Burkina Faso, 1–3 December 2020; pp. 1–8, ISBN 978-1-7281-9685-5. [Google Scholar]
- Cherkassky, V. Fuzzy Inference Systems: A Critical Review. In Computational Intelligence: Soft Computing and Fuzzy-Neuro Integration with Applications; Kaynak, O., Zadeh, L.A., Türkşen, B., Rudas, I.J., Eds.; NATO ASI Series; Springer: Berlin/Heidelberg, Germany, 1998; Volume 162, pp. 177–197. ISBN 978-3-642-63796-4. [Google Scholar]
- Sugeno, M. On stability of fuzzy systems expressed by fuzzy rules with singleton consequents. IEEE Trans. Fuzzy Syst. 1999, 7, 201–224. [Google Scholar] [CrossRef]
- Mamdani, E.; Assilian, S. An experiment in linguistic synthesis with a fuzzy logic controller. Int. J. Man-Mach. Stud. 1975, 7, 1–13. [Google Scholar] [CrossRef]
- Takagi, T.; Sugeno, M. Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 1985, SMC-15, 116–132. [Google Scholar] [CrossRef]
- Sugeno, M.; Kang, G. Structure identification of fuzzy model. Fuzzy Sets Syst. 1988, 28, 15–33. [Google Scholar] [CrossRef]
- Kosko, B. Fuzzy systems as universal approximators. IEEE Trans. Comput. 1994, 43, 1329–1333. [Google Scholar] [CrossRef]
- Pappis, C.; Mamdani, E.H. A Fuzzy Logic Controller for a Trafc Junction. IEEE Trans. Syst. Man Cybern. 1977, 7, 707–717. [Google Scholar] [CrossRef]
- Pomares, H.; Rojas, I.; Gonzalez, J.; Prieto, A. Structure identification in complete rule-based fuzzy systems. IEEE Trans. Fuzzy Syst. 2002, 10, 349–359. [Google Scholar] [CrossRef] [Green Version]
- Tong, R. The Construction and Evaluation of Fuzzy Models. In Advances in Fuzzy Set Theory and Applications; Gupta, M.M., Ragade, R.K., Yager, R.R., Eds.; North-Holland Publishing Company: Amsterdam, The Netherlands, 1979; pp. 559–576. ISBN 978-0444853721. [Google Scholar]
- Wang, L.; Mendel, J. Generating fuzzy rules by learning from examples. IEEE Trans. Syst. Man Cybern. 1992, 22, 1414–1427. [Google Scholar] [CrossRef] [Green Version]
- Nomura, H.; Hayashi, I.; Wakami, N. A learning method of fuzzy inference rules by descent method. In Proceedings of the IEEE International Conference on Fuzzy Systems, San Diego, CA, USA, 8–12 March 1992; pp. 203–210, ISBN 0-7803-0236-2. [Google Scholar]
- Jang, J. ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans. Syst. Man Cybern. 1993, 23, 665–685. [Google Scholar] [CrossRef]
- Horikawa, S.I.; Furuhashi, T.; Uchikawa, Y. On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm. IEEE Trans. Neural Netw. 1992, 3, 881–886. [Google Scholar] [CrossRef]
- Jang, J.S.R. Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm. In Proceedings of the Ninth National Conference on Artificial Intelligence, Anaheim, CA, USA, 14–19 July 1991; pp. 763–767, ISBN 0-262-51059-6. [Google Scholar]
- Lin, C.; Lee, C. Neural-network-based fuzzy logic control and decision system. IEEE Trans. Comput. 1991, 40, 1320–1336. [Google Scholar] [CrossRef]
- Berenji, H.; Khedkar, P. Learning and tuning fuzzy logic controllers through reinforcements. IEEE Trans. Neural Netw. 1992, 3, 724–740. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Nauck, D.; Kruse, R. A fuzzy neural network learning fuzzy control rules and membership functions by fuzzy error backpropagation. In Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA, 28 March–1 April 1993; pp. 1022–1027, ISBN 0-7803-0999-5. [Google Scholar]
- Vieira, J.; Morgado-Dias, F.; Mota, A. Neuro-Fuzzy Systems: A Survey. Wseas Trans. Syst. 2004, 3, 414–419. [Google Scholar]
- Shihabudheen, K.; Pillai, G. Recent Advances in Neuro-Fuzzy Systems: A Survey. Knowl.-Based Syst. 2018, 152, 136–162. [Google Scholar] [CrossRef]
- Lin, C.T. FALCON: A fuzzy adaptive learning control network. In Proceedings of the NAFIPS/IFIS/NASA ’94—First International Joint Conference of the North American Fuzzy Information Processing Society Biannual Conference, the Industrial Fuzzy Control and Intelligence Conference, San Antonio, TX, USA, 18–21 December 1991; pp. 228–232, ISBN 0-7803-2125-1. [Google Scholar]
- Berenji, H.R.; Khedkar, P.S. Using fuzzy logic for performance evaluation in reinforcement learning. Int. J. Approx. Reason. 1998, 18, 131–144. [Google Scholar] [CrossRef] [Green Version]
- Nauck, D. Beyond Neuro-Fuzzy: Perspectives And Directions. In Proceedings of the Third European Congress on Intelligent Techniques and Soft Computing (EUFIT’95), Aachen, Germany, 28–31 August 1995. [Google Scholar]
- Tano, S.; Oyama, T.; Arnould, T. Deep combination of fuzzy inference and neural network in fuzzy inference software—FINEST. Fuzzy Sets Syst. 1996, 82, 151–160. [Google Scholar] [CrossRef]
- Sulzberger, S.; Tschichold-Gurman, N.; Vestli, S. FUN: Optimization of fuzzy rule based systems using neural networks. In Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA, 28 March–1 April 1993; pp. 312–316, ISBN 0-7803-0999-5. [Google Scholar]
- Juang, C.F.; Lin, C.T. An online self-constructing neural fuzzy inference network and its applications. IEEE Trans. Fuzzy Syst. 1998, 6, 12–32. [Google Scholar] [CrossRef]
- Figueiredo, M.; Gomide, F. Design of fuzzy systems using neurofuzzy networks. IEEE Trans. Neural Netw. 1999, 10, 815–827. [Google Scholar] [CrossRef]
- Kasabov, N.; Sung, Q. Dynamic Evolving Fuzzy Neural Networks with ‘m-out-of-n’ Activation Nodes for On-Line Adaptive Systems; Technical Report 99/04; University of Otago, Department of Information Science: Dunedin, New Zealand, 1999. [Google Scholar]
- Jang, R. Neuro-Fuzzy Modelling: Architectures, Analysis and Applications. Ph.D. Thesis, University of California, Berkeley, CA, USA, 1992. [Google Scholar]
- Naceur, F.B.; Telmoudi, A.J.; Mahjoub, M.A. A proposal ANFIS estimation algorithm for optimal sizing of a PVP/Battery system. In Proceedings of the 2020 7th International Conference on Control, Decision and Information Technologies (CoDIT’2020), Prague, Czech Republic, 29 June–2 July 2020; pp. 206–210, ISBN 978-1-7281-5954-6. [Google Scholar]
- Acosta, K.M.Y.; Baldovino, R.G. Predicting Acute Aquatic Toxicity Towards Fathead Minnow (Pimephales Promelas) Using Neuro-Fuzzy Inference System (ANFIS). In Proceedings of the 2020 12th International Conference on Information Technology and Electrical Engineering (ICITEE’2020), Yogyakarta, Indonesia, 6–8 October 2020; pp. 329–332, ISBN 978-1-7281-1098-1. [Google Scholar]
- İnan, T.; Baba, A.F. Prediction of Wind Speed Using Artificial Neural Networks and ANFIS Methods (Observation Buoy Example). In Proceedings of the 2020 Innovations in Intelligent Systems and Applications Conference (ASYU’2020), Istanbul, Turkey, 15–17 October 2020; pp. 1–5, ISBN 978-1-7281-9137-9. [Google Scholar]
- Kalyani, K.; Kanagalakshmi, S. Control of Trms using Adaptive Neuro Fuzzy Inference System (ANFIS). In Proceedings of the 2020 International Conference on System, Computation, Automation and Networking (ICSCAN’2020), Pondicherry, India, 3–4 July 2020; pp. 1–5, ISBN 978-1-7281-6203-4. [Google Scholar]
- Hammam, R.E.; Solyman, A.A.A.; Alsharif, M.H.; Uthansakul, P.; Deif, M.A. Design of Biodegradable Mg Alloy for Abdominal Aortic Aneurysm Repair (AAAR) Using ANFIS Regression Model. IEEE Access 2022, 10, 28579–28589. [Google Scholar] [CrossRef]
- Angelov, P.; Filev, D.; Kasabov, N. Guest Editorial Evolving Fuzzy Systems–Preface to the Special Section. IEEE Trans. Fuzzy Syst. 2008, 16, 1390–1392. [Google Scholar] [CrossRef] [Green Version]
- de Oliveira, J.V.; Pedrycz, W. (Eds.) Advances in Fuzzy Clustering and Its Applications; John Wiley & Sons, Inc.: Chichester, UK, 2007; ISBN 978-0-470-02760-8. [Google Scholar]
- Lughofer, E.D. FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi–Sugeno Fuzzy Models. IEEE Trans. Fuzzy Syst. 2008, 16, 1393–1410. [Google Scholar] [CrossRef]
- Filev, D.; Angelov, P. Algorithms for Real-Time Clustering and Generation of Rules from Data. In Advances in Fuzzy Clustering and its Applications; de Oliveira, J.V., Pedrycz, W., Eds.; John Wiley & Sons, Inc.: Chichester, UK, 2007; pp. 353–370. ISBN 978-0-470-02760-8. [Google Scholar]
- Rong, H.J.; Sundararajan, N.; Huang, G.B.; Saratchandran, P. Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst. 2006, 157, 1260–1275. [Google Scholar] [CrossRef]
- Lima, E.; Hell, M.; Ballini, R.; Gomide, F. Evolving Fuzzy Modeling Using Participatory Learning. In Evolving Intelligent Systems: Methodology and Applications; Angelov, P., Filev, D.P., Kasabov, N., Eds.; John Wiley & Sons, Ltd.: Chichester, UK, 2010; Chapter 4; pp. 67–86. ISBN 978-0-470-56996-2. [Google Scholar]
- Kasabov, N.; Song, Q. DENFIS: Dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans. Fuzzy Syst. 2002, 10, 144–154. [Google Scholar] [CrossRef] [Green Version]
- Angelov, P.; Filev, D. An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans. Syst. Man Cybern. Part (Cybernetics) 2004, 34, 484–498. [Google Scholar] [CrossRef] [Green Version]
- Nguyen, N.N.; Zhou, W.J.; Quek, C. GSETSK: A generic self-evolving TSK fuzzy neural network with a novel Hebbian-based rule reduction approach. Appl. Soft Comput. 2015, 35, 29–42. [Google Scholar] [CrossRef]
- de Jesus Rubio, J. SOFMLS: Online Self-Organizing Fuzzy Modified Least-Squares Network. IEEE Trans. Fuzzy Syst. 2009, 17, 1296–1309. [Google Scholar] [CrossRef]
- Baruah, R.D.; Angelov, P. DEC: Dynamically Evolving Clustering and Its Application to Structure Identification of Evolving Fuzzy Models. IEEE Trans. Cybern. 2014, 44, 1619–1631. [Google Scholar] [CrossRef]
- Sa’ad, H.H.Y.; Isa, N.A.M.; Manjur, M.; Ahmed, M.M.; Sa’d, A.H.Y. A robust structure identification method for evolving fuzzy system. Expert Syst. Appl. 2018, 93, 267–282. [Google Scholar] [CrossRef]
- Sa’ad, H.H.Y.; Isa, N.A.M.; Ahmed, M.M. A Structural Evolving Approach for Fuzzy Systems. IEEE Trans. Fuzzy Syst. 2020, 28, 273–287. [Google Scholar] [CrossRef]
- Theocharis, J. A high-order recurrent neuro-fuzzy system with internal dynamics: Application to the adaptive noise cancellation. Fuzzy Sets Syst. 2006, 157, 471–500. [Google Scholar] [CrossRef]
- Juang, C.F.; Lin, Y.Y.; Tu, C.C. A recurrent self-evolving fuzzy neural network with local feedbacks and its application to dynamic system processing. Fuzzy Sets Syst. 2010, 161, 2552–2568. [Google Scholar] [CrossRef]
- Luo, C.; Tan, C.; Wang, X.; Zheng, Y. An evolving recurrent interval type-2 intuitionistic fuzzy neural network for online learning and time series prediction. Appl. Soft Comput. 2019, 78, 150–163. [Google Scholar] [CrossRef]
- Eyoh, I.; John, R.; Maere, G.D.; Kayacan, E. Hybrid Learning for Interval Type-2 Intuitionistic Fuzzy Logic Systems as Applied to Identification and Prediction Problems. IEEE Trans. Fuzzy Syst. 2018, 26, 2672–2685. [Google Scholar] [CrossRef]
- Yuan, W.; Chao, L. Online Evolving Interval Type-2 Intuitionistic Fuzzy LSTM-Neural Networks for Regression Problems. IEEE Access 2019, 7, 35544–35555. [Google Scholar] [CrossRef]
- Juang, C.F.; Lin, Y.Y.; Huang, R.B. Dynamic system modeling using a recurrent interval-valued fuzzy neural network and its hardware implementation. Fuzzy Sets Syst. 2011, 179, 83–99. [Google Scholar] [CrossRef]
- Juang, C.F.; Huang, R.B.; Lin, Y.Y. A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing. IEEE Trans. Fuzzy Syst. 2009, 17, 1092–1105. [Google Scholar] [CrossRef] [Green Version]
- El-Nagar, A.M. Nonlinear dynamic systems identification using recurrent interval type-2 TSK fuzzy neural network—A novel structure. Isa Trans. 2018, 72, 205–217. [Google Scholar] [CrossRef]
- Zhao, J.; Lin, C.M. Wavelet-TSK-Type Fuzzy Cerebellar Model Neural Network for Uncertain Nonlinear Systems. IEEE Trans. Fuzzy Syst. 2019, 27, 549–558. [Google Scholar] [CrossRef]
- Jo, J.M. Effectiveness of Normalization Pre-Processing of Big Data to the Machine Learning Performance. J. Korea Inst. Electron. Commun. Sci. 2019, 14, 547–552. [Google Scholar]
- Nguyen, A.T.; Taniguchi, T.; Eciolaza, L.; Campos, V.; Palhares, R.; Suge, M. Fuzzy Control Systems: Past, Present and Future. IEEE Comput. Intell. Mag. 2019, 14, 56–68. [Google Scholar] [CrossRef]
- Li, W.; Qiao, J.; Zeng, X.J. Online and Self-Learning Approach to the Identification of Fuzzy Neural Networks. IEEE Trans. Fuzzy Syst. 2022, 30, 649–662. [Google Scholar] [CrossRef]
- Pérez García, A. Servocontrol Visual de Una Cámara Activa Usando Técnicas Geno-Difusas. B. Eng. Thesis, Universidad de Guanajuato, Guanajuato, Mexico, 2002. (In Spanish). [Google Scholar]
- Jacobson, S.H.; Yücesan, E. Analyzing the Performance of Generalized Hill Climbing Algorithms. J. Heuristics 2004, 10, 387–405. [Google Scholar] [CrossRef]
- Lemos, A.; Caminhas, W.; Gomide, F. Multivariable Gaussian Evolving Fuzzy Modeling System. IEEE Trans. Fuzzy Syst. 2011, 19, 91–104. [Google Scholar] [CrossRef]
Method | RMSE |
---|---|
DENFIS [42] | 0.1749 |
eTS [43] | 0.0682 |
GSETSK [44] | 0.0661 |
LI [60] | 0.0649 |
ANFIS [30] | 0.0558 |
Proposed method | 0.0402 |
Method | RMSE |
---|---|
SAFIS [40] | 0.0221 |
FLEXFIS Var A [38] | 0.0176 |
FLEXFIS Var B [38] | 0.0171 |
SOFMLS [45] | 0.0201 |
eMG [63] | 0.0058 |
DeTS [46] | 0.0172 |
RSIM [47] | 0.0006 |
SEA [48] | 0.0004 |
LI [60] | 0.0025 |
ANFIS [30] | 0.0080 |
Proposed method |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Morales-Viscaya, J.A.; Alonso-Ramírez, A.A.; Castro-Liera, M.A.; Gómez-Cortés, J.C.; Lazaro-Mata, D.; Peralta-López, J.E.; Coello Coello, C.A.; Botello-Álvarez, J.E.; Barranco-Gutiérrez, A.I. Fuzzy Model Parameter and Structure Optimization Using Analytic, Numerical and Heuristic Approaches. Symmetry 2023, 15, 1417. https://doi.org/10.3390/sym15071417
Morales-Viscaya JA, Alonso-Ramírez AA, Castro-Liera MA, Gómez-Cortés JC, Lazaro-Mata D, Peralta-López JE, Coello Coello CA, Botello-Álvarez JE, Barranco-Gutiérrez AI. Fuzzy Model Parameter and Structure Optimization Using Analytic, Numerical and Heuristic Approaches. Symmetry. 2023; 15(7):1417. https://doi.org/10.3390/sym15071417
Chicago/Turabian StyleMorales-Viscaya, Joel Artemio, Adán Antonio Alonso-Ramírez, Marco Antonio Castro-Liera, Juan Carlos Gómez-Cortés, David Lazaro-Mata, José Eleazar Peralta-López, Carlos A. Coello Coello, José Enrique Botello-Álvarez, and Alejandro Israel Barranco-Gutiérrez. 2023. "Fuzzy Model Parameter and Structure Optimization Using Analytic, Numerical and Heuristic Approaches" Symmetry 15, no. 7: 1417. https://doi.org/10.3390/sym15071417
APA StyleMorales-Viscaya, J. A., Alonso-Ramírez, A. A., Castro-Liera, M. A., Gómez-Cortés, J. C., Lazaro-Mata, D., Peralta-López, J. E., Coello Coello, C. A., Botello-Álvarez, J. E., & Barranco-Gutiérrez, A. I. (2023). Fuzzy Model Parameter and Structure Optimization Using Analytic, Numerical and Heuristic Approaches. Symmetry, 15(7), 1417. https://doi.org/10.3390/sym15071417