# Recommendation System Using Autoencoders

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Methodology

#### 3.1. Business Understanding

- Increase the number of sales;
- Improve company’s revenue;
- Encourage engagement and activity on products and services;
- Gain competitive advantage;
- Calibrate user preferences;
- Make personalized recommendations;
- Find the recommendation algorithm and parameterization that leads to the highest overall performance of the product recommendation system.

#### 3.2. Data Understanding

#### 3.3. Data Preparation

#### 3.4. Modelling

#### 3.4.1. Autoencoder Overview

**Encoder**has the function of compressing input information into a different latent space;**Code**is a part of the network that represents the compressed input which is fed to the decoder;**Decoder**does the reverse work and reconstructs the original information, moving from the latent space to the original information space.

#### 3.4.2. Initialization of Parameters

- For the loss function, because we are not dealing with a classification problem where binary cross-entropy can be used, the loss function used had to be the Mean Square Error (MSE);
- The selection of the optimizer had to do with the fact that ADAM presents itself as one of the best, as it is very fast, requires little memory and is ideal for handling large amounts of data;
- The dropout is only used during training, and is then automatically disabled during execution;
- Finally, the choice of the activation function was based on the range to which the network values went, as well as the results of the experiments performed while running the autoencoder with other functions that were tested, but the loss and RMSE results were worse than the ones obtained with Tanh for the encoder layer, latent space and Linear for the output layer.

#### 3.4.3. Architecture

#### 3.5. Evaluation

## 4. Results and Discussion

## 5. Conclusions and Future Work

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

CF | Collaborative Filtering |

CRISP-DM | Cross Industry Standard Process for Data Mining |

CVAE | Collaborative Variational Autoencoder |

DAE | Deep Autoencoder |

DM | Data Mining |

ML | MovieLens |

MSE | Mean Square Error |

PCA | Principal Component Analysis |

RBM | Restricted Boltzman Machines |

RMSE | Root Mean Square Error |

SVD | Singular Value Decomposition |

## References

- Zaslavsky, A.; Perera, C.; Georgakopoulos, D. Sensing as a service and big data. arXiv
**2013**, arXiv:1301.0159. [Google Scholar] - Brandão, A.; Pereira, E.; Esteves, M.; Portela, F.; Santos, M.F.; Abelha, A.; Machado, J. A benchmarking analysis of open-source business intelligence tools in healthcare environments. Information
**2016**, 7, 57. [Google Scholar] [CrossRef][Green Version] - Cardoso, L.; Marins, F.; Portela, F.; Santos, M.; Abelha, A.; Machado, J. The next generation of interoperability agents in healthcare. Int. J. Environ. Res. Public Health
**2014**, 11, 5349–5371. [Google Scholar] [CrossRef] [PubMed][Green Version] - Adomavicius, G.; Tuzhilin, A. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng.
**2005**, 17, 734–749. [Google Scholar] [CrossRef] - Said, A.; Bellogín, A. You are What You Eat! Tracking Health Through Recipe Interactions. In Proceedings of the 6th Workshop on Recommender Systems and the Social Web (RSWeb 2014), Foster City, CA, USA, 6 October 2014. [Google Scholar]
- Miranda, R.; Ferreira, D.; Abelha, A.; Machado, J. Intelligent nutrition in healthcare and continuous care. In Proceedings of the 2019 International Conference in Engineering Applications (ICEA), Azores Island, Portugal, 8–11 July 2019; pp. 1–6. [Google Scholar]
- Ge, M.; Elahi, M.; Fernaández-Tobías, I.; Ricci, F.; Massimo, D. Using tags and latent factors in a food recommender system. In Proceedings of the 5th International Conference on Digital Health 2015, Florence, Italy, 18–20 May 2015; pp. 105–112. [Google Scholar]
- Rich, E. User modeling via stereotypes. Cogn. Sci.
**1979**, 3, 329–354. [Google Scholar] [CrossRef] - Powell, M.J.D. Approximation Theory and Methods; Cambridge University Press: Cambridge, UK, 1981. [Google Scholar]
- Salton, G. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer; Addison-Wesley: Upper Saddle River, NJ, USA, 1989; Volume 169. [Google Scholar]
- Armstrong, J.S. Principles of Forecasting: A Handbook for Researchers And Practitioners; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2001; Volume 30. [Google Scholar]
- Murthi, B.; Sarkar, S. The role of the management sciences in research on personalization. Manag. Sci.
**2003**, 49, 1344–1362. [Google Scholar] [CrossRef][Green Version] - Lilien, G.L.; Kotler, P.; Moorthy, K.S. Marketing Models; Prentice Hall: Upper Saddle River, NJ, USA, 1992. [Google Scholar]
- Felfernig, A.; Burke, R. Constraint-based recommender systems: Technologies and research issues. In Proceedings of the 10th International Conference on Electronic Commerce, Innsbruck, Austria, 19–22 August 2008; pp. 1–10. [Google Scholar]
- Costa, A.; Guizzardi, R.; Guizzardi, G.; Pereira Filho, J. COReS: Context-aware, Ontology-based Recommender system for Service recommendation. In Proceedings of the 19th International Conference on Advanced Information Systems Engineering (CAISE), Trondheim, Norway, 11–15 June 2007. [Google Scholar]
- Bobadilla, J.; Ortega, F.; Hernando, A.; Gutiérrez, A. Recommender systems survey. Knowl.-Based Syst.
**2013**, 46, 109–132. [Google Scholar] [CrossRef] - Breese, J.S.; Heckerman, D.; Kadie, C. Empirical analysis of predictive algorithms for collaborative filtering. arXiv
**2013**, arXiv:1301.7363. [Google Scholar] - Koren, Y.; Bell, R.; Volinsky, C. Matrix factorization techniques for recommender systems. Computer
**2009**, 42, 30–37. [Google Scholar] [CrossRef] - Zhang, A.; Lipton, Z.C.; Li, M.; Smola, A.J. Dive into Deep Learning. 2020. Available online: https://d2l.ai (accessed on 10 May 2020).
- Bobadilla, J.; Alonso, S.; Hernando, A. Deep Learning Architecture for Collaborative Filtering Recommender Systems. Appl. Sci.
**2020**, 10, 2441. [Google Scholar] [CrossRef][Green Version] - Van den Oord, A.; Dieleman, S.; Schrauwen, B. Deep content-based music recommendation. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2013; pp. 2643–2651. [Google Scholar]
- Salakhutdinov, R.; Mnih, A.; Hinton, G. Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, USA, 20–24 June 2007; pp. 791–798. [Google Scholar]
- Haghighi, P.S.; Seton, O.; Nasraoui, O. An Explainable Autoencoder for Collaborative Filtering Recommendation. arXiv
**2019**, arXiv:2001.04344. [Google Scholar] - Ren, Y.; Gong, S. A collaborative filtering recommendation algorithm based on SVD smoothing. In Proceedings of the 2009 Third International Symposium on Intelligent Information Technology Application, Nanchang, China, 21–22 November 2009; Volume 2, pp. 530–532. [Google Scholar]
- Sarwar, B.; Karypis, G.; Konstan, J.; Riedl, J. Incremental singular value decomposition algorithms for highly scalable recommender systems. In Proceedings of the Fifth International Conference on Computer and Information Science, Research Triangle Park, NC, USA, 8–13 March 2002; Volume 1, pp. 27–28. [Google Scholar]
- Ba, Q.; Li, X.; Bai, Z. Clustering collaborative filtering recommendation system based on SVD algorithm. In Proceedings of the 2013 IEEE 4th International Conference on Software Engineering and Service Science, Beijing, China, 23–25 May 2013; pp. 963–967. [Google Scholar]
- Zhang, G.; Liu, Y.; Jin, X. A survey of autoencoder-based recommender systems. Front. Comput. Sci.
**2020**, 14, 430–450. [Google Scholar] [CrossRef] - Dai Tran, H.; Hussain, Z.; Zhang, W.E.; Dang Khoa, N.L.; Tran, N.H.; Sheng, Q.Z. Deep Autoencoder for Recommender Systems: Parameter Influence Analysis. arXiv
**2018**, arXiv:1901.00415. [Google Scholar] - Sedhain, S.; Menon, A.K.; Sanner, S.; Xie, L. Autorec: Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, 18–22 May 2015. [Google Scholar]
- Cao, S.; Yang, N.; Liu, Z. Online news recommender based on stacked auto-encoder. In Proceedings of the 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), Wuhan, China, 24–26 May 2017; pp. 721–726. [Google Scholar]
- Ouyang, Y.; Liu, W.; Rong, W.; Xiong, Z. Autoencoder-based collaborative filtering. In Proceedings of the International Conference on Neural Information Processing, Kuching, Malaysia, 3–6 November 2014; pp. 284–291. [Google Scholar]
- Li, X.; She, J. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017; pp. 305–314. [Google Scholar]
- Kuchaiev, O.; Ginsburg, B. Training deep autoencoders for collaborative filtering. arXiv
**2017**, arXiv:1708.01715. [Google Scholar] - Aqra, I.; Abdul Ghani, N.; Maple, C.; Machado, J.; Sohrabi Safa, N. Incremental Algorithm for Association Rule Mining under Dynamic Threshold. Appl. Sci.
**2019**, 9, 5398. [Google Scholar] [CrossRef][Green Version] - Neto, C.; Brito, M.; Lopes, V.; Peixoto, H.; Abelha, A.; Machado, J. Application of data mining for the prediction of mortality and occurrence of complications for gastric cancer patients. Entropy
**2019**, 21, 1163. [Google Scholar] [CrossRef][Green Version] - Ferreira, D.; Peixoto, H.; Machado, J.; Abelha, A. Predictive Data Mining in Nutrition Therapy. In Proceedings of the 2018 13th APCA International Conference on Automatic Control and Soft Computing (CONTROLO), Azores, Portugal, 4–6 June 2018; pp. 137–142. [Google Scholar]
- Singh, H.; Kaswan, K.S. Clinical decision support systems for heart disease using data mining approach. Int. J. Comput. Sci. Softw. Eng.
**2016**, 5, 19. [Google Scholar] - Ribeiro, A.; Portela, F.; Santos, M.; Abelha, A.; Machado, J.; Rua, F. Patients’ Admissions in Intensive Care Units: A Clustering Overview. Information
**2017**, 8, 23. [Google Scholar] [CrossRef][Green Version] - Krishnaiah, V.; Narsimha, G.; Chandra, N.S. A study on clinical prediction using Data Mining techniques. Int. J. Comput. Sci. Eng. Inf. Technol. Res.
**2013**, 1, 239–248. [Google Scholar] - Iavindrasana, J.; Cohen, G.; Depeursinge, A.; Müller, H.; Meyer, R.; Geissbuhler, A. Clinical data mining: A review. Yearb. Med. Inform.
**2009**, 18, 121–133. [Google Scholar] - TensorFlow. About TensorFlow. Available online: https://www.tensorflow.org/ (accessed on 24 May 2020).
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX symposium on operating systems design and implementation (OSDI 16), Savannah, GA, USA, 1–4 November 2016; pp. 265–283. [Google Scholar]
- Zanker, M.; Bricman, M.; Gordea, S.; Jannach, D.; Jessenitschnig, M. Persuasive online-selling in quality and taste domains. In Proceedings of the International Conference on Electronic Commerce and Web Technologies, Kraków, Poland, 5–7 September 2006; pp. 51–60. [Google Scholar]
- Nurmaini, S.; Darmawahyuni, A.; Sakti Mukti, A.N.; Rachmatullah, M.N.; Firdaus, F.; Tutuko, B. Deep Learning-Based Stacked Denoising and Autoencoder for ECG Heartbeat Classification. Electronics
**2020**, 9, 135. [Google Scholar] [CrossRef][Green Version] - Hinton, G.E.; Zemel, R.S. Autoencoders, minimum description length and Helmholtz free energy. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 1994; pp. 3–10. [Google Scholar]
- Santana, M. Deep Learning para Sistemas de Recomendação (Parte 1)-Introdução. 2018. Available online: https://medium.com/data-hackers/deep-learning-para-sistemas-de-recomenda%C3%A7%C3%A3o-parte-1-introdu%C3%A7%C3%A3o-b19a896c471e (accessed on 12 May 2020).
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.
**2014**, 15, 1929–1958. [Google Scholar] - Jannach, D.; Zanker, M.; Felfernig, A.; Friedrich, G. Recommender Systems: An Introduction; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
- Schröder, G.; Thiele, M.; Lehner, W. Setting goals and choosing metrics for recommender system evaluations. In Proceedings of the UCERSTI2 Workshop at the 5th ACM Conference on Recommender Systems, Chicago, IL, USA, 23–27 October 2011; Volume 23, p. 53. [Google Scholar]
- Herlocker, J.L.; Konstan, J.A.; Terveen, L.G.; Riedl, J.T. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS)
**2004**, 22, 5–53. [Google Scholar] [CrossRef] - Isinkaye, F.; Folajimi, Y.; Ojokoh, B. Recommendation systems: Principles, methods and evaluation. Egypt. Inform. J.
**2015**, 16, 261–273. [Google Scholar] [CrossRef][Green Version] - Hug, N. Available online: http://surpriselib.com/ (accessed on 15 June 2020).

**Figure 4.**Distribution ratings of the ML-1M dataset: (

**A**) Distribution of the quantity of products by a certain number of evaluations; (

**B**) Total number of ratings for each scale value [1–5].

ML-1M | ML-10M | |
---|---|---|

Items | 3043 | 8940 |

Users | 6022 | 69,838 |

Activation Function | Tanh for encoder layer, latent space and Linear for output layer |
---|---|

Optimizer | Adam with a learning rate of 0.0001 |

Loss Function | Mean Square Error (MSE) |

Dropout | 0.1 |

**Table 3.**Comparison of Root Mean Square Error (RMSE) with a different layer number in the autoencoder.

RMSE | ||
---|---|---|

ML-1M | ML-10M | |

1Layer + 1Latent_Space | 0.029 | 0.010 |

2layers + 1Latent_Space | 0.038 | 0.012 |

3Layers + 1Latent_Space | 0.050 | 0.015 |

RMSE | ||
---|---|---|

ML-1M | ML-10M | |

SVD | 0.184 | 0.176 |

Autoencoder | 0.029 | 0.010 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ferreira, D.; Silva, S.; Abelha, A.; Machado, J. Recommendation System Using Autoencoders. *Appl. Sci.* **2020**, *10*, 5510.
https://doi.org/10.3390/app10165510

**AMA Style**

Ferreira D, Silva S, Abelha A, Machado J. Recommendation System Using Autoencoders. *Applied Sciences*. 2020; 10(16):5510.
https://doi.org/10.3390/app10165510

**Chicago/Turabian Style**

Ferreira, Diana, Sofia Silva, António Abelha, and José Machado. 2020. "Recommendation System Using Autoencoders" *Applied Sciences* 10, no. 16: 5510.
https://doi.org/10.3390/app10165510