# A Recommender System for Programming Online Judges Using Fuzzy Information Modeling

^{1}

^{1}/

_{2}, 65100 Ciego de Ávila, Cuba

^{2}

^{1}/

_{2}, 70100 Camagüey, Cuba

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- The user selects a problem to solve from the online judge (Figure 1).
- The code for solving such a problem is written.
- This source code is uploaded to the online judge (Figure 2).
- The judge automatically indicates whether the solution is correct or not. If not, classifies the failure mainly in Wrong Answer, Time limited exceeded or Runtime Error, allowing new solution attempts to the same problem.

- A fuzzy based uncertainty-aware approach for recommending problems to solve in POJs.
- A data preprocessing strategy to be used on the user profile to remove anomalous behaviors that could affect the later recommendation generation.

## 2. Background

#### 2.1. Programming Online Judges

#### 2.2. Fuzzy Tools in Recommender Systems

#### 2.3. Recommender Systems in POJs Scenarios

## 3. Problems Recommendation in Programming Online Judges Using Fuzzy Tools

**Data preparation**, to put data into a suitable format for recommendation generation (Section 3.1),- The
**data preprocessing strategy**to remove anomalous behaviors over the initial data (Section 3.2), - The
**fuzzy transformation**of the POJ data, to provide flexibility to the representation of the interaction between any user and problem (Section 3.3), and - The users similarity calculation and final
**problems recommendation**(Section 3.4).

#### 3.1. Data Preparation

#### 3.2. Data Preprocessing Strategy

- A pair $[\alpha \times ({x}_{u}^{s}-de{v}_{u}^{s});\alpha \times ({x}_{u}^{s}+de{v}_{u}^{s})]$ characterizing accepted problems, and
- A pair $[\alpha \times ({x}_{u}^{f}-de{v}_{u}^{f});\alpha \times ({x}_{u}^{f}+de{v}_{u}^{f})]$ characterizing problems attempted but failed.

#### 3.3. Fuzzy Transformation

#### 3.4. Users Similarity Calculation and Problems Recommendation

## 4. Experimental Study

- The amount of k nearest neighbors: It will be used $k=80$, considering it is a value previously used in related works presented in the background section. As future works, it will be explored the behavior of the proposal for other values of k.
- The value of t, in the fuzzy membership function presented in Figure 5. This value is connected to the number of attempts where the difficulty of the problems can be considered as maximum (see Section 3.3). Therefore, it could be linked with the global average of ${x}_{u}$, being ${x}_{u}$ the average number of attempts for user u (see Table 1). Regarding that the average of ${x}_{u}$ for the current data was 1.18 with a mean deviation of 1.17, here it will be considered $t\ge 3$. Specifically, it will be reported the results for $t=4$, $t=5$, and $t=6$. Further experiments were developed considering other values (e.g., $t=3$), but their performances were under the associated to the mentioned ones.
- The value of $\alpha $ in the data preprocessing strategy. To verify whether the flexibility in the definition of the interval for the outliers characterization leads to positive results, it will be considered $\alpha =1$ (no flexibility), and $\alpha =0.8$ (a small flexibility, considering correct values to some small amounts of attempts treated as outliers by $\alpha =1$).

- At first it will be considered recommendation lists with large sizes, varying such size n in the range [5, 25] with step 10, and
- At second it will be considered smaller recommendation lists, varying n in the range [1, 5] with step 1.

**Objective 1:**Determine whether the fuzzy recommendation approach outperforms previous approaches that do not consider the management of the uncertainty associated to POJ data.

**Objective 2:**Determine the effect of the proposed data preprocessing strategy, in contrast to previous approaches that also consider data preprocessing in POJ scenarios.

## 5. Conclusions

- Both the fuzzy approach and the data preprocessing strategy outperform previous related works in relation to the accuracy of the problem recommendation task, according to the F1 metric.
- The best performances were globally associated to smaller sizes of recommendation list ($n\le 5$).
- The use of more sophisticated data preprocessing strategy (the evaluation-based strategy), leads to the best results in small sizes of recommendation list ($n\le 5$).
- The global and the failed-eval-based data preprocessing strategies modify always under 17% of the data, guaranteeing a low intrusiveness level of the proposal.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Kurnia, A.; Lim, A.; Cheang, B. Online judge. Comput. Educ.
**2001**, 36, 299–315. [Google Scholar] [CrossRef] - Wang, G.P.; Chen, S.Y.; Yang, X.; Feng, R. OJPOT: Online judge & practice oriented teaching idea in programming courses. Eur. J. Eng. Educ.
**2016**, 41, 304–319. [Google Scholar] - Seaborn, K.; Fels, D.I. Gamification in theory and action: A survey. Int. J. Hum.-Comput. Stud.
**2015**, 74, 14–31. [Google Scholar] [CrossRef] - Bobadilla, J.; Ortega, F.; Hernando, A.; Gutiérrez, A. Recommender systems survey. Knowl.-Based Syst.
**2013**, 46, 109–132. [Google Scholar] [CrossRef] - Ortega, F.; Hernando, A.; Bobadilla, J.; Kang, J.H. Recommending items to group of users using Matrix Factorization based Collaborative Filtering. Inf. Sci.
**2016**, 345, 313–324. [Google Scholar] [CrossRef] - Venkatraman, S. A Proposed Business Intelligent Framework for Recommender Systems. Informatics
**2017**, 4, 40. [Google Scholar] [CrossRef] - Noguera, J.; Barranco, M.; Segura, R.; Martínez, L. A Mobile 3D-GIS Hybrid Recommender System for Tourism. Inf. Sci.
**2012**, 215, 37–52. [Google Scholar] [CrossRef] - Lu, J.; Wu, D.; Mao, M.; Wang, W.; Zhang, G. Recommender system application developments: A survey. Decis. Support Syst.
**2015**, 74, 12–32. [Google Scholar] [CrossRef] - Sergis, S.; Sampson, D. Learning Object Recommendations for Teachers Based On Elicited ICT Competence Profiles. IEEE Trans. Learn. Technol.
**2016**, 9, 67–80. [Google Scholar] [CrossRef] - Salehi, M.; Kamalabadi, I. Hybrid recommendation approach for learning material based on sequential pattern of the accessed material and the learner’s preference tree. Knowl.-Based Syst.
**2013**, 48, 57–69. [Google Scholar] [CrossRef] - Sobecki, J. Comparison of Selected Swarm Intelligence Algorithms in Student Courses Recommendation Application. Int. J. Softw. Eng. Knowl. Eng.
**2014**, 24, 91–109. [Google Scholar] [CrossRef] - Yera, R.; Caballero Mota, Y. An e-Learning Collaborative Filtering Approach to Suggest Problems to Solve in Programming Online Judges. Int. J. Distance Educ. Technol.
**2014**, 12, 51–65. [Google Scholar] - Yera, R.; Martínez, L. A recommendation approach for programming online judges supported by data preprocessing techniques. Appl. Intell.
**2017**, 47, 277–290. [Google Scholar] [CrossRef] - Badaracco, M.; Martínez, L. A fuzzy linguistic algorithm for adaptive test in Intelligent Tutoring System based on competences. Expert Syst. Appl.
**2013**, 40, 3073–3086. [Google Scholar] [CrossRef] - Miranda, S.; Orciuoli, F.; Sampson, D.G. A SKOS-based framework for Subject Ontologies to improve learning experiences. Comput. Hum. Behav.
**2016**, 61, 609–621. [Google Scholar] [CrossRef] - Adomavicius, G.; Tuzhilin, A.T. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng.
**2005**, 17, 734–749. [Google Scholar] [CrossRef] - Leal, J.P.; Silva, F. Mooshak: A Web-based multi-site programming contest system. Softw. Pract. Exp.
**2003**, 33, 567–581. [Google Scholar] [CrossRef] - Llana, L.; Martin-Martin, E.; Pareja-Flores, C.; Velázquez-Iturbide, J.Á. FLOP: A User-Friendly System for Automated Program Assessment. J. Univers. Comput. Sci.
**2014**, 20, 1304–1326. [Google Scholar] - Verdú, E.; Regueras, L.M.; Verdú, M.J.; Leal, J.P.; de Castro, J.P.; Queirós, R. A distributed system for learning programming on-line. Comput. Educ.
**2012**, 58, 1–10. [Google Scholar] [CrossRef] - Ala-Mutka, K.M. A survey of automated assessment approaches for programming assignments. Comput. Sci. Educ.
**2005**, 15, 83–102. [Google Scholar] [CrossRef] - Caiza, J.; Del Amo, J. Programming Assignments Automatic Grading: Review of Tools and Implementations. In Proceedings of the INTED 2013, Valencia, Spain, 4–6 March 2013; pp. 5691–5700. [Google Scholar]
- Nadolski, R.J.; Van den Berg, B.; Berlanga, A.J.; Drachsler, H.; Hummel, H.G.; Koper, R.; Sloep, P.B. Simulating light-weight personalised recommender systems in learning networks: A case for pedagogy-oriented and rating-based hybrid recommendation strategies. J. Artif. Soc. Soc. Simul.
**2009**, 12, 4. [Google Scholar] - Regueras, L.M.; Verdú, E.; Muňoz, M.F.; Pérez, M.A.; De Castro, J.P.; Verdú, M.J. Effects of competitive e-learning tools on higher education students: A case study. IEEE Trans. Educ.
**2009**, 52, 279–285. [Google Scholar] [CrossRef] - Ning, X.; Desrosiers, C.; Karypis, G. A Comprehensive Survey of Neighborhood-Based Recommendation Methods. In Recommender Systems Handbook; Springer: Berlin, Germany, 2015; pp. 37–76. [Google Scholar]
- Yera, R.; Martínez, L. Fuzzy Tools in Recommender Systems: A Survey. Int. J. Comput. Intell. Syst.
**2017**, 10, 776–803. [Google Scholar] [CrossRef] - Al-Shamri, M.Y.H.; Al-Ashwal, N.H. Fuzzy-weighted Pearson Correlation Coefficient for Collaborative Recommender Systems. In Proceedings of the 15th International Conference on Enterprise Information Systems (ICEIS), Angers, France, 4–7 July 2013; pp. 409–414. [Google Scholar]
- Zhang, Z.; Lin, H.; Liu, K.; Wu, D.; Zhang, G.; Lu, J. A hybrid fuzzy-based personalized recommender system for telecom products/services. Inf. Sci.
**2013**, 235, 117–129. [Google Scholar] [CrossRef] - Cheng, L.C.; Wang, H.A. A fuzzy recommender system based on the integration of subjective preferences and objective information. Appl. Soft Comput.
**2014**, 18, 290–301. [Google Scholar] [CrossRef] - Menhaj, M.B.; Jamalzehi, S. Scalable user similarity estimation based on fuzzy proximity for enhancing accuracy of collaborative filtering recommendation. In Proceedings of the 2016 4th International Conference on Control, Instrumentation, and Automation (ICCIA), Qazvin, Iran, 27–28 January 2016; pp. 220–225. [Google Scholar]
- Son, L.H. HU-FCF: A hybrid user-based fuzzy collaborative filtering method in Recommender Systems. Expert Syst. Appl.
**2014**, 41, 6861–6870. [Google Scholar] [CrossRef] - Lu, J. A Personalized e-Learning Material Recommender System. In Proceedings of the 2nd International Conference on Information Technology for Applications, Harbin, China, 9–11 January 2004; pp. 1–6. [Google Scholar]
- Wu, D.; Lu, J.; Zhang, G. A Fuzzy Tree Matching-Based Personalized E-Learning Recommender System. IEEE Trans. Fuzzy Syst.
**2015**, 23, 2412–2426. [Google Scholar] [CrossRef] - Poorni, G.; Balaji, K.; DeepthiNivetha, C. A Personalized E-Learning Recommender System Using the Concept of Fuzzy Tree Matching. Int. J. Adv. Res. Comput. Eng. Technol.
**2015**, 4, 4039–4043. [Google Scholar] - Myszkorowski, K.; Zakrzewska, D. Using Fuzzy Logic for Recommending Groups in E-Learning Systems. In Computational Collective Intelligence. Technologies and Applications; Springer: Berlin, Germany, 2013; pp. 671–680. [Google Scholar]
- Caro, M.; Jimenez, G. Similar Users or Similar Items? Comparing Similarity-Based Approaches for Recommender Systems in Online Judges. In Proceedings of the ICCBR 2017, Rondheim, Norway, 26–28 June 2017; Springer: Berlin, Germany, 2017; pp. 92–107. [Google Scholar]
- Gunawardana, A.; Shani, G. A Survey of Accuracy Evaluation Metrics of Recommendation Tasks. J. Mach. Learn. Res.
**2009**, 10, 2935–2962. [Google Scholar] - Castro, J.; Yera, R.; Martínez, L. An empirical study of natural noise management in group recommendation systems. Decis. Support Syst.
**2017**, 94, 1–11. [Google Scholar] [CrossRef] - Castro, J.; Yera, R.; Martínez, L. A fuzzy approach for natural noise management in group recommender systems. Expert Syst. Appl.
**2018**, 94, 237–249. [Google Scholar] [CrossRef] - Yera, R.; Caballero Mota, Y.; Martínez, L. Correcting noisy ratings in collaborative recommender systems. Knowl.-Based Syst.
**2015**, 76, 96–108. [Google Scholar] - Prabhakar, S.; Spanakis, G.; Zaïane, O. Reciprocal Recommender System for Learners in Massive Open Online Courses (MOOCs). In Proceedings of the International Conference on Web Learning (ICWL), Cape Town, South Africa, 20–22 September 2017; Springer: Berlin, Germany, 2017; pp. 157–167. [Google Scholar]
- Symeonidis, P.; Malakoudis, D. MoocRec.com: Massive Open Online Courses Recommender System. In RecSys Posters, Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, 15–19 September 2016; ACM: New York, NY, USA, 2016. [Google Scholar]
- De Meo, P.; Quattrone, G.; Terracina, G.; Ursino, D. An XML-Based Multiagent System for Supporting Online Recruitment Services. IEEE Trans. Syst. Man Cybern. Part A
**2007**, 37, 464–480. [Google Scholar] [CrossRef]

Notation | Meaning |
---|---|

$M[u,p]$ | Binary value indicating whether the user u solves problem p |

$F[u,p]$ | Number of attempted solutions the user u has tried over the problem p |

${F}^{\ast}[u,p]$ | Fuzzy membership value associated to the number of failures $F[u,p]$ |

${x}_{u}$ | Average number of attempts for user u |

$de{v}_{u}$ | Average deviation of ${x}_{u}$ |

${x}_{u}^{s}$ | Average number of attempts for solved problems, by user u |

$de{v}_{u}^{s}$ | Average deviation of ${x}_{u}^{s}$ |

${x}_{u}^{f}$ | Average number of attempts for finally failed problems, by user u |

$de{v}_{u}^{f}$ | Average deviation of ${x}_{u}^{f}$ |

${C}_{u,v}$ | Set of problems which solutions have been attempted by both users u and v |

$SimPro{b}_{p}(u,v)$ | Similarity between the users u and v according to a specific problem p |

$Sim(u,v)$ | Global similarity between two users u and v |

${w}_{p}$ | Recommendation score for the problem p |

Recommended | Not Recommended | |
---|---|---|

Solved | True-Positive (tp) | False-Negative (fn) |

Not solved | False-Positive (fp) | True-Negative (tn) |

**Table 3.**Comparison between the current proposal and previous approaches, without considering data preprocessing step at Section 3.2. Large recommendation lists.

5 | 15 | 25 | |
---|---|---|---|

UCF-OJ [12] | 0.3568 | 0.3555 | 0.3237 |

PCF-OJ [35] | 0.3187 | 0.2965 | 0.2657 |

UCF-OJ++ [13] | 0.3613 | 0.3610 | 0.3277 |

Fuzzy-UCF-OJ (t = 6) | 0.3663 | 0.3600 | 0.3281 |

Fuzzy-UCF-OJ (t = 5) | 0.3640 | 0.3613 | 0.3286 |

Fuzzy-UCF-OJ (t = 4) | 0.3625 | 0.3607 | 0.3279 |

**Table 4.**Comparison between the current proposal and previous approaches, without considering data preprocessing step at Section 3.2. Smaller recommendation lists.

1 | 2 | 3 | 4 | 5 | |
---|---|---|---|---|---|

UCF-OJ [12] | 0.2454 | 0.2990 | 0.3316 | 0.3471 | 0.3568 |

PCF-OJ [35] | 0.2327 | 0.2869 | 0.3127 | 0.3182 | 0.3187 |

UCF-OJ++ [13] | 0.2407 | 0.2958 | 0.3268 | 0.3469 | 0.3613 |

Fuzzy-UCF-OJ (t = 6) | 0.2494 | 0.3010 | 0.3305 | 0.3517 | 0.3663 |

Fuzzy-UCF-OJ (t = 5) | 0.2487 | 0.3024 | 0.3320 | 0.3512 | 0.3640 |

Fuzzy-UCF-OJ (t = 4) | 0.2466 | 0.3021 | 0.3297 | 0.3488 | 0.3625 |

**Table 5.**Statistical analysis of the results, without considering data preprocessing at Section 3.2. Wilcoxon test (p < 0.05).

Significance Values | |
---|---|

Fuzzy-UCF-OJ (t = 6) vs. UCF-OJ++ [13] | 0.043 |

Fuzzy-UCF-OJ (t = 5) vs. UCF-OJ++ [13] | 0.018 |

Fuzzy-UCF-OJ (t = 4) vs. UCF-OJ++ [13] | 0.043 |

**Table 6.**Comparison between the current proposal and previous approaches, considering data preprocessing. Large recommendation lists.

5 | 15 | 25 | |
---|---|---|---|

UCF-OJ++ + preproc [13] | 0.3615 | 0.3619 | 0.3280 |

Fuzzy-UCF-OJ + preproc(t = 6, $\alpha $ = 1, global) | 0.3652 | 0.3607 | 0.3288 |

Fuzzy-UCF-OJ + preproc (t = 5, $\alpha $ = 1, global) | 0.3652 | 0.3604 | 0.3285 |

Fuzzy-UCF-OJ + preproc (t = 4, $\alpha $ = 1, global) | 0.3643 | 0.3612 | 0.3288 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 0.8, global) | 0.3649 | 0.3624 | 0.3291 |

Fuzzy-UCF-OJ + preproc (t = 5, $\alpha $ = 0.8, global) | 0.3649 | 0.3614 | 0.3289 |

Fuzzy-UCF-OJ + preproc (t = 4, $\alpha $ = 0.8, global) | 0.3640 | 0.3606 | 0.3288 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 1, eval-based) | 0.3633 | 0.3626 | 0.3280 |

Fuzzy-UCF-OJ + preproc (t = 5, $\alpha $ = 1, eval-based) | 0.3649 | 0.3609 | 0.3273 |

Fuzzy-UCF-OJ + preproc (t = 4, $\alpha $ = 1, eval-based) | 0.3662 | 0.3611 | 0.3275 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 0.8, eval-based) | 0.3659 | 0.3620 | 0.3279 |

Fuzzy-UCF-OJ + preproc (t = 5, $\alpha $ = 0.8, eval-based) | 0.3644 | 0.3601 | 0.3274 |

Fuzzy-UCF-OJ + preproc (t = 4, $\alpha $ = 0.8, eval-based) | 0.3667 | 0.3605 | 0.3268 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 1, failed-eval-based) | 0.3674 | 0.3620 | 0.3290 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 0.8, failed-eval-based) | 0.3664 | 0.3613 | 0.3287 |

**Table 7.**Comparison between the current proposal and previous approaches, without considering data preprocessing. Smaller recommendation lists.

1 | 2 | 3 | 4 | 5 | |
---|---|---|---|---|---|

UCF-OJ++ + preproc [13] | 0.2430 | 0.2967 | 0.3268 | 0.3485 | 0.3615 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 6, global) | 0.2501 | 0.3044 | 0.3302 | 0.3523 | 0.3652 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 5, global) | 0.2481 | 0.3038 | 0.3313 | 0.3515 | 0.3652 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 4, global) | 0.2471 | 0.3017 | 0.3318 | 0.3505 | 0.3643 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 6, global) | 0.2479 | 0.3029 | 0.3299 | 0.3532 | 0.3649 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 5, global) | 0.2493 | 0.3026 | 0.3318 | 0.3525 | 0.3649 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 4, global) | 0.2476 | 0.3014 | 0.3308 | 0.3511 | 0.3640 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 6, eval-based) | 0.2545 | 0.3047 | 0.3296 | 0.3529 | 0.3633 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 5, eval-based) | 0.2509 | 0.3051 | 0.3307 | 0.3520 | 0.3649 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 4, eval-based) | 0.2535 | 0.3035 | 0.3316 | 0.3513 | 0.3662 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 6, eval-based) | 0.2518 | 0.3020 | 0.3326 | 0.3541 | 0.3659 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 5, eval-based) | 0.2530 | 0.3058 | 0.3318 | 0.3538 | 0.3644 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 0.8, t = 4, eval-based) | 0.2521 | 0.3059 | 0.3304 | 0.3520 | 0.3667 |

Fuzzy-UCF-OJ + preproc ($\alpha $ = 1, t = 6, failed-eval-based) | 0.2530 | 0.3050 | 0.3334 | 0.3549 | 0.3674 |

Fuzzy-UCF-OJ + preproc ($\alpha $=0.8, t = 6, failed-eval-based) | 0.2510 | 0.3045 | 0.3365 | 0.3540 | 0.3664 |

**Table 8.**Statistical analysis of the results, considering data preprocessing at Section 3.2. Wilcoxon test (p < 0.05).

Significance Values | |
---|---|

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 0.8, global) vs. UCF-OJ++ + preproc [13] | 0.018 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 1, eval-based) vs. UCF-OJ++ + preproc [13] | 0.028 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 1, failed-eval-based) vs. UCF-OJ++ + preproc [13] | 0.018 |

Fuzzy-UCF-OJ + preproc (t = 6, $\alpha $ = 0.8, failed-eval-based) vs. UCF-OJ++ + preproc [13] | 0.028 |

Fuzzy-UCF-OJ + Preproc (Global) | ||
---|---|---|

$\mathit{\alpha}$ = 0.8 | $\mathit{\alpha}$ = 1 | |

14.82% | 8.74% | |

Fuzzy-UCF-OJ + preproc (eval-based) | ||

$\mathit{\alpha}$ = 0.8 | $\mathit{\alpha}$ = 1 | |

36.69% | 28.3% | |

Fuzzy-UCF-OJ + preproc (failed-eval-based) | ||

$\mathit{\alpha}$ = 0.8 | $\mathit{\alpha}$ = 1 | |

16.09% | 13.44% |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yera Toledo, R.; Caballero Mota, Y.; Martínez, L.
A Recommender System for Programming Online Judges Using Fuzzy Information Modeling. *Informatics* **2018**, *5*, 17.
https://doi.org/10.3390/informatics5020017

**AMA Style**

Yera Toledo R, Caballero Mota Y, Martínez L.
A Recommender System for Programming Online Judges Using Fuzzy Information Modeling. *Informatics*. 2018; 5(2):17.
https://doi.org/10.3390/informatics5020017

**Chicago/Turabian Style**

Yera Toledo, Raciel, Yailé Caballero Mota, and Luis Martínez.
2018. "A Recommender System for Programming Online Judges Using Fuzzy Information Modeling" *Informatics* 5, no. 2: 17.
https://doi.org/10.3390/informatics5020017