Next Article in Journal
An Effective Resource Matching Scheme Based on a Novel Unified Descriptive Model for Modern Manufacturing Industry Systems
Next Article in Special Issue
Applying Collective Intelligence in Health Recommender Systems for Smoking Cessation: A Comparison Trial
Previous Article in Journal
Maximizing Channel Capacity of 3D MIMO System via Antenna Downtilt Angle Adaptation Using a Q-Learning Algorithm
Previous Article in Special Issue
A Cascade Framework for Privacy-Preserving Point-of-Interest Recommender System
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Using LinkedIn Endorsements to Reinforce an Ontology and Machine Learning-Based Recommender System to Improve Professional Skills

María Cora Urdaneta-Ponte
Ibon Oleagordia-Ruíz
1 and
Amaia Méndez-Zorrilla
eVIDA Research Group, Faculty of Engineering, University of Deusto, 48007 Bilbao, Spain
Faculty of Engineering, Andres Bello Catholic University (UCAB), Ciudad Guayana 08050, Venezuela
Author to whom correspondence should be addressed.
Electronics 2022, 11(8), 1190;
Submission received: 28 February 2022 / Revised: 3 April 2022 / Accepted: 6 April 2022 / Published: 8 April 2022
(This article belongs to the Special Issue Recommender Systems: Approaches, Challenges and Applications)


Nowadays, social networks have become highly relevant in the professional field, in terms of the possibility of sharing profiles, skills and jobs. LinkedIn has become the social network par excellence, owing to its content in professional and training information and where there are also endorsements, which are validations of the skills of users that can be taken into account in the recruitment process, as well as in the recommender system. In order to determine how endorsements influence Lifelong Learning course recommendations for professional skills development and enhancement, a new version of our Lifelong Learning course recommendation system is proposed. The recommender system is based on ontology, which allows modelling the data of knowledge areas and job performance sectors to represent professional skills of users obtained from social networks. Machine learning techniques are applied to group entities in the ontology and make predictions of new data. The recommender system has a semantic core, content-based filtering, and heuristics to perform the formative suggestion. In order to validate the data model and test the recommender system, information was obtained from web-based lifelong learning courses and information was collected from LinkedIn professional profiles, incorporating the skills endorsements into the user profile. All possible settings of the system were tested. The best result was obtained in the setting based on the spatial clustering algorithm based on the density of noisy applications. An accuracy of 94% and 80% recall was obtained.

Graphical Abstract

1. Introduction and Related Studies

Employability requirements have changed to address the new realities and challenges faced by organizations. According to [1] cited by [2], in several fields, some discrepancies emerge between employee competences and the labor market. In view of this, professionals must improve and/or develop their competences and skills according to trends in the labor market in order to be competitive.
Nowadays, organizations can use different Social Networks Sites (SNSs) to research into candidates [3], as well as to gather information on labor trends so as to adjust academic plans, analyze student profiles and extract their skills, and identify the most desired skills for the purpose of adjusting curricula [4].
In the professional field, LinkedIn has played a very important role in terms of the dissemination of jobs, where, according to the work presented by one of its missions is to match jobs with suitable professionals [5].
To support this process, the Recommender Systems (RSs) have emerged, which collect the information linking users to items and use it to make relevant and meaningful suggestions. By [6] the RSs base the prediction of user interests, according to their explicit or implicit preferences. RSs are not only used to find the ideal candidate for a job position, but also to offer educational services to professionals to develop professional skills.
RSs use different filtering algorithms, in which the predominant approaches are collaborative filtering, content-based filtering and hybrid filtering. This combines the two approaches so as to overcome any problems that may arise through individual use of the different techniques, such as the problem of data sparsity, or owing to the lack of information provided by users about the recommended items, which is known as cold start [7], and thus improve recommendation performance [8].
On the other hand, the quality of recommendations is highly dependent on information retrieval and representation. According to [2], RSs handle a large amount of data that, depending on their nature, can derive from different sources such as SNSs, which are characterized by their heterogeneity. In this regard, semantic RSs have emerged, which make use of the semantic web to represent reality in a specific domain, in a machine-readable way [9].
The work presented by [2] involves a review of articles that propose the use of ontology in order to create a model to describe the relationship between a domain area and the users that belong to it.
In this review, the importance of the representation and management of information is highlighted. Studies [10,11] pointed out that ontologies allow information of interest to users to be inferred, which can be used to enrich recommendations. Likewise, [12] indicated that the amount of information handled is a key factor in the success of recommendations. Studies [11,13] pointed out that the use of ontology improved cold start and data sparsity problems, which limit RS performance. [14] also suggested that the use of ontologies facilitates the analysis, reasoning, sharing and reuse of knowledge.
In recent years, artificial intelligence techniques have been introduced in RSs, where Machine Learning (ML) techniques are included [15]. According to [16], the use of these techniques has proved to be a promising solution when designing RSs in the era of Big Data.
According to [17] ML algorithms have become dominant in big data analysis, visualization and modeling. For [18] ML applies mathematical models and algorithms to gradually improve the performance of a task and can be categorized into two main groups: supervised and unsupervised learning. ML algorithms are performing tasks like classification, regression, clustering, dimensionality reduction, and ranking.
In the systematic review presented by [19], attention should be drawn to the work shown in Table 1 in this area.
In some cases, ML is used in RSs to make inferences about how past actions correspond to future outcomes. [23] applies this technique to predict future performance based on students’ academic record in order to re-schedule courses for target course preparation and exam preparation. [24] proposes an RS for different users in academia, using ML to extract information about research areas of interest to professors from publications and curricula, and then combining them with their background to assign courses and research work for supervisory purposes.
Additionally, semantic web techniques have been combined with ML techniques, allowing knowledge to be exploited from available information, updated and new relationships between data inferred [14]. The combination of an ontology-based recommender system with ML techniques has been used as an approach to improve the accuracy of recommendations, in addition to addressing information overload, with a view to solving cold start problems [26].
Based on the above [2], they propose a hybrid RS to suggest lifelong learning (LLL) courses to improve professional competences for users whose profile is built from their LinkedIn profile data. The RS core involves semantic filtering that uses ontology to model employment sectors and areas of knowledge to represent professional competences. The ontology is updated using events based on professional record profile data extracted from LinkedIn, using ML to cluster entities in order to make predictions about the new data. As a line of future work, they recommend considering the use of LinkedIn endorsements to enrich the user profile and thus improve recommendations.
Recently, there have been several research work oriented analyses of data provided by LinkedIn, which in addition to data related to academic and professional training, also highlights endorsements, which is one of the key features of LinkedIn, asking the viewer to endorse a skill for the candidate as proof of their skill level [27].
In a review of relevant literature, we found the research shown in Table 2, related to the use of LinkedIn endorsements.
This is highlighted in [4], in which an RS is proposed that classifies profiles according to skills and peer endorsements, identifying the most desired skills that should be covered by curricula and areas of learning, and suggesting possible corrections in learning programs. [32] indicates that in addition to this information, endorsements of the skills of LinkedIn users can be analyzed to complement the profile.
The system proposed by [32] recommends relevant job opportunities for users seeking employment in the area of information technology, based on the curriculum to be entered into the system. In addition, it makes it easy for recruiters to search for the best talent based on job requirements by analyzing not only the resume, but also LinkedIn endorsements, which could improve the user profile.
Based on the above, an improvement in RS of [2] is proposed, incorporating LinkedIn endorsements as an additional system input. These are taken into account in order to validate the skills determined in the RS profiles to improve recommendations.
In literature on the subject, we found some RSs aimed at developing professional skills through a wide range of training. In this respect, the objective of the RSs proposed by [33,34,35] is to identify the professional competencies that users need to develop in their current job, so that they can be taken into account in their training plan.
For [4], along with RSs, the analysis of SNSs data has become a common practice for collection of information about users, since [36], SNSs such as Facebook and LinkedIn have become integrated into everyday life.
Ref. [37] indicates that social media provide innovative pedagogical frameworks for teaching and learning that allow students to develop digital skills deemed useful for a successful professional career. He also points out that semantic social networks offer a series of advantages related to their strategic use, such as the creation of a network of contacts, which can have an impact on their professional development by connecting with professionals and following labor market trends.
As the use of SNSs has increased decade by decade, so has interest in using SNSs in recruitment and hiring processes [3]. In this sense, for [28], LinkedIn is the most influential web tool in terms of professional use, unlike other SNSs such as Facebook or Twitter, which focus on social relationships. In LinkedIn, users have greater visibility in professional terms, as they can share their training, skills and work experience [4].
For [37], LinkedIn focuses on professional networking and career development, designed to help people make connections, share experiences and resumes, and find jobs. It is also a tool that can be used to find relevant, quality content. According to [3], allowing users to provide written recommendations and endorsements of skills that appear on a user’s profile, LinkedIn has also introduced endorsements, in which users are asked to endorse the skills of other users [27].
The data contained in SNSs are very diverse [3], and RSs also obtain information from multiple sources; to handle heterogeneity, they have adopted semantic knowledge representation as one of the theories to help deal with this [26].
Based on this review and lending continuity to the RS presented by [2], it is proposed that LinkedIn skills endorsements be incorporated in order to validate the skills obtained in the user profiling stage to optimize performance and prediction of the continuing education course recommendation system. The system uses LinkedIn user records in the area of software development.
This article is structured as follows: Section 2 details the RS proposal; Section 3 describes the evaluation of the proposal; and Section 4 provides conclusions and recommendations for future research.

2. Materials and Methods

2.1. Materials

In the field of learning and in personalized recommendation services, different user dimensions, such as personal data, interests, knowledge levels, and context, among others, must be taken into account in order to respond appropriately to their needs [38]. Based on this, LinkedIn records used by [2] were employed to create user profiles, which contained personal, academic and professional data of users, as they in turn contain relevant information to help determine the area of knowledge, skills and labor sector that are taken into account when recommending courses. Additionally, LinkedIn endorsements were incorporated. For the purposes of evaluating the proposal, this was confined to the specific domain of software development.
For user and course profiles, professional skills and job sectors were coded using the taxonomies defined in [2].
The ontology used was stored in Neo4j, and selection of the tool in [2] was based on the fact that it is a NoSQL graph database manager, whereby the relationship is the most important database element representing the interconnection between nodes. This makes it ideal to represent knowledge graphs, allowing nodes and relationships and efficient operations to be managed on them.
As for the different programs that make up the system, the ones developed by [2] were used as a basis and programmed using Python 3. Cypher, the Neo4j query language, was used to query the graph.

2.2. Methods

In order to recommend continuing education courses to improve and develop professional skills, a hybrid RS based on ontology and ML is planned to determine the skills to be updated and/or developed according to labor market trends. User profiles are created from their LinkedIn records, taking into account endorsements, in order to validate and determine the level of skills. The recommendation process has three filtering stages at whose core is a semantic filtering, which is combined with filtering by content for the initial prediction of courses, and a heuristic one to obtain the final range of courses to be recommended.
The following is established as a basis for designing the RS:
The system input data is obtained from LinkedIn, to create user profiles, and from the web for courses, without requiring any additional information upload by the users.
Courses will be recommended using the skills they develop as a criterion. Skills in the user and course profiles need to be defined under the same terms in the recommendation process.
Modular design allows new algorithms to be incorporated into the system and multiple configuration options offered, as well as including parameterizations for the different algorithms, by enabling or disabling filtering stages in order to validate and evaluate the proposals.
The RS architecture proposed is shown in Figure 1.
The system consists of two main phases: the off-line phase (I), in which the user profiles (A) and (B) are built, definitions are loaded and data models are built from the ontology (C) and ML (D) update processes; an on-line phase (II), in which the course recommendation process is performed, for which purpose the user profile is to be given the recommendation (1) must be built, and which is performed in three filtering stages (2, 3 and 4).

2.2.1. Off-Line Phase

Profile Creation

A semantic profiling based on ontology is performed in order to generate the different profiles used, making use of taxonomies. This process consists of the user profiling process (A) and the course profiling process (B).
After analyzing the records of LinkedIn users and LLL courses extracted from the Web, two taxonomies were then defined to hierarchically code the areas of knowledge and job performance sectors needed for the recommendation process. The levels defined for the areas of knowledge are: area; sub-area; specialty; sub-specialization; and knowledge. For job performance sectors, the hierarchy levels are: sector; field; type; domain; and position.
The following are coded with a taxonomy of areas of knowledge: user skills with their level of specialization (LS) and their degree of updating (DU), the skills to be developed on LLL courses with an LS, and also the skills required to qualify for a course. The job performance sector taxonomy is used to hierarchically code the different positions held by users.
The user profile information is given by:
Demographic information.
Job performance sectors. This is a set of hierarchical codes defined by a taxonomy for job performance sectors, their description and dates.
Skill sets, where each skill is coded with three arguments: a hierarchical code that is defined by a taxonomy of areas of knowledge, a level of specialization and degree of update.
Course profile information is provided by:
Demographic information.
Skills to be developed. Each skill is coded with two arguments: a hierarchical code defined by a taxonomy of areas of knowledge and a level of specialization.
Required skills, where each is coded with a hierarchical code that is defined by a taxonomy of areas of knowledge.
Related skills. These are given by a set of codes of areas of knowledge, where the skills developed on the course may be of interest or complement them.

Ontology and ML

A key element for smooth performance of RSs is to ensure that there is a wealth of semantic data mining, and to prevent the loss of information obtained from data retrieval. Another process associated with this phase aims at building data models to represent the domain of skills and job performance sectors, according to areas of knowledge.
The ontology, in its broadest sense, seeks to represent positions, skills and existing relationships between them. Additionally, it should offer the capacity to represent different groupings of positions under the criterion of similarity according to the skills they use. Likewise, the different relationships that can be found between skills must be represented. The classes defined in the ontology are shown in Table 3.
Likewise, class hierarchy is given by the different relationships between classes of ontology as described in Table 4.
The ontology is updated in two stages from user profiles: through events (C), where the model is updated with the relationships between job performance sectors and areas of knowledge according to user profile competences; through machine learning (D), the positions of the entities in the ontology are grouped together using the density based spatial clustering of applications with noise algorithm (DBSCAN) and K-means algorithms.
For [39] K-means algorithm mainly uses the Euclidean distance function to measure the similarity between data objects, and the sum of squared errors is used to found spherical clusters with more uniform distribution of data.
The work presented by [40] defines DBSCAN as a clustering algorithm based on data point distribution density. The algorithm can identify the degree of data density and classify the data points in the distribution. At the same time, sporadic data points can be identified as noise, rather than being classified within a class.
In the event driven update, the ontology stores the relationship between the characters and areas of knowledge, which is given by skills with attributes, as well as the number of times that skill is present, its LS average and the promise of DU, which is calculated taking into account the number of users that possess the skill. The relationships are determined, and their attributes calculated, from a training dataset of user profiles via events in the ontology.
For the ontology substrate, new entities, relationships and attributes are defined in the taxonomies and inferences that are allowed to be made, which will be used in semantic filtering. Among the new relationships, synonyms and uses of terms in other languages are contemplated—for the purposes of testing this proposal, the use of terms in Castilian Spanish and English were both considered, due to their common use in the domain used. In the case of areas of knowledge, relationships of the type “is of interest” are used in their entities to indicate that users who possess a skill related to those entities may be interested in developing other skills with other knowledge, in order to obtain a more comprehensive profile.
Additionally, in work performance sectors, the position entities can have “opt for” relationships, where from one position one can opt for similar or higher positions in an orderly relationship defined by a ladder.
Job performance clusters are created in the machine learning update. An alternative for calculation of related skills involves using ML. It is proposed that unsupervised clustering algorithms be used in instances of position entities, in order to group similar positions, with these groupings being based on the similarity of skills associated with the relevant positions. Job performance charge clustering, based on the similarity of the skill set, can be used to determine the set of related positions for a particular position, or a particular user’s skill set. From the job grouping determined, related skills can be established as those skills that appear most frequently in the positions in the grouping, and which the user does not possess.

2.2.2. On-Line Stage

The course recommendation process is carried out at this stage. As shown in Figure 1, this is made up of the objective user profile construction process (1) and recommendation process, which is carried out in three filtering stages (2), (3) and (4).
  • (1) Process involved in creating the target user profile
This process is similar to the profile construction of the off-line stage. It differs in the semantic transduction stage in that it incorporates coding of the endorsements in the user profile. In the information obtained from the LinkedIn record of the user to whom the recommendation is to be made, are the endorsements and use of skill coding algorithms, and the hierarchical code of the area of knowledge is determined. To determine the LS and DU, heuristics are used that take into account the relative frequency of the number of users who endorse the skill. The endorsements are part of the profile of the user to whom the recommendation is to be made and are taken into account in the recommendation process as skills that the user has acquired.
  • Recommendation process
The recommendation process followed is the one proposed by our previous work [2], where the first stage corresponds to a modification of the semantic filtering algorithm (2) in order to determine the user’s own, related and ontological skills, as given by the incorporation of LinkedIn endorsements. Then a second content-based filtering (3) is applied for initial course prediction, and finally, filtering and sorting heuristics (4) are applied for the final recommendation of LLL courses to improve and/or develop professional skills based on a user’s record.
The process diagram represented in Figure 2 is followed in order to prepare course recommendations for each user.
  • (2) Semantic filtering to determine similar skills
In order to determine related skills for a user, we first determine his or her related work performance areas using the ontology, based on his or her skills. For this process, the set of work performance sectors where the user’s own skills appear is determined and filtered as follows: those related work performance sectors whose skills are covered by a given percentage of the user’s skills are selected, and/or those related work performance sectors whose associated skills cover a percentage of the user’s own skills are selected. In any case, the user’s job performance sectors are excluded from the set of related job performance sectors.
Alternatively, from the user’s job performance sectors it can be determined to which groupings (clusters) they belong, constructed according to ML algorithms. In the case of new data, i.e., a job performance sector that is not part of any of the clusters obtained during training, it is possible to predict the cluster, to which the user belongs, by making use of the user’s skills. Given the groupings, the set of job performance sectors that belong to each of them will form the user’s related performance sectors.
In the case of DBSCAN, a core node with a distance less than an epsilon value is located and associated with the cluster, to which it belongs, to predict clustering for new data; in the case of k-means, the centroid with the smallest distance is located and associated with that cluster. In both cases, the difference with one of the similarity functions is used as distance, according to Equation (1).
d = 1 S i m S K u , S K c ,
Finally, which related work performance sectors are deemed prior to the user’s work performance sectors are determined in an orderly relationship imposed by a ladder, and these are removed from related performance areas. Subsequently, with sectors of work performance, the user’s own and related skills are determined, which are the most frequent skills associated with related sectors of work performance that add value, in other words, that the skill does not exist among the set of skills or, if it does, that it has a higher level of specialization or degree of update than the user’s skills; finally, those skills that add value and are included under the relationship “is of interest” in the knowledge network are added to related skills.
Given the inputs:
UJS , Set of user job performance sectors.
USK , Set of user skills.
UESK , Set of validated user skills.
The following Algorithm 1 is used to semantically determine related skills (RSK), the result of modifying the algorithm described in [2], after taking endorsements into consideration:
Algorithm 1: Semantic determinining of similar skills.
Input :   U J S ,   U S K ,   U E S K
1: R J S  // Related job sector
2: R S K    
3:To s k U K S  obtain
4: R J S   R J S O n t o _ G e t J o b s j s
5: R J S   R J S U J S
6: R J S   O n t o _ C u t O f f _ J o b s R J S , U S K , u j k P , j s k P
7: R J S   R J S O n t o _ P r e v i o u s R J S , U J S
8:To js ∈ RJS obtain
9: R S K   R S K   s k ,   S V U S K UESK , s k s k O n t o _ S k i l l s j s
10:To s k U S K M S 2 S R S K , φ  obtain
11: R S K R S K s k ,   S V U S K UESK , s k s k O n t o _ I n t S k i l l s s k
12: R S K M S 2 S R S K , λ
Output :   R S K
The following Algorithm 2 is used to alternately determine similar skills using ML clustering:
Algorithm 2: Calculation of similar skills.
Input :   U J S ,   U S K ,   U E S K
1: R J S    // Related job sector
2: R S K    
3:To j s U J S  obtain
4: R J S   R J S O n t o _ C l u s t e r _ P r e d j s j s
5: R J S   R J S O n t o _ P r e v i o u s R J S , U J S  
6:To j s R J S  obtain
7: R S K   R S K   s k ,   S V U S K U E S K , s k s k O n t o _ S k i l l s j s
8:To s k U S K M S 2 S R S K , φ  obtain
9: R S K   R S K   s k ,   S V U S K   U E S K , s k s k O n t o _ I n t S k i l l s s k
10: R S K M S 2 S R S K , λ
Output :   R S K
Onto_Get_Jobs( s k ) performs job performance sectors whose users have the sk skills registered via events in the ontology and can be configured to use registered skills belonging to the same subspecialty sub-specialization of sk when no job performance sectors are obtained for the same.
Onto_CutOff_Jobs(jobs,sks,uskp,jskp) performs filtering of job performance sectors, js,js ∈jobs, according to a percentage jskp coverage of sks skill set over the skills associated with job performance sector js,js ∈jobs or a percentage skp coverage of sector skills js,js ∈jobs over the sks skill set.
Onto_Cluster_Pred( j s ) predicts the cluster for a js job sector according to clustering by Machine Learning in the ontology.
Onto_Previous( R J S , U J S ), the previous sectors in R J S to U J S according to the orderly relationship given by the ladder in the ontology.
Onto_Skills( J S i ) are the skills of J S i represented in the ontology, a product of update via events involving user profiles.
Onto_IntSkills( s k ) are the skills of interest associated with the sk skills represented in the ontology.
SV( S K , s k ) refers to the function that verifies that the sk skill is of value for SK skills.
MS2S( M S , c ) constructs a set with the most frequent c elements in the MS multiset.
RSK’, Multiset of related skills.
RSK refers to the set of similar skills.
  • (3) Content-based filtering to predict initial course recommendation
In this stage, the algorithm proposed in [2] is applied, where filtering by content is used to filter the courses from the course catalog that raise the level of specialization and/or degree of update of the user’s own skills, or help develop the user’s related skills. From this filtering, an initial recommendation is obtained, which is refined in the next stage.
  • (4) Heuristics of filtering and ordering courses for final prediction
The initial course prediction from the previous phase is complemented and filtered using heuristics proposed by [2], while the following is verified for the user profile and course catalog:
That the user has the necessary skills to approach the courses, and otherwise, courses that develop them are selected by applying demographic restrictions;
User demographic restrictions are applied to initial course prediction and to the result of the previous stage;
In the course prediction, the courses whose skills to be developed are the same or a subset of another are eliminated, retaining those with the highest score, to thus determine course recommendation for the user.

3. Evaluation and Results

According to [7], RS evaluation can be conducted through on-line and off-line experimentation. The work presented by [2] performed off-line RS evaluation. This type of evaluation allows different algorithms and approaches to be assessed, which is used in experimental environments, since the utmost consistency is desired in order to compare the performance of different proposals under the same conditions. Likewise, the off-line evaluation involves metrics to reflect the effectiveness of the system from the user perspective and provide a widely accepted evaluation, due to the robustness of the metrics used.
To evaluate our proposal, and to compare the results with our previous work [2], we performed an off-line evaluation and calculated the metrics used in it (Table 5).
RSs are evaluated in batches, with a set of data containing the profiles of both users and courses, as well as user preferences. These were taken from a survey conducted using a Google form, where it was requested that a list of courses be classified in the categories of desirable, preferred, novel and serendipitous, in order to ascertain the preferences of users regarding choice of LLL courses.
The off-line phase of the system was implemented, whereby data were loaded into the Neo4j database that had previously been populated by the ontology substrate definitions. This was updated via events using the training dataset, and clusters of job performance sectors were determined using ML, k-Means and DBSCAN algorithms.
In order to evaluate RS performance and compare it to our previous work, metrics were calculated for each of the following configurations [2]:
1: Content filtering using only the user’s own skills; no related skills are determined, although skills of interest are considered;
2: Collaborative filtering using only the user’s own work performance sectors; semantically related skills are determined;
3: Semantic filtering using rules to determine related job performance sectors and related skills;
4: Semantic filtering using 75% coverage of user skills to determine related job performance sectors. In other words, those positions that cover 75% of the user’s skills were selected as related job performance sectors;
5: Semantic filtering using 50% coverage of skills from the job performance sector to determine related job performance sectors. In other words, those job performance sectors in which the user’s skills cover 50% of the skills associated with the position;
6: Semantic filtering using DBSCAN clustering with ε = 0.3, to determine related job performance sectors and semantic rules to determine related skills;
7: Semantic filtering using k-Means clustering to determine related job performance sectors and semantic rules to determine related skills;
8: Semantic filtering using DBSCAN and k-Means clustering to determine related job performance sectors and semantic rules to determine related skills.
Each of the configurations described above were run through the different data sets, training, testing and total, obtaining the following results.
The test results are next shown along with the results obtained in the previous work [2], to compare the results and evaluate improvement in the RS.
The results, using the training data (70% of the data set) to validate the model performance, are shown in Table 6.
The results obtained from use of the different configurations with the total number of samples from the dataset are shown in Table 7.
Finally, the system was run under the different configurations, using the test portion (30%) of data, with the results shown in Table 8.
From the results obtained, it can be observed that both this proposal and the system proposed in [2] evidence similar behavior. As such, the configurations showing the best performance in the case of all data sets were semantic filtering using rules to determine related job performance sectors and related skills (3) and semantic filtering using DBSCAN clustering with ε = 0.3 to determine related job performance sectors, and semantic rules to determine related skills (6). However, there was an improvement in MAE and RMSE scores, and a slight increase in precision.
A measure that summarizes both precision and recall is the harmonic mean between them (F1). Table 9 shows the harmonic means for the best performing configurations.
From the harmonic mean comparison table, it can be seen in Figure 3, that there is a slight improvement in configuration 6, which makes use of the DBSCAN algorithm, while configuration 3, which corresponds to semantic filtering, remains unchanged.
From the different tests, the clustering performed by k-Means evidences an inferior performance, which could be explained by the nature of the domain and distribution of the positions in space. In general, in the graph analysis we observe better clustering (related positions) performed by DBSCAN than that performed by k-Means. Improvements in the recall and serendipity metrics can be associated with use in the ontology and the estimation of associated skills or skills of interest and filtering of positions prior to the current one in an orderly relationship imposed by a ladder.
In our review of different works, we found some similar RS proposals, and so to evaluate the performance of ours it is advisable to compare results. The work by [41] is geared to university students and company coordinators, recommends jobs according to user skills, while that by [21] is designed to recommend courses to students, using multiple data sources. On the other hand, the work by [42] recommends career paths and skills required for different jobs to users, based on their skills and interests. Finally, [35] recommends online courses to professionals according to their professional competences and professional development preferences. Table 10 provides a summary of the results obtained from each of these papers, which use the precision and recall metrics for the evaluation of their RSs, and the best performing configuration from our proposal.
When comparing these works with the best results obtained from our proposal, an improvement in metrics of all similar RS proposals can be seen, as can an increase in accuracy, recall and harmonic mean; the latter measures being the combined performance between accuracy and recall.

4. Discussion and Conclusions

In this paper we have presented a new version of the RS proposed by [2], taking the incorporation of endorsements of user LinkedIn skills to evaluate RS performance into consideration. In the review of literature on LinkedIn endorsements, we noted that very few papers analyze this element. These are more oriented to analyze the validity of endorsements, as an element to be taken into account in the data extracted from SNSs.
The strategy proposed for recommendation of LLL courses was based on establishing a relationship between users according to work performance sectors and professional skills in order to identify those skills that should be improved or developed for their current job, or to access another, higher level job and, based on these identified skills, to determine an initial prediction of courses that may develop them. This strategy made it possible to establish a mechanism for relating the data, which generally speaking, did not initially have relationships on which to base recommendations.
By incorporating the endorsements, it was possible to obtain more information on the user profile, which in turn made it possible to incorporate skills that were not evident in the data related to current employment, and which were useful when refining course recommendation. When evaluating the incorporation of endorsements in creating user profiles, an improvement in RS performance was observed. The configuration that makes use of the DBSCAN algorithm improves the precision value by 3%, and results in a decrease in the root mean square error (RMSE) and mean absolute error (MAE).
When we wanted to compare our proposal with similar works, we did not find RSs that made use of endorsements, therefore, we selected some similar proposals in terms of RSs objectives. One of the main differences focuses on the nature of the data as it was observed that most work was with structured data. The work by [42] obtained data directly from a university database, while [41] obtained them from a data repository used for ML experimentation, and [35] extracted them from employee profiles and the training plan directly from a company. The work by [21], on the other hand, proposed using information from LinkedIn to complement the data entered directly from users through the application or obtained from the university database. From these data a taxonomy was proposed to represent course, job and student information.
The metrics used by most works were found to be recall and accuracy when they were analyzed to compare results with this proposal. In terms of the scores obtained, we can conclude that if we compare them to the results obtained from the tests performed on the total set of data, this proposal improves on the metrics obtained by these previous works by two percentage points in terms of accuracy and 10 percentage points in recall.
Research on LinkedIn endorsements is oriented towards assessing the veracity of endorsements on LinkedIn profiles. [30] propose a framework for assessing the trustworthiness of endorsements. The work of [31] proposes to measure the trustworthiness of job candidates based on their skills and endorsements. Based on this research, one line of future work is to incorporate the validation of endorsements into the RS to determine their veracity when taking them into account for recommendations.
In order to evaluate system behavior for multiple domains, the ontology could be updated with new instances associated with new domains. In order to evaluate system behavior for multiple domains, the ontology could be acted on with new instances associated with new domains, and the use of ontologies already built could also be evaluated, together with new ways to represent areas of knowledge and job performance sectors.
With regard to information sources and given the continuous changes, the use of other SNSs should be evaluated, as well as different platforms such as, RocketReach and DataLead, from where professional profiles can be obtained.
Online applications could also be offered for course recommendations, not only for online evaluation of the system, but also to determine opportunities for improvement based on user suggestions.

Author Contributions

The contributions to this paper are as follows: research, software, validation, formal analysis, data curation writing—original draft preparation, M.C.U.-P.; supervision, A.M.-Z. and I.O.-R.; conceptualization, methodology, writing—review and editing, M.C.U.-P., A.M.-Z. and I.O.-R. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Data Availability Statement

The data presented in this study are available on request from the corresponding authors.


This document has the support of the Unesco Chair in Human Resources and the eVida group recognized by the Basque Government, at the University of Deusto, Bilbao-Spain.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Raitskaya, L.; Tikhonova, E. Skills and Competencies in Higher Education and Beyond. J. Lang. Educ. 2019, 5, 4–8. [Google Scholar] [CrossRef]
  2. Ponte, M.C.U.; Zorrilla, A.M.; Ruíz, I.O. Lifelong Learning Courses Recommendation System to Improve Professional Skills Using Ontology and Machine Learning. Appl. Sci. 2021, 11, 3839. [Google Scholar] [CrossRef]
  3. Cubrich, M.; King, R.; Mracek, D.; Strong, J.; Hassenkamp, K.; Vaughn, D.; Dudley, N. Examining the criterion-related validity evidence of LinkedIn profile elements in an applied sample. Comput. Hum. Behav. 2021, 120, 106742. [Google Scholar] [CrossRef]
  4. Constantinov, C.; Popescu, P.S.; Poteras, C.M.; Mocanu, M.L. Preliminary results of a curriculum adjuster based on professional network analysis. In Proceedings of the 19th International Conference on System Theory, Control and Computing (ICSTCC), Cheile Gradistei, Romania, 14–16 October 2015. [Google Scholar]
  5. Li, S.; Shi, B.; Yang, J.; Yan, J.; Wang, S.; Chen, F.; He, Q. Deep Job Understanding at LinkedIn. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA, 25–30 July 2020. [Google Scholar]
  6. Bakhshinategh, B.; Spanakis, G.; Zaïane, O.; ElAtia, S. A Course Recommender System based on Graduating Attributes. In Proceedings of the 9th International Conference on Computer Supported Education, Porto, Portugal, 21–23 April 2017. [Google Scholar]
  7. Aggarwal, C. Recommender Systems. The Textbook, 2016; Springer: New York, NY, USA, 2016. [Google Scholar]
  8. Suhaim, A.B.; Berri, J. Context-Aware Recommender Systems for Social Networks: Review, Challenges and Opportunities. IEEE Access 2021, 9, 57440–57463. [Google Scholar] [CrossRef]
  9. Shao, J. Research on Fuzzy Ontology E-learning Based on User Profile. In Proceedings of the International Conference on E-Education, E-Business and E-Technology (ICEBT 2017), Toronto, Canada, 10–12 September 2017. [Google Scholar]
  10. Koh, M.; Chew, Y. Intelligent Job Matching with Self-learning Recommendation Engine. Procedia Manuf. 2015, 3, 1959–1965. [Google Scholar] [CrossRef] [Green Version]
  11. Valentin, C.; Emrich, A.; Lahann, J.; Werth, D.; Loos, P. Adaptive Social Media Skills Trainer for Vocational Education and Training: Concept and Implementation of a Recommender System. In Proceedings of the 48th Hawaii International Conference on System Sciences, Kauai, HI, USA, 5–8 January 2015. [Google Scholar]
  12. Balachander, Y.; Moh, T. Ontology Based Similarity for Information Technology Skills. In Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), Barcelona, Spain, 28–31 August 2018. [Google Scholar]
  13. Gugnani, A.; Kasireddy, V.; Ponnalagu, K. Generating Unified Candidate Skill Graph for Career Path Recommendation. In Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), Singapore, 17–20 November 2018. [Google Scholar]
  14. Obeid, C.; Lahoud, I.; El Khoury, H.; Champin, P.-A. Ontology-Based Recommender System in Higher Education. In Proceedings of the International World Wide Web Conferences Steering Committee, Lyon, France, 23–27 April 2018. [Google Scholar]
  15. Anusha, U.; Biradar, S. Recommender Systems: A Survey. Int. J. Latest Technol. Eng. 2016, 5, 2278–2540. [Google Scholar]
  16. Karakaya, M.; Aytekin, T. Effective methods for increasing aggregate diversity in recommender systems. Knowl. Inf. Syst. 2018, 56, 355–372. [Google Scholar] [CrossRef]
  17. INti, I.K.; Quarcoo, J.A.; Aning, J.; Fosu, G.K. A mini-review of machine learning in big data analytics: Applications, challenges, and prospects. Big Data Min. Anal. 2022, 5, 81–97. [Google Scholar]
  18. Hosseini, M.P.; Hosseini, A.; Ahi, K. A Review on Machine Learning for EEG Signal Processing in Bioengineering. IEEE Rev. Biomed. Eng. 2021, 14, 204–218. [Google Scholar] [CrossRef] [PubMed]
  19. Urdaneta Ponte, M.C.; Méndez Zorrilla, A.; Oleagordia Ruíz, I. Recommendation Systems for Education: Systematic Review. Electronics 2021, 10, 1611. [Google Scholar] [CrossRef]
  20. Zheng, X.; Chen, C.; Hung, J.; He, W.; Hong, F.; Lin, Z. A Hybrid Trust-Based Recommender System for Online Communities of Practice. In Proceedings of the 2015 IEEE 15th International Conference on Advanced Learning Technologies, Hualien, Taiwan, 6–9 July 2015. [Google Scholar]
  21. Ibrahim, M.; Yang, Y.; Ndzi, D.; Yang, G.; Almaliki, M. Ontology-based personalised course recommendation framework. IEEE Access 2018, 7, 5180–5199. [Google Scholar] [CrossRef]
  22. Huang, Z.; Liu, Q.; Yin, Y.; Chen, E.; Gao, W.; Hu, G. Exploring Multi-Objective Exercise Recommendations in Online Education. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM ’19), Beijing, Chine, 3–7 November 2019. [Google Scholar]
  23. Jiang, W.; Pardos, Z. Time slice imputation for personalized goal-based recommendation in higher education. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys’19), New York, NY, USA, 16–20 September 2019. [Google Scholar]
  24. Samin, H.; Azim, T. Knowledge Based Recommender System for Academia Using Machine Learning: A Case Study on Higher Education Landscape of Pakistan. IEEE Access 2019, 7, 67081–67093. [Google Scholar] [CrossRef]
  25. Liu, H.; Ding, J.; Yang, L.; Guo, Y.; Wang, X.; Deng, A. Multi-Dimensional Correlative Recommendation and Adaptive Clustering via Incremental Tensor Decomposition for Sustainable Smart Education. IEEE Trans. Sustain. Comput. 2020, 5, 389–402. [Google Scholar] [CrossRef]
  26. Akmaliah, F.; Krisnadhi, A.; Sensuse, D.; Rahayu, P.; Wulandari, I. Role of Ontology and Machine Learning in Recommender Systems. In Proceedings of the 2018 Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS), Batu, Indonesia, 9–11 October 2018. [Google Scholar]
  27. Yan, X.; Jaewon, Y.; Obukhov, M.; Zhu, L.; Bai, J.; Wu, S.; He, Q. Social Skill Validation at LinkedIn. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, New York, NY, USA, 4–8 August 2019. [Google Scholar]
  28. Rapanta, C.; Cantoni, L. The LinkedIn Endorsement Game: Why and How Professionals Attribute Skills to Others. In Business and Professional Communication Quarterly; SAGE Publications: Thousand Oaks, CA, US, 2017. [Google Scholar]
  29. Pérez-Rosés, H.; Sebé, F. Iterated endorsement deduction and ranking. Computing 2017, 99, 431–446. [Google Scholar] [CrossRef]
  30. Wu, Y.; Dhakal, N.; Xu, D.; Cho, J.-H. Analysis and Prediction of Endorsement-Based Skill Assessment in LinkedIn. In Proceedings of the 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Tokyo, Japan, 23–27 July 2018. [Google Scholar]
  31. Drakopoulos, G.; Kafeza, E.; Mylonas, P.; Katheeri, H. Building Trusted Startup Teams from LinkedIn Attributes: A Higher Order Probabilistic Analysis. In Proceedings of the 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI), Baltimore, MD, USA, 9–11 November 2020. [Google Scholar]
  32. Dissanayake, K.; Mendis, S.; Subasinghe, R.; Geethanjana, D.; Lunugalage, D.; Kasthurirathna, D. Career Aura—Smart Resume and Employment Recommender. In Proceedings of the 3rd International Conference on Advancements in Computing (ICAC), Colombo, Sri Lanka, 9–11 December 2021. [Google Scholar]
  33. Bañeres, D.; Conesa, J. eOrient@—A Recommender System to Address Life-Long Learning and Promote Employability. In Proceedings of the International Conference on Intelligent Networking and Collaborative Systems (INCoS), Ostrava, República Checa, 7–9 September 2016. [Google Scholar]
  34. Tavakoli, M.; Mol, S.; Kismihók, G. Labour Market Information Driven; Personalized; OER Recommendation System for Lifelong Learners. In Proceedings of the 12th International Conference on Computer Supported Education, Online, 2–4 May 2020. [Google Scholar]
  35. Wang, C.; Zhu, H.; Zhu, C.; Zhang, X.; Chen, E.; Xiong, H. Personalized Employee Training Course Recommendation with Career Development Awareness. In Proceedings of the The Web Conference 2020, Taipei, Taiwan, 20–24 April 2020. [Google Scholar]
  36. He, Q.; Yang, J.; Shi, B. Constructing Knowledge Graph for Social Networks in A Deep and Holistic Way. In Proceedings of the Companion Proceedings of the Web Conference 2020, New York, NY, USA, 20–24 April 2020; Association for Computing Machinery: New York, NY, USA, 2020. [Google Scholar]
  37. López-Carril, S.; Anagnostopoulos, C.; Parganas, P. Social media in sport management education: Introducing LinkedIn. J. Hosp. Leis. Sport Tour. Educ. 2020, 27, 100262. [Google Scholar] [CrossRef] [PubMed]
  38. Bedjou, K.; Azouaou, F.; Berkani, L. Semantic recommendation of web services in the context of on-line training. In Proceedings of the International Symposium ISKO-Maghreb: Concepts and Tools for knowledge Management (ISKO-Maghreb), Algiers, Algeria, 9–10 November 2014. [Google Scholar]
  39. Xu, H.; Yao, S.; Li, Q.; Ye, Z. An Improved K-means Clustering Algorithm. In Proceedings of the 2020 IEEE 5th International Symposium on Smart and Wireless Systems within the Conferences on Intelligent Data Acquisition and Advanced Computing Systems (IDAACS-SWS), Dortmund, Germany, 17–18 September 2020; pp. 1–5. [Google Scholar]
  40. Zhang, M. Use Density-Based Spatial Clustering of Applications with Noise (DBSCAN) Algorithm to Identify Galaxy Cluster Members; IOP Publishing: Bristol, UK, 2019. [Google Scholar]
  41. Kolekar, A. SKYNET A platform for maximizing career opportunities. In Proceedings of the International Conference on Nascent Technologies in Engineering (ICNTE 2021), NaviMumbai, India, 15–16 January 2021. [Google Scholar]
  42. Patel, B.; Kakuste, V.; Eirinaki, M. CaPaR: A Career Path Recommendation Framework. In Proceedings of the 2017 IEEE Third International Conference on Big Data Computing Service and Applications (BigDataService), San Francisco, CA, USA, 6–9 April 2017. [Google Scholar]
Figure 1. Hybrid RS architecture based on ontology and ML with endorsements.
Figure 1. Hybrid RS architecture based on ontology and ML with endorsements.
Electronics 11 01190 g001
Figure 2. General recommendation algorithm.
Figure 2. General recommendation algorithm.
Electronics 11 01190 g002
Figure 3. Harmonic mean (F1) for configurations with the best RS performance.
Figure 3. Harmonic mean (F1) for configurations with the best RS performance.
Electronics 11 01190 g003
Table 1. Use of ML in recommender systems.
Table 1. Use of ML in recommender systems.
Ref.YearObjectiveFiltering TechniqueML TechniqueMetrics
[20]2015Recommend LLL courses according to common professional interestsCollaborative + MLML supervisedAccuracy/Recall
[21]2018Recommend courses according to academic profile and jobs on offerCollaborative + Content + MLML supervisedAccuracy/Recovery
[22]2019Recommend exercises based on information regarding academic performanceMLML non-supervisedNDCG **, MAE * and F1 ***
[23]2019Recommend courses to maximize learning based on past performanceMLML by way of supportAccuracy/F1
[24]2019Recommend consultancy according to academic-industrial research interestsMLML supervisedNDCG
[25]2020Recommend learning resources according to context, taking colleagues’ learning into accountMLML non-supervisedAccuracy/Recall/F1
MAE *: Mean absolute error; NDCG **: Normalized Discounted Cumulative Gain; F1 ***: Harmonic mean between precision and recall.
Table 2. Research related to the use of LinkedIn endorsements.
Table 2. Research related to the use of LinkedIn endorsements.
Ref.YearObjectiveUse of EndorsementsRS
[4]2015Determine a set of competences required by the market, in which more focus should be placed on the curriculum.To determine the weight of a skill (beginner, intermediate, advanced).YES
[28]2016Identify the motivation behind behavior in the use of LinkedIn.To analyze the reliability of skills endorsements. NO
[29]2017Determine new skills endorsements.To deduce new endorsements.NO
[30]2018Propose an evaluation framework for evaluation methods of competences via regression.To analyze endorsements, to determine which ones are false.NO
[27]2019Estimate the experience gained by LinkedIn members and objectively identify skills.To measure the level of skills.NO
[31]2020Measure reliability of a candidate for LinkedIn profile startups.To determine a user’s skills.NO
[32]2021Job recommendation.To determine false endorsements.YES
Table 3. Classes of ontology.
Table 3. Classes of ontology.
Class Description Type
AreaGeneral category, into which an area of knowledge is divided, e.g.,: ComputingEntity
Sub-areaDifferent general categories, which contain a sub-area of knowledge, e.g.,: LanguagesSubstrate
SpecializationDifferent branches of specific knowledge, which contain a sub-area of knowledge, e.g., Programming LanguagesSubstrate
Sub-specializationDifferent categories, which contain a specialization. Groups together common knowledge of different specific areas of knowledge, e.g., Object-oriented Programming LanguagesSubstrate
KnowledgeSpecific knowledge belonging to a sub-specialization. Corresponds to the most specific level (leaves) in the area of knowledge hierarchy. May denote the domain of technologies, tools, software, etc., e.g., Java Programming LanguageSubstrate
SectorGeneral category, into which labor domain is divided, e.g., Information and Communication TechnologySubstrate
FieldDifferent general categories, which contain a labor sector, e.g., Software DevelopmentSubstrate
TypeNature of the different positions contained in field, e.g., DeveloperSubstrate
DomainSpecific areas contained in a type of position, e.g., Development of Mobile AppsSubstrate
PositionSpecific position in a position domain: Corresponds to the most specific level (leaves) in the job performance sector hierarchy, e.g., Development of iOS Native Mobile AppsSubstrate
RequirementSpecific requirements of a position. There may be different types (desirable, mandatory). Related to age, experience, and academic background, among othersSubstrate
SkillLevel of experience and degree of update associated with an area of knowledgeUpdated via event
ClusterGroup of similar positionsML
Table 4. Ontology relationships.
Table 4. Ontology relationships.
Relationship Description Cardinality Classes
Is a/anBelongs to a category1:1Between underlying classes belonging to a hierarchy
Ia the sameDescriptions are synonymous 1:1Between the same classes belonging to a hierarchy
Is of interestKnowledge is of interest to other areas of knowledge1:NBetween areas of knowledge
InvolvesA skill involves or refers to a specific area of knowledge1:1Skill ⟶ Knowledge
PossessesA position makes use of a set of skills
A cluster possesses a set of skills such as Centroide
1:NPosition ⟶ Skill
Cluster ⟶ Skill
IncludesA cluster includes a set of positions 1:NCluster ⟶ Position
HasA position has a set of requirements 1:NPosition ⟶ Requirements
Table 5. Evaluation metrics for recommendation systems.
Table 5. Evaluation metrics for recommendation systems.
Metrics Description Calculation for the System
Coverage Users   to   whom   the   system   has   made   a   recommendation ,   C R i   set   of   recommendations   from   user   u i U C S = u i U , C R i U
PrecisionThe fraction of the recommendation that is relevant to the user. P S = 1 U u i U P u i
MAE Mean   absolute   error   of   the   given   recommendation   vs .   the   recommendation   expected   by   the   user ,   C E i M A E = i = 1 n | ( C E i C R i ) C E i C R i | i = 1 n | C E i C R i |
RMSERoot mean square error R M S E = i = 1 n | ( C E i C R i ) ( C E i C R i ) | 2 i = 1 n | C E i C R i |
RecallCalculates the ratio between recommendation and user preference R S = 1 U u i U R u i
NoveltyThe portion of recommendations made to the user that the user is not familiar with or has not seen before N S = 1 U u i U N u i
SerendipityThe fraction of the recommendation that is unexpected and valuable to the user. S S = 1 U u i U S u i
F1Harmonic mean between the precision and recall F 1 = 2 P r e c i s i o n R e c a l l P r e c i s i o n + R e c a l l
Table 6. Metrics for the different RS configurations used to train data.
Table 6. Metrics for the different RS configurations used to train data.
Author’s Proposal0.589.130.641.000.360.140.09
Author’s Proposal0.518.170.730.970.410.210.07
Author’s Proposal0.244.140.910.850.820.460.07
Author’s Proposal0.213.380.820.950.710.450.07
Author’s Proposal0.366.350.910.830.740.390.07
Author’s Proposal0.162.580.910.940.800.520.07
Author’s Proposal0.489.050.910.690.630.310.04
Author’s Proposal0.284.950.950.820.800.480.04
Table 7. Metrics for the different RS for the whole sample.
Table 7. Metrics for the different RS for the whole sample.
Setting Ref. MAE RMSE Coverage Precision Recall Novelty Serendipity
Author’s Proposal0.5911.050.610.990.370.110.06
Author’s Proposal0.509.450.710.970.450.170.05
Author’s Proposal0.234.820.940.840.870.390.06
Author’s Proposal0.224.260.810.960.730.390.05
Author’s Proposal0.336.780.870.860.750.340.06
Author’s Proposal0.255.050.940.860.860.450.05
Author’s Proposal0.5011.360.940.650.650.250.04
Author’s Proposal0.337.260.970.770.860.420.03
Table 8. Metrics for the different configurations for the set of test data.
Table 8. Metrics for the different configurations for the set of test data.
Setting Ref. MAE RMSE Coverage Precision Recall Novelty Serendipity
Author’s Proposal0.626.240.560.960.410.040.00
Author’s Proposal0.474.750.670.960.560.070.00
Author’s Proposal0.222.481.000.801.000.240.04
Author’s Proposal0.262.630.780.960.780.260.00
Author’s Proposal0.252.600.780.940.780.210.02
Author’s Proposal0.404.921.000.681.000.280.00
Author’s Proposal0.556.901.000.540.690.120.06
Author’s Proposal0.435.541.000.631.000.280.00
Table 9. Harmonic mean (F1) for configurations with the best RS performance.
Table 9. Harmonic mean (F1) for configurations with the best RS performance.
Configuration Ref. F1
Training Data
Total Data
Test Data
3 [2]0.820.840.89
Author’s Proposal0.830.840.89
6 [2]0.850.850.80
Author’s Proposal0.860.850.81
Table 10. Comparison of evaluation metrics in similar work.
Table 10. Comparison of evaluation metrics in similar work.
RSs Related Precision Recall F1
Author’s Proposal 0.940.800.85
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Urdaneta-Ponte, M.C.; Oleagordia-Ruíz, I.; Méndez-Zorrilla, A. Using LinkedIn Endorsements to Reinforce an Ontology and Machine Learning-Based Recommender System to Improve Professional Skills. Electronics 2022, 11, 1190.

AMA Style

Urdaneta-Ponte MC, Oleagordia-Ruíz I, Méndez-Zorrilla A. Using LinkedIn Endorsements to Reinforce an Ontology and Machine Learning-Based Recommender System to Improve Professional Skills. Electronics. 2022; 11(8):1190.

Chicago/Turabian Style

Urdaneta-Ponte, María Cora, Ibon Oleagordia-Ruíz, and Amaia Méndez-Zorrilla. 2022. "Using LinkedIn Endorsements to Reinforce an Ontology and Machine Learning-Based Recommender System to Improve Professional Skills" Electronics 11, no. 8: 1190.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop