Ontology-Based Personalized Job Recommendation Framework for Migrants and Refugees

Round 1

Reviewer 1 Report

The paper presents a job recommendation framework based on the IMMERSE platform. The framework was first built on ontologies of expectations, languages, educational background, job experiences, and skills. Then, the framework provides recommendations based on job matching similarity based on user locations. Although this paper does not propose novel technical improvements from the current literature on this domain, the author adopted the existing methods to claim new improvements for the relevant application. This paper can be improved by comparing performance with other baselines or existing methods for the same tasks and extending the literature review.

Some minor problems in the paper need to be fixed:

A. One of my main concerns about this paper is the effectiveness of the proposed method. The framework has been discussed in good detail in sections 3-6. However, the experimental results section is quite short. I would like to see the performance comparison with a baseline method or relevant existing methods (machine learning or deep learning recommendation systems for job matching).

B. The literature review covered quite a detail of ontology and knowledge-based recommendation systems for job recruitment. Please extend the literature review to other existing methods for recommendation systems (using machine learning or deep learning methods of content-based and collaboration filtering techniques). Please look at the following related works and others:

- Kenthapadi, K., Le, B., & Venkataraman, G. (2017, August). Personalized job recommendation system at linkedin: Practical challenges and lessons learned. In Proceedings of the eleventh ACM conference on recommender systems (pp. 346-347).

- Musale, D. V., Nagpure, M. K., Patil, K. S., & Sayyed, R. F. (2016). Job recommendation system using profile matching and web-crawling. International Journal, 1(2).

- Yang, S., Korayem, M., AlJadda, K., Grainger, T., & Natarajan, S. (2017). Combining content-based and collaborative filtering for job recommendation system: A cost-sensitive Statistical Relational Learning approach. Knowledge-Based Systems, 136, 37-45.

- Koh, M. F., & Chew, Y. C. (2015). Intelligent job matching with self-learning recommendation engine. Procedia Manufacturing, 3, 1959-1965.

- Rodriguez, L. G., & Chavez, E. P. (2019, February). Feature selection for job matching application using profile matching model. In 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS) (pp. 263-266). IEEE.

- Drigas, A., Kouremenos, S., Vrettos, S., Vrettaros, J., & Kouremenos, D. (2004). An expert system for job matching of the unemployed. Expert Systems with Applications, 26(2), 217-224.

- Malinowski, J., Keim, T., Wendt, O., & Weitzel, T. (2006, January). Matching people and jobs: A bilateral recommendation approach. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06) (Vol. 6, pp. 137c-137c). IEEE.

- Mine, T., Kakuta, T., & Ono, A. (2013, August). Reciprocal recommendation for job matching with bidirectional feedback. In 2013 Second IIAI International Conference on Advanced Applied Informatics (pp. 39-44). IEEE.

C. You need to cite the references for the following terms/phrases: IMMERSE, OntoMetrics (line 209), GraphD (line 243), RDF4J (line 247), and SPARQL (line 246).

D. Multiple acronyms/terms, such as IMMERSE (abstract line 3), ICT (lines 21), W3C (line 66), DL (line 26), SI(D) (line 217), and BN (Figure 7) need to be spelled in full the first time you used them, and the related reference sources need to be cited.

E. The quality of the tables and figures in this paper need to be improved.

- Figures 1, 2, 3, 4, 5, 6, and 17 can be enlarged to take the full width of the page. Currently, the text font inside those figures is quite small.

- Some figures, such as Figures 2, 6, and 7, are skewed

F. Some places in this paper need to be improved in written English or formats.

Please check the comments in the attached PDF.

Comments for author File: Comments.pdf

Author Response

Response to Reviewer 1 Comments

Point 1: The framework has been discussed in good detail in sections 3-6. 

Response 1: We would like to thank you for the positive feedback.

Point 2: However, the experimental results section is quite short. I would like to see the performance comparison with a baseline method or relevant existing methods (machine learning or deep learning recommendation systems for job matching).

Response 2: Performance comparison is not feasible since every method uses its own dataset that is not available or possible to recreate. In addition, each method depends on some implementation that is not available, and it was not in scope for the particular work to implement it for the sake of performance comparison. For that reason, we have qualitatively compared our framework to other similar solutions in section 2.  All our amendments in section 2 are highlighted.

Point 3: The literature review covered quite a detail of ontology and knowledge-based recommendation systems for job recruitment. Please extend the literature review to other existing methods for recommendation systems (using machine learning or deep learning methods of content-based and collaboration filtering techniques). Please look at the following related works and others.

Response 3: We would like to thank you for providing the list of related works. Although such works approach the problem from a different perspective (data-driven approaches), we included a relevant subsection in section 2 mentioning the key concepts of these approaches (lines 164-181). 

Point 4: You need to cite the references for the following terms/phrases: IMMERSE, OntoMetrics (line 209), GraphDB (line 243), RDF4J (line 247), and SPARQL (line 246).

Response 4: Missing citations were added for all terms/phrases suggested in the comment. In particular the following citations were added:

• IMMERSE [7]
• OntoMetrics [29]
• GraphDB [33]
• RDF4J [34]
• SPARQL [19]

Point 5: Multiple acronyms/terms, such as IMMERSE (abstract line 3), ICT (lines 21), W3C (line 66), DL (line 26), SI(D) (line 217), and BN (Figure 7) need to be spelled in full the first time you used them, and the related reference sources need to be cited.

Response 5: The full names of the suggested acronyms/terms were added in text and relevant citations were added. In particular the full names of each terms can be found in the following lines:

• IMMERSE : Line 3
• ICT : Line 22
• W3C : Line 78
• DL : Line 261
• SI(D) : Each letter describes an operation, not necessarily starting from this letter, thus each letter was explained in text. Check Lines 264-269
• BN : This term was not found in Figure 7

Point 6: The quality of the tables and figures in this paper need to be improved. Figures 1, 2, 3, 4, 5, 6, and 17 can be enlarged to take the full width of the page. Currently, the text font inside those figures is quite small. Some figures, such as Figures 2, 6, and 7, are skewed.

Response 6: The suggested Figures were enlarged. In particular Figures 1,2,3,4,5,6 was enlarged (and centered) to fit in the column’s width while Figure 17 was enlarged to take the full width of the page.

Point 7: Some places in this paper need to be improved in written English or formats.

Response 7: Thank you very much for sharing the manuscript with your inline comments and corrections. We have considered your changes and also revisited the whole document and fixed vocabulary and grammar issues. All our amendments in text are highlighted.

Reviewer 2 Report

This paper describes an ontology-based reasoning framework of a platform namely IMMERSE, which acts as a matching tool that enables the contexts of individual refugees and migrants, including their languages, educational background, job experience and skills, so as to match with job opportunities available in their host country.

Although there is added value in this work, overall, I found that the authors have recently published a related work (IMMERSE: A Matching Platform Improving Migrant Integration with Semantic Technologies, Information and Communications Technology in Support of Migration. Security Informatics and Law Enforcement. Springer, Cham. https://doi.org/10.1007/978-3-030-93266-4_13), which is not included in the Background and Related Work section, not even in the reference list.

As a result, I cannot evaluate the added value of this particular work.

Although I was thinking of rejecting the paper due to plagiarism reasons, I give the authors a 2^nd chance and suggest to thoroughly rewrite the paper, with explicit reference to their previous work(s), as well the added value of this paper.

Furthermore, the reference list is very poor. There are paragraphs and even sections (e.g. the Introduction section) without a single reference, supporting the text (the content of the Introduction section is not a general truth that does not need references).

Author Response

Response to Reviewer 2 Comments

Point 1: Although there is added value in this work, overall.

Response 1: We would like to thank you for the positive feedback.

Point 2: I found that the authors have recently published a related work (IMMERSE: A Matching Platform Improving Migrant Integration with Semantic Technologies, Information and Communications Technology in Support of Migration. Security Informatics and Law Enforcement. Springer, Cham. https://doi.org/10.1007/978-3-030-93266-4_13), which is not included in the Background and Related Work section, not even in the reference list. 

Response 2: Thank you for pointing out the missing work. We have updated the introduction and actually included two related works. In particular in line 33-39 we present an overview of the work you have suggested in your comment (i.e., reference [7]). In addition, in lines 39-40 we provide an additional citation that describes a preliminary version of the recommendation framework. (i.e. reference [8])

Point 3: Although I was thinking of rejecting the paper due to plagiarism reasons, I give the authors a 2nd chance and suggest to thoroughly rewrite the paper, with explicit reference to their previous work(s), as well the added value of this paper.

Response 3: We have modified the introduction (lines 28-49) to describe the contribution of this study in greater depth, in comparison to earlier ones. More precisely, we included two new references. One refers to a prior work in which we provided the architecture of the platform on which our framework is based, and another to a previous work in which we showed a very basic version of the matchmaker. Much progress has been achieved from the prototype version. First, we finished and updated the ontology to include information about a user's skills, job interests, and interactions with existing postings, and we expanded the matching service with new rules that consider the aforementioned ontology improvements.

Point 4: Furthermore, the reference list is very poor. There are paragraphs and even sections (e.g., the Introduction section) without a single reference, supporting the text (the content of the Introduction section is not a general truth that does not need references).

Response 4: The reference list was significantly extended (19 new references were added). In particular, the introduction was updated with 8 new references [1-8] and the related work with 5 more references [24-28] about existing works that approach the problem of job matching from a different perspective using other techniques (e.g., machine learning etc.). In addition, several references were added in the following lines:

• Ref [11] concerning W3C term (Line 78)
• Ref [7] was reused in Figure 1
• Ref [29] concerning OntoMetrics term (Line 254)
• Ref [30] concerning Description Logic and SI(D) expressivity terms (Line 264)
• Ref [33] concerning GraphDB term (Line 292)
• Ref [34] concerning RDF4J term (Line 296)
• Ref [33] concerning OWL 2 RL term (Line 281)

Round 2

Reviewer 2 Report

I would like to thank the authors for the corrections. 

In my opinion, the current version of the paper is satisfactory.