Service Discovery Method Based on Knowledge Graph and Word2vec

Round 1

Reviewer 1 Report

This paper described a service discovery system (named SDKG) that employed the knowledge graph to recommend service based on semantic information.

Pros

1. The idea was well-explained and the logic flow was easy to follow. 

2. The application has real scenario needs that are interesting to end-users.

3. The precision and recall of the system showed significant advantages compared to baseline approaches.

Cons

• The knowledge graph building was the key of the system. However, there were not many details about how it was constructed. If it had been built purely manually, there would be concern about the extensibility of the work.

• It was not clear how the matching template was built and how effective each template was.

• The test data set was not from a third-party, which might be biased to the selected systems and making the final result less convincing. 

• Lack of detailed analysis of the result. E.g., there were no good or bad examples in the paper. Without detailed analysis, it was difficult to evaluate the effectiveness of each component of the system. Furthermore, it would be valuable to figure out the future work that was not addressed in the paper.

Specific comments

• There were several left over comments in the PDF that should not be a final submission version.

• The important link doesn’t exist: https://github.com/zhoujunkai/Service_Axioms that makes the results non-reproducible.

• The “Cyber query” was used without description.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

1) Knowledge graph is currently a good method to represent linked knowledge/information, especially for the data base having heterogeneous data. It means, however, for most of the cases, the SQL might be more efficient for regular databases. Could you please explain why KG is more suitable than SQL in your special case?

2) The knowledge graph was contructed by using Chyper/Neo4j. It is actually still a munual process to convert the information to triples. In another word, is it means that the conversion of the information may take a lot of time? e.g., heterogeneous data may require different Chyper statement?

3) How would you maintain and further enrich the knowledge graph?

4) Text of FIgure 3 is not that clear, some annotations might be needed.

5) Chyper, SPARQL, etc. are not simple query languages, it might cause difficulties for people to learn and use. Any idea regarding how to convert those queries to questions in natural languages for people to use?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The article entitled “Service Discovery Method Based on Knowledge Graph and Word2vec” deals with an interesting and timely problematic, the service discovery and matching. Authors used knowledge graph to create a recommendation system. 

Overall, the document is well written and easy to read however, there are some sections in orange (without any apparent reason) that are out of place. Also, several variables and acronyms are not introduced or are introduced only after used several times. 

The biggest flaw of the work is the empirical validation and the references. These are old, and some are incomplete. Apart from six references, all others are prior to 2014, with several being conference works. This outdated perspective is also present in the Empirical validation. The developed technique is compared with the results obtained in [5], [25,26], and [27] from 2007, 2009, and 2003, respectively. In this section, there are also various repetitions in the text and the information presented in the tables and charts. 

With this outdated comparison, the authors fail to provide proof that the proposed solution improves in the current state of the art. An update of the performed comparison will improve the validity and interest in the proposed solution. Therefore, I encourage the authors to improve section 4 and resubmit the work. In its present form, I do not recommend this work for publication. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

The manuscript entitled "Service Discovery Method Based on Knowledge Graph and Word2vec" authors propose a service discovery approach based on a knowledge map (SDKG). They embed service-related information into the knowledge graph, alleviating the impact of data sparsity and mining deep relationships between services, improving the accuracy of service discovery.

The manuscript is well-written, and well-justified and it should be reproduced thanks to the information exposed by the authors.

Congrats for your research and for this manuscript.

I only have some minor suggestions:

Minor changes:

General:

- Authors should unify some words in lowercase: Web Services => web services.

- It is important to show unabbreviated words the first time they appear in the text.

- Github URL is broken.

Abstract:

- Some words should be in lowercase form and others : 

line 4 - "for Mashup developers" => "for mashup developers"

line 5 - "Existing methods" => "existing methods"

line 10 - "accu-racy" => "accuracy"

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

n/a

Author Response

Reviewer 3 Report

The authors provided some improvements from the previous version. However, some issues remain. Such as:
Improvements in the edition:
L213 – why a new paragraph?
L270 – why a new paragraph?
L283 - “based on the relationship between entities and entities.” this sentence is confusing, please revise;
L322 – why a new paragraph?
L326 – rewrite the sentence between (1) and (2) to prevent this “lost line”
 
Terms not presented: LSI, WSD, VSM, FBWSD, SRMWSD-LDA

Figure 2, API3 does not have a tag, is this intentional? Please elaborate.

Table 1, In the text below (line 232 onwards) are referenced: service name, service label, service category, and service function. However, these “names” are not present in the table itself. Also, the query word type labels create some confusion, please revise.

Table 1 has several repeated cells, please revise it to avoid confusion

Figure 4 is unnecessary.

The SDKG network could be present in more detail, please provide a link for easy access, GITHUB, or any other.

Figure 5 font is too small, thus difficult to read

Table 3 and Figures 6, 7, and 8 present the same data, please choose, a table or figures. The Y-axis values should be integers if choosing figures 6, 7, and 8.

The comparison is made with techniques that are considerably old. Please update.

Table 4 are the ranking values correct? Why present them with one decimal point?

Tables 5 and 6, These two tables are very similar, please merge them. Are the p values the same for both cases?

L431 – authors state that there is a significant difference between the methods however, the p-value for FBWSD vs SDKG is 0.2733. This indicates resemblance, contrary to the said. Please revise, not omitting this result.

Concerning the literature review, below is seven examples of works done in the last three years. Please significantly update the literature review:
Ko, H.; Lee, S.; Park, Y.; Choi, A. A Survey of Recommendation Systems: Recommendation Models, Techniques, and Application Fields. Electronics 2022, 11, 141. https://doi.org/10.3390/electronics11010141
Wang, X., Liu, X., Liu, J. et al. A novel knowledge graph embedding based API recommendation method for Mashup development. World Wide Web 24, 869–894 (2021). https://doi.org/10.1007/s11280-021-00894-3
Botangen, K. A., Yu, J., Sheng, Q. Z., Han, Y., Yongchareon, S.: Geographic-aware collaborative filtering for Web service recommendation. Exp. Sys. App. 151, 113347 (2020)
Zhang, Y., Yin, C., Wu, Q., He, Q., Zhu, H.: Location-aware deep collaborative filtering for service recommendation. IEEE Trans. Sys. Man Cybern., 1–12 (2020)
C. Chen et al., "Holistic Combination of Structural and Textual Code Information for Context based API Recommendation," in IEEE Transactions on Software Engineering, doi: 10.1109/TSE.2021.3074309.
NuriAlmarimi, “Web service API recommendation for automated mashup creation using multi-objective evolutionary search,” in Applied Soft Computing, doi:10.1016/j.asoc.2019.105830,  https://doi.org/10.1016/j.asoc.2019.105830
Duan, L.; Tian, H.; Liu, K. A Novel Approach for Web Service Recommendation Based on Advanced Trust Relationships. Information 2019, 10, 233. https://doi.org/10.3390/info10070233

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 3 Report

The previous issues pointed to the authors were addressed. Therefore I recommend the work for publication.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Some kind of cross validation should be used for the experiments.

A statistical test should be used for the comparison of the examined methods.

The authors should better explain why the proposed methodology seems to work well and present some information about

the time efficiency of their method.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Manuscript summary

In this manuscript, the authors present a service discovery approach based on a knowledge graph. The results presented show a significant increase in accuracy compared to benchmark approaches.

Review summary

The study uses a well-known methodology, but I find it lacks scientific soundness in terms of motivating the study and using the right methodology.

In particular, I cannot find a rationale for using the original word2vec algorithm on the graph here. I wonder why the authors do not use knowledge graph embeddings (e.g., the TransX family) directly?

Moreover, and most importantly, the methods and results of the study cannot be replicated because the authors do not publish source code. I am afraid I can not properly understand their contribution (including accuracy) without the associated programming code.

Concerns/Weaknesses

1. Authors need to provide programming code to ensure reproducibility of their results.

2 Rewrite the motivation paragraph in the introduction and provide an illustrative example.

3. Explain the rationale for word2vec; as I said above, I will go for KG embeddings here.

4. A more thorough proofreading is needed.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The paper could be accepted in the current form

Reviewer 2 Report

The authors have improved the manuscript.