Review Reports - Explainable B2B Recommender System for Potential Customer Prediction Using KGAT

Round 1

Reviewer 1 Report

Per my understanding, this paper mentions two new perspectives on how recommender systems can be utilized in B2B: (1) Recommending buyers to sellers, and (2) Explainability must be guaranteed. The paper proposes an explainable B2B recommender system using KGAT that assists supplier companies to predict potential customers. The paper also highlights the importance of recommendation in B2B e-commerce and suggests the application of an explainable recommender system in B2B for recommending potential customers. Below shows the reviewer’s major concerns.

1. The authors are recommended to revise the manuscript and to answer the question better that what is KGAT and how does it contribute to potential customer prediction in B2B?

2. This paper does not explicitly mention any limitations of the system. However, it is worth noting that the effectiveness of the system may depend on the quality and completeness of the data used to construct the knowledge graph. Additionally, the system may require significant computational resources to train and deploy, which could be a challenge for some organizations.

3. This system may not be suitable for all B2B contexts, as the effectiveness of the system may depend on the specific characteristics of the industry or market being analyzed. How would the authors answer this question in the summary?

4. This system may require significant domain expertise to construct the knowledge graph and interpret the results, which could be a barrier to adoption for some organizations. Can the authors answer this question in their conclusion?

A moderate proofreading is recommended for this work.

Author Response

Reviewer #1

The authors are recommended to revise the manuscript and to answer the question better that what is KGAT and how does it contribute to potential customer prediction in B2B?

In section 2.2, we highlighted the unique advantages of KGAT compared to other models when applied within the B2B.

This paper does not explicitly mention any limitations of the system. However, it is worth noting that the effectiveness of the system may depend on the quality and completeness of the data used to construct the knowledge graph. Additionally, the system may require significant computational resources to train and deploy, which could be a challenge for some organizations.

Taking the reviewer's comments into consideration, we incorporated future work into Section 5, "Conclusion and Future Work," where we explicitly address both the limitations and potential enhancements for KGAT.

This system may not be suitable for all B2B contexts, as the effectiveness of the system may depend on the specific characteristics of the industry or market being analyzed. How would the authors answer this question in the summary?

The paper presented in this study provides an overall analysis of the B2B market in South Korea, but it may yield different results in other environments. This research did not differentiate between specific industry markets. Each company can engage in transactions across multiple industry markets, and we believe this could offer additional insights for recommendation from various sectors and transaction data. This study was presented from a conceptual perspective, and we consider the impact of B2B recommendation systems on industries to be a task that should be researched in the future.

This system may require significant domain expertise to construct the knowledge graph and interpret the results, which could be a barrier to adoption for some organizations. Can the authors answer this question in their conclusion?

This research demands domain expertise in both KGAT and B2B. Additionally, without B2B transaction data, the research would be infeasible. However, the fact that deep neural networks can, with minimal human effort, comprehend the network relationships among numerous companies and recommend potential trading partners, signifies that there are numerous challenges within the B2B research domain that need to be addressed using AI. We believe that AI experts, through further research in the B2B domain, can reduce the adoption barriers for technology, and this paper is a part of that ongoing process. Thank you for your comments.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper builds an explainable B2B recommender system. This is a very interesting topic and is rarely discussed. My concerns are as follows,

1) In the related work section, I cannot find which papers are you refer to “The second paper” in line 93 page 3 and the “the third paper” in line 99 of page 3. Please complete the references.

2) In eq(10), page 9, the subscript of the \sum is ‘ X in X ’, which I think is incorrect.

3) Please give a short description of the “random” method in table 4, is it randomly recommend items with equal probability? Or set some variables in the algorithm to be random rather than to be calculated?

4) Section 3.4 is not clearly written. First, what is the purpose of section 3.4, is it a preprocess step for building the knowledge graph? then why put it after the section of building knowledge graph? Second, the variables in the equation 9 and 10 are not specifically described. Third, what do you mean by ‘mecab noun’ in table 4? Mecab is a software library as inferred from the context, or is it a dictionary with a bunch of frequently used words?

To sum up, this is a pretty nice application paper, though with little theoretical contributions. The paper is valuable for researchers in this area. I would like to see the author’s responses.

no comment.

Author Response

Reviewer #2

This paper builds an explainable B2B recommender system. This is a very interesting topic and is rarely discussed. My concerns are as follows,

In accordance with the reviewer's comments, we completed the references for "The second paper" and "The third paper".

2) In eq(10), page 9, the subscript of the \sum is ‘ X in X ’, which I think is incorrect.

To rectify this issue, we corrected eq(10) on page page9. Additionally, we added a reference to a relevant article that explains the concept of branching entropy equation in detail.

In Table 4, the "random" method represents a recommendation approach where items are randomly recommended to users with equal probability. As mentioned, we added a brief explanation about the random method. The reason for including the random method in the comparison is to provide a baseline or reference point for individuals who are not familiar with recommendation systems.

According to the reviewer's suggestion, section 3.4 has been moved under "Building Knowledge Graph" as it describes the preprocess step for constructing the knowledge graph.

Additionally, we added a reference to provide detailed explanations for equations 9 and 10. The main focus of the paper is not on these equations, which is why we did not provide extensive explanations in the paper. To keep this section concise and avoid making it overly lengthy, we opted to provide a brief conceptual explanation.

Lastly, as suggested, we replaced "mecab nouns" with "nouns extracted using Mecab" to clarify that it refers to the extracted nouns using the Mecab library.

To sum up, this is a pretty nice application paper, though with little theoretical contributions. The paper is valuable for researchers in this area. I would like to see the author’s responses.

Thank you for your comments.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments for author File: Comments.pdf

Author Response

Reviewer #3

Abstract:

Abstract needed to be updated. You should explain more the objective of KGAT and define what is the acronym before the use of KGAT. especially here : “This paper presents a new perspective on the explainable B2B recommender system using”

In accordance with reviewer's comment, we mentioned the full name of KGAT. The term "new perspective" refers to the approach of recommending potential buyers to sellers in B2B, rather than recommending items to buyers. KGAT facilitates this process while providing the advantage of explainability. We incorporated this explanation into the abstract.

“Unlike traditional recommendation systems that suggest products to consumers, this study focuses on recommending potential buyers to sellers. Additionally, the utilization of KGAT's attention mechanisms enables the provision of explanations for each company's recommendations.”

Introduction:

The introduction section is very weak. Enhance it by splitting it into sub-paragraphs to highlight exactly what you are working on and introduce your research problems. Also, explicitly state the novelty of the B2B recommender system adopted in the paper.

We divided the text into sub-paragraphs to enhance clarity. Additionally, I included the novelty of this paper in the final paragraph of the introduction.

Theoretical Background/Literature Review:

The sub-section B2B Recommender System should be inside the literature review section. I don’t see any work that you reference
Before the "Related Works" section, include a new section titled "Theoretical Background" or "Literature Review" to provide a theoretical foundation for the subsequent discussion.
In the theoretical background section of your article about B2B recommender systems, you should provide a comprehensive overview of the relevant concepts, theories, and fundamental principles that underpin the topic. This section aims to give readers the necessary knowledge and context to understand the rest of your article.

In accordance with the reviewer's comments, we completed the references for "The second paper" and "The third paper".

Related Works:

In the "Related Work" section of your article, it is essential to include a substantial number of references to relevant academic papers, conference proceedings, industry reports, and other reputable sources. This section should demonstrate that you have conducted a thorough review of the existing literature on B2B recommender systems and are familiar with the key research and developments in the field.

Unfortunately, there are not many studies in a similar genre to this research, which is why additional ones could not be included.

Recommender System for Identifying B2B prospects

You are advised to substitute this portion with the "Proposed Approach" in order to elevate the paper's overall quality. While describing your suggested approach, remember to infuse it with your personal insights. Additionally, ensure to explicitly state the study's objectives.

In accordance with the reviewer's comments, we revised the paper.

Methodology Description:

It is preferable to use a global suggested model that describes the process from the beginning. Then describe each phase or step very carefully. The subsection Market Transaction Dataset should not be inside this section.

Based on the reviewer's advice, we made the necessary adjustments to the structure of the paper. Specifically, we included the Market Transaction Dataset as a subsub section in Building B2B Knowledge graphs of buyers within section 3.2.1. This modification was done to ensure a coherent and logical flow of information between sections 3.2.1 and 3.2.2, which are related to each other.

Figures

Figure 6 is not clear. What is the idea this figure is trying to convey? Also the WRITING SIZE is too small.

The size of Figure 6 has been increased to show the process of extracting keywords from unprocessed item names. This section focuses on extracting keywords from unrefined item names in the transaction data. Although item names in the transaction data are unrefined, the process of extracting neologisms has been somewhat challenging. As a result, this content has been included to provide insights into how to extract keywords, which is an important variable from the knowledge graph.

Motivation:

Specify the importance of this paper by adding a research problem and detail your proposed solution.

Distinctions:

Include distinctions to highlight the unique aspects or contributions of the proposed method compared to existing approaches.
Consider incorporating additional datasets (if exists) for a more comprehensive evaluation.

In the Results section, we added the KGAT results for the MovieLens dataset conducted by another researcher.

Proofreading:

The paper should undergo proofreading to eliminate grammatical and typographical errors.

Thank you for your comments. As you mentioned, we will proceed with proofreading and correcting English grammar.

Author Response File: Author Response.pdf

Reviewer 4 Report

Abstract: The article presents a novel Recommender System called KGAT for identifying B2B prospects. The KGAT model utilizes a Collaborative Knowledge Graph (CKG) to combine user-item bipartite and knowledge graphs. The system incorporates graph embedding techniques and attention propagation layers to learn low-dimensional representations and capture high-order relations. The article explains the overall process, data preparation, and dataset used for evaluation. Experimental results demonstrate the KGAT's superiority over baseline models regarding recall and hit rate. The article is well-structured, and the proposed KGAT model appears to be promising.

General Assessment: The article introduces an exciting and relevant topic on a Recommender System for B2B prospects identification. The KGAT model's concept is well-defined, and the overall process is presented. The evaluation methodology is appropriate, and the experimental results indicate the model's effectiveness. Integrating a knowledge graph for better recommendations and the attention scores visualization for analysis adds value to the research.

Specific Comments:

Clarity and Structure: The article's structure is generally clear, with sections covering data preparation, model architecture, dataset description, and evaluation. The explanations are mostly clear, but some parts can be further improved for better readability. Additionally, it would be helpful to provide an overview or flowchart of the KGAT model in the introduction to give readers a better understanding before diving into the technical details.
Related Works: The article lacks a comprehensive review of related works in the field of Recommender Systems, particularly in the context of B2B. Including a review of existing literature and discussing how the KGAT model differs from previous approaches will strengthen the significance of the proposed model.
Data Preprocessing: The data preprocessing steps are briefly mentioned. Providing more details on data cleaning, handling missing values (if any), and the rationale behind the 10-core setting filtering applied to the dataset would be beneficial.
Equation Presentation: The equations are referenced in the text, but their meanings and significance could be better explained. Consider providing additional context or explanation for the key equations, especially Equation (1), which seems critical to the KGAT model.
Experimental Results: The experimental results are presented in tables and graphs, which is appropriate. However, the analysis of the results is relatively limited. The authors should include a discussion of why the KGAT model outperforms the baselines, highlighting the strengths and limitations of the proposed approach.
Attention Scores Visualization: The visualization of attention scores in Figure 8 is insightful. It would be beneficial to provide more such examples and elaborate on how attention scores contribute to personalized prospect recommendations.
Limitations and Future Work: The article does not discuss the limitations of the KGAT model and potential directions for future work. Identifying and addressing the limitations will strengthen the paper's overall credibility and guide future research.
Clarity of Language: Overall, the language is mostly clear and technical terms are well-defined. However, some sentences are long and complex, making them harder to comprehend. Consider simplifying the language and breaking down complex sentences for better readability.

Conclusion: The article presents a well-defined KGAT Recommender System for identifying B2B prospects. The model's integration of a knowledge graph and attention scores visualization enhance its value. The experimental results show promising performance compared to baseline models. However, some areas, such as related works, data preprocessing details, and result analysis, need improvement. Addressing these points will significantly enhance the quality and impact of the article.

Recommendation: The article shows potential, and with some revisions, it can be a valuable contribution to the field of Recommender Systems for B2B applications. I recommend acceptance with revisions. Please address the specific comments and suggestions provided to improve the clarity, structure, and comprehensiveness of the article.

Author Response

Reviewer #4

Specific Comments:

Clarity and Structure: The article's structure is generally clear, with sections covering data preparation, model architecture, dataset description, and evaluation. The explanations are mostly clear, but some parts can be further improved for better readability. Additionally, it would be helpful to provide an overview or flowchart of the KGAT model in the introduction to give readers a better understanding before diving into the technical details.

Figure 1 depicts the flowchart of the KGAT model.

Related Works: The article lacks a comprehensive review of related works in the field of Recommender Systems, particularly in the context of B2B. Including a review of existing literature and discussing how the KGAT model differs from previous approaches will strengthen the significance of the proposed model.

Regrettably, the field of B2B recommendation systems is currently underexplored, which has led to a limited body of research. In response to the reviewer's comments, rather than providing an exhaustive review of the B2B recommendation field, we addressed why models other than KGAT may not perform as effectively in this study.

Data Preprocessing: The data preprocessing steps are briefly mentioned. Providing more details on data cleaning, handling missing values (if any), and the rationale behind the 10-core setting filtering applied to the dataset would be beneficial.

In accordance with reviewer's comment, we provided a more detailed explanation of the 10-core setting approach.

Equation Presentation: The equations are referenced in the text, but their meanings and significance could be better explained. Consider providing additional context or explanation for the key equations, especially Equation (1), which seems critical to the KGAT model.

We provided a more detailed explanation of Equation (1) through further modifications.

Experimental Results: The experimental results are presented in tables and graphs, which is appropriate. However, the analysis of the results is relatively limited. The authors should include a discussion of why the KGAT model outperforms the baselines, highlighting the strengths and limitations of the proposed approach.

We added further explanations regarding the reasons for KGAT's superior performance and its relationship to attention scores. We also addressed the limitations of KGAT in the "Future Work" section.

Attention Scores Visualization: The visualization of attention scores in Figure 8 is insightful. It would be beneficial to provide more such examples and elaborate on how attention scores contribute to personalized prospect recommendations.

Based on the reviewer's comments, we added one more example related to visualizing Attention scores. Additionally, we added further explanations to the figures 8.

Limitations and Future Work: The article does not discuss the limitations of the KGAT model and potential directions for future work. Identifying and addressing the limitations will strengthen the paper's overall credibility and guide future research.

As you mentioned, we wrote about future research directions in the "conclusion" section.

[future work]

To further advance this study, there are several aspects that should be taken into consideration. Firstly, it's crucial to tackle the performance drop of KGAT with increasing triples. Efficient graph learning and embedding strategies can play a vital role in optimizing performance. Moreover, to further extend the potential of the KGAT model, the incorporation of additional side information from users could be explored. For instance, integrating supplementary details such as user interests into the model can provide more sophisticated recommendations. Additionally, the efficiency of the KGAT model is substantially dependent on the quality and availability of the fundamental knowledge graph and data. Hence, focusing on improving data quality and reducing noise can significantly enhance performance. Considering these avenues of development, the KGAT model can evolve into a more effective and accurate B2B recommendation system.

Clarity of Language: Overall, the language is mostly clear and technical terms are well-defined. However, some sentences are long and complex, making them harder to comprehend. Consider simplifying the language and breaking down complex sentences for better readability.

Complex sentences have been shortened, and some grammar errors have been corrected.

Thank you for your comments.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Your responses are okay with me.

The English is readable but a moderate revision of language is suggested to proofread the overall manuscript.

Author Response

According to the reviewer's comment, the whole/partial English has been corrected. Thank you.

Reviewer 3 Report

The paper has not been adjusted according to my recommendations, resulting in a lack of improvement in its quality. Therefore, I suggest that another round of review should be conducted before publication.

● The abstract requires revision. There has been no response regarding the KGAT acronym.

● The introduction section is very weak. Enhance it by splitting it into sub-paragraphs to highlight exactly what you are working on and introduce your research problems. Also, explicitly state the novelty of the B2B recommender system adopted in the paper. Try to remove sub-section titles.

● Before the "Related Works" section, include a new section titled "Theoretical Background" or "Literature Review" to provide a theoretical foundation for the subsequent discussion.

● In the theoretical background section of your article about B2B recommender systems, you should provide a comprehensive overview of the relevant concepts, theories, and fundamental principles that underpin the topic. This section aims to give readers the necessary knowledge and context to understand the rest of your article.

● You are advised to substitute this portion with the "Proposed Approach" in order to elevate the paper's overall quality. While describing your suggested approach, remember to infuse it with your personal insights. Additionally, ensure to explicitly state the study's objectives.

Author Response

Reviewer #3

The paper has not been adjusted according to my recommendations, resulting in a lack of improvement in its quality. Therefore, I suggest that another round of review should be conducted before publication.

The abstract requires revision. There has been no response regarding the KGAT acronym.

The original full name of the acronym "KGAT" has been provided. However, due to LaTeX errors, the modifications made to the abstract couldn't be highlighted.

The introduction section is very weak. Enhance it by splitting it into sub-paragraphs to highlight exactly what you are working on and introduce your research problems. Also, explicitly state the novelty of the B2B recommender system adopted in the paper. Try to remove sub-section titles.

According to the reviewer's comments, the contribution of the proposed method was clearly described in the introduction part. We also removed subsection headings that obscured explicit descriptions.

Before the "Related Works" section, include a new section titled "Theoretical Background" or "Literature Review" to provide a theoretical foundation for the subsequent discussion.
In the theoretical background section of your article about B2B recommender systems, you should provide a comprehensive overview of the relevant concepts, theories, and fundamental principles that underpin the topic. This section aims to give readers the necessary knowledge and context to understand the rest of your article.

The "Theoretical Background" section has been added based on the reviewer's comments. This section delves into the concepts, role, and significance of the B2B recommendation system in this project. Additionally, a concise overview of the relevant theory, specifically KGAT, is provided.

Recommender System for Identifying B2B prospects

You are advised to substitute this portion with the "Proposed Approach" in order to elevate the paper's overall quality. While describing your suggested approach, remember to infuse it with your personal insights. Additionally, ensure to explicitly state the study's objectives.

As per the reviewer's comment, we have repositioned the explanation of the KGAT that was originally in the section titled "Recommender System for Identifying B2B prospects." This section now focuses on describing the process undertaken in this paper to develop a B2B recommendation system.