Enhancing Knowledge-Aware Recommendation with Dual-Graph Contrastive Learning

Abstract

Incorporating knowledge graphs as auxiliary information to enhance recommendation systems can improve the representations learning of users and items. Recommendation methods based on knowledge graphs can introduce user–item interaction learning into the item graph, focusing only on learning the node vector representations within a single graph; alternatively, they can treat user–item interactions and item graphs as two separate graphs and learn from each graph individually. Learning from two graphs has natural advantages in exploring original information and interaction information, but faces two main challenges: (1) in complex graph connection scenarios, how to adequately mine the self-information of each graph, and (2) how to merge interaction information from the two graphs while ensuring that user–item interaction information predominates. Existing methods do not thoroughly explore the simultaneous mining of self-information from both graphs and effective interaction information, leading to the loss of valuable insights. Considering the success of contrastive learning in mining self-information and auxiliary information, this paper proposes a dual-graph contrastive learning recommendation method based on knowledge graphs (KGDC) to explore a more accurate representations of users and items in recommendation systems based on external knowledge graphs. In the learning process within the self-graph, KGDC strengthens and represents the information of different connecting edges in both graphs, and extracts the existing information more fully. In interactive information learning, KGDC reinforces the interaction relationship between users and items in the external knowledge graph, realizing the leading role of the main task. We conducted a series of experiments on three standard datasets, and the results show that the proposed method can achieve better results.

1. Introduction

In the era of information explosion, users are confronted with an overwhelming number of choices. Recommendation systems alleviate this issue by providing personalized recommendations [1,2,3], helping users quickly find content or products of interest, thus reducing information overload and enhancing user satisfaction and experience. For e-commerce and content platforms, recommendation systems can efficiently guide users to content they are more likely to purchase or consume, significantly boosting conversion rates and sales, thereby directly increasing economic benefits for businesses. Collaborative filtering algorithms, widely employed in recommendation systems, leverage users’ historical behavior data to provide personalized recommendations for each user, improving the relevance of recommendations and user satisfaction [4,5,6,7]. However, real-world data often present challenges such as cold start and sparse data, which greatly limit the performance of collaborative filtering-based recommendation algorithms [8,9,10]. To address the issue of sparse data, an effective approach is to integrate auxiliary information into the collaborative filtering recommendation model, and knowledge graph-based recommendation is a typical method in this regard [11,12,13,14,15].

In knowledge graph-based recommendations, in addition to the interaction information between users and items, there is also knowledge graph information about the items. Knowledge graphs contain rich entity relationship information about items, enabling the construction of comprehensive item features and helping to uncover hidden relationships between items, thereby improving the accuracy of recommendations. To make it easier to understand, we use collaborative filtering (CF) to represent the interaction graph between users and items, and use knowledge graph (KG) to represent the connectivity graph between items. CF is a heterogeneous graph composed of users and items, and KG is a graph composed of huge items and multiple types of connection relationships. Cf and KG are two important graphs in KG-based recommendation, and contain rich information.

There already exists much research on sufficiently utilizing information in CF and KG. Earlier studies [11,16,17] focused on independently learning from the two graphs, which mainly represent the triplet information from the item knowledge graph as embedding and its use as contextual information for enhancing recommendations in the CF graph. These methods typically employ knowledge graph embedding (KGE) models (such as TransE [18], TransH [19]) to learn representations of entities in the KG. However, these approaches have limitations in extracting meaningful information from entities, and they can only extract information from a single graph structure, failing to integrate information from the CF graph. Therefore, subsequent works [20,21,22] have increasingly focused on how to extract more relevant information for recommendations from the KG. One productive approach is to represent the interactions with multi-hop paths from users to items, which is hard to optimize because of the manually designed meta-paths. Recently, graph neural networks (GNNs) have demonstrated strong capabilities in representing structural knowledge in graphs [12,13,23,24]. They are widely used in recommendation methods based on knowledge graphs and have achieved excellent performance.

Although knowledge graph-based recommendation has achieved promising results, it still faces the following issues:

• Insufficient mining of the two graphs’ own information: Existing methods often use the interaction data between users and items as supervision signals to derive user and item representation vectors from the entities in the KG for learning and training. However, these methods do not fully exploit the information inherent in the two graphs, especially the strong and effective features of user and item IDs in the recommendation domain. This oversight can lead to the loss of valuable information, adversely affecting the recommendation performance.
• Unbalanced information between the two graphs: Unlike the sparse behavioral data between users and items, the connections in knowledge graphs are dense, containing a wealth of information. The difference in the amount of knowledge contained in the two graphs can cause issues in the subsequent utilization of the information. The supervision signals in CF are directly related to the predictions, whereas the abundant redundant information in the KG can weaken these CF supervision signals. If the dominance of CF information is not maintained, it can lead to a decline in recommendation accuracy.

Inspired by the success of contrastive learning (CL) methods in sparse data scenarios, this paper proposes a Dual-graph Contrastive learning recommendation model based on the Knowledge Graph (KGDC) to address the aforementioned issues. To fully exploit the effective information from each of the two graphs, KGDC utilizes information propagation and aggregation techniques from GNNs to learn the vector representations of IDs in the CF graph and the entities and relations in the KG, respectively. To integrate information from both graphs, the proposed method leverages the concept of contrastive learning to fuse information from two aspects. The first method treats the items interacted with by the same user in the CF graph as pseudo-positive item sets, using them as positive supervision signals in the KG while using other non-similar, non-connected items as negative samples to further learn and train the representation vectors of entities in the KG. The second method considers the corresponding items and entities in the CF graph and KG as similar samples, with other non-corresponding samples as negative samples, to further enhance the information fusion and transfer between the two graphs. Finally, we employ a multi-objective training mode, where the vector representations of different components are used to calculate the loss according to different objectives, and the losses are summed with different weights to optimize and train the model parameters. We conducted extensive experiments on two public datasets, and the results show that our proposed method outperforms some state-of-the-art methods.

The contributions of this paper are as follows:

• We propose the dual-graph conception to fully exploit the information within each graph while effectively integrating information between the graphs. It firstly enhances the deep exploration of each graph’s inherent information, particularly strengthening the representation learning of user and item IDs in CF graph. Then, during the information fusion process, it ensures the strong dominance of target consistency information, preventing interference from redundant information in the KG graph.
• We introduce the contrastive learning to both individual and integrated learning stages. On the one hand, sample construction in contrastive learning prevents overfitting caused by excessive emphasis on the target. On the other hand, it incorporates the loss calculation based on the comparative learning sequence relationship to improve the effectiveness of ranking orders.
• We conduct extensive experiments on public datasets, further validating the superior performance of the proposed method.

The rest of this paper is organized as follows. Section 2 reviews the related work on graph-based recommendation and contrastive learning. Section 3 defines the problem formulation and presents a detailed introduction of KGDC. Section 4 presents experimental results, and subsequent discussions are provided in Section 5. Section 6 concludes the paper and looks forward to future work.

2. Related Work

The study in this paper is mainly relevant to two parts of works: knowledge graph-based recommendation and contrastive learning.

2.1. Knowledge Graph-Based Recommendation

Research into knowledge graph-based recommendation systems has explored various methodologies, focusing on leveraging the structure and semantics of knowledge graphs to enhance recommendation performance. This part reviews notable contributions from two aspects: non-GNN-based methods and GNN-based methods.

2.1.1. Non-GNN-Based Methods

Early work on integrating knowledge graphs with recommendation systems primarily focused on leveraging the rich semantic relationships inherent in knowledge graphs to improve recommendation accuracy [11,17,25]. Zhang [17] introduced the Collaborative Knowledge Base Embedding (CKE) framework, which combines collaborative filtering techniques with knowledge graph embeddings to enhance recommendation performance by incorporating additional contextual information. Similarly, KTUP [25] considers both explicit user–item interactions and implicit knowledge graph relationships to improve the recommendation process with the TransH [19] method. Wang [11] proposed the RippleNet model, which propagates user preferences over a knowledge graph to uncover multi-hop relational paths, thereby enriching the user’s preference representation.

Another significant contribution in this area is the use of path-based methods. These approaches often employ random walk techniques or path ranking algorithms to identify relevant connections between entities. PER [26] extracts meta-path or meta-graph latent features from the knowledge graph to exploit the connectivity between users and items along different types of relation paths or graphs. KPRN [22] goes beyond predefined meta-paths by dynamically extracting paths that are relevant to user–item interactions, thereby improving the ability to capture complex relationships.

Embedding-based methods and path-based methods are both useful and effective. However, embedding-based methods may not fully capture the nuances of user preferences in recommendation scenarios where user–item interactions are sparse or when personalized recommendation accuracy is critical. Path-based methods rely heavily on the quality of path extraction and the ability to capture relevant semantic relationships.

2.1.2. GNN-Based Methods

GNN-based methods in knowledge graph-based recommendation systems have evolved significantly, addressing challenges related to sparse interactions, unbalanced graph utilization, and knowledge extraction. KGCN [12] focuses on leveraging user preferences to aggregate neighborhood information of items in the KG. Similarly, KGAT [13] uses GNNs to recursively aggregate information across the unified heterogeneous graph, emphasizing the importance of both collaborative filtering signals and knowledge signals. CKAN [15] highlights the significance of collaborative filtering signals in addition to knowledge signals. In KGIN [24], GNNs are applied on the user-intent–item-entity graph to capture nuanced user preferences. CG-KGR [27] uses pre-trained collaborative signals to guide the aggregation process on the knowledge graph, focusing on extracting sufficient knowledge for recommendations. KGIC [28] employs GNNs to capture hierarchical and structural dependencies within the KG.

Most GNN-based methods operate under supervised learning paradigms, which can be challenging when dealing with sparse user–item interactions. This limitation affects recommendation accuracy, particularly in scenarios with limited data. Moreover, the connections in knowledge graphs are dense, containing a wealth of information. These methods often overly emphasize the knowledge graph, potentially neglecting the crucial collaborative filtering signals inherent in user–item interactions.

2.2. Contrastive Learning

Contrastive learning, as an effective unsupervised learning method, has been widely studied and applied in recent years across domains such as image and natural language processing. Its core idea is to compare positive examples (similar samples) and negative examples (dissimilar samples), aiming to bring similar samples closer together in the representation space while pushing dissimilar samples further apart. This approach helps alleviate issues related to insufficient learning caused by sparse data. SimCLR [29] utilizes data augmentation techniques to generate multiple views of the same instance and employs a contrastive loss function to maximize the similarity between different views of the same instance while minimizing the similarity between different instances.

In recent research, several approaches have applied contrastive learning to enhance knowledge graph (KG)-based recommendation systems. SGL introduces three operators to generate augmented views and employs a multi-task strategy, jointly optimizing contrastive loss and recommendation loss to improve the robustness of representation learning in KG-based recommendation systems. SEPT [30] proposes a socially aware contrastive learning framework, and mines additional social information of users to enrich the learning process, aiming to better capture user preferences and interactions in recommendation tasks. CKER [31] integrates a contrastive learning module to enhance the sharing of user preferences, and derives additional supervision signals to improve the collaborative filtering aspect of KG-based recommendation systems. KGCL [32] introduces a knowledge-guided contrastive learning paradigm to derive more robust node representations, enhancing the quality of recommendations. KGIC [28] applies contrastive learning to learn embeddings or representations that distinguish relevant (positive) interactions from irrelevant (negative) ones based on the KG. KACL [33] utilizes contrastive learning to discern meaningful patterns and relationships within the knowledge graph.

Existing recommendation methods based on knowledge and contrastive learning mostly directly learn the integrated information of the two graphs, which can result in insufficient exploration of the inherent information in each graph. Moreover, when there is an imbalance of information between the CF and KG graphs, it is easy to introduce too much redundant information from the KG, which can negatively impact the prediction results.

Our proposed KGDC utilizes the dual-graph concept, simultaneously addressing the exploration of inherent information in both graphs and the integration of information between them. During the information fusion process, we ensure that the supervised signals in CF, which is more relevant to the target, maintain a dominant position.

3. Methodology

In this section, we present our proposed KGDC recommendation method. The purpose of the proposed KGDC is to fully exploit the individual information and interactive information from CF and KG. Figure 1 presents the framework of KGDC, which consists of three parts: Individual Graph Constructing and Encoding, Interactive Graph Constructing and Encoding, and Multi-task Learning. It performs contrastive learning in both individual graph and interactive graph learning to extract more effective information. Then we introduce the details of the three components.

3.1. Problem Formulation

To facilitate understanding, we describe the problem formulation here. We first introduce the structural data: the collaborative filtering graph of user–item interactions and the knowledge graph, and then formulate the problem statement of KG-based recommendation.

Collaborative Filtering Graph: To maintain consistency with the settings in GNN-based recommendation systems, we construct a bipartite graph $G_{cf}=\left\{(u,y_{uv},v)\right\}$ based on the historical interactions between users and items, such as the consumption, viewing, and clicking. In the construction of this graph, there are connections only between item and user nodes, with edges existing between nodes of users and items that have had historical interactions, where $u\in U$ and $v\in V$ denote the user and item involved in the interaction, and U and V are the sets of users and items, respectively. $y_{uv}$ is an indicator as to whether there is a connection between user u and item v based on the historical interactions. $y_{uv}=1$ indicates an interaction, while $y_{uv}=0$ indicates no interaction.

Knowledge Graph: In the knowledge graph, the connections are denoted as a set of triplets $G_{kg}=\left\{(h,r,t)\right\}$, and each triplet describes a connection from head entity $h\in I$ to tail entity $t\in I$ with the relationship $r\in R$. There, it usually denotes I and R as the sets of entities and relations in the knowledge graph, and I includes items V and non-item entities $I\setminus V$.

KG-based Recommendation: Based on the user–item interaction graph $G_{cf}$ and item knowledge graph $G_{kg}$, the target of KG-based recommendation is to learn a function $y(u,v)$ that predicts the probability how likely a user u would interact with an item v.

3.2. Individual Graph Constructing and Encoding

Different from most existing methods, KGDC learns node and entity embedding from KG and CF, respectively, with GNN methods. In KG, KGDC uses contrastive learning to overcome the problem of missing explicit labels. In CF, KGDC uses propagation and aggregation methods for learning ID embedding of users and items.

3.2.1. Graph Constructing and Encoding in KG

During the individual graph constructing and encoding in KG, only triplets $G_{kg}=\left\{(h,r,t)\right\}$ are valid here. Considering that there are no explicit signals, KGDC utilizes contrastive learning to construct samples for information aggregation, loss calculation, and parameter optimization.

In terms of sample construction, the training entities, which are denoted as E, at different stages are also different. For example, the goal of the pre-training stage is to adequately train all nodes in the graph, and all nodes will participate in the training, $E=I$. In the subsequent model fine-tuning process, the entities to be trained are those involved in user–item interactions $E\in I$, with the aim of performing better for the prediction target. For any entity $e_i\in E$, the positive samples of $e_i$ are sampled from the connected tails and relations from $G_{kg}$, denoted as $(r_{i,p},t_{i,p})$∈$G_{kg}$. Then the negative samples are sampled from all entities that are not connected in $G_{kg}$, denoted as $(r_{i,n},t_{i,n})$∉$G_{kg}$. Combining the positive and negative samples, we can obtain the samples for KG, denoted as $S_{kg}=\{(r_{i,p},t_{i,p}),(r_{i,n},t_{i,n})\dots ,\dots \},e_i\in E$.

With the constructed samples, KGDC uses the attentive mechanism in GNNs to propagate and aggregate information in KG. Moreover, for any entity $e_i\in E$, we first compute the representations of positive samples by linear summation,

$$h_{kg,i}^{+}=\sum _{j\in (r_{i,p},t_{i,p})}{\alpha }_{ij}{h}_{kg,j},$$

(1)

where $h_{kg,j}$ is the representation of entity j in KG, and ${\alpha }_{ij}$ denotes the attention score between entity $e_i$ and entity $e_j$. ${\alpha }_{ij}$ is calculated as follows:

$${\alpha }_{ij}={\displaystyle \frac{exp(h_{kg,i}·r_j)}{{\sum }_{k\in (r_{i,p},t_{i,p})}exp(h_{kg,k}·r_k))}}.$$

(2)

Then, the positive representation of entity i is calculated as follows:

$$h_{kg,i}^{+}=\sum _{j\in (r_{i,p},t_{i,p})}{\displaystyle \frac{exp(h_{kg,i}·r_j)}{{\sum }_{k\in (r_{i,p},t_{i,p})}exp(h_{kg,k}·r_k))}}{h}_{kg,j}.$$

(3)

Similarly, the negative representation of entity i is calculated as follows:

$$h_{kg,i}^{-}=\sum _{j\in (r_{i,n},t_{i,n})}{\displaystyle \frac{exp(h_{kg,i}·r_j)}{{\sum }_{k\in (r_{i,n},t_{i,n})}exp(h_{kg,k}·r_k))}}{h}_{kg,j}.$$

(4)

With these representations of entity $e_i$, we calculate the similarity score between positive samples and the target sample through the inner product, $p_i^{+}=h_{kg,i}·h_{kg,i}^{+}$. Similarly, the similarity score between negative samples and the target sample is denoted as $p_i^{-}=h_{kg,i}·h_{kg,i}^{-}$. Then, we adopt a pair-wise loss BPR to encourage the positive samples to be more similar with the target sample than the negative samples:

$$L_{kg}=\sum _{e_i\in E}-log\sigma (p_i^{+}-p_i^{-}),$$

(5)

where $\sigma$ is the Sigmoid function.

3.2.2. Graph Constructing and Encoding in CF

In the individual graph constructing and encoding in CF, KGDC learns the embedding of users and items by user–item interactions. This does not differentiate between users and items, and KGDC performs the same neighbor sampling and embedding aggregation on call nodes in CF to achieve information propagation.

During representation learning, the main idea is to update the vector representation of each vertex based on its own representation and that of its neighbors using some aggregation function. This process is repeated for all vertices for T iterations to obtain the representation of each node. Before representation learning, the neighbors of each vertex in CF are stored globally, denoted as $N\left(s_i\right)$, where the nodes $s_i$ can be either a user node or an item node. Assuming it is the t-th iteration, KGDC directly retrieves its neighbor set from global storage, denoted as $N\left(s_i\right)$. Then, it randomly selects a fixed number K of neighbors from $N\left(s_i\right)$, denoted as $N{\left(s_i\right)}^t$. If the number of neighbors is less than K, sampling with replacement is used to reach the desired number; if it is greater than K, sampling without replacement is used.

With the sampled neighbors of the node $s_i$ in the t-th iteration, we can obtain the embedding set of these nodes using the equation as follows:

$$h_{cf,N\left(s_i\right)}^{\left(t\right)}={\mathrm{AGGREGATE}}_t(h_{cf,j}^{\left(t\right)},\forall j\in N{\left(s_i\right)}^t),$$

(6)

where ${\mathrm{AGGREGATE}}_t$ is the aggregation method at the t-th iteration, such as mean pooling, sum pooling, max pooling, RNN, transformer, and other methods. Obtaining the presentation of neighbors, we perform a linear transformation and enhance the representation of the original node $s_i$ by using a custom activation function. The vector representation of vertex $s_i$ at the $t+1$-th iteration is calculated as:

$$h_{cf,s_i}^{(t+1)}=\sigma \left(W_t\left(\mathrm{CONCAT}(h_{cf,s_i}^{\left(t\right)},h_{cf,N\left(s_i\right)}^{\left(t\right)})\right)+b_t\right),$$

(7)

where $W_t\in R^{2d\ast d}$ and $b_t\in R^d$ are the trainable weights and the trainable bis at the $(t$-th iteration.

After T iterations, the node representation $h_{cf,s_i}^{\left(T\right)}$ is L2-normalized to obtain the final representation vector:

$$h_{cf,s_i}={\displaystyle \frac{h_{cf,s_i}^{\left(T\right)}}{\parallel h_{cf,s_i}^{\left(T\right)}{\parallel }_2}}.$$

(8)

With the nodes’ embedding in CF, we compute the individual loss of CF as follows:

$$L_{cf}=-\sum _{(s_i,y_{i,j},s_j)\in G_{cf}}(y_{i,j}log\left(\widehat{y_{i,j}}\right)+(1-y_{i,j})(1-log\left(\widehat{y_{i,j}}\right)))),$$

(9)

where $\widehat{y_{i,j}}=\mathrm{Sigmoid}\left(h_{cf,s_i}^T{h}_{cf,s_j}\right)$ is the predicted score of node $s_i$ and node $s_j$, $y_{i,j}$ is namely the edges in CF, and the value is the same as $y_{u,v}$.

3.3. Interactive Graph Constructing and Encoding

KGDC also leverages contrastive learning to learn the interactive information from two graphs. To ensure the dominant position of the CF graph, KGDC takes the signals in CF to guide the learning of entities in the KG. To further enhance the information fusion between the two graphs, KGDC uses the entities corresponding to the items in both graphs as similar samples for alignment.

3.3.1. Graph Encoding in KG with Signals of CF

In this section, KGDC extracts the signals of user/item from CF to guide the training of nodes in KG. The supervised signals extracted from CF are first-order connections between users and items. For any user u in CF, all of the items that have interacted with u are collected. Therefore, the signals from CF are denoted as:

$$V_{cf}\left(u\right)=\{v\mid (u,v)\in G_{cf},\mathrm{and}v\in \{v\mid y_{uv}=1\}\}.$$

(10)

With the signals above, the initial entities learned in the KG are those that correspond to the items in the node set $V_{cf}\left(u\right)$, which are denoted as:

$$V_{kg,0}^{+}\left(u\right)=\{e\mid (u,e)\in G_{cf}\}.$$

(11)

In the collaborative filtering recommendation, each user has his own interests and preferences, and there are commonalities in preferences among different users, which means that the items that interact with the same user tend to be more similar. Therefore, for any sample in CF $(u_i,v_j)$ and $y_{u_i,v_j}=1$, the items in $V_{kg,0}\left(u_i\right)$ are more similar with $v_j$ than others. Furthermore, KGDC takes the items randomly selected from non-connected items across the entire dataset as negatives, denoted as $V_{kg,0}^{-}\left(v_j\right)$. Taking the positive samples $V_{kg,0}\left(u_i\right)$ and negative samples $V_{kg,0}^{-}\left(v_j\right)$ as the initial entity sets of the KG, KGDC constructs the embedding through a natural propagation in the KG. The propagation is the same as in Section 3.2.2, and aggregation is similar to that in Section 3.2.1, which randomly selects a fixed number of connected (relation, tail) as neighbors and aggregates the information using the attentive method as in Equations (1) and (2). At the l-th layer, the embedding of any entity e (including the positive samples and negative samples) defined above is as follows:

$$h_{kg,e}^{(l+1)}=\mathrm{CONCAT}\left(h_{kg,e}^l{\mathrm{AGGREGATE}}_l\left(\sum _{j\in (r_e,t_e)}{\alpha }_{ij}{h}_{kg,j}^{\left(l\right)}\right)\right),$$

(12)

where $h_{kg,*}$ is the embedding in KG, and the definition of ${\mathrm{AGGREGATE}}_l$ is the same as in Equation (6).

Similarly, after L iterations and L2-normalization as in Equation (8), we can obtain the entity presentation, denoted as $h_{kg,e}$. Then, we utilize the triplet loss function, which is usually used in contrastive learning to compute the loss. The distance score between two entities is calculated through the inner product. Therefore, the loss function is as follows:

$$L_{kic}=\sum _{(u,v)\in G_{cf}}\sum _{e_i\in V_{kg,0}^{+}\left(u\right)}\sum _{e_j\in V_{kg,0}^{-}\left(v\right)}\mathrm{Max}(h_{kg,v}^T{h}_{kg,e_j}-h_{kg,v}^T{h}_{kg,e_i}+m,0.0)$$

(13)

where m is the margin.

3.3.2. Align Encoding with CF and KG

To further enhance the interaction between the two graphs, we propose to map the item in CF and the corresponding entity in KG into the same space. For better understanding, we use $h_{cf}\left(v\right)$ to represent the item embedding in CF, and use $h_{kg}\left(e\right)$ to represent the entity embedding in KG, where e and v correspond to the same item. Therefore, for any item v in CF, the positive sample is the corresponding item e in KG, and the negative sample is the randomly selected item j from batch-wise in the training process. With the batch-wise negative sampling strategy, we do not need to sample globally, addressing computational power constraints and reducing time consumption.

Before the alignment between the two graphs, KGDC utilizes two MLPs to map them into the same space $(z_{cf}\left(v\right),z_{kg}\left(e\right))$ to achieve the information shared across interaction and knowledge views. Thus, the positive pair is denoted as $\{z_{cf}\left(v\right),z_{kg}\left(e\right)\}$, and the negative pairs are denoted as $\{z_{cf}\left(v\right),z_{kg}\left(j\right)\}$ and $\{z_{cf}\left(j\right),z_{kg}\left(e\right)\}$. With these samples, we adopt the InfoNCE function to calculate the loss, which is commonly used in contrastive learning. InfoNCE loss aims to encourage the consistency between the representations of its different views while enforcing the divergence of negative pairs. The calculation formula is as follows:

$$L_{\mathrm{ali}}\left(v\right)=-log{\displaystyle \frac{exp(cos(z_{cf}\left(v\right),z_{kg}\left(e\right))}{{\sum }_{j\in V\cup \left\{v\right\}}exp(cos(z_{cf}\left(v\right),z_{kg}\left(j\right))+exp\left(cos(z_{cf}\left(j\right),z_{kg}\left(e\right))\right)}}$$

(14)

where $cos(·)$ measures the cosine similarity of two vectors, and V (namely the items in CF) is the set of negative samples.

3.4. Multi-Task Learning

Considering that some of the losses obtained above are not directly related to the target, we take a portion of the representations to reconstruct embedding for users and items to ensure the update is more directly to the target. For any data $(u,v)$ in the interaction dataset, the representations of individual CF in Section 3.2.2 are $h_{cf,u}$ and $h_{cf,v}$, and the representations of interactive learning in Section 3.3.1 are $\{h_{kg,e\}\mid e\in V_{kg,0}^{+}\left(u\right)}$ and $h_{kg,v}$. Specifically, for the set of embedding $\{h_{kg,e}\mid e\in V_{kg,0}^{+}\left(u\right)\}$, we utilize the mean pooling method to aggregate them to obtain the user’s presentation, denoted as ${\overline{h}}_{kg,V_{kg,0}^{+}\left(u\right)}$. By concatenating these vectors, we can obtain the final presentations of the users and items, which are also the model’s predictions. The formulations of reconstruction and the matching score are as follows:

$$h_u=\mathrm{CONCAT}(h_{cf,u},{\overline{h}}_{kg,V_{kg,0}^{+}\left(u\right)}),$$

(15)

$$h_v=\mathrm{CONCAT}(h_{cf,v},h_{kg,v}),$$

(16)

Then, we obtain the predicted score using the inner product and Sigmoid function, denoted as $\widehat{y}(u,v)=\mathrm{Sigmoid}\left(h_u^T{h}_v\right)$. With $\widehat{y}(u,v)$ and the real labels $y(u,v)$, we utilize the cross-entropy function to compute the loss as Equation (9), denoted as $L_{target}$.

To combine the individual learning as well as the interactive learning proposed above with the recommendation task, we proposed a multi-task learning framework to jointly train the multiple losses as in Algorithm 1. In the proposed framework, the individual learning of KG is conducted in the pre-train process, which enables all of the entities and relations in KG to be fully trained compared with learning it in the training process. Moreover, the equation of $L_{total}$ consists of four parts, where $L_{target}$ and $L_{cf}$ are directly related to the final objective and the other two are indirectly related. This ensures that the user–item interaction signal maintains a dominant position.

Algorithm 1 KGDC Algorithm

1: Input: user–item interaction graph $G_{cf}$; knowledge graph $G_{kg}$; trainable parameters: ${\{h_{cf,u},h_{cf,v}\}}_{u\in U,v\in V}$, ${\left\{e\right\}}_{e\in E}$, ${\left\{r\right\}}_{r\in R}$, ${\{W_i,b_i\}}_{i=1}^L$; hyper-parameters and functions: $L,dim,\sigma (·),\mathrm{CONCAT}(·),\mathrm{AGGREGATE}(·)$ 2: Output: Prediction vectors 3: for  $h=1,\dots ,epoch\left(\mathrm{pre}‐\mathrm{train}\right)$do 4:     for $e\in E$ do 5:         Construct samples and presentations as Equations (1)–(4) 6:         Calculate and obtain loss $L_{kg}$ as in Equation (5) 7:         Update parameters by gradient descent 8:     end for 9: end for 10: for  $h=1,\dots ,epoch\left(\mathrm{train}\right)$do 11:     for $(u,v)\in G_{cf}$ do 12:         Individual learning as Equations (3), (4) and (6)–(8) 13:         Calculate loss $L_{kg}$ as Equation (5), $L_{cf}$ as Equation (9) 14:         Interactive learning as Equation (12) and Section 3.3.2 15:         Calculate loss $L_{kic}$ as Equation (13), $L_{ali}$ as Equation (14) 16:         Reconstruct $h_u$, $h_v$ as Equation (15), Equation (16), and calculate target loss $L_{target}$ as Equation (9) 17:         Calculate $L_{total}=L_{target}+\alpha L_{cf}+\beta L_{kg\_}+\gamma L_{CL}$ 18:         Update parameters by gradient descent 19:     end for 20: end for 21: return  $h_u,h_v$

4. Experiments

In this section, we conduct extensive experiments on two real-world datasets to evaluate the proposed KGDC. Next, we will sequentially provide detailed introductions to the experimental datasets, comparison baselines, experiment settings, and experimental results.

4.1. Datasets

We evaluate KGDC on two distinct scenarios: books and music. These datasets are publicly available and vary significantly in size and sparsity, and they are often used in the field of knowledge graph-based recommendation. Below are the details of each dataset:

• Amazon-book (http://jmcauley.ucsd.edu/data/amazon/, accessed on 1 April 2024): This dataset includes user ratings and extensive metadata for books, such as descriptions, category information, price, and brand, sourced from Amazon.
• Last.FM (https://grouplens.org/datasets/hetrec-2011/, accessed on 1 April 2024): Collected from the Last.FM online platform, this music dataset consists of user-generated music listening events.

The basic statistics of the two datasets are presented in

Table 1. Basic statistics of the two datasets.

		Book-Crossing	Last.FM
User–item Interaction	# users	17,860	1872
	# items	14,967	3846
	# interactions	139,746	42,346
Knowledge Graph	# entities	77,903	9366
	# relations	25	60
	# triplets	151,500	15,518

.

We adopt the method in [11] to convert explicit feedback across three datasets into implicit feedback, where a label of 1 denotes positive samples. In the Book-Crossing dataset, the ratings greater than or equal to 4 are labeled as 1, while the threshold is 0 in the Last.FM dataset. For every dataset, the ratio of training, evaluation, and test set is 6:2:2. Each experiment is repeated 3 times, and the average performance is reported. For constructing sub-KGs, we utilize Microsoft Satori4, which is similar to RippleNet [11] and KGCN [12]. Each sub-KG adheres to the triple format and constitutes a subset of the entire KG with confidence levels exceeding 0.9. From the sub-KG, we extract Satori IDs of all valid movies, books, or musicians by matching their names with the tail of triples. Subsequently, we match item IDs with the head of all triples and select well-matched triples from the sub-KG.

4.2. Baselines

To demonstrate the effectiveness of the proposed KGDC, we compare it with the recent state-of-the-art methods, including BPRMF, CKE, KGCN, KGNN-LS, KGAT, CKAN, KGIN, and KGIC. The descriptions of these methods are as follows:

• BPRMF [34]: It aims to rank user–item interactions in a way that places higher preference scores on items that users have interacted with, which is a traditional CF-based method;
• CKE [17]: It leverages structured information from knowledge graphs to enhance the quality of recommendations by embedding both user–item interactions and the rich semantic relationships present in the knowledge graph;
• KGCN [12]: It captures both the structural and semantic information from the knowledge graph, enabling it to learn more comprehensive and rich representations of items;
• KGNN-LS [14]: It integrates knowledge graph information with neural networks, incorporating the technique of label smoothing to improve model performance;
• KGAT [13]: It integrates knowledge graph information using attention mechanisms;
• CKAN [15]: It employs attention mechanisms to dynamically focus on the most relevant entities and relationships within the knowledge graph, enhancing the collaborative filtering process;
• KGIN [24]: It applies a GNN to the user-intent–item-entity graph, allowing for a more granular and nuanced understanding of these interactions;
• KGIC [28]: It enhances traditional collaborative filtering by leveraging rich semantic relationships from knowledge graphs with the contrastive learning method;
• CG-KGR [27]: It employs a collaborative guidance mechanism, which encodes historical interactions as guidance for personalized knowledge extraction, making it particularly effective for tasks like Top-K recommendation.

4.3. Experiment Settings

The settings of hyper-parameters for baselines and the proposed KGDC are as follows. To ensure the fairness of the experimental results, some parameters are set to fixed values. The dimensions of nodes in CF and nodes (and relations) in KG are set as 64, and the batch size is set as 1024. Model parameters are initialized using Xavier, and optimization is performed using Adam.

For some parameters, we search for the optimal values within a certain range. We explore learning rates in the range {0.0001, 0.0005, 0.001, 0.0015}, and vary the number of neighbors between 2 and 50. The depth exploration of GNNs ranges from 1 to 4. Baselines retain their default hyper-parameter settings except for the aforementioned parameters. To mitigate overfitting, we ensure training convergence by limiting the number of steps to 50.

To evaluate the effectiveness of our proposed KGDC in the knowledge graph-based recommendations, we employ four widely used metrics: AUC, F1, Recall@K, and NDCG@K, where the K values of Recall@K and NDCG@K are set to [5, 10, 20, 50, 100]. AUC and F1 are two widely used metrics in click-through rate (CTR) prediction, while Recall@K and NDCG@K are two methods usually used in the Top-K recommendation.

The AUC can be calculated using the following formula:

$$\mathrm{AUC}={\displaystyle \frac{{\sum }_{i=1}^P{\mathrm{rank}}_i-\frac{P(P+1)}{2}}{P\times N}},$$

where ${\mathrm{rank}}_i$ is the rank of the i-th positive sample, P is the number of positive samples, and N is the number of negative samples.

The F1 score combines precision and recall and is calculated as:

$$\mathrm{F}1=2\times {\displaystyle \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}},$$

where $\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$, and $\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$. The definitions of “TP, FP, FN” are as follows:

• TP (True Positives) is the number of correctly predicted positive outcomes;
• FP (False Positives) is the number of incorrectly predicted positive outcomes;
• FN (False Negatives) is the number of incorrectly predicted negative outcomes.

The formula of Recall@K is as follows:

$$\mathrm{Recall}\text{@}\mathrm{K}={\displaystyle \frac{\mathrm{Number}\mathrm{of}\mathrm{relevant}\mathrm{items}\mathrm{in}\mathrm{Top}‐\mathrm{K}\mathrm{recommendations}}{\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{relevant}\mathrm{items}}}$$

NDCG@K is calculated as follows:

$$\mathrm{NDCG}\text{@}\mathrm{K}={\displaystyle \frac{\mathrm{DCG}\text{@}\mathrm{K}}{\mathrm{IDCG}\text{@}\mathrm{K}}},$$

$$\mathrm{DCG}\text{@}\mathrm{K}=\sum _{i=1}^{min(K,n)}{\displaystyle \frac{2^{re{l}_i}-1}{{log}_2(i+1)}},$$

$$\mathrm{IDCG}\text{@}\mathrm{K}=\sum _{i=1}^{min(K,n)}{\displaystyle \frac{2^{max\left(re{l}_i\right)}-1}{{log}_2(i+1)}},$$

where n is the number of items in the recommendation list, and $re{l}_i$ is the relevance score of the i-th item in the recommendation list.

4.4. Results

In this section, we present a comprehensive performance analysis between our proposed KGDC and all representative baselines in the tasks of CTR prediction and Top-K recommendation.

4.4.1. Results of CTR Prediction

We present the experimental results of the CTR prediction task across all datasets in

Table 2. Average results of AUC and F1 in CTR prediction task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	AUC	F1	AUC	F1
BPRMF	0.6583	0.6117	0.7563	0.7010
CKE	0.6759	0.6235	0.7471	0.6740
KGCN	0.6841	0.6313	0.8027	0.7086
KGNN-LS	0.6762	0.6314	0.8052	0.7224
KGAT	0.7314	0.6544	0.8293	0.7424
CKAN	0.7420	0.6671	0.8418	0.7592
KGIN	0.7273	0.6614	0.8486	0.7602
KGIC	0.7473	0.6690	0.8592	0.7753
CG-KGR	0.7472	0.6794	0.8368	0.7424
Our KGDC	0.7656	0.6872	0.8652	0.7771
%Improv	2.45%	1.15%	0.7%	0.23%

. From these results, we draw the following observations and analyses.

4.4.2. Results of the Top-K Recommendation

Similarly, we conduct experiments of Top-K recommendations across all datasets, and vary K in the range of { 5, 10, 20, 50, 100}. For a better comparison between KGDC and these baselines, we firstly show the average results of the Top-K, K = 5, 10, 20, 50, 100, in

Table 3. Average results of Recall@5 and NDCG@5 in Top-K recommendation task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	Recall	NDCG	Recall	NDCG
BPRMF	0.0267	0.0195	0.0514	0.0461
CKE	0.0304	0.0167	0.059	0.0404
KGCN	0.0316	0.0471	0.078	0.0569
KGNN-LS	0.0352	0.0518	0.066	0.0512
KGAT	0.0276	0.0181	0.067	0.0511
CKAN	0.0273	0.0179	0.0901	0.0724
KGIN	0.0294	0.0234	0.0923	0.0812
KGIC	0.0388	0.0316	0.0906	0.0906
CG-KGR	0.0567	0.0747	0.0987	0.0711
Our KGDC	0.0614	0.0758	0.1143	0.1018
%Improv	8.29%	0.15%	15.81%	12.36%

,

Table 4. Average results of Recall@10 and NDCG@10 in Top-K recommendation task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	Recall	NDCG	Recall	NDCG
BPRMF	0.0341	0.0218	0.1014	0.0699
CKE	0.0315	0.0171	0.0784	0.0431
KGCN	0.0573	0.0511	0.1243	0.0823
KGNN-LS	0.0694	0.0581	0.1047	0.0734
KGAT	0.0359	0.0249	0.1111	0.0749
CKAN	0.0342	0.0251	0.1212	0.0994
KGIN	0.0401	0.0245	0.1351	0.0978
KGIC	0.0450	0.0318	0.1426	0.1136
CG-KGR	0.0811	0.0725	0.1675	0.1012
Our KGDC	0.0963	0.0819	0.1773	0.1264
%Improv	18.74%	12.97%	5.85%	11.27%

,

Table 5. Average results of Recall@20 and NDCG@20 in Top-K recommendation task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	Recall	NDCG	Recall	NDCG
BPRMF	0.0467	0.0280	0.1684	0.0875
CKE	0.0438	0.0217	0.1151	0.0496
KGCN	0.0785	0.0593	0.1825	0.0973
KGNN-LS	0.0851	0.0606	0.1773	0.0911
KGAT	0.0534	0.0301	0.1822	0.0931
CKAN	0.0619	0.0347	0.2078	0.1194
KGIN	0.0659	0.354	0.2257	0.1347
KGIC	0.0666	0.0397	0.2179	0.1405
CG-KGR	0.1081	0.0769	0.2442	0.1410
Our KGDC	0.1172	0.0821	0.2585	0.1502
%Improv	8.418%	6.762%	5.856%	6.525%

,

Table 6. Average results of Recall@50 and NDCG@50 in Top-K recommendation task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	Recall	NDCG	Recall	NDCG
BPRMF	0.0597	0.0371	0.2693	0.1201
CKE	0.0502	0.0259	0.2415	0.1026
KGCN	0.1100	0.0613	0.2783	0.1097
KGNN-LS	0.1347	0.0701	0.2834	0.1273
KGAT	0.0671	0.0349	0.2912	0.1235
CKAN	0.0896	0.0407	0.3101	0.1516
KGIN	0.1063	0.0414	0.2942	0.1567
KGIC	0.1227	0.0488	0.3769	0.1765
CG-KGR	0.1527	0.0901	0.2894	0.137
Our KGDC	0.1792	0.1048	0.3951	0.1705
%Improv	17.35%	16.32%	5.856%	-0.34%

and

Table 7. Average results of Recall@100 and NDCG@100 in Top-K recommendation task. The best results are in boldface and the second best results are underlined.

Model	Book-Crossing		Last.FM
	Recall	NDCG	Recall	NDCG
BPRMF	0.0612	0.0423	0.3711	0.1348
CKE	0.0517	0.0263	0.3793	0.1294
KGCN	0.1451	0.0687	0.4001	0.1323
KGNN-LS	0.1599	0.0787	0.3772	0.1371
KGAT	0.0912	0.0410	0.3997	0.1426
CKAN	0.1304	0.0510	0.4234	0.1785
KGIN	0.1321	0.0579	0.4167	0.1815
KGIC	0.1719	0.0544	0.4842	0.1981
CG-KGR	0.2187	0.1073	0.3874	0.1544
Our KGDC	0.2440	0.1154	0.4979	0.2112
%Improv	10.37%	7.55%	2.83%	6.612%

. Then, we make use of four subfigures to further illustrate the complete results for the Recall@K and NDCG@K metrics, shown in Figure 2.

5. Discussion

We next conduct a detailed analysis and discussion of the results in Section 4.

5.1. Discussion on Results of the CTR Prediction

For the CTR prediction task, our KGDC demonstrates significant improvements in performance. Particularly on the Book-Crossing dataset, KGDC improves by 2.45% and 1.15% in AUC and F1 compared to the best result of the baselines, respectively, while on the Last.FM dataset, KGDC only improves AUC by 0.7% and improves F1 by 0.23%.

This is likely due to the relatively limited impact of knowledge supplementation on the Music dataset. We use the ratio of the number of KG triplets to the number of items to measure the average knowledge contribution to enrich item embedding. A higher ratio typically indicates richer semantics in the KG, enhancing item backgrounds. For the Music dataset, this ratio is 4.03, while the ratio of the Book dataset is 10.12. Thus KGDC can perform significantly better on the Book-Crossing dataset by effectively utilizing the rich semantics in these KGs for accurate predictions.

5.2. Discussion on Results of the Top-K Recommendation

For the Top-K recommendation task, KGDC achieves better results compared with the CTR task. Taking K = 20 as an example, KGDC improves Recall@20 and NDCG@20 by 8.418% and 6.762% compared to the state-of-the-art method on the Book-Crossing dataset; on the Last.FM dataset, it improves Recall@20 and NDCG@20 by 5.856% and 6.525%, respectively. We believe that the significant improvements are attributed to the extensive integration of contrastive learning techniques and pair-wise loss formulations in the algorithm. The introduction of contrastive learning and pair-wise loss have significantly improved the effectiveness of Top-K recommendation. Compared to traditional methods, introducing contrastive learning and pair-wise loss can achieve better results. It also demonstrates that contrastive learning can help the model to fully and effectively mine information when the supervision signals are insufficient. Meanwhile, the pair-wise loss function enhances the model’s learning of local ranking, which is particularly beneficial for Top-K recommendation scenarios.

As the value of K ranges, KGDC shows better performance compared to baselines, except NDCG@50 in Last.FM, and it demonstrates competitive performance in the evaluation of NDCG@K. By explicitly propagating interaction information between users, items, and entities, KGDC effectively learns latent representations of user preferences and item attraction patterns from CF and KG graphs. Additionally, instead of directly integrating the individual graph information, KGDC employs a collaborative guidance mechanism and an alignment mechanism to enhance the interaction between CF and KG. Moreover, KGDC adopts a multi-task framework to ensure the dominance of the supervision signals in CF. These results prove that KGDC has a significant advantage in Top-K recommendation.

In the Top-K recommendation task, BPRMF, as a traditional CF-based method, always performs better than knowledge graph-based CKE on both the Book-Crossing and Last.FM datasets. It also demonstrates that simply integrating KGs into recommendation systems does not always guarantee improved performance. Both the CF and KG graphs contain rich information, while not all information within KGs may contribute effectively to recommendations. Therefore, optimizing recommendation effectiveness requires making comprehensive and coherent use of CF and KG. This also indicates that the proposed KGDC can fully leverage the individual information and the interactive information.

Similar to CTR-based recommendations, KGDC shows greater improvement on the Book-Crossing dataset compared to the Last.FM dataset when K = 10, 20, 50, and 100. This further demonstrates that the proposed method can extract more valuable information from semantically rich knowledge graphs while maintaining the dominant role of CF information and preventing interference from irrelevant redundancies of KG.

5.3. Limitation

KGDC achieved good results on both CTR and Top-K recommendation tasks, but there are still noticeable problems. KGDC does not show stable improvement in the NDCG@K evaluation metric for the Top-K recommendation problem, and there is even no improvement in some cases. This means the model is unable to recommend the items that users like the most at the top of the list. We think this is because KGDC does not distinguish the different ratings of users in CF, leading to fluctuations in the more stringent NDCG performance, which evaluates the ranking order more strictly.

6. Conclusions and Future Work

In this paper, we propose a new method, KGDC, to tackle the aforementioned challenges. KGDC leverages GNN-based information propagation and aggregation techniques to effectively utilize the individual information from CF and KG, and also enhance the interaction of the two graphs by leveraging contrastive learning and multi-task learning. The extensive experiments show that KGDC significantly improves the recommendation performance over state-of-the-art methods on both tasks of CTR recommendation and Top-K prediction.

In this paper, we do not distinguish the different types of interactions between users and items, meaning that all connections in the CF graph are considered to have the same weight. However, in real-world scenarios, different behaviors of users towards items always represent varying attitudes. For example, a user’s preference for a movie rated 5 stars is higher than for one rated 1 star. Therefore, representing and learning different types of behavior in the CF graph can help the model distinguish and explore more nuanced user preferences, thereby improving the accuracy of recommendations. In future work, we will introduce the learning of different user behavior types in the CF graph during both independent and integrated learning processes in the dual-graph framework, further improving the accuracy of recommendations.