Cloud–Edge Collaborative Personalized Deployment of Knowledge Bases in Semantic Communications

Abstract

With the rapid evolution of next-generation mobile communications, semantic communication has emerged as an intelligent communication paradigm capable of surpassing the Shannon limit. A fundamental prerequisite for this paradigm is the synchronization of background knowledge between the transmitter and receiver, making the semantic knowledge base (SKB) a critical cornerstone. However, effectively selecting appropriate content from massive cloud-based knowledge repositories for edge deployment remains a significant challenge. This paper conducts systematic research to address the key issues in the flow deployment of SKBs at the edge, including insufficient adaptation to personalized preferences, inadequate timeliness management, and the complexity of multi-objective optimization. First, a comprehensive system model is constructed, integrating user preferences, knowledge relevance, transceiver matching degree, and the Age of Information (AOI). Second, the Generative Adversarial Network (GAN)-assisted Preference-based Reinforcement Learning (GaPbRL) algorithm is proposed. The experimental results demonstrate that this method outperforms traditional schemes in terms of knowledge-base hit rate, transceiver matching degree, and algorithm convergence speed, while significantly reducing the overhead of manual fine-tuning. This study provides a robust framework for the personalized and efficient cloud–edge collaborative deployment of SKBs.

1. Introduction

With the vigorous advancement of next-generation mobile communications, traditional communication systems based on Shannon’s information theory are increasingly encountering inherent performance bottlenecks. Semantic communication [1,2], as an innovative intelligent communication paradigm, transcends the constraints of bit-level transmission by prioritizing the delivery of the core information semantics over redundant raw data. By integrating communication with intelligence, this approach reshapes critical technical processes, including information extraction, semantic representation, encoding, decoding, and content reconstruction [3]. As a promising technology poised to break the Shannon limit, semantic communication has evolved into a key research direction in future network domains.

Currently, SKBs [4] have achieved significant progress in three technical dimensions, knowledge extraction, dynamic management, and performance optimization, thus overcoming the bottlenecks of traditional applications. In [5], the SKB serves as the fundamental cornerstone of efficient semantic communication, since effective semantic interaction requires that the sender and the receiver have consistent background knowledge. It encompasses structured entities, relational associations, and task-specific domain knowledge, providing critical support for semantic comprehension, logical reasoning, and accurate decoding throughout the communication process. Whether resolving semantic ambiguities or supplementing contextual semantic details, the rationality of its deployment and the effectiveness of its application directly determine the overall performance of the semantic communication system, including communication accuracy, timeliness, and quality of experience (QoE). In [6], a joint construction scheme that utilizes SKBs and large language models (LLMs) has demonstrated the ability to defend effectively against poisoning attacks.

Moreover, the performance of an SKB [7] is governed primarily by two core dimensions: completeness and preference responsiveness. Completeness refers to the extent to which the knowledge base covers semantic information relevant to the task, including the richness of entity sets, the accuracy of relational descriptions, and the comprehensiveness of attribute information. Incomplete knowledge bases often induce semantic ambiguity or information loss, thereby undermining reliability. In contrast, preference responsiveness reflects the knowledge base’s capacity to adapt to individual user demands and dynamic application scenarios. Different users exhibit divergent backgrounds and usage preferences that may evolve dynamically over time. A knowledge base incapable of accommodating these personalized and time-varying preferences will inevitably lead to mismatches between communication content and user needs [8]. Balancing completeness with flexible preference responsiveness constitutes a key technical challenge in current research. In [9], LLMs are leveraged to construct a novel communication framework. The rise of LLMs provides core technical support for the construction and optimization of SKBs. As artificial intelligence models with massive parameters and powerful semantic understanding capabilities, LLMs can automatically extract structured entities, relationships, and task-related knowledge from large-scale unstructured data, efficiently complementing semantic associations while significantly reducing manual construction costs. Meanwhile, LLMs possess excellent contextual reasoning and preference mining capabilities, enabling them to dynamically identify personalized needs from interaction data and provide accurate references to optimize responsiveness [10]. Their generative capabilities also assist in resolving semantic ambiguities and supplementing context-dependent details—for instance, generating multi-dimensional interpretations for ambiguous expressions to align the background knowledge of communication parties. Furthermore, LLMs can be deeply integrated with dynamic deployment mechanisms: by analyzing the timeliness of knowledge elements and shifting user needs in real time, they offer intelligent decision support for AoI-driven update and replacement strategies, thereby further enhancing the freshness and adaptability of edge-side SKBs.

To address the two main issues in the deployment of SKBs [11]—insufficient personalized preference matching and a lack of knowledge timeliness management—the AoI is integrated into the proposed dynamic deployment framework. As a key metric for quantifying the freshness of the information, AoI is defined as the time elapsed since the generation of the knowledge-base data. Traditional static deployment methods typically optimize only for cache hit rates, ignoring heterogeneous user preferences, causal correlations between knowledge entities, and the degree of matching between the sender and receiver, which is prone to issues such as outdated knowledge and semantic ambiguity. In contrast, the AoI-incorporated dynamic deployment method adjusts strategies for retaining, deleting, and replacing edge-side knowledge by monitoring the timeliness of elements in real time. Adapting to personalized needs ensures high knowledge freshness, effectively mitigates semantic deviations caused by outdated information, and ultimately improves the comprehensive performance of the system.

SKBs exhibit prominent multi-layered and multi-preference characteristics in practical applications [12]. The multi-layered nature is reflected in the classification of knowledge into shared public knowledge and personalized private knowledge. The multi-preference characteristic manifests itself as the diverse and evolving interests of different users [13]. These inherent characteristics give rise to an intrinsic trade-off between SKB scale and operational efficiency: expanding the scale improves completeness but imposes excessive storage burdens on edge devices and reduces query and update efficiency. In contrast, reducing the scale to optimize efficiency may result in insufficient coverage of key knowledge [14]. Balancing this trade-off to achieve efficient operation under constrained edge resources has become an urgent technical issue.

• The SKB edge deployment must simultaneously satisfy four core requirements with inherent trade-offs: (1) adaptation of personalized preferences; (2) semantic consistency; (3) information timeliness; (4) constraints on edge resources. Traditional edge caching methods mostly focus on a single objective and lack systematic design for multi-objective optimization, making it difficult to achieve a dynamic balance among these factors.
• The reward engineering needs to integrate multi-dimensional indicators, e.g., hit rate, matching degree, and AoI, where manually quantifying weights is difficult and prone to reward hacking. However, implementing reinforcement learning from human feedback (RLHF) requires a large volume of expert annotations for trajectory preferences, which involves high costs, low efficiency, and limited generalization. This makes it difficult to adapt to universal semantic communication scenarios.

Compared with standard deep reinforcement learning and vanilla PbRL methods, the distinct technical contributions of this work are summarized as follows:

• We introduce a GAN-driven preference expansion mechanism that encapsulates human-expert evaluation logic into a stable mathematical distribution. By substituting active human labeling with simulated adversarial evaluations, our framework achieves data-driven completeness and breaks the scalability bottleneck of traditional preference learning in dynamic communication networks.
• Unlike conventional edge caching that focuses entirely on data-level hit rates, our framework jointly optimizes semantic comprehension alignment (transceiver matching degree) and information staleness (AoI) under limited edge storage capacities.

The structure of this paper is as follows. First, in Section 2, we introduce the system model for collaborative deployment of SKBs on the cloud edge, including the representation form and the definition of key metrics. Next, we formulate the optimization problem, clarifying the objective function and constraints. Subsequently, in Section 3, we elaborate on the detailed design of the GaPbRL method, covering MDP modeling, GAN-assisted preference expansion, and policy learning. In Section 4, we present simulation experiments that verify the superiority of the proposed method. Finally, in Section 5, we summarize the core research findings.

2. System Model

2.1. Scenario

A significant distinction between semantic communication systems and conventional communication systems is that semantic communication systems require an SKB that conforms to the personalized preferences of communication users. This section first introduces the performance gains brought about by knowledge bases in semantic communication systems. At the transmitting end, the semantic information S is encoded in a signal X. After transmission through the channel, the received signal Y is obtained, from which the receiver decodes the estimated semantic information $\widehat{S}$. Defining the decoding precision as $F(S,\widehat{S})$, mutual information as $I(S;Y)$, and the knowledge base as K, from the perspective of information theory the literature [14,15] demonstrates that

$$\begin{array}{c}\hfill F(S,\widehat{S}\mid K)\ge F(S,\widehat{S}),\end{array}$$

(1)

$$\begin{array}{c}\hfill I(S;Y\mid K)\le I(S;Y).\end{array}$$

(2)

These expressions imply that an SKB-assisted system improves the accuracy of encoding and decoding model training while reducing transmission overhead [16]. To theoretically formalize the performance gain of limiting the information age, let $F(S,\widehat{S}\mid K)$ denote the decoding precision conditioned on the deployed knowledge base K. Because semantic information fields drift dynamically over time, the real-world utility and accuracy of edge-cached knowledge decline as a function of the elapsed time since its generation in the cloud. Let $\Delta f\left(t\right)=f_n(t+1)-f_n\left(t\right)$ be the variation in knowledge age. The instantaneous semantic hit rate $\eta \left(t\right)$ degrades according to an exponential decay model, formulated as

$$\Delta \eta \left(t\right)=k_3{e}^{\Delta f\left(t\right)}+k_4,$$

(3)

where $k_3<0$ is a negative scaling constant determined by the dynamic volatility of the specific knowledge category. This formulation guarantees that constraining the average age to $f_n\left(t\right)\le \lambda$ mathematically minimizes the semantic ambiguity boundary and protects the transceiver matching degree $C\left(t\right)$ from sudden drop-offs. However, maintaining a singular centralized SKB is prohibitively expensive, and directly deploying large-scale centralized knowledge bases to the edge is similarly impractical. Taking into account the graph-structured characteristics of semantic data [17], this paper focuses on introducing how the SKB is collaboratively deployed at the edge and in the cloud [18].

Figure 1 illustrates the cloud–edge collaborative architecture for semantic communications. The specific knowledge sub-bases cached on the edge server are collectively referred to as edge sub-bases, whereas the cloud-side repository is defined as the cloud knowledge master base [19]. Due to the limited storage capacity of edge servers, the content of edge sub-bases is selectively extracted from the master base and organized by categories. Semantic codecs can be trained specifically according to distinct semantic knowledge categories [20]. The edge sub-base is assumed to support up to N distinct categories [21]. A buffer zone is structurally established at the edge server–node interface. The primary function of the buffer zone is to serve as an asynchronous staging cache that temporarily accommodates incoming knowledge chunks streamed dynamically from the cloud master repository during cache misses or updates. By acting as a traffic smoothing layer, it buffers variations in transmission throughput and prevents cloud–edge synchronization latencies from disrupting the local, real-time semantic codec pipelines executing on edge devices.

Figure 1. SKBs in cloud–edge collaboration for semantic communications.

During initialization, the edge server bootstraps its local semantic cache by selectively loading knowledge sub-bases that simultaneously align with historical user preferences and the factual relevance of anticipated data requests, subject to the available storage budget. Whenever a mobile device issues a semantic-based understanding request [22], in [23] the server queries its resident knowledge base. In the event of a cache failure, the server streams the required segment from the cloud and updates the local repository atomically to ensure seamless downstream inference.

The system continuously monitors observable metrics, including user preferences, causal relationships, sender–receiver SKB affinity, and instantaneous cache occupancy. These metrics are utilized to recalibrate the cache at the start of every time slot. Oriented by the matching affinity, the remaining capacity, and the AoI of each data chunk, the edge controller decides which semantic partitions to retain, evict, or reuse, thus formulating the SKB caching problem as a constrained Markov decision process (CMDP) [24].

In this paper, we assume that each user device corresponds to an edge server. The knowledge base on the cloud end is defined as $G_{\mathrm{total}}=\{V\left(G_{\mathrm{total}}\right),E\left(G_{\mathrm{total}}\right),L\}$, where the knowledge base stored on the sending edge server is represented by $G_s=\{V\left(G_s\right),E\left(G_s\right),L_s\}$; and the knowledge base stored on the receiving edge server is $G_j=\{V\left(G_j\right),E\left(G_j\right),L_j\}$. The relationship between the subsets g and the total set G is $g_1\cup g_2\cup \dots \cup g_N=G$. $V\left(G\right)$ represents the set of entities, including source entities and target entities, referring to entities or concepts of the real world, and $E\left(G\right)$ represents the relationships between different entities. $L\left(G\right)$ represents the label of the knowledge base, which is automatically generated when the knowledge base is extracted [25]. The label consists of multiple components, primarily including the category and the data volume of the knowledge base. For a given category, the extraction degree of the master knowledge base at the cloud server is maintained at 100%. The timeliness requirement of the knowledge base of this category, quantified as the value of the AoI weight w, and a current timestamp are generated when transmitted to the edge server to record the AoI.

In this paper, we define user preferences as related to the frequency of the user’s requests for corresponding categories of knowledge bases. Each user’s preference can be represented by a preference label $P=[p_1,p_2,\dots ,p_N]$, where the value of each element represents the user’s degree of preference for the content feature label, as the probability of being ranked at the top. It is assumed that the probability of a user’s preference for a certain category of content follows a Zipf distribution with parameter $\varphi$ [26], and its calculation can be expressed as

$$\begin{array}{c}\hfill p_n\left(t\right)={\displaystyle \frac{{\left[y_n\left(t\right)\right]}^{-\varphi }}{{\sum }_{n\in \mathbb{N}}{\left[y_n\left(t\right)\right]}^{-\varphi }}},\varphi \ge 0,\end{array}$$

(4)

where n represents the descending rank of the access frequency of the knowledge base of the n-th category at time t, and $\varphi$ is a real constant used to control the degree of aggregation of the interest of the knowledge base [27]. A larger $\varphi$ indicates that the knowledge base with a higher rank has a higher request frequency.

We define a relevance label to determine the causal relationship of the content browsed by the user. The relevance label is expressed as $q=(q_1,q_2,\dots ,q_h)$. The causal relevance between two categories of knowledge bases is expressed using cosine similarity. We assume $I=(I_1,I_2,\dots ,I_N)$ represents the content of various knowledge sub-bases stored on the edge server, where I is an N-dimensional vector, with each dimension representing a knowledge-base category. The relevance between category m and category n is expressed as

$$\begin{array}{c}\hfill q_t(m,n)={\displaystyle \frac{I_m\left(t\right)·I_n\left(t\right)}{\parallel I_m\left(t\right)\parallel \parallel I_n\left(t\right)\parallel }}.\end{array}$$

(5)

Consequently, the knowledge-base hit rate at time t, $\eta \left(t\right)$, is defined as

$$\begin{array}{c}\hfill \eta \left(t\right)=k_1\sum _n{p}_n\left(t\right)+k_2\sum _m\sum _n{q}_t(m,n).\end{array}$$

(6)

Furthermore, the degree of matching of the knowledge base $C\left(t\right)$ is defined as the sum of the ratios of the degrees of extraction M of the corresponding categories of knowledge bases [28] extracted by the receiver edge server from the cloud and is expressed as

$$\begin{array}{c}\hfill C\left(t\right)={\displaystyle \sum _{n=1}^N}{\displaystyle \frac{M_n^r\left(t\right)}{M_n^s\left(t\right)}}.\end{array}$$

(7)

Considering that the receiver optimizes extraction based on private parameters, the degree of extraction $M_n^r\left(t\right)$ is typically smaller than the degree of the sender $M_n^s\left(t\right)$. To quantify the efficiency of an SKB over time, we employ the AoI, $f\left(t\right)$:

$$\begin{array}{c}\hfill f\left(t\right)=t-t_0,\end{array}$$

(8)

where t is the current time and $t_0$ is the generation timestamp in the cloud. To ensure data freshness, an AoI threshold $\lambda$ is defined [29]. Sub-bases exceeding this threshold are updated, triggering operational adjustments such as modified extraction degrees. Importantly, for communication services, AoI requirements may deviate from conventional definitions; while edge interfaces require dynamic adjustment, general-purpose knowledge may not necessitate extreme freshness. However, personalized sub-bases tailored to user preferences are highly dynamic and require a low AoI to maintain system efficiency [30].

In this paper, the maximum value of the AoI knowledge base is assumed to be $f_n\left(t\right)\in \{1,2,\dots ,\lambda \}$. When $f_n\left(t\right)$ exceeds the threshold, it indicates that the knowledge base stored in the edge server’s n-th category is inefficient, which may result in generating ambiguous meanings. The equation can be expressed as follows:

$$\begin{array}{c}{f}_n(t+1)=\left\{\begin{array}{cc} 1,\hfill & a_n\left(t\right)=A_t,\hfill \\ \min \left\{\lambda ,f_n\left(t\right)+1\right\},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\hfill \end{array}$$

(9)

where $a_n\left(t\right)$ represents the action of the intelligent body in the knowledge base of the n-th category at time t, and $a_n\left(t\right)=A_t$ indicates that the intelligent body is performing action $A_t$ in the knowledge base of the n-th category at time t. Under the operation of strategy $\pi$, starting from the initial state $s\left(0\right)$, the long-term average AoI of the knowledge base of the n-th category is expressed as

$$\begin{array}{c}\hfill \overline{f_n\left(t\right)}\triangleq \underset{d\to \infty }{\mathrm{lim\; sup}}{\displaystyle \frac{1}{d+1}}{\displaystyle \sum _{t=0}^d}\left[f_n\left(t\right)\mid s\left(0\right)\right].\end{array}$$

(10)

Therefore, the upper limit for the AoI is represented by $\overline{f_n\left(t\right)}\le \sigma$ and ${\sum }_n{w}_n\overline{f_n\left(t\right)}\le \psi$, where $\sigma$ represents the average value of the AoI of each knowledge base. $w_n$ represents the weight of the n-th knowledge base, and $w_n\ge 0$, with ${\sum }_n{w}_n=1$. The weight can be assigned by the edge server based on the importance of the different categories of the knowledge base [31]. When focusing on the efficiency of knowledge bases, a lower AoI weight is assigned to commonly used knowledge bases, while higher AoI weights are assigned to more dynamic, personalized knowledge bases that require more attention to be given to their efficiency.

2.2. Problem Formulation

Taking into account the constraints of cache capacity and AoI, the objective function of this paper is given as follows:

$$\begin{array}{cc}\hfill \max \hspace{1em}& \underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle \sum _{t=0}^{T-1}}\left[\alpha C\left(t\right)+\beta \eta \left(t\right)\right]\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \mathrm{C}1:\sum _n\left[x_n\left(t\right)D_n\left(t\right)\right]\le S,\hfill \\ \hfill & \mathrm{C}2:\sum _n{x}_n\left(t\right)\le N,\hfill \\ \hfill & \mathrm{C}3:x_n\left(t\right)\in \{0,1\},\forall n\in \{1,2,\dots ,N\},\hfill \\ \hfill & \mathrm{C}4:0\le M_n^r\left(t\right)\le M_n^s\left(t\right)\le 1,\hfill \\ \hfill & \mathrm{C}5:\overline{f_n\left(t\right)}\le \sigma ,\hfill \\ \hfill & \mathrm{C}6:\sum _n{w}_n\overline{f_n\left(t\right)}\le \psi ,\hfill \\ \hfill & \mathrm{C}7:\sum _n{w}_n=1,\forall w_n\ge 0,\hfill \\ \hfill & \mathrm{C}8:0\le f_n\left(t\right)\le \lambda ,\forall f_n\left(t\right)\in \mathbb{Z},\hfill \\ \hfill & \mathrm{C}9:\alpha +\beta =1,\alpha >0,\beta >0.\hfill \end{array}$$

(11)

$x_n\left(t\right)$ is a control variable, where if $x_n\left(t\right)=1$ it indicates that the corresponding knowledge base is cached, and if not, it is not cached. $C1$ indicates that the knowledge-base data stored on the edge server do not exceed S. $C2$ indicates that the knowledge-base category stored on the edge server does not exceed N. $C3$ is used to judge whether the knowledge base is cached. $C4$ limits the knowledge-base extraction process. $C5$ restricts the AoI of each knowledge base, and the average AoI in the knowledge-base extraction process cannot exceed $\sigma$. If the AoI is too large, it will lead to inefficient network usage. $C6$ represents the weight limit of each knowledge base. $C7$ is the weight limit for each knowledge base and $C8$ limits the rights of each knowledge base. $C9$ is the constraint of the objective function.

The deterministic constraints C1 to C9 mapped from the physical edge architecture are managed via a two-layer safety mechanism within the DRL agent. Hard physical storage limits (C1, C2) are enforced by an action-validation filter: if the proposed action $a_t$ causes ${\sum }_n{x}_n\left(t\right)D_n\left(t\right)>S,$ the environment rejects the transition, inflicts a structural negative step penalty, and rolls back the agent’s state to re-sample valid actions. Time-freshness boundaries (C5, C6) are explicitly integrated into the CMDP state transition rules, where sub-bases exceeding the threshold $\lambda$ are dynamically flagged for high-priority eviction.

The objective function reflects that the AoI in the knowledge-base cache problem [32] is constrained by the decision-making process. The smaller the AoI, the better the efficiency of the knowledge base, which can be expressed as the change in the AoI that affects the rate of the knowledge base in the network $\Delta \eta \left(t\right)=k_3{e}^{\Delta f\left(t\right)}+k_4$. The time-varying behavior of the edge server is modeled by the change in data storage as $\Delta C\left(t\right)=k_5\Delta {\sum }_n\left[x_n\left(t\right)D_n\left(t\right)\right]+k_6$, where $D_1,D_2,\dots ,D_n$ are the data storage amounts of different knowledge categories. Meanwhile, we prove that the optimization problem in this paper is NP-hard in Appendix A.

3. GAN-Assisted Preference Reinforcement Learning for SKB Deployment

3.1. Markov Decision Process on SKB Deployment

Given that the optimization problem was proven to be NP-hard in Section 2, deep reinforcement learning (DRL) is used to find the optimal solution. At each time step, the SKB controller determines the currently cached content and evaluates newly acquired content based on the state. We explore an optimal strategy to achieve efficient SKB caching within specific storage constraints [33], aiming to maximize the degree of matching between the receiver and sender knowledge bases. First, the Markov decision process (MDP) and the system state space S are defined, where the state at time t is denoted as $s_t=G_t=\{g_1,g_2,\dots ,g_n\}$. The system’s state space encompasses the knowledge content G, while the action space A is characterized by an initial state distribution $\mu$ and a bias rate $\rho$. The following sections provide a comprehensive explanation of these components [34].

The state space S represents the content of the SKB G. The state at time t is represented as $s_t=G_t=\{g_1,g_2,\dots ,g_n\}$, which indicates the content of the knowledge base stored on the edge server. From the knowledge content $G_t$, we can read the knowledge signature L, such as the usage of the knowledge base ${\sum }_n{x}_n\left(t\right)D_n\left(t\right)$, the category of knowledge ${\sum }_n{x}_n\left(t\right)$, the moment $t_0$, the AoI weight $w_n$, and the degree of extraction $M_n\left(t\right)$, which is represented as $L={\sum }_n{x}_n\left(t\right)D_n\left(t\right),{\sum }_n{x}_n\left(t\right),t_0,w_n,M_n\left(t\right)$. $t_0$ is the time when the knowledge is transferred to the edge server, used to calculate the AoI of the n-th SKB category [35].

The action space $A=\{a_1,a_2,\dots ,a_n\}$ corresponds to the set of sub-bases $\{g_1,g_2,\dots ,g_n\}$. Each action a is determined by the controller by evaluating the degree of matching between transceivers in conjunction with storage and efficiency constraints. The controller dynamically adjusts the degree of extraction of each SKB. When storage blocks are removed while maintaining the same extraction degree, the system can effectively select newly extracted knowledge for replacement. The controller first observes historical cloud-based requests for different SKB categories. The initial probability distribution model is given by $p_n\left(t\right)={\displaystyle \frac{{\left[y_n\left(t\right)\right]}^{\phi }}{{\sum }_{n\in N}{\left[y_n\left(t\right)\right]}^{\phi }}},$ where the categories are ranked according to the user’s preference [36]. The edge device randomly samples content from the cloud to obtain the initial state distribution $\mu \left(s\right)$. In preference reinforcement learning, a sampled trajectory can be expressed as $\tau =\{s_0,a_0,s_1,a_1,\dots ,s_{n-1},a_{n-1},s_n\}$ [37]. We define ${\tau }_i\succ {\tau }_j$ as meaning that human experts prefer the trajectory ${\tau }_i$ over the trajectory ${\tau }_j$. If ${\tau }_i\succ {\tau }_j$, these two trajectories are stored in the SKB preference set (KBPS), $\zeta$, in the form of the triple $({\tau }_i,{\tau }_j,1)$ [38]. ${\tau }_i\sim {\tau }_j$ indicates that human experts cannot distinguish the preference between these two trajectories. If ${\tau }_i\sim {\tau }_j$, the two trajectories are stored in $\zeta$ as the triple $({\tau }_i,{\tau }_j,0.5)$. The preference probability $\rho ({\tau }_i\succ {\tau }_j)$ is defined as the probability that ${\tau }_i\succ {\tau }_j$ given the trajectories $({\tau }_i,{\tau }_j)$.

3.2. Personalized Bias Reinforcement Learning with GAN

This paper incorporates a generative adversarial network (GAN) to assist the preference-based learning method (PbRL) in determining the weighting parameters $\alpha$ and $\beta$ in the objective function. Subsequently, deep Q-networks (DQNs) are utilized to solve the NP-hard problem. By integrating a normal distribution and expert-guided strategies, the model achieves personalized cloud–edge collaboration [39].

The general framework of the algorithm in this paper is shown in Figure 2. The agent selects trajectories from the environment, which are then compared by human experts. To simulate the subjective behavior of human preference, the system processes the set of preferences $\zeta$ to train a neural network that emulates human decision-making, producing the set of GAN-enhanced biases $\Omega$. The PbRL network integrates $\zeta$ and $\Omega$ to train a reward function $Q^{*}(C,\eta )$, and then a Q function via the DQN algorithm of the Q network, while the DQN algorithm optimizes the policy for the optimal SKB placement strategy. The rationale for adopting a DQN rather than continuous control algorithms like PPO or SAC lies in the inherently discrete nature of the SKB caching action space (retention, deletion, replacement). Furthermore, the weighting coefficients ($\alpha$ and $\beta$) were selected via grid search to optimally balance the cache efficiency and matching degree. To validate the robustness of these hyperparameters, a sensitivity analysis was conducted, demonstrating that the system performance remains stable within a $\pm 10\%$ fluctuation of the chosen weights. In Figure 2, the GaPbRL algorithm for the deployment of the collaborative SKB on the cloud edge is shown. The GaPbRL algorithm uses three distinct components of the neural network: the GAN, the PbRL reward generation network, and the DQN [40]. Their interaction follows these procedural rules:

Figure 2. Diagram of the GaPbRL algorithm for SKB cloud–edge collaborative deployment.

• Generate the current strategy $\pi$ for the adversarial network to generate the trajectory $({\tau }_n,{\tau }_m)$.
• Update the adversarial network and then map it to the GAN-assisted bias set $\Omega$, where $\Omega$ is the set of {1, 2}, representing the trajectory preference.
• Train the reward model to predict the Q value of human preference.
• Update the parameters using the DQN algorithm to obtain the new control strategy and maximize the Q value of human preference. This step proceeds with three iterations [41] to stabilize the results generated by the neural networks, i.e., the continuous difference between two returns is less than a threshold. The detailed implementation of the GaPbRL algorithm is shown in Algorithm 1.

Algorithm 1 GaPbRL Algorithm for Cloud–Edge Collaborative Deployment of semantic knowledge bases

1. Initialization - Initial Zipf distribution model: $p_n\left(t\right)={\displaystyle \frac{{\left[y_n\left(t\right)\right]}^{-\phi }}{{\sum }_{n\in \mathbb{N}}{\left[y_n\left(t\right)\right]}^{-\phi }}}\to \mu \left(s\right)=\{G_{s,0},G_{j,0}\}$ - Satisfy the optimization problems from C1 to C4. 2. Train  $\alpha$  and  $\beta$  in the reward function - Sample: Select trajectories ${\tau }^{{\pi }_i},{\tau }^{{\pi }_j}$ from policies ${\pi }_i,{\pi }_j$. - Query: Label preferences $y\in \{0,1\}$ via human expertise. - Update GAN: ${\theta }_{di}\leftarrow {\theta }_{di}+{\nabla }_{{\theta }_{di}}{E}_{{\tau }_i\in {\Omega }_{np}}\log (1-Y_{di}\left({\tau }_i\right))+{\nabla }_{{\theta }_{di}}{E}_{{\tau }_i\in \Omega }\log Y_{di}\left({\tau }_i\right)$. - Optimize: Compute losses $L_{\alpha },L_g$ and update coefficients $\alpha ,\beta$. 3. Execute DQN algorithm     Use the $ϵ$-greedy strategy to decide the action a as $$a=\left\{\begin{array}{cc}\arg \max Q(s,a),\hfill & P=1-ϵ\hfill \\ \mathrm{random},\hfill & \mathrm{other}.\hfill \end{array}\right.$$ - If ${\sum }_n\left[x_n\left(t\right)D_n\left(t\right)\right]\le S$ and ${\sum }_n{w}_n{f}_n\left(t\right)\le \psi$, then calculate: $\left[\alpha C\right(t)+\beta \eta (t\left)\right]$ else, go back one step and re-decide action a. end if - While $\left[\alpha C\right(t)+\beta \eta (t\left)\right]$ is less than the revenue or the iteration count is greater than 200     do output the maximum reward value $\max \left[\alpha C\right(t)+\beta \eta (t\left)\right]$ and the optimal strategy $\arg {\max }_{\pi }\sum \mathrm{Pr}(\pi \to {\pi }^{\prime })$

This paper defines trajectory preferences in the following three ways:

(1)

The sampled knowledge bases of both the transmitter and the receiver are provided to human experts for comparison. The human experts return a binary feedback indicating whether ${\tau }_m\succ {\tau }_n$. If the human experts cannot compare the results, this round is skipped, and the trajectory pair is not included in the KBPS.

(2)

The adversarial neural network [42] returns a binary feedback based on existing human behavior to determine whether ${\tau }_m\succ {\tau }_n$. Since the GAN is two-dimensional, a trajectory pair preference must be distinguished.

(3)

Based on the generated reward function $Q:S\times A\to Q$ (the reward function in this paper takes the form $Q=\alpha C\left(t\right)+\beta \eta \left(t\right)$, where generating the reward function refers to generating $\alpha$ and $\beta$), if the environmental feedback satisfies ${\sum }_{i=1}^k{\gamma }^{i-1}Q(s_{t+i}^m,a_{t+i}^m)>{\sum }_{i=1}^k{\gamma }^{i-1}Q(s_{t+i}^n,a_{t+i}^n)$, then ${\tau }_m\succ {\tau }_n$.

The objective of GAN training is to develop a model that effectively simulates the logic of human preference [43], thus reducing the dependency on human experts. Furthermore, the GAN can correct erroneous human decisions caused by subjective factors. We train a discriminative neural network model $Y_{di}$, with its model parameters as ${\theta }_{di}$. This model learns the probability of selecting the best trajectory from the set of bias trajectories $\Omega$. ${\Omega }_{np}$ represents the set of preferred trajectories and ${\Omega }_{all}$ represents all selected trajectories. The model parameter ${\theta }_{di}$ can be updated by using the stochastic gradient ascent algorithm: ${\theta }_{di}\leftarrow {\theta }_{di}+{\nabla }_{{\theta }_{di}}{E}_{{\tau }_j\in {\Omega }_{np}}\log (1-Y_{di}\left({\tau }_i\right))+{\nabla }_{{\theta }_{di}}{E}_{{\tau }_j\in {\Omega }_{np}}\log Y_{di}\left({\tau }_j\right).$

The training process of the reward function is represented. We use human experts’ assistance and the GAN-assisted PbRL method to generate the reward function. First, human experts use the evaluation criteria to perform trajectory selection. Then, the selected bias data is used to train the adversarial neural network. If the adversarial network can correctly simulate the behavior of a human expert, it can be used as input for PbRL.

Then, we use the Q-function as the evaluation bias correctness indicator, where $Q(s,a_t)={\sum }_i{\gamma }^{i-1}r(s_{t+i},a_{t+i})$, assuming the bias preference, where Q increases as the function grows and is represented as

$$\begin{array}{c}\hfill \widehat{\rho }({\tau }_1\to {\tau }_2)={\displaystyle \frac{1}{1+\exp [Q(s_t^2,a_t^2)-Q(s_t^1,a_t^1)]}}.\end{array}$$

(12)

Here, $\widehat{\rho }({\tau }_1\to {\tau }_2)$ refers to the predicted preference to select the trajectory ${\tau }_1$ instead of ${\tau }_2$ after the intelligent agent computes the value Q. This trajectory is stored in the GAN-enhanced bias set of the form $({\tau }_1,{\tau }_2,x=1)$. Then, we update the Q-value function and minimizes human bias in the Q-function based on the cross-entropy loss $L_a$, as shown below:

$$\begin{array}{c}\hfill L_a=-\sum _{({\tau }_1,{\tau }_2,\zeta )\in \Omega }\left[x\left(1\right)\log \widehat{\rho }({\tau }_1\to {\tau }_2)+x\left(2\right)\log \widehat{\rho }({\tau }_2\to {\tau }_1)\right].\end{array}$$

(13)

In the adversarial neural network, $\rho ({\tau }_1\to {\tau }_2)=1-\rho ({\tau }_2\to {\tau }_1)$ generally holds, because the two endpoints are selected independently of the adversarial neural network, expressed as

$$\begin{array}{c}\hfill \rho ({\tau }_1\to {\tau }_2)=Y_{di}\left({\tau }_1\right),\end{array}$$

(14)

$$\begin{array}{c}\hfill \rho ({\tau }_2\to {\tau }_1)=Y_{di}\left({\tau }_2\right).\end{array}$$

(15)

Therefore, to better simulate human behavior, we also minimize the cross-entropy loss between the bias preference based on the Q-function and the GAN’s bias preference, which is represented by the auxiliary bias loss $L_g$:

$$\begin{array}{cc}\hfill L_g=-\sum _{({\tau }_1,{\tau }_2,x)\in \Omega }& \left[\rho ({\tau }_1\succ {\tau }_2)\log \widehat{\rho }({\tau }_1\succ {\tau }_2)+\left(1-\rho ({\tau }_1\succ {\tau }_2)\right)\log \widehat{\rho }({\tau }_2\succ {\tau }_1)\right]\hfill \\ \hfill -\sum _{({\tau }_1,{\tau }_2,x)\in \Omega }& \left[\rho ({\tau }_2\succ {\tau }_1)\log \widehat{\rho }({\tau }_2\succ {\tau }_1)+\left(1-\rho ({\tau }_2\succ {\tau }_1)\right)\log \widehat{\rho }({\tau }_1\succ {\tau }_2)\right].\hfill \end{array}$$

(16)

In this way, it can be predicted that the preference for selection based on the Q-function or the preference obtained through GAN will be higher. A higher preference score corresponds to a higher reward value.

Using GaPbRL, the reward function $Q=\alpha C\left(t\right)+\beta \eta \left(t\right)$ is obtained with the parameters $(\alpha ,\beta )$, and the optimization problem becomes the dynamic storage optimization problem of the knowledge base. The action space includes the storage, deletion, replacement of knowledge blocks, and changes in the degree of extraction in the environment [44]. In this paper, actions are performed to sample the extraction process (ensuring that the extraction degree is an integer). We use an $ϵ$-greedy strategy to determine the actions, which can be written as

$$\begin{array}{c}\hfill a=\left\{\begin{array}{cc}\arg \max Q(s,a),\hfill & P=1-ϵ\hfill \\ \mathrm{random},\hfill & \mathrm{other}.\hfill \end{array}\right.\end{array}$$

(17)

We use deep Q-networks to solve this optimization problem. First, the intelligent agent obtains the initial state distribution $\mu \left(s\right)$; next, the intelligent agent receives the state parameters $s_t=\{G_t,W_t\}$ at time t, and the state parameters at the next time step. Due to storage capacity S, category volume N, and constraint of AoI, different strategies are required to modify the degree of knowledge extraction in $G_t$.

Under the assumption that the SKB deployment process is modeled as a finite constrained Markov decision process (CMDP), and the reward function $Q=\alpha C\left(t\right)+\beta \eta \left(t\right)$ is bounded, the proposed GaPbRL framework inherits the convergence properties of deep Q-networks (DQNs) under experience replay and target network stabilization. Specifically, the state space is finite with a limited number N of SKB categories; the action space is discrete, including cache, replace, and extract operations; and the reward is bounded where both $C\left(t\right)$ and $\eta \left(t\right)$ fall within the interval $[0,1]$. Therefore, the Q-function update in GaPbRL converges to a fixed point with probability 1 under standard stochastic approximation conditions.

The training stability of GaPbRL is ensured by two mechanisms: the target network and experience replay in the DQN, which reduce temporal correlation in updates; and the bounded adversarial loss in the GAN-based preference modeling, which prevents reward explosion. Since both $C\left(t\right)$ and $\eta \left(t\right)$ are bounded and normalized, the reward signal remains stable during training.

4. Simulation Experiments

Due to the knowledge graph-like characteristics of the SKB, most studies on semantic communication systems use knowledge graph-based knowledge bases. We use NELL-995, a large-scale knowledge base based on a knowledge graph, to evaluate the SKB deployment scheme proposed in this paper. NELL-995 consists of 754,920 entities and 200 types of relationships. In this paper, these entity–relation pairs are divided into 57 categories, corresponding to 57 types of knowledge base, with each category containing approximately 15 entity–relation pairs. The simulation assumes that all 867 entity–relation pairs can be retrieved from the cloud, and that the edge server can store at most 10 different categories of knowledge bases (setting $N=10$). The degree of extraction M of each knowledge base is a continuous value in the interval $[0,1]$, which follows a Pareto distribution with parameter $\phi =1.3$. The parameters are set as $k_1=0.75$, $k_2=0.25$, $k_3=-0.00054$, $k_4=0.67$, $k_5=-0.01$, $k_6=1.2$, and $\epsilon =0.05$. The simulations are implemented based on the TensorFlow open-source platform and are conducted on a workstation equipped with an Intel(R) Core(TM) i5-12400F CPU @ 2.50 GHz, 16.0 GB RAM @ 3200 MHz, a 1 TB SSD, and an NVIDIA GeForce RTX 4080 SUPER GPU. In this paper, the definition of the average reward is given as follows:

$$\begin{array}{c}\hfill \mathrm{average}\text{\_}\mathrm{reward}\left(X\right)={\displaystyle \frac{{\displaystyle {\displaystyle \sum _{x=1}^X}\mathrm{reward}\left(x\right)}}{X}}.\end{array}$$

(18)

where X is the number of training rounds.

Figure 3 illustrates a comparison of the average reward versus training episodes between GaPbRL-generated and conventionally designed reward functions. The average AoI threshold for each knowledge base is set to $\sigma =15\mathrm{s}$, and the threshold for the weighted sum of the average AoI in all knowledge bases is set to $\Psi =200\mathrm{s}$. The instantaneous AoI threshold for each knowledge base is set to $\lambda =20\mathrm{s}$. The storage capacity of the edge server is set to $S=120\mathrm{GB}$. The number of comparison trajectories is set to 50 for manual comparison and 50 for GAN-based comparison. The weighting coefficients of the output of the objective function of the PbRL reward function generation network are set to $\alpha =0.467$ and $\beta =0.533$ (red curve). After multiple rounds of fine-tuning by human experts, the reward functions for the DQN algorithm are set to $r=0.4C+0.6\eta$ (purple curve) and $r=0.48C+0.52\eta$ (blue curve). It can be observed that when the GaPbRL-generated reward function is applied to the DQN algorithm, a clear convergence trend appears around the 60th and 80th training episodes, while the two reward functions manually designed by human experts exhibit clear convergence only around the 100th and 120th training episodes, respectively. It can also be observed that, in most training episodes, the average reward obtained using the GaPbRL-generated reward function is higher than the average weighted reward achieved by traditional manual-designed reward functions. From these observations, it can be concluded that the GaPbRL method proposed in this paper converges faster and yields a higher weighted sum of the knowledge-base hit rate and the degree of matching in the final output policy. The reason is that it is difficult for humans to design a reward function that is fully suitable for the DQN algorithm, and manual design typically relies on repeated fine-tuning and experimental trials. In contrast, the GaPbRL method can learn human preferences through the knowledge-base preference set $\zeta$ and the GAN-assisted preference set $\Omega$, allowing the design of more accurate reward functions.

Figure 3. Comparison of average reward versus training episodes between GaPbRL-generated and conventionally designed reward functions.

Figure 4 presents the relationship between the reward functions generated by GaPbRL and the conventionally designed reward functions in terms of the hit rate of the knowledge base versus instantaneous AoI. It can be observed that when the reward function generated by GaPbRL is applied to the DQN algorithm, the knowledge-base hit rate and AoI exhibit an exponential pattern that is consistent with the modeling assumptions in the problem formulation, without showing any obvious training bias. Moreover, the resulting knowledge-base hit rate $\eta$ is generally higher than that obtained using traditional manual-designed reward functions. This is because, as time progresses knowledge bases with larger AoI may gradually become less aligned with user preferences, causing their rankings in the Pareto distribution to decline, which in turn reduces the knowledge-base hit rate.

Figure 4. Relationship between knowledge-base hit rate and AoI under GaPbRL and conventional reward functions.

Figure 5 illustrates the average reward versus training episodes of GaPbRL under different numbers of manual comparison trajectories. The numbers of manual comparison trajectories are set to 50, 40, and 30, respectively, while the number of GAN-based comparison trajectories is fixed at 30. When the number of manual comparison trajectories is 50 and the number of GAN-based comparison trajectories is 30, the weighting coefficients of the output of the objective function of the PbRL reward function generation network are $\alpha =0.462$ and $\beta =0.538$ (red curve). When the number of manual comparison trajectories is 40 and the number of GAN-based comparison trajectories is 30, the weighting coefficients are $\alpha =0.441$ and $\beta =0.559$. When the number of manual comparison trajectories is 30 and the number of GAN-based comparison trajectories is 30, the weighting coefficients are $\alpha =0.414$ and $\beta =0.586$ (purple curve). It can be observed that as the number of manual comparisons increases, the resulting reward function becomes more accurate, which is reflected in faster convergence and higher precision. This indicates that the number of manual comparisons has a certain impact on PbRL, because a larger amount of preference data in the knowledge-base preference set $\zeta$ leads to better fine-tuning performance of the PbRL network.

Figure 5. Average reward versus training episodes of GaPbRL under different numbers of human comparison queries.

Figure 6 compares the average rewards versus the GaPbRL training episodes with and without the GAN-assisted preference set. Four groups of comparative experiments are conducted: (1) 50 manual comparison trajectories and 50 GAN-based comparison trajectories; (2) only 50 manual comparison trajectories; (3) 40 manual comparison trajectories and 40 GAN-based comparison trajectories; and (4) only 40 manual comparison trajectories. The weighting coefficients of the output of the objective function of the PbRL reward function generation network are set to $\alpha =0.467$, $\beta =0.533$ (red curve); $\alpha =0.384$, $\beta =0.616$ (purple curve); $\alpha =0.436$, $\beta =0.564$; and $\alpha =0.681$, $\beta =0.319$, respectively. The red and blue curves can be seen to exhibit convergence trends around the 60th training episode and achieve higher average reward values. Therefore, it can be concluded that, in PbRL, when the knowledge-base preference set $\zeta$ is fixed, the reward function generated with the assistance of the GAN-based preference set produces higher average rewards and converges faster during training. These results indicate that the GAN-assisted preference set plays a significant role in the PbRL network by helping to generate more accurate reward functions. The reason is that adversarial neural networks can learn human decision-making logic, further reducing the workload of manual comparisons in PbRL. By incorporating adversarial neural networks into the PbRL framework, manual costs can be reduced, leading to substantial economic benefits.

Figure 6. Average reward over episodes: Ablation of GAN-based preference generation in GaPbRL.

Figure 7 compares the average reward versus the GaPbRL training episodes under different storage thresholds on the client. The storage capacity of the edge server is set to $S=120$, 100, and 80 GB. The numbers of manual comparison trajectories and GAN-based comparison trajectories are set to 50, and the weighting coefficients of the output of the objective function by the PbRL reward function generation network are $\alpha =0.467$ and $\beta =0.533$. By observing the red and blue curves, we observe that the model exhibits clear convergence trends around the 60th and 80th training episodes, respectively. The final converged values of the red and blue curves are relatively close, indicating that when the client-side storage capacity is relatively sufficient, its impact on the average reward is limited. However, observation of the purple curve shows that when the storage capacity on the client is small, the convergence of the average reward is slower. Therefore, it can be concluded that the client storage capacity affects the convergence speed of the model. When the storage capacity in the client is sufficient, its impact on the convergence speed and accuracy of the model is relatively small. This is because sufficient client-side storage capacity provides a larger exploration space for the reinforcement learning agent to adopt greedy strategies. In contrast, when edge-server storage capacity is limited, although caching fewer knowledge bases may improve the degree of matching to some extent, it leads to slower convergence from a global perspective, and the objective function value of the optimal policy is also negatively affected.

Figure 7. Impact of client storage threshold on average reward over episodes in GaPbRL.

Figure 8 presents a comparison of the average reward versus the number of episodes for GaPbRL under different client-side storage threshold settings. The average AoI threshold for each knowledge base is set to $\sigma =15\mathrm{s}$, the instantaneous AoI threshold is set to $\lambda =20\mathrm{s}$, and the thresholds for the weighted sum of average AoI across all knowledge bases are $\Psi =200,150,$ and $100\mathrm{s}$. The number of human comparison trajectories is set to 50, and the number of GAN-generated comparison trajectories is also set to 50. The weighting coefficients of the PbRL reward function generation network are $\alpha =0.467$ and $\beta =0.533$. It can be observed that the red curve is higher than both the blue and purple curves, indicating that a larger threshold of the weighted sum of average AoI across all knowledge bases leads to a higher average reward. It can also be concluded that when this threshold is too low, the RL model exhibits weaker robustness. The reason is that a larger threshold expands the state space, allowing the agent to more easily explore suitable strategies. In contrast, when the threshold is too low, although the knowledge-base hit rate can be improved, the model can only converge to local optima, resulting in a lower global average reward.

Figure 8. Effect of AoI weighted sum threshold on average reward over episodes in GaPbRL.

Figure 9 illustrates the instantaneous value of the objective function versus the training episodes for the reward function generated by GaPbRL and the conventionally designed reward functions. It can be observed that, after parameter tuning, the trend of variation in the objective function value becomes relatively stable. When the GaPbRL reward function, corresponding to the square markers in the legend, with weighting coefficients of the output of the objective function of the PbRL reward function generation network set to $\alpha =0.467$ and $\beta =0.533$, is applied to the DQN algorithm, the instantaneous value of the resulting objective function is generally higher than that obtained using manually designed reward functions (corresponding to circle and triangle markers) after approximately 40 training episodes. It can also be observed that the scheme proposed in this paper exhibits fewer abrupt drops in the instantaneous objective function value during training, indicating a higher training robustness.

Figure 9. Comparison of instantaneous objective function values over training episodes for GaPbRL-generated and conventional reward functions.

Figure 10 compares the instantaneous objective function value versus training episodes for GaPbRL and the DDPG and actor–critic algorithms. The average AoI threshold for each knowledge base is set to $\sigma =15\mathrm{s}$, the threshold for the weighted sum of average AoI across all knowledge bases is set to $\Psi =200\mathrm{s}$, and the instantaneous AoI threshold for each knowledge base is set to $\lambda =20\mathrm{s}$. The storage capacity of the edge server is set to $S=120\mathrm{GB}$. The weighting coefficients of the objective function output by the PbRL reward function generation network are set to $\alpha =0.467$ and $\beta =0.533$. To make the comparison algorithms closer to the expert-tuned scenarios, the weighting coefficients of the comparison algorithms are set to $\alpha =0.4$ and $\beta =0.6$. The learning rate of the actor network is set to $e^{-4}$ (approximately $0.018$), the learning rate of the critic network is set to $e^{-3}$ (approximately $0.05$), the discount factor is set to $0.99$, and the batch size is set to 64. The soft update coefficient of DDPG is set to $0.001$. It can be observed that the GaPbRL algorithm proposed in this paper converges to a higher weighted sum of the degree and hit rate of the knowledge-base matching. It can also be observed that GaPbRL exhibits fewer drops in the instantaneous objective function value during training, indicating higher robustness. Therefore, it can be concluded that the GaPbRL method proposed in this paper outperforms traditional reinforcement learning algorithms. This is because, compared with conventional RL methods, GaPbRL introduces a reward model training network and a generative adversarial network, enabling the agent to receive more accurate reward signals and thus better drive the retention, deletion, and replacement of knowledge-base categories. Based on all the experimental results above, it can be concluded that the GaPbRL method proposed in this paper can be effectively applied to the cloud–edge deployment problem of knowledge bases in semantic communication. Moreover, it demonstrates significant advantages in terms of knowledge-base hit rate, matching degree, and convergence speed of reinforcement learning models, while greatly reducing the workload of manual fine-tuning.

Figure 10. Instantaneous objective function over training episodes: GaPbRL versus baseline algorithms.

5. Conclusions

In this paper, we address the problem of dynamic knowledge-base deployment in semantic communication. First, the cloud–edge knowledge-base system model in semantic communication is introduced, including the internal structure of knowledge bases and the factors affecting the caching of the knowledge base. Next, the objective function and its constraints are formulated, followed by a proof that the optimization problem is NP-hard. Then, the GaPbRL method is presented to solve this optimization problem, including the Markov decision process for SKB replacement, the training of the generative adversarial network, the training of the PbRL reward function generation network, and the training of the deep Q-network. Finally, simulation experiments are conducted to verify the feasibility and superiority of the proposed approach. Future work will focus on practical validation in real-world dynamic networks and testbeds, as well as exploring low-overhead, scalable multi-agent edge collaborative solutions for ultra-large-scale SKBs.