An Optimized Semantic Matching Method and RAG Testing Framework for Regulatory Texts

Abstract

To enhance the accuracy and reliability of large language models (LLMs) in regulatory question-answering tasks, this study addresses the complexity and domain-specificity of regulatory texts by designing a retrieval-augmented generation (RAG) testing framework. It proposes a dimensionality reduction-based semantic similarity measurement method and a retrieval optimization approach leveraging information reasoning. Through the construction of the technical route of the intelligent knowledge management system, the semantic understanding capabilities of multiple mainstream embedding models in the text matching of financial regulations are systematically evaluated. The workflow encompasses data processing, knowledge base construction, embedding model selection, vectorization, recall parameter analysis, and retrieval performance benchmarking. Furthermore, the study innovatively introduces a multidimensional scaling (MDS) based semantic similarity measurement method and a question-reasoning processing technique. Compared to traditional cosine similarity (CS) metrics, these methods significantly improved recall accuracy. Experimental results demonstrate that, under the RAG testing framework, the mxbai-embed-large embedding model combined with MDS similarity calculation, Top-k recall, and information reasoning effectively addresses core challenges such as the structuring of regulatory texts and the generalization of domain-specific terminology. This approach provides a reusable technical solution for optimizing semantic matching in vertical-domain RAG systems, particularly for MDSs such as law and finance.

1. Introduction

In recent years, with the rapid development of LLMs, increasing attention has been paid by both academic and industrial communities to the accuracy of their generated content. Although LLM performance has improved through parameter scaling and training optimization, their outputs have still frequently exhibited factual inaccuracies and outdated domain knowledge. To address these issues, RAG [1,2,3] has emerged as a promising solution and has been increasingly integrated with LLMs. By incorporating dynamic retrieval mechanisms from external knowledge bases, RAG enables LLMs to access real-time, domain-relevant knowledge, thereby mitigating factual errors and significantly improving output reliability.

RAG has evolved from basic implementations (e.g., “Naive RAG”) to advanced architectures such as “Modular RAG” incorporating retrieval optimization techniques and multi-module fusion. These methodologies enhance response accuracy and timeliness within intelligent question-answering systems, including applications in customer service and document generation. However, the effectiveness of RAG is critically dependent on comprehensive knowledge base construction and optimized retrieval pipeline design. This requires multi-stage optimization, including high-quality data source curation, domain-adapted text chunking, and precise semantic retrieval algorithms. In high-stakes domains with strict regulatory requirements, such as law and finance, the accuracy, completeness, and logical rigor of generated outputs directly affect decision-making risks, placing greater performance demands on RAG systems. Compared to general domains, knowledge management in vertical domains has presented three core challenges: knowledge base construction [4], semantic retrieval [5,6], and generation control.

• The knowledge base construction process is complex. Professional texts such as financial regulations [7] exhibit high clause interconnectivity and logical rigor. Conventional text segmentation methods [8] tend to fracture critical contextual relationships, resulting in fragmented retrieval outcomes. Concurrently, the high-frequency update characteristic of regulatory provisions requires the knowledge base to support real-time updates and version control capabilities. Recent improvements in knowledge base construction are primarily achieved through two approaches: structured data storage [4,9] and verification of clause-level logical consistency. Common knowledge processing methods include, but are not limited to, text chunking, information extraction [10,11,12], knowledge graphs [13], and Knowledge-Augmented Generation (KAG). Optimization methods for knowledge base construction have also been devised. For instance, dynamic semantic segmentation (such as dependency parsing in the txtai library [14]) has been adopted to address the strong logical coherence of regulatory provisions. This approach has progressively replaced fixed-length text chunking [15] with dynamic semantic segmentation techniques. This method applies dynamic segmentation to knowledge base texts, where clause boundaries are identified through dependency parsing using frameworks such as SpaCy 3.0.0, as proposed by Kumar et al. [16,17]. This ensures logical units maintain integrity as discrete chunks during storage. Regarding knowledge structure construction, multi-hop clause citation networks are developed by integrating information extraction with knowledge graphs [18]. Zuo et al. [19] demonstrated this approach by constructing datasets using the Scrapy framework and storing knowledge graphs in Neo4j, thereby establishing traceable knowledge-augmented structures [20]. Liang et al. [21] proposed KAG, which enhances the synergy between knowledge representation and retrieval through a logic-guided hybrid reasoning engine. Concurrently, Jiang et al. [22] introduced knowledge-augmented dialog generation, which employs a divergent knowledge selector to pre-optimize candidate knowledge and a knowledge-aware decoder to generate contextually coherent, knowledge-rich responses. To ensure the timeliness of regulatory knowledge bases, researchers have introduced a version control mechanism. This approach utilizes twin networks (e.g., MERIT) where the target network caches historical clause representations, achieving smooth version transitions through momentum updates [23] while preventing the inclusion of repealed provisions in the knowledge base. Under this mechanism, outdated provisions are tagged as “abrogated” and replaced by updated versions, enabling automated identification of content modifications during regulatory revisions [24,25].
• Semantic retrieval accuracy in general-domain technical solutions is constrained. General-domain embedding models [26] exhibit limited representational capacity for vectorizing domain-specific terminology. The implicit logical complexity of regulatory provisions poses significant challenges for both high-dimensional sparse vector representations [27] and traditional cosine similarity algorithms [28,29], leading to critical omissions during regulatory queries and potentially invalidating analytical outcomes. Current research primarily concentrates on domain-adapted embedding models and improvements in retrieval accuracy [30]. General-domain embedding models exhibit constrained representational capacity for relevant terminology, resulting in insufficient retrieval accuracy. Additionally, different embedding models demonstrate varied capacities for representing domain-specific terms [31], which necessitates comparative testing of domain-adapted models [32] (e.g., bge-m3, mxbai-embed-large) to improve matching precision and enhance comprehension of context-specific semantics. Furthermore, Weller et al. [5,33,34,35,36] proposed a reranking model that enables multi-level semantic reranking, thereby improving the precision of retrieved passages. Simultaneously, Bai et al. [37] developed a multi-hop reasoning approach incorporating relative temporal encoding to enhance path retrieval capability for complex regulatory logic.
• Stringent demands are imposed on response generation control. The generation phase must strictly adhere to inter-provision citation logic, and outputs must be traceable and interpretable to ensure credibility. Regarding generation control, current research efforts are focused on domain-constrained generation [38] and interpretability enhancement [39]. Prompt engineering optimization techniques, exemplified by the Collaborative Legal Expert Framework [40,41], ensure that generated content complies with regulatory formatting and citation logic through intent identification and legal foundation tracing. Additionally, a multi-tier posterior verification mechanism has been introduced [42] to validate provision validity through knowledge graph backtracking. This is further integrated with reasoning path visualization tools, such as ReasonGraph developed by Li et al. [43], which generated reasoning graphs that visualize the full trace: “problem keywords” → “retrieved provisions” → “generation basis”, thereby significantly enhancing the traceability of generated results [44] while markedly improving their credibility and auditability.

A comprehensive analysis of current research reveals a significant disparity between generic RAG frameworks and the temporal accuracy requirements of regulatory knowledge bases. Key limitations include insufficient semantic retrieval precision, inadequate traceability and reliability of generated results, and constrained flexibility in debugging and selecting embedding models. Consequently, research on specialized RAG testing frameworks and optimization methods for regulatory applications remains notably underdeveloped. To address the limited applicability of generic solutions in the regulatory domain, it is essential to fully consider the structural characteristics and domain-specific semantics of regulatory texts, thereby necessitating the development of collaborative retrieval-generation testing and optimization frameworks with enhanced domain adaptability.

Based on these challenges, this study proposes an optimized semantic matching method for regulatory texts and a dedicated RAG testing framework. Specifically, by focusing on RAG testing and semantic matching optimization for regulatory texts, this study investigates the principles of domain-specific adaptation for retrieval-augmented generation technology. A comparative analysis of multiple knowledge base processing strategies and embedding model performance is conducted, providing critical theoretical foundations and methodological guidance for the design of domain-specific RAG systems. Existing research indicates the absence of open-source RAG testing frameworks or software platforms specifically designed for the regulatory domain. The regulatory RAG testing framework developed in this study achieves comprehensive optimization throughout the entire workflow, from knowledge base construction to generation verification. This solution not only resolves technical constraints in regulatory retrieval but also establishes a reusable technical paradigm for developing domain-specific RAG systems.

The main contributions are summarized as follows:

• An open-source regulatory RAG testing framework has been constructed. Multiple embedding models are deployed to preprocess knowledge fragments, with similarity computation strategies configured to facilitate test in regulatory text semantic matching.
• A semantic matching optimization method based on dimensionality reduction has been proposed. Within the RAG testing framework, semantic matching accuracy for regulatory texts has been enhanced through denoising high-dimensional sparse vectors.
• Developed a retrieval optimization method based on information reasoning. By inferring potential questions, this approach enhances the accuracy of knowledge fragment matching.
• The applicability of different knowledge base processing methods and retrieval strategies for regulatory question-answering tasks has been validated using real-world regulatory datasets.

The remainder of this paper is organized as follows. Section 2 introduces the design and functionality of the RAG testing framework. Section 3 compares test data from multiple embedding models. Section 4 proposes a dimensionality reduction-based similarity computation method and evaluates the accuracy of different similarity metrics using various embedding models. Section 5 presents an information reasoning-based retrieval approach, and analyzes its applicability through real-world dataset testing. Section 6 concludes the paper and outlines potential applications and future research directions.

2. RAG Testing Framework Design

To support RAG testing and evaluate the effectiveness of different semantic matching approaches for regulatory texts, this section extends the AI-KM 5.4.18 [41] platform, which already integrates functionalities such as multi-view knowledge management and LLM invocation. In response to domain-specific requirements in the regulatory field, additional modules are developed, including knowledge preprocessing, RAG orchestration, cosine similarity computation, dimensionality reduction-based similarity calculation, and visualization components. The software architecture, centered on RAG testing, is illustrated in Figure 1.

Figure 1 demonstrates the latest functional design of the RAG testing framework. Addressing domain-specific requirements for regulatory text processing, core capabilities including LLM deployment, knowledge preprocessing, and similarity computation have been implemented through enhancements centered around RAG functionality.

• LLM Deployment. The software implements localized deployment and invocation of latest LLM through the Ollama 0.9.5 platform, with support extended to general-purpose language models (e.g., Deepseek-R1, LLaMA3.3), embedding models including bge-m3 and nomic-embed-text, and multimodal models such as Gemma3. This architecture enables rapid integration of newly released LLM products while reducing access barriers via standardized deployment interfaces, ensuring operational simplicity and flexibility in model deployment.
• Knowledge Base Management. The software converts structured text into multiple interactive interfaces such as tree diagrams, tag-based classification views, knowledge graphs, and mind maps by parsing Markdown documents. Efficient browsing, editing, and maintenance of the knowledge base are enabled through these views. Additionally, the knowledge base can be subjected to various preprocessing operations, including intelligent segmentation of Markdown documents, vectorization using custom embedding models, and semantic enhancement through information reasoning.
• RAG Module. The integration of embedding models with similarity computation enables high-precision knowledge retrieval, while domain-specific answers are generated by LLM based on the knowledge base. Fundamental matching between queries and knowledge fragments is implemented through cosine similarity, with semantic information reasoning being further enhanced through latent semantic information extraction. For a given knowledge segment, the software can automatically infer relevant information or expand contextual meaning through manual annotation, followed by similarity computation that combines original text with information reasoning. Additionally, the integrated MDS-based semantic similarity measurement method enhances semantic matching capability by eliminating interference from irrelevant semantic dimensions.
• Evaluation and Debugging Module. The software implements visual debugging, supporting the graphical display of both cosine similarity computation results and Dimensionality reduction results. Configuration of slicing strategies, switching between embedding models, comparison of similarity algorithms and hidden information editing capabilities are enabled to establish a debugging workflow covering parameter configuration, experiment execution, result visualization, and performance comparison, providing intuitive support for technical optimization decisions.

The corresponding knowledge base processing and RAG testing framework interfaces within the software are presented in Figure 2.

Figure 2 presents the data processing and debugging interface of the RAG testing framework. Users can input queries, view generated responses, and visually inspect the associations between knowledge base fragments and corresponding questions. Parameters such as slicing strategies, embedding models, and similarity computation methods can be adjusted, and the processes of cosine similarity computation and MDS-based semantic similarity measurement are visualized to support transparent evaluation and debugging.

Considering the structured nature of regulatory texts, which typically include nested clauses, domain-specific terminology, and complex logical relationships, a multi-stage knowledge base processing framework has been developed. This framework builds upon the functionalities outlined in Figure 1 and enhances retrieval precision by configuring slicing strategies, embedding models, information reasoning and editing modules, and similarity computation methods at distinct processing stages. It also improves software operability and enhances retrieval visualization capabilities. The complete workflow for knowledge base processing and RAG evaluation is illustrated in Figure 3.

Figure 3 illustrates the knowledge base processing and RAG workflow, comprising the following stages.

• The directory containing the target knowledge base is selected. The knowledge base primarily consists of files in Markdown format. Documents in other formats, such as PDF, can be converted using open-source tools such as MinerU 0.4.1.0. Manual verification and editing of Markdown files may be conducted to improve textual accuracy and consistency.
• The knowledge base is segmented using predefined delimiters, such as consecutive line breaks. If the segmentation results are determined to be suboptimal, manual refinement of the Markdown structure can be applied to enhance the segmentation quality.
• Vectors corresponding to each segmented knowledge fragment are computed using the specified embedding model and stored in the database.
• For each knowledge fragment, implicit semantic information is extracted using the configured large language model. By default, this process involves prompt-based question generation to derive potential questions associated with the corresponding content. Alternatively, customized prompt instructions can be applied to elicit semantically enriched outputs. The extracted information may also be manually refined to enhance the semantic granularity of key content. (This step is optional and may be selectively executed depending on task requirements.)
• Vectors are computed for both the original knowledge fragments and the associated implicit information using the designated embedding model, and subsequently stored in the database.
• For user-submitted queries, similarity computations are performed using either cosine similarity or dimensionality reduction based on MDS. Vectors corresponding to the original knowledge fragments, implicit information, and queries are calculated and ranked in descending order according to similarity scores.
• Relevant knowledge fragments are retrieved according to one of three predefined recall strategies: similarity-based recall, top-k recall, or length-constrained recall. Prompt templates are constructed accordingly, and each knowledge fragment is annotated with metadata such as regulatory source and temporal information to improve interpretability and ensure that updated regulations maintain backward traceability.
• The finalized prompts are processed by the LLM to generate responses aligned with the regulatory knowledge base, thereby enhancing domain relevance and answer precision.

3. Embedding Models Testing

To evaluate the performance of various embedding models within the RAG testing framework and determine the most suitable model for semantic matching of regulatory texts, several embedding models are selected from the Ollama model library for benchmarking. The results provide guidance for optimal embedding model selection and configuration in domain-specific RAG systems.

Embedding models convert textual content into semantic representations in high-dimensional vector space [45,46]. By computing the cosine similarity between user queries and knowledge fragments, the most semantically relevant fragments can be retrieved to support accurate domain-specific answer generation. However, different embedding models vary significantly in their ability to capture general versus domain-specific semantics. To evaluate the performance of different embedding models, the open-source platform Ollama is adopted, which supports efficient installation and deployment of diverse embedding models through standardized interfaces.

Based on model maturity and practical applicability, four embedding models are selected for performance benchmarking using the proposed RAG testing framework. The characteristics of these models are summarized in

Table 1. Characteristics of different embedding models.

Model	Parameters	Default Dimensions	Context Length	Features
mxbai-embed-large	335M	1024	512 tokens	The model supports multilingual retrieval and is well-suited for short-text tasks, though its capability for processing long texts is relatively limited [47].
Bge-m3	420M	1024	8192 tokens	The model demonstrates strong capabilities in long-text processing and cross-lingual tasks, making it particularly suitable for knowledge base question-answering, though its larger model size results in higher resource consumption [48].
ALL-MiniLM	33M	384	256 tokens	The model supports deployment on lightweight devices with primary English-language support, though it exhibits performance degradation on long-text processing [49].
Nomic-embed-text	137M	768	8192 tokens	The model features transparent training data and processes, while demonstrating average performance on complex tasks [50].

.

To evaluate the suitability of different embedding models for semantic matching in the regulatory texts, an experimental procedure is designed to test the models’ ability to interpret regulatory provisions. The experimental setup is defined as follows:

• The knowledge base incorporates regulations such as the “Government Procurement Goods and Services Bidding Management Measures” for preliminary testing of LLM and embedding model performance. Multiple embedding models are selected for comparative evaluation.
• The test question dataset comprised 50 financial regulation domain questions. This dataset is generated using DeepSeek-V3-0324 based on the knowledge base, followed by manual verification.
• The selected embedding models are listed in Table 1. These models are evaluated using a streamlined regulatory corpus to assess their performance.
• Cosine similarity is employed to calculate the similarity between user-submitted questions and knowledge base segments. The matching accuracy of embedding models is evaluated by setting recall thresholds. A question is considered correctly answered if its corresponding knowledge segment is matched under the specified recall threshold, with accuracy rates are calculated accordingly.
• The framework implemented three distinct retrieval strategies: (1) similarity-based retrieval, where knowledge segments are recalled when their similarity scores exceed a predetermined threshold; (2) Top-k retrieval, which selects a fixed number of top-ranked knowledge segments sorted by descending similarity; and (3) length-constrained retrieval, where segments are similarly ranked by similarity but are dynamically aggregated through prompt engineering until reaching a predefined character count limit.

Following the aforementioned testing conditions, the collected data are systematically organized and subjected to one-way analysis of variance (ANOVA) across the three datasets. The F-value represents the ratio of between-group mean differences to within-group variations, while the p-value indicates the probability of observed differences occurring by chance. A p-value below 0.05 is considered statistically significant. The resulting retrieval accuracy rates are presented in the table below.

The table presented test results of various embedding models on regulatory domain question-answering data. Based on the data from

Table 2. Accuracy rates under different recall strategy and threshold conditions (the cosine similarity method).

Recall Strategy	Embedded Model	Threshold
		0.58	0.55	0.52	0.49	0.46
Similarity-based recall	mxbai	100.00%	100.00%	100.00%	100.00%	100.00%
	bge-m3	18.00%	20.00%	24.00%	30.00%	46.00%
	all-minilm	14.00%	24.00%	36.00%	54.00%	70.00%
	nomic	58.00%	70.00%	84.00%	98.00%	100.00%
		$F=24.41,p<0.001$
		2	3	4	5	6
Top-k recall	mxbai	48.00%	54.00%	62.00%	68.00%	78.00%
	bge-m3	24.00%	40.00%	54.00%	62.00%	72.00%
	all-minilm	40.00%	50.00%	58.00%	66.00%	74.00%
	nomic	36.00%	44.00%	52.00%	62.00%	74.00%
		$F=0.57,p=0.65$
		2500	3000	3500	4000	4500
Length-constrained recall	mxbai	38.00%	42.00%	48.00%	50.00%	54.00%
	bge-m3	30.00%	38.00%	40.00%	40.00%	44.00%
	all-minilm	32.00%	34.00%	38.00%	46.00%	50.00%
	nomic	28.00%	38.00%	42.00%	42.00%	46.00%
		$F=1.53,p=0.245$

, Figure 4 is generated.

Under the similarity-based recall strategy, the accuracy differences among models demonstrate high statistical significance (F = 24.41, p < 0.001), indicating that model selection significantly impacts retrieval performance. The mxbai model achieves optimal performance, maintaining 100% accuracy across all thresholds, while the bge-m3 and all-minilm model exhibit poorer results, requiring threshold reduction to recall sufficient segments. Analysis reveals substantial variations in similarity score distributions among models (Figure 4: Similarity-based Recall): the mxbai model consistently generates higher similarity scores, enabling effective recall even under strict thresholds. In contrast, bge-m3 shows only 18% accuracy at high thresholds (e.g., 70%), requiring threshold reduction to 46% for performance improvement. This finding demonstrates that the evaluation results of similarity-based recall strategies are highly dependent on the inherent similarity distribution characteristics of each model, making direct cross-model performance comparisons impractical.

Under the Top-k recall strategy, there are no statistically significant differences in accuracy among the models (F = 0.57, p = 0.65), yet their performance rankings remain consistent: mxbai > all-minilm > nomic > bge-m3. The mxbai model demonstrates optimal overall performance when Top-k recall is fixed, though the performance gap with other models narrows significantly (Figure 4: Top-k Recall). By directly controlling the number of retrieved segments, this strategy eliminates interference from similarity distribution disparities, thereby more equitably reflecting models’ ranking capabilities. Consequently, for scenarios requiring a balance between recall and precision, the Top-k recall strategy is recommended as the primary evaluation approach.

Under the length-constrained recall strategy, the performance differences among the models further diminish (F = 1.53, p = 0.245), and all models exhibit relatively low correct-answer rates (Figure 4: Length-constrained Recall). This suggests that length-constrained recall may impair retrieval effectiveness, possibly due to truncating critical information or introducing noise—particularly impacting high-precision models like the mxbai model more significantly. Although the mxbai model maintained a slight advantage, the practical utility of this strategy remains limited and is only recommended for scenarios with strict text-length constraints.

The comprehensive comparison demonstrates that the mxbai model consistently achieves optimal performance across all three strategies, making it particularly suitable for high-precision regulatory retrieval tasks. When employing similarity-based recall, model-specific threshold optimization is recommended. In contrast, the Top-k recall strategy, owing to its superior stability and fairness, is better suited as a benchmark method for model performance evaluation. Length-constrained recall requires cautious implementation to avoid performance degradation caused by information loss. In practical applications, strategy selection should be demand driven: for high-stakes scenarios, priority should be given to the mxbai model paired with similarity-based recall, while Top-k recall is recommended for general evaluation to enhance result comparability.

4. Semantic Matching Optimization Methodology

Within the RAG testing framework for embedding model configuration strategies, this section proposes a semantic matching optimization method based on dimensionality reduction results. The method enhances conventional cosine similarity computation through systematic testing of alternative similarity calculation strategies, with rigorous analysis conducted to validate the effectiveness of the proposed dimensionality-reduction-enhanced similarity metric for semantic matching tasks.

For high-dimensional sparse vector matching, dimensionality reduction techniques, such as Uniform Manifold Approximation and Projection (UMAP), t-Distributed Stochastic Neighbor Embedding (t-SNE), MDS, and Principal Component Analysis (PCA) [51,52,53,54,55], serve as an effective approach for vector visualization. These methods transform high-dimensional vectors into low-dimensional representations while preserving their primary distance relationships. By reducing noise from irrelevant dimensions in semantic understanding, they consequently enhance retrieval accuracy. Among these dimensionality reduction methods, t-SNE and UMAP prioritize local structure preservation at the expense of global distance comparability. For regulatory semantic matching tasks—where understanding the overall spatial relationships, relative positions, and distance metrics between queries and knowledge segments is crucial—classical MDS and PCA yield more reliable and interpretable distance representations. Furthermore, while t-SNE’s strong dependence on random initialization leads to inconsistent outputs across multiple runs, and UMAP (though more stable than t-SNE) still exhibits result variability, both MDS and PCA guarantee deterministic and reproducible similarity results. Consequently, dimensionality reduction methods based on MDS and PCA are selected for experimental validation.

4.1. Cosine Similarity

In conventional RAG frameworks, the standard approach involves transforming text into vector representations using embedding models. Specifically, the user’s query (denoted as vector $q$) and knowledge segments (each represented as vector $d_i$ for the $i$ segment) undergo cosine similarity computation. The semantic similarity measurement between two vectors is calculated as follows:

$$CS\left(q,d_i\right)={\displaystyle \frac{q·d_i}{‖q‖·‖d_i‖}}$$

(1)

4.2. MDS

The core principle of MDS-based semantic similarity measurement lies in preserving the distance relationships between high-dimensional data points within a low-dimensional space, reconstructing the data using a reduced-dimension distance matrix. This approach aligns with text semantic matching requirements, where the objective is to capture core semantic relationships while eliminating noise from peripheral semantics. Consequently, the optimization strategy for semantic matching involves projecting high-dimensional retrieval vectors into a lower-dimensional space while maintaining their original distance relationships, thereby reducing interference from irrelevant dimensions and improving both the accuracy and efficiency of semantic matching. Within the RAG testing framework, Figure 5 illustrates the computational workflow that employs the classical MDS algorithm for similarity measurement.

The operational steps of the method using MDS for semantic similarity measurement are as follows:

• During the knowledge base processing phase, structured Markdown documents are utilized as the primary data source. These documents are segmented into text fragments based on predefined rules, such as paragraph boundaries, section headings, or fixed-length chunking. Each text fragment is then transformed into a high-dimensional vector $d_i$ using designated embedding models (for example, mxbai-embed-large or bge-m3), which are designed to capture deep semantic features. The resulting vectors are stored in a vector database along with their corresponding text fragments to support efficient similarity-based retrieval. To preprocess n vectors of dimension m, a cosine distance matrix $D\in R^{n\times n}$ is computed, where each element is given by the following:

  $$D_{ij}=1-CS(v_i,v_j)=1-{\displaystyle \frac{v_i·v_j}{‖v_i‖‖v_j‖}}$$

  (2)
• The classical MDS method is then applied for dimensionality reduction. A centering matrix is constructed as follows:

  $$H=I_n-{\displaystyle \frac{1}{n}}11^T$$

  (3)

  and used to double centering the squared distance matrix $D$, yielding the inner product matrix:

  $$B=-{\displaystyle \frac{1}{2}}H{D}^{\left(2\right)}H$$

  (4)

  This transformation ensures that both the row and column sums of the matrix $B$ are zero, which eliminates dependence on the coordinate origin and facilitates subsequent eigen decomposition. In this context, $I_n$ denotes the identity matrix of dimension $n\times n$; $1$ is a column vector of ones with dimension $n\times 1$; $11^T$ is an $n\times n$ matrix with all entries equal to one; and $D^{\left(2\right)}$ represents the element-wise squared distance matrix. Matrix $B$ is then subjected to eigendecomposition:

  $$B=U\Lambda U^T$$

  (5)

  where $\Lambda$ is the diagonal matrix of eigenvalues. The eigenvectors associated with the two largest eigenvalues (ranked in descending order) are selected to construct the projection matrix. The low-dimensional coordinate $d_i^{MDS}$ of each data point is obtained from the corresponding row of the projection matrix. This approach is justified by the fact that eigenvalues quantify the variance explained along each dimension. Larger eigenvalues correspond to directions with greater variance (thus carrying more significant semantic information), while smaller eigenvalues correspond to less informative directions, which can be interpreted as semantic noise. Selecting only the top eigenvectors enables preservation of the most meaningful geometric relationships while suppressing irrelevant components.
• The semantic similarity between texts can be obtained by calculating the Euclidean distances between different coordinate points. Following the same methodology, the embedding vector corresponding to the user’s query (denoted as $q$ undergoes centralization and projection transformation to be mapped into the low-dimensional space as $q^{MDS}$. This process effectively preserves the original spatial distance relationships in the reduced-dimensional space. Finally, the semantic similarity between the retrieved knowledge segments and the user’s query is determined through distance computation, with detailed operations specified in Formula (6).

  $$MDS(q,d_i)={\displaystyle \frac{1}{1+{‖q^{MDS}-d_i^{MDS}‖}_2}}$$

  (6)

Figure 6 illustrates a schematic representation of the similarity results derived from MDS-based calculations between retrieved knowledge fragments and user queries.

Figure 6 provides a two-dimensional visualization on the left, illustrating the semantic distances between the user query and each individual knowledge fragment. Notably, the clause directly relevant to the query yields a similarity score of 67.17%, ranking second among all retrieved segments. According to the configured Top-k recall threshold, the tender process query is not only successfully linked to the most relevant regulatory clause, but the system also demonstrates the ability to effectively extract and summarize the complete tendering procedure based on retrieved content.

4.3. Principal Component Analysis

In RAG systems, the principal objective of PCA is to project high-dimensional embedding vectors into a lower-dimensional space through linear transformation, preserving the most significant semantic features while reducing noise interference. Similarly to MDS-based semantic similarity measurement, PCA achieves dimension reduction by retaining the principal variance directions of the data. However, PCA directly decomposes the covariance matrix of the original vectors rather than a distance matrix, making it more suitable for processing linearly structured data. The implementation procedure consists of the following steps:

• During the knowledge base processing phase, structured Markdown documents are segmented into text fragment collections $D=\{d_1,d_2,...,d_n\}$ according to predefined rules. Each text fragment $d_i$ is then transformed into a high-dimensional semantic vector $v_i=Embedding\left(d_i\right)\in R^m$ using an embedding model.
• The computational procedure involves the following steps: first, compute the mean vector and perform centralization: $\overline{v}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{v}_i,v_i\prime =v_i-\overline{v}$, constructing the centralized matrix $V\prime \in R^{n\times m}$.
• Next, calculate the covariance matrix: $\sum ={\displaystyle \frac{1}{n-1}}{V\prime }^{T}V\prime \in R^{m\times m}$. Subsequently, decompose the covariance matrix: $\sum =P\Lambda P^T$ where $\Lambda =diag\left({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_m\right)$ and $P$ is the eigenvector matrix. The target dimension $k$ is determined based on the cumulative explained variance ratio. Finally, generate the dimensionality reduction projection matrix $P_k\in R^{m\times k}$ by selecting the top k eigenvectors, and project into the lower-dimensional space: $v_i^{PCA}=P_k^T{v}_i\prime \in R^k$.
• For user query q, its embedding vector is similarly generated, and dimensionality reduction is applied: $q^{PCA}=P_k^T\left(Embedding\right(q)-\overline{v})$. In the low-dimensional space, retrieval is performed using cosine similarity: $PCA\left(q,d_i\right)={\displaystyle \frac{q^{PCA}·d_i^{PCA}}{‖q^{PCA}‖·‖d_i^{PCA}‖}}$.

4.4. Experimental Analysis

The semantic matching accuracy of CS, MDS, and PCA methods is evaluated using identical query datasets and regulatory knowledge bases, with the resulting accuracy metrics presented in Figure 7.

As shown in Figure 7, with the cosine similarity-based results serving as the baseline, the performance of embedding models, the effectiveness of semantic similarity measurement, and computational efficiency are analyzed, leading to the following conclusions: different embedding models demonstrate varying capabilities in comprehending regulatory texts. The mxbai model exhibits optimal performance, achieving 80% accuracy under the specified experimental conditions (Top-k recall strategy, threshold = 6, MDS), highlighting its superior understanding of regulatory semantics. Within this RAG testing framework, the approach provides quantifiable citation requirements, offering empirical support for recall strategy threshold selection. This solution is recommended for regulatory RAG frameworks, supplemented with a dynamic threshold adjustment mechanism. When either recall volume or accuracy rates reach predetermined thresholds, a manual review process should be initiated to balance operational efficiency with risk control requirements.

Under the Top-k recall strategy, when employing the mxbai embedding model with recall thresholds between 2 and 6, the performance ranking of semantic matching methods follows: MDS > CS > PCA. However, with lower-performing embedding models (e.g., bge-m3 and all-minilm), CS outperforms MDS by 2–16%. This indicates that MDS method requires higher-quality embedding models to be effective—its application may amplify semantic matching deficiencies when used with suboptimal embedding models. Across all tested embedding models, PCA consistently demonstrated inferior performance. This is primarily attributed to PCA’s inherent limitation in capturing nonlinear semantic relationships, resulting in comparatively poorer matching accuracy.

The computational time complexity of these three methods is systematically evaluated. Cosine similarity demonstrates $O\left(Nd\right)$ complexity. For MDS, the requirement to compute pairwise cosine similarities between all vectors results in $O\left(n^{2}d\right)$ complexity, with additional $O\left(n^3\right)$ complexity for singular value decomposition (SVD) during matrix operations. PCA involves mean calculation and centering operations with $O\left(nd\right)$ complexity, followed by covariance matrix computation at $O\left(d^{2}n\right)$ complexity and its subsequent SVD at $O\left(d^3\right)$ complexity, culminating in principal component projection with $O\left(2nd\right)$ complexity (where $n$ represents vector count and $d$ denotes vector dimensionality). Under large-scale knowledge base conditions, both MDS and PCA exhibit significantly lower computational efficiency compared to CS, which may adversely impact practical application performance, necessitating methodological optimizations for deployment scenarios.

5. Retrieval Method Based on Information Reasoning

To enhance retrieval accuracy for knowledge fragments, the RAG framework incorporates an information reasoning-based optimization method. Within this framework, segmented knowledge fragments undergo processing by a large language model to deduce their potentially implicit information. By default, this hidden information reasoning approach is configured to derive the probable user questions corresponding to each knowledge fragment. The system then computes a composite similarity score by evaluating the relationships between the original knowledge fragment, its inferred information, and the user’s query, ultimately generating a ranked list of knowledge fragments according to the specified recall strategy. The precise computational method for determining this comprehensive similarity metric is formally expressed in Formula (7).

$$Merge\left(q,v_i,u_i\right)=\left\{\begin{array}{cc}{\displaystyle \frac{\frac{q·v_i}{‖q‖·‖v_i‖}+\frac{q·u_i}{‖q‖·‖u_i‖}}{2}} ,& \mathrm{C}\mathrm{S}\\ {\displaystyle \frac{\frac{1}{1+{‖q^{MDS}-v_i^{MDS}‖}_2}+\frac{1}{1+{‖q^{MDS}-u_i^{MDS}‖}_2}}{2}} ,& \mathrm{M}\mathrm{D}\mathrm{S}\\ {\displaystyle \frac{\frac{q^{PCA}·v_i^{PCA}}{‖q^{PCA}‖·‖v_i^{PCA}‖}+\frac{q^{PCA}·u_i^{PCA}}{‖q^{PCA}‖·‖u_i^{PCA}‖}}{2}} ,& \mathrm{P}\mathrm{C}\mathrm{A}\end{array} \right.$$

(7)

In the formula, $q$ represents the vector corresponding to the user’s query, $v_i$ denotes the vector of the knowledge fragment, and $u_i$ indicates the vector of the inferred information.

To evaluate the applicability of traditional cosine similarity versus MDS-based semantic similarity measurement for similarity computation, six financial regulation documents are selected to test different retrieval approaches. The testing conditions are as follows:

• The mxbai embedding model is utilized, which demonstrates optimal performance for this domain according to the tests in Section 3.
• The Top-k recall strategy is implemented, and shows broader applicability based on the test results in Section 3.
• The financial regulation knowledge base and query dataset specifications are detailed in

  Table 3. Configuration of the regulatory knowledge base and query dataset.

  Regulation	Name of Regulation	Number of Fragments	Number of Questions
  Regulation 1	Regulations on the Bidding and Tendering Process for Government Procurement of Goods and Services	12	50
  Regulation 2	Financial Department Supervision Measures	6	36
  Regulation 3	Fiscal Bill Management Measures	7	35
  Regulation 4	Management Measures for Loans and Grants from International Financial Institutions and Foreign Governments	7	39
  Regulation 5	Administrative Measures for the Transfer of State-owned Assets in Financial Enterprises	8	64
  Regulation 6	Approval and Supervision Measures for Asset Appraisal Institutions	6	48

  .

Using the corresponding regulatory knowledge base, each query in the dataset is systematically tested to evaluate question-answering performance, yielding the accuracy distribution presented in Figure 8.

The mxbai model with Top-k recall and traditional cosine similarity serves as the baseline for comparison. Figure 8 demonstrates that hidden information reasoning can enhance RAG accuracy in small-scale knowledge bases. Additionally, the data reveals that MDS-based computation achieves higher accuracy than cosine similarity for most regulatory documents.

To investigate the applicability of four retrieval approaches for large-scale financial regulation datasets—traditional cosine similarity (Method Normal CS), cosine similarity with information reasoning (Method Reasoning of CS), MDS-based semantic similarity measurement (Method Normal MDS), and MDS-based semantic similarity measurement with information reasoning (Method Reasoning of MDS)—a comprehensive study is conducted. Fifteen categories of financial regulations are collected to construct a domain-specific large-scale knowledge base. For this knowledge base, a corresponding query dataset is generated through DeepSeek and manually verified methods. The resulting distribution is presented as a histogram in Figure 9, illustrating the ranking positions of correctly retrieved knowledge fragments across different retrieval methods and reasoning conditions.

The results demonstrate that for large-scale knowledge bases without information reasoning, MDS exhibits compromised matching accuracy compared to CS, as evidenced by the posterior shift in its distribution median. This indicates limited suitability of MDS for large knowledge repositories. However, test data from Section 4.4 and Section 5 confirm the method’s applicability to small-scale knowledge bases.

The semantic information reasoning significantly enhances the effectiveness of traditional CS matching, as evidenced by the highest and most forward-shifted distribution peak, indicating that the correct knowledge fragments achieve higher similarity rankings under this approach. However, the information reasoning does not improve the matching accuracy of MDS-based semantic similarity measurement and even exhibits detrimental effects. This phenomenon primarily occurs because while the semantic information reasoning combined with MDS enriches the semantics of correct knowledge fragments, it simultaneously enhances the semantic representations of other knowledge fragments and aligns them toward more generic information. Consequently, the similarity rankings of correct knowledge fragments are shifted backward, thereby reducing matching accuracy. Therefore, for large-scale knowledge bases, semantic information reasoning can be employed to process the knowledge base. The cosine similarity measurement is then utilized to compute the similarity between each knowledge fragment and the user’s query, through which the most relevant knowledge fragments are selected. Subsequently, MDS is applied to optimize the ranking of the top-ranked fragments, thereby further improving the accuracy of the retrieval process. This approach shares conceptual similarities with re-rank models, where critical information is reordered to enhance the accuracy of semantic matching. Additional computational time is allocated to key processing stages to achieve higher precision. Notably, this solution requires no additional training. For domain-specific RAG systems, implementation after thorough testing is recommended.

6. Conclusions

To enhance the accuracy of semantic matching for regulatory texts, it is essential to address three core challenges in RAG technology: knowledge base construction, semantic retrieval, and generation control. First, an open-source RAG testing framework specifically designed for regulatory domains is developed. Subsequently, within this framework, comprehensive performance evaluations are conducted on domain-specific embedding models. Furthermore, a dimensionality-reduction-based semantic matching optimization method is proposed, along with an information-reasoning-enhanced retrieval approach that improves matching accuracy by latent semantic information extraction from knowledge fragments. Finally, the effectiveness of this combined strategy in enhancing regulatory text semantic matching is rigorously validated across multiple real-world regulatory datasets. Specifically, for regulatory texts, chapter-based segmentation effectively addresses the issue of logical unit fragmentation caused by conventional text splitting methods. Regarding embedding model selection, the mxbai-embed-large model demonstrates superior semantic comprehension of regulatory terminology, significantly outperforming other embedding models. The proposed MDS-based semantic similarity measurement method enhances the identification accuracy of key regulatory provisions in small-scale knowledge bases by preserving high-dimensional semantic relationships while eliminating semantic noise. The quantity-based recall strategy proves effective in adapting to different embedding models while maintaining consistent retrieval quality. Additionally, the accuracy of semantic matching can be further improved through latent information inference processing of the knowledge base, which deduces potential user queries in advance.

This study adopts financial regulation question-answering as a representative scenario, implementing dynamic knowledge base management to achieve timely updates of regulatory provisions. The RAG testing framework systematically validated the implementation effectiveness of various combined approaches, with the optimal solution demonstrating significant practical value in real-world regulatory question-answering applications. Currently deployed in reimbursement and auditing scenarios, the framework has effectively addressed critical challenges including imprecise professional term matching and low accuracy in RAG systems. The open-source RAG testing framework achieves comprehensive optimization across the entire “knowledge processing-retrieval-generation” pipeline. It supports dynamic configuration of text segmentation strategies and embedding models, incorporates a visual debugging interface, and implements hidden information enhancement mechanisms—leveraging LLMs to inversely deduce implicit queries or supplement critical information, thereby expanding semantic coverage. This framework ensures interpretable retrieval processes while providing multi-dimensional evaluation metrics for comparative analysis, significantly lowering the barrier to RAG testing. By facilitating industry–academia–research collaboration, it accelerates the translation of regulatory intelligence applications from laboratory research to industrial practice, delivering reliable technical infrastructure and implementation pathways for sectoral digital transformation.

Future research will focus on developing multi-phase and hybrid recall strategies, along with multimodal regulatory text analysis, to further refine the lifecycle management mechanisms for regulatory knowledge and RAG methodologies. Additionally, during embedding model training, MDS-based semantic noise filtration could be implemented to preserve core semantic relationships and enhance model performance. This study establishes a replicable methodological framework for domain-specific RAG systems, with technical approaches offering significant reference value for processing highly specialized texts in regulatory, financial, and healthcare domains.