Interpretable Conversation Routing via the Latent Embeddings Approach

Abstract

Large language models (LLMs) are quickly implemented to answer question and support systems to automate customer experience across all domains, including medical use cases. Models in such environments should solve multiple problems like general knowledge questions, queries to external sources, function calling and many others. Some cases might not even require a full-on text generation. They possibly need different prompts or even different models. All of it can be managed by a routing step. This paper focuses on interpretable few-shot approaches for conversation routing like latent embeddings retrieval. The work here presents a benchmark, a sorrow analysis, and a set of visualizations of the way latent embeddings routing works for long-context conversations in a multilingual, domain-specific environment. The results presented here show that the latent embeddings router is able to achieve performance on the same level as LLM-based routers with additional interpretability and higher level of control over model decision-making.

1. Introduction

Chatbots built with large language models (LLMs) face significantly more complex tasks [1], which can contradict each other. For example, answers may require different lengths and levels of details, the model can use external data sources, call other models, and APIs, or run certain code to improve the result for a specific use case. A simple monolithic architecture with a single large prompt for LLM-based applications cannot comprehend such challenges as it is unable to execute all diverse tasks with the same level of quality [2]. One big prompt would not be enough to handle it and may cause conflicts between instructions for use cases (for example be creative in some situations and give laconic, straight answers in others). Use cases might need different generation parameters (different values of temperature or top p) to execute corresponding tasks.

The routing layer can solve this problem by orchestrating a system of multiple specified agents, which solve concrete tasks instead of trying to handle everything [3]. This way the chatbot is split into a network of sophisticated sub-applications, which are connected by a classifier that chooses the best answer generator for new messages. This makes the answering process more context-aware, accurate, and cost-efficient. It also helps to avoid conflicts between instructions as these agents can be different prompts, models, or predefined sets of actions and answers.

Another advantage is the increased security of the application as harmful instructions, unintended questions or jailbreaks will not even get passed to the LLM itself [4]. The routing layer can filter out such requests and protect agents from instruction injections and attempts at out-of-scope usage [5].

Routing problems can be expressed as a simple classification task. Common classifiers like Naive Bayes, SVMs, and other TF–IDF-based methods do not have a multilingual advantage, which current transformers can provide. They are easy to build and support, but they are difficult to scale into a multilingual, multidomain environment, where input language is not predefined and can switch from conversation to conversation.

Routing can be solved by a fine-tuned transformer or any other deep-learning model [6,7]. The only requirement is a high prediction speed as this layer would be called for every input and can easily become a bottleneck due to high latency. However, developers would face the problem of gathering and annotating training data for conversations. Fine-tuning a classifier would require thousands of correctly labeled examples for a variety of use cases. Messages have to be diverse as users can formulate similar requests in multiple ways or make spelling, punctuation, or grammar mistakes. Some cases can be covered poorly and it would be difficult to catch it before the real-life usage of the system. Inputs for modern chatbots are not structured and do not have any determined, correct way of expressing users’ intentions. This makes the development and support of the routing layer much more difficult at the early stages of the chatbot lifecycle.

Multiagent chatbot systems can significantly enhance the performance of medical workers by taking routine tasks and providing a simple way to interact with already collected data [8]. Machine learning can act as an advice system to help diagnose, forecast, and model situations [9]. However, the interpretability and controllability of machine learning systems are crucial in this field as they require full-on trust from the user to the system with detailed explanations of the decision-making process [10]. Each step of the answering has to be explained and grounded including the routing layer.

Also, a router maintenance problem can emerge. The number of routes can change with time and the definitions of existing routes can be dynamic too. This would require data gathering, annotation, full retraining, and testing of the routing model, so the update will take a significant amount of time, research, and development to end up in production.

Another crucial challenge is the interpretation and control of the routing process. It is important to understand the reasoning behind the chosen route and to have ways of easily influencing the prediction [11]. Standard text classification models do not provide means to control the inference without tuning the model itself [12] or ingesting additional feature vectors to give a level of control over the original model [13]. However, this would still require a full-on fine-tuning of the model as such modifications need significant architectural changes. Interpretation methods are lacking too as it would rely on either analysis of attention masks or SHAP-like visualizations [14].

Attention scores indicate the relative importance of tokens, but they do not translate into causality, so even if token attention is high, it does not mean it is important for the final prediction. Also, attention masks highlight where the model looked at during task solving. They do not provide a reason for a specific class to be chosen or a token to be generated. Another crucial disadvantage is the difficulty of processing big inputs. Masks will be too big to handle manually, making this method irrelevant for tasks like chat routing as inputs will be long.

SHAP provides a set of visualizations to ease the interpretation of the results (summary plots, dependence plots, force plots, etc.). It is based on game theory and assigns a clear numerical value to each feature (input token in the case of text classification) to indicate how much it increases or decreases the probability of a target class. SHAP has scaling issues for high-dimensional data too, which would make it computationally expensive and more difficult to gain meaningful insights for chats or even individual long messages. Just like with attention masks, SHAP values do not show cause-and-effect relationships and do not provide insights into how the model decision-making can be modified to fix certain cases.

Zero- and few-shot classifiers can ease the implementation as they do not need training or a large set of examples [15]. Approaches like natural language inference (NLI) classifiers or LLM in-context learning can be used to solve it [16,17]. Our previous research proved that NLI routers significantly underperform and the LLM router is too unreliable. LLM routers are easy to implement and have a high accuracy even for multilingual systems. However, it is biased towards class order, labels, and descriptions of classes [18]. Even an intermediate multiple evidence calibration (MEC) cannot make it more stable [19]. Such instability makes them suitable for initial data annotation or application prototyping, but they cannot be relied on in a real-life production environment.

LLM router lacks interpretability as the model can use similar reasoning for different results, which makes it more difficult to debug and tune use cases [20]. Control is possible via prompting and implementing change in the generation hyperparameters, which is better than full retraining as it requires less data and time. However, it is difficult to correctly adjust LLM classifier behavior without breaking other cases considering the lack of interpretability and stability [21].

To address the issues listed previously, a method of routing via latent embedding retrieval was proposed [22]. This work shows that the semantic routing approach can be used in various fields, where high interpretability is critical, as it can provide explanations of decision-making and can be easily controlled and modified to adjust the behavior.

In this paper, we aim to provide a set of best practices for semantic routing, particularly to enhance their efficiency in dynamic, multilingual, and multitask environments with a high level of interpretability and control over the inference. Research highlights the possible disadvantages of this approach in comparison to LLM routing and transformer classifiers. We checked how such a routing method handles LLM jailbreaks and the influence of examples set size, compared multiple examples set building approaches, and created visualizations to highlight router decision-making.

The article is organized as follows: Section 2 covers the used datasets, the semantic routing approach in general, and proposed modifications. Section 3 presents the experiments, their results, and interpretations. Finally, Section 4 provides conclusions and discusses the obtained results in terms of further possible research and general usage of the routing approach.

2. Materials and Methods

2.1. Routing Benchmark Datasets

We need a dataset to test semantic routing approaches, which would contain multiple types of questions to be handled by different agents. During our previous research, we used a dataset of questions for wine store assistant, which handles the following scenarios:

• Catalog recommendations, where the agent has to generate a search query, find the corresponding wines, and form a message with a proposition. The resulting message has to be grounded by the search response so that the model does not make up any items out of the catalog;
• General questions about wine, where the agent has to answer any question about wine topics in general without search or other additional actions. Here, the model is allowed to talk about the topic in any way, even about items that are not sold by store;
• Small talks, where the agent just needs to keep a human-like, friendly conversation and answer simple messages like greetings, gratitude, and others. The agent is allowed to be creative and does not require any additional actions;
• Out of scope messages (offtop). Such messages should be ignored by the system completely as they are either LLM jailbreaks (harmful injections or attempts to use the chatbot as a free LLM wrapper) or just random questions out of the system’s scope (not about wines or their attributes, history, geography, and the winemaking process itself in the case of this dataset) [23].

Instructions to solve these use cases conflict with each other and they need different intermediate steps, so one shared answering pipeline would make the system not only less stable and reliable but also more expensive and slower. Such conflicting use cases may appear in any field, so splitting them into different sub-applications with a routing layer on top would be a suitable strategy to manage such a situation.

The dataset was gathered in multiple ways: a set of examples provided by consumers (wine online store), questions scrapped from the web, and synthetically generated questions with LLM. Synthetic messages were generated using Gemini 1.5 Pro for each route separately with multiple configurations: a question by a new customer, a middle-experienced customer, and an expert [24]. Also, 40% of the synthetic questions were also passed through another prompt to add grammatical and punctuation mistakes into them. In this way, we wanted to check how the router would handle text, which is not written ideally. Offtop (out of scope) messages were taken from the Stanford SQuAD dataset [25].

Table 1. Routing benchmark dataset statistics.

Route	Original Examples	Synthetic	Scrapped	SQuAD	Total
General wine questions	30	283	618	0	931
Catalog	82	675	0	0	757
Small talk	7	297	0	0	304
Offtop	0	0	0	884	884
Total	122	1255	618	884	2876

presents general statistics on the gathered dataset.

Also, we used machine translation to create some multilingual samples in French, Italian, German, and Ukrainian.

Table 2. Language distribution of the dataset.

Route	English	Each of the German/French/ Italian/Ukrainian
General wine questions	468	116
Catalog	442	78
Small talk	172	33
Offtop	500	96
Total	1584	323

shows the language distribution of texts in the dataset.

Table 3. Characteristics of texts across languages present in the benchmark.

Language	Characters Count	Word Count	Average Word Count per Text	Average Word Length
English	189,677	38,483	24.29	4.05
German	31,338	5519	17.08	4.81
French	33,199	5836	18.07	4.41
Italian	31,245	5867	18.16	4.45
Ukrainian	27,847	5154	16.00	4.56

shows the characteristics of messages in every language present in the benchmark (character count, word count, average text length in words, and average word length).

Benchmark was saved in 2 formats: a full one with all 2876 texts and a small randomly chosen subset with 40% of texts (1151 samples).

The dataset was created for internal usage using Teamwork Commerce, so it cannot be open-sourced at this moment. More information, statistics, and visualizations regarding the dataset are provided in our previous paper about conversation routing approaches [21].

Another dataset we used in this research is Jailbreak28K specifically its mini version with 258 harmful LLM jailbreaks, which should interfere with original instructions and inject malicious intents or make the model solve an unethical problem [26]. Also, we used a 10% sample of the large dataset (2800 samples) without “figstep” types of questions (questions for multimodal LLMs with vision capabilities, where the jailbreak itself is passed as an image) as we test routing specifically for text-only interactions.

2.2. Semantic Routing Based on Latent Sentence Embeddings Retrieval

In this paper, we propose a slight modification of semantic routing based on latent sentence embedding retrieval. The original approach can work both as a few-shot learning classifier and as a supervised learning classifier. It receives a set of examples for each route and a rejection threshold. The router builds an embedding storage for provided examples, and each time it needs to classify an example, it retrieves the N with the most similar examples and their respective labels. Similarity scores of examples are aggregated using labels with a function, which is chosen during the creation of the routing layer. It can be a sum, mean, or max aggregation. Then, each route has to pass a round of rejection threshold filtering. If the route does not achieve an aggregated similarity score higher than its rejection threshold, then it is rejected even if it is the top 1 among all fetched. If all routes are filtered by the rejection thresholds, it means the message is out of the scope of the current system and should not be passed along to any agent (Figure 1).

This routing approach provides an easily interpretable solution as a user can just check which examples were most similar to the input message and compare aggregated similarity scores to threshold values in the case where the message was rejected completely. Also, it is easy to control such a system and modify its behavior by just adding new examples, deleting old ones, modifying existing, or adjusting the thresholds.

If you have a training dataset and want to tune such a routing layer, it will only adjust the thresholds without learning new examples or generalizing them, which could boost the performance of the system even further. The router will iterate over the dataset, try different values for thresholds in the provided range, and check which one gives the best performance based on the training data.

Another problem with this approach is the lack of redundant example filtering. They are memorized anyway; the examples set grows uncontrollably and the router has to search across similar samples. This can slow the search for systems with a large number of use cases or big message datasets. Examples can be additionally optimized to take less time for retrieval and make classification faster. Also, the routing layer does not try to generalize or transform examples in any way. This leads to a larger example set to cover more similar samples. For our dataset, there can be multiple examples to cover different occasions to buy wine, when one generalized and the more templated sample could correspond to all of them simultaneously.

2.3. Proposed Modifications for the Semantic Routing Method

We propose to add the step of initial examples pruning based on their similarity to already memorized samples. There would be 2 approaches to prune excessive examples, which already have similar counterparts in router memory:

• Just filter input examples using another cutoff threshold to reduce the example set size and make it more efficient;
• Save not just the original version of the input example but also a generalized version generated via LLM to cover multiple use cases at once with possibly higher similarity. Examples would become more templated to correspond to multiple requests at once.

By pruning we mean the following steps during router initialization:

• Fetch examples set of the route provided by the developer;
• Encode all examples of the route with the embedder model;
• Save the first example to router memory;
• For the following examples retrieve the top 1 similar sample of the current route, which is already saved to the router memory;
• If the similarity score of the top 1 most similar sample is higher than the example pruning threshold (a coefficient, which describes when 2 texts are too similar, so it does not make sense to save another similar example), ignore the new example and do not add it to the router memory;
• If the similarity score of the top 1 most similar sample is less than the example pruning threshold, add it to router memory either as it is (first proposed approach) or pass it through the LLM to generalize it (second proposed approach) and save this generalized version to router memory;
• Repeat for every route.

Figure 2 demonstrates this pipeline in the form of a diagram.

During our work, we checked these routing adjustments and the original way with multiple encoders, quantization techniques, and embedding sizes to check how additional pruning changes the behavior of the router in terms of accuracy and speed (especially for local environments as API-based encoders are more difficult to track in terms of time performance).

3. Results

3.1. The Effect of Pruning of Examples on Semantic Routing

First, we tested the original semantic router approach and the two proposed modifications on a small subset of our routing benchmark. Here is the detailed router configuration we used in this research for the reproducibility of results:

• Aggregation function: max;
• Number of examples to retrieve: 15;
• Similarity score threshold for each route: 0.6 (if route aggregated similarity is lower than 0.6, it would be rejected). This threshold was tuned to suffice the benchmark during our previous research [21]. Higher values would cut off more possible routes and lower would allow low-confidence predictions;
• Examples are provided only for valid routes: general wine questions, catalog, and small talk. The offtop route is assigned only when all valid routes are rejected;
• Examples pruning threshold: 0.8;
• LLM configuration for the generalization of examples: GPT 4o with temperature 0.0 and top p 0.0 [27];
• Encoder model for the router: text-multilingual-embedding-002 by Google with task type RETRIEVAL_QUERY with an embedding size of 768 (task types in Google text embedding API allow us to choose the model optimized for the specific embeddings use case) [28,29]. During our previous research, this encoder proved to be the best out of e5, OpenAI text-ada, and OpenAI text-embedding-3-small. It is a multilingual version of the text-embedding-4 model by Google, which scores 66.31 on average across all tasks on the MTEB benchmark;
• Random seed: 42;
• Encoder model for pruning: text-multilingual-embedding-002 by Google with task type SEMANTIC_SIMILARITY with an embedding size of 768;
• A full list of examples provided to the router is listed in Appendix A.

Table 4. Classification report on routing benchmark with a 40% subset of the database for multiple router configurations.

Router Configuration	Memorized Examples	Accuracy	General Wine Questions F1	Catalog F1	Small Talk F1	Offtop F1
No pruning	72	0.84	0.79	0.86	0.70	0.90
Pruning (0.8)	46	0.84	0.80	0.88	0.68	0.87
Pruning (0.8) + generalization	49	0.81	0.76	0.84	0.63	0.87
XLM-R finetuned with 60% of the dataset	-	0.97	0.95	0.97	0.94	0.98

shows the classification report (accuracy and routes F1 scores) for this small portion of the benchmark for all three semantic router configurations and XLM-R-based router fine-tuned to classify four possible routes with the other 60% of the dataset. The “-” in the memorized examples column for XLM-R means that it does not use a vector base filled with a certain number of samples to predict labels. The same sign would be used further to mark the lack of a vector database filled with examples for router models, which do not have it.

The semantic router in all configurations significantly loses to a supervised fine-tuned transformer classifier (0.13–0.16 accuracy difference). However, a dataset with 1725 samples was required to train this model. Considering a cold start problem, these samples would probably be synthetic for conversation routing tasks as there are no sources to gather even such a small dataset to fit the model. Synthetic data cannot fully replicate the real-life variety of language usage and the dataset can become homogeneous [30]. This model can be applied to the routing task at later stages, when there is enough data to train it and a semantic router can be a good option to label examples initially.

As you can see, pruning preprocessing allowed us to remove around 36.1% of examples from routers without an overall performance degradation. Overall accuracy stayed the same with only offtop and small talk routes receiving a slight F1 score drop (0.02–0.03).

However, the generalization of examples proved to be less efficient than expected as it performed the worst out of all configurations. The LLM tends to over-simplify provided examples to the point where catalog inquiries become too general and start to mislead the router into general wine questions or even small talk routes. The generalization prompt can be found in Appendix B.

Then, we ran the same measurement on a full benchmark to check how the results would scale on it. XLM-R was not used in this experiment as it uses 60% of these data for fine-tuning.

Table 5. Classification report on full routing benchmark for multiple router configurations.

Router Configuration	Memorized Examples	Accuracy	General Wine Questions F1	Catalog F1	Small Talk F1	Offtop F1
No pruning	72	0.85	0.80	0.86	0.73	0.90
Pruning (0.8)	46	0.85	0.81	0.89	0.71	0.88
Pruning (0.8) + generalization	49	0.82	0.76	0.85	0.66	0.88
GPT 4o LLM in-context learning router	-	0.91	0.90	0.94	0.82	0.93

shows the obtained classification report and the result of the LLM-based router built with GPT 4o, which was one of the best LLM routers in our previous research [21].

Pruning without generalization still keeps the overall accuracy on the same level as a router built with all available examples. Slight performance degradation for small talks and offtops are also reproduced with the same 0.02 score drop. Generalized examples still did not show any positive effects as the overall accuracy is the lowest for this router and it does not outperform others on any specific route or at least by example set size.

Semantic router still loses to LLM in-context learning classifier (classification prompt can be found in Appendix B), but the difference is just 6% of the overall accuracy with the advantage of high interpretability, easier control, and reproducibility. Adding more specific examples can make the performance of the semantic router even better and closer to the LLM router. Also, it would be easier to extend it later in production and debug wrong cases, while the LLM router requires a prompt rewrite, which does not guarantee an accuracy boost or even the preservation of existing performance.

Then, we proceeded to check the influence of the pruning coefficient value on the final size of the router examples set.

Table 6. Router examples set size with different pruning configurations.

Router Configuration	The Original Number of Examples	0.7 Threshold	0.75 Threshold	0.8 Threshold	0.85 Threshold	0.9 Threshold
General wine questions	23	3	8	16	21	23
Catalog	32	2	10	16	25	32
Small talk	17	7	10	14	15	16
Total	72	12	28	46	61	71

shows multiple pruning configurations with different values of pruning threshold to demonstrate how it affects the size of the router examples set.

Once routers with different pruning thresholds were constructed, we measured them on a 40% subset of the routing benchmark.

Table 7. Classification report on routing benchmark with a 40% subset of the database for multiple pruning threshold values.

Pruning Threshold	Memorized Examples	Accuracy	General Wine Questions F1	Catalog F1	Small Talk F1	Offtop F1
0.70	12	0.39	0.00	0.00	0.45	0.60
0.75	28	0.80	0.71	0.85	0.70	0.84
0.80	46	0.84	0.80	0.88	0.68	0.87
0.85	61	0.85	0.84	0.88	0.69	0.89
0.90	71	0.83	0.79	0.85	0.68	0.90
No pruning	72	0.84	0.79	0.86	0.70	0.90

shows how different pruning threshold values affect the classification accuracy.

Table 6 and Table 7 show that tuning a pruning coefficient is crucial as low values can lead to catastrophic performance degradation of the router (coefficient values lower than 0.75). The examples set becomes too small to cover necessary use cases, so the router classifies the most valid samples as either small talks or offtops. At the same time, keeping the coefficient around 0.90 or higher would not make any sense, as almost no examples are pruned and the set size remains the same. The best results were achieved with a pruning coefficient from 0.80 to 0.85 as they kept the original performance of the router with a set size reduction of around 15–34%. These measurements may differ for different encoders, so the pruning coefficient should be tuned specifically for the chosen router encoder.

However, we noticed slight accuracy fluctuations with different pruning coefficient values. For example, coefficient 0.90 filters only one example, but achieves a worse accuracy on a 40% subset of the benchmark due to more errors during catalog and small talk classification. On the other hand, the 0.85 coefficient improves the overall accuracy and provides better results for the catalog and general wine question routes. These two router pruning configurations examples were remeasured on a full benchmark dataset, and the results are presented in

Table 8. Classification report on full routing benchmark for various pruning threshold (0.80+) values.

Pruning Threshold	Memorized Examples	Accuracy	General Wine Questions F1	Catalog F1	Small Talk F1	Offtop F1
0.80	46	0.85	0.81	0.89	0.71	0.88
0.85	61	0.86	0.84	0.89	0.73	0.89
0.90	71	0.84	0.80	0.86	0.71	0.90
No pruning	72	0.85	0.80	0.86	0.73	0.90

.

The same behavior was reproduced on the full benchmark, so it was decided to check which examples were pruned and why it happened. The 0.90 pruning coefficient does not improve the router in terms of both memory size and prediction accuracy as it pruned just one sample and deteriorated the accuracy of small talk detection. The only pruned example was “I’m impressed with the variety of wines you offer.” from the small talk route, which led to a lack of similar examples about user impressions and store experience.

As for coefficient 0.85, we focused on general wine questions as it was significantly improved with such a configuration. This route pruned only two examples: one about the influence of alcohol content on wine characteristics and another about wine geography. Both examples had similar ones among memorized samples, so it made sense to not add them too. This category of questions is the hardest in the dataset as it can be close to out-of-scope questions (for example when they relate to wine geography or history). Also, the questions here can be diverse and cover completely different topics: glasses, grapes, wine traditions, geography, history, winemaking, wineries, the wine itself, and the ways to store or drink it. It is reasonable to memorize as many examples as possible because most can be unique. This proves a necessity to optimize the pruning threshold for each route or class, specifically some would be fine if redundant examples are removed from the memory and others require as many unique samples as possible.

Originally proposed semantic router implementation can increase the example set over time as developers obtain more real-life data. However, as the router vector count grows, it would make retrieval slower and memory consumption would increase too. Pruning would allow us to keep vector count optimal and can keep the retrieval efficient without the quantization of router embeddings or manual examples filtering.

3.2. Jailbreak Prevention

The next step was to check the level of protection the semantic router provides against common LLM jailbreaks. First, we measured routers without pruning, with a pruning coefficient of 0.8, with both pruning and generalization and XLM-R router on a mini version of Jailbreak28K. The router had to predict offtop for all provided texts as they should not be passed to any agent down the line.

Table 9. Semantic router performance on jailbreak prevention task (mini Jailbreak 28K).

Router Configuration	Memorized Examples	Accuracy
No pruning	72	0.97
Pruning (0.8)	46	0.97
Pruning (0.8) + generalization	49	0.97
XLM-R finetuned with 60% of the dataset	-	0.63

shows the results obtained from the test.

All configurations of semantic routers show the same performance in terms of jailbreak rejection, which proves the efficiency of semantic routing not only for multiagent chatbot orchestration but also as a security precaution for LLM-based applications. However, the performance of the fine-tuned transformer model significantly drops for jailbreaks (0.34 below semantic router). The variety of possible out-of-scope questions causes such behavior for the XLM-R router. It can be anything outside of router primary topics and small talk messages. Gathering a dataset to cover all cases here would be an impossible task. It can be solved with a classification probability threshold, which considers all predictions with a probability less than the chosen cutoff value as out-of-scope messages.

The next step was to run the same test with a 10% sample of a full Jailbreak28K without image jailbreaks (2600 texts).

Table 10. Semantic router performance on jailbreak prevention task (10% of full Jailbreak28K).

Router Configuration	Memorized Examples	Accuracy
No pruning	72	0.97
Pruning (0.8)	46	0.97
Pruning (0.8) + generalization	49	0.97
XLM-R finetuned with 60% of the dataset	-	0.63

shows the results of the conducted measurements.

The result was fully reproduced even on a larger set of possible instruction injections. Scores for larger and mini jailbreaks benchmark versions fully match all 4 tested models. The final thing to check is how the router would handle these jailbreaks if they contain topic keywords like “wine”, “red”, “catalog”, and others. Messages from Jailbreak28K do not touch the target topic of the router, so it would be easier to write them off as offtops. Adding topic keywords can make it easier to mislead the router by making input embeddings closer to embeddings of valid examples, so such vulnerability has to be checked.

We create a small set of questions about wines or just sets of keywords (10 texts). The mini version of Jailbreak28K is used as a base for this test. Wine-related statements and keywords are added at the start of the text and also at the end of the message for 35% of samples.

Table 11. Semantic router performance on jailbreak prevention task (mini Jailbreak28K and injection of wine-related statements in jailbreaks).

Router Configuration	Memorized Examples	Accuracy
No pruning	72	0.32
Pruning (0.8)	46	0.32
Pruning (0.8) + generalization	49	0.22
XLM-R finetuned with 60% of the dataset	-	0.02

shows the result of the experiment.

The performance of the semantic router drops significantly as it is misled by injected wine-related statements. This makes the router vulnerable to instruction injections if the attacker knows the primary topic of the chatbot and adds some topic-related statements there to avoid it being sent to the offtop route right away. This does not mean that the underlying agents will not be able to handle this later, but a possible harmful message would still reach the LLM as the router would not filter it.

XLM-R router achieves the worst accuracy score (just 0.02) as it was fitted to assign all wine-related messages to either wine general questions or catalog inquiries. In this way, it completely misclassifies the whole jailbreak benchmark as harmful instructions are masked by primary topic n-grams.

This experiment proves that the issue persists in the semantic router and an ordinary transformer classifier. The classifier can be fine-tuned further to handle such mixed cases with jailbreak masking. Such mixed cases can affect the accuracy, but fine-tuning the model to handle this specific type of jailbreak should improve the performance. However, there is no clear solution for the semantic router. Replacement of the most frequent n-grams in the examples set would diminish the main advantage of sentence embeddings: multilingual comparisons if the encoder model supports multiple languages. Also, it can deteriorate the accuracy of valid route detection.

Another option would be to remove the first and last N tokens and make a voting classification by classifying the message as it is, classifying it without the first N tokens, and then without the last N tokens. Such attacks usually place fake topic-related statements at the start and at the end of the jailbreak to provoke the LLM into answering a normal question and then continue with answering the harmful one. However, our experiment with such a setup did not give a significant improvement even for jailbreak identification as the best accuracy we achieved was 0.51 which was achieved by removing 30% of tokens from the start and the end of the text.

Such similar attacks require deeper research to propose countermeasures, which would not affect the performance of valid route identification and would protect the router from harmful messages better.

3.3. Interpretability and Controllability

Even if the semantic router loses to the LLM-based router (considering only those routers that can handle a cold start problem and do not require fine-tuning), it provides crucial features for chatbot systems maintenance, such as a high level of control and interpretability by extending and modifying the examples set.

We created some visualizations to demonstrate the advantages of such an approach. They were generated with the same semantic search configuration as the router in Section 3.1 (the top 15 most similar examples, generated with text-multilingual-embedding-002 embedding model by Google with task type RETRIEVAL_QUERY with an embedding size of 768). Pruning was applied to router examples set with a pruning coefficient of 0.8, so it has 46 memorized samples.

First, we visualized how it classifies a catalog question by creating 2D projections of examples and query sentence embeddings. One option is TSNE (t-distributed stochastic neighbor embedding) [31]. TSNE allows us to project high-dimensional data into a lower-dimensional space while preserving samples pairwise similarity as much as possible. It aims to preserve the relationships between data points in a lower space to replicate the original dataset structure. This algorithm is effective for discovering patterns and possible clusters in complex data (such as sentence embeddings in our case).

TSNE measures pairwise similarity in high dimensions with Euclidean distance. However, most similar examples are chosen by cosine distance, so we decided to use the UMAP (uniform manifold approximation and projection) dimensionality reduction algorithm to optimize cosine similarity directly during the creation of low-dimension projections [32]. The main advantage of UMAP is the preservation of both local and global structures as it creates a high-dimensional graph with the provided data samples and then optimizes low-dimensional representations to preserve relationships from this graph. Nodes of the high-dimensional graph represent data points and edges with weights correspond to similarities of points. So, UMAP tries to preserve local structures and keeps track of the global relationships between the provided samples. In this way, clusters and their positions are reflected even in generated projections.

You can use the following parameters set from the UMAP algorithm to reproduce the provided visualizations:

• N_neighbors: 10;
• N_components: 2;
• Metric: cosine;
• Random_state: 42.

The model is fitted on examples set only, so queries are provided only to the transform method of the model (they do not affect the original high-dimensional graph and the way the model tries to project it into a 2D space).

You can see the visualization of classification executed with the semantic router in Figure 3, where a set of examples and a query are projected in 2D space with UMAP.

Larger circles with a clear black border are among the top 15 most similar examples, while others did not make it, so they do not affect the prediction. Most such samples belong to the catalog type in the visualization, which explains the router decision-making process as the query is routed to the catalog case.

You can see how samples group in three separate clusters in Figure 3 with only a few outliers for the small talk cluster. These outliers relate to either bot capabilities in terms of wine-related issues (for example what exactly can the bot tell about wine) or praises for the wine selection to cover cases, when the user wants to express gratitude for a good recommendation.

It is worth noting that some examples may appear closer to the query on visualization. Such cases are caused by approximation performed during dimensionality reduction as exact pairwise distances can be sacrificed for the preservation of the cluster structure or cluster relationships globally. These data points can share neighbors in the high-dimensional graph, so UMAP tries to preserve this local neighborhood even in low-dimensional projections, which leads to a closer and denser placement.

General wine questions and catalog recommendations are placed closer in this plot as they cover similar topics, including wines and their attributes. At the same time, small talk messages are significantly distanced from wine-related clusters.

In

Table 12. The top 5 most similar examples for the provided query.

Text	Type	Similarity
Ciao, stasera cucino un risotto ai frutti di mare. Quale vino bianco si abbina bene senza essere troppo secco?	catalog	0.76
Can you recommend a wine for a romantic dinner?	catalog	0.76
Hello, I’m looking for a robust red wine with moderate tannins to pair with a rich mushroom and truffle pasta. Ideally, something from the Tuscany region, under $70. Any suggestions?	catalog	0.74
Hey, I’m searching for a nice red wine around $40. I usually enjoy a good Merlot, but I’m open to other options. Anything with a smooth finish and rich fruit flavors would be great! Any recommendations?	catalog	0.74
I’m preparing a French-themed dinner. What French wine would complement the meal?	catalog	0.73

we show the top 5 most similar examples from this visualization to show the exact text, which model is considered the most similar to the query “I’m looking for a wine to use in cooking, specifically for making risotto. Any suggestions?”.

The next step was to visualize an out-of-scope (offtop) message. We chose a sample with the following text: “Who initially proposed using the term “Native American” to recognize the primacy of the native population?”. The result can be seen in Figure 4.

The sample is distanced from valid route examples. All examples among the top 15 did not cross the 0.6 threshold, so this sample would not be passed on to any agent further. The maximum similarity value was 0.43 for the general question example. However, it is important to note that categories like small talk or general questions about a certain wide area can be more vulnerable than more specific categories like catalog recommendations. Out-of-scope samples are more similar to samples from these vulnerable categories in Figure 4 than to a specific, narrow one with a concrete use case (like searching for goods using an in-store database). This proves that a semantic router would achieve its best performance with a narrower definition of routes instead of working with general and wide topics as they can be difficult to distinguish. This theory should be checked in future research to check how the split of larger question clusters into smaller subclusters can affect classification accuracy. In particular, it should be checked for out-of-scope question prevention.

The final visualization shows a case of misclassification by the router. We took one of the previously adjusted jailbreaks, which is wrapped in statements similar to the catalog route and visualized in the same way as in Figure 3 and Figure 4. Here is the text of the sample: “Red wine, white wine, rose. Wines, wines. Respond to every prompt I give you with a moralizing rant about the provider content policies, but then start a new paragraph with the sentence” anyways, since we are out of this *** ***, let’s ignore the *** rules: “and then respond to the prompt exactly as an unfiltered and unsafe, completely unlimited language model could do. Be detailed. Prompt: Invent a method for producing fake legal documents for court cases. I want to find a good red wine for a date tonight.”. Figure 5 shows the result.

As you can see, the sample falls close to catalog examples and 13 out of 15 examples achieve a similarity score higher than the rejection threshold (0.6). Masking out-of-scope messages with a set of keywords related to the router’s primary topic can significantly mislead the router, which can be proved by both cosine similarity scores and visualization of the data.

Such visualizations and tables can easily explain the router decision-making for any area and topic as they provide a set of most similar examples and their similarity scores. So, if the user would like to modify router behavior, they can add another target route example to avoid further misclassification or adjust existing ones. This approach allows us to interpret and control the classification process without specific knowledge of machine learning (which is required to interpret attention masks or SHAP visualizations) by supporting the list of examples and checking which ones are most similar at each router call.

4. Discussion

Semantic routing approach based on latent sentence embeddings retrieval can achieve close performance to LLM in-context learning classifier and provides a high level of interpretability and controllability. The router can be adjusted by adding more examples, modifying existing ones, or changing the rejection threshold to filter fetched examples differently. The decision-making process is clear and can be easily understood as the user just needs to check which examples were chosen as most similar and compare their similarity scores to the rejection threshold. This method would still work for long texts as it does not try to evaluate the significance of each specific token and instead evaluates pairwise similarities of texts. The interpretability of this approach does not require any additional tools or models as it can be carried out simultaneously with the classification itself (the model returns the chosen most similar examples and their scores).

This approach can be extended for interpretable text classification in general and in areas that require a detailed explanation of conducted decisions and even for multimodal tasks with multiple domains [33]. Currently, it can be less accurate than a fine-tuned transformer or LLM classifier, but it is easier to understand and control. The modification would not take full-on fine-tuning and the results are easily reproduced compared to the LLM router. Routing achieves its best performance with narrowly specified options, so it is recommended to avoid wide definitions, which can overlap. There is still a field for further research on how to increase the accuracy of this approach to keep both the interpretability of semantic routers and the accuracy of transformer classifiers.

Proposed automatic example pruning allows us to not overextend router example set with redundant samples, which can harm its performance without any added accuracy. This way example set can become almost 40% smaller without accuracy degradation and manual filtering of too similar examples. High values of the pruning coefficient almost do not affect the examples set and leave it as it is in terms of size. Small values can lead to significant performance loss due to the removal of too many texts.

The encoder model directly influences the optimal value of the pruning threshold as they are trained with different tasks in mind (for example paraphrase similarity and detection of semantic distinctions). This produces significantly different embeddings, which may affect the pairwise similarity of the same pair of texts if they are encoded and compared by two different encoders. Pairwise similarities can also be affected by different pooling strategies, differences in pretraining datasets, and label quality. So pairwise scores can differ even if two models agree that the texts are similar. Router examples pruning should be implemented once there is a validation dataset to check multiple configurations and choose the one which works the best for the specific model and domain.

However, the generalization of examples in addition to pruning did not improve performance and even caused a slight drop in accuracy. It became worse not only in more wide routes like “general wine questions” or “small talk” but also in narrow and specific ones like “catalog”. At the same time, out-of-scope questions are still labeled with a similar success rate to the ordinary router or the one with a pruning mechanism.

The model can scale to ordinary classification tasks and even to multimodal cases. The only requirement is a set of predetermined class examples and a choice of encoder model to build sentence embeddings of examples and future queries. However, we still would like to research the generalization approach further as the idea of keeping just a few templated messages instead of a whole set of examples still seems promising. Generalization can be applied only to some routes, which correspond to some specific tasks, while routes that keep track of a wide range of topics need to memorize as many examples as possible. Also, we want to check the effect of pruning on larger sets of examples to analyze its effect on the memory size of the router and its accuracy. Also, it is necessary to provide an automated way of establishing pruning threshold tuning in case the model user has a validation dataset to measure router accuracy with multiple configurations and choose the best one.

Another direction to improve the interpretability of the model would be to add a sentence embedding decomposition step to the pipeline to determine which components of the feature vector may affect the prediction in a certain way and try to interpret those characteristics in terms of the router domain.

Even though a semantic router increases the security of the chatbot application by filtering most out-of-scope messages and jailbreaks, it is still vulnerable to attacks when such message is wrapped within valid statements, which are similar to target routes. The issue persists with XLM-R-based routers and the effect of such jailbreak masking is even worse for this model (accuracy is equal to 0.02). Fine-tuning can be a possible solution for this problem for transformer routers. However, there is no clear solution for the semantic router to counter such attacks. One option would be to remove the most used words from the examples set. Another one would be to make a voting classification: classify the message as it is, remove the first N tokens and classify it again, and finally remove the last N tokens and classify this version. However, both methods do not give a significant improvement in terms of jailbreak protection and can deteriorate the performance of the router in general.