Web Search-Enhanced Small Language Models: A Case Study for a Kazakh-Centric Language Model

Small language models (SLMs) are valued for their computational efficiency and suitability for edge deployment, but often underperform in localized linguistic and cultural contexts due to their limited parameter size. This study explores integrating real-time web search into Qolda, a 4B-parameter Kazakh-centric SLM, to close the performance gap with larger models. We assess two strategies: Naïve Retrieval-Augmented Generation (RAG), which uses raw benchmark questions as search queries, and Query-Refined RAG, which applies various refiner models, including a supervised distillation-tuned Qolda, to optimize queries. On the KazCulture and KazMMLU benchmarks, the Naïve RAG approach in reasoning-enabled mode achieved an average accuracy of 76.00%, improving on the Zero-Shot evaluation result of 60.37%, and, in this system-level comparison, exceeding the Zero-Shot accuracy of larger open-source models such as Qwen3-32B (64.72%) and Gemma-3-27b-it (60.24%), which were evaluated without retrieval augmentation. Query refinement improved the accuracy by about 3%, from 76.00% to 79.46%, but nearly doubled the computational cost. Inference time analysis shows that Naïve RAG adds approximately 1 s of retrieval latency per question. Query refiners introduce up to 4 s of additional overhead. However, the retrieved context reduces the time required for model reasoning in think mode. The most notable gains were observed in localized cultural knowledge, where web search integration correctly answered 32.9% of KazCulture questions that the Zero-Shot baseline failed on, while losing only 16.9% in return. These results suggest that retrieval-augmented SLMs can offer a cost-effective and competitive alternative to larger models for tasks in the domains of Kazakh language and Kazakh culture.