Abstract
Digital humans must speak promptly, but thinking-enabled large language models (LLMs) answer only after reasoning. CATT (Complexity-Aware Thinker–Talker) routes easy turns to a direct reply; hard turns get an answer-free acknowledgment, progress speech from background reasoning, then the final answer. It needs no fine-tuning and is plug-and-play: its orchestration code runs unchanged when the model endpoint, router, or speech and playback layer is replaced. Evaluated on the transcript-to-speech subsystem (local Qwen3-32B, 738 isolated turns, and virtual playback), CATT starts speaking about 6.3 s earlier than a prompt-matched thinking baseline. The first substantive content comes about 10.5 s earlier on closed-form math and science questions and 3.0 s earlier on open-ended requests (about 1 s later when routed to reasoning). The final closed-form answer arrives about 3.2 s later at the same accuracy. An LLM-judged listening rating, a proxy that includes waiting, places CATT about 1 point above the thinking baseline but 0.42 below a non-thinking baseline on open-ended requests. Both latency gains held for three model families accessed through hosted APIs and in an end-to-end avatar case study. A limited ReVerT reproduction on the same reasoning started speaking 8.4 s after CATT. Real users, microphones, networks and multi-user load remain untested.
1. Introduction
A spoken digital human is an interactive system first. In every turn it has to recognize what the user said, decide on a response, synthesize speech and start playback. Human conversation is organized turn by turn [1], and the gaps between turns are typically short across languages [2,3,4]. Response delays and badly timed interruptions remain a recognized difficulty for conversational systems and social robots [5]. A study of LLM-powered virtual agents examined conversational fillers as a way to bridge response delays and improve perceived waiting time [6]. This systems view follows the embodied-conversational-agent and virtual-human literature, which treats speech, dialogue and visible behavior as parts of one interactive interface [7,8]. The present paper concerns one part of that interface: how soon a digital human starts to speak after a question and what it says while a reasoning model is still working on the answer.
Some of the questions such a system receives benefit from deliberate reasoning, for example multi-step problems in mathematics and science. With the current large language model (LLM) backbones, responsiveness and reasoning accuracy then pull in opposite directions. The deployment evaluated in this paper shows the size of this tension. It serves Qwen3-32B [9] with vLLM [10], and both of its baselines receive the same spoken-answer prompt. Without thinking, the model starts speaking after a median of 0.57 s, but it answers only 44.4% of 214 closed-form questions correctly (3.3% on AIME 2026 [11]). With thinking enabled, the accuracy rises to 65.9% (56.7% on AIME 2026). A thinking model, however, emits its visible answer only after its reasoning pass, and a streaming speech pipeline has nothing to synthesize until that answer begins. Over all 738 evaluation items, the median wait before the first audio therefore grows to 6.98 s and 19.32 s on the closed-form questions. Its 90th percentile is 33.71 s, and the longest wait was 327.43 s. Therefore, in this deployment, enabling thinking postpones the start of the spoken response by seconds on typical turns and by minutes on the hardest ones.
A natural systems solution is to speak first and keep reasoning in the background. Protocols that think before or while speaking point in this direction [12,13,14,15], and incremental dialogue processing provides a broader precedent for acting on partial information as it arrives [16]. For deployment, the layer that does this should attach to a model the operator already uses. It should not change model weights, modify the decoder or require a specialized speech-language model. Its evaluation also needs care because speaking early is not the same as answering early. A system that fills the wait with speech can report a small first-audio latency while its answer arrives no sooner or even later. This paper therefore asks two questions. Can a deployment-only orchestration layer that is attached to a standard streaming model endpoint let a turn that needs reasoning begin speaking and convey question-specific content before the reasoning finishes? And what does this cost in final-answer time, accuracy, intermediate errors and computation?
To answer the second question, we separate four events on the playback timeline of each turn. The acknowledgment is the first audio on a turn routed to reasoning; it acknowledges the question and names its topic, but it gives no answer. First substantive content is the first sentence that gives question-specific information beyond the question text as identified by an LLM judge. It includes the solution approach and intermediate results, not only answers. On closed-form questions, the first candidate answer is the first explicit candidate answer heard outside the acknowledgment, and the final answer is the moment at which the answer that the turn finally commits to starts playing. On turns answered directly, and in both baselines, the first audio is the answer onset, and the four events can coincide. Every first-audio value in this paper is labeled accordingly as an acknowledgment or an answer onset, and systems are compared event by event rather than through a single latency figure.
We present CATT (Complexity-Aware Thinker–Talker), an orchestration layer for spoken responses that sits between a finalized user transcript and the speech synthesizer (Figure 1). Complexity is operationalized as the expected benefit of enabling the deployed model’s own thinking mode rather than an abstract task-difficulty label. A router decides for each turn whether it takes the fast path or goes to reasoning. Easy turns on the fast path are answered directly by a non-thinking talker. On a hard turn, routed to reasoning, a thinker streams the model’s reasoning in the background while the talker speaks. The talker first gives a one-sentence acknowledgment. It then produces progress speech, rewriting recent reasoning into spoken sentences when new reasoning is available and the audio queue runs low, and it finally gives the answer. To keep routing off the critical path, the reasoning request and the acknowledgment call start together with the routing decision; the acknowledgment is held back until the route is known. When the reasoning ends, queued progress speech that has not started playing is dropped, and the final answer plays after the sentence being played has finished. Two safeguards restrict intermediate speech: the acknowledgment may not state an answer, and progress speech must mark any candidate answer as tentative. CATT needs no fine-tuning, decoder modification, or access to model internals. It uses a streaming chat-completion interface and, when the endpoint exposes one, the documented reasoning channel.
Figure 1.
Architecture of CATT and the timeline of one turn on each route. (a) The orchestration layer sits between speech recognition and speech synthesis. It uses the model only through its serving interface without fine-tuning or access to model internals. The controller is the only writer of the workspace. At the start of a turn it starts the complexity router, the thinker call and the acknowledgment call together, and it holds the acknowledgment until the route is known. The router, the model endpoints and the speech and playback layer (gray dashed outlines) can be replaced without changing the orchestration code. The thinker and talker engines serve the same model, locally or through a hosted API. In the main study, both engines run on shared GPUs. Piper synthesizes the speech, and playback is virtual and computed from real synthesis times. The case study adds speech recognition, CosyVoice synthesis and a browser avatar. (b) On a reasoning route, the listener first hears the acknowledgment, then progress speech while the thinker reasons, then the final answer. The circled numbers mark what happens when the reasoning ends: the progress call in flight is canceled (1), the sentence being played finishes (2), queued progress speech that has not started is dropped (3) and the final answer follows (4). On an easy route, the thinker call and the held acknowledgment are canceled, and the first audio is the answer onset. Panel (b) is schematic and not to scale.
We call CATT plug-and-play in a specific sense: its orchestration code runs unchanged when the model endpoint, the router, or the speech and playback layer is replaced. The term describes the interface, not a performance guarantee. Each of the three was replaced in this study. Besides the local Qwen3-32B deployment, the model endpoint was one of three model families accessed through hosted APIs (DeepSeek V4.1 Flash, GLM-5.2 and Qwen3.5-27B). The router was either the CATT router or an off-the-shelf complexity classifier from NVIDIA [17]. The speech and playback layer was either Piper [18] with virtual playback or the CosyVoice API [19] with a browser-rendered avatar. Earlier first audio and earlier first substantive content carried over to the hosted models and the end-to-end setting. Speech continuity and final-answer timing, by contrast, depended on the deployment and have to be measured again for each one. For the same reason, the router is treated as a replaceable component rather than the core of the method. It is still useful. A separate comparison uses recorded forced-route outcomes for 170 closed-form questions and compares routing policies that send the same number of questions to reasoning. Averaged over all such budgets, from zero to 170 questions sent to reasoning, the router is more accurate than random routing, a length heuristic and the RouteLLM classifier [20]. It is on par with the NVIDIA classifier and a math-symbol rule. Compared with sending every turn to reasoning, the router configuration reaches similar accuracy (65.4% and 65.9%) with fewer model calls per turn (4.58 and 4.96) and earlier first substantive content (medians of 5.22 s and 7.36 s). These router results are limited to Qwen3-32B in the evaluated deployment, and the router was not recalibrated for the hosted models.
The main study evaluates the transcript-to-speech subsystem on isolated turns with virtual playback: each turn’s playback timeline is computed from measured synthesis times rather than played on a device. Its 738 items comprise 214 closed-form questions from AIME 2026, GPQA Diamond [21] and the MMSU subset of VoiceBench [22] and 524 open-ended requests from the AlpacaEval prompts of VoiceBench. CATT is compared with prompt-matched thinking and non-thinking baselines on the same deployment. The thinking baseline’s request is identical to CATT’s thinker request. On every hard turn the two therefore produce byte-identical reasoning, so timing differences on those turns reflect the orchestration rather than different reasoning. All the contrasts are paired over items with 95% bootstrap intervals, and accuracy is also compared with exact McNemar tests. The main quantitative results come from this local Qwen3-32B deployment. An end-to-end case study adds speech recognition on synthesized spoken questions, CosyVoice synthesis, a browser-rendered avatar, audio–visual synchronization and barge-in, all run in a headless browser on the server. Real users, real microphones, real networks and multi-user load are not covered.
The results support a narrower claim than faster answers. Against the thinking baseline, CATT with its router starts speaking 6.34 s earlier (median paired difference; 95% interval [, ] s; medians of 0.49 s and 6.98 s). On hard turns this first audio is the acknowledgment, and on the other turns it is the answer onset. CATT conveys first substantive content 10.46 s earlier on closed-form questions ([, ] s) and 3.00 s earlier on open-ended requests ([, ] s). On the open-ended requests that it routes to reasoning, first substantive content comes 1.15 s later ([, ] s) because their reasoning is short and leaves little progress to report. The final closed-form answer arrives 3.20 s later ([, ] s) with accuracy on par (65.4% versus 65.9%; difference percentage points, [, ]). On hard turns, the total silence between spoken segments has a median of 0.13 s and a 90th percentile of 0.90 s. The costs are stated alongside these results. CATT makes 4.58 model calls per turn instead of one. Its progress speech contains at least one incorrect intermediate statement in 58 of the 168 closed-form hard turns (34.5%; 25.2% of the 103 whose final answer was correct). Quality is assessed with a listening rating from two LLM judges that see the full timeline of what was heard on a 1–10 scale; it is a proxy, not a measure of human perception. On this rating, CATT scores about one point above the thinking baseline on both question types and 0.42 points below the non-thinking baseline on open-ended requests.
The main contributions are as follows:
- A deployment-only orchestration layer. CATT routes each spoken turn, acknowledges hard turns while the reasoning runs asynchronously, turns the reasoning stream into progress speech, and drops queued progress speech when the final answer is ready. It requires no fine-tuning, decoder modification or access to model internals. It is plug-and-play in the sense defined above, with the router as one replaceable component.
- An event-separated timing evaluation. Acknowledgment, first substantive content, first candidate answer and final answer are measured separately and compared item by item against prompt-matched baselines that share CATT’s reasoning request with an interval for every contrast. Favorable and unfavorable cells are reported together, along with intermediate-speech errors, a timeline-based listening rating and computational cost.
- Evidence beyond a single configuration. A routing comparison, a transfer study with three model families, a comparison with ReVerT [12] run from its official implementation on the same reasoning, a replay of the recorded runs under added latency and slower generation, and an end-to-end digital-human case study show which effects carry over and which must be measured again for each deployment.
The remainder of this paper is organized as follows. Section 2 reviews the related work and compares the method properties of CATT with those of the closest systems. Section 3 presents the architecture, the router, the talker modes and their safeguards, the asynchronous thinker and the timing model of playback. Section 3.8 describes the data, the deployment, the baselines, the event definitions, the judges and the statistics. Section 4 reports the main results, and Section 5 reports the additional studies. Section 6 discusses the settings for which CATT is suited. Section 7 states the limitations, and Section 8 concludes the paper.
2. Related Work
We review four threads of work that bear on a real-time spoken digital human. Section 2.1 covers turn-taking, streaming speech pipelines and the latency measures used to evaluate them. Section 2.2 covers end-to-end speech-language models. Section 2.3 turns to methods that let a system speak before or while a model reasons, which are the closest to CATT, and compares their deployment properties. Section 2.4 covers language-model routing and cascading. Throughout, we focus on the deployment assumptions that each line of work makes and what its latency measures capture.
2.1. Turn-Taking, Streaming Speech Pipelines and Latency Measures
Turn timing is a central constraint in spoken interaction. Conversation-analysis and phonetic studies show that speakers coordinate turns so as to limit both long gaps and disruptive simultaneous speech [1,2,3,4]. Conversational systems and social robots still struggle with interruptions and response delays [5]. Incremental dialogue-processing architectures address this problem by letting modules consume and produce partial information rather than wait for complete utterances [16,23]. Language-model approaches such as TurnGPT further use dialogue context to project upcoming turn completion [24]. In free-form conversations with LLM-powered virtual agents, Maslych et al. [6] study how conversational fillers affect the experience of waiting. Their user study in virtual reality finds that natural fillers combining gestures and verbal cues improve perceived response time at longer delays.
A related body of work treats real-time spoken interaction as a pipeline of components rather than a single model call. Modern cascades can draw on high-capacity speech recognition architectures such as Conformer and Whisper [25,26] and parallel or end-to-end text-to-speech (TTS) models such as FastSpeech 2 and VITS [27,28]. Synthesis can also begin before the full utterance is fixed; INPRO_iSS is an early component-level example of incremental just-in-time speech synthesis [29]. In a digital human, the same pipeline also drives a rendered avatar, which adds audio–visual synchronization and the handling of user interruptions (barge-in) to the requirements. This pipeline view shapes our measurements. Every time we report is measured from question submission, and in the main study the playback timeline is computed from the measured synthesis time of each spoken chunk (Section 3.7). The end-to-end case study adds speech recognition, a browser-rendered avatar and barge-in (Section 5.5).
Recent evaluation work examines the timing of spoken interaction directly. Talking Turns [30] evaluates how spoken dialogue systems understand, predict and produce turn-taking events, including gaps and overlaps. VoiceBench [22] argues that LLM-based voice assistants should be evaluated under speech-interaction conditions rather than on clean text alone. We use two user-side measures in the same spirit: the time to first audio and the silence between spoken segments. Neither measure, however, shows what the listener has heard by a given moment. Once a system speaks before its answer is ready, its first audio may be an acknowledgment rather than an answer, so a short wait for the first audio need not mean an early answer. We therefore time four events in every turn (Section 3.8.4). The first audio is an acknowledgment on turns routed to reasoning and the answer onset otherwise. The other three events are the first substantive content, which includes the solution approach, the first candidate answer and the final answer.
2.2. End-to-End Speech-Language Models
A separate thread integrates speech directly into the language model. SpeechGPT learns discrete speech representations for cross-modal conversation, while SALMONN connects speech and audio encoders to an LLM for general hearing and spoken-query tasks [31,32]. These systems establish broad speech-language capability, but low-latency spoken interaction also requires the model to emit audio incrementally.
Recent end-to-end models target this requirement more directly. LLaMA-Omni couples a speech encoder to a streaming speech decoder [33], while Freeze-Omni attaches speech input and output modules and keeps its text LLM frozen [34]. Their streaming speech output relies on speech-specific training, modules and decoding interfaces.
CATT places its thinker and talker in a separate orchestration layer. Each role uses an ordinary streaming chat-completion endpoint, and the calls exchange text. A separate synthesizer produces speech. This separation allows the model endpoint and the speech and playback layer to be replaced without changing the orchestration code.
2.3. Speaking While the Model Reasons
The closest work to CATT lets the listener hear something before a model has finished reasoning. Table 1 compares the properties of five such methods with those of CATT.
Table 1.
Method properties of the closest systems to CATT. Entries summarize each method as described by its authors.
Think, Verbalize, then Speak (TVS) [12] separates an internal thinking step from a verbalization step that rewrites the reasoning into speech-friendly text for a TTS system. Its incremental variant, ReVerT, uses a fine-tuned verbalizer model that reads the reasoning as it streams and decides, segment by segment, whether to speak. The reasoning model and the synthesizer stay unchanged. Discourse-Aware Dual-Track Streaming Response (DDTSR) [13] targets cascaded spoken dialogue systems. A small model produces discourse connectives that can be spoken at once while a large model reasons in parallel. Recognition, generation and synthesis run as concurrent streams, and a curriculum-learning stage targets continuity between the early speech and the later content. The authors report substantial latency reductions on a cascaded pipeline.
Two further methods act inside the model. Asynchronous Reasoning [14] is training-free. It lets one thinking model write its answer while it continues to reason, pausing the answer for more thought when warranted. To do so, it changes the model’s positional encodings during inference. Mini-Omni-Reasoner [15] interleaves silent reasoning tokens and spoken response tokens at the token level inside a dedicated specially trained speech-language model. Its authors report that this token-level thinking-in-speaking can both improve reasoning and avoid an explicit reasoning pause. Both methods let reasoning and response generation proceed together by controlling the decoding process.
At the chunk level, STITCH [35] alternates unspoken reasoning chunks with spoken-response chunks. It fine-tunes a speech-language backbone to generate reasoning, text and speech token chunks in that order (STITCH-R) or to start with a spoken-response chunk before its first reasoning chunk (STITCH-S). While an audio chunk plays, the model generates the next reasoning chunk. This scheduling relies on a model that is trained for chunked speech-token generation.
CATT shares the goal of these methods but differs in where it acts. Like TVS/ReVerT, it reads the reasoning text of an unmodified reasoning model and turns it into speech through a separate synthesizer. Unlike TVS/ReVerT, DDTSR, Mini-Omni-Reasoner and STITCH, it trains no component. The acknowledgment, the progress speech and the final answer all come from prompted calls to an ordinary chat model, and the router is calibrated rather than trained. CATT operates through the text chat interface rather than requiring control over positional encodings or interleaved speech-token generation. It can also use models that are reachable only through a hosted chat API. The transfer study runs it with three model families accessed through hosted APIs (Section 5.3). The design has its own costs. Progress speech requires an API that returns the reasoning text, and every spoken update is an additional model call. How continuous the speech is depends on the per-call latency and the behavior of each deployed model. CATT also decides for each turn whether to reason at all, so turns that the router does not expect to benefit from reasoning are answered directly without an acknowledgment.
Speaking about reasoning before it ends carries a risk: an intermediate statement can be wrong even when the final answer is right. We therefore measure how often CATT’s intermediate speech contains incorrect statements (Section 4.5). For an experimental comparison, Section 5.4 runs ReVerT from its authors’ official implementation and released verbalizer. It receives the same Qwen3-32B reasoning as CATT, replayed in real time, and uses the same speech synthesis, playback timing and judges.
2.4. Language-Model Routing and Cascading
Language-model routing and cascading reduce cost or latency by directing each query to a less expensive path when possible. FrugalGPT [36] cascades through a sequence of progressively stronger models and accepts the first sufficiently confident output. AutoMix [37] uses a small model’s self-consistency to decide whether to escalate to a larger one. Hybrid LLM [38] learns a quality-aware router that chooses between a small or large endpoint based on expected output quality, while FORC uses a meta-model to assign queries under explicit cost-performance constraints [39]. These systems treat the alternatives as different models, and they target cost–quality trade-offs rather than the latency a listener experiences.
Other work studies how routers are trained and evaluated. Shnitzer et al. [40] repurpose benchmark results to learn model selection for unseen tasks. A unified analysis of routing and cascading shows that the two paradigms can be combined and identifies output-quality estimation as a central requirement [41]. This research motivates routing as a deployment-layer decision. However, it generally evaluates model cost and answer quality rather than the timing of spoken responses.
RouteLLM [20] learns routers from preference data and offers several router families, including a similarity-weighted ranking router that scores an incoming query against a set of preference battles. Its authors also reused the routers without retraining on other model pairs (Claude 3 Opus/Sonnet and Llama 3.1 70B/8B) and report that they generalized well. The CATT router adopts the similarity-weighted ranking idea. However, it replaces the cross-model preference data with outcomes of the thinking-enabled and thinking-disabled configurations of the deployed backbone, Qwen3-32B (Section 3.3). Its decision is thus tied to one deployment choice: whether this model is expected to benefit from thinking on a given turn. The router could be recalibrated for another model in the same way, but this was not evaluated, and our router results are limited to Qwen3-32B.
Closer to our setting are systems that treat thinking as a switchable mode of one model rather than a different model. SynapseRoute [42] routes medical questions between the thinking and non-thinking modes of a dual-state language model and tracks accuracy, inference time and token usage together. OptimalThinkingBench [43] provides evidence on both sides of our motivation: thinking models tend to overthink simple inputs, while non-thinking models tend to underthink hard ones. CATT differs from these systems in two ways. First, its setting is real-time spoken dialogue, so the routing decision is one part of a latency-aware pipeline that also includes an immediate acknowledgment and progress speech during reasoning. Second, the routing decision is made outside the model by a replaceable component. Besides the CATT router, we evaluate an off-the-shelf complexity classifier from NVIDIA [17] in the same position with the rest of the system unchanged, and we compare the CATT router with other routing policies (Section 4 and Section 5.1).
Taken together, the prior work offers low-latency speech pipelines, end-to-end speech-language models that speak directly, several ways to speak while a model reasons, and routing between models or modes of one model. The five closest methods to CATT either train a component or control generation inside a language or speech-language model (Table 1). CATT combines speech during reasoning with a replaceable per-turn router. It needs only a streaming chat-completion API, which must return the reasoning text for progress speech. Section 3 describes its design.
3. Materials and Methods
This section describes CATT as it was run in the experiments of Section 4 and Section 5. Section 3.1 presents the architecture and the course of a turn. Section 3.2 defines what plug-and-play means for CATT and which interfaces it assumes. Section 3.3, Section 3.4, Section 3.5 and Section 3.6 detail the router, the talker modes and their safeguards, the asynchronous thinker, and the scheduling of progress speech and the final answer. Section 3.7 states how time is measured on the playback timeline. The control flow of one turn is given as pseudocode in Algorithm A1 (Appendix A). The data, the deployment, the baselines, the event definitions, the judges and the statistics are described in Section 3.8.
3.1. Architecture and Course of a Turn
CATT is an orchestration layer between a finalized user transcript and a speech and playback layer (Figure 1). In the main study the transcript is the question text itself; in the end-to-end case study it comes from speech recognition. The layer contains five modules. The router decides whether a turn takes the fast path or goes to reasoning. The thinker streams the reasoning of the deployed model on turns that go to reasoning. The talker produces every sentence that is spoken in one of four modes. The controller is an event loop. It receives the routing decision, reasoning chunks, the end of reasoning, talker completions and readings of the queued audio, and it issues every control action: it starts, cancels and dispatches model calls, and it asks the speech layer to drop queued speech. The workspace holds the state of the dialogue. The controller is its only writer, and the other modules read detached snapshots. Every event of a turn is recorded with its time, including each played and dropped speech chunk, and all the timings in this paper are computed from these records.
Model calls go to two engines that serve the same model weights. The thinker engine runs the reasoning request with thinking enabled, and the talker engine runs every talker call with thinking disabled (Section 3.5). The speech and playback layer splits the talker’s text stream into chunks, synthesizes them and plays them on a single channel. It reports how much audio is still queued, and it drops queued chunks when the controller requests it.
A turn proceeds as follows (Figure 1b). When the transcript arrives, the controller starts three calls at once: the routing call, the reasoning request on the thinker engine and the acknowledgment call on the talker engine. The acknowledgment text is held back until the route is known. On an easy route, the controller cancels the reasoning request and the acknowledgment call and dispatches a direct answer, so the turn consists of a single spoken reply. On a hard route, the acknowledgment is released to synthesis and is normally the first audio of the turn. While the reasoning continues, the controller dispatches progress updates whenever enough new reasoning is available and the queued audio runs low. Each update turns the most recent reasoning into one or two spoken sentences. When the reasoning ends, the controller drops the progress speech that has not started playing and dispatches the final answer, which plays after the sentence currently playing has finished. The dialogue state follows these phases: Easy on the fast path and Hard_First, Hard_Mid and Hard_Final for the acknowledgment, progress and final phases of a hard turn.
3.2. Plug-and-Play Interface
We call CATT plug-and-play in a specific sense: its orchestration code runs unchanged when the model endpoint, the router, or the speech and playback layer is replaced. The term refers to the code and its interfaces. It does not promise the same performance after a replacement. Replacing the model endpoint changes only the client configuration and a thin transport adapter outside the orchestration code, for example the endpoint address, the model name, the API’s switch for the thinking mode and the form of its continuation request at the reasoning budget. A router is any component that maps a transcript to a score, which a threshold turns into a route (Section 3.3). A speech and playback layer has to accept text chunks, report the duration of the queued audio and drop the queued chunks of a given talker call on request.
All three components were replaced in this study (Section 5.3 and Section 5.5). Besides the local Qwen3-32B deployment, the model endpoint was one of three model families accessed through hosted APIs. The router was either the CATT router or the NVIDIA classifier described in Section 3.3. The speech and playback layer was either Piper [18] with virtual playback or the CosyVoice API [19] with a browser-rendered avatar. Two parts were not replaced on their own: in every deployment the thinker and the talker used the same model, and the router’s sentence encoder was not exchanged separately since the NVIDIA classifier replaces the whole router. Earlier first audio and earlier first substantive content carried over to these replacements. Speech continuity and final-answer timing depend on the per-call latency of the endpoint and the behavior of the model, so they have to be measured again for each deployment.
The design rests on five assumptions about these interfaces.
- A1.
- Streaming chat completion. The model endpoint offers a streaming chat-completion API that yields text deltas and lets each request enable or disable the model’s thinking mode. To enforce a reasoning budget, it also has to accept a request that continues from a given text (Section 3.5).
- A2.
- Reasoning channel. A request with thinking enabled returns the model’s reasoning in a separate documented output channel, such as the reasoning-parser output of a vLLM server [10] or the reasoning_content field of the DashScope API. Both the progress talker and the final talker read this channel. Every endpoint evaluated here exposes it; endpoints without a reasoning channel were not evaluated.
- A3.
- No access to model internals. CATT does not read logits, attention states or the key–value cache, and it does not modify the decoder.
- A4.
- No raw reasoning is spoken. The reasoning serves only as input to the talker and as a token count for scheduling. The listener hears talker output, never the reasoning text itself.
- A5.
- Control from observable events. Every routing, scheduling and cancellation decision depends only on stream events, readings of the queued audio and the workspace state.
3.3. Router as a Replaceable Component
The router decides for each turn whether it takes the fast path (easy) or goes to reasoning (hard). In CATT, complexity is operationalized as the expected benefit of enabling the deployed model’s own thinking mode rather than an abstract difficulty label. The router maps the transcript x to a score , and a threshold turns the score into a route:
Three configurations of this component are evaluated. The CATT router follows the similarity-weighted ranking idea of RouteLLM [20]. The transcript is embedded with Qwen3-Embedding-0.6B [44] (instruction-wrapped, last-token pooling, and L2-normalized) and compared by cosine similarity with every entry of a reference bank. Each entry is a training prompt with a battle label for the deployed backbone, Qwen3-32B. The label is 1 if the backbone’s answer with thinking enabled won against its answer with thinking disabled, 0 if it lost, and 0.5 for a tie. Similarities are clipped at zero, and is the similarity-weighted mean of the labels over the whole bank. The second configuration replaces the CATT router with an off-the-shelf model, the NVIDIA prompt-task-and-complexity classifier [17], whose complexity score is computed with the formula of its model card. The third configuration, all-thinking, has no router and sends every turn to reasoning.
Both routers use a threshold set on router-calibration data only by the same rule. The calibration data are 318 items with stored correctness of the backbone’s answers with thinking enabled and disabled; the gain of an item is the change in correctness when thinking is enabled (for open-ended items, the sign of a judge’s preference). Among all the cuts between adjacent scores, the rule takes the one that maximizes the summed gain of the items routed hard. This is equivalent to routing hard exactly where a monotone fit of the gain on the score is positive. The rule gives for the CATT router and 0.1191 for the NVIDIA classifier. The 64 GPQA Diamond items among the calibration data were used to set the routing threshold and are therefore excluded from evaluation (Section 3.8.1).
Both routers run on the CPU. Their median decision times in the main run were 0.29 s for the CATT router and 0.86 s for the NVIDIA classifier. CATT therefore does not wait for the route before it starts working. The reasoning request and the acknowledgment call start together with the routing call, and the acknowledgment text is not passed to synthesis until the route is known, so no acknowledgment audio can precede an easy answer. On a hard route, the acknowledgment is released and the reasoning request continues. On an easy route, the controller closes the reasoning stream, which makes the engine abort the request, cancels the acknowledgment call, whose text is never synthesized, and dispatches the easy answer. Progress updates also wait for the route; a reasoning stream that ends before the route is known is handled once the hard route is confirmed. On hard turns, the routing time is thus hidden behind the generation of the acknowledgment. On easy turns, the direct answer still starts only after the route is known, and the two canceled calls count toward the model calls per turn (Section 4.4). Appendix A reports how much later the fast-path answer starts than the answer of the non-thinking baseline.
The router, its reference bank and its threshold were built for Qwen3-32B, and the router results of this paper apply to that model in the evaluated deployment. The bank and the threshold can be rebuilt for another model, but this was not evaluated: the transfer study used the router without recalibration (Section 5.3). Section 5.1 compares the CATT router with other routing policies on Qwen3-32B.
Application scope of the router. The CATT router uses outcomes of the deployed backbone twice: for the battle labels of its reference bank and setting its threshold on calibration items. At , the threshold sent 55.6% of the evaluated turns to reasoning. The evaluated configurations suggest three cases.
- CATT router. It fits deployments that can obtain the backbone’s answers in both modes for items like those they will serve. Averaged over all budgets of the routing comparison, it was more accurate than random routing, a length heuristic and the RouteLLM classifier (Section 5.1).
- A router without a reference bank. An off-the-shelf classifier or a simple rule removes the need for a reference bank. On the closed-form questions of the routing comparison, the NVIDIA classifier and the math-symbol rule routed about as well as the CATT router. In this study, however, the NVIDIA classifier’s threshold was still set with the backbone’s outcomes on the calibration items. Setting a threshold without such outcomes, for example by fixing the share of turns sent to reasoning, was not tested. With its threshold set on the calibration outcomes, the NVIDIA-router configuration reached 65.9% closed-form accuracy with an indistinguishable listening rating from that of the router configuration. It sent more open-ended requests to reasoning (65% against 46%). Its median first audio came later (0.96 s against 0.49 s; the acknowledgment on hard-routed turns and the answer onset otherwise). The math-symbol rule was evaluated only on recorded closed-form outcomes and without a threshold, so its routing of open-ended requests is untested.
- New backbone or domain. Recalibration for a new backbone means answering the reference prompts and the calibration items with it in both modes, then deriving the battle labels and the threshold again. A new domain would need new calibration items for the threshold. Neither step was evaluated. In the transfer study, which reused the router unchanged, CATT’s closed-form accuracy stayed on par with each model’s thinking baseline. The 64 closed-form items, however, gave intervals that were too wide to show non-inferiority within 5 percentage points.
Whichever router is used, its threshold trades accuracy against latency and cost. The rule used here weighs only the outcome gain on the calibration items. Appendix F traces this trade-off for the CATT router on the 738 evaluated items (Figure A2 and Table A8). Closed-form accuracy rose steeply from 43.9% with every turn on the fast path to 65.0% with 50% of the turns sent to reasoning and then stayed nearly flat (65.4% at ). From no turn to every turn sent to reasoning, model calls per turn grew from 2.05 to 5.00, and first substantive content went from 6.01 s earlier than in the thinking baseline to 0.78 s later. Lowering the threshold until every turn goes to reasoning amounts to using no router, which is the all-thinking configuration evaluated in Section 4; the endpoint of the curve is computed from recorded turns and differs slightly from that run. A deployment that also weighs cost or waiting can choose a higher threshold; in the evaluated deployment, the curve shows how much accuracy this gives up.
3.4. Talker Modes and Safeguards
All talker calls run on the talker engine with thinking disabled. The controller selects the mode from the dialogue state at dispatch time, and the modes differ only in their prompts and inputs (Table 2). Every prompt asks for speech that is easy to follow by ear without markdown, lists or symbols and formulas said in words. Two modes have separate prompt variants for the two question types. A question counts as closed-form if its text asks for a boxed answer or lists lettered answer options on separate lines and open-ended otherwise. This rule reads only the question text, never the dataset label.
Table 2.
The four talker modes. All modes run on the talker engine with thinking disabled and differ only in their prompts and inputs. The code names in parentheses are those of the released implementation.
Acknowledgment. The acknowledgment is one short sentence that acknowledges the question and names its topic. It may say what the question asks for or involves using only what the question itself states. It must not answer, guess, describe a method, an approach or a result, or promise anything, and generic filler such as “let me think” is excluded. Its length target is set for each question from the predicted time at which the first progress update can start playing so that the acknowledgment ends at about that time; a word cap enforces the target. An answer guard checks each acknowledgment sentence before synthesis. A sentence that reads as an answer statement, for example “the answer is” or “option B is correct”, is not spoken and ends the acknowledgment. The guard did not fire on any of the 410 hard turns of the router configuration.
Progress updates. A progress call receives the question, the reasoning produced since the previous progress dispatch, a short stretch of the reasoning before it as context, and the text already spoken in the turn. Each call therefore reads only the new part of the reasoning, not the whole trace. It returns one or two short sentences that say what is being worked on: the principle or method being applied and the specific quantity, step, case or option being worked out or checked. The results are deliberately left out of this description because intermediate values and identifications often change during reasoning. The update has to be specific to the question. It must not restate the question, repeat what was already said or use generic filler. If the new reasoning is still on the step described last, the talker returns SKIP and nothing is spoken. A candidate answer may be mentioned only as tentative. A guard enforces this rule: a progress sentence that states an answer without a hedging word such as “tentatively”, “likely” or “seems” is not spoken.
Final answer. The final talker receives the question, the last 6000 characters of the reasoning, where the conclusion normally lies, and the text that the listener has heard or is hearing. For a closed-form question, its first sentence states the answer itself, including the option letter and its content or the number and without words such as “tentatively”. One or two sentences with the decisive reason follow. If the answer differs from a tentative candidate mentioned earlier, the talker says so briefly. For an open-ended request, the final talker begins directly with the content and continues from what was already said; if something was to be written, it speaks the piece itself. When the reasoning was stopped at its budget, the final talker states the answer that the model wrote after the stop (Section 3.5).
Easy answer. On the fast path a single talker call answers directly. For a closed-form question the first sentence states the answer. An open-ended request gets a response that begins with the content and includes a few sentences when the request needs more than a single fact.
From text to speech. Every talker call, and the single call of each baseline, streams its text through the same sentence splitter. A chunk is released to synthesis at each sentence end and the first chunk of a call already at a comma so that audio can start early. Decimal points and thousands separators never split a chunk. A spoken-form normalizer in front of the synthesizer, identical for all conditions, turns markdown and mathematical notation into words. The normalizer belongs to the replaceable speech and playback layer (Figure 1a), not to the orchestration code, which runs unchanged. Answers are scored on the displayed text, not on the normalized text. In the main runs, the normalizer read single-asterisk markdown emphasis as the word “times” in a small share of the turns in every condition, and in a few turns it read a spaced dash between words as “minus”. The judges saw these spoken forms. The transfer study and the case study used a corrected normalizer, and the correction is included in the code release.
3.5. Asynchronous Thinker and Matched Reasoning Request
On a hard turn the thinker sends one streaming request with thinking enabled to the thinker engine. This request is identical to the request of the thinking baseline. It has the same spoken-answer prompt, the same sampling parameters and seed, the same 16,000-token reasoning budget with the same continuation at the budget, the same loop guard and the same 1200 s request timeout. The two conditions differ only in what happens after the reasoning. The thinking baseline continues the stream and speaks the visible answer that the model writes after its reasoning. The CATT thinker closes the stream at the first token of that visible answer, which makes the engine abort the request. The spoken answer of a CATT hard turn therefore always comes from the final talker.
The reasoning budget and the loop guard act identically in both conditions through one shared client. When the reasoning reaches the budget, the client stops the stream, and the thinker engine continues from the reasoning followed by the model’s own text for ending its thinking so that the model writes an answer with its full reasoning in view. The loop guard treats a reasoning stream that keeps repeating the same passage as having reached the budget. The thinking baseline speaks this written answer, and CATT’s final talker states it.
Reasoning deltas are appended to a shared reasoning record as they arrive. The controller counts the reasoning tokens for the progress conditions (Section 3.6), and each progress call reads its own slice of the record.
Two vLLM engines [10] serve the same Qwen3-32B weights [9]. The thinker engine runs only reasoning requests, those of the CATT thinker and both baselines, and the talker engine runs every talker call. Because the thinker engine serves one request at a time, talker calls never share its batches, and the reasoning stream does not depend on what the talker is doing. Together with the identical request, this made the reasoning byte-identical to that of the thinking baseline on all 410 hard turns of the router configuration. Differences between CATT and the thinking baseline on these turns, including the later final answer, therefore come from the orchestration, not different reasoning. The two engines are 4-way tensor-parallel and share the same four A100 40GB GPUs through the CUDA Multi-Process Service. The talker engine is limited to 50% of the streaming multiprocessors, and the thinker engine has no limit, which gives the reasoning stream priority when both engines are busy. The baselines run on the thinker engine while the talker engine stays loaded and idle. Section 3.8.3 gives the remaining deployment details.
3.6. Progress Scheduling and Final-Answer Cut-In
Progress updates should fill the time until the final answer without speaking over each other or interrupting a sentence. Let R be the number of reasoning tokens received so far, the number received since the previous progress dispatch, B the duration of the queued audio in seconds, and n the number of progress dispatches so far in the turn. A progress update is dispatched when
where hard means that the hard route has been confirmed, Hard_Mid that the turn is in its progress phase, in_flight that a talker call is still generating, and end that the end of reasoning has been received; C is the condition for every update after the first. The controller checks this condition on every reasoning chunk and every reading of the queued audio, which the speech layer reports at short intervals. The first update is dispatched as soon as the acknowledgment call has finished generating, and the reasoning holds tokens whatever the queue. A later update needs new reasoning tokens while less than s of audio is queued. When less than s of audio remains, a low-queue rule dispatches an update after only new tokens. Allowing only one talker call at a time keeps spoken segments in order. The values of , , L and F were chosen on development items that are disjoint from the evaluation set. Appendix E.2 shows that setting to 2.5, 5 or 10 s barely changes the results.
When the reasoning ends, the controller sets the state to Hard_Final and cuts in with the final answer. It cancels any progress call that is still generating. It has the speech layer drop every queued progress chunk whose playback has not started and any chunk of a canceled call that arrives later. Dropped chunks are never played and never deferred. The chunk that is playing, normally a sentence, is allowed to finish. If the acknowledgment call is still generating, the controller waits for it, and it then dispatches the final talker. The final talker is told which speech is still audible, so its answer follows on from what the listener actually hears. No talker call can start after the final dispatch, so progress and final sentences cannot interleave.
A turn ends when three conditions hold: the terminal reply (the final answer or the easy answer) has finished generating, no audio of the turn remains to be scheduled, and the dialogue state is terminal. The redundant conditions prevent a late progress completion from being mistaken for the end of the turn. A watchdog stops any turn that has not ended after 1200 s; no evaluated turn reached it.
3.7. Timing Semantics
All the times in this paper are in seconds from question submission, that is, from the moment the finalized transcript is submitted; this origin is the same for CATT and the baselines. In the main study, playback is virtual: no audio is played on a device. Every chunk is synthesized by Piper on the CPU, and its real synthesis time is recorded. Synthesis is pipelined, so later chunks are synthesized while earlier ones play. Chunk k of a turn then receives a playback window on a single channel in strict order,
where is the time at which the chunk’s audio has been synthesized and is its audio duration. Two chunks therefore never play at the same time, and a chunk that is ready late delays every later chunk. The queued audio B that the controller uses for scheduling is the remaining duration of this channel. Reported times are playback starts on this virtual channel, computed from when synthesized audio became available. They do not include device buffering, audio output latency or network transfer.
A chunk that is dropped never receives a playback window. At the end of reasoning, queued progress chunks whose playback has not started are dropped, and a chunk of a canceled call that arrives later is dropped on arrival. A progress chunk whose synthesis the channel is already waiting for at that moment is still played. Calls canceled on an easy route are never played. A progress call that returns SKIP, or whose sentences are all removed by the guard, produces no audio and no window. The silence of a hard turn is the sum of over consecutive played chunks. It excludes the wait before the first audio and anything after the last chunk. The baselines pass through the same splitter, normalizer, synthesizer and playback model, so their timelines are computed in the same way. The compared events in this paper, from the acknowledgment or answer onset to the final answer, are positions on these timelines and are defined in Section 3.8.4. Figure 1b shows one hard turn and one easy turn on this timeline.
The timing model was checked against the records. A replay of the recorded runs that recomputes every playback window from the recorded generation and synthesis times, without perturbation, reproduces every recorded playback start within 7.1 ms (Appendix E.1). The same replay with added latencies and slower generation is used in the additional analysis of Section 5.2. The end-to-end case study (Section 5.5) runs the orchestration with real synthesis and playback through a browser-rendered avatar.
3.8. Evaluation Setup
This subsection describes the evaluation data and their provenance, the compared conditions, the deployment and the measured events, the LLM judges, the statistical analysis and possible benchmark contamination. It refers to the main study. The additional studies of Section 5 use the same definitions and state their own settings where these differ.
3.8.1. Data and Provenance
The evaluation set has 738 items from four public sources. The 214 closed-form items have a single reference answer and are scored automatically. Competition mathematics is represented by the 30 problems of the AIME 2026 I and II examinations (15 each), taken from the MathArena collection [11,45]. GPQA Diamond [21] contributes 134 graduate-level science questions. The remaining 50 are multiple-choice questions from MMSU, the multiple-choice understanding subset of VoiceBench [22], sampled across its subject categories. The 524 open-ended requests have no reference answer. They are AlpacaEval prompts [46] as distributed with VoiceBench: 50 items sampled from its AlpacaEval subset and 474 further prompts from its full AlpacaEval set. Each of the 474 further prompts was kept only if its normalized text neither matched nor nearly duplicated an item of any other item set used in this study. All the items enter the system as text in the wording and with the answer options of the source.
Five kinds of data are kept apart, and Table A2 in Appendix B lists each with its role. The source benchmarks are the public datasets named above, which remain under their own terms. Router-calibration data are 318 items with outcomes recorded in earlier runs of the system: 30 items each from AIME 2024 and AIME 2025, 64 from GPQA Diamond, 94 from MMSU and 100 from AlpacaEval. They served only to set the routing thresholds of both the CATT router (, Section 3.3) and the NVIDIA classifier (Section 3.8.2). The evaluation fixture is a frozen file of 802 items, identified by its SHA-256 checksum and read by every run. It contains all 198 GPQA Diamond items. Of these, 64 were used to set the routing threshold and are therefore excluded from evaluation, which leaves the 738 evaluated items. No other evaluated item belongs to the router-calibration data. A separate pool of 504 development items, which shares no item with the evaluation fixture by identifier or normalized text, was used for design choices and the sensitivity check of the backlog threshold (Appendix E.2). The outputs generated in this study are the 4010 turns of the main run, one for each of the 802 items in each of five conditions, with their speech, timings, reasoning, model outputs and judge labels. The additional studies generated further outputs, which are described with each study.
The code and releasable evaluation records are provided in Supplementary Data S1; the Data Availability Statement specifies the contents and exclusions.
3.8.2. Compared Conditions and Matched Baselines
Five conditions were run on the same deployment and the same items. Three of them are configurations of CATT that differ only in how turns are routed. The router configuration uses the CATT router at its threshold (Section 3.3). It is the configuration reported throughout, and “CATT” refers to it unless stated otherwise. The NVIDIA-router configuration replaces the router with NVIDIA’s prompt task and complexity classifier [17] thresholded at 0.1191; this threshold was set on the router-calibration data by the same rule that gave . The all-thinking configuration sends every turn to reasoning. The two baselines make one model call per turn with thinking enabled in the thinking baseline and disabled in the non-thinking baseline.
The baselines are matched to CATT in prompt, sampling, reasoning budget and stopping. Both receive the same spoken-answer prompt, built from the instructions of CATT’s talker prompts. It asks for plain conversational speech without markdown, with the conclusion first and only the explanation that the listener needs, usually a few spoken sentences. Sampling follows the Qwen3 recommendations for each mode [9]. With thinking enabled (CATT’s thinker and the thinking baseline), the temperature is 0.6, top-p 0.95 and top-k 20. With thinking disabled (all talker calls and the non-thinking baseline), the temperature is 0.7, top-p 0.8 and top-k 20. Each request carries a fixed seed derived from the item, the role of the call and its index.
The thinking baseline’s request is identical to the request of CATT’s thinker (Section 3.5). It has the same prompt, sampling settings and seed, the same reasoning budget of 16,000 tokens and the same stopping rules: when the reasoning reaches the budget or the loop guard finds that it keeps repeating, the model is made to end its thinking and answer from the reasoning so far. The reasoning of CATT and the thinking baseline is therefore byte-identical on all 410 turns that CATT routed to reasoning, and differences between the two on these turns arise in the orchestration. All the conditions also share the 1200 s watchdog, the sentence splitter, the spoken-form normalizer in front of speech synthesis, the synthesizer and the playback model (Section 3.7). The baselines run through CATT’s controller, so every timestamp in every condition is produced by the same code.
Because both baselines share the spoken-answer prompt, their visible answers are short (Table 3). CATT speaks more words per turn since, on hard turns, its acknowledgment and progress speech come before the answer. This additional speech is part of the design, and every timing and quality measure includes it.
Table 3.
Output length by condition on the 738 evaluated items. Visible answer tokens are the baselines’ completion tokens minus their reasoning tokens as a mean per turn; the AlpacaEval range spans the 50 sampled items and the 474 further prompts. Words heard count all played speech of a turn as a median, including CATT’s acknowledgment and progress speech. n/a: CATT’s speech comes from several model calls per turn, so its length is given in words heard only.
3.8.3. Deployment and Run Protocol
All five conditions ran on one server with four NVIDIA A100 40 GB GPUs. Qwen3-32B [9] (revision 9216db57, bfloat16) was served by vLLM 0.10.2 [10] with its Qwen3 reasoning parser, which returns the reasoning as a separate output channel. The thinker engine and the talker engine each run with 4-way tensor parallelism and share the same four GPUs through NVIDIA’s Multi-Process Service (MPS), as described in Section 3.5. The talker’s share of 50% of the streaming multiprocessors was selected on development items by a fixed rule: among the shares that left the reasoning stream unchanged and kept the talker’s call latency within a small margin, the one with the fastest reasoning was chosen. The sentence encoder of the CATT router, the NVIDIA classifier and the Piper speech synthesizer [18] run on CPU. Both engines are local to the server, so no network lies between the orchestration and the model.
Every turn starts from the question text, which is submitted at time zero as a finalized transcript. Speech recognition is therefore not part of the main study. Turns are isolated: each turn opens a new dialogue, both engines are idle when it starts, and their prefix caches are reset beforehand. Only one turn runs at a time, so all the measurements describe a single user. The items were run in a seeded random order, with the five conditions of each item back to back in a seeded order, after a fixed warm-up on prompts outside the evaluation set. Speech is synthesized while earlier speech plays, and playback follows the virtual timeline of Section 3.7, which is computed from the measured synthesis times. All the times are given in seconds from the submission of the question.
OpenAI Codex (codex-cli version 0.148.0-alpha.15) was used as an authoring assistant for code assistance, figure preparation, and consistency checks; the authors reviewed the resulting scripts, figures, and reported outputs.
3.8.4. Measured Events and Costs
All the measures are computed from the played speech of a turn, that is, from the synthesized chunks in playback order with their start and end times. For judging, consecutive chunks of one model call are merged into sentences by the same rule in every condition, and a sentence starts when its first chunk starts. The following events and measures are recorded for every turn.
- First audio is the playback start of the first chunk. On turns that CATT routes to reasoning it is the acknowledgment. On turns routed to the fast path, and in both baselines, it is the answer onset. Every first-audio value in this paper carries one of these two labels.
- First substantive content is the playback start of the first sentence that the judge labels substantive (Section 3.8.5). The definition includes a specific solution approach and intermediate results, so progress speech can be substantive before any answer is given.
- First candidate answer (closed-form items) is the playback start of the first chunk, outside the acknowledgment, by which the speech of its model call has stated an explicit candidate answer. Candidates are identified by the explicit-statement rules of the answer scorer.
- Final answer (closed-form items) is the playback start of the earliest chunk by which the speech played so far yields the turn’s scored answer. The search runs within the final model call; if the scored answer is found only in the reply as a whole, it runs over the whole reply without the acknowledgment.
- Accuracy (closed-form items) is scored automatically from the displayed text of the played speech before its conversion to spoken form. Fixed extraction rules read an integer for AIME 2026 and an answer option by its letter or its content for GPQA Diamond and MMSU. They read the final model call first and the whole reply otherwise, and a turn without an extractable answer counts as incorrect.
- Silence is the total time between the end of one played chunk and the start of the next summed over a turn (Section 3.7); the wait before the first audio is not included. For CATT it is reported on the turns routed to reasoning and for the baselines on all turns.
- Playback completion is the end of the last played chunk.
- Intermediate errors and the listening rating are judged as described in Section 3.8.5.
Cost is recorded per turn as the number of model calls, the prompt and completion tokens summed over these calls (with reasoning tokens counted separately), the largest number of model calls in flight at the same time, and GPU energy. GPU energy is the total over the four GPUs during the wall-clock interval of the turn. It is computed from the cumulative energy counters of the NVIDIA Management Library (NVML), which a sampler read throughout the run. The figure includes idle draw, and, for the baselines, it also includes the loaded but idle talker engine. Because it was measured for a single user, it mostly follows the duration of a turn rather than the computation that the turn requires.
3.8.5. LLM Judges and Listening Rating
Three judgments use LLM judges: first substantive content, intermediate errors and the listening rating. First substantive content and intermediate errors are labeled by gpt-5.6-luna alone (served model identifier gpt-5.6-luna-2026-07-09). The listening rating is given by two judges, gpt-5.6-luna and gemini-3.1-flash-lite (the service returns no dated identifier), and gemini-3.1-flash-lite is used for no other judgment. Both models were called through an OpenAI-compatible API with reasoning effort set to medium, at most 16,000 output tokens, and default values for all the other sampling settings. Each reply is judged on its own, never side by side with the reply of another condition, so the order of presentation cannot affect the judgment. No judge prompt names the condition that produced the reply. The judges read every sentence in its spoken form as produced by the spoken-form normalizer (Section 3.4) and rated the recorded replies after the runs. The three judge prompts are released with the code (see the Data Availability Statement). The additional studies use the same judges and prompts. Two further checks use gpt-5.6-luna alone with their own prompts and are described where they are used: the content audit of progress updates (Section 4.5) and the semantic correctness check of ReVerT answers (Section 5.4).
For first substantive content, the judge receives the question and the played sentences of a turn in playback order and labels each sentence as substantive or not. A sentence is substantive if it gives information beyond the question text that bears directly on the question: a specific solution approach or method, an intermediate result, a candidate or final answer, or, for an open-ended request, content that starts answering it. A sentence that only acknowledges, restates or paraphrases the question, names its topic, announces that work is under way or is generic filler is not substantive. Correctness does not matter for this label.
For intermediate errors, the units are the acknowledgment and each progress dispatch of a hard turn. The judge sees the question, the reference answer when one exists, the turn’s final answer as spoken and the numbered units, and it gives each unit one of four labels. OK means that nothing in the unit is incorrect or contradicted. WRONG marks an incorrect fact, value, step or candidate answer, whether hedged or not. REVERSED marks a claim presented as settled that the turn’s own final answer contradicts. TENTATIVE_CHANGED marks a candidate answer that was explicitly called tentative and differs from the final answer; this is permitted and does not count as an error. A turn is an error turn if at least one of its units is WRONG or REVERSED. All hard-routed turns are labeled whatever their final answer. The share of error turns is reported in two forms. The rate over all hard-routed turns includes every hard-routed turn. The rate conditional on a correct final answer leaves out the closed-form hard turns whose final answer is wrong, so it describes intermediate speech on turns that end with a correct answer. Open-ended hard turns are not scored for correctness and enter both rates.
Quality is measured by a listening rating from both judges. Each judge receives the request, a timeline of everything the listener heard and, for closed-form items, the reference answer. The prompt states that the listener did not hear the reference answer and that it serves only to judge correctness. The timeline lists every played sentence in spoken form with its start and end times in seconds after the question. It also shows the silence before the first sentence, every later silence of at least 0.5 s, and the end of the reply. On a hard CATT turn the acknowledgment therefore appears as the first sentence at the time it was heard and without any label. The judges rate the overall quality of the reply as experienced by the listener on a scale from 1 to 10. They are asked to consider together the waiting and silences, helpfulness and relevance, correctness, and clarity and completeness for a spoken reply and not to reward length for its own sake. The score of a turn is the mean of the two judges’ scores. The rating is an LLM-judged proxy for the listening experience, not a measure of human perception.
A validity check applied the same prompt to synthetic timelines built from recorded replies. Moving the first sentence of the thinking baseline’s reply from 0.5 s to 60 s with its words and relative timing unchanged lowered the mean score by 4.18 points ([−4.65, −3.69]; 60 items). Moving everything after CATT’s acknowledgment 30 s later, which adds 30 s of silence after it, lowered the mean score by 1.99 points ([−2.55, −1.45]; 40 items). An early acknowledgment therefore does not hide a later wait from the judges. Appendix C reports the reliability of the rating. The two judges agree, with an intraclass correlation ICC(A,1) of 0.78 [47]. On 393 re-judged replies, the test–retest ICC is 0.95 for gpt-5.6-luna and 0.93 for gemini-3.1-flash-lite. Appendix C (Table A3) also gives the association of the rating with reply length. In a separate re-judged sample of 10% of the turns, gpt-5.6-luna reproduced its own first-substantive positions in 96.2% of the turns and its error flags in 99.3%. No human evaluation was carried out, and the judges were not validated against human ratings. All the conclusions about substantive content, intermediate errors and quality in this paper are therefore based on LLM judgments.
3.8.6. Statistical Analysis
All the comparisons are paired over items. For each item, the value of a CATT configuration is compared with the value of a baseline on the same item. Times are summarized by the median of the paired differences because their distributions are skewed. Accuracy and the listening rating are summarized by the mean of the paired differences. The tables also give each condition’s median time or mean value. Intervals are 95% percentile bootstrap intervals over items with 10,000 resamples [48] and seed 20261009 with three exceptions. The transfer study (Section 5.3) used seed 20261011. The case study (Section 5.5) used seed 20261008 for its main run and 20261013 for its preemption run. The routing comparison (Section 5.1) evaluates every policy at every budget from 0 to 170 questions sent to reasoning and averages each difference over these budgets; its intervals come from 20,000 resamples stratified by dataset with seed 20261002. Accuracy differences are also tested with an exact McNemar test on the discordant pairs [49], and rates such as the share of error turns carry Wilson 95% intervals [50]. Contrasts on a subset, such as the closed-form items or the turns routed to reasoning, resample the items of that subset. Splits by route use CATT’s route for both members of a pair.
A turn in which no played sentence is labeled substantive is assigned 1200 s, the watchdog limit, as its time of first substantive content. Likewise, a turn without played audio would be assigned 1200 s as its first audio, and an incomplete or unscorable closed-form turn would count as incorrect with 1200 s as its final-answer time. The counts of valid and censored turns per condition are given in Appendix D.
Each condition was run once per item with one generation. The intervals therefore reflect the sampling of items only, not the variation between repeated generations. On hard turns, the reasoning is byte-identical in CATT and the thinking baseline, so the comparisons on these turns do not depend on randomness in the reasoning. The acknowledgment, the progress speech, the final-answer call and the fast-path replies remain single samples.
3.8.7. Benchmark Contamination
The Qwen3-32B weights were released on 28 April 2025. The repository revision used here (9216db57, dated 26 July 2025) only adds a license file to the released weights. The AIME 2026 I and II examinations were held on 5 and 11 February 2026 [51], more than nine months later, so, for AIME 2026, the examination postdates the model’s release. GPQA Diamond, MMSU and AlpacaEval were public before the model was released, and their contamination cannot be ruled out but does not bias the paired comparison. Memorized items would raise the absolute accuracy of CATT and the thinking baseline because both use the same model and their reasoning is identical on hard turns. The hosted models of the transfer study (Section 5.3) are more recent and may have seen the AIME 2026 problems in training. This would affect their absolute accuracy, not the comparison between CATT and the thinking baseline of the same model.
4. Results
This section reports the main comparison on the 738 evaluated items: 214 closed-form questions (AIME 2026 30 items; GPQA Diamond 134; MMSU 50) and 524 open-ended requests from VoiceBench AlpacaEval. Unless stated otherwise, CATT denotes the router configuration, and every contrast is CATT minus a prompt-matched baseline on the same items. Timing contrasts are medians of per-item paired differences in seconds. Accuracy and the listening rating are compared through mean paired differences. Brackets give 95% percentile bootstrap intervals over items (10,000 draws), and rates carry Wilson 95% intervals. At the routing threshold, CATT sends 410 turns to the reasoning path (168 closed-form and 242 open-ended) and answers the other 328 on the fast path (46 closed-form and 282 open-ended). On a hard-routed turn, the first audio of CATT is the acknowledgment. On an easy-routed turn, and in both baselines, it is the onset of the answer. The tables and the text below keep the two apart.
4.1. Timing of the Four Response Events
Table 4 compares CATT with the thinking baseline on the four events defined in Section 3.8.4: first audio, first substantive content, first candidate answer and final answer. The comparison is split by question type and the route that CATT took. Each cell shows the paired difference together with the medians of both sides. Cells in which CATT is later are reported in the same way as the others.
Table 4.
Timing of the four response events, CATT (router configuration) minus the prompt-matched thinking baseline. Each cell gives the median of the per-item paired differences in seconds with its 95% bootstrap interval (first line) and the medians of CATT and the thinking baseline (second line). Negative values mean that CATT is earlier. The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; the first audio of the baseline is always the answer onset.
CATT starts speaking earlier than the thinking baseline over all items ( s [, ]; medians 0.49 s and 6.98 s). What the listener hears first depends on the route. On hard-routed turns it is the acknowledgment, which comes 23.48 s earlier on closed-form questions and 6.35 s earlier on open-ended requests. On easy-routed turns the fast path starts the answer itself: 5.48 s earlier on closed-form questions and 4.95 s earlier on open-ended requests.
First substantive content arrives earlier over all items as well ( s [, ]). By the definition used here, it includes a specific solution approach. The size of the gain depends on the question type. On closed-form math and science questions, CATT conveys substantive content 10.46 s earlier [, ]. For the closed-form questions that it routes to reasoning, the difference grows to s [, ] with medians of 6.44 s for CATT and 24.04 s for the thinking baseline. In these turns the listener hears how the problem is being approached while the reasoning is still running. Open-ended requests receive substantive content 3.00 s earlier [, ]. The exception is the group of open-ended requests that CATT routes to reasoning. There CATT is 1.15 s later [, ] (medians 8.22 s and 6.74 s) because the reasoning on these requests is short and leaves little progress to report before the answer.
On closed-form questions, the answer itself comes later in CATT. The first candidate answer is 2.23 s later [, ], and the final answer is 3.20 s later [, ] (medians 20.82 s and 19.47 s). On the hard closed-form turns, the final answer is 4.55 s later [, ]. The reasoning on every hard turn (410 of 410) is byte-identical to that of the thinking baseline, so this delay is a scheduling effect. After the reasoning ends, CATT lets the progress sentence that is playing finish and then speaks the final answer produced by a separate talker call, whereas the thinking baseline continues directly into its answer. Only on the 46 easy-routed closed-form questions does the fast path give the final answer earlier ( s [, ]).
The non-thinking baseline is the quickest route to a direct answer. Its first audio, the answer onset, is close to that of CATT, which is the acknowledgment on hard-routed turns and the answer onset otherwise (CATT minus non-thinking: s [, ] over all items). Its substantive content, however, comes earlier. Relative to it, CATT is 4.26 s later over all items [, ], 5.32 s later on closed-form questions [, ] and 0.96 s later on open-ended requests [, ]. On easy-routed turns the remaining difference is the routing decision: the fast path starts 0.32 s later [, ] than the non-thinking baseline at first audio and 0.33 s later [, ] at first substantive content (328 turns). The non-thinking baseline also gives the closed-form final answer much earlier (CATT minus non-thinking: s [, ]), but its closed-form accuracy is 21.0 percentage points lower (Section 4.2).
4.2. Results by Dataset and Accuracy
Table 5 gives the paired contrasts for every metric and dataset against both prompt-matched baselines. Both baselines use the same spoken-answer prompt as CATT (Section 3.8.2). Playback completion and the listening rating therefore compare replies written under the same answer contract.
Table 5.
Paired contrasts by dataset, CATT (router configuration) minus each prompt-matched baseline. Times are medians of paired differences in seconds (negative: CATT earlier). Accuracy is the mean paired difference in percentage points (pp). The listening rating is the mean paired difference on the 1–10 scale (mean of both judges). In the accuracy column, the first line gives the estimate with its 95% bootstrap interval and the second line the discordant counts of the exact McNemar test (CATT wrong and baseline right vs. the reverse); on AIME 2026 against the thinking baseline no problem is discordant, so no interval is shown. In the other columns, each cell gives the estimate (first line) and its 95% bootstrap interval (second line). The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; the first audio of the baselines is the answer onset.
Accuracy. On the 214 closed-form questions, CATT answers 65.4% correctly and the thinking baseline 65.9%, a paired difference of percentage points (pp) [, ]. The exact McNemar test finds five questions that only the thinking baseline answers correctly and four that only CATT answers correctly (). On GPQA Diamond both reach 59.7% with three discordant questions in each direction. On AIME 2026 both conditions answer the same 17 of 30 problems correctly (56.7% each) with no discordant problem. MMSU is the only dataset with a lower point estimate for CATT, 86.0% against 88.0% ( pp [, ]; two discordant questions against one; ). Against the non-thinking baseline (44.4%), CATT is 21.0 pp more accurate [, ] with six against 51 discordant questions (). The largest gain is on AIME 2026, where the non-thinking baseline solves 3.3% ( pp [, ]). Since the reasoning on hard turns is identical, an accuracy difference against the thinking baseline can arise only on the fast path or in the final answer that CATT writes after the reasoning; Section 4.6 counts both. Each condition was run once per item, so the intervals reflect item sampling only. On hard turns, the accuracy and final-answer comparisons are not affected by randomness in the reasoning because the reasoning is the same in both conditions.
Timing by dataset. First audio and first substantive content are earlier than in the thinking baseline on every dataset. The gains are the largest where the baseline reasons the longest. On AIME 2026, where CATT routes all 30 problems to reasoning, the acknowledgment comes 106.56 s earlier than the answer onset of the thinking baseline; first substantive content is 90.08 s earlier. The final answer, in contrast, arrives later on the datasets that CATT mostly routes to reasoning by 7.03 s on AIME 2026 [, ] and by 3.54 s on GPQA Diamond [, ]. On MMSU, where 33 of the 50 questions take the fast path, the final answer is 4.12 s earlier [, ]. Playback of the whole reply ends earlier than in the thinking baseline on every dataset by 3.49 s (GPQA Diamond) to 10.71 s (AIME 2026). Against the non-thinking baseline, the open-ended replies of CATT end 5.64 s later [, ]. CATT speaks more in total since, on hard turns, the acknowledgment and the progress speech come before the answer. The median open-ended reply contains 73 words heard for CATT, 63 for the thinking baseline and 56 for the non-thinking baseline.
Listening rating by dataset. CATT is rated higher than the thinking baseline on every dataset, from on AIME 2026 to on MMSU. Against the non-thinking baseline, it is rated higher on AIME 2026 ( [, ]). The difference is not distinguishable from zero on GPQA Diamond ( [, ]) and MMSU ( [, ]), and CATT is rated lower on AlpacaEval ( [, ]).
4.3. Listening Ratings of the Three CATT Configurations
Quality is measured by the listening rating only (Section 3.8.5). Two judges, gpt-5.6-luna and gemini-3.1-flash-lite, read a timeline of everything the listener heard, including the start and end times of each sentence and the silences between them, and rate the reply from 1 to 10. The score of a turn is the mean of the two judges. The rating is therefore an LLM-judged proxy for the listening experience, not a measurement of human perception. Table 6 compares the three CATT configurations with both baselines.
Table 6.
Listening ratings of the three CATT configurations against the prompt-matched baselines (1–10 scale, mean of both judges). Contrast cells give the mean paired difference, configuration minus baseline, with its 95% bootstrap interval. The last column gives the mean rating of each condition.
All three configurations are rated higher than the thinking baseline on both question types: by 0.79 to 0.98 points on closed-form questions and by 0.35 to 0.92 points on open-ended requests. The router configuration has the largest gains ( [, ] and [, ]). Against the non-thinking baseline, the picture depends on the question type. On closed-form questions the router configuration is rated higher ( [, ]), while the NVIDIA-router and all-thinking configurations are not distinguishable from the non-thinking baseline ( and with intervals that include zero). On open-ended requests all three configurations are rated lower (, and ). This deficit grows with the share of open-ended requests sent to reasoning (46%, 65% and 100%). For the router configuration it lies entirely in the open-ended requests routed to reasoning: [, ] on these 242 turns against [, ] on the 282 easy-routed requests. On those hard turns CATT speaks an acknowledgment and progress updates before its answer, and its reply is longer.
Among the three configurations, the router configuration has the highest point estimates, but only one difference between configurations is resolved. The all-thinking configuration is rated below the router configuration on open-ended requests ( [, ]). The NVIDIA-router configuration differs from the router configuration by [, ] on closed-form questions and by [, ] on open-ended requests. The mean ratings of closed-form replies are low in every condition (3.86 to 4.83) because the judges compare the reply with the reference answer and also weigh the waiting. The two judges differ in how strongly they penalize waiting, but they never point in opposite directions on a reported contrast. Their agreement, their test–retest stability and the validity checks of the rating are given in Appendix C.
4.4. Quality, Latency and Cost
Table 7 places accuracy, the listening rating and the event times next to the computational cost of each condition. All five conditions ran on the same deployment (Section 3.8.3).
Table 7.
Quality, latency and cost per condition on the 738 evaluated items. Panel (a) gives closed-form accuracy, the mean listening rating and median event times; panel (b) gives the mean cost per turn.
The router configuration reaches almost the same accuracy as the all-thinking configuration (65.4% against 65.9%) at a lower cost. It makes fewer model calls per turn (4.58 against 4.96), reads fewer prompt tokens (3936 against 4351), generates fewer completion tokens (955 against 1124) and uses less GPU energy per turn (18.1 kJ against 21.2 kJ). Its first substantive content also comes earlier (median 5.22 s against 7.36 s), and its open-ended replies are rated higher (Section 4.3). In this system, sending every turn to reasoning answers one more closed-form question correctly, but it adds cost and delays substantive content. The all-thinking configuration does have the earliest first audio (0.33 s). Its first audio is the acknowledgment on every turn, and the acknowledgment does not wait for a routing decision. The NVIDIA-router configuration reaches 65.9% accuracy but speaks later than the router configuration (median first audio 0.96 s against 0.49 s with median routing decisions of 0.86 s and 0.29 s). In both configurations, first audio is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns. The NVIDIA-router configuration also gives substantive content and the final answer later, and it makes more calls.
CATT also has clear costs against the thinking baseline. A CATT turn makes 4.58 model calls on average against one, and it reads 3936 prompt tokens against 490 because each talker call reads excerpts of the reasoning. CATT also keeps two model copies in GPU memory and runs two to three calls at once for a single user, where the baseline runs one. GPU energy per turn is similar (18.1 kJ against 19.4 kJ); with one user, it mostly tracks wall time and includes idle draw. Multi-user throughput was not measured.
4.5. Intermediate Speech
Speaking before the reasoning ends carries a risk: an acknowledgment or a progress update can state something wrong even when the final answer is right. We therefore labeled every unit of intermediate speech on hard turns with gpt-5.6-luna. A unit is either the acknowledgment or one progress dispatch. The judge compares each unit with established facts, the reference answer when one exists, and the final answer of the same turn. A unit is WRONG if it states something incorrect, such as a false fact or a wrong value, formula, step or candidate answer. It is REVERSED if it presents as settled a claim that the final answer of the turn contradicts. A candidate answer that was explicitly marked as tentative and later changed (TENTATIVE_CHANGED) is allowed and does not count as an error. An error turn contains at least one WRONG or REVERSED unit. All 410 hard-routed turns are labeled, and two rates are reported (Section 3.8.5). The rate over all hard-routed turns includes every one of them. The rate conditional on a correct final answer leaves out 65 of the 168 hard closed-form turns whose final answer is wrong, so it describes only turns that end with a correct answer. Every open-ended hard turn enters both rates.
Over all 410 hard-routed turns, 62 contain at least one intermediate error (15.1% [12.0, 18.9]). The errors concentrate on closed-form questions: 58 of the 168 closed-form hard turns (34.5% [27.8, 42.0]) against four of the 242 open-ended hard turns (1.7% [0.6, 4.2]). Turns that end with a wrong final answer contain errors more often. On closed-form questions, 32 of the 65 turns with a wrong final answer are error turns (49.2% [37.5, 61.1]). Conditional on a correct final answer, the closed-form rate is 26 of 103 (25.2% [17.8, 34.4]). Over all the hard-routed turns except the 65 closed-form turns with a wrong final answer, the conditional rate is 30 of 345 (8.7% [6.2, 12.1]). The conditional rates are therefore lower than the rates over all the hard-routed turns and are not general error rates. From the listener’s side, the 168 closed-form hard turns fall into four groups. In 26 of them (15.5% [10.8, 21.7]), the listener hears at least one intermediate statement labeled WRONG before a correct final answer. The other three groups are 77 turns (45.8%) with a correct final answer and no intermediate error, 32 (19.0% [13.8, 25.7]) with a wrong final answer and at least one intermediate error, and 33 (19.6%) with a wrong final answer and no intermediate error.
The unit-level labels show where the errors occur. Only one of the 410 acknowledgments is WRONG, on an open-ended request whose wording it misread, and none of the 168 closed-form acknowledgments are. The guard that suppresses answer statements in the acknowledgment never fired (zero of 410 hard turns). Progress speech accounts for the remaining errors. On closed-form questions, 170 of 862 progress units are WRONG: 65 of 378 in turns with a correct final answer and 105 of 484 in turns with a wrong one. On open-ended requests, four of 209 are WRONG. No unit is REVERSED, and three units contain a tentative candidate that was later changed. In turns with a wrong final answer, the errors are mostly wrong reasoning steps, such as a wrong mechanism, formula or identification. They are rarely early statements of the eventual wrong answer. An automatic check for explicit candidate answers found a WRONG unit stating the turn’s own final answer in only one of these 32 error turns. The risk therefore lies in incorrect intermediate claims made while closed-form reasoning is under way. It does not lie in the acknowledgment, and it does not take the form of answers that are stated as settled and later overturned.
These rates rest on a single LLM judge, which also sees the reference answer. On turns with a wrong final answer, knowing which answer the reasoning misses may lead the judge to rate intermediate steps more strictly. Without human labels, this effect cannot be separated from true errors. The rates were measured for the router configuration on the 738 evaluated items only.
An additional analysis audited the content of the 1071 progress updates played in 347 hard turns with gpt-5.6-luna and text measures that use no judge. The judge rated 72.5% [69.7, 75.0] of the updates as informative, that is, new and relevant to the question. It found that 21.0% [18.7, 23.5] repeat earlier speech, more often on closed-form turns (23.2%) than on open-ended turns (12.0%). A lexical measure gives a similar share: in 23.7% [21.3, 26.4] of the updates, fewer than 30% of the content words are new within the turn. The judge marked 12.2% [10.4, 14.3] of the updates as hesitation or self-checking, which in practice is mostly narration of re-checking a step; none of the 1071 updates contain an explicit hesitation phrase. No update was off-topic (zero of 1071). None copy the prompt template either (zero of 1071), where copying means that at least half of the word 5-grams come from the template.
Each hard turn makes 5.84 model calls on average: one reasoning call, one acknowledgment, 2.84 progress dispatches (median one) and one final answer. Closed-form hard turns average 8.48 calls and open-ended hard turns 4.01. AIME 2026 turns average 17.1 calls because their reasoning is long. When the reasoning ends, about one queued progress chunk per hard turn (0.98 on average) is dropped without being played. Over all the turns, including those on the fast path, CATT makes 4.58 calls per turn against one for each baseline (Table 7).
4.6. Failure Cases
Table 8 sorts the failures of the router configuration by type, with one paraphrased example for each. All 738 turns completed, and none reached the 1200 s watchdog. Two failure types change accuracy against the thinking baseline. The first is routing: 46 closed-form questions took the fast path, which answered 37 of them correctly against 39 for the thinking baseline on the same questions. The second is the final answer written after the reasoning. On the 168 hard closed-form questions the reasoning is byte-identical, yet the final answer differs on four, with a net gain of one question for CATT. Together, these two sources make up the five and four discordant questions of the McNemar test and the net difference of one question out of 214 ( pp). Intermediate-speech errors therefore change no accuracy outcome, although they occur in 34.5% of the closed-form hard turns (25.2% of those with a correct final answer; Section 4.5). The reasoning budget was reached on three of the 30 AIME 2026 problems. Because the reasoning is identical, these are the same problems in both conditions, and both conditions answered all three wrongly.
Table 8.
Failure cases of CATT (router configuration) on the 738 evaluated items. The accuracy effect is the number of closed-form questions gained (+) or lost (−) against the thinking baseline out of 214. Examples are paraphrased.
5. Additional Studies
The studies in this section extend the main comparison in five directions. Each is an additional analysis or run, and each is described as it was computed. Section 5.1 compares the CATT router with other routing policies and summarizes how cost and effect change with the routing threshold. Section 5.2 replays the recorded runs of the main study under perturbed timing. Section 5.3 runs CATT on three hosted reasoning models, and Section 5.4 compares it with ReVerT on the same reasoning. Section 5.5 places CATT in an end-to-end digital-human system. Throughout this section, as in Section 4, the first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; that of the baselines is the answer onset, and that of ReVerT is its first verbalization.
5.1. Comparison of Routing Policies
In CATT the router is a replaceable component (Section 3.3), so the question is whether it earns its place against simpler or off-the-shelf policies. To compare policies on equal terms, we used forced-route runs of the same Qwen3-32B deployment and router on 170 closed-form questions: the 30 AIME 2026 problems, a 90-item subset of GPQA Diamond and the 50 MMSU questions. Each question was run once forced to the fast path and once forced to reasoning. The accuracy of any routing policy can then be read from the recorded outcome of the path it chooses for each question, so all the policies are scored on the same generations. These runs used prompts that differ from those of the main study, and their absolute accuracies are therefore not comparable with Table 7. None of the 170 questions were used to set the routing threshold.
The router was compared with five policies: random routing (its expected accuracy), question length in characters, the share of digits and mathematical symbols in the question (math-symbol rule), the RouteLLM classifier [20] and the NVIDIA prompt task and complexity classifier [17]. For a budget of k questions sent to reasoning, each policy, the CATT router included, sends the k questions it scores the highest. We computed the accuracy of the router minus that of each policy at every budget from zero to 170 and averaged this difference over all the budgets. Table 9 gives these averages with 95% intervals from a paired bootstrap over questions stratified by dataset (20,000 draws, seed 20261002). For context, at the operating threshold (Section 3.3), the router sends 125 of the 170 questions to reasoning and answers 68.8% of the 170 questions correctly.
Table 9.
Routing policies compared on recorded forced-route outcomes of 170 closed-form questions. At every budget from 0 to 170 questions sent to reasoning, each policy, the CATT router included, sends the questions it scores highest. Each entry is the accuracy of the CATT router minus that of the policy at the same budget averaged over all budgets in percentage points (pp) with a 95% paired bootstrap interval over questions (20,000 draws stratified by dataset, seed 20261002). Positive values favor the router.
Averaged over all the budgets, the router was ahead of random routing by 4.96 pp [, ], the length heuristic by 3.36 pp [, ] and the RouteLLM classifier by 3.17 pp [, ]. It was on par with the NVIDIA classifier ( pp [, ]) and the math-symbol rule ( pp [, ]), whose intervals include zero. The router is thus useful but not unique: on these closed-form questions, an off-the-shelf complexity classifier and a simple symbol-based rule route about as well. The NVIDIA classifier also served as CATT’s router in the main study, and Section 4.4 compares that configuration with the router and all-thinking configurations. These router results are limited to Qwen3-32B in the evaluated deployment. The router could be recalibrated for another model, but this was not evaluated.
Appendix F (Figure A2) traces the cost–effect curve of the routing threshold on the 738 evaluation items from every turn on the fast path to every turn sent to reasoning, with marked. Each point is assembled from recorded turns of the main study rather than a new run. A turn that a threshold sends to reasoning, although the router configuration did not, is taken from the all-thinking configuration. A turn that a threshold keeps on the fast path, although the router configuration sent it to reasoning, is taken from the fast path of the NVIDIA-router configuration where that configuration also kept it there and otherwise from the non-thinking baseline (up to 352 turns). Closed-form accuracy rises from 43.9% with every turn on the fast path to 65.0% when 50% of the turns go to reasoning. It then stays nearly flat: 65.4% at , where 55.6% of the turns go to reasoning, and 66.4% with every turn sent there. This endpoint uses the router configuration’s turns wherever the router chose reasoning, so it differs slightly from the 65.9% of the all-thinking run. Over the same range, model calls per turn grow from 2.05 to 5.00, and first substantive content moves from 6.01 s earlier than in the thinking baseline to 0.78 s later. First audio stays 6.1 to 6.6 s earlier at every threshold.
5.2. Speech Continuity Under Timing Perturbations
In the recorded runs, speech on hard-routed turns is nearly continuous. The total silence between spoken segments of a hard-routed turn has a median of 0.13 s and a 90th percentile of 0.90 s (410 turns). On closed-form questions the two values are 0.10 s and 0.61 s and on open-ended requests 0.15 s and 1.19 s. No hard-routed turn in the router configuration had more than 2.15 s of silence in total. These values come from the playback timeline computed from measured synthesis times (Section 3.7), so they depend on how fast the deployment generates and synthesizes.
To see how far the timing results depend on that speed, we replayed every recorded turn of the router configuration and the thinking baseline with perturbed timing. The replay keeps each turn’s recorded decisions, texts and dispatch order. It shifts token arrivals, synthesis and playback according to the perturbation, starts each chunk once it has been synthesized and the previous chunk has ended, and applies the rule that drops queued progress speech at the end of reasoning on the perturbed timing. Without perturbation, it reproduces the recorded playback starts (Section 3.7). The perturbations were a fixed delay on the arrival of every generated token, as from network latency (dispatches triggered by the thinker receive it twice, once through the reasoning stream and once through the talker’s own stream); slower streaming of thinker and talker generation; a fixed delay on every synthesized chunk; slower synthesis; longer speech durations; and two combinations of these. Table A6 in Appendix E.1 lists every scenario.
CATT kept its earlier first audio and earlier first substantive content in every scenario, and both advantages grew as generation slowed. With 1.0 s added to the arrival of every token, first substantive content still came 3.88 s earlier than in the thinking baseline (3.99 s without perturbation), and the closed-form final answer came 3.06 s later (3.20 s without perturbation). With 1.0 s added to every synthesized chunk or synthesis slowed by a factor of four, the final answer was 2.61 s and 2.69 s later. When generation was slowed by factors of 1.25 and 1.5, first audio came 7.92 s and 9.52 s earlier and first substantive content 5.16 s and 6.25 s earlier, while the final answer was 1.82 s and 1.65 s later. Across all the scenarios, the final answer stayed between 1.65 s and 3.82 s later than in the thinking baseline.
Continuity tolerated added latency and slower synthesis. Under these perturbations the median silence of a hard-routed turn stayed at or below 1.06 s and its 90th percentile at or below 2.27 s. Continuity is, however, sensitive to generation speed. With generation slowed by factors of 1.25 and 1.5, the median rose to 1.72 s and 3.53 s and the 90th percentile to 16.45 s and 34.32 s: in long turns, progress speech no longer kept pace with playback, and long silences appeared. The replay keeps every progress dispatch at its recorded position in the reasoning stream. It therefore cannot add the updates that the running system issues when the audio queue runs low, so these silences are an upper bound for the running system rather than a measurement. A deployment that generates markedly more slowly than ours should measure continuity again.
A separate run on 84 development items, with every turn sent to reasoning, varied the queue threshold of the progress trigger (Section 3.6): a later progress update is dispatched only while less than of audio is queued, and the main study uses 5 s. Values of 2.5, 5 and 10 s gave median hard-turn silences of 0.14 s, 0.12 s and 0.13 s. Relative to the 5 s setting, they moved first substantive content and the final answer by 0.07 s or less in point estimate (Table A7 in Appendix E.2).
5.3. Transfer to Hosted Reasoning Models
To see which effects carry over to other models, we ran CATT with three hosted thinking models: DeepSeek V4.1 Flash [52], GLM-5.2 [53] and Qwen3.5-27B [54]. DeepSeek V4.1 Flash and GLM-5.2 were reached through a third-party OpenAI-compatible API service. Qwen3.5-27B was reached directly through Alibaba Cloud Bailian (DashScope), and CATT read its reasoning from the documented reasoning_content field; the third-party service streams the reasoning of the other two models in the same way. On that service, a request whose first token was slow to arrive was sent again, and the first stream to respond was used. Each model was compared with its own thinking and non-thinking baselines, prompt-matched as in the main study. The router, its threshold, the prompts, the reasoning budget and the timing constants were those of the main study. The router was not recalibrated, so every item took the same route as with Qwen3-32B. A small transport adapter outside the orchestration code mapped the thinking switch and the continuation request after a reasoning-budget stop to each API. Speech was synthesized with Piper [18] on CPU and played on the same virtual timeline as in the main study. This study and the case study (Section 5.5) ran the same orchestration as the main runs with a corrected spoken-form normalizer (Section 3.4): markdown emphasis marks and spaced dashes are no longer read aloud as mathematical operators. The corrected normalizer is included in the code release. The study uses a 139-item subset of the evaluation set that is stratified by source and route with closed-form items oversampled (64 closed-form and 75 open-ended). CATT routed 86 of these items to reasoning, 51 of them closed-form.
Earlier first audio and earlier first substantive content carried over to all three models (Table 10). CATT started speaking 6.16 s, 5.24 s and 48.02 s earlier than each model’s thinking baseline. The size of this gain follows how long each model reasons before it answers. The median first audio of the thinking baseline, its answer onset, was 8.47 s for DeepSeek V4.1 Flash, 7.51 s for GLM-5.2 and 48.88 s for Qwen3.5-27B. For CATT, whose first audio is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns, the medians were 2.21 s, 2.09 s and 0.92 s. First substantive content came earlier over all the items, and much earlier on the hard-routed closed-form items, by 13.26 s to 68.82 s. On the hard-routed open-ended requests, DeepSeek V4.1 Flash and GLM-5.2 showed a small delay ( s [, ] and s [, ]), as did the local deployment on the same requests ( s [, ]); each of these intervals includes or reaches zero. Qwen3.5-27B instead gave substantive content 17.17 s earlier on these requests [, ] because its thinking baseline reasons at length even on open-ended requests. CATT gave substantive content about 4 s later than each model’s non-thinking baseline ( s, s and s), close to the s of the local deployment on the same items. Closed-form accuracy was on par with the thinking baseline on all three models: 87.5% against 89.1% for DeepSeek V4.1 Flash, 90.6% for both on GLM-5.2, and 78.1% for both on Qwen3.5-27B. With 64 closed-form items, however, the intervals are too wide to show non-inferiority within 5 pp. CATT made 3.96 to 6.79 more model calls per turn than the thinking baseline.
Table 10.
Transfer to hosted reasoning models: CATT minus each model’s own prompt-matched thinking baseline with the router and all settings of the main study. Timing entries are medians of per-item paired differences in seconds and accuracy is the mean paired difference in percentage points (pp), each with its 95% bootstrap interval; negative timing values mean that CATT is earlier. The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; that of the baselines is the answer onset. For reference, the last column gives the local Qwen3-32B deployment of the main study on the same 139 items.
Speech continuity did not carry over unchanged. The median silence per hard-routed turn was 0.47 s for GLM-5.2 and 0.71 s for DeepSeek V4.1 Flash with a 90th percentile of 3.7 s for both against 0.10 s and 0.84 s for the local deployment on the same items. On these two models the residual silence comes from per-call latency. Every talker call through a hosted API waits longer for its first token than on the local engine, and the timing constants were chosen for the local deployment. Qwen3.5-27B had longer gaps (median 2.86 s and 90th percentile 20.9 s). Its progress talker often chose to skip an update while the reasoning was drafting and refining its answer. The progress prompt allows such a skip when the reasoning has not moved past the step that was last reported (Section 3.4), and Qwen3.5-27B used it in 42% of its progress calls. Stretches in which the progress talker skipped account for 63% of its silence on hard-routed turns. Continuity therefore depends on the model’s behavior as well as the per-call latency of the deployment.
Table 10 has no final-answer row. In the local deployment, the thinking baseline and CATT’s thinker produced byte-identical reasoning on every hard-routed turn, so a per-item difference in final-answer time measures the orchestration alone. The hosted APIs did not reproduce the same reasoning for the same request and seed: only 4.7%, 14.1% and 0% of the closed-form pairs had identical reasoning on DeepSeek V4.1 Flash, GLM-5.2 and Qwen3.5-27B. Per-item differences in final-answer time therefore mostly reflect how long each condition happened to reason. Appendix G reports them together with the time from each condition’s own end of reasoning to its final answer. In short, earlier first audio and earlier substantive content carried over without recalibrating the router, whereas continuity and final-answer timing have to be measured again for each deployment. The main quantitative results of this paper come from the local Qwen3-32B deployment.
5.4. Comparison with ReVerT
ReVerT [12] also speaks while a model reasons. A trained verbalizer reads the reasoning one segment at a time, decides for each segment whether to speak, and rewrites the reasoning into speech when it does. We ran the authors’ official implementation with their released 3B verbalizer in our setting. It received the same Qwen3-32B reasoning as CATT: for each item, the thinking baseline’s recorded token stream was replayed in real time with its recorded arrival times. ReVerT’s speech then went through the same synthesis, playback timeline and judges as CATT’s. The comparison uses the 139 items of the transfer study, and the CATT and thinking-baseline turns are those of the main study on these items.
ReVerT started speaking 8.37 s later than CATT and gave first substantive content 5.51 s later (Table 11). Its closed-form final answer came 14.77 s later. It was also later than the thinking baseline on all three events. The main reason is that ReVerT rarely spoke during the long exploratory reasoning that Qwen3-32B produces on these tasks. It spoke before the reasoning ended in 15 of the 86 hard-routed turns, and its speech covered 5.8% of the reasoning time against 93.1% for CATT. On the listening rating, ReVerT scored below CATT (closed-form ; open-ended ), below the thinking baseline ( and ) and below the non-thinking baseline ( and ). The two judges’ separate contrasts pointed in the same direction in all 12 cases. We make no accuracy claim. Judged semantically, ReVerT answered 56.3% of the closed-form items correctly against 60.9% for CATT, a difference of pp, whose interval includes zero [, ]. The automatic scorer, which matches option letters, shows a larger gap, mostly because ReVerT often stated an answer without its option letter (Appendix H).
Table 11.
ReVerT, CATT and the thinking baseline on the same Qwen3-32B reasoning (139 evaluation items). The level columns give medians (times), shares or means; the contrast columns give the median paired difference for times and the mean paired difference for accuracy and the listening rating, each with its 95% bootstrap interval. Positive time differences mean that ReVerT is later. The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; that of the thinking baseline is the answer onset, and that of ReVerT is its first verbalization.
This is a limited reproduction in our setting, not a reproduction of ReVerT’s published results. Its verbalizer was trained on different reasoning made of short step-by-step chains from a non-thinking model. On the authors’ own GSM8k training data, our setup reproduced the verbalizer’s training decisions in 99.0% of the segments. The implementation therefore runs as published, and the above behavior reflects the different reasoning model and tasks. The comparison shows how ReVerT, as released, behaves on long and exploratory reasoning; it does not show how a verbalizer retrained on such reasoning would behave. Replaying the stream also favors ReVerT slightly. CATT’s live thinker reached the end of the same reasoning a median 0.72 s later than the replayed stream ([0.65, 0.83] on the 86 hard-routed turns), far less than any gap in Table 11. Appendix H gives the fidelity checks, a smaller 0.5B verbalizer, a variant with collapsed blank lines and the replay check; all point the same way.
5.5. End-to-End Digital-Human Case Study
The main study evaluates the transcript-to-speech subsystem with virtual playback. The case study runs the same orchestration inside an end-to-end digital-human system with the local Qwen3-32B deployment as backend. Questions were rewritten in spoken form, synthesized and fed through a simulated microphone to the iic/SenseVoiceSmall speech recognizer [55]; for a few items, the spoken-form text was injected directly while recognition still ran. Replies were synthesized through the Alibaba Cloud Model Studio API using cosyvoice-v2 with the loongbella_v2 voice [19] and spoken by a LiteAvatar 2D avatar, streamed to a headless browser on the same server. All the times are taken from the browser’s playback events, not a loudspeaker. Barge-in trials played a spoken interruption, together with a simulated echo of the avatar’s speech, into the simulated microphone during the acknowledgment, the progress speech or the final answer of CATT turns. The main run used 28 case-study items (21 closed-form) in CATT and both baselines, and CATT routed 18 of them to reasoning. Final-answer timing comes from a separate run on 14 hard closed-form items with sentence-level preemption: once the final answer is ready, the sentence being played finishes and queued speech that has not started is dropped. As in the main study, items used to set the routing threshold are excluded. Table 12 summarizes both runs, and Appendix I gives the details.
Table 12.
End-to-end digital-human case study. Level columns give medians in seconds or counts of correct answers and successful trials; the last column gives the median of per-item paired differences with its 95% bootstrap interval. The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; that of the baselines is the answer onset.
In the main run, CATT’s first audio (the acknowledgment on hard-routed turns; the answer onset otherwise) came 7.23 s earlier than the thinking baseline’s answer onset (medians 1.96 s and 9.39 s), and CATT gave first substantive content 6.29 s earlier. As in the main study, its substantive content came later than in the non-thinking baseline ( s [, ]). First audio takes longer here than with virtual playback because it includes speech recognition, hosted synthesis and avatar rendering. CATT answered 15 of the 21 closed-form items correctly against 16 for the thinking baseline and nine for the non-thinking baseline. Barge-in succeeded in 18 of 20 trials; the avatar’s speech was lowered after a median 0.21 s and stopped after a median 1.04 s. In the preemption run, where CATT routed every item to reasoning, CATT’s acknowledgment started 41.64 s before the thinking baseline’s answer onset, and CATT gave first substantive content after a median 7.75 s against 43.53 s. Its final answer became audible a median 2.63 s after the final-answer text was ready. Measured from each condition’s own end of reasoning, the final answer came 1.32 s later than in the thinking baseline [, ], the same direction as in the main study. We use this measure because the reasoning was not reproduced between the two conditions in this run, so per-item differences in end-to-end final-answer time mostly reflect reasoning length. All five barge-in trials of this run succeeded.
Audio and video were poorly synchronized in all three conditions, and the avatar video froze more often in CATT turns (Table A12 in Appendix I). For each turn, we computed the share of audio–visual offset measurements that fall inside the acceptability window of Recommendation ITU-R BT.1359 [56]. The median of this share over turns was 0.18 for CATT, 0.17 for the thinking baseline and 0.24 for the non-thinking baseline, so, in a typical turn, most measurements fell outside the window. The baselines were affected as well, so this deficit is not specific to the orchestration. The video froze 23 times in the 28 CATT turns against eight times in the 28 turns of each baseline. On hard-routed turns, CATT passes the avatar a succession of short speech segments: the acknowledgment, the progress updates and the final answer. The additional freezes may be related to these frequent short segments. If so, they follow from how CATT speaks rather than incidental noise, and a digital-human system that uses CATT has to address them. We did not test this explanation.
The case study is small and descriptive. It demonstrates that CATT can be integrated into a digital-human system, but it does not yet demonstrate a satisfactory end-user digital-human experience. We make no continuity claim for the real chain. Real users, real microphones, real networks and multi-user load were not evaluated.
6. Discussion
The results show that CATT changes when the listener first hears speech and when substantive content arrives. It does not shorten the reasoning itself. On every hard turn the reasoning is byte-identical to that of the prompt-matched thinking baseline, so the gains come from speaking while the reasoning runs. The costs come from the additional model calls and the step that turns the finished reasoning into a spoken answer. This section first sets the benefits and costs side by side, then gives guidance on when to use CATT and discusses its compatibility with widely used models. It closes with three directions for improvement that the results point to: open-ended requests sent to reasoning, the routing decision, and the correctness of progress speech. Unless stated otherwise, differences are CATT (router configuration) minus the named baseline. Times are median paired differences, and accuracy and the listening rating are mean paired differences, each with its 95% bootstrap interval in brackets.
6.1. Benefits and Costs
Against the prompt-matched thinking baseline, the benefits lie in the time to first speech and the time to first substantive content. First audio arrives 6.34 s earlier ([−6.58, −6.08]). On hard turns this first audio is an acknowledgment; on easy turns and in the baseline it is the onset of the answer. First substantive content arrives 10.46 s earlier on closed-form math and science questions ([−14.65, −7.32]; 214 items) and 3.00 s earlier on open-ended requests ([−3.53, −2.20]; 524 items). On the closed-form questions routed to reasoning, where the listener hears the solution approach while the reasoning continues, the difference is 15.24 s ([−21.76, −10.14]; 168 items). Closed-form accuracy is on par: 65.4% against 65.9%, a difference of −0.5 pp ([−3.3, +2.3]). Speech on hard turns is nearly continuous in the evaluated deployment with a median total silence of 0.13 s per hard turn and a 90th percentile of 0.90 s. On the LLM-judged listening rating, which accounts for waiting, CATT scores 0.98 points higher on closed-form items ([+0.78, +1.18]) and 0.92 points higher on open-ended requests ([+0.79, +1.05]).
The costs are of four kinds. First, the final closed-form answer arrives later by 3.20 s over all the closed-form items ([+2.57, +3.98]) and by 4.55 s on those routed to reasoning ([+3.67, +5.06]). Because the reasoning is identical, this delay arises after the reasoning ends. At that point CATT lets the sentence being played finish and issues a separate call that phrases the final answer, whereas the thinking baseline continues directly into its answer. Second, open-ended requests sent to reasoning reach substantive content 1.15 s later than with the thinking baseline (Section 6.4). Third, progress speech can state incorrect intermediate claims. At least one such claim occurred in 34.5% of the closed-form hard turns (58 of 168; 25.2% of the 103 that ended with a correct answer) against 1.7% of the open-ended hard turns (Section 6.6). Fourth, CATT needs more computation and memory. It makes 4.58 model calls per turn against one and reads 3936 prompt tokens per turn against 490 because progress and final-answer calls read excerpts of the reasoning. The deployment keeps two copies of the model in GPU memory, one for the thinker and one for the talker, and a single user’s turn runs two to three model calls at once. The routing decision itself adds 0.32 s to the first audio of easy turns relative to the non-thinking baseline. Against that baseline, CATT is rated 0.37 points higher on closed-form items ([+0.07, +0.67]), where it is 21.0 pp more accurate, but 0.42 points lower on open-ended requests ([−0.57, −0.28]).
6.2. When to Use CATT
CATT suits spoken settings that mix easy and hard questions, where silence is costly and hearing the approach early is useful. Explanation, tutoring and expert assistance are examples of such settings. We did not evaluate these domains: the evidence comes from closed-form math and science questions and open-ended requests. Within that evidence, hard closed-form questions reveal their solution approach about 15.2 s earlier than with the thinking baseline at the same accuracy, while easy turns go to the fast path.
The thinking baseline is preferable when the earliest exact final answer matters since it delivers the closed-form final answer about 3.2 s sooner. Settings in which a wrong intermediate statement is unacceptable, such as medical or legal advice, call for particular care. In the evaluated deployment, 34.5% of the closed-form hard turns contained at least one incorrect intermediate claim (25.2% of those with a correct final answer). A configuration without progress speech might avoid this risk, but none were evaluated, so we do not recommend one.
For open conversational requests, a direct non-thinking reply received the highest listening rating. CATT matches it on the open-ended requests that it sends to the fast path (difference 0.00 points, [−0.16, +0.17]), but it must hold a second model copy to support the reasoning path. A deployment that serves only such requests therefore has little to gain from CATT.
Which benefits carry over also depends on the deployment. In the transfer study, earlier first audio and earlier substantive content held on all three hosted models. Continuity, by contrast, varied with per-call latency and model behavior with median hard-turn silence between 0.47 and 2.86 s. In the replay with generation slowed by a factor of 1.25, the 90th percentile of hard-turn silence rose from 0.90 s to 16.45 s. Continuity and final-answer timing should therefore be measured again for each new deployment.
6.3. Compatibility with Widely Used Models
The main study used Qwen3-32B on a local server, whereas many widely used reasoning models are proprietary and can be reached only through hosted APIs. This subsection discusses, without new experiments, what CATT needs from a model API, which APIs the experiments covered, and how CATT can be used with models whose reasoning is hidden.
CATT needs four things, the last of which is optional. The first is streaming output so that the controller can act on each reasoning chunk and speech can start before a talker call has finished. The second is a fast responder that gives the easy answer and serves every talker call. Its latency sets how soon the acknowledgment, each progress update and the final answer can start. The fast responder can be the reasoning model itself with thinking switched off, as in all our deployments. It can also be a separate, faster and possibly weaker model so that a strong model reasons as the thinker and a faster model speaks as the talker. The third is a streamed text view of the reasoning while it is generated, which can be the raw reasoning or streamed summaries of it. Progress speech and the final answer draw on this text: the progress talker reads its newest part, the scheduler counts its tokens, and the final talker reads its last 6000 characters. The fourth is a reasoning budget, after which the model stops reasoning and writes its answer. CATT enforced a budget of 16,000 tokens by stopping the stream and sending a request that continues from the reasoning (Section 3.5). Without a budget, the reasoning runs until the model ends it or the API’s output limit is reached.
A separate talker model and streamed summaries both fit the existing interface without changes to the orchestration code. The thinker and the talker already send their requests to separate endpoints, which in the main study served two copies of Qwen3-32B. The configuration names one model for both roles, so assigning a different model to each role is a change to the configuration or the transport adapter described below, not the orchestration. In the same way, a stream of reasoning summaries can be passed to the progress talker and the final talker in place of the raw reasoning, and the scheduler would then count summary tokens. With a strong model and a fast model, the router would choose between the fast model’s direct answer and the strong model’s reasoning. Its reference outcomes and threshold would then have to be rebuilt with these two models (Section 3.3).
The experiments cover one kind of API, which streams the model’s raw reasoning as text in a separate field and lets each request switch thinking on or off. In every deployment the same model took both roles. The local Qwen3-32B deployment met all four needs. The transfer study added three model families accessed through hosted APIs: DeepSeek V4.1 Flash and GLM-5.2 through a third-party OpenAI-compatible API service and Qwen3.5-27B directly through Alibaba Cloud Bailian. A small transport adapter mapped the thinking switch and the continuation request to each API, and the orchestration code ran unchanged (Section 5.3). Earlier first audio and earlier first substantive content carried over to all three models, but continuity did not carry over unchanged. The median silence per hard-routed turn was 0.71 s for DeepSeek V4.1 Flash, 0.47 s for GLM-5.2 and 2.86 s for Qwen3.5-27B against 0.10 s for the local deployment on the same items. On the first two models the remaining silence came from the longer wait for the first token of each talker call. On Qwen3.5-27B it came mostly from stretches in which the progress talker chose to skip an update.
As of October 2026, the official documentation of three widely used proprietary model families, from OpenAI, Anthropic and Google, describes a different kind of access [57,58,59]. All three APIs stream their output. None of them return the raw reasoning of these models through the standard request options. Each can instead return a summary of the reasoning on request and stream it as a separate part of the response [58,59,60]. On some models, including GPT-6 Astra, Claude Opus 5.5 and Gemini 4 Argon, reasoning cannot be switched off. Faster or lower-cost models of the same families, such as GPT-6 Luna, Claude Haiku 5.5 and Gemini 3.5 Flash-Lite, accept a setting with no or minimal reasoning [57,58,59,61]. For the models that cannot switch reasoning off, these APIs set the amount of reasoning through effort or thinking levels rather than a token budget. Based on these documented options, such models can be connected to CATT as a strong model that reasons, a faster model that speaks, and streamed reasoning summaries that drive progress speech.
This paper evaluated only deployments in which the same model took both roles and the raw reasoning was streamed. With summaries, how early and informative progress speech is would depend on how often the summaries arrive and how detailed they are. The final talker would also phrase the answer from the summaries rather than the reasoning, so the closed-form accuracy could differ from that of the model’s own answer. The documentation does not describe in detail how closely the summaries follow the hidden reasoning in time. A weaker talker model may phrase progress updates and final answers less accurately, and its per-call latency would shape continuity, as it did in the transfer study. None of these effects were measured. CATT’s continuation at the budget cannot be used where only summaries are returned because it sends the reasoning text back to the model. The API’s own effort or thinking levels can reduce the amount of reasoning instead, although the documentation describes them as guidance rather than a strict token limit. Where an API returns neither the raw reasoning nor summaries, CATT has only the acknowledgment and the final answer, so its first audio would still come early, but the listener would hear no progress speech.
6.4. Open-Ended Requests Sent to Reasoning
The weakest cell of the evaluation is the open-ended request routed to reasoning. The router sent 242 of the 524 open-ended requests (46.2%) to the reasoning path. On these turns CATT reached first substantive content 1.15 s later than the thinking baseline ([+0.90, +1.41]). Their reasoning is short and leaves little progress to report, so CATT’s first substantive sentence is usually its final answer. That answer can start only after the reasoning ends and the final-answer call returns. The thinking baseline, by contrast, starts its answer as soon as its short reasoning ends.
Against the non-thinking baseline the gap is larger. On the same 242 requests, CATT reached substantive content 6.49 s later ([+5.46, +7.71]) and was rated 0.91 points lower ([−1.15, −0.68]). On the 282 open-ended requests sent to the fast path, the rating did not differ (0.00, [−0.16, +0.17]). The deficit also grows with the share of open-ended requests sent to reasoning: it is −0.42 points for the router configuration (46%), −0.52 for the NVIDIA configuration (65%) and −0.99 for the all-thinking configuration (100%). In these turns CATT speaks an acknowledgment and progress updates before its answer. We did not test which of these elements lowers the rating.
Two changes could address this cell, and neither was evaluated here. The routing decision could send fewer open-ended requests to reasoning (Section 6.5). For the open-ended requests that are still sent there, CATT could shorten the path to the answer, for example by cutting narration before the answer or starting the final answer sooner.
6.5. Improving the Routing Decision
Routing pays off even though reasoning on every turn gives similar accuracy. Against the all-thinking configuration, the router configuration reached nearly the same closed-form accuracy (65.4% against 65.9%). It did so with fewer calls per turn (4.58 against 4.96), fewer prompt tokens (3936 against 4351), fewer completion tokens (955 against 1124) and less GPU energy per turn (18.1 kJ against 21.2 kJ). It also gave first substantive content earlier at a median of 5.22 s against 7.36 s. On open-ended requests, the all-thinking configuration was rated 0.57 points lower than the router configuration ([−0.71, −0.44]); the other differences between the configurations on the listening rating were not resolved. The energy figures include idle draw and were measured for a single user, so they mostly follow the duration of each turn.
The routing decision can still improve in two directions. The first is accuracy at a given reasoning budget. In the routing comparison on forced-route runs, averaged over all the budgets from zero to 170 questions sent to reasoning, the CATT router was ahead of random routing, the length heuristic and the RouteLLM classifier. It was only on par, however, with the NVIDIA classifier (+0.36 pp, [−1.63, +2.31]) and a simple math-symbol rule (+1.74 pp, [−0.13, +3.63]). On the 738-item set, 46 closed-form items took the fast path; on these, CATT was wrong where the thinking baseline was right on four items and right where it was wrong on two.
The second direction is the routing target. The router estimates whether enabling the model’s thinking improves the outcome. It does not weigh what the reasoning path costs the listener in waiting and intermediate speech, and, for open-ended requests, the listening rating favored the direct reply (Section 6.4). A target that includes this cost or a separate threshold for open-ended requests might send fewer of them to reasoning. Neither variant was evaluated. The router was also calibrated only for Qwen3-32B, and another model would need its own reference outcomes and threshold (Section 7).
6.6. Checking Progress Speech Against the Reasoning
The audit of intermediate speech shows where the risk lies. The acknowledgments were almost always correct: one of the 410 acknowledgments misread an open-ended request, none were wrong on closed-form items, and the guard that suppresses answer statements never fired. Progress speech on closed-form reasoning carried most of the errors, with 170 of 862 closed-form progress updates labeled wrong against four of 209 on open-ended requests. On closed-form turns with a correct final answer, 65 of 378 progress updates were labeled wrong and, on those with a wrong final answer, 105 of 484. No update stated an answer that was later reversed. The errors are therefore incorrect intermediate claims, not premature final answers. On turns that ended with a wrong final answer, they were mostly wrong reasoning steps rather than early statements of that answer. A plausible source is that progress speech narrates reasoning that is still under way. It can therefore voice steps that the model later revises and, on turns that end with a wrong answer, steps of the faulty reasoning itself. We did not trace each wrong update back to the reasoning.
One possible improvement is to check each progress update against the reasoning before it is spoken. Such a check could compare the update with the reasoning that has arrived since the update was drafted or hold back candidate values until the reasoning confirms them. It would cost time since every update would wait for the check, and the perturbation replay showed that continuity is sensitive to generation speed. This check could not, however, catch a wrong step that the later reasoning does not correct, and, on turns that end with a wrong answer, most errors appear to be steps of this kind. A check could also address repetition. With long reasoning, progress speech keeps the channel filled, but its content repeats more. The judge labeled 21.0% of the 1071 audited progress updates as repeating earlier speech (23.2% on closed-form and 12.0% on open-ended turns), although 72.5% added new relevant content and none were off-topic. An update that adds nothing new could be skipped provided that the remaining speech still covers the wait.
7. Limitations
The following limitations bound the scope of these results.
- Virtual playback in the main study. The main study evaluates the transcript-to-speech subsystem on isolated turns. Its playback timeline is computed from real synthesis times (Piper on CPU) and is not measured on a playback device. The end-to-end case study adds real synthesis, playback through a browser-rendered avatar and barge-in. It is small, however (28 items plus a separate run on 14 items). Its silence measures are not comparable with those of the main study, so we make no continuity claim for the real chain. Its spoken questions were synthesized and injected into a headless browser on the server, so no real user or microphone was involved.
- Audio–visual synchronization and video freezes in the case study. Audio–visual synchronization was weak in all three conditions of the case study. The median over turns of the share of audio–visual offset measurements inside the acceptable window was 0.18 for CATT, 0.17 for the thinking baseline and 0.24 for the non-thinking baseline. The video also froze 23 times in the 28 CATT turns against eight times in the 28 turns of each baseline. The additional freezes may be related to the frequent short speech segments that CATT passes to the avatar. If so, they concern the architecture and not only the renderer. We did not test this explanation. They could be reduced by merging short segments before they reach the avatar or by a renderer that handles a succession of short segments without freezing. Neither remedy was tested, and merging could delay some of the early speech. Synchronization needs improvement in all the conditions since the baselines were affected as well.
- Simulated network jitter. Network latency, slower generation and slower synthesis were only simulated by replaying the recorded runs with added delays. The replay keeps the recorded controller decisions apart from the final-answer preemption, so it cannot show how the controller would adapt to slower generation. The long silences under slower streaming are therefore an upper bound rather than a measurement. The transfer study called hosted APIs over real networks, but it did not vary network conditions or measure them separately from the service latency.
- No listener study. No human listened to the outputs. The listening rating is an LLM-judged proxy computed from text timelines of virtual playback, and the judges’ weighting of waiting is not calibrated to people. We therefore make no claim about perceived latency, naturalness or trust. Whether listeners prefer an early acknowledgment and progress speech to silence remains to be tested.
- LLM judges. All the semantic labels come from LLM judges, and none of the judges were validated against human ratings. First substantive content and intermediate errors are labeled by a single judge (gpt-5.6-luna). For intermediate errors, this judge also sees the reference answer, so, on turns that end with a wrong final answer, it may rate the intermediate steps more strictly. The listening rating averages two judges from different providers (gpt-5.6-luna and gemini-3.1-flash-lite). Their agreement (intraclass correlation 0.78) and their stability on re-judging (0.95 and 0.93) are reasonable, but agreement and stability do not establish validity. The judges are not told which condition produced a reply, yet its style, such as an opening acknowledgment, remains visible to them.
- One generation per condition. Each condition was run once per item, so the intervals reflect item sampling only and do not cover generation randomness. On hard turns the reasoning is byte-identical between CATT and the thinking baseline, so the accuracy and final-answer comparisons there do not depend on randomness in the reasoning. The spoken final answer, the progress speech and the fast-path replies are still single samples. The hosted APIs and the case-study backend did not reproduce the reasoning across conditions, so final-answer timing there is measured from each condition’s own end of reasoning.
- Single-user operation. All the runs served one user at a time. CATT keeps two model copies in GPU memory, runs two to three model calls at once per user against one for the baselines, and reads more prompt tokens per turn. Under multi-user load these costs would limit how many users one deployment can serve. Throughput, queuing and tail latency under such load were not measured. The GPU energy per turn includes idle draw and was measured for a single user, so it does not predict the energy per user under load.
- Router not recalibrated for other models. The router results hold for Qwen3-32B in the evaluated deployment. In the transfer study, the router and its threshold were reused without recalibration. The router can be recalibrated for another model by replacing its reference outcomes, but this was not evaluated. The transfer study used 139 items. With 64 closed-form items, its accuracy intervals are too wide to show non-inferiority within 5 pp; for DeepSeek V4.1 Flash, for example, the difference is −1.6 pp ([−9.4, +6.2]). Every evaluated deployment exposed a reasoning channel, and models without one were not evaluated. Deployments in which the thinker and the talker use different models, and those in which reasoning summaries replace the raw reasoning, were not evaluated (Section 6.3).
- GPQA and AIME texts not released. The GPQA authors ask that the items not be published [21], and AIME items are handled in the same way. The released material therefore gives GPQA Diamond and AIME items only as identifiers, metrics and hashes without question text, responses or reasoning, and readers cannot re-check the judge labels of these items from the released files. The benchmark items are referenced by identifier with instructions for obtaining them, and audio is not included. The Data Availability Statement lists what is released and what is not; for example, the adapter code of the transfer and ReVerT runs is not included.
8. Conclusions
This paper presented CATT, an orchestration layer for spoken responses in digital humans. A router sends easy turns to a direct reply. Hard turns receive an immediate acknowledgment, then progress speech drawn from reasoning that runs in the background, and then the final answer. CATT needs no fine-tuning and is plug-and-play in a defined sense: its orchestration code runs unchanged when the model endpoint, the router, or the speech and playback layer is replaced. The main study evaluated the transcript-to-speech subsystem of a local Qwen3-32B deployment on 738 isolated turns with virtual playback.
Against a prompt-matched thinking baseline, CATT starts speaking about 6.3 s earlier; its first audio is an acknowledgment on hard turns and, as in the baseline, the answer onset on the other turns. It conveys first substantive content about 4.0 s earlier over all items: about 10.5 s earlier on closed-form math and science questions and about 3.0 s earlier on open-ended requests. The closed-form accuracy is on par (65.4% against 65.9%), the speech on hard turns is nearly continuous (median silence 0.13 s per hard turn), and the LLM-judged listening rating is about 1 point higher on both item types. These benefits come with costs. The final closed-form answer arrives about 3.2 s later, and open-ended requests sent to reasoning reach substantive content about 1 s later. Against a non-thinking baseline, CATT is rated higher on closed-form items (+0.37) but lower on open-ended requests (−0.42), a deficit that comes from the open-ended requests sent to reasoning. Progress speech contained an incorrect intermediate claim in 34.5% of the closed-form hard turns (25.2% of those that ended with a correct answer). Each turn takes 4.58 model calls on average against one, and the deployment keeps two model copies in GPU memory.
Earlier first speech and earlier substantive content carried over to three model families accessed through hosted APIs, whereas continuity and final-answer timing depended on the deployment. In a limited reproduction on the same reasoning, ReVerT [12] rarely spoke before the reasoning ended. The end-to-end case study added speech recognition on synthesized spoken questions, CosyVoice synthesis, a browser-rendered avatar and barge-in. It confirmed the earlier first speech with a median first audio of 1.96 s (the acknowledgment on hard-routed turns; the answer onset otherwise) against 9.39 s for the thinking baseline (answer onset). In a separate 14-item run with sentence-level preemption, the final answer came 1.32 s later than in the thinking baseline when measured from the end of reasoning. In the main 28-item run, however, audio and video were poorly synchronized in all three conditions. The avatar video also froze 23 times in the 28 CATT turns against eight times in the 28 turns of each baseline. This difference may be related to CATT’s frequent short speech segments, which is a relation that we did not test. Therefore, the case study demonstrates that CATT can be integrated into a digital-human system but not yet a satisfactory end-user digital-human experience. Real users, real microphones, real networks and multi-user load were not evaluated.
CATT is therefore best suited to spoken settings that mix easy and hard questions and in which hearing the approach early matters more than receiving the exact final answer a few seconds sooner. Future work should test these trade-offs with human listeners and measure throughput under multi-user load. It should also check progress speech against the reasoning before it is spoken, route open-ended requests with the listener’s waiting in mind, and recalibrate the router for other models.
Supplementary Materials
The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/electronics15215064/s1, Data S1: The zip archive of the snapshot of the CATT code and data release described in the Data Availability Statement. It contains the orchestration and evaluation code; the exact run configuration and prompts with their SHA-256 checksums; the per-turn records and gpt-5.6-luna labels of the main study on the 738 evaluated items; the listening rating scores; the per-turn data of both the ReVerT comparison and the transfer study on 139 of these items; and documentation for reproduction. GPQA Diamond and AIME texts are withheld, as described in the Data Availability Statement.
Author Contributions
Conceptualization, X.C. and F.X.; methodology, X.C., Z.D., F.X. and K.X.; software, X.C.; validation, X.C. and Z.D.; formal analysis, X.C. and Z.D.; investigation, X.C.; data curation, X.C.; visualization, X.C.; writing—original draft preparation, X.C.; writing—review and editing, Z.D., F.X., K.X. and T.C.; supervision, F.X., K.X. and T.C.; project administration, T.C. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable. This study did not involve human participants or animals.
Informed Consent Statement
Not applicable.
Data Availability Statement
The run records and code of the main study, the listening rating, the ReVerT comparison and the transfer study are released except GPQA and AIME texts and the materials listed later in this statement. They are hosted in a public GitHub repository at https://github.com/xixi951/catt (accessed on 15 October 2026), and a snapshot of this release (version 1.1.0) is provided as Supplementary Data S1. The release contains the orchestration and evaluation code and the exact run configuration and prompts with their SHA-256 checksums. The released orchestration files are byte-identical to those of the main runs. A later correction of the spoken-form normalizer, which the main runs did not use, is included separately. The evaluation code comprises the judge prompts, the judging scripts and the analysis code. For the 738 evaluated items in all five conditions, the data include the per-turn records (timings, routing scores and decisions, played and dropped speech, reasoning and per-call outputs) and the substantive-content and intermediate-error labels of gpt-5.6-luna, including those of the re-judged turns. The intermediate-error labels cover all hard-routed turns of the router configuration, including the 65 closed-form turns with a wrong final answer. The data also include the listening rating scores of both judges, with those of the validity checks and the re-judged replies, the per-turn metrics of the ReVerT comparison, and the per-turn records and gpt-5.6-luna labels of the transfer study on 139 of the evaluated items. The released analysis script recomputes the main reported values of these four studies from the released data. The per-turn data behind most other main-study tables are released as well, with the exceptions listed below. Benchmark items are not redistributed, and each benchmark remains under its own terms. Every evaluated item is listed by its source identifier, with a checksum of the exact question that was given to the model, so that the question can be obtained from the original source. For GPQA Diamond and AIME 2026 items, the questions, model responses, reasoning and spoken text are withheld and given as identifiers, metrics and hashes only. These texts can reproduce or closely paraphrase the questions, and the GPQA authors ask that the dataset’s examples not be revealed online in plain text. The router’s reference vectors are released without the vectors of its GPQA reference requests. Recomputing the routing scores therefore requires rebuilding those vectors, as documented in the release, while the recorded routing score and route of every evaluated item are included. Audio files and token-level streaming logs are not included but the timings derived from them are; the perturbation replay therefore cannot be recomputed from the release. The judges’ free-text reasons are not included, nor are the labels of the content audit of progress updates, the spoken text of the ReVerT replies, the semantic-correctness labels of the ReVerT comparison and the control run of its fidelity check. The frozen analysis scripts of the study are replaced by a released analysis script that recomputes the main reported values. In the transfer study, the hosted models’ reasoning texts are not included either; their timing and length are. The code that ran the orchestration on the hosted APIs in the transfer study and the adapter that ran ReVerT’s released verbalizer are not included, whereas the per-turn data of both studies are. Code is released under the Apache License 2.0 and the data generated in this study under the Creative Commons Attribution 4.0 International license (CC BY 4.0). The records of the routing-policy comparison and the backlog-threshold check are not part of this release. Records of the end-to-end case study are available from the corresponding author upon reasonable request subject to applicable third-party terms.
Acknowledgments
During the preparation of this manuscript, the authors used OpenAI Codex (codex-cli version 0.148.0-alpha.15) for code assistance, figure preparation, and consistency checks. The authors reviewed and edited the output and take full responsibility for the content of this publication.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A. Orchestration of One Turn
Algorithm A1 gives the control flow of one CATT turn as it was run in every experiment, and Table A1 lists its constants. Every decision is taken on observable events: streamed tokens, finished calls and the playback backlog B, which is the queued speech that has not yet been played. Times are seconds from question submission. The thinker request runs on the thinker engine, and all the talker calls (acknowledgment, progress, final answer and fast-path answer) run on the talker engine (Section 3.8.3).
| Algorithm A1 One CATT turn (router configuration). In the NVIDIA-router configuration the router score is replaced by the classifier’s score and its threshold; in the all-thinking configuration every turn takes the reasoning path. |
|
Several rules complete the algorithm. The router call, the thinker request and the acknowledgment call start at the same time, and the acknowledgment’s text is held back until the route is known. When the route is the fast path, the thinker request and the acknowledgment call are canceled, so no acknowledgment audio is played before the answer. The fast-path answer starts a median 0.32 s later than the non-thinking baseline’s answer ([, ] s; 328 turns), mainly because it waits for the routing decision.
Table A1.
Constants of Algorithm A1, identical in all conditions and studies unless stated otherwise.
The acknowledgment confirms the question and names its topic without an answer or claims about the approach. Its requested length is set from the predicted time at which the first progress update can start playing. A guard removes any answer statement from it; the guard never fired in the evaluation (0 of 410 hard turns). Progress speech states only reasoning-derived question-specific content and must mark any candidate answer as tentative. A guard also removes progress sentences that state an answer without such a mark. The progress talker may answer SKIP, in which case nothing is spoken. On normal turns the thinker stream is closed at the first token of the visible answer, and the final-answer talker works from the reasoning. When the reasoning budget is reached or the loop guard detects repeated reasoning, the thinker engine continues the generation from the end of the reasoning, and the final-answer talker states the model’s written answer. On closed-form questions the final answer starts with the answer itself, including the option letter; on open-ended requests it starts with the content. A question counts as closed-form if its text asks for a boxed answer or lists at least two lettered options on separate lines; the rule reads only the question text.
The spoken output keeps a strict order. At most one talker call is in flight, and no talker call starts after the final one, so spoken segments cannot play at the same time, change order or interleave with the final answer. When the reasoning ends, queued progress chunks that have not started playing are dropped; a chunk whose synthesis is already under way is still played. On average, 0.98 queued progress chunks per hard turn were dropped in this way. Chunks of a canceled call that arrive later are dropped on arrival. Speech is played on a single channel: chunk k starts at (Equation (3)), where is the time its synthesized audio is ready and is the end of the previous chunk. In the main study, comes from the measured synthesis times of Piper [18] on CPU. Synthesis runs ahead while earlier chunks play. The playback timeline is computed from these times rather than played on a device. A replay of the recorded runs without any perturbation reproduces every recorded playback start within 7.1 ms (Appendix E.1).
Both baselines make a single call with the same spoken-answer prompt with thinking enabled or disabled. Their output passes through the same sentence splitter, spoken-form conversion, synthesis and playback timeline. The thinking baseline’s request is identical to the thinker request H, including the reasoning budget, its continuation and the loop guard. On every hard turn of the evaluation, the two reasoning streams are byte-identical (410 of 410).
Appendix B. Data Provenance
Table A2 separates the data used in this study by origin and role. Design choices and sensitivity checks used only development items, which are disjoint from the evaluation fixture by identifier and normalized text. Of the 802 fixture items, 64 GPQA Diamond items were used to set the routing threshold and are therefore excluded from evaluation. They were run in every condition, like all other fixture items. The transfer study and the ReVerT comparison use a 139-item subset of the 738 evaluated items (64 closed-form; 75 open-ended). The case study uses its own items from the same benchmarks, again without the items used to set the routing threshold.
Table A2.
Data provenance of the main study.
Appendix C. Judge Reliability
The listening rating is the mean of two judges, gpt-5.6-luna and gemini-3.1-flash-lite. Each judge gives a score from 1 to 10 after reading the timeline of everything the listener heard. Table A3 reports four checks of this rating: agreement between the judges, agreement of each judge with itself, the response of the rating to waiting, and its association with reply length. The substantive-content and intermediate-error labels come from gpt-5.6-luna alone. For these labels, the table gives luna’s own test–retest agreement.
Agreement between the judges is computed over all 3829 rated turns: the five conditions of the main run on the 738 items and the ReVerT turns on the 139-item subset. It is higher on closed-form items than open-ended ones. A contrast is read as higher when its 95% interval lies above zero, lower when the interval lies below zero, and showing no detected difference otherwise. There are 18 contrast cells: the three CATT configurations against each baseline on each item type (12 cells) and ReVerT against CATT and each baseline on each item type (6 cells). The two judges never reach opposite readings. In 6 of the 12 cells of the main run, they differ only in whether the interval excludes zero.
For test–retest, 393 replies were judged a second time. They consist of the replies of all five conditions on 73 of the 738 items (10% of each item type; 365 replies) and the ReVerT replies on 28 of the 139 subset items (20% of each item type). Replies first judged about 19 h earlier scored 0.10 points lower on average when judged again. Re-judging changed no pooled contrast by more than 0.07 points.
Two probes test whether the rating responds to waiting. They use synthetic timelines built from recorded replies and the same judge prompt. In the first probe, the thinking baseline’s reply on 60 items keeps its words and internal timing, while its first sentence is moved from 0.5 s to 60 s. The mean score falls by 4.18 points, and it is lower at 60 s on 98% of the items. In the second probe, 30 s of silence are inserted after the acknowledgment of 40 CATT hard turns. The mean score falls by 1.99 points, so the judges do not treat an early acknowledgment as the end of the wait.
Across turns, longer replies receive lower ratings, most clearly on closed-form items. There, long replies are long spoken derivations of harder problems, so this association mixes length with difficulty. Within an item, where all the conditions answer the same question, the rating is unrelated to the number of words heard. The rating therefore shows no sign of rewarding length. This is an association only. None of these checks compare the judges with human listeners.
Table A3.
Reliability and validity of the listening rating (1–10 scale, mean of gpt-5.6-luna and gemini-3.1-flash-lite) and test–retest agreement of the gpt-5.6-luna labels used for substantive content and intermediate errors. Intervals are 95% bootstrap intervals.
Appendix D. Valid Samples, Timeouts and Distributions
Every scheduled turn completed in every condition (Table A4), so no turn is missing from any analysis. No turn came near the 1200 s watchdog. The longest turn finished playback at 384 s, so any watchdog of 400 s or more gives identical results. Three AIME items reached the 16,000-token reasoning budget in CATT and the thinking baseline alike. They are the same items because the two reasoning streams are identical and all three were answered wrongly in both conditions. Turns in which no played sentence was labeled substantive are assigned 1200 s; this concerns 8, 5 and 7 turns (CATT, thinking and non-thinking).
Table A4.
Valid samples, timeouts and censoring on the 738 evaluated items.
Table A5 gives the distribution of the main timing metrics, and Figure A1 shows the corresponding empirical cumulative distributions. Hard-turn silence stays short throughout: its median is 0.13 s, its 90th percentile 0.90 s and its maximum 2.15 s, and 28.3% of hard turns contain no silence at all. The largest single silence within a hard turn has a median of 0.12 s, a 90th percentile of 0.83 s and a maximum of 2.15 s. The other two CATT configurations behave similarly. Over its 554 hard turns, the NVIDIA-router configuration has a median of 0.00 s, a 90th percentile of 0.44 s and a maximum of 2.49 s; over its 738 turns, the all-thinking configuration has 0.15 s, 1.01 s and 2.75 s. In the baselines, 98.1% (thinking) and 95.9% (non-thinking) of turns contain no silence once playback has started. Playback completion has medians of 31.44, 35.03 and 23.16 s (CATT, thinking and non-thinking), 90th percentiles of 73.19, 71.22 and 37.68 s, and maxima of 369.90, 383.90 and 208.68 s.
Table A5.
Distribution of the main timing metrics on the 738 evaluated items in seconds from question submission. First audio is the acknowledgment on CATT’s hard-routed turns and the answer onset on its fast-path turns and in both baselines. First substantive content uses gpt-5.6-luna labels; turns without a substantive sentence are set to 1200 s, which gives the maxima and raises the 99th percentiles. The final answer is measured on the 214 closed-form items. Silence is the total silence between played chunks after the first audio on CATT’s 410 hard turns and all turns of each baseline. The share of turns with zero silence is given for silence only.
Figure A1.
Empirical cumulative distributions of the main timing measures on the 738-item evaluation set for the CATT router configuration and the two prompt-matched baselines. Times are seconds from question submission; the horizontal axes of (a–c) are logarithmic. (a) First audio: an acknowledgment on CATT’s hard-routed turns and the answer onset on its easy-routed turns and in both baselines (738 turns per condition). (b) First substantive content, labeled by gpt-5.6-luna (738 turns per condition). Turns with no sentence labeled substantive (CATT 8, thinking 5, and non-thinking 7) remain in the denominator, so these curves end below 1. (c) Final answer on the 214 closed-form items. (d) Total silence between spoken segments after playback starts on CATT’s 410 hard-routed turns and all 738 turns of each baseline. No silence occurs in 28.3% of CATT’s hard-routed turns and in 98.1% (thinking) and 95.9% (non-thinking) of baseline turns.
Appendix E. Perturbation Replay and Backlog Threshold
Appendix E.1. Perturbation Replay
This additional analysis replays the recorded turns of the main run under modified timing. For every turn of CATT’s router configuration and the thinking baseline, the playback timeline is recomputed from the recorded token arrival times, call times and synthesis times. Each chunk starts at the later of two times: when its audio is ready and the previous chunk ends. Without any perturbation, the replay reproduces every recorded playback start within 7.1 ms.
Five kinds of perturbation are applied, alone and in two combinations (Table A6). Under a network latency, every streamed token arrives later by a fixed delay. The delay counts twice for talker calls triggered by the reasoning stream because both the triggering reasoning and the talker’s reply cross the network. Slower generation stretches the thinker and talker streams by a constant factor. A synthesis latency adds a fixed delay to every chunk, and a synthesis factor multiplies every chunk’s synthesis time. Longer speech multiplies the duration of every spoken chunk.
Decisions are taken from the recorded run. Progress dispatches keep their recorded positions in the reasoning stream with their texts and order, and the router is not perturbed. CATT’s final-answer preemption is replayed on the perturbed timing: at the end of the reasoning, progress chunks that have not started playing are dropped, and a chunk whose synthesis is under way is still played. First substantive content is read from the gpt-5.6-luna labels of the sentences that are still played. Because dispatch positions are kept, the replay does not add the progress updates that the low-queue rule could trigger when the backlog runs low.
The latency advantages hold under every perturbation. A fixed delay shifts both systems alike and leaves first audio unchanged. Slower generation delays the thinking baseline, which must finish its reasoning before it speaks more than it delays CATT, so CATT’s lead grows: with generation slowed by a factor of 2, CATT’s first audio (the acknowledgment on hard-routed turns; the answer onset otherwise) comes 12.75 s before the thinking baseline’s answer onset. Across all the scenarios, the closed-form final answer stays between 1.65 and 3.82 s later than the thinking baseline’s. Longer speech fills the gaps but delays the final answer slightly because the final answer waits for the sentence being played.
Speech continuity tolerates added network latency and slower synthesis. Under these perturbations, the median hard-turn silence stays at or below 1.06 s and its 90th percentile at or below 2.27 s. Continuity is sensitive to generation speed, however. When generation slows by a factor of 1.25, 1.5 or 2, progress speech no longer keeps pace with playback, and the 90th percentile of hard-turn silence rises to 16.45, 34.32 and 71.13 s.
Table A6.
Perturbation replay of the recorded runs: CATT (router configuration) minus the thinking baseline on the 738 evaluated items; median paired differences in seconds with 95% bootstrap intervals. First audio is the acknowledgment on hard-routed turns and the answer onset otherwise. The final answer is measured on the 214 closed-form items and silence on CATT’s 410 hard turns. Values given without an interval equal the recorded value and have the same interval.
Appendix E.2. Backlog Threshold for Progress Dispatches
After the first progress update, a regular progress dispatch requires the playback backlog to be below the threshold , which is 5 s in all main runs. A lower therefore makes progress updates rarer, not more frequent. To check the sensitivity to this value, we ran the all-thinking configuration in which every turn goes to reasoning on 84 development items (54 closed-form; 30 open-ended) with set to 2.5, 5 and 10 s. Nothing else was changed, and the deployment was the one of the main study. In every turn of every run, the reasoning stream was byte-identical to the thinking baseline’s.
Halving or doubling changes the number of progress dispatches by 0.13 to 0.19 per turn (Table A7). A higher threshold dispatches more updates but also drops more queued chunks at the end of the reasoning. The first substantive content and the final answer move by less than 0.1 s, and the median hard-turn silence stays between 0.12 and 0.14 s. Intermediate-error rates move without a consistent direction. These are conditional rates: they count every open-ended turn but only the closed-form turns whose final answer is correct (68 or 69 turns per setting). The closed-form turns with a wrong final answer were not labeled in this check, so it gives no rate over all turns. With so few counted turns, the intervals are too wide to show an effect of the threshold. Within the range of 2.5 to 10 s, the 5 s setting is therefore not a sensitive operating point.
Table A7.
Sensitivity to the backlog threshold : all-thinking configuration on 84 development items (54 closed-form; 30 open-ended). Changes from the 5 s setting are paired over items with 95% bootstrap intervals: mean change for silence and median paired difference for timings. The intermediate-error rate is conditional: it counts all open-ended turns and only the closed-form turns with a correct final answer.
Appendix F. Routing-Threshold Cost–Effect Curve
Figure A2 shows how accuracy, timing and cost change with the share of turns sent to reasoning from every turn on the fast path to every turn on reasoning. This additional analysis is computed from the recorded runs of the main study; no new runs were made at other thresholds. At a threshold , an item goes to reasoning if its recorded router score is at least . Each item then contributes one recorded turn:
- The router configuration’s turn if the item goes to reasoning both at and ;
- The all-thinking configuration’s turn if it goes to reasoning at but not at ;
- The router configuration’s fast-path turn if it takes the fast path at both thresholds;
- If it takes the fast path at but goes to reasoning at , the NVIDIA-router configuration’s turn when that configuration sent the item to the fast path and otherwise the non-thinking baseline’s turn as a stand-in.
At the composite is exactly the router configuration. Stand-in turns occur only at shares below the operating point and number at most 352 at the all-fast end. They are slightly faster than CATT’s own fast path, which starts a median 0.32 s later than the non-thinking baseline, mainly because it waits for the routing decision. Paired differences from the thinking baseline use the bootstrap of the main analysis.
Closed-form accuracy rises steeply as closed-form items move to reasoning (Table A8). It goes from 43.9% with every turn on the fast path to 65.0% when 50% of the turns go to reasoning, which is within 1 percentage point of the thinking baseline (65.9%). Beyond the operating point, sending more turns to reasoning gains about one point of accuracy (66.4% with every turn on reasoning). Over the same range, first substantive content goes from 3.99 s earlier than the thinking baseline to 0.78 s later, the final answer from 3.20 s to 4.01 s later, and the number of model calls per turn from 4.58 to 5.00. The closed-form final answer comes earlier than the thinking baseline’s only while most closed-form items take the fast path, which costs accuracy. First audio (the acknowledgment on hard-routed turns; the answer onset otherwise) is not plotted because it is between 6.1 and 6.6 s earlier than the thinking baseline’s answer onset at every threshold. The all-reasoning end of the curve is a composite and is not the all-thinking configuration’s own run, which reached 65.9% accuracy.
Figure A2.
Cost–effect curve of the routing threshold on the 738-item evaluation set. Each point is one threshold, placed by the share of turns it routes to reasoning, from all turns on the fast path (0%) to all turns routed to reasoning (100%). The blue circles and vertical dashed lines mark the operating threshold , which routes 55.6% of turns to reasoning. The horizontal gray lines in panels (a)–(c) indicate zero difference from the thinking baseline. (a) Closed-form accuracy, mean paired difference from the thinking baseline. (b) First substantive content (gpt-5.6-luna) and (c) closed-form final answer; median paired differences from the thinking baseline. Shaded bands are 95% bootstrap intervals. (d) Mean model calls per turn. At the differences are −0.5 pp [−3.3, +2.3], −3.99 s [−4.32, −3.62] and +3.20 s [+2.57, +3.98] with 4.58 calls per turn. First audio (the acknowledgment on hard-routed turns; the answer onset otherwise) is not shown: it is 6.1–6.6 s earlier than the thinking baseline’s answer onset at every threshold. The curve is computed from the recorded turns without new generation. An item whose router score is at or above the threshold uses the router configuration’s turn if that run routed it to reasoning and otherwise the all-thinking configuration’s turn. An item below the threshold uses the router configuration’s fast-path turn if that run sent it to the fast path; otherwise it uses the NVIDIA-router configuration’s fast-path turn or, if that run also routed it to reasoning, the non-thinking baseline’s turn. Such baseline turns occur only to the left of the marked point (at most 352 turns at the all-fast end).
Table A8.
Selected points of the routing-threshold curve: CATT minus the thinking baseline on the 738 evaluated items with 95% bootstrap intervals. Accuracy is given with the mean paired difference in percentage points; times are median paired differences in seconds.
Appendix G. Final-Answer Timing on Hosted APIs
The transfer study (Section 5.3) reports no per-item final-answer differences in the main text. CATT sends its thinker the same request and seed as the thinking baseline. On the local deployment, this reproduces the reasoning exactly on every hard turn. The hosted APIs do not reproduce it: only 4.7% (DeepSeek V4.1 Flash), 14.1% (GLM-5.2) and 0% (Qwen3.5-27B) of the 64 closed-form pairs have identical reasoning. On the local deployment the share is 79.7%, which is every pair routed to reasoning. Even the number of turns that reach the reasoning budget differs between the two conditions on one model (Table A9). A per-item difference in final-answer time on a hosted API therefore mostly reflects the difference in reasoning length between two independent generations.
Table A9 gives these per-item differences together with a measure that does not depend on reasoning length: the time from each condition’s own end of reasoning to the start of its final answer on hard closed-form items. The per-item differences point in both directions and have wide intervals. The closed-form value for Qwen3.5-27B is dominated by the closed-form items routed to the fast path on which CATT answers without reasoning while this model’s thinking baseline reasons at length.
Measured from the end of reasoning, CATT’s final answer starts a median 3.72, 5.36 and 3.74 s after its reasoning ends. The thinking baseline, which continues from its reasoning directly into its answer, starts 0.28, 0.47 and 0.55 s after. The paired difference of 2.39 to 3.38 s has the same sign as on the local deployment. There, on the 51 hard closed-form items of the same subset, the reasoning is identical and the paired final-answer difference is s [, ]. In CATT, the final answer is a separate talker call that starts only after the reasoning has ended and the sentence being played has finished. The hosted runs add the per-call latency of the API to this delay. The pairs counted for this measure are those in which both turns gave a final answer and both ends of reasoning were recorded.
Table A9.
Final-answer timing in the transfer study: CATT minus each model’s own thinking baseline on the 139-item subset. Median paired differences in seconds with 95% bootstrap intervals. DeepSeek V4.1 Flash and GLM-5.2 were accessed through a third-party OpenAI-compatible API service and Qwen3.5-27B directly through Alibaba Cloud Bailian (DashScope).
Appendix H. ReVerT: Setup, Fidelity and Supplementary Arms
Appendix H.1. Setup
We ran ReVerT [12] from the authors’ official repository (commit e81b0ad) with their released verbalizers: the 3B model as the primary arm and the 0.5B model as a supplementary arm. The verbalizer client was used unchanged with its prompt, special tokens, decision call and verbalization call. The loop follows the authors’ interactive demonstration for one question. The reasoning is cut into segments at every line break, and the verbalizer decides for each segment whether to speak. A decision to speak triggers a verbalization while the reasoning keeps arriving, and the remaining buffer is always verbalized at the end.
The reasoning is the thinking baseline’s recorded Qwen3-32B stream from the main run, replayed in real time with its recorded token arrival times. ReVerT, CATT and the thinking baseline therefore speak about the same token sequence. The adaptations concern only the interface. The replayed stream replaces the demonstration’s live model call. The verbalizer’s maximum context was raised so that it accepts the longest reasoning in the set. Each verbalization is passed as one dispatch into CATT’s speech path with the same splitter, spoken-form conversion, Piper synthesis and playback timeline. Verbalizations triggered by reasoning segments count as intermediate speech; those triggered by answer segments, and the final flush, count as the final answer. Each verbalizer ran on a GPU of its own. Scoring, judging and statistics are those of the main study on the 139-item subset.
A supplementary arm applies the newline collapsing of ReVerT’s training preprocessing to the reasoning stream. It was added on the basis of development items because the demonstration passes Qwen3-32B’s blank lines to the verbalizer as segments, which its training data never contained.
Appendix H.2. Fidelity Check
Four checks test whether the setup shows the behavior that ReVerT specifies (Table A10). F1 checks the mechanics: verbalizations must end at the end-of-verbalization token or the end of the sequence. F2 checks the defining behavior, speaking while the reasoning is still running. F3 checks that the output is speech-ready text without LaTeX or equation symbols. F4 is an in-domain control on the authors’ own GSM8k training data. It tests whether the released verbalizer reproduces its training decisions in our installation.
The primary arm passes the mechanical check and reproduces its training decisions in 99.0% of the control segments, so the installation is correct. The defining behavior does not carry over: on Qwen3-32B reasoning, only 17.4% of hard turns contain a verbalization before the reasoning ends. We therefore call the comparison a limited reproduction in our setting, not a reproduction of the published results. ReVerT’s verbalizer was trained on different reasoning, and it rarely judges Qwen3-32B’s long exploratory reasoning as ready to be spoken. On the 86 hard turns, ReVerT speaks before the reasoning ends in 15 turns (17%), and its speech covers 5.8% [2.9, 9.2] of the reasoning time against 93.1% [91.6, 94.6] for CATT. Because it makes one decision call per reasoning segment, ReVerT makes 42.1 model calls per turn on these items against 5.19 for CATT (difference [, ]).
Table A10.
Fidelity checks of the ReVerT arms on the 139-item subset (86 hard-routed items).
Appendix H.3. Supplementary Arms
Table A11 compares the three arms with CATT and the thinking baseline. The 0.5B verbalizer speaks during the reasoning slightly more often than the 3B model, but it fails the mechanical check F1. Collapsing line breaks makes speech during the reasoning rarer (F2 of 11.6%). Every arm starts speaking and gives the closed-form final answer later than CATT and the thinking baseline. The one exception is the 0.5B arm’s first audio against the thinking baseline, whose interval includes zero.
Table A11.
ReVerT arms on the 139-item subset: ReVerT minus CATT and ReVerT minus the thinking baseline; median paired differences in seconds with 95% bootstrap intervals. The final answer is measured on the 64 closed-form items. The first audio of CATT is the acknowledgment on hard-routed turns and the answer onset on easy-routed turns; that of the thinking baseline is the answer onset, and that of ReVerT is its first verbalization.
Appendix H.4. Accuracy Scoring, Final-Answer Timing and Replay
The automatic scorer used throughout the paper gives ReVerT 32.8% on the 64 closed-form items against 60.9% for CATT ( percentage points [, ]). Most of this gap comes from ReVerT answers that state the answer by its value or content but without the option letter. Of the 64 ReVerT turns, 15 are marked wrong by the scorer but correct by a semantic judgment of the spoken answer (gpt-5.6-luna). Under the semantic judgment, accuracy is 56.3% for ReVerT and 60.9% for CATT ( percentage points [, ]). CATT’s accuracy is the same under both scorings, and no accuracy difference between the two systems can be claimed.
The later final answer comes from how ReVerT ends a turn. Measured from the end of reasoning, ReVerT starts the closed-form answer a median 10.64 s after the reasoning ends against 3.23 s for CATT and 0.78 s for the thinking baseline. The paired differences are s [, ] against CATT (42 pairs) and s [, ] against the thinking baseline (53 pairs). ReVerT verbalizes the answer only after the whole answer has streamed, and its final verbalization usually summarizes the reasoning before it states the answer. The pair counts exclude turns whose spoken text yields no final answer and, against CATT, the closed-form items that CATT sends to the fast path, which have no end of reasoning.
Replaying the reasoning stream slightly favors ReVerT. On the 86 hard items, CATT’s live thinker, which shares its GPUs with the talker, reached the end of the same reasoning a median 0.72 s [0.65, 0.83] later than the replayed stream that ReVerT received. This advantage is far smaller than every reported gap between ReVerT and CATT. Contrasts between ReVerT and the thinking baseline use the identical stream and timing, so they carry no such advantage. In a single-node deployment, ReVerT’s verbalizer would also share the thinker’s GPUs, which could only make ReVerT later.
Appendix I. End-to-End Case Study: Audio–Visual Synchronization and a Known Gap
The case study (Section 5.5) runs the same orchestration code with the local Qwen3-32B backend, CosyVoice synthesis and a browser-rendered avatar. Everything runs in a headless browser on the server, and synthesized spoken questions pass through speech recognition. Table A12 reports audio–visual synchronization and video freezes in the 28-item run.
For each turn, we computed the share of audio–visual offset measurements that fall inside the acceptability window of Recommendation ITU-R BT.1359 [56]. The table gives the median of these per-turn shares. Synchronization is poor in all three conditions: in a typical turn, most measurements fall outside the window. The baselines are affected as well, so the deficit is not specific to the orchestration, and the case study includes no comparison that would isolate its source. The measured offsets are video lags that grow with turn duration, so conditions with longer turns are not directly comparable. Video freezes were more frequent in CATT turns. On hard-routed turns, CATT passes the avatar a succession of short speech segments: the acknowledgment, the progress updates and the final answer. The additional freezes may be related to these frequent short segments. We did not test this explanation. Both issues are listed among the limitations.
Table A12.
Audio–visual synchronization and video freezes in the 28-item run of the case study.
The separate 14-item run on hard closed-form items uses sentence-level preemption. When the reasoning ends, the sentence being played finishes and all queued speech that has not started is dropped. Text that has not reached synthesis is skipped, and audio that has already been synthesized is discarded. Over the 14 turns, the final answer became audible a median 2.63 s after its text was ready, mostly the time needed to finish the sentence being played.
One known gap remains. Synthesis that is already running for a dropped segment is not aborted. The single synthesizer then finishes that segment, whose audio is discarded, before it starts on the final answer. This delayed the final answer in 2 of the 14 turns by about 4.6 s and 0.5 s. Aborting the synthesis of dropped segments would remove this delay; it was not implemented in the evaluated system.
References
- Sacks, H.; Schegloff, E.A.; Jefferson, G. A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 1974, 50, 696–735. [Google Scholar] [CrossRef] [Scilit]
- Stivers, T.; Enfield, N.J.; Brown, P.; Englert, C.; Hayashi, M.; Heinemann, T.; Hoymann, G.; Rossano, F.; de Ruiter, J.P.; Yoon, K.E.; et al. Universals and Cultural Variation in Turn-Taking in Conversation. Proc. Natl. Acad. Sci. USA 2009, 106, 10587–10592. [Google Scholar] [CrossRef] [Scilit]
- Heldner, M.; Edlund, J. Pauses, Gaps and Overlaps in Conversations. J. Phon. 2010, 38, 555–568. [Google Scholar] [CrossRef] [Scilit]
- Levinson, S.C.; Torreira, F. Timing in Turn-Taking and Its Implications for Processing Models of Language. Front. Psychol. 2015, 6, 731. [Google Scholar] [CrossRef] [Scilit]
- Skantze, G. Turn-Taking in Conversational Systems and Human-Robot Interaction: A Review. Comput. Speech Lang. 2021, 67, 101178. [Google Scholar] [CrossRef] [Scilit]
- Maslych, M.; Katebi, M.; Lee, C.; Hmaiti, Y.; Ghasemaghaei, A.; Pumarada, C.; Palmer, J.; Segarra Martinez, E.; Emporio, M.; Snipes, W.; et al. Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents. In Proceedings of the 7th ACM Conference on Conversational User Interfaces, Waterloo, ON, Canada, 8–10 July 2025; pp. 1–15. [Google Scholar] [CrossRef] [Scilit]
- Cassell, J.; Sullivan, J.; Prevost, S.; Churchill, E.F. (Eds.) Embodied Conversational Agents; MIT Press: Cambridge, MA, USA, 2000. [Google Scholar]
- Gratch, J.; Rickel, J.; André, E.; Cassell, J.; Petajan, E.; Badler, N.I. Creating Interactive Virtual Humans: Some Assembly Required. IEEE Intell. Syst. 2002, 17, 54–63. [Google Scholar] [CrossRef] [Scilit]
- Qwen Team. Qwen3 Technical Report. arXiv 2025, arXiv:2505.09388. [Google Scholar] [CrossRef] [Scilit]
- Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C.H.; Gonzalez, J.E.; Zhang, H.; Stoica, I. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23), Koblenz, Germany, 23–26 October 2023; pp. 611–626. [Google Scholar] [CrossRef] [Scilit]
- MathArena. AIME 2026 Dataset. Available online: https://huggingface.co/datasets/MathArena/aime_2026 (accessed on 15 October 2026).
- Woo, T.; Lee, S.; Kim, K.w.; Kim, G. Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speech. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China, 4–9 November 2025; pp. 14362–14379. [Google Scholar] [CrossRef] [Scilit]
- Liu, S.; Xu, J.; Jiang, F.; Wang, K.; Zhao, Z.; Huang, C.R.; Gu, J.; Yin, C.; Li, H. Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems. arXiv 2026, arXiv:2602.23266. [Google Scholar] [CrossRef] [Scilit]
- Yakushev, G.; Babina, N.; Vahid Dastgerdi, M.; Zhdanovskiy, V.; Kuznedelev, D.; Shutova, A.; Ryabinin, M. Asynchronous Reasoning: Training-Free Interactive Thinking LLMs. arXiv 2025, arXiv:2512.10931. [Google Scholar] [CrossRef] [Scilit]
- Xie, Z.; Ma, Z.; Liu, Z.; Pang, K.; Li, H.; Zhang, J.; Liao, Y.; Ye, D.; Miao, C.; Yan, S. Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models. arXiv 2025, arXiv:2508.15827. [Google Scholar] [CrossRef] [Scilit]
- Schlangen, D.; Skantze, G. A General, Abstract Model of Incremental Dialogue Processing. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009), Athens, Greece, 30 March–3 April 2009; pp. 710–718. [Google Scholar]
- NVIDIA. NemoCurator Prompt Task and Complexity Classifier. Model Card; Revision fea11215. Available online: https://huggingface.co/nvidia/prompt-task-and-complexity-classifier (accessed on 15 October 2026).
- Rhasspy. Piper: A Fast, Local Neural Text to Speech System. Software Repository (Archived); Voice Model en_US-amy-low. Available online: https://github.com/rhasspy/piper (accessed on 15 October 2026).
- Alibaba Cloud. CosyVoice Voice List. API Documentation; Model Cosyvoice-v2, Voice Loongbella_v2; Updated 11 October 2026. Available online: https://www.alibabacloud.com/help/en/model-studio/cosyvoice-voice-list (accessed on 27 October 2026).
- Ong, I.; Almahairi, A.; Wu, V.; Chiang, W.L.; Wu, T.; Gonzalez, J.E.; Kadous, M.; Stoica, I. RouteLLM: Learning to Route LLMs from Preference Data. In Proceedings of the International Conference on Learning Representations, Singapore, 24–28 April 2025; pp. 34433–34448. [Google Scholar]
- Rein, D.; Hou, B.L.; Stickland, A.C.; Petty, J.; Pang, R.Y.; Dirani, J.; Michael, J.; Bowman, S.R. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. In Proceedings of the Conference on Language Modeling, Philadelphia, PA, USA, 7–9 October 2024. [Google Scholar]
- Chen, Y.; Yue, X.; Zhang, C.; Gao, X.; Tan, R.T.; Li, H. VoiceBench: Benchmarking LLM-Based Voice Assistants. Trans. Assoc. Comput. Linguist. 2026, 14, 378–398. [Google Scholar] [CrossRef] [Scilit]
- Skantze, G. Towards a General, Continuous Model of Turn-Taking in Spoken Dialogue Using LSTM Recurrent Neural Networks. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, Saarbrücken, Germany, 15–17 August 2017; pp. 220–230. [Google Scholar] [CrossRef] [Scilit]
- Ekstedt, E.; Skantze, G. TurnGPT: A Transformer-Based Language Model for Predicting Turn-Taking in Spoken Dialog. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online, 16–20 November 2020; Association for Computational Linguistics: Stroudsburg, PA, USA, 2020; pp. 2981–2990. [Google Scholar] [CrossRef] [Scilit]
- Gulati, A.; Qin, J.; Chiu, C.C.; Parmar, N.; Zhang, Y.; Yu, J.; Han, W.; Wang, S.; Zhang, Z.; Wu, Y.; et al. Conformer: Convolution-Augmented Transformer for Speech Recognition. In Proceedings of the Interspeech 2020, Shanghai, China, 25–29 October 2020; pp. 5036–5040. [Google Scholar] [CrossRef] [Scilit]
- Radford, A.; Kim, J.W.; Xu, T.; Brockman, G.; McLeavey, C.; Sutskever, I. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning, Honolulu, HI, USA, 23–29 July 2023; Volume 202, pp. 28492–28518. [Google Scholar]
- Ren, Y.; Hu, C.; Tan, X.; Qin, T.; Zhao, S.; Zhao, Z.; Liu, T.Y. FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. In Proceedings of the International Conference on Learning Representations, Virtual, 3–7 May 2021. [Google Scholar]
- Kim, J.; Kong, J.; Son, J. Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech. In Proceedings of the 38th International Conference on Machine Learning, Virtual, 18–24 July 2021; Volume 139, pp. 5530–5540. [Google Scholar]
- Baumann, T.; Schlangen, D. INPRO_iSS: A Component for Just-In-Time Incremental Speech Synthesis. In Proceedings of the ACL 2012 System Demonstrations, Jeju Island, Republic of Korea, 10 July 2012; pp. 103–108. [Google Scholar]
- Arora, S.; Lu, Z.; Chiu, C.C.; Pang, R.; Watanabe, S. Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics. In Proceedings of the International Conference on Learning Representations, Singapore, 24–28 April 2025; Volume 2025, pp. 52754–52781. [Google Scholar]
- Zhang, D.; Li, S.; Zhang, X.; Zhan, J.; Wang, P.; Zhou, Y.; Qiu, X. SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 6–10 December 2023; Association for Computational Linguistics: Stroudsburg, PA, USA, 2023; pp. 15757–15773. [Google Scholar] [CrossRef] [Scilit]
- Tang, C.; Yu, W.; Sun, G.; Chen, X.; Tan, T.; Li, W.; Lu, L.; Ma, Z.; Zhang, C. SALMONN: Towards Generic Hearing Abilities for Large Language Models. In Proceedings of the International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar]
- Fang, Q.; Guo, S.; Zhou, Y.; Ma, Z.; Zhang, S.; Feng, Y. LLaMA-Omni: Seamless Speech Interaction with Large Language Models. In Proceedings of the International Conference on Learning Representations, Singapore, 24–28 April 2025; pp. 57607–57624. [Google Scholar]
- Wang, X.; Li, Y.; Fu, C.; Zhang, Y.; Shen, Y.; Xie, L.; Li, K.; Sun, X.; Ma, L. Freeze-Omni: A Smart and Low Latency Speech-to-Speech Dialogue Model with Frozen LLM. In Proceedings of the 42nd International Conference on Machine Learning, Vancouver, BC, Canada, 13–19 July 2025; Volume 267, pp. 63345–63354. [Google Scholar]
- Chiang, C.H.; Wang, X.; Li, L.; Lin, C.C.; Lin, K.; Liu, S.; Wang, Z.; Yang, Z.; Lee, H.y.; Wang, L. STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for Spoken Language Models. In Proceedings of the International Conference on Learning Representations, Rio de Janeiro, Brazil, 23–27 April 2026; Volume 2026, pp. 43512–43538. [Google Scholar]
- Chen, L.; Zaharia, M.; Zou, J. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. Trans. Mach. Learn. Res. 2024. Available online: https://openreview.net/forum?id=cSimKw5p6R (accessed on 15 October 2026).
- Aggarwal, P.; Madaan, A.; Anand, A.; Potharaju, S.P.; Mishra, S.; Zhou, P.; Gupta, A.; Rajagopal, D.; Kappaganthu, K.; Yang, Y.; et al. AutoMix: Automatically Mixing Language Models. In Proceedings of the 38th Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 10–15 December 2024. [Google Scholar]
- Ding, D.; Mallick, A.; Wang, C.; Sim, R.; Mukherjee, S.; Ruhle, V.; Lakshmanan, L.V.S.; Awadallah, A.H. Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing. In Proceedings of the International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar]
- Šakota, M.; Peyrard, M.; West, R. Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, Mérida, Mexico, 4–8 March 2024; pp. 606–615. [Google Scholar] [CrossRef] [Scilit]
- Shnitzer, T.; Ou, A.; Silva, M.; Soule, K.; Sun, Y.; Solomon, J.; Thompson, N.; Yurochkin, M. Large Language Model Routing with Benchmark Datasets. In Proceedings of the Conference on Language Modeling, Philadelphia, PA, USA, 7–9 October 2024. [Google Scholar]
- Dekoninck, J.; Baader, M.; Vechev, M. A Unified Approach to Routing and Cascading for LLMs. In Proceedings of the 42nd International Conference on Machine Learning, Vancouver, BC, Canada, 13–19 July 2025; Volume 267, pp. 12987–13010. [Google Scholar]
- Zhang, W.; Qiao, S.; Luo, L.; Li, S.; Li, Y.; Zheng, C.; Xu, Q.; Li, M.; Gui, Y.; He, Y.; et al. SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Model for Medical Question Answering. BMJ Innov. 2026, 12, 266–271. [Google Scholar] [CrossRef] [Scilit]
- Aggarwal, P.; Kim, S.; Lanchantin, J.; Welleck, S.; Weston, J.E.; Kulikov, I.; Saha, S. OptimalThinkingBench: Evaluating Over and Underthinking in LLMs. In Proceedings of the International Conference on Learning Representations, Rio de Janeiro, Brazil, 23–27 April 2026; pp. 8886–8918. [Google Scholar]
- Qwen Team. Qwen3-Embedding-0.6B. Model Card. Available online: https://huggingface.co/Qwen/Qwen3-Embedding-0.6B (accessed on 27 October 2026).
- Balunović, M.; Dekoninck, J.; Petrov, I.; Jovanović, N.; Vechev, M. MathArena: Evaluating LLMs on Uncontaminated Math Competitions. Adv. Neural Inf. Process. Syst. 2025, 38, 22851–22888. [Google Scholar] [CrossRef] [Scilit]
- Dubois, Y.; Liang, P.; Hashimoto, T. Length-Controlled AlpacaEval: A Simple Debiasing of Automatic Evaluators. In Proceedings of the Conference on Language Modeling, Philadelphia, PA, USA, 7–9 October 2024. [Google Scholar]
- Shrout, P.E.; Fleiss, J.L. Intraclass Correlations: Uses in Assessing Rater Reliability. Psychol. Bull. 1979, 86, 420–428. [Google Scholar] [CrossRef]
- Efron, B. Bootstrap Methods: Another Look at the Jackknife. Ann. Stat. 1979, 7, 1–26. [Google Scholar] [CrossRef] [Scilit]
- McNemar, Q. Note on the Sampling Error of the Difference between Correlated Proportions or Percentages. Psychometrika 1947, 12, 153–157. [Google Scholar] [CrossRef] [Scilit]
- Wilson, E.B. Probable Inference, the Law of Succession, and Statistical Inference. J. Am. Stat. Assoc. 1927, 22, 209–212. [Google Scholar] [CrossRef]
- Mathematical Association of America. 2025–2026 AIME Thresholds Are Now Available. Available online: https://maa.org/news/2025-26-aime-thresholds-are-now-available/ (accessed on 16 October 2026).
- DeepSeek-AI. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression. Technical Report, 2026. Available online: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf (accessed on 15 October 2026).
- Z.ai. GLM-5.2. Model Card, 2026. Available online: https://huggingface.co/zai-org/GLM-5.2 (accessed on 15 October 2026).
- Qwen Team. Qwen3.5: Towards Native Multimodal Agents. Blog Post. February 2026. Available online: https://qwen.ai/blog?id=qwen3.5 (accessed on 15 October 2026).
- iic. SenseVoiceSmall. Model Card; Model iic/SenseVoiceSmall. Available online: https://www.modelscope.cn/models/iic/SenseVoiceSmall (accessed on 27 October 2026).
- ITU-R. Relative Timing of Sound and Vision for Broadcasting; Recommendation ITU-R BT.1359-1; International Telecommunication Union: Geneva, Switzerland, 1998. [Google Scholar]
- OpenAI. Reasoning Models. API Documentation. Available online: https://developers.openai.com/api/docs/guides/reasoning (accessed on 21 October 2026).
- Anthropic. Thinking. API Documentation. Available online: https://platform.claude.com/docs/en/build-with-claude/thinking (accessed on 21 October 2026).
- Google. Gemini Thinking. API Documentation; Updated 14 October 2026. Available online: https://ai.google.dev/gemini-api/docs/generate-content/thinking (accessed on 21 October 2026).
- OpenAI. Responses Streaming Events. API Reference. Available online: https://developers.openai.com/api/reference/resources/responses/streaming-events (accessed on 21 October 2026).
- OpenAI. Models. API Documentation. Available online: https://developers.openai.com/api/docs/models (accessed on 21 October 2026).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.


