A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving

Abstract

Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closed-loop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving.

1. Introduction

In recent years, Large Language Models (LLMs) have achieved rapid progress in natural language understanding, text generation, and complex reasoning, driving related research from traditional single-turn question answering toward complex task-solving scenarios. In such settings, models are typically required to perform multi-step reasoning, invoke external tools, and continuously interact with the environment to accomplish a given objective. Consequently, task solving is no longer limited to generating a final answer in a single pass, but instead relies on sustained modeling and control of intermediate states, subgoal progression, and execution processes. Therefore, improving planning capability, execution capability, and process stability in multi-step tasks has become an important research problem.

Existing studies have shown that explicit intermediate reasoning can improve performance on complex tasks. Chain-of-Thought (CoT) enhances reasoning ability by eliciting stepwise inference [1], while plan-then-solve methods decompose complex problems into simpler subproblems to reduce omission and misunderstanding in one-shot generation [2]. With the development of tool-use research, language models have further acquired capabilities such as retrieval, calculation, and program invocation, extending multi-step task solving from pure text generation to the joint process of reasoning and execution [3]. Moreover, interleaving reasoning with action generation enables the model to adjust subsequent decisions according to environmental feedback, thereby improving execution quality and interpretability [4]. Unified representation and modular training strategies further enhance model transferability across heterogeneous task settings [5].

Despite these advances, current multi-step agents remain limited in process-level control. Although they can generate subgoals or action sequences, they usually lack an explicit mechanism to determine whether the current step is consistent with the task context and whether it effectively supports the current subgoal. Once local errors occur, such as improper action selection, incorrect parameter settings, or execution deviations, they are likely to be propagated into subsequent contexts and accumulate over the reasoning process, ultimately degrading final task performance. This issue is particularly pronounced in complex question answering and multi-turn tool-use scenarios.

To mitigate this problem, prior studies have introduced various feedback-based strategies for improving multi-step task solving. Self-correction methods enhance output quality through iterative refinement [6], reflective agents exploit language feedback and memory to adjust subsequent decisions across trials [7], and tool-assisted critique methods further improve reliability by examining generated outputs with external tools after execution [8]. Step-level supervision has also been shown to be beneficial for reasoning verification [9]. However, many existing studies mainly operate at the level of output refinement, post-execution critique, or round-level reflective feedback [6,7,8], while explicit assessment of step validity during the ongoing execution trajectory, together with hierarchical recovery strategies that combine local correction with re-planning, remains relatively underexplored. This limitation makes it difficult to prevent invalid intermediate steps from being incorporated into the historical context in time, thereby increasing the risk of error accumulation in subsequent task solving.

To address these limitations, this paper studies process control in multi-step task solving and proposes a closed-loop modular language agent framework, termed CL-SPF (Closed-Loop Step Verification and Planning Framework), integrating task planning, action execution, step verification, local regeneration, and replanning. The framework introduces explicit step-level validity assessment into the execution trajectory and uses the verification result to govern subsequent recovery decisions. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. In addition, a process-supervision data construction method is developed to support step validity prediction by preserving real execution steps as valid samples and automatically synthesizing invalid samples from realistic process perturbations. Through this design, the proposed method aims to reduce error propagation, improve execution stability, and strengthen recovery capability in multi-step task solving.

The main contributions of this work are summarized as follows:

• Step-level consistency verification in tool-augmented multi-step reasoning.

A verification formulation is established to evaluate the consistency among the current subgoal, action, action input, observation, and task context, enabling intermediate-step assessment before erroneous states are incorporated into the subsequent reasoning trajectory.

2.

Structured step supervision via negative sample synthesis.

A structured supervision scheme is constructed by combining real valid steps with automatically synthesized invalid steps, covering action substitution, input perturbation, observation replacement, and subgoal mismatch for fine-grained step validity learning.

3.

Hierarchical recovery driven by verification feedback.

A hierarchical recovery mechanism is introduced, where step verification acts as a process-level control signal for local regrounding and subgoal-level replanning, enabling online correction of invalid intermediate steps while reducing error propagation and improving execution stability.

2. Related Work

In LLM-driven multi-step task-solving frameworks, existing studies can generally be grouped into three directions, namely reasoning and planning, tool-based execution, and feedback-based verification. Early methods mainly treated language models as single-turn generators and focused on the correctness of final outputs. As task complexity increased, subsequent research gradually shifted toward intermediate process modeling, where the model is required to generate executable intermediate steps aligned with task objectives and complete the task through tool invocation and interaction with external environments [3,4,7]. Under this trend, multi-step task solving has gradually evolved into a general paradigm of task planning, action execution, and result feedback, with explicit reasoning, action trajectory modeling, and tool augmentation introduced to improve execution capability and robustness [10,11].

The first line of work focuses on explicit reasoning and task planning. Representative studies such as ReAct [4] interleave reasoning and action generation, allowing the model to adjust subsequent decisions according to intermediate states. Toolformer [3] enables autonomous tool invocation through self-supervised learning, thereby extending language model generation from pure text prediction to tool-assisted reasoning. Reflexion [7] further introduces language feedback and memory to reduce repeated errors, while SELF-DISCOVER [12] improves complex problem solving by automatically composing reasoning structures. These studies have strengthened the expressiveness of intermediate reasoning processes and promoted a shift from static generation to process-oriented task solving. However, their main emphasis remains on step generation itself, while explicit constraints on whether a generated step is consistent with the current task stage are still limited. As a result, step deviation and error accumulation remain common in long-horizon reasoning and multi-stage tasks [7,12].

The second line of work emphasizes tool use and execution capability. API-Bank [10] constructs executable environments and corresponding datasets for evaluating API usage ability, and ToolLLM [13] further develops a complete training and evaluation pipeline based on real-world API data. In task-specific settings, ToRA [14] combines language reasoning with symbolic computation and demonstrates the effectiveness of tool interaction in mathematical problem solving. AutoAct [15] and AgentTuning [11] improve agent capability through trajectory synthesis and parameter optimization, respectively. In addition, prior work has explored autonomous tool integration for open-domain task solving [16] and incorporated tool generation into a unified framework [17], further expanding the model action space. Overall, these methods significantly enhance execution capability. Nevertheless, their effectiveness is still highly dependent on action selection and tool invocation quality. Once an intermediate action deviates from the current subgoal, the resulting error is easily propagated to subsequent steps and may ultimately reduce overall task performance [10,13,17].

The third line of work focuses on feedback-based verification and process correction. Existing agent systems such as ReAct, Reflexion and CRITIC have demonstrated that feedback-based correction can improve multi-step task solving. Therefore, the contribution of the present work does not lie in introducing the general idea of closed-loop verification or correction itself. Instead, the focus here is on a more specific formulation for tool-augmented multi-step reasoning, where verification is performed at the level of subgoal–action–input–observation consistency before an erroneous step is absorbed into the subsequent execution trajectory. CRITIC [8] improves model outputs through tool-assisted critique, and Let’s Verify Step by Step [9] demonstrates the value of step-level supervision for reasoning verification. Related studies have also shown that language models remain limited in self-diagnosis and often require external verifiers to support reliable correction [18,19]. More recent work has shifted attention from whether to verify to how to verify efficiently. For example, Derailer-Rerailer [20] introduces adaptive triggering to balance verification accuracy and computational cost, VerifiAgent [21] proposes a unified verification framework for diverse task types, and SCoRe [22] explores reinforcement learning to enhance self-correction ability. Meanwhile, benchmarks such as Mind2Web [23], WebArena [24], AppWorld [25], and OSWorld [26] provide evaluation platforms closer to real-world environments, and AgentBoard [27] further emphasizes fine-grained evaluation of process-level performance. Nevertheless, most existing feedback and verification methods remain focused on result-level correction [8,22], turn-level reflection [7], or auxiliary verification stages [19,20,21], with limited integration of step-level signals into the current action generation process, thereby constraining their ability to suppress local errors and perform online correction. Compared with these approaches, the present framework emphasizes step-level consistency verification for executable tool steps rather than only trajectory-level reflection, post-execution critique, or offline reasoning-step scoring. In particular, the verifier evaluates whether the current subgoal, action, action input, and returned observation remain consistent under the task context, and its output is directly used to trigger local regrounding or subgoal-level replanning during execution. To clarify the relationship between CL-SPF and existing feedback-based or modular agent frameworks, Table 1 summarizes their differences in feedback level, verification target, and recovery mechanism. The proposed framework does not claim the general idea of feedback-driven correction as entirely new. Instead, its main distinction lies in introducing step-level executable validity verification into the ongoing agent trajectory and using the verification result as an online control signal for local regrounding and subgoal-level replanning.

Table 1. Comparison with existing feedback-based and modular agent frameworks.

Method	Verification Target	Step-Level Online Interception	Recovery	Main Distinction
ReAct	Environment feedback on action-observation process	Partial	None	Interleaves reasoning and action but lacks explicit step-validity verification
Reflexion	Trial/episode-level failure reflection	No	Reflection memory	Uses reflection memory rather than online step-level correction
CRITIC	Output-level verification of generated outputs	No	Tool critique	Critiques outputs but does not regulate each executable step
Agent Lumos	Modular plan-ground modeling of subgoal/action	No	None	Provides Plan-Ground modules but lacks verification-guided correction
Let’s Verify Step by Step	Reasoning-step verification of reasoning trace correctness	No	None	Scores reasoning traces rather than executable tool steps
CL-SPF	Step-level process control over subgoal-action-input-observation consistency	Yes	Reground + replan	Uses verification as an online control signal

This comparison shows that CL-SPF is not intended to replace existing feedback-based paradigms, but to extend modular language agents with step-level executable verification and hierarchical online recovery.

Despite significant progress in reasoning modeling, tool usage, and feedback mechanisms, fine-grained control in multi-step task execution remains underexplored. First, existing methods generally lack explicit mechanisms to determine whether the current step effectively supports the current subgoal, making intermediate processes observable but weakly constrained [4,7,9]. This distinction is particularly important in tool-augmented settings, where an incorrect action, a mismatched input, or an inconsistent observation may enter the history state and propagate to later decisions if not intercepted in time. Second, most feedback mechanisms operate at the result or turn level, making it difficult to intervene in real time during step generation, thereby limiting local correction capability [8,18,19]. Third, in complex task scenarios, local deviations tend to propagate along the task trajectory, leading to accumulated errors that adversely affect subsequent planning and execution [11,20,21]. Therefore, constructing a tightly coupled closed-loop control mechanism across task planning, action execution, and process verification to enable step-level validity assessment and online correction remains a key challenge in multi-step task solving. This design differs from process-supervision methods such as Let’s Verify Step by Step, which focus on reasoning traces; here, verification targets executable tool steps under the current subgoal and context, and serves as an online control signal in the agent loop rather than an offline reward or score.

To address the limitations of existing methods in step validity modeling and local error control, this paper presents a closed-loop framework integrating task decomposition, action execution, and step verification. Unlike previous studies that mainly focus on intermediate step generation or result-level correction, the proposed framework embeds step validity assessment into the execution process, allowing it to influence current action selection and subsequent process progression. This design supports dynamic constraints of intermediate steps and timely local correction, thereby improving process consistency and execution stability in complex task solving.

3. Proposed Method

3.1. Problem Definition and Overall Framework

3.1.1. Problem Definition and Framework Structure

Multi-step task solving in complex environments is essentially a sequential and state-dependent process. Rather than generating the final answer in a single inference pass, the system is required to progressively determine the current subgoal, map it to executable actions, interpret the resulting observations, and further decide whether the current step should be incorporated into the task trajectory. Without explicit control over intermediate steps, local deviations may enter the historical context and continue to propagate across subsequent stages, thereby degrading overall task performance.

To address this issue, this study formulates multi-step task solving as a closed-loop process conditioned on historical states. The architectural presentation of the proposed framework is organized with reference to the software-architecture-description approach proposed by Górski [28], in which system functionality, logical structure, and runtime behavior are considered in a coordinated manner. As shown in Figure 1, under this formulation, the framework is no longer treated as a simple serial composition of planning and execution, but rather as a coordinated architecture in which task planning, action execution, step verification, and recovery jointly regulate trajectory evolution.

Figure 1. Closed-loop multi-step task-solving framework with step verification and error recovery.

Let the input task be denoted by $x$, and let the valid history before step $t$ be denoted by $h_t$. The accepted trajectory before the current step is defined as

$$h_t=\left\{\right(g_i,a_i,o_i\left)\right|v_i=\mathrm{1,1}\le i<t\}$$

(1)

where $g_i$, $a_i$, and $o_i$ denote the subgoal, action, and observation at step $i$, respectively.

Based on the current task input and valid history, the single-step state transition is written as

$$g_t=P_{\theta }\left(x,h_t\right),a_t=G_{\varphi }\left(g_t,h_t\right),o_t=E\left(a_t\right),v_t=V_{\psi }\left(x,g_t,a_t,o_t,h_t\right)$$

(2)

where $P_{\theta }$ denotes the task planning model, $G_{\varphi }$ denotes the action execution model, $V_{\psi }$ denotes the step verification model, and $E(\cdot )$ denotes the action executor or environment interface. The parameters $\theta$, $\varphi$, and $\psi$ correspond to the three modules, respectively.

If the current step is accepted, the valid history is updated as

$$h_{t+1}=\left\{\begin{array}{cc}{h}_t\cup \left\{\right(g_t,a_t,o_t\left)\right\},& v_t=1\\ h_t,& v_t=0\end{array}\right.$$

(3)

Thus, subsequent planning is conditioned on verified intermediate states rather than on raw model outputs.

3.1.2. Hierarchical Recovery Mechanism and Trajectory Modeling

A key characteristic of the proposed framework lies in its hierarchical recovery mechanism. When the current step is judged invalid, the system first retains the current subgoal and performs local correction at the execution level. Specifically, the next candidate action is regenerated under the same subgoal constraint with invalidity feedback:

$$a_t^{\left(k+1\right)}=G_{\varphi }(g_t,h_t,f_t)$$

(4)

where $a_t^{\left(k+1\right)}$ denotes the $\left(k + 1\right)$-th regenerated action and $f_t$ denotes the feedback signal returned by the step verification module.

If repeated local correction still fails to restore valid progress, the framework further escalates to the planning layer and triggers replanning according to the current valid history and failure feedback:

$$g_t^{\prime }=P_{\theta }(x,h_t,f_t)$$

(5)

where $g_t^{\prime }$ denotes the replanned subgoal. In this way, execution-level deviations and planning-level deviations are handled at different levels of control.

The inference process terminates when the planning module outputs a stop signal or when the task completion condition is satisfied. The final answer is then generated from the accepted trajectory: $y=F(x,h_T)$, where $F(\cdot )$ denotes the final answer generation function and $h_T$ denotes the final valid trajectory state.

For completeness, the accepted multi-step trajectory can be written as

$$\tau =\left\{\right(g_t,a_t,o_t,v_t){\}}_{t=1}^T$$

(6)

This trajectory formulation highlights that the proposed framework is a closed-loop architecture for controlled multi-step task solving, in which trajectory evolution is regulated by planning, execution, verification, and recovery jointly.

3.2. Task Planning Model

The task planning module is responsible for generating the current subgoal conditioned on the task input and the accumulated valid history. Rather than producing a complete solution in a single pass, the framework organizes task solving as a sequence of stage-wise decisions. This reduces the reasoning span at each step and provides a clearer interface for subsequent execution and verification.

The current subgoal is generated as

$$g_t=P_{\theta }(x,h_t)$$

(7)

where $P_{\theta }$ denotes the planning model.

The generated subgoal is not the final answer, but an intermediate stage objective. It should therefore satisfy two requirements. First, it should exhibit local goal orientation, meaning that it focuses on the current stage rather than on the entire solving process. Second, it should be executable, meaning that it can be grounded into explicit actions by the downstream execution module. Accordingly, the planning problem can be written as selecting the most suitable subgoal from the candidate space $\mathcal{G}$:

$$g_t^{*}=\mathrm{a}\mathrm{r}\mathrm{g} \underset{g\in \mathcal{G}}{\mathrm{m}\mathrm{a}\mathrm{x}}{P}_{\theta }(g\mid x,h_t)$$

(8)

Since multi-step task solving is not unbounded, the stop decision is incorporated into the same planning framework by extending the output space as ${\mathcal{G}}^{\prime }=\mathcal{G}\cup \left\{\mathrm{STOP}\right\}$. This allows the planning module to determine not only the next subgoal, but also whether the task no longer requires further decomposition.

During training, given a reference subgoal sequence, the planning model is optimized using the supervised objective

$${\mathcal{L}}_{\mathrm{plan}}=-{\displaystyle \sum _{t=1}^T}\mathrm{l}\mathrm{o}\mathrm{g} P_{\theta }(g_t^{*}\mid x,h_t)$$

(9)

Under this objective, the model learns to generate appropriate stage-wise subgoals under dynamic historical constraints. The key point is not to re-derive standard sequence generation, but to emphasize that planning is history-aware, iterative, and coupled with verified execution feedback.

3.3. Action Execution Model

After the current subgoal is obtained, the framework further determines how that subgoal should be completed. The action execution module maps a high-level stage objective to a structured executable action, thereby connecting task planning with environment interaction. Compared with direct natural-language generation, this structured grounding process makes the intermediate execution path more explicit and more suitable for tool use and interactive decision making.

Given the current subgoal $g_t$ and valid history $h_t$, the execution model generates the corresponding action as

$$a_t=G_{\varphi }(g_t,h_t)$$

(10)

where $G_{\varphi }$ denotes the action execution model.

For modeling convenience, each action is represented as a combination of an action type and its associated parameters:

$$a_t=(c_t,u_t)$$

(11)

where $c_t$ denotes the action type and $u_t$ denotes the corresponding parameter information. This structured representation is retained because it directly supports both the verification stage and the construction of invalid samples. In particular, it allows the framework to distinguish between different sources of failure, such as inappropriate action selection and incorrect parameter generation.

The generated action is then executed through the external environment or tool interface:

$$o_t=E(a_t,s_{t-1})$$

(12)

where $o_t$ denotes the resulting observation and $s_{t-1}$ denotes the environment state before execution.

The resulting observation provides the basis for subsequent verification. Therefore, the execution module is not treated as an isolated generator. Its outputs remain subject to process-level validation before they are allowed to affect later planning and execution decisions.

During training, the execution model is optimized using subgoal–action supervision constructed from transformed trajectories:

$${\mathcal{L}}_{\mathrm{exec}}=-{\displaystyle \sum _{t=1}^T}\mathrm{l}\mathrm{o}\mathrm{g} P_{\varphi }(a_t^{*}\mid g_t,h_t)$$

(13)

where $a_t^{*}$ denotes the reference action associated with subgoal $g_t$. This objective establishes a stable mapping from stage-wise task intentions to executable behaviors without introducing unnecessary probabilistic expansion.

3.4. Step Verification Model

Although the planning and execution modules can generate intermediate steps, they do not guarantee that each step effectively supports the current subgoal. If invalid steps are directly incorporated into the historical state, the system may continue along an erroneous trajectory, thereby amplifying local deviations. To address this issue, a step verification model is introduced to explicitly assess the validity of each step.

At step $t$, the step state is jointly determined by the task input, current subgoal, executed action, observation, and valid history:

$$z_t=(x,g_t,a_t,o_t,h_t)$$

(14)

Based on this representation, the step verification model predicts whether the current step is valid:

$$v_t=V_{\psi }\left(z_t\right),v_t\in \left\{\mathrm{0,1}\right\}$$

(15)

where $v_t=1$ indicates a valid step and $v_t=0$ indicates an invalid step. This representation is designed to capture step-level consistency in tool-augmented reasoning. Rather than evaluating a reasoning trace only after completion, the verifier examines whether the current executable step remains compatible with the local subgoal and the historical task state before the step is incorporated into the subsequent trajectory.

Unlike final-answer evaluation, step verification focuses on whether the current step is reasonable at the current stage, including whether the action is consistent with the subgoal, whether the action parameters are appropriate, and whether the resulting observation indicates effective progress.

To construct supervision signals, valid steps from real trajectories are retained as positive samples:

$$z_t^{+}=(x,g_t,a_t,o_t,h_t),y_t^{+}=1$$

(16)

Invalid samples are automatically synthesized by perturbing the subgoal, action, or observation:

$$z_t^{-}=(x,{\tilde{g}}_t,{\tilde{a}}_t,{\tilde{o}}_t,h_t),y_t^{-}=0$$

(17)

where ${\tilde{g}}_t$, ${\tilde{a}}_t$, and ${\tilde{o}}_t$ denote perturbed subgoals, actions, and observations, respectively. The design of these negative samples follows the internal structure of a tool-augmented reasoning step. A valid step requires consistency among the planned subgoal, the selected action, the action input, and the returned observation. Violating each of these components naturally gives rise to one major type of invalid step, namely action substitution, input perturbation, observation replacement, and subgoal mismatch. These four categories are not claimed to be theoretically exhaustive, but they provide practical coverage of major consistency failures in step-level tool execution.

The verification model is trained using binary supervision:

$${\mathcal{L}}_{\mathrm{ver}}=-{\displaystyle \sum _{i=1}^N}\left[y_i\mathrm{l}\mathrm{o}\mathrm{g} p_i+(1-y_i)\mathrm{l}\mathrm{o}\mathrm{g}(1-p_i)\right]$$

(18)

where $p_i$ denotes the predicted probability that sample $z_i$ is valid.

In the proposed framework, step verification is not treated as a standalone filtering module, but as a process-level control signal for trajectory regulation. Rather than merely assigning an isolated validity label, it determines whether the current action–observation pair is trustworthy under the current subgoal and context, and whether execution should proceed, undergo local regrounding, or be escalated to replanning.

3.5. Verification-Guided Local Correction Mechanism

If step verification is used only for offline assessment, its outputs cannot directly influence the online solving trajectory. To make verification effective during execution, the proposed framework incorporates step-level judgments into the control loop and introduces a verification-guided hierarchical recovery mechanism. The purpose of this mechanism is to transform validity judgments into actionable recovery decisions, so that different forms of deviation can be handled at appropriate levels of the framework.

Recovery is organized into two levels. When the current step is judged invalid, the framework first performs local correction by preserving the current subgoal and regenerating the action under the same subgoal constraint. If repeated local correction still fails, the failure is treated as evidence that the deviation may originate from the planning level rather than from action grounding alone. In that case, the framework escalates to replanning and revises the current subgoal. This layered design allows execution-level errors and planning-level mismatches to be handled differently within a unified closed-loop process.

3.5.1. Local Action Regeneration Under the Current Subgoal

When the current step is judged invalid, the framework does not immediately discard the current subgoal. Instead, it retains $g_t$ and regenerates the action using the invalidity feedback returned by the verifier:

$$a_t^{\left(k+1\right)}=G_{\varphi }(g_t,h_t,f_t)$$

(19)

where $a_t^{\left(k+1\right)}$ denotes the $\left(k + 1\right)$-th regenerated action and $f_t$ denotes the feedback signal produced by the step verification module.

This choice is motivated by the observation that an invalid step does not necessarily imply that the current subgoal is inappropriate. In many cases, the error arises from an unsuitable grounding of the subgoal into executable action. It is therefore more reasonable to first correct the action while preserving the current stage objective, rather than modifying the planning state prematurely.

If a regenerated step is subsequently verified as valid, it is accepted and incorporated into the valid history:

$$h_{t+1}=h_t\cup \left\{\left(g_t,a_t,o_t\right)\right\}, \mathrm{if} v_t=1$$

(20)

Otherwise, the invalid step is rejected and the valid history remains unchanged:

$$h_{t+1}=h_t, \mathrm{if} v_t=0$$

(21)

As a result, only verified steps are allowed to update the trajectory state. This prevents invalid intermediate outputs from being propagated into subsequent planning and execution stages.

Let $q_t$ denote the probability that a single attempt under the current subgoal both succeeds and passes verification. If at most $K$ local retries are allowed, the probability that the current subgoal is eventually completed through local correction is given by

$$P_{\mathrm{succ}}^{\left(t\right)}=1-{(1-q_t)}^K$$

(22)

This expression indicates that local regeneration can improve the completion probability of the current stage without requiring an immediate revision of the task decomposition.

3.5.2. Replanning After Repeated Local Failure

Local regeneration is not always sufficient. If repeated attempts under the current subgoal still fail to produce a valid step, the deviation is no longer treated as a purely execution-level problem. Instead, the framework regards the failure as a signal that the current subgoal itself may be inappropriate, or that the current local progression path is no longer suitable for advancing the overall task. The recovery process is then escalated to the planning layer through replanning:

$$g_t^{\prime }=P_{\theta }(x,h_t,f_t)$$

(23)

where $g_t^{\prime }$ denotes the replanned subgoal and $f_t$ serves as high-level failure feedback accumulated from unsuccessful local correction.

Compared with the original planning step, replanning is performed under the same task input $x$ and current valid history $h_t$, but additionally incorporates failure information. This allows the planner to revise the current stage objective when continued repair under the original subgoal is no longer productive.

After replanning, the execution process resumes under the updated subgoal:

$$a_t^{\prime }=G_{\varphi }(g_t^{\prime },h_t)$$

(24)

Through this mechanism, the framework is able to recover not only from action-level grounding errors, but also from subgoal-level deviations that cannot be resolved through local correction alone.

The two recovery modes can be summarized as

$$\mathrm{Recovery}(t)=\left\{\begin{array}{cc}\mathrm{Reground},& \mathrm{if} v_t=0\wedge k<K\\ \mathrm{Replan},& \mathrm{if} v_t=0\wedge k\ge K\end{array}\right.$$

(25)

where $k$ denotes the current retry count under subgoal $g_t$.

This formulation captures the central principle of the recovery strategy: the framework prioritizes low-cost local correction and resorts to higher-level replanning only when local regeneration is no longer sufficient. Such a strategy improves robustness while limiting unnecessary disruption to the overall solving trajectory.

3.5.3. Effect on Error Propagation

To improve the stability of multi-step execution, CL-SPF introduces a verification-guided recovery mechanism that intercepts invalid states before they are written into the valid trajectory. After each action–observation pair is produced, the step verifier evaluates whether the step is consistent with the current subgoal, action input, returned observation, and preceding valid history. A valid step is appended to the execution history and used for subsequent decision-making. In contrast, an invalid step is excluded from the valid history and used only as a feedback signal for recovery. The system first attempts local action regeneration under the current subgoal. If local repair fails repeatedly, the failure is escalated to the planning layer for subgoal-level replanning. This design prevents erroneous intermediate states from being directly accumulated in the agent history and provides a process-level control mechanism for reducing error propagation during long-horizon task execution.

This formulation is intentionally kept compact. It is not intended as a full probabilistic analysis of every recovery case, but rather as a concise characterization of the main intuition: if invalid steps can be filtered before entering the valid history, and if both local regeneration and replanning are available as recovery paths, then the likelihood of sustained error propagation along the trajectory is reduced.

Overall, the contribution of this section lies not in re-deriving standard conditional generation formulas, but in clarifying the hierarchical recovery architecture of the proposed framework. Within this architecture, step verification functions as an online process-level control signal, local regeneration handles execution-level deviations, and replanning addresses higher-level subgoal mismatch when local correction is no longer sufficient.

4. Experiments

4.1. Experimental Setup

4.1.1. Data Sources and Experimental Tasks

To systematically evaluate the effectiveness of the proposed closed-loop language agent framework in complex multi-step tasks, experiments are conducted on two representative task scenarios: mathematical reasoning and web interaction. These two task types correspond to two typical agent application paradigms, namely static structured problem solving and dynamic interactive decision making. Mathematical reasoning tasks emphasize the consistency and verifiability of intermediate reasoning steps, and can therefore reflect the stability and logical coherence of the model in multi-step reasoning processes. In contrast, web interaction tasks emphasize sequential decision making over a candidate action space and can be used to evaluate the execution reliability and generalization ability of the proposed method in realistic environments. By conducting unified evaluation on both task types, the effectiveness and applicability of the proposed method for multi-step task solving can be analyzed from different perspectives.

For dataset construction, the publicly available mathematical reasoning datasets GSM [29], ASDiv [30], and PRM800K [9] are selected as the data sources for the mathematical reasoning domain, while the publicly available web interaction dataset Mind2Web [23] is selected as the data source for the web interaction domain. For Mind2Web, the Test-Domain split is adopted following the original benchmark setting. This split is specified because the three official Mind2Web test splits differ in their requirements for task-, website-, and domain-level generalization. Step Success Rate is used as the primary evaluation metric, consistent with the action-level evaluation protocol commonly adopted in Mind2Web-based agent studies. Since these original datasets differ substantially in annotation format and organizational structure, and their raw formats are typically designed either for final-answer prediction or for recovering original action trajectories, they cannot directly satisfy the unified modeling requirements of the proposed task decomposition–action execution–step verification closed-loop framework for planning data, execution data, and verification data. Therefore, the original samples must first be transformed into a unified representation.

Specifically, following the modular data construction idea of Lumos, the original samples from different task domains are first converted into an intermediate representation in the form of task + original process + transformed subgoal trajectory. Based on this intermediate representation, data for task planning and task execution are further constructed for training the planning module (Plan) and the execution module (Ground), respectively. For step verification data, the intermediate representation is further processed at the level of real execution steps, where the task description, current subgoal, action, action input, and execution result are retained and labeled as valid samples. Furthermore, based on the real steps, invalid steps that are inconsistent with the current context are automatically constructed through action substitution, input perturbation, observation replacement, and subgoal mismatch, and are uniformly labeled as invalid. In this way, binary supervised data for step validity prediction are constructed, providing fine-grained process-level supervision for the step verification module (Verify). Table 2 summarizes the construction of the step verification data and model, and Table 3 reports the statistics of the resulting verification datasets.

Table 2. Overview of Step Verification Data and Model Construction.

Item	Description
Module name	Step verification module
Module objective	To determine whether the current execution step is consistent with the task objective, the current subgoal, and the contextual state
Input unit	Task description + current subgoal + action + action input + execution result/ observation
Output form	Binary label: valid/invalid
Source of positive samples	Process-level samples converted from real execution steps in the original data
Information retained in positive samples	Task description, current subgoal, action, action input, and execution result
Negative sample construction methods	Action substitution, input perturbation, observation replacement, and subgoal mismatch
Objective of negative sample construction	To generate invalid steps that are inconsistent with the current context
Supervision granularity	Process-level and step-level supervision
Training task formulation	Binary classification task
Task domains of data sources	Mathematical reasoning and web interaction

Table 3. Statistics of the Step Verification Datasets.

Task Domain	Data Variant	Train/Val/Test	Description
Mathematical reasoning	Full	280,772/35,095/35,099	Math step verification dataset
	No Negative	21,176/2647/2647	Valid steps only
	Full Negative	280,772/35,095/35,099	Full negative-sample setting
	Single Action	86,075/10,759/10,760	Action substitution only
	Single Input	86,075/10,759/10,760	Input perturbation only
	Single Observation	86,075/10,759/10,760	Observation replacement only
	Single Subgoal	86,075/10,759/10,760	Subgoal mismatch only
	w/oAction	215,873/26,983/26,986	Without action substitution
	w/oInput	215,873/26,983/26,986	Without input perturbation
	w/oObservation	215,873/26,983/26,986	Without observation replacement
	w/oSubgoal	215,873/26,983/26,986	Without subgoal mismatch
Web interaction	Full	29,205/3650/3655	Web process-level verification data

The invalid samples are generated by perturbing a single component of a valid executable step while keeping the other components unchanged. Specifically, for action substitution, the original action type is replaced with the action type from another step in the same trajectory, whereas the current subgoal, action input, and observation are retained. For input perturbation, the action input is replaced with that of a donor step while preserving the current subgoal, action type, and observation. For observation replacement, the returned observation is substituted with the observation from a donor step, with the current subgoal, action type, and action input unchanged. For subgoal mismatch, the original action–observation pair is preserved, but the current subgoal is replaced by the subgoal of another step, thereby creating an inconsistency between the intended objective and the executed behavior. In the default Full setting adopted in the data-construction scripts, one donor step is selected for each valid step, and the four types of perturbations are instantiated based on this donor step, leading to balanced proportions among the negative-sample categories. In the current implementation, the donor step is uniformly sampled from another executable step within the same trajectory, excluding the current step, and the substitution is not constrained to the same action or tool type. As a result, action-level perturbations may involve different action or tool types when such alternatives are available. This design is intended to increase the diversity of invalid samples and to encourage the verifier to model contextual consistency among the subgoal, action, input, and observation, rather than relying only on superficial action-pattern matching.

Different data variants in the mathematical reasoning task are used to analyze the impact of various invalid-step construction strategies on the performance of the step verification model, while the web interaction task is used to evaluate the generalization ability of the module in dynamic interactive decision-making scenarios.

4.1.2. Baseline Methods

To comprehensively evaluate the effectiveness of the proposed closed-loop framework, four categories of baseline methods are considered.

• Integrated Training. In this setting, task solving is modeled as a single end-to-end generation process. Given an input task, the model directly produces the final output without explicitly representing intermediate reasoning steps or execution procedures. This method serves as a basic baseline for evaluating task-solving performance in the absence of process-level modeling.
• CoT Training [1]. This method follows the chain-of-thought training paradigm, where the model is trained to generate intermediate reasoning in natural language before arriving at the final answer. Compared with direct generation, such a setting can improve performance on complex tasks by making the reasoning process more explicit. However, the intermediate process is still expressed only in free-form natural language, rather than in structured and executable representations such as subgoals or actions.
• ReFT [31]. This method adopts a reinforcement fine-tuning strategy designed to improve reasoning performance. The model is first fine-tuned on chain-of-thought data and is then further optimized through online reinforcement learning. For each problem, multiple candidate reasoning trajectories are sampled, and reward signals are assigned according to answer correctness. In comparison with standard CoT training, this method is better able to exploit diverse reasoning trajectories for the same task, which helps improve generalization in reasoning.
• Agent Lumos [5]. This method employs a modular framework for task solving. It first decomposes the input task into a sequence of structured subgoals and then grounds each subgoal into a corresponding action. In this way, both task structure and execution flow are explicitly modeled. Owing to its modular design, this method provides a more controllable and interpretable problem-solving process, while also introducing structured intermediate representations for complex tasks.

The baseline methods were chosen with two goals in mind: covering representative paradigms and supporting progressive comparison. On the one hand, they include several typical settings for complex task solving, ranging from end-to-end generation and natural language reasoning to reinforcement-based reasoning optimization and modular decomposition with action grounding. On the other hand, these methods reflect a gradual increase in process modeling, which makes it easier to examine how intermediate reasoning and structured execution influence overall performance. Hence, this baseline setting provides a clear and interpretable basis for evaluating the proposed method.

4.1.3. Evaluation Metrics

The proposed system is evaluated from three levels, namely the task level, step level, and process level.

At the task level, Accuracy is adopted as the primary metric for mathematical reasoning tasks to measure whether the final answer is correct. For web interaction tasks, Step Success Rate or action matching accuracy is used as the primary metric to measure whether the predicted action and target element are consistent with the reference trajectory.

At the step level, the step verification task is formulated as a binary classification problem, i.e., determining whether the current step is valid or invalid. Accordingly, Accuracy, Precision, Recall, F1, and Macro-F1 are adopted as the main evaluation metrics. Among them, Precision, Recall, and F1 are mainly used to assess the model’s ability to identify invalid steps, while Macro-F1 is used to evaluate the overall classification performance under class imbalance.

At the process level, to further analyze the effect of the closed-loop mechanism on system behavior, several process-oriented metrics are introduced, including Avg Steps, Avg Action Attempts, Invalid Verdict Rate, Reground Attempts, and Reground Success Rate.

Let the test set contain $N$ tasks. For the $i$-th task, let $S_i$ denote the number of actual execution steps, $A_i$ denote the total number of action attempts, $V_i$ denote the number of steps judged as invalid, and $R_i$ denote the number of times local regeneration is triggered, among which $R_i^{\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{c}}$ denotes the number of successful corrections after regeneration. The above metrics are defined as follows:

$$\mathrm{A}\mathrm{v}\mathrm{g} \mathrm{S}\mathrm{t}\mathrm{e}\mathrm{p}\mathrm{s}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{S}_i$$

(26)

$$\mathrm{A}\mathrm{v}\mathrm{g} \mathrm{A}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{t}\mathrm{s}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{A}_{i }$$

(27)

$$\mathrm{I}\mathrm{n}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{d} \mathrm{V}\mathrm{e}\mathrm{r}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t} \mathrm{R}\mathrm{a}\mathrm{t}\mathrm{e}={\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N}{V}_i}{{\displaystyle {\sum }_{i=1}^N}{S}_i}}$$

(28)

$$\mathrm{R}\mathrm{e}\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d} \mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}\mathrm{t}\mathrm{s}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{R}_i$$

(29)

$$\mathrm{R}\mathrm{e}\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d} \mathrm{S}\mathrm{u}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s} \mathrm{R}\mathrm{a}\mathrm{t}\mathrm{e}={\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N}{R}_i^{\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{c}}}{{\displaystyle {\sum }_{i=1}^N}{R}_i}}$$

(30)

where Avg Steps and Avg Action Attempts denote the average numbers of execution steps and action attempts, respectively. Invalid Verdict Rate is defined as the proportion of steps judged invalid during execution. Reground Attempts denotes the average number of local action regenerations triggered per task, and Reground Success Rate measures the success rate of execution recovery after regeneration is triggered. These metrics jointly evaluate final task performance, step-level discrimination ability, and process control capability.

4.1.4. Experimental Details

To ensure reproducibility and avoid ambiguity in model identification, all modules were initialized from the publicly available Hugging Face checkpoint (Hugging Face, Inc., New York, NY, USA) Qwen/Qwen3.5-4B (released by the Qwen Team, Alibaba Cloud, Hangzhou, China), which is denoted as Qwen3.5-4B in the following sections for brevity. This checkpoint corresponds to the official post-trained 4B model released by the Qwen team on Hugging Face. The same initialization checkpoint was used for all compared methods and experimental settings to ensure a fair comparison. The task planning and action execution modules were optimized through full-parameter supervised fine-tuning, whereas the step verification module was trained using LoRA-based parameter-efficient fine-tuning, with the rank, alpha, and dropout set to 16, 32, and 0.1, respectively. All experiments were conducted under the same data splits, optimization settings, and hardware environment to maintain comparability across methods. To improve the robustness of the reported results, the main task-level experiments were repeated using three different random seeds. For the primary performance metrics, the average results and standard deviations are reported as mean ± std.

The planning module and the action grounding module followed the same training protocol, including the same number of training epochs, batch-size settings, and input-length constraints, so as to ensure consistency in training cost and optimization conditions. The step verification module was trained with an independent configuration. Its maximum input length was set to 512, the maximum number of training steps was set to 5000, and gradient checkpointing was enabled to reduce GPU memory consumption. To improve training efficiency and reduce parameter update cost, the step verification module adopted LoRA-based parameter-efficient fine-tuning, with the LoRA rank set to 16, alpha set to 32, and dropout set to 0.1. Unless otherwise specified, the planning and action grounding modules were trained with full-parameter supervised fine-tuning.

Unless otherwise noted, all methods were conducted under the same data splits, the same training budget, and consistent experimental conditions, such that performance differences can be attributed, as much as possible, to module design and process modeling strategies rather than training configuration differences. The main hyperparameter settings are summarized in Table 4.

Table 4. Main Experimental Hyperparameters.

No.	Parameter	Value
1	Maximum input length for training	1024
2	Number of training epochs	2
3	Maximum generated tokens per step for the planning module	128
4	Maximum generated tokens per step for the action grounding module	128
5	Maximum number of planning/execution iterations	16
6	Maximum retry number after regrounding, $R$	2
7	Scaling factor, $\alpha$	0.625

4.2. Experimental Results

4.2.1. Experiments on Mathematical Reasoning Tasks

To evaluate the effectiveness of the proposed method in structured reasoning scenarios, four baseline methods and the proposed CL-SPF were compared on mathematical reasoning tasks. The experimental results are presented in Figure 2.

Figure 2. Main experimental results on mathematical reasoning tasks. The figure compares Accuracy, Task Completion, and Step Success across different methods. Accuracy values are shown above the bars, and error bars indicate the standard deviation over independent runs.

As shown in Figure 2, the overall performance of different methods on the GSM task exhibited clear differences. Integrated Training achieved a task accuracy of only 0.317, whereas CoT Training improved this value to 0.460, indicating that the introduction of explicit intermediate reasoning is beneficial for mathematical problem solving. ReFT further increased task accuracy to 0.568, suggesting that process-oriented reinforcement fine-tuning can further improve the quality of intermediate reasoning steps and thus enhance both solution stability and final answer accuracy.

Agent Lumos, which adopts task decomposition, further improved the task success rate to 0.617. This result indicates that decomposing complex problems into subgoals and executing them in a stepwise manner can effectively strengthen task-solving performance. On this basis, CL-SPF further increased the task success rate to 0.650, demonstrating the positive contribution of the step verification and recovery mechanism to final task performance. Meanwhile, Agent Lumos and CL-SPF both achieved a task completion rate of 0.975, and their step success rates were both close to 0.992. This suggests that the main difference between the methods does not lie in whether the reasoning process can be completed, but rather in whether the completed process leads to the correct answer. Therefore, the performance gain of CL-SPF is mainly attributable to its ability to identify and correct erroneous intermediate steps, rather than to a simple improvement in process completion.

4.2.2. Experiments on Web Interaction Tasks

To further evaluate the applicability of the proposed method in interactive decision-making environments, experiments were conducted on a web interaction dataset. Compared with mathematical reasoning tasks, web interaction tasks place greater emphasis on multi-step decision making over candidate interface elements and actions. Therefore, they are more suitable for evaluating the generalization ability of the proposed method in dynamic interactive environments. The experimental results are presented in Table 5.

Table 5. Main experimental results on web interaction tasks.

Method	Step Success Rate	Avg Action Attempts	Invalid Verdict Rate	Reground Success Rate	Replan Success Rate
Integrated Training	0.246 ± 0.007	-	-	-
Agent Lumos	0.310 ± 0.005	5.8	-	-	-
Agent Lumos + fixed retries (R = 6)	0.315 ± 0.006	6	-	-	-
Agent Lumos + fixed retries (R = 7)	0.324 ± 0.009	7	-	-	-
CL-SPF	0.424 ± 0.007	6.6	0.444	0.857	0.533

Table 5 presents the results on web interaction tasks. Integrated Training achieves a Step Success Rate of 0.246, suggesting that end-to-end generation provides only limited capability for maintaining stable decisions across multi-step interaction trajectories. Agent Lumos improves the Step Success Rate to 0.310 with an Avg Action Attempts of 5.8, indicating that structured subgoal decomposition and action grounding are beneficial. However, because its execution remains largely forward-driven, invalid intermediate steps are not explicitly detected or repaired.

To determine whether CL-SPF benefits mainly from additional execution attempts, we include two fixed-retry variants of Agent Lumos. Increasing the retry budget to 6 and 7 attempts only raises the Step Success Rate to 0.315 and 0.324, respectively. These marginal gains suggest that repeated execution alone is insufficient. By contrast, CL-SPF achieves a Step Success Rate of 0.424 with an Avg Action Attempts of 6.6. Compared with Agent Lumos + fixed retries (R = 7), CL-SPF achieves a 0.100 absolute improvement while using fewer average attempts than the fixed seven-attempt setting, indicating that its improvement is not merely the result of a larger action budget.

The advantage of CL-SPF can be attributed to its verification-guided recovery process. The Invalid Verdict Rate of 0.444 shows that the verifier identifies a substantial portion of potentially invalid steps. Once recovery is triggered, local regrounding succeeds in 85.7% of attempts, while replanning succeeds in 53.3% of attempts. These results suggest that many errors can be corrected locally without changing the current subgoal, while replanning remains useful when local repair is insufficient. Overall, CL-SPF improves web interaction performance by selectively detecting and correcting intermediate errors, thereby reducing the accumulation of invalid states and enhancing the stability of multi-step decision making in dynamic environments.

Since the present study adopts the Test-Domain split of Mind2Web, the obtained Step Success Rate should be interpreted under the more challenging domain-level generalization setting. Prior results on Mind2Web show that Step Success Rate varies substantially across different evaluation splits. For example, the original Mind2Web study reported a Step Success Rate of 52.0% under the Cross-Task setting, whereas the results decreased to 38.9% and 39.6% under the more difficult Cross-Website and Cross-Domain settings, respectively. Agent Lumos further reported Step Success Rates of 27.6% and 31.3% for its 7B and 13B variants on Mind2Web. Against this background, the 42.4% Step Success Rate achieved by CL-SPF indicates competitive performance for a 4B-scale modular agent under the Test-Domain setting. Nevertheless, because published results may differ in candidate generation, data processing, model backbone, and evaluation split, this comparison is used only to contextualize the result rather than as a strictly controlled leaderboard comparison.

4.3. Step Verification and Ablation Analysis

To further verify the effectiveness of the key mechanisms in the proposed closed-loop framework, supplementary experiments are conducted from two aspects, namely overall module contribution and step verification performance. First, a module ablation study is performed to examine the effects of step verification, local regrounding, and high-level replanning on overall task-solving performance. Second, a step verification ablation study is conducted to analyze the verification performance of the verifier under different settings, thereby further clarifying its role in process control.

4.3.1. Module Ablation Study

As shown in Table 6, the ablation study considers three settings, namely w/o Verifier, w/o Reground, and w/o Replan, where the step verification module, the local regrounding module, and the high-level replanning module are removed from the full framework, respectively, to examine the contribution of each component to overall performance. The full model achieves the highest Answer Accuracy (0.650) and consistently outperforms all ablated variants, indicating that the coordination of multiple modules effectively improves task-solving performance.

Table 6. Results of the Module Ablation Study.

Method	Answer Accuracy	Step Success Rate	Avg Action Attempts	Invalid Verdict Rate	Reground Success Rate	Replan Success Rate
w/oVerifier	0.567 ± 0.007	0.993	4.36	-	-	-
w/o Reground	0.608 ± 0.006	0.995	7.87	0.672	-
w/o Replan	0.633 ± 0.005	0.989	8.73	0.672	0.333	-
Full	0.650 ± 0.005	0.992	7.96	0.636	0.347	0.571

After removing the step verification module, Answer Accuracy drops from 0.650 to 0.567, showing the largest decline among all settings. This suggests that step-level validity assessment plays a critical role in suppressing the accumulation of intermediate errors and maintaining the stability of the solving process. Removing the local regrounding module reduces Answer Accuracy to 0.608, indicating that error detection alone is insufficient for effective recovery, and that a local correction mechanism is needed to translate verification signals into actual corrective capability. After removing the high-level replanning module, Answer Accuracy becomes 0.633, while Avg Action Attempts reaches the highest value, suggesting that without high-level path adjustment, the system is more likely to fall into repeated local repairs, thereby limiting recovery efficiency.

Overall, verifier, Reground, and Replan serve different roles in intermediate state assessment, local correction, and global adjustment, respectively. The results indicate that the performance gain of the proposed method mainly arises from the closed-loop coordination of these three components, rather than from the isolated contribution of any single module.

4.3.2. Analysis of Step Verification Results

To evaluate the effect of negative-sample construction on the performance of the step verifier, ablation experiments were conducted on the GSM dataset. The analysis was carried out from three aspects, including overall classification performance, binary discrimination between valid and invalid steps, and error-type-specific detection ability.

Figure 3 presents the overall performance of the step verifier under different negative-sample construction settings. Among all configurations, the Full setting achieved the best overall performance, with a Macro-F1 of 0.6094, which was substantially higher than that of No Negative (0.0584). This result indicates that relying only on valid-step samples is insufficient for learning a stable decision boundary for invalid-step detection, and negative-sample supervision is essential for establishing effective verification capability. Further comparison of the single-type negative settings shows that Single Input achieved relatively better overall performance, whereas Single Observation and Single Subgoal were less effective, suggesting that different negative-sample types provide unequal supervision signals during training.

Figure 3. Overall Performance Comparison of the Step Verifier under Different Negative-Sample Construction Strategies.

Figure 4 further compares the classification results for the valid and invalid classes. Most settings achieved relatively high precision on the invalid class. For example, the Full setting reached a precision-invalid of 0.9361, indicating that the verifier was generally reliable when predicting a step as invalid. However, the performance on the valid class remained relatively weak. The F1-valid of Full was 0.4000, while that of Full Negative was only 0.0954. This suggests that when the training process overly emphasizes invalid-step detection, the verifier tends to become biased toward the invalid class, thereby weakening its ability to preserve valid steps. In contrast, although No Negative maintained a relatively high recall on the valid class, it was almost incapable of detecting invalid steps. Therefore, achieving a balance between invalid-step detection and valid-step retention is a critical issue in training the step verifier.

Figure 4. Comparison of Precision and Recall for the Valid and Invalid Classes under Different Negative-Sample Construction Strategies.

This result also indicates a non-negligible class imbalance in the verifier’s behavior. Under the Full setting, the verifier performs more reliably on the invalid class than on the valid class, as reflected by the relatively low F1-valid score. Nevertheless, in the end-to-end GSM pipeline, the verifier is not employed as a hard rejection module. Instead, invalid predictions are mainly used as recovery signals to guide subsequent regrounding or replanning. In this setting, steps predicted as invalid are not directly discarded from the reasoning process. Accordingly, the downstream improvement should not be attributed simply to aggressive rejection. Rather, it should be understood as the combined effect of verification-guided recovery, candidate revision, and subsequent trajectory regulation.

Figure 5 shows the recall changes for different error types by taking the Full setting as the reference. The Full configuration achieved relatively balanced performance across the four invalid-step categories, with recalls of 0.8694 and 0.9002 for action swap and observation swap, respectively. In contrast, the recall of No Negative was close to zero for nearly all error types, further confirming that negative-sample supervision is fundamental to the development of error detection ability. For the removal-based ablations, w/o Action dropped sharply to 0.0843 on action swap, indicating that action-swap samples provide direct support for recognizing action-level errors. This finding suggests that the step verifier does not merely learn a generic notion of incorrectness but instead develops differentiated responses to different categories of invalid steps.

Figure 5. Heatmap of Recall Changes across Error Types under Different Ablation Settings.

Figure 6 further indicates that different negative-sample types are not simply additive. Single Action still achieved relatively high recall on action swap, whereas w/o Action showed clear degradation, indicating strong type-specificity. In contrast, input swap and observation swap remained relatively detectable even after their corresponding negative samples were removed, suggesting a certain degree of cross-type complementarity. Single Subgoal also performed well on its corresponding error type, implying that subgoal-mismatch samples contain relatively dense discriminative information.

Figure 6. Comparison of Error-Type Recognition under Single-Type Retention and Removal-Based Ablations.

Overall, the results show that negative-sample construction is essential for improving step verification performance on GSM. The Full setting achieved a better balance between invalid-step detection and valid-step retention, while different negative-sample types contributed unequally to error recognition. In particular, action-swap samples showed stronger type-specificity, whereas input-, observation-, and subgoal-related negatives provided more complementary supervision.

4.3.3. Effect of Negative-Sample Construction on End-to-End Task Performance

To further assess the effect of step-verification ablations on end-to-end reasoning, the verifiers trained with different negative-sample construction strategies were integrated into the full GSM pipeline. Rather than evaluating binary classification alone, this experiment examines whether the verification signal can effectively support regrounding and replanning during reasoning. Table 7 reports the task accuracy, task completion rate, step success rate, and recovery success rates under different training settings.

Table 7. Effect of Negative-Sample Construction on End-to-End Task Performance on GSM.

Method	Task Accuracy	Task Completion Rate	Step Success Rate	Reground Success	Replan Success
Full	65.0 ± 0.5%	97.5%	99.2%	34.7%	57.1%
Full Negative	56.0 ± 0.7%	96.7%	99.0%	24.7%	42.0%
No Negative	33.3 ± 0.9%	95.0%	98.6%	1.08%	2.01%
Single Action	56.0 ± 0.8%	98.3%	99.5%	23.0%	46.6%
Single Input	62.7 ± 0.6%	81.7%	94.2%	34.7%	69.2%
Single Observation	54.3 ± 0.7%	98.3%	99.5%	29.4%	64.5%
Single Subgoal	59.3 ± 0.6%	83.3%	94.8%	37.8%	70.6%
w/o Action	61.0 ± 0.7%	96.7%	99.0%	29.5%	77.5%
w/o Input	49.3 ± 0.8%	96.7%	99.0%	22.3%	43.5%
w/o Observation	59.3 ± 0.7%	95.0%	98.4%	24.9%	47.5%
w/o Subgoal	57.7 ± 0.8%	95.0%	98.5%	22.8%	45.6%

The end-to-end ablation results indicate that negative-sample construction affects not only the classification capability of the step verifier, but also the final task performance of the complete reasoning pipeline. As shown in the table, the Full setting achieved the highest task accuracy on GSM (65.0%), while maintaining a high task completion rate (97.5%) and step success rate (99.2%). This suggests that the combination of step-level verification with regrounding and replanning improves final answer correctness without substantially reducing execution stability.

In contrast, the task accuracy of No Negative dropped to 33.3%, substantially lower than that of the Full setting. Although its task completion rate and step success rate still reached 95.0% and 98.6%, respectively, the absence of invalid-step supervision prevented the system from forming effective error detection and recovery capability. This indicates that positive-only training may preserve process executability, but is insufficient to support effective closed-loop correction.

The effects of different negative-sample types were also unequal. In the removal-based ablations, w/o Input showed the largest decline, with task accuracy decreasing to 49.3%, whereas w/o Action, w/o Observation, and w/o Subgoal achieved 61.0%, 59.3%, and 57.7%, respectively. This suggests that, in mathematical reasoning tasks, consistency between the action input and the current subgoal and context has a more direct influence on the correctness of the subsequent calculation chain.

The single-type settings further show that relying on only one error type is insufficient for stable end-to-end reasoning. For example, Single Input achieved a task accuracy of 62.7%, close to that of the Full setting, but its task completion rate and step success rate dropped to 81.7% and 94.2%, respectively. Similarly, although Single Subgoal achieved a relatively high replanning success rate, its task completion rate was only 83.3%. In contrast, the Full setting showed a better balance among task accuracy, task completion rate, and step success rate.

From the perspective of recovery dynamics, regrounding and replanning played complementary roles across different settings. For example, the w/o Action setting obtained a replanning success rate of 77.5%, which was substantially higher than that of the Full setting, but its final task accuracy was still lower. This suggests that improvement in a single recovery indicator does not necessarily translate into better overall task performance. More importantly, broader negative-sample coverage provides the verifier with richer supervision signals, which helps maintain more reliable error detection and more stable recovery during the reasoning process. Taken together, these findings indicate that the design of negative samples affects not only verifier-level behavior, but also end-to-end accuracy, process completion, and recovery effectiveness. This further supports the view that structured negative-sample construction is an important factor in strengthening the closed-loop correction capability of the overall framework.

4.4. Qualitative Error Analysis

To better understand the role of step-level verification in the proposed framework, we further examine several representative cases from end-to-end execution traces. These examples, summarized in Table 8, provide qualitative insights into how verification signals guide local correction and replanning under different error conditions.

Table 8. Qualitative Error Analysis in Mathematical Reasoning Tasks.

Case	Field	Description
Case A	Type	Successful correction
	Original Task	Original Task: Claire eats 3 eggs every morning. How many dozens of eggs will she eat in 4 weeks?
	Key Verification and Recovery Steps	The initial step relied on a missing intermediate result, produced an empty observation, and was judged as invalid. After two failed reground attempts, the system triggered replan and directly used the relation among 3 eggs/day, 7 days/week, 4 weeks, and 12 eggs/dozen, yielding 7.
	Interpretation	The verifier detects empty observations and broken dependencies, and guides recovery from local correction to replanning.
Case B	Type	Detected but not corrected
	Original Task	In a class of 20 students, 20% choose contemporary, 25% of the remaining choose jazz, and the rest choose hip-hop. What percentage choose hip-hop?
	Key Verification and Recovery Steps	The initial step depended on unreliable intermediate values, produced an empty observation, and was judged as invalid. Repeated reground failed. A later replan step produced 55, but incorrectly interpreted “25% of the remaining” as “25% of all students.”
	Interpretation	The verifier exposes execution errors, but recovery fails to preserve the original quantitative relation.
Case C	Type	Detected but not corrected
	Original Task	Kylar buys 16 glasses. One costs $5, and every second glass costs 60% of that price. What is the total cost?
	Key Verification and Recovery Steps	A correct percentage-based step produced 3 but was judged as invalid. After reground, it was rewritten in decimal form and marked unknown. Likewise, a compact step yielding 64 was rejected before being expanded.
	Interpretation	The verifier is conservative toward percentage-based or compact expressions, causing extra correction attempts.

To complement the case-based analysis, we further examine the recovery outcomes on the GSM evaluation results. At the task level, 55.0% of the cases are solved after recovery is triggered by step verification, whereas 32.5% remain unresolved after the correction process. The remaining 12.5% do not activate the correction mechanism; among these cases, 10.0% of all samples are ultimately answered correctly, while 2.5% still fail. For the subset of cases in which correction is invoked, the successful-recovery rate is 62.9%, and the persistent-failure rate is 37.1%. These statistics are intended to characterize the behavior of the recovery mechanism, rather than to redefine task-level performance. The reported Accuracy and Task Success Rate are still computed according to final-answer correctness.

Over-correction is further analyzed using a conservative proxy. A case is treated as over-correction-like when an intermediate step is rejected by the verifier, but a later rewritten step yields a semantically equivalent non-failure observation and the task is eventually solved. According to this criterion, 10.0% of the cases exhibit over-correction-like behavior.

These cases suggest that the step verifier functions primarily as a process-level control component rather than as a judge of final-answer correctness. Its main role is to identify problematic intermediate states, such as empty observations, disrupted step-to-step dependencies, and execution failures, and then initiate either local regrounding or higher-level replanning when needed.

For more semantically involved problems, the verifier is still useful for revealing abnormal intermediate steps, but the effectiveness of recovery depends on whether the regenerated action preserves the original quantitative structure of the problem. In some cases, even when an expression is semantically correct, the verifier may respond conservatively if the expression is overly compressed, which can lead to additional correction attempts.

Overall, the verification–correction mechanism mainly targets three categories of intermediate problems: failures associated with tools or missing observations, broken reasoning chains caused by incomplete intermediate dependencies, and valid but non-standard expressions that require normalization before further execution.

4.5. Interpretability and Engineering Analysis

Beyond the performance improvements, the proposed framework also offers benefits in terms of interpretability and practical implementation. In contrast to black-box generation methods that produce only a final output, this framework structures the solving process into several explicit stages, including task planning, action execution, step verification, and local correction. Such a design makes the intermediate reasoning and execution process more transparent. Because each stage serves a relatively distinct role, it becomes easier to examine how decisions are formed and where errors emerge.

In terms of interpretability, the framework makes several key intermediate states visible, such as the current subgoal, the generated action, and the corresponding validity judgment. Consequently, errors are not observed only at the final output level, but can be traced back to particular stages of the process, for example, subgoal deviation, action generation failure, or inconsistency between action and observation. In addition, process-level metrics such as Invalid Verdict Rate, Avg Action Attempts, and Reground Success Rate provide further evidence for analyzing system behavior. Compared with approaches that report only final task outcomes, this framework supports a more detailed examination of how errors arise, propagate, and are corrected in multi-step solving.

From an engineering perspective, the framework adopts a modular design in which each component has a relatively clear functional responsibility. This makes the system easier to train, modify, and deploy, and is also advantageous for later maintenance and extension. Another practical aspect is that recovery is performed locally whenever possible. When an invalid step is detected, the framework first revises the action under the current subgoal, rather than restarting the entire task process. This strategy improves robustness by avoiding full-task restarts, while the actual computational overhead is still reflected in the increased number of action attempts and requires further runtime and token-level profiling. The experimental results also suggest that although local regeneration increases the number of action attempts, it can recover part of the erroneous trajectories and thus provides a reasonable balance between performance and cost.

Furthermore, the results on both mathematical reasoning and web interaction tasks indicate that the framework is not restricted to a single type of task structure. It remains effective in both structured reasoning settings and interactive decision-making environments. This suggests that the planning–execution–verification–correction closed-loop mechanism has a certain degree of cross-task applicability and practical value.

5. Discussion and Limitations

To provide a clearer interpretation of the empirical findings, this section further discusses the effectiveness, limitations, and scalability of the proposed framework across different task scenarios.

The experimental results indicate that the proposed framework consistently improves performance on both mathematical reasoning and web interaction tasks, although the pattern of improvement is not the same in the two settings. In mathematical reasoning, the framework brings relatively stable gains in answer accuracy, suggesting that step-level verification can identify and correct problematic intermediate reasoning before those errors propagate further along the trajectory. In web interaction, the framework also improves step success rate, which indicates that verification-guided recovery remains useful in sequential decision-making tasks involving interaction with an external environment. However, although the absolute improvement in the web setting appears larger, it is also associated with more recovery operations and a higher number of action attempts. This suggests that, in interactive tasks, the observed gains mainly reflect stronger robustness under environmental uncertainty, whereas in mathematical reasoning the improvement more directly reflects the benefit of step-level validation itself.

The ablation results further show that the increase in action attempts appears in both task types, but its role differs across them. In mathematical reasoning, introducing verification and recovery increases the average number of action attempts from 4.36 to 7.96, while answer accuracy rises from 0.567 to 0.650. This suggests that the additional attempts are not simply redundant retries, but part of an effective correction process that prevents invalid intermediate steps from entering the reasoning trajectory. In contrast, in web interaction, repeated attempts are more closely tied to the dynamic and context-dependent nature of the environment. Variations in page content, interface state, and historical interaction context may all influence execution outcomes, making step validity inherently more difficult to judge in a stable manner. Under such conditions, repeated correction serves mainly to enhance execution robustness, rather than improving reasoning quality alone.

Despite these differences, the proposed framework still maintains clear advantages over purely forward generation baselines. Its central contribution lies in introducing explicit feedback into the execution process, so that local errors can be detected and addressed before they propagate to later steps. By combining step verification with a hierarchical recovery mechanism, the framework offers a practical way to reduce cumulative error propagation without fundamentally altering the modular planning–execution paradigm. These findings further suggest that step verification in the proposed framework should be understood primarily as a process-level control signal rather than as a checker of final-answer correctness. Its main function is to regulate whether intermediate states should be accepted, locally repaired, or revised at the planning level before they affect subsequent execution. At the same time, the verifier does not achieve uniformly high classification performance across both classes. The relatively low F1 score for the valid class suggests that the verifier tends to behave conservatively and may reject some semantically acceptable but compact or non-standard steps. This reflects a trade-off between invalid-step interception and valid-step preservation. A more conservative verifier can reduce the risk of accepting erroneous states, but it may also increase action attempts and cause over-correction. This observation is consistent with the end-to-end ablation results, where higher recovery rates do not always lead to higher final task accuracy. Future work will investigate verifier calibration and confidence-aware verification to better balance error detection and unnecessary rejection.

Several limitations should nevertheless be noted. First, verification and recovery inevitably bring additional computational overhead, because invalid steps may trigger repeated action attempts or replanning. Although the additional cost is bounded by the maximum retry number in the current setting, its practical efficiency still requires further runtime and token-level profiling. Second, the verifier is trained on automatically constructed negative samples. While this strategy supports scalable data construction and captures several common error patterns, it may not fully cover the diversity of failures encountered in real-world dynamic environments.

From the scalability perspective, the proposed framework inevitably brings additional computational cost, as invalid intermediate steps may lead to local regeneration or, in more difficult cases, high-level replanning. In the current experiments, this cost is mainly observed through the increase in Avg Action Attempts. Direct measurements of wall-clock latency and token consumption are not included. Therefore, the reported results should be interpreted as evidence for the effectiveness of verification-guided recovery in improving task performance and process robustness, rather than as a complete evaluation of execution efficiency.

The overhead introduced by the framework is also related to the interaction among step verification, local regeneration, and replanning, rather than to the retry mechanism alone. Since recovery is activated only when a step is judged invalid, and the maximum retry number is explicitly limited, the additional cost is partially controlled in the present setting. Nevertheless, the practical efficiency of the framework still requires further investigation. Future work will include runtime profiling, token-cost analysis, and adaptive verification strategies, with the aim of improving the trade-off among robustness, correction capability, and computational cost in longer and more complex task scenarios.

6. Conclusions

This study addressed process control in multi-step task solving and presented a closed-loop modular language agent framework that integrates task planning, action execution, step verification, local regeneration, and replanning. To support step validity prediction, a process-supervision scheme was introduced by retaining real execution steps as valid samples and constructing invalid samples through action substitution, input perturbation, observation replacement, and subgoal mismatch.

The experimental results on mathematical reasoning and web interaction tasks support the effectiveness of the proposed framework. On mathematical reasoning tasks, CL-SPF achieved an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, CL-SPF achieved a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Conditional on triggered recovery attempts, the reground and replan success rates were 0.857 and 0.533, respectively.

The ablation results further show that the gains of the proposed framework do not arise from any single component in isolation. Instead, they depend on the interaction between step verification and the recovery mechanisms. In particular, step verification does not mainly function as an independent filtering module. Within the proposed framework, it serves as a process-level control signal for hierarchical recovery, where local regeneration addresses execution-level deviations and replanning addresses higher-level subgoal mismatch when local correction is insufficient.

Overall, the proposed framework provides a structured approach to process control in multi-step task solving. The current study remains limited to two representative task settings and a fixed base model. Future work will consider improving verifier calibration, strengthening recovery strategies in more complex interactive environments, and extending the framework to broader task domains and stronger model settings.