Improving the Robustness of Scene-Aware Neuro-Symbolic Solving for Arithmetic Word Problems Under Input Perturbations

Abstract

Robust Arithmetic Word Problem (AWP) solving is important for applying mathematical reasoning systems in educational scenarios, where problem statements may contain changed numerical values, paraphrased descriptions, or irrelevant distracting information. Although Large Language Models (LLMs) have shown strong potential in solving AWPs, their reasoning processes may still be sensitive to surface-form variations and perturbation-induced noise. To address this issue, this paper proposes a Scene-Aware Neuro-Symbolic solver designed to improve the robustness of AWP solving under perturbations. The proposed method extends the existing scene-aware framework by introducing perturbation-oriented mechanisms at the scene, relation, and symbolic-solving levels. A Chain-of-Scene (CoS) prompting strategy first generates candidate scenes, after which goal-guided filtering retains target-related and bridge scenes while removing distractor-induced scenes. The retained scenes are then processed by the Scene-Aware Syntax-Semantics ($S^2$) method to extract explicit and implicit relations, and relation consistency checking is applied to remove locally plausible but globally irrelevant relations. Finally, the symbolic solver performs iterative equation-based reasoning over the filtered relation sets, with fallback recovery activated when standard solving does not produce a target-compatible answer. Experiments on AGG, MAWPS, and GSM8K show an average accuracy of 92.8% on clean datasets. On GSM-Perturb and AWP-Perturb, the solver achieves perturbed accuracies of 80.8% and 87.5%, with robustness drops of 8.3% and 6.8%, respectively. Ablation results show that scene filtering and relation consistency checking are the main contributors to reducing perturbation-induced errors. These findings suggest that combining LLM-based scene understanding with symbolic relation reasoning is a promising direction for improving the robustness and interpretability of AWP solvers in the evaluated perturbation settings.

1. Introduction

Problem solving has become an important research topic with the rapid development of artificial intelligence, and solving Arithmetic Word Problems (AWPs) is one of the most representative and challenging tasks in this area [1]. An AWP describes quantitative conditions and solution goals in natural language, requiring a solver to jointly perform language understanding, numerical reasoning, and equation-level calculation [2,3]. In recent years, neural models and Large Language Models (LLMs) have substantially improved the solving accuracy of AWPs [4,5,6]. However, high accuracy on clean benchmark problems does not necessarily indicate that a solver is reliable in practical educational scenarios. A problem statement may be paraphrased by a student, contain additional irrelevant information, or use changed numerical values while preserving the same reasoning structure. Therefore, beyond solving clean AWPs, it is also important to examine whether AWP solvers can remain stable under input perturbations.

The robustness problem is particularly important for LLM-based mathematical reasoning. Prompting methods such as Chain-of-Thought (CoT) encourage LLMs to solve mathematical tasks through step-by-step natural language reasoning [6]. A series of robustness-oriented reasoning methods have further improved LLM-based mathematical reasoning from different perspectives, including self-consistency voting over multiple reasoning paths [7], subproblem decomposition through Least-to-Most prompting [8], program-based reasoning such as Program-of-Thoughts and Program-Aided Language Models [9,10], and tool-augmented or code-execution-based verification methods [11,12]. Nevertheless, recent studies have shown that mathematical reasoning models can still be sensitive to surface-form variations and distracting contexts. For example, GSM-Plus evaluates the robustness of LLMs on GSM8K variants and shows that LLMs are far from robust under different mathematical perturbations [13]. Shi et al. [14] further show that LLMs can be distracted by irrelevant context in grade-school mathematical reasoning. GSM-Symbolic also reports that changing numerical values or adding seemingly relevant but unnecessary clauses can lead to noticeable performance degradation in LLM reasoning [15]. These findings suggest that robust AWP solving should not rely only on direct natural language reasoning by LLMs.

Another important line of AWP-solving research is the relation-flow approach, which represents the mathematical structure of a problem as explicit and implicit relations and then derives answers through symbolic reasoning [16]. Compared with direct answer generation, relation-flow methods are more interpretable because the intermediate relations and symbolic operations are aligned with human mathematical reasoning. They are, therefore, suitable for educational applications, where transparency and reliability are important [17,18]. However, traditional relation-flow solvers depend heavily on relation extraction from the original problem text. When the problem contains paraphrased expressions or irrelevant numerical statements, relation extraction may become incomplete or noisy. Thus, relation-flow methods provide reliable symbolic reasoning, but they still require stronger text understanding and noise suppression mechanisms under perturbations.

Existing work attempted to combine the advantages of LLMs and relation-flow methods by proposing a Scene-Aware Neuro-Symbolic solver [19]. In that framework, an LLM decomposes the input AWP into scenes through Chain-of-Scene (CoS) prompting, a scene-aware $S^2$ method extracts relations from each scene, and a symbolic solver reasons over the extracted relation sets. This design improves solving accuracy and interpretability by using the LLM for language understanding rather than direct calculation. However, the previous framework mainly considers clean AWP inputs and does not explicitly handle perturbation-induced noise. Under irrelevant-information perturbation, the LLM may generate scenes from distracting sentences; under paraphrase perturbation, key relations may be expressed in unfamiliar forms; under numerical perturbation, the solver must preserve the reasoning structure while correctly recomputing the answer. Therefore, simply decomposing an AWP into scenes is not sufficient for robust solving. A perturbation-oriented solver should further decide which scenes are relevant to the final goal, which relations are consistent with the target-oriented reasoning process, and how to recover from local extraction or solving failures.

To address these limitations, we study how to improve the robustness of Scene-Aware Neuro-Symbolic Solving for AWPs under input perturbations. The method keeps the scene-relation-equation reasoning pipeline of the previous scene-aware framework, but adapts each stage to the perturbation setting. In the Scene Decomposer, perturbation-aware CoS prompting is combined with goal-guided scene filtering so that target-related and bridge scenes are retained while distractor-induced scenes are reduced. In the Relation Extractor, the Scene-Aware $S^2$ method is applied to the retained scenes, and relation consistency checking is introduced to remove locally plausible but globally irrelevant relations. In the Symbolic Solver, inter-scene answer passing is used for iterative reasoning, and conservative fallback recovery is activated only when the standard solving process fails to produce a target-compatible answer. With this design, the LLM is mainly used for semantic scene decomposition, whereas symbolic relation reasoning handles structured inference and equation-level calculation.

The main contributions are summarized as follows:

• We formulate robust AWP solving under perturbations and evaluate solvers from both solving accuracy and robustness perspectives. By considering numerical substitution, problem-understanding reformulation, and distractor insertion, this study examines reliability issues that are not fully reflected by clean benchmark accuracy alone.
• We extend the existing scene-aware framework with perturbation-oriented mechanisms at the scene, relation, and symbolic-solving levels. Specifically, goal-guided scene filtering, relation consistency checking, and conservative fallback recovery are introduced to reduce the propagation of perturbation-induced errors.
• We provide experimental results on standard and perturbed AWP datasets to evaluate the effectiveness of the proposed solver. The results show competitive clean-data accuracy and smaller performance degradation under perturbations, while the ablation and diagnostic analyses further identify the contributions of scene filtering and relation consistency checking.

The remainder of the paper is organized as follows. Section 2 reviews related studies on LLM-based AWP solving, robustness evaluation under perturbations, and relation-flow approaches. Section 3 introduces the proposed method. Section 4 presents the experimental settings, results, ablation study, diagnostic analysis, and case study. Section 5 discusses the experimental results, analyzes limitations, and concludes the paper with future research directions.

2. Related Work

2.1. LLM-Based Methods for Solving AWP

Large Language Models (LLMs) have become an important direction for solving mathematical reasoning tasks because they can interpret natural language problem descriptions and generate intermediate reasoning steps from prompts [4,5]. In AWP solving, Chain-of-Thought (CoT) prompting encourages LLMs to produce step-by-step reasoning before giving the final answer [6]. Self-consistency further improves CoT by sampling multiple reasoning paths and selecting the most frequent or consistent answer [7]. These methods show that LLMs can solve many AWPs without task-specific model training.

Several studies have further enhanced LLM-based mathematical reasoning by changing the form of intermediate reasoning or by introducing additional verification mechanisms. Program-of-Thought prompting asks the LLM to generate executable programs, so that numerical calculation can be delegated to a program interpreter [9]. Program-Aided Language Models similarly use the LLM to translate natural language reasoning into runnable programs and offload the computation to an external interpreter [10]. Least-to-Most prompting decomposes a complex problem into simpler subproblems and solves them progressively [8]. Tool-augmented LLMs further improve reasoning by enabling the model to call external tools such as calculators or APIs [11]. Code-based self-verification methods use code execution and verification signals to check and revise mathematical solutions [12]. Equation-of-Thought, X-of-Thought and Discriminator-Guided reasoning methods introduce equation generation, multiple reasoning thoughts, verifier-based selection or step-level correctness guidance to improve reasoning reliability [12,20,21]. These approaches are effective because they reduce some calculation errors and encourage more structured reasoning than direct answer generation.

However, LLM-based AWP solvers still have inherent limitations under perturbations. Most prompting methods express intermediate reasoning as free-form natural language or executable code, and these intermediate states are not always aligned with the explicit mathematical relations in the problem text. As a result, a paraphrased problem, an irrelevant numerical statement, or a numerical substitution may lead the LLM to produce a fluent but incorrect reasoning chain. Self-consistency and verifier-based methods can reduce the effect of unstable individual reasoning paths, but they usually do not identify which scenes or relations are required by the final solving goal. Program-based methods improve calculation reliability, yet they still depend on the LLM to generate correct program logic from the perturbed text. Therefore, although LLMs are useful for semantic understanding, relying on LLM-generated reasoning alone is insufficient for robust AWP solving. This motivates the use of LLMs mainly for scene-level understanding, while relation extraction and equation solving are handled by more explicit symbolic components.

2.2. Robustness of Mathematical Reasoning Under Perturbations

Robustness evaluation studies whether a model can preserve its expected behavior under controlled input changes. In natural language processing, adversarial and behavioral testing methods have shown that models may rely on superficial textual cues rather than stable task-relevant reasoning [22,23]. For AWP solving, this problem is especially important because a mathematically equivalent problem may be expressed using different wording, different numerical values, or additional irrelevant statements. A robust AWP solver should, therefore, maintain correct reasoning under numerical substitution, problem-understanding reformulation, and distractor insertion.

Recent mathematical reasoning benchmarks have shown that LLMs are sensitive to such perturbations. GSM-IC demonstrates that irrelevant contexts can distract LLMs in grade-school mathematical reasoning [14]. GSM-Plus extends GSM8K with different perturbation types and evaluates whether LLMs can preserve correct reasoning under controlled variants [13]. GSM-Symbolic further reports that changing numerical values or adding seemingly relevant but unnecessary clauses may lead to noticeable performance degradation [15]. These studies indicate that clean-data accuracy alone cannot fully measure the reliability of mathematical reasoning systems.

Existing robustness-oriented methods mainly improve LLM reasoning at the prompting, sampling, decomposition, tool-use, or verification level [7,8,9,10,11,12,14]. However, these strategies generally leave the selection of useful mathematical conditions to the LLM itself. In particular, they do not explicitly separate useful mathematical conditions from distractors before reasoning, nor do they maintain a structured relation memory to check whether newly extracted information is compatible with the final solving target. Therefore, perturbation-induced errors may still propagate through the reasoning chain. In contrast, our method places perturbation handling inside the intermediate reasoning process: scenes are filtered before relation extraction, relation consistency is checked before symbolic solving, and the final answer is derived from explicit equation systems. In this way, robustness is handled at the scene, relation, and symbolic-solving levels rather than only through input-level prompting.

2.3. Relation-Flow and Neuro-Symbolic AWP Solving

Relation-flow methods solve AWP by extracting mathematical relations from problem text and reasoning over these relations [16]. Different from direct neural answer generation, relation-flow methods represent intermediate reasoning as explicit or implicit quantitative relations, which makes the solving process more interpretable. The $S^2$ method maps textual syntax and semantic patterns into mathematical relations and has been extended to different educational problem-solving tasks [16,17,24,25,26]. Because the intermediate states are relation sets rather than hidden vectors or free-form text, relation-flow methods are suitable for educational applications where transparency is important.

Nevertheless, traditional relation-flow methods also have limitations. They usually assume that the input problem can be converted into a coherent relation set from the original text. This assumption works reasonably well for clean AWPs, but becomes less reliable when the problem contains paraphrased expressions, irrelevant numerical statements, or multiple local scenes. In such cases, relation extraction may become incomplete or noisy: necessary relations may be missed because the wording changes, while irrelevant relations may be extracted from distractor sentences. Therefore, although relation-flow reasoning provides interpretability and precise equation solving, it still needs stronger text understanding and noise suppression mechanisms under perturbations.

Neuro-symbolic AWP solving combines neural language understanding with symbolic reasoning. Neural components can help identify quantities, entities, or problem structures, while symbolic components derive answers through explicit mathematical operations [27,28,29,30]. The previous Scene-Aware Neuro-Symbolic solver further introduced LLM-based COS prompting to decompose an AWP into scenes before relation extraction [19]. This design organizes complex problem conditions into local scene contexts, which is useful for relation extraction. Its main weakness in the perturbation setting is that all generated scenes and the relations extracted from them may still enter the later symbolic reasoning process. A distractor sentence may, therefore, lead to a locally plausible but globally irrelevant relation, while a weakly connected intermediate scene may be difficult to keep or remove.

For this reason, we extend the previous scene-aware framework by adding goal-guided scene filtering, relation consistency checking, and conservative fallback recovery. These mechanisms retain target-related and bridge scenes, suppress spurious relations introduced by perturbations, and reduce the impact of local extraction failures.

Table 1 summarizes representative existing AWP-solving methods in terms of their main approach, use of LLMs, symbolic solving, and perturbation handling. Compared with these existing methods, the proposed method further introduces perturbation-oriented scene filtering and relation consistency checking into the scene-aware neuro-symbolic pipeline.

Table 1. Comparison of representative AWP-solving methods.

Method	Main Approach	LLM	Symbolic Solver	Perturbation Handling
JieAlg [30]	Deductive reasoning for expression construction	No	No	Not explicitly considered
YuAlg [16]	Relation-flow reasoning with explicit and implicit relation extraction	No	Yes	Not explicitly considered
CoT [6]	Multi-step reasoning path generation	Yes	No	Implicitly improves reasoning stability through step-by-step generation, does not explicitly handle perturbations
Self-Consistency [7]	Multiple reasoning path sampling with answer aggregation	Yes	No	Reduces instability of single CoT path reasoning.
Least-to-Most [8]	Decomposes problems into subproblems	Yes	No	Improves compositional reasoning, depends on correct generation of subproblems from perturbed text.
GRACE [21]	Discriminator-guided CoT reasoning	Yes	No	Improves reasoning reliability by using a correctness discriminator to guide multi-step reasoning
PoT [9]	Program generation with external execution	Yes	Yes	Reduces calculation errors through program execution, depends on correct program generation from perturbed text
PAL/Tool-Augmented [10,11,12]	External tool use and code-based self-verification	Yes	Yes	Reduces calculation and verification errors, depends on correct tool-use logic generation under perturbations
XoT [20]	Planning and switching among multiple reasoning paths	Yes	Yes	Combines multiple reasoning paths to reduce reasoning instability under perturbations
PengAlg [19]	LLM-based scene generation combined with relation-flow method	Yes	Yes	Partially improves stability through symbolic relation reasoning, does not explicitly filter perturbation-induced scenes or relations

3. Methodology

3.1. Problem Definition and Overview

This paper studies robust AWP solving under input perturbations. Different from conventional AWP solving, which assumes that the problem is presented in its original form, the setting considered in this paper allows the problem text to be transformed before solving. In robustness and behavioral testing for natural language processing, a perturbation is usually regarded as a controlled input transformation used to examine whether a model relies on task-relevant reasoning rather than superficial textual cues [22,23]. For AWPs, such perturbations may involve numerical substitution, problem-understanding reformulation, or distractor insertion.

To introduce the proposed method, we first define Arithmetic Word Problem, Perturbed Arithmetic Word Problem, and Scene.

Definition 1 

(Arithmetic Word Problem). An arithmetic word problem is a set of known conditions and at least one final solution goal described using natural language and numerical symbols.

Definition 2 

(Perturbed Arithmetic Word Problem). Given an original AWP T, a perturbed AWP $T^{\prime }$ is produced by applying a perturbation operator $\mathcal{P}(·)$ to T, i.e., $T^{\prime }=\mathcal{P}\left(T\right)$. In this paper, $\mathcal{P}$ includes three perturbation types: Numerical Substitution, Problem-Understanding Reformulation, and Distractor Insertion. The perturbed problem remains solvable and is associated with a gold answer $y^{\prime }$.

Definition 3 

(Scene). A scene is a subset of conditions described in an AWP together with a sub-goal that can be inferred from these conditions. The sub-goal is either directly required by the final solution goal or serves as an intermediate condition for a later scene. Therefore, an AWP contains at least one scene.

Let T denote an input problem text and y denote its target answer. For its perturbed version $T^{\prime }$, the corresponding target answer is denoted as $y^{\prime }$. In problem-understanding reformulation and distractor insertion, the answer remains unchanged, while in Numerical Substitution, the answer may change with the modified quantities. A robust solver $f(·)$ is expected to solve both the original and perturbed problems correctly with minimal performance degradation while preserving interpretable symbolic reasoning.

The proposed method further develops the scene-aware framework [19] for perturbed AWP solving. It strengthens the scene-aware pipeline by introducing perturbation-oriented mechanisms at the scene, relation, and symbolic-solving stages. In the scene stage, goal-guided scene filtering reduces distractor-induced scenes while retaining scenes that are relevant to the target. In the relation stage, relation consistency checking suppresses locally plausible but goal-irrelevant relations before they enter symbolic solving. In the symbolic-solving stage, fallback recovery serves as a controlled backup when the standard solving process fails to produce a target-compatible answer. These extensions are intended to reduce the propagation of perturbation-induced errors while preserving the interpretability of relation reasoning. Our method consists of the following three modules:

• Scene Decomposer: This module inherits the CoS-based scene generation idea, but does not directly accept all generated scenes. It generates candidate scenes from the input problem and then applies goal-guided filtering to remove irrelevant or weakly supported scenes while retaining scenes that may support intermediate sub-goals.
• Relation Extractor: The Relation Extractor adapts the Scene-Aware $S^2$ relation extraction paradigm to the retained scenes. It further applies a relation consistency check to suppress candidate explicit relations that are locally plausible but weakly connected to the final solving goal or the retained scene context.
• Symbolic Solver: The Symbolic Solver extends symbolic solving over relation sets by maintaining an answer buffer for inter-scene answer passing. Fallback recovery is activated only when the standard answer-passing process fails to produce a target-compatible answer.

The overall workflow is shown in Figure 1 and summarized in Algorithm 1. In Figure 1, $s_i$ denotes the i-th candidate scene generated by the LLM, $s_i^{*}$ denotes a retained scene after filtering, $R_i$ denotes the relation set extracted from $s_i^{*}$, and $A_i$ denotes the intermediate answer obtained from $R_i$. The following subsections describe the three modules in detail.

Figure 1. Overview of the proposed Scene-Aware Neuro-Symbolic Solver.

3.2. Scene Decomposer

3.2.1. Perturbation-Aware CoS Prompting

The original CoS method treats an AWP as a chain of scenes and prompts an LLM to generate these scenes progressively. This idea is retained in this paper, but the decomposing process is adapted to the perturbation setting. Instead of directly accepting all LLM-generated scenes, the Scene Decomposer first generates candidate scenes and then applies goal-guided filtering before relation extraction.

Algorithm 1: Robust AWP Solving under Perturbations
	Input: A problem text T
	Output: A solution A
1	Module 1: Scene Decomposer: Use Procedure 1 to decompose T into candidate scenes and obtain the filtered scene list $S^{*}$;
2	Module 2: Relation Extractor: Use Procedure 2 to extract an ordered list of relation sets $\mathcal{R}$ from $S^{*}$;
3	Module 3: Symbolic Solver: Use Procedure 3 to reason over $\mathcal{R}$ and obtain the solution A.

The decomposing prompt is represented by a unified decomposing query $Q_d$. It contains three parts: (1) a decomposition instruction that asks the LLM to identify solvable scenes rather than directly solve the whole problem; (2) a goal reminder that emphasizes the final question; (3) a negative instruction that discourages treating pure background descriptions or isolated numerical statements as valid scenes unless they support a later sub-goal. This design helps prevent paraphrased expressions and irrelevant numerical cues from being converted into unnecessary candidate scenes.

3.2.2. Exemplar Retrieval and Candidate Scene Generation

To construct few-shot prompts for scene generation, the decomposer retrieves exemplars from an annotated exemplar pool following the retrieval setting of the previous study [19]. Given an input problem text T, MWP-BERT [31] is used to obtain a global problem representation

$$\mathbf{h}\left(T\right)=\mathrm{MeanPool}\left(\text{MWP-BERT}\right(T\left)\right)$$

(1)

For each exemplar problem $O_i$ in the exemplar dataset $\mathcal{D}={\left\{(O_i,C_i)\right\}}_{i=1}^m$, where $C_i$ is the annotated scene list of $O_i$, the representation $\mathbf{h}\left(O_i\right)$ is computed in the same way. The retrieval score between T and $O_i$ is defined as

$$\mathrm{sim}(T,O_i)=\cos \left(\mathbf{h}\left(T\right),\mathbf{h}\left(O_i\right)\right)$$

(2)

The top-K exemplars are selected according to Equation (2) and inserted into the few-shot prompt together with $Q_d$ and T. The LLM then generates an ordered candidate scene list $S={\left\{s_i\right\}}_{i=1}^{n_s}$. The exemplar pool is manually constructed by domain experts according to school mathematical textbooks, rather than being directly sampled from the evaluation datasets in our experiment. We also removed duplicate problems between the exemplar pool and the evaluation datasets before evaluation.

3.2.3. Goal-Guided Scene Filtering

The candidate scene list may still contain weakly supported or distractor-induced scenes. Therefore, a goal-guided filter is applied before relation extraction. This filter uses shallow linguistic information and contextual representations, so it does not depend on the later relation extraction results.

Given the problem text T and its candidate scene list $S={\left\{s_i\right\}}_{i=1}^{n_s}$, the filter first extracts the target question clause g from T. For the target question, it constructs a target signature $\mathrm{Sig}\left(g\right)$ using entities, units, and available temporal cues. For each candidate scene $s_i$, it constructs a scene signature $\mathrm{Sig}\left(s_i\right)$ in the same way.

The filtering process has two steps: recall-oriented candidate pre-screening and score-based retention. The pre-screening step identifies scenes that may be useful for solving, but it does not determine final retention by itself. A scene is treated as a target-connected candidate only when it shares compatible signature elements with the target question. Entity-unit overlap is, therefore, used only as an initial cue; if the temporal cues conflict, such overlap alone is not sufficient for marking the scene as target-connected. A scene is treated as a bridge candidate when it is not directly connected to the target question but shares compatible signature elements with other candidate scenes. This design is intended to identify intermediate scenes that may not directly mention the final target entity but may support later relations.

For each candidate scene, the target relevance score and the scene linkage score are computed as

$${\mathrm{Rel}}_g\left(s_i\right)=\cos \left(\mathbf{h}\left(g\right),\mathbf{h}\left(s_i\right)\right)$$

(3)

where $\mathbf{h}\left(g\right)$ and $\mathbf{h}\left(s_i\right)$ are obtained using the same MWP-BERT encoding and mean-pooling strategy.

$$\mathrm{Link}\left(s_i\right)=\underset{s_j\in S,j\ne i}{\max }{\displaystyle \frac{|\mathrm{Sig}\left(s_i\right)\cap \mathrm{Sig}\left(s_j\right)|}{|\mathrm{Sig}\left(s_i\right)\cup \mathrm{Sig}\left(s_j\right)| + ϵ}}$$

(4)

where $ϵ$ is a small constant used to avoid division by zero. The final filtering score is

$$\mathrm{Score}\left(s_i\right)=\alpha {\mathrm{Rel}}_g\left(s_i\right)+(1-\alpha )\mathrm{Link}\left(s_i\right)$$

(5)

After candidate pre-screening, each candidate scene is assigned a filtering score. A candidate scene is retained if its filtering score is not lower than the threshold $\tau$. In addition, a bridge candidate whose sub-goal is used as an intermediate condition by another scene retained by the filtering score is retained even when its filtering score is lower than $\tau$. This rule protects intermediate scenes that are weakly connected to the final question but are explicitly consumed by later reasoning steps. In this paper, $\alpha$ is set to 0.7 and $\tau$ is set to 0.6 (the diagnostic sensitivity analysis is discussed in Section 4.2.4). If all candidate scenes are removed, the highest-scoring scene is retained as a conservative fallback for scene filtering.

The complete process is summarized in Procedure 1.

Procedure 1: Scene Decomposer
1	Input: a problem text T and a unified decomposing query $Q_d$;
2	Output: a filtered scene list $S^{*}$;
3	Step 1: Exemplar Retrieval
4	▹ Use MWP-BERT to encode T and obtain $\mathbf{h}\left(T\right)$;
5	▹ Load the exemplar dataset $\mathcal{D}={\left\{(O_i,C_i)\right\}}_{i=1}^m$;
6	▹ For each exemplar $O_i$, compute $\mathrm{sim}(T,O_i)$ using Equation (2);
7	▹ Retrieve the top-K exemplars according to $\mathrm{sim}(T,O_i)$;
8	Step 2: Candidate Scene Generation
9	▹ Build the few-shot prompt using $Q_d$, the retrieved exemplars, and T;
10	▹ Input the prompt into the LLM and generate a candidate scene list $S={\left\{s_i\right\}}_{i=1}^{n_s}$;
11	Step 3: Goal-Guided Scene Filtering
12	▹ Extract the target question clause g and construct $\mathrm{Sig}\left(g\right)$ using entities, units, and available temporal cues;
13	▹ For each scene $s_i\in S$, construct $\mathrm{Sig}\left(s_i\right)$ in the same way;
14	▹ Apply recall-oriented candidate pre-screening to identify target-connected candidates and bridge candidates;
15	▹ Compute ${\mathrm{Rel}}_g\left(s_i\right)$, $\mathrm{Link}\left(s_i\right)$, and $\mathrm{Score}\left(s_i\right)$ using Equation (5);
16	▹ Retain candidate scenes whose filtering scores are not lower than $\tau$;
17	▹ Retain bridge candidates whose sub-goals are used as conditions by scenes retained through filtering score;
18	▹ If $S^{*}=\varnothing$, retain the highest-scoring scene in S;
19	▹ Output $S^{*}$.

3.3. Relation Extractor

3.3.1. Scene-Aware $S^2$ Method

The Relation Extractor transforms each retained scene in $S^{*}$ into a relation set for symbolic reasoning. Different from extracting relations from the original problem text, this paper performs relation extraction only on filtered scenes. This design reduces input noise and makes relation extraction more stable under perturbations.

The extractor is applied after goal-guided scene filtering. Therefore, each retained scene is associated with its scene signature and sub-goal information. The extractor uses this information as context to check whether a candidate relation is connected to the target-oriented memory or supports the scene sub-goal.

The proposed extractor extends the Scene-Aware $S^2$ method used in previous work [19]. For each retained scene $s_i^{*}\in S^{*}$, the extraction process contains three steps: (1) scene type classification; (2) explicit relation extraction by Scene-Aware $S^2$ models; (3) implicit relation instantiation by scene-type-specific templates. Since a retained scene may contain multiple quantities or clauses, explicit extraction may produce several candidate relations, not all of which are necessarily useful for the final solution. Therefore, a consistency check is used to remove relations that are locally plausible but weakly connected to the target question or to the retained scene context.

Definition 4 

(Relation). A relation is an equation-like representation that may include numbers, entities, attributes, units, and variables. A relation is called an explicit relation if it is explicitly stated in the text, and an implicit relation if it is required for solving but not explicitly stated.

Definition 5 

(Relation Slot). A relation slot is an entity-centered or attribute-centered quantitative role appearing in a relation.

Definition 6 

(Target Slot). A target slot is a relation slot corresponding to the final question, including the entity to be solved, the attribute or quantitative role to be solved, the associated unit, and the temporal cue if available.

For example, $\mathrm{Price}\left(apple\right)$, $\mathrm{Distance}\left(car\right)$, and $\mathrm{Rate}\left(worker\right)$ can be treated as relation slots when they appear in clauses. Among them, if the final question asks for the price of apples or the rate of a worker, then $\mathrm{Price}\left(apple\right)$ or $\mathrm{Rate}\left(worker\right)$ is further treated as a target slot. In addition, if temporal cues are available, they are attached to the corresponding slots to distinguish quantities with the same entity and unit but different temporal scopes.

Let g denote the target question clause. The extractor first derives the target slot set $\mathrm{Slot}\left(g\right)$ from the target entity set $E_g$, the attribute set $A_g$, the unit set $U_g$, and the temporal cue set if available. The retained scene list $S^{*}$ follows the order generated by the CoS-based scene decomposition. The relation memory ${\mathcal{M}}^{*}$ is initialized by $\mathrm{Slot}\left(g\right)$ and is updated along this scene order only after a candidate relation is retained. The scene signature $\mathrm{Sig}\left(s_i^{*}\right)$ identified during scene filtering is used as context for relation consistency checking. In the following equations, slot overlap means compatible slot overlap rather than exact string identity. Two slots are considered compatible when their entities, attributes, units, and temporal cues do not conflict.

3.3.2. Scene Type Classification

For each retained scene $s_i^{*}$, MWP-BERT first produces contextualized token representations. Let ${\mathcal{Q}}_i$ denote the set of quantity tokens in $s_i^{*}$. Following the quantity-aware classification idea in previous study [16], the extractor computes a quantity-aware scene vector

$${\mathbf{v}}_i=\sum _{q\in {\mathcal{Q}}_i}\beta \left(q\right)\mathbf{h}\left(q\right)$$

(6)

where $\mathbf{h}\left(q\right)$ is the contextual representation of quantity token q, and $\beta \left(q\right)$ is the normalized attention weight of q with respect to the scene representation. The scene type is predicted by

$${\widehat{l}}_i=\underset{l}{\mathrm{arg\; max}}{(W{\mathbf{v}}_i+b)}_l$$

(7)

where W and b are trainable parameters. The predicted scene type ${\widehat{l}}_i$ is used to select explicit relation models and implicit templates.

3.3.3. Explicit Relation Extraction

The extractor retains the syntax-semantics matching paradigm of Scene-Aware $S^2$ because it is interpretable and precise for quantitative relation extraction.

Definition 7 

(Scene-Aware $S^2$ Model). A Scene-Aware $S^2$ model is a quintuple $\delta =(L,P,K,Q;R)$, where L denotes applicable scene types, P denotes the Part-of-Speech (PoS) structure, K denotes keywords, Q denotes matching rules, and R denotes the relation template. The model pool is denoted by $\Delta ={\left\{{\delta }_j\right\}}_{j=1}^{N_m}$.

Table 2 shows representative models in the model pool. Each row is an instantiated example of a Scene-Aware $S^2$ model; the table is illustrative rather than exhaustive.

Table 2. Examples of Scene-Aware $S^2$ models.

Scene Type	Model Structure	Matched Sentence	Explicit Relation
General	n m [more than] n a = n; b = m; c = n Count(a) = Count(c) + b	Pens are 5 more than erasers.	Count(Pen) = Count(Eraser) + 5
General	n [of] n m [times] n a = n; b = n; c = m; d = n a(b) = c*a(d)	The price of durian is 5 times that of apple.	Price(durian) = 5 × Price(apple)
Change	n m q [Remain] a = n; b = m; c = q Remain(a) = b*c	The match is still 2 cm left.	Remain(match) = 2 × cm
Rate	n [average] m q [per] q a = n; b = m; c = q; d = q Rate(a) = b*(c/d)	The factory binds an average of 90 books per day.	Rate(factory) = 90 × (book/day)
Discount	n [at] m [discount] a = n; b = m Discount(a) = b	A product is sold at a 10% discount.	Discount(product) = 10%
Mixture	n [made of] n n [ratio of] q a = n; b = n; c = n; d = q Solution(a), Solute(b)/Solvent(c) = d	An alcohol is made of ethanol and water in a mass ratio of 1:5.	Solution(alcohol), Solute(ethanol) ÷ Solvent(water) = 1:5

Note: $[\ast ]$ indicates keywords in a model. a(b) denotes the attribute a of entity b.

For each retained scene $s_i^{*}$, explicit relation extraction is performed in three stages. First, only models whose applicable scene types contain ${\widehat{l}}_i$ are considered. Second, for each sentence $u_k$ in $s_i^{*}$, a PoS parser obtains its PoS structure, and the extractor matches the sentence against $(P,K)$ of each candidate model. Third, if the match succeeds, the matched entities, attributes, numbers, and units in $u_k$ are substituted into the relation template R, yielding a candidate explicit relation $r_{ij}^{exp}$.

Since a retained scene may contain multiple quantities or clauses, explicit extraction may produce several candidate explicit relations. Some candidate explicit relations may be textually plausible but disconnected from the target question or from the intermediate reasoning path. Therefore, each candidate explicit relation is checked by considering its connection to the target-oriented relation memory and sub-goal-based reasoning.

Let $\mathrm{Slot}\left(r\right)$ denote the slot set appearing in a candidate relation r. Because candidate relations are instantiated from the current scene by the extraction rules, local scene support is treated as the premise of consistency checking. The relation memory ${\mathcal{M}}^{*}$ is initialized by $\mathrm{Slot}\left(g\right)$ and is updated only after a candidate relation is retained. For retained scenes, the extractor also checks whether a candidate relation supports the scene sub-goal. Let ${\mathrm{SubgoalUse}}_i\left(r\right)=1$ if r provides a slot or quantitative condition required by the sub-goal of $s_i^{*}$ and this sub-goal is connected to the final goal or to a later retained scene; otherwise, ${\mathrm{SubgoalUse}}_i\left(r\right)=0$.

The consistency function is defined as:

$${\mathrm{Cons}}_i\left(r\right)=\mathbb{I}\left(\mathrm{Slot}\left(r\right)\cap {\mathcal{M}}^{*}\ne \varnothing \vee {\mathrm{SubgoalUse}}_i\left(r\right)=1\right)$$

(8)

Because ${\mathcal{M}}^{*}$ is initialized by the target slot set, the first condition retains relations connected to the final solving goal or to previously retained relations. The sub-goal-use condition prevents necessary intermediate relations from being removed only because they are not directly connected to the final target slot at the current step. If ${\mathrm{Cons}}_i\left(r\right)=1$, the relation is retained and ${\mathcal{M}}^{*}$ is updated by adding $\mathrm{Slot}\left(r\right)$.

3.3.4. Implicit Relation Instantiation

Some scenes cannot be solved by explicit relations alone. For scene types such as rate, discount, and mixture, additional domain-specific quantitative relations may be required even if they are not explicitly stated in the text. Therefore, implicit relations are instantiated from scene-type-specific templates.

Procedure 2: Relation Extractor

The implicit template pool is denoted as $\mathcal{T}={\left\{{\tau }_l\right\}}_{l\in \mathcal{L}}$, where each template is indexed by a scene type l and represented as ${\tau }_l=(C{C}_l,T{P}_l)$. Here, $C{C}_l$ is the concept set required by the template and $T{P}_l$ is the relation template. For the current scene type ${\widehat{l}}_i$, the extractor retrieves the corresponding template ${\tau }_{{\widehat{l}}_i}$. If no template exists for ${\widehat{l}}_i$, implicit relation instantiation is skipped for this scene.

For a retrieved template ${\tau }_{{\widehat{l}}_i}=(C{C}_i,T{P}_i)$, the extractor derives the slot candidate set ${\mathcal{E}}_i$ from the retained explicit relations $R_i^{exp}$ in the current scene. Specifically, ${\mathcal{E}}_i$ collects the slots appearing in all retained explicit relations of $s_i^{*}$. Then, each concept $cc\in C{C}_i$ is aligned to a slot candidate in ${\mathcal{E}}_i$ through deterministic slot alignment. The alignment process first applies direct text matching and alias matching between the concept name and the slot description. If several candidates remain, type and unit compatibility are used to remove incompatible candidates. When multiple compatible candidates are still available, the extractor selects the candidate with the highest contextual similarity computed from the MWP-BERT representations already used in scene classification. If no compatible candidate can be found, or if the best candidates cannot be distinguished, the template is not instantiated for the current scene.

Once all necessary concepts in $C{C}_i$ are aligned to slot candidates, the template $T{P}_i$ is converted into an implicit relation $r_i^{imp}$. Because the implicit relation is instantiated only from retained explicit relations and a scene-type-specific template, it is directly added to $R_i^{imp}$. After $r_i^{imp}$ is retained, the relation memory is updated as ${\mathcal{M}}^{*}\leftarrow {\mathcal{M}}^{*}\cup \mathrm{Slot}\left(r_i^{imp}\right)$, so that later scenes can use the newly instantiated intermediate relation.

The complete relation extraction process is summarized in Procedure 2.

3.4. Symbolic Solver

The Symbolic Solver derives the final answer from the list of relation sets produced by the Relation Extractor. Unlike direct neural answer generation, this component performs explicit symbolic reasoning over quantitative relations, so that the final answer is obtained through equation solving.

Let $\mathcal{R}=[R_1,R_2,\dots ,R_{|S^{*}|}]$ denote the ordered list of relation sets. The order of $\mathcal{R}$ follows the retained scene order generated by the Scene Decomposer. The solver maintains an answer buffer ${\mathcal{A}}^{*}$, where each element is a solved quantitative relation. For each relation set $R_i$, the solver first merges the current answer buffer into $R_i$ and obtains an augmented relation set $R_i^{*}$. Then, $R_i^{*}$ is converted into a symbolic equation system $E_i$ together with a slot-symbol mapping table $F_i$. The equation system $E_i$ is solved by a symbolic equation solver such as SymPy, and the solution is restored into answer relations $A_i$ according to $F_i$.

A relation set that cannot be solved in the current pass remains in the unresolved set and is revisited after the answer buffer is updated. This repeated traversal continues until all relation sets are resolved or no new answer can be produced in a full pass. After the iteration terminates, the solver extracts the target-compatible answers:

$$\mathcal{A}=\{a\in {\mathcal{A}}^{*}\mid \mathrm{Slot}\left(a\right)\cap \mathrm{Slot}\left(g\right)\ne \varnothing \}$$

(9)

Here, the intersection between slots follows the compatible slot matching rule used in the Section 3.3.

If $\mathcal{A}$ is empty but unresolved relation sets remain, the solver activates fallback recovery after the standard answer-passing process terminates. Fallback recovery is used for cases where the constraints required for solving are distributed across several scenes and cannot produce a new intermediate answer during the standard process. It constructs a relaxed global relation set:

$$R_{fb}=\left(\bigcup _{R_i\in \mathcal{U}}{R}_i\right)\cup {\mathcal{A}}^{*}$$

(10)

where $\mathcal{U}$ denotes the unresolved relation set collection. The solver then solves $R_{fb}$ once using the same conversion and symbolic solving process. The fallback result is accepted only when it produces a unique target-compatible answer whose slot is compatible with $\mathrm{Slot}\left(g\right)$. If no target-compatible answer is obtained, or if multiple conflicting target-compatible candidates are produced, the fallback result is not accepted.

The complete symbolic solving process is shown in Procedure 3.

Procedure 3: Symbolic Solver

4. Experiments

This section evaluates the proposed algorithm from two perspectives: solving accuracy on standard AWP datasets and robustness under input perturbations. Following previous studies on AWP solving and mathematical reasoning robustness, answer accuracy is used as the main evaluation metric, while robustness is measured by the performance gap between seed problems and their perturbed variants [6,9,13,16,20]. In addition, we conduct incremental ablation and diagnostic sensitivity analyses to examine how the perturbation-oriented mechanisms affect robustness.

4.1. Experimental Settings

4.1.1. Datasets and Perturbation Construction

Table 3 summarizes the datasets used in the experiments. GSM8K, AGG, and MAWPS are used to evaluate solving accuracy on standard AWPs. For perturbation robustness, we use two perturbation datasets: GSM-Perturb and AWP-Perturb.

Table 3. Datasets used in the experiments.

Dataset	Size	Source	Usage	Reference
GSM8K	1319	web	accuracy	[32]
AGG	1492	web, exam	accuracy	[33]
MAWPS	1987	web	accuracy	[34]
GSM-Perturb	1319/3957	web	perturbation robustness	[13]
AWP-Perturb	600/1800	web, exam	perturbation robustness	This paper

Note: For perturbation datasets, “Size” is reported as seed problems/perturbed problems. GSM-Perturb denotes the selected GSM-Plus subset containing Numerical Substitution, Problem-Understanding Reformulation, and Distractor Insertion.

GSM-Perturb denotes the selected subset of GSM-Plus used in this paper. GSM-Plus is an adversarial extension of GSM8K for evaluating the robustness of mathematical reasoning models [13]. To keep the perturbation categories consistent with GSM-Plus, we use three perturbation types: Numerical Substitution, Problem-Understanding Reformulation, and Distractor Insertion. Numerical Substitution changes numerical values and requires the answer to be recomputed. Problem-Understanding Reformulation changes the problem description while preserving the underlying mathematical structure. Distractor Insertion adds solution-irrelevant information to the problem text. GSM-Perturb contains 1319 GSM8K seed problems and $1319\times 3=3957$ perturbed variants.

AWP-Perturb (The dataset is available at: https://github.com/FerretDoll/AwpPerturb (accessed on 6 June 2026).) is constructed from sampled seed problems in AGG and MAWPS using the same three perturbation categories. During construction, answers are recomputed for Numerical Substitution, the mathematical structure is preserved for Problem-Understanding Reformulation, and the inserted sentences in Distractor Insertion are constrained to be topic-related but solution-irrelevant. The perturbation design follows the general idea of behavioral testing and adversarial evaluation in NLP, where controlled input variants are used to test whether a model preserves the expected behavior under surface-form changes or irrelevant distractions [22,23].

4.1.2. Baselines and Implementation Details

Four baseline algorithms are selected for comparison:

• WeiAlg: a Chain-of-Thought (CoT) prompting method that asks the LLM to generate intermediate natural language reasoning steps before producing the final answer [6].
• ChenAlg: a Program-of-Thoughts (PoT) prompting method that asks the LLM to generate executable programs for numerical reasoning, with the final computation performed by a program interpreter [9].
• LiuAlg: an X-of-Thought (XoT) method that switches among multiple prompting methods with a verifier for mathematical reasoning [20].
• PengAlg: the previous scene-aware solver proposed in [19], which combines LLM-based CoS prompting, relation extraction, and symbolic solving.

The proposed algorithm in our work is denoted as PROP. For all LLM-based algorithms, the base generation model is set to gpt-4 through the official OpenAI API. All LLM-based algorithms are evaluated under the same default decoding configuration. Due to the non-deterministic nature of API-based inference, each algorithm is evaluated five times on each dataset, and the results are reported as mean ± standard deviation. For a fair comparison with PengAlg, PROP uses the same MWP-BERT encoder (mwp-bert-en), exemplar retrieval setting, and $S^2$ model pool.

4.1.3. LLM Token Consumption

To report the inference cost of LLM-based components, we record the average number of LLM calls and token consumption for each algorithm. All algorithms use the same 3-shot prompting setting, with algorithm-specific output formats. For each problem, the input tokens and output tokens are obtained from the API usage records. The total token consumption is computed as the sum of input and output tokens over all LLM calls for the problem. The reported values are averaged over all evaluated problems.

Table 4 also shows the difference in LLM usage among the compared algorithms. WeiAlg, ChenAlg, PengAlg, and PROP use one LLM call per problem, while LiuAlg requires more calls because its XoT-style reasoning process involves iterative planning, reasoning, and verification. PROP has the same LLM-call and token-consumption cost as PengAlg, because both methods use the LLM only for scene generation. The additional components in PROP are implemented as local modules and do not introduce extra LLM calls.

Table 4. Average LLM calls and token consumption per problem.

Algorithm	LLM Calls	Input Tokens/Call	Output Tokens/Call	Token Consumption
WeiAlg	1.00	521	68	589
ChenAlg	1.00	613	61	674
LiuAlg	4.28	587	63	2782
PengAlg	1.00	632	75	707
PROP	1.00	632	75	707

Note: “LLM Calls” denotes the average number of LLM API calls per problem. “Input Tokens/Call” and “Output Tokens/Call” denote the average token usage per LLM call. “Token Consumption” denotes the average total number of input and output tokens consumed per problem.

4.1.4. Evaluation Metrics

The main metric for standard datasets is answer accuracy:

$$\mathrm{Acc}\left(D\right)={\displaystyle \frac{N_{\mathrm{correct}}}{N_{\mathrm{all}}}}\times 100\%$$

(11)

For perturbation robustness, we report the accuracy on seed problems, the accuracy on perturbed problems, and the robustness drop:

$$\mathrm{Drop}=\mathrm{Acc}\left(D_{\mathrm{seed}}\right)-\mathrm{Acc}\left(D_{\mathrm{pert}}\right)$$

(12)

This performance-drop-based setting follows GSM-Plus and is also consistent with previous robustness evaluations that test mathematical reasoners under irrelevant context or controlled input variations [13,14].

4.2. Experimental Results

4.2.1. Evaluation on Accuracy

The reported values are averaged over five runs. As shown in Table 5, PROP achieves the highest average accuracy across the three standard datasets. Compared with PengAlg, PROP improves the average accuracy from 91.0% to 92.8%, with the largest gain observed on GSM8K. On AGG and MAWPS, where the baseline accuracies are already relatively high, the improvements are smaller but still consistent. These results suggest that the perturbation-oriented mechanisms do not noticeably reduce solving accuracy on clean AWPs and can provide moderate gains under the standard evaluation setting.

Table 5. Solving accuracy on standard AWP datasets.

Dataset	WeiAlg	ChenAlg	LiuAlg	PengAlg	PROP
GSM8K	82.6 ± 0.6	81.5 ± 0.2	84.3 ± 0.5	85.6 ± 0.5	89.1 ± 0.2
AGG	90.2 ± 0.3	90.5 ± 0.3	91.0 ± 0.4	91.8 ± 0.4	92.8 ± 0.4
MAWPS	93.2 ± 0.4	94.0 ± 0.2	95.3 ± 0.4	95.5 ± 0.3	96.6 ± 0.1
Avg	88.7	88.7	90.2	91.0	92.8

Note: The unit of value in this table is %. Values are reported as mean ± standard deviation over five runs.

4.2.2. Evaluation on Perturbation Robustness

Table 6 reports answer-level robustness on GSM-Perturb and AWP-Perturb. For each perturbation dataset, the table compares the accuracy on seed problems with the accuracy on perturbed problems. The results are reported as mean ± standard deviation over five runs. A smaller robustness drop indicates that the solver is less sensitive to controlled perturbations.

Table 6. Answer-level robustness on perturbation datasets.

Dataset	Algorithm	Seed Acc.	Pert. Acc.	Drop
GSM-Perturb	WeiAlg	82.6 ± 0.6	71.5 ± 0.3	11.1
	ChenAlg	81.5 ± 0.2	70.6 ± 0.3	10.9
	LiuAlg	84.3 ± 0.5	73.7 ± 0.4	10.6
	PengAlg	85.6 ± 0.5	72.9 ± 0.1	12.7
	PROP	89.1 ± 0.2	80.8 ± 0.3	8.3
AWP-Perturb	WeiAlg	91.8 ± 0.4	80.8 ± 0.4	11.0
	ChenAlg	92.5 ± 0.3	81.6 ± 0.4	10.9
	LiuAlg	93.0 ± 0.2	82.7 ± 0.5	10.3
	PengAlg	93.2 ± 0.1	81.1 ± 0.3	12.1
	PROP	94.3 ± 0.2	87.5 ± 0.4	6.8

Note: “Pert. Acc.” denotes the accuracy on perturbed problems. “Drop” denotes the absolute accuracy decrease from seed problems to perturbed problems.

PROP achieves the highest perturbed accuracy and the smallest robustness drop on both perturbation datasets. Compared with PengAlg, the perturbed accuracy increases from 72.9% to 80.8% on GSM-Perturb and from 81.1% to 87.5% on AWP-Perturb. The corresponding robustness drops are reduced from 12.7% to 8.3% and from 12.1% to 6.8% percentage points, respectively. Since the gain on perturbed problems is more evident than the gain on seed problems, the improvement is mainly related to perturbation handling rather than only to clean-data solving ability. Compared with direct LLM prompting methods, PROP also maintains a smaller accuracy drop, which indicates that filtered scene decomposition and symbolic relation solving help limit the influence of irrelevant or rephrased problem information in this experimental setting.

4.2.3. Ablation Study

The ablation study examines how the previous scene-aware solver PengAlg changes when the perturbation-oriented mechanisms are added step by step. PengAlg + SF adds goal-guided scene filtering before relation extraction; PengAlg + SF + GC further adds relation consistency checking based on target-oriented memory and sub-goal-based reasoning; and PROP denotes the complete method with scene filtering, relation consistency checking, and fallback recovery. The experiments are conducted on GSM-Perturb and AWP-Perturb, and seed accuracy, perturbed accuracy, and robustness drop are reported.

Table 7 shows that the main robustness gain comes from the two stages before symbolic solving. After goal-guided scene filtering is added to PengAlg, perturbed accuracy increases on both GSM-Perturb and AWP-Perturb, which means that removing weakly connected or distractor-induced scenes before relation extraction is beneficial under input perturbations. Relation consistency checking brings a further improvement by preventing candidate explicit relations from being retained only because they are locally plausible. This result supports the use of target-oriented memory and sub-goal reasoning for reducing the propagation of irrelevant relations. The complete PROP model gives an additional but smaller gain, which is consistent with the role of fallback recovery as a conservative recovery step rather than the main solving path.

Table 7. Incremental ablation study on perturbation datasets.

Variant	GSM-Perturb			AWP-Perturb
	Seed	Pert.	Drop	Seed	Pert.	Drop
PengAlg	85.6 ± 0.4	72.9 ± 0.1	12.7	93.2 ± 0.1	81.1 ± 0.3	12.1
PengAlg + SF	87.0 ± 0.3	76.5 ± 0.4	10.5	93.8 ± 0.2	84.5 ± 0.2	9.3
PengAlg + SF + GC	88.2 ± 0.1	79.7 ± 0.3	8.5	94.2 ± 0.3	87.3 ± 0.2	6.9
PROP	89.1 ± 0.2	80.8 ± 0.3	8.3	94.3 ± 0.2	87.5 ± 0.4	6.8

Note: SF denotes goal-guided scene filtering, and GC denotes relation consistency checking.

4.2.4. Diagnostic Sensitivity Analysis for Scene Filter

To further examine how the goal-guided scene filter behaves under different parameter settings, Table 8 reports a diagnostic sensitivity analysis of $\alpha$ and $\tau$. The parameter $\alpha$ controls the balance between target relevance and scene linkage, while $\tau$ controls the threshold for retaining candidate scenes. We report seed accuracy, perturbation accuracy, robustness drop, and the average number of retained scenes to show how different filtering settings affect both answer robustness and scene retention.

Table 8. Diagnostic sensitivity analysis of scene-filtering parameters.

$\mathit{\alpha }$	$\mathit{\tau }$	Seed Acc.		Pert. Acc.		Drop		Avg. Retained Scenes
		GSM-P	AWP-P	GSM-P	AWP-P	GSM-P	AWP-P	GSM-P	AWP-P
0.6	0.5	88.5 ± 0.5	93.6 ± 0.3	76.4 ± 0.4	82.0 ± 0.3	12.1	11.6	3.61	2.34
	0.6	89.0 ± 0.3	93.9 ± 0.4	77.8 ± 0.2	83.4 ± 0.2	11.2	10.5	3.48	2.25
	0.7	89.1 ± 0.4	94.2 ± 0.3	80.2 ± 0.3	86.2 ± 0.4	8.9	8.0	3.45	2.19
0.7	0.5	89.2 ± 0.2	94.2 ± 0.4	78.7 ± 0.2	84.6 ± 0.1	10.5	9.6	3.42	2.21
	0.6	89.1 ± 0.2	94.3 ± 0.2	80.8 ± 0.3	87.5 ± 0.4	8.3	6.8	3.36	2.10
	0.7	88.6 ± 0.1	93.5 ± 0.2	79.5 ± 0.3	85.5 ± 0.4	9.1	8.0	3.29	1.99
0.8	0.5	89.1 ± 0.3	94.4 ± 0.2	80.9 ± 0.4	87.2 ± 0.1	8.2	7.2	3.31	2.05
	0.6	88.4 ± 0.2	93.3 ± 0.3	78.1 ± 0.1	83.6 ± 0.4	10.3	9.7	3.14	1.99
	0.7	87.3 ± 0.2	92.7 ± 0.1	76.5 ± 0.3	82.6 ± 0.3	10.8	10.1	3.07	1.96

Note: Drop denotes the accuracy difference between seed problems and perturbed problems. GSM-P and AWP-P are abbreviations for GSM-Perturb and AWP-Perturb, respectively.

The results show that the filtering behavior changes with both $\alpha$ and $\tau$. When $\tau$ is relatively small, more scenes are retained, but the perturbed accuracy is not always improved, suggesting that some distractor-induced scenes may enter relation extraction. When $\tau$ is too large, the average number of retained scenes decreases, but seed accuracy and perturbed accuracy may also decline, suggesting that some necessary intermediate scenes may be removed. The parameter $\alpha$ shows a similar trade-off. A larger $\alpha$ increases the influence of target relevance and usually reduces the number of retained scenes, but an overly large value may weaken the preservation of bridge scenes. Overall, $\alpha =0.7$ and $\tau =0.6$ provide a near-best setting on evaluated datasets, with relatively small robustness drops and a moderate number of retained scenes.

4.3. Case Study

To further analyze how the proposed mechanisms affect the solving process, this subsection presents both successful and failed cases. The successful cases illustrate how PROP handles Numerical Substitution, Problem Understanding, Distractor Insertion, and a fallback-recovery scenario. The failure cases further discuss representative boundary cases where the current method may still fail.

4.3.1. Successful Cases

As shown in Table 9, both PengAlg and PROP correctly solve the numerically perturbed problem. This result is consistent with the characteristics of relation-based solvers. The perturbation changes the values from “5 sprints, 4 times, 80 m” to “3 sprints, 3 times, 60 m”, but the underlying rate relation is unchanged. Once the rate scene is correctly identified, both algorithms can substitute the new numerical values into the same relation structure and obtain the correct answer. Therefore, numerical substitution alone is not the main source of errors for relation-flow algorithms; more challenging cases usually arise when the perturbation changes the linguistic expression of key relations or introduces irrelevant but locally plausible relations.

Table 9. Case study on numerical substitution.

Seed AWP Text	James decides to run 5 sprints 4 times a week. He runs 80 m each sprint. How many total meters does he run a week?
Perturbed AWP Text	James decides to run 3 sprints 3 times a week. He runs 60 m each sprint. How many total meters does he run a week?
Solution
PengAlg & PROP	1. {Type: Rate, Relation Set: {Amount(James) = (3*3)*sprint, Rate(James) = 60*(meter/sprint)}, Answer: {Total(James) = (3*3)*60 = 540*meter)}}; Final Answer: James runs 540 m a week. (Correct)

As shown in Table 10, the seed problem states the paired pricing relation explicitly: one glass in each pair costs HKD 5 and the other costs HKD 3. Therefore, the relation structure can be directly extracted. In the perturbed problem, the same mathematical structure is preserved, but the price of the second glass is paraphrased as “60% of the regular price”. This rewriting makes an originally explicit price relation become implicit. PengAlg extracts the regular unit price and the total number of glasses, but fails to instantiate the paired pricing relation under the paraphrased expression. As a result, it treats all 16 glasses as regular-price glasses and obtains the incorrect answer. In contrast, PROP decomposes the perturbed problem into connected scenes: computing the discounted price, computing the price of one pair, and computing the total price. This scene-level decomposition helps preserve the implicit pairing relation under paraphrase, allowing the relation extractor and symbolic solver to derive the correct total cost.

Table 10. Case study on problem-understanding reformulation.

Seed AWP Text	Kylar bought 16 glasses for his apartment. The glasses are sold in pairs. In each pair, one glass costs HKD 5 and the other glass costs HKD 3. How much should Kylar pay in total?
Perturbed AWP Text	Kylar bought 16 glasses for his apartment. The regular price of each glass is HKD 5. However, for every pair of glasses, one glass is charged at the regular price and the other is charged at only 60% of that price. How much should Kylar pay in total?
Solution
PengAlg	1. {Type: Rate, Relation Set: {Rate(price) = 5*(dollar/glass), Amount(glass) = 16}, Answer: {Total(price) = 16*5 = 80*dollar}}; Final Answer: Kylar needs to pay HKD 80. (Incorrect)
PROP	1. {Type: General, Relation Set: {Price(Regular) = 5*dollar, Price(Other) = Price (Regular)*60%}, Answer: {Price(Other) = 5*60% = 3*dollar}}; 2. {Type: Part-Whole, Relation Set: {Part(Regular Price) = 5*dollar, Part(Other Price) = 3*dollar}, Answer: {Whole(Pair Price) = 5 + 3 = 8*dollar}}; 3. {Type: Rate, Relation Set: {Rate(glass) = 8*(dollar/pair), Amount(glass) = (16/2)*pair}, Answer: {Total(glass) = 8*(16/2) = 64*dollar}}; Final Answer: Kylar needs to pay HKD 64. (Correct)

As shown in Table 11, the irrelevant sentence “She ate 6 eggs for lunch yesterday” introduces a numerical statement that is topically related to eggs but outside the temporal scope of the target question. PengAlg incorrectly treats this sentence as an additional consumption relation for today’s eggs and subtracts it when computing the remaining eggs, which leads to an incorrect final income. In contrast, PROP retains only the scenes that support the target goal of computing today’s selling income. The breakfast and muffin events are kept because they affect the number of eggs remaining today, while the lunch event from yesterday is filtered out as irrelevant to the current goal. Therefore, PROP avoids introducing the spurious relation into symbolic reasoning and obtains the correct answer.

Table 11. Case study on distractor insertion.

Seed AWP Text	Lisa had 16 eggs today. She ate three eggs for breakfast this morning and used four eggs to bake muffins. She sold the remaining eggs at the farmers’ market for HKD 2 per egg. How much money did she make today?
Perturbed AWP Text	Lisa had 16 eggs today. She ate three eggs for breakfast this morning and used four eggs to bake muffins. She ate 6 eggs for lunch yesterday. She sold the remaining eggs at the farmers’ market for HKD 2 per egg. How much money did she make today?
Solution
PengAlg	1. {Type: Change, Relation Set: {Have(egg) = 16, Lost(breakfast egg) = 3, Lost(muffins egg) = 4, Lost(lunch egg) = 6}, Answer: {Remain(egg) = $16-6-3-4$ = 3}}; 2. {Type: Rate, Relation Set: {Amount(egg) = 3, Rate(egg) = 2*(dollar/egg)}, Answer: {Total(egg) = 3*2 = 6*dollar}}; Final Answer: Lisa made HKD 6 today. (Incorrect)
PROP	1. {Type: Change, Relation Set: {Have(egg) = 16, Lost(breakfast egg) = 3, Lost(muffins egg) = 4}, Answer: {Remain(egg) = $16-3-4$ = 9}}; 2. {Type: Rate, Relation Set: {Amount(egg) = 9, Rate(egg) = 2*(dollar/egg)}, Answer: {Total(egg) = 9*2 = 18*dollar}}; Final Answer: Lisa made HKD 18 today. (Correct)

As shown in Table 12, fallback recovery is useful when the constraints required for solving are distributed across several retained relation sets and no single relation set can independently produce a target-compatible answer. This case differs from ordinary multi-scene chain reasoning, where an earlier scene first produces a numerical intermediate answer and later scenes consume it through the answer buffer. In this example, the numbers of apples and bananas and their corresponding revenues are mutually constrained across four relation sets. The standard answer-passing process cannot directly obtain the target answer because the intermediate variables remain unresolved. Fallback recovery merges the unresolved relation sets and solves the resulting global equation system, producing the correct number of apples. This example uses explicitly stated unit prices rather than commonsense knowledge, so the recovery process depends only on extracted quantitative relations. Such coupled-equation cases are relatively infrequent in the current perturbation datasets, but they are common in classroom-style AWPs.

Table 12. Case study on fallback recovery.

AWP Text	A total of 20 fruits were sold at a market, all either apples or bananas. Each apple was sold for HKD 2, and each banana was sold for HKD 4. The total revenue was HKD 70. How many apples were sold?
Solution
Relation Sets	1. {Type: Part-Whole, Relation Set: {Part(apple) = [A], Part(banana) = [B], Whole(fruit) = 20, #Whole (fruit) = Part(apple) + Part(banana)}}; 2. {Type: Rate, Relation Set: {Amount(apple) = [A], Unit(apple sold) = 2 *(dollar/apple), Total(apple sold) = [C], #Total(apple) = Amount(apple)*Unit(apple sold)}}; 3. {Type: Rate, Relation Set: {Amount(banana) = [B], Unit(banana sold) = 4*(dollar/banana), Total (banana sold) = [D], #Total (banana sold) = Amount(banana)*Unit(banana sold)}}; 4. {Type: Part-Whole, Relation Set: {Part(apple sold) = [C], Part(banana sold) = [D], Whole(revenue) = 70*dollar, # Whole(revenue) = Part(apple sold) + Part(banana sold)}}.
Fallback recovery	The fallback relation set combines the unresolved relation sets: $\left\{\right[A]+[B]=20,[C]=2[A],[D]=4[B],[C]+[D]=70\}$. Solving the global equation system gives $\left[A\right]=5$, $\left[B\right]=15$, $\left[C\right]=10$, and $\left[D\right]=60$. Final Answer: The seller sold 5 apples. (Correct)

4.3.2. Failure Cases

This subsection presents representative boundary cases to illustrate three typical failure patterns: highly confusable distractors, non-equation relations, and long numerical-logic problems. These cases are not intended to exhaust all possible failures, but to clarify where the current pipeline may still be insufficient.

The cases in Table 13 show that highly confusable distractors are difficult when they form locally solvable relations that share the same entity type, unit, operation, or target slot with the correct reasoning path. Once such distractor relations pass scene filtering and relation checking, they can enter symbolic solving and lead to wrong results or conflicting equations. This remains a limitation of the current framework, which does not explicitly perform fine-grained scope disambiguation or cross-scene contradiction resolution.

Table 13. Failure cases caused by highly confusable distractors.

Failure Example	Failure Reason
A fruit stand has 18 red apples and 12 green apples. The red apples are packed equally into 6 sale bags. The owner also plans to pack all the apples equally into 5 sale bags for a promotion. How many red apples are in each sale bag?	The distractor shares the same entity category, unit, and packing operation with the target scene. It is, therefore, retained and its all-apples-per-bag relation enters symbolic solving, causing the solver to use a broader quantity scope than the target red-apples scope.
A boat can go 60 miles per hour in still water. The route sheet states that going 560 miles upstream takes the same time as going 640 miles downstream. For a check case, the same sheet says that going 300 miles upstream takes the same time as going 420 miles downstream. How fast is the current?	The check case forms a complete target-compatible relation set with the same upstream–downstream structure as the main route. It is retained together with the main route relation, causing symbolic solving to produce conflicting current-speed equations and target-compatible answers.

The second failure type (as shown in Table 14) is related to the coverage of the current relation representation. The Scene-Aware $S^2$ model is designed to extract equation-form quantitative relations, which are effective for many standard arithmetic operations but do not explicitly represent inequalities, ceiling operations, or integer constraints. As a result, problems involving expressions such as “at most”, “minimum”, or indivisible packages may be incorrectly converted into ordinary division or multiplication relations.

Table 14. Failure cases involving non-equation relations.

Failure Example	Failure Reason
A bus can carry at most 40 students. There are 146 students going on a trip. What is the minimum number of buses needed?	The solution requires an inequality and a ceiling operation. An equality-oriented relation may produce $146/40=3.65$ instead of the integer answer 4.
The fruit store sells apples in packs of 6. Each pack of apples costs HKD 5. A teacher needs to buy 20 apples. How much does the teacher need to pay?	The solution requires an integer pack constraint and a ceiling operation. An equality-oriented relation may produce $20/6$ packs and obtain a fractional cost, instead of inferring that 4 packs are needed and the total cost is HKD 20.

The third failure type (as shown in Table 15) involves algebraic or numerical-logic problems that do not naturally match the entity-centered scene-relation representation used in PROP. In the first example, the sentence can be solved by direct algebraic translation, but it does not provide clear real-world entities, attributes, or units for constructing relation slots. In the second example, the reasoning requires assignment search over digit positions rather than arithmetic relation extraction over scenes. These cases fall outside the main problem structure assumed by the current scene-aware pipeline.

Table 15. Failure cases involving algebraic or numerical-logic reasoning.

Failure Example	Failure Reason
7 times a number added to 5 amounts to 19 less than the product of 3 and the number. Find the number.	The problem is an algebraic word sentence over an unnamed variable and lacks stable entity-centered slots. As a result, PROP cannot construct a valid scene and an effective relation set for symbolic solving.
A three-digit code uses the digits 1, 3, and 4 exactly once. The first digit is 3 less than the last digit, and the middle digit is 2 more than the first digit. What is the code?	This case requires searching over positional digit constraints, which is outside the current scene-relation-equation pipeline.

Overall, these failure cases show that the current method is most reliable when the problem can be decomposed into entity-centered quantitative scenes and equation-like relations. Errors arise when distractors are nearly indistinguishable from target relations, when the required relation is non-equational, or when the problem is better modeled as algebraic parsing or constraint satisfaction.

5. Final Remarks

5.1. Discussion on Experiment Results

The experimental results show that PROP improves robustness under controlled perturbations while maintaining competitive solving accuracy on clean AWP datasets. On the standard datasets, PROP obtains an average accuracy of 92.8%, which indicates that the perturbation-oriented mechanisms do not weaken the basic solving ability of the scene-aware neuro-symbolic framework. This result is important because the added filtering and checking steps could otherwise remove useful information if they were too aggressive.

The larger gains appear on GSM-Perturb and AWP-Perturb, where the input problems include numerical substitution, problem-understanding reformulation, and distractor insertion. This pattern shows that the main benefit of PROP comes from perturbation handling rather than from a general increase in clean-data accuracy. In particular, goal-guided scene filtering prevents weakly connected or distractor-induced scenes from entering relation extraction, while relation consistency checking suppresses locally plausible but globally irrelevant relations before symbolic solving. These two steps reduce the chance that perturbation-induced noise will be propagated into the final equation system.

The ablation and diagnostic results further show that the robustness improvement mainly comes from the filtering and checking stages before symbolic solving. Fallback recovery provides only a smaller additional gain, which is consistent with its role as a conservative backup rather than the main solving path. The sensitivity analysis of $\alpha$ and $\tau$ also shows that scene filtering requires a balance between removing distracting scenes and preserving necessary bridge scenes. In terms of inference cost, PROP uses the same number of LLM calls as PengAlg because both methods use the LLM only for scene generation. Thus, the robustness gain is mainly obtained from local filtering, relation checking, and symbolic reasoning rather than from additional LLM queries.

5.2. Limitations

Although the proposed method improves robustness in the evaluated datasets, several limitations remain.

The first limitation is its dependence on LLM-based scene generation. If the LLM misses a necessary scene or generates a distractor scene, the later relation extraction and symbolic solving stages may not fully recover the correct reasoning path. The goal-guided filter reduces this risk, but it cannot perfectly distinguish all useful and irrelevant scenes. For example, an irrelevant sentence may still be retained when it shares the same entities, units, and temporal scope as the target question. Conversely, a necessary intermediate scene may be removed when its bridge role is not recognized.

The scene filtering and relation consistency mechanisms are also heuristic. They rely on target/scene signatures, slot compatibility, and sub-goal usage, all of which depend on the correct identification of cues such as entities or attributes. When these cues are ambiguous or incomplete, the filter and consistency checker may retain irrelevant information or suppress useful relations. The sensitivity analysis further shows that $\alpha$ and $\tau$ control the balance between retaining necessary scenes and removing distracting scenes. Although the selected setting works well on the experimental datasets, adaptive parameter selection may be needed for problems with different lengths, domains, or perturbation distributions.

Another limitation is the scalability of the current $S^2$ model pool. The method works best when a problem can be decomposed into entity-centered quantitative scenes and represented by equation-like relations. Problems involving inequalities, ceiling operations, indivisible quantities, integer constraints, or positional digit constraints are not naturally covered by the current relation templates. In these cases, even correct scene decomposition may not be sufficient, because the relation extractor may fail to instantiate the relations required for symbolic solving.

Finally, the evaluation setting and system complexity also remain limited. The current framework mainly evaluates AWPs under three perturbation categories. These perturbations cover common robustness challenges, but they do not exhaust all possible variations in educational scenarios (such as implicit commonsense assumptions and multiple valid interpretations). Furthermore, despite PROP utilizing the same number of LLM invocations as PengAlg, its process remains more intricate than the direct prompt method due to its retention of scene signatures, relation slots, and template pools. For large-scale or real-time educational applications, further optimization is necessary.

5.3. Conclusions and Future Work

This paper studies robust AWP solving under input perturbations and proposes a method for improving the robustness of Scene-Aware Neuro-Symbolic Solving for AWPs under input perturbations. The proposed method extends the previous scene-aware framework by introducing perturbation-oriented mechanisms at three levels. At the scene level, Chain-of-Scene prompting and goal-guided scene filtering are used to obtain target-related and bridge scenes. At the relation level, the Scene-Aware $S^2$ method extracts explicit and implicit relations from retained scenes, while relation consistency checking suppresses spurious relations introduced by perturbations. At the solving level, symbolic equation solving and conservative fallback recovery are combined to derive target-compatible answers.

The main contribution of this work is not to replace LLM reasoning with a purely symbolic solver, nor to rely on the LLM to directly produce the final answer. Instead, the proposed framework assigns different roles to neural and symbolic components: the LLM is used for semantic scene decomposition, while relation extraction and symbolic solving provide structured and interpretable mathematical reasoning. Experimental results on clean and perturbed datasets show that this division of labor can improve robustness against numerical substitution, paraphrasing, and irrelevant-information perturbations in the evaluated settings. The ablation and diagnostic analyses further indicate that robustness gains mainly come from reducing perturbation-induced scene noise and suppressing inconsistent relations before symbolic solving.

Future work will focus on several extensions. Scene generation and filtering can be improved through self-verification, multi-sample scene voting, and adaptive filtering strategies that better preserve bridge scenes while removing distractors. The relation representation should also be extended beyond equation-like relations to cover inequalities, integer constraints, ceiling operations, and constraint-satisfaction mechanisms. In addition, semi-automatic or data-driven construction of Scene-Aware $S^2$ models will be explored to improve the scalability of the relation model pool. We also plan to extend the proposed method to more complex educational reasoning tasks, such as solving multimodal mathematical problems like geometric figure-word problems and chart-word problems.