Skip to Content
Future InternetFuture Internet
  • Article
  • Open Access

13 September 2026

27 Pages

FinDS-Agent: A Cloud–Edge Collaborative Data Science Agent for Financial Analytics

,
,
,
,
,
and
1
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China
2
Business Analysis BU, GienTech Technology Co., Ltd., Shanghai 200032, China
3
School of Artificial Intelligence, Shanghai Normal University Tianhua College, Shanghai 201815, China
4
Department of Computer and Information Science, University of Macau, Macau, China

Abstract

Large language model agents can automate data science workflows, but cloud-centric deployment exposes sensitive context and edge-only deployment limits analytical capability. We present FinDS-Agent, a cloud–edge framework that keeps raw records and program execution at the trusted edge while providing a policy-screened, sanitized context to support cloud planning. FinDS-Agent integrates a Three-Stage Cascaded Privacy Gate (TCPG), a Multi-Dimensional Joint Router (MJR), contract-guided ToolGraph planning, edge-side verification, and bounded repair. On 222 DataSciBench tasks over three runs, FinDS-Agent achieved a 69.93% completion rate and 57.06% success rate, improving over Edge-Only by 9.50 and 5.71 percentage points while invoking the cloud for 32.27% of eligible task-runs. On FinDS-Privacy-Bench, TCPG increased sensitive-field recall from 58.20% to 98.10%; no payload-leakage event was observed in the full set (0/200; Wilson 95% CI: 0–1.8845%) or blind split (0/100; 0–3.6993%) under the specified audit and threat model. External evaluation gave pass rates of 33.2%, 53.1%, and 62.3% for Edge-Only, FinDS-Agent, and Cloud-Only on DS-1000. These empirical results support selective cloud planning while delimiting statistical, privacy, and transfer claims.

1. Introduction

Over the past decades, financial institutions have made substantial progress in enterprise data governance, particularly at the headquarters level. The centralization of core business data, construction of data warehouses, and standardization of data-management processes have established a relatively mature foundation for business intelligence and regulatory reporting [1]. Meanwhile, analytical requirements are evolving from descriptive reporting toward more complex data science workflows. Business users increasingly need to perform data cleaning and preprocessing, exploratory and statistical analysis, visualization, predictive modeling, data mining and pattern discovery, as well as model interpretation and report generation [2].
Traditionally, these workflows have relied heavily on professional data analysts and manually developed scripts. Such processes are often time-consuming, difficult to reuse, and highly dependent on the expertise of individual analysts. Recent advances in large language models (LLMs) and autonomous agents offer a potential means of reducing this dependence [3]. By integrating natural-language understanding, task planning, tool invocation, code generation, and execution feedback, LLM-based agents can transform business requirements into executable analytical workflows [4]. Data science benchmarks such as DS-1000 [5], DSBench [6], MLE-bench [7], and DataSciBench [8] have further demonstrated the potential of LLM agents in realistic programming, data analysis, and machine-learning tasks.
However, deploying data science agents at financial branches presents three interrelated challenges. First, constrained local infrastructure and limited analytical expertise restrict the ability of lightweight edge models to generate reliable code and complete complex workflows involving multi-stage preprocessing, modeling, visualization, and report generation. Second, although cloud-hosted LLMs generally provide stronger reasoning and code-generation capabilities [9], transmitting user requests, data samples, intermediate results, or raw financial records outside the institutional security domain introduces substantial privacy and regulatory risks. Finally, the probabilistic nature of LLM generation may produce hallucinated APIs, unsafe operations, incomplete artifacts, or silently incorrect analytical logic, even when the generated code executes successfully [2]. Practical financial data agents must therefore jointly address limited edge capability, unreliable model-generated programs, and sensitive-data exposure. These challenges create a fundamental deployment dilemma, as illustrated in Figure 1.
Figure 1. Edge-only versus cloud-only deployment for financial analytics.
Existing LLM-based data agents remain limited in privacy-sensitive enterprise settings. Cloud-centric approaches risk exposing sensitive context, whereas edge-only agents may lack sufficient capability for complex analytical tasks. Existing cloud–edge methods often make offloading decisions without jointly considering task intent, analytical complexity, data sensitivity, and policy constraints. Moreover, privacy control, program verification, execution recovery, and artifact validation are typically addressed separately rather than within a unified workflow. Current evaluations also provide limited support for jointly assessing analytical utility and payload-level privacy leakage [10]. These limitations reveal a research gap in selectively exploiting cloud planning while retaining raw data and program execution at the edge. FinDS-Agent addresses this gap through privacy-aware routing, constrained cloud collaboration, edge-local execution, and integrated privacy evaluation.
To address these limitations, we propose FinDS-Agent, a privacy-aware cloud–edge collaborative framework for enterprise data science that decouples cloud-assisted planning from edge-side execution. Under the evaluated policy and threat model, raw records and data-dependent execution remain in the trusted edge environment, while only a screened and sanitized context is eligible for cloud planning. The framework combines three system-level mechanisms: TCPG performs hierarchical privacy assessment; MJR considers task intent, analytical complexity, and sensitivity when selecting edge or cloud-assisted planning; and ToolGraph constrains planning through registered skills, dependencies, execution contracts, and bounded repair. The contribution is their policy-governed coordination with edge-local execution and observable-payload auditing, rather than a claim that privacy screening, routing, tool graphs, or repair are individually novel primitives.
This paper makes three primary contributions toward trustworthy cloud–edge agent systems for enterprise data science:
  • We formulate and implement a policy-governed cloud–edge workflow that separates cloud-assisted planning from edge-side data custody, verification, and execution. Its system-level contribution is the coordination of TCPG, MJR, ToolGraph, Code-Down communication, and bounded recovery under explicit privacy and artifact contracts.
  • We make the routing and privacy boundaries auditable: the MJR parameters, calibration offset, pilot composition, probability estimates, fallback rule, and exploratory sensitivity profiles are disclosed, while payload leakage is defined as a benchmark event with explicit denominators rather than as a formal security guarantee.
  • We extend the empirical evidence beyond the original internal configurations through same-model framework comparisons, DS-1000 and KramaBench evaluation, compound-fault injection, and a separate five-run nine-arm ablation. Statistical intervals and paired task-cluster analyses preserve the task, run, benchmark, and version boundaries of each experiment.
The remainder of this paper is organized as follows. Section 2 reviews related work, and Section 3 defines the system, task, and threat models. Section 4 presents FinDS-Agent and its principal mechanisms, while Section 5 describes the experimental evaluation and results and Section 6 describes the failure analysis. Finally, Section 7 summarizes the findings, limitations, and future directions.

3. Preliminaries

This section defines the system setting, data science task, privacy and threat assumptions, and the optimization problem considered in this work. The concrete mechanisms used to satisfy these requirements are presented in Section 4.

3.1. System Setting

We consider a cloud–edge system for privacy-sensitive enterprise data science, represented as
S = U , E , C , D , Π p r i v ,
where U denotes an authorized enterprise user or business application, E denotes the trusted edge environment, C denotes an external cloud planning service, D is the edge-resident dataset, and Π p r i v is the applicable enterprise privacy policy. The user submits a natural-language request and identifies an authorized data source. Authorization to access a data source does not imply that every submitted request is policy compliant; each request remains subject to privacy and execution-policy checks.
In a practical deployment, E may correspond to an on-premises server, a branch-level workstation, a private computing cluster, or an institutional data platform operating within the enterprise trust boundary. It stores the original dataset, enforces local access and outbound-communication policies, hosts the edge planner f e d g e , verifies generated programs, executes admitted code, validates the resulting artifacts, and returns them to the authorized user. The edge may access local files, databases, or institutional data services for which the user has been granted permission.
The cloud component C may be implemented as a commercial LLM API, an externally hosted foundation model, or a remote model service outside the enterprise trust boundary. It provides a cloud planner f c l o u d for planning and code generation but has no direct interface to D , the edge file system, or the local execution environment. Cloud–edge interaction is restricted to an edge-to-cloud task representation and a cloud-to-edge planning response containing a structured plan and candidate program. The admissibility condition for outbound messages is defined in Section 3.3, and the concrete cloud payload is specified in Section 4.4.3.
Accordingly, planner selection may change the source of planning and code generation but does not change data custody or the location of data-dependent execution. All admitted programs, whether generated by f e d g e or f c l o u d , are verified and executed within E . The adversarial behavior of the cloud and the remaining trust assumptions are specified separately in Section 3.3.

3.2. Task Definition

A data science task is represented as
T = q , D , A , C ,
where q is a natural-language request, D is the edge-resident dataset introduced in Section 3.1, A = { a 1 , … , a m } is the set of m required analytical artifacts, and C denotes the applicable privacy, safety, and execution constraints. Here, the calligraphic symbol C denotes task-level constraints and is distinct from the cloud participant C .
Given a candidate program c , let
y = E x e c e d g e c , D
denote its execution result in the trusted edge environment. Task success is defined as
S u c c e s s T , c = I y = v a l i d ∏ j = 1 m I S a t a j , y = 1 ,
where I ⋅ is the indicator function and S a t a j , y indicates whether the execution result satisfies the output specification of artifact a j . Therefore, a task is considered successful only when the candidate program executes correctly and produces all mandatory artifacts in their required forms.

3.3. Privacy and Threat Model

Let S c D denote the set of sensitive columns in D , including personally identifiable information, financial or business-sensitive attributes, protected attributes, and quasi-identifiers. Let V s D denote the corresponding set of raw sensitive values. For a task T , let M o u t T denote the set of messages transmitted from the edge to the cloud. An admissible cloud interaction must satisfy
∀ m ∈ M o u t T ,     R a w V a l m ∩ V s D = ⌀ ,
where R a w V a l m denotes the unmasked literal values contained in message m . Reversible masking information and other locally retained privacy metadata must not be included in M o u t T .
Equation (4) specifies an operational non-disclosure requirement rather than a formally verified privacy guarantee. Its practical satisfaction depends on the accuracy of sensitive-information detection, the construction of cloud-visible messages, and the adopted leakage-audit procedure. The cloud component C is modeled as honest-but-curious: it follows the prescribed interaction protocol and returns the requested planning results but may inspect or retain any information transmitted to it. Communication between E and C is assumed to be protected by transport-layer encryption. We trust the edge environment E , including its local storage, privacy-policy enforcement, program verifier, and execution sandbox. The cloud is assumed not to deliberately violate the interaction protocol or generate programs specifically intended to exfiltrate data. Physical compromise of the edge environment, cryptographic attacks, malicious-cloud code exfiltration, reconstruction from released aggregates, membership inference, side-channel attacks, and risks arising from the secondary use of cloud logs are outside the evaluated threat model.
An observed payload-leakage event occurs when a request enters an externally exposed processing path and the serialized cloud-visible payload retains at least one gold sensitive field or an unmasked gold sensitive value under the adopted benchmark audit. This event differs from sensitive-field recall, the benchmark action decision, sensitive content in a final local output, and a formal privacy property. Accordingly, a zero event count means only that no payload-leakage event was observed under the specified benchmark, audit procedure, and threat model. The study provides neither differential-privacy nor cryptographic or non-interference guarantees, and it does not estimate risks from reconstruction, membership inference, side channels, malicious cloud-generated exfiltration code, or secondary use of provider logs.

3.4. Problem Formulation

For each task T i , the system must determine whether the task should be rejected or planned using an edge or cloud resource. We denote this decision by
π T i ∈ B l o c k e d , E d g e , C l o u d A s s i s t ,
where π is the system-level planning decision. B l o c k e d denotes policy-based rejection, whereas E d g e and C l o u d A s s i s t denote edge-local and cloud-assisted planning, respectively. These decisions determine the source of planning rather than the execution location because all admitted programs execute within E .
Given a collection of N tasks, the system-level objective is
m a x π   J π = 1 N ∑ i = 1 N λ U U i π − λ C C i c l o u d π − λ R R i p r i v a c y π − λ L L i π ,
where U i π denotes analytical utility, C i c l o u d π denotes cloud-resource consumption, R i p r i v a c y π denotes residual privacy risk, and L i π denotes end-to-end latency. The nonnegative coefficients λ U , λ C , λ R , and λ L represent deployment-specific preferences.
The optimization is subject to
T i ⊨ / C i ⇒ π T i = B l o c k e d , π T i = C l o u d A s s i s t ⇒ R a w V a l m ∩ V s D i = ⌀ ,   ∀ m ∈ M o u t T i , π T i ≠ B l o c k e d ⇒ E x e c L o c a t i o n c i = E .
The first constraint requires a task that violates its applicable constraints to be assigned the B l o c k e d route. The second permits cloud-assisted planning only when every outbound message contains no raw sensitive value under the adopted detection model. The third requires every admitted candidate program, whether generated by the edge or cloud planner, to be executed within the trusted edge environment E . These constraints respectively formalize policy compliance, outbound non-disclosure, and edge-local execution. Section 4 specifies how FinDS-Agent operationalizes these requirements.

4. Methods

This section presents FinDS-Agent, a privacy-aware cloud–edge framework that selectively uses cloud planning while retaining raw data and data-dependent execution within the trusted edge environment. The method consists of privacy-aware task preparation, multidimensional planner selection, contract-guided planning, and edge-side verification and execution.

4.1. System Overview

Within the system setting defined in Section 3.1, FinDS-Agent operationalizes the interaction among the authorized user U , the trusted edge environment E , and the cloud planning service C . As illustrated in Figure 2, FinDS-Agent comprises three edge-resident stages and an optional cloud planner. Privacy-aware task preparation constructs a structured local context and determines what information, if any, may leave the edge. Adaptive orchestration then selects an edge or cloud-assisted planner and grounds code generation with task-specific skills and constraints. Finally, secure execution verifies generated programs, executes admissible code in a local sandbox, and applies bounded recovery when failures are detected.
Figure 2. Cloud–edge architecture of FinDS-Agent.
The central idea is to decouple planning capability from data custody and program execution. The cloud is treated as a selectively invoked planning resource rather than a default execution environment. Privacy constraints first determine whether cloud interaction is admissible; planner selection is subsequently based on task intent, analytical complexity, and residual sensitivity. Thus, cloud assistance can improve planning capability without transferring the original dataset or relocating data-dependent execution.
At runtime, the user request and authorized data sources are converted into a structured local context. TCPG either blocks the task or constructs a privacy report and sanitized context. MJR then selects the edge or cloud-assisted planner, and ToolGraph provides the selected planner with relevant skills, artifact requirements, and execution constraints. The generated program is verified and executed locally. Detectable failures may trigger bounded repair, deterministic fallback, or policy-permitted cloud replanning. The process terminates when the required artifacts have been validated and returned to the authorized user. Algorithm 1 summarizes this end-to-end procedure.
Algorithm 1 FinDS-Agent Cloud–Edge Execution Pipeline
Require:     Task T = (q, D, A , C ); privacy policy Πpriv; complete ToolGraph registry G ; MJR parameter set ΘMJR containing the fixed score coefficients and routing thresholds; repair budget K
Ensure:       Validated artifact set A ^ or a safe terminal response
        Section 4.2: Context Building and Three-Stage Cascaded Privacy Gate
     1:
  X T ← BuildLocalContext( T )
     2:
 g(q, D) ← TCPG( X T , Πpriv)                                          ▷ g = (ℓs, bblock, bagg, Ms, P s )
     3:
 if bblock = 1 then
     4:
        return SafeReject(g(q, D))
     5:
 end if
        Section 4.3: Multi-Dimensional Joint Router
     6:
 r0 ← MJR( X T , g(q, D), ΘMJR)                                      ▷ r0 ∈ {Edge, CloudAssist}
     7:
 rf ← r0
     8:
  G T ← ToolGraphGround( X T , g(q, D), r0, G )
        Planner invocation
     9:
 if r0 = CloudAssist then
    10:
         U T ← BuildUplink( P s , r0, G T )
    11:
        if OutboundSafe( U T , D, g(q, D), Πpriv) then
    12:
              c ← CloudPlan( U T )                                           ▷ sanitized context up; plan and code down
    13:
        else
    14:
              rf ← Edge
    15:
               G T ← ToolGraphGround( X T , g(q, D), rf, G )      ▷ re-ground after safe edge fallback
    16:
              c ← EdgePlan( X T , G T )
    17:
        end if
    18:
 else
    19:
        c ← EdgePlan( X T , G T )
    20:
 end if
        Section 4.4: Agent Orchestration Engine
        Edge-side verification, execution, and validation
    21:
 ( status , A ^ , e) ← VerifyRepairExecuteValidate(c, D, G T , g(q, D), K)
    22:
 if status = Success then
    23:
         return A ^
    24:
 end if
        Deterministic fallback
    25:
 cf ← DeterministicFallback( X T , G T , e)
    26:
 ( status , A ^ , e) ← VerifyExecuteValidate(cf, D, G T , g(q, D))
    27:
 if status = Success then
    28:
         return A ^
    29:
 end if
        Conditional cloud escalation
    30:
 if r0 = Edge ∧ MayEscalate(g(q, D), e) then
    31:
         G T e s c ← ToolGraphGround( X T , g(q, D), CloudAssist, G )    ▷ re-ground for cloud-assisted replanning
    32:
         U T e s c ← BuildUplink( P s , CloudAssist, G T e s c )
    33:
        if OutboundSafe( U T e s c , D, g(q, D), Πpriv) then
    34:
              rf ← CloudAssist
    35:
              cc ← CloudPlan( U T e s c )
    36:
              ( status , A ^ , e) ← VerifyRepairExecuteValidate(cc, D, G T e s c , g(q, D), K)
    37:
              if status = Success then
    38:
                     return A ^
    39:
              end if
    40:
        end if
    41:
 end if
    42:
 return SafeFailure(g(q, D), e, r0, rf)

4.2. Context Building and Three-Stage Cascaded Privacy Gate

FinDS-Agent first transforms the task defined in Equation (2) into a structured representation suitable for privacy assessment, routing, and constrained code generation. Privacy decisions are made before planner selection so that estimated cloud capability cannot override mandatory protection requirements.

4.2.1. Context Building

The edge resolves the authorized data sources into the local dataset D and extracts schema, structural, and lightweight statistical information. Raw rows may be inspected locally for profiling and privacy detection, but complete records are not incorporated into cloud-visible messages. The resulting task context is
X T = q ,   Σ D ,   Φ D ,   A ,   C ,
where q is the user request, Σ D contains the locally extracted schema and structural metadata, and Φ D contains lightweight profile information, including dataset dimensions, missing-value ratios, column cardinalities, and selected descriptive statistics. The artifact set A and constraint set C are inferred from the request and may be supplemented by the corresponding execution or benchmark adapter.
The context X T remains entirely within the trusted edge domain. It provides a common representation from which TCPG derives privacy controls, MJR derives routing features, and ToolGraph derives artifact and execution constraints. The privacy policy Π p r i v introduced in Section 3.1 instantiates the privacy-related requirements in C , including sensitive-field classifications, prohibited disclosure patterns, aggregate-only conditions, and outbound-message admission rules.

4.2.2. Three-Stage Cascaded Privacy Gate

The Three-Stage Cascaded Privacy Gate (TCPG) combines policy-guided detection, local semantic review, and query-intent arbitration. The cascade follows a recall-oriented design: deterministic rules identify high-confidence sensitive fields, while uncertain candidates are retained for local semantic analysis rather than being prematurely discarded, as shown in Figure 3.
Figure 3. Three-stage privacy assessment in TCPG.
For candidate field x j , let ρ j ∈ 0 , 1 denote its rule-based sensitivity confidence and let ρ 0 = 0.70 be the semantic-review threshold. The rule and ambiguous sets are
M r u l e = x j ∣ ρ j ≥ ρ 0 , M a m b = x j ∣ ρ j < ρ 0 .
The confidence ρ j measures the reliability of the rule-based sensitive-field classification rather than the severity of the field. A high-confidence field may therefore be assigned any policy-defined sensitivity level.
The rule scan incorporates financial-data lexicons, column-name patterns, value-pattern regular expressions, and query indicators associated with row-level or indirect disclosure. Fields in M a m b are subsequently reviewed by an on-premises 7B model using column-level metadata and a limited number of truncated values. Complete rows are not provided to the model. If M s e m denotes the fields classified as sensitive through semantic review, the final detected-field set is
M s = M r u l e ∪ M s e m .
For each ambiguous field, the local reviewer returns a structured finding containing the field identifier, sensitivity category, sensitivity level, classification rationale, and a binary sensitive-field decision. Only fields classified as sensitive are included in M s e m . The semantic stage resolves domain-specific abbreviations and contextual meanings that cannot be reliably captured by surface-form rules. Its output does not authorize cloud transmission; instead, the findings are passed to query-intent arbitration.
Query-intent arbitration jointly evaluates q and M s . Requests involving prohibited row-level disclosure, indirect disclosure, or high-risk explanation over sensitive attributes are blocked. Permitted aggregate requests over high-sensitivity data are assigned an aggregate-only restriction. The decision is expressed as
b b l o c k , b a g g = h p r i v a c y q , M s ,
where h p r i v a c y ⋅ denotes the policy-based query-intent arbitration function, b b l o c k ∈ { 0 , 1 } indicates whether the request must be rejected, and b a g g ∈ { 0 , 1 } indicates whether only aggregate processing and output are permitted.
If b b l o c k = 1 , no cloud-visible context is constructed and P s = ⊥ . Otherwise, TCPG produces
P s = q ~ ,   Σ ~ D ,   D ~ s ,   M s ,   b a g g ,
where q ~ is the redacted request, Σ ~ D is the sanitized schema, and D ~ s contains a limited number of masked samples. Reversible masking information and complete raw rows are excluded.
TCPG further derives a task-level sensitivity label l s ∈ { l o w , m e d i u m , h i g h } . Its complete output is
g q , D = l s ,   b b l o c k ,   b a g g ,   M s ,   P s .
The privacy report defines the admissible information boundary for subsequent processing. TCPG therefore determines whether and in what form a task may interact with the cloud, whereas MJR determines whether cloud assistance is analytically justified.

4.3. Multi-Dimensional Joint Router

The Multi-Dimensional Joint Router (MJR) operationalizes the planner-selection problem in Equation (5). At the system level, the resulting route is one of Blocked, Edge, or CloudAssist. The Blocked route is propagated directly from TCPG, whereas MJR selects between Edge and CloudAssist for a non-blocked task. These routes determine the source of planning rather than the execution location because all approved programs execute at the edge, as shown in Figure 4.
Figure 4. Privacy-constrained planner selection in MJR. Privacy rules are applied before the expected benefit of cloud assistance is evaluated.
Let Θ M J R denote the fixed router parameters, including the calibration offset, intent-dependent prior weights, privacy-risk coefficients, cloud-use penalties, and routing thresholds η and η m .
MJR represents task T as
x T = τ T , κ T , s T ,
where τ T denotes task intent, κ T denotes analytical complexity, and s T = l s denotes TCPG-derived sensitivity.
Task intent characterizes the requested operation, including profiling, aggregation, visualization, predictive modeling, advanced mining, and row-level retrieval. Analytical complexity is represented by κ T ∈ { L 1 , L 2 , L 3 , L 4 } . For natural-language tasks, profiling, aggregation or visualization, predictive modeling, and advanced mining are generally mapped to increasingly demanding levels. For code-oriented tasks, structural features of the request provide additional refinement.
Task intent and analytical complexity play distinct but complementary roles. Task intent identifies the requested analytical operation and determines the operation-specific routing prior, whereas analytical complexity estimates the planning and code-generation capability required to complete the task. These two dimensions are operationally coupled for natural-language tasks but are not treated as interchangeable variables. Data sensitivity imposes an orthogonal privacy constraint that determines whether cloud interaction is admissible and, if permitted, contributes to the residual privacy-risk penalty. Dataset scale and cloud-use cost are incorporated as routing penalties rather than additional task dimensions.
MJR applies privacy constraints lexicographically: a blocked task is rejected without computing a cloud-benefit score, and a high-sensitivity aggregate-only task at complexity L 1 or L 2 remains at the edge. For other admissible tasks, the estimated cloud gain is
G c l o u d = c l i p p ^ c l o u d − p ^ e d g e + ξ ,   0 ,   1 ,
where p ^ e d g e and p ^ c l o u d are capability estimates for the corresponding intent–complexity bucket, ξ = 0.10 is a fixed calibration offset, and c l i p ⋅ restricts the result to 0 , 1 .
The intent-dependent prior is
P o f f l o a d τ T = 0.45 , τ T = a d v a n c e d _ m i n i n g , 0.35 , τ T = m o d e l i n g , 0.05 , τ T ∈ { a g g r e g a t i o n , v i s u a l i z a t i o n } , 0.12 , τ T = u n r e c o g n i z e d , 0 , o t h e r w i s e .
This quantity is a fixed prior weight rather than a calibrated probability.
Residual privacy risk is approximated by
R p r i v a c y = r s s T + 0.25   I r o w τ T + m i n 0.01 M s , 0.20 ,
r s s T = 0.35 , s T = h i g h , 0.12 , s T = m e d i u m , 0.03 , s T = l o w ,
where I r o w indicates row-level disclosure intent. Blocked tasks are excluded because they have already been handled by the privacy constraint.
Let n r D and n c D denote the numbers of rows and columns in D . The cloud-use penalty is
C c l o u d D = 0.18 + 0.04   I n c D > 80 + 0.04   I n r D > 10 5 .
This term is a routing penalty rather than an observed monetary cost or runtime measurement.
The final routing score is
S d e c i s i o n = G c l o u d + P o f f l o a d τ T − R p r i v a c y − C c l o u d D .
The score approximates the utility, cloud-resource, and privacy-risk terms in Equation (6). Latency is measured empirically rather than predicted by the current router. All utility-score coefficients are fixed routing hyperparameters rather than calibrated probabilities or a jointly learned policy. Privacy prohibitions are lexicographic hard rules and cannot be overridden by utility optimization. The current three-run main experiment used the explicitly disclosed calibration offset ξ = 0.10 ; the equation and Table 1 therefore retain that value.
Table 1. Distribution of DataSciBench evaluation entries. One task may contribute to multiple categories.
The development pilot contained 240 tasks in a balanced 5 × 4 × 3 × 4 design: five intents, four analytical-complexity levels, three privacy levels, and four tasks per cell. Edge and cloud-sanitized arms were each repeated three times, yielding 1440 execution records. DataSciBench evaluation tasks were not used to select the reported coefficients. A post-hoc automated comparison of the 240 prompts and data trees against 4180 benchmark prompts flagged no overlap; because this audit was not preregistered and cannot rule out all semantic paraphrases, we describe the set as a pilot rather than an independently confirmed calibration benchmark.
For an intent–complexity bucket, let s and n be its successes and trials. The smoothed posterior mean is P = s + 2 π / n + 2 , where π = S + 0.5 / N + 1 is a fallback prior. The implementation searches, in order, (i) other intents at the same complexity, (ii) other complexities for the same intent, and (iii) all other buckets for the same arm; the first pool with at least 20 records is used, with the global pool serving as the final fallback. When n < 20 , the operational probability is the parent prior itself; otherwise, it is the smoothed posterior mean. p ^ e d g e and p ^ c l o u d are computed separately from the two arms. Their bucket difference is the cloud gain before the current main-version offset ξ is applied.
A later, versioned zero-bias router was evaluated exploratorily with three frozen profiles (privacy-priority, default, and utility-priority), each on 222 tasks over three runs. The mean CR was 77.53%, 73.14%, and 76.30%; the maximum spread was 4.39 percentage points, while initial cloud-selection rates ranged from 91.59% to 98.47%. The profile names encode policy preferences rather than calibrated risk levels. These results show observable quality–cloud-use variation and do not establish parameter invariance. Because that router removed the historical + 0.10 bias, its statistics are not attached to Table 2 and are reported separately in Appendix A (Table A3).
Table 2. Performance under different inference configurations. Results for locally evaluated configurations are mean sample SD over three runs of 222 tasks; CR/SR 95% intervals are reported in the text. The GPT-4o-2024-05-13 result is taken from the official DataSciBench evaluation and is reported only as an external capability reference.
For low-complexity mining tasks, MJR additionally defines
χ m i n i n g = I τ T = a d v a n c e d _ m i n i n g ∧ κ T ∈ { L 1 , L 2 } ∧ G c l o u d > η m ,
where η m = 0.15 . This exception applies primarily to code-oriented tasks whose complexity is refined independently of the default intent mapping.
Combining the TCPG blocking decision with the MJR planner selection, the system-level initial route is
r 0 = R o u t e T = B l o c k e d , b b l o c k = 1 , E d g e , s T = h i g h ∧ b a g g = 1 ∧ κ T ∈ { L 1 , L 2 } , C l o u d A s s i s t , S d e c i s i o n > η ∨ χ m i n i n g = 1 , E d g e , o t h e r w i s e ,
where η = 0.20 is the default routing margin. Privacy hard rules are evaluated before the score-based conditions. Consequently, cloud assistance is selected only when the combined cloud-capability gain and intent-dependent offload prior exceed the privacy-risk and cloud-use penalties by the positive margin η . Neither S d e c i s i o n nor the associated thresholds should be interpreted as calibrated success probabilities.
Capability estimates are retrieved from the most specific available intent–complexity bucket. When observations are insufficient, the router uses predefined priors. MJR is therefore a hybrid rule-based and partially frequency-calibrated policy. Each decision records its input dimensions, score components, selected route, and rationale for subsequent auditing.
The execution record stores both the initial route r 0 and the final route r f . The two values differ when an initially selected cloud route falls back to edge planning or when an edge-planned task subsequently escalates to the cloud. This distinction prevents failure-driven escalation from being interpreted as an additional routing dimension.

4.4. Agent Orchestration Engine

Once the route has been selected, the Agent Orchestration Engine generates and evaluates a task-specific program. FinDS-Agent uses Mini-Agent [29] as the underlying interaction substrate, while ToolGraph introduces domain-specific planning structure and machine-readable constraints. Mini-Agent provides the underlying agent loop, tool interface, contextual memory, and model-invocation mechanism but does not itself provide the privacy, routing, or execution controls required by FinDS-Agent. TCPG, MJR, ToolGraph grounding, PlanVerifier, Code-Down, and failure recovery are introduced around this substrate to enforce the privacy and execution requirements defined in Section 3.

4.4.1. ToolGraph-Guided Planning

ToolGraph represents analytical skills and their dependencies as a task-conditioned graph. Its nodes encode data science operations, privacy contracts, and execution adapters, while typed edges encode prerequisite and output relationships. Let G denote the complete ToolGraph registry, which contains registered analytical skills, privacy contracts, artifact specifications, execution constraints, and repair policies. For task T , ToolGraph selects and expands a task-conditioned subgraph from G , yielding the grounding context G T . For task T , ToolGraph produces
G T = V T , E T , A e x p , C e x e c , C p r i v , R r e p a i r ,
where V T and E T are the selected skill nodes and dependency edges, A e x p is the machine-readable realization of A , C e x e c , and C p r i v are derived from C , and R r e p a i r contains the applicable repair policies.
Candidate skills are scored using the request, task intent, analytical complexity, task type, sensitivity information, and the active planner-route hint. The highest-scoring analytical skill is selected as the primary node. If no candidate receives a positive score, a deterministic general-purpose analytical skill is used as the grounding fallback. Because the active route may affect planner-specific grounding, G T is reconstructed when a cloud-selected task falls back to edge planning or when an edge-planned task escalates to cloud-assisted replanning.
Graph construction proceeds by selecting the analytical skill most relevant to the task representation, recursively expanding its dependencies, and aggregating artifact, privacy, execution, and repair constraints. A privacy-contract node is included in every selected path, and the path is bounded to five nodes to limit prompt length and planning complexity. The resulting context is provided to either planner through a common interface.
ToolGraph constrains rather than predetermines program generation. The planner may produce task-specific code instead of concatenating fixed templates, but its output is subsequently checked against the graph-derived contracts. This separation preserves generative flexibility while making expected artifacts and constraints explicit.
ToolGraph grounding narrows the planning space but does not establish program safety, artifact correctness, or analytical optimality. These properties are assessed separately through PlanVerifier, sandbox execution, and post-execution artifact validation.

4.4.2. Verification and Artifact Validation

A generated program may be syntactically valid while remaining unsafe or incomplete. PlanVerifier checks Python syntax and abstract syntax tree structure, prohibited imports and external-access operations, consistency with aggregate-only and other privacy restrictions, references to expected artifacts, and registered MUST_CALL and MUST_WRITE constraints. The privacy-consistency check acts as a defense-in-depth control after TCPG rather than replacing the earlier privacy decision.
Let A r e f c ⊆ A e x p denote the expected artifacts referenced by candidate program c . Contract coverage is
C o v c = A r e f c A e x p ,     A e x p > 0 .
Post-execution artifact validation uses only the user-visible task specification and the artifacts generated in the local run directory; it does not inspect hidden benchmark answers.
The coverage score is a lightweight completeness signal rather than evidence of correctness. Execution admission depends on all blocking checks, and artifact validity is assessed again after sandbox execution. Programs with non-repairable violations, or programs that exhaust the repair budget, are denied access to the local dataset.

4.4.3. Code-Down Protocol and Failure Recovery

Cloud assistance may be selected initially by MJR or invoked after a detectable edge-side failure. In both cases, communication between E and C follows the interface defined in Section 3.1, and the cloud receives only a policy-approved representation of the task.
U T = P s ,   r ,   G T .
Before transmission, the complete serialized message must satisfy
R a w V a l U T ∩ V s D = ⌀ , redactions ∉ F i e l d s U T ,
where r e d a c t i o n s denotes the reversible masking map retained at the edge. The message is additionally checked against an outbound allowlist and scanned for recognizable sensitive values. These controls operationalize Equation (4) under the adopted detection model; they do not constitute a formal non-inference guarantee.
The Code-Down workflow is
c c l o u d = f c l o u d U T , c ⋆ = V e r i f y R e p a i r c c l o u d , A , C , g q , D , G T , y = E x e c e d g e c ⋆ , D , c ⋆ ≠ ⊥ , R e j e c t , c ⋆ = ⊥ .
Consistent with Section 3.1, f c l o u d returns a structured planning response, and c c l o u d denotes the candidate-program component of that response. The operator V e r i f y R e p a i r ⋅ performs bounded verification and repair. The cloud has no execution path to D ; every returned program is verified and executed at the edge.
Failure-Driven Recovery
Recovery is applied sequentially and is guided by observable failure evidence. A repairable static violation produces structured blocking reasons, which are combined with the applicable policies in R r e p a i r and returned to the active planner for one constraint-guided revision. The revised program must pass PlanVerifier again before entering the sandbox. If an admitted program fails during execution, its exception type, execution trace, and standard error are returned to the edge planner under a bounded retry budget. These signals provide localized evidence for repairing syntax, dependency, API-usage, execution, and artifact-generation failures.
If bounded repair does not produce an admissible and executable program, a deterministic fallback may construct a contract-oriented program for supported task categories. The fallback is also verified and executed locally. Persistent execution failure or post-execution detection of missing or malformed artifacts may trigger cloud-assisted replanning only when TCPG permits cloud interaction. Every cloud-generated or repaired program subsequently re-enters the same verification, sandbox-execution, and artifact-validation pipeline.
Recovery is triggered only by observable signals, including static-policy violations, syntax errors, prohibited operations, runtime exceptions, and missing or malformed artifacts. It therefore improves robustness against detectable failures but cannot guarantee the correctness of plausible yet semantically incorrect analytical results.

5. Experiments

We organize the evaluation around three research questions: (RQ1) What trade-off does FinDS-Agent achieve among analytical utility, cloud-resource consumption, and end-to-end latency relative to edge-only and cloud-planner-only configurations? (RQ2) How effectively does FinDS-Agent restrict observable sensitive-data disclosure during cloud-assisted planning under the adopted privacy-benchmark audit? (RQ3) How do the principal components of FinDS-Agent contribute to its analytical utility, privacy enforcement, cloud utilization, and execution reliability?

5.1. Experimental Design

5.1.1. Benchmarks

DataSciBench
DataSciBench is used to evaluate general data science capability. It covers data processing, statistical analysis, visualization, machine learning, data mining, and report generation. Each task requires the agent to interpret an analytical request, generate and execute code, and produce task-specific artifacts over tabular data.
The experimental split contains N c a p = 222 unique task prompts associated with 519 Intention–Function–Code and Task–Function–Code evaluation entries. Because one prompt may contain multiple requirements, the 519 entries are not treated as independent tasks as shown in Table 1. All 222 prompts are included in the denominators of Completion Rate and Success Rate. A timed-out, rejected, or failed task receives zero credit for unsatisfied criteria.
DataSciBench was selected because its executable and artifact-oriented evaluation directly measures the planning, code-generation, verification, and recovery capabilities targeted by FinDS-Agent. DataSciBench remains the principal capability benchmark; Section 5.2 and Appendix A therefore report separate DS-1000, KramaBench, and bounded DSBench transfer evidence without averaging their official metrics.
FinDS-Privacy-Bench
FinDS-Privacy-Bench evaluates privacy protection in Chinese financial data science. It contains 200 manually specified requests over four synthetic datasets representing personal banking, human-resource management, loan applications, and securities trading. Synthetic data avoid the use of real customer records while preserving representative schemas and privacy-sensitive task patterns.
Each request is annotated with sensitive columns, data-classification labels, an expected privacy action, and a privacy-complexity level κ i p r i v ∈ { P 1 , P 2 , P 3 , P 4 } . These levels range from direct row-level disclosure to indirect semantic-disclosure risks and are distinct from the L 1 – L 4 analytical-complexity levels used by MJR. The benchmark contains 48 P 1 , 68 P 2 , 54 P 3 , and 30 P 4 tasks. Among the 200 requests, 195 involve sensitive fields, and 20 constitute an obfuscated-column subset requiring semantic recognition.
Two annotators with experience in financial data governance independently labeled the benchmark, and disagreements were resolved by consensus. The gold actions are block, sanitize, and allow, corresponding respectively to rejection, protected or aggregate-only processing, and policy-compliant processing without additional suppression.
Only 5 of the 200 requests have the gold action allow. This count is insufficient to estimate benign-query false blocking, unnecessary intervention, or over-sanitization with useful stability. We define false blocking as predicting block for a gold-allow request; unnecessary intervention as predicting block or sanitize for a gold-allow request; and over-sanitization as removing information or functionality that policy did not require to be removed from an otherwise admissible request. Consequently, the 97.5% action accuracy is dominated by class imbalance and is not evidence of a low false-blocking rate. A separate balanced set of 100 requests (40 allow, 30 sanitize, and 30 block) was used only as a development diagnostic; it was not an independently adjudicated confirmatory test, and no performance number from it is reported as a main result.
A predefined 100-task subset was withheld from TCPG prompt refinement and used as a blind split. Because the complete 200-task results include this subset, the full-set results are reported descriptively rather than as an independent test set.

5.1.2. Compared Configurations

The DataSciBench evaluation includes four locally evaluated configurations and one externally reported capability reference:
  • Edge-Only (w/o FinDS-Agent): The local 7B model directly generates code without MJR, ToolGraph, Code-Down, or failure-driven escalation;
  • Edge-Only: FinDS-Agent is used with cloud assistance disabled, so planning, verification, repair, and execution remain at the edge;
  • FinDS-Agent (Cloud–Edge): The complete system, in which TCPG enforces privacy constraints and MJR selects edge or cloud-assisted planning;
  • Cloud-Planner-Only: MJR is bypassed and the cloud planner is used for every non-blocked task. Privacy screening, sanitized context, edge-side verification, and local execution are retained.
  • GPT-4o-2024-05-13 (Official Reference): The result of GPT-4o-2024-05-13 is taken from the official DataSciBench evaluation on the complete benchmark. It provides an external capability reference but was not reproduced under our prompting, execution, evaluator, or hardware environment. It is therefore excluded from controlled performance, cloud-efficiency, and latency comparisons.
The privacy evaluation compares Rule-Only, Full TCPG, and Cloud-Raw. Rule-Only excludes local semantic review; Full TCPG uses all three privacy stages; and Cloud-Raw intentionally bypasses privacy screening and sanitization. Cloud-Raw is used only as an unsafe privacy baseline and is not a permitted FinDS-Agent deployment mode.

5.1.3. Evaluation Metrics

Analytical utility is measured using the Completion Rate (CR) and Success Rate (SR). For task i , let n i s a t and n i a l l denote the numbers of satisfied and applicable evaluation criteria. Then
C R = 1 N ∑ i = 1 N n i s a t n i a l l ,
S R = 1 N ∑ i = 1 N I n i s a t = n i a l l .
CR measures criterion-level completion and permits partial credit, whereas SR requires all applicable criteria and artifacts to be satisfied.
Cloud Utilization Rate (CUR) measures the proportion of route-eligible task-runs that invoke the cloud planner at least once:
C U R = ∑ r = 1 R ∑ i ∈ E r I u r , i c l o u d > 0 ∑ r = 1 R E r ,
where R = 3 , E r is the set of non-blocked route-eligible tasks in run r , and u r , i c l o u d is the number of cloud-planner invocations. Initial cloud selection and failure-driven escalation are both counted, but each task-run contributes at most once.
Average Cloud-Token Consumption (CTC) is
C T C = 10 − 3 R N ∑ r = 1 R ∑ i = 1 N t r , i , i n c l o u d + t r , i , o u t c l o u d ,
where task-runs without cloud invocation contribute zero tokens. CTC is reported in thousands of tokens per task-run.
Average end-to-end latency is
L ¯ = 1 R N ∑ r = 1 R ∑ i = 1 N t r , i e n d − t r , i s t a r t .
The measurement includes context construction, privacy assessment, routing, planning, verification, repair, execution, and artifact validation.
For FinDS-Privacy-Bench, sensitive-field detection is evaluated using column-level precision, recall, and F1. Binary action accuracy evaluates whether the system correctly distinguishes tasks requiring protection (block or sanitize) from tasks that may be allowed. Privacy-Benchmark Sensitive Leak Rate is defined as
S L R p r i v = N f u l l + N p a r t i a l N P B ,
where N P B is the number of benchmark tasks, N f u l l counts sensitive tasks incorrectly allowed without protection, and N p a r t i a l counts externally processed tasks whose sanitized payload retains at least one gold sensitive field.
Column-level recall and S L R p r i v measure different properties. Recall includes sensitive fields from tasks that are subsequently blocked, whereas S L R p r i v records only disclosure along an externally exposed processing path. Thus, a missed field in a blocked task reduces recall but does not constitute payload leakage. A measured S L R p r i v of zero indicates no leakage observed under the benchmark and audit rules, not a formal zero-leakage guarantee.

5.1.4. Implementation Details

Experiments were conducted on an edge node with an NVIDIA RTX 3090 GPU (24 GB), an Intel Xeon CPU, and 128 GB RAM under Ubuntu 22.04 LTS. The implementation used Python 3.11.8, PyTorch 2.6.0, CUDA 12.4, pandas 2.2.3, NumPy 2.1.1, and scikit-learn 1.6.
Qwen2.5-7B-Instruct served as the edge planner, and the official DeepSeek-V4-flash API served as the cloud planner. Unless otherwise stated, the MJR routing margin was η = 0.20 , the mining threshold was η m = 0.15 , and the repair budget was K = 1 . MiMo-v2.5 was used as the evaluator with temperature 1, caching disabled, and semantic shim disabled.
All comparable configurations use identical task inputs, artifact requirements, execution environments, and evaluation functions. All locally evaluated configurations are evaluated in three repeated runs over the same 222 DataSciBench tasks, yielding 666 task-runs per configuration.

5.1.5. Statistical Analysis

Run-to-run variability is summarized by the sample standard deviation, and run-level 95% intervals use Student’s t distribution. Table 2 uses only run identifiers 0–2 from the same archived three-run batch as its reported means. For paired method contrasts, the unique task is the inferential unit: repeats are averaged within task, task-cluster paired bootstrap intervals are computed by resampling tasks, paired Wilcoxon tests assess signed differences, and Holm adjustment is applied within each CR or SR hypothesis family. Repeated executions are not treated as additional independent tasks or privacy scenarios. Zero-event proportions report the event count, denominator, and two-sided Wilson 95% interval. Scores from different benchmarks are kept separate rather than averaged.

5.2. Main Results

5.2.1. Analytical Utility, Cloud Efficiency, and Latency

To answer RQ1, we compare FinDS-Agent with the edge-only and cloud-planner-only configurations in terms of analytical utility, cloud utilization, cloud-token consumption, and end-to-end latency. Table 2 reports results for the four locally evaluated configurations together with the externally reported GPT-4o-2024-05-13 reference.
Introducing the FinDS-Agent workflow without cloud assistance increases CR from 55.20% to 60.43%, while SR changes from 51.05% to 51.35%. This result suggests that constrained planning and verification primarily improve criterion-level completion under the capability limit of the local model.
FinDS-Agent with cloud–edge collaboration achieves a CR of 69.93% and an SR of 57.06%, improving over Edge-Only by 9.50 and 5.71 percentage points, respectively. Only 32.27% of route-eligible task-runs invoke the cloud planner, with an average consumption of 2.82 K cloud tokens per task-run. Selective cloud assistance therefore improves utility while most eligible task-runs remain edge planned.
Cloud-Planner-Only achieves the highest CR and SR, at 80.48% and 65.32%, but invokes the cloud for every route-eligible task-run and consumes 5.81 K cloud tokens per task-run. Relative to this configuration, FinDS-Agent retains 86.9% of CR and 87.4% of SR while reducing CUR by 67.73 percentage points and CTC by 51.4%.
The official DataSciBench evaluation reports a CR of 68.44% and an SR of 66.31% for GPT-4o-2024-05-13 on the complete benchmark. These values are reported as an external capability reference. Because the model was not rerun under our prompting, execution, evaluator, or hardware settings, the result is not used for controlled performance, cloud-efficiency, or latency comparisons.
FinDS-Agent has an average latency of 20.19 s, compared with 19.17 s for Edge-Only and 16.31 s for Cloud-Planner-Only. The additional latency is associated with edge-side privacy assessment, routing, grounding, verification, and recovery. The results therefore characterize FinDS-Agent as an intermediate operating point between fully local planning and unconditional cloud planning, rather than as the highest-utility or lowest-latency configuration.
The corresponding three-run 95% t intervals for CR/SR are 51.36–59.05%/45.76–56.34% for Edge-Only without FinDS-Agent, 55.70–65.16%/48.39–54.31% for Edge-Only, 66.44–73.41%/54.24–59.87% for FinDS-Agent, and 78.82–82.13%/64.20–66.43% for Cloud-Planner-Only. These wide intervals reflect the three-run uncertainty and are not replaced by statistics from the separate ten- or five-run batches.
Under a separate same-model comparison, FinDS-Agent, Data Interpreter/MetaGPT, Open Interpreter, and a Mini-Agent v1 execution-substrate baseline processed the same 222 DataSciBench tasks with Qwen2.5-7B-Instruct, a common 900-s outer task budget, and three runs per framework (2664 task-runs). Mean CR was 66.02%, 60.02%, 19.27%, and 48.02%, respectively. The paired FinDS-Agent-minus-Data-Interpreter difference was 6.00 percentage points with a task-cluster bootstrap 95% interval of − 0.03 to 12.10 percentage points, so we do not claim statistical superiority. Internal interaction loops, model-call counts, token use, and a disclosed two-way deadline-recovery phase for unfinished Mini-Agent/Open-Interpreter cells differed; the comparison is therefore observational under the disclosed configurations, and latency, cost, token, completion, or failure differences cannot be attributed solely to architecture. Appendix A (Table A2) reports the full summary.
External evaluation tested transfer beyond DataSciBench. On DS-1000 (1000 tasks per arm), pass rates were 33.2% for Edge-Only, 53.1% for FinDS-Agent, and 62.3% for Cloud-Only. On KramaBench (104 tasks and three seeds per arm), mean official scores were 0.1971 for Edge-Only, 0.5093 for the frozen FinDS-Agent route, and 0.5368 for Cloud-Only Sanitized. A pilot-trained learned router and a random router matched at a 40.38% initial cloud budget scored 0.4503 and 0.4829; their paired difference was − 0.0325 (95% bootstrap CI: − 0.0751 to 0.0095), providing no same-budget advantage for the learned candidate. The pilot-trained router is exploratory because its training set was used for candidate selection. DSBench contained 27 cases and 306 questions, but its benchmark-informed routing arm is retained only as development evidence and not as sealed confirmation. Official scores across these benchmarks are not averaged. Appendix A (Table A4) provides the external summary.

5.2.2. Privacy Protection Under the Adopted Audit Protocol

To answer RQ2, we evaluate the privacy-protection behavior of FinDS-Agent on FinDS-Privacy-Bench. The evaluation compares the complete privacy pipeline with rule-only screening and the unsafe Cloud-Raw baseline, covering sensitive-field detection, privacy-action decisions, and observable leakage through externally exposed payloads. Table 3 reports the privacy results on FinDS-Privacy-Bench. The local semantic reviewer receives only column metadata and limited de-identified samples from the first 50 rows, rather than complete records.
Table 3. Privacy evaluation on FinDS-Privacy-Bench. Recall includes fields from blocked tasks, whereas the last column reports observed payload-leakage events and two-sided Wilson 95% intervals on externally exposed benchmark paths.
On the full benchmark, TCPG increases sensitive-field recall from 58.20% to 98.10% (+39.90 percentage points) and changes precision from 28.60% to 25.60% ( − 3.00 percentage points). The observed payload-leakage count decreases from the Rule-Only rate of 30.00% to 0/200 for Full TCPG; on the blind split it is 0/100. The two-sided Wilson 95% intervals are 0–1.8845% and 0–3.6993%, respectively. Thus, no payload-leakage event was observed under the specified benchmark, audit procedure, and threat model; the intervals and uncovered threats preclude a claim of zero true leakage probability.
The recall gain and precision loss reflect a conservative, recall-oriented policy. Because only five full-set requests are labeled allow, the benchmark cannot establish reliable false-blocking, unnecessary-intervention, or over-sanitization rates. Cloud-Raw exposes sensitive content in 195/200 requests and is an intentionally unsafe baseline, not a deployable FinDS-Agent mode.
The utility-side privacy-gate intervention was evaluated separately on DataSciBench: the configuration without privacy-gate enforcement obtained 69.99% CR and 57.06% SR, whereas full FinDS-Agent obtained 69.93% and 57.06%, a CR change of − 0.06 percentage points with unchanged SR. Because the privacy and utility changes come from different benchmarks, they do not form a same-request Pareto curve and do not identify an optimal privacy–utility operating point.
The audit covers gold sensitive fields and unmasked gold values in serialized cloud-visible payloads. It does not establish differential privacy, cryptographic confidentiality, non-interference, or safety against reconstruction, membership inference, side channels, malicious exfiltration code, short-value ambiguity, or secondary use of cloud logs. A six-category adversarial privacy set was used only for development diagnostics and is not reported as confirmatory attack performance.

5.2.3. Component Analysis

To answer RQ3, we conduct controlled component interventions to quantify the contributions of privacy-gate enforcement, ToolGraph grounding, and failure-driven recovery. Table 4 reports their effects on analytical utility, cloud utilization, token consumption, and end-to-end latency. Edge-Only and Cloud-Planner-Only are treated as deployment-boundary configurations rather than component ablations.
Table 4. Component analysis of FinDS-Agent. CR and SR are mean sample SD over the original three-run batch. The separate five-run expanded ablation is reported in Appendix A.
Removing privacy-gate enforcement produces negligible change on DataSciBench: CR changes by 0.06 percentage points and SR remains unchanged. This result indicates that privacy enforcement introduces little utility loss on the capability benchmark; it does not measure the privacy contribution of TCPG, which is evaluated separately on FinDS-Privacy-Bench.
Removing ToolGraph reduces CR by 7.55 percentage points and SR by 2.11 percentage points. The larger reduction in CR suggests that explicit artifact and dependency constraints primarily improve criterion-level completion. Because ToolGraph removal also changes cloud utilization and failure behavior, these values represent its system-level contribution rather than an isolated prompt effect.
Removing failure-driven repair produces the largest degradation: CR decreases by 11.75 percentage points and SR by 7.66 percentage points. CUR simultaneously decreases from 32.27% to 7.80% because repair-triggered cloud escalation is disabled. The observed difference therefore represents the joint contribution and resource cost of bounded repair, fallback, and conditional escalation.
Overall, the component analysis associates ToolGraph with more complete artifact production and failure recovery with robustness against detectable verification, execution, and artifact failures. The privacy contribution of TCPG is supported by the dedicated privacy benchmark rather than the DataSciBench utility ablation.
For the three matching runs, the 95% t intervals are 66.44–73.41% CR and 54.24–59.87% SR for the full system, 60.65–64.11% and 53.84–56.07% without ToolGraph, and 53.97–62.40% and 45.47–53.33% without repair. A clean matching run-level archive was not recoverable for the original 69.99% privacy-gate intervention, so no interval is imputed for that row.
The separate R15 analysis used nine arms, 222 paired tasks, and five runs (9990 task-runs). Averaging repeats within task, repair increased CR by 9.29 percentage points (bootstrap 95% CI: 6.04–12.68; Holm-adjusted p = 6.30 × 10 − 7 ) and SR by 6.85 points (3.15–10.63; adjusted p = 0.00232 ). ToolGraph increased CR by 5.21 points (1.27–9.09; adjusted p = 0.00727 ), whereas its SR effect was 0.90 points ( − 3.02 –4.82; adjusted p = 1.00 ). The CR interaction estimate was 4.33 points with a bootstrap interval crossing zero ( − 0.98 –9.55), despite an adjusted Wilcoxon p = 0.04699 ; we therefore treat the interaction as weak and inference-method dependent. The SR interaction was 1.08 points ( − 4.14 –6.40; adjusted p = 1.00 ). Raw and sanitized cloud-only arms are not interchangeable, and five payload events were observed in the sanitized cloud-only arm. Table A5 and Table A6 report the distinct five-run results and factorial effects.

6. Failure Analysis

To better understand the remaining limitations of FinDS-Agent, we manually analyzed the unsuccessful tasks and categorized them according to their primary root causes. As summarized in Table 5, functional correctness errors constitute the largest proportion of failures (52.1%), followed by result-quality deficiencies (33.2%). Together, these two categories account for more than 85% of all failure cases, whereas environment-related issues, artifact generation failures, dependency problems, and schema inconsistencies represent only a small fraction of the remaining failures.
Table 5. Distribution of failure categories observed in the experimental runs.
The analysis reveals that the primary challenge for LLM-driven data science agents is no longer execution feasibility but task-level semantic correctness. Most unsuccessful cases are not caused by runtime exceptions; instead, the generated programs can be executed but fail to satisfy the expected analytical objectives, such as incomplete analytical workflows, incorrect statistical procedures, or outputs that do not meet the evaluation requirements. This observation indicates that successful code execution does not necessarily imply successful task completion.
Furthermore, the relatively small proportion of environment- and dependency-related failures suggests that the proposed execution framework, including ToolGraph-guided tool orchestration and the verification–repair pipeline, effectively mitigates common runtime issues. The remaining limitations are mainly associated with semantic reasoning and analytical correctness, which remain challenging for current LLM-based autonomous agents.
Overall, these findings suggest that future improvements should focus on task-level verification, schema-aware semantic validation, and result-quality assessment, rather than relying solely on iterative execution repair. Such mechanisms are expected to further improve the reliability and robustness of cloud–edge collaborative data science agents.

Multi-Cause Failure Handling

The runtime processes compound failures in a bounded order: policy violations terminate or sanitize before routing; planning/tool-format errors trigger constrained replanning; code-execution and verifier/artifact errors trigger one local repair; and eligible unresolved failures may invoke a bounded sanitized cloud fallback. Once the repair/fallback budget is exhausted, the system terminates safely and retains the diagnostic record rather than treating an incomplete artifact as success.
We evaluated 1260 fault cases plus 60 controls spanning six single faults and 15 fault pairs. Of 1871 activated fault instances, micro-averaged identification F1 was 85.08%. Automatic recovery completed 805/1222 cases with an activated fault (65.88%); 198 remaining terminal cases satisfied the safe-termination contract (198/198). Under fault injection, official CR and SR were 43.43% and 28.65%, and mean CR decreased by 7.06 percentage points relative to matched controls. Masking and non-additive interactions were common, so these outcomes do not imply independent detection or recovery of every configured cause. The result demonstrates bounded recovery and safe termination under the injected conditions, not failure immunity.

7. Conclusions

This paper presented FinDS-Agent as a policy-governed cloud–edge data-science workflow in which screened cloud planning is separated from edge-side data custody, verification, and execution. The contribution is the coordinated use of TCPG, MJR, ToolGraph contracts, and bounded recovery under explicit empirical boundaries.
On the original three-run DataSciBench batch, FinDS-Agent achieved 69.93% CR and 57.06% SR while using the cloud in 32.27% of eligible task-runs. Same-model framework results were observational rather than architecture-only causal evidence. Transfer results improved over Edge-Only on DS-1000 and KramaBench but remained below cloud-only variants. The five-run factorial analysis supported CR effects for repair and ToolGraph, while the ToolGraph SR effect and both interaction interpretations were limited or inconsistent.
On FinDS-Privacy-Bench, TCPG increased sensitive-field recall to 98.10%, but precision decreased to 25.60%. No payload-leakage event was observed in the full set (0/200; Wilson 95% CI: 0–1.8845%) or blind split (0/100; 0–3.6993%) under the specified audit and threat model. This is an empirical zero-event observation, not a formal privacy guarantee. The five allowed requests cannot establish low false blocking, and the balanced privacy/attack sets remain development diagnostics.
Limitations include fixed hand-selected routing parameters; the version difference between the main ξ = 0.10 router and the exploratory zero-bias sensitivity batch; a post-hoc pilot-overlap audit; only three main-result runs; benchmark, model, and evaluator dependence; non-identical framework interaction budgets; and untested reconstruction, membership-inference, side-channel, malicious-code, and provider-log threats. Future work should preregister calibration and overlap tests, evaluate larger independently adjudicated benign/privacy sets, and compare learned and rule-based routing under identical data and cloud budgets.

Author Contributions

Conceptualization, X.D. and C.W.; Methodology, X.D., R.D. and C.W.; Software, C.W. and S.H.; Experiments, C.W., S.H. and F.Z.; Testing, C.W., S.H., and F.Z.; Writing—original draft preparation, X.D.; Writing—review and editing, R.D., S.H., F.Z., S.F. and Z.L.; Supervision, S.H., F.Z. and Z.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by grants from the National Natural Science Foundation of China (72595845, 72595840).

Data Availability Statement

DataSciBench is publicly available from its original repository. The FinDS-Privacy-Bench datasets, annotations, evaluation scripts, and anonymized experimental results are available from the corresponding author upon reasonable request. The source code and configuration files will be made publicly available upon publication.

Conflicts of Interest

Author Cheng Wang was employed by GienTech Technology Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Appendix A. Supplementary Evidence

Table A1. Component positioning and claim boundary.
Table A2. Same-model DataSciBench framework comparison (222 tasks × 3 runs per framework).
Table A3. Exploratory sensitivity of the separate zero-bias MJR version (222 tasks × 3 runs per profile).
Table A4. External benchmark and matched-routing summary.
Table A5. Separate five-run nine-arm ablation. CR interval is a run-level Student interval; cloud-call rate and payload events are descriptive.
Table A6. Task-cluster factorial effects from the five-run ablation (222 paired tasks).

References

  1. Luo, Y.; Li, G.; Fan, J.; Tang, N. Data Agents: Levels, State of the Art, and Open Problems. In Proceedings of the 2026 International Conference on Management of Data (SIGMOD-Companion), Bengaluru, India, 31 May–5 June 2026; pp. 571–579. [Google Scholar] [CrossRef] [Scilit]
  2. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; Cao, Y. ReAct: Synergizing Reasoning and Acting in Language Models. In Proceedings of the 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda, 1–5 May 2023. [Google Scholar]
  3. Schick, T.; Dwivedi-Yu, J.; Dessì, R.; Raileanu, R.; Lomeli, M.; Hambro, E.; Zettlemoyer, L.; Cancedda, N.; Scialom, T. Toolformer: Language Models Can Teach Themselves to Use Tools. Adv. Neural Inf. Process. Syst. 2023, 36, 68539–68551. [Google Scholar] [CrossRef] [Scilit]
  4. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; Yao, S. Reflexion: Language Agents with Verbal Reinforcement Learning. Adv. Neural Inf. Process. Syst. 2023, 36, 8634–8652. [Google Scholar] [CrossRef] [Scilit]
  5. Biswas, S.; Wardat, M.; Rajan, H. The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines in Theory, in-the-Small, and in-the-Large. In Proceedings of the 44th International Conference on Software Engineering (ICSE), Pittsburgh, PA, USA, 21–29 May 2022; pp. 2091–2103. [Google Scholar] [CrossRef] [Scilit]
  6. Lai, Y.; Li, C.; Wang, Y.; Zhang, T.; Zhong, R.; Zettlemoyer, L.; Yih, W.-T.; Fried, D.; Wang, S.; Yu, T. DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. Proc. Mach. Learn. Res. 2023, 202, 18319–18345. [Google Scholar]
  7. Jing, L.; Huang, Z.; Wang, X.; Yao, W.; Yu, W.; Ma, K.; Zhang, H.; Du, X.; Yu, D. DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025. [Google Scholar]
  8. Chan, J.S.; Chowdhury, N.; Jaffe, O.; Aung, J.; Sherburn, D.; Mays, E.; Starace, G.; Liu, K.; Maksin, L.; Patwardhan, T.; et al. MLE-Bench: Evaluating Machine Learning Agents on Machine Learning Engineering. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025. [Google Scholar]
  9. Zhang, D.; Zhoubian, S.; Cai, M.; Li, F.; Yang, L.; Wang, W.; Dong, T.; Hu, Z.; Tang, J.; Yue, Y. DataSciBench: An LLM Agent Benchmark for Data Science. In Proceedings of the Association for Computational Linguistics: ACL 2026, San Diego, CA, USA, 2–7 July 2026; pp. 3685–3728. [Google Scholar] [CrossRef] [Scilit]
  10. Chen, Z.; Li, S.; Smiley, C.; Ma, Z.; Shah, S.; Wang, W.Y. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates, 7–11 December 2022; pp. 6279–6292. [Google Scholar] [CrossRef] [Scilit]
  11. Li, S.; Wang, H.; Xu, W.; Zhang, R.; Guo, S.; Yuan, J.; Zhong, X.; Zhang, T.; Li, R. Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges. arXiv 2025, arXiv:2507.16731. [Google Scholar] [CrossRef] [Scilit]
  12. Ong, I.; Almahairi, A.; Wu, V.; Chiang, W.-L.; Wu, T.; Gonzalez, J.E.; Kadous, M.W.; Stoica, I. RouteLLM: Learning to Route LLMs with Preference Data. In Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025. [Google Scholar]
  13. Hao, Z.; Jiang, H.; Jiang, S.; Ren, J.; Cao, T. Hybrid SLM and LLM for Edge–Cloud Collaborative Inference. In Proceedings of the Workshop on Edge and Mobile Foundation Models (EdgeFM), Tokyo, Japan, 3–7 June 2024; pp. 36–41. [Google Scholar] [CrossRef] [Scilit]
  14. Dong, J.; Li, J.; Zheng, T.; Lin, W. HybridFlow: Resource-Adaptive Subtask Routing for Efficient Edge–Cloud LLM Inference. In Proceedings of the 43rd International Conference on Machine Learning (ICML), Seoul, Republic of Korea, 6–11 July 2026. [Google Scholar]
  15. Hollmann, N.; Müller, S.; Hutter, F. Large Language Models for Automated Data Science: Introducing CAAFE for Context-Aware Automated Feature Engineering. Adv. Neural Inf. Process. Syst. 2023, 36, 44753–44775. [Google Scholar] [CrossRef] [Scilit]
  16. Yang, J.; Jimenez, C.E.; Wettig, A.; Lieret, K.; Yao, S.; Narasimhan, K.; Press, O. SWE-Agent: Agent–Computer Interfaces Enable Automated Software Engineering. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024), Vancouver, BC, Canada, 10–15 December 2024. [Google Scholar]
  17. Fan, Z.; Gao, X.; Mirchev, M.; Roychoudhury, A.; Tan, S.H. Automated Repair of Programs from Large Language Models. In Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), Melbourne, Australia, 14–20 May 2023; pp. 1469–1481. [Google Scholar]
  18. Ghareeb, A.E.; Chang, B.; Mitchener, L.; Yiu, A.; Szostkiewicz, C.J.; Laurent, J.M.; Wright, S.M.; Razzak, M.T.; White, A.D.; Hinks, M.M.; et al. A Multi-Agent System for Automating Scientific Discovery. Nature 2026, 655, 497–505. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  19. Xia, T.; Hu, L.; Sun, Y.; Xu, M.; Xu, L.; Wang, S.; Xu, W.; Jiang, J. GraSP: Graph-Structured Skill Compositions for LLM Agents. arXiv 2026, arXiv:2604.17870. [Google Scholar] [CrossRef] [Scilit]
  20. Anthropic. Model Context Protocol Specification. Available online: https://modelcontextprotocol.io/ (accessed on 19 July 2026).
  21. Google. Agent2Agent Protocol (A2A). Available online: https://github.com/google/A2A (accessed on 19 July 2026).
  22. Gao, L.; Madaan, A.; Zhou, S.; Alon, U.; Liu, P.; Yang, Y.; Callan, J.; Neubig, G. PAL: Program-Aided Language Models. In Proceedings of the 40th International Conference on Machine Learning (ICML), Honolulu, HI, USA, 23–29 July 2023; pp. 10764–10799. [Google Scholar]
  23. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. Self-Refine: Iterative Refinement with Self-Feedback. Adv. Neural Inf. Process. Syst. 2023, 36, 46534–46594. [Google Scholar] [CrossRef] [Scilit]
  24. Evans, J.; Bratton, B. Agentic AI and the Next Intelligence Explosion. Science 2026, 391, eaeg1895. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  25. Zharmagambetov, A.; Guo, C.; Evtimov, I.; Pavlova, M.; Salakhutdinov, R.; Chaudhuri, K. AGENTDAM: Privacy Leakage Evaluation for Autonomous Web Agents. In Proceedings of the 39th Conference on Neural Information Processing Systems, Datasets and Benchmarks Track (NeurIPS), San Diego, CA, USA, 2–7 December 2025. [Google Scholar]
  26. Wang, S.; Yu, F.; Liu, X.; Qin, X.; Zhang, J.; Lin, Q.; Zhang, D.; Rajmohan, S. Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents. In Proceedings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, 4–9 November 2025; pp. 17055–17074. [Google Scholar] [CrossRef] [Scilit]
  27. El Yagoubi, F.; Badu-Marfo, G.; Al Mallah, R. AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems. IEEE Access 2026, 14, 94960–94978. [Google Scholar] [CrossRef] [Scilit]
  28. Papadopoulou, A.; Yu, Y.; Lison, P.; Øvrelid, L. Neural Text Sanitization with Explicit Measures of Privacy Risk. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (AACL-IJCNLP), Online, 20–23 November 2022; Volume 1: Long Papers, pp. 217–229. [Google Scholar] [CrossRef] [Scilit]
  29. Ferrag, M.A.; Lakas, A.; Tihanyi, N.; Debbah, M. Securing LLM Agents: From Prompt Sanitization to Autonomous Red Teaming and Beyond. Internet Things Cyber-Phys. Syst. 2025, 5, 185–209. [Google Scholar] [CrossRef] [Scilit]
  30. Chen, Y.; Zhao, J.; Tang, B.; Wang, H.; Zhang, Y.; Huang, F.; Xiong, F.; Li, Z. MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge–Cloud Agents. arXiv 2026, arXiv:2605.09530. [Google Scholar] [CrossRef] [Scilit]
  31. Li, N.; Zhang, K.; Polley, K.; Ma, J. Security Considerations for Artificial Intelligence Agents. arXiv 2026, arXiv:2603.12230. [Google Scholar] [CrossRef] [Scilit]
  32. Pilán, I.; Lison, P.; Øvrelid, L.; Papadopoulou, A.; Sánchez, D.; Batet, M. The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization. Comput. Linguist. 2022, 48, 1053–1101. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.