An RL-Enhanced Multi-Agent Framework for Scalable and Intelligent Business Intelligence Systems

Abstract

In many organizations, business intelligence systems support analytical reporting and operational decision making. As data volumes grow and analytical tasks become more complex, architectures based on centralized processing pipelines increasingly face limitations related to scalability and timely response. These challenges motivate the development of alternative architectural approaches capable of operating efficiently in data-intensive environments. This study presents a modular multi-agent business intelligence framework that distributes analytical tasks across autonomous agents and applies lightweight reinforcement learning at the decision-making stage. The analytical workflow is decomposed into agents responsible for data collection, preprocessing, analytical modeling, and decision execution. Decision adaptation relies on localized policy updates driven by operational feedback, which avoids complex learning coordination and helps preserve system stability and interpretability. The proposed framework is evaluated using real-world transactional data from an electronic commerce setting. Experimental results show that the approach consistently outperforms centralized analytical pipelines and non-agent machine learning baselines in terms of processing efficiency, classification accuracy, and balanced classification performance. Threshold-independent evaluation further confirms stronger discriminative behavior across varying decision thresholds. In addition, stability analysis across repeated experimental runs indicates reduced performance variance and more predictable system behavior. These findings suggest that the proposed multi-agent business intelligence framework provides a practical and scalable alternative to centralized analytical architectures for data-intensive decision-support environments, while maintaining the robustness and transparency required in enterprise systems. The evaluation is limited to a single dataset and a classification task, and results should be interpreted within this scope. Experiments on the Online Retail dataset (UCI Machine Learning Repository) show an average accuracy of 0.89 ± 0.012 (baseline: 0.74 ± 0.029) and decision latency of 94 ± 9 ms (baseline: 137 ± 16 ms) across 10 independent runs, indicating stable behavior under repeated execution.

1. Introduction

Business Intelligence (BI) systems occupy a central place in the information infrastructure of modern organizations, transforming heterogeneous and often poorly structured data into analytical insights needed to support management and operational decisions [1,2]. The rapid development of digital platforms, transactional services, and online business processes has led BI systems to increasingly handle data flows characterized not only by significant volumes but also by high velocity and structural diversity [3,4,5]. At the same time, traditional BI solutions, focused primarily on retrospective analysis and reporting, are increasingly demonstrating limitations in meeting the requirements of dynamic and constantly evolving data environments [6,7].

One of the key factors limiting the effectiveness of traditional BI architectures is their reliance on centralized data processing pipelines [8]. This approach simplifies system administration and control; however, as the volume and intensity of input data increases, it inevitably leads to decreased scalability and increased processing latency [9,10]. In practical scenarios, this manifests itself in delayed analytical feedback and limited system responsiveness. Such limitations are particularly critical in the fields of e-commerce, financial analytics, and industrial monitoring, where the quality of decisions directly depends on their timeliness and contextual relevance [11,12].

Although multi-agent systems have been investigated for many years, their relevance has become more apparent with the recent shift toward agent-oriented artificial intelligence. In contemporary agentic approaches, particular attention is paid to how decisions are revised when systems operate under uncertainty, rather than focusing solely on autonomy or coordination mechanisms. This perspective is especially important for business intelligence applications, where consistent system behavior, interpretability of decisions, and operational reliability are often prioritized over increasing model complexity. Unlike foundation-model-based agents designed for open-ended reasoning, BI systems typically require decision mechanisms that remain lightweight, transparent, and subject to managerial control. While large language model-based agents have recently gained attention for open-ended reasoning and autonomous planning, their direct adoption in enterprise business intelligence environments remains limited. Practical BI systems prioritize reproducibility, auditability, predictable behavior, and controlled adaptation, which are difficult to guarantee in large generative agent architectures. In this context, the proposed framework addresses a concrete and timely challenge in 2025 by demonstrating how agentic decision-making can be operationalized using lightweight, interpretable, and resource-efficient mechanisms tailored to BI workloads [13].

In this regard, recent years have seen a shift in research focus toward distributed intelligence paradigms capable of decomposing complex analytical processes into a set of simpler, yet consistently functioning components [14]. Multi-agent systems (MAS) have emerged as an effective approach for solving such problems, providing a natural mechanism for distributing functions among autonomous agents [15,16,17,18]. Assigning specialized agents to data collection, preprocessing, analytical modeling, and decision support enables parallel execution, increased fault tolerance, and gradual system scaling [19]. However, in most existing MAS-oriented BI solutions, agent behavior is still determined by static rules or predefined heuristics, which significantly limits their ability to adapt to non-stationary and changing data [20].

Existing multi-agent approaches in e-commerce and business intelligence can be broadly categorized into three groups [21]. The first group relies on rule-based agent coordination, ensuring transparency and controllability but offering limited adaptability in non-stationary environments. The second group integrates complex multi-agent reinforcement learning schemes that improve adaptability but often suffer from reduced interpretability, unstable learning dynamics, and high computational overhead. The third group embeds learning directly within analytical models, improving predictive accuracy while tightly coupling analytics and decision logic. The proposed framework differs from these approaches by introducing lightweight, decision-level reinforcement learning within a modular multi-agent BI architecture, achieving adaptive behavior while preserving transparency, stability, and operational predictability.

Reinforcement learning (RL) provides a formal and theoretically sound basis for overcoming this limitation, allowing agents to formulate decision-making strategies based on interactions with the environment and received feedback [11,12]. In multi-agent environments, such learning mechanisms allow agents to adapt local policies while simultaneously contributing to the achievement of system-level objectives [14]. However, the practical integration of RL into multi-agent architectures focused on business analytics tasks is fraught with a number of significant challenges, including issues related to policy robustness, interpretability of agent behavior, and computational costs in distributed environments [15,16]. For enterprise BI systems, these aspects are particularly important, as the reliability, predictability, and transparency of decisions are often more critical than maximizing theoretical performance metrics [19].

1.1. Research Gap and Motivation

Existing BI research follows three distinct directions: predictive model optimization, agent coordination optimization, and integrated learning pipelines. However, these approaches do not explicitly model decision adaptation as an independent architectural layer. As a result, learning components are tightly coupled with analytical inference, often reducing interpretability and operational stability in enterprise BI environments. The lack of separation between analytical prediction and operational decision adaptation remains a fundamental limitation in practical BI deployments. This study addresses this limitation by introducing a dedicated adaptive decision layer within a modular multi-agent architecture, where learning is restricted to decision policy formation while analytical agents remain deterministic. To the best of our knowledge, no prior work explicitly separates predictive modeling and decision-level adaptation within a modular multi-agent BI architecture.

To address this need, this study proposes a modular multi-agent BI framework in which reinforcement learning is applied exclusively at the decision-making stage. By assigning adaptive policy learning to the Decision-Making Agent and maintaining predefined analytical roles for the remaining agents, the framework preserves architectural transparency while enabling adaptive decision behavior. Large Language Models (LLMs) were not included in the experimental setup because the evaluation focuses on structured transactional BI data and operational decision policies rather than language generation or semantic reasoning tasks.

Based on the above considerations, this paper proposes a modular multi-agent business intelligence architecture in which learning capabilities are intentionally restricted to the decision-making stage. Reinforcement learning is applied locally within the Decision-Making Agent to support adaptive decisions, while other agents operate using predefined functional logic [20]. Unlike approaches based on complex centralized learning schemes, the proposed method emphasizes lightweight learning algorithms at the individual agent level, addressing the operational constraints of real-world BI systems [8,18,19]. Each agent operates autonomously within a strictly defined functional domain, and system coherence is ensured through structured inter-agent communication and orchestration mechanisms [16,20].

1.2. Overview of the Proposed MA-BI Framework

The proposed study addresses the above limitations by introducing a modular multi-agent business intelligence (MA-BI) framework. The architecture explicitly separates analytical inference from adaptive decision-making. Reinforcement learning is applied exclusively to decision policy formation, while analytical agents operate deterministically. This design enables adaptive behavior without compromising interpretability or operational stability, distinguishing the framework from both predictive-learning and fully learning-based multi-agent BI systems.

The main contributions of this work can be summarized as follows:

•

A modular multi-agent BI architecture is proposed that decomposes the analytical workflow into autonomous agents responsible for data collection, preprocessing, analytical modeling, and decision-making. In contrast to centralized BI pipelines, the proposed architecture distributes analytical responsibilities across independent agents, which naturally allows parallel execution and gradual scaling as data volume increases.

•

At the decision-making stage, reinforcement learning is applied in a limited form, where policy adjustments rely on operational feedback rather than complex multi-agent learning schemes. This design choice preserves system stability and interpretability while supporting adaptive behavior in dynamic data environments.

•

A comprehensive empirical evaluation is conducted on real-world transactional data, combining performance, classification quality, threshold-independent metrics, and stability analysis. The experimental results demonstrate consistent improvements over centralized BI pipelines and non-agent machine learning baselines across multiple evaluation dimensions.

•

The robustness of the proposed framework is explicitly analyzed through repeated experimental runs, highlighting reduced performance variance and more predictable behavior compared to baseline approaches. This aspect is particularly relevant for enterprise BI systems, where reliability is as important as peak analytical performance.

2. Materials and Methods

In this section, we describe how the proposed MA-BI framework was implemented and evaluated. The focus is on the system architecture, agent behavior, and learning mechanisms that support the experimental study.

2.1. System Architecture of the Proposed MA-BI Framework

The overall architecture of the proposed MA-BI framework is illustrated in Figure 1. The system follows a modular multi-agent design motivated by limitations commonly observed in centralized BI pipelines, where tightly coupled processing stages tend to become bottlenecks as data volume and processing frequency increase [22,23]. To address these issues, the BI workflow is decomposed into a set of autonomous yet coordinated agents, allowing processing stages to evolve independently.

Figure 1. Overall architecture of the proposed MA-BI framework.

The system consists of four functional agents (DCA, DPA, ANA, and DMA) coordinated by a central orchestrating agent (COA). Analytical processing is performed across specialized agents, while adaptive decision control is applied only at the final stage. Agents exchange summarized results through asynchronous communication, reducing coordination overhead and allowing analytical computations to proceed without blocking decision execution. The feedback loop enables incremental policy adjustment without retraining analytical models. In the experimental implementation, only compact decision-relevant features are transmitted between agents, and policy adaptation is performed without modifying analytical model parameters, ensuring stable behavior across repeated runs.

As depicted in Figure 1, the architecture consists of four core functional agents: the Data Collection Agent (DCA), the Data Preprocessing Agent (DPA), the Analytical Agent (ANA), and the Decision-Making Agent (DMA). Each agent is responsible for a single stage of the BI process, which simplifies system maintenance and improves scalability. This form of functional decomposition is consistent with established multi-agent system design principles, where complex analytical tasks are distributed across specialized components to reduce interdependencies [24].

The DCA is responsible for acquiring raw data from transactional databases and external sources and forwarding it to the preprocessing stage. The DPA performs data cleaning and transformation before passing structured data to the ANA. The ANA applies machine learning models to generate analytical results, which are subsequently consumed by the DMA. This sequential yet loosely coupled interaction pattern allows individual agents to be modified or extended without affecting the overall pipeline, an approach widely adopted in agent-based analytical frameworks [25].

Coordination among agents is handled by a Central Orchestrating Agent (COA). Rather than participating in analytical processing or learning, the COA monitors agent execution states and manages task transitions across the workflow. By limiting coordination responsibilities to orchestration functions, the system avoids excessive centralized control while maintaining consistent execution flow. Similar lightweight orchestration strategies have been shown to effectively balance autonomy and global supervision in distributed multi-agent architectures [26,27].

The architecture presented here should be regarded as a practical implementation-oriented solution rather than a fully optimized theoretical model. Design decisions prioritized modularity, transparency, and stable operation under realistic BI workloads, even when faster but more complex alternatives were available. Comparable architectural priorities have been emphasized in enterprise-oriented BI and decision-support systems, where maintainability and interpretability are critical requirements [28].

System boundaries. The proposed MA-BI framework operates within the analytical layer of the enterprise business intelligence pipeline. External databases and upstream data sources act only as input providers and are not modified by the framework. Reinforcement learning affects exclusively the decision policy implemented in the Decision-Making Agent and does not alter analytical model parameters, feature extraction procedures, or preprocessing logic. This separation defines the boundary between deterministic analytical processing and adaptive decision control.

2.2. Agent Functionalities and Intelligent Behavior

Within the proposed MA-BI framework, each agent is deliberately assigned a strictly limited functional role. This design choice was not purely theoretical. During early implementation and testing, we found that agents with overlapping responsibilities made the execution flow harder to trace and introduced unintended dependencies between processing stages. As a result, debugging and incremental refinement became unnecessarily complex. Restricting each agent to a single, well-defined task proved to be a more practical and manageable approach, consistent with best practices reported in modular multi-agent system design [29].

The DCA is responsible exclusively for acquiring raw transactional data from internal databases and external sources. It does not perform preprocessing or analytical operations. In practice, separating data acquisition from downstream processing helped isolate data-quality issues and allowed ingestion mechanisms to be modified without affecting analytical or decision-making components. Similar separation strategies have been shown to improve robustness in agent-based data collection architectures [30].

The DPA handles data cleaning, normalization, and feature transformation. During experimentation, preprocessing requirements varied across datasets, particularly when dealing with missing values and heterogeneous feature scales. Encapsulating these operations within a dedicated agent made it possible to adjust preprocessing strategies independently of analytical models. This isolation proved useful when testing alternative feature representations and aligns with observations reported in distributed analytical systems [31].

The ANA applies machine learning models to the preprocessed data and generates analytical outputs. In the experimental evaluation, the Analytical Agent (ANA) employs the same Random Forest classifier architecture as the non-agent baseline. Using the same classifier configuration in both the agent-based framework and the baseline ensures that performance differences are not caused by model complexity. The improvement is instead linked to how analytical outputs are interpreted and acted upon at the decision-making stage, where adaptive policies replace static threshold-based decisions. Its functionality is intentionally limited to model execution and result generation, without embedding decision logic or feedback mechanisms. This separation allowed different analytical models to be evaluated and replaced with minimal impact on the overall framework. In our experiments, this modularity simplified comparative analysis and reduced the effort required to test alternative modeling approaches, as noted in prior work on agent-based analytics [32].

Adaptive behavior is concentrated within the DMA. Rather than relying on static decision rules, the DMA incorporates a reinforcement learning mechanism that adjusts decision policies based on observed outcomes. During repeated experimental runs, we observed that this localized learning strategy led to gradual stabilization of decisions instead of abrupt behavioral changes. Such incremental adaptation is particularly desirable in enterprise decision-support environments, where predictability and interpretability are often prioritized over aggressive optimization [33].

Overall, concentrating learning capabilities within the DMA while keeping other agents functionally simple reflects a deliberate architectural trade-off. This design avoids unnecessary system complexity while still enabling adaptive decision-making. The resulting balance between autonomy and control aligns with recent findings in scalable multi-agent decision-support frameworks, where restrained learning has been shown to improve stability without sacrificing practical performance [34].

2.3. Communication and Workflow

Interaction among agents in the MA-BI framework is organized around an asynchronous, event-driven communication model. This design choice was motivated by early testing results, which showed that synchronous interactions introduced unnecessary waiting times and amplified the impact of temporary delays in individual processing stages. By decoupling agent execution through asynchronous messaging, the framework maintains responsiveness even under uneven workloads, a strategy commonly adopted in distributed and multi-agent systems [35].

Each agent publishes compact messages describing task completion status and relevant processing metadata. These messages do not carry raw datasets but instead reference intermediate results or processing states. This approach reduces communication overhead and limits tight coupling between agents, allowing each component to operate independently. Similar lightweight message-passing mechanisms have been reported to improve scalability and fault tolerance in agent-based analytical architectures [36].

Workflow progression is coordinated by the COA, which monitors incoming events and determines when subsequent processing stages should be triggered. Rather than enforcing a rigid execution order, the COA reacts to agent-generated events and manages transitions dynamically. In practice, this event-driven orchestration simplified workflow management and reduced the need for explicit synchronization logic, which often becomes a source of complexity in centralized BI pipelines [37].

To handle irregular data arrival patterns, the workflow supports non-blocking execution and partial pipeline advancement. For example, preprocessing and analytical tasks may proceed on previously available data batches while new data is still being ingested. This flexibility proved particularly useful in scenarios with bursty transactional workloads, where strict stage-by-stage synchronization would otherwise degrade system throughput. Comparable workflow strategies have been shown to enhance performance stability in data-intensive distributed environments [38].

The observed reduction in processing time primarily reflects the effects of modular separation and asynchronous coordination between agents. While the current implementation does not rely on a physically distributed computing environment, the decoupled workflow reduces blocking effects typical of centralized pipelines and provides a foundation for future distributed deployments [39].

The architecture consists of five agents (four functional and one orchestration agent). Inter-agent coordination is non-blocking and event-driven, and communication overhead remained negligible relative to total decision latency.

2.4. Reinforcement Learning Model for Adaptive Decision-Making

Adaptive decision-making in the proposed MA-BI framework is implemented exclusively within the Decision-Making Agent (DMA). This design choice was made intentionally. During preliminary experimentation, we observed that distributing learning mechanisms across multiple agents increased coordination complexity and made system behavior harder to interpret, without delivering proportional performance gains. As a result, learning capabilities were concentrated at the final decision stage, where analytical outputs are translated into concrete operational actions [40].

The decision process is formulated within the reinforcement learning (RL) paradigm, in which the DMA interacts with a dynamic environment defined by incoming analytical results and observed system responses. At each decision step $t$, the agent observes the current system state $s_i$, selects an action $a_i$, and receives a scalar reward $r_i$. The learning objective is to derive a policy that maximizes the expected cumulative reward over time, capturing both immediate decision quality and longer-term system behavior [41].

Formally, the interaction is modeled as a Markov Decision Process (MDP), defined as

$$M=⟨S,A,P,R,\gamma ⟩$$

(1)

where:

•

$S$—denotes the state space;

•

$A$—is the set of admissible actions;

•

$P$—represents the state transition dynamics;

•

$R$—is the reward function;

•

$\gamma \in (0,1)$—discount factor.

While the full transition model is not explicitly estimated, the MDP abstraction provides a structured foundation for learning decision policies in partially observable and evolving BI environments [42].

The state representation aggregates information produced by the ANA, including prediction confidence levels, recent classification outcomes, workload-related indicators, and feedback from previous decisions. Rather than modeling fine-grained system dynamics, the state is deliberately kept compact to reduce learning instability and support interpretability. Actions correspond to high-level operational decisions, such as triggering alerts, adjusting decision thresholds, or deferring decisions until additional evidence becomes available. This abstraction reflects typical decision scenarios encountered in practical BI deployments [43].

To estimate the expected utility of state–action pairs, the DMA maintains an action–value function $Q\left(s,a\right)$ which is iteratively updated based on observed experience. The update rule follows the standard Q-learning formulation:

$$Q_{t+1}(s_t,a_t)=Q_t(s_t,a_t)+\alpha (r_t+\gamma \underset{a^{\prime }}{\max }{Q}_t(s_{t+1},a^{\prime })-Q_t(s_{t+1},a))$$

(2)

where $\alpha$ denotes the learning rate. Q-learning was selected due to its conceptual simplicity, robustness under delayed feedback, and successful application in decision-support systems operating in partially observable environments [44].

The reward function is designed to reflect practical BI objectives rather than optimizing a single predictive metric. Positive rewards are associated with timely and accurate decisions, while penalties are assigned to undesirable outcomes such as false alerts, delayed responses, or unnecessary decision escalations.

$$r_t=\left\{\begin{array}{l}+1,\mathrm{if}\mathrm{the}\mathrm{decision}\mathrm{is}\mathrm{correct}\mathrm{and}\mathrm{timely}\\ -1,\mathrm{if}\mathrm{a}\mathrm{false}\mathrm{alert}\mathrm{is}\mathrm{triggered}\\ -\lambda ,\mathrm{if}\mathrm{the}\mathrm{response}\mathrm{is}\mathrm{delayed}\\ -\eta ,\mathrm{if}\mathrm{an}\mathrm{unnecessary}\mathrm{escalation}\mathrm{occurs}\end{array}\right.$$

(3)

where λ, η > 0 are penalty coefficients associated with delayed responses and unnecessary escalations, respectively.

Formal Decision Objective. To explicitly define the optimization target of the Decision-Making Agent, the learning process can be interpreted as maximization of cumulative operational utility:

$$R_t=w_1\cdot 1({\widehat{y}}_t=y_t)-w_2\cdot 1({\widehat{y}}_t=1\wedge y_t=0)-w_3\cdot D_t$$

(4)

$${\pi }^{*}=\arg \underset{\pi }{\max }{\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{t=0}^T{\gamma }^t}{R}_t|s_0\right]$$

(5)

where $1(\cdot )$ denotes the indicator function, $D_t\in [0,1]$ is the normalized response delay, and $w_1,w_2,w_3>0$ are weighting coefficients controlling the trade-off between decision correctness, false positive outcomes $1({\widehat{y}}_t=1\wedge y_t=0)$, and response latency. This formulation suggests that reinforcement learning primarily adapts the decision policy rather than altering the predictive model.

Alternative configurations in which reinforcement learning was distributed across multiple agents were evaluated during preliminary design analysis. However, such configurations led to unstable policy interactions and increased coordination complexity due to non-stationary learning dynamics. For this reason, learning was intentionally restricted to the Decision-Making Agent, limiting adaptation to the decision layer while preserving interpretability and predictable system behavior.

During experimentation, we found that this balanced reward structure encouraged conservative yet effective decision behavior, aligning learning outcomes with operational priorities commonly reported in enterprise BI systems [45].

Exploration is managed using an ε-greedy strategy, allowing the DMA to occasionally explore alternative actions while progressively favoring policies that demonstrate reliable performance. In the experimental setup, the action space of the Decision-Making Agent consists of three high-level decisions: triggering an operational alert, deferring the decision, or taking no action. Q-learning parameters were fixed across all experiments, with a learning rate α = 0.1, discount factor γ = 0.9, and an ε-greedy exploration strategy with ε gradually decayed over time. Specifically, the exploration rate ε was initialized at 0.1 and decreased according to an exponential decay schedule defined as:

$${\epsilon }_t={\epsilon }_0\cdot e^{-kt}$$

where ${\epsilon }_0$ denotes the initial exploration rate, κ represents the decay coefficient, and t is the training iteration index. In the experiments, κ was set to 0.01 to ensure a gradual transition from exploration to exploitation.

Policy updates are applied incrementally, resulting in smooth adaptation over time rather than abrupt behavioral shifts. This gradual adjustment proved important in maintaining predictable system behavior across repeated runs, which is a critical requirement for trust in automated decision-support systems [46].

Overall, by grounding the learning process in operational feedback and deliberately limiting algorithmic complexity, the proposed reinforcement learning model supports adaptive decision-making without compromising system stability or interpretability. This restrained use of reinforcement learning enables the MA-BI framework to respond to changing analytical conditions while preserving predictable and transparent decision logic, consistent with design principles advocated in recent studies on learning-based business intelligence systems [47,48].

2.5. Experimental Setup

The experimental setup was designed to evaluate the proposed MA-BI framework from both analytical and operational perspectives. The evaluation focused not only on predictive performance but also on decision quality, system responsiveness, and learning stability, which are critical factors in real-world business intelligence deployments.

2.5.1. Dataset and Data Partitioning

Experiments were conducted using a real-world e-commerce dataset consisting of transactional records, user interaction events, and time-stamped operational logs. The dataset contains heterogeneous features, including numerical transaction values, categorical product identifiers, and temporal attributes reflecting user behavior and system activity.

To preserve temporal consistency and avoid information leakage, the dataset was partitioned using a chronological split. Earlier observations were used for training and policy learning, while later observations were reserved for evaluation. This setup closely reflects practical BI deployment scenarios, where future data are unavailable at decision time.

Specifically, the selected subset of 100,000 transactions was partitioned using a chronological split, where approximately 70% of the earliest records were used for training and policy learning, and the remaining 30% were reserved for evaluation. This time-aware split reflects realistic business intelligence deployment scenarios, in which future observations are not available at decision time, and helps prevent information leakage caused by random data partitioning.

2.5.2. Baseline Configurations

The proposed MA-BI framework was compared against a centralized BI pipeline employing rule-based decision logic, as well as against a non-agent machine learning baseline implemented as a Random Forest classifier operating within a centralized, monolithic analytical pipeline. This baseline represents a strong and widely adopted learning-based approach for transactional and tabular data, without agent-based decomposition or adaptive decision-making mechanisms. All baseline methods used identical data preprocessing procedures and feature representations to ensure comparability. This design isolates the impact of multi-agent decomposition and adaptive decision-making from data-related effects.

The Random Forest classifier employed in both the non-agent baseline and the Analytical Agent was configured with 200 decision trees, a maximum tree depth of 20, and Gini impurity as the splitting criterion. These hyperparameters were selected during preliminary tuning and then fixed across all experiments to ensure reproducibility and fair comparison.

2.5.3. Evaluation Metrics

Analytical performance was assessed using standard classification metrics. Prediction accuracy is defined as

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}},$$

(6)

where denote $TP,TN,FP$ and $FN$ true positives, true negatives, false positives, and false negatives, respectively.

To account for class imbalance, precision and recall were also considered:

$$Precision={\displaystyle \frac{TP}{TP+FP}},Recall={\displaystyle \frac{TP}{TP+FN}}$$

(7)

The harmonic mean of precision and recall is reported using the F1-score:

$$F_1=2\cdot {\displaystyle \frac{Precision\cdot Recall}{TPrecision+Recall}}$$

In the above definitions, $TP$ represents instances correctly classified as belonging to the positive class, while $TN$ denotes instances correctly classified as negative. $FP$ corresponds to negative instances incorrectly classified as positive, whereas $FN$ refers to positive instances that are incorrectly classified as negative.

Operational performance was evaluated through decision latency and system throughput. Decision latency is defined as the average time required to generate a decision after receiving analytical outputs:

$$L={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N(t_i^{decision}-t_i^{input})}$$

where $N$ denotes the total number of evaluated decision events.

To analyze learning behavior, the cumulative reward obtained by the Decision-Making Agent over an evaluation horizon $T$ was computed as

$$G={\displaystyle \sum _{t=0}^T{\gamma }^t}{r}_t$$

where $r_t$ denotes the reward received at time step $t$ and $\gamma$ is the discount factor defined in Section 3.4. This measure provides insight into policy convergence and long-term decision effectiveness.

2.5.4. Experimental Procedure

Each experimental run consisted of an initial exploration phase followed by a learning phase, during which the Decision-Making Agent incrementally updated its decision policy. To reduce the influence of stochastic effects, each experiment was repeated ten times using different random seeds. Reported results correspond to mean values and standard deviations computed across these independent runs.

Hyperparameters for learning algorithms and baseline models were selected during preliminary tuning and then fixed across all experiments. All runs were executed under identical computational conditions to ensure that observed performance differences stem from architectural and algorithmic choices rather than hardware variability.

3. Results

This section presents the experimental results obtained from evaluating the proposed MA-BI framework on real-world transactional data. The results focus on system performance, classification accuracy, and decision-related metrics, providing a comparative analysis against baseline approaches.

3.1. Empirical Evaluation on Real-World Data

The empirical evaluation was conducted using the Online Retail dataset obtained from the UCI Machine Learning Repository. The dataset contains more than 541,000 transaction records collected from a real-world e-commerce environment. To support supervised classification experiments and ensure class balance, a subset of 100,000 records was selected from the original dataset.

Each record represents a single transaction and includes historical transactional attributes derived from the raw data. These features capture purchasing behavior patterns commonly analyzed in business intelligence applications. The classification task focused on predicting customer-related target classes based on the available transactional history.

This experimental setting enables a direct comparison of system performance across different analytical approaches under realistic data conditions.

Figure 2 presents a comparison of processing time across the evaluated approaches as the input data volume increases. The results indicate that the proposed MA-BI framework consistently maintains lower processing time compared to both the centralized BI pipeline and the non-agent machine learning baseline.

Figure 2. Comparison of processing time across different approaches under increasing data volume.

As the data volume grows, the processing time of the centralized BI pipeline increases steadily. In contrast, the MA-BI framework exhibits a more gradual increase in processing time across all evaluated configurations. The non-agent machine learning baseline shows competitive performance at lower data volumes but experiences higher latency as the workload increases.

Across all evaluated data sizes, the MA-BI framework achieves the lowest average processing time, as illustrated in Figure 2.

Confidence intervals corresponding to repeated runs are reflected in the reported mean ± standard deviation values and confirm consistent latency differences between the compared approaches.

3.2. Classification Performance

This subsection reports the classification performance of the evaluated approaches using accuracy-based metrics. The comparison focuses on the predictive behavior of the proposed MA-BI framework relative to the centralized BI pipeline and the non-agent machine learning baseline under identical experimental conditions.

The classification results were obtained on the same evaluation subset described in Section 3.1, ensuring consistency across performance and accuracy measurements.

Figure 3 shows the classification accuracy achieved by each approach on the evaluation dataset. The proposed MA-BI framework attains higher accuracy compared to both the centralized BI pipeline and the non-agent machine learning baseline.

Figure 3. Comparison of classification accuracy across different approaches.

The centralized BI pipeline exhibits the lowest accuracy among the evaluated methods, while the non-agent machine learning baseline achieves intermediate performance. The MA-BI framework consistently records the highest accuracy values across the reported results.

Across all reported accuracy measurements, the proposed MA-BI framework maintains the highest performance, followed by the non-agent baseline and the centralized BI pipeline (Figure 3).

To assess the robustness of the observed accuracy differences, experiments were repeated under identical conditions across multiple runs. The consistent performance gap between the proposed framework and the baseline approaches indicates that the improvement is systematic rather than incidental. Additionally, a paired comparison across repeated runs confirmed that the observed accuracy improvement is statistically consistent across executions, reinforcing the reliability of the reported performance difference.

3.3. Confusion Matrix and F1-Score Analysis

This subsection examines the classification behavior of the evaluated approaches using confusion matrix representations and F1-score metrics. These results complement the accuracy-based comparison by providing a more detailed view of classification outcomes across positive and negative classes.

Figure 4 presents the confusion matrices obtained for the centralized BI pipeline, the non-agent machine learning baseline, and the proposed MA-BI framework. The matrices illustrate the distribution of correctly and incorrectly classified instances across both classes.

Figure 4. Confusion matrix of classification results for the evaluated approaches.

The MA-BI framework shows a higher concentration of correct predictions along the main diagonal, indicating improved classification consistency compared to the baseline approaches. In contrast, the centralized BI pipeline exhibits a larger number of misclassified instances, while the non-agent baseline demonstrates intermediate performance.

To summarize classification quality in a single metric that accounts for both precision and recall, the F1-score was computed for each approach. The MA-BI framework achieves the highest F1-score among the evaluated methods, followed by the non-agent machine learning baseline and the centralized BI pipeline.

3.4. ROC Curve and AUC Analysis

This subsection evaluates the classification performance of the proposed MA-BI framework using receiver operating characteristic (ROC) curves and the area under the curve (AUC). ROC analysis provides a threshold-independent view of model behavior and complements accuracy- and F1-based evaluations by capturing the trade-off between true positive and false positive rates across different decision thresholds.

Figure 5 compares the ROC curves obtained for the centralized BI pipeline, the non-agent machine learning baseline, and the proposed MA-BI framework. The MA-BI framework consistently achieves the highest area under the curve, indicating stronger class separation capability across the full range of decision thresholds.

Figure 5. ROC curves for the evaluated approaches.

The ROC curve corresponding to the non-agent machine learning baseline occupies an intermediate position, while the centralized BI pipeline demonstrates the lowest overall AUC values. The separation between the curves is maintained throughout the evaluation range, suggesting stable relative performance among the compared approaches.

These results confirm that the proposed MA-BI framework sustains superior discriminative behavior independently of a specific classification threshold, reinforcing the robustness of the observed accuracy and F1-score improvements reported in the preceding subsections.

3.5. Stability and Robustness Analysis

To assess the stability of the evaluated approaches, additional experiments were conducted using multiple independent runs with different random initializations. This analysis focuses on the consistency of classification performance and decision-related metrics rather than peak values obtained in a single run.

Table 1 reports the mean and standard deviation of key evaluation metrics across repeated experimental runs. The proposed MA-BI framework exhibits lower variance compared to both the centralized BI pipeline and the non-agent machine learning baseline, indicating more stable behavior under repeated executions.

Table 1. Mean and standard deviation of performance metrics across repeated experimental runs.

Method	Accuracy (Mean ± Std)	F1-Score (Mean ± Std)	Decision Latency (ms, Mean ± Std)
Centralized BI Pipeline	0.68 ± 0.041	0.65 ± 0.038	182 ± 21
Non-Agent ML Baseline	0.74 ± 0.029	0.72 ± 0.031	137 ± 16
MA-BI Framework (proposed)	0.89 ± 0.012	0.87 ± 0.015	94 ± 9

In particular, the MA-BI framework maintains consistent accuracy and F1-score values across runs, while the baseline approaches show higher variability. This pattern suggests that the agent-based decomposition and localized decision-making reduce sensitivity to stochastic effects introduced by data sampling and model initialization.

The centralized BI pipeline demonstrates the highest variability among the evaluated approaches, reflecting its sensitivity to workload fluctuations and sequential processing constraints. The non-agent machine learning baseline exhibits moderate stability, positioned between the centralized pipeline and the proposed MA-BI framework.

Overall, the results summarized in Table 1 indicate that the proposed MA-BI framework achieves not only higher performance but also improved robustness, providing more predictable behavior across repeated experimental trials.

The results summarized in Table 1 demonstrate that the proposed MA-BI framework maintains consistent performance across repeated experimental runs, with lower variance compared to the baseline approaches. This stability complements the performance gains reported in the preceding subsections and provides additional confidence in the reliability of the observed results.

To further quantify stability, 95% confidence intervals were estimated based on the repeated experimental runs (n = 10). The relatively narrow intervals for accuracy, F1-score, and decision latency confirm that the observed performance gains remain consistent across executions and are not driven by isolated runs.

A paired t-test across repeated runs confirmed that the accuracy difference between the MA-BI framework and the baseline is statistically significant (p < 0.01). Given the limited number of repeated runs, the statistical results should be interpreted as indicative rather than definitive.

The observed performance improvement is unlikely to result from overfitting, since the predictive model parameters remain unchanged and evaluation is performed on chronologically separated unseen data.

4. Discussion

4.1. Impact of Multi-Agent Decomposition

The reduction in processing time observed in the experimental results is closely related to how analytical responsibilities are distributed across agents. By separating data ingestion, preprocessing, analytical modeling, and decision-making into independent components, the framework reduces the accumulation of coordination overhead that typically occurs in centralized BI pipelines. This effect becomes more pronounced as data volume increases, where parallel execution helps contain latency growth rather than eliminate it entirely.

In practice, we observed that centralized pipelines tend to amplify delays when individual processing stages experience temporary workload spikes. In contrast, the proposed multi-agent structure allows bottlenecks to remain localized, preventing delays from propagating across the entire workflow. While this decomposition does not remove all sources of latency, it provides a more graceful performance degradation as workload intensity grows, which is particularly relevant for data-intensive BI environments.

4.2. Classification Performance and Decision Quality

The improvements in classification accuracy and F1-score indicate that efficiency gains are not achieved at the expense of analytical quality. One contributing factor is the clear separation between analytical modeling and decision execution. Predictions generated by the Analytical Agent are consumed by the Decision-Making Agent without being tightly coupled to static decision rules, allowing decisions to adapt to observed outcomes over time.

Analysis of the confusion matrices shows that misclassifications are less concentrated in specific regions compared to the baseline approaches. This behavior is important in BI contexts, where the cost of errors is often asymmetric and depends on business context. While the framework does not eliminate classification errors, the more balanced error distribution suggests that adaptive decision policies can mitigate systematic biases introduced by fixed decision thresholds.

Importantly, these performance improvements cannot be attributed to a stronger classifier, as identical predictive models are employed in both the agent-based framework and the non-agent baseline. Instead, the observed gains arise from the modular agent-based decomposition and the adaptive decision-making mechanism implemented at the Decision-Making Agent level, which enables context-aware and policy-driven responses to analytical outcomes.

The magnitude of improvement is partly influenced by class imbalance and asymmetric decision costs present in the dataset. While the baseline applies a fixed 0.5 classification threshold, the DMA dynamically adjusts decision thresholds based on reward signals. This adaptive thresholding reduces false positives and false negatives in a context-sensitive manner, which explains the observed performance gap.

4.3. Threshold-Independent Behavior

The ROC and AUC analysis provides a complementary perspective to accuracy-based metrics. The consistently higher AUC values achieved by the proposed framework indicate stronger class separation across a range of decision thresholds. This property is particularly relevant in operational BI systems, where decision thresholds are often adjusted dynamically in response to changing business objectives rather than fixed at deployment time.

During experimentation, we found that the relative advantage of the proposed framework persisted even when thresholds were shifted to favor precision or recall. This stability suggests that performance improvements are not tied to a specific threshold configuration but instead reflect more robust underlying decision behavior.

4.4. Role of Reinforcement Learning

Reinforcement learning in the proposed framework plays a deliberately constrained role. Rather than introducing complex multi-agent learning dynamics, policy adaptation is limited to the Decision-Making Agent and driven by operational feedback. This design choice reflects a trade-off between adaptability and stability.

Empirically, this restrained learning strategy resulted in smoother policy evolution and reduced performance variance across repeated runs. While more aggressive learning schemes might yield higher peak performance under controlled conditions, they also increase the risk of unstable or hard-to-interpret behavior. In enterprise BI settings, where trust and predictability are critical, the observed balance between adaptability and stability is a practical advantage rather than a limitation.

4.5. Practical Implications and Limitations

From an implementation perspective, the results suggest that the proposed framework can be integrated into existing BI infrastructures without extensive architectural redesign. The modular agent-based structure supports incremental deployment, enabling individual components to be introduced, replaced, or scaled independently. This property is particularly important for enterprise BI environments, where system evolution must occur without service disruption.

At the same time, the current study has several limitations. The experimental evaluation is limited to a single dataset and a specific classification task, which constrains the generality of the conclusions. In addition, the reinforcement learning model is intentionally lightweight and does not explicitly capture complex long-term temporal dependencies that may arise in certain business intelligence scenarios. These limitations are not indicative of methodological weaknesses, but rather define clear directions for future research.

While the experimental comparison includes a strong and widely adopted learning-based non-agent baseline, extending the evaluation to additional datasets, deeper end-to-end learning models, and alternative agentic learning architectures represents a natural and important direction for future work. Such extensions will enable a more comprehensive assessment of scalability, robustness, and generalization across diverse business intelligence contexts.

5. Conclusions

This study examined a modular multi-agent framework for business intelligence in which reinforcement learning is applied only at the decision-making stage. The proposed MA-BI architecture was designed as an alternative to fully centralized analytical pipelines, motivated by practical scalability and responsiveness limitations observed in data-intensive BI environments. Instead of optimizing the entire pipeline through a single learning mechanism, the framework decomposes the BI workflow into autonomous agents with clearly separated responsibilities.

Experimental evaluation was conducted using transactional data from an e-commerce setting. Under the tested conditions, the proposed framework consistently reduced decision latency and improved classification-related metrics when compared to both a centralized BI pipeline and a non-agent machine learning baseline. These improvements were observed across multiple runs and remained stable when decision thresholds were varied, indicating that performance gains were not limited to a specific operating point.

An important design decision in this work was to restrict reinforcement learning to the Decision-Making Agent. Early experiments with more distributed learning configurations were explored but resulted in unstable policy behavior and increased coordination overhead. By limiting learning to a single agent and relying on operational feedback, the final implementation achieved smoother adaptation and lower performance variance across repeated runs, which is critical for enterprise-oriented BI systems.

Despite these positive results, several limitations should be noted. The experimental analysis focused on a single dataset and one classification task, and the number of agents was evaluated within a fixed range. As a result, the reported findings should be interpreted within this scope. In addition, the lightweight reinforcement learning model used in this study does not explicitly model long-term temporal dependencies, which may be relevant in more complex BI scenarios.

Future work will extend the evaluation to additional datasets and business intelligence tasks, including scenarios with higher agent densities and more heterogeneous workloads. Further investigation of coordination strategies and learning mechanisms that preserve system stability while improving adaptability remains an open direction. These efforts are intended to clarify how learning-enabled multi-agent architectures can be reliably deployed in real-world business intelligence systems.

From a theoretical standpoint, this study demonstrates that reinforcement learning can be integrated into modular multi-agent BI architectures without requiring fully distributed learning across agents. The clear separation between predictive modeling and decision-level adaptation provides a structured alternative to conventional end-to-end optimization strategies, contributing to the interpretability and controllability of learning-enabled BI systems.

Practically, the proposed architecture offers a balanced approach for organizations seeking to enhance decision responsiveness while maintaining system transparency and manageable computational complexity. By confining adaptive learning to the decision layer, enterprises can introduce intelligent behavior incrementally without disrupting existing analytical pipelines.