Quantifying Maladaptive Cognitive Offloading: A Behavioral Framework for AI-Supported Decision Making Under Complexity

Abstract

The integration of AI tools into professional knowledge work has created a critical measurement gap: organizations track AI adoption but cannot distinguish adaptive cognitive support from maladaptive dependency, a distinction with direct implications for how professionals adapt their decision-making processes under complex and time-pressured conditions. We ask whether maladaptive cognitive offloading can be measured from behavioral traces, and which signals matter most. We introduce COB-ML, a behavioral framework combining a six-layer feature architecture with a three-branch stacking ensemble of language-model, tabular, and sequential models. Across five datasets (70,673 survey records, 1,837,989 AI conversation logs, and 69,362 physiological windows), behavioral features outperformed an occupation-only baseline by $\mathrm{\Delta }\mathrm{F}1=+0.200$ (best tabular model). The ensemble reached a Macro-F1 of 0.762 under a cross-dataset protocol that is in-domain for its language-model branch, and an XGBoost branch showed $\mathrm{ICC}(A,1)=0.611$ against WESAD stress labels, preliminary and indirect evidence only. SHAP analysis associates trust disposition and verification behavior, rather than usage frequency, with maladaptive offloading. Monitoring adoption alone may therefore miss the strongest signals of cognitive dependency risk, and COB-ML offers a transparent starting point for measuring them.

1. Introduction

The integration of artificial intelligence tools into professional knowledge work has accelerated beyond the measurement frameworks available to organizations and researchers (Stack Overflow, 2024). The 2024 Stack Overflow Developer Survey reports that 76% of professional developers now use or plan to use AI tools in their workflows (Stack Overflow, 2024), with comparable diffusion documented across legal, medical, and productivity-critical domains (Brynjolfsson et al., 2023; Noy & Zhang, 2023; Rajpurkar et al., 2022). This diffusion is occurring in work environments that are themselves increasingly volatile, uncertain, complex, and ambiguous, where professionals must adapt their decision-making processes to shifting task demands, incomplete information, and compressed timelines. AI tools are frequently adopted as a coping mechanism for this complexity, promising to reduce cognitive load and accelerate judgment under pressure. Alongside this adoption, however, a less examined phenomenon has emerged: the progressive externalization of cognitive functions, including reasoning, verification, and judgment, from human professionals to AI systems. This process, termed cognitive offloading (Risko & Gilbert, 2016), is not inherently problematic. Its consequences depend on how offloading occurs and whether human judgment remains actively engaged in the decision-making process.

The critical distinction is between adaptive and maladaptive cognitive offloading. Adaptive offloading delegates routine processing to AI while preserving human critical evaluation and can be understood as a functional adjustment of decision-making style under complexity. Maladaptive offloading transfers the verification process itself to the AI system. In this case, the worker accepts outputs without critical review and delegates epistemic authority rather than execution, representing a breakdown in adaptive decision-making capacity rather than an efficient reallocation of cognitive resources. This distinction is invisible to usage frequency metrics, which constitute the majority of current AI adoption frameworks deployed by organizations. Human factors research has established that overreliance on automation produces systematic blind spots (Parasuraman & Riley, 1997), accountability failures (J. D. Lee & See, 2004), and performance degradation under high-stakes conditions (Hancock et al., 2011). The game-theoretic analysis in Zahedi et al. (2025) shows that humans systematically underinvest in verification behavior when interacting with automated agents, even when monitoring costs are made explicit. This dynamic applies with even greater force to AI-assisted cognitive work, where the costs of uncritical acceptance are diffuse and delayed, and where the pace of task-switching characteristic of complex professional environments leaves little room for deliberate verification.

Existing measurement approaches fall short along three dimensions. First, prior frameworks rely on synthetic datasets that lack ecological validity, producing accuracy values approaching 1.0 that reflect generative process reconstruction rather than genuine behavioral learning. Second, single-modality models cannot simultaneously represent survey self-report, AI conversation logs, and physiological cognitive load signals, which capture distinct facets of the offloading construct. Third, no prior framework cross-validates behavioral proxies against physiological reference data (Lim et al., 2018; Schmidt et al., 2018), leaving claims about construct validity empirically unsubstantiated. These gaps leave open whether maladaptive cognitive offloading can be measured from behavioral traces rather than usage frequency, and which signals are most informative. This paper addresses that question through three contributions:

• A six-layer behavioral feature architecture that operationalizes cognitive offloading from naturalistic interaction traces across participant attributes, work context, task properties, AI system characteristics, interaction traces, and human factors states. This architecture provides a formal operationalization of the adaptive and maladaptive offloading distinction at the feature level.
• The COB-ML stacking ensemble (Cognitive Offloading Behavioral Machine Learning), which unifies fine-tuned language model embeddings, gradient-boosted and attention-based tabular models, and sequential deep learning through a stacking meta-learner. COB-ML is evaluated across five real-world datasets spanning 70,673 labeled survey records and 1,837,989 AI conversation logs under four evaluation protocols, including user-based and cross-dataset generalization splits.
• Preliminary physiological construct evidence for Cognitive Offloading Index (COI) predictions against Wearable Stress and Affect Detection (WESAD) stress reference data (Schmidt et al., 2018) and Simultaneous Task EEG Workload (STEW) workload measures (Lim et al., 2018). This evidence offers an indirect, convergent link between behavioral proxies and physiological stress and workload signals in the context of professional AI use, rather than direct validation of the offloading construct itself.

A principal finding motivates the framework’s practical value. AI trust disposition, defined as a worker’s self-assessed confidence in AI output accuracy, is the strongest correlate of maladaptive offloading. Verification behavior constitutes the strongest observable interaction trace signal. Both outrank AI usage frequency as cognitive risk indicators by a SHapley Additive exPlanations (SHAP)-confirmed margin. This result challenges the prevailing assumption that adoption rates constitute meaningful proxies for cognitive dependency risk. It also points to trust calibration as an observable marker associated with whether professionals adapt or fail to adapt their decision-making processes to AI-integrated work; establishing a causal role for trust calibration would require experimental or longitudinal evidence beyond the present cross-sectional design. The remainder of this paper is organized as follows. Section 2 reviews related work on cognitive offloading, human–automation trust, and behavioral analytics in AI interaction. Section 3 formalizes the COI construct, operationalizes adaptive versus maladaptive offloading, and states four falsifiable hypotheses. Section 4 describes the COB-ML three-branch stacking ensemble and six-layer feature architecture. Section 5 presents the datasets, evaluation protocols, and baseline models. Section 6 reports classification performance, SHAP feature importance, and physiological validation results. Section 7 interprets verification behavior dominance and examines organizational implications. Section 8 addresses privacy and consent considerations. Section 9 presents methodological constraints and future work. Section 10 concludes.

2. Related Work

This section reviews five bodies of literature that converge on the present work: cognitive offloading theory, trust and overreliance in human–automation systems, behavioral analytics in human–AI interaction, adaptive decision making under complexity, and explainable machine learning for human-centered measurement. Each stream contributes theoretical grounding, empirical precedent, or methodological tools that COB-ML synthesizes and extends.

2.1. Cognitive Offloading Theory

The theoretical foundation of this work builds on the extended mind thesis, which argues that cognitive processes are not confined to the brain but can extend into the external environment through tools and artifacts (Clark & Chalmers, 1998). Under this view, a professional who delegates reasoning steps to an AI system is not merely using a tool. The professional is redistributing cognitive labor across a broader human and AI system boundary. Cognitive offloading has been operationalized as a behavioral strategy for managing cognitive resource constraints under conditions of limited working memory and time pressure (Risko & Gilbert, 2016). This perspective distinguishes prospective offloading, in which the environment is structured in advance to reduce future cognitive load, from active offloading, in which cognitive processing is delegated in real time. This distinction is central to the present work because offloading is not uniformly adaptive. When offloading extends to the evaluation and verification of externally generated outputs, the human agent may lose the feedback required for error detection and skill maintenance (Risko & Gilbert, 2016). Prior work has also shown that habitual offloading of judgment-intensive tasks can produce measurable degradation in unaided performance on those same tasks over time (Gilbert et al., 2020). In the context of professional AI use, this suggests that maladaptive offloading carries both immediate accuracy risks and longer-term consequences for cognitive capability.

2.2. Trust and Overreliance in Human–Automation Systems

The human factors literature has extensively examined trust and reliance as mediators of human–automation interaction outcomes. Trust has been defined as a multidimensional construct involving purpose, process, and performance dimensions, each of which contributes to a worker’s willingness to rely on an automated system (J. D. Lee & See, 2004). Appropriate reliance requires calibration between the degree of trust and the actual reliability of the system. Both undertrust and overtrust can therefore produce performance costs. Automation misuse research identifies overreliance as a persistent failure mode in which humans apply automation in conditions that require independent judgment (Parasuraman & Riley, 1997). Meta-analytic evidence further shows that automation reliability, system transparency, and operator experience are major moderators of trust calibration in human–robot interaction (Hancock et al., 2011). However, this literature has not fully addressed AI-supported cognitive work, where outputs are linguistic rather than physical and where output quality is often difficult for users to assess independently. Zahedi et al. (2025) introduce a Bayesian game-theoretic framework for modeling trust between a human supervisor and a worker robot. The framework shows that pure-strategy Nash equilibria often fail to exist in monitoring scenarios and that humans systematically underallocate observation resources relative to the normative optimum. The related trust boundary concept identifies a mixed-strategy region in which the supervised agent is incentivized to behave safely. COB-ML extends this insight into a measurement context. Rather than prescribing optimal monitoring strategies, COB-ML detects and quantifies the verification behavior that workers actually exhibit.

2.3. Behavioral Analytics in Human AI Interaction

Empirical research on behavioral patterns in AI-supported professional work has accelerated with the public release of large-scale AI tools. Studies of GitHub Copilot (GitHub, San Francisco, CA, USA) have documented acceptance rates, edit distances, and task completion times across professional software developers, revealing substantial individual variation in the extent to which developers integrate AI suggestions without modification (Ziegler et al., 2022). Controlled trial evidence further shows that AI assistance can reduce task completion time by 55.8% on average, while also exhibiting high cross-individual variance that suggests different reliance strategies across the developer population (Peng et al., 2023). At the conversation level, the WildChat dataset provides 837,989 real ChatGPT (OpenAI, San Francisco, CA, USA) conversations from users in naturalistic settings (Zhao et al., 2024). The LMSYS Chat 1M corpus contributes one million conversations across 25 language models collected from the Chatbot Arena platform (Zheng et al., 2023). Analysis of these corpora reveals that approximately 60% of AI conversations involve a single user turn, suggesting widespread single-turn acceptance behavior consistent with maladaptive offloading. However, neither dataset has previously been analyzed through a cognitive offloading lens. In addition, conversation-level behavioral features have not been cross-validated against physiological cognitive load measures. Survey-based behavioral data from the Stack Overflow Developer Survey (Stack Overflow, 2024) provides complementary self-report signals, including AI tool usage frequency, perceived AI accuracy, trust calibration, and verification habits, across 65,437 professional developers in 2024 and 49,191 in 2025. These self-report measures are distinct from the behavioral traces observable in conversation logs, but they are theoretically related to them. Their combination therefore represents a methodological contribution of this study.

2.4. Adaptive Decision Making Under Complexity

A separate but converging literature examines how decision processes adapt, or fail to adapt, when task environments become volatile, uncertain, complex, and ambiguous. This work has generally distinguished intuitive decision styles, which favor rapid, experience-based judgment, from analytic styles, which favor deliberate, sequential evaluation of evidence, and has asked which combination of styles best supports performance as context shifts from calm to demanding conditions (Kahneman & Klein, 2009). Naturalistic and dynamic decision-making research in high-stakes operational settings has shown that effective decision makers do not rely on a fixed style but shift adaptively between rapid pattern recognition and deliberate verification as situational demands change, as formalized in the recognition-primed decision model developed from studies of fireground commanders operating under time pressure and uncertainty (Klein, 1993). Cognitive offloading to an AI system can be understood within this frame as a further degree of freedom in that adaptation: delegating routine processing to AI may free cognitive resources for deliberate evaluation under complexity, consistent with adaptive decision making, or it may substitute for deliberate evaluation altogether, consistent with a maladaptive collapse toward uncritical acceptance under time pressure. The present work treats verification behavior and trust calibration as observable markers of where a given worker’s offloading pattern falls along this adaptive to maladaptive continuum, extending the decision-style literature into AI-integrated professional contexts where it has not yet been empirically operationalized.

2.5. Explainable Machine Learning for Human-Centered Measurement

SHAP has emerged as a dominant framework for post hoc explanation of machine learning model predictions (Lundberg & Lee, 2017). In behavioral measurement contexts, SHAP values identify the relative contribution of individual features to each prediction. This enables global feature importance rankings and instance-level explanations that support theoretical interpretation. SHAP values also satisfy the axioms of local accuracy, missingness, and consistency, making them preferable to earlier importance measures such as impurity-based ranking (Lundberg & Lee, 2017). TabNet’s sequential attention mechanism provides complementary instance level interpretability for tabular behavioral data by selecting a sparse subset of features at each decision step rather than applying global feature weights (Arik & Pfister, 2021). This property is valuable for behavioral measurement because predictive features may vary across occupational subgroups and usage contexts. Fine-tuned language model embeddings from RoBERTa (Liu et al., 2019) and Sentence BERT (Reimers & Gurevych, 2019) enable semantic feature extraction from AI conversation logs. These embeddings can capture verification language, challenge patterns, and acceptance signals that are not visible through frequency-based behavioral features alone.

2.6. Research Gap

The literature reviewed above reveals a consistent structural gap. Cognitive offloading theory provides the construct definition but lacks computational operationalization for professional AI contexts. Human–automation trust research establishes the behavioral dynamics of reliance but has not been applied to cognitive offloading in AI-supported knowledge work. Recent work has begun to measure AI reliance and overreliance directly, through reliance drills (Hunter et al., 2024), decision-theoretic reliance frameworks (Guo et al., 2024), and self-reported reductions in critical thinking among knowledge workers (H.-P. H. Lee et al., 2025), but none combine behavioral traces, self-report, and physiological signals in one framework. The adaptive decision-making literature has characterized how intuitive and analytic styles shift under complexity, but has not examined AI-mediated cognitive offloading as a correlate of that adaptation, and explainable machine learning methods have not been applied specifically to this construct. No prior work unifies behavioral log analysis, survey self-report, physiological reference data, and the adaptive decision-making perspective into a single validated framework for measuring cognitive offloading in professional AI use. COB-ML fills this gap.

3. Problem Formulation

Measuring cognitive offloading in professional AI use requires a formal construct definition that is both theoretically grounded and computationally tractable. We define the Cognitive Offloading Index as a continuous latent variable estimable from observable behavioral features, discretize it into three ordinal levels for classification, and distinguish two qualitatively distinct offloading regimes that carry different implications for cognitive risk. Four falsifiable hypotheses operationalize these definitions for empirical evaluation.

3.1. Formal Definition of Cognitive Offloading

Let $\mathcal{W}$ denote the set of professional workers and $\mathcal{T}$ the set of work tasks. For a worker $w\in \mathcal{W}$ performing task $t\in \mathcal{T}$ with the assistance of an AI system $\mathcal{A}$, we define the COI as a continuous latent variable:

$$\mathrm{COI}(w,t,\mathcal{A})\in \mathbb{R}$$

(1)

where higher values indicate greater externalization of cognitive function to $\mathcal{A}$. Equation (1) assigns one real number to each worker, task, and system combination, so COI describes a specific instance of AI-assisted work rather than a fixed trait of the person. COI ranges conceptually from full cognitive engagement, in which the worker performs all reasoning and verification independently, to full cognitive transfer, in which the worker accepts all AI-generated outputs without evaluation.

COI is a latent construct and not directly observable. We adopt the measurability assumption that COI is estimable from a vector of behavioral features $\varphi (w,t,\mathcal{A})\in {\mathbb{R}}^d$ drawn from observable interaction traces, without requiring direct neurological measurement. This assumption is grounded in the behavioral indicators of cognitive load identified in the psychophysiology literature (Lim et al., 2018; Schmidt et al., 2018) and is subject to empirical validation through construct validity analysis in Section 6.

The continuous COI is discretized into three ordinal levels for classification purposes:

$$\ell =\left\{\begin{array}{cc}\mathrm{Low}\hfill & \mathrm{if}\mathrm{COI}\le q_{33}\hfill \\ \mathrm{Medium}\hfill & \mathrm{if}{q}_{33}<\mathrm{COI}\le q_{67}\hfill \\ \mathrm{High}\hfill & \mathrm{if}\mathrm{COI}>q_{67}\hfill \end{array}\right.$$

(2)

where $q_{33}$ and $q_{67}$ are the empirical tertile boundaries of the COI proxy distribution in the training data. Equation (2) therefore places each observation in the lowest, middle, or highest third of the sample, so the levels express relative rather than absolute offloading. A secondary regression objective estimates the continuous COI proxy directly, providing a fine-grained measure for organizational monitoring applications.

3.2. Adaptive Versus Maladaptive Offloading

We formally distinguish two qualitatively different offloading regimes within the COI continuum, extending the recognition-primed view of adaptive expertise (Klein, 1993) to a context where part of the recognition and verification process may be delegated to an AI system rather than performed by the human alone. Let $v(w,t,\mathcal{A})\in \{0,1\}$ denote the verification indicator, where $v=1$ if the worker critically evaluates AI-generated output prior to acceptance and $v=0$ otherwise. Let $h(w,t,\mathcal{A})\in {\mathbb{Z}}_{\ge 0}$ denote the human help count, representing the number of conditions under which the worker retains human judgment rather than delegating to $\mathcal{A}$. Adaptive offloading is characterized by:

$$v(w,t,\mathcal{A})=1\mathrm{and}h(w,t,\mathcal{A})\ge {\theta }_{\mathrm{adapt}}$$

(3)

where ${\theta }_{\mathrm{adapt}}$ is a data-driven threshold estimated from the empirical distribution of human_help_count across the Stack Overflow Developer Survey dataset ($N=\mathrm{29,147}$ complete responses; range $[0,8]$; mean $=4.33$, SD $=2.22$). Both conditions in Equation (3) must hold: the worker verifies AI output and also keeps human judgment in at least ${\theta }_{\mathrm{adapt}}$ conditions. Following the tertile discretization applied to COI in Equation (2), ${\theta }_{\mathrm{adapt}}$ is set to the lower tertile boundary:

$${\theta }_{\mathrm{adapt}}=P_{33}\left(h\right)=3$$

(4)

Here $P_{33}\left(h\right)$ is the 33rd percentile of the help-count distribution. Because the threshold is a sample percentile rather than a fixed psychometric cutoff, it may shift in other populations. Workers retaining human judgment in three or more conditions satisfy the adaptive offloading criterion, representing 79.1% of respondents with complete data. The remaining 20.9% fall below the threshold and constitute the population at risk of maladaptive offloading targeted by COB-ML. In adaptive offloading, the worker delegates routine or syntactically bounded processing while retaining critical evaluation. AI amplifies throughput without displacing judgment.

Maladaptive offloading is characterized by:

$$v(w,t,\mathcal{A})=0\mathrm{and}d(w,t,\mathcal{A})=1$$

(5)

where $d\in \{0,1\}$ is the full dependency indicator, set to 1 when a worker reports no conditions under which human judgment would be sought. Equation (5) requires both the absence of verification and full dependency. The two regimes are not exhaustive: workers meeting neither definition occupy an intermediate zone. In maladaptive offloading, the verification process itself is transferred to $\mathcal{A}$. Epistemic authority, rather than execution, is delegated. This regime corresponds to the highest COI values and constitutes the primary target of the COB-ML detection framework.

3.3. Research Hypotheses

Following the hypothesis-driven evaluation structure advocated in human–machine systems research (Zahedi et al., 2025), we state four falsifiable hypotheses prior to presenting any experimental results:

• H1: Behavioral proxies derived from interaction traces predict cognitive offloading level significantly better than occupation-based demographic features alone, measured by an improvement in Macro-F1 exceeding 0.15 on the user-based evaluation split.
• H2: The trust verification flag is the dominant observable interaction trace predictor of maladaptive cognitive offloading, confirmed by the highest mean absolute SHAP value among Layer 5 features and Spearman rank correlation $r>0.90$ across ten random seeds.
• H3: The COB-ML stacking ensemble significantly outperforms any single-modality baseline model on cross-dataset generalization, as established by the Friedman test ($p<0.05$) with Nemenyi post hoc pairwise comparisons and critical difference diagram.
• H4: Behavioral COI predictions correlate significantly with physiological reference measures of acute stress and cognitive workload. This is achieved via an Intraclass Correlation Coefficient $\mathrm{ICC}(A,1)\ge 0.60$ against WESAD stress labels (Schmidt et al., 2018) and STEW EEG workload measures (Lim et al., 2018), with convergent Pearson $r\ge 0.50$. This evidence is preliminary and indirect, supporting construct validity rather than directly validating the offloading construct.

The present framework operates under four bounding assumptions that define its scope and inform the interpretation of results:

• Cross-sectional measurement: COB-ML estimates COI from a single interaction session or survey response. It does not model intra-individual change in offloading behavior over time. Longitudinal extension is identified as a priority for future work in Section 9.
• Professional knowledge work context: The framework is validated on datasets drawn from software development, knowledge work, and general AI-assisted professional tasks. Generalization to physical labor or clinical decision-making contexts is not claimed and requires separate validation.
• Text-based AI interaction: The interaction trace features assume text-based AI tool interfaces, including chat-based language models and code completion assistants. Voice-based or multimodal AI interfaces require additional feature engineering beyond the present scope.
• Proxy-based COI estimation: The COI proxy score is a theoretically grounded but formally derived behavioral estimate, not a direct neurological measurement, and its weights are not yet derived from first principles; Section 9 discusses this as a priority for future refinement. Construct validity is established empirically in Section 6 but does not constitute proof of identity between the behavioral proxy and the underlying neurophysiological construct.

Section 4 now turns from this construct definition to the computational architecture used to estimate COI from observable behavioral features.

4. The COB-ML Framework

COB-ML addresses the COI measurement problem across three levels: feature organization, modality-specific modeling, and interpretable prediction aggregation. Figure 1 presents an overview of the complete architecture, from multi-source data inputs through the six-layer feature pipeline and three-branch ensemble to the final COI predictions and explainability outputs. Each level is described in the subsections below.

Figure 1. Overview of the COB-ML framework: multi-source behavioral data flows through the six-layer feature architecture and three-branch stacking ensemble to produce COI predictions with TreeSHAP attribution.

4.1. Six-Layer Behavioral Feature Architecture

COB-ML organizes behavioral features into six conceptual layers, each corresponding to a distinct and theoretically motivated aspect of the human–AI interaction context. Together, the layers capture 29 engineered features that operationalize the COI construct defined in Section 3.

• Layer 1 encodes static professional and demographic characteristics, including occupation DevType, years of experience WorkExp and YearsCode, education level EdLevel, organization size OrgSize, and role type ICorPM. These features establish the baseline professional context and serve as the occupation-only lower bound against which H1 is evaluated.
• Layer 2 captures environmental and organizational moderators of offloading tendency, including remote work status RemoteWork, employment type Employment, industry context Industry, and job satisfaction JobSat. Remote and asynchronous work conditions are associated with higher AI tool reliance (Stack Overflow, 2024), making these features theoretically motivated additions to the feature set.
• Layer 3 characterizes the cognitive demands of the work being performed through AI complexity rating AIComplex and AI tool diversity ToolCountWork and ToolCountPersonal. Higher complexity ratings combined with high tool diversity indicate a worker who delegates cognitively demanding work across multiple AI systems, a pattern associated with elevated COI.
• Layer 4 captures properties of the AI tools in use, including model tier model_tier, which encodes the capability level of the AI system derived from conversation-log model names, and AI tool selection behavior AISelect and AIToolCurrently. These features reflect the degree to which workers have integrated AI into their professional workflow at the system level.
• Layer 5 constitutes the behavioral core of the feature architecture. Features are extracted directly from AI conversation logs and include the number of user turns per session num_user_turns, single-turn acceptance flag single_turn, user-to-AI message length ratio length_ratio, and the trust verification flag trust_verification_flag. This flag indicates whether the worker explicitly seeks human judgment when they distrust AI outputs. The trust verification flag is the primary operationalization of verification behavior and is the feature under investigation in H2.
• Layer 6 encodes self-reported cognitive and attitudinal states derived from survey instruments, including perceived AI accuracy ai_trust_score, AI sentiment ai_sentiment, AI threat perception ai_threat_perceived, full dependency indicator ai_full_dependency, and frustration count frustration_count. The full dependency indicator is set to 1 when a worker reports no conditions under which human judgment would be sought. It is the primary operationalization of the maladaptive offloading condition defined in Section 3.

Table 1 maps each engineered feature to its source survey question or extraction rule, ensuring full reproducibility of the feature engineering pipeline described above.

Table 1. Feature-to-survey-item mapping. SO column names refer to the Stack Overflow Developer Survey 2024/2025 raw schema.

Feature	SO Column	Layer
DevType	DevType	L1
WorkExp, YearsCode	WorkExp, YearsCode	L1
EdLevel	EdLevel	L1
OrgSize	OrgSize	L1
ICorPM	ICorPM	L1
RemoteWork	RemoteWork	L2
Employment	Employment	L2
Industry	Industry	L2
JobSat	JobSat	L2
AIComplex	AIComplex	L3
ai_complex_avoidance	from AIComplex	L3
model_tier	from log model names	L4
AISelect	AISelect	L4
num_user_turns	WildChat session turns	L5
single_turn	from num_user_turns	L5
length_ratio	message length fields	L5
trust_verification_flag	from AIHuman	L5
human_help_count	count of AIHuman	L5
learning_verification_flag	from AIHuman	L5
ai_trust_score	AIAcc	L6
ai_sentiment	AISent	L6
ai_threat_perceived	AIThreat	L6
ai_full_dependency	from AIHuman	L6
ai_benefit_count	count of AIBen	L6
frustration_count	count of Frustration	L6

Algorithm 1 summarizes the complete COB-ML training and inference procedure.

Algorithm 1 COB-ML Training and Inference

Require: Datasets ${\mathcal{D}}_{\mathrm{SO}}$, ${\mathcal{D}}_{\mathrm{WC}}$, ${\mathcal{D}}_{\mathrm{LM}}$, ${\mathcal{D}}_{\mathrm{WE}}$, ${\mathcal{D}}_{\mathrm{ST}}$; seeds $S=\{0,\dots ,9\}$ Ensure: COI predictions $\widehat{y}$, SHAP values $\mathit{\varphi }$ 1: Extract 29 COI features across six layers; compute proxy scores; assign labels via $q_{33}$, $q_{67}$ 2: Apply modality-specific encoding, median imputation, and training-split normalization 3: Apply GroupKFold by user ID for conversation data; subject-wise split for physiological data 4: for each seed $s\in S$ do (a) Branch A: Fine-tune RoBERTa on ${\mathcal{D}}_{\mathrm{WC}}$ text; compute ${\widehat{\mathbf{p}}}_A$ via mean-pooled embeddings (b) Branch B: Train XGBoost and TabNet on ${\mathcal{D}}_{\mathrm{SO}}$ structured features; compute ${\widehat{\mathbf{p}}}_B$ (c) Branch C: Train LSTM on interaction sequences and 1D-CNN on physiological windows; compute ${\widehat{\mathbf{p}}}_C$ 5: end for 6: Form meta-vector ${\mathbf{z}}^{\left(i\right)}=[{\widehat{\mathbf{p}}}_A^{\left(i\right)};{\widehat{\mathbf{p}}}_B^{\left(i\right)};{\widehat{\mathbf{p}}}_C^{\left(i\right)}]\in {\mathbb{R}}^{3K}$ 7: Train logistic regression meta-learner via 5-fold CV minimizing ${\mathcal{L}}_{\mathrm{cls}}$; train ridge regression minimizing ${\mathcal{L}}_{\mathrm{reg}}$ 8: Predict ${\widehat{y}}^{\left(i\right)}=\arg {\max }_k\mathrm{softmax}{(\mathbf{W}{\mathbf{z}}^{\left(i\right)}+\mathbf{b})}_k$ 9: Compute TreeSHAP $\mathit{\varphi }$ and rank stability ${\rho }_{\mathrm{stability}}$; report stable predictors where ${\rho }_{\mathrm{stability}}>0.90$ Return: $\widehat{y}$, $\widehat{\mathrm{COI}}$, $\mathit{\varphi }$

4.2. Data Pipeline

4.2.1. Preprocessing per Modality

Survey data from the 2024 and 2025 Stack Overflow Developer Surveys undergo ordinal encoding of Likert-scale responses, binary encoding of multi-select AI tool items, and extraction of multi-select features through semicolon-delimited parsing. Conversation-log data from WildChat-1M and LMSYS-Chat-1M undergo role-based message segmentation, message length computation, and turn-count extraction per session. Physiological data from WESAD undergo sliding-window segmentation at 700 samples per window with 50% overlap, with per-window statistical feature extraction across chest electrodermal activity (EDA), electrocardiogram (ECG), and wrist blood volume pulse (BVP) channels, computed as the mean, standard deviation, range, and skewness of each signal. STEW electroencephalogram (EEG) data undergo analogous windowing at 256 samples with 14-channel per-window statistics.

4.2.2. Missing Value Handling and Normalization

Year-specific features present in only one survey wave are retained as NaN for the opposing year and imputed using median imputation within each modality prior to model training. All continuous features are normalized to zero mean and unit variance using statistics computed on the training split only, with the same parameters applied to validation and test splits to prevent data leakage.

4.2.3. IID and Non-IID Split Strategy

A fundamental methodological concern in behavioral datasets derived from conversation logs is the violation of the independent and identically distributed (IID) assumption. WildChat-1M and LMSYS-Chat-1M each contain multiple conversation records per unique user. Random train-test splitting allows the same user’s behavioral patterns to appear in both training and test sets, producing optimistically biased performance estimates that do not reflect generalization to genuinely unseen users. To address this, conversation-log datasets are split using GroupKFold with five folds, keyed on hashed user identifiers, ensuring that no user appears in both training and evaluation splits. Physiological datasets (WESAD, STEW) are split subject-wise. The Stack Overflow survey data, in which each row corresponds to a unique respondent, is treated as approximately IID and split by random stratified sampling. Four evaluation protocols are applied to all models: random IID split, user-based GroupKFold, temporal split (training on 2024 survey, testing on 2025), and cross-dataset leave-one-out. The cross-dataset protocol trains on Stack Overflow structured features and physiological windows and evaluates on WildChat conversation text; consequently, for Branch A this protocol is in-domain rather than cross-domain, since Branch A is fine-tuned directly on WildChat. This distinction is carried through consistently in the reporting and interpretation of H3 in Section 6.

4.3. Three-Branch Model Architecture

COB-ML employs a three-branch architecture in which each branch is specialized for a distinct data modality. Let ${\widehat{\mathbf{p}}}_A,{\widehat{\mathbf{p}}}_B,{\widehat{\mathbf{p}}}_C\in {\mathrm{\Delta }}^{K-1}$ denote the predicted class probability vectors from Branches A, B, and C respectively, where $K=3$ is the number of offloading levels and ${\mathrm{\Delta }}^{K-1}$ is the $(K-1)$-simplex. Each branch produces a distribution over offloading levels independently before being passed to the stacking meta-learner.

4.3.1. Branch A: Semantic Language Branch

Branch A processes AI conversation text using fine-tuned RoBERTa (Liu et al., 2019). For a session i containing $T_i$ user turns $\{x_1^{\left(i\right)},\dots ,x_{T_i}^{\left(i\right)}\}$, the session-level semantic embedding is computed as:

$${\mathbf{e}}_i={\displaystyle \frac{1}{T_i}}{\displaystyle \sum _{t=1}^{T_i}}\mathrm{RoBERTa}\left(x_t^{\left(i\right)}\right)\in {\mathbb{R}}^{768}$$

(6)

The branch prediction is then:

$${\widehat{\mathbf{p}}}_A^{\left(i\right)}=\mathrm{softmax}\left({\mathbf{W}}_A{\mathbf{e}}_i+{\mathbf{b}}_A\right)$$

(7)

Fine-tuning minimizes the cross-entropy loss with AdamW, learning rate $\eta =2\times {10}^{-5}$, and a linear warmup over the first 10% of training steps. Branch A captures verification language patterns such as challenge, doubt, and follow-up questioning signals that are invisible to the frequency-based features in Branches B and C. Because Branch A is fine-tuned and evaluated on WildChat conversation text, its performance under the cross-dataset protocol reflects in-domain generalization to unseen users rather than transfer to an unrelated data distribution; this scoping is made explicit here so that later cross-domain claims are not overstated relative to what this branch actually demonstrates.

4.3.2. Branch B: Tabular Behavioral Branch

Branch B processes structured behavioral features $\mathbf{x}\in {\mathbb{R}}^d$ from the Stack Overflow survey data. XGBoost (Chen & Guestrin, 2016) produces predictions as an additive ensemble of M regression trees:

$$F\left(\mathbf{x}\right)={\displaystyle \sum _{m=1}^M}{f}_m\left(\mathbf{x}\right),f_m\in \mathcal{F}$$

(8)

where $\mathcal{F}$ is the space of classification and regression trees. TabNet (Arik & Pfister, 2021) complements XGBoost through sequential attentive feature selection. At each decision step l, TabNet computes a sparse attention mask ${\mathbf{a}}^{\left[l\right]}\in {[0,1]}^d$ over input features and a step output:

$${\mathbf{h}}^{\left[l\right]}=f_l\left({\mathbf{a}}^{\left[l\right]}\odot \mathbf{x}\right),{\mathbf{a}}^{\left[l\right]}=\mathrm{sparsemax}\left({\mathbf{P}}^{\left[l\right]}{\mathbf{h}}^{[l-1]}\right)$$

(9)

where ⊙ denotes element-wise multiplication, ${\mathbf{P}}^{\left[l\right]}$ is a learned projection matrix, and sparsemax enforces feature sparsity with regularization coefficient ${\lambda }_{\mathrm{sparse}}={10}^{-4}$.

4.3.3. Branch C: Sequential and Physiological Branch

Branch C processes temporal interaction sequences using an LSTM (Hochreiter & Schmidhuber, 1997). For a sequence of per-turn feature vectors $({\mathbf{x}}_1,\dots ,{\mathbf{x}}_T)$, the LSTM hidden state evolves as:

$$\begin{array}{cc}\hfill {\mathbf{f}}_t& =\sigma \left({\mathbf{W}}_f[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_f\right)\hfill \\ \hfill {\mathbf{i}}_t& =\sigma \left({\mathbf{W}}_i[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_i\right)\hfill \\ \hfill {\mathbf{c}}_t& ={\mathbf{f}}_t\odot {\mathbf{c}}_{t-1}+{\mathbf{i}}_t\odot \tanh \left({\mathbf{W}}_c[{\mathbf{h}}_{t-1},{\mathbf{x}}_t]+{\mathbf{b}}_c\right)\hfill \\ \hfill {\mathbf{h}}_t& ={\mathbf{o}}_t\odot \tanh \left({\mathbf{c}}_t\right)\hfill \end{array}$$

(10)

where ${\mathbf{f}}_t$, ${\mathbf{i}}_t$, and ${\mathbf{o}}_t$ are the forget, input, and output gates respectively, and $\sigma$ is the sigmoid function. The final hidden state ${\mathbf{h}}_T$ is passed to a classification head. Physiological signal windows from WESAD and STEW are processed by a 1D-CNN with three convolutional layers (64, 128, 256 filters), global average pooling, and a fully connected head. Both the LSTM and the 1D-CNN use the Adam optimizer with dropout $p=0.3$ and early stopping with a patience of 10 epochs.

4.4. Stacking Meta-Learner

Since the three branches operate on disjoint datasets, a uniform prior $[1/K,\dots ,1/K]$ is substituted for any branch whose modality is unavailable for a given instance. The meta-learner is trained on out-of-fold branch predictions via five-fold cross-validation, learning to weight branch contributions by their reliability on each data distribution.

The branch probability vectors are concatenated to form a meta-feature vector:

$${\mathbf{z}}^{\left(i\right)}=\left[{\widehat{\mathbf{p}}}_A^{\left(i\right)};{\widehat{\mathbf{p}}}_B^{\left(i\right)};{\widehat{\mathbf{p}}}_C^{\left(i\right)}\right]\in {\mathbb{R}}^{3K}$$

(11)

A logistic regression meta-learner is trained on $\mathbf{z}$ via five-fold cross-validation on the training set, minimizing the regularized cross-entropy objective:

$${\mathcal{L}}_{\mathrm{cls}}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{k=1}^K}{y}_{ik}\log {\widehat{y}}_{ik}+\lambda {\parallel \mathbf{W}\parallel }_2^2$$

(12)

where $y_{ik}\in \{0,1\}$ is the one-hot label for instance i and class k, and $\lambda$ is the ${\ell }_2$ regularization coefficient. The final classification prediction is:

$${\widehat{y}}^{\left(i\right)}=\arg \underset{k}{\max }\mathrm{softmax}{\left(\mathbf{W}{\mathbf{z}}^{\left(i\right)}+\mathbf{b}\right)}_k$$

(13)

For the continuous COI regression objective, a ridge regression meta-learner is trained on the same meta-feature vector, minimizing:

$${\mathcal{L}}_{\mathrm{reg}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\left({\mathrm{COI}}_i-{\mathbf{w}}^{\top }{\mathbf{z}}^{\left(i\right)}-b\right)}^2+\alpha {\parallel \mathbf{w}\parallel }_2^2$$

(14)

Logistic regression is chosen deliberately: its linearity ensures interpretable branch contributions through $\mathbf{W}$, and its ${\ell }_2$ penalty prevents over-weighting any single branch.

4.5. Explainability Layer

TreeSHAP (Lundberg & Lee, 2017) is applied at the ensemble level to produce feature importance attributions. For a feature set F and model f, the SHAP value for feature j is defined as:

$${\varphi }_j=\sum _{S\subseteq F\setminus \left\{j\right\}}{\displaystyle \frac{\left|S\right|!\left(\right|F|-|S|-1)!}{\left|F\right|!}}\left[f(S\cup \{j\left\}\right)-f\left(S\right)\right]$$

(15)

where S is a subset of features excluding j and $f\left(S\right)$ denotes the model prediction using only the features in S. SHAP values satisfy the axioms of local accuracy, missingness, and consistency (Lundberg & Lee, 2017), making them preferable to impurity-based importance rankings, which are known to exhibit bias toward high-cardinality features.

Three explainability outputs are produced: a beeswarm plot of ${\varphi }_j$ values for the top 10 features across all test instances; a dependence plot of ${\varphi }_{\mathrm{verif}}$ against the COI output, revealing the functional form of the verification rate effect; and a rank stability report computed as:

$${\rho }_{\mathrm{stability}}={\displaystyle \frac{1}{\left(\genfrac{}{}{0pt}{}{S}{2}\right)}}\sum _{s<s^{\prime }}{r}_s\left(\mathit{\varphi },{\mathit{\varphi }}^{\prime }\right)$$

(16)

where $r_s$ is the Spearman rank correlation between SHAP importance orderings across seed pairs $(s,s^{\prime })$ over $S=10$ random seeds. A feature is reported as a stable predictor only if ${\rho }_{\mathrm{stability}}>0.90$.

5. Datasets and Experimental Setup

Rigorous evaluation of a behavioral measurement framework requires careful separation of data sources, label construction, and evaluation protocols to avoid circular assessment. Five datasets spanning survey self-report, AI conversation logs, and physiological signals are used across training and construct-validation roles, with strict feature separation between label-generating and training variables.

5.1. Datasets

Table 2 summarizes the five datasets used across training, validation, and construct validation roles. Together they span 70,673 labeled survey records, 1,837,989 AI conversation logs, and 69,362 physiological measurement windows.

Table 2. Dataset summary.

Dataset	Role	Size	Key Features
SO Survey 2024–25 (Stack Overflow, 2024)	Training	70,673	AI trust, verification, occupation
WildChat-1M (Zhao et al., 2024)	Training	837,989	Turn count, length ratio, single-turn flag
LMSYS-Chat-1M (Zheng et al., 2023)	Training	1,000,000	Acceptance patterns, multi-turn signals
WESAD (Schmidt et al., 2018)	Validation	55,154	EDA, ECG, BVP, stress labels
STEW (Lim et al., 2018)	Validation	14,208	EEG 14-channel, workload levels

Stack Overflow Developer Surveycombines the 2024 (65,437 respondents) and 2025 (49,191 respondents) waves. After feature engineering and filtering for complete core COI features, 70,673 labeled records are retained. The 2025 wave introduces AIHuman, a direct self-report of human–AI judgment balance, and ToolCountWork, absent from the 2024 instrument. Year-specific features are retained with median imputation for the opposing wave.

WildChat-1M (Zhao et al., 2024) provides 837,989 real ChatGPT conversations from naturalistic user sessions. Interaction trace features are extracted per session and a user-based GroupKFold split is applied to prevent user-level data leakage.

LMSYS-Chat-1M (Zheng et al., 2023) contributes one million conversations across 25 language models. Model tier encoding distinguishes GPT-4, Claude, and open-source model families, providing model diversity absent from WildChat.

WESAD (Schmidt et al., 2018) provides multimodal physiological recordings from 15 subjects under validated baseline and stress conditions. Sliding window segmentation at 700 samples (50% overlap) yields 55,154 windows with chest EDA, ECG, and wrist BVP features. Subjects S2–S17 are used; S12 is excluded due to missing data.

STEW (Lim et al., 2018) provides EEG recordings from 48 subjects during rest and high-workload multitasking. Windowing at 256 samples yields 14,208 balanced windows across 14 EEG channels, with workload binary labels derived from subject-rated nine-point scales.

The five sources are treated as complementary indicators of one multidimensional construct, not as interchangeable measures of a single signal. Survey self-report captures attitudinal facets such as trust. Conversation logs capture behavioral traces of verification and acceptance. Physiological signals capture construct-adjacent correlates: cognitive and affective load. Section 6 reports what this treatment predicts. Convergence appears as significant Pearson correlations between COB-ML predictions and both physiological reference sets ($r=0.604$ for WESAD and $r=0.971$ for STEW). The STEW value reflects its balanced rest versus high-workload design, so it is not treated as independent evidence. Divergence appears as a Cronbach’s $\alpha$ of 0.460 for the COI feature set, below the conventional 0.70 threshold, which is the pattern expected if the facets are partly independent rather than redundant. It also appears in the leading features, which differ between the construct-interpretation analysis (AI trust disposition) and the leakage-controlled predictive analysis (human_help_count). These figures do not show that the datasets measure an identical variable. WESAD measures acute stress and STEW measures workload, so each provides only partial, non-redundant evidence about a shared construct.

5.2. COI Proxy Construction and Leakage Control

The COI proxy is an author-defined weighted sum of five behavioral indicators. For a worker w with observed feature values $x_f$, it is computed as:

$$\widehat{\mathrm{COI}}\left(w\right)=0.30x_u+0.25x_t+0.20x_c+2.00x_d-1.50x_v$$

(17)

where $x_u$ = ai_usage_freq, $x_t$ = ai_trust_score, $x_c$ = ai_complex_rating, $x_d$ = ai_full_dependency, and $x_v$ = trust_verification_flag. The verification term is negated because verification behavior suppresses offloading. Full dependency (2.00) and verification (1.50) receive the largest weights as the primary operationalizations of maladaptive offloading in Equations (3) and (5). The weights are theoretically motivated rather than empirically fit (Section 9). The continuous score is discretized into tertile labels following Equation (2). Because tertiles depend only on rank order, rescaling all weights by a constant leaves every label unchanged.

Sensitivity analysis across ±20% perturbations of each weight shows label stability with a mean agreement of 97.0% (Spearman $r>0.996$ across all single-weight scenarios). Under an extreme equal-weight perturbation, 87.7% of labels are preserved at $r=0.826$. Table 3 reports the full results.

Table 3. COI Proxy Weight Sensitivity Analysis. Agreement measured against baseline tertile labels ($N=\mathrm{70,673}$). Equal-weight scenario represents extreme perturbation bound.

Feature (w)	+20% Perturbation			−20% Perturbation
	Agree.	r	N chg	Agree.	r	N chg
ai_usage_freq ($w=0.30$)	99.5%	0.998	354	96.6%	0.996	2419
ai_trust_score ($w=0.25$)	99.3%	0.997	518	92.6%	0.996	5252
ai_complex_rating ($w=0.20$)	100.0%	1.000	4	99.7%	0.999	246
ai_full_dependency ($w=2.00$)	100.0%	1.000	19	99.9%	1.000	44
trust_verif_flag ($w=-1.50$)	92.3%	0.998	5412	99.3%	0.997	521
Equal weights (extreme)	87.7%    $r=0.826$			N changed $=8712$
Mean (±20% scenarios)	97.0%			$r=0.997$

The tertile labels are therefore robust to moderate uncertainty in the weights, but this does not show that the proxy validly measures maladaptive offloading. The weights are not fit to an external criterion, so the models evaluated here learn an author-defined proxy. The sensitivity is also uneven. The largest changes come from the verification flag at $+20\%$ (92.3% agreement, 5412 labels changed) and the trust score at $-20\%$ (92.6%, 5252 changed), whereas no dependency or complexity scenario changes more than 246 labels. The labels thus depend most on the verification and trust indicators. The equal-weight case is the least favorable: 87.7% of labels are preserved (8712 of 70,673 changed), which bounds but does not remove the sensitivity of downstream results to the weighting. The construct-validity evidence in Section 6 should accordingly be read as indirect and convergent rather than confirmatory. No public dataset pairs AI interaction behavior with an independently validated offloading label, and building one through controlled human-subjects experiments is the highest-priority direction in Section 9.

The five proxy inputs are the label-generating variables and are excluded from model training, so that reported performance reflects behavioral prediction rather than reconstruction of the label formula (Table 4). The SHAP analysis in Section 6 uses the full 22-feature set including the proxy inputs for construct interpretation. A separate leakage-controlled analysis uses independent features only. Table 5 reports Pearson correlations between the 16 training features and the five proxy inputs. The maximum absolute correlation is $r=0.581$ and the mean across all 80 cells is 0.082, which indicates limited indirect leakage. ai_sentiment and ai_complex_avoidance exceeded $\left|r\right|>0.40$ and were excluded from the predictive feature set. The next largest, human_help_count with the verification flag ($r=0.40$), lies at the threshold and was retained. Figure 2 presents the full heatmap.

Table 4. Feature separation for leakage control. COI label features are excluded from model training. Features marked ^† are used only in the construct-interpretation SHAP analysis and are excluded from the predictive models.

Role	Feature	Layer
COI Label (excluded)	ai_trust_score	L6
	ai_complex_rating	L3
	ai_usage_freq	L4
	ai_full_dependency	L6
	trust_verification_flag	L5
Training Features (used)	EdLevel, YearsCode, WorkExp	L1
	DevType, OrgSize, ICorPM	L1
	RemoteWork, Employment	L2
	Industry, JobSat	L2
	ai_complex_avoidance †	L3
	human_help_count	L5
	learning_verif_flag	L5
	ai_sentiment †	L6
	ai_benefit_count	L6
	frustration_count	L6

Table 5. Pearson correlations between training features and COI label-generating variables. Bold values indicate $\left|r\right|>0.40$; features marked ^† were excluded from the predictive model. Max $\left|r\right|=0.581$; mean $\left|r\right|=0.082$.

Feature	${\mathit{x}}_{\mathit{u}}$	${\mathit{x}}_{\mathit{t}}$	${\mathit{x}}_{\mathit{c}}$	${\mathit{x}}_{\mathit{d}}$	${\mathit{x}}_{\mathit{v}}$
EdLevel	$-0.03$	$-0.02$	$0.00$	$0.00$	$0.02$
YearsCode	$0.02$	$0.05$	$0.04$	$-0.01$	$0.03$
WorkExp	$0.03$	$0.09$	$0.04$	$-0.05$	$0.14$
DevType	$-0.03$	$0.03$	$0.02$	$-0.01$	$0.06$
OrgSize	$-0.03$	$0.02$	$0.05$	$0.04$	$-0.05$
ICorPM	$0.01$	$0.09$	$0.06$	$-0.01$	$0.08$
RemoteWork	$-0.03$	$0.02$	$0.02$	$0.03$	$0.00$
Employment	$-0.03$	$0.03$	$0.03$	$0.01$	$0.02$
Industry	$0.03$	$0.06$	$0.03$	$-0.03$	$0.09$
JobSat	$-0.02$	$0.07$	$0.05$	$-0.04$	$0.07$
ai_cx_avoid †	$-\mathbf{0.58}$	$-0.26$	$0.05$	$0.02$	$-0.09$
human_help_cnt	$-0.16$	$-0.17$	$-0.12$	$-0.17$	$0.40$
learn_verif_fl	$-0.02$	$-0.01$	$-0.04$	$-0.25$	$0.34$
ai_sentiment †	$\mathbf{0.57}$	$\mathbf{0.57}$	$0.38$	$0.03$	$0.00$
ai_benefit_cnt	$-0.02$	$0.16$	$0.21$	$0.03$	$-0.06$
frustrat_cnt	$0.00$	$0.00$	$0.00$	$0.00$	$0.00$

$x_u$ = ai_usage_freq, $x_t$ = ai_trust_score, $x_c$ = ai_complex_r. $x_d$ = ai_full_dep, $x_v$ = trust_verification_flag.

Figure 2. Pearson correlation heatmap between 16 training features and 5 COI label-generating variables. Red indicates positive correlation, blue negative. Bold cells mark $\left|r\right|>0.40$; features marked † were excluded from the predictive model.

5.3. Evaluation Protocol

All models are evaluated under four split strategies applied in order of increasing generalization difficulty:

• Random IID: Stratified 80/20 split, providing an optimistic performance ceiling.
• User-based GroupKFold: Five folds keyed on hashed user identifiers, preventing user-level leakage in conversation datasets.
• Temporal: Training on 2024 survey data, testing on 2025, assessing cross-year stability.
• Cross-dataset leave-one-out: Training on all datasets except one, testing on the held-out dataset, repeated for each dataset.

Each model is trained with ten independent random seeds. Mean ± standard deviation is reported for all metrics. Pairwise model comparisons use McNemar’s test with Bonferroni correction. Multi-model ranking uses the Friedman test with Nemenyi post hoc comparisons and a critical difference diagram.

5.4. Evaluation Metrics

Classification performance is measured using the Matthews Correlation Coefficient (MCC) (Matthews, 1975), Macro-F1, and Cohen’s Kappa (Cohen, 1960):

$$\mathrm{MCC}={\displaystyle \frac{TP·TN-FP·FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}$$

(18)

where $TP$, $TN$, $FP$, and $FN$ are the true and false positive and negative counts. Equation (18) gives the binary form; for the three offloading levels, MCC is computed in its multiclass generalization from the full $3\times 3$ confusion matrix. It ranges from $-1$ to 1, with 0 for chance-level prediction.

$$\kappa ={\displaystyle \frac{p_o-p_e}{1-p_e}}$$

(19)

where $p_o$ is the observed agreement between predicted and reference labels and $p_e$ is the agreement expected by chance, so $\kappa =0$ indicates chance-level agreement.

MCC is the primary metric as it accounts for all confusion matrix cells under class imbalance. Macro-F1 is the unweighted mean of the per-class F1 scores, so each offloading level counts equally. For the regression objective, Mean Absolute Error (MAE) and $R^2$ are reported. Calibration is assessed via Expected Calibration Error (ECE) over ten equal-width probability bins. Construct validity is assessed using the Intraclass Correlation Coefficient ICC$(A,1)$ (Koo & Li, 2016), Pearson r for convergent validity against physiological reference data from WESAD and STEW, and Cronbach’s $\alpha$ for internal consistency of the COI feature set. All metrics are reported as mean ± SD across ten random seeds with 95% bootstrap confidence intervals over 1000 resamples. Accuracy is omitted, as it is misleading under the observed class imbalance and is subsumed by MCC.

5.5. Baseline Models

Eight models are evaluated: Logistic Regression (Bishop, 2006), Random Forest (Breiman, 2001), XGBoost (Chen & Guestrin, 2016), LightGBM (Ke et al., 2017), TabNet (Arik & Pfister, 2021), LSTM (Hochreiter & Schmidhuber, 1997), fine-tuned RoBERTa (Liu et al., 2019), and the proposed COB-ML stacking ensemble. An occupation-only baseline (Logistic Regression on Layer 1 participant attributes only) is included as the lower-bound comparator for H1. All baselines share identical preprocessing, normalization, and split strategies as COB-ML. Primary metrics are Matthews Correlation Coefficient (MCC), Macro-F1, and Cohen’s Kappa, reported as mean ± SD across ten independent random seeds with 95% bootstrap confidence intervals over 1000 resamples.

6. Results

This section reports experimental results for all four hypotheses and the occupational parity check. Results are presented in order of increasing methodological complexity: from single-dataset behavioral classification through cross-dataset generalization to physiological construct-validation. All metrics are reported as mean ± SD across ten random seeds unless otherwise stated.

6.1. Behavioral Feature Superiority (H1)

Table 6 reports classification performance for all models on the Stack Overflow behavioral dataset under stratified ten-seed evaluation. The occupation-only baseline, trained exclusively on Layer 1 participant attributes, achieves a Macro-F1 of $0.375\pm 0.003$ and MCC of $0.070\pm 0.005$, confirming that demographic and occupational features alone carry negligible predictive signal for cognitive offloading level.

Table 6. Classification performance on SO behavioral dataset (H1). Mean ± SD across ten random seeds. ^† Occupation-only lower bound for H1. ^‡ RoBERTa trained and evaluated on WildChat-1M conversation text; evaluated across three independent seeds; reported as a within-domain cross-modality reference, not a directly comparable baseline. Pairwise McNemar tests with Bonferroni correction ($\alpha =0.05/6=0.0083$): all six within-modality model pairs are significantly different ($p\le 0.008$, see Table 7).

Model	MCC	Macro-F1	$\mathit{\kappa }$
Panel A: Within-modality (SO survey data, 10 seeds)
Logistic Regression	0.329 ± 0.004	0.544 ± 0.002	0.328 ± 0.004
Random Forest	0.367 ± 0.006	0.571 ± 0.004	0.365 ± 0.006
XGBoost	0.373 ± 0.007	0.564 ± 0.005	0.371 ± 0.007
LightGBM	0.371 ± 0.007	0.575 ± 0.004	0.370 ± 0.007
TabNet	0.355 ± 0.006	0.553 ± 0.008	0.354 ± 0.006
LSTM	0.358 ± 0.006	0.553 ± 0.004	0.357 ± 0.005
COB-ML Ensemble	0.373 ± 0.004	0.577 ± 0.003	0.371 ± 0.004
Occ.-only †	0.070 ± 0.005	0.375 ± 0.003	0.069 ± 0.005
Panel B: Within-domain cross-modality reference (WildChat-1M text, 3 seeds ‡)
RoBERTa	0.635 ± 0.006	0.746 ± 0.005	0.614 ± 0.007

Behavioral proxy features produce consistent and substantial improvements across all model families. Logistic Regression achieves a Macro-F1 of $0.544\pm 0.002$, representing a gain of $+0.169$ over the occupation-only baseline and confirming H1 at the threshold of $\mathrm{\Delta }\mathrm{F}1>0.15$. Gradient-boosted models (XGBoost: $0.564\pm 0.005$, LightGBM: $0.575\pm 0.004$) outperform linear classifiers, consistent with the nonlinear interaction structure expected among behavioral features. TabNet ($0.553\pm 0.008$) and LSTM ($0.553\pm 0.004$) perform comparably, suggesting that attentive feature selection and sequential modeling offer marginal advantages over gradient boosting on cross-sectional survey data. Pairwise McNemar tests with Bonferroni correction confirm that all six within-modality model pairs are statistically distinguishable ($p\le 0.008$, Table 7), establishing that performance differences are not attributable to sampling variance.

Table 7. Pairwise McNemar significance tests between within-modality models. Bonferroni-corrected threshold $p<0.0083$. A checkmark (✔) indicates that the difference is statistically significant after correction. All pairs significant.

Model A	Model B	${\mathit{\chi }}^2$	p	Sig?
Logistic Regression	Random Forest	9.025	0.003	✔
Logistic Regression	XGBoost	53.324	<0.001	✔
Logistic Regression	LightGBM	23.903	<0.001	✔
Random Forest	XGBoost	22.486	<0.001	✔
Random Forest	LightGBM	8.309	0.004	✔
XGBoost	LightGBM	7.139	0.008	✔
Bonferroni threshold: $\alpha =0.05/6=0.0083$. All comparisons significant.

The fine-tuned RoBERTa model, operating on real conversation text from WildChat-1M, achieves the highest single-modality performance at Macro-F1 $=0.746\pm 0.005$ and MCC $=0.635\pm 0.006$ across three independent seeds, demonstrating that semantic conversation content carries substantially richer cognitive offloading signal than structured behavioral features alone. The $+0.372$ F1 gap to the occupation-only baseline is reported for reference only, since the two models are evaluated on different data. This RoBERTa result is reported as a within-domain reference rather than a directly comparable baseline, since Branch A is both fine-tuned and evaluated on WildChat text, whereas all other models in this table are trained and evaluated on the structurally distinct Stack Overflow survey data; the two are not measuring generalization to the same test distribution. Figure 3 presents normalized confusion matrices for all models, revealing consistent patterns in class-level error structure across the full model family. Recall for the Medium class is the lowest for every model (34.2–49.2%, against 59.0–68.5% for High and Low), and High and Low are confused with each other (9.0–15.6%) less often than with Medium.

Figure 3. Normalized confusion matrices for all COB-ML models, arranged left-to-right, top-to-bottom: (a) Logistic Regression, (b) Random Forest, (c) XGBoost, (d) LightGBM, (e) TabNet, (f) LSTM, (g) RoBERTa (fine-tuned, within-domain reference), (h) COB-ML Ensemble ($n=\mathrm{14,135}$ test instances). Green diagonal cells indicate correct classification. Medium is the hardest class across all models, consistent with the continuous nature of the COI construct.

The COB-ML stacking ensemble achieves a Macro-F1 of $0.577\pm 0.003$, comparable to the best tabular models. The modest ensemble gain over individual tabular models reflects a domain alignment limitation: the RoBERTa branch was fine-tuned on WildChat conversation text while tabular branches operate on Stack Overflow survey features, creating a domain mismatch at the meta-learner level. The full contribution of combining branches trained on distinct datasets is evaluated under the generalization protocol in Section 6 (H3), where this same domain-mismatch caveat is carried through explicitly.

H1 is confirmed: behavioral proxy features outperform the occupation-only baseline on the same Stack Overflow data by $\mathrm{\Delta }\mathrm{F}1=+0.200$ (LightGBM) and $+0.169$ (Logistic Regression), both exceeding the pre-specified threshold of $\mathrm{\Delta }\mathrm{F}1>0.15$. The RoBERTa gain ($+0.372$) is a within-domain reference.

6.2. Verification Behavior as Dominant Predictor (H2)

Figure 4 presents the SHAP beeswarm plot for the top ten features ranked by mean absolute SHAP value across all 14,135 test instances. The top-five feature ranking is perfectly stable across ten independent random seeds (Spearman $r=1.000$, min $=1.000$), confirming that the identified predictors are not artifacts of initialization variance. The SHAP analysis uses the full 22-feature set including COI label-generating variables, as its purpose is construct interpretation rather than independent predictive discovery. A separate leakage-controlled analysis on independent predictive features is reported in the subsection below.

Figure 4. SHAP beeswarm plot showing mean absolute SHAP values for the top 10 features across all test instances. Top-5 ranking is perfectly stable across ten random seeds (Spearman $r=1.000$).

SHAP analysis reveals a consistent hierarchy of cognitive offloading predictors. ai_trust_score emerges as the dominant feature (mean $\left|\varphi \right|=2.001$), reflecting that a worker’s self-assessed trust calibration toward AI outputs is the strongest signal of offloading level. Because ai_trust_score is one of the five proxy inputs (Equation (17)), its rank is partly by construction and is read as construct structure, not as independent predictive evidence. ai_complex_rating ranks second (mean $\left|\varphi \right|=1.661$), indicating that workers who rate AI as highly capable of handling complex tasks exhibit systematically higher offloading. ai_usage_freq ranks third (mean $\left|\varphi \right|=1.185$), confirming that usage frequency carries predictive signal but is subordinate to trust and complexity perception.

trust_verification_flag ranks fourth (mean $\left|\varphi \right|=0.963$), constituting the highest-ranked purely behavioral interaction trace feature and the only feature derived directly from observable verification behavior rather than self-reported attitude. The dependence plot in Figure 5 confirms a strong directional effect: workers with trust_verification_flag$=0$ (no verification behavior) receive large positive SHAP values toward High offloading, while workers with trust_verification_flag$=1$ receive negative SHAP values, actively suppressing the High offloading prediction regardless of trust score. ai_full_dependency ranks fifth (mean $\left|\varphi \right|=0.178$), consistent with its role as the binary maladaptive offloading indicator defined in Section 3.

Figure 5. SHAP dependence plot for trust_verification_flag. Workers with no verification behavior (=0) receive large positive SHAP values toward High offloading. Color encodes ai_trust_score.

Occupation, experience, and organizational features (WorkExp, OrgSize, Employment) collectively account for mean $\left|\varphi \right|<0.05$, confirming that demographic characteristics contribute negligible predictive signal relative to behavioral and attitudinal features. This result directly corroborates H1. The construct-interpretation analysis and its associated figures therefore serve a theoretical purpose only and are not used to evaluate predictive model performance.

Leakage-Controlled Predictive SHAP

To address the concern that construct-interpretation SHAP rankings are influenced by label-generating variables, a separate analysis was conducted on the 14 independent predictive features only. ai_sentiment and ai_complex_avoidance were additionally excluded following Pearson correlation screening against label-generating variables, with $r=0.572$ and $r=0.581$ respectively; full results are reported in Table 5. Under strict leakage control, human_help_count emerges as the dominant predictor at mean $\left|\varphi \right|=0.165$, followed by ai_benefit_count at mean $\left|\varphi \right|=0.159$. human_help_count correlates at $r=0.40$ with the verification flag and was retained at the screening threshold. Both features rank above all demographic and organizational features, with the top-5 ranking stable across ten seeds at Spearman $r=1.000$. These signals do not require the self-reported trust score, which suggests the framework could be applied without a dedicated trust instrument, although both features are themselves survey items. Figure 6 presents the leakage-controlled rankings.

Figure 6. Leakage-controlled SHAP feature importance. Label-generating variables, ai_sentiment and ai_complex_avoidance, were excluded. human_help_count and ai_benefit_count are the two highest-ranked predictors (mean $\left|\varphi \right|=0.165$ and $0.159$ respectively).

H2 is supported: trust_verification_flag ranks first among all Layer 5 features under construct interpretation (mean $\left|\varphi \right|=0.963$, Spearman $r=1.000$ across all seeds). Under the leakage-controlled predictive model, human_help_count (mean $\left|\varphi \right|=0.165$) is the dominant Layer 5 predictor and indicates conditions under which workers retain human judgment. The two analyses point the same way, but the first is partly by construction (the verification flag is a proxy input) and the second excludes usage frequency (also a proxy input). H2 is therefore supported rather than conclusively established.

6.3. Cross-Dataset Generalization (H3)

Table 8 reports Macro-F1 across four evaluation protocols of increasing generalization difficulty: random IID split, user-based GroupKFold, temporal split training on 2024 survey and testing on 2025, and a cross-dataset split training on Stack Overflow structured features and testing on WildChat conversation behavioral features. This cross-dataset protocol is out-of-domain for the tabular branches (Branch B, trained on Stack Overflow) but in-domain for Branch A (RoBERTa, fine-tuned directly on WildChat); this asymmetry is discussed explicitly below and should be kept in mind when interpreting the ensemble’s reported gain. The Friedman test confirms statistically significant performance differences across split conditions ${\chi }^2=8.857$, $p=0.012$, establishing that generalization difficulty varies meaningfully across protocols and that model rankings are not invariant to distribution shift.

Table 8. Macro-F1 Across Generalization Protocols (H3). Friedman ${\chi }^2=8.857$, $p=0.012$. $\mathrm{\Delta }$F1 = Cross-DS minus IID. RoBERTa and the ensemble’s Cross-DS column reflect a split that is within-domain for Branch A; see text.

Model	IID	GroupKFold	Temporal	Cross-DS	$\mathbf{\Delta }$F1
Log. Regression	0.544	0.544	0.526	0.382	−0.162
Random Forest	0.571	0.571	0.535	0.446	−0.125
XGBoost	0.564	0.563	0.467	0.446	−0.118
LightGBM	0.575	0.575	0.491	0.446	−0.129
TabNet	0.553	0.553	0.531	0.498	−0.055
LSTM	0.553	0.553	0.437	0.402	−0.151
RoBERTa	0.747	—	0.747	0.747	0.000
COB-ML Ens.	0.577	—	0.469	0.762	+0.185

—: RoBERTa/Ensemble trained on WildChat; GrpKF n/a on SO data.

Multi-model ranking across ten cross-validation folds confirms statistically significant performance differences among models, with Friedman ${\chi }^2=39.280$, $p<0.0001$, $k=5$, and $N=10$ folds. Nemenyi post hoc comparisons at CD $=1.929$ and $\alpha =0.05$ identify three non-significant cliques: LightGBM, Random Forest, and XGBoost form a statistically equivalent performance group at $\mathrm{\Delta }\mathrm{rank}\le 1.90$, with XGBoost and Logistic Regression, and Logistic Regression and the Demographic Baseline, each differing by only 1.00 rank, while the Demographic Baseline is significantly outperformed by all three tree-based ensemble models, with $\mathrm{\Delta }\mathrm{rank}=3.90$ for LightGBM, $3.10$ for Random Forest, and $2.00$ for XGBoost. Logistic Regression is significantly outperformed by LightGBM (2.90) and Random Forest (2.10). Figure 7 presents the critical difference diagram.

Figure 7. Critical difference diagram; Nemenyi post hoc test (CD = 1.929). Connected models are not significantly different. Numbers in parentheses are mean ranks (1 = best). LightGBM, Random Forest, and XGBoost form a statistically equivalent clique; XGBoost and Logistic Regression, and Logistic Regression and the Demographic Baseline, are also not significantly different; the Demographic Baseline is significantly outperformed by all three tree-based ensemble models.

All tabular models exhibit consistent F1 degradation under distribution shift. Logistic Regression sustains the largest absolute drop $\mathrm{\Delta }\mathrm{F}1=-0.162$, reflecting its limited capacity to capture nonlinear behavioral patterns that do not transfer across survey waves. Tree-based ensemble models XGBoost $-0.118$, LightGBM $-0.129$, and Random Forest $-0.125$ show moderate but consistent degradation, all reaching 0.446 under the cross-dataset split, suggesting a shared behavioral signal that generalizes to WildChat interaction patterns. TabNet degrades least (a drop of 0.055, reaching 0.498), whereas Logistic Regression (0.382) and LSTM (0.402) fall below this level. The fine-tuned RoBERTa model shows zero degradation $\mathrm{\Delta }\mathrm{F}1=0.000$ across all protocols, maintaining Macro-F1 $=0.747$ from IID through cross-dataset conditions. This stability is not evidence of true cross-domain generalization: RoBERTa was fine-tuned directly on WildChat conversation content, so the cross-dataset condition is equivalent to its training domain, and its $\mathrm{\Delta }\mathrm{F}1=0.000$ should be read as a within-domain evaluation artifact rather than a demonstration of robustness to distribution shift.

GroupKFold evaluation confirms that user-level leakage does not inflate IID results. Tabular model F1 scores under GroupKFold are within 0.001 of their IID counterparts, indicating that the behavioral signal generalizes to unseen respondents.

The COB-ML stacking ensemble achieves the strongest reported cross-dataset performance at Macro-F1 $=0.762$. Because this figure is driven substantially by the meta-learner upweighting Branch A’s within-domain WildChat signal, it should be interpreted as the ensemble’s performance under this specific evaluation protocol rather than as unambiguous evidence of genuine cross-domain robustness; the tabular branches alone, which are the components actually tested out-of-domain in this split, degrade to a range of 0.382 to 0.498 (Table 8). A cross-dataset split that is out-of-domain for every branch, including Branch A, is identified as a priority for future evaluation and is discussed further in Section 9.

H3 is partially confirmed: the COB-ML stacking ensemble achieves the highest reported cross-dataset Macro-F1 of $0.762$ among all models, with statistically significant performance differentiation across protocols ($p=0.012$). The tabular-only branches, which are genuinely evaluated out-of-domain under this protocol, are outperformed by the ensemble (0.762 versus 0.382 to 0.498; no pairwise test was run for this contrast), supporting H3 for that comparison. However, because Branch A is evaluated within its own training domain under this same split, the ensemble’s headline gain is not unambiguous evidence of cross-domain generalization at the full-architecture level, and this qualification should accompany any citation of the $0.762$ figure. Figure 8 and Table 8 report the full results.

Figure 8. Macro-F1 across three generalization protocols for all COB-ML models. The ensemble achieves the highest reported cross-dataset F1 ($0.762$), while single-modality tabular models, genuinely evaluated out-of-domain, degrade consistently under distribution shift. Friedman ${\chi }^2=8.857$, $p=0.012$.

6.4. Physiological Construct Validity (H4)

Physiological construct validity was evaluated for all eight models against both WESAD and STEW reference data, using the Intraclass Correlation Coefficient, convergent Pearson correlation, and internal consistency as complementary indicators of alignment between behavioral COI predictions and physiological signals. Figure 9 presents ICC(A,1) values across all eight models and both physiological validation datasets, and Figure 10 reports convergent validity and calibration metrics.

Figure 9. ICC(A,1) heatmap across all models and physiological datasets. Blue borders indicate $\mathrm{ICC}\ge 0.60$. At subject level on WESAD, XGBoost reaches ICC $=0.611$, whereas Logistic Regression and Random Forest yield negative values.

Figure 10. H4 construct validity summary. (Left): $\mathrm{ICC}(A,1)$ with 95% CI. (Center): Pearson r convergent validity. (Right): Cronbach’s $\alpha$ and ECE calibration.

XGBoost achieves $\mathrm{ICC}(\mathrm{A},1)=0.611$ ($p=0.008$, 95% CI $[0.15,0.85]$) against WESAD at the subject level ($n=15$), marginally exceeding the pre-specified threshold of 0.60. Negative ICC values for Logistic Regression ($-0.024$) and Random Forest ($-0.054$) show that alignment with WESAD is not obtained by every model. STEW window-level $\mathrm{ICC}(A,1)$ values should be interpreted as a design artifact rather than an independent validity finding. STEW employs a perfectly balanced design in which each subject contributes equal proportions of rest and high-workload windows ($\overline{x}=0.50$, $\sigma =0.00$), which maximizes between-window variance and inflates ICC estimates by construction. The near-perfect values observed (LightGBM: 0.982, XGBoost: 0.963, LSTM: 0.837, Random Forest: 0.792) therefore reflect this balanced design property rather than genuine construct alignment evidence. Subject-level ICC is not computable for STEW for the same reason. These results are retained for completeness but are not used to support H4.

Convergent Pearson correlations indicate that COB-ML predictions covary with the physiological reference data: WESAD Pearson $r=0.604$ ($p=0.017$) and STEW Pearson $r=0.971$ ($p<0.001$), both exceeding the pre-specified threshold of $r\ge 0.50$. As with the ICC, the STEW value reflects a balanced window-level design and is not treated as independent evidence.

Cronbach’s $\alpha =0.460$ for the COI feature set is below the conventional 0.70 threshold, reflecting the theoretically expected multidimensionality of the cognitive offloading construct rather than measurement failure; high internal consistency would imply feature redundancy, which is not expected for indicators that capture distinct attitudinal and behavioral facets. ECE $=0.015$ indicates good probability calibration over ten bins. Table 9 shows that four of the five counted metrics meet their thresholds (WESAD ICC and Pearson, STEW Pearson, ECE). Cronbach’s $\alpha$ does not, and the STEW ICC is excluded.

Table 9. Summary of construct validity metrics for H4. Checkmarks indicate the pre-specified threshold was met; the STEW ICC value is reported for completeness but reflects the dataset’s balanced experimental design rather than independent evidence and is therefore not used to support H4.

Metric	Dataset	Value	p	Met?
$\mathrm{ICC}(A,1)$	WESAD (subj.)	0.611	0.008	✔
$\mathrm{ICC}(A,1)$	STEW (win.)	0.964	<0.001	†
Pearson r	WESAD	0.604	0.017	✔
Pearson r	STEW	0.971	<0.001	✔
Cronbach’s $\alpha$	SO	0.460	—	✕
ECE	Ensemble	0.015	—	✔

^† Balanced-design artifact; not used to support H4.

H4 is preliminarily and indirectly supported: $\mathrm{ICC}(A,1)=0.611$ against WESAD physiological reference data exceeds the pre-specified threshold of 0.60, falling within the moderate reliability range under standard interpretation (Koo & Li, 2016). This agreement should not be read as physiological validation of the offloading construct itself: WESAD was designed to capture acute stress responses under a laboratory induction protocol, not maladaptive AI reliance, and no physiological signal was used to train the deployed predictor. The 95% confidence interval $[0.15,0.85]$ is wide, reflecting the inherent statistical constraint of $n=15$ physiological subjects in WESAD rather than instability in COB-ML predictions. Convergent Pearson $r=0.604$ ($p=0.017$) independently corroborates this indirect alignment between behavioral COI predictions and physiological stress signals. Full validation against in situ professional AI use physiological data remains a priority for future work.

6.5. Occupational Parity Check

Table 10 and Figure 11 report Macro-F1 across eight developer occupation groups on the held-out test set. Performance ranges from $0.534$ (Front-end Developer, $n=710$) to $0.585$ (Other, $n=487$), yielding a Disparate Impact Ratio of:    

$$\mathrm{DIR}={\displaystyle \frac{F{1}_{\min }}{F{1}_{\max }}}={\displaystyle \frac{0.534}{0.585}}=0.913$$

(20)

Table 10. Macro-F1 per Occupation Group. DIR $=0.913\ge 0.80$ (parity threshold met).

Occupation	n	Macro-F1
Other	487	0.585
Desktop/Enterprise Developer	540	0.574
Back-end Developer	2163	0.567
Mobile Developer	445	0.567
Full-stack Developer	4292	0.559
Student	1050	0.552
Software Architect	412	0.537
Front-end Developer	710	0.534
DIR	$0.913$

Figure 11. Macro-F1 per developer occupation group. Green badge confirms Disparate Impact Ratio $=0.913\ge 0.80$. Color gradient encodes relative F1 from red (lowest) to green (highest).

The DIR of $0.913$ exceeds the standard fairness threshold of $0.80$, indicating no performance disparity across occupational groups beyond that threshold. The $0.051$ absolute F1 range across eight groups is narrow, consistent with the framework’s reliance on behavioral interaction traces rather than occupation-specific demographic features.

Four groups perform below the overall Macro-F1 of $0.564$: Front-end Developers ($F1=0.534$), Software Architects ($F1=0.537$, $n=412$), Students ($F1=0.552$, $n=1050$), and Full-stack Developers ($F1=0.559$). This may reflect more heterogeneous offloading patterns that are harder to classify from behavioral proxies alone.

COB-ML maintains equitable performance across all eight developer occupation groups (DIR $=0.913$, threshold $\ge 0.80$), with no group falling below the 80% relative performance floor. This result supports occupational parity in classification performance.

Fairness evaluation was further extended beyond occupational groups to four demographic dimensions: employment type, education level, remote work status, and organization size. Employment type achieves DIR $=0.922$, education level DIR $=0.901$, and organization size DIR $=0.831$, all exceeding the 0.80 threshold. Remote work status produces DIR $=0.777$, the only dimension falling below threshold, driven by a performance gap between in-person workers at $F1=0.503$ and hybrid remote-lean workers at $F1=0.391$. This disparity may reflect greater behavioral heterogeneity among hybrid workers whose AI interaction patterns span both collaborative and independent work contexts not fully distinguished by the current feature set. Targeted feature engineering for hybrid work contexts is identified as a priority for future work. Table 11 reports the full expanded fairness results.

Table 11. Expanded fairness evaluation across four demographic dimensions. DIR $\ge 0.80$ indicates equitable performance. Remote work status falls below threshold.

Group	Subgroup	N	F1	DIR
Employment	Employed, full-time	4313	0.431	0.922 ✔
	Independent contractor	1505	0.444
	Student	542	0.425
Education	Professional degree	677	0.484	0.901 ✔
	Bachelor’s degree	5879	0.479
	Something else	108	0.436
Remote Work	In-person	2075	0.503	0.777 ✕
	Remote	4197	0.470
	Hybrid (remote-lean)	1053	0.391
Org Size	20–99 employees	2326	0.495	0.831 ✔
	500–999 employees	769	0.491
	<20 employees	850	0.411

6.6. Branch Contribution Ablation Study

Table 12 and Figure 12 report Macro-F1 and MCC across four branch configurations evaluated under stratified 80/20 splits across ten independent seeds, examining what each branch contributes to the COB-ML architecture.

Table 12. Ablation study: branch contribution analysis. Mean ± SD across ten seeds. McNemar tests confirm Branch B + C significantly outperforms each single-branch configuration ($p<0.0001$, Bonferroni corrected).

Configuration	n	MCC	Macro-F1	$\mathit{\kappa }$
Cross-modality reference
Branch A (RoBERTa, WildChat)	—	$0.635\pm 0.006$	$0.746\pm 0.005$	$0.614\pm 0.007$
Within-modality ablation (SO survey, 10 seeds)
Demographic Baseline (L1) ***	6	$0.146\pm 0.004$	$0.415\pm 0.003$	$0.140\pm 0.004$
Branch B (Tabular L1 + L2) ***	10	$0.157\pm 0.008$	$0.427\pm 0.005$	$0.153\pm 0.007$
Branch C Proxy (Behavioral L5 + L6) ***	4	$0.209\pm 0.004$	$0.400\pm 0.005$	$0.183\pm 0.004$
Branch B + C (Full Clean)	14	$0.244\pm 0.006$	$0.474\pm 0.004$	$0.239\pm 0.006$
COB-ML Full Ensemble	—	$0.373\pm 0.004$	$0.577\pm 0.003$	$0.371\pm 0.004$

*** McNemar $p<0.0001$ vs. Branch B + C, Bonferroni $\alpha =0.0167$.

Figure 12. Branch contribution ablation study. (Top panel): Macro-F1 with error bars across ten seeds. (Bottom panel): MCC. Branch C proxy achieves the highest MCC among single-branch configurations despite using only four features, indicating that behavioral interaction traces carry relatively strong signal per feature. Branch B + C combination produces statistically significant gains over each single branch ($p<0.0001$, McNemar, Bonferroni corrected). *** denotes significance vs. Branch B + C.

The Demographic Baseline at Layer 1 only ($n=6$ features) achieves Macro-F1 $=0.415\pm 0.003$, which serves as the demographic lower bound for this ablation. Branch B (tabular contextual features, Layers 1–2, $n=10$) produces a marginal gain to $0.427\pm 0.005$, indicating that organizational context features add limited independent signal beyond demographics alone. Branch C proxy (behavioral features only, Layers 5–6, $n=4$) achieves Macro-F1 $=0.400\pm 0.005$ but the highest single-branch MCC at $0.209\pm 0.004$, indicating that behavioral interaction traces carry the most discriminative signal per feature despite limited coverage. The combination of Branches B and C (full clean set, $n=14$) produces a statistically significant gain to Macro-F1 $=0.474\pm 0.004$, with McNemar tests confirming significance against all single-branch configurations at $p<0.0001$, Bonferroni corrected at $\alpha =0.05/3=0.0167$. The full COB-ML ensemble incorporating Branch A (RoBERTa semantic embeddings) achieves Macro-F1 $=0.577\pm 0.003$, a further gain of $+0.103$ over the tabular-only combination, supporting the multi-branch architecture.

7. Discussion

7.1. Trust Disposition and Verification Behavior: A Dual Account

The most consequential finding of this study is not the support for H2 but its productive revision: the data reveal a richer predictor hierarchy than anticipated, with the relevant signal differing depending on whether self-reported attitude is included. Under construct interpretation, ai_trust_score is the dominant SHAP-ranked feature (mean $\left|\varphi \right|=2.001$), more than twice that of trust_verification_flag (0.963). This suggests that trust disposition, not observable verification behavior alone, is the strongest correlate of maladaptive offloading in this dataset: workers who report uncritical trust in AI outputs tend not to verify them. Both features are proxy inputs, so this ranking is partly by construction.

Under leakage-controlled predictive evaluation, which excludes ai_trust_score and other label-generating variables, human_help_count emerges as the dominant leakage-controlled predictor (mean $\left|\varphi \right|=0.165$). This indicates that low trust calibration’s behavioral correlate, retaining fewer conditions for human judgment, is an observable signal that does not require self-reported trust measures. Together, trust disposition is the strongest theoretical correlate of the construct, while human_help_count is what a monitoring system can observe without a trust survey.

These associations are consistent with, but do not establish, a causal account. SHAP identifies predictive features; it does not isolate trust calibration as a cause, nor rule out confounds such as task difficulty, individual ability, prior AI experience, and time pressure, any of which could independently influence both reported trust and verification behavior. A causal role would require experimental or longitudinal evidence beyond this cross-sectional design.

This pattern is consistent with the game-theoretic insight that humans systematically underinvest in verification resources with automated agents (Zahedi et al., 2025), and with the recognition-primed decision lens introduced in Section 2 (Klein, 1993), under which trust calibration may be one factor associated with whether deliberate verification is engaged at all. Organizations monitoring only behavioral traces such as prompt count may be observing a downstream correlate of a cognitive disposition rather than its cause. Addressing trust calibration through system transparency, AI uncertainty communication, and structured verification prompts is a plausible design implication, though not an experimentally validated recommendation.

7.2. Convergence with Recent Literature

These findings are broadly consistent with recent work that measures AI reliance through methods other than classification. H.-P. H. Lee et al. (2025) surveyed 319 knowledge workers and found that higher self-reported confidence in AI was associated with less critical thinking, paralleling the high rank of ai_trust_score in the present SHAP analysis. Jain et al. (2026) compared AI-assisted and manual task performance in young adults alongside measures of cognitive offloading, critical thinking, and attitudes toward AI, a design that likewise contrasts work with and without delegation to AI. Hunter et al. (2024) proposed reliance drills as exercises that test whether users can recognize mistakes in AI-generated advice, operationalizing verification capacity directly rather than inferring it from usage frequency; this is conceptually aligned with trust_verification_flag and human_help_count, though COB-ML derives these from naturalistic interaction traces rather than a controlled drill. Guo et al. (2024) defined reliance as the probability that a decision-maker follows the AI recommendation and separated losses from mis-reliance from losses from misreading the available signals, a separation conceptually related to the present work’s distinction between adaptive and maladaptive offloading in Section 3.

The present study differs from these studies in two respects. Each cited study uses a single design, whereas COB-ML integrates survey, conversation-log, and physiological reference data under one label definition that can be classified and audited at scale. The cost is that COB-ML predicts an author-defined proxy, whereas the cited designs observe reliance directly. Convergence across survey, task-performance, behavioral-test, and decision-theoretic methodologies is consistent with treating trust disposition and verification capacity, rather than adoption frequency, as more informative targets for organizational AI risk monitoring.

7.3. Semantic Conversation Content

The RoBERTa branch reaches Macro-F1 $=0.747$ on WildChat, against $0.575$ for the best tabular model on Stack Overflow data. The datasets differ, so the two scores are not directly comparable, but the result is consistent with semantic content of AI conversations carrying a cognitive offloading signal that structured behavioral features miss. Users who critically evaluate AI outputs are expected to produce linguistically distinct conversation patterns such as follow-up questions, explicit challenges, and requests for justification that are detectable by a fine-tuned language model but invisible to frequency-based features such as turn count. This result suggests implications for AI interface design: conversation content may serve as a real-time behavioral signal of cognitive engagement quality and not merely as an interaction record.

The ensemble’s positive reported gain under the cross-dataset protocol ($\mathrm{\Delta }\mathrm{F}1=+0.185$) is consistent with multimodal stacking being more than additive. The meta-learner learns to weight RoBERTa’s within-domain semantic signal more heavily under this protocol, compensating for tabular model degradation and numerically surpassing RoBERTa’s own score on the same split (0.762 versus 0.747, Table 8). As qualified in Section 6, this comparison should be read as the ensemble’s performance advantage under a protocol that is in-domain for Branch A, rather than as unqualified evidence that the ensemble generalizes better across genuinely unseen domains; the tabular branches, which are evaluated out-of-domain under this same split, are outperformed by the ensemble, although no pairwise test was run for this contrast. This distinction motivates the fully out-of-domain evaluation protocol identified as a priority in Section 9, and in the interim supports maintaining modality separation through the branch structure so that each branch’s domain status remains individually interpretable, rather than early-fusing features into a single opaque representation.

7.4. Construct Validity and Practical Scope

The WESAD $\mathrm{ICC}(A,1)=0.611$ result, achieved by the XGBoost branch, provides indirect cognitive load alignment evidence rather than direct AI offloading validation, and should be interpreted conservatively. The 95% confidence interval $[0.15,0.85]$ reflects the limited number of physiological subjects ($n=15$) available in WESAD and the inherent challenge of aligning behavioral proxies derived from survey self-report with physiological signals measured in a laboratory stress-induction protocol. The behavioral and physiological contexts are not identical: WESAD captures acute stress responses in a controlled setting, while the COI proxy captures chronic AI reliance patterns in professional work. That an ICC of $0.611$ is obtained under these conditions is a modest, indirect signal that the behavioral proxy covaries with physiological stress, even across this methodological gap; it does not by itself establish that the proxy measures the same construct as the physiological reference data, nor that WESAD’s acute stress response is equivalent to maladaptive cognitive offloading.

Convergent Pearson r values of $0.604$ (WESAD) and $0.971$ (STEW) are consistent with this alignment. The STEW window-level $\mathrm{ICC}(A,1)$ values reported in Table 9 and the STEW Pearson value are substantially higher but reflect the balanced rest/high-workload design of that dataset rather than independent evidence of construct validity, as discussed in Section 6, and are not treated as confirmatory here. The low Cronbach’s $\alpha =0.460$ is compatible with a multidimensional construct. High internal consistency would suggest a unidimensional construct, whereas cognitive offloading is expected to span attitudinal, behavioral, contextual, and physiological facets that are partially independent. It is not, however, evidence of validity on its own.

8. Ethical Considerations

The deployment of behavioral monitoring frameworks in organizational contexts raises ethical considerations that extend beyond technical performance. Four issues warrant explicit attention.

COB-ML is designed as a population-level research and organizational monitoring tool, not as an individual surveillance instrument. Deployment in any organizational context must be accompanied by informed worker consent, transparency about what is being measured and why, and clear communication of how COI predictions will and will not be used. COB-ML scores must not be used as the basis for individual employment decisions, including hiring, termination, promotion, performance review, or disciplinary action. This prohibition is absolute and applies regardless of whether independent validation of individual-level predictions has been conducted. The framework is designed exclusively for population-level research and organizational monitoring purposes, and its individual-level predictive validity has not been established to a standard that would justify consequential employment decisions.

The critical ethical boundary for this framework is the distinction between measurement for worker support, which identifies employees who may benefit from AI literacy training or interface modifications, and measurement for performance management. The latter application, in which COI scores are used as productivity metrics or disciplinary instruments, is outside the scope of this framework and would require substantially stronger individual-level validity evidence than we have established.

Expanded fairness evaluation indicates no performance disparity beyond the 0.80 threshold across occupational groups (DIR $=0.913$), employment type (DIR $=0.922$), education level (DIR $=0.901$), or organization size (DIR $=0.831$). Remote work status produces DIR $=0.777$, falling below the 0.80 threshold, which may reflect behavioral heterogeneity among hybrid workers. The framework has not been validated on non-developer professional populations, hourly workers, or workers in non-English-speaking contexts. Deployment outside the validated population scope requires group-specific fairness evaluation before use.

COB-ML requires access to AI conversation logs and survey self-report data. Organizations implementing this framework should apply data minimization principles, retaining only the behavioral features necessary for COI estimation and anonymizing records at the point of collection, rather than retaining raw conversation content for extended periods.

9. Limitations and Future Work

The COI proxy score is derived from theoretically motivated but informally weighted behavioral features. The proxy construction involves assumptions, particularly the relative weighting of trust verification behavior and full dependency indicators, that were not formally derived from behavioral economic theory. Performance is therefore reported against this author-defined proxy and not against directly observed reliance. Future work should derive these weights from utility-theoretic first principles, following the game-theoretic cost estimation approach in Zahedi et al. (2025).

The cross-dataset evaluation protocol used to assess H3 is out-of-domain for the tabular branches but in-domain for the fine-tuned RoBERTa branch, since Branch A is both trained and evaluated on WildChat conversation text. Consequently, the ensemble’s reported cross-dataset Macro-F1 of $0.762$ (Table 8) reflects genuine out-of-domain robustness for the tabular components but not for the architecture as a whole. A cross-dataset split that is out-of-domain for every branch, including a language-model branch fine-tuned or evaluated on conversation data distinct from WildChat, is identified as a priority for future evaluation and is required before the ensemble’s cross-domain generalization claim can be considered fully established.

COB-ML estimates COI from single-session or single-survey snapshots. Cognitive offloading is a dynamic construct that evolves with AI tool familiarity, task demands, and organizational context. Longitudinal session modeling, which tracks COI trajectory within individuals over weeks or months, is the highest priority extension of this framework.

The WESAD validation involves 15 laboratory subjects under controlled stress induction. Field-level physiological validation of COI predictions using wearable EDA or EEG monitoring during naturalistic professional AI use remains an important open problem.

COB-ML is a computational framework evaluated entirely on archival and observational data. A controlled human experiment in which participants complete AI-assisted tasks under varying cognitive load conditions, with COB-ML predictions compared against both physiological measures and independently observed behavioral criteria, would strengthen construct validity claims. This experiment constitutes the primary planned future work and is designed as a two-study program: Study I, establishing natural offloading patterns, and Study II, testing whether COB-ML-guided feedback improves cognitive engagement, following the experimental design paradigm in Zahedi et al. (2025).

Validation is limited to software developers and knowledge workers in English-language AI interaction contexts. Extension to non-developer professional populations and non-English conversation logs requires additional feature engineering and separate validation.

10. Conclusions

This paper introduced COB-ML, a multi-source framework for quantifying human cognitive offloading in AI-integrated professional environments, and evaluated it as a behavioral marker of adaptive versus maladaptive decision making under AI-mediated complexity. The framework operationalizes cognitive offloading as a measurable behavioral construct through a six-layer feature architecture, detects it through a three-branch hybrid stacking ensemble, and explains it through dual TreeSHAP feature attribution with confirmed rank stability across random seeds. Across five real-world datasets spanning 70,673 labeled survey records, 1,837,989 AI conversation logs, and 69,362 physiological measurement windows, COB-ML shows that behavioral interaction traces carry genuine and, for the tabular branches under the cross-dataset protocol tested here, generalizable cognitive offloading signal.

The dominant finding is twofold. Under construct interpretation, AI trust disposition is the strongest signal associated with the COI construct. Under leakage-controlled predictive evaluation, human_help_count is the dominant independent predictor, indicating that directly observable behavioral signals carry meaningful and deployable predictive power without requiring self-reported trust measures. Both analyses refine the prevailing assumption that monitoring behavioral frequency alone is sufficient for cognitive risk assessment.

This distinction matters for three groups. Organizations can add verification and trust-calibration indicators to adoption dashboards, so that dependency risk becomes visible before errors accumulate. Interface designers can test prompts that make verification an explicit step, and training programs can target critical AI evaluation. Researchers gain an auditable, leakage-controlled benchmark for the adaptive versus maladaptive distinction, which synthetic-data frameworks cannot provide. These are design implications rather than tested interventions, and they matter most in high-stakes professional work, where the costs of unverified acceptance are diffuse and delayed.

COB-ML achieves $\mathrm{ICC}(A,1)=0.611$ against WESAD physiological reference data as preliminary, indirect evidence of construct validity, a reported cross-dataset Macro-F1 of $0.762$ under a protocol that is in-domain for the language-model branch (Section 9), and a Disparate Impact Ratio of $0.913$ across eight developer occupation groups. These results position COB-ML as a methodologically transparent framework for measuring cognitive offloading in AI-integrated professional environments, with clearly identified next steps, including a fully out-of-domain generalization test and human-subject validation, toward establishing it on firmer ground.