Quantifying the Fluency Illusion in AI-Augmented Design Education: A Behavioral Soft-Sensor Framework for Decoding Human–AI Collaboration Patterns

Abstract

Generative artificial intelligence (GenAI) has transformed design education, yet growing evidence suggests that the fluency of AI-generated outputs may create a “fluency illusion”—a metacognitive bias whereby learners conflate polished AI artifacts with genuine cognitive mastery. A critical unresolved question is how to quantitatively diagnose this AI-induced fluency illusion without disrupting the natural learning process. This study introduces MBS-AIGC, a purpose-built AI-supported design education platform grounded in the Meaning–Behavior–Spirit (MBS) cultural cognition model for Chinese intangible cultural heritage. Drawing on the industrial soft-sensor paradigm, we computationally formalized six behavioral soft-sensor indicators from the digital interaction traces of 71 undergraduate design students over a four-week instructional period and applied K-means clustering to identify latent engagement patterns. Three distinct human–AI collaboration profiles emerged: Deep Explorers (n = 41), Progressive Builders (n = 16), and Surface Operators (n = 14). Crucially, expert-assessed cognitive flexibility significantly differentiated the three groups (F(2, 68) = 5.66, p = 0.005, η² = 0.143), whereas a conventional self-report questionnaire failed to distinguish among them (F(2, 36) = 0.29, p = 0.748), providing preliminary empirical evidence for the fluency illusion in design education. By addressing the lack of objective diagnostic tools for metacognitive miscalibration, this research contributes a scalable, zero-intrusion behavioral soft-sensor framework that enables educators to decode human–AI collaboration patterns and mitigate the fluency illusion in creative learning environments.

1. Introduction

The rapid proliferation of generative artificial intelligence (GenAI) has fundamentally reshaped the pedagogical landscape [1,2,3,4,5,6,7]. In design education specifically, GenAI enables students to produce visually polished artifacts with unprecedented speed. However, this rapid generation raises a critical question: do these outputs reflect genuine cognitive mastery, or merely superficial engagement [8,9]? A growing body of evidence suggests that the high visual quality of AI outputs may induce a metacognitive bias termed the “fluency illusion” [10]. Rooted in the processing fluency heuristic [11], this illusion occurs when the ease of processing information leads learners to overestimate their actual understanding. In AI-augmented design education, this phenomenon becomes particularly pronounced. When AI systems generate photorealistic images that appear professionally competent, they completely bypass the traditional metacognitive signals of conceptual struggle, such as failed sketches and iterative dead ends [12,13]. Consequently, learners may mistakenly conflate the algorithmic fluency of the AI tool with their own cognitive mastery [14]. Therefore, a core challenge in current educational research is how to effectively quantify and diagnose this AI-induced fluency illusion. Jose [14] demonstrated that AI-driven cognitive offloading significantly erodes critical thinking capacity. Similarly, Lodge and Loble [13] argued that unstructured AI use risks “cognitive atrophy” when metacognitive scaffolding is absent. In the specific context of learning analytics, Ye and Pennisi [15] found that digital-trace data predicted student performance more accurately than self-reported self-regulated learning (SRL) measures. Furthermore, Choi et al. [16] documented significant misalignment between behavioral trace data and self-report surveys. Han and Ellis [17] similarly reported weak alignment between self-reported and digital-trace clusters in blended learning. These converging findings establish that the gap between perceived and actual learning engagement is a robust, cross-context phenomenon. Yet, its specific manifestation in AI-augmented design education remains unexplored, primarily due to the lack of appropriate quantitative measurement tools.

To address these challenges, researchers have increasingly turned to learning analytics to monitor and support student learning processes [18]. Traditional educational measurement relies heavily on self-report instruments, yet the validity of such measures in GenAI contexts is increasingly questioned [15,16]. Winne [18] argued that trace data provide observable indicators supporting valid inferences about metacognitive monitoring and control, offering a more ecologically valid window into self-regulated learning than retrospective surveys. In parallel, multimodal learning analytics (MMLA) has emerged as a promising approach for capturing the complexity of learning processes [19,20,21]. However, most MMLA implementations require specialized hardware—eye trackers, EEG headsets, or wearable physiological sensors [22,23,24]—that are impractical for deployment in naturalistic classroom ecologies. Recent advances in student–AI interaction profiling have demonstrated the value of clustering approaches for identifying behavioral patterns: Fan and Ouyang [25] used learning analytics to unveil human–AI collaborative patterns in instructional design, while Jin et al. [26] proposed RelianceScope, an analytical framework that operationalizes nine fine-grained reliance patterns based on help-seeking and response-use engagement. These studies underscore the growing recognition that understanding how students interact with AI tools requires systematic behavioral analysis beyond simple usage frequency metrics.

In this context, the concept of behavioral soft-sensors offers a compelling alternative. Originating in industrial process control, soft-sensors are computational models that infer hard-to-measure quality variables from readily available process data [27,28]. Kadlec et al. [27] defined data-driven soft-sensors as mathematical models that use easily measured variables (e.g., temperature, pressure, flow rates) to estimate difficult-to-measure quality variables in real time. We propose that this paradigm can be productively transferred to educational settings: just as industrial soft-sensors infer product quality from process signals, behavioral soft-sensors in education can infer cognitive engagement depth from digital interaction traces. The analogy is grounded in three shared characteristics: (1) both rely on readily available, continuously sampled signals (process data/interaction logs) to estimate latent quality variables (product quality/cognitive depth); (2) both operate non-intrusively, without disrupting the process under observation; and (3) both require domain-specific formalization to translate raw signals into meaningful indicators. To clarify the novelty of this approach, it is essential to distinguish behavioral soft-sensors from conventional learning analytics features. While standard analytics often rely on descriptive, low-level metrics (e.g., total click counts, overall time-on-task) that lack direct cognitive mapping, the six behavioral soft-sensors proposed in this study are diagnostic and domain-specific. They are computationally formalized with explicit mathematical definitions tailored to the cognitive structure of the MBS cultural design framework. This formalization translates raw, high-frequency interaction logs into higher-order indicators of cognitive engagement, thereby achieving a level of pedagogical specificity that standard feature engineering in educational data mining typically lacks [29,30].

To bridge this gap, the present study introduces MBS-AIGC, an integrated system innovation comprising (a) a purpose-built AI-supported design education platform grounded in the Meaning–Behavior–Spirit (MBS) cultural cognition model for Chinese intangible cultural heritage, and (b) a behavioral soft-sensor analytics framework that enables zero-intrusion diagnosis of human–AI collaboration patterns. The MBS-AIGC platform represents a system-level innovation that embeds cultural heritage pedagogy into AI-augmented design workflows—a contribution that aligns directly with the scope of applied system innovation. Unlike studies that apply generic AI tools (e.g., ChatGPT, Midjourney) to existing curricula, the MBS-AIGC platform was designed from the ground up to structure student engagement along three culturally grounded cognitive dimensions (Meaning, Behavior, Spirit), enabling the extraction of domain-specific behavioral indicators that capture the unique cognitive demands of cultural translation in design education [31,32,33,34].

Specifically, this study addresses the following research questions (RQs):

RQ1: What distinct behavioral engagement patterns can be identified among design students in an AI-supported learning environment using computationally formalized behavioral soft-sensors?

RQ2: How do these behavioral patterns differ in terms of cognitive outcomes—specifically expert-assessed cognitive flexibility and behavioral indicators of AI dependence—and how does the objective diagnostic efficacy of soft-sensors compare with that of conventional self-report instruments?

To answer these questions, we extracted and formalized six behavioral soft-sensor indicators from the interaction logs of 71 students and applied unsupervised K-means clustering. The resulting three-cluster solution was validated against expert-assessed cognitive flexibility and a self-report Cultural Translation Depth (CTD) questionnaire. The divergence between objective and subjective measures provides the first empirical quantification of the fluency illusion [10] in design education—extending prior theoretical accounts [12,14] with domain-specific behavioral evidence.

The contributions of this study are situated at three levels:

• System innovation: We design and validate the MBS-AIGC platform, an integrated AI-supported design education system. By embedding cultural heritage pedagogy (the Meaning–Behavior–Spirit model) into AI-augmented creative workflows, it demonstrates a novel pathway for digitizing intangible cultural heritage education.
• Methodological contribution: We conceptualize and formalize six domain-specific behavioral soft-sensors with explicit mathematical definitions. This provides a scalable, zero-intrusion alternative to hardware-dependent MMLA approaches for capturing latent cognitive engagement in naturalistic educational settings.
• Empirical contribution: We provide the first quantitative evidence of the fluency illusion in AI-augmented design education. By extending prior theoretical characterizations [10,14] with behavioral trace data, we reveal a systematic divergence between objective cognitive indicators and subjective self-assessments across three distinct human–AI collaboration profiles. This aligns with the emerging research agenda on generative AI and transversal competencies in higher education [35].

2. Materials and Methods

2.1. The MBS-AIGC Platform and Data Acquisition System

The MBS-AIGC platform is a closed-access, web-based learning environment specifically constructed to support design education. It integrates a hierarchical learning-aid system derived from the Meaning–Behavior–Spirit (MBS) theoretical model with generative-AI services enabled by large pre-trained models.

The platform is built on a contemporary web technology stack, including React (v18.2) and Node.js (v18.16) with MariaDB (v10.11), forming an integrated system architecture that incorporates Doubao/Volcengine AI APIs. Unlike traditional LMSs that only track page access, the MBS-AIGC platform implements a higher-frequency log-collection mechanism. It unobtrusively captures fine-grained user operation events—including mouse clicks, hover and dwell times, text inputs, prompt modifications, and AI image-generation requests—at millisecond-level precision. These raw digital records, stored in dedicated database tables (e.g., operation_sequences and mbs_dwell_time), form the fundamental data stream for subsequent soft-sensor computation, as illustrated in Figure 1.

Figure 1. System architecture of the MBS-AIGC platform.

The MBS-AIGC platform differs fundamentally from approaches that simply deploy commercial GenAI tools (e.g., ChatGPT, Midjourney) within existing curricula. Three design principles distinguish it as a system innovation. First, cultural cognitive structuring: the platform enforces a three-phase workflow aligned with the MBS model—students must complete cultural “Meaning” analysis before accessing AI-generation features, ensuring that AI serves as a complement to, rather than a substitute for, cultural cognition. Second, integrated behavioral logging: every interaction—page navigation, module completion, AI API calls, image-editing actions, and time allocation—is automatically captured in a structured event log without requiring any additional instrumentation, enabling the extraction of behavioral soft-sensors as a natural byproduct of the learning process. Third, controlled AI scaffolding: the platform mediates student–AI interaction through structured prompts derived from the MBS dimensions, rather than allowing unconstrained free-form prompting, thereby creating a pedagogically meaningful interaction space that can be systematically analyzed [31,32]. To illustrate this workflow, students first analyze the cultural symbolism using structured MBS cards (Material, Behavior, Spirit) during the Meaning phase. The platform’s prompt-structuring engine then automatically transforms these inputs into a structured visual prompt that explicitly specifies the subject, required cultural elements, style, composition, and constraints, rather than relying on a single free-form template. The logging engine continuously captures events such as ’module_enter’, ’prompt_edit’, ’api_generate’, and ’canvas_modify’, ensuring a comprehensive behavioral trace.

2.2. Computational Formalization of Behavioral Soft-Sensors

To transform raw interaction records into continuous monitoring metrics with educational significance, the behavioral soft-sensors (BS-1 to BS-6) were established (as illustrated in Figure 2). Unlike conventional descriptive statistics, these soft-sensors are computational formulas grounded in time-series feature-extraction algorithms, capable of dynamically reflecting students’ cognitive engagement states.

Figure 2. Behavioral soft-sensor data pipeline.

The adoption of the soft-sensor paradigm from industrial process control [27,28] is motivated by a structural analogy between manufacturing quality monitoring and educational cognitive assessment. In industrial settings, soft-sensors address the challenge of inferring hard-to-measure quality variables (e.g., product composition, viscosity) from easily measured process variables (e.g., temperature, pressure, flow rates) [27]. In educational settings, the analogous challenge is inferring hard-to-measure cognitive variables (e.g., depth of understanding, metacognitive awareness) from easily captured digital interaction traces (e.g., click sequences, time stamps, API calls). Curreri et al. [28] noted that the transferability of soft-sensor models across domains depends on the preservation of the underlying input–output relationship structure. We argue that this condition is satisfied in the educational context: just as process variables causally relate to product quality through physical–chemical mechanisms, interaction behaviors causally relate to cognitive outcomes through learning mechanisms [18]. The six behavioral soft-sensors defined below operationalize this cross-domain transfer by mapping platform interaction signals to educationally meaningful cognitive constructs within the MBS framework.

BS-1: MBS Completeness (${\mathit{C}}_{\mathbf{m}\mathbf{b}\mathbf{s}}$)

The sensor records how many structured-learning task proportions have been acquired in the three-dimensional cognition domain of Meaning, Behavior and Spirit.

$$C_{\mathrm{m}\mathrm{b}\mathrm{s}}={\displaystyle \frac{{\displaystyle {\sum }_{i=1}^3}{w}_i\hspace{0.17em}\delta \left({\mathrm{T}\mathrm{a}\mathrm{s}\mathrm{k}}_i\right)}{{\displaystyle {\sum }_{i=1}^3}{w}_i}}, i=1,2,3$$

(1)

where $i$ is an indicator function (1 if completed, 0 otherwise) and denotes the preset cognitive weight for each dimension. In this study, equal weights ($w_i$ = 1/3) are assigned to each dimension. This egalitarian approach was adopted because there is currently no established empirical standard for differential weighting in this context; future research may calibrate these weights based on expert consensus.

BS-2: Navigation Flexibility (${\mathit{F}}_{\mathbf{n}\mathbf{a}\mathbf{v}}$)

To capture nonlinear exploratory behavior across cognitive dimensions, the sensor definition is the normalized Shannon entropy of the Markov chain transition probability matrix. $p$ is the transition matrix between $N$ pages or modules, and $p_{ij}$ reflects the degree to which it transitions from State $i$ to State $j$;

$$F_{\mathrm{n}\mathrm{a}\mathrm{v}}={\displaystyle \frac{-1}{{\mathrm{l}\mathrm{o}\mathrm{g}}_{2}N}}{\sum }_i{\sum }_j{\pi }_i\hspace{0.17em}{p}_{ij}\hspace{0.17em}{\mathrm{l}\mathrm{o}\mathrm{g}}_2{p}_{ij}$$

(2)

${\pi }_i$ is the stationary distribution probability of State $i$. Higher $F_{\mathrm{n}\mathrm{a}\mathrm{v}}$ values indicate a more complex and nonlinear exploration path, whereas values close to 0 indicate simple linear navigation. In Equation (2), all logarithms are base-2 and normalized to the interval from 0 to 1.

BS-3: Dimension Spread (${\mathit{D}}_{\mathbf{s}\mathbf{p}\mathbf{r}\mathbf{e}\mathbf{a}\mathbf{d}}$)

The indicator determines whether or not all the cognitive modules are distributed fairly. As the inverse of the coefficient of variation for a vector of dwell times $\mathbf{T}=\left[t_{meaning},t_{behavior},t_{spirit}\right]$:

$$D_{\mathrm{s}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{d}}={\displaystyle \frac{\mu \left(\mathbf{T}\right)}{\sigma \left(\mathbf{T}\right)+\epsilon }}$$

(3)

$\mu$ and $\sigma$ represent the mean value and standard deviation of the three-dimensional dwell time distribution, respectively. The term $\epsilon$ is a conservatively small constant (e.g., ${10}^{-6}$) added strictly to prevent division-by-zero mathematical errors in cases where the standard deviation is zero. High $D_{\mathrm{s}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{a}\mathrm{d}}$ indicates even distribution of cognitive efforts in each category.

BS-4: AI Interaction Rate (${\mathit{R}}_{\mathbf{a}\mathbf{i}}$)

The frequency at which artificial intelligence (AI)-assisted features are activated compared with total interactions:

$$R_{\mathrm{a}\mathrm{i}}={\displaystyle \frac{\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{AI} \mathrm{Prompts}\right)+\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{AI} \mathrm{Generations}\right)}{\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{Total} \mathrm{Interactions}\right)}}$$

(4)

It is a continuous-sampling sensor that captures how closely a student has been using artificial intelligence tools at present.

BS-5: Critical Intervention (${\mathit{I}}_{\mathbf{c}\mathbf{r}\mathbf{i}\mathbf{t}}$)

This sensor measures how frequently a student has revised, rejected or modified the AI-generated material in active revision:

$$I_{\mathrm{c}\mathrm{r}\mathrm{i}\mathrm{t}}={\displaystyle \frac{\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{Prompt} \mathrm{Revisions}\right)+\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{Manual} \mathrm{Overrides}\right)}{\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}\left(\mathrm{Total} \mathrm{AI} \mathrm{Generations}\right)}}$$

(5)

A higher $I_{\mathrm{c}\mathrm{r}\mathrm{i}\mathrm{t}}$ level shows that the student has been more proactive in intervening after AI-generated content; conversely, a lower score implies a preference for accepting AI output as is. To handle mathematical edge cases, if a student makes zero AI-generation requests (denominator = 0), the indicator is strictly defined as 0, reflecting an absence of intervention. Regarding its robustness boundary, it is important to note that BS-5 primarily captures intervention frequency. While it effectively identifies active engagement, distinguishing between superficial intervention (e.g., minor batch tweaking) and deep cognitive reconstruction remains a boundary limitation of this purely behavioral metric, highlighting the need for future semantic analysis.

BS-6: Analysis Time Ratio (${\mathit{T}}_{\mathbf{a}\mathbf{n}\mathbf{a}\mathbf{l}\mathbf{y}\mathbf{s}\mathbf{i}\mathbf{s}}$)

The proportion of time that is purely text-based analysis and self-reflection before any action taken by an AI system.

$$T_{\mathrm{a}\mathrm{n}\mathrm{a}\mathrm{l}\mathrm{y}\mathrm{s}\mathrm{i}\mathrm{s}}={\displaystyle \frac{T_{\text{pre-generation}}}{T_{\mathrm{total} \mathrm{session}}}}$$

(6)

Higher values indicate more extensive pre-planning before requesting AI assistance.

All six types of the soft-sensors’ computation formula directly use raw data collected by the logging engine of the MBS-AIGC platform. The data collection and preprocessing methods are described in Section 2.3 below.

2.3. Data Collection and Preprocessing

During a four-week undergraduate design course, data were obtained from students who engaged in a design education project based on the MBS-AIGC platform. In total, 80 students participated. Nine students’ records were omitted because of absence-related missing logs or system technical failures; thus, only N = 71 students had fully recorded soft-sensor data series for clustering. Data cleaning involved removing biologically implausible outlier events, such as module dwell times shorter than 5 s (accidental clicks) or exceeding 60 min of inactivity (idle sessions), ensuring the integrity of the behavioral trace.

Before clustering, the six soft-sensor features (BS-1 to BS-6) were z-score normalized; that is, they all had a mean of zero and unit variance, so as to maintain consistency when calculating distances.

$$z={\displaystyle \frac{x-\mu }{\sigma }}$$

(7)

where $x$ is the original sensor value, $\mu$ is the sample mean, and $\sigma$ is the sample standard deviation.

2.4. Cluster Analysis and Multi-Algorithm Robustness Verification

To identify hidden behavioral patterns, we used unsupervised machine-learning clustering algorithms. Specifically, K-means clustering was implemented using the k-means++ initialization algorithm to ensure convergence stability, with the maximum number of iterations set to 300 and n_init set to 100 to mitigate local minima traps. The optimal number of clusters (K) was selected using the elbow method and silhouette score across K values from 2 to 10. While the silhouette score was moderate, this is characteristic of human behavioral data, which typically exhibit continuous distributions rather than discrete, widely separated clusters [29,36]. The three-cluster solution (K = 3) was ultimately selected because it provided the most robust pedagogical interpretability. Alternative K values either oversimplified the behavioral spectrum (K = 2) or fragmented the sample into excessively small, overlapping subgroups lacking distinct theoretical meaning (K ≥ 4).

To ensure the classification’s generalizability, we compared K-means with Gaussian Mixture Models (GMMs) and Agglomerative Hierarchical Clustering. K-means exhibited both high computational efficiency and high-quality scores, including the silhouette coefficient, compared with the other algorithms, making it more suitable for pedagogical interpretation. All three algorithms showed relatively high consistency in core clustering and member assignment (Adjusted Rand Index = 0.82; normalized mutual information = 0.79). In addition, based on a 1000-iteration bootstrap resampling test, we found that cluster membership assignment was stable in 92.3% of trials. Therefore, the final analysis is based on the K-means result.

2.5. External Validation Indicators

To verify the pedagogical value of the soft-sensor-derived behavioral patterns, three extrinsic indexes were adopted.

(1)

Cognitive flexibility: Two independently blinded expert designers from the field of design education evaluated each student’s final design concept using five-point scales ranging from 1 to 5. The evaluation standards covered five aspects: accuracy of cultural-element recognition, degree of semantic transformation, creativity in visual representation, cross-cultural applicability, and design coherence. The intraclass correlation coefficient (ICC) between the two assessors was 0.87 (95% CI = 0.79–0.92), indicating good agreement. The mean of the two judges’ scores was used as the final cognitive flexibility score.

(2)

AI-generated images: The objective cumulative count of each student’s calls to the AI image-generation API during the project.

(3)

Cultural Translation Depth (CTD): A 15-item self-report questionnaire was completed by a subsample of students (n = 39). Importantly, this questionnaire was not administered uniformly to all participants; rather, it was conditionally triggered by the system. The survey automatically popped up upon exit only if a student had actively engaged with the design canvas for a continuous duration exceeding 15 min. The CTD subsample included 23 Deep Explorers, 8 Progressive Builders, and 8 Surface Operators, which was representative of the full cluster distribution (χ² = 0.09, p = 0.954). The adapted questionnaire, based on a cross-cultural Design Cognition Scale, showed good internal consistency among participants in this study (α = 0.82).

One-way ANOVA was employed to determine whether there were significant differences among the three clusters on the external validation indicators. Effect size was quantified using eta-squared (η²). Prior to ANOVA, the normality of each variable’s distribution was assessed using the Shapiro–Wilk test and homogeneity of variances with Levene’s test. For variables satisfying the normality assumption (e.g., cognitive flexibility), parametric ANOVA with post hoc Tukey HSD tests was applied; for variables violating the assumption (e.g., AI-generated image counts), the non-parametric Kruskal–Wallis H test was used as an alternative.

3. Results

3.1. Descriptive Statistics and Behavioral Soft-Sensor Profiles

Prior to clustering, descriptive statistics were computed for the six behavioral soft-sensors extracted from the 71 students (N = 71). As illustrated in Table 1, these computationally derived indicators exhibited substantial inter-individual variability, attesting to their sensitivity in capturing the diversity of individual learning strategies.

Table 1. Descriptive statistics of the six behavioral soft-sensors (N = 71).

Soft-Sensor Indicator	Mean	SD	Min	Max
BS-1: MBS Completeness (Cmbs)	0.95	0.16	0.25	1.00
BS-2: Navigation Flexibility (Fnav)	0.10	0.13	0.00	0.91
BS-3: Dimension Spread (Dspread)	1.74	2.06	0.00	16.38
BS-4: AI Interaction Rate (Rai)	0.10	0.10	0.00	0.35
BS-5: Critical Intervention (Icrit)	0.74	0.41	0.00	1.00
BS-6: Analysis Time Ratio (Tanalysis)	0.86	0.15	0.26	1.00

The data indicate that students completed an average of 95% of the MBS cognitive scaffold (BS-1, M = 0.95), suggesting high structural compliance. However, the AI Interaction Rate (BS-4, M = 0.10) and Navigation Flexibility (BS-2, M = 0.10) remained relatively low, indicating that most students adopted linear, scaffold-compliant pathways with limited iterative AI engagement. The Critical Intervention ratio (BS-5, M = 0.74) suggests that most students actively edited AI-generated outputs rather than accepting them passively.

3.2. Cluster Analysis: Identification of Behavioral Patterns

To identify pedagogically meaningful latent behavioral patterns, K-means clustering was applied. The within-cluster sum of squares (WCSS) computed across different K values revealed a clear elbow at K = 3 (Figure 3a). Silhouette analysis indicated that K = 3 yielded the strongest interpretable solution, with a mean silhouette score of 0.350 (Figure 3b). Although alternative K values were examined, solutions with more clusters fragmented the sample into excessively small subgroups lacking clear pedagogical interpretation. Accordingly, on the basis of a trade-off between statistical indicators and theoretical interpretability, the three-cluster solution (K = 3) was adopted.

Figure 3. Elbow method (a) and silhouette score (b) for determining the optimal number of clusters.

Figure 4a presents the detailed silhouette plot for the three-cluster solution. Despite a degree of inter-cluster overlap—typical and expected in human behavioral data—the majority of samples (62 of 71) exhibited positive silhouette coefficients, confirming appropriate cluster assignment.

Figure 4. Cluster validation: (a) silhouette plot for cluster assignment; (b) PCA-based visualization of three-cluster separation. The dotted line in (a) represents the mean silhouette value.

Principal component analysis (PCA) was applied for dimensionality reduction to visualize the three clusters in two-dimensional space (Figure 4b). The PCA scatter plot clearly delineates the separation of the three behavioral patterns in feature space.

3.3. Characterization of Three Human–AI Collaboration Behavioral Profiles

Cluster analysis partitioned the 71 students into three distinct groups. One-way ANOVA confirmed that four of the six soft-sensor indicators differed significantly across clusters (BS-1: F(2, 68) = 3.45, p = 0.037; BS-4: F(2, 68) = 42.55, p < 0.001; BS-5: F(2, 68) = 254.29, p < 0.001; BS-6: F(2, 68) = 44.72, p < 0.001), whereas BS-2 (F(2, 68) = 2.15, p = 0.124) and BS-3 (F(2, 68) = 1.43, p = 0.247) did not reach statistical significance, as summarized in Table 2.

Table 2. ANOVA comparison of the three clusters on six behavioral soft-sensors.

Soft-Sensor Indicator	Deep Explorers (n = 41)	Progressive Builders (n = 16)	Surface Operators (n = 14)	F(2, 68)	p
BS-1: MBS Completeness	0.99 (0.05)	0.89 (0.22)	0.89 (0.23)	3.45	0.037
BS-2: Navigation Flexibility	0.09 (0.08)	0.16 (0.23)	0.07 (0.07)	2.15	0.124
BS-3: Dimension Spread	1.71 (1.20)	1.20 (0.47)	2.46 (4.15)	1.43	0.247
BS-4: AI Interaction Rate	0.09 (0.06)	0.22 (0.11)	0.00 (0.01)	42.55	<0.001
BS-5: Critical Intervention	0.99 (0.06)	0.76 (0.28)	0.00 (0.00)	254.29	<0.001
BS-6: Analysis Time Ratio	0.90 (0.07)	0.66 (0.16)	0.97 (0.07)	44.72	<0.001

Figure 5a (radar chart) and Figure 5b (Z-score heatmap) visually depict the standardized feature profiles of the three groups. On the basis of these profiles, the clusters were labeled as follows:

Figure 5. Behavioral soft-sensor profiles of the three clusters: (a) radar chart of normalized values; (b) heatmap of cluster centroids (Z-scores).

Cluster 1: Deep Explorers (n = 41, 57.7%). This group epitomizes a highly self-regulated cognitive mode. They scored highest on Critical Intervention (BS-5 = 0.99) and maintained a high Analysis Time Ratio (BS-6 = 0.90), while showing a relatively low AI Interaction Rate (BS-4 = 0.09). Their Dimension Spread (D_spread = 1.71) indicated comparatively balanced engagement across the MBS cognitive dimensions. Their use of AI was restrained, treating it as an occasional verification tool rather than a primary generative engine.

Cluster 2: Progressive Builders (n = 16, 22.5%). This group was characterized by the highest AI Interaction Rate (BS-4 = 0.22) and moderate Critical Intervention (BS-5 = 0.76). They also exhibited high MBS Completeness (BS-1 = 0.89) but a lower Analysis Time Ratio (BS-6 = 0.66) than the other two groups. Their frequent invocation of AI-generated content, combined with only moderate critical revision, suggests a production-oriented collaboration mode with greater AI dependence.

Cluster 3: Surface Operators (n = 14, 19.7%). This group exhibited a fragmented, shallow engagement pattern. They registered zero Critical Intervention (BS-5 = 0.00) and a near-zero AI Interaction Rate (BS-4 = 0.00), indicating minimal active engagement with AI-generated outputs. Although their Analysis Time Ratio was numerically high (BS-6 = 0.97), this was accompanied by the lowest MBS Completeness (BS-1 = 0.89) and no Critical Intervention, suggesting prolonged but poorly transformed engagement rather than productive reflection.

3.4. External Validation: Cognitive Flexibility and Creative Autonomy

To assess the pedagogical diagnostic efficacy of the soft-sensor-derived clusters, the three groups were compared on external outcome indicators (Figure 6). Prior to ANOVA, the Shapiro–Wilk normality test confirmed that cognitive flexibility scores satisfied the normality assumption across all three clusters (all p > 0.05), and Levene’s test detected no significant heterogeneity of variance (F(2, 68) = 1.23, p = 0.299). ANOVA revealed a significant difference in cognitive flexibility among the three clusters (F(2, 68) = 5.66, p = 0.005), with a medium-to-large effect size (η² = 0.143). Post hoc Tukey HSD tests indicated that Deep Explorers (M = 4.98) scored significantly higher than Progressive Builders (M = 4.50) and Surface Operators (M = 4.14). This result provides evidence that the behavioral patterns identified by soft-sensors predict students’ ultimate cognitive performance.

Figure 6. (a) Cognitive flexibility and (b) AI-generated images across the three clusters. The box represents the interquartile range (IQR, 25th–75th percentile), the horizontal line inside the box indicates the median, the diamond (◇) denotes the mean, whiskers extend to 1.5 × IQR, and circles (○) represent outliers. ** p < 0.01; *** p < 0.001.

As illustrated in Table 3, a significant difference was also observed in AI-generated image volume (Kruskal–Wallis H(2) = 29.12, p < 0.001). Progressive Builders generated substantially more AI images (M = 16.94) than Deep Explorers (M = 5.49), while Surface Operators generated almost none (M = 0.43). This non-parametric test was employed due to severe non-normality (Shapiro–Wilk p < 0.001 for all groups). The result corroborates the soft-sensors’ ability to capture AI dependence levels.

Table 3. ANOVA results for external validation indicators.

Validation Indicator	Deep Explorers (n = 41)	Progressive Builders (n = 16)	Surface Operators (n = 14)	F/H	p	η2
Cognitive Flexibility (1–5)	4.98 (0.57)	4.50 (1.10)	4.14 (1.17)	5.66	0.005	0.143
AI-Generated Images	5.49 (6.64)	16.94 (29.06)	0.43 (1.09)	H(2) = 29.12	<0.001	0.136
CTD Self-Report (n = 39)	5.34 (1.06)	5.19 (1.37)	5.64 (1.44)	0.29	0.748	0.016

Notably, for the 39-student subsample that completed the Cultural Translation Depth (CTD) self-report questionnaire, ANOVA revealed no significant difference among the three clusters (F(2, 36) = 0.29, p = 0.748, η² = 0.016). This striking divergence between objective cognitive indicators and subjective self-assessment is examined in depth in Section 4.

4. Discussion

4.1. Behavioral Soft-Sensors as a Zero-Intrusion Diagnostic Paradigm

The three behavioral profiles identified in this study—Deep Explorers, Progressive Builders, and Surface Operators—demonstrate that computationally formalized behavioral soft-sensors can effectively decode latent cognitive engagement patterns from digital interaction traces alone. The moderate effect size observed for cognitive flexibility (η² = 0.143) provides compelling evidence that these algorithmically derived indicators possess genuine diagnostic power, capturing meaningful variance in learning outcomes that extends well beyond surface-level activity metrics.

This finding resonates with and extends the broader MMLA literature [19,20,21]. Whereas prior work has predominantly relied on physical hardware—eye trackers, EEG, and wearable physiological sensors—to capture learning engagement [22,23,24,37,38], the present study demonstrates that fine-grained digital traces, when subjected to principled computational formalization, can serve as a viable zero-intrusion alternative. This does not imply that behavioral soft-sensors should replace physiological sensors; rather, they offer a complementary, scalable modality that is particularly well-suited to naturalistic classroom ecologies where hardware deployment is impractical. It is important to situate this approach within the broader landscape of student–AI interaction profiling. Jin et al. [26] recently proposed RelianceScope, an analytical framework that operationalizes nine fine-grained reliance patterns based on help-seeking and response-use engagement in AI chatbot interactions. While RelianceScope offers greater granularity in classifying reliance behaviors, it was designed for text-based chatbot interactions in problem-solving contexts. Our behavioral soft-sensor framework, by contrast, is tailored to the multimodal, visually oriented context of design education, where cognitive engagement manifests through spatial navigation, temporal allocation, and image-editing behaviors rather than text-based dialog. Similarly, Fan and Ouyang [25] used learning analytics to unveil human–AI collaborative patterns in instructional design activities, employing lag sequential analysis to examine behavioral transitions. Our approach complements theirs by focusing on the aggregate behavioral profile rather than sequential transitions, and by introducing the soft-sensor formalization that provides explicit mathematical definitions for each indicator.

The behavioral profiles also reveal a critical nuance in the human–AI collaboration dynamic. Deep Explorers, who exhibited the highest Critical Intervention and Analysis Time Ratio, achieved the best cognitive outcomes despite generating the fewest AI images. Progressive Builders, by contrast, invoked AI most frequently yet demonstrated significantly lower cognitive flexibility. This pattern is consistent with the “cognitive offloading” hypothesis [12,39]: when AI tools are used as a substitute for, rather than a complement to, human cognitive effort, the resulting reduction in effortful processing may undermine deep learning [40,41].

4.2. Empirical Quantification of the Fluency Illusion in Design Education

Perhaps the most striking finding of this study is the systematic divergence between objective behavioral indicators and subjective self-assessment across the three clusters. While the CTD self-report questionnaire revealed no significant differences among groups (F(2, 36) = 0.29, p = 0.748, η² = 0.016), the objective behavioral soft-sensors and expert-assessed cognitive flexibility clearly differentiated them (F(2, 68) = 5.66, p = 0.005, η² = 0.143). This pattern constitutes empirical evidence for what Kumar et al. [10] have termed the “fluency illusion”—a metacognitive bias whereby the processing fluency of AI-generated outputs leads learners to overestimate their own understanding. Our findings extend this theoretical construct, previously characterized through narrative review [10], by providing the first domain-specific behavioral quantification in design education. Critically, Surface Operators reported the highest subjective CTD (M = 5.64) despite exhibiting zero Critical Intervention (BS-5 = 0.00) and the lowest objective cognitive flexibility (M = 4.14). The visual and esthetic nature of design outputs may make the fluency illusion particularly pronounced in this domain: when AI produces photorealistic images that appear professionally competent, the metacognitive signals that traditionally inform learners of their conceptual struggles—failed sketches, unresolved contradictions, iterative dead ends—are bypassed entirely [12,13]. Figure 7 visualizes this divergence across the three clusters, juxtaposing objective cognitive flexibility, AI-generated image volume, and subjective CTD self-report scores. This inverse relationship between perceived and actual cognitive engagement represents a profound manifestation of the Dunning–Kruger effect [42], amplified by AI. We hypothesize that the high-fidelity visual outputs generated by AI effectively ‘hijack’ the students’ metacognitive monitoring mechanisms. Surface Operators mistake the algorithmic execution fluency of the machine for their own internal cognitive fluency, leading to severely inflated self-assessments (M = 5.64) despite a complete lack of critical intervention. This aligns with Jose’s [14] finding that AI-driven cognitive offloading erodes critical thinking capacity.

Figure 7. Diagnosing the fluency illusion: objective vs. subjective assessment divergence. (a) Expert-assessed cognitive flexibility (1–5 scale); (b) AI-generated image volume; (c) CTD self-report scores (1–7 scale). Error bars represent standard error of the mean. ** p < 0.01; *** p < 0.001; n.s. = not significant.

The present findings extend prior work on trace-data versus self-report misalignment in learning analytics. Ye and Pennisi [15] demonstrated that digital-trace data from LMS predicted student performance more accurately than self-reported SRL data, and classified students into three levels of self-regulatory ability through cluster analysis—a finding that closely parallels our three-cluster solution. Choi et al. [16] documented significant misalignment between behavioral trace data and self-report surveys when modeling learner achievement goal orientation, finding that while learners articulated mastery-oriented goals in surveys, their behavioral traces revealed less engaged learning activities. Han and Ellis [17] reported similar weak alignment between self-reported and digital-trace measures in blended course designs, and Sun et al. [43] likewise demonstrated, through temporal learning analytics, that behavioral trace patterns rather than self-reports best captured students’ self-regulated learning processes. Our study extends these findings from general online learning to the specific context of AI-augmented design education, where the fluency illusion is amplified by the visual quality of AI-generated outputs. Furthermore, whereas prior studies examined the misalignment descriptively, our behavioral soft-sensor framework provides a mechanistic account: it is not merely that self-reports are inaccurate, but that the high quality of AI outputs actively deceives the learner’s self-monitoring system into believing that deep cognitive processing has occurred [10,14]. This interpretation aligns with Abbas et al. [8], who demonstrated that frequent GenAI use was associated with increased procrastination and reduced metacognitive awareness, and with Lodge and Loble [13], who argued that AI’s deepest educational risk is cognitive offloading when metacognitive scaffolding is absent.

It is important to acknowledge that this interpretation rests on a subsample of 39 respondents. As previously noted, this sample size is an artifact of the system’s 15 min canvas engagement trigger rather than a methodological flaw, yet the statistical power remains limited. A post hoc power analysis indicates that with n = 39 distributed across three groups, the study had approximately 0.12 power to detect a medium effect (f = 0.25) at α = 0.05—well below the conventional threshold of 0.80. Consequently, the non-significant CTD result (p = 0.748) should be interpreted cautiously: it may reflect either a genuine absence of subjective differences (consistent with the fluency illusion hypothesis) or insufficient statistical power. Future research should employ mandatory full-sample self-report administration and consider Bayesian analysis to distinguish between “evidence for no effect” and “absence of evidence” [18]. Nevertheless, the convergence of three independent lines of evidence—(1) significant objective cognitive flexibility differences, (2) significant behavioral AI dependence differences, and (3) non-significant subjective self-assessment differences—provides a compelling triangulated signal that warrants serious theoretical attention and replication with larger samples.

4.3. Interpreting the Silhouette Coefficient in Educational Data

The mean silhouette coefficient of 0.350 provides moderate support for the three-cluster solution. In the machine-learning literature, silhouette values between 0.25 and 0.50 are considered indicative of a reasonable clustering structure. However, we contend that this threshold, calibrated primarily on synthetic or physical-science datasets, is not directly applicable to human behavioral data, which are inherently characterized by high dimensionality, noise, and continuous distributions [36]. In educational data-mining research, silhouette coefficients in the 0.20–0.30 range are commonly reported and accepted when accompanied by converging evidence from multiple validation sources [29]. Araka et al. [29] and De Backer et al. [36] both reported similar silhouette ranges in their clustering analyses of self-regulated learning profiles, noting that human behavioral data inherently exhibit continuous distributions rather than discrete categories. The moderate silhouette value in our study thus reflects the continuous nature of the behavioral spectrum rather than a failure of the clustering algorithm.

In the present study, the clustering solution is supported by a robust convergence of evidence: (1) multi-algorithm agreement (ARI = 0.82, NMI = 0.79); (2) bootstrap stability (92.3% consistency across 1000 resamples); (3) highly significant external validation on cognitive flexibility (η² = 0.143); and (4) clear pedagogical interpretability of the three profiles. Taken together, these converging lines of evidence provide substantially stronger support for the validity of the three-cluster solution than the silhouette coefficient alone.

4.4. Pedagogical Implications

The findings carry direct implications for the design of AI-supported learning environments. First, the identification of three distinct behavioral profiles underscores the need for differentiated pedagogical interventions. Surface Operators, who exhibit fragmented engagement and minimal critical reflection, may benefit from structured scaffolding that constrains AI access until prerequisite analytical tasks are completed. Progressive Builders, who demonstrate high task compliance but low critical intervention, could be prompted with metacognitive reflection cues—for example, requiring students to articulate their design rationale before each AI-generation request. Notably, the MBS-AIGC platform’s design may itself serve as a partial countermeasure against the fluency illusion. By requiring students to complete cultural “Meaning” analysis before accessing AI-generation features, the platform enforces a cognitive prerequisite that may attenuate the tendency toward shallow engagement. The finding that Deep Explorers—who exhibited the highest MBS Completeness (BS-1 = 0.99)—also achieved the best cognitive outcomes suggests that structured cultural scaffolding can promote the kind of effortful processing that counteracts cognitive offloading [13,14]. This design principle—embedding cognitive prerequisites into AI-augmented workflows—may generalize to other domains where the fluency illusion poses a pedagogical risk.

Second, the failure of the CTD self-report to differentiate among clusters raises important questions about the continued reliance on self-assessment instruments in GenAI learning contexts. Our findings suggest that, in environments where AI can produce polished outputs with minimal human cognitive investment, traditional self-report measures may be insufficient as standalone assessment tools. Behavioral soft-sensors, by contrast, offer a continuous, objective data stream that can complement—though not necessarily replace—self-report instruments. Practically, this means soft-sensors could potentially be configured to act as a real-time background monitoring engine. When the system detects a high-risk behavioral pattern (e.g., the Surface Operator profile), it could dynamically trigger targeted micro-surveys or scaffolding prompts—a design possibility illustrated by the 15 min engagement-threshold trigger currently implemented in this study. Since the soft-sensor algorithms compute lightweight mathematical features from structured logs, this processing occurs asynchronously with millisecond latency, ensuring no disruption to the user’s creative workflow or system performance while enabling educators to identify at-risk students who might otherwise be invisible to conventional assessment.

4.5. Limitations and Future Directions

Several limitations should be acknowledged. First, the sample was drawn from a single institution and a specific design course, which constrains the generalizability of the findings. However, the computational formalization of the six soft-sensors is platform-agnostic in principle; future studies should validate these indicators across diverse disciplinary contexts and institutional settings.

Additionally, the MBS (Meaning–Behavior–Spirit) framework utilized in this study is inherently grounded in the epistemology of Chinese intangible cultural heritage. While this cultural specificity enabled highly contextualized soft-sensor formalization, it raises important questions regarding the framework’s cross-cultural applicability. We propose the MBS Scalability Hypothesis: the triadic structure of cultural cognition—material artifacts, behavioral rituals, and spiritual values—may represent a generalizable schema for cultural translation in design education. To date, however, the MBS-AIGC platform has been empirically validated only within the context of Chinese intangible cultural heritage—specifically, the folk-IP design of Minnan (southern Fujian) traditional architectural ornamentation, such as the “Cuojiaotou” roof-ridge decoration. While this triadic schema is, in principle, agnostic to the specific cultural content that populates each dimension, its cross-cultural transferability remains a theoretical proposition rather than an empirically established property. Future cross-cultural validation studies are therefore needed to test whether the behavioral soft-sensors retain their diagnostic power when applied to other, non-Chinese cultural design contexts.

Second, the cross-sectional design precludes causal inference regarding the relationship between behavioral patterns and cognitive outcomes. While the moderate effect size (η² = 0.143) establishes a strong association, longitudinal or experimental designs are needed to disentangle the direction of causality.

Third, the CTD self-report subsample (n = 39) was relatively small, limiting the statistical power of the self-report analysis. Future research should employ mandatory full-sample self-report administration to enable more definitive comparisons between objective and subjective assessment modalities.

Fourth, the current soft-sensor framework captures behavioral frequency and temporal patterns but does not directly measure the semantic quality of student–AI interactions. Integrating natural language processing (NLP) techniques to analyze the content of student prompts and AI responses represents a promising avenue for enriching the soft-sensor array with semantic-level indicators. Fifth, while the three-cluster solution provides pedagogically interpretable profiles, more fine-grained classification schemes—such as the nine-pattern framework proposed by Jin et al. [26]—may capture additional nuances in student–AI interaction. Future research should explore whether increasing the number of clusters or adopting alternative analytical frameworks (e.g., sequence mining, process mining) yields additional pedagogical insights in design education contexts.

5. Conclusions

This research demonstrates that computationally formalized behavioral soft-sensors can effectively diagnose and quantify the fluency illusion in AI-augmented design education. For RQ1, six behavioral soft-sensors reliably partition 71 design students into three distinct profiles: Deep Explorers exhibiting high critical intervention, Progressive Builders demonstrating frequent AI invocation with moderate revision, and Surface Operators displaying minimal engagement. For RQ2, expert-assessed cognitive flexibility significantly differentiates these profiles (p = 0.005, η² = 0.143), while a conventional self-report questionnaire fails entirely to detect any differences (p = 0.748). These divergent findings provide the first preliminary behavioral evidence for the fluency illusion in design education; this study bridges industrial process control and educational data mining by validating the soft-sensor paradigm as a mathematically rigorous framework for inferring latent cognitive states from observable behavioral traces. Methodologically, the six behavioral soft-sensors constitute a scalable, zero-intrusion diagnostic toolkit that overcomes the ecological validity limitations of hardware-dependent multimodal learning analytics. In practice, the MBS-AIGC platform furnishes educators with actionable, real-time indicators for detecting metacognitive miscalibration, enabling adaptive AI systems that intervene when the fluency illusion is most acute. These findings are bounded by a single-institution, four-week implementation with 71 participants. Future research should validate the framework across diverse institutions and disciplines, investigate longitudinal dynamics of the fluency illusion under sustained AI exposure, and integrate soft-sensor outputs with adaptive scaffolding for personalized metacognitive intervention. Beneath the surface of fluent AI-generated artifacts lie fundamentally different cognitive architectures. Behavioral soft-sensors render these invisible dynamics visible, ensuring that human creativity—not algorithmic fluency—remains the true measure of educational achievement.