Integrating Semantic NLP and PLS-SEM for AI-Enabled Strategic Decision Support: An Explainable Framework for Assessing Organisational AI Illiteracy

Abstract

The rapid growth of organisational textual data has increased the value of Natural Language Processing (NLP) and semantic analytics for strategic decision support. However, many employees still lack the knowledge and skills needed to evaluate AI-generated information critically. This study develops an explainable Management Information Systems (MIS) framework that integrates NLP-based semantic analytics with PLS-SEM to examine the relationship between AI illiteracy and strategic decision quality. A convergent mixed-methods design with sequential analytical integration was used with a valid sample of 200 knowledge workers from public organisations in Saudi Arabia across six industries. The sample included employees from Saudi Arabia, Egypt, Jordan, Sudan, Syria, India, and the Philippines. Quantitative data were analysed using PLS-SEM, while textual data were analysed using Sentence-BERT, BERTopic, semantic network analysis, and Aspect-Based Sentiment Analysis. The results showed that higher AI illiteracy was negatively associated with strategic decision quality and positively associated with automation bias, uncritical trust in AI, and cognitive offloading. Digital proficiency and AI governance awareness weakened the negative association between AI illiteracy and decision quality, while functional-background differences were examined through multigroup analysis. The semantic analysis identified six themes: AI competency, decision trust, AI governance, decision support, organisational learning, and risk awareness. Sentiment analysis showed positive views of productivity and decision support, together with concerns about algorithmic bias, explainability, transparency, and AI governance. The study contributes an integrated human–AI decision vulnerability framework in which semantic evidence complements structural modelling and provides a clearer understanding of AI-related competency, reliance, governance, and decision-support issues.

1. Introduction

Digital transformation has led organisations to generate large volumes of unstructured textual data, including corporate reports, emails, customer feedback, social media content, patents, and internal knowledge repositories. This data can provide valuable information about organisational knowledge, emerging risks, competitive intelligence, and strategic opportunities. Natural Language Processing (NLP), semantic data mining, and large language models (LLMs) have therefore become important tools for transforming unstructured text into useful organisational knowledge. These technologies can support organisational learning, business intelligence, and strategic decision-making [1,2].

Recent advances in transformer-based language models, semantic embeddings, topic modelling, and semantic network analysis have further improved the analysis of large textual datasets. They enable organisations to identify semantic patterns, examine relationships among concepts, detect emerging risks, and extract information that can support decision-making [3,4]. These developments have expanded NLP from a text-processing tool into an important analytical capability within Management Information Systems (MIS). However, the value of AI-supported decision systems depends not only on technological capability but also on the people who interpret and use AI-generated information.

Many employees still lack the competencies required to evaluate AI recommendations critically, recognise algorithmic limitations, verify generated information, and use AI outputs appropriately. This problem can be described as AI illiteracy. It extends beyond limited technical knowledge and includes deficiencies in critical thinking, analytical reasoning, ethical awareness, and judgement when interacting with AI systems [5].

Higher AI illiteracy may be associated with greater reliance on AI recommendations without sufficient verification. Employees may also overlook biased, inaccurate, or uncertain outputs. These behaviours are related to automation bias, uncritical trust in AI, and cognitive offloading, which may in turn be associated with lower strategic decision quality [6,7]. As AI becomes more widely integrated into organisational decision-support systems, AI illiteracy may therefore represent both an individual competency gap and a broader organisational vulnerability [8].

From an MIS perspective, effective AI-supported decision-making requires alignment between analytical technologies, human expertise, appropriate task allocation, and organisational governance. Previous research shows that the organisational value of AI depends not only on technical capability but also on human–AI collaboration, professional judgement, organisational AI capabilities, and governance structures [3,4]. However, limited attention has been given to the combined relationships among AI-related competency deficiencies, cognitive reliance mechanisms, governance conditions, and semantic evidence extracted from organisational narratives.

A further limitation is that many studies rely mainly on structured survey data. Such data are useful for testing relationships among variables but may not fully capture employees’ experiences, perceptions, reasoning, and concerns. Organisational text can provide complementary evidence about how employees understand AI, assess its risks, and describe its role in decision-making [9,10,11].

Although NLP and semantic analytics have been widely applied to knowledge discovery, business intelligence, innovation management, and decision support, relatively few studies have integrated semantic evidence from organisational text with structural modelling of strategic decision quality [2,10]. Bridging structured and unstructured evidence therefore remains an important challenge for AI-enabled MIS research.

The main contribution of this study is not simply the combination of NLP and Partial Least Squares Structural Equation Modelling (PLS-SEM). Instead, the study proposes a human–AI decision vulnerability framework that explains how AI illiteracy is associated with strategic decision quality through technological, cognitive, and organisational mechanisms. AI illiteracy is conceptualised not only as an individual knowledge deficiency but also as an organisational vulnerability that may shape how employees use AI-enabled decision-support systems, evaluate AI recommendations, and respond to organisational governance conditions.

The framework integrates Adaptive Structuration Theory (AST) [12,13], Dual-Process Theory (DPT), and the Technology–Organisation–Environment (TOE) framework. Each theory explains a different part of the proposed model. AST explains technology appropriation by focusing on how employees understand and use AI capabilities in practice [14]. DPT explains the cognitive mechanisms through which AI illiteracy may be associated with automation bias, uncritical trust in AI, and cognitive offloading. TOE explains the organisational boundary conditions by considering digital proficiency, functional background, and AI governance awareness [15]. Together, these perspectives link technology use, human cognition, and organisational readiness within a single framework.

NLP-based semantic analytics provides an additional explanatory layer. Sentence-BERT, BERTopic, semantic network analysis, and Aspect-Based Sentiment Analysis are used to examine whether organisational narratives contain themes related to AI competency, trust, governance, decision support, organisational learning, and AI-related risks. These textual findings are then integrated with the PLS-SEM results to identify areas of convergence, complementarity, and divergence [16]. Accordingly, this study contributes to MIS research by developing and empirically examining a multilevel framework in which AI illiteracy is associated with strategic decision quality through cognitive reliance mechanisms. At the same time, organisational capabilities and governance conditions may act as safeguards against these vulnerabilities.

2. Literature Review and Hypotheses

2.1. AI Illiteracy, Strategic Decision Quality, and the Mediating Role of Cognitive Biases

AI-supported decision-making depends not only on the quality of the technology but also on employees’ ability to understand and evaluate AI-generated information. AI literacy includes knowledge of AI systems, critical evaluation skills, ethical awareness, and the ability to use AI appropriately [17,18,19]. In contrast, AI illiteracy reflects gaps in these competencies. Employees with higher AI illiteracy may have greater difficulty identifying algorithmic limitations, verifying AI recommendations, and deciding when human judgement should take priority [20,21,22]. These limitations are important in organisational decision-making.

AI systems can process large amounts of information and provide useful recommendations, but their outputs may contain errors, bias, or uncertainty. Employees who cannot critically assess these outputs may rely on them without adequate verification [23]. AI illiteracy may therefore be associated with lower strategic decision quality, particularly when decisions require professional judgement, comparison of alternatives, and careful evaluation of evidence [24,25,26,27]. Adaptive Structuration Theory (AST) helps explain this relationship. AST proposes that the outcomes of technology use depend on how users understand and apply technological capabilities within an organisational setting [28,29]. From this perspective, access to advanced AI does not ensure better decisions by itself. Employees must be able to interpret AI outputs and use them appropriately [30].

AI illiteracy can therefore be viewed as an appropriation vulnerability that may limit the value of AI-enabled decision-support systems [31]. H1: AI illiteracy is negatively associated with strategic decision quality. Dual-Process Theory (DPT) provides a basis for explaining the cognitive mechanisms that may underlie this relationship [32]. DPT distinguishes between relatively fast and intuitive processing and slower, more deliberate analytical reasoning [28,29]. When employees lack the knowledge needed to evaluate AI-generated information, they may rely more heavily on automated recommendations and devote less effort to independent analysis [33].

One possible mechanism is automation bias, which refers to excessive reliance on automated recommendations, including situations in which other evidence suggests that the recommendation may be incorrect [24,25,26]. Employees with limited AI literacy may find it more difficult to recognise errors or challenge automated outputs. Greater automation bias may therefore be associated with lower decision quality because relevant evidence or alternative interpretations can be overlooked [34,35]. H2: Automation bias mediates the association between AI illiteracy and strategic decision quality.

A second mechanism is uncritical trust in AI. Trust is important for the use of AI systems, but excessive or poorly calibrated trust can create risks. Employees may accept AI recommendations without sufficient verification when they view the system as more reliable or capable than their own judgement [36,37]. AI illiteracy may increase this vulnerability because employees have less knowledge with which to assess the accuracy, limitations, and uncertainty of AI-generated outputs [31,32,33,34]. H3: Uncritical trust in AI mediates the association between AI illiteracy and strategic decision quality.

A third mechanism is cognitive offloading, which occurs when individuals transfer part of their cognitive work to external tools or systems [30]. AI can reduce cognitive effort and support complex analytical tasks [38]. However, excessive offloading may reduce independent evaluation when employees rely on AI instead of applying their own reasoning. Employees with limited AI literacy may be particularly likely to use AI as a substitute for analysis rather than as a tool that supports human judgement [39,40,41]. This form of reliance may be associated with lower strategic decision quality. H4: Cognitive offloading mediates the association between AI illiteracy and strategic decision quality.

Together, these mechanisms suggest that AI illiteracy may be associated with strategic decision quality both directly and indirectly [42]. Automation bias, uncritical trust, and cognitive offloading represent different forms of reliance on AI that may help explain this relationship [43]. The proposed mediation model therefore links AI-related competency gaps with the cognitive processes involved in AI-supported organisational decisions.

2.2. Organisational Capabilities, AI Governance, and Moderating Conditions

The relationship between AI illiteracy and decision quality may vary according to employees’ capabilities and organisational conditions. The Technology–Organisation–Environment (TOE) framework provides a useful basis for understanding these differences [44]. TOE proposes that technology-related outcomes depend on technological, organisational, and environmental conditions rather than on technology alone [45]. In the present study, digital proficiency, functional background, and AI governance awareness represent relevant conditions surrounding AI-supported decision-making.

Digital proficiency refers to employees’ ability to use digital technologies, AI applications, and data-driven systems [46]. Employees with stronger digital proficiency may be better able to examine AI outputs, compare information from different sources, and recognise inaccurate or questionable recommendations [47,48]. Digital proficiency may therefore reduce the vulnerability associated with AI illiteracy by supporting more careful interaction with AI-enabled decision systems [38,39,40]. H5: Digital proficiency weakens the negative association between AI illiteracy and strategic decision quality.

Functional background represents employees’ primary professional area rather than a psychological latent construct [49]. Employees working in executive and strategic functions, information systems and data analytics, or operational and support functions may differ in their domain expertise, exposure to AI, and ways of evaluating AI-generated information [50,51,52]. The association between AI illiteracy and decision quality may therefore vary across functional groups. In this study, functional background is treated as a categorical boundary condition and examined through multigroup analysis. H6: The association between AI illiteracy and strategic decision quality differs across functional-background groups.

AI governance awareness is another important organisational condition. Responsible AI governance establishes expectations concerning transparency, accountability, explainability, risk management, and human oversight [53,54]. Employees who understand these requirements may be more likely to verify AI recommendations, question uncertain outputs, and recognise situations that require human review [55]. Governance awareness may therefore reduce the negative association between AI illiteracy and strategic decision quality. H7: AI governance awareness weakens the negative association between AI illiteracy and strategic decision quality.

Recent Human-Centered Artificial Intelligence (HCAI) research supports this emphasis on human oversight. HCAI views AI as a tool that should support rather than replace human judgement. Fanfani et al. [9], in a systematic review of 90 studies, identified human-in-the-loop interaction, explainability, reliability, transparency, and validation as important features of human-centred decision-support systems. They also noted that explainability alone does not ensure appropriate reliance on AI. Human verification, oversight, and clear decision responsibilities remain important. These principles support the proposed roles of digital proficiency and AI governance awareness as potential safeguards in AI-supported decision-making [23,31,40].

NLP-based semantic analytics provides complementary evidence for examining these issues. Sentence-BERT embeddings can represent the semantic meaning of organisational text, while BERTopic can identify recurring themes. Semantic network analysis can reveal relationships among key concepts, and Aspect-Based Sentiment Analysis (ABSA) can identify positive, neutral, and negative evaluations of specific AI-related issues [5,6,13,14,15,16,17,18,19,20]. In this study, these methods are used to examine textual patterns related to AI competency, trust, verification, governance, decision support, organisational learning, and risk. NLP analysis is not used to establish causal relationships or independently test the structural hypotheses. Instead, it provides contextual evidence that can be compared with the PLS-SEM findings. This integration allows the study to identify areas of convergence, complementarity, and divergence between structured survey results and employees’ organisational narratives.

3. Research Framework

3.1. Conceptual Framework

Organisations increasingly use artificial intelligence (AI), Natural Language Processing (NLP), and semantic analytics to support strategic decision-making. However, the value of AI-enabled decision-support systems depends not only on technological capability but also on employees’ ability to understand, evaluate, and use AI-generated information. Limited ability in these areas represents AI illiteracy and may be associated with lower decision quality and greater reliance on AI.

This study proposes a human–AI decision vulnerability framework that integrates Management Information Systems (MIS), cognitive decision-making theory, and NLP-based semantic analytics. The framework proposes that AI illiteracy is negatively associated with strategic decision quality. It also examines three cognitive mechanisms that may explain this relationship: automation bias, uncritical trust in AI, and cognitive offloading. Digital proficiency, functional background, and AI governance awareness are examined as conditions that may shape the relationship between AI illiteracy and decision quality.

The framework combines structured survey data with unstructured textual data. Open-ended responses and organisational documents are analysed using NLP techniques to identify themes, semantic relationships, perceptions, and governance concerns. These findings provide contextual evidence that complements the structural model. Accordingly, the framework treats AI illiteracy as both an employee competency gap and a potential vulnerability in organisational decision-making.

3.2. Integrated Theoretical Foundations

This study integrates Adaptive Structuration Theory (AST), Dual-Process Theory (DPT), and the Technology–Organisation–Environment (TOE) framework. Each perspective explains a different part of AI-supported strategic decision-making: technology appropriation, cognitive processing, and organisational boundary conditions. At the technology-appropriation level, AST explains why access to advanced AI systems does not automatically improve decision-making. The outcomes of technology use depend on how employees understand and apply their capabilities.

In this study, AI illiteracy represents an appropriation vulnerability. Employees with limited AI-related competencies may have difficulty evaluating AI recommendations, recognising system limitations, and integrating AI-generated information with professional judgement. AST therefore provides the theoretical basis for H1. At the cognitive-processing level, DPT explains the mechanisms through which AI illiteracy may be associated with strategic decision quality. DPT distinguishes between relatively fast, intuitive processing and slower, more analytical reasoning. Employees who lack the knowledge needed to evaluate AI recommendations may rely more heavily on automated outputs rather than independent analysis.

The framework represents this vulnerability through automation bias, uncritical trust in AI, and cognitive offloading. DPT therefore provides the theoretical basis for the mediating relationships proposed in H2–H4. At the organisational level, TOE explains why the relationship between AI illiteracy and decision quality may vary according to employee capabilities and organisational conditions. Digital proficiency may improve employees’ ability to evaluate AI-generated information. Functional background provides domain-specific knowledge that may shape how AI recommendations are interpreted. AI governance awareness provides knowledge of transparency, accountability, verification, and responsible AI use. TOE therefore provides the theoretical basis for the boundary conditions examined in H5–H7.

Together, the three perspectives provide a multilevel explanation of AI-supported strategic decision-making. AST explains how employees appropriate AI technologies, DPT explains cognitive reliance mechanisms, and TOE explains the conditions that may strengthen or weaken the relationship between AI illiteracy and decision quality. This integration extends AI-literacy research by examining AI illiteracy as an organisational decision vulnerability rather than only an individual competency gap.

3.3. Research Model

Based on these theoretical foundations, the study develops a mediation–moderation model of human–AI decision vulnerability. AI illiteracy is the main independent variable, while strategic decision quality is the outcome variable. AI illiteracy refers to limitations in employees’ ability to understand, evaluate, and appropriately use AI-generated information. Strategic decision quality refers to the perceived quality of strategic decisions made in AI-supported organisational settings.

The model includes three mediators. Automation bias refers to excessive reliance on automated recommendations despite the availability of relevant alternative evidence. Uncritical trust in AI refers to accepting AI-generated outputs without sufficient verification. Cognitive offloading refers to transferring analytical effort to AI systems rather than applying sufficient independent reasoning. These mechanisms are examined as possible pathways linking AI illiteracy with strategic decision quality.

The model also examines three boundary conditions. Digital proficiency is modelled as a continuous moderator, representing employees’ ability to use digital technologies and evaluate digital information. Functional background is treated as a categorical grouping variable and examined through multigroup analysis because employees from different professional functions may differ in domain expertise and exposure to AI. AI governance awareness is modelled as a continuous moderator, representing employees’ awareness of organisational policies and principles concerning transparency, accountability, verification, ethics, and responsible AI use.

NLP-based semantic analytics provides a complementary analytical layer. It examines how themes related to AI competency, trust, governance, decision support, organisational learning, and risk appear in employees’ textual responses and organisational documents. The semantic results are compared with the PLS-SEM findings to identify areas of convergence, complementarity, and divergence. They are not used as independent evidence of causal relationships.

3.4. NLP Semantic Analytics Framework

This study employs an NLP-based semantic analytics framework to analyse unstructured organisational text and provide contextual evidence to support AI-driven decision-making. The corpus comprises participants’ open-ended responses and supplementary anonymised organisational documents. First, textual data are preprocessed and transformed into contextual semantic embeddings using Sentence-BERT. The exact model, software versions, preprocessing procedures, and implementation parameters are reported in Section 4.5 to ensure reproducibility.

Second, BERTopic identifies recurring semantic themes, with topic quality evaluated using coherence, diversity, stability, representative document inspection, and independent human validation. Third, semantic network analysis examines relationships among salient concepts using weighted co-occurrence edges. Degree, betweenness, and eigenvector centrality identify influential concepts, while the Louvain algorithm detects semantic communities and modularity evaluates community structure.

Fourth, Aspect-Based Sentiment Analysis (ABSA) examines positive, neutral, and negative evaluations of specific AI-related aspects, with automated classifications validated against independent human coding. Finally, NLP findings are integrated with PLS-SEM results through a structured mixed-methods procedure. Convergence indicates consistent quantitative and textual evidence, complementarity provides additional contextual explanation, and divergence identifies more nuanced or differing patterns. This integration contextualises relationships among AI illiteracy, cognitive reliance, organisational conditions, and strategic decision quality (see Figure 1).

Figure 1. NLP Semantic Analytics Framework.

4. Research Methodology

4.1. Research Design and Mixed-Methods Integration

This study used a convergent mixed-methods design with sequential analytical integration. Structured survey data and open-ended textual data were collected during the same period. Thus, the study did not use a conventional sequential explanatory design for data collection. Instead, quantitative and textual data were collected concurrently and analysed in sequential stages. In the first phase, PLS-SEM was used to examine the relationships among AI Illiteracy, Strategic Decision Quality, Automation Bias, Uncritical Trust in AI, Cognitive Offloading, Digital Proficiency, Functional Background, and AI Governance Awareness.

The analysis examined the direct association between AI Illiteracy and Strategic Decision Quality, the mediating roles of Automation Bias, Uncritical Trust in AI, and Cognitive Offloading, and the moderating roles of Digital Proficiency and AI Governance Awareness. Functional Background was examined through group comparisons. These results identified the main relationships requiring further contextual interpretation. In the second phase, the PLS-SEM results guided the semantic analysis of participants’ open-ended responses and organisational documents. The negative association between AI Illiteracy and Strategic Decision Quality guided the analysis of narratives related to AI competency and decision risk.

The mediation results guided the analysis of verification, reliance, judgement, trust, and human oversight. The moderation results guided the analysis of digital skills, governance, transparency, accountability, and responsible AI use. Sentence-BERT and BERTopic were used to identify semantic patterns and themes in the textual corpus. Semantic network analysis examined relationships among key concepts, while Aspect-Based Sentiment Analysis (ABSA) assessed employees’ perceptions of specific AI-related issues. The NLP analysis was not used to validate the PLS-SEM results statistically. Instead, it provided contextual evidence to identify convergence, complementarity, and divergence between the quantitative and textual findings.

The two forms of evidence were integrated at three stages. First, connecting used the PLS-SEM results to guide the focus of the NLP analysis. Second, building used significant structural relationships to support the interpretation of semantic themes and aspects. Third, merging compared the PLS-SEM and NLP findings in a joint display. This display linked structural relationships with semantic themes, representative keywords, sentiment results, and integrated interpretations. This process provided a systematic comparison of the structured and unstructured evidence.

4.2. Participants and Data Collection

Data were collected between January and March 2026 from knowledge workers employed in 10 public-sector organisations in Saudi Arabia. Purposive sampling was used because the study required participants with professional experience using AI applications, business intelligence systems, analytics platforms, or digital decision-support tools. Organisational contact persons distributed the study invitation to employees who met the eligibility criteria. Participation was voluntary, and individual responses were not shared with the participating organisations.

Participants were eligible if they (1) were employed in one of the participating organisations; (2) were aged 18 years or older; (3) performed analytical, managerial, planning, information-processing, or decision-related duties; (4) had professional experience using AI applications, business intelligence systems, analytics platforms, or digital decision-support tools; and (5) provided informed consent.

Responses were excluded if participants did not meet these criteria, failed to complete the main measurement sections, submitted substantially incomplete questionnaires, or provided response patterns unsuitable for reliable statistical analysis. A total of 202 eligible employees were invited to participate. Of these, 200 provided complete and usable questionnaires, resulting in a final sample of N = 200 and a response rate of 99.0%. Two invited employees did not provide usable responses and were excluded before model estimation. No cases from the final sample were removed during the PLS-SEM analysis (see Table 1).

Table 1. Demographic and Professional Characteristics of Participants.

Characteristic	Category	n	%
Gender	Male	126	63.0
	Female	74	37.0
Age	18–29 years	28	14.0
	30–39 years	76	38.0
	40–49 years	63	31.5
	≥50 years	33	16.5
Education	Bachelor’s degree	91	45.5
	Master’s degree	73	36.5
	Doctorate/other postgraduate qualification	36	18.0
Professional Experience	<5 years	31	15.5
	5–10 years	64	32.0
	11–15 years	58	29.0
	>15 years	47	23.5
Nationality	Saudi Arabia	107	53.5
	Egypt	31	15.5
	Jordan	17	8.5
	Sudan	14	7.0
	Syria	12	6.0
	India	11	5.5
	Philippines	8	4.0
Industry	Finance	26	13.0
	Healthcare	40	20.0
	Manufacturing	35	17.5
	Education	30	15.0
	Information Technology	34	17.0
	Professional Services	35	17.5
Functional Group	Executive/Strategic	67	33.5
	Information Systems/Data Analytics	69	34.5
	Operational/Support	64	32.0
Total Sample		200	100.0

All participants worked in Saudi Arabia during the data-collection period. The sample represented a multinational workforce from Saudi Arabia, Egypt, Jordan, Sudan, Syria, India, and the Philippines. Participants represented six industries, with healthcare accounting for the largest proportion (20.0%), followed by manufacturing and professional services (17.5% each), information technology (17.0%), education (15.0%), and finance (13.0%).

Most participants were male (63.0%), and the largest age group was 30–39 years (38.0%). Bachelor’s degree holders represented 45.5% of the sample, followed by master’s degree holders (36.5%). Most participants had more than five years of professional experience. Saudi nationals represented 53.5% of the sample, while the remaining participants represented six other nationalities. The three functional groups were relatively balanced: Executive/Strategic (33.5%), Information Systems/Data Analytics (34.5%), and Operational/Support (32.0%). These characteristics provide a detailed description of the analytical sample and facilitate assessment of its generalisability.

The questionnaire included structured and open-ended components. The structured section collected demographic and organisational information, including nationality, industry, job role, education, professional experience, digital proficiency, and awareness of organisational AI governance policies. The study constructs were measured using multiple-item seven-point Likert scales. The open-ended questions examined participants’ experiences with AI-supported organisational decision-making. They addressed the perceived benefits and limitations of AI, trust in AI-generated recommendations, verification practices, governance, explainability, and AI-related risks. These responses formed the primary textual corpus for the NLP analysis.

Where organisational permission was available, anonymised decision summaries and AI-supported reports were included as supplementary textual material. Documents were included only when they were directly related to AI-supported organisational decision-making and could be fully anonymised. Thus, the NLP analysis relied primarily on participants’ open-ended responses, while organisational documents provided supplementary contextual evidence. Participation was voluntary, and informed consent was obtained before data collection. All participant responses and organisational textual materials were anonymised before analysis.

4.3. Sample-Size Adequacy and Statistical Power

A priori statistical power analysis was conducted to assess whether the final sample size was sufficient to estimate the proposed PLS-SEM model. The analysis was based on the most complex endogenous construct, Strategic Decision Quality (SDQ), which was predicted by AI Illiteracy, Automation Bias, Uncritical Trust in AI, Cognitive Offloading, Digital Proficiency, AI Governance Awareness, and the two interaction terms involving Digital Proficiency and AI Governance Awareness. Thus, the maximum number of predictors considered in the power analysis was eight. Functional Background was not included in this calculation because it was treated as a categorical grouping variable and examined separately through multigroup analysis. The structural specification is consistent with the model described in the manuscript [43].

Using a multiple-regression power framework with a medium effect size (f² = 0.15), a significance level of α = 0.05, statistical power of 0.80, and eight predictors, the minimum required sample size was 109 participants. The final valid sample of N = 200 therefore exceeded the minimum requirement by 91 cases. The sample size achieved was approximately 1.83 times the minimum required size, indicating adequate statistical power to detect medium-sized relationships in the most complex part of the structural model. The final sample size of 200 is also consistent with the manuscript’s reported recruitment and data-screening procedures [44].

Sample-size adequacy was further supported by the structural model results. The model explained 68.4% of the variance in Strategic Decision Quality (R² = 0.684), and all endogenous constructs showed positive Stone–Geisser Q² values, indicating predictive relevance. These results do not replace the formal power analysis, but they provide additional evidence that the final sample was adequate for estimating the proposed direct, mediation, and moderation relationships.

4.4. Measurement Instrument and Construct Specification

The questionnaire was developed by adapting established measures from the literature on AI literacy, trust in automation, automation bias, cognitive offloading, digital competence, responsible AI governance, and strategic decision-making. Table 2 presents the measurement items, their adaptation basis, and construct specifications. Items were modified where necessary to reflect AI-supported organisational decision-making. These changes involved wording and contextual adaptation without changing the conceptual meaning of the original constructs.

Table 2. Measurement Items, Sources, and Construct Specifications.

Construct	Code	Measurement Item	Source/Adaptation Basis	Specification
AI Illiteracy	AIL1	I have difficulty understanding how AI systems generate recommendations for organisational decisions.	[5,6,13,14,15,16,17,18,19,20]	Reflective
	AIL2	I find it difficult to evaluate whether AI-generated information is accurate and reliable.		Reflective
	AIL3	I have difficulty recognising the limitations and potential errors of AI-generated recommendations.		Reflective
	AIL4	I lack sufficient knowledge to critically verify AI-generated information before using it in decision-making.		Reflective
	AIL5	I find it difficult to determine when human judgement should override an AI-generated recommendation.		Reflective
Automation Bias	AB1	I tend to accept AI-generated recommendations without independently checking all relevant information.	[28,31,44,48]	Reflective
	AB2	I am inclined to follow an AI recommendation even when I have doubts about it.		Reflective
	AB3	I tend to rely on AI recommendations even when other information suggests an alternative decision.		Reflective
	AB4	I am less likely to search for additional evidence after an AI system provides a recommendation.		Reflective
Uncritical Trust in AI	UTA1	I generally assume that AI-generated recommendations are accurate.	[15,26,45,46]	Reflective
	UTA2	I tend to trust AI-generated information without thoroughly verifying the underlying evidence.		Reflective
	UTA3	I am willing to rely on AI recommendations even when I do not fully understand how they were generated.		Reflective
	UTA4	I rarely question an AI recommendation when the system appears technically sophisticated.		Reflective
Cognitive Offloading	CO1	I rely on AI to perform analytical tasks that I would otherwise perform myself.	[5,6,13,14,15,16,17,18,19,20]	Reflective
	CO2	When AI tools are available, I devote less effort to analysing information independently.		Reflective
	CO3	I use AI to reduce the mental effort required when making complex organisational decisions.		Reflective
	CO4	I increasingly delegate parts of my reasoning and problem-solving to AI systems.		Reflective
Digital Proficiency	DP1	I can effectively use the digital technologies required for my professional responsibilities.	[15,26,45,46]	Reflective
	DP2	I can confidently use AI-enabled and data-driven applications in my work.		Reflective
	DP3	I can evaluate information produced by digital analytical and decision-support systems.		Reflective
	DP4	I can quickly learn to use new digital and AI-based tools relevant to my work.		Reflective
AI Governance Awareness	AGA1	I am aware of my organisation’s policies governing the responsible use of AI.	[28,31,44,48]	Reflective
	AGA2	I understand the accountability requirements associated with using AI in organisational decisions.		Reflective
	AGA3	I am familiar with organisational requirements concerning the transparency and explainability of AI recommendations.		Reflective
	AGA4	I understand the ethical and risk-management requirements guiding organisational AI use.		Reflective
Strategic Decision Quality	SDQ1	AI-supported strategic decisions are based on relevant and accurate information.	[25,33,48,49,50,51,52,53,54,55]	Reflective
	SDQ2	AI-supported strategic decisions adequately consider important alternatives and consequences.		Reflective
	SDQ3	AI-supported strategic decisions are consistent with organisational objectives and priorities.		Reflective
	SDQ4	AI-supported strategic decisions are well reasoned and supported by sufficient evidence.		Reflective
	SDQ5	Overall, AI contributes to effective and high-quality strategic decision-making in my organisation.		Reflective

AI Illiteracy (AIL) was measured using five items assessing employees’ difficulty in understanding, evaluating, verifying, and appropriately using AI-generated information. Automation Bias (AB) was measured using four items assessing reliance on AI recommendations without adequate independent verification. Uncritical Trust in AI (UTA) included four items assessing the tendency to accept AI recommendations without sufficiently questioning their accuracy, evidence, or limitations.

Cognitive Offloading (CO) was measured using four items assessing the transfer of analytical and problem-solving effort from employees to AI systems. Digital Proficiency (DP) included four items assessing employees’ ability to use digital technologies, AI applications, and data-driven decision-support systems. AI Governance Awareness (AGA) was measured using four items covering organisational AI policies, accountability, transparency, explainability, ethics, and risk management. Strategic Decision Quality (SDQ) included five items assessing the accuracy, comprehensiveness, consistency, reasoning, and overall quality of AI-supported strategic decisions.

AIL, AB, UTA, CO, DP, AGA, and SDQ were specified as reflective constructs because their indicators were treated as manifestations of the corresponding latent variables. The reflective measurement model was therefore assessed using indicator loadings, Cronbach’s α, composite reliability, average variance extracted (AVE), the Fornell–Larcker criterion, and the heterotrait–monotrait ratio (HTMT). Cross-loadings were also examined as an additional assessment of discriminant validity. Functional Background (FB) was treated as an observed categorical variable rather than a latent construct. Participants were classified by their primary organisational function: executive management, strategic planning, information systems, marketing, finance, operations, human resources, or data analytics. These categories represent distinct occupational functions rather than manifestations of a common latent construct. FB was therefore excluded from the reflective measurement-model assessment and examined separately through group comparisons.

Before full data collection, the questionnaire was reviewed for content relevance, wording clarity, comprehensibility, and contextual suitability. A pilot test was then conducted with 30 knowledge workers who had characteristics similar to those of the target population. Participants evaluated the clarity, relevance, comprehensibility, and possible ambiguity of the items. Based on their feedback, four items underwent minor wording revisions. These changes improved clarity without altering the conceptual meaning of the measures.

The questionnaire was administered in Arabic and English. A qualified translator translated the English version into Arabic. An independent bilingual specialist then back-translated the Arabic version into English. The original and back-translated English versions were compared to assess semantic and conceptual equivalence. Any discrepancies were discussed and resolved before pilot testing. This process was used to improve linguistic accuracy and conceptual consistency between the two versions.

4.5. NLP-Based Semantic Analytics and Validation

The NLP analysis used unstructured text collected mainly from participants’ open-ended responses about AI-supported organisational decision-making. Anonymised organisational materials were added as supplementary data where permission was available. After data-quality screening, the final corpus contained 224 textual units: 200 participant responses and 24 supplementary organisational documents. The corpus contained approximately 24,860 words and 31,420 tokens. The mean document length was 130.2 words (SD = 58.7), and the median was 119 words.

Responses were collected in Arabic and English. Arabic responses were translated into English before computational analysis. The same bilingual validation procedure used for the questionnaire was applied to check the translated text. This process provided a common English-language representation for the NLP analysis. The analysis was implemented in Python 3.11. Sentence-Transformers was used to generate semantic embeddings, BERTopic 0.16.0 for topic modelling, UMAP 0.5.5 for dimensionality reduction, HDBSCAN 0.8.33 for clustering, Hugging Face Transformers for sentiment classification, and NetworkX for semantic-network analysis.

Before analysis, all personal and organisational identifiers were removed. The text was converted to lowercase, and punctuation and non-informative symbols were removed. Tokenisation, stop-word removal, and lemmatisation were then performed. Semantic embeddings were generated using the Sentence-Transformers `all-mpnet-base-v2` model. This model represents each textual unit as a 768-dimensional dense vector. The resulting embeddings were used for dimensionality reduction and clustering.

Topic modelling was performed using BERTopic 0.16.0. UMAP 0.5.5 was configured with `n_neighbors = 15`, `n_components = 5`, `min_dist = 0.0`, `metric = cosine`, and `random_state = 42`. HDBSCAN 0.8.33 was configured with `min_cluster_size = 10`, `min_samples = 5`, `metric = euclidean`, `cluster_selection_method = eom`, and `prediction_data = True`. Topic representations were generated using class-based TF-IDF (c-TF-IDF). The ten highest-weighted terms were retained to support the interpretation of each candidate topic.

The initial BERTopic solution identified eight substantive clusters and 11 outlier textual units (4.9%). The final number of topics was not predetermined. Candidate solutions were compared using topic coherence, semantic interpretability, topic redundancy, and inspection of representative documents. Two pairs of semantically overlapping topics were merged after examining their c-TF-IDF terms, representative documents, and cosine similarities. Topic quality was evaluated using the (C_v) coherence metric and topic diversity. The final six-topic solution achieved a mean (C_v) coherence score of 0.71, compared with 0.63 for the initial eight-topic solution. Topic diversity was 0.87, indicating limited overlap among the final topics.

Topic stability was assessed by repeating the modelling procedure across 10 random seeds and repeated 80% subsamples of the corpus. Corresponding topics across runs were compared using semantic similarity between their topic representations. The final solution achieved a mean stability score of 0.84, indicating that the main semantic structure was consistently recovered across repeated estimations. Human validation was also conducted. Two independent bilingual researchers with experience in AI and organisational research reviewed the ten highest-weighted terms and representative documents for each topic. They independently assigned conceptual labels to the topics.

Inter-coder agreement was high (Cohen’s κ = 0.86). Disagreements were resolved through discussion. This procedure produced six final themes: AI Competency, Decision Trust, AI Governance, Decision Support, Organisational Learning, and Risk Awareness. Semantic-network analysis was then conducted using keyword and concept co-occurrence. Nodes represented salient organisational concepts, and edges represented concepts that occurred within the same textual unit. Terms occurring fewer than five times were removed to reduce noise. Degree centrality, betweenness centrality, and eigenvector centrality were calculated to identify influential concepts.

Community structure was identified using the Louvain community-detection algorithm applied to the weighted semantic network. The algorithm partitions nodes by maximising modularity, while edge weights represent concept co-occurrence frequency. The quality of the resulting partition was assessed using the modularity coefficient Q. Aspect-Based Sentiment Analysis (ABSA) was used to assess sentiment toward specific AI-related aspects rather than assigning a single sentiment label to each complete response. The analysis included predefined and topic-informed aspects: productivity enhancement, decision support, knowledge accessibility, organisational learning, AI competency, trust, governance, transparency, explainability, algorithmic bias, and risk awareness.

Sentiment classification was performed using the `cardiffnlp/twitter-roberta-base-sentiment-latest` transformer model. For each aspect-level instance, the model generated probabilities for positive, neutral, and negative sentiment. The class with the highest probability was assigned when the maximum probability was ≥0.60. Predictions below this threshold were treated as uncertain and manually reviewed. Sentiment classification was validated using a stratified sample of 250 aspect–sentiment instances. Two independent bilingual researchers manually coded these instances. Inter-coder agreement was high (Cohen’s κ = 0.88).

Disagreements were resolved to produce an adjudicated human-coded reference set. The automated classification was then compared with this reference set. The principal sentiment evaluation metrics were accuracy, macro-precision, macro-recall, and macro-F1. Accuracy was defined as the proportion of correctly classified instances among all validated instances. Precision measured the proportion of predicted instances in a sentiment class that were correctly classified, whereas recall measured the proportion of actual instances in that class that were correctly identified. The F1-score was calculated as the harmonic mean of precision and recall.

Macro-precision, macro-recall, and macro-F1 were calculated by giving equal weight to the positive, neutral, and negative classes. Against the human-coded reference set, the automated procedure achieved 89.6% accuracy, 89.1% macro-precision, 88.7% macro-recall, and 88.9% macro-F1. Together, topic coherence, topic diversity, stability testing, expert topic validation, inter-coder agreement, and sentiment-classification metrics provided complementary computational and human validation of the NLP results.

4.6. Assessment of Common-Method Bias

Because the predictor, mediator, moderator, and outcome variables were collected from the same respondents, procedural and statistical measures were used to reduce and assess potential common-method bias. Several procedural safeguards were applied. Respondent anonymity was maintained, participation was voluntary, and items were written in clear and neutral language.

Participants were informed that there were no correct or incorrect answers. The questionnaire was also pilot tested to reduce ambiguity and evaluation apprehension. Statistically, common-method bias was assessed using full-collinearity variance inflation factors (VIFs). All construct-level VIF values were below the conservative threshold of 3.3, suggesting that substantial common-method variance was unlikely to dominate the model. Harman’s single-factor diagnostic was also examined.

The first unrotated factor explained 34.7% of the total variance, below the conventional 50% threshold. Together, these results provided no strong indication that common-method variance dominated the observed relationships. However, these diagnostics cannot completely rule out common-method bias. Because several focal constructs were self-reported by the same respondents at a single point in time, potential common-method bias remains a limitation of the study.

4.7. Statistical Analysis and Evaluation Metrics

PLS-SEM analysis followed two stages: assessment of the reflective measurement model and assessment of the structural model [54,55]. The measurement model was evaluated for indicator reliability, internal consistency, convergent validity, discriminant validity, and collinearity. The structural model was then assessed using standardised path coefficients, bootstrapping, coefficients of determination (R²), predictive relevance (Q²), specific indirect effects, moderation effects, and multigroup comparisons. For reflective construct (j), Average Variance Extracted (AVE) was calculated as

$${AVE}_j={\displaystyle \frac{{\sum }_{i=1}^{kj}{\lambda }_{ij}^2}{k_j}}$$

where ${\lambda }_{ij}$ is the standardised loading of indicator (i) on construct (j), and $kj$ is the number of indicators. An AVE above 0.50 indicated adequate convergent validity.

Composite Reliability (CR) was calculated as

$${CR}_j={\displaystyle \frac{{\left({\sum }_{i=1}^{kj}{\lambda }_{ij}^2\right)}^2}{{\left({\sum }_{i=1}^{kj}{\lambda }_{ij}^2\right)}^2+{\sum }_{i=1}^{kj}{(1-\lambda }_{ij}^2)}}$$

Cronbach’s α and CR were used to assess internal consistency, with values of 0.70 or higher considered satisfactory. Discriminant validity was primarily evaluated using the Heterotrait–Monotrait ratio (HTMT), with values below 0.85 indicating adequate construct distinctiveness. The Fornell–Larcker criterion and cross-loadings were examined as supplementary diagnostics [54,55].

Structural relationships were represented by standardised path coefficients (β). Statistical inference was based on non-parametric bootstrapping with 5000 resamples. A relationship was considered statistically supported when the 95% bootstrap confidence interval excluded zero and the two-tailed (p)-value was below 0.05. The coefficient of determination was calculated as

$$R^2={\displaystyle \frac{{\sum }_{i=1}^n{(y_i-\widehat{y_i})}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}$$

where $y_i$ is the observed value, $\widehat{y_i}$ is the model-predicted value, and $\overline{y}$ is the sample mean. (R²) represents the proportion of variance in an endogenous construct explained by its predictors.

Predictive relevance was assessed using Stone–Geisser’s (Q²):

$$Q^2=1-{\displaystyle \frac{SSE}{SSO}}$$

where (SSE) is the sum of squared prediction errors and (SSO) is the sum of squared observations. A (Q²) value greater than zero indicated predictive relevance for the endogenous construct.

Mediation was assessed using specific indirect effects. For mediator (M), the indirect effect was calculated as

$${IE}_M={\beta }_{AIL\to M}\times {\beta }_{M\to SDQ}$$

The significance of each indirect effect was assessed using 95% bootstrap confidence intervals based on 5000 resamples. An indirect effect was considered significant when its confidence interval excluded zero. When both the direct and indirect effects were significant and had the same direction, the pattern was interpreted as complementary partial mediation. Moderation by Digital Proficiency and AI Governance Awareness was tested using interaction terms:

$$SDQ={\beta }_0{+\beta }_{1}AIL+{\beta }_3 \left(AIL\times Z\right)+\epsilon$$

where (Z) represents the moderator and β₃ is the interaction coefficient. A moderation effect was considered significant when the 95% bootstrap confidence interval for β₃excluded zero. Significant interactions were interpreted using simple slopes at one standard deviation below the moderator mean (−1 SD), at the mean, and one standard deviation above the mean (+1 SD). The magnitude of the interaction effect was additionally evaluated using (F²). For predictor (x), the effect size was calculated as

$$F^2={\displaystyle \frac{R_{included}^2-R_{excluded}^2}{{1-R}_{included}^2}}$$

where $R_{included}^2$ is obtained from the model containing the predictor and $R_{excluded}^2$ from the model without that predictor.

Functional Background was analysed as an observed categorical grouping variable rather than a reflective construct. Before PLS multigroup analysis, measurement invariance of the reflective constructs was assessed using the three-stage Measurement Invariance of Composite Models (MICOM) procedure.

The procedure examined (1) configural invariance, (2) compositional invariance, and (3) equality of composite means and variances. Multigroup differences were interpreted only after establishing the level of measurement invariance required for meaningful group comparison. These procedures provided explicit criteria for evaluating measurement quality, structural relationships, mediation, moderation, predictive relevance, and group differences.

5. Results

5.1. Measurement Model Evaluation

Functional Background was excluded from the reflective measurement-model assessment because it was operationalised as an observed categorical variable. Therefore, indicator loadings, Cronbach’s α, Composite Reliability (CR), Average Variance Extracted (AVE), cross-loadings, the Fornell–Larcker criterion, and HTMT were evaluated only for AI Illiteracy (AIL), Automation Bias (AB), Uncritical Trust in AI (UTA), Cognitive Offloading (CO), Digital Proficiency (DP), AI Governance Awareness (AGA), and Strategic Decision Quality (SDQ). Functional Background was examined separately through multigroup analysis.

Figure 2 shows satisfactory reliability and convergent validity for all reflective constructs. Indicator loadings ranged from 0.812 to 0.901 and were above the recommended threshold of 0.70. AVE values ranged from 0.706 to 0.775 and exceeded 0.50. CR values ranged from 0.905 to 0.945, while Cronbach’s α values ranged from 0.861 to 0.927. These results support the reliability and convergent validity of the measurement model.

Figure 2. Quality of the Conceptual Model.

Discriminant validity was assessed using cross-loadings, the Fornell–Larcker criterion, and the Heterotrait–Monotrait ratio (HTMT). Discriminant validity was subsequently examined using cross-loadings, the Fornell–Larcker criterion, and the Heterotrait–Monotrait ratio (HTMT) (see Figure 3).

Figure 3. Cross-loading analysis.

Each reported indicator loaded more strongly on its intended construct than on the other constructs. This pattern provides additional support for discriminant validity.

Figure 4 shows that the square root of AVE for each construct was greater than its correlations with the other constructs. The Fornell–Larcker criterion therefore provided support for discriminant validity.

Figure 4. Fornell–Larcker criterion.

Figure 5 shows that all HTMT values ranged from 0.346 to 0.741 and remained below the conservative threshold of 0.85. The HTMT results therefore support discriminant validity. The cross-loading and Fornell–Larcker results provided additional evidence that the reflective constructs were empirically distinct. All VIF values were below 3.3, indicating no serious collinearity concerns.

Figure 5. HTMT criterion.

Overall, the measurement model showed satisfactory reliability, convergent validity, discriminant validity, and collinearity levels. These results supported subsequent evaluation of the structural model.

5.2. Structural Model Evaluation

The structural model was assessed using PLS-SEM with 5000 bootstrap resamples. The analysis examined the direct structural relationships and the interaction terms for Digital Proficiency and AI Governance Awareness.

Figure 6 shows that AI Illiteracy was negatively associated with Strategic Decision Quality (β = −0.281, p < 0.001). AI Illiteracy was also positively associated with Automation Bias (β = 0.612, p < 0.001), Uncritical Trust in AI (β = 0.573, p < 0.001), and Cognitive Offloading (β = 0.551, p < 0.001). Automation Bias (β = −0.194, p < 0.001), Uncritical Trust in AI (β = −0.223, p < 0.001), and Cognitive Offloading (β = −0.187, p < 0.001) were each negatively associated with Strategic Decision Quality.

Figure 6. Structural model.

The interaction terms for Digital Proficiency and AI Governance Awareness were also statistically significant. Functional Background was not included as an interaction term because it was treated as a categorical grouping variable and examined separately through multigroup analysis. The model’s explanatory and predictive relevance was assessed using the coefficient of determination (R²) and Stone–Geisser’s (Q²).

As shown in Figure 7, the model explained 37.4% of the variance in Automation Bias, 32.8% in Uncritical Trust in AI, 30.4% in Cognitive Offloading, and 68.4% in Strategic Decision Quality. All (Q²) values were above zero, ranging from 0.240 to 0.439. These results indicate predictive relevance for all endogenous constructs within the PLS-SEM assessment. Strategic Decision Quality showed the highest (R²) and (Q²) values.

Figure 7. Predictive Relevance and Coefficient of Determination.

5.3. Mediation Analysis

The mediating roles of Automation Bias, Uncritical Trust in AI, and Cognitive Offloading were assessed using 5000 bootstrap resamples. Mediation was evaluated using specific indirect effects and their 95% bias-corrected bootstrap confidence intervals. The indirect effect of AI Illiteracy on Strategic Decision Quality through Automation Bias was β = −0.119. The indirect effect through Uncritical Trust in AI was β = −0.128, while the indirect effect through Cognitive Offloading was β = −0.103. All three indirect effects were statistically significant because their confidence intervals did not include zero.

The total indirect effect across the three mediators was β = −0.350. The direct association between AI Illiteracy and Strategic Decision Quality remained negative and statistically significant after the mediators were included (β = −0.281, p < 0.001). Because the direct and indirect effects were significant and operated in the same direction, the results indicate complementary partial mediation. Uncritical Trust in AI showed the strongest indirect effect, followed by Automation Bias and Cognitive Offloading (see Table 3).

Table 3. Specific Indirect Association, where causality is implied and Mediation Analysis.

Indirect Association, Where Causality Is Implied	β	t	p	95% Bootstrap CI	Mediation Type
AIL → AB → SDQ	−0.119	3.842	<0.001	[−0.181, −0.064]	Complementary partial
AIL → UTA → SDQ	−0.128	4.176	<0.001	[−0.193, −0.071]	Complementary partial
AIL → CO → SDQ	−0.103	3.517	<0.001	[−0.164, −0.052]	Complementary partial
Total indirect association, where causality is implied	−0.350	5.736	<0.001	[−0.467, −0.239]	—
Direct association, where causality is implied: AIL → SDQ	−0.281	4.891	<0.001	[−0.394, −0.169]	—
Total association, where causality is implied	−0.631	8.214	<0.001	[−0.754, −0.508]	Complementary partial mediation

The mediation results support H2, H3, and H4. Higher AI Illiteracy was associated with lower Strategic Decision Quality both directly and indirectly through Automation Bias, Uncritical Trust in AI, and Cognitive Offloading.

5.4. Moderation Analysis

The moderating roles of Digital Proficiency and AI Governance Awareness in the association between AI Illiteracy and Strategic Decision Quality were examined using interaction terms. Interaction strength was assessed using (f²), and significant interactions were interpreted using simple-slope analysis at low (−1 SD), mean, and high (+1 SD) levels of each moderator. Digital Proficiency significantly moderates the association between AI Illiteracy and Strategic Decision Quality (β = 0.162, t = 2.781, p = 0.005, (f²) = 0.034). The interaction effect was small.

Simple-slope analysis showed that the negative association between AI Illiteracy and Strategic Decision Quality was strongest when Digital Proficiency was low and became weaker as Digital Proficiency increased. This result suggests that stronger digital proficiency may reduce the vulnerability associated with AI illiteracy.

AI Governance Awareness also significantly moderated the association between AI Illiteracy and Strategic Decision Quality (β = 0.209, t = 3.751, p < 0.001, (f²) = 0.061). This interaction was also small but stronger than the interaction for Digital Proficiency. The negative association between AI Illiteracy and Strategic Decision Quality was strongest at low levels of AI Governance Awareness and became weaker at higher levels. This pattern suggests that greater awareness of transparency, accountability, verification, ethics, and responsible AI use may act as a protective organisational condition (see Table 4).

Table 4. Moderation Results.

Hs	Moderating Association, Where Causality Is Implied	β	t	p	f2	Association, Where Causality Is Implied Size	Decision
H5	DP × AIL → SDQ	0.162	2.781	0.005	0.034	Small	Supported
H7	AGA × AIL → SDQ	0.209	3.751	<0.001	0.061	Small	Supported

The interaction results support H5 and H7. Higher Digital Proficiency and AI Governance Awareness were associated with a weaker negative relationship between AI Illiteracy and Strategic Decision Quality.

5.5. Measurement Invariance and Multigroup Analysis

Functional Background was treated as an observed categorical variable and examined using PLS multigroup analysis (PLS-MGA). To obtain meaningful group sizes, participants were classified into three broader categories: Executive/Strategic functions, Information Systems/Data Analytics functions, and Operational/Support functions. Before comparing structural coefficients across these groups, measurement invariance of the reflective constructs was assessed using the Measurement Invariance of Composite Models (MICOM) procedure. MICOM was conducted in three stages.

First, configural invariance was assessed by confirming that the same indicators, data-treatment procedures, algorithm settings, and measurement specifications were used across groups. Second, compositional invariance was evaluated using permutation testing. Third, equality of composite means and variances was examined. The MICOM results supported full measurement invariance for AIL, AB, UTA, CO, DP, AGA, and SDQ. This allowed meaningful comparison of the structural relationships across the functional-background groups (see Figure 8).

Figure 8. Complete MICOM Invariance Testing Results.

Following MICOM, PLS-MGA was used to test whether the association between AI Illiteracy and Strategic Decision Quality differed across functional groups. A significant difference was found between the Executive/Strategic and Information Systems/Data Analytics groups (Δβ = −0.200, p = 0.018). The negative association was stronger for participants in Executive/Strategic functions (β = −0.361) than for those in Information Systems/Data Analytics functions (β = −0.194).

The difference between Executive/Strategic and Operational/Support functions was not significant (Δβ = −0.087, p = 0.146). The difference between Information Systems/Data Analytics and Operational/Support functions was also not significant (Δβ = 0.080, p = 0.193). These results provide partial support for H6 and indicate that the association between AI Illiteracy and Strategic Decision Quality varies across some functional-background groups.

5.6. NLP-Driven Semantic Mining Results

The semantic analysis was conducted on 224 textual units, comprising 200 participant-generated open-ended responses and 24 supplementary organisational documents. The corpus contained approximately 24,860 words. BERTopic initially identified eight substantive clusters, while 11 textual units (4.9%) were classified as outliers. The candidate topics were examined using representative documents, c-TF-IDF terms, semantic similarity, topic redundancy, and coherence. Two pairs of overlapping topics were merged, resulting in a final six-topic solution.

The final solution achieved a mean (C_v) coherence score of 0.71, compared with 0.63 for the initial eight-topic solution. Topic diversity was 0.87, indicating limited overlap among the final topics. Topic stability was assessed across repeated model estimations and 80% corpus subsamples. The mean stability score was 0.84, suggesting that the main topic structure remained consistent across repeated analyses. Two independent bilingual researchers also reviewed the final topic labels. They examined the representative keywords and documents for each topic and achieved high agreement (Cohen’s κ = 0.86). Disagreements were resolved through discussion.

The six final themes were AI Competency, Decision Trust, AI Governance, Decision Support, Organisational Learning, and Risk Awareness. AI Competency included references to AI literacy, skills, knowledge, and training. Decision Trust focused on confidence, verification, recommendations, and judgement. AI Governance included policy, ethics, transparency, and accountability. Decision Support reflected analytics, planning, reporting, and forecasting. Organisational Learning focused on collaboration, knowledge sharing, and improvement. Risk Awareness included bias, hallucination, misinformation, and uncertainty (see Table 5).

Table 5. BERTopic semantic themes identified from organisational textual data.

Semantic Theme	Representative Keywords	Organisational Interpretation	Strategic Implication
AI Competency	AI literacy, skills, knowledge, training	Organisational capability to understand and utilise AI	Continuous AI education strengthens organisational readiness
Decision Trust	confidence, verification, recommendation, judgement	Human trust in AI-supported decisions	Verification mechanisms reduce blind reliance on AI
AI Governance	policy, ethics, transparency, accountability	Responsible organisational AI governance	Governance improves trustworthy AI adoption
Decision Support	analytics, planning, reporting, forecasting	AI-assisted strategic decision support	AI augments managerial decision-making
Organisational Learning	collaboration, knowledge sharing, improvement	Continuous organisational learning	AI supports knowledge creation and innovation
Risk Awareness	bias, hallucination, misinformation, uncertainty	Organisational AI-related risks	Risk management enhances strategic resilience

The semantic results complemented the quantitative findings by identifying organisational issues that were not fully captured by the structured measures. The themes highlight the importance of employee competency, trust, governance, decision support, organisational learning, and risk awareness in AI-supported decision environments.

5.7. Semantic Network Analysis

Semantic network analysis was conducted to examine the relationships among the main concepts identified in the organisational AI corpus. Following the preprocessing and minimum-frequency criteria described in Section 4.5, the final network contained 58 nodes and 214 weighted edges. The network density was 0.129, the mean degree was 7.38, and the average clustering coefficient was 0.462. The network formed a single connected component, indicating that the main concepts were connected within a common semantic structure. Centrality measures were used to identify the most influential concepts. AI Literacy had the highest degree centrality (0.684), followed by Verification (0.649), Governance (0.623), and Decision Support (0.596).

These concepts therefore had the strongest direct connections with other concepts in the network. Verification had the highest betweenness centrality (0.286), followed by AI Literacy (0.241) and Governance (0.218), indicating that these concepts served as important semantic bridges between different areas of organisational AI discourse. Eigenvector centrality was also highest for AI Literacy (0.412), Verification (0.397), and Governance (0.381), indicating that these concepts were connected to other influential concepts.

Community structure was identified using the Louvain community-detection algorithm applied to the weighted semantic network. The algorithm groups nodes by maximising network modularity, so that concepts within the same community have stronger connections with each other than with concepts in other communities. Edge weights represented the frequency of concept co-occurrence.

The quality of the resulting partition was assessed using the modularity coefficient Q. The Louvain procedure identified six semantic communities, with a modularity score of Q = 0.438. These communities broadly corresponded to the six themes identified through BERTopic: AI Competency, Decision Trust, AI Governance, Decision Support, Organisational Learning, and Risk Awareness. This correspondence indicates that the thematic structure identified through topic modelling was also reflected in the relational structure of the semantic network.

The network showed close relationships among AI literacy, training, verification, trust, governance, transparency, decision support, knowledge sharing, and risk. AI Literacy occupied a central position and was strongly connected with competency- and training-related concepts. Verification linked competency-related concepts with trust and decision-support concepts, while Governance and Transparency connected responsible AI practices with verification, accountability, and risk-related concerns. Algorithmic Bias and Risk were located within the Risk Awareness community and were connected with governance, trust, and verification concepts.

Figure 9 shows the semantic network of the organisational AI corpus. Node size represents degree centrality, edge width represents co-occurrence strength, and community membership represents the main semantic themes. The semantic network complements the BERTopic findings by showing that organisational AI discourse is organised around interconnected issues of AI competency, verification, trust, governance, decision support, organisational learning, and risk awareness.

Figure 9. Semantic Network of Organisational AI Corpus.

In particular, the central positions of AI literacy, verification, and governance are consistent with the broader structural and semantic findings of the study. These results highlight employee competency, critical verification of AI recommendations, and organisational governance as important conditions associated with effective AI-supported strategic decision-making.

5.8. Aspect-Based Sentiment Analysis and Validation

Aspect-Based Sentiment Analysis (ABSA) was used to assess sentiment toward specific aspects of organisational AI use. Unlike document-level sentiment analysis, ABSA classified evaluations at the aspect level as positive, neutral, or negative. The analysis used the cardiffnlp/twitter-roberta-base-sentiment-latest transformer model. A sentiment label was assigned automatically when the highest predicted probability was at least 0.60. Predictions below this threshold were treated as uncertain and manually reviewed.

Model performance was validated using a stratified sample of 250 aspect–sentiment instances. Two independent bilingual researchers coded the sample and achieved high agreement (Cohen’s κ = 0.88). After disagreements were resolved, the human-coded data were used as the reference standard. The automated ABSA procedure achieved 89.6% accuracy, 89.1% macro-precision, 88.7% macro-recall, and 88.9% macro-F1.

The strongest positive sentiment was found for Productivity Enhancement (78.1%), Decision Support (74.6%), Knowledge Accessibility (71.4%), and Organisational Learning (69.2%). These results show that employees generally viewed AI positively in relation to productivity, access to knowledge, learning, and decision support. Negative sentiment was strongest for Algorithmic Bias (62.6%), Explainability (59.7%), Transparency (53.0%), and Risk Awareness (52.2%). These findings indicate concerns about biased recommendations, opaque AI processes, limited explainability, and AI-related risks. (see Table 6 and Table 7).

Table 6. Semantic Network Centrality Results.

Rank	Concept	Degree Centrality	Betweenness Centrality	Eigenvector Centrality	Community
1	AI Literacy	0.684	0.241	0.412	AI Competency
2	Verification	0.649	0.286	0.397	Decision Trust
3	Governance	0.623	0.218	0.381	AI Governance
4	Decision Support	0.596	0.173	0.364	Decision Support
5	Transparency	0.561	0.201	0.348	AI Governance
6	Trust	0.544	0.194	0.337	Decision Trust
7	Training	0.509	0.136	0.318	AI Competency
8	Risk	0.491	0.166	0.302	Risk Awareness
9	Knowledge Sharing	0.456	0.109	0.286	Organisational Learning
10	Algorithmic Bias	0.421	0.127	0.271	Risk Awareness

Table 7. Aspect-Based Sentiment Analysis (ABSA) across organisational dimensions.

Organisational Aspect	Positive (%)	Neutral (%)	Negative (%)	AI Is Associated with Collaboration and Learning.
Productivity Enhancement	78.1	13.4	8.5	Trust appears to depend on verification and user competency.
Decision Support	74.6	15.8	9.6	Employees show mixed perceptions of AI competency.
Knowledge Accessibility	71.4	18.7	9.9	Governance remains an area of concern.
Organisational Learning	69.2	18.1	12.7	Employees express concern about opaque AI processes.
Trust in AI	43.5	24.8	31.7	Limited explainability is a major concern.
AI Competency	36.9	29.7	33.4	Employees recognise misinformation and AI-related risks.
AI Governance	31.8	26.7	41.5	Algorithmic bias is the most negatively perceived aspect.
Transparency	24.6	22.4	53.0	AI is associated with collaboration and learning.
Explainability	21.8	18.5	59.7	Trust appears to depend on verification and user competency.
Risk Awareness	22.7	25.1	52.2	Employees show mixed perceptions of AI competency.
Algorithmic Bias	16.9	20.5	62.6	Governance remains an area of concern.

Overall, the validated ABSA results show a mixed view of organisational AI use. Employees recognise clear benefits for productivity, knowledge access, and decision support, while also expressing concern about governance, transparency, explainability, bias, and risk.

5.9. Mixed-Methods Integration: Joint Display of PLS-SEM and NLP Findings

The quantitative and textual findings were integrated using a joint display. The purpose was to examine whether the two forms of evidence showed convergence, complementarity, or divergence. Convergence was identified when the semantic evidence was consistent with the direction and meaning of a structural relationship. Complementarity occurred when the textual findings added information that the structural model did not directly capture. Divergence was identified when the textual evidence showed a different or more nuanced pattern. Table 8 links the main PLS-SEM findings with the corresponding semantic themes, keywords, sentiment evidence, and integrated interpretations.

Table 8. Joint Display Integrating PLS-SEM and NLP Findings.

PLS-SEM Finding	Semantic Theme(s)	Representative Keywords/ Semantic Evidence	ABSA/Textual Evidence	Integration	Integrated Interpretation
AIL → SDQ β = −0.281, p < 0.001	AI Competency; Risk Awareness	AI literacy, skills, knowledge, training; bias, hallucination, misinformation, uncertainty	AI Competency: 33.4% negative; Risk Awareness: 52.2% negative	Convergence	The negative structural relationship is reflected in narratives describing competency deficiencies and risks associated with unreliable AI outputs.
AIL → AB β = 0.612, p < 0.001; AB → SDQ β = −0.194, p < 0.001	Decision Trust; Risk Awareness	verification, recommendation, judgement; bias, misinformation	Trust in AI showed mixed sentiment; employees emphasised verification and human oversight	Convergence/Complementarity	The semantic evidence supports automation-bias concerns by showing that employees recognise the need to verify AI recommendations rather than accept them automatically.
AIL → UTA β = 0.573, p < 0.001; UTA → SDQ β = −0.223, p < 0.001	Decision Trust; AI Governance	confidence, verification, recommendation, judgement; transparency, accountability	Trust in AI: 43.5% positive, 24.8% neutral, 31.7% negative	Partial convergence	The quantitative model identifies uncritical trust as detrimental, whereas the text distinguishes between blind trust and conditional trust based on verification and governance.
AIL → CO β = 0.551, p < 0.001; CO → SDQ β = −0.187, p < 0.001	Decision Support; Organisational Learning	analytics, planning, reporting, forecasting; collaboration, knowledge sharing	Decision Support: 74.6% positive; Organisational Learning: 69.2% positive	Divergence/Complementarity	Although cognitive offloading is negatively related to decision quality, employees generally value AI assistance. The divergence suggests that AI support becomes problematic primarily when augmentation shifts into substitution for human reasoning.
DP × AIL → SDQ β = 0.162, p = 0.005	AI Competency; Organisational Learning	skills, knowledge, training; collaboration, improvement	Positive perceptions of knowledge accessibility and organisational learning	Convergence	Textual emphasis on training and capability development supports the protective role of digital proficiency.
AGA × AIL → SDQ β = 0.209, p < 0.001	AI Governance; Risk Awareness	policy, ethics, transparency, accountability; bias, uncertainty	AI Governance: 41.5% negative; Transparency: 53.0% negative; Explainability: 59.7% negative	Convergence with contextual tension	Governance awareness mitigates the negative association, where causality is implied of AI illiteracy, while negative sentiment indicates that employees still perceive existing governance and transparency arrangements as insufficient.
SDQ/AI-enabled decision support	Decision Support	analytics, planning, reporting, forecasting	Decision Support: 74.6% positive; Productivity Enhancement: 78.1% positive	Complementarity	Employees recognise substantial organisational benefits from AI-enabled decision support despite the structural risks associated with poor AI literacy and excessive reliance.
Overall AI vulnerability mechanism	AI Competency; Decision Trust; AI Governance; Risk Awareness	literacy, verification, transparency, accountability, bias, uncertainty	Positive benefit perceptions coexist with strong concern about bias and explainability	Mixed convergence and divergence	AI is perceived simultaneously as a valuable decision-support capability and a source of cognitive and governance risk, supporting a conditional rather than uniformly positive or negative interpretation of organisational AI use.

The joint display shows that the two analytical streams were not fully identical. Several findings converged, especially those related to AI competency, governance, verification, and risk. Other findings were more nuanced. For example, employees viewed AI-supported decision support positively, while the structural model showed that Cognitive Offloading was negatively associated with Strategic Decision Quality. This suggests an important distinction between AI augmentation, where AI supports human reasoning, and cognitive substitution, where AI replaces independent analysis.

A similar pattern emerged for trust. The structural model identified Uncritical Trust in AI as a risk, but the textual findings showed that trust itself was not uniformly negative. Instead, employees distinguished between trust based on verification and blind reliance on AI outputs. Overall, the mixed-methods integration shows that the NLP findings did more than repeat the PLS-SEM results. They added context to the structural relationships and helped explain when AI support may be useful and when it may create decision risks.

6. Discussion

This study developed and examined an explainable Management Information Systems (MIS) framework that integrates Partial Least Squares Structural Equation Modelling (PLS-SEM) with NLP-based semantic analytics. The aim was to examine how AI illiteracy is associated with strategic decision quality and how this relationship is linked to cognitive reliance mechanisms and organisational conditions.

The structural results showed that higher AI illiteracy was negatively associated with Strategic Decision Quality (β = −0.281, p < 0.001). AI illiteracy was also positively associated with Automation Bias (β = 0.612), Uncritical Trust in AI (β = 0.573), and Cognitive Offloading (β = 0.551). The model explained 68.4% of the variance in Strategic Decision Quality (R² = 0.684), while Q² = 0.439 indicated predictive relevance within the PLS-SEM assessment. These findings are consistent with previous MIS research showing that the value of AI depends not only on technology adoption but also on employees’ ability to interpret and evaluate AI-generated information [31,32,33,34,35,36,37,38,39,40]. They also support the view that human capability remains important in AI-supported decision environments. Advanced AI systems may provide useful analytical support, but employees still need sufficient knowledge to identify errors, question recommendations, and apply professional judgement.

The mediation results provide further insight into this relationship. Automation Bias, Uncritical Trust in AI, and Cognitive Offloading each showed significant indirect effects between AI Illiteracy and Strategic Decision Quality. This pattern supports Dual-Process Theory by suggesting that limited AI-related knowledge may be linked to greater reliance on automated recommendations and less independent evaluation. Among the three mediators, Uncritical Trust in AI showed the strongest indirect effect. The findings also show that trust should not be treated as uniformly negative.

Trust becomes problematic when employees accept AI recommendations without adequate verification. This distinction is important because appropriate trust can support human–AI collaboration, while uncritical trust may increase decision risk. The semantic results support this interpretation by showing mixed sentiment toward trust and frequent references to verification, judgement, and human oversight.

Cognitive Offloading showed a similar pattern. The structural results linked greater offloading with lower Strategic Decision Quality, while the textual analysis showed positive views of Decision Support, Productivity Enhancement, Knowledge Accessibility, and Organisational Learning. This difference is not necessarily contradictory. It suggests a distinction between AI augmentation and cognitive substitution.

AI may be useful when it supports analysis and knowledge access, but greater risk may arise when employees allow AI to replace independent reasoning. The moderation results also highlight the role of organisational conditions. Digital Proficiency and AI Governance Awareness weakened the negative association between AI Illiteracy and Strategic Decision Quality. These findings are consistent with the Technology–Organisation–Environment framework, which emphasises the importance of organisational readiness, employee capabilities, and governance conditions in technology use [38,40,55].

AI Governance Awareness showed the stronger interaction. This finding suggests that employees who understand transparency, accountability, verification, ethics, and responsible AI practices may be better positioned to manage the risks associated with limited AI literacy. However, the textual evidence also showed negative sentiment toward governance, transparency, explainability, and algorithmic bias. This indicates that awareness of governance requirements should not be interpreted as evidence that existing governance practices are sufficient.

Functional Background also provided useful contextual evidence. The relationship between AI Illiteracy and Strategic Decision Quality differed significantly between Executive/Strategic and Information Systems/Data Analytics groups. The negative association was stronger among employees in Executive/Strategic functions. This may reflect differences in technical familiarity, domain expertise, or the type of decisions made across occupational groups. However, because only one group comparison was statistically significant, this finding should be interpreted cautiously.

The mixed-methods integration added important context to the structural results. Several findings converged across the quantitative and textual evidence. AI Competency and Risk Awareness reflected the negative association between AI Illiteracy and Strategic Decision Quality. Decision Trust was related to the mediation results for Automation Bias and Uncritical Trust in AI. AI Governance themes were also consistent with the moderating role of governance awareness.

Other findings showed complementarity or divergence. In particular, employees expressed positive views of AI-supported decision support despite the negative structural relationship involving Cognitive Offloading. This suggests that employees may value AI as a support tool while still facing risks when reliance becomes excessive. Similarly, the textual evidence distinguished between trust based on verification and blind reliance on AI recommendations. These differences show the value of integrating PLS-SEM with NLP-based semantic analysis.

The NLP results did not simply repeat the structural findings. Instead, they helped explain how employees describe competency gaps, trust, governance, decision support, organisational learning, and AI-related risks. The joint display therefore provides a more detailed interpretation of the structural relationships. From a theoretical perspective, the findings support the integrated use of AST, DPT, and TOE. AST helps explain AI illiteracy as a technology-appropriation vulnerability. DPT explains the cognitive reliance mechanisms represented by Automation Bias, Uncritical Trust in AI, and Cognitive Offloading.

TOE explains why organisational conditions such as Digital Proficiency, Functional Background, and AI Governance Awareness may shape the strength of these relationships. From an MIS perspective, the study contributes by treating AI illiteracy as more than an individual knowledge gap. The findings suggest that it can also be viewed as an organisational decision vulnerability linked to how employees interpret AI outputs, rely on automated recommendations, and respond to governance requirements.

From an NLP perspective, the study shows how semantic analytics can complement structural modelling. Sentence-BERT, BERTopic, semantic-network analysis, and ABSA provided contextual evidence that helped interpret the quantitative results. The contribution therefore lies not in combining methods alone but in using structured and unstructured evidence to examine the same human–AI decision process from different analytical perspectives.

Overall, the findings suggest that responsible AI-supported decision-making depends on a balance between technological capability, employee competency, human verification, and organisational governance. AI can support productivity, learning, and decision analysis. Still, these benefits may be limited when employees lack sufficient AI-related knowledge, rely on AI without verification, or operate within weak governance environments. Because the study is cross-sectional and mainly based on self-reported data, these findings should be interpreted as associations rather than causal relationships.

7. MIS Contributions

This study makes three main contributions to Management Information Systems (MIS) research. First, it develops and empirically examines a human–AI decision vulnerability framework that conceptualises AI illiteracy as an organisational decision-support vulnerability rather than only an individual competency gap. The findings show that higher AI illiteracy is associated with lower Strategic Decision Quality, both directly and indirectly through Automation Bias, Uncritical Trust in AI, and Cognitive Offloading. This perspective extends AI literacy research by linking employee competency to cognitive reliance patterns in AI-supported organisational decisions.

Second, the study integrates structured PLS-SEM evidence with semantic evidence extracted from organisational narratives. The contribution therefore goes beyond applying PLS-SEM and NLP as separate methods. The joint analysis shows how semantic themes, concept relationships, and aspect-level sentiment can contextualise structural relationships and identify convergence, complementarity, and divergence between structured and unstructured evidence. This approach provides a richer understanding of how employees experience and interpret AI-supported decision-making.

Third, the findings identify organisational conditions that may reduce decision vulnerability. Digital Proficiency and AI Governance Awareness weakened the negative association between AI Illiteracy and Strategic Decision Quality. Functional-background analysis also showed that this relationship differed between selected occupational groups. The textual evidence further highlighted training, verification, transparency, accountability, explainability, and human oversight as important organisational concerns. These findings provide a basis for developing more human-centred and responsible AI decision-support practices.

The study does not establish causal relationships or long-term organisational performance effects. Because the research used a cross-sectional and predominantly self-reported design, the structural coefficients represent theoretically specified associations. Longitudinal, experimental, and multi-source studies are needed to determine whether these relationships persist over time and whether improvements in AI competency and governance lead to better decision outcomes.

8. Conclusions

This study developed and empirically examined an explainable MIS framework integrating NLP-based semantic analytics with PLS-SEM to investigate the relationships among AI Illiteracy, cognitive reliance mechanisms, organisational conditions, and Strategic Decision Quality. The results show that higher AI Illiteracy is negatively associated with Strategic Decision Quality. Automation Bias, Uncritical Trust in AI, and Cognitive Offloading showed significant indirect relationships between AI Illiteracy and Strategic Decision Quality. Digital Proficiency and AI Governance Awareness weakened the negative association, while functional-background analysis indicated differences between selected occupational groups.

The NLP analysis complemented these findings by identifying six organisational themes: AI Competency, Decision Trust, AI Governance, Decision Support, Organisational Learning, and Risk Awareness. Employees generally viewed AI positively in relation to productivity, decision support, knowledge accessibility, and organisational learning. However, they also expressed concerns about algorithmic bias, explainability, transparency, governance, and AI-related risks. The main contribution is not the combination of NLP and PLS-SEM itself.

Rather, the study uses these methods to examine a human–AI decision vulnerability framework in which AI illiteracy is associated with strategic decision quality through cognitive reliance mechanisms and organisational boundary conditions. The semantic evidence adds context by showing how these relationships are reflected in employees’ descriptions of trust, verification, competency, governance, and risk.

From a practical perspective, the framework can help organisations identify AI competency gaps, examine patterns of reliance on AI recommendations, assess governance awareness, and detect textual signals related to trust, verification, transparency, and AI risk. The findings also highlight the importance of distinguishing AI augmentation from cognitive substitution. AI may support decision-making when it complements human judgement, but excessive reliance may create additional decision vulnerabilities. Because the study used a cross-sectional observational design, the findings should be interpreted as associations rather than causal effects. Further evidence is required before concluding that changes in AI literacy, governance, or reliance behaviour lead to subsequent improvements in strategic decision outcomes.

9. Limitations and Future Studies

This study has several limitations that provide directions for future research. First, the cross-sectional design does not establish temporal order or causal relationships among AI Illiteracy, cognitive reliance mechanisms, organisational conditions, and Strategic Decision Quality. Longitudinal studies could examine how these relationships change as employees gain experience with AI. Experimental and quasi-experimental studies could also test whether AI literacy training, verification protocols, or governance interventions change reliance behaviour and subsequent decision outcomes.

Second, the final sample consisted of 200 knowledge workers employed in public-sector organisations in Saudi Arabia. Although participants represented several industries, functional roles, and nationalities, the findings may not generalise to private organisations, other countries, or different regulatory and cultural environments. Future research should test the framework using larger samples across countries, sectors, and organisational settings. Third, several focal constructs were measured through self-reported data collected from the same respondents.

Although procedural and statistical checks did not indicate that common-method variance dominated the results, it cannot be completely excluded. Future studies should combine employee surveys with supervisor assessments, behavioural measures, system logs, decision outcomes, and other independent data sources. Fourth, the NLP corpus was relatively small and consisted mainly of participants’ open-ended responses, supplemented by a limited number of organisational documents. Future studies could examine larger and more diverse textual corpora, including decision reports, policy documents, internal communications, meeting records, and organisational knowledge repositories. Larger corpora would also allow more extensive evaluation of topic stability and semantic generalisability.

Fifth, the NLP pipeline used specific embedding, topic-modelling, network, and sentiment-analysis methods. Although computational and human validation procedures were applied, different models or parameter settings may produce different semantic structures. Future studies should compare alternative embedding models, topic-modelling approaches, multilingual models, and sentiment classifiers to assess robustness.

Finally, future research could extend the framework by examining trust calibration, human oversight, explainability, organisational AI maturity, resilience, and ethical AI capability. Advanced methods such as knowledge graphs, retrieval-augmented generation, and explainable language models may also be investigated where they provide clear analytical value. These extensions could help determine how human competency, AI design, and organisational governance jointly shape responsible AI-supported decision-making.