How Does Prompt Anchoring Affect Large Language Model Outputs?

Abstract

This study examines how different prompt anchoring strategies influence the conceptual representation of LLM-generated keywords and compares those effects with the effects of model selection. A controlled exploratory experiment evaluated four prompt conditions—No Examples, Brief Keywords, Detailed Explanations, and Author-Based Examples—across T. D. Wilson’s four information behavior dimensions using 1068 abstracts. Four LLMs (GPT-4o-mini, Claude-3-haiku, Gemini-2.0-flash-lite, and DeepSeek V3) were evaluated under all prompt conditions, yielding 17,036 valid observations. Results indicate that model identity accounts for substantially more variance in keyword generation (η² = 0.309) than prompt condition (η² = 0.069), although these estimates should be interpreted with caution given the repeated-measures design and assumption violations. Prompt anchoring, however, consistently reconfigured the conceptual distribution of outputs across all models, indicating that it influences conceptual representation even when model effects are larger. Author-Based Examples substantially increased representation of the typically underrepresented Information Sharing dimension, whereas Detailed Explanations produced the highest overall generation rates and the broadest dimensional coverage. These findings further indicate that different anchoring strategies involve consistent trade-offs in dimensional coverage. The study thereby identifies prompt anchoring as a source of methodological variation in LLM-assisted content analysis, indicating that anchoring strategies should be explicitly specified, justified, and reported as part of the study methodology.

1. Introduction

Large language models (LLMs) are increasingly used to support keyword generation for content analysis tasks such as systematic literature reviews, bibliometric mapping, and automated coding of scholarly texts. Their outputs are sensitive to prompt design, and different prompt strategies can systematically influence which concepts are emphasized or neglected. For researchers designing reproducible and theoretically grounded analytical workflows, understanding how prompt structure shapes model output is an important methodological concern.

One aspect of this sensitivity involves what this study terms prompt anchoring. Prompt anchoring, as operationalized here, refers to the systematic influence of prompt content, specificity, and example type on the conceptual distribution of LLM-generated outputs. This usage is distinct from the classic anchoring heuristic (Tversky & Kahneman, 1974), in which initial numerical or categorical information disproportionately biases subsequent judgments, and from studies that examine whether anchor hints shift LLM responses toward specific values (Huang et al., 2025; Lou & Sun, 2024). Rather, prompt anchoring describes how different forms of semantic guidance embedded in a prompt guide which conceptual areas a model prioritizes in its output. Empirical research on anchoring effects within LLMs remains nascent (Huang et al., 2025; Lou & Sun, 2024), and no widely adopted taxonomy of prompt anchoring strategies currently exists. This study contributes to this emerging literature through a systematic descriptive comparison of distinct prompt conditions. It does not seek to explain the cognitive or computational mechanisms underlying these effects but rather to characterize the resulting output patterns.

Information behavior was selected as the experimental domain because T. D. Wilson’s framework delineates four bounded dimensions, Information Seeking, Information Use, Information Sharing, and Information Needs, with relatively clear conceptual distinctions. This framework provides a controlled setting for examining how prompt variation affects coverage across multiple dimensions within a theoretical framework. Beyond these structural properties, information behavior research represents a mature area of Library and Information Science (LIS) scholarship with well-established theoretical terminology and practical relevance for bibliometric preprocessing and automated content analysis. Although prompt sensitivity in LLMs has received increasing attention, systematic comparisons of structurally distinct prompt anchoring strategies within a theoretically grounded analytical framework remain limited. As a result, prompt choices may introduce consistent variation into automated analytical workflows without researchers being aware of it.

Thus, this study aims to systematically examine how different prompt anchoring strategies influence the conceptual representation of LLM-generated keywords and to compare those effects with those of model selection. To address this objective, four prompt conditions varying in guidance intensity were evaluated, ranging from minimal direction in the No Examples condition to lexical cues in Brief Keywords, conceptual scaffolding in Detailed Explanations, and explicit theoretical references in Author-Based Examples. Output patterns were compared across four LLM architectures—GPT-4o-mini, Claude-3-haiku, Gemini-2.0-flash-lite, and DeepSeek V3—using quantitative measures of generation volume, dimensional coverage, and keyword similarity. This study is exploratory and descriptive by design. Rather than evaluating whether any prompt or model produces objectively correct keywords, it characterizes systematic patterns in how prompt anchoring shapes LLM-generated outputs. This approach is appropriate given the interpretive nature of keyword generation, which lacks a single ground truth for evaluation. In doing so, it identifies prompt anchoring as a source of methodological variation in LLM-assisted content analysis and provides an empirical basis for its systematic consideration in future studies.

2. Literature Review

This literature review addresses three domains relevant to LLM-based keyword generation. These include conceptual frameworks, applied uses of LLMs, and prompt design and evaluation. Together, these domains frame how prompt variation influences keyword generation across distinct dimensions of information behavior.

This study adopts Wilson’s (1999) information behavior framework, which organizes human interaction with information into four dimensions. These include Information Seeking (Wilson, 1999; Bates, 1989; Ellis, 1989; Marchionini, 2006; Kuhlthau, 1991), Information Use (Dervin, 1983; Taylor, 1986), Information Sharing (Chatman, 1996; Granovetter, 1973; Wenger, 1998; Haythornthwaite, 1996), and Information Needs (Belkin, 1980; Taylor, 1968; Case & Given, 2016). These dimensions are well-defined theoretically and widely cited in information science research. They provide a structured basis for categorizing information behavior constructs found in scholarly texts and serve two functions in this study: they define the conceptual categories used across all prompt conditions and provide a testbed for examining dimensional bias given natural variation in representation across academic discourse.

Prior scientometric analyses of the information behavior literature have documented that these four dimensions are represented unevenly across the published record. Information Seeking receives substantially more empirical attention than Information Sharing (Wilson, 1999; Case & Given, 2016). This documented imbalance provides an independent basis for assessing dimensional bias in LLM outputs, linking the present experimental design to established patterns in the scholarly literature rather than treating dimension frequencies as purely model-determined. The framework’s combination of terminological stability, theoretical maturity, and active use in bibliometric preprocessing workflows makes it well suited for evaluating LLM keyword generation in scientometric contexts.

In applied contexts, prior work has examined the use of LLMs to identify information behavior patterns in scholarly texts. Mannuru et al. (2024) describe how conversational interfaces introduce new affordances for querying, summarization, and text generation, stressing the role of prompt optimization in reducing bias and inaccuracy. Williams et al. (2024) show that prompt formulation shapes interpretive capacity in qualitative analysis. Ramanathan et al. (2025) demonstrate that structured prompts aligned with analytic codes improve LLM performance in content analysis. Tian et al. (2024) report that prompt synthesis influences output quality in conceptual generation tasks.

Research on prompt design shows that prompt engineering plays a central role in shaping LLM output quality, and that model behavior is highly sensitive to prompt variation (Chatterjee et al., 2024; Lu et al., 2024; Zhuo et al., 2024). Semantically diverse examples improve output quality and generalization (Chen & Wang, 2024). Optimization approaches range from structured methods for long prompts (Kumar et al., 2024) to prompt distillation for domain-specific applications (Li et al., 2023; Zeng et al., 2025). Requirement articulation outperforms templated strategies (Ma et al., 2025). Prompt informativeness is systematically related to response uncertainty (Zhang et al., 2024). Sahoo et al. (2024) and Fagbohun et al. (2024) provide typologies of prompting techniques and Atreja et al. (2025) show that prompt design significantly affects annotation compliance and accuracy in complex analytical tasks. Additional work links prompt formulation to response length and energy efficiency (Poddar et al., 2025), reliability and consistency (Wang et al., 2024), and output diversity (Kusano et al., 2025). Prompt phrasing also shapes model bias profiles (Torres et al., 2024; Sun & Kok, 2025), uncertainty expression (Ulmer et al., 2025), and hallucination rates (Rawte et al., 2024).

Although most prior studies examine classification, annotation, or generation tasks rather than keyword generation specifically, they collectively demonstrate that prompt structure systematically influences LLM outputs.

The literature points to three converging gaps. First, prompt engineering research has examined sensitivity, efficiency, and bias but few studies test structurally distinct prompt conditions within a single, theoretically grounded framework. Second, LLM applications in information behavior research have rarely examined how prompt anchoring shapes dimensional coverage across Wilson’s four categories. Third, similarity analyses of LLM outputs rely largely on lexical measures and overlook conceptual convergence that can be captured through semantic embeddings. This study addresses these gaps by evaluating four prompt conditions across four LLM architectures using Wilson’s information behavior framework as the analytic lens. It combines lexical and semantic similarity analysis to provide a more complete account of how anchoring shapes automated keyword generation.

3. Methodology

This study employed a controlled experimental design to evaluate the impact of different prompt conditions on LLM keyword generation behavior. A within-subjects design was implemented, in which each of the 1068 abstracts was processed through all four prompt conditions across four different LLMs. The workflow comprised four stages: literature search, experimental design, API-based data collection across multiple models and trials, and statistical analysis. The workflow is described in Figure 1, with further details provided in the following sections.

Figure 1. Research Process Flow: Literature search → Experimental design (4 prompt conditions × 4 LLMs) → API-based data collection (1068 abstracts × 16 model-condition combinations) → Multi-metric statistical analysis.

3.1. Literature Search Strategy

A systematic literature search was conducted using Scopus to identify theoretical frameworks in information behavior research and empirically ground the experimental design. The search strategy employed a multi-faceted approach targeting peer-reviewed articles published between 2021 and 2024 to capture contemporary theoretical discourse and ensure recent relevance. The search query was:

(TITLE(“information behavior” OR “information behaviour” OR “information use behavior” OR “information seeking behavior” OR “information sharing behavior” OR “information needs”)) AND DOCTYPE(ar) AND LANGUAGE(English) AND PUBYEAR > 2020 AND PUBYEAR < 2025 AND NOT (TITLE(“algorithm” OR “machine learning” OR “artificial intelligence” OR “system architecture” OR “database design” OR “network protocol”) OR AUTHKEY(“algorithm” OR “machine learning” OR “artificial intelligence” OR “system architecture” OR “database design” OR “network protocol”)).

This search yielded 1068 abstracts that met the inclusion criteria and served as experimental stimuli. The abstracts contained references to information behavior concepts and provided authentic academic discourse for evaluating LLM outputs across different prompt conditions.

3.2. Experimental Conditions

Table 1 presents the four prompt conditions used in this study. Each condition introduces a distinct form of conceptual guidance that functions as an anchor. The conditions vary in specificity, ranging from bare dimension labels to explicit theoretical references: dimension-only labels, surface-level keywords, elaborated definitions, and theory-based references. The table clarifies how each condition structures the prompt and frames the interpretive task expected of the model.

Table 1. Overview of Prompt Conditions and Anchoring Types.

Condition ID	Condition Name	Anchoring Type	Description
no_examples	No Examples (Baseline)	None	Uses only the four dimension names without any examples, definitions, or theoretical cues.
brief_keywords	Brief Keywords	Lexical Cues	Provides two representative keywords per dimension to test surface-level lexical anchoring.
detailed_explanations	Detailed Explanations	Conceptual Scaffolding	Offers full conceptual definitions of each dimension to guide semantic understanding.
author_based_examples	Author-Based Examples	Theoretical References	Cites canonical scholars and theories associated with each dimension for theoretical anchoring.

Only the Author-Based Examples condition incorporates references to specific scholars, thereby embedding disciplinary context, whereas the Brief Keywords condition relies solely on lexical cues. Across conditions, conceptual depth increases progressively from the baseline configuration to the most theory-driven format. By detailing both structural features and the type of prompt guidance each condition provides, the table facilitates systematic comparison across prompt types.

Table 2 presents the prompt that contains only the four dimension names—Information Seeking, Information Use, Information Sharing, and Information Needs—without any accompanying descriptions, examples, or contextual information. This condition requires the model to rely solely on its internalized knowledge of the categories to perform keyword classification, without anchoring cues. This minimal configuration functions as a neutral reference point, isolating the model’s unprompted behavior and helping to establish the baseline accuracy, coverage, and category overlap in generated terms. It is especially useful for detecting how the model allocates terms across closely related dimensions by default.

Table 2. Prompt Structure for No Examples Condition (Baseline).

Extract keywords from this abstract and categorize them into these four information behavior dimensions: - Information Seeking - Information Use - Information Sharing - Information Needs Abstract: [TARGET ABSTRACT] Extract up to 5 relevant keywords for each dimension if applicable. If a dimension is not relevant to the abstract, leave that dimension empty. Return as JSON: {“Information Seeking”: [“keyword1”, “keyword2”], “Information Use”: [“keyword3”], “Information Sharing”: [“keyword4”], “Information Needs”: [“keyword5”]}

Table 3 shows the prompt where each dimension is accompanied by exactly two keywords, such as “search strategies” for Information Seeking and “gap analysis” for Information Needs. These examples act as lexical anchors designed to cue the model toward specific terminology. Unlike the baseline, this configuration introduces targeted lexical guidance without offering definitions or theoretical framing. It permits assessment of whether minimal keyword cues are sufficient to guide the model’s classification or whether they result in overly literal or repetitive outputs. This condition allows examination of prompt sensitivity at the surface linguistic level.

Table 3. Prompt Structure for Brief Keywords Condition (Lexical Anchoring).

Extract keywords from this abstract and categorize them into these four information behavior dimensions: - Information Seeking: search strategies, information foraging - Information Use: information evaluation, sense-making - Information Sharing: knowledge dissemination, collaborative filtering - Information Needs: needs assessment, gap analysis Abstract: [TARGET ABSTRACT] Extract up to 5 relevant keywords for each dimension if applicable. If a dimension is not relevant to the abstract, leave that dimension empty. Return as JSON: {“Information Seeking”: [“keyword1”, “keyword2”], “Information Use”: [“keyword3”], “Information Sharing”: [“keyword4”], “Information Needs”: [“keyword5”]}

Table 4 provides conceptual descriptions that emphasize cognitive and behavioral processes such as browsing, sense-making, and credibility assessment. These definitions are intended to expand the semantic boundaries of each category, enabling the model to generate keywords based on broader conceptual features rather than lexical similarity alone. This condition allows examination of whether conceptual scaffolding produces more precise dimensional alignment, greater terminological variation, or more nuanced inferential behavior. It is particularly useful for detecting shifts in abstraction level across model responses.

Table 4. Prompt Structure for Detailed Explanations Condition (Conceptual Anchoring).

Extract keywords from this abstract and categorize them into these four information behavior dimensions: - Information Seeking: How people actively search for and discover information, including search strategies, browsing patterns, query formulation, and information foraging behaviors in various contexts - Information Use: How people evaluate, interpret, and apply discovered information, including credibility assessment, sense-making processes, information integration, and decision-making based on information - Information Sharing: How people distribute and communicate information to others, including knowledge dissemination, collaborative filtering, social sharing behaviors, and information exchange in communities - Information Needs: How people identify and articulate their information requirements, including needs assessment, gap analysis, problem definition, and requirements specification processes Abstract: [TARGET ABSTRACT] Extract up to 5 relevant keywords for each dimension if applicable. If a dimension is not relevant to the abstract, leave that dimension empty. Return as JSON: {“Information Seeking”: [“keyword1”, “keyword2”], “Information Use”: [“keyword3”], “Information Sharing”: [“keyword4”], “Information Needs”: [“keyword5”]}

Table 5 presents well-established theories and scholars across all dimensions, including Wilson’s information behavior models (Wilson, 1999), Bates’ berrypicking strategies (Bates, 1989), Ellis’ information seeking patterns (Ellis, 1989), Marchionini’s exploratory search (Marchionini, 2006), Dervin’s sense-making methodology (Dervin, 1983), Kuhlthau’s information search process (Kuhlthau, 1991), Taylor’s value-added processes (Taylor, 1986), and question-negotiation (Taylor, 1968), Chatman’s small world theory (Chatman, 1996), and Belkin’s anomalous state of knowledge (Belkin, 1980), among others. By grounding each dimension in specific intellectual traditions, this prompt introduces a disciplinary interpretive frame that may shift the model’s keyword selection toward theory-aligned concepts or terminology. This condition enables evaluation of whether theoretical anchoring enhances the model’s generation of framework-aligned terms. It also provides a way to assess whether citation-based prompts alter observable categorization patterns in model outputs.

Table 5. Prompt Structure for Author-Based Examples Condition (Theoretical Anchoring).

Extract keywords from this abstract and categorize them into these four information behavior dimensions: - Information Seeking: Wilson’s information behavior models, Bates’ berrypicking strategies, Ellis’ information seeking patterns, Marchionini’s exploratory search - Information Use: Dervin’s sense-making methodology, Kuhlthau’s information search process, Taylor’s value-added processes, Belkin’s cognitive approach - Information Sharing: Chatman’s small world theory, Granovetter’s weak ties concept, Wenger’s communities of practice, Haythornthwaite’s social networks - Information Needs: Belkin’s anomalous state of knowledge, Taylor’s question-negotiation, Case’s information behavior framework, Wilson’s problem-solving model Abstract: [TARGET ABSTRACT] Extract up to 5 relevant keywords for each dimension if applicable. If a dimension is not relevant to the abstract, leave that dimension empty. Return as JSON: {“Information Seeking”: [“keyword1”, “keyword2”], “Information Use”: [“keyword3”], “Information Sharing”: [“keyword4”], “Information Needs”: [“keyword5”]}

3.3. Data Collection (API)

The experiment evaluated four prompt conditions across four LLMs: GPT-4o-mini, Claude-3-haiku, Gemini-2.0-flash-lite, and DeepSeek V3. These models were selected because lightweight models are widely used in automated content analysis, where cost, speed, and API accessibility constrain model choice, and because they span multiple providers and training approaches. Whether the prompt sensitivity patterns observed here generalize to frontier-scale models remains an open question and represents an important direction for future work. API configuration parameters are summarized in Table 6. Each of the 1068 abstracts was processed under every model and prompt condition, yielding 16 model-condition combinations per abstract. This produced 17,036 valid observations from 17,088 possible model-condition-abstract combinations. Fifty-two responses were excluded due to API errors or malformed JSON output. All valid responses were checked for structural completeness across the four information behavior dimensions and were subsequently processed to derive keyword counts, dimensional coverage, specificity, diversity, and entropy measures for statistical analysis.

Table 6. API Configuration Parameters for Each Model.

Model	API Model String	Provider	Temperature	Max Tokens
GPT-4o-mini	gpt-4o-mini-2024-07-18	OpenAI (San Francisco, CA, USA)	0.2	800
Claude-3-haiku	claude-3-haiku-20240307	Anthropic (San Francisco, CA, USA)	0.2	800
Gemini-2.0-flash-lite	gemini-2.0-flash-lite	Google (Mountain View, CA, USA)	0.2	800
DeepSeek V3	deepseek-chat	DeepSeek (Hangzhou, Zhejiang, China)	0.2	800

3.4. Data Analysis

This study used a multi-metric analytical framework to evaluate both quantitative anchoring effects and qualitative semantic changes in LLM keyword generation behavior. First, density distribution analysis examined the shape, spread, and modality of keyword generation patterns to reveal underlying distributional differences between models and anchoring conditions, identifying whether responses clustered tightly or were distributed more evenly.

Second, Shannon entropy calculations quantified response balance across the four information behavior dimensions, measuring distributional evenness where higher entropy indicates more even distribution across dimensions and lower values reveal dimensional specialization or anchoring-induced bias.

Third, keyword similarity was assessed using two complementary measures to capture both lexical and semantic dimensions of overlap. Jaccard similarity was computed to quantify exact keyword overlap, directly relevant to detecting anchoring-induced duplication. The Jaccard index measures the proportion of shared keywords:

J(A,B) = |A ∩ B|/|A ∪ B|

where |A ∩ B| represents shared keywords and |A ∪ B| represents total unique keywords across both sets. Because the similarity scores were very low (0.01–0.05), Min-Max normalization was applied to render small differences visible in the visualization:

J_norm = (J − J_min)/(J_max − J_min)

This rescaling helped highlight the relative differences between model pairs while maintaining perfect self-similarity at 1.0. To complement Jaccard similarity, semantic cosine similarity was computed using sentence embeddings generated by the all-MiniLM-L6-v2 model (based on the Sentence-BERT architecture (Reimers & Gurevych, 2019)) via the sentence-transformers library. This embedding model is widely used for sentence-level semantic similarity tasks because it provides efficient and robust semantic representations. For each model-condition pair, all unique keywords were concatenated into a single string and encoded into a dense vector representation. Cosine similarity was then calculated between all pairwise combinations, capturing conceptual proximity beyond surface-level lexical matching. Both similarity measures were normalized to a [0, 1] scale using Min-Max normalization and visualized together in a divergence scatter plot, enabling direct comparison of lexical and semantic convergence across model-condition pairs.

Fourth, dimensional diversity analysis assessed the variety of unique keywords generated within each information behavior category, revealing whether anchoring effects created broad or narrow semantic representations within specific dimensions.

In addition, statistical analysis employed both parametric and non-parametric approaches to test for significant differences and relationships:

One-way ANOVA and Kruskal–Wallis tests—Detected overall group differences in keyword generation across conditions and models, with non-parametric confirmation due to heteroscedasticity

Chi-square tests—Examined associations between experimental conditions/models and dimensional presence patterns to identify consistent biases

Bartlett tests—Assessed homogeneity of variance assumptions, revealing significant heteroscedasticity that warranted robust non-parametric approaches

Post hoc t-tests with Bonferroni correction—Identified specific between-group contrasts while controlling Type I error inflation across multiple comparisons

Effect size calculations—Cohen’s d for pairwise comparisons and eta-squared (η²) for ANOVA results to quantify practical significance beyond statistical significance

Finally, qualitative analysis of keyword characteristics was conducted to examine semantic progression patterns and theoretical reproduction across anchoring conditions. Focusing on the Information Seeking dimension as the most dominant category, this analysis provided concrete evidence of how prompts systematically influence vocabulary selection, shifting from descriptive terminology to direct reproduction of theory-attributed references.

4. Results

4.1. Keyword Generation by Anchoring Condition

Figure 2 compares keyword generation patterns across models and prompting conditions, identifying which conditions and models yield the most comprehensive keyword generation. The figure reports mean keyword counts by model and condition on the left, and by model and information behavior dimension on the right. The left heatmap shows consistent output differences across models. DeepSeek V3 produces the highest average keyword counts across all conditions, ranging from 13.8 to 14.7. GPT-4o-mini follows at 9.4 to 12.4, then Gemini-2.0-flash-lite at 7.3 to 10.3, and Claude-3-haiku at 5.1 to 10.3. The Detailed Explanations condition increases keyword output for most models. It enables DeepSeek V3 to reach its maximum of 14.7 keywords and allows Claude-3-haiku to attain its highest value of 10.3. These patterns indicate that detailed conceptual examples provide effective guidance for keyword generation, although the magnitude of improvement differs by model.

Figure 2. LLM Keyword Generation Across Anchoring Conditions and Information Dimensions.

The right heatmap reveals a consistent imbalance across the information behavior dimensions. Information Sharing receives the fewest keywords at 0.7 to 1.4, whereas Information Needs and Information Seeking receive higher counts at 2.3 to 4.1. This pattern indicates consistent underrepresentation of sharing-related content. The evaluated LLMs generated relatively more keywords related to individual information needs and search behavior than to information sharing. This imbalance persists across all prompting conditions. These findings suggest that structural tendencies in model outputs may contribute more to these disparities than prompt design alone. These results have implications for the design of LLM-based tools in information science applications that require balanced coverage across behavioral dimensions.

4.2. Category Dominance Patterns

Category distribution analysis across the four theoretical dimensions reveals observable variations in categorical preferences that may affect effectiveness in information behavior research. Figure 3 shows distinct patterns of categorical dominance across LLMs. Information Seeking emerges as the dominant dimension for three of four models (GPT-4o-mini, DeepSeek V3, and Gemini-2.0-flash-lite) under the majority of prompt conditions tested. In contrast, Information Needs dominance is observed primarily in Claude-3-haiku, where this dimension consistently prevails across all prompt conditions. This divergence suggests observable differences in how these models prioritize information behavior categories, potentially reflecting differences among model architectures or training procedures that cannot be isolated in the present design.

Figure 3. Dominance Patterns of Information Behavior Category by Model and Condition.

The predominance of Information Seeking across most models indicates a consistent bias toward search and discovery behaviors over need identification processes. However, Claude-3-haiku demonstrates a contrasting pattern, showing consistent Information Needs dominance across conditions. Information Sharing and Information Use remain consistently underrepresented across all models and prompt conditions, appearing as minimal segments in the dominance distribution. These patterns suggest that although anchoring effects are present across models, baseline categorical preferences vary significantly between models, with most favoring seeking behaviors whereas Claude-3-haiku uniquely emphasizes needs assessment.

4.3. Response Consistency and Variability

Response stability analysis examined whether models consistently produce balanced outputs across different prompt conditions. Figure 4 presents the effects of prompt conditions on dimensional distribution using Shannon entropy. Claude-3-haiku demonstrates the highest entropy values (1.4–1.6 bits), particularly under the “No Examples” and “Detailed Explanations” conditions, indicating a more even distribution across the four information dimensions. This suggests that Claude-3-haiku exhibits greater lexical diversity and shows less sensitivity to prompt variation. In contrast, DeepSeek V3, Gemini-2.0-flash-lite, and GPT-4o-mini display lower and more variable entropy values (0.8–1.5 bits), reflecting a stronger tendency to concentrate keyword output within specific dimensions. Together, these observations indicate that prompt conditions exert varying degrees of influence across models, with Claude-3-haiku showing comparatively higher resistance to prompt-induced dimensional imbalances.

Figure 4. Response Balance Measured by Shannon Entropy.

A notable pattern emerges under the “Author-Based Examples” condition, where all models except Claude-3-haiku show reduced entropy, suggesting that highly specific examples create stronger anchoring effects that concentrate responses in fewer dimensions. The “No Examples” baseline consistently produces higher entropy across most models, consistent with the expectation that example-free conditions allow for more natural, balanced dimensional coverage. This entropy analysis reveals that anchoring effects not only shift which dimensions dominate (as shown in Figure 2) but also alter the balance of attention across all information behavior categories, with some models maintaining more robust dimensional diversity than others.

4.4. Keyword Distribution and Similarity

Figure 5 presents keyword distribution densities, revealing distinct response-variability patterns across LLMs and anchoring conditions that reflect clear differences in model consistency and sensitivity to anchoring effects. Claude-3-haiku demonstrates remarkably tight distributions with pronounced peaks centered around 7–8 keywords across most conditions, indicating highly consistent but potentially constrained generation with minimal variability between trials. By contrast, DeepSeek V3 exhibits broader distributions centered around 13–15 keywords, suggesting higher overall output volume but with greater response variability. This pattern indicates that although DeepSeek V3 generates more comprehensive responses on average, its broader response distributions also show it to be less predictable across individual trials.

Figure 5. Distribution of Total Keywords by Condition and Model.

The multi-modal and broader distributions observed in Gemini-2.0-flash-lite and GPT-4o-mini reveal more complex anchoring dynamics, with multiple peaks, consistent with different response patterns depending on subtle prompt variations. The condition-specific patterns show that the No Examples condition often produces the most consistent distributions, whereas the Author-Based Examples condition frequently generates the most variable responses across models. This variability pattern supports the anchoring hypothesis by demonstrating that specific examples not only shift mean output volume but also alter the consistency and predictability of LLM responses, with some models maintaining consistency whereas others become more erratic under strong anchoring conditions.

Figure 6 presents a divergence scatter plot comparing normalized Jaccard similarity (lexical overlap) against normalized semantic cosine similarity (sentence embeddings) across all model-condition pairs. The majority of pairs cluster in the upper-left quadrant, indicating high semantic cosine similarity but low Jaccard similarity. This pattern reveals that models consistently converge on the same conceptual territory while using different surface vocabulary, demonstrating that anchoring effects operate primarily at the conceptual level rather than the lexical level. A small number of pairs fall below the diagonal, exhibiting higher Jaccard than cosine similarity, suggesting occasional surface-level duplication without deeper conceptual alignment. Two outlier observations in the far lower-right represent pairs with high lexical overlap but low semantic similarity, consistent with generic or boilerplate keyword reproduction rather than genuine conceptual convergence. Point shapes indicate the four models, revealing that DeepSeek V3 and Claude-3-haiku contribute more frequently to the high-cosine cluster, whereas GPT-4o-mini and Gemini-2.0-flash-lite show greater dispersion across divergence types.

Figure 6. Divergence Between Lexical (Jaccard) and Semantic (Cosine) Similarity Across LLM-Condition Pairs. Note. Similarity scores normalized to [0, 1] scale. Semantic embeddings computed via all-MiniLM-L6-v2.

The divergence pattern is consistent across all four models, confirming that anchoring effects operate at the conceptual rather than lexical level regardless of model architecture. This distinction has methodological implications: relying solely on lexical overlap metrics such as Jaccard similarity would substantially underestimate the degree of convergence among model outputs, underscoring the value of complementary semantic similarity measures in evaluating LLM keyword generation.

4.5. Keyword Diversity Analysis

Figure 7 presents keyword diversity across the four information behavior dimensions for each LLM (Claude-3-haiku, DeepSeek V3, Gemini-2.0-flash-lite, GPT-4o-mini) under four prompt conditions. The metric measures the average number of unique keywords generated per dimension, aggregated across successful runs. Claude-3-haiku and DeepSeek V3 show consistently high and stable diversity, often exceeding 0.85 in the dimensions of Information Seeking and Information Needs. In contrast, Gemini-2.0-flash-lite and GPT-4o-mini exhibit greater variability, with notable declines in diversity under the Author-Based Examples condition. For instance, Gemini-2.0-flash-lite’s diversity in Information Use drops from approximately 0.80 in the No Examples condition to below 0.55 with theoretical prompts. GPT-4o-mini’s diversity in Information Sharing falls from around 0.75 to slightly above 0.50, suggesting that dense theoretical prompts may narrow lexical variety in dimensions with weaker terminological foundations.

Figure 7. Keyword Diversity by Model, Condition, and Information Dimension.

Across all models, the dimensions of Information Seeking and Information Needs consistently support higher lexical diversity than Information Sharing and Information Use. This pattern suggests that the models produced more diverse terminology in cognitively oriented behaviors than in those involving collaboration or evaluation. GPT-4o-mini shows the lowest diversity in Information Sharing, particularly under author-based prompts, indicating reduced lexical diversity for socially framed concepts when guided by established theoretical references. Claude-3-haiku maintains more consistent diversity across all dimensions, with values often near 0.90, suggesting more balanced response patterns across information behavior categories. DeepSeek V3 performs similarly, with a modest decline in Information Sharing under theoretical prompts but still exceeding 0.70, outperforming Gemini-2.0-flash-lite and GPT-4o-mini in that condition. These results emphasize both the sensitivity of LLM outputs to prompt structure and model-specific differences in representational flexibility across conceptual categories.

4.6. Statistical Validation

Table 7 summarizes the statistical comparisons of keyword generation across prompt conditions and LLMs. Overall group differences were assessed using One-way ANOVA, Kruskal–Wallis, and Chi-square tests, followed by Bonferroni-adjusted pairwise comparisons. Because each abstract was evaluated under every model and prompt condition, the results should be interpreted as descriptive rather than confirmatory.

Table 7. Omnibus Tests and Homogeneity Checks.

Comparison	Test	N	Test Statistic	p-Value	Effect Size
Condition × Total Keywords	One-way ANOVA	17,036	419.59	<0.0001 ***	η2 = 0.069
Model × Total Keywords	One-way ANOVA	17,036	2542.94	<0.0001 ***	η2 = 0.309
Condition × Total Keywords	Kruskal–Wallis	17,036	1248.62	<0.0001 ***	NA
Model × Total Keywords	Kruskal–Wallis	17,036	5141.33	<0.0001 ***	NA
Condition Homogeneity of Variance	Bartlett Test	17,036	67.48	<0.0001 ***	NA
Model Homogeneity of Variance	Bartlett Test	17,036	439.20	<0.0001 ***	NA
Seeking Count Presence × Condition	Chi-Square Test	17,036	657.08	<0.0001 ***	NA
Seeking Count Presence × Model	Chi-Square Test	17,036	960.05	<0.0001 ***	NA
Use Count Presence × Condition	Chi-Square Test	17,036	917.07	<0.0001 ***	NA
Use Count Presence × Model	Chi-Square Test	17,036	1174.37	<0.0001 ***	NA
Sharing Count Presence × Condition	Chi-Square Test	17,036	157.39	<0.0001 ***	NA
Sharing Count Presence × Model	Chi-Square Test	17,036	853.47	<0.0001 ***	NA
Needs Count Presence × Condition	Chi-Square Test	17,036	206.59	<0.0001 ***	NA
Needs Count Presence × Model	Chi-Square Test	17,036	750.63	<0.0001 ***	NA
Dimensional Coverage × Condition	Chi-Square Test	17,036	1173.89	<0.0001 ***	NA
Dimensional Coverage × Model	Chi-Square Test	17,036	2650.23	<0.0001 ***	NA

Note. NA = not applicable. *** p < 0.001. Non-parametric tests do not produce effect sizes directly comparable to η². Pairwise effect sizes (Cohen’s d) are reported in Table 8.

Across all outcome variables, statistically significant differences were observed (p < 0.0001). Bartlett tests indicated significant heteroscedasticity (p < 0.0001); therefore, the Kruskal–Wallis results are emphasized. Model type exhibited substantially stronger effects than prompt condition (H(3) = 5141.33 vs. H(3) = 1248.62), indicating that differences among LLMs contributed more to keyword generation than differences in prompting strategy. ANOVA results showed the same pattern (F = 2542.94 for Model; F = 419.59 for Condition), although effect sizes (η² = 0.309 and η² = 0.069) should be interpreted with particular caution given the repeated-measures design and significant heteroscedasticity. They are presented primarily as descriptive indicators of relative magnitude rather than as precise estimates of variance explained.

The Chi-square tests further demonstrated that both model and prompt condition were significantly associated with the categorical distribution of keywords. All dimensional presence variables (Seeking, Use, Sharing, and Needs) showed statistically significant associations with both factors. The Seeking and Use dimensions yielded the largest Chi-square values for model comparisons (χ² = 960.05 and χ² = 1174.37, respectively), indicating that model differences were most pronounced for these dimensions. Dimensional Coverage also exhibited a large association with model type (χ² = 2650.23), suggesting substantial variation in the comprehensiveness with which different models covered the four information behavior dimensions. Overall, model type was more strongly associated with keyword generation than prompt condition, particularly in total keyword output and dimensional emphasis. The consistency of the parametric and non-parametric analyses supports this overall pattern.

Pairwise post hoc comparisons were also performed for prompt conditions using t-tests with Bonferroni correction (Table 8). All condition contrasts involving the No Examples baseline are statistically significant with p < 0.0001 except the comparison between Author-Based Examples and Brief Keywords, which shows no meaningful difference with Cohen’s d = 0.012. The largest contrast is observed between Detailed Explanations and Brief Keywords with t = 30.30 and d = 0.657, suggesting that conceptual anchoring is associated with higher keyword output than lexical anchoring. Detailed Explanations also show higher output than No Examples with a small to moderate effect size of d = 0.253. In contrast, both Brief Keywords and Author-Based Examples are associated with moderate negative effects relative to the baseline with d = −0.409 and d = −0.380, suggesting that conceptual guidance is more consistently associated with generation breadth than either lexical or theoretical anchoring.

Table 8. Pairwise Post Hoc Comparisons.

Category	Comparison	N	t	p-Value	Cohen’s d
Condition Contrasts	Brief_keywords vs. No_examples	8517	−18.89	<0.0001 ***	−0.409
	Detailed_explanations vs. No_examples	8519	11.68	<0.0001 ***	0.253
	Author_examples vs. No_examples	8518	−17.52	<0.0001 ***	−0.380
	Detailed_explanations vs. Brief_keywords	8518	30.30	<0.0001 ***	0.657
	Author_examples vs. Brief_keywords	8517	0.54	=1.000	0.012
	Author_examples vs. Detailed_explanations	8519	−28.46	<0.0001 ***	−0.617
Model contrasts	DeepSeek V3 vs. Claude-3-haiku	8492	69.90	<0.0001 ***	1.516
	Gemini-2.0-flash-lite vs. Claude-3-haiku	8493	−2.73	=0.0406 *	−0.059
	GPT-4o-mini vs. Claude-3-haiku	8493	39.25	<0.0001 ***	0.852
	Gemini-2.0-flash-lite vs. DeepSeek V3	8543	−69.33	<0.0001 ***	−1.500
	GPT-4o-mini vs. DeepSeek V3	8543	−37.89	<0.0001 ***	−0.820
	GPT-4o-mini vs. Gemini-2.0-flash-lite	8544	39.81	<0.0001 ***	0.861

Note. * p < 0.05; *** p < 0.001.

Pairwise post hoc comparisons were also performed for model-level differences (Table 8). DeepSeek V3 compared with Claude-3-haiku shows the largest effect with d = 1.516. GPT-4o-mini also shows substantially higher output than Claude-3-haiku with d = 0.852 and Gemini-2.0-flash-lite with d = 0.861. Gemini-2.0-flash-lite shows lower output than DeepSeek V3 with d = −1.500 and GPT-4o-mini with d = −0.820 but marginally higher output than Claude-3-haiku with d = −0.059. These findings indicate that model type accounts for large differences in keyword output. The asymmetric pattern of effect sizes suggests nonlinear scaling that may reflect differences in how models are optimized for semantic generation tasks.

4.7. Qualitative Keyword Analysis: Anchoring Effects on Information Seeking Dimension

To examine how anchoring affects lexical and theoretical alignment, keywords were aggregated across all models and abstracts for each condition, then ranked by frequency to identify the top 10 terms for the Information Seeking dimension, selected as the most consistently dominant category. Table 9 reveals consistent anchoring effects on keyword reproduction and theoretical alignment across the four prompt conditions. The analysis demonstrates a clear methodological progression: from basic descriptive terminology under minimal prompting to direct reproduction of theoretical references under structured guidance. In the “No Examples” baseline condition, models generated predominantly generic terms such as “information sources” and “information-seeking behavior,” reflecting default categorical understanding without external cues. The intermediate conditions—“Brief Keywords” and “Detailed Explanations”—produced process-oriented terminology including “search strategies,” “information foraging,” and “query formulation,” indicating that moderate anchoring successfully shifts models toward more behaviorally specific vocabulary.

Table 9. Top 10 Information Seeking Keywords by Prompt Anchoring Condition.

Rank	No Examples	Brief Keywords	Detailed Explanations	Author-Based Examples
1	information sources	search strategies	search strategies	Bates’ berrypicking strategies
2	information needs	information foraging	browsing patterns	Wilson’s information behavior models
3	information-seeking behavior	information sources	information foraging	Marchionini’s exploratory search
4	social media	information-seeking behavior	query formulation	Ellis’ information seeking patterns
5	information seeking	information seeking	information foraging behaviors	exploratory search
6	internet	information-seeking behavior	information sources	information-seeking behavior
7	information-seeking behavior	social media	information-seeking behavior	information sources
8	health information	information seeking behavior	social media	information seeking
9	information seeking behavior	information-seeking behaviors	information seeking	information-seeking behavior
10	information-seeking behaviors	internet	information seeking behavior	information seeking behavior

Most notably, the “Author-Based Examples” condition yielded qualitatively distinct results, reproducing scholar-attributed concepts such as “Bates’ berrypicking strategies,” “Wilson’s information behavior models,” and “Ellis’ information seeking patterns”—terms that directly correspond to the theoretical anchors provided in the prompt (Table 5). Notable contextual differences emerge across conditions: technology-specific terms like social media appear in the first three conditions but are absent from Author-Based Examples, whereas domain-specific applications such as health information appear only in the unanchored condition. Despite substantial differences in theoretical attribution, core terminology such as information-seeking behavior variants appears across all conditions at different rank positions, suggesting that foundational concepts remain stable whereas peripheral vocabulary shifts according to anchoring intensity, demonstrating that keyword generation is systematically influenced by prompt content through direct reproduction mechanisms.

5. Discussion

5.1. Model Effects and Prompt Effects

The findings indicate that both model identity and prompt anchoring are associated with differences in LLM keyword generation, although model type exerts a substantially stronger influence. Model type accounted for approximately 30.9% of the variance in keyword volume (η² = 0.309), compared with 6.9% for prompt condition (η² = 0.069), although these effect sizes should be interpreted cautiously given the dependency structure of the data. This pattern is consistent with the expectation that baseline generation behavior is primarily determined by model characteristics, whereas prompts primarily influence conceptual emphasis rather than overall output volume. Consequently, model selection is the primary determinant of output quantity, whereas prompt anchoring shapes the conceptual configuration of generated keywords. The observed shift from generic terms such as “information sources” under the No Examples condition to theory-oriented concepts such as “Bates’ berrypicking strategies” under the Author-Based Examples condition illustrates how prompt anchoring can alter conceptual focus without substantially changing overall output volume.

The analyses also revealed consistent dimensional patterns across models and prompt conditions. Information Sharing consistently produced the fewest keywords, whereas Information Seeking dominated in three of the four models. These recurring patterns are consistent with the possibility that dimensional preferences reflect characteristics of model training, although this study cannot determine the underlying causes. Model-specific differences were nevertheless apparent. DeepSeek V3 generally produced larger keyword sets, whereas Claude-3-haiku exhibited greater Shannon entropy, suggesting broader distribution of keywords across dimensions. Because the models differ in architecture, provider, training data, alignment procedures, and model size, these differences should not be attributed to any single technical characteristic.

The similarity analyses further demonstrated a consistent pattern across all four models. Lexical overlap, measured using the Jaccard index, remained relatively low, whereas semantic cosine similarity was substantially higher. This finding suggests that different models frequently converge on similar conceptual interpretations despite employing different vocabulary. The Author-Based Examples condition increased semantic convergence across models and substantially improved representation of the Information Sharing dimension, although this occurred alongside reduced overall dimensional breadth. Collectively, theoretical prompting appears to strengthen coverage of underrepresented concepts, although potentially at the expense of broader conceptual diversity.

5.2. Implications for LLM-Assisted Content Analysis

The findings have several practical implications for researchers using LLMs in automated content analysis. When output quantity is the primary concern, model selection appears more consequential than prompt anchoring. Conversely, when theoretical alignment or balanced dimensional coverage is the objective, prompt anchoring becomes the more influential methodological decision. Detailed Explanations were associated with greater keyword generation and broader dimensional coverage, making this condition well suited for comprehensive thematic exploration. In contrast, Author-Based Examples produced greater theoretical specificity and stronger representation of underrepresented dimensions, making it more appropriate when targeted dimensional coverage is the priority. It should also be noted that the Author-Based Examples condition cites scholars and theories that are almost certainly densely represented in model training data. The improved dimensional coverage observed under this condition therefore reflects a confound between the conceptual guidance provided by the prompt and the model’s pre-existing parametric knowledge activated by those names. Because LLMs draw simultaneously on parametric knowledge encoded during training and contextual knowledge provided in the prompt, and because these two sources interact in ways that are difficult to isolate (Augenstein, 2026), it cannot be determined from the present data whether the observed effects reflect prompt-induced anchoring, retrieval of memorized associations, or some combination of both. Future research employing counterfactual prompts with invented theoretical frameworks could help disentangle these mechanisms. No single prompting strategy proved universally preferable, and the appropriate choice depends on the analytical objective.

These recommendations should nevertheless be interpreted cautiously. The present analyses evaluate structural and distributional properties of generated keywords rather than their substantive quality. The study therefore cannot determine whether any prompt condition produces keywords that are more accurate, relevant, complete, or useful for a particular content analysis task. Even so, the results demonstrate that prompt anchoring constitutes a meaningful methodological choice rather than a minor implementation detail. Prompt specifications should therefore be reported with the same transparency as other analytical parameters in LLM-assisted content analysis.

5.3. Implications for Scientometric Applications

The findings also suggest potential implications for scientometric applications, although these remain provisional pending downstream validation. LLM-generated keywords increasingly serve as inputs for thematic mapping, topic clustering, and automated analysis of scholarly corpora. The present results indicate that prompt anchoring may systematically influence which conceptual dimensions are represented before subsequent analytical stages begin. If these effects extend to downstream bibliometric analyses, prompt specification may represent an additional source of methodological variation analogous to database selection or search query formulation. Whether these differences meaningfully alter thematic structures, clustering outcomes, or bibliometric interpretations remains an important question for future empirical investigation.

6. Limitations

This study is exploratory and descriptive in design. It documents consistent patterns in LLM output under different prompt conditions but does not determine whether any condition produces objectively superior keywords, as keyword generation is inherently interpretive and lacks a single ground truth. The findings should therefore be viewed as empirical reference points rather than definitive performance benchmarks. Several methodological limitations should also be considered.

First, the corpus consists exclusively of information behavior abstracts, limiting the generalizability of the findings to other disciplines, document types, or content analysis tasks.

Second, only four LLMs were evaluated using fixed parameter settings. Different models, parameter configurations, or future generations of LLMs may exhibit different prompt sensitivity profiles.

Third, the evaluation relied on structural and distributional metrics rather than external validation. Comparison with author keywords or expert human judgments would provide a valuable benchmark for assessing keyword quality.

Fourth, the prompt conditions differed simultaneously in specificity, length, theoretical content, and semantic guidance. Consequently, the study cannot isolate which prompt characteristics produced the observed differences; a fully factorial design varying these elements independently would be needed to disentangle their contributions. Likewise, the design examined content-based guidance rather than position-based anchoring, leaving order effects unexplored (Tang et al., 2026).

Fifth, the statistical analyses treated observations as independent even though each abstract was evaluated under every model and prompt condition. The reported analyses should therefore be interpreted as descriptive rather than fully confirmatory. Mixed-effects models or generalized estimating equations would provide a more appropriate analytical framework in future research.

Sixth, the study assumes that Wilson’s four information behavior dimensions are conceptually separable and consistently interpretable across all abstracts. In practice, these categories overlap substantially, and a single abstract may involve behaviors that span multiple dimensions simultaneously (Case & Given, 2016). This conceptual ambiguity in the framework itself may affect how keywords are assigned to dimensions independently of prompt design, and this study does not assess construct validity for the four-category structure.

Seventh, all models were evaluated at a single temperature (0.2), and no test–retest reliability assessment was conducted. Whether anchoring effects diminish or amplify at higher temperatures remains an open empirical question. Temperature affects response randomness and may interact with anchoring effects (Messina & Scotta, 2026), and the fixed temperature setting limits generalizability to other decoding configurations.

Finally, the study did not assess the extent to which generated keywords reproduced anchor terms directly from the prompts, nor did it examine the temporal stability of the observed patterns across repeated or longitudinal analyses. Both issues warrant investigation in future research.

7. Conclusions

This study provides evidence that prompt anchoring systematically influences LLM keyword generation by shaping conceptual emphasis rather than simply increasing output volume. Although model type accounted for substantially more variance in keyword volume than prompt condition (30.9% vs. 6.9%), prompt anchoring consistently affected the conceptual configuration of generated keywords. Author-Based Examples substantially increased representation of the Information Sharing dimension, whereas Detailed Explanations produced the highest keyword output and the broadest dimensional coverage. These findings suggest that different anchoring strategies involve distinct trade-offs rather than a single optimal approach. Model selection and prompt anchoring therefore serve complementary methodological functions: the former primarily influences output volume, whereas the latter influences conceptual emphasis and theoretical orientation. More broadly, the findings suggest that prompt anchoring should be recognized as a potential source of methodological variation in LLM-assisted content analysis, at least within the domain of information behavior research. Studies employing different anchoring strategies may produce systematically different conceptual representations even when analyzing the same underlying texts.