1. Introduction
Artificial Intelligence (AI) is poised to substantially increase productivity across virtually all sectors of the global economy, affecting professions ranging from medicine and law to engineering and economics (
Brynjolfsson et al., 2025;
Noy & Zhang, 2023). In high-stakes industries such as healthcare, which constitutes nearly 18 percent of U.S. GDP (
Dumont et al., 2015), AI systems are increasingly embedded in diagnostic, screening, and treatment workflows (
Chen et al., 2025;
Flora & Maniago, 2025;
Starke, 2025). While these technologies promise efficiency gains and cost reductions, they also introduce material operational risk because stochastic model errors can generate severe downstream consequences. However, as AI becomes increasingly integrated into professional workflows, more information and guidance is needed to ensure that individuals optimally leverage AI when cognitive resources are scarce. Existing studies provide valuable qualitative guidance on prompt engineering (
Dell’Acqua et al., 2026;
Jahani et al., 2024;
Sahoo et al., 2024;
Zamfirescu-Pereira et al., 2023) and quantitative analyses of technology adoption in general settings (
Autor et al., 1998,
2003;
Comin & Hobijn, 2010;
Li & Peter, 2021). However, previous work either remains qualitative or only considers users’ endogenous mistakes. This area of literature leaves unexplained the role of the exogenous stochastic errors that are intrinsic to contemporary large language models (
Vaswani et al., 2017), particularly in regulated environments where liability for algorithmic misclassification cannot be fully insured and remains with human decision makers. None of these studies endogenize the trade-off between effort spent producing and effort spent hedging against AI errors. As a result, there is no normative, micro-founded framework for how professionals should optimally divide effort between direct work and prompt refinement when AI performance is inherently stochastic.
In this paper we develop such a framework, treating prompt design as a risk-mitigating investment (
Dionne & Eeckhoudt, 1985;
Ehrlich & Becker, 1972) and showing how an agent’s risk preferences determine the optimal allocation of effort between prompting and direct production. Our central result (Theorem 1) is that, because prompting effort is a substitute for exogenous shocks to AI performance, risk-averse agents rationally “over-invest” in AI enhancement, risk-neutral agents invest until the expected marginal product equals that of unassisted task effort, and risk-seeking agents “under-invest.” Consequently, under the model’s assumptions and controlling for prompting ability, observed prompting effort can serve as an informative indicator of individual attitudes toward downside AI risk. We illustrate the framework’s mechanics in a simple Cobb–Douglas setting, where the closed-form comparative statics follow directly from Theorem 1. This example shows how departures from the risk-neutral benchmark map onto risk aversion, neutrality, or risk-seeking preferences within the model, thereby clarifying the conditions under which prompting behavior can be informative about underlying risk preferences.
Our framework rationalizes empirical results that users increase monitoring after witnessing an AI error (
Dietvorst et al., 2015,
2018;
Lenskjold et al., 2023), as a negative realized shock in AI performance raises the marginal product of a follow-up, corrective prompt. This mechanism connects the growing “algorithm-aversion” literature in human–AI interaction with classic models of self-insurance and self-protection in risk (
Dionne & Eeckhoudt, 1985;
Ehrlich & Becker, 1972), showing that precautionary prompting is a cognitive form of self-protection, analogous to physical risk-mitigation behaviors like wearing a seatbelt or purchasing antivirus software. A unique feature of our decision environment distinguishing it from previous works involving self-protection, however, involves the lack of an insurance market to protect against the errors made by noisy AI systems. For instance, a lawyer who relies on GPT to draft a legal brief faces reputational or disciplinary risk that cannot be offloaded through conventional insurance. Hence, in the absence of insurance markets protecting against errors involving AI, users of AI
must self-insure to manage this downside risk to hedge against the AI’s inherent randomness. This market failure helps motivate Assumption 1 in the class of environments studied, rather than serving merely as a technical assumption for mathematical convenience.
In heavily regulated environments such as healthcare, law, and finance, the liability associated with algorithmic errors ultimately remains with human decision makers and the institutions that employ them. Clinical providers, attorneys, and financial managers cannot fully transfer the consequences of AI misclassification, biased recommendations, or model failure through conventional insurance mechanisms, particularly when harms arise from context-specific misuse or insufficient oversight (
Goldberg et al., 2026). As a result, AI adoption in these settings introduces a form of non-diversifiable operational risk that must be managed internally. The effort devoted to monitoring, refining prompts, and verifying outputs therefore functions as a form of self-insurance against stochastic AI performance. In this sense, corrective prompting and oversight are not merely technical adjustments but economically meaningful protective investments undertaken to mitigate downside exposure in environments where reputational, financial, and regulatory consequences are material.
Although we frame the analysis in terms of ‘prompting’ a noisy AI assistant, the same risk-sensitive allocation logic applies whenever a decision maker chooses between (i) direct production effort and (ii) monitoring effort that reduces downside risk from exogenous errors in any supporting technology or personnel. For example, a principal investigator (PI) deciding how much to code personally versus writing detailed instructions and checks for a research assistant; a medical physician deciding how much to rely on the recommendation of their nursing staff versus double-checking test results themselves; or a manager trading off direct task work with overseeing the work completed by their administrative assistant. In all cases, Assumption 1 is intended to capture settings in which protective effort is most valuable in bad states, so risk-averse agents optimally allocate more to protection than the risk-neutral benchmark, while risk-seeking agents allocate less. This broader view enhances our understanding of heterogeneous adoption and oversight well beyond text prompting.
The remainder of the paper is organized as follows:
Section 2 reviews the relevant literature in psychology, economics, and human–AI interaction.
Section 3 presents the formal model of effort allocation.
Section 4 walks through a simple Cobb–Douglas application of the model.
Section 5 provides concluding remarks.
2. Literature Review
The concept of resource-limited cognition, which posits that humans have finite cognitive resources for processing information and performing tasks, provides the foundation for optimization of resources between humans and technology. Early research on dual-task interference framed performance limits in terms of serial processing constraints, suggesting that prioritizing processing for one task can impair performance on a concurrent second task (
Telford, 1931;
Vince, 1949;
Welford, 1952). Seminal work by
Norman and Bobrow (
1975) differentiated between data-limited and resource-limited processes, arguing that performance is often constrained by the availability of cognitive resources, not just information. The influential model of attention and effort created by
Kahneman (
1973) further established that cognitive capacity is a limited resource that must be strategically allocated. The concept of “economy of the human-processing system” by
Navon and Gopher (
1979) is especially relevant to our model because it frames dual-task performance as an allocation problem of limited cognitive resources across interfering tasks according to their preferences. Later dual-task research expanded this early resource-limited framework by studying parallel processing (
Fischer & Plessow, 2015), while more recent work has also investigated learning effects and error patterns (
Hommel, 2020;
Salvucci & Taatgen, 2008;
Strobach et al., 2015).
Indeed, dual-task performance can be understood as a constrained optimization problem in which agents must decide how much cognitive effort to allocate to direct production versus technology-enhancing activities such as prompting and oversight. These studies provided the groundwork for understanding cognitive resources as analogous to other scarce resources, subject to allocation decisions that impact performance. This perspective is further reinforced by more recent psychological research exploring the implications of cognitive resource limitations in various domains (
Engle, 2018). In the model presented here, this penalty on cognitive effort is captured by the Lagrange multiplier which serves as the shadow price of the agent’s cognitive budget constraint.
While the core principle of resource limitation is widely accepted, its precise implications for decision-making remain an active area of research. For instance,
Deck and Jahedi (
2015) demonstrated that individuals under high cognitive load tend to rely more on heuristics, indicating suboptimal resource allocation, while
Hagger et al. (
2010) showed that cognitive resource depletion can impair self-regulation. These results suggest that cognitive limitations can lead to deviations from purely rational behavior. Although alternative perspectives on cognitive resources exist (
Inzlicht et al., 2021;
Tuk et al., 2015), they do not negate the fundamental constraint of limited cognitive capacity, which underscores the inherent limitations of human cognition (
Palma et al., 2018).
Complementing the psychological perspective, economic models have long recognized the importance of resource allocation under constraints.
Becker (
1965), modeling the allocation of time, highlighted the trade-off between labor and leisure, mirroring the tension between direct task execution and AI development in the model presented here. Modern economic theory, particularly the concept of opportunity cost, emphasizes constrained optimization as a central element of decision-making, with applications ranging from consumer choice to firm behavior (
Varian, 1992). Behavioral economics further develops these principles by demonstrating how cognitive limitations influence economic choices (
Thaler, 2016). For example, research on intertemporal choice highlights the challenges of resource allocation when decisions involve trade-offs between immediate and future benefits, a challenge central to the allocation of effort between direct tasks and AI development (
Kim & Zauberman, 2019;
Loewenstein, 1992).
Building on the foundational expected-utility framework of
Von Neumann and Morgenstern (
1947), our analysis expands upon the covariance approach to production under uncertainty. The idea of this approach is best understood by examining two primary areas of literature concerning how an agent’s effort interacts with risk. In the price uncertainty literature, effort is a
complement to a positive shock in production: a higher market price increases the marginal return of producing more output (
Kihlstrom & Mirman, 1974;
Leland, 1972;
Sandmo, 1971). For a risk-averse agent with a concave production function, this positive relationship between the random shock and the marginal product of effort leads to a negative covariance between marginal utility and marginal product. As a result, risk-averse firms produce
less than their risk-neutral and risk-seeking counterparts.
Other research, though, considers inputs as
substitutes for positive shocks in production. This assumption is fundamental to models of self-insurance and self-protection, where effort is a ‘protective activity’ (
Dionne & Eeckhoudt, 1985;
Ehrlich & Becker, 1972). In this case, a positive shock reduces the marginal value of the input. The idea is that, because protective effort is a substitute for good luck, a positive shock leaves less risk to offset, so each additional unit of protection yields lower marginal value if a positive state is realized. This
reverses the sign of the covariance, leading risk-averse agents to optimally ‘over-invest’ in the risk-reducing activity, a result often termed a ‘precautionary’ increase in effort (
Jullien et al., 1999).
Li and Peter (
2021) examined how technology risk affects a single risk-mitigation effort within a self-insurance and prudence framework (
Kimball, 1989), allowing inputs to be either substitutes for or complements to technology shocks. We, in contrast, model the trade-off between optimal allocation of effort between direct work and prompting an AI system whose output is stochastic. This distinction is crucial because our model shifts the focus from the agent’s
endogenous misjudgments of the AI system’s ability to the
exogenous errors made by noisy AI systems (
Zamfirescu-Pereira et al., 2023). This framework is better suited to understand real-world trade-offs in modern AI workflows (
Brynjolfsson et al., 2017).
A sizable area of literature in consumption-based capital asset pricing modeling (CCAPM) has also studied the same covariance mechanism from
Section 3 extensively. Starting with exchange and intertemporal pricing results (
Breeden, 1979;
Lucas, 1978) and their empirical implementation (
Hansen & Singleton, 1982), CCAPM values payoffs by how they co-move with the intertemporal marginal rate of substitution; insurance-like payoffs that deliver in ‘bad’ states are more valuable and therefore require a lower expected return (
Cochrane, 2009). In this context, our setting treats normalized utility as the stochastic discount factor and the marginal product of AI enhancement as the return.
When prompting and AI shocks are substitutes, as assumed in
Section 3, the marginal product of prompting is highest when the AI realizes a bad state. For a risk-averse agent, marginal utility is also highest in those states, so the covariance between marginal utility and the marginal product of prompting is positive. As a result, the risk-adjusted value of prompting is high, and the agent optimally accepts a lower expected marginal product than a risk-neutral benchmark, what we term rational “over-investment” in protection. By contrast, leveraging the same AI system, a risk-loving agent values additional performance more in good states; because the marginal product is largest in bad states, the covariance turns negative, and the optimum sets a higher expected marginal product, leading to an ‘under-investment’ relative to the risk-neutral benchmark. Similarly, production-based asset models ‘price’ technology by how their marginal products co-vary with the stochastic discount factor, placing our dual-task model squarely within that tradition (
Cochrane, 1991,
2021).
Taken together, these separate areas of literature converge to a single point. Notably, when the marginal return on an input is highest in the most favorable states, rational agents invest more in that input, not less. A central contribution of the research presented here is the application of this idea to human–AI workflows, characterizing prompt-engineering effort as a form of self-protection that minimizes the downside risk of AI errors. By deriving the covariance condition from a simple production-risk model within a dual-task cognitive budget, our framework rationalizes why risk-averse individuals rationally “over-invest” into prompting.
This insight suggests that, controlling for prompting ability and task conditions, variation in prompting effort may be informative about underlying attitudes toward downside AI risk. At the same time, the behavioral literature does not imply a uniform response to AI outputs. Studies regarding this have demonstrated that algorithm aversion emphasizes increased vigilance after observed errors (
Dietvorst et al., 2015) and that users may accept imperfect algorithms more readily when they retain limited control over the output (
Dietvorst et al., 2018). However, the broader automation literature cautions that under-reliance and over-reliance can both arise depending on context (
Parasuraman & Riley, 1997). The model presented here is intended for environments in which poor realized AI states induce corrective prompting and monitoring. Thus, the paper bridges resource-limited cognition and risk-sensitive production theory while providing a disciplined theoretical basis for future work on the economic forces shaping AI adoption, monitoring, and governance.
3. Theoretical Model
To formalize how an agent allocates effort between tasks, consider an agent with a finite cognitive bandwidth
(
Navon & Gopher, 1979;
Simon, 1955;
Sims, 2003). This agent allocates scarce cognitive resources between two competing tasks (Task 1 and Task 2) analogous to a dual-task paradigm studied extensively in psychology (
Pashler, 1994) and economics (
Buser & Peter, 2012;
Holmstrom & Milgrom, 1991). In Task 1, the agent can complete the task themselves with effort,
, or use an AI system to complete Task 1 on their behalf, but must allocate resources
to write and refine prompts. The effort
e directed toward enhancing the AI system on Task 1 generates a product which the unassisted agent could have completed with
units of effort, where
is a strictly increasing, concave function and
is a ‘temperature’-like shock random variable with the integrable cumulative density function
.
1 At the same time, the agent must complete Task 2 on their own with the effort
. In both tasks, the agent derives utility from their performance. Task 1 and 2 performances are denoted as
and
, respectively, according to the utility function
(
Drichoutis & Nayga, 2020), which is assumed to be twice differentiable and strictly increasing.
It is useful to interpret the choice variable e as overall prompting effort. In practice, this includes both the quantity of interaction with the model, such as repeated prompting or follow-up refinement, and the quality of prompt design, such as specificity, structure, and contextualization of instructions. Likewise, although the model treats the AI production function as fixed, prompting ability could differ across users. Holding the AI system fixed, users with greater prompting ability obtain a larger marginal increase in Task 1 performance from an additional unit of prompting effort, so is larger at a given model state. We abstract from that additional source of heterogeneity to isolate the role of risk preferences in prompting behavior, so the comparative statics below should be interpreted conditional on prompting ability. The shock term is likewise a reduced form, possibly capturing miscalibration, latent bias, or context-specific misalignment that affects AI performance independently of the user’s chosen effort. These simplifications are deliberate. The model holds prompting ability fixed, collapses prompt quantity and prompt quality into a single reduced-form choice variable, and treats AI error with a reduced-form stochastic term in order to isolate the risk-preference mechanism at the center of Theorem 1. This framework can be extended to allow for heterogeneous prompting ability, separate intensive and qualitative margins of prompting, and richer AI error distributions.
In a clinical setting, Task 1 can represent AI-assisted diagnosis or risk prediction, where performance depends jointly on human input and algorithmic output, while Task 2 represents non-automatable human judgment tasks such as contextual evaluation, ethical deliberation, or direct patient interaction. Within this interpretation, the stochastic shock captures model miscalibration, training bias, or context-specific misalignment that affects AI performance independently of the user’s chosen effort. This framing illustrates how the formal structure applies to a multitude of regulated environments.
Since the AI system’s exact response is unknown before the prompt is submitted, the agent must make decisions to maximize the expected utility derived from the AI system’s response (
Savage, 1972). It follows that the decision problem faced by the agent is given by Expression (
1):
For the model illustration, all costs of effort are normalized to 1 in the budget constraint
since all forms of cognitive effort are ultimately measured in uniform “units” drawn from the same finite resource pool (e.g., attention, working memory, mental bandwidth, etc.). This is supported by foundational work in cognitive psychology and economics on attention and working memory (
Baddeley, 1992;
Kahneman, 1973), data-limited and resource-limited processes (
Norman & Bobrow, 1975), and the concept of cognitive bandwidth as an economic resource (
Caplin, 2016;
Wojtowicz & Loewenstein, 2023). Fundamentally, the effort exerted to write and refine AI prompts is equivalent to the effort exerted to perform the task directly as both draw from the same limited cognitive reserves (
Zamfirescu-Pereira et al., 2023). Whether an agent exerts effort crafting a prompt to elicit a desired response from an AI or is manually performing the task unassisted, both utilizing activities draw from the same constrained mental resources
C, similar to those outlined in the limited resource model (
Baumeister et al., 2018) and in multi-task settings (
Borghini et al., 2012). This approach mirrors standard microeconomic theory, where diverse resources are often aggregated into a single budget constraint and assigned a unitary “price” for analytical tractability, simplifying the analysis without loss of generality (
Varian, 1992). By expressing all activities in terms of a common pool resource, the model focuses on the essential trade-offs between competing uses of effort and attention, rather than on arbitrary scaling factors that do not affect the underlying optimization problem (
Mas-Colell et al., 1995). Therefore, the cognitive effort expended to improve an AI system is directly comparable to the effort required to complete the task independently, as they are essentially interchangeable claims on the same constrained cognitive resource.
This analysis focuses on the interior solution where all choice variables are strictly positive. Notably, when focusing on an interior solution, standard first-order optimality conditions often suffice to characterize how the agent distributes resources across tasks. Intuitively, as AI systems become increasingly advanced, corresponding to a larger , people will integrate more AI enhancements into their workflows. This increase in productivity corresponds to a greater incentive for having AI systems assist with all of the work on people’s behalf. However, a trade-off of reallocating cognitive resources from completing the task unassisted to improving AI systems is the reduced oversight of the latent process determining the observed performance. As people spend a greater amount of time improving the AI system to complete a task on their behalf, rather than improving their own performance on the task, there will be reduced human oversight to ensure the validity of the final product. In essence, an interior solution in the case of the model’s framework guarantees that the agent leveraging AI (i.e., ) has some control over the process in Task 1 with their input , while also attending to tasks which cannot be replaced with AI with their input .
Further strengthening this concept, the interior solution to the decision problem (
1) can be found using the Lagrangian multiplier method. Proposition 1 illustrates the first-order condition of the agent’s decision problem, providing insight into how people optimize the use of AI while ensuring that its work is accurate.
Proposition 1. A unique interior stationary allocation that solves the decision problem (1) satisfies Expression (
2) describes the relationship between the expected marginal product of AI enhancement (i.e.,
) and the expected marginal utility of increased task performance (i.e.,
) at the optimal effort allocation
.
2 While the former is the technological marginal product measure describing how an extra unit of prompting effort boosts performance of the AI, the latter is a preference-based marginal utility measure describing how an incremental increase in Task 1 performance boosts expected utility. If the AI system is degenerate, the error structure of the stochastic AI system is additive, or the agent is
risk-neutral, the covariance term would be zero, leading to the standard condition
. We formalize this intuition with Corollary 1 of Proposition 1.
Corollary 1. If the AI system is degenerate (i.e., ), the AI error structure is additive (i.e., ), or the agent is risk neutral (i.e., ), and the optimal expected marginal product of AI enhancement equals the marginal product of working on Task 1 unassisted, i.e., Similarly to the expected value of lotteries,
serves as the ‘rational’ baseline for risk-neutral agents. Much like how traditional risk-neutral agents are indifferent to the variance of lotteries, our risk-neutral agent is indifferent to the
covariance between the marginal product of AI enhancement and the marginal utility of increased Task 1 performance. Because the marginal utility of performance is constant for a risk-neutral agent, they are indifferent to the AI’s ability to be more productive in high-stakes situations (
Otten, 2009), as all situations hold the same marginal utility for them. Simply, if AI systems are unable to affect the marginal utility of task performance with their exogenous shocks, agents behave rationally in the traditional sense by exerting prompting effort to a level such that the expected marginal product of AI enhancement equals the marginal product of working on Task 1 unassisted.
This formulation implies that prompting effort carries a risk adjustment analogous to the pricing of risky assets in financial markets. When the marginal product of prompting covaries positively with marginal utility, the agent accepts a lower expected marginal product in exchange for insurance-like performance in adverse states, much like accepting a lower expected return for holding an asset that performs well during downturns. Conversely, when the covariance is negative, the agent requires a higher expected marginal product to justify additional effort. In institutional settings, this mechanism suggests that observed monitoring intensity may reflect an underlying risk adjustment embedded in governance decisions surrounding AI deployment.
However, the central premise of this paper hinges on how the AI shock
interacts with the marginal product of prompting effort
. A large amount of literature on
algorithm aversion finds that once people observe an algorithm make an error, they penalize it more harshly than they would a human and become more vigilant in monitoring subsequent outputs (
Dietvorst et al., 2015). That vigilance, combined with the well-documented tendency to notice errors in others’ work more readily than in one’s own (
Pronin & Hazel, 2023;
Pronin & Kugler, 2007;
Pronin et al., 2002), means that negative realizations of
trigger quick low-cost “local” fixes. These would be follow-up prompts that nudge the model back on track. This corrective mechanism is asymmetric: users actively correct negative AI shocks due to algorithm aversion but simply accept positive ones (
Dietvorst et al., 2018;
Lenskjold et al., 2023). Because the shock effectively crowds out ex-ante prompting effort, they are
substitutes of each other. We formalize this idea with Assumption 1.
Assumption 1. The AI shock ξ is a substitute for prompting effort e if the cross-partial derivative is negative: Our interpretation of Assumption 1 is deliberately local rather than universal. Evidence on algorithm aversion shows that users often penalize algorithms after observing errors, while related work also shows that willingness to rely on algorithmic output can increase when users retain even limited control over how that output is adjusted (
Dietvorst et al., 2018). More broadly, the automation literature emphasizes that both under-reliance and overreliance are possible, depending on the task environment and the design of human oversight (
Parasuraman & Riley, 1997). Accordingly, Assumption 1 should be read as a scope condition for settings in which poor realized AI states induce corrective prompting and monitoring. In environments dominated by automation bias, passive reliance, or weak verification, the substitution between shocks and protective effort may be weaker, which is a natural extension for future work.
Assumption 1 implies that, as the AI performs better (worse) due to a positive (negative) shock, the incremental benefit of investing more prompting effort decreases (increases). When the AI underperforms by making an error—corresponding to a ‘low’ realized —a corrective prompt fixing the error leads to a big boost in AI performance, corresponding to ‘high’ marginal product of AI enhancement. Conversely, when the AI overperforms by producing something incredible—corresponding to a ‘high’ realized —a follow-up prompt leads to no further AI productivity gains (as you already have a high-quality response from the AI), corresponding to ‘low’ marginal product of AI enhancement. Put differently, Assumption 1 captures the idea that effort is most valuable when the AI needs fixing. In practice, this reflects how corrective prompts are most valuable when AI outputs are incorrect or misaligned, a phenomenon documented in algorithm-aversion research and clinical AI monitoring environments.
Building on this idea, an increase in must lead to a decrease in the marginal product of AI enhancement under Assumption 1, since a follow-up corrective prompt is least effective when the AI does a good job. To determine the effect of a positive shock in on the marginal utility of Task 1 performance , one can take the cross-partial derivative . Under a common-sense assumption that a positive shock must increase AI output (i.e., ), the sign of is entirely determined by the sign of and differentiates whether a positive shock in AI task performance increases or decreases the marginal utility of Task 1 performance. Specifically, is zero when the agent is risk-neutral (), negative when the agent is risk-averse (), and positive when the agent is risk-seeking (); this implies marginal utility is (i) invariant to the shock in the case of risk neutrality, (ii) decreasing in a positive shock under risk aversion, and (iii) increasing in a positive shock for a risk-seeking agent. Since an increase in leads to a decrease in the marginal product of AI enhancement under Assumption 1, must be (i) zero under risk neutrality, (ii) positive under risk aversion, and (iii) negative under risk loving.
From Proposition 1 and positive marginal utility of Task 1 performance , it follows that (i) under risk neutrality if and only if , (ii) under risk aversion if and only if , and (iii) under risk loving if and only if . Put differently, if Assumption 1 holds and one observes the agent choose either , , or , that choice is informative about the sign of and thus about the curvature of the utility function over Task 1 performance at the stationary point . We formalize this intuition below with Theorem 1.
Theorem 1. If the AI system is stochastic and Assumption 1 holds, then
- 1.
if and only if the agent is risk-averse.
- 2.
if and only if the agent is risk-neutral.
- 3.
if and only if the agent is risk-seeking.
Economically, Theorem 1 states that prompting is valued not only by its average marginal product, but also by when that marginal product is realized. When prompting is most productive in bad AI states, it provides insurance-like performance exactly when a risk-averse agent values additional Task 1 performance most, so the agent optimally accepts a lower expected marginal product than the risk-neutral benchmark. The opposite logic applies for a risk-seeking agent.
Figure 1 illustrates this benchmark-relative comparison under diminishing returns: risk-averse agents rationally “over-invest” in AI enhancement relative to the risk-neutral benchmark
, while risk-seeking agents “under-invest.” Within the model, investment in AI enhancement is therefore informative about the curvature of utility over task performance.
While the AI production function
f is the same across risk-averse and risk-seeking agents (strictly increasing and exhibiting diminishing returns), the covariance alignment on the LHS of Expression (
2) might be taken as effectively ‘inflating’ and ‘deflating’ the AI production function, as shown in
Figure 2. AI systems with positive covariance alignment are effectively more productive because their marginal output is greatest in conditions where that output is most valuable to the agent. Conversely, if a system’s marginal output is highest when that output is least valuable, its effective productivity is diminished because each unit of prompting effort will likely pay off only when the outcome is least important. Conversely, agents have a strong incentive to invest more into AI systems that deliver in states that matter most, as better performance leads to even more payoffs when deploying positively aligned AI systems. Put together, since AI systems exhibit diminishing returns (
Thompson et al., 2021), AI systems perform best (worst) in states that matter most (least) to risk-averse (risk-seeking) agents, leading to a hidden ‘boost’ (‘contraction’) of productivity in the use of AI systems.