The Q-Exponential Decay of Subjective Probability for Future Reward: A Psychophysical Time Approach

Abstract

This study experimentally examined why subjective probability for delayed reward decays non-exponentially (“hyperbolically”, i.e., q < 1 in the q-exponential discount function) in humans. Our results indicate that nonlinear psychophysical time causes hyperbolic time-decay of subjective probability for delayed reward. Implications for econophysics and neuroeconomics are discussed.

PACS Codes: 87.10.Mn

1. Introduction

1.1. Subjective Probability and Intertemporal Choice

In order to account for hyperbolic discounting in intertemporal choice [1–4], evolutionary theorist Peter Sozou proposed that subjective probability of obtaining the delayed reward in intertemporal choice decays hyperbolically [5,6]. Although the causes of hyperbolic decay of subjective probability for delayed reward is important for econophysics and neuroeconomics [7] in addition to quantitative finance, no definitive answer has been obtained as to whether Sozou’s theory is correct or not. For instance, we have previously examined Sozou’s theory and obtained results which only partially support it. In this study, we alternatively examined the applicability of Takahashi’s (2005) nonlinear time perception theory of hyperbolic discounting [2] to the hyperbolic time-decay of subjective probability of obtaining delayed reward. In this study, we hypothesized that the reason why subjective probability for delayed reward decays hyperbolically is that logarithmic psychophysical time distorts exponential decay of subjective probability for delayed reward in physical time.

1.2. Theoretical Background

In economics, econophysics, and neuroeconomics, several types of intertemporal choice models have been proposed. In most models, the subjective value of delayed reward is assumed to be temporally discounted (i.e., delayed reward is devalued according to an increase in delay until its receipt). The most classical model is an exponential discounting model [8]:

$$\text{V}(\text{D})=\text{V}(0)\text{exp}(-{\text{k}}_1\text{D})$$

(1)

where V(D) is a value (i.e., time-discounted utility) of delayed reward obtained at delay D and k₁ is an exponential time-discount rate. In exponential discounting, the agent is time-consistent [1,3]. However, later studies in behavioral economics and behavioral psychology reported that people and animals do not follow the exponential time discounting function [1,3]. Rather, the following simple hyperbolic function fit better than the exponential discount function:

$$\text{V}(\text{D})=\frac{\text{V}(0)}{1+{\text{k}}_0\text{D}}$$

(2)

where k₀ is a hyperbolic time-discount rate at delay D = 0. In this hyperbolic discounting, the agent’s preference reverses over time [1,3]. In order to generalize the two discount models above, the following q-exponential time discount model has been proposed [1]:

$${\text{V}}_{\text{q}}(\text{D})=\frac{\text{V}(0)}{{(1+{\text{k}}_{\text{q}}(1-\text{q})\text{D})}^{\frac{1}{1-\text{q}}}}$$

(3)

where k_q is the q-exponential time-discount rate and 1 − q indicates the deviation from the exponential function. Specifically, q→ 1 corresponds to the exponential time-discount model and q = 0 corresponds to the simple hyperbolic time-discount model. Please note that the Equation (3) is different from V(0)exp_q(−k_qD) (see [9–12] for details). Our previous studies revealed that the q-exponential time discount model fitted best among the three discount models (i.e., exponential, simple hyperbolic, and q-exponential) [4,7,8]. It is to be noted that the q-exponential time-discount model is derived from exponential discounting with logarithmic psychophysical time [2–4,13,14].

Several authors speculated that temporal discounting in intertemporal choice occurs due to an increase in uncertainty associated with delay, because future is risky. Evolutionary theorist, Peter Sozou (1998, [5]), proposed that Bayesian-estimated subjective probability of obtaining the delayed reward follows the simple hyperbolic function in terms of delay. Although this hypothesis was partially supported (certainty for delayed reward temporally decayed in a hyperbolic manner, rather than an exponential manner) by our previous experiment [6], one of the predictions from Sozou’s theory (i.e., relationship between precautious risk aversion and time preference) was rejected in the experiment [6]. Therefore, it is the logical next step to explore alternative accounts of the hyperbolic time-decay of subjective probability of obtaining the delayed reward. This exploration is important for advances in behavioral and neuroeconomics and finance, because finance is a theoretical framework for mitigating the risk due to uncertainty associated with future.

We have previously proposed [2], and experimentally confirmed [4,13], that “hyperbolic” (i.e., q < 1 in the q-exponential discount model, q = 0 corresponds to the simple hyperbolic function introduced in Equation (2)) temporal discounting is due to nonlinear (logarithmic) psychophysical time. Hence, it is probable that hyperbolic time-decay of subjective probability for delayed reward is also originated from psychophysical characteristics of subjective time. We, therefore, experimentally examined this hypothesis in the present study. If our current hypothesis is correct, (i) subjective probability for delayed (future) reward decays hyperbolically in terms of delay; (ii) subjective time for (objective) delay until receipt of the future reward follows Weber-Fechner law (i.e., subjective time is logarithmic in terms of physical time); and (iii) the hyperbolicity of the functional form of the subjective probability for delayed reward is reduced when subjective time is employed instead of (objective) delay (i.e., q will be increased when delay is replaced with subjective time). Mathematically, in testing the hypothesis, we utilized the following q-exponential time-decay function of subjective probability:

$${\text{SP}}_{\text{q}}(\text{D})=\frac{\text{SP}(0)}{{(1+{\text{k}}_{\text{q}}(1-\text{q})\text{D})}^{\frac{1}{1-\text{q}}}}$$

(4)

It is to be noted that V(D) = SP₀(D) × V(0) with SP₀(0) = 1 corresponds to Sozou’s hypothesis’ assumption.

For functional forms of psychophysical laws of time perception, we examined the following functional forms: i.e., linear, power (Stevens power law), and logarithmic (Weber-Fechner law) functions for time perception in waiting for future reward.

A linear function is:

$$\mathrm{\tau }(\text{D})=\mathrm{\alpha }\text{D}$$

(5)

Steven’s power law is:

$$\mathrm{\tau }(\text{D})=\mathrm{\alpha }{\text{D}}^{\mathrm{\beta }}$$

(6)

Weber-Fechner (logarithmic) law is:

$$\mathrm{\tau }(\text{D})=\mathrm{\alpha }\hspace{0.17em}\text{ln}(1+\mathrm{\beta }\text{D})$$

(7)

In Equations (5)–(7), τ(D) is subjective (psychophysical) time as a function of delay (physical time). α and β are positive free parameters.

Then, our hypothesis’ prediction (i) and (iii) corresponds to the statement that q is larger for SP_q(τ) in comparison to that for SP_q(D).

We now mention the relationship of the current study to previous studies [13,15,16] on hyperbolic time discounting. These previous studies examined the roles of subjective time only in time discounting; i.e., how subjective time can account for Equation (3), not Equation (4) (note again that Equation (3) describe time discounting; in contrast, Equation (4) proposes the time-dependency of subjective probability). According to standard economic theory [17], attitudes for risk is unrelated to time preference; i.e., time discounting has not been supposed to a decrease in subjective probability in the discipline of neoclassical economics. In this study, however, we for the first time tried to extend the nonlinear time-perception theory of non-exponential time discounting [2] into the domain of time-dependency of subjective probability for delayed reward, by considering Sozou’s hypothesis [5] in evolutionary biology (rather than economics).

2. Results and Discussion

2.1. Physical Time-Decay of Subjective Probability for Future Reward

First, we plotted the group data of subjective probability for future reward (group data are expressed as mean +/− standard error of the mean, see Figure 1a). Then, we fitted the three types of the time-decay models of subjective probability for future reward; i.e., the exponential model (q = 1 in Equation (4), similar to Equation (1)), the simple hyperbolic model (q = 0 in Equation (4), similar to Equation (2)), and the q-exponential model (Equation (4)) (Figure 1b). Among the three models, the q-exponential function best fitted the behavioral data (see

Table 1. Parameters and AIC (Akaike Information Criterion) for time-decay of subjective probability.

	Exponential	Hyperbolic	q-Exponential
AIC	67.54205	64.26132	23.16847
Parameter	k1 = 0.0018215	k0 = 0.027814	kq = 0.027814, q = 6.295212

). Because AIC of the simple hyperbolic model was smaller than that of the exponential model (and q was smaller than 1), the time-decay of subjective probability for future reward was “hyperbolic” rather than exponential, consistent with our prediction (i).

2.2. Psychophysical Time for Delay until Receipt of Reward

Next, we plotted group data of subjective time for delay until receipt of future reward (group data are expressed again as mean +/− standard error of the mean, see Figure 2a). Then, we fitted the three types of psychophysical laws (Equations (5)–(7)) (see Figure 2b). The logarithmic function best fitted the psychological data (see

Table 2. Parameters and AIC (Akaike Information Criterion) for physical time (delay/day) vs. subjective probability (%) for group data.

	Linear Function	Power Function	Logarithmic Function
AIC	89.05314	61.38481	58.18276
Parameter	α = 0.02149	α = 77.98593, β = 0.08858	α = 12.222, β = 171.403

, the logarithmic function had the smallest AIC), indicating the subjective time for delay until receipt of future reward follows Weber-Fechner law. This verifies our prediction (ii).

2.3. Psychophysical Time-Decay of Subjective Probability for Future Reward

Finally, we examined decay dynamics of subjective probability for future reward in a psychophysical dimension; i.e., psychophysical time-decay of subjective probability for future reward. Figure 3a shows group mean (+/− standard error of the mean) of subjective probability for each subjective time (for each corresponding delay). We further examined the manner of temporal dynamics of subjective probability in this purely psychophysical (subjective) dimension (see Figure 3b), as can be seen from the Figure 3b, the decay speed of the subjective probability is increasing toward the psychophysical future. In addition, the best fit model was the q-exponential model with q > 1 (

Table 3. Parameters and AIC (Akaike Information Criterion) for psychophysical time-decay of subjective probability.

	Exponential	Hyperbolic	q-Exponential
AIC	57.00633	58.59702	37.25649
Parameter	k1 = 0.0038631	k0 = 0.004874	kq = 0.001593, q = 4.819

), consistent with our prediction (iii). Taken together, it can be confirmed that hyperbolic time decay of subjective probability for future reward is due to nonlinear psychophysical time perception.

3. Experimental Section

In our current experiment, subjects were asked to draw a line on a 180 mm scale to indicate their subjective time for seven delays (i.e., 1 week, 2 weeks, 6 months, 1 year, 5 years, 25 years) until receipt of future reward (1000 yen = about 10 dollars) and subjective probability (also on a 180 mm scale) for obtaining the delayed rewards at the corresponding seven delays. The subjects were Hokkaido University students (N = 27, male 15 and female 12) whose average age was 20.1. For analysis, we employed group averaged data of subjective time and probability. After plotting the group data of their subjective time and probabilities, we utilized nonlinear curve fitting (with R statistical language) for fitting mathematical models for dynamics of subjective probability and subjective time. The goodness-of-fit (tradeoff between overfitting and poor fitting) was parameterized with AIC (Akaike Information Criterion = 2 × (the number of free parameters) – 2 ln (likelihood)), following our previous studies [6,13,14]. Note that AIC punishes an increase in the number of free parameters in the models for exploring parsimonious explanations for observed data [18].

4. Conclusions

This study is the first to demonstrate that hyperbolic time-decay of subjective probability for future reward is due to nonlinear distortion of delay until receipt of the future reward. Our present results suggest that, in psychological time, future is increasingly more risky. Future studies in neuroeconomics and neurofinance [3,14] should examine neural basis of this psychological tendency. Furthermore, recent advances in high throughput genomic data in neurobiology by Changeux’s group can help us to analyze brain’s organization by utilizing Tsallis’ statistics [19]. Future neuroeconomic studies can also utilize this type of research strategies by utilizing q-generalized statistics.

With respect to subjective probability in economic decision making, a recent study [20] proposed the relevance of Tsallis’ thermostatisical model to one of the most well-established behavioral economic model of economic decision under uncertainty (i.e., Tversky and Kahneman’s cumulative prospect theory [21]). Our present finding on subjective probability for delayed reward could be analyzed with this model in future studies. Moreover, time-inconsistency in temporal discounting has been shown to be disentangled into nonlinearity in time perception and non-exponentiality in time-discounting function [22]. Future studies may also be able to disentangle the non-exponential decay of subjective probability for delayed reward in a similar manner.