Stochastic Optimal Control for Online Seller under Reputational Mechanisms

Abstract

In this work we propose and analyze a model which addresses the pulsing behavior of sellers in an online auction (store). This pulsing behavior is observed when sellers switch between advertising and processing states. We assert that a seller switches her state in order to maximize her profit, and further that this switch can be identified through the seller’s reputation. We show that for each seller there is an optimal reputation, i.e., the reputation at which the seller should switch her state in order to maximize her total profit. We design a stochastic behavioral model for an online seller, which incorporates the dynamics of resource allocation and reputation. The design of the model is optimized by using a stochastic advertising model from [1] and used effectively in the Stochastic Optimal Control of Advertising [2]. This model of reputation is combined with the effect of online reputation on sales price empirically verified in [3]. We derive the Hamilton-Jacobi-Bellman (HJB) differential equation, whose solution relates optimal wealth level to a seller’s reputation. We formulate both a full model, as well as a reduced model with fewer parameters, both of which have the same qualitative description of the optimal seller behavior. Coincidentally, the reduced model has a closed form analytical solution that we construct.

1. Introduction

In this work, we consider sellers in an online market like Amazon without a bidding mechanism. The setting may include concurrent online auctions on a site like eBay, for a product or service with only a “buy-it-now” option, under the assumption that buyers can compare reputation among online platforms. In this online environment, the buyer and seller have a certain amount of anonymity, which has implications on the fairness of the auction [4]. One of the only counter measures to ensure fairness is the use of feedback forums, such as Amazon, eBay, and Yahoo!, at online auction sites. The feedback provided by customers provides a ranking system for the sellers, which in turn rewards fair sellers and penalizes unfair or unreliable sellers [5,6,7]. Since it is in the sellers best interest to maximize their profits, it follows that a seller will seek to use reputation as a means to optimize their decision making [8,9]; e.g., for the optimal bidding strategies in sequential auctions, see [10].

Online auction sellers can be broadly categorized by two types of behavior: those focused on advertising and customer service, which includes following up with past customers; and those who instead focus only on processing existing orders. While the latter behavior will likely lower the seller reputation on average, this immediate increase in wealth may offset the long-term damage from a lower reputation. With this in mind, it is compelling to ask whether there exists an optimal long term strategy, in which a seller attempts to maximize her profit by achieving a fixed reputation.

We propose a simple behavioral model to study the relationship between a seller’s wealth and reputation in the auctions environment with only the “buy-it-now” option. Following the standard Nerlove-Arrow construction [11], which was extended to a stochastic setting by Sethi [1], the wealth and growth of the seller are described by continuous stochastic processes. Note that our model is a continuous approximation of a discrete auctioning process. The discrete model of the process may be less tractable but still very interesting and useful. We propose an optimal infinite horizon strategy, and incorporate the wealth-growth model of Mink and Seifert [3], based on empirical studies of ebay sellers. Also, we use the model for reputation proposed by Sethi [1] and Rao [12] which has been adopted by many others in optimal advertising models, such as Raman’s recent work in Boundary Value Problems in the Stochastic Optimal Control of Advertising [2]. This work incorporates the idea that humans do not multi task well, but rather switch, see [13]. In other words, switching costs do not factor in for an individual agent, but only for large online retailers who have marketing department reorganization costs. Additionally, the model in this work allows all sellers to have a little extra capacity/money for either shipping or advertising (e.g., writing a letter or similar individual campaign), which is rather small and not unlimited.

The rest of the paper is laid out as follows. In Section 2 we define the stochastic Nerlove-Arrow model, along with the growth rate due to Mink and Seifert. Using the principal of optimality we derive the corresponding Hamilton Jacobi Bellman (HJB) boundary value problem satisfied by the value function. In Section 3, we propose several important modifications to the existing methodology, and in doing so derive a new model which is normalized with respect to the seller reputation, and has a linear growth rate. In making these modifications, we also find a reduced version of our new model, which is governed by a geometric Brownian motion. We prove that both models admit unique piecewise continuous solutions, and are qualitatively similar. However the reduced model, surprisingly, has a closed form piecewise defined solution. This allows us to prove that pulsing behavior is an optimal strategy for sellers and also serves as a benchmark to our numerical solution of the full model in Section 4, which does not have a closed form analytic solution. Our results are compelling, and warrant further investigation for the finite horizon case, which we discuss with concluding remarks in Section 5.

2. Explicit Resource Allocation Mechanism

Consider a seller in an online marketplace as a triple (W, R, μ) representing her wealth, reputation, and excess rate, respectively. Here, the excess rate is extra effort that can be allocated to either expedite payment or increase her reputation. In this setting, the reputation R is a positive number reflecting customer satisfaction. As an initial approach, we consider only the effects of reputation on the optimal way to grow her wealth, and leave the more general model with shipping costs for future work. Also, in our mathematical formulation, we consider the reputation mechanism first suggested by the work of Nerlove and Arrow [11] and later generalized to stochastic settings in [1,2,12].

The seller is one of many competing in the online marketplace, and has her transactions verified and processed automatically via the marketplace. All sellers competing in this marketplace have an expected speed for mailing out completed orders to be able to sell in the marketplace, with some time allowed for delayed shipping before the marketplace intervenes. In return, the marketplace guarantees the transaction. Payment is modeled to be released to the seller once the purchased item is mailed to the buyer.

However, the seller can choose to go beyond these expected standards. For example, she may prepare packages and drive to the post office earlier than expected to expedite the payment from the marketplace. Or she may spend time and money communicating with buyers in a bid to increase their favorable ratings. This may include sending a small unexpected gift in the package to be mailed out or spending extra time to individually craft and send emails out to purchasers. Our model assumes that at most only one of these extra actions can happen with noticeable return at any given time. The seller may also decide to do nothing extra, and in this case μ = 0.

For example, if she is focused on getting her payment released, then she is not able to focus on creative ways to engage her buyers post sale. We also assume that her extra capacity to do either extra action is limited by absolute minimum mailing rates and her pool of resources she is willing to contribute to going beyond expected standards. By choosing to shift up her resources from promotion to expedited mailing and back, the seller can influence her wealth and reputation levels via the excess rate μ. Positive μ corresponds to a promotional state, while negative μ corresponds to an expedited mailing state.

We note that if our seller is in fact a large company with many such agents that can work on either processing or sales, then the company can choose to shift the ratio of workers who work in processing or sales, and this model is the subject of future work. The work we present here is limited to a one agent seller. We note here that our model also assumes no cost for the individual to switch behavior from promotion to expedited mailing. In a large company, one would expect such costs to arise in restricting the size of advertising and shipping departments. One related paper that addresses such switching costs, albeit in a real options investment setting, is the paper by Duckworth et al. [14].

We utilize a continuous time formulation for wealth and reputation, as well as control of these processes. The seller will be determined to behave optimally if she maximizes her expected present value of total earnings, until her reputational value reaches R = 0. This produces a fully consistent formulation of both the value function V, and the strategy to achieve this optimal value.

2.1. Mathematical Formulation

Following the framework in [15,16], reputation R is observed in $\mathcal{O}:=(0,\infty )$ and the control μ is constrained to $U:=[-ϵ,ϵ]$. Since this process will be stochastic, we introduce a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ and corresponding reference probability systems $\nu =\left(\left(\Omega ,\left\{{\mathcal{F}}_s\right\},\mathbb{P}\right),B\right)$ where ${\mathcal{F}}_s\subset \mathcal{F}$ and B is an ${\mathcal{F}}_s$-adapted Brownian motion. The evolution of wealth and reputation is hence modeled by a controlled two-state Markov process $(W_{.}^{{\mu }_{.}},R_{.}^{{\mu }_{.}})$, where wealth W grows at a rate proportional to a function h(R) that links reputation to revenue per sale, subject to a controlled extra processing rate μ per unit time chosen from the admissible class $A_{\nu }$,

$${\mathcal{A}}_{\nu }:=\left\{\mu \in {\mathcal{F}}_s\mid \mu \text{is}{\mathcal{F}}_s\text{-p.m.},{\mu }_{.}\in U\text{on}[0,\infty ),\mathbb{E}\left[{\int }_0^{{\tau }_{\mathcal{O}}}{e}^{-\rho s}\left|(1-{\mu }_s)h\left(R_s^{\left(\mu \right)}\right)\right|\text{ds}\right]<\infty \right\}$$

(1)

Here, p.m. denotes progressively measurable, ρ is the constant discount factor to account for the time value of money, and ${\tau }_{\mathcal{O}}$ is the first exit time of reputation R from $\mathcal{O}$, or ∞ if $R\in \mathcal{O}$ for all $s\ge 0$. We set ∞ as an absorbing state for reputation. Symbolically,

$${\tau }_{\mathcal{O}}:=inf\left\{t\ge 0\mid R_t^{\left(\mu \right)}\notin \mathcal{O}\right\}$$

(2)

Formally, our two state controlled Markov process is

$$\begin{array}{ccc}\hfill d{W}_t^{\left(\mu \right)}& =& (1-{\mu }_t)h\left(R_t^{\left(\mu \right)}\right)\text{dt}\hfill \\ \hfill d{R}_t^{\left(\mu \right)}& =& ({\mu }_t-\kappa R_t^{\left(\mu \right)})\text{dt}+\sigma {\text{dB}}_{\mathrm{t}}\hfill \end{array}$$

where κ is a proportionality constant which accounts for mean reversion [11] and σ is the constant volatilty accounting for random effects to reputation.

The major difficulty is in defining the growth rate h(R). However, an explicit mechanism has been proposed by Mink and Seifert [3]. There the authors not only propose, but empirically justify a growth rate of

$$\begin{array}{cc}\hfill h\left(R\right)& =A+C\left(1-\frac{1}{ln(e+R)}\right)\hfill \\ & =A\left[1+\frac{C}{A}\left(1-\frac{1}{ln(e+R)}\right)\right]\hfill \end{array}$$

(3)

where A relates to the inherent value of the object for sale and C is a parameter to be fitted. This is accomplished by obtaining data using an auction robot and then computing a single regression, which gives C = 2.50 in Equation (3). To the best of our knowledge, the model in [3] is among the first to give an explicit relationship between reputation and price. We also note that the parameter $\frac{C}{A}$ represents the maximal reputational effect of a seller as a fraction of inherent value A, and is what we choose in the numerical examples below. The model also suggests a multiple regression formula where other factors, such as shipping costs and whether a “buy-it-now" price is offered, are considered as well, in which case C = 1.93. The authors in [3] also comment that highly experienced sellers have higher feedback scores and design the auction more favorably, which reflects to their higher revenue. In fact, they show that the coefficient attributed to shipping costs is larger than one, implying that customers put a high value on shipping when deciding on their bids, and that savvy agents take this into consideration. Notice that since h is bounded on $\mathcal{O}$, our admissible class reduces to

$${\mathcal{A}}_{\nu }:=\left\{\mu \in {\mathcal{F}}_s\mid \mu \text{is}{\mathcal{F}}_s\text{-p.m.},{\mu }_{.}\in U\text{on}[0,\infty )\right\}.$$

(4)

Finally, the work in [3] posits that the horizon does not affect the revenue stream as much as the shipping cost and reputation factors, and so we consider an infinite horizon model here.

2.2. Hamilton-Jacobi-Bellman Formulation

With the stochastic dynamics for reputation proposed in [1], a growth rate model for reputational effect on sales [3], and an infinite horizon, we expect that switching would depend only on the current reputational state. We now seek a twice-continuously differentiable, polynomially growing function $V\in C^2[0,\infty )\cap C_p[0,\infty )$ which is a candidate solution of the optimal control problem

$$\begin{array}{cc}\hfill \overline{V}\left(R\right)& =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{{\tau }_0}{e}^{-\rho s}(1-{\mu }_s)h\left(R_s^{\left(\mu \right)}\right)\text{ds}\mid R_0=R\right]\hfill \\ & =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{{\tau }_{\mathcal{O}}}{e}^{-\rho s}(1-{\mu }_s)h\left(R_s^{\left(\mu \right)}\right)\text{ds}+1_{\left\{{\tau }_{\mathcal{O}}<\infty \right\}}{e}^{-\rho {\tau }_{\mathcal{O}}}\frac{1+ϵ}{\rho }h\left(\infty \right)1_{\left\{R_{{\tau }_{\mathcal{O}}}^{\left(\mu \right)}=\infty \right\}}\mid R_0=R\right]\hfill \\ \hfill h\left(R\right)& =A+C\left(1-\frac{1}{ln(e+R)}\right)\hfill \\ \hfill {\tau }_0& :=inf\left\{t\ge 0\mid R_t=0\right\}\hfill \end{array}$$

(5)

One approach to finding $\overline{V}$ would be to solve Equation (5) directly. For example, by the definition of $\overline{V}$, it follows that

$$\begin{array}{cc}\hfill \overline{V}\left(0\right)& =0\hfill \\ \hfill \overline{V}\left(\infty \right)& =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{\infty }{e}^{-\rho s}(1-{\mu }_s)h\left(R_s^{\left(\mu \right)}\right)\text{ds}\mid R_0=\infty \right]\hfill \\ & =\frac{1+ϵ}{\rho }h\left(\infty \right)\hfill \end{array}$$

(6)

However, we shall instead apply the principle of optimality, and in doing so arrive at the following nonlinear Hamilton-Jacobi-Bellman (HJB) boundary value problem (suppressing the explicit dependence of R on μ)

$$\begin{array}{cc}\hfill 0& =\underset{-ϵ\le \mu \le ϵ}{max}\left[(\mu -\kappa R)\frac{\partial V}{\partial R}+\frac{{\sigma }^2}{2}\frac{{\partial }^{2}V}{\partial R^2}+(1-\mu )h\left(R\right)-\rho V\right]\hfill \\ \hfill V\left(0\right)& =0\hfill \\ \hfill V\left(\infty \right)& =\frac{1+ϵ}{\rho }h\left(\infty \right)\hfill \\ \hfill h\left(R\right)& =A+C\left(1-\frac{1}{ln(e+R)}\right)=A+C\frac{ln\left(1+\frac{R}{e}\right)}{1+ln\left(1+\frac{R}{e}\right)},\hfill \end{array}$$

(7)

which simplifies to

$$\begin{array}{cc}\hfill \rho V& =h\left(R\right)-\kappa R\frac{\partial V}{\partial R}+\frac{{\sigma }^2}{2}\frac{{\partial }^{2}V}{\partial R^2}+ϵ|\frac{\partial V}{\partial R}-h\left(R\right)|\hfill \\ \hfill V\left(0\right)& =0\hfill \\ \hfill V\left(\infty \right)& =\frac{1+ϵ}{\rho }(A+C)\hfill \\ \hfill h\left(R\right)& =A+C\frac{ln\left(1+\frac{R}{e}\right)}{1+ln\left(1+\frac{R}{e}\right)}\hfill \end{array}$$

(8)

We therefore instead solve the HJB problem, which is justified by the following standard theorem (Corollary IV.5.1 in [15]):

Theorem 1. 

Let $V\in C^2\left(\mathcal{O}\right)\cap C_b\left(\overline{\mathcal{O}}\right)$ be a twice-continuously differentiable and bounded solution to an associated HJB equation of a control problem

$$V_{\mathrm{PM}}\left(x\right):=\underset{\nu }{inf}\underset{A_{\nu }}{inf}\mathbb{E}\left[{\int }_0^{{\tau }_{\mathcal{O}}}{e}^{-\rho s}G(R_s^{\mu },{\mu }_s)\text{ds}\right]$$

(9)

for a process R whose SDE has drift and volatility that are Lipschitz in μ and R, and a $G(R,\mu )$ that is continuous and polynomially growing for all $R\in \mathcal{O}$ and continuous for all $\mu \in U$. Moreover, assume either that $\beta >0$ or that ${\tau }_{\mathcal{O}}<\infty$ with probability 1 for every admissible progressively measurable control process μ. Then $V\left(x\right)=V_{\mathrm{PM}}\left(x\right)$ for an optimal control $u^{*}\left(s\right)\in argmin[L_R\left[V\right]+G(R,u)]$, where L is the generator of the process R.

We now have our verification theorem:

Theorem 2. 

Let $V\in C^2\left(\mathcal{O}\right)\cap C_b\left(\overline{\mathcal{O}}\right)$ be a twice-continuously differentiable and bounded solution to the boundary value problem Equation (8). Then V is an optimal solution to the optimal control problem Equation (5).

Proof. 

By Corollary IV.5.1 in [15], since

• (i) the drift and volatility are (uniformly) Lipschitz in R and μ,
• (ii) ρ > 0, and
• (iii) we impose an additional boundary condition on the solution as R → ∞ as our indefinite horizon is now $[0,{\tau }_{\mathcal{O}})=[0,{\tau }_0)=[0,\infty )$.

it follows directly that our bounded classical solution $V=\overline{V}$ and the control

$${\mu }^{*}\left(R\right)\in \underset{\mu \in [-ϵ,ϵ]}{argmax}\left[(\mu -\kappa R)\frac{\partial V}{\partial R}+\frac{{\sigma }^2}{2}\frac{{\partial }^{2}V}{\partial R^2}+(1-\mu )h\left(R\right)-\rho V\right]$$

(10)

leads to an optimal, stationary Markov control ${\mu }^{*}\left(R_s^{\left({\mu }^{*}\right)}\right)$.

Remark 1. 

We note here that it is sufficient to require polynomial growth on V, not bounded growth. However, as h is bounded above by A+C on $\mathcal{O}$, we can restrict our attention to bounded V. Furthermore, an upper bound (and in fact limit as R → ∞) for V is the value obtained by a seller earning maximum value A+C on her items on the time interval [0, ∞] while always processing orders.

3. Market Share Based Pricing Mechanism

We observe that under its current definition, the reputation $R\in \mathcal{O}$. In this section, we employ the mapping

$$Y:=f\left(R\right)=\frac{R}{1+R}$$

(11)

which results in a normalized market share reputation $Y\in \mathcal{Q}:=(0,1)$. This is consistent with prior assumptions [3] that the value function $h\left(R\right)$ be concave, and bounded. We therefore also simplify (3) accordingly, replacing logarithms with a rational function

$$\begin{array}{cc}\hfill \overline{h}\left(R\right)& =A+C\frac{R}{1+R}\hfill \\ & =A\left[1+\frac{C}{A}\frac{R}{1+R}\right]\hfill \end{array}$$

(12)

This modification, when combined with the mapping from a reputational score to a market share score, $R\to Y$, produces a more intuitive growth rate, $\tilde{h}\left(Y\right)$ which grows linearly with respect to market share reputation. Recall that ∞ is an absorbing state for R; consequently, we have $d{Y}_t\equiv 0$ if $Y_t\ge 1$. For $Y_t<1$, it follows from Ito calculus that $d{Y}_t=f^{\prime }\left(R\right)d{R}_t+\frac{1}{2}{f}^{\prime \prime }\left(R\right)d{R}_{t}d{R}_t$ and so

$$\begin{array}{cc}\hfill d{Y}_t& ={(1-Y)}^2\left[\left(\mu -\kappa \frac{Y}{1-Y}\right)\text{dt}+\sigma {\text{dB}}_{\mathrm{t}}\right]+\frac{1}{2}(-2{(1-Y)}^3){\sigma }^2\text{dt}\hfill \\ & =\left[\mu {(1-Y)}^2-\kappa Y(1-Y)-{\sigma }^2{(1-Y)}^3\right]\text{dt}+\sigma {(1-Y)}^2{\text{dB}}_{\mathrm{t}}\hfill \\ \hfill \tilde{h}\left(Y\right)& =A+CY\hfill \\ \hfill \tilde{V}\left(y\right)& =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{{\tau }_0}{e}^{-\rho s}(1-{\mu }_s)\tilde{h}\left(Y_s\right)\text{ds}\mid Y_0=y\right]\hfill \end{array}$$

(13)

Note that a function that is bounded for market share $Y\in (0,1)$ is also correspondingly bounded for all reputation $R\in (0,\infty )$. We therefore study the market share model below, confident that our results will hold for the full reputation model by virtue of the inverse map of Equation (11).

3.1. Rescaled HJB Model

After market share rescaling, the HJB now takes a form which is in fact degenerate at both endpoints $y=0,1$:

$$\begin{array}{cc}\hfill & -\frac{{\sigma }^2}{2}{(1-y)}^4{V}^{\prime \prime }\left(y\right)+\rho V=\tilde{h}\left(y\right)-\left[\kappa y(1-y)+{\sigma }^2{(1-y)}^3\right]V^{\prime }\left(y\right)+ϵ|{(1-y)}^2{V}^{\prime }\left(y\right)-\tilde{h}\left(y\right)|\hfill \\ & V\left(0\right)=0,V\left(1\right)=\frac{1+ϵ}{\rho }\tilde{h}\left(1\right)\hfill \end{array}$$

(14)

In theory, a closed form solution for this boundary value problem can be formally constructed piecewise, to the left and right of the special point $y^{*}$, for which $\tilde{h}\left(y^{*}\right)={(1-y^{*})}^2{V}^{\prime }\left(y^{*}\right)$. We shall refer to $y^{*}$ as the switching point below, since it is the reputational value at which the seller switches from advertising to processing. In practice, we shall resort to numerical computation to approximate this solution, and in doing so estimate the value $y^{*}$.

Theorem 3. 

If $V^{**}\in C^2\left(\mathcal{Q}\right)$ is a monotonically increasing solution to the HJB problem Equation (14), then $V^{**}=\tilde{V}$.

Proof. 

Enforcing the boundary condition on $V^{**}\left(1\right)$ enforces bounded growth on our monotonically increasing $V^{**}$. By the inverse map of Equation (11), the result follows as a corollary to Theorem 2. Note that $\mathcal{Q}=(0,1)$ is open, but $y=1$ is not reached in finite time from anywhere in $\mathcal{Q}$ and the diffusion Y is in fact absorbed for all time in state $Y=1$. Hence, the exact value of $V^{**}\left(1\right)$ imposed.

3.2. Reduced Model

With the previous mapping, note that the market (reputation) share Y is absorbed at $Y=1$, which corresponds to $R\to \infty$. Moreover, as in the Stochastic Nerlove evolution, we have that the probability that $Y_t<0$ for some positive t is greater than 0, and the drift term is a third-order polynomial in Y. Based on these observations, we propose a reduced model for Stochastic Reputation Share, expressed by the following stochastic differential equation and associated stochastic optimal control problem

$$\begin{array}{cc}\hfill \check{V}\left(y\right)& =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{{\tau }_0}{e}^{-\rho s}(1-{\mu }_s)(A+C{Y}_s)\text{ds}\mid Y_0=y\right]\hfill \\ \hfill \frac{d{Y}_t}{1-Y_t}& =\mu \text{dt}+\sigma {\text{dB}}_{\mathrm{t}}\hfill \end{array}$$

(15)

This leads to a corresponding nonlinear HJB boundary value problem,

$$\begin{array}{cc}\hfill 0& =\underset{-ϵ\le \mu \le ϵ}{max}\left[\mu (1-y)V^{\prime }\left(y\right)+\frac{{\sigma }^2}{2}{(1-y)}^2{V}^{\prime \prime }\left(y\right)+(1-\mu )(A+Cy)-\rho V\right]\hfill \\ \hfill V\left(0\right)& =0\hfill \\ \hfill V\left(1\right)& =\underset{\nu }{sup}\underset{\mu \in {\mathcal{A}}_{\nu }}{sup}\mathbb{E}\left[{\int }_0^{\infty }{e}^{-\rho s}(1-{\mu }_s)\tilde{h}\left(1\right)\text{ds}\right]={\rho }^{-1}(1+ϵ)\tilde{h}\left(1\right)\hfill \end{array}$$

(16)

which can be recast as

$$\begin{array}{cc}\hfill -\frac{{\sigma }^2}{2}{(1-y)}^2{V}^{\prime \prime }\left(y\right)+\rho V& =\tilde{h}\left(y\right)+ϵ|(1-y)V^{\prime }\left(y\right)-\tilde{h}\left(y\right)|\hfill \\ \hfill V\left(0\right)& =0\hfill \\ \hfill V\left(1\right)& =\frac{1+ϵ}{\rho }\tilde{h}\left(1\right)\hfill \end{array}$$

(17)

One of the most attractive features of the reduced model (17) is that it has a closed form, piecewise defined analytic solution, which we derive in the Appendix. Here we state the following theorem.

Theorem 4. 

There exists a solution $V^{*}\in C^2\left(\mathcal{Q}\right)\cap C_b\left(\overline{\mathcal{Q}}\right)$ to the reduced model (17). Furthermore, $V^{*}=\check{V}$, and is given by the piecewise solution

$$\begin{array}{c}\hfill V^{*}\left(y\right)=\left\{\begin{array}{cc}{V}_{\ell }\left(y\right),\hfill & 0\le y\le y^{*}\hfill \\ V_r\left(y\right),\hfill & y^{*}\le y\le 1\hfill \end{array}\right.\end{array}$$

(18)

where

$$\begin{array}{}\mathrm{(19)}& \hfill V_{\ell }& =c_1{(1-y)}^{{\gamma }_{-}^{\ell }}+c_2{(1-y)}^{{\gamma }_{+}^{\ell }}+{\alpha }_{\ell }+{\beta }_{\ell }y\hfill \mathrm{(20)}& \hfill V_r& =c_3{(1-y)}^{{\gamma }_{+}^r}+{\alpha }_r+{\beta }_{r}y\hfill \end{array}$$

with constants defined in the Appendix.

Proof. 

The construction of this piecewise solution is found in the Appendix. The solution constructed is monotonically increasing in y. Enforcing the boundary condition on $V^{*}\left(1\right)$ uniformly bounds $V^{*}$ on $\mathcal{Q}$. The proof of equality $V^{*}=\check{V}$ then follows from Corollary IV.5.1 in [15], as in our Theorem 2 above.

Remark 2. 

Since the reduced model has a known closed form solution, we can measure the error made in constructing numerical solutions in Section 4, which in turn acts as a benchmark for the code we use to study the full model. It is uncommon to find a closed form solution for most optimal control problems.

4. Numerical Results

In this section, we construct numerical solutions of the market share scaled model Equation (14), as well as our proposed reduced model Equation (8). In Sections A and A.1, the piecewise analytic solution and a nonlinear equation for the reduced switching point $y^{*}$ are found. These will be used to validate the numerical results for the reduced model, which act as a benchmark for the full problem. Please note that in what follows, the symbol $y^{*}$ is used to denote the switching point in reputational share for both the reduced and full model and their corresponding ODEs.

4.1. Numerical Results for the Reduced Model

The boundary value problem Equation (8) is discretized using a standard finite difference scheme. Let $y_k=k/N_y$, and $V_k=V\left(y_k\right)$, for $k=0,1,\dots ,N_y$. Then we solve a linear system of $N_y+1$ equations of the form $M\mathbf{v}=\mathbf{f}$, where

$$M\mathbf{v}\approx \rho V-\frac{{\sigma }^2}{2}{(1-y)}^2{V}^{\prime \prime },\text{and}\mathbf{f}\approx h+ϵ|(1-y)V^{\prime }-h|.$$

Since $\mathbf{f}$ depends on V, the solution is implicit, and therefore must be obtained by using a fixed point iteration $M{\mathbf{v}}^{(k+1)}={\mathbf{f}}^{\left(k\right)}$. In our numerical experiments, we let $N_y=1000$, and set the maximum iteration count at $K=20$. Convergence is observed in all tested cases. In Figure 1, several numerical solutions $V\left(y\right)$ are shown, for fixed $A=1$, $C=0.15$, $ϵ=0.02$, and various values of $\rho =0.1,0.2,0.5,2.0$ and $\sigma =0.2,0.5,1.0,5.0$. The values of A and C are chosen via $\frac{C}{A}=0.15$ to reflect a maximal 15% reputational premium for sellers above the inherent value A.

Figure 1. A plot of several numerical solutions $V\left(y\right)$ for the reduced boundary value problem Equation (17) are shown with $A=1$, $C=0.15$, and $ϵ=0.02$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 1. A plot of several numerical solutions $V\left(y\right)$ for the reduced boundary value problem Equation (17) are shown with $A=1$, $C=0.15$, and $ϵ=0.02$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

It follows that both ρ and σ have a dramatic effect not only the shape of the solution, but also, as is demonstrated in Figure 2, on the switching point $y^{*}$. These numerical results suggest that both the discount rate ρ and the volatility σ of a seller’s reputation can “dramatically” affect her behavior, particularly the critical reputation at which she switches from the processing to advertising mode.

Figure 2. A plot of the quantity $(1-y)V^{\prime }\left(y\right)$, where $V\left(y\right)$ is a numerical solution of the the reduced boundary value problem Equation (17). The value $y^{*}$ is defined as the intersection of these curves with $h\left(y\right)$ (the solid line). (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 2. A plot of the quantity $(1-y)V^{\prime }\left(y\right)$, where $V\left(y\right)$ is a numerical solution of the the reduced boundary value problem Equation (17). The value $y^{*}$ is defined as the intersection of these curves with $h\left(y\right)$ (the solid line). (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

4.2. Numerical Results for the Full Model

The same numerical discretization is employed to solve the full Nerlove-Arrow model Equation (14), where we solve a linear system of the form $M\mathbf{v}=\mathbf{f}$, with

$$M\mathbf{v}\approx \rho V+\left[\kappa y(1-y)+{\sigma }^2{(1-y)}^3\right]V^{\prime }\left(y\right)-\frac{{\sigma }^2}{2}{(1-y)}^4{V}^{\prime \prime }\left(y\right),\mathbf{f}\approx h+ϵ|{(1-y)}^2{V}^{\prime }-h|$$

We first set $\kappa =0$, and hold all remaining parameters fixed, and plot the results in Figure 3 and Figure 4. The full solutions have more curvature than those of the reduced model, but nonetheless remain monotone, and have a single unique switching point $y^{*}$. The additional curvature is due to the appearance of $V^{\prime }$ terms in the differential operator, which now depend on σ, as well as κ, which we recall incorporates mean reversion. That is, independent of the sellers strategy, reputation will tend to decrease to a smaller amount of the market share, with constant rate κ.

In Figure 5 and Figure 6, the same solutions are shown with $\kappa =1$. Here it becomes apparent that both the rate of mean reversion, as well as the volatility will affect the seller’s optimal strategy.

Remark 3. 

Our numerical results illustrate that a wide range of seller behaviors can be described by varying the parameters, and therefore that reputational value is a strong indicator of optimal seller behavior.

Figure 3. A plot of several numerical solutions $V\left(y\right)$ for the full boundary value problem Equation (14) are shown with $A=1$, $C=0.15$, $ϵ=0.02$, and $\kappa =0$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 3. A plot of several numerical solutions $V\left(y\right)$ for the full boundary value problem Equation (14) are shown with $A=1$, $C=0.15$, $ϵ=0.02$, and $\kappa =0$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 4. A plot of the quantity ${(1-y)}^2{V}^{\prime }$, where V is a numerical solution of the the full boundary value problem Equation (14). The value $y^{*}$ is defined as the intersection of these curves with $h\left(y\right)$ (black solid line). (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 4. A plot of the quantity ${(1-y)}^2{V}^{\prime }$, where V is a numerical solution of the the full boundary value problem Equation (14). The value $y^{*}$ is defined as the intersection of these curves with $h\left(y\right)$ (black solid line). (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 5. The same as Figure 3, but with $\kappa =1$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 5. The same as Figure 3, but with $\kappa =1$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 6. The same as Figure 4, but with $\kappa =1$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

Figure 6. The same as Figure 4, but with $\kappa =1$. (a) $\rho =0.1$; (b) $\rho =0.2$; (c) $\rho =0.5$; (d) $\rho =2.0$.

5. Conclusions and Future Work

In this work, we have proposed a model which describes the behavior of sellers participating in online auctions (markets). Our model is based on the premise that buyer feedback greatly impacts the sales rate, which motivates our assumption that the revenue per sale (or the price per unit) is solely dependent upon seller reputation. We assumed that a seller has a fixed amount of resources that must be allocated either to (i) advertising to new buyers or (ii) processing orders for current customers. In doing so, we were able to design an optimal selling strategy, wherein the seller switches their behavior when an optimal market share reputation is reached, which depends on statistical modeling parameters.

These modeling parameters were introduced through the wealth-reputation mechanisms in [11], which have been generalized to stochastic settings in [1,2,12]. We have additionally modified the empirical model $h\left(R\right)$ found in [3], relating the price per unit to the reputation. Rather than viewing reputation R as an unbounded quantity, we instead introduced a market scaled reputation $Y=R/(1+R)$, where $Y\in [0,1)$, which reduces the price per unit to a simple linear model $h\left(Y\right)=A+CY$.

We then optimized the stochastic model over the control of the excess rate μ using the Hamilton-Jacobi-Bellman equation, yielding a deterministic equation relating the value per unit to the seller reputation. The resulting boundary value problem is then solved numerically, and we qualitatively see that the value of sellers goods increase monotonically with reputation, but that a unique optimal reputation $y=y^{*}$ determines when the seller should switch from advertising to processing. The numerical scheme was validated with a reduced model, which has a closed form piecewise analytic solution, and permits direct determination of $y^{*}$. The numerical results validate our modeling assumptions, and provide a framework for studying seller behavior based on seller reputation. Although the used techniques are standard, we believe that the optimal strategy presented both analytically and numerically has implications on the way reputational information can be used to predict the behavior of an individual seller in an online market.

This work can naturally be generalized in a variety of ways. For instance, the model could be developed in the case of finite horizon time T, since a more realistic assumption is that a seller strategy depends on both time and the current reputation state. Unlike our current infinite horizon model, the resulting HJB equation will lead to a time-dependent partial differential equation, which will be the subject of future work.