# A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background and Motivation

## 3. Underlying Concepts

#### 3.1. QRSE

#### 3.1.1. Deriving Decisions

#### 3.1.2. Deriving Statistical Equilibrium

#### 3.1.3. Limitations of Logit Response

#### 3.2. Thermodynamics of Decision-Making

## 4. Model

#### 4.1. Maximum Entropy Component

#### 4.2. Feedback Between Observed Outcomes and Actions

#### 4.3. Priors and Decisions

#### 4.4. Rolling Prior Beliefs

## 5. Australian Housing Market

#### 5.1. Model

#### 5.1.1. Priors

#### Uniform

#### Previous

#### Mean

#### Extreme Priors

#### 5.2. Results

#### 5.3. Role of Parameters

#### 5.3.1. Decision Temperature

#### 5.3.2. Agent Expectations

#### 5.4. Temporal Effects of Data Granularity on Decisions

## 6. Discussion and Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Derivations

#### Appendix A.1. Decision Duality

#### Appendix A.2. Decision Function

## Appendix B. Australian Housing Market Data

**Figure A2.**Density plots of returns grouped by year. We can see each year follows a different shape, but shows some striking regularities representing a statistical equilibrium.

## Appendix C. Relation to Rational Inattention

## Appendix D. Additional Parameters

#### Appendix D.1. Impact of Decisions on Outcomes

#### Appendix D.2. Skewness

**Figure A5.**Resulting fitted marginals distributions $f\left[x\right]$ for each year. Each coloured line represents a different prior (with the legend given in the top left). The blue bars show the (discretized) actual return distribution.

## Appendix E. Probability Plots

**Figure A6.**Resulting Joint Distributions. Red lines represent $f[\mathrm{sell},x]$, and green lines represent $f[\mathrm{buy},x]$. Each plot from top to bottom shows: Uniform, previous, mean and extreme buy and extreme sell priors (in that order).

**Figure A7.**Decision functions for selling. Buying curves are excluded as they are simply the complement ($1-\mathrm{sell}$). The green lines represent the extreme buy a priori preference, which means the resulting probabilities of selling are shifted far to the right, i.e., the majority of the area comprises buying actions, and only the extreme positive growth rates for sell. In contrast, the red lines represent the sell preference, which “pulls” the area to the left, resulting in a strong resulting conditional preference for selling.

## References

- Simon, H.A. Models of Man; Social And Rational; Wiley: Hoboken, NJ, USA, 1957. [Google Scholar]
- Sims, C.A. Implications of rational inattention. J. Monet. Econ.
**2003**, 50, 665–690. [Google Scholar] [CrossRef][Green Version] - Matějka, F.; McKay, A. Rational inattention to discrete choices: A new foundation for the multinomial logit model. Am. Econ. Rev.
**2015**, 105, 272–298. [Google Scholar] [CrossRef][Green Version] - Ortega, P.A.; Braun, D.A. Thermodynamics as a theory of decision-making with information-processing costs. Proc. R. Soc. A Math. Phys. Eng. Sci.
**2013**, 469, 20120683. [Google Scholar] [CrossRef] - Scharfenaker, E.; Foley, D.K. Quantal response statistical equilibrium in economic interactions: Theory and estimation. Entropy
**2017**, 19, 444. [Google Scholar] [CrossRef] - Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 620. [Google Scholar] [CrossRef] - Yang, J. Information theoretic approaches in economics. J. Econ. Surv.
**2018**, 32, 940–960. [Google Scholar] [CrossRef] - Yakovenko, V.M. Econophysics, Statistical Mechanics Approach to. In Encyclopedia of Complexity and Systems Science; Meyers, R.A., Ed.; Springer: New York, NY, USA, 2009; pp. 2800–2826. [Google Scholar]
- Scharfenaker, E.; Semieniuk, G. A statistical equilibrium approach to the distribution of profit rates. Metroeconomica
**2017**, 68, 465–499. [Google Scholar] [CrossRef] - Scharfenaker, E.; Yang, J. Maximum entropy economics. Eur. Phys. J. Spec. Top.
**2020**, 229, 1577–1590. [Google Scholar] [CrossRef] - Wolpert, D.H.; Harré, M.; Olbrich, E.; Bertschinger, N.; Jost, J. Hysteresis effects of changing the parameters of noncooperative games. Phys. Rev. E
**2012**, 85, 036102. [Google Scholar] [CrossRef][Green Version] - Dragulescu, A.; Yakovenko, V.M. Statistical mechanics of money. Eur. Phys. J. Condens. Matter Complex Syst.
**2000**, 17, 723–729. [Google Scholar] [CrossRef][Green Version] - Yakovenko, V.M.; Rosser Jr, J.B. Colloquium: Statistical mechanics of money, wealth, and income. Rev. Mod. Phys.
**2009**, 81, 1703. [Google Scholar] [CrossRef][Green Version] - Foley, D.K. Unfulfilled Expectations: One Economist’s History. In Expectations; Springer: Berlin/Heidelberg, Germany, 2020; pp. 3–17. [Google Scholar]
- Harré, M.S. Information Theory for Agents in Artificial Intelligence, Psychology, and Economics. Entropy
**2021**, 23, 310. [Google Scholar] [CrossRef] [PubMed] - Foley, D.K. Information theory and behavior. Eur. Phys. J. Spec. Top.
**2020**, 229, 1591–1602. [Google Scholar] [CrossRef] - Ömer, Ö. Maximum entropy approach to market fluctuations as a promising alternative. Eur. Phys. J. Spec. Top.
**2020**, 229, 1715–1733. [Google Scholar] [CrossRef] - Yang, J.; Carro, A. Two tales of complex system analysis: MaxEnt and agent-based modeling. Eur. Phys. J. Spec. Top.
**2020**, 229, 1623–1643. [Google Scholar] [CrossRef] - Jaynes, E.T. Information theory and statistical mechanics. II. Phys. Rev.
**1957**, 108, 171. [Google Scholar] [CrossRef] - Jaynes, E.T. Probability Theory: The Logic of Science; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Scharfenaker, E. Implications of quantal response statistical equilibrium. J. Econ. Dyn. Control.
**2020**, 119, 103990. [Google Scholar] [CrossRef] - Ömer, Ö. Dynamics of the US Housing Market: A Quantal Response Statistical Equilibrium Approach. Entropy
**2018**, 20, 831. [Google Scholar] [CrossRef][Green Version] - Ömer, Ö. Essays on Modeling Housing Markets, Income Distribution, and Wealth Concentration. Ph.D. Thesis, The New School, New York, NY, USA, 2018. [Google Scholar]
- Ömer, Ö. Equilibrium-Disequilibrium Dynamics of the US Housing Market, 2000–2015: A Quantal Response Statistical Equilibrium Approach. Working Papers 1809, New School for Social Research, Department of Economics. 2018. Available online: https://econpapers.repec.org/paper/newwpaper/1809.htm (accessed on 30 September 2020).
- Yang, J. A quantal response statistical equilibrium model of induced technical change in an interactive factor market: Firm-level evidence in the EU economies. Entropy
**2018**, 20, 156. [Google Scholar] [CrossRef][Green Version] - Wiener, N. Measuring Labor Market Segmentation from Incomplete Data. Working Paper 2018-01, Amherst, MA. 2018. Available online: https://scholarworks.umass.edu/econworkingpaper/238/ (accessed on 3 October 2020).
- Wiener, N. Essays on Labor Mobility and Segmentation. Ph.D. Thesis, The New School, New York, NY, USA, 2019. [Google Scholar]
- Wiener, N.M. Labor market segmentation and immigrant competition: A quantal response statistical equilibrium analysis. Entropy
**2020**, 22, 742. [Google Scholar] [CrossRef] - Blackwell, K. A Behavioral Foundation for Commonly Observed Distributions of Financial and Economic Data. Working Papers 1912, New School for Social Research, Department of Economics. 2019. Available online: https://ideas.repec.org/p/new/wpaper/1912.html (accessed on 8 October 2020).
- Blackwell, K. Entropy Constrained Behavior in Financial Markets A Quantal Response Statistical Equilibrium Approach to Financial Modeling. Ph.D. Thesis, The New School, New York, NY, USA, 2018. [Google Scholar]
- Scharfenaker, E. Statistical Equilibrium Methods in Analytical Political Economy. J. Econ. Surv.
**2020**. [Google Scholar] [CrossRef] - Smith, A. The Wealth of Nations: An inquiry into the nature and causes of the Wealth of Nations; Harriman House Limited: Petersfield, UK, 2010. [Google Scholar]
- McKelvey, R.D.; Palfrey, T.R. Quantal response equilibria for normal form games. Games Econ. Behav.
**1995**, 10, 6–38. [Google Scholar] [CrossRef] - McKelvey, R.D.; Palfrey, T.R. Quantal response equilibria for extensive form games. Exp. Econ.
**1998**, 1, 9–41. [Google Scholar] [CrossRef] - Lord, C.G.; Ross, L.; Lepper, M.R. Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. J. Personal. Soc. Psychol.
**1979**, 37, 2098. [Google Scholar] [CrossRef] - K Levine, D. Is Behavioral Economics Doomed?: The Ordinary Versus the Extraordinary; Open Book Publishers: Cambridge, UK, 2012. [Google Scholar]
- DellaVigna, S. Psychology and economics: Evidence from the field. J. Econ. Lit.
**2009**, 47, 315–372. [Google Scholar] [CrossRef][Green Version] - Daunizeau, J.; Den Ouden, H.E.; Pessiglione, M.; Kiebel, S.J.; Stephan, K.E.; Friston, K.J. Observing the observer (I): Meta-bayesian models of learning and decision-making. PLoS ONE
**2010**, 5, e15554. [Google Scholar] [CrossRef] [PubMed] - Khalvati, K.; Park, S.A.; Mirbagheri, S.; Philippe, R.; Sestito, M.; Dreher, J.C.; Rao, R.P. Modeling other minds: Bayesian inference explains human choices in group decision-making. Sci. Adv.
**2019**, 5, eaax8783. [Google Scholar] [CrossRef] [PubMed][Green Version] - Kruis, J.; Maris, G.; Marsman, M.; Bolsinova, M.; van der Maas, H.L. Deviations of rational choice: An integrative explanation of the endowment and several context effects. Sci. Rep.
**2020**, 10, 1–16. [Google Scholar] [CrossRef] - Debreu, G. Review of individual choice behavior by RD Luce. Am. Econ. Rev.
**1960**, 50, 186–188. [Google Scholar] - McFadden, D. Conditional logit analysis of qualitative choice behavior. In Frontiers in Econometrics; Zarembka, P., Ed.; Academic Press: Cambridge, MA, USA, 1973; pp. 105–142. [Google Scholar]
- Golan, A. Foundations of Info-Metrics: Modeling, Inference, and Imperfect Information; Oxford University Press: Oxford, UK, 2018. [Google Scholar]
- Hafner, D.; Ortega, P.A.; Ba, J.; Parr, T.; Friston, K.; Heess, N. Action and perception as divergence minimization. arXiv
**2020**, arXiv:2009.01791. [Google Scholar] - Ortega, P.A.; Stocker, A.A. Human decision-making under limited time. In Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; pp. 100–108. [Google Scholar]
- Gottwald, S.; Braun, D.A. The two kinds of free energy and the Bayesian revolution. PLoS Comput. Biol.
**2020**, 16, e1008420. [Google Scholar] [CrossRef] [PubMed] - Wilson, A. Boltzmann, Lotka and Volterra and spatial structural evolution: An integrated methodology for some dynamical systems. J. R. Soc. Interface
**2008**, 5, 865–871. [Google Scholar] [CrossRef] - Crosato, E.; Nigmatullin, R.; Prokopenko, M. On critical dynamics and thermodynamic efficiency of urban transformations. R. Soc. Open Sci.
**2018**, 5, 180863. [Google Scholar] [CrossRef][Green Version] - Slavko, B.; Glavatskiy, K.; Prokopenko, M. Dynamic resettlement as a mechanism of phase transitions in urban configurations. Phys. Rev. E
**2019**, 99, 042143. [Google Scholar] [CrossRef] - Harding, N.; Spinney, R.E.; Prokopenko, M. Population mobility induced phase separation in SIS epidemic and social dynamics. Sci. Rep.
**2020**, 10, 1–11. [Google Scholar] [CrossRef] - Shore, J.; Johnson, R. Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Inf. Theory
**1980**, 26, 26–37. [Google Scholar] [CrossRef][Green Version] - Kesavan, H.; Kapur, J. Maximum Entropy and Minimum Cross-Entropy Principles: Need for a Broader Perspective. In Maximum Entropy and Bayesian Methods; Springer: Berlin/Heidelberg, Germany, 1990; pp. 419–432. [Google Scholar]
- Glavatskiy, K.S.; Prokopenko, M.; Carro, A.; Ormerod, P.; Harre, M. Explaining herding and volatility in the cyclical price dynamics of urban housing markets using a large-scale agent-based model. SN Bus. Econ.
**2021**, 1, 1–21. [Google Scholar] [CrossRef] - Evans, B.P.; Glavatskiy, K.; Harré, M.S.; Prokopenko, M. The impact of social influence in Australian real estate: Market forecasting with a spatial agent-based model. J. Econ. Interact. Coord.
**2021**, 1–53. [Google Scholar] - Hommes, C.H. On the consistency of backward-looking expectations: The case of the cobweb. J. Econ. Behav. Organ.
**1998**, 33, 333–362. [Google Scholar] [CrossRef] - Genewein, T.; Leibfried, F.; Grau-Moya, J.; Braun, D.A. Bounded rationality, abstraction, and hierarchical decision-making: An information-theoretic optimality principle. Front. Robot. AI
**2015**, 2, 27. [Google Scholar] [CrossRef][Green Version] - Friedman, M. Theory of the Consumption Function; Princeton University Press: Princeton, NJ, USA, 2018. [Google Scholar]
- Hommes, C.; Wagener, F. Complex evolutionary systems in behavioral finance. In Handbook of Financial Markets: Dynamics and Evolution; Elsevier: Amsterdam, The Netherlands, 2009; pp. 217–276. [Google Scholar]
- Evans, G.W.; Honkapohja, S. Learning and Expectations in Macroeconomics; Princeton University Press: Princeton, NJ, USA, 2012. [Google Scholar]
- Chow, G.C. Usefulness of Adaptive and Rational Expectations in Economics; Center for Economic Policy Studies, Princeton University: Princeton, NJ, USA, 2011. [Google Scholar]
- Soofi, E.S.; Retzer, J.J. Information indices: Unification and applications. J. Econom.
**2002**, 107, 17–40. [Google Scholar] [CrossRef] - Dziewulski, P. Just-noticeable difference as a behavioural foundation of the critical cost-efficiency index. J. Econ. Theory
**2020**, 188, 105071. [Google Scholar] [CrossRef] - Pfeifer, R.; Bongard, J. How the Body Shapes the Way We Think: A New View of Intelligence; MIT Press: Cambridge, MA, USA, 2006. [Google Scholar]
- Polani, D.; Sporns, O.; Lungarella, M. How information and embodiment shape intelligent information processing. In 50 Years of Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2007; pp. 99–111. [Google Scholar]
- Ay, N.; Bernigau, H.; Der, R.; Prokopenko, M. Information-driven self-organization: The dynamical system approach to autonomous robot behavior. Theory Biosci.
**2012**, 131, 161–179. [Google Scholar] [CrossRef] - Montúfar, G.; Ghazi-Zahedi, K.; Ay, N. A theory of cheap control in embodied systems. PLoS Comput. Biol.
**2015**, 11, e1004427. [Google Scholar] [CrossRef][Green Version] - Polani, D.; Nehaniv, C.L.; Martinetz, T.; Kim, J.T. Relevant information in optimized persistence vs. progeny strategies. In Proceedings of the Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, Bloomington, IN, USA, 3–6 June 2006; MIT Press: Cambridge, MA, USA, 2006. [Google Scholar]
- Prokopenko, M.; Gerasimov, V.; Tanev, I. Measuring spatiotemporal coordination in a modular robotic system. In Proceedings of the Artificial Life X: Proceedings of the 10th International Conference on the Simulation and Synthesis of Living Systems, Bloomington, IN, USA, 3–6 June 2006; pp. 185–191. [Google Scholar]
- Capdepuy, P.; Polani, D.; Nehaniv, C.L. Maximization of potential information flow as a universal utility for collective behaviour. In Proceedings of the 2007 IEEE Symposium on Artificial Life, Honolulu, HI, USA, 1–5 April 2007; pp. 207–213. [Google Scholar]
- Tishby, N.; Polani, D. Information theory of decisions and actions. In Perception-Action Cycle; Springer: Berlin/Heidelberg, Germany, 2011; pp. 601–636. [Google Scholar]
- Tishby, N.; Pereira, F.C.; Bialek, W. The information bottleneck method. arXiv
**2000**, arXiv:physics/0004057. [Google Scholar] - Spinney, R.E.; Lizier, J.T.; Prokopenko, M. Transfer entropy in physical systems and the arrow of time. Phys. Rev. E
**2016**, 94, 022135. [Google Scholar] [CrossRef] [PubMed][Green Version] - Caplin, A.; Dean, M.; Leahy, J. Rational inattention, optimal consideration sets, and stochastic choice. Rev. Econ. Stud.
**2019**, 86, 1061–1094. [Google Scholar] [CrossRef][Green Version]

**Figure 1.**The effect of decision temperature T on the resulting expected payoffs (

**a**). for the limits given by Equation (17). The inverse temperature $\frac{1}{T}$ (

**b**) conveys the same information but may offer a more useful visualisation due to the continuity.

**Figure 2.**Decision Functions. All cases have equivalent utility functions. Each row has equivalent temperatures, showing how with matched parameters and utility, having an alternate prior can shift the decision-makers preference. Each column has different priors, given along the top of the first row to show how decision-makers decisions change based on their prior beliefs. On the left-hand side, preference is shifted towards the buying case. Likewise, on the right-hand side, preference is given to the selling case. The uniform case with equal preference is shown in the middle.

**Figure 3.**In the three-action case, the priors can introduce asymmetries by biasing the decision functions. This allows for separate indifferent points (

**right**) vs. the uniform priors implying a single intersect (

**left**).

**Figure 4.**Resulting marginal probabilities $f\left[a\right]$ for varying priors. Green represents $f\left[\mathrm{buy}\right]$, and red represents $f\left[\mathrm{sell}\right]$.

**Table 1.**Resuling likelihood and percentage of variability explained for each year, when compared to the actual underlying distribution (i.e., those given in Figure A2). Optimisation is done by minimising the negative log-likelihood between the resulting distributions and the actual distribution of returns.

Uniform | Previous | Mean | Extreme Buy | Extreme Sell | |
---|---|---|---|---|---|

2006 | 1082 (93%) | 1082 (93%) | 1082 (93%) | 885 (59%) | 1005 (74%) |

2007 | 1089 (92%) | 1089 (92%) | 1090 (90%) | 939 (68%) | 1042 (83%) |

2008 | 998 (95%) | 905 (78%) | 998 (95%) | 998 (95%) | 998 (95%) |

2009 | 918 (96%) | 918 (96%) | 866 (88%) | 880 (85%) | 875 (85%) |

2010 | 857 (95%) | 857 (95%) | 857 (95%) | 740 (62%) | 857 (95%) |

2011 | 1045 (92%) | 1044 (91%) | 1047 (92%) | 1045 (91%) | 873 (62%) |

2012 | 1067 (96%) | 1067 (96%) | 1067 (96%) | 162 (6%) | 142 (8%) |

2013 | 1080 (90%) | 1076 (90%) | 1083 (90%) | 983 (77%) | 1075 (91%) |

2014 | 938 (98%) | 851 (74%) | 938 (98%) | 875 (71%) | 938 (98%) |

2015 | 860 (96%) | 860 (96%) | 860 (96%) | 33 (10%) | 808 (71%) |

2016 | 873 (84%) | 932 (95%) | 908 (86%) | 817 (70%) | 932 (95%) |

2017 | 916 (97%) | 916 (97%) | 916 (97%) | 812 (76%) | 916 (97%) |

2018 | 989 (88%) | 932 (85%) | 933 (85%) | 955 (82%) | 998 (91%) |

2019 | 1101 (92%) | 1103 (92%) | 1067 (94%) | 1101 (92%) | 952 (76%) |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Evans, B.P.; Prokopenko, M.
A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback. *Entropy* **2021**, *23*, 669.
https://doi.org/10.3390/e23060669

**AMA Style**

Evans BP, Prokopenko M.
A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback. *Entropy*. 2021; 23(6):669.
https://doi.org/10.3390/e23060669

**Chicago/Turabian Style**

Evans, Benjamin Patrick, and Mikhail Prokopenko.
2021. "A Maximum Entropy Model of Bounded Rational Decision-Making with Prior Beliefs and Market Feedback" *Entropy* 23, no. 6: 669.
https://doi.org/10.3390/e23060669