How Active Inference Could Help Revolutionise Robotics
1. Active Inference
2. Solutions to Technical Challenges in Robotics
- Accurate and robust state tracking. Filtering schemes developed for neuroimaging time-series  enable accurate state-tracking in highly complex and volatile environments [27,36]. This allows for continuous refinement of past, present, future state-estimation and the estimated precision of sensors as new information arrives  (c.f., Bayes optimal estimators of Kalman gain ). Moreover, AIF fuses multiple sensory streams, by weighing incoming sensory information by their estimated precision [36,39]. This enables accurate and robust inferences.
- Adaptive model-based and shared control. Describing the agent’s behaviour with a generative model—prescribing attracting states and trajectories—ensures robustness and adaptivity in the presence of noise, external fluctuations, and parameter changes. AIF humanoid robots  and industrial manipulators  show improved behaviour in the presence of internal and external parameter changes  and shared compliance control . The robot’s autonomy—in shared control—can also be dynamically tuned. In particular, the operator may be given high-level control and the robot low-level control.
- Learning and grounding. AIF agents learn from sparse and noisy observations by actively sampling informative data points, enabling few-shot learning. Learning latent structure by optimising model evidence, subject to prior preferences in the generative model, leads to organising knowledge in hierarchical, sparsely interconnected modular (i.e., factorised) representations with temporal depth, usually represented with a graphical model . This offers a promising pathway for biologically plausible neurosymbolic technologies [42,43].
- Operational specification, safety and explainability. AIF behaviour is explainable as a mixture of information and goal-seeking policies that are explicitly encoded (and evaluated in terms of expected free energy) in the generative model as priors—which can be specified by the user. Planning, which proceeds by generating counterfactual actions and assessing their consequences , can be monitored online and control can be returned to the user if necessary (i.e., policy switching). Moreover, the generative model can be specified as a directed graph (i.e., a Bayesian network), which entails the causal relationships between agent’s representations [44,45]. This affords an explicit and transparent explanation of sentient behaviour.
3. Practical Perspectives
- Context adaptive robots. AIF agents build generative (world) models by continuously optimising free energy with regard to incoming data. This optimisation process maximises model accuracy while minimising complexity, which enables generalisation and context-adaptivity . Contrariwise, robots that solely optimise accuracy risk overfitting, which could lead to catastrophic outcomes when the context changes, such as when performing assistive surgery on a new patient. The ability to generalise and adapt is necessary for robotic skills such as scene understanding and adaptive control and should facilitate robots to operate in volatile (e.g., social) environments (e.g., hospitals) [36,46]. In industrial applications, this allows robots to operate freely while adapting to real world conditions—once the designer has specified preferences over the final outcome.
- Safer robots. AIF agents continuously resolve uncertainty by selecting informative actions that minimise risk , which is important for high-stakes, high-uncertainty tasks, such as human-robot interaction . Actions are selected to minimise expected free energy, which minimises risk (expected cost) and ambiguity (expected inaccuracy) . This allows for information seeking behaviour that is accompanied with an explicit and quantifiable measure of risk. Additionally, when uncertain about current states of affair, robots should automatically seek advice and guidance from the user, e.g., via shared control.
- Social and collaborative robots. AIF robots model others’ intentions to predict others’ actions, such as movements , enabling intentional understanding . This allows robots to operate safely in social environments by constantly resolving uncertainty about others’ intentions and implicit goals . This embodiment  is crucial for social robots, such as personal aides, auxiliary robot nurses and companions, e.g., assisting the disabled and elderly. In collaborative robotics, AIF allows for imitation learning and intentional blending, whence robot goals and intentions can be guided by the user before and during the task [41,50].
- Wearable devices. The belief updating process that underwrites AIF is energetically efficient , which should aid the development of wearable devices with a degree of autonomy, such as exoskeletons . This follows as optimising model free energy decreases the movement from prior to posterior, which corresponds to the computational (and hence energetic) cost of inference [1,2]. In addition, wearables directed by human intention  should benefit from AIF’s intentional understanding , and adaptive and shared control capabilities .
- Regulatory processes. Generative models with temporal depth induce allostatic control, whence the robot acts on its environment to pre-empt homeostatic control [54,55]. This should benefit regulatory processes subject to strong external perturbations [16,36], such as closed-loop medical applications such as artificial organs (e.g., the artificial pancreas).
- Neurotechnology. The neurological functional plausibility of specific AIF algorithms [1,46,56] should facilitate integration with the nervous system. This opens new opportunities for neurotechnology, BCI-enabled sensorimotor restoration, perceptual body extension and brain or body enhancement using prosthetics and implants . Currently, AIF provides testable hypotheses for optimising neural excitatory-inhibitory balance using deep brain stimulation to alleviate functional deficits induced by brain lesions . Soon, monitoring of brain activity may predict aberrant neural responses, such as seizures, and anticipate the required intervention.
Data Availability Statement
Conflicts of Interest
- Da Costa, L.; Parr, T.; Sajid, N.; Veselic, S.; Neacsu, V.; Friston, K. Active inference on discrete state-spaces: A synthesis. J. Math. Psychol. 2020, 99, 102447. [Google Scholar] [CrossRef] [PubMed]
- Barp, A.; Da Costa, L.; França, G.; Friston, K.; Girolami, M.; Jordan, M.I.; Pavliotis, G.A. Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents. In Geometry and Statistics; Academic Press: Cambridge, MA, USA, 2022. [Google Scholar]
- Friston, K.J.; Daunizeau, J.; Kilner, J.; Kiebel, S.J. Action and behavior: A free-energy formulation. Biol. Cybern. 2010, 102, 227–260. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Buckley, C.L.; Kim, C.S.; McGregor, S.; Seth, A.K. The free energy principle for action and perception: A mathematical review. J. Math. Psychol. 2017, 81, 55–79. [Google Scholar] [CrossRef]
- Friston, K. The free-energy principle: A unified brain theory? Nat. Rev. Neurosci. 2010, 11, 127–138. [Google Scholar] [CrossRef] [PubMed]
- Friston, K.; Kilner, J.; Harrison, L. A free energy principle for the brain. J. Physiol. 2006, 100, 70–87. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Friston, K.; Da Costa, L.; Sajid, N.; Heins, C.; Ueltzhöffer, K.; Pavliotis, G.A.; Parr, T. The free energy principle made simpler but not too simple. arXiv 2022, arXiv:2201.06387. [Google Scholar]
- Parr, T. The Computational Neurology of Active Vision. Ph.D. Thesis, University College London, London, UK, 2019. [Google Scholar]
- Millidge, B. Deep active inference as variational policy gradients. J. Math. Psychol. 2020, 96, 102348. [Google Scholar] [CrossRef][Green Version]
- Fountas, Z.; Sajid, N.; Mediano, P.A.M.; Friston, K. Deep active inference agents using Monte-Carlo methods. arXiv 2020, arXiv:2006.04176. [Google Scholar]
- Tschantz, A.; Millidge, B.; Seth, A.K.; Buckley, C.L. Reinforcement Learning through Active Inference. arXiv 2020, arXiv:2002.12636. [Google Scholar]
- Sajid, N.; Tigas, P.; Zakharov, A.; Fountas, Z.; Friston, K. Exploration and preference satisfaction trade-off in reward-free learning. arXiv 2021, arXiv:2106.04316. [Google Scholar]
- Mazzaglia, P.; Verbelen, T.; Dhoedt, B. Contrastive Active Inference. Available online: https://openreview.net/forum?id=5t5FPwzE6mq (accessed on 18 February 2022).
- Lanillos, P.; Meo, C.; Pezzato, C.; Meera, A.A.; Baioumy, M.; Ohata, W.; Tschantz, A.; Millidge, B.; Wisse, M.; Buckley, C.L.; et al. Active Inference in Robotics and Artificial Agents: Survey and Challenges. arXiv 2021, arXiv:2112.01871. [Google Scholar]
- Blei, D.M.; Kucukelbir, A.; McAuliffe, J.D. Variational Inference: A Review for Statisticians. J. Am. Stat. Assoc. 2017, 112, 859–877. [Google Scholar] [CrossRef][Green Version]
- Meo, C.; Lanillos, P. Multimodal VAE Active Inference Controller. In Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech, 27 September–1 October 2021; pp. 2693–2699. [Google Scholar] [CrossRef]
- Silver, D.; Huang, A.; Maddison, C.J.; Guez, A.; Sifre, L.; van den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016, 529, 484–489. [Google Scholar] [CrossRef] [PubMed]
- Vinyals, O.; Babuschkin, I.; Czarnecki, W.M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D.H.; Powell, R.; Ewalds, T.; Georgiev, P.; et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 2019, 575, 350–354. [Google Scholar] [CrossRef] [PubMed]
- Van Der Himst, O.; Lanillos, P. Deep Active Inference for Partially Observable MDPs. In Active Inference, IWAI 2020, Communications in Computer and Information Science; Verbelen, T., Lanillos, P., Buckley, C.L., De Boom, C., Eds.; Springer: Cham, Switzerland, 2020; Volume 1326. [Google Scholar] [CrossRef]
- Cullen, M.; Davey, B.; Friston, K.J.; Moran, R. Active Inference in OpenAI Gym: A Paradigm for Computational Investigations Into Psychiatric Illness. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 2018, 3, 809–818. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Sajid, N.; Ball, P.J.; Parr, T.; Friston, K.J. Active Inference: Demystified and Compared. Neural Comput. 2021, 33, 674–712. [Google Scholar] [CrossRef]
- Marković, D.; Stojić, H.; Schwöbel, S.; Kiebel, S.J. An empirical evaluation of active inference in multi-armed bandits. Neural Netw. 2021, 144, 229–246. [Google Scholar] [CrossRef]
- Paul, A.; Sajid, N.; Gopalkrishnan, M.; Razi, A. Active Inference for Stochastic Control. arXiv 2021, arXiv:2108.12245. [Google Scholar]
- Friston, K.; Da Costa, L.; Hafner, D.; Hesp, C.; Parr, T. Sophisticated Inference. Neural Comput. 2021, 33, 713–763. [Google Scholar] [CrossRef]
- Smith, R.; Friston, K.J.; Whyte, C.J. A step-by-step tutorial on active inference and its application to empirical data. J. Math. Psychol. 2022, 107, 102632. [Google Scholar] [CrossRef]
- Friston, K.; Trujillo-Barreto, N.; Daunizeau, J. DEM: A variational treatment of dynamic systems. NeuroImage 2008, 41, 849–885. [Google Scholar] [CrossRef] [PubMed]
- Meera, A.A.; Wisse, M. Free Energy Principle Based State and Input Observer Design for Linear Systems with Colored Noise. In Proceedings of the 2020 American Control Conference (ACC), Denver, CO, USA, 1–3 July 2020; pp. 5052–5058. [Google Scholar] [CrossRef]
- Baltieri, M.; Isomura, T. Kalman filters as the steady-state solution of gradient descent on variational free energy. arXiv 2021, arXiv:2111.10530. [Google Scholar]
- da Costa, L.; Sajid, N.; Parr, T.; Friston, K.; Smith, R. The relationship between dynamic programming and active inference: The discrete, finite-horizon case. arXiv 2020, arXiv:2009.08111. [Google Scholar]
- Imohiosen, A.; Watson, J.; Peters, J. Active Inference or Control as Inference? A Unifying View. arXiv 2020, arXiv:2010.00262. [Google Scholar]
- Millidge, B.; Tschantz, A.; Seth, A.K.; Buckley, C.L. On the Relationship Between Active Inference and Control as Inference. In International Workshop on Active Inference; Springer: Cham, Switzerland, 2020; pp. 3–11. [Google Scholar]
- Sajid, N.; da Costa, L.; Parr, T.; Friston, K. Active inference, Bayesian optimal design, and expected utility. arXiv 2021, arXiv:2110.04074. [Google Scholar]
- Smith, R.; Schwartenbeck, P.; Parr, T.; Friston, K. An Active Inference Approach to Modeling Structure Learning: Concept Learning as an Example Case. Front. Comput. Neurosci. 2020, 14, 41. [Google Scholar] [CrossRef]
- Çatal, O.; Wauthier, S.; De Boom, C.; Verbelen, T.; Dhoedt, B. Learning Generative State Space Models for Active Inference. Front. Comput. Neurosci. 2020, 14, 103. [Google Scholar] [CrossRef]
- Friston, K.J.; Lin, M.; Frith, C.D.; Pezzulo, G.; Hobson, J.A.; Ondobaka, S. Active Inference, Curiosity and Insight. Neural Comput. 2017, 29, 2633–2683. [Google Scholar] [CrossRef]
- Oliver, G.; Lanillos, P.; Cheng, G. An empirical study of active inference on a humanoid robot. IEEE Trans. Cogn. Dev. Syst. 2021, 4, 1–17. [Google Scholar] [CrossRef]
- Meera, A.A.; Wisse, M. Dynamic Expectation Maximization Algorithm for Estimation of Linear Systems with Colored Noise. Entropy 2021, 23, 1306. [Google Scholar] [CrossRef]
- Baltieri, M.; Buckley, C.L. PID Control as a Process of Active Inference with Linear Generative Models. Entropy 2019, 21, 257. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Lanillos, P.; Cheng, G. Adaptive Robot Body Learning and Estimation Through Predictive Coding. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 4083–4090. [Google Scholar] [CrossRef][Green Version]
- Pezzato, C.; Ferrari, R.M.G.; Corbato, C.H. A Novel Adaptive Controller for Robot Manipulators Based on Active Inference. IEEE Robot. Autom. Lett. 2020, 5, 2973–2980. [Google Scholar] [CrossRef][Green Version]
- Chame, H.F.; Tani, J. Cognitive and motor compliance in intentional human-robot interaction. In Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May–31 August 2020; pp. 11291–11297. [Google Scholar] [CrossRef]
- Friston, K.; Moran, R.J.; Nagai, Y.; Taniguchi, T.; Gomi, H.; Tenenbaum, J. World model learning and inference. Neural Netw. 2021, 144, 573–590. [Google Scholar] [CrossRef] [PubMed]
- Taniguchi, T.; Piater, J.; Worgotter, F.; Ugur, E.; Hoffmann, M.; Jamone, L.; Nagai, T.; Rosman, B.; Matsuka, T.; Iwahashi, N.; et al. Symbol Emergence in Cognitive Developmental Systems: A Survey. IEEE Trans. Cogn. Dev. Syst. 2018, 11, 494–516. [Google Scholar] [CrossRef][Green Version]
- Pearl, J. Graphical Models for Probabilistic and Causal Reasoning. In Quantified Representation of Uncertainty and Imprecision; Smets, P., Ed.; Springer: Dordrecht, The Netherlands, 1998; pp. 367–389. [Google Scholar] [CrossRef][Green Version]
- Friston, K.J.; Parr, T.; De Vries, B. The graphical brain: Belief propagation and active inference. Netw. Neurosci. 2017, 1, 381–414. [Google Scholar] [CrossRef]
- Verbelen, T.; Lanillos, P.; Buckley, C.; Boom, C.D. (Eds.) Active Inference. In Proceedings of the First International Workshop, IWAI 2020, Co-Located with ECML/PKDD 2020, Ghent, Belgium, 14 September 2020; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar] [CrossRef]
- Wirkuttis, N.; Tani, J. Leading or Following? Dyadic Robot Imitative Interaction Using the Active Inference Framework. IEEE Robot. Autom. Lett. 2021, 6, 6024–6031. [Google Scholar] [CrossRef]
- Horii, T.; Nagai, Y. Active Inference Through Energy Minimization in Multimodal Affective Human–Robot Interaction. Front. Robot. AI 2021, 8, 684401. [Google Scholar] [CrossRef]
- Lanillos, P.; Pages, J.; Cheng, G. Robot Self/Other Distinction: Active Inference Meets Neural Networks Learning in a Mirror. In Proceedings of the ECAI 2020-24th European Conference on Artificial Intelligence, Compostela, Spain, 29 August–8 September 2020; IOS Press: Amsterdam, The Netherlands, 2020; pp. 2410–2416. [Google Scholar] [CrossRef]
- Shin, J.Y.; Kim, C.; Hwang, H.J. Prior preference learning from experts: Designing a reward with active inference. Neurocomputing 2021, 12, 42. [Google Scholar] [CrossRef]
- Friston, K. Complexity and Computation in the Brain: The Knowns and the Known Unknowns. Available online: https://direct.mit.edu/books/book/4588/chapter/204732/Complexity-and-Computation-in-the-Brain-The-Knowns (accessed on 18 February 2022).
- Lanillos, P.; van Gerven, M. Neuroscience-inspired perception-action in robotics: Applying active inference for state estimation, control and self-perception. arXiv 2021, arXiv:2105.04261. [Google Scholar]
- Ajoudani, A.; Zanchettin, A.M.; Ivaldi, S.; Albu-Schäffer, A.; Kosuge, K.; Khatib, O. Progress and prospects of the human–robot collaboration. Auton. Robot. 2018, 42, 957–975. [Google Scholar] [CrossRef][Green Version]
- Çatal, O.; Verbelen, T.; Van de Maele, T.; Dhoedt, B.; Safron, A. Robot navigation as hierarchical active inference. Neural Netw. 2021, 142, 192–204. [Google Scholar] [CrossRef]
- Tschantz, A.; Barca, L.; Maisto, D.; Buckley, C.L.; Seth, A.K.; Pezzulo, G. Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference. Biol. Psychol. 2022, 169, 108266. [Google Scholar] [CrossRef]
- Friston, K.J.; Fitzgerald, T.H.B.; Rigoli, F.; Schwartenbeck, P.; Pezzulo, G. Active Inference: A Process Theory. Neural Comput. 2017, 29, 1–49. [Google Scholar] [CrossRef][Green Version]
- Cheng, G.; Ehrlich, S.K.; Lebedev, M.; Nicolelis, M.A.L. Neuroengineering challenges of fusing robotics and neuroscience. Sci. Robot. 2020, 5, eabd1911. [Google Scholar] [CrossRef]
- Sajid, N.; Parr, T.; Gajardo-Vidal, A.; Price, C.J.; Friston, K.J. Paradoxical lesions, plasticity and active inference. Brain Commun. 2020, 2, fcaa164. [Google Scholar] [CrossRef]
- Tschantz, A.; Baltieri, M.; Seth, A.K.; Buckley, C.L. Scaling active inference. arXiv 2019, arXiv:1911.10601. [Google Scholar]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Da Costa, L.; Lanillos, P.; Sajid, N.; Friston, K.; Khan, S. How Active Inference Could Help Revolutionise Robotics. Entropy 2022, 24, 361. https://doi.org/10.3390/e24030361
Da Costa L, Lanillos P, Sajid N, Friston K, Khan S. How Active Inference Could Help Revolutionise Robotics. Entropy. 2022; 24(3):361. https://doi.org/10.3390/e24030361Chicago/Turabian Style
Da Costa, Lancelot, Pablo Lanillos, Noor Sajid, Karl Friston, and Shujhat Khan. 2022. "How Active Inference Could Help Revolutionise Robotics" Entropy 24, no. 3: 361. https://doi.org/10.3390/e24030361