Next Article in Journal
EvoDevo: Past and Future of Continuum and Process Plant Morphology
Next Article in Special Issue
Facing Immersive “Post-Truth” in AIVR?
Previous Article in Journal
The Cognitive Philosophy of Communication
Article

An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis

1
Economics Department, Smith College, Northampton, MA 01063, USA
2
School of Engineering, University of Louisville, Louisville, KY 40292, USA
3
Mathematical Sciences Department, Chalmers University of Technology, 405 30 Gothenburg, Sweden
*
Author to whom correspondence should be addressed.
Philosophies 2020, 5(4), 40; https://doi.org/10.3390/philosophies5040040
Received: 25 September 2020 / Revised: 10 November 2020 / Accepted: 20 November 2020 / Published: 1 December 2020
(This article belongs to the Special Issue The Perils of Artificial Intelligence)
An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the strong orthogonality thesis since the modifications would be influenced by the AGI’s intelligence. AGIs in highly competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory. Nothing in our analysis weakens arguments concerning the risks of AGI. View Full-Text
Keywords: artificial general intelligence; orthogonality thesis; instrumental drives artificial general intelligence; orthogonality thesis; instrumental drives
MDPI and ACS Style

Miller, J.D.; Yampolskiy, R.; Häggström, O. An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis. Philosophies 2020, 5, 40. https://doi.org/10.3390/philosophies5040040

AMA Style

Miller JD, Yampolskiy R, Häggström O. An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis. Philosophies. 2020; 5(4):40. https://doi.org/10.3390/philosophies5040040

Chicago/Turabian Style

Miller, James D., Roman Yampolskiy, and Olle Häggström. 2020. "An AGI Modifying Its Utility Function in Violation of the Strong Orthogonality Thesis" Philosophies 5, no. 4: 40. https://doi.org/10.3390/philosophies5040040

Find Other Styles

Article Access Map by Country/Region

1
Back to TopTop