Ethical reasoning, when it is done effectively, involves prioritizing between competing options. Philosophical thought experiments such as the “trolley problem” highlight the human tendency to avoid making decisions about situations in which there is no good outcome. However, the history of both religion and science fiction provide ample illustration of the need to organize competing values, and to do so in advance of concrete situations where our moral and/or ethical commitments are put to the test. From Jesus’ identification of the greatest commandment and the parable of the good Samaritan, to Isaac Asimov’s Three Laws of Robotics and Star Trek’s Kobayashi Maru scenario, religion and science fiction converge on the importance of confronting no-win situations of a variety of sorts, situations in which we cannot fully implement all of our competing ethical commitments. These stories can help us weave the fabric of our diverse communities and values, as well as integrate autonomous machines into a forward-looking social structure.
1.1. A Framework for Technical Discussion and Some Basic Definitions
The topic that we hope to address can be unwieldy because it spans many different fields and points of view. We therefore wish to define certain “loaded” words and concepts at the outset so that we may present our thesis more clearly. Our intent here is not to ignore or downplay the importance of debates related to the definition of these terms. Rather, the goal is to provide a baseline for meaningful discussion that allows us to avoid devoting the majority of our attention to semantic matters. We begin by defining, somewhat trivially, that a human is an agent capable of making a decision based upon some criteria or values, which may change over time based upon instinct, experience, education, or whimsy. A human agent may modify its set of values as well as the relative priority of those values. We define an autonomous machine or robot as an agent that makes a decision based upon some static set of values together with specified priorities which are not allowed to change. However, we do still allow a robot to learn how to optimize its behavior within these parameters by improving its decision-making ability. In some cases, we may refer to a self-aware robot as a non-biological human agent, which may or may not have some weak constraints on its values and their relative priority. Hopefully the context will make clear whether we are referring to imagined machines of science fiction, or machines that we can expect to exist in the real world based on current trajectories of technology.
For convenience, we make the simplifying assumption that a robot’s values are hard-wired (literally or metaphorically) so that neither they nor their priority can be changed. We refer to an algorithm as a series of (software) instructions that the robot uses to decide to perform some action (or inaction) based upon the value system encoded in those instructions. Algorithms are typically evaluated based upon the speed with which they arrive at a decision and the accuracy of the result as compared to whatever may be deemed the “best” solution. Many factors may affect the accuracy of the result, such as incorrect input from sensors (like cameras or radar) or mechanical output errors (like slipping wheels, incorrect turn radius, or brake failure). Our article focuses primarily on accuracy in terms of the theoretical result based on the given input, rather than on its implementation in practice.
Algorithms operate on problems of varying difficulty, often (but not solely) represented by the size of a problem’s state space, which enumerates all possible results or outcomes for a given problem. Algorithmic approaches to problems typically fall into two broad categories that represent this difficulty, deterministic and non-deterministic. Briefly, deterministic algorithms are always able to arrive at the best or optimal solution. For example, the algorithm for tic-tac-toe is deterministic, since we can always find a line of play that leads optimally to a draw. Non-deterministic algorithms feature a state space for a problem (such as chess) that is so large that we simply cannot compute all outcomes in reasonable time, and thus, we must introduce heuristics or educated guesses about the nature of an optimal solution. Ethical value systems of the sort that we discuss in this article are one form of heuristic for making the right decision. Our heuristic guess can be flawed, but within the scope of that heuristic, our goal is to optimize the resulting decision.
In some cases, various techniques in machine learning
or deep learning
may be used to incorporate past success or failure to improve the success rate of a future action (or inaction) in accordance with its hard-wired values. Often, these techniques involve providing the robot with a training set
of prior events, in which the success or failure is already known, which is used by the robot to establish a baseline upon which to make future decisions. This baseline is then iterated upon further with a validation set
, and then finally tested for accuracy with a testing set
. The term artificial intelligence
) refers to the overall ability of a robot to learn, plan, and solve problems autonomously.1
Alan Turing reasoned about the nature of artificial intelligence with the design of the famous Turing test
(Turing 1950, p. 433
), setting the stage for one of the most hotly-debated topics in philosophy and computer science—can a robot think? Our focus is on significantly narrower but no less difficult questions: Can we program autonomous machines to act consistently in accordance with human ethical guidelines? What does it mean, if anything, for a machine to be ethical
There has been significant discussion, from a religious, sociological, psychological, and philosophical point of view, about the distinction (or lack thereof) between two of our more common words denoting value systems, morality and ethics. Defining these terms has proven nearly as challenging as determining whether a given observed action is both ethical and moral, one or the other, or neither, based on particular definitions of the terms. Since our focus is on how to navigate complex decision-making when faced with competing values, we need not distinguish between the two in the context of the present study (and, in this sense, we use them interchangeably). Thus, we define ethical and moral behavior as belonging to a set of preferred or prescribed behaviors, whether specified by a higher being, religion, social norm, or a need for self-preservation. We expect that a human agent would decide upon or take for granted a set of values and assign a priority to them that is meaningful or “right” for that agent. This could happen at any point, including retrospectively from the perspective of hindsight; in the case of a robot, however, it would be assigned in advance.
Finally, we define a few relevant terms used in computer science. A decision tree
, roughly speaking, is a graph
composed of nodes
(connecting lines) that illustrates every point at which a decision needs to be made before arriving at a course of action. For example, in Figure 1
, every node progresses the flow of decision until a final and unambiguous action (or inaction) is determined. Such a system works very well when the values are clearly prioritized. Notice that there are no “loops” in the image, in that no later decision can affect a prior one. As we will discuss later in the article, such “loops” are often the reason for confusion in human reasoning (just as in computer science).
Another concept we use is that of a Hasse diagram or a partial order, which describes a series of known comparisons among objects of a set. The set of integer numbers is an example of a total order, because for any two integers, one can compare them and determine which of the two is larger. For a set of objects that cannot be completely compared, a Hasse diagram helps us visualize the relationships that we do know. In the context of this article, we use them to specify the relative priority among values used in both human and robot decision-making, especially in moral and ethical matters.
For example, we read the graph in Figure 2
from top to bottom, where a given node A
has greater priority than node B
if and only if A
is above B
and an edge connects A
. In Figure 2
, each node contains a bitstring
, a lexical (rather than numerical) sequence of symbols 0
. Bitstrings are ordered in the same spirit as words in a dictionary, where symbols are compared pairwise by position; in other words, the first symbols of each bitstring are compared, and then the second, and so on. In this example, we prioritize the bitstring 1011
over the bitstring 1001
, because every position of the first bitstring is no less than that of the second bitstring. However, the bitstrings 0101
are incomparable, since each bitstring has at least one symbol that is larger than its corresponding symbol in the other bitstring. The edges in Figure 2
provide a visual representation of the properties of the partial ordering we defined just above. The data configured in this way may be of any sort, including ethical commitments and objects of value, as we shall explore below.
1.2. Key Challenges to Human Ethical Reasoning and Ethical Robot Programming
As we discuss throughout the article, human agents have a fundamental discomfort with a priori decision-making, especially when all available options are less than ideal. Part of the discomfort stems from our reluctance to set aside the influence of contextual clues, emotional input, or other not-strictly-logical personal bias or feeling in a specific circumstance. In fact, even a human agent with a firm theoretical view of a specific course of action may defy or contradict that viewpoint when that concrete situation arises in practice. Human agents also tend to believe that they make decisions quickly and precisely in difficult situations, yet we are also relatively quick to forgive poor outcomes that result from innate human limitations. It is therefore somewhat ironic that human agents, in general, have significantly higher expectations of robots than of themselves, so much so that humans harshly criticize machines if the outcome falls even slightly short of perfection, despite a faithful execution of their algorithms.
The lack of clarity in human moral and ethical reasoning presents real challenges when designing robots to follow ethical guidelines and is exacerbated by our emotional responses to their implementation in practical scenarios. Navigating the murky gray areas in no-win situations is even more complex, and often leads to disagreement that has historically led to conflict of the highest order. Even though autonomous machines are the natural evolution towards technological advancement and increased human comfort and safety, it is no surprise that human legal quandaries and distrust of programming appear as hurdles along the path to acceptance of autonomous machines as part of mainstream human society. By bringing together perspectives from religion, philosophy, computer science, and science fiction, the authors hope to clarify how human agents and robots should think and act, and how we should judge (or refrain from judging) both human agents and robots in their implementation of ethical and moral values.
Any engagement with matters that are ethical, robotic/computational, or both will eventually find itself at the intersection of philosophical and narrative structure. Science fiction stories and parables can aid in creating a “sandbox” in which one can explore and study ethical and other philosophical questions. There is, to be sure, an inherent danger in using a story where the “conclusion” is all but pre-determined and leads to circular logic. For instance, if we depict machines as self-aware in a story, and use that story to advocate for robot rights, we are assuming what is yet unproved, namely that machines may one day have this capability. Our aim is not to wrestle with the speculative problems related to the most advanced machines that science fiction has imagined, but with the steps towards autonomous machines that reflect present technology and its realistic progression into the future. It is this topic—at the intersection between computing, science fiction, and ethical reasoning—that is our concern here.