Machine Intelligence, Artificial General Intelligence, Super-Intelligence, and Human Dignity

Our temptation to personify machine intelligence is not unexpected. As a child we named our dolls and took our Teddy Bear to bed with us. Today we ask death bots to comfort us with post-mortem conversation. All the while we know this to be pretend. Yet we must ask: if Artificial General Intelligence (AGI) or even Artificial Super-Intelligence (ASI) become available, will our game of pretend continue? Or will intelligent robots actually become selves deserving of dignity that hitherto could be ascribed only to human persons? If government-imposed guardrails shut the door on development of AGI and ASI in order to preserve human safety and even dignity, we might never learn whether AGI or ASI could develop selfhood, personhood, virtue, or religious sensibilities. As we approach the future, can we live without knowing whether AGI or ASI would be capable of developing selfhood and commanding dignity?