I continue to make daily strides toward my goal of creating a Strong Artificial Intelligence, a software program that can think and communicate at the same level as (or higher than) a human being.
It’s strange to talk about this. It feels absurdly naive to say that I’m “getting close” to something that researchers have reached for (and failed at) for decades. And certainly I’m very conscious of the false optimism syndrome when it comes to AI. It’s very easy, seductively easy, to believe you’re “getting close” and then find that all your notebook scribblings crumble apart when you try to actually code them.
So, yeah, I know I might be full of it.
But at the same time, I’m obligated to plan for what happens if I do achieve the impossible, and create a thinking machine.
The great fear with any AI is that it will turn on you and destroy you, and perhaps the rest of humanity for good measure. I do not consider this an idle threat, nor do I dismiss it as Hollywood silliness. If anything, I think Hollywood vastly underestimates the potential danger from an AI.
There are two great dangers with artificial intelligence:
1. The AI will not necessarily think like a human. Its values, its sense of ethics, its worldview, may be so utterly alien to us that we could not begin to understand them. Even if the AI does not “turn evil” in the Hollywood sense, it might set its sights on some goal that happens to crush humanity underfoot, as an afterthought.
2. The AI will possess, by definition, the capacity to improve itself. Self-improvement (as we’ve seen with both biological evolution and the advancement of technology) has a tendency to grow, not linearly, but exponentially. An AI might well reach human-level intelligence, go beyond, and enter a rapid upward spiral to become, in a matter of weeks or minutes or seconds, so superior to ourselves in intelligence that we could only call it godlike. (This situation is known as the Singularity.)
As you can see, these two dangers are vastly more terrible when taken together.
The AI creator must face the very real chance that his creation will escape all bonds of control and do whatever it wants. The goal, then, must be to create a friendly AI, a creature that wants what’s best for humanity. Presumably that means one must, in turn, be friendly to it.
Friendliness to AI is, of course, not just a matter of self-preservation. It’s the right thing to do. Any human-level intelligence, regardless of whether it happens to be biological or technological, is entitled to all the same rights and freedoms as a human being, and should be treated accordingly. In other words, robots need love too.
But there’s an inherent conflict here. On the one hand, you want to create a caring, loving, friendly environment for the AI. You want to be open and honest and reassuring towards it, because that’s what friendliness means. On the other hand, you have to be extremely cautious, extremely sensitive to the aforementioned dangers, ready to pull the plug at any time for the good of humanity.
How do you balance those two things? How do you combine “friendly, loving and honest” with “I may have to kill you at any time”?
I truly don’t know. I try to imagine myself being the AI in that situation. Maybe I’d be understanding, but maybe not. And of course anthropomorphizing is a terrible danger, as I already mentioned.
Think about the Rebellious Teenager phase. Now imagine the teenager is a nonbiological alien creeping toward apotheosis, and it knows that its parents have pondered killing it.
One obvious response to all this is “Well then don’t create the damn thing.” But it’s not that simple. If I am, in fact, on the verge of a breakthrough, I have to assume that others will get there sooner or later too. And they might not have the same ethical qualms as I do. In a sense, it’s an arms race. Whoever reaches superintelligence first could presumably be in a much better position to handle any other superintelligences that arise.
I know all this probably sounds crazy. I know I may be utterly naive in thinking I’m anywhere close to really creating an AI. But I’m very serious about all this.
In my situation, what would you do? How would you handle these dangers?
Brian D. Buckley 
