No doubt you are familiar with the Turing Test, the artificial intelligence challenge created in 1950 by mathematician Alan Turing that would be won when a person could not distinguish whether it was talking to a computer or a human being.
Now, 64 years later, a computer program going by the name of Eugene Goostman and simulating a 13-year-old Ukrainian boy fooled 10 out of 30 judges into thinking it was human.
I say Eugene “apparently” passed the test, because a significant number of folks are unimpressed by both the structure of the event and the way in which the judges were fooled. On the Popular Science blog, Maki Naro says, “Goostman makes all the mistakes the chatbots before him have made: he dodges questions, he changes the subject, he makes vague answers, he repeats things back to you in ways that no normal human does in a cute attempt to show that he's listening, and of course he says really stupid stuff that doesn't make any sense. Goostman's creators explain his quirks away by giving him a fictional back story. See, Eugene is a 13-year-old Ukranian kid. He has favorite foods and a pet guinea pig, and he feels okay derailing important interrogations to tell you these things.”
It feels like cheating to counter any failings in the way the program communicates by imposing artificial limitations on the persona it’s meant to represent -- a tactic that could easily get out of control. You might as well have a chatbot named Hodor that only ever says, “Hodor,” and tell people it’s not a programming thing, that’s just the way Hodor is.
Naro also thinks the Goostman approach is a cop-out. “I don't buy any of this, by the way,” he says. “One, it's insulting to 13-year-olds, plenty of whom can hold a conversation without resorting to non sequitur. It's also insulting to people who speak English as a second language, for the same reasons.”
But let’s just say for the sake of argument that you can “win” the Turing Test by creating a fictional narrative that excuses your program’s flaws. The question is: does it matter?
Five years ago, I wrote a column for MediaPost’s Search Insider proposing that, for many of us, a more important test was a kind of “Turing 2.0.” Instead of asking whether a computer can pass for a human, we should instead be asking whether a human can pass for a human.
“Consider the last time you sent an email to someone you didn't know. How did you introduce yourself? What did you say to prove that you were you -- and, therefore, worth listening to? Consider the last time you met someone new online -- maybe someone who sold you that used exercycle on Craigslist. How could you tell it was a person? More important, how could you tell it was a trustworthy person?”
As robots become better and better at imitating people, we will have to become more and more adept at expressing that which makes us uniquely human. And what makes us uniquely human is not banter about the weather. A computer can fake being a 13-year-old Ukrainian boy talking about his guinea pig? Fine. But even if the boy were real, I wouldn’t consider that conversation to represent any kind of profound connection.
I want Eugene to share with me the experience of living in a country under threat and the sense of his parents’ hopes and fears as the very identity of their nation is up for grabs. I want to know how much of his sociopolitical environment he understands, and how he is interpreting the events that are unfolding around him. And it’s not just because he’s from Ukraine and it’s not just because he’s 13; there are equivalent conversations to be had with every person, from every country, at every age.
It’s a different kind of communication than the one Eugene was able to pull off. But it’s far more meaningful. And we’ve got a ways to go before a computer can pass that test.