AAARGH A COMPUTER HASN’T PASSED THE SODDING TURING TEST, IT WASN’T THE FIRST TIME, AND IT WASN’T A SUPERCOMPUTER, AAAAAARGH!!!!
AAAARGH!!!
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARRRGGGH!
*Ahem*
No, seriously, those “supercomputer passes Turing Test” news stories that *EVERYONE* is running are nonsense. The bot in question, “Eugene Goostman” is not a supercomputer, it’s just a chatbot, no different from ELIZA or Parry. It could easily run on your mobile phone, if unlike me you have one.
And it didn’t “pass the Turing test”. It won the “Turing test competition”, which is different. They have a criterion of convincing 30% of people in a five minute conversation, which is not the same in any way as Turing’s actual test, which involves a computer convincing people it’s a man as often as a woman could, or a woman as often as a man could, and doesn’t have an arbitrary five-minute time limit.
It’s not the first time it’s won it either. The same software — the *exact same program* — won the Turing Test competition in 2012. There it only convinced 29%, rather than 30%, of judges, but given the small number of judges AND THE FACT THAT IT’S THE SAME PROGRAM that’s just statistical noise, not a new result.
It also only manages to get even that score, under conditions in no way comparable to Turing’s, by being programmed to simulate not the adult that Turing talked about, but a thirteen-year-old boy whose first language isn’t English.
This is no more “passing the Turing test” than running a hundred yards in thirty seconds is running a four-minute mile.
*twitch*
*twitch*
I think I’m better now…
Sci-Ence! Justice Leak! 
The Turing paper actually does contain the five minutes and it does contain the 30% pass but not as criteria for success. Turing writes that he believes that in 50 years then computers will be good enough at the imitation game that people would only be able to guess with 70% accuracy in a five minute test. He made this prediction in 1950. I think the Loebner prize had a 29% pass rate for 5 minute tests around 2002, where Turing said 2000… pretty damn good.
The section I’m referring to is about 3/4 of the way through the original paper after he goes through his original objections.
Yeah, exactly — that was a prediction of time-scale for progress towards passing the test, not a set of criteria to pass. If the articles had said “Turing’s predictions about right” rather than “Turing test passed by a supercomputer for the first time EVAH!!!11!” then I wouldn’t have a problem with them — but then, they’d no more have been “news” than “raining in Manchester again”, and they wouldn’t have had a misleading quote from Cyberkev…
This story was best dealt with by whoever it was in my twitter stream suggesting along the lines that the Turing Test is whether a film has two named gay men talking about programming without mentioning the war.