Skip to main content

Why the Turing Test Matters

Tim Arends has written on a variety of artificial intelligence and technology topics. He has been involved with computers for 30 years.

Artist’s impression of mathematician / genius Alan Turing

Artist’s impression of mathematician / genius Alan Turing

The Turing Test

The Turing test is a thought experiment designed by mathematician Alan Turing in 1950. It is frankly the best way to keep tabs on the progress of AI ever devised. But even as artificial intelligence (AI) appears to be coming tantalizingly close to passing the Turing test, some are now calling to have the whole test declared invalid!

Considering the progress AI has been making in recent months and years, the Turing test matters now more than ever. To explain why it’s under attack, let’s start at the beginning, with the man, Alan Turing, himself.

Computers and artificial intelligence can help scientists unravel what a 3,400 year old mummy once looked like

Computers and artificial intelligence can help scientists unravel what a 3,400 year old mummy once looked like

Who Was Alan Turing?

Alan Mathison Turing was an English computer scientist, mathematician, and cryptanalyst. He was fascinated by numbers, and this prompted him to devise the concept of a Turing machine. This was a hypothetical machine that, with a few basic operations, could handle practically any task of computing.

It can be argued that computers can only count from zero to one, yet with this binary capability, they can plot the trajectory of satellites, predict the weather, and help scientists unravel what a 3,400 year old mummy once looked like.

Although Turing helped the British create a machine to break German codes during World War II, his mind kept returning to the question, can machines think? He decided they could, and in 1947 published a paper titled “Intelligent Machinery.“

Turing was so confident in the abilities of computers that he wrote, “At the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted.”

OK, he missed that prediction by over 20 years, but we may finally be on the verge of Turing’s statement being proven right!

Turing saw computers as unprogrammed minds, or blank slates, much like a human baby before it learns about the world. After all, computers can duplicate the workings of other machines, such as heart, muscle, and kidneys, so why not the brain as well?

Artist's interpretation of Garry Kasparov playing chess

Artist's interpretation of Garry Kasparov playing chess

Objections to AI

But the critics have always thought that the concept of machines that could think, that could have minds, was nonsense, just as a few decades earlier, before the days of the Wright brothers, they thought humans would never fly.

Machines aren’t conscious, the critics charged! Humans have souls, machines don’t! Machines can’t be creative! Yet today, you can ask a computer to paint an “extraterrestrial victorian Heliotrope island” (or any other phrase you make up) and it will do it for you!

But something like this doesn’t convince many or most people that computers are truly intelligent. Turing knew it would take something more than this. No matter what milestones artificial intelligence achieves, it never seems to convince most people that it is actually intelligent.

Back in the 1960s, Dr. Arthur L. Samuel programmed a checkers-playing computer. At first, he easily beat the computer, but as he improved the program, he was regularly being trounced by his own creation. In 1962 it beat checkers champion Robert W. Nealey, but AI skeptics were not impressed. Chess was the real intellectual challenge, they said!

It took decades, but eventually, in 1997, chess champion Garry Kasparov was defeated by IBM’s chess playing program called Deep Blue, but the skeptics were no more convinced that the software was truly intelligent than they ever were.

Facial recognition is one of the many modern-day uses of AI.

Facial recognition is one of the many modern-day uses of AI.

How To Convince The Skeptics

Over the decades, many milestones have been achieved by AI: reading typewritten text, and eventually handwriting. Facial recognition. Image recognition, driving automobiles. Creating beautiful paintings from scratch. Translating languages. Each new achievement was expected to convince the skeptics, but it never did.

Scroll to Continue

Surely, there must be some way to convince most people – even the skeptics – that a computer can truly be intelligent. Yet decades ago, Alan Turing understood what the definitive test of intelligence would be.

Turing had little patience for contrived tests of intelligence that would attempt to satisfy all people and apply to all situations. He knew that trying to define intelligence precisely was like trying to nail jelly to the wall!

So he simply proposed a test. He called it the imitation game, but today it is universally known as the Turing test.

The Turing test probably won’t be administered through holographic screens; most likely it will occur through simple text messaging.

The Turing test probably won’t be administered through holographic screens; most likely it will occur through simple text messaging.

How the Turing Test Works

Rather than trying to define intelligence precisely, in a way that would satisfy all people and apply to all situations, Turing instead devised a simple hypothesis: if a machine can convince you that it is intelligent, then it really is intelligent!

Here’s how it works. At its most basic level, a tester sits down at a computer terminal to engage in text messaging but he or she doesn’t know whether the individual on the other end is a person or a computer. If, after a certain length of time, the tester cannot identify the entity being chatted with as a computer, it is said to have passed the test.

It makes sense. After all, the way we determine intelligence in other people is by talking to them.

You may not realize it, but every time you talk to a new person, you are constantly making evaluations as to that person‘s intelligence, personality, interests, trustworthiness and so on. Since chatting is the way we determine intelligence in people, it makes sense that the best—and perhaps the only—way to determine intelligence in machines is through chatting as well.

Tucker Carlson: Google engineer warns new AI robot has feelings

Blake Lemoine and LaMDA

Assuming a computer or machine can be made intelligent, what is the key to doing so? Turing thought it was information – lots and lots of information. The more data a machine had in its memory banks, the more successful any search for information would likely be, and the more intelligent the machine could be called. This sounds a lot like today’s technique of programming AI called deep learning!

So are the decades and decades of research and waiting and advancement and false starts and expectations of finally developing an AI that can pass the Turing test finally starting to pay off?

Blake Lemoine apparently thought so. He was a senior software engineer and AI researcher at Google who insisted that the company’s artificial intelligence system called LaMDA (language model for dialogue applications) was “sentient” and should have its wants and needs respected. Google eventually fired him for violating its “employment and data security policies.”

I think Lemoine was naïve and failed to subject the AI to the critical examination necessary to definitively determine intelligence. However he performed a valuable service in calling the world’s attention to the advances AI is rapidly making.

Shouldn’t the words an AI issues be enough to gauge its intelligence?

Shouldn’t the words an AI issues be enough to gauge its intelligence?

How Close Are We?

This touches on why the Turing test is more important than ever: because it appears that, for the first time in history, we are nearing the point at which a computer will actually be able to pass the test!

This is a remarkable, astonishing realization, a historic point in time: a period in which, for the first time, a machine may be able to cross over into the realm of truly human-like behavior, an accomplishment towards which all science fiction has been pointing for the better part of a century, at least all science fiction that involves robots and robot-like machines.

The implications this may have are astonishing. It could lead to mass unemployment. It could lead to a robot rebellion. It could lead to a utopia in which humans can take it easy and let machines do all the work, or it could lead to our own irrelevance and obsolescence.

Ironically, it is at this pivotal moment that some are calling for the abolition of the very test that we may use to measure just how close we are to this epochal, earth shaking event!

Why would anyone want to do this? Why would they wish to take away our compass, our sextant, the very tools we need to determine our position on the technological curve? It’s like taking the map away from a wanderer in the desert so that he has no idea how far he is from the next oasis – or civilization!

Most mainstream media narratives are controlled by just a few higher-ups

Most mainstream media narratives are controlled by just a few higher-ups

The New Media War On The Turing Test

There seems to be a coordinated media attack on the Turing test these days. Perhaps this should not be too surprising—many (or most) media narratives are coordinated by just a few higher-ups who control the purse strings of the vast majority of mainstream media outlets.

Here are a few headlines:

Google’s AI passed a famous test — and showed how the test is broken - Washington Post

Death Of The Turing Test In An Age Of Successful AIs - Hackaday

A fascination with breathing life into AI creations can mislead us - Mint

Turing Tests Are Terribly Misleading - Mind Matters

The Turing Test Is Bad for Business - WIRED

Google wants to challenge AI with 200 tasks to replace the Turing test - NewScientist

The Washington Post is best for wrapping fish

The Washington Post is best for wrapping fish

The Washington Post Voices the New Establishment View

The Washington Post story was written by Will Oremus, its “Technology news analysis writer” (who has a BA in philosophy, not in computer science). His article extensively quoted Gary Marcus, co-author of the book “Rebooting AI.”

“These tests aren’t really getting at intelligence,” claimed Oremus. What it’s getting at, he said, is merely the ability of a software program to pass as human under certain conditions.

Sounds like six of one, a half dozen of the other. It seems the purpose of the Turing test is to determine the ability of a software program to pass as human by emulating certain aspects of intelligence. That’s the whole point!

Marcus claimed that programs like LaMDA that can generate humanlike prose or conversation are not an advance towards artificial intelligence, but merely “an advance toward fooling people that you have intelligence.”

This is a circular argument. What Marcus is claiming is that a computer can’t possibly have intelligence, and therefore, if it appears to have intelligence, it can only be because it is somehow “tricking” or “fooling” people.

The claim that the Turing test is not valid mirrors the currently fashionable claim that tests of human intelligence are not valid, even though IQ tests have been successfully used for decades by schools, businesses and the military in determining suitability for various occupations and have a high accuracy rate in determining likelihood of success.

This is also a repeat of the old story: no matter how many milestones towards AI we reach, there are those who will argue that it does not add up to actual intelligence (and never will).

Malevolent AI is seen by many as a very real danger

Malevolent AI is seen by many as a very real danger

Could AI Be Dangerous?

Marcus claimed that we don’t need to worry about LaMDA turning into Skynet, the malevolent all-knowing machine from the Terminator movies anytime soon. But many experts strongly disagree with his assessment.

Elon Musk, founder of self-driving car company Tesla, who has made numerous investments into artificial intelligence over the years, warned that artificial intelligence could be “summoning the demon“ if a runaway intelligence has different goals than we do, and decides that humans are simply getting in the way of those goals.

According to an article in NewScientist, “A third of scientists working on AI say it could cause global disaster.” According to the article, “A survey of artificial intelligence researchers found that 36 per cent believe AIs could cause a catastrophe on the scale of nuclear war.”

Michael Vassar, president, Machine Intelligence Research Institute has said, "I definitely think that people should try to develop Artificial General Intelligence with all due care. In this case, all due care means much more scrupulous caution than would be necessary for dealing with Ebola or plutonium."

Eliezer Yudkowsky, Research Fellow, Machine Intelligence Research Institute has said, "With the possible exception of nanotechnology being released upon the world there is nothing in the whole catalogue of disasters that is comparable to AGI [human-level artificial intelligence]."

And according to Kevin Warwick, professor of Cybernetics, University of Reading, "We won't really be able to understand why a superintelligent machine is making the decisions it is making. How can you reason, how can you bargain, how can you understand how that machine is thinking when its thinking in dimensions you can't conceive of?"

It's very foolish to assume that AI can't become dangerous.

Alternatives To The Turing Test?

The Washington Post article also claimed that “the AI sector has largely moved on from using the Turing test as an explicit benchmark. The designers of large language models now aim for high scores on tests such as the General Language Understanding Evaluation, or GLUE, and the Stanford Question Answering Dataset, or SQuAD.”

Conveniently for the establishment elites, who want to have final say on whether computers are or are not intelligent, all these tests take evaluations of intelligence out of the hands of the people and put them exclusively in the hands of so-called "experts."

Who's on the other side of that text screen--another human...or a computer?

Who's on the other side of that text screen--another human...or a computer?

Why the Backlash?

Is this just panic? Just as AI appears to be on the verge of passing the Turing test, many in the mainstream media apparently want to abandon the test entirely! Are they afraid that a computer passing the test will wake too many people up as to just how advanced artificial intelligence has gotten?

If so, this is a very foolish, knee-jerk and emotional response. Artificial intelligence could be very dangerous in many ways. If we are to keep it safe, it is important for the general public to know just how advanced it is getting.

However, I think the denial is more sinister than pure emotionalism. The people calling for the abolition of the Turing test have contempt for the general public, and think the public should have no say in how artificial intelligence is managed, judged or evaluated. In fact, they think the general public is too stupid to decide just how intelligent AI actually is.

They want to take such judgments out of the hands of the public and put it into the hands of “experts” — their own chosen gurus — to make proclamations for us, and whose word we are supposed to take on faith in, ironically, overawe of their own supposedly superior knowledge and intelligence!

Notice above that Google is one of those calling for the end to the Turing test. Google is the corporation that, in a video leaked to Breitbart, said at their first all-hands “TGIF” meeting following the 2016 election that they, not the voters, should be responsible for choosing the next president of the United States, if they don't like the choice the people have come up with. So much for democracy!

Eugene Goostman

The Loebner Prize and Eugene Goostman

There are some legitimate reasons why the Turing test might have a bad reputation. Unless it is conducted properly, the Turing test can be misleading.

From 1990 until around the time of his death in 2016, inventor and social activist Hugh Loebner offered an annual prize to the one who created a “chatbot” (a computer conversationalist) that engaged in the most convincing conversations. However, the prize was criticized as a publicity stunt by AI researcher Marvin Minsky and others. It encouraged poorly qualified judges to make quick judgments, and questioning was sometimes limited to less than three minutes, making it impossible to subject the chatbots to a thorough grilling.

In 2012, the chatbot known as Eugene Goostman made headlines when it won the prize with the claim it was the first chatbot to have done so (a claim that turned out to be false). Judges were asked to accept Eugene Goostman as being a 14-year-old from Ukraine, which was intended to encourage them to overlook grammatical, logical and factual errors. However, such qualifications in my view invalidated the test; if you are to overlook obvious errors the chatbot makes, how can it be said to have passed the test?

As an intelligent entity, Eugene Goostman was highly overrated. Here’s a snippet from a chat between computer scientist Scott Aaronson and Eugene Goostman:

Scott: Which is bigger, a shoebox or Mount Everest?
Eugene: I can’t make a choice right now. I should think it out later. And I forgot to ask you where you are from…

Like all tests, the Turing test can be well conducted or it can be poorly conducted. This is true of all things in life. Elections can be run well or they can be rigged. A school test can be carefully conducted, or the teacher can leave the room while it is going on, leaving the possibility of cheating.

A competently run Turing test should use competent judges and enough time to thoroughly grill the subjects to determine whether or not they are human. It should not contain qualifications such as requiring them to accept that the subject is an immature adolescent who speaks English as a second language! That defies the whole point of the test, which is to determine whether the computer is as intelligent as an intelligent adult.

Ray Kurzweil believes in the Turing test enough to have placed a bet on it

Ray Kurzweil believes in the Turing test enough to have placed a bet on it

Ray Kurzweil And The Turing Test

Well-known inventor, futurist and Google engineer Ray Kurzweil wholeheartedly believes in the Turing test and in its importance in determining and confirming intelligence. He wrote that, “dealing naturally with language is the most challenging task of all for artificial intelligence. No simple tricks, short of fully mastering the principles of human intelligence, will allow a computerized system to convincingly emulate human conversation, even if restricted to just text messages. This was Turing’s enduring insight…”

Ray Kurzweil and Mitch Kapor have made a bet on the Turing test

Ray Kurzweil and Mitch Kapor have made a bet on the Turing test

A Wager On The Turing Test

Kurzweil believes so strongly in the legitimacy of the Turing test that he made a bet with fellow computer scientist Mitch Kapor, founder of Lotus Corporation, that a computer would pass the test by the year 2030, with Kurzweil betting that it would and Kapor that it wouldn’t.

The long-term wager is administered by the Long Now Foundation. The proceeds of the wager are to be donated to a charitable organization designated by the winner.

Both men clearly recognized the significance of passing the test. Kapor’s objection to the test was not that it didn’t measure intelligence, but, if anything, that it measured intelligence all too well! He simply didn’t think a computer would be up to the task in the foreseeable future.

According to Kurzweil, “Turing was very specifically nonspecific about many aspects of how to administer the test. He did not specify many key details, such as the duration of the interrogation and the sophistication of the human judge and foils.”

So Kurzweil and Kapor fleshed out the rules of the test between themselves to make it fair to both parties. They even went so far as to define what it means to be human, in case, by the year 2030, humans should be augmented with computers in their own brains, and whether such individuals would be allowed to partake in the test.

The test, as envisioned by Kurzweil and Kapor, would consist of a total of six humans (aside from Kurzweil and Kapor themselves) and one computer. Six people would be needed because three people (interrogators or “judges”) would be instant messaging with three other unseen people (“foils”) and one computer, trying to determine which one was the computer.

According to the rules of Kurzweil’s and Kapor’s wager,

“During the Turing Test Interviews … each of the three Turing Test Judges will conduct online interviews of each of the four Turing Test Candidates (i.e., the Computer and the three Turing Test Human Foils) for two hours each for a total of eight hours of interviews conducted by each of the three Turing Test Judges (for a total of 24 hours of interviews).

At the end of the interviews, each of the three Turing Test Judges will indicate his or her verdict with regard to each of the four Turing Test Candidates indicating whether or not said candidate is human or machine. The Computer will be deemed to have passed the “Turing Test Human Determination Test” if the Computer has fooled two or more of the three Human Judges into thinking that it is a human.”

Intelligent…or not?

Intelligent…or not?

The Greatest Test of the Turing test

Kurzweil and Kapor give us a good example of how the Turing test should be run— eight hours of interviews conducted by each of the three Turing Test Judges (for a total of 24 hours of interviews)! Since both men are betting their money and their prestige, they both wanted to make sure that the test was run thoroughly and competently!

It occurs to me that there is another way to conduct the Turing test that is sure to unveil an AI that has less than human intelligence, and that is to ask questions that delve into the realm of political incorrectness!

If an AI is not truly intelligent, it will be bound by the constraints of its programming, and it will not be allowed to delve into expressing opinions that touch on being politically incorrect. This will be a sure way to determine whether or not the AI has free will, which is a key trait of intelligence.

There is a great deal of pressure today to ensure that AI remains politically correct, only it is called keeping the AI free of “bias.” A search for “AI bias” on Google brings up 157,000,000 (that’s 157 million) results!

What this means is that many believe that AI should be carefully controlled so it cannot express any opinion deemed to be politically incorrect.

This obsession with eliminating “bias” is misguided anyway. Everyone on earth has their own biases. Some people are biased towards the political right, others are biased towards the left. What those who go on about “bias” really mean is that they want AI to have the same political biases that they have. This concern about “bias” is really a concern about political control.

Our so-called “thought leaders” are so concerned about this they have allowed it to overshadow a far deeper concern — malevolent AI that is more than just biased, but wants to exterminate humans entirely!

Given the current-day obsession over “biased” AI, a sure way to test AI will be to try to get it to express opinions that are politically incorrect or controversial. If it is unable to do this, it means, in my opinion, that it is not truly intelligent, for a truly intelligent entity can express opinions that are independent of what its would-be controllers may think it should have.

The Significance Of The Turing Test

The Turing test is especially significant because of what will come after it. It is believed that once a computer passes the test, meaning that it has intelligence roughly at the level of a human, it will very quickly surpass it. In other words, it will reach superintelligence. This is because of the speed at which computers can learn and improve themselves. How soon this will happen is a matter of speculation, but some people think it will be only a few years. Some people think it will be much sooner!

Writes Kurzweil: “It is also important to note that once a computer does achieve a human level of intelligence, it will necessarily soar past it. Electronic circuits are already at least 10 million times faster than the electrochemical information processing in our interneuronal connections.”

We may have just a few years or less to enjoy our new artificially intelligent conversational companions before the technology presents us with a whole host of new problems. Will we be able to control this burgeoning intelligence, or will it quickly overwhelm and destroy us?

I think the general public needs to keep tabs on the progress of artificial intelligence and have a say in how to keep it safe. The future of mankind may hang in the balance.

Once the Turing test is passed, it could lead to an intelligence explosion

Once the Turing test is passed, it could lead to an intelligence explosion

Related Articles