Lesson 2 - The Turing Test and Searle's Chinese Room Flashcards
Covers the Tuing Test, Searle's Chinese Room Argument and rebuttals
When did Turing propose his test?
1950
How many electronic computers were in existence at the time Turing proposed his Turing Test?
4
when was the Dartmouth Conference?
1956
In which paper did Turing propose his Imitation Game?
Computing Machinery and Intelligence (1950)
in his 1950 paper Computing Machinery and Intelligence, what question was Turing attempting to rephrase?
Can machines think?
Who are the players in the imitation game?
A - a man, B - a woman, C - an interrogator of any gender
alternative to having typed communication in the Imitation Game?
an intermediary
which player would the machine take the role of?
player A (the deceptor)
what is the specific question that replaces the old “Can machines think?”
will the interrogator decide wrongly as often when the game is played with a machine than as when he does when the game is played between a man and a woman?
how does Turing regard the question “Can machines think?”
meaningless to discuss due to the ambiguity of the question
What did Turing believe would happen in 50 years time?
it’ll be impossible for interrogators to make the correct distinction more than 70% of the time
what is the computing capacity Turing predicts for the 2000s?
10^9
what is Eugene Goostman
chatbot developed in St Petersburg by 3 Russian/Ukranian programmers to simulate a 13 year old Ukranian boy with EFL
significance of the age of Eugene Goostman
not too old to know eveything but not too young to know nothing
what happened with Goostman on the 60th anniversary of Turing’s death
claimed to have passed the Turing Test
how many judges did Eugene fool after a 5 minute conversation?
10 out of 30
what was important about the Loebner Competition
no computer had ever won the top prize for outright fooling the judges
according to Daniel Bennett what was/wasn’t the Turing Test intended as?
WAS - a conversation stopper/a thought experiment
WASN’T - a platform for serious scientific research
name of the 2023 study that used the online game to simulate the Imitiation Game
Human or Not? A Gamified Approach To The Turing Test
how many participants in the first month of the May 2023 study
1.5 million
how many guesses in the first month of the May 2023 study
10 million
probability of a correct guess overall in the game for the May 2023 study
68%
probability of a correct guess for the bot in the game for the May 2023 study
60%
4 main objections to Turing’s Test
Theological , ‘Head in the Sand’, Argument from Conciousness, Lady Lovelace
total # of objections that Turing had prepared in Computing Machinery and Intelligence
9
Theological objection
thinking is a function of a man’s immortal soul. God has given an immortal soul to every man and women but not to any other animal or to machines. Hence no animal or machine can think
Turing’s dismissal of the theological objection of the Imitation Game
it is entirely up to God whether or not a particular body is imbued with a thinking soul (perhaps He is waiting for an intelligent enough machine to give a soul to)
The Head in the Sand Objection
the consequences of machines thinking would be too dreadful. Let us hope and believe that they cannot do so
Turing’s dismissal of the Head in the Sand Objection
the argument is insufficiently substantial. humans tend to believe they are superior to the rest of the creation
The Argument from Consciousness is cited from who/where?
Professor Goeffrey Jefferson’s Lister Oration for 1949
The Argument from Consciousness
it’s not until a machine can carry out actions based on its emotions (pleasure, grief, anger) not mere artificial signalling that we can conclude “machine equals brain”
Turing’s dismissal of the Argument from Consciousness
how can we tell that a machine that passes the Turing Test is not conscious? this risks falling into a solipsistic mode of thinking
solipsism
the philosophy that only one’s own mind can be known to exist
Lady Lovelace’s objection
“The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform”
where does the most detailed information of Babbage’s Analytical Engine come from?
a 1842 memoir by Lady Lovelace
variants of Lady Lovelace’s Objection
a machine can never do anything really new nor take us by surprise
Turing’s dismissal of Lady Lovelace’s Objection (humans)
all creative acts can be seen as following from “seeds” planted from teaching - humans can never really do anything new either
Turing’s dismissal of Lady Lovelace’s Objection (computers)
computers can surprise their programmers by producing answers that were not expected
how many objections to the Turing Test did Copeland have?
4
names of Copeland’s 4 objections to the Turing Test
the Chimpanzee Objection, the Sense Organs objection, the simulation objection and the black box objection
What sets Copeland’s first objection apart from the other 3?
a negative outcome to the test would show nothing definite.
The chimpanzee objection
if thinking animals can fail the Turing Test, then presumably, a thinking machine can too
what does the chimpanzee objection imply about the relationship between the question “Can machines think?” and the Turing Test
the Test cannot completely replace that question
Using terms of propositional logic, explain what the chimpanzee objection implies about the relationship between the question “Can machines think?” and the Turing Test
the Turing Test is not a necessary condition for thinking but might still be a sufficient one
the sense organs objection
the Test only focuses on the computer’s ability to use words, not of its ability to relate words to things in the world
the sense organs objection implies that the Test is too…
weak
how does the sense organs objection suggest that the Test can be improved?
give the machine sense organs such as a television eye
Turing himself felt that the best way for a computer to be prepped for the Test is for it to do what?
have the best sense organs that money can buy and then subject it to an appropriate course of education
the sense organ objection question whether verbal quizzing …
is an adequate test of comprehension
The Analytical engine was Turing Complete - true or false?
True
the simulation objection
a computer that passes the Turing test has shown a good simulation of being a thinking entity but it is not the same as BEING a thinking entity
simulation-1 type
the simulated object lack the essential features of the thing it is simulating (e.g. death, leather)
simulation-2 type
the simulated objection is just like what is being simulated but produced in a different way (coal, voice)
the black box objection
the intelligence of the machine is only concluding based on its output and not its inner workings
why does Copeland refutes Turing’s belief that we treat fellow humans as a black box?
our judgement is based on the fact of our biological similarity, not just an observation of their behaviour
the two modifications Copeland would suggest for a machine to pass the Turing Test
outward and design criteria
what would be the proposed outward criterion for the Turing Test
it passes the interrogation test
what would be the proposed design criterion for the Turing Test
it passes a test to ensure the output is not just based on a giant lookup table or a pattern matching chatbot
in what 2 ways could the deisgn criterion be met
if the machine works like a human. if the design is modular enough to extend to a robot with motor/sensory systems
what is the key feature of creatures we are willing to deduce as “thinking”
their action-directing inner processes are massively adaptable
intentional language
explaining/predicting an objects behaviour in terms of what it thinks/believes/wants
have LLMs been doing well on reasoning tests
no
who created the Abstration and Reasoning Corpus
Francois Chollet
what is the Abstraction and Reasoning Corpus
a new kind of logic test for AI systems
what is the main type of challenge proposed in the Abstraction and Reasoning Corpus
detect patterns in visual examples and apply them to new examples
most researchers agree that the best way to test LLMs for abstract reasoning abilities and other signs of intelligence….
remains and open, unsolved problem
other than reasoning, what other traits might we want to assess in AI
empathy, moral decision-making
what thought experiment does Searle use to determine that the Strong AI Hypothesis cannot be true
Searle’s Chinese Room Argument
True or False: Searle claims the question (whether computers can think) is an empirical question?
FALSE
what is an empirical question
one that can be settled by experiementation
True or False: Searle claims his Chinese Room argument better settles the question than the Turing Test
True
no amount of syntactic manipulation…
can lead to semantics
what are arguments A and B for Searle’s Chinese Room
A: the man doesn’t know Chinese
B: the room doesn’t know Chinese
a priori
knowledge independent of experience
True or False: Searle’s Chinese Room Argument is ignored by the AI community
True
