Introduction

The Gender Guessing Game took place on April 16th between 3-6pm ET. This test was based upon Alan Turing's paper, Computing Machinery and Intelligence, written in 1950. Although the popularised version of the "Turing Test" has been conducted many times over the last decade, no experiment has been conducted of the original "Imitation Game" described in Turing's paper.

The popularised "Turing Test" of common understanding is that a judge interviews a computer software program and a human via computer chat, and attempts to discern who the real human is, and who the machine is.

The original Imitation Game introduced a "gender guessing game", wherein two human subjects, a male and a female, would communicate via computer chat to a "judge". Both the male and the female would try to convince the judge that they were female. Turing's original question was, if a gender guessing game was done, and an AI replaced the male, would the judge be more accurate in guessing who the real female was?

About the researchers

Three students at Simon's Rock designed the experiment: Cameo Wood, a Linguistics, and Cognitive Science Junior, has been interested in neuroscience, philosophy of mind, and artificial intelligence for many years, and the original Imitation Game was a experiment she had wanted to perform for many years. Melissa Leventhal, a 17-year-old college sophomore in Psychology, was interested in the definition of humanity as explored through online communication. Allyson Sgro, an 18-year-old college senior in Chemistry attending the University of Washington's Ph.D. Program in Chemistry and Nanotechnology next year, had been interested in the Turing Test after taking a seminar course concerning Douglas Hofstadter's book, Gödel, Escher, Bach.

Frequently Asked Questions

Frequently Asked Questions:

  1. What are your results? It has been five days since the experiment. I'm curious, what what was the success rate for the man? For the machine?
    Whilst we would love to immediately disclose our findings to interested public, scientific research for publication or presentation is required to go through a system of peer review before it can be published. The average length of time for peer review is 6-8 weeks but this may be lengthened depending on referee availability and the number of reviews already in peer review. Until our paper has been accepted for publication or presentation, we are very limited in what we can disclose.
  2. What do you mean, this hasn't been done before? I hear about the Turing test all the time!
    Susan Sterrett at Duke University wrote a paper in 2000 titled "Turing's Two Tests for Intelligence" wherein she discussed the Standard Turing Test interpretation as well as a more literal interpretation named the Original Imitation Game. We performed the later.
  3. Was the bot actually good enough to fool people?
    Yes. Some people were definately fooled.
  4. Was this experiment reviewed by an IRB?
    Yes, the Simon's Rock Human Research Review Committee reviewed our research proposal prior to our experiment.

Credits

The research team for The Guessing Game wish to thank those that assisted our success conducting our experiment.

  • This experiment was funded in part by grants from the Simon's Rock College of Bard departments of Natural Science and Social Science.
  • The experiments wish to thank the Dr. Richard Wallace and the A.L.I.C.E. AI Foundation for his assistance onsite during the experiment and the use of his bot, Alice.
  • Anne O'Dwyer, the research advisor for this project, for her invaluable help in designing and analysing our research.
  • Our confederates that posed as themselves and the men that posed as women— this experiment could not have occured without your dedication.
  • To our web host, Bjørn-Erik Townsend, for waking up at ungodly hours throughout our experiment and the following interest.
  • Thank you to Wayne Sæwyc at Wikinews for his help in documenting our experiment.

About the experiment

Three students from Simon's Rock College of Bard decided to conduct this experiment as part of their Research Methods in the Social Sciences class. During their literature review, they were shocked to discover that Turing's original "Imitation Game" had never been conducted. They then read in a Wired Magazine article with Artificial Intelligence designer Dr. Richard Wallace that he was very eager to participate in such an experiment. After contacting a very enthusiastic Wallace, the students began to finalise their preparations for their test.

The experiment utilised A.L.I.C.E., kindly furnished by the ALICE Artificial Intelligence Foundation. Dr. Richard Wallace was on hand during the experiment to troubleshoot the AI robot, and later, to give a lecture about on The Anatomy of A.L.I.C.E.. His coverage of the event can be found here.

Six human subjects from Simon's Rock composed the human players in the game. The judges were recruited from various non-technical internet communities. Roughly eighty individuals participated in the experiment. The data is currently being analysed, and a paper is forthcoming.