Competition description and rules

The task

The competition task is to create a computer game bot which is indistinguishable from a human player. Those entries that pass this test will share the major prize of A$7,000 cash, and will also be offered a trip to 2K Australia's studio in Canberra. If the major prize is not won, a minor prize of A$2,000 plus a trip to the studio will be awarded.

The competition will be run at the IEEE Symposium on Computation Intelligence and Games, in Milan, Italy.

In a change from the 2008 competition, entrants can participate remotely, rather than having to attend the conference.

The game used for the competition will be based on a modified version of the DeathMatch game type for the First-Person Shooter, Unreal Tournament 2004. This modified version provides a socket-based interface (called Gamebots) that allows control of bots from an external program. In addition, several extra modifications will be made especially for the competition:

  • Chatting will be disabled.
  • Some aspects of the game play will be modified.
Chatting will be disabled lest the competition revert to a chat bot competition! Other modifications will be made to encourage the submission of bots that can learn how best to play, as human players are able to do.

To enter

Competitors must advise their intention to enter the competition on or before 1st June 2009, by email to the competition organisers. If there are a large number of entries, the organisers reserve the right to carry out qualification trials to select the entries to take part in the finals at the conference. In that case, competitors will be required to take part in these trials by a remote mechanism that has yet to be decided, at a date and time to be specified. If needed, these will probably be before 1st July. The programs for the trials need not be final versions.

Other conditions of entry are:

  • Individuals or teams may enter.
  • No-one can enter more than one bot (either as an individual or as part of a team).
  • No-one associated with 2K or with the organisation of the competition may enter.
  • Entrants must affirm that they have intellectual rights to their entry and that it and its components comply with all artistic licenses.
  • Entrants younger than 18 years of age must provide a written statement of permission by at least one parent or guardian.
  • Entrants must be willing to allow videos/mpegs of their entries in action at the competition to be published and become public domain.

Testing protocol

The protocol for the competition will be similar to that of the Loebner Prize. There will be a panel of judges, who will test the submitted bots and attempt to identify whether they are bots or human players. The judges will be provided with a computer each, set up in one room, and a group of confederates (human players) will be set up in a separate room, such that the only communication between the two rooms is through the medium of the game over a local network. Testing will take place on PC's provided by the organisers, running Windows XP or Vista.

Each judge will play a number of games against each bot entered in the competition, and also against each human confederate, but will not know which is which. Each game will be between three players: a judge, a human and a bot. After each game, the judge will be asked to nominate whether each opponent was human or a bot, and if a bot, to rate its "human-ness" on a rating scale. The rules permit judges to nominate both opponents as human, or both as bots, or one of each.

To win a share of the major prize, an 80% majority of the judges must be convinced that the bot is a human. If no-one wins the major prize, then the minor prize will be awarded to the bot with the highest average "human-ness" on the judges' ratings. If there is a tie, the prize will be split evenly. Here is an example of a judge's scoring sheet.

A member of the winning team (either major or minor) will be invited to visit 2K Australia's studio, at 2K Australia's expense, up to an amount of A$5,000, in addition to the cash prize.

Practice sessions

The bots will be trained by the organisers in the week before the final, and the trained bots will be taken to Milan. The human confederates and judges will be given a practice session to practice the modified game the day before the final.

Human confederates will be moderately competent UT players, neither novice nor expert. Confederates will be instructed to play the game as they normally would, and will play for a 100 Euro prize awarded to the confederate with the best score.