The development of artificially intelligent agents, including agents with general linguistic capabilities, will require agents that learn how to behave intelligently within the context of a realistic environment. However, it is currently not possible, for various technological, economic, and ethical reasons, to embed learning robots in the real world. An attractive alternative is to embed agents in artificial, but realistic, worlds, such as those contained in video games. Recent work from, for example, DeepMind, has demonstrated how computers can effectively teach themselves to play simple video games, using the techniques of deep reinforcement learning. The goal of the project was to investigate whether agents can develop more meaningful, semantic representations by interacting in a video game environment. A less ambitious goal was to investigate the extent to which the use of modalities other than text, e.g. vision, sound, can lead to richer semantic representations.
The goal of full embedding in a video game environment turned out to be too ambitious for this short pilot project. Creation of the virtual environments turned out to be a technically challenging task, for which games companies typically employ a large team of dedicated programmers. In addition, the existence of any publicly available environments only began to emerge part way through the duration of the project. Further, training of agents in such environments, using deep reinforcement learning, requires large-scale compute resources, at a scale typically only found in large tech companies, rather than university departments.
However, we were able to perform a number of more controlled experiments, investigating a range of modalities: vision, audio, and psychological feature norms. In addition, we also investigated the extent to which the induced semantic representations correlate with representations observed in the brain, through the use of brain imaging techniques such as fMRI and EEG. Through these studies we were able to clearly demonstrate the added value in using additional modalities (other than text) when inducing semantic representations for use by an artificially intelligent agent. Given the richer context provided by a video game environment, we can expect these results to carry over into the more ambitious scenario envisaged in the original proposal.
