Online game reveals something fishy about mathematical models

Mathematical models allow us to understand how patterns and processes in the real world are generated and how complex behaviour, such as the collective movement of animal groups, can be produced from simple individual level rules. Fitting models based on the large scale properties of the data is one way to choose between different models, but can we be satisfied with our model when this has been achieved? How can we apply other methods to see how good our model fit is?

James Herbert-Read, researcher at the Department of Mathematics at Uppsala University, and his colleagues highlight and propose a solution to this problem by implementing a Turing test to assess how good their models were at reproducing collective motion.

They designed an online game where members of the public (over 1700 players online) and a small group of experts were asked to differentiate between the collective movements of real fish schools and those simulated by a model.

'By putting the game online, and though crowd sourcing this problem, the public have not only become engaged in science, they have also helped our research,' says James Herbert-Read.

Even though the statistical properties of the model matched those of the real data, both experts and members of the public could differentiate between simulated and real fish. The researchers asked the online players that answered all six questions correctly to give feedback on how they differentiated between the real schools and the simulated ones.

'These players commonly suggested that the spatial organization of the groups and smoothness of the trajectories appeared different between the simulated and real schools. These are aspects of the model we can try to improve in the future', says James Herbert-Read.

'Our results highlight that we can use ourselves as Mechanical Turks through 'citizen science' to improve and refine model fitting'.

Turing test

Alan Turing provided a means of assessing whether a machine's behaviour was equivalent or indistinguishable from that of a human. In the Turing test, if a human observer could not determine between which one of two interacting players was a machine (the other a human), then the machine had passed the test and exhibited intelligent behaviour. The test is designed to assess the ability of a model (the machine) to reproduce the real world (human behaviour).