In the game, Airport Scanner, players were both research participants and baggage screening officers as they pretended to operate X-ray scanners to detect illegal items in luggage. Players tapped the touchscreen of their mobile devices to identify banned objects from a logbook of prohibited items (such as guns, crossbows and bottles) and permitted items (including sunglasses, headphones and clothing).
Researchers examined players’ ability to identify “ultra-rare” targets, which appeared in only 0.1% of the luggage, according to the American Psychological Association (APA), which published the report in December 2014.
In a traditional lab setting, such an experiment would require 1,000 trials to get a single instance of the target item, a potentially costly and time-consuming process. However, by incorporating crowdsourcing, the researchers from the Center for Cognitive Neuroscience at Duke University were able to gather data from more than 2 billion trials encompassing 7 million mobile devices over 22 months.
Among the findings: Participants were better able to identify two prohibited items when the items were identical compared to when the two items were different.
“Questions that could have taken decades to answer in a laboratory setting, or that could not be realistically answered in a lab, can be examined using big data gathered in a relatively short time,” lead researcher Stephen R. Mitroff told the APA.
The detection of prohibited items has been an enormous undertaking for the Transportation Security Administration (TSA) since its creation in response to the terrorist attacks of Sept. 11, 2001.
In 2013, the federal agency’s 50,000 officers screened almost 640 million travelers at more than 450 airports. The screeners discovered 1,813 firearms; more than 80% of the guns were loaded.
In their report – “What Can 1 Billion Trials Tell Us About Visual Search?” – Mitroff and his colleagues noted that crowdsourcing allows for automatic and continuous data collection in an environment that imitates real-life situations. Airport Scanner players consented to anonymous data collection with installation of the app on their mobile device. The game, which was developed by the Kedlin Company, is available in the App Store on iTunes and on Google Play.
But the researchers also cautioned that there are disadvantages to crowdsourced data, including the need to create an enjoyable app and the inability to control who is playing the game. Additionally, the collected data may not be of high quality.
Still, Mitroff told the APA that “mobile technology offers a phenomenal opportunity to examine cognitive processes on a large scale.”