Using Games to Explore the Mind

January 9, 2024 • Institute Update

The National Institute of Mental Health (NIMH) and the non-profit organization The Many Brains Project  partnered with the National Institutes of Health (NIH) All of Us Research Program  to adapt a series of new game-like tasks that are now part of the All of Us Research Program’s participant experience. The quizzes, puzzles, and other engaging exercises measure abilities like attention span, decision-making, and emotion recognition.

The All of Us Research Program

The All of Us Research Program is a large-scale effort spearheaded by NIH to collect and study data from at least one million people in the United States. People who participate in the program are asked to complete surveys on different topics, share data from their electronic health records, complete online tasks, and provide biological samples (such as saliva).

Why did NIMH partner with All of Us?

In 2009, NIMH developed the Research Domain Criteria Initiative, known as RDoC. RDoC was created as a new way of conceptualizing and studying mental disorders. In this framework, mental disorders are studied through the lens of different functional processes, such as cognition, mood, social interactions, and so forth, with traits that exist on a continuum ranging from typical to extreme.

NIMH is interested in collecting data on some of the factors that are part of the RDoC framework, such as cognition, social, and reward processes, from a large, diverse pool of participants—just the type of participant cohort being built through All of Us. As a bonus, All of Us gathers a lot of other mental and physical health data. When combined with the information learned from the new gamified tasks, these data could help researchers understand what leads to mental disorders, how they develop, and the most effective ways to treat them.

In 2016, RDoC staff convened a workgroup to identify behavioral tests that could measure factors in the RDoC framework. After examining the list of tests, RDoC staff chose several with strong normative data—baseline scores or measurements against which new scores can be compared. RDoC staff then issued several year-long contracts to different groups, asking them to transform the tests into easy-to-use gamified tasks that were still valid for measuring the intended concepts.

Four tasks—Guess the Emotion, Left or Right, City or Mountain, and Now or Later—were ultimately adapted to be part of the All of Us Research Program. These four tasks were created as part of the nonprofit TestMyBrain platform , supported by The Many Brains Project.

What are the new tasks, and what do they measure?

NIMH worked with All of Us to introduce four different tasks to their platform: Guess the Emotion, Left or Right, City or Mountain, and Now or Later.

• Guess the Emotion: The Guess the Emotion task asks participants to look at pictures of people and name the people’s emotions based on their facial expressions. This task is a version of the Face Emotion Identification task, which is used to learn about social processes, social communication, and how people receive information communicated through facial expressions.
• Left or Right: The Left or Right task measures how well people can focus within distracting environments. The task begins with a screen showing five arrows—one center arrow between two sets of side arrows. The five arrows change direction during the task, and the participant must indicate the direction of only the middle arrow. This can be tricky because the side arrows might point in a different direction than the middle arrow. This task is a version of the Flanker task, which measures attention and response inhibition, or the ability to stop a response that results from inappropriate information (in this case, the side arrows).

• City or Mountain: The City or Mountain task tests how fast participants respond—or how well participants resist responding—to a changing scene. During the task, city or mountain scenes fade in and out. Participants are asked to press a response key only when they see an image of a city. This task is a type of gradual onset continuous performance task. It is used to understand cognitive systems such as attention, cognitive control, and the ability to respond to only one type of information while ignoring others.
• Now or Later: In the Now or Later task, participants are presented with different pretend scenarios where they are offered a certain amount of money after specific waiting times. The amount of money and waiting time change in each scenario. In essence, participants decide whether they would like a smaller monetary reward in a short time frame or a larger monetary reward in a longer time frame. This is a delay discounting task, which measures a person’s individual level of temporal discounting or how long they are willing to wait for a certain reward. This information helps researchers understand how people assess the value of rewards.

All these tasks measure aspects of behavior that can vary from person to person. For example, some people may quickly identify the emotion in a face, whereas others may be slower. Some people may feel better about receiving a larger reward, even if they must wait, whereas others may see the more immediate reward as more valuable. Mental disorders can impact these behaviors in different ways. Understanding how these behaviors change with mental health offers insights into how mental disorders affect the brain. It could also inform interventions or treatments so that they have the most benefit for patients.

How will the information from these tasks be used?

All the data collected as part of the All of Us Research Program, including information collected using these new tasks, will be made available in the future for researchers to analyze. The vast size of the All of Us dataset and the many types of information collected through the program—genetic, physical, mental health, etc.—allow researchers to make new connections between these health factors and expand our understanding of how they work together in health and illness. Researchers can visit ResearchAllofUs.org  to learn more or register to use the All of Us dataset.

Participation in All of Us is free and open to anyone 18 or older. People can learn more about joining the All of Us Research Program at JoinAllofUs.org .