IRP Podcast: Dr. Christopher Baker: How does the brain categorize the visual world and change with learning? (NIMH Podcast)

Summary

Dr. Christopher Baker, a vision neuroscientist at NIMH, discusses research on how the brain organizes visual input, and how the brain changes with learning.

Transcript

PETER BANDETTINI: Welcome to the Brain Experts podcast, where we meet neuroscience experts and talk about their work, the field in general, and where it's going. We hope to provide both education and inspiration. I am Peter Bandettini with the National Institute of Mental Health. Please note that the views expressed by the guests do not reflect NIMH policy. This is episode four with Chris Baker. We will discuss, among other things, how the brain categorizes the visual world and change with learning. Let's chat.

PETER BANDETTINI: Okay. Here we are with, uh, Chris Baker. And, uh, Dr. Baker is a principal investigator at NIMH. He started in 2006. And, uh, received his PhD in 1999 at, at, uh, University of St. Andrews in the in the UK. he studies, uh, visual processing. what motivated you to get interested in neuroscience?

CHRISTOPHER BAKER: So it's really-- uh, really a bit of a, a long trek in a way. when I was in high school, effectively, in the UK, I was very much into animal behavior. I was going to be David Attenborough. That was exactly where I wanted to go.

CHRISTOPHER BAKER: I went to Cambridge University. And the way the, the course works there, I just studied natural sciences that allows you to take different, um, subject groups. And in my second year, I had to choose a particular group and I didn't know what to choose. And I chose, um, experimental psychology. And I--

PETER BANDETTINI: Okay.

CHRISTOPHER BAKER:  --think going into experimental psychology-- at that particular time, I had a, supervisor who does a lot of color vision research. And I just got really engaged by experimental psychology. I was studying human behavior. And that's any sort of human cognition through human behavior-- really captivated me in the same way that I'd been sort of interested in animal behavior and zoology.
And from that point, I realized that I wanted to go into the, the human behavior, but then, also, look in the underlying mechanisms so I got drawn into neuroscience. And so when I finished at, uh, Cambridge, uh, my [inaudible] at the end was neuroscience. Even though I went in thinking I was going to be doing animal behavior.

PETER BANDETTINI: Huh.

CHRISTOPHER BAKER: So it's not so different. But that was really kind of the start of my, my path in this direction.

PETER BANDETTINI: What we all end up doing with our lives is a little chance and just these moments when you have an opportunity to do something, you realize, "This is something that I really find interesting."

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And I feel that's-- actually, my whole career has been a little bit like that.

PETER BANDETTINI: It's nice that, that as scientists we can sort of do that sort of thing. You know, try to keep track of what we're-- what we're interested in and then kind of move in those directions

CHRISTOPHER BAKER: That's kind of like one of the great freedoms that I've had in my career is being able to do that.

PETER BANDETTINI: So after St. Andrews, you first-- so you ac-- you actually did two postdocs. So you went to Carnegie Mellon and then you went to MIT. Uh, how were they similar, different? Why did you do two as opposed to one or, or [laughter]--?

CHRISTOPHER BAKER: Well, th-they're very different. I think-- I think it's, it's worth sort of saying a little bit about what I did during my PhD because, actually, every step that I've taken there's been quite large transitions. So during my-- PhD in St. Andrews, I was studying neurophysiology of vision. We were recording, uh, with single units-- single neurons in the brain while, um, non-human primates are looking live humans moving around the lab. It was a very exploratory kind of lab. So often we would-- we would find a-- find a neuron. So we're listening for the sound of a neuron firing. We find a neuron in this particular part of the brain where we know that we find a lot of neurons that tend to   respond well to faces. And then we'd try and drive this neuron as best we could. So we had, within the lab-- we had cells. We could give our cells stimuli. And our goal, at that time, was to often try and drive this neuron the best we can to learn--

PETER BANDETTINI: Yeah. One electrode--

CHRISTOPHER BAKER: --"What is this neuron doing?"

CHRISTOPHER BAKER: And so just incredibly exploratory work.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: It was a lot of fun, actually, [laughter] when you have this sort of challenge there [laughter] and you-- and we'd do a num-number of things. So sometimes we would sort of walk in front of the, the animal and suddenly the neuron goes crazy. And we're like, "Wait a minute. What was it about what I was doing that made this neuron fire?" And then you try and sort of work it out., as you progress, you try and make your tests more and more controlled. And, and that approach, I found really engaging. And at that time, I spent a lot of time thinking about how the neurophysiology of sort of, in that particular case, face and body recognition-- how that related to our understanding of development and to sort of more cognitive capacities and all these kind of things. And, and so I was doing a mixture of animal behavior, thinking about development, recording from single neurons. I see that as being like the core of where I am right now. I still keep coming back to those kind of ideas that I was looking at then and how I've used those in my career now.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: But I went from there, into a lab at Carnegie Mellon University that had a very different approach, where the approach was much more about doing kind of vision experiments. And that time, I started doing experiments more about learning. How do we learn to recognize objects?

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: But where the emphasis was very much on sort of control. And so I moved from this highly exploratory environment into this environment of, like, very deep control over, over the experiments we were running. It was just the complete polar opposite of what I'd done to earn my PhD.

PETER BANDETTINI: Yeah. Yeah. Yeah.

CHRISTOPHER BAKER: And then--

PETER BANDETTINI: You had a very specific hypotheses. You do it and then--

CHRISTOPHER BAKER: And you go through and it's just-- i-it's just a different approach to the science. And I think, for me, that experience and that change from going from one environment to another, where I went from one country to another, and realizing that there's no one way to do science.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: There's no one way everybody thinks about science, all these different ideas that people have.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: This really informs what I do, where it's like we try and find a balance between those two. And at that time, I was going to the meetings, like a neuroscience meeting, and started to see people who were doing human brain imaging, who were asking similar questions.

CHRISTOPHER BAKER: I-I wanted to try and see what this was all about out.

PETER BANDETTINI: Yeah. Okay.

CHRISTOPHER BAKER: This is what motivated me doing a second postdoc. I was like, "I just want to jump into a lab." That's when I joined Nancy Kanwisher's lab at MIT.

PETER BANDETTINI: You know, a-a-as neurophysiologists often say you're measuring, millions of neurons at a time.

CHRISTOPHER BAKER: Right. It's--

PETER BANDETTINI: Is it even going to be meaningful?

CHRISTOPHER BAKER: Exactly. It's such a complete change. you just have to accept that it's completely different.

CHRISTOPHER BAKER: And so you have these pros and cons and different techniques. And, and this is something that, again, I mean, you ment-
- you know, mentioned that I've worked in lots of different techniques. And partly, the drive for that is because there's no technique that's going--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --answer all of our questions.

PETER BANDETTINI: Right. Right.

CHRISTOPHER BAKER: And so th-then 2006, you came to the NIH. And we were very lucky to, to have you join. I-- uh, so Chris is part of the laboratory, brain and cognition and-- which, I'm also part of. you've established this, this wonderful lab that complements all the other labs at the NIH. there were two papers, that your group has written. the first one was published in NeuroImage. in 2019, the title of it is Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images..

CHRISTOPHER BAKER: One of the driving forces behind this paper was, the desire to test the brain's responses to, different images using a, diverse image set. I think one of the problems that we have in, trying to understand the brain is that we can only test this small number of stimuli. And the difficulty of that is that, well, then the stimuli we choose to test influences a little bit of what we're gonna see. So if we only test a small number of images and we think, "Oh, well, maybe we think faces are important, maybe put in a lot of faces." but we're kind of guaranteeing we're going to see something about faces because we've put them in our stimuli set. And maybe we didn't put cars in. And so one of the things w-we were trying to do here is to say, "Okay. W-when we do these tests, we need to use a larger set of images

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: We took, categories of things from, image databases that are used for advertising.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And we picked different types of, of categories that came from that. So they're not necessarily categories that a neuroscientist would necessarily think about but they're categories that we generally sort of understand. And we picked images from these different categories. And we had 48 different categories. We picked three images for each category. And this gave us kind of a broader sampling of what we see. And what we were sort of trying to address is, like, okay, we can-- we can look at these-- this set of, um, 48 categories and this set of images and say, "Well, how do we understand these images? Like, conceptually, how do the-- how do we think about these different images?" So w had categories that were things like houses, and farm animals, bags, and mountains. So it was a mixture of scene and object categories.
And you can say, "Well, okay. Maybe you think of things like houses and say, household objects, they would tend to go together because they-- you find them in similar locations. And maybe mountains and wild animals go together." And-- so we have this general sense. We want to understand both. When we've got a broader set of different types of stimuli how do we understand then, behaviorally, what's happening in the brain? And where do we see the sort of strongest links between what we see behaviorally and what we're seeing in the brain? And where do we see that?

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: So we asked people to, to arrange them on the computer screen, put things they think are similar, close together, things they think are very different, further apart.

PETER BANDETTINI: Yeah. so that's the behavioral side. So then the imaging side, the pattern of activation would be similar for similar categories?

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: "Doesn't it depend on the task that you're doing?"

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: If yours said, "Well, look for only brown objects," suddenly, you'll lump bears and trees together

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: So, so do you think that when people are doing this they're sort of making their own heuristics or they're making their own task? Or-- i-it depends on the task? Or do you think it's something more innate?

CHRISTOPHER BAKER: So I think-- I mean, the question of task is undoubtedly really, really important in some ways. And behaviorally, yes, if you ask people to arrange them according to color--

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: --they'll do something completely differently to what they would say if we asked them to do it based on, um, how   attractive this-- the, the items are, for example. We've been sort of trying to say, "Well, okay, there's lots of different ways in which you can view objects but     what's your general stance?" So we're not pushing people towards any one particular dimension or another because, actually, we don't know which way to think about it is the-- maybe the right way to think about it.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: So we're sort of relying on people having this-- the general sense of what they know about these items—

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER:  --and put them around. Um, the other thing that's sort of worth saying in this-- in this context is that, actually, task, in some ways, has bigger effects than behavior. But task in the brain is a little different in the sense that-- um, you know, we've actually done some, some experiments in the past where we have people looking at the same set of visual stimuli but doing different tasks.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And in some of the brain areas that are involved in sort of visual recognition, the effects of tasks are actually quite small.

PETER BANDETTINI: Huh, that's interesting.

CHRISTOPHER BAKER: And that's been kind of intriguing. It's not that there are none but there are some there. Now, there are other brain regions sort of that are involved in more parietal and frontal cortex but the [inaudible] a much bigger effect of task.

PETER BANDETTINI: Okay. Okay.

CHRISTOPHER BAKER: But, actually, the effect of task appears to be quite different across different parts of the brain.

PETER BANDETTINI: Huh.

CHRISTOPHER BAKER: So I think it's a really important question. I think, here, we're trying to get a basic sense because we didn't want to push people in one direction or another, initially.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: The way people group the stimuli, it made a lot of sense. I mean, one of the things that was very clear is people could see the categories. We had multiple images for each category. But people, behaviorally, tended to group those together.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: But also, the larger-scale organization of these things went together. So, the fact that—

PETER BANDETTINI: Okay.

CHRISTOPHER BAKER: --um, you know, animals and outdoor scenes tended to go together.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: Or household appliances and thing-- other things related to the houses tended to go together. PETER BANDETTINI: 20:16       
we have the behavior, we have the brain imaging, and if we do the brain imaging and we look at regions that people think are involved in -- visual object recognition, what we saw there was there wasn't a huge amount of agreement between the behavior and the brain imaging.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: Which in some ways, you could think of it as being maybe that's a little bit of a puzzle. Shouldn't these be similar? If people understand these images in this particular way, shouldn't the brain region involved in that recognition also show a very similar pattern?

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And they were related but they were completely different. But it wasn't a very strong relationship.

PETER BANDETTINI: Yeah. Yes.

CHRISTOPHER BAKER: And so that's when we turned to sort of the deep neural networks to say, "Well, okay, let's take a model of visual processing--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER:  --and say, well, what does that do with those images?"

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And then how does that relate to both the behavior and to the brain imaging data?

PETER BANDETTINI: Yeah. Yeah. Okay.

CHRISTOPHER BAKER: And that was really trying to get a sense of like, "Well, maybe how-- you know, maybe this will give us some extra sense of how to understand these kind of data."

PETER BANDETTINI: This mathematical model that had many different layers showed, you know, there's layers, it's sequential processing of the images. And then, finally, at the, the highest layers, represent sort of maybe the, the final ability to sort of categorize the objects?

CHRISTOPHER BAKER: Right. Classify what's in the image.

PETER BANDETTINI: And so you found that with the behavioral judgments, , the final layer -- showed a pretty strong agreement, up to 80% agreement, uh, correlation -- of that with the behavioral judgments. But, um, but it was interesting that when you compared to, the fMRI data itself, the pattern, it seems that it sort of went up quickly and then about layer five it sort of peaked as to--

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: S-so there's some intermediate step in which the fMRI data most agreed with.

CHRISTOPHER BAKER: Right. and this paper and some other work that we've done, we've been comparing the behavior, the fMRI, and the deep neural networks. And it seems there's no simple relationship between them. Because on the one hand you might think, "Oh, look. This deep neural network really agrees-- so there's nice strong agreement with behavioral data." So you assume that it should just, then, look like the fMRI data. But, actually, that three-way relationship is kind of sort of complicated to understand.

PETER BANDETTINI: Yeah. there's no reason to believe, necessarily, that you might have, you know, very different pools of neurons activating the same voxels in some sense. So maybe it--

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: Maybe we're looking at a scale that, you know, is not really relevant, uh, in that regard. But, luckily—

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: --it seems like it's close. It's close enough that -- there's some agreement. But, , within a voxel you have a million neurons and they could all be categorizing th-themselves. I mean, with, with--

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: --you know, grouping with other neurons that are all over the place in the brain. And so, so you might be able to get some different pattern if you looked at a higher spatial resolution in some sense, so.

CHRISTOPHER BAKER: Right. I mean, I think there's a-- there's a general assumption on the fMRI that, that we should see that in a sense we're looking somewhat local representation.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: 'Cause we're looking in a particular region and we're saying, "Okay. Let's look at the pattern response in this area."

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And if things are more distributed then, actually, that's not going to show us a strong relationship.

PETER BANDETTINI: Yes. Yes.

CHRISTOPHER BAKER: And I think, you know, one of the things here when we look between the behavior, the fMRI, and the deep neural network is that there all capturing different aspects of what's going on. And that's why they have some agreement between them. But, actually, when you look closely, they're capturing slightly different things.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And that's kind of important to understand that's it's sort of looking at deep neural networks, people tend to look at, like, the bottom line. Like, you know, how similar is this on average--

PETER BANDETTINI: Right. Yeah.

CHRISTOPHER BAKER: --to, say, behavior or the brain? And show, "Oh, look. We get a significant relationship between this model and the brain." But actually, we need to go a little bit deeper and understand what the sort of-- well, we might think of it-- what the representational structure is. And how--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: -- similar is it? And which parts of it are being captured and which parts aren't being captured?

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: And, actually, that kind of data can be, can be really useful in saying, "Okay. What does it say that the deep neural networks aren't capturing about brain activity or behavior? And then what would it take, then, to make them capture that part of the-- of that data?"

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And that may be a way to say, "This is how we need to sort of think about the models, how we might modify the models to capture other parts of the data."

PETER BANDETTINI: So that brings up, actually, another interesting point is that, um, that in some way you're using deep neural nets to the degree that it can agree with the patterns in fMRI activation, it seems that, there's sort of an implicit sort of thought that, that this might be representing what's potentially, at least at, some rough level, how the brain is organized itself.

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: And, and if you can start to generate these deep neural nets and maybe say, "Oh, well, maybe there is this, this sort of hierarchical organization in the brain that takes, you know, color, and shape, and texture, and other higher-level context and analyzes them at different levels and then, finally, comes up with something. And if it can agree with this, you can start to say something more about principles of how the brain is organized--

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: --at least the visual processing in, in the brain.

CHRISTOPHER BAKER: Well, I think it's, it's in, in a particular way. there's one way you can say, "Okay. Look.  the performance of these networks in like an object recognition task, they are, actually, arguably, super-human now. Right? They actually, certainly, do as well as    most people in, say, labeling an image in terms of what objects are there.

PETER BANDETTINI: And, and they're robust [crosstalk].

CHRISTOPHER BAKER: And they're robust.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: On the one hand, that's fantastic. And it's fantastic from an engineering viewpoint.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And it's fantastic in what I'm able to use these deep networks for in, in, you know, all aspects of our life. Um, but there's this question, well, what does that tell us? You know, how then do we use that understanding? What are we gaining from understanding from that in terms of looking at the brain?

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And, um, the way I think of it is w-where I think there's, uh, there's an element of caution that we need, which is to say, "Well--" it's an analogy that I've used, um, is to say, well, let's take a, um, a baseball pitcher and let's take a baseball pitching machine. And you look at, like, the output. And the output of these two things, they're very similar, right? You can have these pitching machines, can produce every different type of pitch that you want.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And for most people, that's gonna be as good or indistinguishable from what a baseball pitcher can throw.

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: But it doesn't make it necessarily a model of how the brain does it. How, , the real pitcher does it.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And the same way these deep, you know, network models may not be a, a precise model of how the brain does it but it doesn't mean they offer no insight. And so I feel that we have to be careful looking only at performance--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --in terms of interpreting this.

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: And I think at the same time there were inspired by thinking about the brain.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: This idea of having layers and the ways in which some of the processing steps go on, they're inspired from neuroscience. And the way that I think about them is being like a very useful tool. So, I don't think about them as necessarily being a model of the brain.

PETER BANDETTINI: Right. Right.

CHRISTOPHER BAKER: But they're a very useful tool to say, "Okay. We have these, these networks showing the millions of images and training them on what, what is actually shown in those images. And they, essentially, learn over time how to relate the image features to what the object is."

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And if you find, as we sometimes find, that, actually, what's captured by those networks, actually, is similar to what we see in the brain. It really is telling us something.

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: And telling is that, in fact, this, this sort of process of linking visual features to, um, object identification—

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: --is producing kind of the types of representations we're seeing in, in, in different parts of the cortex. So it's telling us something there but then maybe we don't take it as like a very precise model. Because a different way in which we were meant to think about it is, well, you know, the, the brain is a-- is a biological organ.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And for some things we want to understand, we actually want to understand it as a biological organ. And I think, particularly, if we, um-- if we want to go in and, say, look at doing an intervention, right? Actually, a computational kind of understanding that might come from a deep neural network is not gonna be that helpful.

PETER BANDETTINI: Right. Right.

CHRISTOPHER BAKER: It's we need a kind of a biological understanding as well.

PETER BANDETTINI: Yeah. Yes.

CHRISTOPHER BAKER: And so I think it's, you know, deep neural nets is one type model but there's other types of models that we might want to be looking at that actually stick closer to sort of the biological side of things.

PETER BANDETTINI: Okay.

CHRISTOPHER BAKER: And somewhere, of course, there's ways in which these can be combined because—

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --in principle, you could take biological principles, or you could take models of the firing of individual neurons and build—

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: --those into the deep neural network models or different things like that.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: But there's a lot of different approaches we can use, and it depends on exactly what aspect of the brain we're trying to u- understand.

PETER BANDETTINI: I think, you know, one reason when people first talked about, "Okay. Here's this, this face area in the brain," I think one reason why people were sort of maybe willing to acknowledge this would like it to be something real-- was that we, we also knew that there were patients who had brain damage, who couldn't recognize faces anymore.

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: -- so when you have evidence of people who may have brain lesions in this part of the brain, who can't recognize faces anymore and you find this activation where it seems to be somewhat specific to faces, then this provides a more compelling eye for what's going on. it's one of the reasons why, within my lab, you know, we started to think about trying to use, um, brain stimulation approaches, approaches where you can go in and say, "Okay. I have an idea about what this brain region might be doing. I'm now going to try and interfere with this brain region using--

PETER BANDETTINI: Yeah. Yes.

CHRISTOPHER BAKER: --magnetic stimulation," for example.

PETER BANDETTINI: Yep.

CHRISTOPHER BAKER: And say, "Well, what effect does that have on behavior?" And provide that kind of more sort of causal kind of connection in that sense.

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: There's a different way in which we can, can ask these questions.

PETER BANDETTINI: Yeah. And it might even be, um-- you know, it might be a potential thing, not only for research, for understanding the causal connections, but, you know, maybe for, you know, therapy, you know?

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: It might be-- you know, once you know exactly, you know, what structures are important for what, you might try to go in with neural modulation of some sort--

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: --and either stimulate or oblate these areas, so-- which could be a great therapy if it-- if it catches on. the other paper is trying to get at how the brain changes with experience and learning why don't you describe what you found in this paper?

CHRISTOPHER BAKER: We had been looking at all these papers that had talked about changes in the structure of the brain with learning. we had some concerns about some of the things they were trying, some of the controls we thought they should've used or different things they were doing. And, you know, we decided that we wanted to actually look at this more deeply ourselves. the idea was to collect the data set we felt you needed to have to really be able to address these questions. Then we brought in volunteers, into the study over a four-week period, we scanned them eight times. And so what would happen is we would scan them in the morning, they'd have a break, and then we'd scan them again in the afternoon. Sometimes    during that break, they would do a task. Sometimes they're doing nothing. Um, so that task is something that we're   training them on. Um, and what we're trying to do is to say, "Okay. When you train on something what's actually   changing in the brain?" And, you know, we, we kind of know from lots of animal studies, that there are changes that happen as you learn different things. lots of different plasticity mechanisms. These can operate at the level of, um, uh, synapses changing, dendrites changing, all these different kind of things that are happening. When you come to MRI and brain imaging, the question is, those are happening at a very small scale. You know, how can we really pick these things up with, with--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --brain imaging. And, and, in principle, this is really important for us to understand, to be able to look at things like if we want to sort of look at like, how can we ameliorate the effects of aging on the brain? Right? Understanding what's exactly changing and exactly how training may affect those changes is really important. We did a number of different types of, of MRI on them. So one way to look at the brain structure, one way to look at the resting-state functional brain activity, another way to look more at, like, um, sort of the white matter, the connections within, within the brain--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --and see how they were changing with learning. And I think, you know, one of the first things that we, we found was something a little bit surprising to us, overall intracranial brain volume it changed from morning to afternoon.

PETER BANDETTINI: Did it go up or down?

CHRISTOPHER BAKER: It went, um, down.

PETER BANDETTINI: Okay. Okay.

CHRISTOPHER BAKER: So the amount of gray matter appeared to go down. Actually, the amount of CSF or cerebrospinal fluid appeared to go up.

PETER BANDETTINI: Yes. Yeah.

CHRISTOPHER BAKER: Um, and so we saw this pattern. And for these people-- they came in on four separate days. And we saw the same pattern in most of the people that we tested. And it was like, "Okay. There's some change that's happening over the course of the day to the brain."

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And suddenly, you look at this and you're like, "Well, okay. This is suddenly really important." Because when you're trying to look and understand what's happening with training, there may be this other effect that's happening, which happened even on the days when people weren't training.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: So we knew it wasn't related to the training they were doing.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: It was just something that's happening over the course of the day. And so we've-- we decided to refer to this as, like, the Time of Day Effect because we don't know exactly what's going on--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --but it has been really kind of important to understand.

PETER BANDETTINI: Yeah. So, over the course of the day, our brains, you know, for whatever reason, I always thought it was hydration changes or things like that but it seems like it's more, more involved than that, uh, that our brains--

CHRISTOPHER BAKER: It's-- yeah.

PETER BANDETTINI: -- change in volume.

CHRISTOPHER BAKER: And I think there's lots of ways this can happen. And some things-- the hydration does have some effect.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: It can also just be, you know, um, something related to cerebrospinal fluid over the course of the day, you're more vertical, there's some compression that happens at the skull [inaudible]. You know--

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: --or something relating to blood flow. I think the challenge is trying to understand exactly what's going on.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: Um, but it just leads you to sort of realize that actually, the brain is not a static structure. I mean, sometimes we tend to think of it as being, like, "Oh--

PETER BANDETTINI: It's just there.

CHRISTOPHER BAKER: --here's this structure of the brain and it's just there." And it's like, no. Actually, this is a biological organ and there's blood flowing through, there's cerebrospinal flowing through. There are all these changes that are happening.

PETER BANDETTINI: Yes. Yeah.

CHRISTOPHER BAKER: --and they have some impact on the things that we measure. And the question for us, then, is to say, "Okay. Once you've done that-- once you realize there is some effect just over the course of a day, well, okay, now, you've got to be really careful and looking for effects of, of brain plasticity and training--

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: --and take this into account." Um, and that's, , what we've been trying to do since then. that, for us, was an unexpected observation but it's one that you can't ignore.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: The way our study was designed, we had some days where people didn't go to training, some days where they did do training, which allows us to sort of try and, you know, factor--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --out the effect of just the time of day—

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --to look at what those training effects are. ‘

PETER BANDETTINI: Yeah. Okay. Okay.

CHRISTOPHER BAKER: This paper, is one where we were focused on the resting-state activity.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And in this one, we can see that there are effects of time of day in resting-state activity. And some other people have found these effects as well before. But important in this particular case, we could look at like, what's the effect of training as well. And so we actually do find some effects of training that are separate from the effects of the time of day.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: But it's just really important to take these two things into account.

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: And so we actually-- we have announced it in the paper where we looked at what if you didn't take time of day into account, what would you find? And you'll find many more changes--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --that are actually unrelated to sort of brain plasticity per se in terms of, like, you know, what we think may be the underlying neural changes--

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: --and may be related to some other aspect of, of, of the physiology of the brain.

PETER BANDETTINI: It was a really interesting paper too., where you tend to see changes-- seemed like they overlapped a little bit but are not really, uh, necessarily coincident with what is active during the task as well.

CHRISTOPHER BAKER: Right. Right.

PETER BANDETTINI: So that's kind of-- that's intriguing in itself.

CHRISTOPHER BAKER: What you often see is, you run a study and you find a sort of particular brain patterns and then you may want to try and interpret them after the fact.        

CHRISTOPHER BAKER: Um, and, yeah, it just highlights some of the challenges of, of, of brain imaging.

PETER BANDETTINI: What is actually changing? I mean, at least with the connectivity change, you can imagine nerves, you know, uh, uh-- connections become stronger. But, but, you know, there's also anatomic changes that people see.

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: That still seems like an open question as to-- what is actually this mechanism of change?

CHRISTOPHER BAKER: One of the things that people have been looking at is the effect of exercise on brain function.

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: There are effects of exercise and going through periods of exercise on, on the brain.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And, and those may be related to effects on, on blood flow. But it doesn't mean that that's not relevant for understanding the impact of exercise. Because it might be that the changes in blood flow actually improve neural function.

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: And that all of this is kind of interlinked together. And I think—

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --that's the other thing that we sort of see is, you know, when you see effects like, oh, there's something happening in the brain over the course of the day, people think that, "Okay. This may mean that it's not interesting or relevant." And I, actually, think, actually, no. I think we have a full understanding of how kind of blood flow, neural activity, CSF flow, all these things are very tightly interlinked--

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: --in this organ. And we need to understand all of those together.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: And, actually, it, it may be through affecting individual components that actually you have some impact on—

PETER BANDETTINI: Yes.

CHRISTOPHER BAKER: --overall sort of behavioral function.

PETER BANDETTINI: Yeah. And also, of course, sleep too. I mean, it—

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: --seems like consolidation or pruning and, and things like that that occur, um, so.

CHRISTOPHER BAKER: Right. So there's all these different areas. And these are really fascinating questions.

PETER BANDETTINI: Yeah. Yeah.

CHRISTOPHER BAKER: I mean, this study is one that got us, you know, thinking a lot about that. For example, sleep's not something we manipulated here or tried to control for but--

PETER BANDETTINI: Yeah.

CHRISTOPHER BAKER: --clearly, we think that's going to have some, some major role.

PETER BANDETTINI: Yeah. That would be interesting, actually, to have them learn, you know-- see their brain as it changes with them-- I mean, it's a really hard study to design given the time of day effects. But if you just keep the same time of day and have them learn, sleep, learn, sleep and, you know, maybe see how things get consolidated or change or whatever. I mean, that's just, yeah, one iteration, but.

CHRISTOPHER BAKER: Yeah.

PETER BANDETTINI: It's really interesting. I agree with you. I think, understanding how the brain changes across temporal scales, uh, either with functional connectivity and where it's occurring relative to the task. And, also, you know, long temporal scales with structural changes. Some people even think structural changes take place in the course of a day too.

CHRISTOPHER BAKER: Right.

PETER BANDETTINI: Um, uh, so it's all kind of wide open. And--

CHRISTOPHER BAKER: Yeah. And I think there are-- and there's studies in, in, um, in rodents showing that you can have growth of new dendrites over the course of hours, right? So there's--

PETER BANDETTINI: Right.

CHRISTOPHER BAKER: --there's some, some things that can change in that time scale.

PETER BANDETTINI: Yeah. That's completely interesting. So, it's wide open for study. this is a really nice paper as well. thanks for spending the time to, to talk with us. I-I really enjoyed talking about your papers, talking about your perspective, how the field sort of is advancing, self-correcting itself, and, and, you know, more deeply going beyond cartography to actually understand the brain itself, so. Well, thanks a lot.

CHRISTOPHER BAKER: Oh, thanks, Peter. It's been very, very fun talking with you.