Identifying New Directions in Mental Health Disparities Research: Innovations with a Multidimensional Lens - Day Two, Part Two

Transcript

CRYSTAL: Good afternoon, and welcome. Welcome back, everyone. I see we are slowly, but surely, welcoming in and accepting our attendees and registrants. Thank you. Welcome back to everyone. Again, good afternoon. My name is Crystal Barksdale, and I am the Chief of Minority Mental Health Research in the National Institute of Mental Health, Office for Disparities Research and Workforce Diversity or the NIMH ODWD. 

On behalf of the NIMH ODWD and the National Institute on Minority Health and Health Disparities, NIMHD, I want to welcome you back to day two of our virtual Disparities Workshop, “Identifying New Direction and Mental health Disparities Research, Innovations with a Multidimensional Lens.” We had a wonderful afternoon yesterday, hearing about the innovative disparities research from several of our presenters and engaging in great discussions about gap areas, the excellent challenges of doing mental health disparity research, and opportunities to advance the science. And we look forward to hearing from the remainder of our dynamic presenters and facilitating robust discussions this afternoon, particularly during our final roundtable discussion, where we aim to hear from presenters collectively about the challenges and opportunities for advancing a multidimensional mental health disparities agenda. 

We encourage registrants to please submit your questions for presenters to disparitiesworkshop@mail.nih.gov to be read as time permits during the roundtable discussion. As a reminder, this workshop will be recorded and made accessible on the NIMH website. This typically takes a bit of time, but we invite you to use this option if you're unable to attend any part of the workshop in real time. We also have several options for you to share your thoughts with us after the meeting. We are currently seeking input from interested parties on innovative research priorities to improve mental health outcomes among minority and health disparities populations through a request for information. You may also send questions or comments to the NIMH OD office e‑mail. Next slide please.

Finally, I just want to review a few housekeeping issues. You're entering into the workshop in listen‑only mode, and, again, questions are being taken exclusively via e‑mail, so we ask that you please, for registrants submit your questions for panelists to the e‑mail address listed. Questions that are not addressed during the meeting may be responded to post‑meeting, and the e‑mail address will be periodically posted in the chat box. For any technical assistance, please e‑mail events@1sourceevents.com. Now, I'm pleased to introduce Dr. Andrea Beckel‑Mitchener, the Director of the National Institute of Mental Health Office for Disparity Research and Workforce Diversity. The Director for the National Institute of Minority Health, Mental Health, please excuse me; Office of Rural Mental Health Research and Acting Deputy Director of the NIH Brain Initiative, who will be providing a few welcome remarks. I warmly welcome and thank Dr. Beckel‑Mitchener for her remarks. Dr. Beckel‑Mitchener?

ANDREA: Thank you. Thank you so much, Crystal. It is a pleasure to be here on day two. Welcome back. Good afternoon for those of you where it is afternoon. This is day two of the Virtual Mental Health Disparities Workshop, and I'm very pleased to be here. I just want to spend a quick moment to thank a few people. Thanks to Crystal Barksdale, Denise Juliano-Bult, and Jennifer Alvidrez for their leadership organizing this workshop, as well as the directors and deputy directors of both NIMH and NIMHD for supporting this endeavor. 

You heard from Drs. Gordon and Webb Hooper yesterday, and there are great many NIH staff involved and many more sort of lurking in the background as active listeners. So, many of you, I know the program staff who volunteered to moderate the sessions, and on a personal note, I want to express my gratitude for these individuals at NIMH and NIMHD. It's a privilege to work with them every day. So, also, a quick thanks to The Bizzell Group, who are helping with the logistics on this meeting. Especially Jonelle and TaRaena and there are others as well. And I want to extend a thank you to Sandra Molina in our office, who is always keeping everything and everyone on track. So, thank you Sandra.

Finally, I'm not going to spend a lot of time, but I do want to recognize all of the presenters and those of you that are attending. We've had a tremendous response to this virtual workshop, and I know we've all spent many months attending meetings on Zoom and other virtual platforms, and I just want to extend my appreciation to those of you who have joined us in the U.S. and around the world. I know many of you are dealing with very significant challenges right now, and we're really pleased that you've been able to spend some of your time with us, both yesterday and today. So, we're all looking forward to another great day of talks and discussion and the wrap‑up session especially, as Crystal just noted. We are here to listen and learn. These meetings are essential as we integrate the knowledge gained into identifying our research priorities and developing paths toward achieving equity. So, again, my thanks to all of you, and I will now pass the baton to my colleague, Dr. Denny Pintello, who will introduce herself, and the first session of the day. So, again, welcome, and on to Denny.

DENNY: Good afternoon. Thank you, Andrea. Good afternoon, everybody, and good morning to the rest of the country. We have a number of folks joining in. I'm so excited, and as Andrea said, I'm Denny Pintello, and I oversee the Dissemination and Implementation Research Portfolio here at NIMH. 

And I'm really excited to moderate the panel, the next panel, which is composed of five implementation scientists who will focus on a number of different areas that address disparities through implementation research. And those five panelists are Miya Barnett at the University of California at Santa Barbara. She's looking at workforce issues to reduce mental health disparities in services access. The next speaker will be Miraj Desai from Yale University School of Medicine, looking at implicit bias, organizational bias. The third speaker will be Tiffany Haynes at the University of Arkansas for Medical Sciences, and she'll be examining, or presenting promoting emotional wellness in rural African American churches. Our fourth speaker will be Rheanna Platt from University, Johns Hopkins University School of Medicine, focusing on redesigning pediatric care for families, and then our last speaker will be Deepa Sekhar from Pennsylvania State College of Medicine, and she will be addressing disparities through depression screening in high schools. So, as you know, each of these presentations will be for about 7 minutes, and we are going to ask our audience to verbally hang on to your questions till, our, our roundtable discussion, and then you can, of course, send your questions to disparitiesworkshop@mail.nih.gov so that we can address them in the subsequent roundtable discussion. So, what I'd like to do is go ahead and turn it over to Miya Barnett, and from there, we will go forward and have a great presentation.

MIYA: Thank you so much Denny, and thank you, everyone, for having me here today. It is really an honor to talk about how lay health workers can be a scalable workforce to reduce child mental health disparities in service access. To start with, I don't have any conflicts of interest to disclose, and I want to thank the generous funding from NIMH so that I'm able to do this work. 

So, the objectives of my talk today are to talk about why we might want to think about mobilizing lay health workers, what different lay health worker models of care are, how lay health workers reduce disparities for access to evidence‑based treatments, and provide a case example of LEEP, which is the work I'm doing currently with my KO1 award. Mental health disparities are global. We know in low and middle‑income countries that individuals who need treatment, 75 percent of them don't receive them, and it's been estimated that we need close to 240,000 additional providers to be able to meet those needs. So, the World Health Organization Mental Health Action Program proposed the use of lay health workers to help address these needs. Oops. Hold on one second. I'm just having a little technical issue. That should do it. Okay, got it. At the same time, disparities are also present in the United States. In a recent study, by Whitney & Peterson, they showed where children are not able to access care and how it varies by region. In the lightest states, 30 to 40 percent of children who need mental health treatment aren't getting it, and in the darkest areas, it's up to 72 percent, which might sound very similar to the rates that were seen in low and middle‑income countries. If you look at regions that have the highest proportion of people by color, you see that many of these places with disparities to access occur are also regions that have larger racial and ethnic minority populations. So, I recently completed a review that talked about how lay health workers could be involved in the delivery of evidence‑based treatment to address disparities in care. In this conceptual model, we're really looking at different drivers to mental health disparities. The supply drivers, talking about the limited workforce, especially workforce to provide culturally and linguistically competent services to populations that might not be fluent in English. The demand side talks about how even when you bring evidence‑based treatments into community settings, stigma, structural barriers, and unfamiliarity with treatments and skills and symptoms can lead to low utilization of these treatments, and both of these sides can really exacerbate mental health disparities.

So, lay health workers could be a workforce solution. They are members of the community. They serve without formalized mental health training, and this is a model that has been used extensively within physical health with recent recognition of the importance it could have in the mental health workforce. We've heard already from many of the panelists different terms that can be incapsulated in the lay health worker workforce, including Parent or Peer Support Partners, Patient Navigators, Community Health Workers, or Promotoras de Salud, in the Latin community, and all of these individuals have different types of work that they do, different roles, but they have lived experience. And they're seen as engagement specialists. So, I'm really talking broadly about this workforce. Lay health workers can address the supply issues because they are more likely to be bilingual providers. They can have specialized roles addressing different issues related to engagement, and in that, they can be conducting outreach, doing work to address stigma, and overall addressing barriers to care. And we're hopeful that this, in turn, can lead to better mental health parity. So, lay health workers really have different roles that they can play in mental health delivery, from more of the navigator and outreach role, where you're doing work in the community, screening and providing referrals, being bridges to services, to sharing those roles with mental health providers, case management, helping to improve engagement and adherence. Task‑shifting, on the other hand, is when you are having the lay health worker serve as the people who provide the evidence‑based treatment, either in stepped‑care models where they are doing more prevention level work, or as the primary providers for all individuals. 

In a recently completed systematic review that I did, we found that the majority of research with lay health workers has studied task‑shifting to have lay health workers be the primary providers. Even though this role is not considered the primary role they would have in the United States, it is still where the research is being done, and that there are some real challenges in identifying what should happen in high‑income countries, that the majority of this work has been done in low and middle‑income countries, and that specifically in the United States, when we're looking at lay health worker models of care, it's less likely to be an evidence‑based treatment or even a treatment that has been supported as having principles of evidence‑based practices.

So, now, I want to talk about the work I'm currently doing on my KO1 award, which is “Mobilizing Lay Health Workers to Improve, PCIT Implementation.” PCIT is parent‑child interaction therapy. It's an evidence‑based treatment for young children with behavior problems that has a long history of being shown to be effective and has now been widely disseminated into community settings. At the same time, there are many challenges with engagement regarding recruitment of appropriate families into care, adherence to the treatment protocol from the family side, that they're practicing the skills at home, and attrition that impacts multiple levels. So, obviously, at the client level, if they're not engaging in care, they are not having the same positive impacts on children's behaviors and long‑term mental health outcomes. Therapists have had a hard time with implementation in getting certified in this treatment, when they have a hard time having appropriate referrals and also graduating cases because of difficulty with families completing treatment. Agencies invest a lot of money when they bring in evidence‑based treatment, like PCIT, into, their services, and if they are not having the success of reaching families, then they are not having the return on investment, and, obviously, public health is impacted when there is not good return on investment with the children that are reached by these evidence‑based treatments.

So, I've developed the LEEP model, which combines professional provision of parent‑child interaction therapy with lay health workers providing the outreach, care coordination, and support of home practice to improve both client and implementation outcomes of PCIT. And I'm currently completing a hybrid type two trial of LEEP as my K01 Award, that's looking at improving, like I said, both clinical and engagement outcomes and implementation outcomes, specifically reach and the number of families that get PCIT. The preliminary work and the first aim of this KO1 award partnered closely with a network of, Promotoras de Salud, which is a lay health worker population for the Latino population, like I said. It was to understand more where implementation challenges exist to leveraging this workforce here in the United States for, linguistic minority individuals, and there are a number of themes that came up in our qualitative interview. First, financing is not guaranteed with these Promotoras, and when we interviewed agency leaders, it was similarly talked about how difficulties with having sustainable funding sources, like billing and claiming, really limits the ability to leverage this workforce. The Promotoras talked about how they received limited training and supervision when they were brought on to projects. And that they struggled because the programs that they would be brought into for different physical and mental health issues rarely were sustained, that they were often solely the, being funded through a grant or through a foundation for a limited time, and then they would have to switch to another issue. And this is exemplified by one quote, where the Promotoras talked about, saying we know Promotoras work with love, this is why we do the work, but Promotoras also have needs, and, so, really, this issue around funding community health workers is critical for us to continue to think about how these individuals can be addressing disparities. So, we have been working hard to have resources available to the Promotoras to help with engagement in PCIT. All of them have an iPad that has an i‑Book with different resources where they're engaging parents, we have different videos they can show parents to help them understand what treatment's going to look like and help them choose to do PCIT opposed to more treatment as usual that doesn’t have a strong of an evidence based. Including parent testimonials, there are scripts to help with their adherence to explaining PCIT and different handouts that they can share with the parents. And what you see here is a video, or a screen‑shot of one of our videos, which was recently shifted when we moved everything to tele‑health. This has required an extensive amount of re‑training of the Promotoras to feel comfortable with tele‑health, and also videos to show parents how tele‑health works, because we are seeing a lower uptake of tele‑health services from the Spanish‑speaking population here. 

I'll leave you with implementation questions that continue to be key and central to the work I do and I think the work that many of us are doing. Which is that we really have to figure out what roles are most appropriate for lay health workers in high‑income countries as we juggle with how there's a lack of providers, but also many reasons why they might not have complete task‑shifting to be the primary providers in a sustainable way. We'll continue to investigate what implementation strategies and supports are necessary for lay health workers to be successful in providing evidence‑based treatment or support of evidence‑based treatments with fidelity, and what policy levers could be used to sustain lay health worker models of care. And with that, I thank you, and I pass it off to Dr. Miraj Desai.

MIRAJ: All right. Thank you. All right, so, thank you for, the genuine, thank you for the introduction and the genuine honor of this invitation and the opportunity to present some of my and my team's work. Today, I'll be giving you some snapshots of our research exploring structural barriers to culturally responsive care, including our recent delineation of a novel concept of implicit organizational bias. I will also discuss our research on more, community‑based research and interventions on mental health disparities. This overall work, of course, follows from growing concern within the health fields that social, institutional, and collective structures are often a major cause or determinant of ill health downstream. So, we see our work as attempting to contribute to this wider health and mental health movement of which many in this room are leaders or members. 

All right, so, our objectives for today will be to briefly introduce our framework of implicit organizational bias, and then to discuss research and interventions in the world outside the clinic.

And finally, to discuss the overall importance of addressing the structural and social determinants of mental health. So, within psychology, psychiatry, and medicine and far beyond, the notion of bias is usually studied by way of implicit bias. Many of you have heard this phrase by now, but implicit bias is often defined as an unconscious form of prejudice believed to negatively influence things like patient‑provider encounters, even for providers with overt commitments to racial and gender equality, for instance. Our most recent work, however, has developed the notion of implicit bias that operates at levels, at higher levels within healthcare, education, law, and beyond. Implicit organizational bias can be defined as the ways in which largely hidden norms, practices, procedures, and shared perceptions of institutions, as a whole can implicitly and systematically bias against people of color and other groups. Here, we view institutions as entities or agents unto themselves. In other words, institutions themselves can hold biases, implicit ones, not just the personnel within them, and, again, this falls from a long line of scholarship and other fields and from scholars of color in particular on institutional bias, which we see our work as contributing to, particularly given that several scientific reviews have noted lack of sustained research into this area, especially from health and mental health.

So, the word structure gets discussed a lot, but I don't see the science as actually addressing the structure as deeply and pervasively as we can be. So, we developed this concept through an empirical research project funded by a NIMH administrative supplement for Minority Health and Mental Health Disparities Research which I should mention upfront. Very grateful for that, and that project was designed to examine cultural and organizational factors influencing how, person‑centered care, planning and other practices in particular for being taken up by members of the Latinx and Asian communities. We articulated this concept through a series of papers, most recently one in America's Psychologist, but there are a few others as well. We don't have time to get into the thick of how we arrived at the concept of implicit organizational bias, but what I will say is that it developed from the ground‑up, by analyzing very basic encounters between providers and their clients of color. Including first meetings, decision‑making patterns, and any practice modifications that were made. So, beginning with this basic point of contact, our attention soon became redirected towards the structures around which this contact took place. So, a clue emerged that began to alert us to the presence of implicit organizational bias. We started noticing, essentially, that individual providers noted similar reasons, this is a qualitative study, so when you see, you interview providers and you start looking at each individual provider interview, they start noting similar reasons when their process with clients of color broke down. Such as when the client was not verbal, did not admit to an illness or problem, did not accept services, or had a presenting problem that did not fit with the predominant individualist ethos of the clinic. So, these seemingly isolated conflicts were indicative of a pattern, and beyond the pattern, a mental health clinic culture, and beyond a culture, a structure with a hidden architecture that preceded, surrounded, and influenced the very reality of the clinic. This hidden architecture was marked by majority group norms, expectations, and biases regarding mental health and correct conduct, and this hidden architecture was not just the simple matter of preference, it was a force that received its potency by being inscribed within policies, procedures, financial and bureaucratic requirements for efficiency. There was, indeed, no system without them. It's what the system required to operate. Thus, in the study of cultural diversity, we found that the culture and the structure of the mental health system may pose the greatest barrier to a more robust engagement through client diversity, local variation and creativity by open and flexible clinicians was possible. But deep adherence to these norms was what you, as a client, had to do or be for the system to work optimally for you rather than, in some cases, against you. This is just a brief, quote, describing this corresponding ideal client for whom the system is actually designed to work most optimally. So, which we termed the ideal client construct. So, at some point, he says, “the client was easy.” What did you mean by easy? “They're low‑maintenance, come in, communicate, are open, are willing to change, they're working on the recovery, you have no issues with them, they're motivated.”

Now, these are the examples of non‑ideal clients by virtue of them conflicting with the expected norms or speech, illness belief, activity level, and so forth. Being verbal was a part of this, so not speaking much. “Like, I'll ask her questions, she won't answer them, so we'll just wait until you answer. Other providers would be like, can you please just answer the question? Like, get irritable at her.” “In a group, he got something out of listening to others, but you also want them to be able to share some of his experiences. He was, quote, “limited in that way, and I just think that's who he is. He's a decent person contributing to society, but his ideas are simple, simple‑minded.” So, here, we actually see the provider negatively interpreting non‑verbal behavior as being indicative of someone who's simple‑minded. I won't have time to get into everything, but another one is when they don't admit to the fact that mental health is a reality, it means you have a problem, it means you need help, you need medication, not doing anything. “He just wanted to talk about going back to school. He didn't understand why he was there, wasn't doing what he said he was doing.” So, implicit organizational bias is thus defined as the unspoken structure, norms, belief, and expectations about the way clients should ideally behave and/or interact with the mental health system to gain optimal benefit, which in turn relates to an ideal client construct that exists within nearly every imaginable institutional space.

And just in closing, it identifies, we believe this concept identifies a relatively undefined mechanism of organizational structure that may strongly hinder minority health. It responds to the lack of focus on racialized structures within research and implementation and organizational social contexts, culture, and the rest, and that extends to influential science of implicit bias to structures beyond the individual. We often talk about adapting practices to other cultures, but we talk less about what culture we are adapting it from and the extent to which that culture's embedded in not only treatments, but in policies, procedures, and in organizational norms. Just briefly, there's, in the world outside the clinic, where we focus on, building partnerships with the New Haven‑based African American faith organization to collaboratively understand and address the social determinants of depression, including racism. And developing interventions and responses that genuinely incorporate the local perspective rather than insisting on predetermined frameworks of understanding, that are church‑led, church‑staffed and drawing on the structure of the church. Lastly, I wrote a book in 2008 that tries to articulate how we can infuse more worldly concerns such as those pertaining to racism, sexism, gross inequality, oppression in mental health fields that typically only focuses on the individual level. These, these are, I'd like to thank my team members, funders, and I now turn it over to Dr. Tiffany Haynes.

TIFFANY: Hello and thank you for having me today. I'm going to be talking about promoting emotional wellness in rural African American churches, and so I think it's going to dovetail very nicely with our last speaker. We'll be using our project, Rejoice, to provide some lessons learned or some case studies, if you will. So, you will hear me speaking, but I do not want in any way for you to think that this work is mine. We have a great team, a family, as I like to call them, that works very hard on this work, and I want to make sure that we shout them out, especially our community PIs, our pastors Jerome Turner and Johnny Smith, and our faith task force. We also want to acknowledge NIMHD for funding this work. 

So now, I'm going to just jump right in, and if you will allow me to deviate from the normal academic talk and to use the words from our community to really showcase what we mean when we're talking about implementation, I'm going to do that. So, picture it, 2010 in Arkansas, Delta, one of the pastors came to our researchers and said I'm concerned about our minds. He had a concern about mental health. He was seeing increase in depression and an increase in suicide, and he wanted to know what we could do about that. So, we did what researchers do, we did the work. We had two projects, a funded pilot that did focus groups with faith leaders, college students, and mental health providers. We also had a faith in the Delta, an NIMD‑funded R34 that did needs assessments across churches across the Delta, and from that data, we heard a lot of information. One, that they, in the communities, were saying, yes, we do need some help, we're having issues with emotional wellness, we're seeing a sense of hopelessness come in our communities, and, two, that they don't know how to get that help, because they don't know where it is in their community or even if it exists. So, we took that data and we went to our faith task forces, and, so, we like to, have almost, like, Thanksgiving dinner conversations with our faith task force. We went to a meeting, we said here's what we found out, what do we need to do now? And one of our leaders said we need to get help here. We have to figure out how to get the help here. And so, that caused a little bit of an issue, because if you know about mental health care disparities, you know we understand that stressors are cause of issues within African American families, but when they try to reach out for help, they're experiencing barriers. Notably, the ones that are highlighted or underlined, availability of services, transportation, stigma, and quality of care are major barriers in the rural African American community and can lead to these disparities.

So, what can we do? We sat down and talked again, we said how do we bridge this gap between the need and the services? And we realized that we needed to look at our problem through two different lenses. One, implementation science, which will allow us to look at the factors that deal with implementation. And two, community engagement, which will allow us to look at how we can co‑create interventions that are going to combine cultural knowledge and scientific evidence, because one of the things we heard continuously is that we need to improve access to services or increase the risk of our services by moving outside the clinic into the community. And that's how we got to Rejoice. Rejoice is a hybrid two implementation trial where we had three specific questions. Will Rejoice work? How can it best be implemented? And one of the questions that specifically is important to our community, how can it be sustained? So, we have three time points, time one, where we look at how does it work, where we do a control versus intervention. Time two, where we answer the question, how can we best implement it or what supports are needed for implementation, where we look at training plus weekly support versus training only. Then time three, that our pastors like to call ministry time, where we see once we remove the, grant support, are they able to move this into ministry without that help.

And so, I want to highlight a few of our lessons learned and the importance of looking through these two lenses, because we got to recruitment, and we noticed that we were having slow numbers, and one of the, calls that we got from our Rejoice leader was that she said one of my participants said “I'm not answering all these folks' questions, they think I'm crazy, I'm not telling them nothing about me.” So, we took that back to our task force and our leaders, and that's the importance of that continuous engagement, because we had that relationship with them, and they were at the table and we could have these conversations. And so, they brought back to us two questions. They said “what if we aren't ready to move from absolutely no emotional wellness to now a whole intervention? And does the research design really match the African American church culture?” And so, we are able to focus on those things and listen to their, suggestions, and we were able to make adaptations to our intervention in time where we changed the content of our orientations that we used to recruit, and we also changed the scope to focus less on effectiveness and reducing depressive symptoms and more on building emotional wellness such as increasing self-esteem and healthy coping.

So, our community wanted to move further back in prevention and less on treatment, which is something we heard a lot about yesterday, is moving to the prevention lens. So, what this means is when we're talking about health disparities and addressing disparities in mental health care, we have to have a level of flexibility that allows us to extend the reach of these studies. Moving to EBPs from the non‑clinical settings, and then when we do that, we need to have infusion of the community engagement, infusion of the community knowledge. And so having our community members at the table with us, talking with us about how we can do that allows us to make meaningful adaptations that will ultimately increase relevance, the acceptability, and, hopefully, the sustainability of these projects. 

I'd like to end with one of our pastor's quotes, Pastor Johnny Smith, when we asked him why this project was important and why he continued to want to work on this despite of these challenges. He said, “you know what? We have a project that is different from what we originally envisioned, but ultimately has the potential to impact more people.” And so, when we married those implementations, science, and community engagement, we have the ultimate potential to impact more people. Thank you again for this opportunity, and I'd like to pass the mic to the next speaker, Dr. Rheanna Platt.

RHEANNA: So, what I was saying was, that I'm really thrilled to be here, and really excited by, what I've already learned this morning, as well as our presenters yesterday, I'm going to talk about, a case study of redesigning pediatric well childcare to address family mental health disparities and particular focusing on postpartum depression. So, and I'm focusing on, the Latino population. There's an asterisk here, because it's, obviously, a very heterogeneous population, but on the whole, studies have shown, high prevalence of postpartum depression relative to the general population, but at the same time, low rates of detection and receipt of treatment. And for both of these, a lot of the things that we've already talked about today, there are sort of root causes. So, pediatric primary care is, an important venue in which to engage mothers, and one of the reasons for that is that there's, you know, at least seven visits during the infant's first year of life. Screening for postpartum depression is recognized as, you know, as very important by the American Academy of Pediatrics, but less than 50 percent of providers routinely screen, and part of that is, you know, for the pediatricians in the room, you know, are well aware of the, incredible number of things that need to happen during well child visits, which really is, more than can realistically be delivered in the standard model, especially when we're talking about families where there may be, limited English proficiency or psychosocial risks. 

And so, there's been this concept of well childcare redesign, where the idea is, really, to think about, a population of children, and then think about the best ways to sort of design the structures, processes, and content of care. So, one form of well childcare, which I've been looking at, of well child care redesign that I've been looking at is called group well child care, and in this model, six to eight families who have same‑age infants, attend together in a cohort. And Typically, what happens, is, the total time is what you would be using for, each of the six families, and it's broken up into brief one‑to‑one assessments, and while families are cycling through the assessments, other families are completing screening or other activities, and then there's a multi‑family group discussion afterwards. And studies have shown, you know, higher attendance and satisfaction, higher, more time discussing, non‑physical aspects of care, and then a little bit of what we've talked about, I think some of the others have talked about really thinking about the hierarchies of care and, the knowledge that the families themselves have. And then this model is, you know, when enough families attend, has been found to be cost‑neutral is already being implemented in many federally‑qualified healthcare centers across the country. And so, what I, wanted to do was really to look at barriers and facilitators to identification and management of postpartum depression. I did this through conducting a, a case study at a clinic that's currently delivering group well childcare to children and immigrant families, Spanish‑speaking children and immigrant families. So, we observed the group discussion portion of the visit, relevant clinic meetings, and then interviewed mothers, providers, and other stakeholders.

And the goal of this work is going to be to think about how we might enhance the model, to improve screening, and initial management of maternal depression, but also risk factors. And so, we framed, the study around, some guiding questions, including, really, just understanding the context, so how do, psychosocial and postpartum depression occur in these visits. How do providers manage planned and spontaneous discussions of psychosocial topics, and how do parent participants sort of view, this model of care with respect to its appropriateness for discussion of psychosocial issues, including maternal depression. And then a little bit about the participants, so we followed, seven groups through, at least the six‑month visit, and, you know, the clinic has continued to offer this type of care, although there's, obviously, been challenges related to COVID, and they're currently attempting to do a hybrid virtual group combined with an in‑person checkup. In terms of the parent participant characteristics, most have been living in the U.S., for a mean of about five years, most came from, Central America. Almost 50 percent had less than an 8^th grade education, and then I think the final point is really the most important, which was really that 10 percent of the moms here, or less than 10 percent actually had their own primary care provider, so really suggesting the need to meet them at the site where they were receiving their child's care.

So, in terms of, what we learned, and I think this kind of dovetails a little bit with what the previous speaker was talking about, where you know, I think one of the, best sort of viewed aspects of these visits was not just a focus on thinking about depression, but, really, thinking about well‑being. So, this one participant described, you know, compared prior experiences with well childcare, with this current, with the group model, saying “when one goes to the individual appointments, then it's only based on the child, and they never ask how you're feeling and how you're doing. These group visits I have now with the doctor, they're also focused on me a little, how I am, what I can do not to feel so frustrated” and, you know, we saw that discussions about well‑being and about parenting, because of kind of the increased time in the room and the group discussion, they were able to really kind of tie, parent well‑being to a lot of other typically more sort of biomedical concepts. So, for example, when talking about vaccines for the children, they talked a lot about, how can parents tolerate their own distress, watching their child cry when receiving vaccines, and then there was this sort of repeated concept that the baby's health is dependent on the mother's health.

And then, finally, you know, there were realistic suggestions for improving well‑being, provided by providers, but then also the mothers themselves. In terms of depression, providers viewed this model as, allowing for, kind of a more valuable discussion on, screening, the rationale for it, and the process for it. I'm not going to read through all the quotes, but you'll be able to see them, here and then on the website. And then, you know, parents, talked about this, providing an opportunity, having these discussions, and seeing other people either talking about it or the providers modeling talking about it, that, depression, you know, maybe, it was something normal. It was something we actually talked about in the group. That's what was happening to me for a while. Sometimes, one believes they're alone, but, no, there are a lot of us. On the other hand, not all the parents wanted to, necessarily talk about, nor was it really their, you know, role or responsibility to talk about it in the group setting. So, one parent talked about that's not something I want to talk about here, it's like a rejection, I might talk about it individually. And then, you know, one of the big distinguishing characteristics between this model of care and, standard individual visits is just, differences in time.

So, both having a shorter amount of, one‑on‑one time with the provider, but a longer amount of total time with the provider. So, some providers talked about, you know, having these brief visits really affected this flow of how they might talk about sensitive topics with parents and how that was sort of truncated. And while other providers talked about how this changed, because of the total amount of time and the kind of context of their interactions, that they felt like they were able to connect with families in a more meaningful way. And then, finally, several providers talked about just, again, having the space for, including valid screeners when they didn't necessarily do so regularly. And you know, we did find that the screening, which I think is oftentimes the first step in treatment and, obviously, in these settings, that there's a lot that can go into that, both with, so, literacy was a big issue, and, so, it was actually quite a time‑intensive process, not merely handing something off to someone to fill out and then receiving it. And that, sometimes, this was done in a group setting, so there were issues about privacy, related to that. And then, you know, the other issue with time, again, having a little bit more time was that it allowed for a group discussion about topics that were, you know, incredibly important for both maternal and child health, so thinking about policy, and we talked a little bit about policies yesterday, so things like changes to the public charge rule and what the implications of that would be.

So, the next step, is that, you know, working with additional sites to really understand what organizational factors might support or impede the ability to leverage this model, to think about addressing the needs of mothers within this model of care, and then setting up an advisory team to guide potential enhancements to the model, and then, finally, testing the enhanced model. And really thinking about are their other populations, other settings, so there are, you know, other, types of care, other settings where group care is used, such as prenatal care or diabetes care, so how might we leverage these existing models of care to, sort of maximize, the mental health of participants. So, with that, I would like to, thank mentors, acknowledge funding, and I will pass it on to, Deepa Sekhar.

DEEPA: Hello, everyone. Can you all see my screen? Okay. Okay. All right, thank you so much for having me. Again, my name is Deepa Sekhar. I am Associate Professor of Pediatrics at Penn State and a General Pediatrician. It's my pleasure to share with you today our studies, screening in high schools to identify, evaluate, and lower depression or the SHIELD trial. SHIELD is actually funded by two complementary but distinct studies from the Patient-Centered Outcomes Research Institute and the Health Resources and Services Administration. So, you will all be well‑aware of the rising prevalence of adolescent depression from 8.3 percent up to 14.4 percent. The U.S. Preventative Services Task Force endorsed universal adolescent depression screening back in 2009, reaffirming the statement in 2016. The issue is that less than half of U.S. adolescents have routine primary care, and even when they do, screening, at least based on national datasets, occurs infrequently. So, for my primary care perspective, this led me to the question, what about schools? Schools already conduct vision and hearing screening with the goal of detecting conditions which might impact a student's academic success, and I don't think any of you would argue that depression impacts academic success.

And with that brief bit of background, this led us to the question for the SHIELD study, which is could we do a better job addressing disparities and mental health identification and treatment engagement by embedding depression screening in schools? So, the two arms of this trial are targeted screening, which is the current process by which you are referred into treatment based on observable behaviors of concern. What does this mean? It means you were a straight‑A student, and, all of a sudden, you're failing out of your classes. And so, someone would pull you aside and get a sense of what’s going on. If you disclose, maybe, your parents are going through a really bad divorce, and you've got multiple symptoms of depression, they might refer you on to appropriate school or community-based services. In Pennsylvania, this is done through the Pennsylvania Student Assistance Program, which exists in all Pennsylvania schools to identify barriers to academic success. When comparing this to universal screening, whereby all students receive screening with a patient health questionnaire nine, which is a well‑validated screening tool for depression in the adolescent population, with referral for those with elevated scores into this Pennsylvania Student Assistance Program.

Our primary outcome is treatment engagement. So, if you screen positive in either arm, the student assistance program confirms depression in need of further follow‑up services and that you engage with one of the recommended treatments or services. And then we have several secondary outcomes as well listed there. This is an overview, because I've gone through a lot, so we've included 14 Pennsylvania public schools and over 12,000 students in this randomized clinical trial. So, within a high school, two out of the four high school grades are randomized to one of the two arms. That's either targeted screening, so screening based on concerning behavior to student assistance program triage, or universal screening with a patient health questionnaire nine. Now, if you have a score under 10, you could still be referred during the school year for concerning behavior as per the standard process. If your score's above 10 or you endorse suicidal thoughts, you're referred into that student assistance program process. The big piece of the success of this project to date has been our stakeholder advisory board. So, we work with a fantastic group of parents, teens, school staff, physicians, and mental health and suicide prevention representatives who have advised us on multiple aspects of the study. And they, in turn, have ties to national networks, which we hope to be able to use as part of our dissemination efforts when the study is completed. We're currently finishing collection of our primary outcome. I just wanted to spend a couple minutes here talking to you a little bit more about our engagement efforts.

So, obviously, there were several challenges with this, recruitment being a big one. You can imagine approaching a school district, saying you wanted to do depression screening, which is great, but then they're also thinking about the additional kids they're going to have to triage and work through, and in many cases, our parent and mental health representatives were great. They sat next to me when I was presenting to school boards, explaining why I thought the project was important, their presence really helped me engage with the school boards and answer their questions, which was fantastic. Our stakeholders had advised us on the logistics of the randomized clinical trial itself, so students said, you know, don't call this a depression screener, call it a mood screener. When we take it, we want to be spaced out in the classroom, and we worked with that. All of our school districts on meeting those requests. My timeline was different than the school's timeline, in some cases. I thought we were going to have all these screenings done in the fall, and it just didn't play out that way, and, of course, COVID has proven to be challenging for many of us, and we didn't complete screenings at two of our schools. We'll still have an adequate sample size for our data to be analyzed effectively, but, obviously, that was frustrating. In terms of successes, anecdotally, we did identify at least two students in crisis that nobody knew about, and, honestly, part of me is, like, if we saved two lives, the whole study was worth it regardless. We, and I in particular, had the opportunity to engage with multiple school communities as part of this, and I'm truly humbled by the limited scope of patients that I'm getting to see in the primary care office compared to the kids who are out there. Thank you for your attention, and I believe we'll next be moving on to the panel discussion.

DENNY: Yes. Those were fantastic presentations. Thank you, Miya, Miraj, Tiffany, Rheanna, and Deepa. They were great. Very provocative, and what I'd like to do is start our roundtable discussion by kind of asking our panelists, you know, now that you've heard from each other and given your different, yet related research threads of research projects you are conducting, do you have any questions for each other? I’d like to open it up. As you're thinking about this, you know, particularly in implementation science, a lot of our focus is on manipulating various levels of modifications to address disparities. So, whether it's systemic, organizational with Miraj's work, you've got, community level with Tiffany's work, you've got workforce, such as Miya's work, Rheanna's doing primary care, school with Deepa. You know, kind of tinkering around the edges to manipulate some of these attributes at various levels, does it feel like that's effective enough? Is it fast enough? Is it too incremental? I'd just love to get your thoughts on that.

MIYA: That's such a good question, and I would say that the more I do the work that I've been doing with lay health workers, and these workforce things, the more it makes it so apparent, how much work we have to do and how many different areas across different systems and across implementation science, need to be addressed. And that can certainly feel overwhelming to me, and I'm sure to many of the other panelists, so I'll just speak to a couple of those things. I think regarding sort of exciting directions for the field of implementation science, and I'd love to hear from the other panelists about how they feel about this, but even for this emerging field, even the measures that we have, have really, I think, been focused on more traditional healthcare settings. Organizations, and workforces, like professional mental health providers, and, so, figuring out how to apply implementation frameworks and measures to less traditional settings, like, especially, I think, the work that Tiffany and myself are doing, I think there's a lot of exciting opportunities there when there's not formalized potentially structures that are helping to keep these practices sustained. Then as far as thinking about different systems, I don't think that it's enough to just tinker around the edges. I do think that it requires much larger changes and innovation on our part around what we're thinking is a solution as to, thinking outside of the box, the box of where mental healthcare is provided and who's providing it, and that requires many different logistical supports. But, but each step is important, I think, and, so, I really want to see more work done in this arena, and I think it is being done to help guide the direction that we'd be going.

DENNY: Thank you, Miya. Anyone else want to comment on that?

TIFFANY: I just want to piggyback, I'm sorry, did I cut you off? 

DEEPA: No, you go ahead.

TIFFANY: To piggyback on what Miya was saying about, one, just the need for innovation and understanding the complexity of disparities and how they work, and so, sometimes, our interventions are going to look complex, our changes are going to look complex. And it's going to take time, and I think we have to remind ourselves of that often in this work, that it's going to be the small steps, but they're going to lead to those bigger changes where we're going to make big changes when we're talking about disparities.

DEEPA: I was just going to add that I find it to be just so practical, right? Like, I feel like you come with your idea of what's going to work, so I had certain ideas of what might work in a school, and it's just so refreshing, like, people just tell you, if you ask them, right? Like, this is not going to work. It's nice to hear that, and I feel like it saves so much time on the back end, coming up with some big plan that just is going to flop, and I love how practical it is, and it's a little stressful sometimes, trying to adjust to stuff, but you learn to meet people in the middle. We learn to explain, you know, the randomized clinical trial setup to some of our school partners, you know, when we talked about certain grades getting screened, they initially asked, well, can we pick the grades? Because we're really concerned about this 9^th grade class, and I was like, well, it doesn't work like that, but let me tell you why. It was nice to bring people together to talk through those issues.

CRYSTAL: Denny, we have a question from e‑mail.

DENNY: If you want to read it for us.

CRYSTAL: Yep, I can read it. I can go ahead. So, for the panel, can you comment on the role of intervention developers? For example, with PCIT in intervention adaptations and how intervention developers can facilitate or impede the adaptation process.

MIYA: I am happy to start this, though I don't want to talk much. I think that that is an important question, and there is a lot of work, I think, to think about intervention developers, almost as part of our stakeholder group around how much flexibility there is in adapting these interventions, both in terms of how the content of the intervention is changed and what flexibility there is around more context issues. Like, who is providing interventions and what is needed to get certified, and, so, certainly, there can be some real barriers to that work. I think that one of the reasons that my, I know that one of the reasons that my systematic review found that evidence‑based treatments are more likely to be delivered by lay health workers in low and middle‑income countries as opposed to high‑income countries is just that many of the different evidence‑based treatments, that you have to have a master's degree to provide them. But they are allowed, they have partnered with different researchers to, like, let up on those requirements when it's delivered in low and middle‑income countries, and, so, what do we do when we have knowledge that, lay health workers can provide these treatments effectively? But that it's not the right fit because of many different regulations, from developers and also from mental health organizations here in the United States, would limit who's providing those treatments. So, those are questions, I think, are really important to get into.

DENNY: Yeah, and I think I was struck by what Tiffany was saying about kind of focusing on combining community knowledge, you know, with the, you know, scientific evidence. I'd love to get your thoughts about how that might help with intervention development for the future.

TIFFANY: Yeah, you know, when we went through our intervention adaptation phase, I thought that, very similar to, I can't remember who said this in their talk, about, you had your thoughts about what was going to work, and then people tell you that's absolutely not going to work. And so, having them at that table and during that adaptation phase really was important for us, to use the correct words, to use, different vignettes and changing even small things, like the race of a person in a vignette, or wait a minute, that person wouldn't say that they were low on money, they would say they were broke. Just those small tweaks, and having someone from the community there to, to do that, but, ultimately, we also had to do another arm of education with our providers and policymakers within our state who were concerned about, wait a minute, are they prepared to handle this? Can they provide this type of treatment? What's going to go on and what's happening with that? So, we had to do a level of education about here's what the research says when we partner, when we let up on those regulations to let lay people do this. This is how we're screening to make sure that we're not doing those things. And so, I would love to see more interventions think about what would happen if this intervention were not just in a clinic, if it was in a barber shop, if it was in a church. Then how can we, what changes can we make to this intervention on the front end that would allow it more easily to be put in those places.

DENNY: I would like to go ahead and promote a question that we just got from one of our folks in our audience his name is Josh Gordon. Can the panelists discuss the tension between customizing for particular community versus generalizability? Kind of that personalization versus broadening out to generalizability.

MIRAJ: Well, I think I could, one thing I would say, what the research that we're doing seems to suggest is that when you, as the phrase goes, when you see the difference, you realize the constant, so I would turn the question around to be, like, what is the constant of the vast majority of mental health treatments? And why is it that so many communities of color conflict with them? So, I would, my focus lately has been looking back in the clinic, not adapting it to the culture, because, until we understand how those kinds of things, we take for granted are actually cultural, then I think we could start, once we start understanding that more, we can start understanding what's being left out. And so then, it can apply to a great many more types of cultures. I don't know if that makes sense. So, for instance, with the focus on verbal-ness, or being verbal, it's a bare minimum of almost every single mental health service you can even possibly imagine. And yet the number one driver of health within certain traditions, for instance, in the whole, traditions arising around the, quote, mindfulness, but I'm a deep practitioner of Zen, for instance, the number one driver of health and transformation is silence, and that perspective is being taken up across the whole gamut of health and mental health services. Mindfulness is everywhere now. The number one driver of healthcare is silence. So, if you have, basically, a mental health culture that depends on there being speech and that gets codified within a system and then, therefore, the whole bureaucratic system around that depends on speech, but not just speech, that there's a problem that you report via diagnostic codes, that there's a menu of services that you also report. So, the whole bureaucratic financial reporting structure is dependent on these basic cultural norms, they're just scratching the surface of how there are so many kinds of powerful practices out there that are just not able to be fitting within this culture, even within our work with, the African American church. A lot of our work is turning back and saying we're not here to tell you, you have protected this community for hundreds of years of racist violence, I am not going to be a mental health professional, coming in, saying we have the solution. For what it is to be protective of this community. So, that's how I would answer it, is just we need to really think about the question of what are the very basic fundamental presuppositions of our field. Then we start having a discussion about what's being left out and what are we not recognizing as powerful mental health practices.

DENNY: Okay. We have other questions, but I want to allow any other panelists to comment on that, that question.

MIYA: I just wanted to say, you know, I think a lot of great work is happening within implementation science, and David Chambers with Adaptome and, Shannon Wiltsey Stirman with different frameworks have really laid out ways that we can look at these tensions around generalizability versus adaptation. I think what we've heard from our panelists is that we can be, that we have to be adapting within community, the work and the way that these interventions are, presented and used and made to work within a community, but we can be learning about how communities work. There's so much for us to learn on the ground as providers of all different levels, within the church providers, mental health providers. They are making these adaptations, and, so, in some ways, there are some generalizable type of changes that have to be made, to these specific interventions. And I think exciting work to be done there really learning from the communities on how they make implementation work.

DENNY: Thank you, Miya, and Deepa, do you want to be the last one to respond to this one?

DEEPA: Sure. I was going to add on a much simpler level, I love how Miraj phrased, like, what could you keep as the constant. I mean, this is where I find the value of community‑based research, right? Like, when you do your drug trial in absolutely controlled settings, then people take the medication, they don't take it like they do in a randomized clinical trial, where everything is controlled. The same applies to research in community settings, there are pieces of the randomized clinical trial that cannot be changed, but there's certainly other elements of it that can be adapted to that community where your results are still valid, and I feel like the challenge and the joy is working with those individual communities to figure out where we can compromise and come together.

DENNY: Perfect. Thank you very much, and then, I think, Crystal, you have some questions?

CRYSTAL: Yes, I do. This is fantastic. So, first question, for the panel, any recommendations for changes that can lead to sustainability? So, there've been a lot of questions, I think in discussion about kind of, engaging communities and different communities and how you all are doing that. But this question is about, what changes can occur that lead to sustainability.

DEEPA: I think you first want to see if it works, so just using the example of our, of our study, the PHQ9, I picked because it would be freely available to schools after we left, so if it's effective, if screening is effective in high schools, we set it up so the schools would be able to redo what we did for the study themselves without us being there. And at least in my knowledge from work with schools, that's been an ongoing complaint of schools, that researchers come in, they bring all their stuff, and then they leave, and the schools are left with, you know, back where they started. So, I think planning for that on the front end, I'm sure all of you have that about that with your work, is very important.

TIFFANY: I just want to you know highlight what you just said, about planning for sustainability at the beginning, and that some of the questions when we are making adaptations about what are the things in the church that's going to allow us to implement, but also allow us to sustain. How are we going to, that move from intervention to ministry. So, we're thinking about that move for every decision that we make, and, a lot of times, we think about sustainability at the end of our projects. We think about it the same time we think about dissemination, year five, month six, oh, I got to disseminate, I have to figure out how we are going to maintain this. It has to be a conversation you're having with your community partners, with your research team from the moment that you start your project.

DENNY: That's great. Great responses. Crystal, did you have some more questions?

CRYSTAL: I do. Yes indeed. So, another question, kind of similar, kind of related, what are the pathways to adoption of these interventions? And who are the key gatekeepers or stakeholders to allow for adoption? I think it was Miya who stated a blockage in making community workers sustainable by getting it reimbursable, but how do we get the, quote unquote, evidence? And what is the, quote unquote, evidence to shift practice? This is a great question, very important. Yeah

MIYA: I can just quickly, start, and then I would love to hear from others too. I think that one thing I'm really looking at is opportunities that are happening at the policy level, like recent changes in, there are a lot of community health workers and peer providers about having more billable codes that can be used within healthcare and mental healthcare systems. And so, I think it is important to be staying on top of where those changes are happening and then helping also make sure that when policy changes happen, that there's a way to be embedding evidence‑based work within in that work. So, I think it requires some nimbleness and work to be building the evidence base as we're changing policy and then helping inform policy as we have more of an evidence base.

RHEANNA: I think that's also really, in some ways, kind of the role of implementation science, is really thinking about doing this work in, like, real‑world practice settings and kind of generating the evidence at the same time, but within a setting, you know, that it's not sort of a stand‑alone research project, but that it's actually occurring within a naturally recurring setting.

MIRAJ: I would just also add that, in our work, I just think just a lot more needs to be done, I think there's a fundamental tension within implementation science about, still trying to understand what are the facilitative conditions with, say if you're in the community, there are remarkable practices happening in these organizations whom we partner with. Remarkable. That are saving people every single day. If you take those out, those people would be in great, great distress. So, I just think there's a tension, that if you're trying to adapt something to a community setting, just make sure you are spending the time working with people, like sociologists, anthropologists, qualitative researchers, to understand the very fundamental conditions and practices that are already existent in the person you're working with. And then you demonstrate genuine appreciation of them, and it's remarkable, how the community responds. They see you as a genuine partner, not just someone trying to bring something in.

DENNY: Thank you. This is great, and I just have, we have just 2 more minutes left, I just have one final question for each of you to just think about of now that you know what you know about research implementation science and some of the challenges in this field, what would you recommend to early‑stage investigators as they begin to think about stepping into this pathway?

DEEPA: That one's easy for me, I guess, to answer. I would say, from the very beginning, find your stakeholders and include them from the beginning. We often think about them partway through writing the proposal, who do I need to ask, but if you have them onboard from the beginning, the relationships are richer, and I think your science is better in terms of what you'll propose and the outcomes you'll pick.

DENNY: Perfect. Thank you. Anybody else?

TIFFANY: I will, go on.

RHEANNA: I was going to say, so, I would say I probably am an early‑stage investigator, so the advice I would give to myself, though, is, also, I think, thinking about the stakeholders, and but, you know, thinking about sort of durable ways, I don't know that I would say this to investigators, but more to put it out, like, what are some durable ways that we can continue to have, these advisory groups beyond one individual project? But thinking, like, partnering over the long‑term.

DENNY: Mm‑hmm. Tiffany?

TIFFANY: I would just add, you know, community engagement is important, but there's also a science behind it, and I think that I would encourage new investigators to learn that science, and it will help you make sure that you're not making those missteps in the beginning of your project.

DENNY: Perfect. Miraj?

MIRAJ: I would say I'm also an early‑stage investigator, but, you know, I think we, those of us who are doing this work, it's not easy. Most funding goes to other work, and it's thus very difficult at times, so you need a community of support, just for yourself as well, I think to provide encouragement that this work takes a lot of time. You are spending lots of time in the community and in other places, and I think that's essential, to have a support community around yourself.

DENNY: Perfect. Well, thank you all very much. I really appreciate it. We, unfortunately, need to end our session, but it was such a provocative and interesting discussion. I really, appreciate you thinking all these issues through. Now, I need to hand the reins over to my NIMH colleague, Dawn Morales, who will moderate the next session, titled “Innovative Methods for Conducting Multidimensional Mental Health Disparities Research.”

DAWN: Thank you Denny. Hello, everyone, and welcome to the last panel for our workshop, which focuses on innovation in analytics, methods, and rigor in mental health disparities research. Our first speaker is Dr. David Chae from Tulane University School of Public Health in Tropical Medicine. Following him is Dr. Bowen Chung from the University of Southern California, and the David Geffen School of Medicine. Next, we will hear from Dr. Alicia Martin, who is at the Broad Institute at MIT and Harvard. Finishing up our panel is Dr. Zachary Warren from the Vanderbilt Kennedy Center Treatment and Research Institute for Autism Spectrum Disorders. Our panel is too exciting to keep waiting, so please let me pass the microphone, microphone control over to Dr. Dave Chae, who will speak to us about Black deaths matter, multi‑level racism, syndemics, and the transmission of grief. Thank you, Dr. Chae.

DAVID: Ok. Can folks hear me? Yes?

DAWN: We can hear you great.

DAVID: Great. Ok. Thanks for that intro. Today, I'll be discussing a framework for studying multi‑level channels through which racism impacts mental health. I'll talk a bit about the utility of using a syndemics lens, which is the idea that multiple co‑occurring health threats results increase vulnerability to poor outcomes, and also about death not only as an outcome but also as a social determinant of health and the transmission of grief. I'd like to begin with this quote by Rudolf Virchow, who is commonly considered one of the founders of social medicine. “Medical statistics will be our standard of measurement. We will weigh life for life and see where the dead lie thicker, among the workers or among the privileged.” So, one thing that I appreciate about this quote is the reference to social determinants of health, the privileged or the working‑class, the rich, the poor, wealth and poverty, the advantage or disadvantage. These are age‑old issues that still have strong relevance today.

Another sentiment expressed in this quote is that of equality, that we will weigh life for life, that the workers' lives should be weighed the same as the privileged life. Today, this would be akin to saying that Black lives matter. This means that Black deaths also matter, and we know that there are tremendous racial inequities in mortality, and that these, racial inequities are driven by racism. So, how does racism impact health? Most of the research in this area has focused on interpersonal experiences of racism, such as experiences of racial discrimination. Neighborhood factors, including various aspects of physical and social disorder have also been associated with greater risk for disease. It's important to note that racial residential segregation, has been intentionally designed to maintain white supremacy. One major consequence has been the creation of predominantly Black neighborhoods that, on average, have greater systematic disadvantages compared to white neighborhoods. Other less commonly studied facets of racism include vicarious racism, which involves hearing about or seeing acts of racism committed against other members of one's racial group. Vicarious racism is a unique experience of threat in that it's not necessarily tied to the individual, but rather the racial group to which the individual belongs. The concept of linked lives, suggests that the experience of other racial group members are shared and that experiences of racism in group members can also be a personal sources of stress.

Now, this source of threats are particularly salient during the COVID‑19 pandemic. For example, witnessing instances of racial insults, harassment, and police brutality committed against Black Americans, which has become increasingly visible with the aid of video and subsequent news coverage. Experience of racial harassment and xenophobic talks against Asians has also been publicized through social media. And this was largely stemmed from, the use of stigmatizing language by the outgoing US President. In data we collected over the summer, we found that experiences of vicarious racism were associated with higher levels of anxiety and depression among both Asian and Black Americans. We also examined racism related vigilance. This involves efforts to prepare one's self for the experience of racism, the need to feel watchful, or changing one's behaviors just to avoid being the target of racism, and we found that racism vigilance was also associated with mental health. So, these findings echo research documenting rises in depression and anxiety among Black Americans following the killing of George Floyd, suggesting that racist acts have more diffuse, community, and population level health effects. These findings also resonate with those reported by Abigail Sewell and colleagues, who found that neighborhood frisks and use of force resulted in greater psychological stress among men. Areas characterized by more aggressive policing may be a risk factor for poorer mental health and vicarious racism and racism related vigilance may be pathways to which this occurs. Deeply entrenched racism has been exposed during the COVID‑19 pandemic in multiple ways. This includes not only acts of racism, but also the inequitable distribution of COVID‑19 cases, hospitalizations, and deaths. We know that communities of color have been disproportionately impacted by COVID‑19.

And part of this is due to greater social inequities that drive exposure, as well as increased socially induced biological susceptibility. The COVID‑19 pandemic has been shaped by racism and has exposed and widened racial inequities. Multiple public health crises, including the co‑occurrence of racism and COVID and unemployment have had a tremendous impact, particularly on Black communities. Black women who were the backbone, not only of the recent presidential election, but I would say also the backbone of this country, have been bearing the weight of multiple pandemics. In recent analyses we conducted on Black women's experiences during the COVID‑19 pandemic, we looked at experiences of racism in addition to COVID‑specific experiences. This included knowing someone diagnosed with COVID, as well as knowing someone who died from COVID. We also looked at their experiences of work and economic disruption, including experiences of unemployment, reduced working hours, and loss of income. So, Black women are very resilient but having to bear the constellation of these three public health crises has resulted in mental health tolls. Among the Black women in our sample, these superwomen unfortunately reported much higher levels of stress, as well as depression and anxiety. We saw particular disparities in the experience of COVID‑19 death by race. Almost a third of Black respondents in our study reported knowing somebody who died from COVID‑19. That's about twice as high as the percent of whites, and as expected, those who experienced COVID‑19 bereavement reported higher levels of depression. So, this is important. U.S. age‑adjusted COVID‑19 mortality rates for Blacks are three times higher than the rate for whites, and, you know, if Black Americans died of COVID‑19 at the same actual rate as whites, about half of the Black lives lost would still be here today. And so, these deaths, represent the loss of sisters and brothers, mothers and fathers, grandparents, daughters, sons, friends, we carry this grief, which impacts us throughout the life course. For example, can you imagine a young kid who loses their parent? Who is going to guide them as they get older? You know, where are they going to find sources of social support as they encounter problems in their life? You know, imagine a parent losing a child. We know that Black parents are much more likely to lose a child compared to white parents. You know, these deaths are not isolated or acute events, and the transmission of grief means that these deaths will have much longer‑term public health implications. So, racism is deadly.

In other work, we looked at racism at the area level proxied by Google search concentration for the N word, and that's shown in this map. Those areas with greater levels of racism had higher Black mortality rates, as well as higher rates of adverse outcomes. Recent data suggests that racism causes pandemic‑scale deaths among Black Americans annually. So, Black deaths matter. Black lives matter. Racism has caused countless deaths from COVID‑19, as well as from overt acts of hate, including racially‑motivated murders and police brutality. These experiences have resulted in major mental health tolls and highlights the need to implement anti‑racist policies and practices as public health strategies. Thanks, and our next speaker is Dr. Bowen Chung.

BOWEN: So, hi. Good morning, everybody. Thanks for inviting me to this talk. This is, an older study, that was funded in 2007 that I'm going to talk about. We're going to talk about how community engagement can be used to address mental health disparities. Next slide. Hold on. This is a depression study, but it's really much more than that. It's a study, you know, everybody here knows depression's the leading cause of disability worldwide. It seems like the epidemiology suggests that at least among adults, it's fairly similar among racial and ethnic minorities with, maybe, actually, slightly lower rates among, you know, immigrants and African Americans. I think it's a big issue now because of the disaster of COVID, the recent CDC reports indicates, from MMWR in August that it's about four times the rate it is, at the same time last year in June, but then it was, in 2019. You know, one of the ways that, has been sort of talked about how to address depression in primary care has been collaborative care, and there's a lot of evidence that suggests that it works, to address depression in adults, but in lower‑income communities or systems that are not integrated or in rural areas, it's really hard to do these kind of interventions given the resources, despite the fact that Medicare has really started supporting some of the, care manager and phone call work that's usually done, especially under COVID. One of the other things, especially talking about social determinants, is that we really don't know about, the independent contribution of community linkages, according to SAMSHA, to improve health and behavioral health outcomes in kind of the same way that I think we do with some of the physical health, like around housing for health in California. 

So, how do we really translate some of the work that the NIMH has funded over the years, in primary care to sort of think about how do you extend some of the, tasks and approaches at the organizational level for minority communities? I do have, one conflict of interest. I get salary support from a company called Chorus Innovations Technologies. Was not involved in the work for this study. So, community partners in care was a study funded in 2007 with the NIMH, through research center. The PI was Ken Wells and the co-PI was Loretta Jones. And, I actually grew up in this study. I did exactly what you're not supposed to do in academics, and I stayed with my mentor, using a team science approach over about, I guess now 17 years, and that's actually kind of what it takes in order to do this kind of work in communities. This study was done using a participatory research approach, that's a variation of CBPR called community partnered participatory research that was developed in Los Angeles by our partners, and we had about 50 organizations, partnering with us to implement this study in Los Angeles and south LA and Hollywood downtown. And these are some of the many agencies that were, there, including Kaiser, some very small community‑based organizations, in south Los Angeles and Hollywood that ran the gamut of, healthcare, barber shops, beauty salons, homeless agencies, food banks. You name it, we had it. And I think many of you have seen this, but the whole study was done under, the conditions of participatory research, so we had, you know, for a lot of people on this call, I think, everybody kind of knows a little bit about this, but, basically, we used principles of transparency, respect, power‑sharing, and co‑leadership and knowledge exchange. This model was developed by Loretta Jones who passed away last year, and it's structured and manualized, and we have an executive council with both academic and, community members, or patients. All the work groups and all the, everything's done in partnership and is reinforced with a memorandum of understanding signed by different institutions and different individuals where we talk about sharing and sort of the principles and all of that. What did the study do? Basically, we took a collaborative care approach that was designed for primary care, called partners in care, which was kind of a sort of earlier, intervention‑based primary care, depression care intervention that was sort of preceded, the work that's being done by Dr. Unitzer at the University of Washington. And we recruited 50 agencies and had 95 programs within those agencies to one of two conditions. This is a cluster randomized trial, so we randomized them to either what we're calling, community engagement and planning versus, resource for services. Basically, mimicking what chronic disease management companies would do to train up, primary care clinics across, a large geographic area within an integrated health system to deliver primary care. With the change that, basically, we also included, you know, non‑healthcare, agencies. In the first condition, we just provided the equivalent of conference and technical support calls for, the organizations around how to implement depression care, collaborative care based on that evidence‑based model, and then the second one, we actually randomized the programs by type to a planning process called community engagement planning. Which it used participatory research principles to adapt the interventions, and I’ll describe the interventions later, and then implement it, as well as create a virtual network for depression care across different organizations. This is the design. We went up to 48‑month follow‑up. Basically, the intervention was a complex intervention, organizational intervention. We had key management that we trained people up in, clinical assessments, like screening, follow‑up, education, medication trainings. We provided training in cognitive behavioral therapy, care management and case management that was adapted from work we'd done in New Orleans post‑Katrina, sort of taking some of the care management tasks that are usually done by social workers or psychologists and adapting for community health workers. We also had, a group CBT that was done by, lay health workers. So, one group basically just got the trainings, and then they got a bunch of, technical support phone calls, like this one, and then the other group met for six months, adapted the trainings and then trained everybody up, separately in Hollywood and South Los Angeles.

One of the key findings we found was the community engaged group versus RS, and we didn't pay any of the organizations, we just gave them food, we gave them MOUs, and what we found was, basically, that, the trainings, the attendance at trainings, and this is an implementation science point, was, you know, like, close to an order of magnitude greater for all of the trainings from the community engaged group. Because we actually had a lot of the community partners co‑lead the trainings, so they became experts, especially in the community engaged condition. This is pre‑ACA, and I know I have very little time, so I'm not going to go too far here, but, you know, this lower‑income, a lot of no health insurance, a lot of single people, middle age, just like me. They were screened. We had about 1300 entered the study. At six‑month follow‑up, we had, in the community engage group, what we found was reductions in poor mental health related quality of life for the community engage group. The community engage group also had, improved mental wellness, which was a community defined outcome. Physical activity improved in the community engage group, and homelessness and homelessness risk factors decreased in the, the community engage, the, clients were recruited from the community engage group. We also reduced hospitalizations by about 50 percent for behavioral health conditions at six months, as well as high utilizers.

We continued to have results at about 12 months with, you know, the same sort of improvements in the primary outcomes of, improved poor mental health related quality of life and reductions in behavioral health hospitalizations. So, for example, community asked us to look at arrests and probations, and community engage group reduced arrests and probations, for the community engage group, we also reduced work loss days for the employed, increased mental wellness, decreased homelessness risks. And a lot of these sort of psychosocial outcomes were developed by the community, and, actually, we had originally designed the study as not a comparative effectiveness study, but rather as a usual care study, and the community said, no, we're not going to be guinea pigs in this situation, so we had to really try to both change the study design, so we had two active interventions, as well as include a lot of, outcomes that we were a little bit worried about actually, because we weren't sure we were going to actually make it. We also had an SMI sample. Because the recruitment criteria were PHQ9 score of 10 and above, 18 or older, English, or Spanish and reliable contact information. That was it. It seems like the training costs were higher, but this was done from a societal perspective, so we included all the time that we prepped for trainings, the driving time, and, the time that people actually went to the meetings, and if you have more participation, you're going to have to include the time from clients and the loss of billing from organizations and any other work activities. But, actually, in cost analyses, we found no difference from a societal perspective, when this is published. Three‑year outcomes, we found very similar things. The interesting thing about this is when we had the network effect was that there was a higher percentage in the community engage group with community sector or faith‑based depression services, so we shifted services away from especially mental health to primary care to community services. Which is also kind of interesting, as you start thinking about how Medicaid is going to be redesigned to address social determinants. There were a lot of community benefits, which I'm not going to go into here because of time, and, you know, we're very proud over the years, and I think this has a lot of implications about how Medicaid and, in terms of how to pay organizations and how to link them up that are not a part of the healthcare system. This is the team. We were funded by the NIMH National Library of Medicine, RWJ, California Community Foundation, UCLA Clinical and Translational Science Institute and we’ve won the awards from the NCATS, which is the trade organization of the Team Science Award 2014. And we also won an award for health equity from the community council partnerships [inaudible] Thank you.

ALICIA: Okay. Again, I'm happy to, take over here. So, I will be talking about the critical importance of diversity in genomics. So first, why talk about genetics in the first place? Well, genetics gives us unique insights into the biomedical basis of disease. And it's particularly informative for psychiatric disorders and mental health disorders, where we don't have a great biomarker for many of these, disorders, and, so, rely on a lot of phenomenological manifestations. So, genetics also forms a fundamental building block of many of our basic research questions, and we can use it as an instrument not inherently confounded by socioeconomic status or environmental factors that contribute to disparities, so we can use it for causal inference to understand the underlying epidemiological factors that contribute to disparities. So, as a geneticist who works in population genetics, I find it really important to start at the beginning. So, humans originated in Africa on the scale of hundreds of thousands of years ago. They migrated out of Africa and took a subset of genetic variation with them, which means that the average out of Africa population has about a million fewer genetic variations on average than the average African population. This human history over the course of the past hundred or thousand or so years informs the landscape of genetic variation in the world, which is really important then to consider when we're designing our genetic studies.

I think it's the most exciting possible time to be working in human genetics, because our studies have grown just explosively over the course of the past decade. These technologies, though, can be, misused, so we've seen a long history of eugenics, but they can also be used for good. They're not an inherently good or bad technology. This inherent use is dependent on the intention. So, one use for good, for example, we see unprecedented, rapid pace of COVID‑19 vaccine development taking place because of genetic technologies, so that's really incredible, and that's really driven by this exponential growth and increasingly powerful, technology. So, we've learned a ton about human history, we've learned a ton about the biomedical basis of disease from this growth, and this really serves our genetics mission of providing biological hypotheses from unbiased associations, forming the basis for downstream functional analysis and eventual therapeutic insight. But if we look at this graph now colored by population, you can see that genomics has a huge diversity problem. So, right now, about 80 percent of participants in these large genetics’ studies are of European descent, and this is far out of step with the global population, where about 16 percent of the world is made‑up of people with primarily European ancestry. And this is not, a trivial problem, it's not easy to fix, and it's not getting better on its own. So, if you look at the bottom part of this figure, you can see that the proportion of people that are primarily from outside of Europe or from non‑European ancestry populations, this progress, this diversifying progress has stalled or perhaps even slid in the wrong direction since around 2014 or so, and that's because our European ancestry population studies have just been getting larger and larger and leaving other populations behind. So, why does this matter? Our fundamental biology is the same across different populations. Our organs, our cells, our molecular pathways are all the same, so why does this matter? Well, if we're blinding ourselves and trying to only understand the genetic basis of disease from a subset of genetic variation, we're only going to find that subset of genetic variation that's most important to the populations we're studying. So, if we fundamentally have less genetic variation, we're destined to find fewer genetic associations. So, I see some really basic needs in the field of genetics and how this sort of plays out, downstream as well. So, on the left side of this slide are areas that we really need to improve on in basic research and in basic science to improve upon where we're at with the blinders we've had due to vast Eurocentric study biases. On the right are areas where we have big scientific opportunities, that we'll begin to realize as our studies become more diverse. So, we need new methods, we need new data and community resources, and we need new research capacity to power our scientific discoveries in an equitable manner, such that the downstream fruits of all of our research are going to, contribute equitably.

So, we have really big, opportunities, both ethically and scientifically, to identify new genetic associations, to really zero in on and pinpoint causal variance, to predict genetic risk of disorders, so we can use these technologies in precision medicine in an equitable way and to tease apart those genetic and environmental determinants of health and understand where these are shared and where these are unique populations specific risk factors. So, one of the areas that I think has gotten the most attention lately in genetics is in the use of polygenic risk scores. So, polygenic scores have been talked about in precision medicine as a really exciting biomarker, and I can't think of a single lab test that's so predictive across such a broad spectrum of disease, compared to genetic technologies and polygenic scores in particular. But I also see this as the biggest challenge to implementation of genetics in precision medicine from emanating from these Eurocentric biases because of really massive disparities in genetic prediction accuracy due to these study biases. So, because we're doing our large‑scale studies primarily in European ancestry populations, we're predicting traits far more accurately in European ancestry populations than elsewhere. So, for example, we're predicting European ancestry traits about twice as accurately as an East Asian ancestry population, and about four to five-fold as African ancestry populations. So, this is a really big issue, because these are being talked about in the context of precision medicine, and I can't think of any lab test that works across the board to predict so much risk of disease but works so inequitably. So, no drugs work this inequitably as a class, no lab test works so inequitably across the board. And we understand the basis of this, and it all comes down to the fact that we're really studying such biased subsets of populations. 

Where we've looked at more diverse populations, for example, in schizophrenia, we've seen that there's really consistent promise that's coming from our diversifying efforts. So, for example, from the Psychiatric Genomics Consortium or the PGC, there have been really massive efforts to, aggregate data, both from European ancestry populations and then increasingly in East Asian populations, more recently. And, so, European ancestry populations still have the largest genetic datasets. However, work from the PGC particularly led by Hailiang Huang and colleagues have amassed datasets that are now about a third the size as the largest European dataset, but if we're trying to predict genetic risk to something like schizophrenia, we see that even with this sample size that's a third as large as in Europeans, we're doing a better job with the ancestry match data in this, underrepresented population. So, that means that we've got a lot of room to grow, and we don't, we can really benefit from these, increasingly, large studies. So, we have the potential to make the biggest gains in African populations. As I mentioned, humans originated in Africa.

African populations in the U.S. suffer the largest health disparities, and so we really should be investing a lot in genetics of African populations if we hope to really close these gaps in terms of where we expect genetic technologies to have the greatest inequities. So, there's a large‑scale study, called the NeuroGAP neuro psychiatric genetics in African populations effort. This is on track to study 35,000 Africans spanning Ethiopia, Kenya, Uganda, and South Africa, areas of the world that have been traditionally left out of psychiatrics genomic studies, and these are particularly interesting from an ancestry point of view compared to other genetic studies, because here in the, the only places that African ancestry, genetic datasets are of any appreciable sample size is in African Americans, which is fantastic. But other parts of Africa are as, genetically unique and contribute as much genetic diversity, to the global breadth as different continents and out of Africa populations. So, there are over 20,000 participants that have been enrolled in this study. It's going to get rolled into an even larger effort across Latin America and Africa, spanning about 120,000 participants, 40,000 of which, have schizophrenia, 40,000 of which have bipolar disorder, and about 40,000 of which are population‑match controls. There's a big capacity‑building effort to ensure that the investigators in these parts of the world are able to conduct analyses and lead these studies themselves and to contribute, study resources to ensure that these genetic technologies can be distributed equitably.

So, we've already begun sequencing large numbers of African and Latin American populations, and we've also, through these training efforts, been taking on studies to understand, what the polygenic score, how accurately we can, predict genetic disorders using these types of technologies. And so, working with some of these African investigators and mentoring them over the course of the past couple of years, we've taken on studies to learn that there's outside promise for genomics in diverse African ancestry populations. I’m going to spare you the technical details of this slide but just say we're already learning that there's some traits that we're going to be able to learn about much more rapidly by contributing these more diverse ancestry populations to these studies. So, we'll just stop there and say thanks. It takes a village to conduct, work at this, scale. I'm happy to take questions after Dr. Zachary Warren talks about addressing disparities in ASD disparities.

ZACHARY: Okay, thanks. I'm honored to be a part of this conference and present our Vanderbilt's teams work related to developing better care systems for young children with Autism Spectrum Disorder. These are my funding sources historically for the sake of full transparency with no conflicts. It's well‑known that autism's a very common neurodevelopmental disorder. You know, the most recent CDC estimates are at one in four in the U.S., and despite the known benefits of early intervention, we know, many children are identified quite late, in families from geographically underserved areas, families living in poverty, as well as racially and ethnically diverse families are much less likely to be identified at young ages. To help address sort of late diagnosis and some of these disparities, American Academy of Pediatrics, for the past 15 years, has endorsed screening parameters for ASD at 18 and 24 months of age, and in the U.S., we have, developed a tremendous capacity essentially to hand the form to families. You see pictured here the MCHAT are the most commonly available ASD screener available in a multitude of different languages, but, unfortunately, the road from screening to diagnosis and service, has not been built to capacity. We screen and refer to specialty services that may, be extremely challenging to access, and as such, we have unintentionally created, extended waits and distress with traditionally underserved groups disproportionately affected. I strongly believe that one of the biggest barriers to being able to shift meaningful identification in service for underserved populations relates to the tools and processes that we've historically relied on so heavily. I contend that we likely need to focus on developing many new tools and innovative approaches for screening and formal diagnosis in order to address early disparities. And moreover, we need to be developing these tools in full partnership with and explicitly for, the groups we want to serve. Much of sort of our early work in this domain initially focused on, you know, things we've heard from others in terms of these developing training processes, to deploy simple tools for identification within primary care settings, right? Where kids are already getting care, and we started a series of projects where we were providing training, and a screening tool for infants and toddlers. A brief 15‑minute interaction assessing social communication skills and ASD sort of risks, and we found, you know, not surprisingly, that supporting STAT training for community PCPs are directly dropping models into primary care settings could, in fact, help accurately identify many cases of ASD. Dramatically shorten waits for service, and we could do this in settings dedicated to serving underserved groups like FQHC’s. And we also found, that on a system level, this type of training, this type of tool, when supported with significant resources, could produce dramatic increases in the number of children receiving ASD‑specific services. Despite successes here of partners whether we exported this training program, all of these studies were really relying on a somewhat outdated proprietary sort of tool, psychometrics not optimized for this use in this setting. We were simply using it at Vanderbilt because it was owned by Vanderbilt. Predecessors had completed it here and hence we were trying to our service research here. It was free to us, but not free to every system. 

And so, what we found is that, you know, sustained system change outside of that context required substantial and sustained funding, which wasn't realistic everywhere. We also found that, you know, asking the busiest of providers, serving multi‑stressed, under‑resourced families to do more, can be a tough ask. So, in many locations, this tool deployment, this type of training didn't really stick, with my own state being one of those, locations, and as such, several years ago, we started working on building and testing models for potential tele‑diagnostic evaluation services. Here in Tennessee with partnership of our early intervention part C system. We started building capacity to, you know, like we're doing here, literally zoom into existing partner developmental centers to conduct, tele‑evaluations. And across the initial pilot work, we found that many children could be identified using structured standardized tool. In this setting, some 75 percent with ASD were identified, and we also found that appropriate use could increase capacity for service, lessen referrals to burden to tertiary care centers, and dramatically reduce waits. We dropped it to about 11 weeks in our diagnostic partner program. We were also doing this in under researched areas, 130‑miles away from any, urban center, families with diverse SES sort of backgrounds, high school education or less, median income of $40,000 or less, and families actually chose this model. They chose this tool and approach over the traditional screening referral. Some 90 percent, 56 out of that 63 of our offered families chose that model as their actually preferred mode of initial interaction. And although, you know, promising, these tele‑medicine sort of service programs were really relying on that same outdated tool from before, and actually, we required two providers for every visit, you know, one at the center and one at the remote location, which created services. so, with this sort of demonstrative potential, we set out to build tools specifically for this population, specifically for this use where we really wanted to design something to enhance our ability to do tele‑medicine and tele‑medicine consultations well. So, to do this, we mined a decade of other assessment data and gold standard sort of instruments to a computational approach where we could quantify key aspects of decision‑making and then, most importantly, had families and clinical experts translate these computational solutions into meaningful observations and ratings that were intentionally constrained by time, by person, by setting, by population, to create a new evaluation tool that we called the TAP, the Tele-ASD-PEDS, designed for actually a provider to walk that family through interactions with their children in order to be able to determine an initial triage assessment of ASD sort of risk. And we were then fortunate to be funded to gather data from families and clinicians about optimal use and accuracy and comparison to both standard assessments. We had already completed some of the initial feasibility user studies, providing what was coming from our families. In March, when you know COVID came and stayed, and not without irony, the pandemic shut down our tele‑medicine trial for a brief period of time, because we actually couldn't have a comparator condition to in‑person to assess some of the accuracy, but the pause gave us time to look at our interim data. And you know, we found, you know, very high levels of initial accuracy for those that had made it through. By the time we pressed pause in our initial classification, we're talking about 95 percent agreement, again, confined to this setting, our clinicians, trained providers, etc., but promising. And you know, because, in March, when every developmental clinic around was suddenly and simultaneously needing a tele‑medicine tool and solution, when our clinics were all kind of shut down all at once, we actually did what we could to make this tool with full transparency of the data limits and where we were and we needed further evaluation to the country, essentially.

So, we posted webinars, offered supplemental consultation training, made everything free and open and tried our best to gather data about what was happening with that process. You know, so, it was like a field trial by necessity, right? We had almost 2000 folks come in and receive training from, you know, 39 states and one territory, and we tried to gather data on sort of use, practice sort of behaviors and accuracy over time. Most were not using tele‑medicine prior to COVID, and now, you know, a majority shifting to that. We surveyed those who started to use the tool. Some 85% who are indicating they will continue to use its post pandemic. And moreover, I think on a system level, it was very interesting to us that, you know, we had about 82 percent of children tracked within our own Vanderbilt system, a couple hundred kids over that time, where we felt just by deployment of a brief assessment like this in a developmental center are actually at home under this circumstance, that these children were able to kind of, be routed without necessarily the clinician desiring sort of the traditional, more in‑depth sort of assessment. When we pushed this out to, those who have been attending the training, some 62 percent of children seen by these providers were noted to have their initial sort of triage decision be made with certainty. So, this was kind of interesting and a stressful time to be pushing some of this out, and it feels very promising for a number of reasons of thinking about modes where tele‑consultation or brief models of consultation may actually be the preferred model of care for families and clinicians.

And ultimately, I think the data really supports this idea that, you know, developing tests that are explicitly designed for the population, the setting, and the barrier, rather than adapting sort of existing tools that have been used for quite a long period of time helps provide better care for that population. And you know, that addressing disparities really requires that, and failure to adequately develop tools for your population, specifically for your population means that that population oftentimes may receive inadequate or suboptimal care. You know, we, obviously, need to understand so many other things about this tool, this was a R21 that were trying to take out, but it was interesting to see how it got launched within the COVID epidemic. And ultimately, we need to understand how tools that are developed really overcome, the most pressing issues for service. You know, in this case, are we providing meaningful answers that drive meaningful action for families? So, I'll conclude there and thank all of the forward‑thinking leaders and my collaborators here in Tennessee and our families. So, thank you.

DAWN: You're welcome, Dr. Warren, and thank you so much for ending the panel, which was already very terrific, on a very high note. And thank you to each of our panelists. I encourage everyone with a question to send it via the e‑mail address found in the chat or in the workshop invitation. And I would like to ask the panelists for their own questions in a moment, but I'd like to start off the discussion with a question of my own. What do you think are the tools to address health disparities? The methodological tools, the analytical tools, the cultural rigor tools that we need, that we don't have, to address mental health disparities. What kinds of tools do you feel are not available that we need? Does anybody want to start with the um…

BOWEN: This is Bowen.

DAWN: Yes?

BOWEN: I can give you some. You know, I do, I don't do genetics research or, you know, I don't do observational research, I do intervention research, and one of the things I've been very struck by from the methodological perspective, you know, just in disparities, but in mental health research in particular is, with intervention research, that we're so focused on medium to large effect sizes, both in dissemination research and in intervention efficacy and effectiveness research. And my suspicion is that, you know, in other areas of medicine, and with the vaccine trials, it's a clear example, we're very comfortable with doing large, you know, very, very small effect sizes, like the women's health initiative study, you know, had 25,000 people per arm in a three‑arm study, you know. And so, and what I'm struck by with disparities, as well as with, especially if you want to try to do something that's not just observational, is really thinking about how we sort of link clinical work to population‑based health outcomes, and I think they're going to require much larger, sample sizes to tease out effects below 0.2 or 0.1 even. You know, I think, I was looking through the, what's the group at the federal level that the American task force, you look at the recommendations that they have, and most of the recommendations they have, especially for physical health conditions, are based on, you know, sample sizes of 10,000, 5,000, 20,000, 100,000, 200,000. Yet as mental health clinicians, we look through the frame of what a usual, good‑size practice would be for an individual clinician. I really think we have to change our mindset. I think the second big area in order to address disparities, the tools, is we need a way to think about, especially in services, is how to link up the social determinants in a meaningful way with clinical outcomes and standardize those measures. And I think getting, and figuring out how to link sources of data that are not just healthcare, you know, how do you take these big, large administrative datasets, where completely, you know, hard to interpret, even for the systems and settings, you know, the public system, how do you link out the housing with the income supplements, with the tax records, with the with the school grades, with incarcerations, Medicaid, and all of that? Like, we have no sense of how to link up that data, you know, and I think looking at figuring out how to streamline and organize that data in a way that's usable, I know there are a lot of efforts that are doing that, but I think that we just don't know. And so, those would be the two, and then third is trust and engagement, you know, tools around how to develop trust in communities, how to develop, effective ways of engagement, and I think, you know, CPPR is one I think Cory’s done a wonderful job of really trying to make an effort to make sure that people who are, you know, effected at the end, but sort of having some sort of template for that. I think there are other places that have done that. Anyway.

DAWN: Excellent. Thank you, Bowen, very much. Does anybody else want to follow‑up with an additional answer of their own? To what tools or strategies do they think are missing from our toolbox in terms of antilogic rigor, in terms of metalogic rigor, in terms of cultural rigor to address mental health disparities?

ALICIA: I would just like to add, I totally agree with what Bowen said, and I feel very lucky as a geneticist that we have these massive sample sizes, and we're able to query the entire genome simultaneously in a very homogeneous way. Every time we ask, what one position in the genome is, we pretty much get the same answer back. There are different ways that you can administer surveys, and you'll get different answers about how people are feeling at a given time. We need lots more longitudinal data and a much more even‑handed way of measuring environmental stressors and in measuring socioeconomic differences that will, I think, enhance our ability to understand health disparities in a way that we don't have a good grasp on at the moment. Then we're going to need, I think, a lot of statistical rigor in how we implement methods to tie together our big epidemiological datasets that have that depth at large scale for teasing apart those disparities’ questions.

DAWN: Thank you, Alicia. Crystal, I'd like to check in with you. Do we have any questions e‑mailed into our inbox? Yet?

JENNIFER: This is Jennifer. I'm monitoring the e‑mail for this session. We do. So, the first question is for Dr. Chae. You mentioned the intentionality of racist policies and practices for maintaining white supremacy. So, in a research context, how do we generate evidence regarding intentionality of policies and practices?

DAVID: Before I begin, I'm a social epidemiologist, so the kinds of questions that I ask are, you know, what can we do on a societal level to address particular inequities along the race axis. And so, going back to Dawn’s initial question of tools that I wish that I had as an investigator would be like the ability to implement societal level kind of social change. For example, like, intervening at the level of policing, things like reparations. So, with that being said, there are things that, I think, we have in our toolbox that we could do, like voting, and voting for people who are going to be supportive of more equitable social policies. And that's not only incumbent on us as individuals but to vote, but also to get the people around us to vote, right? So, talking with our parents, our friends, our neighbors, you know I think it's kind of an unfair expectation that people who are oppressed are responsible for dismantling the systems that oppress them, and, so, it's part of explicitly naming who benefits from racism and, you know, we should also talk about who's accountable. And so anyway, I forget the question now.

JENNIFER: Let me sort of ask a follow‑up question. The question was about intentionality, and to put, the study of racism and discrimination, sort of to make it a legitimate respected area of study, do we need to be able to document or measure intentionality? Or is focusing on differential impact an outcome sufficient?

DAVID: Right. So, again, to take a step back, like, I don't think we need to do a study to show that racism is bad. Like, racism is bad, period. It's, it's a social and moral hazard, regardless of whether it has an impact on health. You know, regarding, like, this idea of intentionality, a lot of the, you know, the racist acts that occur like on an individual or personal level are unintentional. You know, I mean, that's kind of one of the very deceptive things about racism, is that it isn't always overt. It's oftentimes subtle. You know, the kind of analogy that I make is that it's kind of like any kind of physical environmental toxin that's there, but we get used to it, and we don't see it. Now, how do we measure something like that? And so, you know, I think, people are coming up with interesting non‑survey‑based ways of assessing it. One method that I used in my work was looking at kind of Google search concentration for the N word, you know, and one of the reasons why I think that could be useful is that a lot of the discrimination, the interpersonal discrimination that people experience is not necessarily observable. So, let's say someone applies for a job and they don't get a call back, you don't quite know whether or not that was racially‑motivated or not, yet we know that the evidence suggests that if you have a Black‑sounding name, you have to send out, like, ten times as many resumes to get the same number of call‑backs. So, it's a real phenomenon, but how do you assess it? Because, again, it's very subtle and sometimes, unclear. So, you know, I think, I think using, like, some of these new big data approaches can also kind of, be one way of capturing racism.

DAWN: Dave, Thank you for that answer. I'd like to call upon Miraj next, please. I believe he has a question for us.

MIRAJ: Well, I was just going to follow‑up on the comment. So, I'm not sure if the person who asked the question saw my talk, but, basically, it was about showing how institutions should can be considered as kinds of agents that are capable of transmitting implicit bias at the level of norms, procedures, policies. So, the follow‑up study actually went further into the kind of ghost in the machine, and we were able to identify group psychological and structural mechanisms that drive implicit organizational bias. So, the take‑home is that we came to an understanding that institutions are kinds of agents that are capable of transmitting actual forms of perception through these things, like bureaucratic norms, codes of conduct and things of that nature. So, going back to the question of intentionality, it's a word we're very comfortable with in our research approach, because it's the very fundamental basis of phenomenological research, which is intentional analysis or the analysis of intentionality. All I'm saying is that basically, what we're trying to find out is, essentially, when you look at an institution, institutions, one of the functions for them is for them to survive. So, we need to look at what does a given organization need to survive, including financially, and then you're going to start seeing what kinds of clients they optimally require to survive, and then once you start getting into that, you start seeing, basically, who they don't want or who requires more resources and therefore are seen as non‑ideal. And the same kinds of manifestations operate within law enforcement, if you look, for instance, at reporting requirements and arrests quotas and things of that nature, these are the kinds of incipient insidious ways in which intentionality plays out at the institutional level.

DAWN: Thank you for joining the conversation, Miraj. We have a few minutes left. Does anybody else want to pose the last question or offer the last answer to one of the questions that's already been posed?

JENNIFER: We do have another question via e‑mail. This is for Dr. Warren. Can you talk about some of the technological challenges you experience with your approach? And is the digital divide, could that potentially worsen disparities in access to these types of assessment tools?

ZACHARY: Yeah, so, it's a great question. So, I think the short answer is we saw it all, right. When you start zooming into patient homes and other community settings, you think you have an appreciation of that as an empathetic sort of clinical psychologist surveying underserved populations, but it, you know, it really gives you a really hammer‑home perspective. There's some wonderful things in our technology, but in the same respect, many of the tools and instruments we've created through the years have been designed to be sort of analogues for actually getting to see what would happen in a real‑world setting, and we have families, some families, who are very comfortable and actually sort of appreciative of this idea of a technology that literally brings you into their world and sees their experience and what's happening with their child. But, yeah, certainly, we were operating, you know, in a time where we had nothing else, and tele‑medicine, I don't think should be construed as the be all, end all for all populations, because the last thing that we would want to be able to happen is to create an additional care disparity. You know, the data is pretty compelling, that more folks than ever are able to utilize one of these things to be on a Zoom call, and we had a really, really high rate across, our catchment, which serves a lot of rural areas that may not have the same access to broadband that other places do, but it definitely highlights the idea, and in our design we were going to be utilizing technologies within clinics where people who might have barriers to this could take advantage of it too. So, you need to think about it on a number of different levels to move forward, and, so, you know, our best data was there was about 12 percent of the patient population that we tried to capture over the past six months that was unable to participate in one of our Zoom‑ oriented sort of visits, so we need to have a mechanism in place for them, or else we're creating a care disparity for that population versus addressing the problem. I got long-winded in my answer but that’s my response.

DAWN: We do have just one-minute left. I invite Victoria to please ask her question, and that will be the last one we can fit in.

VICTORIA: I actually want to just make a comment about your question, Dr. Morales, where you were asking about, you know, sort of where are the gaps in methods, and, you know, as I look back to some of the work that I've done and that was reflected today, I think, you know, we need something sort of in between the qualitative, the purely qualitative and purely quantitative and some more mixed methods research that can really provide that depth and nuance to the quantitative work. One of the strategies that I've been using is really just embedding open‑ended questions within surveys, and those have been yielding sort of really rich, you know, rich data that we can then follow‑up in a more profound way, you know, either quantitatively or qualitatively, but it gives us sort of the sense of the breadth and depth of an issue that might be important for our communities. Another point I wanted to make was about the lack of homogeneity in the definitions, and this is something I've encountered and talked about yesterday. So, a good example would be, in measurement of, for example, something like housing, right, which we struggle really quite profoundly to measure in the re‑entry program that I'm leading through funding by OMH. And when you look at what is available for federal measures versus research measures, they don't often overlap, and it leads to really disparate kinds of interpretations and conclusions, you know, with the data, and I think that it would be nice to see more standardization of measures across programs that are federally funded so that we could kind of match our research to have real‑life implications and, you know, benefit to policymakers. I'll stop there but thank you very much for the opportunity to share that.

DAWN: You're very welcome, and I really appreciate you ending our panel on such a valuable note. I'd like to once again thank all the panelists for a very, robust and stimulating discussion. I believe I turn the control back over to my colleague, Dr. Crystal Barksdale.

CRYSTAL: Thank you, Dawn, and thank you to our panelists again for a wonderful presentation and discussion. So, we are now going to take our break, a 20‑minute break. I invite panelists and speakers to please turn your cameras off and please mute yourself. We will return at 2:40 for our final roundtable discussion, again, where we will be discussing the challenges and opportunities for framing a multidimensional mental research, mental health disparities research agenda. So, we look forward to wrapping up the day with a very enlightening and invigorating discussion. We'll see you back here at 2:40. Thank you.

BREAK

JENNIFER: OK. So welcome back, everyone. Hope you enjoyed the break. And we do appreciate everyone for being part of this workshop the past two days and for hanging in there for this final session. Crystal, did you want to say anything or since I see you on the screen?

CRYSTAL: Yes. No, I echo your sentiments, Jennifer. Welcome back, everyone, and thank you all for hanging in there for this last roundtable. We are looking forward to an engaging discussion, so I will turn it over to you, Jennifer and you, Denise, to kick us off, and I will be in the background, monitoring questions and looking for our raised hand from our panelists or presenters. So, presenters, again, please feel free to chime in and share your opinions to our questions. Jennifer and Denise?

JENNIFER: Okay. So, and I see Denise is in motion there. So, yeah, in this final session, we wanted to really sort of pick your brains a little more about the directions that we should be heading regarding mental health disparities, and by we, we mean not just NIH, but the field as a whole, what we need to be doing, and we really want to hear your thoughts. The goal is not to come up with a consensus or top priorities or synthesize anything, but, really, just to hear all your individual thoughts and opinions. We are going to focus on the panelists, as Crystal mentioned, who were invited to share their expertise, as well as, some discussants we have in the audience who, you're a discussant, if you have an asterisk by your name in the participant list. We probably will mostly be using the raise‑hand function so that we can hear directly from folks. We definitely encourage other participants in the audience to let us know any comments or thoughts in the email, the email box that we have been using for two days. Though just to note we’re sort of finished with the Q&A session where you ask questions of the panelist about the research. This is more where we are really looking for thoughts and feedback about that. So, and Denise did you want to add anything?

DENISE: No. When we get around to it, I have a couple questions, but I want to hear from folks first.

JENNIFER: Okay. And the first. We just wanted to open the floor, if there were discussions that we had the past two days, that you had an opinion about something and we ran out of time, we wanted to open the door for you, not questions, but comments and thoughts, so just wanted to see if there were any, if there was any unfinished business. Okay, well, why don't we, Denise, if you want to ask your questions?

DENISE: Oh Sure. So, our broad theme was about multidimensional research in this area, and I'm wondering what folks see as particularly potentially influential, but neglected dimensions in research or combinations of dimensions that might be fruitful pathways for moving the research forward.

JENNIFER: And I think Dr. Thames had a comment but can't raise her hand. So, I don't know if it's possible to un‑mute.

APRIL: Hi. Sorry, I tried to do the more function, and it wasn't working. So, I guess, the one question I have, or comment is, you know, thinking about there are these large studies, and I think this was mentioned at some point, that already exists with these very large cohorts. And for funding purposes, it's often strategic for people to sort of piggyback or to, you know, kind of use existing cohorts, but they were often not designed for looking at the issues that many of us are interested in when it comes to health disparities. These large cohorts, and I'm just thinking, you know, in the work that I do with dementia being a component, and there are these large, like, Alzheimer's disease consortium data. And I've actually looked at it, but it has no data related to discrimination experiences, the sample's predominantly European American, and yet if I were to propose, you know, starting a cohort to collect very rich data, it's likely not to get funded. So, how do we, I guess, move from the political issues of trying to obtain funding, and all you know, to where we can actually start to look in‑depth at these issues? Is it that we should be proposing to go back to these cohorts that exist and collect this data or collect the new data with the health disparities framework in mind? So, if anyone, that's my question and comment.

JENNIFER: And just a technical note, the raise‑hand, some people find it in the more, but if you open the participants list and you see some buttons at the bottom, invite and raise hand, that's where the raise‑hand button would be, just in case. So, do any panelists want to address that question? Or we could just leave it as a question that the field needs to grapple with.

JESSICA: I think that's a really important question, and I think it relates to another sort of tension that I see in doing this research, which is also about whether, I mean, I think when we partner with communities, we know that so many communities have been studied, and they feel like they've been studied so much, and, so, the usually, the kind of research that communities want to partner on is intervention research. I think this is important, because we are really working together with communities to create change, and often, if we have, if we're doing intervention studies, we end up collecting a lot of data that enables us to analyze and look at other questions, more etiological questions and mechanism questions. So, I think that's important, but then that requires even more funding in a way, to really be able to do a large enough intervention study that we have the sample size to also answer these kinds of questions. So, I have a similar question about, really, how do we do that and what are the most strategic ways to fund this kind of research, but I think it's important to recognize what the urgencies are with our community our partners as well.

DAWN: So, this is Dawn Morales, and before COVID‑19 struck, me and my rural colleagues at CDC were, we wanted to try and hold a workshop/conference about, the, well, ostensibly, the main focus of the workshop was going to be the many definitions of rural and what the implications are for them. I don't know if you're aware, but, like, the word rural has more definitions at the federal level than like almost, like, any other word, and, you know, in any way related to health, and, we wanted to talk about that, but we, at the conference or workshop, we also wanted to just really talk about the practicality of trying to build a large database of social determinants of health by some kind of index that was helpful in the United States. For example, ZIP code, so, like, you know just whatever, like, evictions or just percentage of people who get food stamps or, just you know, whatever, I mean, there's a lot of federal data around floating around that you could get and would be a very helpful resource for researchers, if they could access something like that, even if they wind up not being able to get at individual or family level data for the specific person they're looking at. If they just even had data for the community that person resided in, which actually is, very interesting, so that's something that I have thought might be of value at some point.

JENNIFER: Okay, thank you.

CRYSTAL: We have a question, we have a comment, from an attendee. There's a note, there seems to be a gap in policy research in health disparities and how policies affect sub‑groups. Do any of the panelists have any thoughts about that? Let's see. Debra? Thank you.

MAGGIE: This is Maggie. Could you put me in the queue?

CRYSTAL: Okay, yes. Debra, then Maggie.

DEBRA: Okay, so, I actually raised my hand to respond to something that Dr. Morales said, so, Maggie, if you want to take the policy question, you can cycle back to me.

CRYSTAL: Maggie, do you have a statement about the policies?

MAGGIE: Yes. I think, I think it is, has been a tenuous situation for people in the institutes to take policy that can be, I don't know, seen as political or advocacy, and therefore, I think there has been this difficulty in, really looking at especially policies that have to do with, like, for example, social determinants and whether that also changes the, role or, the role, the assigned roles that the institutes have that has to be around illness or healthcare. And I think that that's the reason that, sometimes, some of the policy issues are not addressed or brought to the forefront, but I think given a lot of what has been discussed today about the importance of focusing on policy and system change, I think that needs to be really, something different that is considered by the institutes. And lastly, I would say we need also to engage more with policymakers. I think the problem has been we've come way after we do the work rather than in partnership to do the work, and therefore, I think we're not really understanding what policymakers need for decision‑making.

JENNIFER: Thank you. Debra, do you want to go ahead?

DEBRA: Thank you. So, one of the things that struck me over the last two days across these terrific and often inspirational presentations is the extent to which, context really matters, and many of the studies, people talked about sounded, you know, really great for their context, but then I tried to imagine transporting them to our central great plains context or someplace, you know, outside of the coast or large urban areas. And it just, like, there's a very, there’s a very different context that matters a lot. So, I was thinking about what Dr. Morales said about having, you know, sort of this nationwide kind of database where you could get some of that context, that seems like a great start to me, and also, as we're thinking about this work, to make sure that we're not just doing it in one place and assuming that it generalizes every place else. Because the level of infrastructure is so different, and it's not that there aren't any people of color who are in other parts of the country, it may not be as dense, but there's certainly LGBTQ folks, rural folks, immigrants, who might have a very different set of needs or need a very different set of solutions, so I guess I was just making a plug for those of us, and I think there were a handful of papers that came from other contexts.

JENNIFER: Thank you.

CRYSTAL: Bowen?

BOWEN: Yeah, this is Bowen. I would say that, you know, one of the things, and I think this is a good reflection of the people here, is we really need to focus on making sure that, you know, sexual gender minority and racial ethnic minority individuals are actually able to pipeline of investigators, especially in the applied work area. You know, what I'm very struck by, as someone who's kind of survived and managed to continue to get funded is that there, you know, the individuals that I have to collaborate with have kind of have gone along have not been able to get funded. I've been very fortunate, and sort of figuring out how to, you know, support people, especially at the junior, mid‑level, to keep this work going, especially in the applied work and disparities area, I think you can't, methods don't help, if you don't have people who can actually figure out how to do it in these funding announcement. But, really paying attention to the pipeline of the types of individuals who are going to get these kind of research grants, is, I think, really important and supporting people in this area.

JENNIFER: All right, thank you. I don't see any more hands raised. Denise, did you have additional questions you wanted to ask?

DENISE: So, I, in thinking about dimensions that we need to pay more attention to, and someone mentioned policy, so that's a pretty broad level dimension, but I do know that some of our folks really do community‑level type of research, not to put Andy Subica on the spot, but I would say that's an outstanding feature of his work, and I don't know if he might want to comment on the challenges of doing that kind of community‑level work. Sorry, Andy.

ANDREW: Can you hear me?

DENISE: Yes.

ANDREW: Yeah, not on the spot at all. Thank you.

[Laughing]

ANDREW: No, yeah, I guess, I can speak to that. One of the things I've still been thinking about is the earlier question around policy and what Maggie said, if I can just actually just jump in there real quick, because it has implications. I have been doing work around grassroots organizing for some of these social determinants, and I've never actually taken it to NIH yet, because I was a little bit concerned that they would be interested in something like this. Now really likes this grassroots organizing, this social determinants, this culture of health type of thing. And so, we had to also navigate some of these policy issues, like, for instance, Maggie was saying some reviewers might be cautious around this, and NIH might be cautious so I would actually be curious to know, because one of the things that we had to be very careful was you can't, you can advocate, but you can't lobby using, was what we were told, using government or foundation funds. So, a lot of the communities I would always work through would advocate to things like public health departments, you know, build these partnerships with hospitals, and that seemed to be ideal, and then they would try and find extra pockets of money for their own advocacy work to elected officials. And I thought that was actually really clever. Now, I'm not sure if NIH would be interested, but to speak now to your point, we have to navigate. I always get, when I work through these community organizations, like, with Pacific Islanders, I have to ask them to do the work for me, to do the surveys in the community, because I just can't access, you know, they wouldn't trust me to answer questions around mental health or stigma or sexual abuse, you know, anything like that. But they would trust somebody from their community who looks like them, and then whenever we do that, you're right, it's challenging, because I always have to be very open to everything they suggest. So, I do find that there's some tension, I don't think it's too much of a challenge to navigate the rigor that we want and that I would write in an application with the actual real‑world implementation of some of these things, and I realize, sometimes, that might be, probably, the biggest issue. Is in order to get things done in the community that they would accept, we have to tamp down some of the rigor and just figure out how can we still get papers and move the field forward without doing randomization, right, or something that they wouldn't accept very easily.

CRYSTAL: Okay, we have another question.

JENNIFER: Sorry, Crystal, can I follow‑up on that question?

CRYSTAL: Yeah, please.

JENNIFER: I just want to, would like to hear about how other panelists have balanced rigor with community needs and priorities. If you would like to comment.

BOWEN: I can talk.

DAWN: Methodological rigor versus cultural rigor. I just, you know, like, rigor goes in multiple directions. That's all I just wanted to say.

BOWEN: I agree with that comment a hundred percent. I agree with that comment that was just said a hundred percent. I think there's both methodological rigor, as well as external validity, and when I say external validity, we usually think about effectiveness or dissemination implementation, but sort of the uptake and sort of whether, I think, trust, which is the big word of the moment, trust in science, right, whether people will trust the results and find them to reflect their values and beliefs. I'm just very struck by that and how problematic that has been, and I think, you know, one of the big questions that's going to come out about services is, you know, even after the pandemic starts slowing down and we get the vaccines, the big crisis we're going to face right now, and this is an opportunity, as well as a big challenge of how to deliver, how to figure out how to deliver the same type of sort of care and put the same type of attention to addressing the behavioral health inequities, as well as the behavioral health pandemic that’s going to, that's already taking place that we haven't been able to track. Or have been tracking on the margins, I mean, this is, you know, I see it as every day as somebody who works in a major public hospital in the emergency room and sort of the, I mean, we're taking over the COVID unit, because we have so many suicidal kids and aggressive kids, but how do we, in real time, really take advantage of these opportunities at a population level to address the mental health sort of, like, epidemic that's already emerging from this. And I think the other big question is, you know, figuring out how to pay community partners for what they actually do. I think PCORI’s figured it out a little bit, and then also figuring out how to sustain these individuals over time and organizations to continue to participate in this work. Because oftentimes, we have to pay organizations to do things that are, you know, data collection's not always the best way to sort of keep them, and they're like any other infrastructure. I think that’s where the clinical translation science institutes sort of think about them. You know, they're relationships that need to be just like an MRI or a lab system, they need to be maintained and groomed over time, and that's the only way you're going to get these kinds of robust sort of end roads into communities that have not been well‑treated by the science that's supposed to help individuals. And how do you fund that, given sort of, you know, what the five‑year cycle or two‑year cycles or three‑year cycles that we have to work with? And people rely on this to make a living and pay the rent and, you know, get food on the table, and most of us are going to be fine. But then if we can't sustain them and something drops after the funding stream ends, they're in big trouble.

CRYSTAL: I think Kiara has her hand up.

KIARA: I agree with everything that was said, and one of the things that really struck me over the last couple of days is seeing all of this amazing community partner and community engaged work and then also knowing how challenging it is to have that work funded and sustain it. I think to the piece about rigor, you know, there are a lot of things we see at the end result of a study that really stem from relationships, and, so, we see, you know, the 80 percent retention rate in a longitudinal study or being able to retain clinical trial participants over two years or someone being able to take, you know, a community‑based intervention from efficacy all the way through implementation and dissemination. And the foundation of that, really, are those relationships and that work that's being described, but that piece is often not recognized in funding, and also in academic work, you know, so that information is being passed down from mentors to mentees or from, you know, collaborators to other collaborators, but it's not always getting accepted or published in journals as being part of the scientific rigor process. And so, I think really focusing on that idea that those relationships and those contributions of all of the research team, not just the one that have the named appointment at a university, are really what's driving that work and allowing the scientific work to meet those standards, I think would be an important shift going forward and was, to me, very clearly demonstrated by so many presentations in varying areas of work.

MIRAJ: That's a good point too. What’s more friendly than respecting what you just said.

CRYSTAL: I see we have, I saw Tiffany's hand, and I see Miya's hand, but we also have a really important comment that I do want to interject here from an attendee. There's a concern that this conversation about meeting community needs being a challenge to rigor is rooted in white supremacy and would challenge panelists to consider that ideology. So, I saw Tiffany's hand and Miya's hand, so please, I invite folks to speak to that as well. Thank you, attendee, participant, registrant who shared that comment. Anybody want to respond? Miya, and then Tiffany. Or Tiffany, I'm sorry, I saw your hand first, and then you put it back down, so, Tiffany, please, and then Miya.

TIFFANY: You know, I think that's a great point about, you know, challenging us to really think about what we bring to the table and how we, the lens that we look at different things, and, so, one of the things that we do with our community partners is we really take the time in the beginning, relationship matters, to build that relationship and really talk about what we all bring to the table. And to let them know what they bring to the table. They have what we call Ph.Ds. in community, and they bring that community science to the table, and so it's not, for us, a compromise of rigor versus community, it is an opportunity for us to say how do we work together, because we've been doing it our way for a very long time, and we haven't seen the needle move in disparities the way that we want to. So, they're bringing something to us that's going to help us move the needle, and by acknowledging that and emphasizing that and understanding that we're able to have compromises and conversations where we're actually moving the science. Shout‑out to Jennifer, because I know, sometimes, she probably gets sick of our conversation when we're like, here's what the community's saying, here's what we're saying, and here is where we think we are going to meet in the middle. She's been very responsive to that. That helps us understand and emphasize to them, when we can go back and say, hey, NIMHD says we agree with you, this is a way that we can go to actually really move the needle.

CRYSTAL: Thank you. Miya and then Bowen and Andy.

MIYA: I just really want to thank the attendee who brought up that point and emphasize what John has said and many other panelists have said which is that we cannot, we cannot say that it is a continuum that goes from rigor to community focus. If we do that, that is very dangerous, I believe, it, it's saying that working with a community is not scientific. And we, all of us doing this work know that it is. I wanted to point to something that I feel like Tiffany had addressed in her presentation and that I have found to be really helpful when you are, maybe, asking a question about how do you work with communities to do some of the research activities that need to happen, collect data, as we've talked about. Or, potentially randomize, if that's the design that we've decided is the most appropriate, and one thing that I think is important is that we are getting our questions from our community that we are working with, you know and that's why I'm pointing to Tiffany's example, is that we can hear from our community partners things that they want to know about how to make things better. And then part of the way that we are helping design our trials and implement them and get them excited about the data collection and wanting to be part of it is by really drawing together how these activities are helping to answer the questions that they themselves have for the community. So, I think that that has been something that I've found to be most effective in getting buy‑in around not rigor, but around what research activities need to happen is just, say, like, here's some ways we could answer these questions that you have or needs that you might have so that a lot of community partners are also applying for funding not to NIH but to private foundations or to different governmental organizations. Really making sure that values are aligning and that in the questions we are asking and thinking about how we can answer them together.

BOWEN: This is Bowen. I don't think that, you know, I think we were able to pull off a randomized study, looking at lots of different outcomes that, that you know, in all the studies we've done, and I did that one, because that's the best example, but we have other studies where, you know it's actually improved the quality of the science. We, in our study, we had depression originally as the primary outcome, we had usual care compared to this community engaged approach, and then the community said, no, right? They said we have to have two conditions, and then we need social outcomes, like incarceration, housing, all this other stuff that came up. And you know, we had no idea what the effect size for some of these things were, and, so, but in the end, they were right, right? Their framework for structural racism, was not viewed as rigorous. I couldn't get an abstract accepted to anything between 2003 and 2012 to the American Psychiatric Association, where I had anything about structural racism. I actually, not to, and, you know, I remember my first call when I was a post‑doc with the with the NIMH and, actually, the program officer at the time said your partnership doesn't exist, this is not science. Loretta was on the phone at the time, Loretta Jones who was on Zerhouni’s advisory committee, and she nearly bit her head off, but I would say that, you know, in order for us to have, quote unquote, rigorous internal validity and rigorous external validity, the challenge is the time and energy it takes to gain trust. And what is one rewarded at the NIH and academics in order to do the activities and write about it in such a way that will get you papers, and it doesn't always lend itself so easily, so all the participatory research out there, the reason why it's all on process, because that's all people get funded for, and that's all people have time for. I mean, they need to generate papers, right? That's why we're the first study that showed that participatory research actually has an effect on health outcomes. The funding cycles of the five years, I think everybody knows this, right? You just can't do it in that time, especially with, or to do something, and then once you have it, how do you maintain it, if you've built it over 10 or 15 years? So, I think you can have your cake and eat it too and have really rigorous methods. I think the challenge is that it's not that the methods aren't, that we're not using rigorous methods, the methods haven't caught up to the way that community partners and stakeholders think about things is my is the way I've looked at it. That we don't have the tools and the types of data and the mixed methods, the econometric methods, how do you think about, you know, health‑related quality of life with really poor people, how do you do cost effectiveness analysis that considers not just the trade‑offs for health, but also social, like, how do you compare housing with an antibiotic or food with eye surgery, right? So, I think our methods haven't caught up, and our data sources and our measures haven't caught up to what the community wants. They're already there, but we, you know, and I think the other thing to think about is despite the internal validity methods, I think the other question is, you know, how do we sort of reward external validity and engagement? You know, how do we reward, how do we track how the science actually improves health outcomes at, you know, using sort of CDC or panel data, right, and sort of say, hey, because of this, we've actually reduced or eliminated health disparities because of this happening? And I think that's the other part that would be helpful to think about and track if that's even possible.

CRYSTAL: Great. I have Andy up and then April. Thank you.

ANDREW: Okay, yeah, I would say to the person who wrote that comment, 100 percent. Thank you for saying that. I just, I intrinsically agree. Bowen was having, had a lot of good points, and what I would probably emphasize is, maybe, internal to me, but I think other people here believe this too. My favorite presentation to give is I name it “Community has the Solution,” and I go to different academic places, I go to different conferences, and I do that presentation, and I really like it, because I truly do believe that in many cases, especially when we're talking about health disparities, communities usually have a really well‑situated knowledge of what's going on, what the needs are. And then they also have some concept of what they would like to do and, maybe, even approaches. Where I think the breakdown happens is I don't think we yet have good models and methods for bringing the community, I guess strategies and needs into this kind of NIH or academic world, and I think we're developing some of these things now, but I really feel like that's the reason we have a translational gap. One of the big reasons, is it's so much going one direction, and I think we could really figure out how to move the needle on these disparities, if we can start also looking a little bit more, ground‑up is a lot of words, culturally grounded, but how do we really accept that that's viable and rigorous, and I think that's probably a conversation this group would have. I'm glad that we're here. I know of Bowen's work, April's work, I now know of Miraj's work, I mean, a lot of really cool people are doing cool things, and I don't know yet that we have really capitalized together to figure this out. And I guess that's what I would say is, maybe, a little bit radical, but I would like to do more of this work, so that we could change the systems, especially from, Dr. Alvidrez has already demonstrated the model that I think would help us figure that out, and I'd just like to have more conversations about that, because I do feel like we're missing the boat a little bit with really focusing like Bowen said on internal validity and less on external validity when they are I guess in conflict with each other from time to time. That's it. I think we can do both.

CRYSTAL: April? Thank you.

APRIL: I just wanted to add that one of the systems that I think stand in the way, unfortunately, is the way we think about science. Science has historically promoted white supremacy through its practices, and I think that when we start to think about the studies that people are trying to do at the community level and this whole idea that it's not as rigorous, I mean, that definitely is embedded within white supremacist thinking. And but I think we have to change the way we review grants. I think the way academic systems promote people and what, and the pressures on early‑stage investigators to produce all these papers, yet if you're doing real community partnered work to address health disparities, it's a slower process. So, every, the way we think about science and the way that early‑stage investigators are funded and promoted, it's all within a system that, unfortunately, has not broken away from white supremist thinking. So, I would just add that to the conversation, because that has been something that I don't see us moving the needle till we actually take a look within our internal system of science and how we go about doing things.

CRYSTAL: Thank you, and I see Vicky's hand, Victoria's hand as well. Thank you.

VICTORIA: Hi, everybody. I wanted to follow‑up a little bit more about sort of the issue of broader community priorities. I think what this year has shown us is, very painfully, that social justice issues and, COVID, for example, intersecting, but, with respect to the social justice piece, it's not really very well‑reflected in the NIH portfolio. I think, you know, colleagues at UCSF, you know, published a manuscript a few years back, saying that 0.1 percent of the NIH portfolio focuses on justice system issues or justice‑involved populations, and I think, you know, that would be sort of an area that I would see really important going forward as NIH thinks about, you know, how do we, deal with these intersectional communities that are facing stigma, structuralized stigma, and discrimination. That intersect at the you know core of physical health, mental health, and, ultimately, community health, and, I think it, you know, again, for certain communities, you know, I know here in San Diego, that's a very big issue in California, it's been sort of a priority to focus on, social justice issues, and, again, without focusing on that side of, you know, on those topics, I think it would be very difficult to move forward kind of in a broader health focused portfolio. But I think, you know, one of the things that I've spent the last four and a half years doing as well is partnering with somebody that, agencies that would be not sort of traditional public health partners, you know, the district attorney's office and probation offices, to engage them, and educate from a public health lens, how do we promote health and well‑being in these communities. And I think that has been a useful approach, some of the tools that we've designed have been, you know, taken up by these agencies as they're interfacing with their clients, and, so, I think just something to think about, that, you know, to have that perspective sort of growing, you know, in the NIH portfolio, but I think also requires greater education of, of our colleagues, of other investigators, to be more aware of social justice issues and particularly the health of justice‑involved populations. So, thank you.

JENNIFER: Thank you. I see Miraj has a hand up.

MIRAJ: Yeah, I think, just, this is an important discussion, and I'm very inspired by the comments. Just wanted to say that. Part of this, part of the answer, I think, to the broader question also has to do with honoring and recognizing those who have been speaking this, these truths for decades and were never considered as science despite being the only ones saying these truths. In 1967, I believe, Dr. King gave a speech to the American Psychological Association, where he basically implored behavioral scientists to study racism more, because up until then, it was social movements who was raising these issues. Around the same time, Stokely Carmichael and colleagues are the ones who developed the very concept of institutional racism and its relationship to health. Prince went on in the 60s, talked about mental health institutions containing a preexisting bias, a preexisting framework, and I could list a dozen Audrey Lorde quotes here about the relationship between oppression and, well‑being. Gandhi, in 1927, wrote of the disease of racism. So, you know, I think that because of so, that's in response to that question, is that, these have not been considered real, I mean, even now, the Buddha came around thousands of years ago, many, many years ago, and only now, the scientific fields are considering it real by validating it suddenly. We've known it has been real forever, but now, and it's wild to me that we've even had, colleagues from Hong Kong visit us, saying, oh, we can only study mindfulness, if it's taught to us by a westerner. So, you're getting into this whole, so, it's really complex issues, once you start opening the question of science and the legacies of history and things of that nature and what has been considered rigorous and scientific and things of that nature. And I think there's just been a lot of non‑recognition and a lot of pain that comes along with that. I know I feel it on a daily basis. Suddenly, in the last several months, people consider work like ours valuable, but even, like, and that's some people. I think for years, it was just not considered valuable, and it was, we were slowly and steadily weaned out of spaces. And it's been an incredibly difficult journey, and I think I'm speaking, I think a lot of my panelists would agree.

JENNIFER: Thank you, and Deepa has a comment.

DEEPA: Hi there. I'm probably changing tack just a little bit, but I was intrigued when we were talking earlier about maintaining partnerships with community and was just curious as to the experience of some of the others. One of the neat things that has come out of our project, and I'm doing this school‑based depression screening again, is that some of the folks we are working with have actually come to us to partner on other opportunities to validate in a more rigorous fashion the work that they're doing. So, some of our mental health partnerships, they're community‑based organizations, they have educational programming out, have asked us to help them design something to evaluate the work they're doing and show that it is effective, and that's been a neat opportunity, sort of flipping things backward from how we've been talking, to take our expertise and help community groups show that what they're doing makes a difference.

DENNY: Can I ask a question here? Deepa, if you could tell us a little bit more about with, did they have, was this just for themselves? Sort of self‑monitoring how well they're doing, or did they have plans to take that validation information further?

DEEPA: So, it's an educational curriculum on depression and suicide recognition that a lot of Pennsylvania schools are using that this non‑profit has put out, and there's no evidence to support that the curriculum is effective, so they asked us to help them design a study to demonstrate that it was effective in increasing student knowledge. Because I think the concern is it's been adopted pretty rapidly across the state, but there's no evidence to support that it is effective, I guess. I hope that answered your question.

CRYSTAL: And I saw Andy's hand was raised. Andy?

ANDREW: Yeah. Hi. That was actually a really good question. I don't exactly have an answer to that, because I feel like it's a much broader or deeper conversation I would love to actually have with you, Dr. Sekhar, about this, and, so, that's why I actually just wanted to ask Dr. Juliano-Bult, Dr. Alvidrez, because I'm really encouraged, and I've spoken with both of you before, you're very supportive of this type of work. And that makes me feel so encouraged. You know, we have this group of people here who are doing so many things that I think we can all learn from, like, how do we continue this, if I did want to talk to Dr. Sekhar as opposed to just trying to reach out to her via e‑mail? You know, is there a way that we can continue this? Because I feel like this is wonderful in a way that other cohorts I have been involved with haven't been, because it's so specific to mental health and creating new methods and figuring out new ideas around this.

JENNIFER: So, I'll defer that to Crystal about potential plans going forward.

DENISE: If I could jump in, I think it would be safe to say we don't have a plan yet, right? We have a lot to talk about from this meeting.

CRYSTAL: Yeah. So, we do, yes, so, there's quite a bit of plans in terms of follow‑up. Thank you, Andy, for the question. There's lots of plans in terms of follow‑up, but one of which is, and actually, we did discuss this, in terms of how to, I think, coalesce and figure out a way to facilitate networking, so we will follow‑up with the presenters in particular to figure out how we can facilitate that. I see, Miraj, you also have your hand raised.

MIRAJ: Yeah. I just wanted to echo that. You know, I have been noticing, so, one of the things I study also is the relationship between social movements and health and mental health, and there's clearly some kind of, kind of a group collective energy here and, you know, in a way, it's a kind of intervention on its own. So, once we start expanding our sense of what intervention means for our field, things like this become that. And so, there's, this is the path of movement in structural change, so I would just support and echo the need to continue building infrastructure, and we, I think most of us would be happy to participate.

JENNIFER: So, I don't see any more hands raised. Just wanted to, so, we have 10 minutes, is there anything that any of you feel is important to think about as we try to move forward to address health disparities through research that we have not…that no one has mentioned that you think is important to mention? And Debra has a hand up.

DEBRA: I was just sitting here, thinking, one thing that we've talked about, kind of indirectly, but hasn't, I haven't heard as explicitly as the issues around intersectionality, and there have been a few talks that kind of addressed it, but just wanted to sort of raise that to our consciousness again about, you know, multiple identities. And there were some nice presentations that included that but wanted to get that word back out on the table.

DAWN: I guess, this is Dawn, something that I haven't heard too many people talk about is I really feel like there's a communication gap, I feel like those of us who are inside this circle or who are in this room today, I think we all understand each other pretty well, and I think we largely agree on a lot of shared priorities and things that are very important. But I feel like I'm not sure we're doing a great job of communicating, a lot of what we know to people who are not stakeholders in our community, and I think that that's something that I know I've been thinking really hard about how to work on that.

JENNIFER: And Dawn, can you say which stakeholders you're referring to?

DAWN: Well, people not here, people who are, people who would say something like, well, you know, the poor have always been with us, you know, the poor are with us today, and they were with us yesterday, they'll be with us tomorrow. I hear people say things like that. Poverty in America today is very different from poverty a few decades ago even, so, you know, I mean, I'm full of all kinds of very specific facts and figures, but I feel like I just, I don't know, I feel like I could do a better job of communicating, that, for example, poverty is very different today, or that, structural, you know, people who want to talk about racism, but their construct of racism is entirely something that happens on an interpersonal level between two people. And they don't think about, know about the idea that there are systemic structural components to racism, for example.

JENNIFER: Other comments about things we haven't covered, but should have covered, or any other comments that you feel like it's important for us to hear?

CRYSTAL: Can I add just one question? I think I want to hear from the group, or, I guess, some of the, we've alluded to this, I think, throughout the two afternoons, but I think just kind of getting some overall idea about, and I think you were probably leading up to this, Jennifer, but some of the challenges that we need to kind of be aware of as we craft the agenda and think about what this agenda looks like. What are the overall challenges that you meet that we should be aware of? And, again, we've heard them kind of in bits and pieces from yesterday and today, but kind of leading from, I think leading from Dawn's point, in messaging and in communicating your work and communicating your experiences and engaging with your partners and your varied partners and in doing it and in doing this work and in doing it so well. I'm just curious, for kind of a summation if you will, of some of your challenges.

BOWEN: I mean, the big challenge is making sure to keep everybody engaged when I think, first, for me at least, you know, when a project is running out of money, especially the community partners that need resources to really work with us. And also, the other challenge that we have is figuring out how to fund engagement with not just, you know, sort of low‑income stakeholders, but sort of the policy and the health system and the, you know, I guess the insurance companies and developing sort of approaches that would, getting the resources to communicate with them effectively about where we're at in different projects. And then what the results are in ways that, you know, people have used creative ways like video or websites, but, you know, sometimes, one has to pull that funding out to, you know, we have to sort of be creative about how to fund it, given sort of the resource limitations at times, and in order to make sure that those activities are supported. And then, I think, you know, there's lots of little things that I won't get into, but the contracting issues, you know, how do you get other sort of methodologists onboard to work with community members, and then once you have them and how do you keep it all together, all this expertise and the teams together and across different disciplines and different stakeholders, that's a real challenge.

CRYSTAL: Yeah. Thank you. I saw Kiara's hand, and then Andy.

KIARA: So, I think, you know, as people have been saying, this work, there's a lot of different methodologies that need to be developed or worked with. There's a lot of potential in different areas, but, you know, there are very few senior disparities researchers, and for early‑career folks, I think the collaboration is really critical, but our grants are often smaller. So, even, I think the number of early‑career people that I have in my network that are a great source of personal support and are reading grants and reading papers and things like that. We can't necessarily fund each other to be co‑investigators on grants, but there are circumstances where that might be extremely beneficial, I think in advancing the work and in bringing together some of this synergy scientifically. So, that seems, to me, to be a major challenge, and I think in speaking to the piece about, you know, there being very few senior researchers and then also the, the number of resources that are concentrated, you know, in particular states, particular urban areas, and certainly something, I'm in Boston at Mass General Hospital, I benefit from that immensely as an early career investigator, and that gives me an advantage that people in less resourced institutions and institutions without a lot of NIH funding don't have. And I think that's a huge area of concern where we are really locking out so many early career investigators from the pipeline, and I see so much interest, you know, from the undergraduate level up, you know, and people that contact us and want to get involved in the work and want to intern and go to medical school and do all of these things. But that the funding, you know, isn't necessarily there in place for them to then be able to get, you know, on an NRSA or a diversity supplement or a T32 to be able to make that trajectory, and so thinking about that allocation of resources in this work, I think is pretty critical.

CRYSTAL: Thank you. Andy, I saw your hand. Then Rheanna, and then Janet.

ANDREW: Yeah. I would endorse what I heard earlier. I think those are all challenges. My biggest concern, what keeps me up at night is sustainability for the work that's being done in the communities after the grant's expired and funding runs out. I, before I even submit a grant, I'm worried about that from the outside, because I feel like when I approach communities, if you don't want to be a helicopter researcher, you basically have to show them something that you're going to give them at the end that's going to help them in a real way. And I think that's always felt like the biggest challenge and responsibility that I have, is figuring out, because I'm not sure I see a lot in the literature about how to do that yet, so I would love to hear more conversations from people about that part.

CRYSTAL: Great. Thanks. Rheanna?

RHEANNA: Sorry, I forgot to un‑mute again. Yeah, I think, also, thinking about sustainability particularly when we think about groups that, who are excluded from, you know, sort of general medical care in general, that we really have to, you know, think even more about that. I was also just going to echo what Kiara said, that even in places that are relatively well‑resourced for NIH funding, they may not be well‑resourced in the sense of people who are doing similar work, so I think the networking is really critical.

CRYSTAL: Great. Janet?

JANET: I don't know that I have an answer, but I've heard some comments from, during some of the amazing presentations and during this conversation that made me think it may be a very valuable next step to consider, I'd say reaching out to those who, you know, participated in this particular workshop and, doing some kind of confidential survey about the barriers that we have experienced trying to navigate our careers, advance the work, you know. I've heard a couple of comments about, you know, Robert Wood Johnson or PCORI may be friendlier to the either methods that we're using or community engaged work, so I think there really may be something to learn there, challenges around senior mentorship. I'd be curious to know, you know, how many people have had experiences, you know, being discouraged in conversations with program officers about pursuing certain types of questions or certain types of methods. I really think there could be something very valuable to be learned that would be very informative for the next steps if we want to be bold and really be able to do some of the most innovative work. It's, you know, the presentations have been incredibly, incredibly inspiring, and I think the potential is there.

CRYSTAL: Great. Andrea?

ANDREA: Yeah. I would love to say something. My connection isn't great, so I will give it a try. I apologize in advance for the connection issues. I just wanted to add a quick comment about the fact that I would love to see more research building upon the work that's already been done around integrated care and patient‑centered medical homes and integrated behavioral health. I think there's been a lot of great work in that area, but that it's kind of just the tip of the iceberg about what can be done in terms of integration across systems just more generally and studying how do you build more kind of long‑term collaborations and integration. I think those kinds of IBH programs, you know, can only be so much, right, and so, I don't have an answer about how to study that, and I think, there are a lot of ways that could be done, but I think that needs to be taken to the next ten levels in terms of access to care research.

DENISE: Crystal, I have a question that someone sent me, if that's okay.

CRYSTAL: Sure.

DENISE: Okay. Alright. Kind of related to this conversation, are there experts from other disciplines we should be inviting or encouraging to join traditional mental health researchers, like social work, medicine, psychology, to really expand the work? Thinking about how, quantitative scientists have been encouraged to join the mental health workforce regarding computational psychiatry. So, for disparities research, who, which disciplines or what other kinds of, scientists or professionals could be good synergistic partners in moving the field?

BOWEN: Informatics. I think informatics, and I think, you know, there's lots of examples, like, you know, of attempts to do things where you collect sort of EHR claims data. I don't think that's the only way to do it, but sort of integrating data across systems. How to think about, you know, normalizing administrative data in healthcare and claims, and, you know, if you really want to get to disparities, it's just, I think it's really sort of important to really think about how to streamline across these administrative datasets at the state, local, and federal level in such a way, on the one hand, that preserves peoples' privacy, but on the other hand, allows for linkages across different systems. Because I don't think we do, that's, I think, the only way we're going to start getting at scale, some sense of how, you know, different services and exposure to different sectors may or may not influence things, like mental health, and I, and I think people in informatics, you know, computation and neuroscience is one way. I do worry a little bit about technology and NIH, especially, you know, just because I kind of feel like the technology moves so quickly, especially digital technologies, that once one has something funded five years, and then seven years writing a paper, it's going to be obsolete, so that's a really big challenge for the use of technology like digital technologies. What we come up with today will be irrelevant in five to six years because these companies putting so much money into it in a way that we can’t

CRYSTAL: I saw, Victoria, I saw your hand was up, and then Miraj, and then we do need to close. We are, so we can wrap‑up.

VICTORIA: Yeah, I just wanted to add, just a huge appreciation for this meeting. I think, I got to know work that I was not familiar with, people that, certainly, I'd look forward to citing and to reaching out to in the future, and I think, you know, certainly, having the opportunity to connect virtually when we don't have, you know, the spaces, and even when I think about the spaces that I might attend, you know, they tend to be even just so much more discipline‑specific or content‑specific or population‑specific. That it's been really great to hear from a breadth of different areas that intersect with the work that I do as well, so I just would like to echo that I loved having this opportunity to be a part of this conference and would love to see more, you know, meetings like this, and I think that also speaks to the fact, like, where I am, there is a very, very small health services research community, even though I think of that as being sort of one of my main areas of interest. And, being in San Diego, that there aren't that many people that do this kind of work. The next biggest place would be UCLA, where I trained, you know, and, so, I think it really does, it's a challenge for folks that have an interest in health services research, for students that are looking to pursue this area, having opportunities like this just are really helpful and I would like to add, you know, from the perspective of who else should we be thinking about adding was potentially geographers or people that have expertise in, you know, estimating area variations or looking at, how disparities and the conditions that we're interested in studying, manifest across different geographic regions. So, you know, kind of building in either statisticians or bio statisticians or geographies that have those areas of expertise, I think would be really valuable as well, but, again, my greatest appreciation to NIMH and NIMHD for putting this together.

CRYSTAL: Thank you, and last, but not least, Miraj.

MIRAJ: That's a lot of pressure, but, so, just on the question of who can we bring into the table, so, I have been pretty, intrigued by the fact that, often, some of the people who are often asking me to join their projects are environmental scientists. And I mean, if you just look at, say, what they've said, if we basically don't address climate change by 2030, there will be unimaginable human suffering. So, if we're looking at some of the major determinants in mental health moving forward, already, they're here, but in any case, what I've been blown away by often are the calls, RFAs coming out of NIEHS and the level of community engagement that's required and the level of environmental, focus on explicit justice issues. I couldn't believe it, really. And so, that's a good example of how that kind of cross‑fertilization can be really powerful, and even in that, they explicitly mention you have to have community partners, even as PIs potentially, and things of that nature. So, let's not forget that one of the major people we need to be bringing into the table are is the community as PIs.

CRYSTAL: Thank you. Thank you so much. Jennifer, Denise, I will turn it to you to close out our roundtable and make our final remarks.

JENNIFER: Okay, I can start. Thank you all for, again, hanging in there till the final hour of this workshop. We know some people have had to leave early, we can see in the chat, but, thank you so much for your participation. From the perspective of NIMHD, it's been great to hear and show off some of our grantees, hear about the NIMH grantees, and hope there's, opportunities for some cross‑fertilization between our institutes, as well as our grantees who are already getting funding from both. And it's just been great to hear, all the exciting research, all the research that is balancing scientific rigor and cultural and community rigor, so it's been, very educational, very exciting, and, I just want to thank you all again for participating.

CRYSTAL: Okay. Denise?

DENISE: I want to thank everyone as well. I have to say, my interest never waned throughout both days, and you can't say that about many meetings, I think. There were just so many interesting ideas and comments, even from people’s whose work I thought I knew pretty well. So, I appreciate the opportunity to learn more and think about, new directions, and everybody's participation was very valuable, so thank you for participating.

CRYSTAL: Yes, and I just echo my co‑chairs' sentiments, and I thank you all for your, fantastic participation. The presentations were amazing and informative, and the discussions were, just really, really well‑rounded and just really thoughtful and engaging. So, I look forward to following up, I look forward to continuing the discussion and continuing this really important work. So, we will be in touch. Thank you all so much. Thank you for hanging in there on this Tuesday afternoon, and, thank you. We will see you all very soon. Take care.