Mobile Technology for Global Mental Health Research

Transcript

>> WEBINAR OPERATOR: Hello and thank you for joining the National Institute of Mental Health Office for Research on Disparities and Global Mental Health 2018 webinar series. This presentation is entitled "Mobile Technology for Global Mental Health Research." Please note all lines are in listen only mode. If you would like to ask a question during today's presentation, you may do so at any time through the Q&A pod located in the right lower hand corner of your screen. This call is being recorded. It is my pleasure to turn the call over to Dr. Makeda Williams. Please go ahead.

>> MAKEDA WILLIAMS: Thank you so much and thank you for giving us the webinar logistics overview, Rachel.

Hello, my name is Dr. Makeda Williams from the National Institute of Mental Health, Office on Research for Disparities and Global Mental Health. Thank you for your participation in our 2018 webinar series. I'm pleased to welcome you to today's webinar titled "Mobile Technology for Global Mental Health Research." This is the third webinar in our 2018 webinar series sponsored by NIMH Office for Research on Disparities and Global Mental Health. Please note this webinar is recorded and will be posted on our website. As Rachel our operator mentioned, you will have an opportunity to submit questions for the speakers at any time during the webinar. The speakers will address your questions after all the presentations are completed.

With the rising ownership of cell phones and use of digital technology in low middle-income countries, there's a great potential for mobile technology to promote healthy behavioral change. Research on innovative, cost effective and scalable mobile technology to improve the accessibility effectiveness or delivery of mental health care in low to middle income countries is imperative to addressing the global mental health burden.

This webinar will highlight three NIMH supported research studies that leverage mobile technology to address mental health challenges such as depression and substance use in lower middle-income countries. The purpose of the webinar is to introduce or enhance knowledge on how mobile health technologies establish research networks, build capacity and improve mental health outcomes in low to middle income countries. Now it gives me great pleasure to introduce our three speakers for today. Our first speaker, Dr. Lisa Marsch is Director of the Dartmouth Center for Technology and Behavioral Health, designated Center of Excellence supported by the NIH.

In addition to directing this national center, Dr. Marsch is an international research and capacity building project funded by NIMH that leverages science based digital therapeutics for mental health and substance abuse disorders to expand access to science based behavioral care and primary care settings in Latin America. She served as consultant to the Department of Mental Health and Substance Abuse at the World Health Organization. She currently serves on the National Advisory Council for the National Institute of Drug Abuse at the NIH.

Dr. Paulo Menezes is professor of preventative medicine at the faculty of medicine and leader of research group on population, mental health at the University of São Paulo. With more than 20 years’ experience working in psychiatric epidemiology, his achievements in research include development of two large population-based research programs on first time psychosis in Brazil investigating the incidence risk factors and prognosis and aspects related to neuroimaging genetics and services. Dr. Menezes was principal investigator for the NIMH funded grant for the Latin America technology and innovation network on mental health, a five year program aimed at developing and evaluating effectiveness on technological interventions and apps for smart phones in treatment of depression symptoms in individuals with diabetes and hypertension in primary care.

Our final speaker is Dr. John Piette, a professor of global public health and internal medicine, director of the University of Michigan Center for Managing Chronic Diseases and the Department of Veteran Affairs senior research career scientists. His research focuses on developing and evaluating novel strategies for using patient-facing health technology to improve accessibility and quality of care for patients with chronic illnesses. Much of the work focuses on the use of health monitoring systems in socio-economic vulnerable populations in Latin America. Dr. Piette has been the principal investigator on multiple NIH, VA and Agency for Healthcare Research and Quality funded trials of disease services supported by mobile communication tools for patients and formal caregivers.

For more information on our speakers, please visit our 2018 webinar series website. I'm delighted to welcome all our speakers today to talk about leveraging mobile technology in global mental health research. I will now turn it over to Dr. Lisa Marsch.

>> LISA MARSCH: Great. Thank you. Can you hear me?

>> MAKEDA WILLIAMS: Yes, we can.

>> LISA MARSCH: Terrific. Well, thank you so much. I particularly would like to thank the Dr. Williams for organizing the session and including me in the session and it's a privilege to be able to participate in this. I'm going to start us off with a brief introduction to what is digital health and what are the opportunities for applying mobile technologies to global mental health research and then I'm going to move into talking about a very specific project in which we are engaged funded by the National Institute of Mental Health, focused on scaling up science based mental health care by centrally using digital technology as part of the care model.

So first I would like to acknowledge our funding for the work I'll be speaking about today, including our grant, our cooperative agreement grant from the U.S. National Institute of Mental Health, and also our center grant for the Center for Technology and Behavioral Health. In addition to my academic affiliation at Dartmouth College in the U.S. I have affiliation with a couple of small businesses that are involved in deploying various technology based digital therapeutic tools for behavioral health, for mental health and substance use, our research center has several industry partners.

So, when you think about digital technology, the theme of this webinar here today, this has fundamentally changed so much of our society, when you think about cloud computing and mobile technology and data analytics. The explosion of digital technologies has changed how we do finance, how we buy things, how we engage in educational activities, how we have social communications with people all over the world. And today we're talking about how digital technology has the potential to transform how we think about delivery tools, resources, science based resources in the health space. And there's a tremendous opportunity, I hope you'll see by the end of this webinar to increase the quality and reach of care by harnessing digital technologies to enable new ways, additional ways to deliver science based health resources to people all over the world.

There are tremendous advances that are happening in digital technologies that allow us to better understand and provide personalized interventions for health behavior, including as we're speaking about today mental health at a population level. These digital interventions are often called digital therapeutics. So in the case of applying digital therapeutics to mental health, we have this great opportunity to extend the reach and the impact of our mental health clinician workforce. So it's like giving clinicians additional tools in their toolbox that they can offer to their patients to give them 24 hours a day, seven days a week access to personalized resources that they can use as they move through their lives, that can extend the reach and the impact of the mental health clinician workforce.

So it's kind of like having a virtual clinician in your pocket. It's kind of like having something on demand accessible to you that can help you in navigating mental health challenges you might be experiencing.

So the opportunity here is to have these scalable science based, anytime, anywhere access to behavioral health interventions and mental health interventions.

All over the world pretty much any population you can think of either has access to digital technology or is getting access to it. This is true in some of the most traditionally underserved and vulnerable populations. This is true in many low/middle income countries, and the statistics are very compelling that mobile technology is becoming ubiquitous in virtually all communities across the globe. And if you haven't heard the statistic, it's pretty startling. There are actually more mobile phone subscriptions in the world than there are people in the world. So it just underscores the tremendous opportunity to leverage the widespread availability of mobile devices to provide personalized science based mental health resources.

So, this space is increasingly being called digital health, and digital health is a broad space that refers to using mobile technology to not only better understand people's health needs but also to provide personalized healthcare resources in unprecedented ways.

There is a lot of exciting research in this space. This slide just summarizes sort of a couple of decades of research in this space of digital therapeutics. In general what we found is if these digital therapeutics are developed well, there's a lot to say about development, and developed in accordance with science based principles, we have seen that we can get high acceptability of these interventions across lots of different populations. We have seen that we can get a very robust impact on all kinds of different health behaviors and health outcomes, robust and replicable impact on that. We've seen in some studies that these tools can produce clinical outcomes that are comparable to outcomes that are delivered by clinicians delivering behavioral healthcare. And this is important because it underscores that there can be active ingredients in the tools that can have a big impact on people's behavior and health outcomes and that can extend the reach and the impact of our clinician workforce.

We've seen that we can increase the quality, reach and personalization of care with these tools. There is growing evidence of cost effectiveness of using these tools in different sectors. And what is exciting, I think, about these tools is that they don't have to be static. They don't have to work the same way every time, but they can be flexibly adaptive and can be responsive to people changing mental health needs and changing clinical trajectories over time.

So I wanted to include a slide here about our center. We have a lot of resources at the center for folks who may be interested in the space. I put the website there for our Center for Technology and Behavioral Health which is an interdisciplinary center that focuses on using science to best understand how to develop, evaluate and create sustainable models of implementation of digital therapeutics for mental health and substance use disorders.

So our work spans many different populations and many different contexts. These are examples. We've seen, for example, that behavioral therapies delivered in an interactive way to an individual through a mobile health platform can markedly reduce drug use among people suffering with substance use disorders. So we've seen we can roughly double abstinence rates from drugs when these mobile interventions are part of an addiction care model versus when they're not. We've seen we can markedly increase people's retention in addiction treatment with these tools.

Another example, chronic pain patients, we've seen that by offering an interactive mobile intervention that can help people better manage chronic pain and reduce the negative impact of chronic pain on their lives, that we can reduce opioid misuse among chronic pain patients. We even seen we can reduce emergency department visits among chronic pain patients. So this has tremendous implications to cost savings if we can reduce unnecessary utilization of costly emergency department services by providing something that people can use in the privacy of their own homes or as they move through their daily lives to help them better manage their health conditions.

And we've also seen that we can entirely change healthcare service delivery models by offloading some of the service delivery model, like for mental health, to a digital platform. And that's what I'm going to give you an example about today.

So I would like to tell you a bit about a project that we are conducting in Latin America. We call it the DIADA project and it's funded by the National Institute of Mental Health, and consistent with the theme of the webinar today, you'll hear about three projects that are being conducted in Latin America. But I want to underscore that this is just an example of one region of the world where we can really realize the power of mobile technology as applied to mental health, and there are tremendous opportunities all over the globe in low and middle income countries to similarly scale up access to science based mental health resources using digital technology.

So in Latin America, we see that the burden of mental health prognosis is high and we also know that service capacity, the mental health workforce is limited, particularly in non urban context. And in the regions that we're working, depression and substance use are particularly striking concerns.

So, we are seeking through this project, which is basically a multi national implementation research initiative and capacity building initiative to expand access to science based mental health healthcare in primary care systems across Latin America, starting in the country of Colombia and also expanding in partnership with colleagues in Peru and Chile. And it this map reflects the six regions of the country of Colombia when starting this project.

This project reflects a collaboration with a very diverse group of stakeholders, and this is really key, I think, to the chances of success and sustainability of this project.

So it reflects a partnership between colleagues at Dartmouth College in the U.S., Javeriana University based out of Bogotá, Colombia. The U.S. National Institute of Mental Health as well as a diverse array of healthcare systems, governmental organizations, patient organizations, insurance company payors in the region, as well as a number of non governmental and multi lateral organizations. And in this project, just briefly, the goal is to work with primary care systems to train the workforce in that context to screen for mental health and substance use problems. And then provide science based resources in response to needs for mental health and substance use in that context. And we're particularly focusing on depression and problematic alcohol use, which are striking concerns in many of the regions in which we're working.

So digital technology is a central part of this in several ways. First of all, we have an automated patient screening process in waiting rooms. So patients coming to primary care go to a kiosk in the waiting room and answer screening questions, so we can get a sense they may have a risk around depression and/or alcohol problematic alcohol use. And then for those who screen positive, the providers are prompted to do a more thorough assessment of those individuals and they have an automated clinical decision support system that helps the providers in offering science based care, so it helps them in understanding if someone is diagnosable with these problems. It helps them provide motivational enhancement to help people, encourage people, to consider making changes in some of these problems. It gives them recommendations about science based resources that they can refer patients to use. And also we the providers also are prompted to routinely follow up on the clinical status of these individuals.

And then the third way in which we're using digital technology in this project is by using a science based digital therapeutic platform offered directly to patients. This is a direct to patient interactive self directed intervention based on science based approaches to helping people manage depression and/or problematic alcohol use. So this program is called Laddr®, and this is a program that we developed based on about two decades of research that we've had the opportunity to be part of funded by the National Institutes of Health, and as we've had the opportunity to work with lots of diverse populations and lots of different contexts, we've seen we have some of the fundamental principles of the science of behavior change and how can we embed functionality and digital technology tools that can help people change self defeating patterns of behavior, like problematic drinking behavior, for example.

So, we created a platform that can flexibly provide these resources to promote self regulation, which is a real key mechanism of behavior change in lots of different populations. Within the context of a mobile platform. So briefly here are some of the ingredients in the tool. This is all delivered, of course, in Spanish. We worked with folks in Colombia to put this in Colombian Spanish and make it culturally relevant to region, but it includes problem solving therapy for depression or helping people understand self defeating patterns of triggers, for example, and patterns of risk factors that might impact problematic drinking and how can people learn new skills and develop new behavioral repertoire to respond differently to triggers or avoid triggers in new ways. How can we help people become the person they want to be, thinking about their future self and working toward long term goals. How can we help people challenge negative cognitions that may be maladaptive or harmful? And what additional strategies and skills can we help people learn that can be helpful in managing mental health challenges?

And so in pairing for this work, before we implemented this model of care, we did a lot of preparatory work. I'll just briefly mention it. We did a national survey across Colombia to understand do people have access to technology, and if so, how are they using it, and would they be interested in using it to access mental health resources?

And so we collected survey data from just under 1600 people in the country of Colombia. Actually, 35% are the sample reported depressive symptoms in the last 30 days. And about 93% had cell phones but only about 56% had smart phones. So we saw wide variation, perhaps not surprisingly, who had smart phones, and although cell phone penetration was quite high. We also did a lot of formative development work, qualitative research to get a sense of people's interest in mental health resources and what their needs are, both from patient perspective as well as healthcare professional perspective.

We started with a pilot study with 50 patients in a single site in Javesalud in Bogotá, Colombia, and the learning from that pilot then informed our protocol for expanding the implementation of this intervention model I described to sites across the country, which is what we're launching now in a staggered way in urban and rural communities, and our plan is to test this with about 2,000 patients, but we're not just looking at we're not just looking at patient outcomes, patient clinical outcomes, we're looking at a lot of implementation outcomes, factors that might impact adoption and sustainability of this in terms of workflow, in terms of provider capacity. It's system level outcomes as well as cost outcomes and qualitative outcomes to understand barriers and facilitators to implementing this.

So, we're additionally focusing heavily on building sustainable capacity. So the goal of this along the way is through the learning of this project, to understand what is working and what is not, to iterate and refine that over time so that the region can take this on and continue this in a way that is most impactful for the region after the life of the research grant funding.

So, in summary, this project illustrates it's just one example of how you can harness science based digital health approaches to create innovative new models of health delivery. This can be applied in many regions of the world, but this is a way in context where the workforce is at a population that you can sort of super charge the workforce by giving them these digital technology tools to extend the reach and impact they can have in the region.

So overall, we've been really excited and encouraged about the considerable interest we've seen among a very diverse group of stakeholders. We have had some challenges in embedding mental health screening into the workflow of primary care and ensuring sufficient Internet connectivity in all the regions, but these challenges are things we can seem to work on and there are lots of strategies that we've been testing successfully and trying to take on these challenges, including offloading a lot of the process as much as possible to a technology platform.

So, overall, I would just like to underscore the opportunities here just to scale up science based mental health care at a population level, and we can say at a global level, by harnessing ubiquity of mobile technology.

Thank you so much.

>> MAKEDA WILLIAMS: Thank you so much, Dr. Marsch. And now we'll have our next speaker. Our next speaker is Dr. John Piette.

>> JOHN PIETTE: Good morning, everyone! Can you hear me?

>> MAKEDA WILLIAMS: Yes, we can hear you.

>> JOHN PIETTE: Okay, great. So I'm going to describe the current status and kind of the thinking behind a project that we're currently developing in Colombia. It's a mobile health solution or kind of a combination of mobile health and community health worker solution to improve access to depression care in that country. I'm going to describe the project. The project was funded as an R21 by NIMH and it really began with developmental work in Bolivia, and I'm going to end today with commenting more on some of the advances that I've seen over the last five years in the cloud based solutions to mobile health that I think are really a game changer in terms of making these sorts of solutions much more scalable and much more realistic, frankly, for researchers around the world and for people in communities around the world, so that they can actually access in a cost effective way the resources they need to make these solutions a reality for patients on the ground.

So, as I mentioned, the core of this project was and is an R21 grant to develop and do pilot testing around a mobile health solution for depression care. The first aim being much more developmental and the second aim being to conduct a pilot randomized control trial, and we're just beginning that process right now. The intervention per se is called AniMovil, and it's a word, for those that speak Spanish, Movil is in many countries, a term they use for like "mobile phone." And AniMovil doesn't have direct translation, but it means, you can do it, you're going to make it, etc. So it's really kind of a perfect way to kind of capture the essence of this project. So this is AniMovil, you're important to us, kind of increasing the importance of social engagement, one of the elements we think is important in depression management.

As I mentioned, the study started with a very diverse group of collaborators that included groups in Bolivia, the University of the Andes in Bolivia, and the University in La Pots, Bolivia. We also have involved the International Medical Corps, some of you may recognize the name. She was really a ground breaking work on cognitive behavioral therapy for groups in depression management in low and middle income countries primarily with important trials in Pakistan and other countries in that region of the world. We're working with Adrian Aguilara, University of California Berkeley and developing a text messaging program that he generously shared with us. And also, with Ricardo Munoz who is very well known for his development of CBT models with Hispanic populations with depression here in the U.S. So, it's a very broad and diverse group of people that has allowed us to really jump start some of the content development in a way that would never have been possible if we tried to do it ourselves.

First let me tell you here that, yeah, TCC, that's really Spanish for cognitive behavioral therapy or CBT. So the idea that randomized trials that we're just getting underway now, is that patients with significant depressive symptoms are being randomized to the intervention, which I'll describe in more detail in a minute, or the control group, and really at the recommendation of reviewers of R21, and I really supported this recommendation. There is not a usual care control group the control group is receiving daily text messages where they're going to be reporting their mood scores randomly throughout the day. 0 10 scale. And there is some evidence that random mood scores, especially if you have several of them, can be a really good sense of how someone is with their depression, with their mood. Just about at the same level validity you get with a PHQ. So, we will have this high frequency measure of mood over the course of patient's 12 weeks of participation that we can use as another outcome in addition to measuring the PHQ scores at the end of the trial. The idea is to identify patients with very, very broad eligibility criteria that have PHQ scores over 10 and we're now doing that in one of the main hospitals in one of the more low income areas of Colombia, as I said.

The overall structure of the intervention is this: It's really multi faceted and multi directional. We've done work with IVR response calls in a variety of countries around the world principally in Latin America. We worked in Honduras and Mexico and Bolivia and Colombia and we worked in Thailand. And we like IVR calls because on one hand you can get pretty detailed clinical data from patients using their touch tone phone. You can do a PHQ readily be an IVR call and patients don't have to have a Smart Phone, which is a huge benefit. In some of our work here domestically I'm a VA investigator as said at the beginning of the presentation. We have shown that cognitive behavioral therapy delivered to people with chronic pain via IVR calls actually has outcomes as good as you can get for people who are seeing face to face visits with a trained CBT therapist. So the core of this is weekly IVR calls to monitor patient's depressive symptoms, give them and clinicians feedback on progress, monitor for suicidality and present a series of skills, CBT skills around self care, behavioral activation, dealing with interpersonal problems, life stresses, cognitions, etc. In addition to the weekly IVR calls intervention patients will also receive daily mood monitoring by text messages, and they'll receive some messages reinforcing some of the information they're getting through IVR with the skill practice. We've gone to great lengths to develop manuals that are patient centered, that, you know, really deal with the various aspects of CBT skill practice. We have road tested these with patients in Bolivia and Colombia and, of course, have translated them into a language and vernacular that is acceptable to patients with limited education. Huh oh...sorry.

I hit a space bar instead of a forward. So can someone help me get back to my...

Thank you very much.

There is a community health worker, and we feel strongly that linking technology with real live support is actually crucial. I mean, with the work we've done in a variety of countries, including the U.S. and Bolivia and elsewhere, when you have patients that are using mobile health technology and they participate with someone that they understand, trust, care about, they know it's kind of following along with them. Their adherence to these technologies is far better. So there is a community health worker involved in this intervention. For example, we found in a lot of our...

[ no audio ]

>> MAKEDA WILLIAMS: Dr. Piette, can you hear us?

[ no one is speaking ]

>> MAKEDA WILLIAMS: Thank you all for your patience. We're trying to reconnect with Dr. Piette at this time, if you could just please give us a couple moments. Thank you.

>> JOHN PIETTE: Are you able to hear me?

>> MAKEDA WILLIAMS: Yes, we can hear you again. Thank you so much.

>> JOHN PIETTE: Okay, I'm not sure what happened. Sorry, seems like I got kicked out.

Okay, I'll just continue.

So there's a community health worker because we've seen in our work in Bolivia and Honduras and other low and middle income countries that having a linkage between technology and live support is really important. Both the patients in the community health workers have details, manuals with information kind of about the content as well as what they can expect from the intervention process.

This is a stepped care model where we wanted to rely heavily on the mobile health tool on the automation and make sure that everyone had as much live support as they actually needed. So step one, if people have a mild to moderate depression, say PHQ scores less than 15, they will get weekly IVR monitoring and self management assistance with the manual, if the scores are above 15 they have a minimum of three weeks of live follow up with the community health worker until those symptoms remit, in which they can go back to step one.

Step three is simply the suicide protocol and we have that in place for patients that express suicidal ideation. So some patients will be getting the trained CBT from the therapist and a lot of patients, we hope, will be able to be managed with the resources that are available from the mobile health tool.

Functionally, briefly, given the time, we put a lot of thought into the CBT weekly communication and self management education. It really follows along to the evidence based CBT programs we used for chronic pain in the U.S. Patients get their PHQ scores monitored once a week and get feedback about trends in the PHQ scores. We check in about CBT skill practice, and the people that are participating have some ability to fast forward or to go more slowly and deeply through the five CBT skills. And these are the standard types of skills for behavioral activation until self care and dealing with interpersonal problems, etc., that are presented for CBT. As I said, Ricardo Munoz, a psychologist from Palo Alto, particularly useful, him and Chaudry in developing that project.

To make sure there's a tight linkage with a live support from the health worker. Community health workers will be monitoring patients progress through a dashboard and be able to leave a recorded message that is automatically attached to the IVR message, and patients will be able to leave a recorded message for their therapist. So even though it's asynchronous, they always feel they have that kind of live support.

We have in both Bolivia and in Colombia used a very community based development process. So we've learned certain things about not using the term "depression," not talking about the diagnosis but rather the symptoms and using terms that are much more socially acceptable like [non English word or phrase]...I'm feeling sad or I don't have any energy. We give very, very concrete examples of the types of self talk people can use in cognitive behavioral approaches used to help people change dysfunctional thinking, and every time we're preventing presenting kind of behavioral suggestions, we're always trying to link those back to patient's mood. So they can make connections between, for example, getting back involved in their daily life, going to church, going to the market, reconnecting with family and friends, and how that is going to affect their mood.

We have done pilot testing that we have been very pleased with. Pilot testing in these projects usually focuses on two aspects. First, can we actually get patients to engage to the number of IVR calls that they complete? And by and large we find we do very well. Completion rates are high in the U.S. where people often complete 90% of their IVR calls. Here in Colombia we found it's about 70%. But adherence to the intervention and factors that are associated with adherence is one aspect of piloting. We also look at the validity of the information that patients are reporting via IVR, via the automated call, and is it the type of information that you would get during a live interview? What you see here, if you focus on the left side, is we're trying to compare what patients told us about their perceived health status when we enrolled them, and they were talking to a native Spanish speaker, and what they said the first time they were called in their IVR call. So, you see that, among people that told us in the live interview that they felt their health was excellent or very good, 67% of those people said in the first IVR call, yeah, my health is excellent or very good, compared to less than half of those who said in the baseline interview that their health was very poor.

Similarly, on the right side, on patients in the live interview with a native speaker had mild or moderate depressive symptoms, 73% of those patients reported mild to moderate depressive symptoms via IVR compared to fewer who reported at baseline their depressive symptoms were severe.

So there is a sense that the patient can report valid and reliable information via IVR, and that is similar to what we found in other studies.

Reading what the graffiti says "USAID, get out of Bolivia," and they were asked to leave in the last few years. Our relationships between our government and the Bolivian government are frankly pretty awful, and our project was sorts of a casualty of that. We thought the project was really dead in the water, you know, that we weren't going to be able to continue. But because of some very creative administrative work and great support by NIMH, particularly Beverly Pringle, our program officer, but everyone at NIMH, we were able to pause the project, regroup and move the project to Colombia, and it turns out that Colombia is a really ideal place for this project to continue. Colombia has enormous mental health problems because of the 50 year civil war that you know happened in that country. It's one of the most mined countries landmines, more than almost anywhere in the world. So being able to continue there is really ideal. We also had the ideal situation that there is a physician here who is our site PI. Diego, he's a physician and professor at the University of the Andes in Bogotá, and he actually got his Ph.D. from our very Department of Health behavior here in the public health school at the University of Michigan. So that was really, really just a perfect thing and part as capacity building, Diego invited me down recently to do training on mobile health as part of a summer series at the University of the Andes. So that has turned out to be a wonderful relationship.

You know, I'm going to go through I'm going to jump ahead because I see we're almost out of time, but my point is I wanted to make this one last important point. We've tried scaling and doing these mobile health systems in a variety of ways. We put IVR and SMS programs on laptops. We've transferred them to university platforms within various universities. We've used small things called raspberry pies. All of these have pluses and minuses, but frankly in terms of scaling, more minuses than pluses. And until very recently, if you wanted to do this sort of work, you had really two choices. You could take off the rack tools like you can find for depression monitoring in the app store, which can be wonderful for people, but it doesn't give you any flexibility in terms of translation or modifying or really tailoring it to the needs of individual patients or healthcare systems.

You could do that or you could, you know, grow your own. And we've done a lot of growing our own, and those are with basic low-level languages. Universities are good at growing things and then making them scalable and disseminatable. We learned an awful lot in the last ten years from doing that, but it has its limits. In the last five years, these options in the middle, particularly things that are available through Twilio have made this sort of development work collaboratively between the university group and a company that has international reach. These developments have become much more possible, much more cost effective. So some of the work that we're doing now in Colombia involves work with a platform that is supported by twilio. In particular with Dr. Luca Mi and the rest of our colleagues, we're working in El Chaco, a Pacific coast of Colombia. If it was its own country it would be Haiti. Extremely poor and isolated, and the population is a mix of African descendent people and indigenous populations, often in communities you can only reach by boat. Our community health worker for NIH funded trial is from El Chaco, and when we describe what we were doing there, she cried. Because this was the first time that she could see that that sort of support in the community could be made available. So really echoing what the last speaker said, that the potential for these interventions is great in general and I think it's now particularly great with the advent of some of these new cloud based tools, which we could talk about if we had more time, that allow scaling and kind of really to expand beyond an individual university.

So let me just say thank you again to NIMH on behalf of myself, my team here at the University of Michigan, and all of our collaborators in Bolivia and Colombia. Without the flexibility and administration and creativity NIMH had with this project it couldn't have gone forward and I really do believe that we have made really groundbreaking progress thanks to that not only that financial support but that flexibility. So thank you again.

>> MAKEDA WILLIAMS: Thank you so much, Dr. Piette. Our final speaker is Dr. Paulo Menezes and I would like to remind everyone, if you do have any questions, please feel free to enter those questions in the Q&A box.

Dr. Menezes.

>> PAULO MENEZES: Thank you very much. I would like to start by thanking Makeda Williams and Beverly Pringle for inviting me to take part in this webinar and talk about the work that we’ve been developing in Latin America resulting in testing a technological intervention for depression among people with chronic disease in two countries, Brazil and Peru. Our work has been funded by NIMH grant U19 and we are reaching now the end of the fifth year of this project, so what I'm going to do is to talk a little bit about the background to the project and then show preliminary key results from our work. Just briefly, we are all working, as the two previous speakers, we're also working in Latin America, subcontinental area constituted by 20 countries. We have a population bigger than 600 million people in total.

Two demographic characteristics are very important for understanding our work. The first is the population region in Latin America. We can see here the projections for the population from 1995 to 2025. And the other thing is the migration process that has been taking place in the last 60 70 years in Latin America, people moving from rural to rural areas to large urban centers. So this is showing an example of working in remote areas. I'm going to talk about the population of the very large urban areas in Latin America. We have been working in Lima, in Peru, and São Paulo in Brazil, and these pictures show how large these urban centers are, and also the social disparities and social inequalities that are present in these large urban centers.

Mobility is a major issue in the centers, and this is important, because there is not only enough to have healthcare centers to deliver care that people need to reach the health centers in order to receive care, and that many times is problematic.

Here are some data about depression in Brazil, which is also applicable probably to all the Latin America countries. A recent national survey estimated that the prevalence of depression is around 8% and it's higher among women, people living in urban areas, and those with lower education and those with chronic disease such as diabetes and hypertension.

And very importantly, only one fifth of these people with depressive symptoms were receiving some kind of care for their depressive symptoms. Women, those who are white, those with better educational status, and those living in the south or southeast were more likely to receive some type of care for depressive symptoms.

And the other thing about...to receive care for depressive symptoms, is the uneven distribution of mental health professionals. A recent study about the distribution of psychiatrists in Brazil show that there is a huge concentration of psychiatrists, as we can see here, in the southeast area of the country as compared to the north and northeastern areas. And not even by a geographical inequity distribution, but also in the richest areas, working in the private sector, not in the public sector, so that most people, about 75% of the population, will use the public health care system do not have access to care delivered by specialist professionals.

And as has been said before, mobile coverage has grown really fast in all these countries, including Brazil and Peru. So there is an opportunity to use mobile technology to use the equipment gap in this population. So, the Latin-MH research objective in these five years were to develop and to test the effectiveness of an intervention for depressive symptoms, for people with diabetes and hypertension. The intervention is delivered through an application for a Smart Phone with support of assistant nurses from the health system.

We went through two phases. The first one was a formative research phase where we developed and tested in a pilot study the intervention, which is based in behavior activation delivered through this app for smart phones, and we call it CONEMO.

And then it was followed by a second phase, effectiveness research, where we conducted assessed the effectiveness of CONEMO intervention compared to usual care in Lima and in São Paulo.

The CONEMO Assistant is a platform that integrates the app that is in the smart phones delivering the intervention, the behavior activation intervention, which is connected to a server that allows nurses to monitor their patients through the app for intervention, and also supervisors to monitor the work of these nurses that are based in the health system.

So the technology is designed to provide people with depression with a way to manage their moods, provide healthcare providers with the ability to monitor patients, provide supervisors with the information that they need to manage the healthcare providers and provide researchers with ability to manage the trial and also working within the healthcare system, as I'm going to talk about quite soon.

This is some screenshots of the CONEMO app sessions. The CONEMO app can deliver text, videos, questions, and answers.

The Nurse Dashboard displays participants assigned to the nurse and it indicates to the nurse when a task is overdue to the patient, so that a nurse can then contact the patient and ask whether there is a problem that he or she can help the patient with in order to improve adherence to the intervention.

This is an example of the Nurse Dashboard. And there is also a Supervisor Dashboard, which is similar to the Nurse Dashboard. I'm not going to go through in details. I'm only going to show the results of the pilot study that we did in Sao Paulo with 51 patients. In dark blue we can see the 51 at the beginning of the intervention for the patients. About half of them had moderate depressive symptoms, and then in the light blue we see how they were six weeks after starting intervention. More than half of them had either no depressive symptoms or only mild depressive symptoms. So that suggested really a good potential for the intervention to be effective.

So we tested the intervention in two large trials. In São Paulo we ran a cluster randomized trial with 20 family health clinics involving about 100 family health teams. And we enrolled 880 participants with hypertension or diabetes and symptoms of depression.

The intervention in São Paulo was delivered and monitored by assistants working in these family health clinics. In Lima we ran an individually randomized trial with seven primary and secondary care services. We enrolled 432 participants, Brazilian participants, and the intervention was delivered and monitored by nurses hired by the research project. Because in Lima, in the pilots, we realized that the Nurse Assistant would not be able to deliver the intervention. They were just too busy and it was too much overload for them.

So, this is a this is the diagram for inclusion and outcome for Lima. Unfortunately, there is more than realized now. But we had very good follow up rate. We had about 95% follow up rate three months after inclusion.

And here is the diagram for São Paulo, where also we had very good follow up rate with 90% completion at three months, which is our primary outcome. This table shows how...[ coughing ]...excuse me. This table shows how randomization worked, and the intervention groups and cultural groups in São Paulo and Lima are very balanced regarding key characteristics. And it also shows that in São Paulo, patients had more severe symptoms of depression than patients in Lima.

And these are the main primary outcome results. We would see that at three months, in the intervention group in São Paulo, 40% had improved, at least 50% had 50% reduction in PHQ scores as compared to 30% of the patients in the controlled group. And in Lima we had 50% of participants in intervention with positive outcome as compared to 30% of the patients in the control group.

The difference, we assess patients at six months, but in the direction that the control group continues to improve to almost reach the patients in the intervention group.

So to conclude this presentation, these were the first large trials testing mobile technological intervention for depression in low middle income countries. We were able to reach a very high methodological quality. The intervention seems to be effective in the short term. And there are differences, important differences between the sites regarding the target populations. In São Paulo we worked at the primary care level, and in Lima we worked more at the kind of secondary care level. And the mode of delivery was also a difference in the two settings. In São Paulo it was more pragmatic trial than in Lima. Therefore we believe that the real effect, if this intervention is scaled, will be probably something between the results that we got for São Paulo and for Lima.

So, I will conclude here thanking my partners in this project, Professor Ricardo Araya from Kings College London who is Co-PI, and Jaime Miranda and David Mohr who are Co-investigators, all the Latin America team and NIMH team that has been very supportive, and I'll stop here. Thank you very much for your attention.

>> MAKEDA WILLIAMS: Thank you so much, Dr. Menezes.

And at this time, if anyone has any questions, please enter your questions in the Q&A box.

So it looks like we have one question. This question says: "Does the system work offline and later sync to the server?"

So I open that up to either Dr. Marsch, Dr. Piette or Dr. Menezes to address that question.

>> PAULO MENEZES: Sorry, Lisa. This is Paulo.

In the CONEMO system, yes, we program the app so that it can work offline, and then at any point when the mobile has connection, it transfers the information, the data, to the server, yes.

>> LISA MARSCH: I was going to say the same thing, we do the same thing. It's really important because you don't have constant connectivity in a lot of regions, but you don't want the app to be nonfunctional, so it's really important to be able to still utilize it even when you're offline and then you can sync data and do upgrades and such when you have connection.

>> JOHN PIETTE: It roughly works similarly. Since we're using the cloud platform for some things, it's less of an issue because you're going through the telecommunication system instead of necessarily through you know, data lines. But, yeah, they have to be able to have periodic ability to sync and upload or exchange data.

>> MAKEDA WILLIAMS: Great. Thank you. Did anyone else have any questions? If so, please feel free to enter those questions in the Q&A box.

Well, I have a question. One of my questions is about the sustainability of the mobile technologies after you've completed your research. What are some of the challenges and how have you addressed sustainability?

And this is for all…

>> LISA MARSCH: I can start if you like.

>> MAKEDA WILLIAMS: Thank you.

>> LISA MARSCH: Hi. Thank you for the question. You know, I think one of the key factors with sustainability is having the right stakeholders and the active dialogue around the topic from the very beginning. So, you know, from the very initial, you know, sort of, you know, planning of a project and conceptualizing what the project could be and trying to understand what might have utility and what might be implementable, in our experience, it seems incredibly important to have a very diverse group of stakeholders at the table that are really key to potential sustainability, and having them involved in the conceptualization of the project, the launch of it, refinement of it, sort of the iteration and evolution of it over time, and that's not only the healthcare systems, you know, and all the relevant stakeholders there, from the patients to clinicians and administrators, but also in administrative health in the region and the insurance company in the region and all the parties that make decisions around what they're going in their region, and have them involved the whole way through and evaluating what we're doing and sharing that and having that perspective on it over time I think is really key.

I think the learning from the project is going to be really...we haven't sustained it yet. We're still launching, but I think the learning from the project about how do you implement this in these systems of care and how you have an impact, I think that's going to be the data are also going to be compelling as we understand it to be stakeholders. So what is the impact on increasing capacity to tackle the mental health challenges in the region, which as you heard on the call today are really large. The need is considerable. But the resources are not sufficient to meet the need. So I think that's one of the compelling aspects around this.

But I do think, as John was saying, too, it's important to have a mechanism to continue to support these tools beyond the life of a grant and to have, you know, a team committed to deployment, you know, after the research has ended in partnership with these stakeholders.

>> JOHN PIETTE: You know, I...I'll say something slightly counter cultural. Sustainability, of course, is something we all think about a great deal, but the challenges are extremely formidable. We're not only might not there be interest in sustaining these programs, but we're actually working against the economic interest in some instances to, you know, fee for service systems that really make money from having people come in being sick and come back for a follow-up a visit. Those are very real challenges.

But I think any right thinking system, whether it's an insurance program, whether it's a healthcare system, whether it's individuals, the first thing they're going to ask is: Does this work? And so I don't think it's a bad thing to have each of us be developing science and the evidence base without having a clear answer to the sustainability issue. And the things we develop, if we're publishing and doing rigorous science like we're supposed to do, you know, it might not be sustained in the environment that we're doing the work, but it might very well be sustained in some other part of the world.

So I think the science stands on its own and is vitally important and it's going to be, as Lisa just said, it's going to be an important element of sustainability even if we don't have, you know, kind of a sustainability plan from the get go, which, of course, we all hope we do.

>> PAULO MENEZES: Yes, I agree with what has been said by Lisa and John. And also, I would like to add that in our trials, it was really very important to have all the supports by the directors and key personnel from the provider organizations with which we worked with, because that guaranteed that we started the project. As soon as it started, then the things started to have less resistance to take part in this, and some became very enthusiastic about the intervention in their routines. So I think this is one key point. The other thing that I think is worth mentioning is that we, in our study, we decided to borrow smart phones for patients, because we knew not all of them would have the equipment available. And it was a good surprise to see that patients really understood it as something that they were getting from the health system. So they would take the smart phones with them for six weeks, during the duration of the intervention, and after they finished it, they would hand it back to the nurses because they knew someone else would need to use it.

So that was another thing that was really very interesting and important for us in terms of sustainability. It's possible to think about having the equipment as part of the resources from the healthcare system that patients can use and take back to the system.

>> MAKEDA WILLIAMS: Thank you so much for that. The next question is for Dr. Menezes. You were targeting depression in persons with hypertension and diabetes, did you see improved management of the chronic medical conditions where depression improved?

>> PAULO MENEZES: We're still starting the analysis from the outcome data we got from the trials. We did measure the treatment for the chronic condition, particularly whether patients how the patients are taking their prescribed medication. We were not able to have biological data from patients such as fasting glucose or direct measures for blood pressure at the outcome. But we do have indirect measures for the outcome, and we hope that we also find some relationship between the improvement of the depressive symptoms and improvement of the chronic condition.

>> MAKEDA WILLIAMS: Thank you. Our next question is for all speakers. What about the issue of reliable Internet connection? What has worked in sites with poor connectivity?

>> JOHN PIETTE: I think we've addressed that somewhat. If it's something that is based in a Smart Phone, you, of course, will need to think more about connectivity, but often there are creative ways to have things done like on people's local device and then sync periodically when connectivity becomes available. So that's not necessarily a deal breaker. And I will say that by and large in most areas, you can get to I almost want to say 90% of the world with Internet connectivity. That's become less of an issue. It's still an important one, but it's becoming less of one. When you have somebody that is a combination of Internet and going through telephone networks, through cell phone networks, there again I think you can kind of get around some of the intermittent connectivity issues. So this is an issue, and it's something we all have to think about, but, you know, I think it's certainly something that is addressable and in the coming years it's going to be less and less of a problem.

>> MAKEDA WILLIAMS: Dr. Marsch or Menezes, did you want to respond?

>> LISA MARSCH: I agree with what John said.

>> MAKEDA WILLIAMS: Okay. Great.

>> PAULO MENEZES: So do I. I have nothing to add to that.

>> MAKEDA WILLIAMS: Okay. Another question for all of you. Comment and question. I would imagine that there are many patients who do not use cell phones or uncertain about using them for healthcare. What sort of training do you provide? And do you have any systems in place to identify and help participants who are having issues with using the sophisticated technology?

>> LISA MARSCH: Go ahead, Paulo.

>> PAULO MENEZES: I can talk about what we did in our trials, and, of course, in particular the elderly people have more difficulty dealing with technological devices. So we had a very as an inclusion criteria, people had at least be able to read. Which may be a problem for some elderly people. But we understand that the familiarity that the population has with these devices, it's changing rapidly. So by the time that we finished the trial and have the results, by the time that these things are to be scaled, we believe that most people who need it can benefit with these kind of interventions, but we're not having too much difficulty dealing with these devices.

>> LISA MARSCH: I'll just chime in. I agree with those comments. I think some of this goes back to the initial design of the digital intervention, and there's a lot that can be done with the user interface and navigational flow of these types of tools to make them comprehensible and fairly easy to use, and I think that's a big part of it, is sort of the initial development piece. But also, yeah, the implementation of it, I think there are things you can do to support the end users. So, for example, you know, the one I described that we're using in Latin America, we had help buttons embedded in everything. At any time you can have help buttons pop up that can help you understand what you're supposed to do next or who this section is about. You don't have to see them but you can activate them whenever you want. We have audio, voice actor speaking, everything that is on the screen. You can have pretty linear navigational flows, and a lot of different paths that can be confusing for some end users, particularly if people have cognitive challenges. We also ask if someone joins our project, we also show them have them contact the app right then and there when they join the project with one of our team members so they can get exposed to it and have the chance to have some time answering questions that they may have. But I really am going to point, the design of these things, design goes a long way in helping people being able to engage with these types of tools.

>> MAKEDA WILLIAMS: Great. We'll go on to the next question. This is also for all of our speakers.

How long do patients using mobile technology take to effectively start using the app involved?

>> JOHN PIETTE: How long...what did you say?

>> MAKEDA WILLIAMS: I can repeat the question. How long do patients using mobile technology take to start effectively using the app involved?

>> JOHN PIETTE: In our experience it's really pretty immediate. We will do a test call when they're enrolled. But it's pretty straightforward. And then we rely I think the other programs as well, we rely on a lot of outgoing calls, so we're not passively waiting for patients to choose to engage as much with the app. It's kind of "don't call us, we'll call you." So they get texts and respond. They get IVR automated calls, and they respond. So people tend to be responsive and they tend to be responsive pretty quickly. If they do have problems the first couple weeks, we can call and help them with troubleshooting.

>> MAKEDA WILLIAMS: Dr. Menezes.

>> PAULO MENEZES: And ours is similar. Participants had one meeting with the nurse that lasted about 5 or 6 minutes that they show them the Smart Phone and explain how it works, and it has all the characteristics that Dr. Marsch describes, including help buttons and these sorts of things. And then the participant is ready to start with it. And every other week the nurse will call the participant to talk a little bit about how he's doing with the app.

>> MAKEDA WILLIAMS: We'll move on to the next question. In more collective cultures where families and communities are important for mental health and de stigmatization, are there aspects of the technology that target those supports?

>> JOHN PIETTE: That is the question is correct, you know, that social engagement is very important, and programs based on cognitive behavioral therapy are really kind of behavioral activation often emphasize that, how to solve communication problems, the negative aspects of family and social engagement that can be troublesome to people, and encourage people to engage or re engage with their community, including going to the market, going to church, spending time with family and friends and other things that can fall away when someone has depression.

So, yes, these programs often encourage people to be community engaged because it is important, you know, in their country as well as ours.

>> LISA MARSCH: So we're developing some functionality right now under a separate grant that is intended to try to add a sort of going social, or adding a social component where people using this type of digital therapeutic can choose to share information with a social support network of their choosing. It could be family, friends, some support network. And there's lots of things, as John is saying, that a social support network can do to support people in process of behavior change or recovery. For example, in accordance with certain principles of behavior economics you can have people provide these types of incentives for achieving certain milestones during behavior change or recovery process. Or have people participate in brainstorming and problem solving therapy types of interventions to help people in making decisions about the activities they engage. There's lots of other things I can describe, but I think there's a tremendous opportunity to have a functionality that can engage a social support network to help encourage people and in an ongoing way as they engage with these types of digital apps.

>> PAULO MENEZES: I would like to add that I think it also contributes indirectly to integrate mental health care into the lives of the people in the community. For example, in our experience, particularly the elderly will take this Smart Phone home and they would say to the nurse, they would ask the children, for example, to help them dealing with the device.

So I think the idea is that someone comes back home from the clinic with this Smart Phone, with the app, to treat emotional symptoms helps people to understand that this is also part of our health and that they have a system that can do something to help people with these problems.

>> MAKEDA WILLIAMS: Great. The next question is for you, Dr. Menezes. Is there any plan to expand the project to other areas in São Paulo and Brazil?

>> PAULO MENEZES: Yes, there are plans about that, but I think that at this point we would need to work more on the programming of the system, because I could say that at this moment the system is a very almost home made, if you like, because it was programmed specifically for the intervention. So it's not as yet developed for large scale use. But I think this will be the next step that we hope to be able to do.

>> MAKEDA WILLIAMS: This question is for all the speakers.

Which measure is put in place in terms of data security?

>> LISA MARSCH: Well, a lot of encryption protocols you can use that are really secure, similar to the kinds of protocols you use in eCommerce and in banking and finance, and you can encrypt data at rest and transit and storage. There's a lot that can be done from a technical level to try to protect data flow. There's also other things you can do to encourage users to, you know, to maintain these apps, but there's a lot we can benefit from other areas outside of health that we can apply to the health domain for data protection.

>> MAKEDA WILLIAMS: Dr. Piette or Dr. Menezes, did you want to answer that as well?

>> JOHN PIETTE: I don't really have anything to add.

>> MAKEDA WILLIAMS: Okay. So our next question is also for all the speakers.

Have you created ways to manage mental health issues for patients with several mental health issues in addition to depression?

>> JOHN PIETTE: We haven’t but there is definitely a movement in this work in general, people at Hopkins and I believe University of Washington, are taking more of a symptom base or function base as opposed to diagnostic base view. Many people with depression also have what we call comorbid anxiety and a global perspective would say that's because they're not different things. Many people have mental health, you know, problems that are affecting their daily lives. And so, you know, dealing with the symptoms is something that people are doing, particularly in environments where people have a lot of posttraumatic stress disorder like they do in places that are often difficult or have repressive governments or have a lot of violence.

And so that kind of work is kind of ongoing and it certainly is something that is very doable and addressable. It makes a lot of sense for these environments where, you know, depression, per se, is either not recognized or it's recognized as something that only crazy people have, if I can be blunt. So, you know, I had someone in Bolivia told me, you know, John, we don't have depression here. That's basically a gringo problem. We don't have the luxury of laying in bed and being depressed. We have to work.

So what they don't recognize depression as an entity and focusing on kind of those feelings or symptoms and kind of functional limitations as opposed to our ICD 9 diagnostic groups makes a lot of sense, and I think that's very doable.

>> LISA MARSCH: I just want to add that one of the beautiful things about digital technology is that you don't have to address one problem in isolation. You don't have to have a siloed app that just does one thing. We know that different types of mental health challenges, there are some overlapping approaches that can be helpful to lots of different populations in terms of cognitive behavioral interventions or economic interventions or motivational enhancement types of approaches. And so you can have tools that can flexibly use those types of functionality for lots of different types of target audiences and then bring in sort of problem specific types of tools for individual mental health problems.

So like what we're using, we've tried to take that approach to have a platform that can embrace, you know, the full spectrum of mental health, so it has panic disorder and anxiety and binge eating and all kinds of things beyond depression. We're not using all those in this particular trial, but there are population specific but also cross population or sort of trans behavioral approaches that have been effective and one in technology is you can have a platform that flexibly embraces that full spectrum.

>> MAKEDA WILLIAMS: Dr. Menezes, did you want to respond as well?

>> PAULO MENEZES: No, I don't have anything to add to what has been said already. Thank you.

>> MAKEDA WILLIAMS: Okay. Sure.

Our final question and comment:

In communities that rely more on spirituality, whether religious or cultural, can these technologies be tailored to include interventions that are not evidence based in the western world? Does anyone have experience working with spiritual healers for consultation and inclusion, especially considering some countries 80% of people with mental illnesses go to spiritual healers before they go to formal healthcare providers?

So that question is for all three of you.

>> JOHN PIETTE: I would say that, you know, cognitive behavioral therapy is very compatible with a spiritual approach and encouraging people to integrate with the belief systems and the spiritual practices that are meaningful to them. So that's certainly something that is very doable Bolivia is one of those places where there's a lot of people are Catholic, and a lot of people are Catholic as well as having indigenous beliefs, and our collaborators in Bolivia, we're quite comfortable that our intervention at least was respectful to those beliefs and I think you could even expand parts of it without much difficulty in a CBT framework.

>> LISA MARSCH: Just a brief comment. You know, we've worked with developing digital therapeutics for Native American communities in the U.S., so it's not like we haven't done it outside the U.S. with spiritual healers per se, but in Native American communities spirituality and religion is a big part of the culture, and what we've done is what John said. A lot of tools work well and are compatible with that perspective. We have modified a little bit of the content to be more culturally embracing in the Native American communities, but overall, we have found and seen in, you know, randomized trials that these types of tools can be very relevant even in those communities.

>> PAULO MENEZES: I agree that there is no incompatibility in our experience we can consider the technological interventions as one more tool that healthcare team has to improve the quality of the care delivered to patients, and the care to patients can be very integrative and comprehensive, including other types of approaches to health.

So I don't see any incompatibility with that.

>> MAKEDA WILLIAMS: Thank you so much to our speakers for answering all of the questions from the participants.

I would like to thank our speakers also for a very informative webinar on Mobile Technology for Global Mental Health Research and I would like to thank NIMH, The Bizzell Group, 1Source for their support of our webinar series and logistics. As I mentioned, this webinar is recorded, and it will be posted on the NIMH Office for Research on Disparities and Global Mental Health website in the near future.

For those of you who are interested in conducting global mental health research that leverages global mental health technology, specifically for low and middle-income countries, the NIH has a funding opportunity announcement led by the Fogarty International Center titled Mobile Health Technology and Outcomes in Low and Middle-Income Countries. The overall goal for this funding opportunity is to contribute to evidence base for the use of mobile technology to improve clinical outcomes in public health while building research capacity in low middle-income countries and establishing research networks in this area.

Applicants are required to propose partnerships between at least one U.S. institution and one low middle-income country institution and the proposed research plan to strengthen the Mhealth capabilities in the low middle-income country institution.

The funding opportunity number is PAR 18 242 and the application deadline is August 31st, 2018. This funding opportunity includes NIMH and other interested NIH institutes and offices. Please review this funding opportunity announcement for more information.

Our next webinar will be Tuesday, July 18, 2018 at 10:00 a.m. U.S. Eastern time titled "Implementation Science and Healthcare: An Introduction."

Visit our 2018 webinar series website for more information. I will turn it back to our operator, Rachel, who will close out the webinar.

>> WEBINAR OPERATOR: This concludes today's program. Thank you for your participation. You may disconnect at any time. Have a wonderful day.