Mental Health Economics: Analyzing Value

Transcript

Webinar Operator: Good day everyone and welcome to today’s Mental Health Economics Analyzing Value webinar. At this time, all participants are in a listen-only mode. At any time, if you would like to submit a topic or technical-related question, please use the Q&A pod at the bottom of your screen. Please note, this webinar is being recorded. I’ll be standing by if you should need any assistance. It is now my pleasure to turn the conference over to Makeda Williams.

Makeda Williams: Thank you so much, Wendy, for the logistics for our webinar today. Hello, my name is Dr. Makeda Williams. I’m from the National Institute of Mental Health. I’m pleased to welcome you to this webinar titled “Mental Health Economics: Analyzing Value.” This is the first in a series of four webinars for the 2017 webinar series focusing on global mental health issues, and the webinar series is sponsored by the National Institutes of Mental Health Office for Research on Disparities and Global Mental Health. Please note that this webinar is being recorded, and it will be posted on our website. It now gives me great pleasure to introduce you to our speaker for today.

Dr. Dan Chisholm is a Programme Manager for Mental Health at the World Health Organization Regional Office for Europe. His main areas of work at WHO have included development and monitoring of global mental health plans and activities, technical assistance to WHO member states on mental health system strengthening, and analysis of the costs and cost-effectiveness of strategies for reducing the global burden of mental disorders and noncommunicable diseases. Most recently, he co-edited the Mental, Neurological, and Substance Use Disorders volume of the third edition of Disease Control Priorities. He is actively involved in the Mental Health Innovation Network and supporting a number of NIMH’s international hubs for collaborative research on mental health. I’m delighted to welcome him today for our talk about mental health economics. Dr. Chisholm, welcome, and over to you.

Dan Chisholm: Well, thank you very much, Makeda, and welcome to everyone joining this webinar, whether you are here today or listening to this after the event. It’s great to see so many people logging in and joining this discussion, and I hope it’s a good start to this series of seminars being organized by NIMH. So, yes, the topic for today’s webinar is in the area of economics, which may not be a topic well-known to you, or maybe even a topic that you like. I think what I shall try and do today is try and make you aware, and we can discuss some of the issues around the need for an economic perspective in health care evaluation and in mental health services research. I’m going to pitch this at a relatively introductory level, because I’m not too sure that people joining the webinar have a detailed or in-depth knowledge of the area.

So I’m going to essentially base this around a number of fairly frequently asked questions and illustrate those, where I can, with appropriate examples from ongoing or recently completed research studies in the area of global mental health. So if I can turn to the next slide. Here is a set of questions, which I think it can be a useful way of structuring the presentation and the discussion that we will have when I’ve finish the presentation. So, I mean, a good place to start is always to ask the question, why? I think this is an important issue to get upfront, because in ... not until the quite recent past, the regular question—and still a question for many people—is, "Well, why do we need an economic perspective?” If I’m an individual clinician doing the best that I can for the patient sitting in front of me, I just want to be able to have the best technology and the best treatment available for that person. Why should cost even be a consideration when trying to do the best that I can as a clinician?

We start with that, the question of why, and then, we’ll move on to the what and the how of doing an economic analysis, in terms of the design, the data collection, data analysis, these kind of issues. Also, issues around the when, so when should you as budding or seasoned health researchers be trying to begin thinking about including these types of questions in your research applications or projects? And then also a question around the who, so who should be doing this work? Is it you? Is it someone else? Is it a specialist? These kind of issues.

So we go through a number of these questions and then leave some time at the end for some questions that you might have. So feel free to use the system to make your questions. These will come up to my attention, and we’ll try and get through as many of those as we can in the time available.

Maybe we can now move to the first of the questions, which is the why question. Why is this topic of analyzing value or health economics more generally, of relevance and interest? The starting point for this and indeed much discourse in the field of economics is the pervading issue of resource scarcity. There is never enough resources or money to do everything that everyone would like to do, or would like to have, or would need. If you think about the health care system, the amount of resources, for example, that are allocated to health, the percentage of GDP in the U.S. is perhaps highest in the world—around 20%. Now, arguably you could double that to 40% of the entire wealth of the U.S., and there might still be ... It’d be a fantastic health system, I would hope. There would still probably be some unmet needs somewhere in that population.

My point is that most countries are operating—and particularly when we’re thinking about low- and middle-income countries—at a very much reduced level of expenditure or as a proportion of the national wealth, perhaps 5%, maybe only 3% of the wealth of a country is devoted to health. Inside of that available budget or envelope of resources, some pretty tough choices are going to have to be made about where to allocate resources to, so that as many of the needs of the population can be addressed and met. So in essence, this brings us into the question of making choices, and making some kind of priorities, and … in order to be efficient with the resources that are available for health care, or for mental health, let’s say.
That can be a difficult conversation, because when you make choices, that means that while some people are winners, there are others who are losers. Or if I put it into the overall, let’s say, government sector, if I decide to double the amount spent on health, or on defense, that means there’s going to be less money available for education, or transport, or other areas of potential public expenditure.

So this can be a difficult issue. It also raises the difficult question sometimes of not just having to make priorities, but in essence rationing, which is not a word which people are wanting to hear necessarily, but is in fact an inevitable part of making choices, is that you are by deciding to devote resources in one direction, you are de facto, you are ... By so doing, you are withholding resources from another avenue of expenditure, or another option, or another way of intervening for a particular disease. So, yes, resource scarcity.

Then the question of efficiency is another important underlying principle and issue in economic analysis and sort of works at a number of levels. I mean, I was talking a moment ago about making decisions across different government sectors, for example. That would be the broadest level of trying to see, “Well, given all the money that is available and collected through taxes and other means, what is the best use of those resources?” and trying to prioritize across agriculture, and health, and space exploration, and everything else. All of those have costs and hopefully all of them have useful benefits as well in terms of meeting the actual or aspirational needs of the population. If we come down a level into the health sector, we would apply the same principles—that across the health sector, allocation to infectious diseases, and injury prevention, and to cardiovascular disease, and to mental health, and everything else. So, how to determine how much resources? Should it at least in part be based on issues of efficiency? So where are you going to get the most value back for the money that is invested? Just like you would in the private sector when thinking about different investment options.

And in mental health specifically, the question might come down to more specific issues around, for example, for the treatment of depression or psychosis. What are the best ways of meeting the needs of those population groups? And what are the intervention strategies? Is it pharmacological treatment? Is it psychosocial treatment? What about social interventions? We need to start thinking, therefore, about the, what are the costs of implementing those different intervention strategies, and then also what are the benefits? What are the outcomes from that, that will flow?

And so that in essence is what economic evaluation in health care or mental health care is trying to do, is simply trying to identify what are the cost implications of different decisions, or allocations, or options, or treatment strategies, and compare those costs to the outcomes or benefits of those different possible options. And in so doing, begin to identify which are the more cost-effective or most efficient strategies or options that appear to come out of an analysis.

So that covers the why, to some extent, and of course there’s an awful lot of materials out there, which you could read more about, about economic evaluation, and we’ll come a bit later in the webinar to some online resources that can help you gravitate more towards these issues, some of which may be a bit new to you.

Let me move to the question number two, which is around … another why question, which is ... I think I’ve partly answered it, but in essence, why should I bother doing this economic analysis alongside my innovation? I think you got a gist from what I already said about the overall purpose of doing economic evaluation is to assess the costs and benefits. And so I think the basic rationale for why you should be seriously considering adding issues of economic analysis into your preparations, into your project work, into your thinking, is that certainly if we’re talking about global mental health and the extreme resource scarcity issue in Africa or elsewhere when it comes to the mental health sector, it’s great to have new ideas, new innovations and evidence coming up about how different disorders or mental conditions can be appropriately managed, or prevented, but as a local decision-maker, I would also want to know what this is going to cost. Is it going to cost me more? Is it going to save me money? If it saves me money, then I’m going to be quite interested.

If I’m ... and I’m not talking here about the mental health program manager in a government or a ministry. I’m maybe talking about the minister of finance, or the planning department, which has nothing to do with the mental health program. So there are other authorities who often have an important say over what is the allocation of resources, in this case to mental health, who are not people who are working in mental health. So it’s important be able to, to an extent, talk their language, so if ... and I’ve certainly suffered this in the past myself as a WHO employee, that when you have an opportunity to talk to decision-makers, and you start talking about quality-adjusted life years or improvements on a depression scale, their eyes glaze over because they don’t really know what you’re talking about. But they certainly understand money, and so if you can clearly demonstrate that by scaling up your innovation, that you are able to not only improve the health of the target group, but also save some money, or compared to the way that money is currently spent—let’s say, hospital-based services—your community-based program or innovation represents a more sensible use of resources. That’s the kind of information which you might be able to communicate quite effectively to people who are not specialized in the rubric of mental health, or in health economics, or anything like that. So it’s part of the evidence, essentially, that goes alongside information on the effectiveness of an intervention.

Also, another important criterion that might enter into the decision-making alongside something being effective and cost-effective is issues of equity and fairness. So is the population that you’re targeting, is your intervention pro poor, for example? Does it improve access to poorer people and more vulnerable populations? And another one might be feasibility, the logistical considerations around scaling up your intervention. So a kind of a rounded process of decision-making would incorporate a number of elements, of which cost-effectiveness or efficiency is just one of the considerations on the table.

I think my point is that if it’s not on the table, if you haven’t got any information on what the cost, the cost-effectiveness of the intervention is likely to be, then that’s an important missing gap in order to make evidence-based and rational kind of decisions around the use of resources. So I think I’ll move on to the next slide.

So let’s move to the what question. And what should I do? And the first thing to do is design a study. This could be anything from, “I want to be able to know what my intervention costs.” So for example, let’s say you’re developing some psychosocial intervention, and you need to develop your protocol, you need to develop a training package, and then you need to train people. Then you need to see how it actually works. You might have to give some, for example, some sessions of psychosocial treatment or counseling. How long do those take? Who needs to be involved? Then there might be issues around supervision, follow-up, and so on.

So You could just focus a study just on that alone, and that will still give you important new information that, as researchers, I think it’s important that you know what is the cost of delivering your intervention to one beneficiary or person, one beneficiary of your intervention. If it’s highly expensive and it’s using a lot of resources—you know, 24 sessions of CBT—it’s highly unlikely that that intervention will be ... it will be possible to scale that up in a very resource-constrained environment. That’s sort of an extreme case, but you get my point.

So just looking at cost alone has some information value, but it doesn’t tell you about whether the intervention represents value for money. For looking at questions of value, of cost-effectiveness, of efficiency, you need to have both information on the costs and also on the outcomes of the intervention. So a full economic evaluation would need to consider both of those. Just looking at costs alone is a partial, if you like, economic evaluation, not a full economic evaluation.

I’ll get to ... When we get to the when question, I’ll mention the point at which it would be appropriate to start thinking about the design of a study. But, in essence, you need to think about what it is that you are interested or already involved in evaluating, in terms of your innovation, and thinking how you can add in an economic component to complement the other domains of assessment or evaluation that might be already part of your project or your plan.

I’ll move to the second question on this slide, which is dealing now in more detail. When we talk about doing a full economic evaluation, we need sort of information on the costs and also the outcomes. So what do we mean by that? On the cost side, there’s a number of possible components or areas that could be costed.

I already mentioned the cost of the intervention itself. Let’s say it’s your psychosocial intervention and a number of sessions of counseling. You can think through what are the actual resources that will be needed or are needed to make that intervention a reality—so time spent training people, time spent per beneficiary providing the counseling, time spent developing the hard-copy materials and training notes, and so on. That’s just one obvious kind of cost that you might need to consider, but when we’re thinking about mental health services and the introduction of an intervention, your intervention, within that we’re also interested in looking at what might happen as a result of ... For example, let’s say the standard treatment is to treat psychosis with antipsychotic drugs, old antipsychotic drugs, and basically patients don’t get much in the way of psychosocial treatment. Maybe your intervention is to add a psychosocial treatment to that pharmacological treatment to see if that can improve the outcomes and recovery of persons with psychosis. You might also hypothesize that by improving people’s functioning maybe quicker … It’s like in depression: If you can hasten, if you can speed up the rate at which people recover from their depressive episode, there’s not only clinical or health gain, there may be some economic impacts as well, and not just in the health care situation, but also beyond that, which I’ll come to in a moment.

If we just think about in the health care system itself, maybe as a result of your innovation, people need to be admitted to hospital less often, or there’s less revolving door. Maybe in Ghana, people were, or their families were going a lot to indigenous practitioners—faith healers or priests—who can certainly provide a liaison role. But maybe families or individuals were spending a lot of money on religious retreats or … there can be quite a sizeable out-of-pocket expenditure, and maybe that reduces following the introduction of your innovation.

In other words, when we do an economic evaluation in the area of mental health care, it’s typical to ask about people’s use of services generally, with a view to detecting whether there are any changes over time—or between the two groups of your trial, for example—with respect to use of services. Particularly, you might hope that, yes, there might be a reduction in hospital admissions and inpatient stays, which are of course highly expensive. Maybe there’s a change in use of other health-seeking behaviors, which may be for the better or for the worse. So we try and get a sense of the totality of the health service costs that are incurred by people receiving the innovation.

Another category of cost is time. Now, I didn’t yet mention the other famous axiom of economics, which is the concept of opportunity cost, but it very much comes up here. The notion of opportunity cost is that if I spend time talking to you now, that’s time which I have now not got available for doing something else. Just like I was saying, if you spend all of your money on health care, that’s less available for education. It’s the same concept that when we think about the value of a service, or a good, or even someone’s time, we think about, "Well, how could it have been used ... If I wasn’t doing it now, what would be the value?” So if I wasn’t, for example, looking after my mother, who is suffering from cognitive impairment or dementia, then maybe I could’ve been doing my job. Maybe I had to give up my job in order to take care of my mother. So I then spend time providing informal caregiving to my mother, or my aunt, or whoever. And that money—sorry, that is time which is not paid, but it of course has a value, and so we’re trying to, in economic assessments of this kind, is to not forget about that. And there may be some intervention that means that young girls spend less time going to fetch water, okay? Outside of the village. Part of an assessment of introducing safe drinking water in villages might be exactly that—that you’re saving on the time that people spend going to fetch water, or firewood from the nearby forest, or something like that.

In essence, what we can think about here is the time, if we can bring it back to the innovation in our minds, of the time cost associated with the innovation. So from a patient or a family point of view, that might mean the time spent traveling to receive care. And again, that could be a big factor, that if you are championing or developing a community-based or primary-health-care-based intervention, meaning that care is much closer to where people actually live, compared to having to go to two or something hours to a nearby town or city for care, then that might be an important saving, which has real value for the people benefiting from it.

These are two obvious categories of time that we can think about—the time accessing and waiting for care and also time spent giving care but which is not paid. If I think about an informal caregiver, I can think about, “Well, what is the value of that time?” It might be the value of the salary or wage that I was earning before I gave that up, but it might also be arrived at by thinking, “Well, if I don’t do this informal caregiving, what would I have to pay someone else to do it? What would be the wage of the home help who comes in helps with my mother’s needs?” There are ways of arriving at, if you like, the price or cost of time. So we can think about adding those into our design.

Then a third one is around productivity and it’s ... I mean, often you can put these together, time and productivity costs, because you’re thinking about the time that may have been lost, either … might be caregiving, but I’m also thinking here about the time that individuals with, let’s say, who become depressed, they have to stop working maybe, or they have to cut down their work. So that of course has a ... is an important cost and consequence of the disorder, so we can also think about what is the value, the lost value of that time and productivity.

So those are some of the key categories of cost. Then moving to the outcome side, there’s ... I mean, ultimately the most important just ... This is no different to any clinical evaluation: You want to be able to demonstrate the improvement to the beneficiaries in terms of reduced symptoms, or disability, improvements in other areas. Level of functioning is a useful concept I think, because it’s a bit more transdiagnostic and so being able to ... and it’s in many ways has more meaning for the end beneficiary than being able to say that their symptom count went down by three. That’s a clinical tool. What they really care about is being able to go back to work, or enjoy the things in life, spend time with their family, whatever it is—so being able to function, basically, in everyday life.

In addition to the obvious health or functioning outcomes that I’m sure are your primary or secondary measures of outcome, you also might want to consider other, what I call non-health outcomes—social or economic outcome measures. So here the relevant metrics would be things like the ability to go back to work—so, the person who lost all those days of work due to depression. You may be familiar with studies which show the huge cost of depression on economies and millions and millions of days of work lost. So how many of those do you get back as a result of an intervention? Do you get all of them back? Seems not. But you know what? I don’t have even a very good idea, because so few innovations or evaluations have actually incorporated that outcome measure in their work. We were scratching our heads about two years ago when we were trying to do a global analysis of this, and we were really struggling to find good measures of return to work, even though that’s an obvious everyday need for many people, including for people who have different mental health problems. So that would be an example of an economic outcome measure, and might be an important element to include in your battery of outcome measures, and you can subsequently put a value on that, a price, a cost, and add that to your economic analysis.

Also, I mention livelihood measures, not just in terms of back to work, but other measures of being able to generate an income, or grow crops, or whatever it might be. These are other important outcomes to include in the thinking on the design of such a study. Then of course these are what you could describe as final outcome measures. They’re really dealing with the actual health, functioning, and livelihood of individuals. But before that, you can of course identify as well a set of intermediate outcome measures, and those deal more with the result of your ... the impact of your intervention in terms of perhaps detection rates, or referral rates, utilization rates—different intermediate measures, which can be very important, but are set apart from the final outcomes of the evaluation. When you do an economic evaluation, it’s important to be able to link these costs to final outcomes, more than intermediate outcomes.

Okay, so time’s whizzing along. I’m going to move to the next slide here.

So the question is, when. I touched on this a bit earlier, but when thinking about designing an economic component to a study, it’s important that it is at the beginning of your thinking. Now, this may be too late for some of you. You may be already halfway through a study and think, “Oh damn, I should’ve been thinking about this earlier.” Because I can tell you a number of times when people have come to me or my brethren, health economists, saying, “Oh, we’ve got this great study going on. Do you think you could just knock out a quick cost-effectiveness analysis for us?” So you say, “Yeah, well that’s interesting. Let’s talk about it. Where have you reached?” “Well, we’ve got about six months of the study to run.” And you think, “Okay. Well, when did it start?” “Oh, about two years ago.” “So what data do you have on the cost of it?” “Well, that’s what we thought you could tell us.” And it goes on like this.

So you can imagine the frustration for all concerned that this dimension has been basically an afterthought, rather than an integral part of the process of designing and planning a study.

To help you to think of remembering to do this at the initial stages, I think we have to thank the funding bodies, because a number of them are now either subtly encouraging or even insisting on an economic dimension being part of the thinking and the planning. As I said, it can vary in terms of the scope, and the importance of the economic question should determine the amount of effort that then is exerted on answering that question. Maybe it’s not a very, maybe a highly important question, but the point I’m trying to make is that you need to think what is? Is there an economic question? If so, what is it, and how important is it to answer it within the confines of your study? Then think through, "Well, what do I need to measure in order to be able to answer that question?"

So, yes, a number of funding bodies are now asking for this, and we will come to some sources of information to help you through that in addition to what I’m discussing now.

So that was that. Now, the other when question relates to data collection. So you’ve got your study design, you’ve got your NIMH funding for your wonderful new innovation. All is good. So then, what next? And the design hopefully is being all worked out nicely. You’ve got your cluster RCT, and you got your facilities, and you’ve got your sample, and everything else, and you’ve got your instruments. The main instrument that you will need to add in to the battery of outcome measures and other process measures in your study will be a service use questionnaire. Ideally, we would just take all of the information off electronic records and the like, but in the context of global mental health that’s not going to be likely feasible. Also, remember that we want to ask about the full range of services that people in the study might have used. You would need to have information across all of those different service providers.

A quick and second-best option is to administer alongside your measures of clinical status, and demographics, and everything else, you need to ask a few questions about what services have you used over the last few months or few weeks in order to get a sense of the service costs. And I’ll say a bit more about that in terms of the instruments in a moment.

So—when. This would just be the same as ... typically, the same as the main clinical measures, so obviously you need one at baseline, and then at your main follow-up points in the trial. It could be at 6 months, in 12 months, or 18 months afterwards. This will then enable you to look at what is the estimated cost of the intervention and the services, and other costs as I just discussed, alongside your assessment of the clinical or other study primary outcomes.

Okay, so that’s in a way trying to demystify the complexity of this. In essence you’re saying, "Right. We have an economic question, which is we want to know whether the intervention is not only more effective, which we’re hypothesizing it to be, but also whether it’s more cost-effective." In other words, "Can we address the question? Can we answer the question of this ... Or can we show that this intervention is not only more effective, but represents a better use of resources than the alternative?," which could be your usual care or no treatment option.

Okay, so then the other piece of information or data that needs to be collected are unit costs. An example of some unit costs would be the cost of an inpatient day, the cost of an outpatient visit, the cost of a day’s medication, the cost of an EEG, or an X-ray. These will all be the cost per unit of resource. It could also be the unit cost of a health worker, which would be, in other words, the salary, the wage, the cost per day of pay of that person.

There’s good news and bad news. The good news is that it’s not something you have to do on a regular basis. Let’s say you’re doing three different studies in over three years in Sri Lanka. Well, you don’t need to do this three different times in Sri Lanka. Once will do, because this data will be ... have a shelf life of a few years. The bad news is that it can be a bit of a pain to get hold of some of this information, particularly when you get down to things like inpatient costs. It can be very complicated working out the cost structures in a large hospital and working out what it really costs one inpatient to spend one night in a psychiatric ward. That can get fiendishly complicated because of all the shared costs in a hospital. So what we’ve developed is a number of simplified templates, which enable you to quickly arrive at some estimate of the unit costs, or if you like, price of the different items or resources, and they’re shown here.

Here’s an example of a resource use schedule questionnaire. It’s asking about, "Did you see a primary health care doctor? If yes, how many times?” “Did someone come along with you?" because that will affect the cost to the family of travel and time. "How much money was spent doing that?" And then, "What was the time spent waiting and actually receiving care?” “Were any out-of-pocket payments made for the fees?” and so on. We obtain this information for the different type of health care providers, and we can then add all of that up and put costs to those different items of service use.

Here’s an example of this template for looking at the costs. This is a case of a ... of, it looks like, a medical doctor in Nigeria, because it’s in naira. This is the wage in naira, or in dollars. Then what do people actually get in terms of some on-costs, so like the pension, and entitlements, and so on. Also, what is the cost of the utilities that they use and the office space and so on—these sort of overheads. So you work out a kind of overall grand cost of that person, associated with that worker, and then divide it through, in this case, by the number of days and hours that they work in a year, how many people they see, let’s say, in a week or in a day, and then you can arrive at these unit costs. The time ... We go right down to the bottom of the page. We’re saying it’s 29 naira per minute of client contact time. So if you’re applying this to your counseling innovation, that would be the value that you would use, or in U.S dollars it would be 17 cents. So you can see that this is a simplified but accessible way of arriving at the cost of these different types of service use.

Now, let’s move to the who question. Who can do this type of work? Well, what I’m basically trying to convey to you today is that this doesn’t need to be undertaken by health economists with Ph.D.s. It can be undertaken by yourselves. It’s not rocket science, and innovative health service researchers can easily undertake this work. As non-specialists, you don’t have to be an economist or anything to do economic evaluation in health care. However, of course there are some peculiarities. You’ll want some advice about the planning stage and the design, and there are some peculiarities to do with the data analysis, not least because of the skewness of a lot of data, so you have to kind of work with that—statistical issues. But in essence I’m saying that don’t undersell yourselves and your ability to add this type of work into your planning and to your work. And then, if needed, identify someone locally or internationally who can provide some maybe some strategic periodic advice on special issues that come up.

And that’s been the model that I’ve been trying to apply with the NIMH research hubs, that all of the teams have got people who are working on this. They’re not health economists, they’re just part of the project team, and then we have the occasional call or follow-up to check in on how things are going, and that seems to be going okay.

So I’ve set up a few of the main players at different stages. At the funding and design stage, obviously the principal investigator and others need to be involved in the design of economic evaluation. Then in the implementation stage an appointed, identified person, a member of the research team. At the analysis, it’s important obviously to have a good data manager and an analyst to help with the data cleaning and the processing. Maybe you need some more special stats advice, depending. Then of course for the reporting of the work as part of the output of the research study, again all involved team members should be reflected there.

Let’s move to a couple more questions. One is around ... Well, it’s actually the last of the kind of interrogatory questions, the how. The first one says, "How do I know if my innovation is affordable or cost-effective?" You’ve done all this work. You worked out the cost. You’ve related it to the outcomes. You’ve got a number and what does it mean? So the most typical way ... I mean, you have your assessment of cost, and even that in itself means different things to different people. If I produce an innovation that costs $100 per beneficiary in the U.S., that’s going to be like a bargain, but if it’s $100 in Zimbabwe, that’s going to be a lot of money. You need to think what is the sort of level of affordability, and it’s very hard to come up with fixed kind of rules about this, but in the resource-constrained settings in which many will be working, suggesting that for population-wide promotional prevention, you definitely want to have a cost of less than a dollar. If you’re looking at the cost per treated case of disorder, let’s say psychosis, then probably $100 is kind of not your limit, but it would be good if it’s less than that.

Yes, so then the other critical question is the cost-effectiveness question. How do you know that your innovation is cost-effective or not? The typical scenario is that your innovation might have cost a little bit more, because you had to do some training, and you’ve built your capacity up, and you had your sessions of counseling. So there was kind of an additional cost, and the question is whether that additional cost was worth it in terms of the additional health gain that it led to. This leads to something called the cost-effectiveness ratio, which is essentially the incremental cost divided by the incremental benefit. Exactly what I said. Is the extra, let’s say, $50 per treated case worth it in terms of the number of days of resource functioning, or number of days seizure-free, if you’re looking at epilepsy? You can then summarize that as a ratio.

Even then there’s a question of, "Well okay, what does that mean?" So health economists have come up with ways of trying to interpret that as well. For example, saying, "Well, if we think that one extra seizure-free day is worth $10 in the local context, then the probability of the intervention being cost-effective is 90%." If on the other hand the willingness to pay for a unit improvement in health was $100 or $1,000, the probability would be that much less. There are these so-called acceptability curves, which can be drawn to illustrate the likelihood that your invention is going to be cost-effective at different levels of willingness of decision-makers to pay for that improvement.

Now, we can come back to this question if you have any. I don’t see any questions popping up in my Q&A panel yet, but don’t forget that you can fire a question out there if you’re interested to do that.

Finally, how do I learn more? I took you through very quickly some of these instruments, and questionnaires, and templates. There’s also some brief guidelines that have been prepared on undertaking economic analysis in the context of low- and middle-income countries in the area of mental health from a long time ago (I think I wrote those). There’s one or two presentations like this one, which will go into some of the issues.

The other resource that I want to draw your attention to is an existing NIH self-study online course, and you can see the link there. On the next couple of slides, I’ll just give you an outline. I’ll show you the outline in fact of the course, because this has already been prepared and covers some of the useful ground. It’s an NIH Health Economics Information Resource, and seeks to do what I’m doing, which is just sort of give an introduction and identify the scope of health economics and what it’s trying to do and where you can get information on it. You’ve got some U.S.-specific health financing issues, which might be of interest to you in the current political climate and debates.

Then what are the issues around assessing quality. That’s a very useful skill actually, because even for those who may not be wanting to incorporate this into their own projects or work, just like being able to read a good clinical evaluation paper, how do you know whether you’re reading a good economic evaluation paper or not? There are these ... Well, there are these checklists that have been published, but also here it’s giving you some advice about how to identify decent economic evaluation studies. Again, some criteria.

So that’s covering some of the overall purposes of it, and then there’s a number of modules covering the scope, I would say, and then basically this is giving a bit more detail about what it goes into. I recommend you having a look at that if you’re interested in pursuing this a bit more, just to brush up your own skills and knowledge in this area.

Okay. Now, I want to move a bit to the application of these techniques and methods. This is just to give you an overview of some of the NIMH-funded hubs that Makeda introduced at the beginning. The point I wanted to cover here is that one of the issues that many of these hubs are seeking to address, because they’re looking at issues of task sharing or shifting ... You can see the type of task-shifting approach, or task-sharing approach that is being pursued, and then in that, often there is a question of the … "Well, what are the cost implications of this, and can we also save money as well as improve health as a result of this?" So you can see that across a number of these different hubs, with an overarching similar goal but different specific designs and target groups, that most of them have or are including an economic dimension.

This was taken from some time ago, when I went to one of the research hub’s annual meetings, and we were trying to assess the commonality across the different hubs. So you’ll see, for example, it’s interesting that all of them are including this measure, the WHO-DAS [WHO Disability Assessment Schedule], so are looking at functioning, and that’s great for being able to look across these different studies, which as you can see have different primary outcome measures, but have a common measure of functioning across them, which enables us to do a cross-study sort of economic analysis.

I thought I’d just put this up just to make it clear what again we’re trying to do. You want your intervention in the middle, which is some intervention compared to some standard care or no treatment. And you can see the inputs and costs that are needed to make that intervention happen, in terms of training, and drugs, and staffing, and so on, and what happens in terms of … as an individual recipient of that, their use of services and so on, leading to these different outputs and outcomes. That’s just a graph, which I thought should ... perhaps should’ve put up a bit earlier.

Then what I wanted to do now was just finish with a case study to try and illustrate some of the concepts and methods that I’ve been going through. This is an economic evaluation of one of these task-shifting or task-sharing interventions for common mental disorders in India. It’s the MANAS study led by Vikram Patel, and the economic evaluation of the trial was published in 2012, about, I don’t know, a year or so after the main clinical evaluation by Christine Buttorff et al., and others. This economic evaluation was included at the beginning of the thinking of the MANAS trial, so all of the instruments were piloted before the main trial started. We thought about the sampling and so on. The hypothesis that we had was that collaborative step care led by lay health workers will promote recovery and restore functioning in a more cost-effective way than conventional care.

That was our economic question here, and our outcomes were depression status and severity, disability (using the WHODAS), and also converting the disability scores into these quality-adjusted life years, or QALYs. (I didn’t say anything about QALYs yet, but we can come to that if there’s time at the end.) Those were our outcome measures, and we looked at a range of different costs, both the health system costs that I mentioned earlier, in terms of the actual cost of the intervention and what that took; all of the health care visits; use of medication; and so on. So these were the health system costs. And then also time costs, so the time spent traveling and help seeking, as well as lost productive time, and we added those together for some of the analyses.

Our findings were that the overall health system costs did not increase. So what that means is that, yes, there was a cost involved in training, and developing, and implementing the intervention, but we found out (a) that it was very low, very small, very modest, and (2) we found that it was offset, so it led to a slight reduction in the use of services compared to the control group. We also found that the time costs that we measured were reduced, so that was a good sign, a nice finding to get. And of course, the symptoms also improved. So if you put that all together, we’re best able to conclude that the intervention was affordable, low cost, and also a cost-effective approach to take.

I think there’s a bit more on this. Here, it’s showing these costs in the ... both in terms of the health system and the time costs between the two groups. You can see it’s been split up into these outpatient costs, inpatient, medication, and so on. And then in the time costs—visiting, lost wages, family caregivers, and so on. You can see that overall the health system costs were no different, and there was a slight reduction in the time costs in the intervention group.

This is the cost-effectiveness summary analysis, that we’ve got our costs here carried over from the previous table, and then looking at the outcome measures between the groups and what that really is. ... we’re able to conclude is that this intervention arm was slightly more costly, but more effective in terms of the health system costs and in terms of overall costs, which is the health system and the time costs, less costly and more effective. This is just giving a very quick example, and you can read this, as you can see, there in the WHO Bulletin. I think, virtually my last slide is to, just to push it back to you, and this is what I would suggest for your homework, following the webinar is to say, okay. let’s say you’re thinking of a study. You got your innovation in mind, and you want to add in an economic component. These are some of the questions that you should be asking yourself: What is the economic question or hypothesis? What is the scope or perspective that you want to take when thinking this through? Whose costs are you talking about? Is it families? Is it the health care system? Is it the employer? Who’s? And what outcomes? You could actually then write down a list of the types of costs that you would need to collect and analyze in order to address that question that you’ve addressed under number one. So you don’t have to start thinking about how you’re actually going to measure it or evaluate it, but just identifying, thinking about what are the likely costs that you need to measure is a good first step. And then you need to then think through how you would relate the costs to the main study outcomes, typically in the form of a cost-effectiveness analysis.

Okay, so my voice is beginning to go. These are some other questions, but I think at this point I’d like to open it up for some questions. I see I’ve got a few in here. I haven’t had a chance to read them yet, but I’m going to do so now in real time.
I’ve got one from Dr. Hamdani in Pakistan. "Hi, Dan, is there a learning resource on statistical analysis of health economic data?" So this man is asking here about what he’s probably getting himself into right now. So this man is someone who’s taken an active interest in this area. He’s not a specialist, a health economist, but he’s applied his not-insignificant brain power to this issue and has now collected data in some studies, some innovation studies, and now is battling with some of the more complicated statistical issues around economic analysis.

Some of that is around, as I mentioned, the fact that the costs of a group of participants in a trial are often very skewed, positively skewed. Meaning that, there’s one or two people who, let’s say, have very high costs because they have a lot, let’s say, hospital care. Whereas, a lot of other people have very little care, so you’ve got skewed data, which means that you have to treat the data in a non-kind of normal way, and you have to use nonparametric techniques to evaluate those kinds of issues. So there are some more specialist textbooks which go into some of these techniques, which are available. I don’t quite know how I’m going to make the … maybe I can add to these to the slide set afterwards, or just communicate directly with people who have this question. But the answer is, yes, there is a learning resource on that.

Now, I’ve got two questions from Victoria de Minel in Chennai on how to do an economic analysis of tele-psychiatry. My question is, "What do you think is the best outcome measure for economic analysis in mental health?" My response to that, as a bit suggested earlier, is that one limitation of having a very disorder-specific measure of outcome is that it makes it hard to compare your results to other people’s results. So often, it’d be good ... you still maybe want to have a measure of symptoms, or something specific to their condition in question, but always try to also include a broader, more transdiagnostic measure, and I think functioning is an excellent transdiagnostic concept, which has good validity and meaning to end users. I think that would be ... and there are various measures like, I mentioned, the WHO-DAS has been developed for use in and across different cultural settings. That would certainly be one possibility.

And a supplementary question was around the measurement of economic outcomes, and there I think there are questions already in the WHO-DAS, which ask about number of days out of role, either full days out of role or partial days out of role. Even that gives you, I think, quite a useful measure of ... It’s not on a scale—these are separate questions just asking about days out of role, and I think give you quite a useful tool to look at those economic outcomes, whether you’re a paid worker, or you’re a home worker, or whatever, than actually just measuring the number of restored days, reduced days out of role, I think is a meaningful measure.

Let me move to the next question. "Which policy would be cost-efficient for governments financially supporting adult children?" I don’t under that. "Adult children who cannot completely be in the labor force of elderly people who are in the need of mental health care also ... " I’m afraid I can’t understand that question, so I’ll move on.

"What do you think is ... " This is another question. "What do you think is the best tool to measure service utilization and cost?" Okay, this is a good question, from Kevin Habtouk. So I put up earlier an example of something called a service use questionnaire, which is loosely based on something called the Client Service Receipt Inventory, which was developed by researchers in the UK quite some years ago when looking at the re-hospitalization ... the community ... the move to move individuals out of institutions into community-based care settings. So that instrument, the CSRI, has been used and adapted now in I think hundreds of studies. I think the point I would stress here is that the first thing you would do with any adaption is think again about who is your target group, what is their likely range of service use, and adapt the questionnaire accordingly.

So if I’m doing a study in Pakistan, there are lady health workers, so I might want to adapt it to make sure I ask about any visits that individuals have had or received from lady health workers, for example. Those don’t exist in the UK. There’s an adaptation process also in terms of the scope of services, so it’s very different if I’m looking at a perinatal depression study compared to if I’m looking at care for the elderly, or psychosis, and so on. But as a basis, an instrument, the CSRI is one that ... the most commonly used in mental health services research to my knowledge, and certainly in developing countries.

Next question, "In cases where unit costs are not available, can we ask the participants the amount they spend for a particular service?" Well, yes. That is a relevant question to ask if they’re paying it all themselves privately out-of-pocket. So if they go to a private hospital, it’s fair to ask how much did you pay? If on the other hand, they went to a government hospital, I’d hope that they’d say “nothing,” they’d say zero if it was free at the point of use. Now, they may have had to pay some money, so they might say, "Well, I had to pay this many shillings, or rupees,” or whatever. But you have no idea whether that’s really the true cost of the government-provided health care. So there’s no way around ... you can’t just rely on the report of individuals where you’re trying to look at the costs of public health facilities. You do need to work out the unit costs of providing an outpatient visit in a government clinic, for example.

Next question, Dristy from Nepal. Hello, Dristy. "Can you share why health economists prefer using EuroQol, then WHOQOL?" Okay. Well, that’s getting quite specific, but the WHOQOL is a scale for looking at the quality of life across a number of domains. You get some overall scores on different domains of quality of life. It’s like any of these other functioning or quality-of-life scales. The EuroQol, on the other hand, has been specifically developed to enable something called cost-utility analysis, where you can convert the scores on this EuroQol, which is very brief and short—just asks for one to five on five domains or something. You can squeeze all of that information onto a zero-to-one scale, and so you can then basically use that to generate quality-adjusted life years, which is the time that people spend alive—that’s the quantity of life—and then it’s adjusted by the quality of life, which is on this sort of zero-to-one level. So … and this is what is used also in things like the global burden of disease—methods used to work out what health states are on a scale of zero to one. For example, psychosis has a very high level of disability or lost health, so on a scale where zero is no disability and the schizophrenia or psychosis is around .6 or .7—so quite a long way from good health, whereas, depression might be around half that amount.

So that’s why economists prefer using EuroQol, because they can then undertake this cost-utility analysis, and the value of that is that, as I was saying before, it means that if you’re study of your task-sharing innovation in Nepal comes up with a cost per QALY, that can then be compared to the cost per QALY results in, not just in mental health, but in compared to malaria, or HIV, or cardiovascular disease, or anything else under the sun. That’s why economists like to use this, because they can compare across different fields of health care.

Okay, next. What have I got here? Yes, I’ve got a question here. "Sometimes it’s observed that after the use of any intervention people become more aware about the available health services, and their health service utilization may also increase due to it." I’m not sure what the question is here. Yes, I think it’s saying that once people realize that services are available, then they start using them more. And I guess it’s possible that you could ... in a baseline, people didn’t realize … you start being asked about whether you use these services, and you might say, "Oh, I didn’t know they existed." And once you have that knowledge, maybe you start using them more. It’s a good question I’ve never thought about before, and I guess it could lead potentially to increased rates of reported service use, which could be very good and needed, but it would also show up as higher costs. But who knows whether that higher use of, let’s hopefully say, good services, appropriate services, leads to a reduction in services which might … are less needed. It’s a bit like good cholesterol and bad cholesterol. I’ve never thought that one through.

I have one more question before turning it back. It’s, "Is economic evaluation of an intervention possible when we don’t have a comparison group?" Okay, that’s a great question to finish with because this is a very important issue. It’s the same in clinical evaluation, that you can have a one-sample, or one-arm study and follow people up. You can certainly do that—an economic analysis, let’s call it, of a group and see what their costs were at the baseline and follow-up.

I’ve done it myself. I remember, for example, doing a study in India with colleagues at NIMHANS [National Institute of Mental Health and Neurosciences] of untreated schizophrenia in the community, and we followed those people up for 18 months, looked at their costs and service use. We looked at their functioning. We looked at their symptoms and were able to do a nice observational study. But of course, what we cannot do in that design is attribute the improvement or the change in cost to the intervention itself. For that, that’s why you need a control group.

So the answer is, yes, you can do an economic analysis, but to do a full, proper economic evaluation you need a control group, because when you say something is cost-effective, it’s always asking the question, compared to what? Is it compared to usual care? Is it compared to ... comparing drug treatment versus psychosocial treatment? Is it comparing to nothing? So it’s always a relative concept: cost-effective compared to ... So if you don’t have a “compared to” group, then you’re not really doing cost-effectiveness analysis.

With that, I’m going to hand it back to Makeda for any closing remarks. Thank you very much for hanging on in there, all of you, and I hope that you’ve generated some utility from attending this webinar. Thanks so much. ‘Bye.

Makeda Williams: Thank you so much, Dan. You’ve done an excellent job. You’ve talked it through for almost an hour and a half. So, great webinar. Thank you so much.
And as I mentioned earlier, this webinar will be posted on our website, the NIMH Office for Research on Disparities and Global Mental Health website. I’d also like to thank our NIMH staff; the Bizzell Group, our contractors; OneSource, who have been doing the logistics for the webinar, for their support for our webinar series.

Our next global mental health webinar will be tomorrow, Wednesday, October 23rd, from 9:00 a.m. to 10:30 a.m., U.S. eastern time. The title of that webinar is “Treatment Targets, Target Engagement, and Target Populations in Mental Health Services Research to Improve Public Health: Examples from the Field.” If you have any questions, please visit our global mental health website for more information. Now, I’ll turn it back over to Wendy to close out today’s webinar.

Webinar Operator: This does conclude today’s program. Thank you for your participation. You may disconnect at any time.