Center for Global Mental Health Research Webinar Series: Grants Management: Post Award

Transcript

MODERATOR: Hello and thank you for joining the Center for Global Mental Health Research 2022 Webinar Series. This session is being recorded and the recording will be made available at www.nimh.gov. And it is now my pleasure to turn the call over to Leo Cubillos. Please go ahead.

LEONARDO CUBILLOS: Thank you very much. Good morning and good afternoon, everyone. I'm Leonardo Cubillos. I'm the director of the Center for Global Mental Health Research at the National Institute of Mental Health. And I would like to welcome you to today's Webinar on Grants Management Post Award aspects of Grants Management.

This is the seventh and last webinar in our 2022 Center for Global Mental Health webinar series. Today's webinar will be hosted by my colleague Tamara Kees and a host of wonderful Grant Management officer specialists who will explain the post award processes and will walk us through how we manage a grant once the NIMH has awarded it. Some of the talk will be ranging from submitting the annual Research Performance Progress Report that we call RPPR to closing out the award once the research has been completed.

As was announced before, these webinars are archived so that you can view them and study them and take notes. The link will be at the end of this presentation, and it is also at the Center's web page. We are uploading them to NIMH webpage, and they are also available in YouTube. So, please study them, disseminate them. We've made them and prepared them for your own benefit. And lastly, the 2023 webinar series will be announced soon.

So, today's main presenter is my colleague, Tamara Kees. Tamara joined us during the webinar we had in May, when Tamara and Heather Weiss presented the pre-award aspects of grants management, and today, again, they will work on the post-award aspects.

Tamara began her NIMH career at the National Institute of Allergy and Infectious Diseases in 2003, where she managed an important and complex portfolio of foreign and domestic grants that included several trials over 40 sites in over a dozen of countries in different continents of the world. Tamara is now a supervisory Grants Management specialist and team leader at NIMH, where she supervises a team of specialists across different portfolios located in different areas of experimental divisions within NIMH.

Tamara will be introducing her colleagues as they present today's webinar. And before I close, I want to thank you all for attending this 2022 webinar series. This has been a delight, a true joy to put it together and to see this wonderful response from the public. I hope you find it useful, and I hope you keep revisiting these videos and disseminating them.

I want to especially thank my colleagues from NIMH, who put this together, the people that are working behind the scenes backstage, as well as the presenters that are on camera. And also, to my colleagues from Bizzell that made this webinar series possible.

So, Tamara, thank you to you and your team for preparing this webinar. And over to you.

TAMARA KEES: Thank you so much. Good morning, everyone. I appreciate everyone who is joining the webinar. Thank you for joining us. And of course, I'm Tamara Kees. And hopefully some of you remember me from the pre-award presentation that Heather Weiss and I did back in May.

Today, I'm happy to introduce three of my colleagues who will also be presenting. Rita Sisco is a supervisory lead here within our grants management branch, and she'll be presenting prior approvals. And I have Bob Munk and Julie Bergerud, who will co-present on the Research Performance Progress Report or RPPR, and they are senior specialists within our branch, and I was just very happy to have them join me because being a supervisor, I don't work on grants as intensely or as often. I'm more above things, working out problems going to meetings, and so having boots on the ground, people who really know what they're doing when it comes to the mechanics and working through a grant is very important. And their knowledge and experience is much appreciated by me.

Not to say I don't know policy; I actually do know policy. However, when you have to get into a grant and actually go through the systems and work with the PIs, trying to help them understand what they're seeing on their end and what we're seeing on our end, it's just very, very different. So, thank you all for joining me today and I guess we can move on.

So, basically now that you have your notice of award, almost all PIs, they go into their lab, and they start working on their research because they work so hard to get their grant and get their notice of award, and say, wow, I've got this money and I want to get going. But there are a ton of different things that need to happen when you're managing an NIH grant. And not all of it will be your responsibility as the PI.

The institution is given the money for your grant. And so, the NIH gives money to an institution. And the reason we do that is because it's unlikely the entire institution is going to pick up and walk away. We have a steady, for sure place where we can go when problems arise, or anything comes up. So, the money is with your institution and that's fine if you are a big institution and you have a huge, sponsored research office. The problem there is that they manage a lot of grants. So, it's very difficult to, I think, keep an eye on every single thing that's going on all the time.

However, there is the opposite end. When you come from a very small institution, you may only have like one or two people who are managing the grant for the institution. So, I believe it's very important for our PIs to have an understanding of what the post award requirements are. And it has been my experience that the PIs that understand a little bit of policy - you don't have to know all of it. In fact, I tell my staff always, you don't have to keep every little thing in your head. If you can do that, great. But if you know where to go to find the information, that is the most important thing that you can know.

And also, just being aware, just reading through the policy so that if something comes up, something in the back of your head will say, hey, I better look into this because I think I need to do something more. So, it is important for the PI to have an idea of what the responsibilities are for his administrative office.

Monitoring expenses. What do I mean by this? First, every NIH grant recipient must have an accounting system in place. Everybody does. If they don't, they get one. And you want to be able to track expenses. But not only that. You have an accounting system. You have money coming in, you have money going out. Somebody is doing the math. It's more than that when you're working on your grant. You want to make sure that the invoices that you're paying are allowable.

And so, I put here in this slide the NIH Grants Policy Statement, Chapter 7, Cost Considerations. It is a laundry list of allowable costs and how you apply them. And I recommend that anyone who is working on a grant visit that page, because if you ever have a question - can I do this, can I provide meals, can I buy a piece of equipment, can I hire this postdoc instead of that postdoc, can I replace a PI - all of that stuff is in the Grant's Management Policy Manual. So, that is where you will find the cost principles. And it's an A-to-Z list of all of the allowable costs and the applicable policies that go with them. And having that idea as a PI in your lab, will enable you to complete your research without a lot of trouble.

If you buy something and it's not allowed, what happens if that happens? Well, the institution has to reimburse us or reimburse the grant. And that whole process can take a long time to work out, which impacts your science. So, knowing what your responsibilities are and what your institution's responsibilities are, having an idea is a really good thing and it will help you in the long run, even though it's very tedious when you first start.

So, your institution is going to have to create a payment management system account. That is done usually by the finance officer. And you have a contact within the Office of Financial Management who will assist you with any issues that come up. So, the payment management system allows your institution to draw down funds to pay your invoices. So, the system will tell you how much money you have, when it was put into your account, and tracks every drawdown that your institution makes.

So, the thing you need to know from our perspective when it comes to the PMS system is we have limited knowledge about how it is set up within the Office of Financial Management. It is handled by the Department of Health and Human Services, and we don't have access to go in and make any changes. All of that is done between the institution and the Office of Financial Management. We have a report that we look at and we can see what you're drawing down, but we can't fix it. So, if you're having issues with the system, you can contact us, but what we're going to say is you need to contact your coordinator or contact person, point of contact person within the Payment Management Office.

Once you have that account, it enables us, the grants managers to review your expenditures. So, the NIH monitors your grant from different aspects. We have the programmatic side where the PI and the program officer discuss the science. I guess we don't get involved in that because we're administrators. But you work through any issues that come up scientifically with your program officer. We as, grant specialists, look at your RPPR to ensure that you are spending your money in the way that you said you were going to when you applied, and we funded your grant. So, we review your expenditures to see how you're drawing down funds.

We look to see if you have a large unobligated balance. If you do have a large unobligated balance, we look within the RPPR to see if there is a reason that we can spot, to understand why you may have that unobligated balance. If we can't figure that out, we go to the program officer and we say, what's going on in the science right now that would make them have this unobligated balance?

We had one country that had a tsunami that couldn't do any research for a long time. We didn't know about it because we didn't hear about it. But the program officer knew because she had been in touch with the PI. So, we're looking at your PMS account to see if you have consistent drawdowns, if you have unobligated balance, or if you have accelerated spending. If you're spending a lot of money - let's say, in the second quarter, you spend more money than you did in the entire last year of your grant, we're going to want to know why. Why is that happening?

So, the first thing we do is look at the science to see what is going on. Did you all of a sudden need to do MRI scans? Did you have a piece of equipment break down? Did we do a prior approval for that? So, we're looking at your finances just to see if you're being consistent with your drawdowns, which tells us that your science is being supported in a responsible manner. So, monitoring the expenses really takes the PI, the grants management specialist, the program officer, and the institution. We all need to look at it to know and handle that money in the best way possible. It's a partnership.

So, with that, that's really all I have about monitoring expenditures. The best thing I can tell you from this is look at the NIH Grants Policy Statement under Cost Considerations and review it to see what it is that's allowed, what isn't allowed so you have just a bit of knowledge in your head, and you know exactly where to go if you have a question that comes up that your kind of iffy on.

So, with that, I'm going to hand it over to Rita Sisco, and she's going to go over prior approvals. And I think all of these, especially prior approvals, it's really great for the PI to have an idea about what they need to do when they want to make a change within their grant.

So, Rita, thank you for being here today and take it away.

RITA SISCO: Hi. Again, I'm Rita Sisco. And we have 17 required post-award prior approvals. These are the 17 required prior approvals. I'm not going to go over all of them, but I'm going to just name several that we see very often. Change in scope. You may not think that there is change in scope, but there may be any time that you add a different colony of mice or rats or any animals, human subjects, you go from children to adults, that would be considered a change of scope. Additional no cost extensions other than the first which is automatic. Carry over requests. Change in PI for key personnel. Change in recipient organization. Questions about pre-award cost greater than 90 days.

Administrative supplements. Extension with funds. Any time that you add a new foreign component, you need to contact us because we have to get prior approval from the FACs(?) Office, which is the Foreign State Department. Any time you deviate from your notice of award terms. Retention of research grant salary where you have received a K award. Some awards based on fixed amounts, so re-budgeting from one to another because what was reviewed and approved was a fixed amount.

Requests for prior approvals must be received in writing or email at least 30 days before proposed change. It must be signed by the AOR and approved by GMO, which is us, Grants Management Office, with concurrence of the program officer. As Tamara said, we work as a partnership. The IC should comply with the timeframe specified. Remember the 30 days.

Other indicators of change of scope would be significant re-budgeting. Transfer of performance of substantial programmatic work to a third party, patient care costs, if not previously approved, and the purchase of equipment greater than $25,000.

Carryover of unobligated balances. Both Grants Management and program staff review this request for approval. We look for if it will benefit the proposed research, if it duplicates the funding that we've already provided. And program will signify if there is an expansion. We also look at, will the approval generate a recurrent cost in future years? Now some ICs routinely offset. NIMH, we do not routinely do this, but we will if we continue to see that the research is progressing and there is not a reason for it.

Second no-cost extensions. So, one of the things I need to make clear is that we don't grant second no-cost extensions if there is no money remaining in the grant. The time required to ensure adequate completion of original approved project, no additional funds required, but funds still exist, the scope of project will not change, and the recipient agrees to update all required assurances. All terms and conditions of the award apply during the extended period.

We often see a change of status of PI, and this can be when there is a MPI and other key personnel that's named on the notice of award. So, withdrawal of a PI or key personnel named on the NOA, absence for a period of three continuous months, reduction of effort by 25 percent or more of the level approved at award includes any time there is a change where it will impact his or her ability to carry out the approved research.

Change of institution, which is the original recipient organization. It is wise to contact your PO to let them know that you are thinking about changing, moving to a new institution. And just because you choose to do so doesn't mean that we will automatically award the grant, or the old recipient will release that grant to a new recipient because you chose to move. So, what is needed? The Official Statement Relinquishing Interest and Rights that the grant should be submitted electronically through the eRA Commons and ultimately a Final Invention Statement and final financial report.

What is needed from the new institution? As soon as the old institution relinquishes the grant, you should start having your AOR work on the new application. So, the face page with appropriate approvals, budget pages with narrative budget information based on previously committed direct costs, updated bios and other support, statement indicating status of research plans, resource page, equipment being moved, checklists, and if appropriate, updated human and animal approvals and electronic submission.

TAMARA KEES: Thank you very much for presenting those prior approvals. What I would like to say or add is why is it important for the PI to know exactly all of those prior approvals? As we just said, the ones we mostly see are second, no cost extensions, transfers, carryovers and change of PIs. The reason is because - let's take transferring to a new institution. If you're going to move, that means that your research is probably going to be on hold while you make your transition to the new institution. And it can take months and months. Sometimes I've seen it take over a year because of the administrative adjustments that need to be made on the PI side.

Sometimes PIs will leave their current institution because they're unhappy and they don't want to go back to the original institution and say, hey, I'm leaving, and they want to secure everything first. So, all of this stuff is happening behind the scenes and once you're ready, we then have to go in and do all the administrative things to make sure you're in compliance. Because when you move to a new institution, you have to have another IRB for your human subjects. We have to ensure that we have an IACUC and that the animal work is going on the way that it was supposed to be going on at the old institution. So, there are a ton of different things that go on behind the scenes.

If it takes a year to transfer a grant, that means your science is a year behind and it can be very problematic. So, any prior approval that you have, it's going to take a little while once it gets to us, because we also manage over 3,000 grants a year and prior approvals are not added to that total. So, these specialists are working the regular work and doing the prior approval. So, sometimes it takes, we say 30 days, sometimes it takes a little longer than that. We try for 30 days.

The best way to circumvent having those issues that stall your research is to have open communication with your program officer and your grant specialist. You need to let us know what you're doing so that we can tell you exactly what you need to send to us, so that when we get that package, we can then move on it and move you in a timely fashion. So, prior approvals are important to be aware of.

Another issue that comes up, if you change something in your research and you don't let us know about it and it was a prior approval, we then have to go back into your grant and look to see what you were supposed to do. And if you didn't have the compliance issues already settled, that impacts the dollars of your grant. And depending on how egregious it is, it can really impact your science if you have to throw out data because you didn't have the correct compliance issues all hammered out at your new place or actually with any change of scope. So, do go in and just look over those prior approvals so you have that in the back of your head.

And also visit your sponsored research office because they're supposed to have policies in place to help guide you through the process. If you're doing a transfer and it's not amicable on the side with your original institution, you can work with your new administrator to ensure that all of those compliance issues and regulations are followed from the get-go. And yeah, it's just really very important for you to have an awareness of these.

So, thanks very much, Rita, for representing the prior approvals and telling them what they need to submit to have those go through. So, now I'm going to hand it over to Bob Munk and Julie Bergerud, and they're going to be going over what we call the RPPR. And as I said, they're very versed at this and have a lot of knowledge.

Let me see if there are any questions. The first question is, will the slides be uploaded? And the answer is yes. So, thank you, Shirley, for asking that question. And another question is, I was a grant specialist for the Texas Health and Human Services Commission and managed a federal Medicare grant for three years. The information today speaks to reminds me about all intricacies of federal grants management as a whole. There is much transferable information seen here. Thank you. Well, thank you. We appreciate that.

With that, I'm going to hand it over to Bob and Julie. Thank you both. I appreciate your help with this.

BOB MUNK: Well, I'm Bob Munk. Julie, I'm going to let you go ahead and start.

JULIE BERGERUD: Hi, I'm Julie Bergerud. Good morning, everyone. Great to be here. Bob and I are going to go through the RPPR. We're going to hopefully give some good tips about what can delay the next notice of award that's coming, things to look out for, and really top five or six things that are often missed when you're going through the RPPR.

So, first, what is RPPR? The RPPR is how grantees submit progress reports to NIH. The RPPR is required for all annual, noncompeting NIH awards, otherwise known as Type 5s. Would you probably seen that number in your notice of awards? So, the progress report must be submitted electronically through the eRA Commons. We switched over to electronic years ago, so everybody should be in pretty good shape, understanding that.

The progress report allows the grantee to document both accomplishments and compliance with the terms of award from the NOA. The RPPR Instruction Guide, which we have linked up with the slides and are available easily, that's the resource that grantees use to complete the RPPR. It's super helpful and has detailed information for you to refer back to and as you're going through each question in the RPPR, it is really helpful, and we'll touch on a lot of that as we go through these slides. And again, is at the end of the PowerPoint, you can link up to that.

So, what is a non-competing application? So, this is the progress report required to continue support of your grant for the second or subsequent budget period within an approved competitive segment. So, as you can see with this slide, your first competing award starts with a one that's your type one competing award. And as you're submitting these RPPRs, you're going to see Type 5 R01 and your grant number. So, that's the next year or subsequent year that you'll be reporting on.

So, the progress report is also referred to as a Type 5 or RPPR. The RPPR reports on the past year's progress, but the Type 5 application RPPR is considered the request for the next year's funding. So, this is mandatory. You have to submit the request. That's a request for funding. If you don't, then we can't make the award. So, sometimes grantees submit late, and we give them a gentle reminder that they have to submit it with a certain within a certain amount of time. And we have a slide for that. But you want to make sure you get this request in, because that's the only way we can actually issue that next notice of award.

RPPR submission. Keep in mind there is no RPPR form available for download. The grantee submits our paper through the eRA Commons. That's the route to take. The PI can delegate their responsibility to another person to initiate the RPPR. The signing officials typically submit the RPPR but may delegate it off to another PI within the organization to prepare the RPPRs. So, it's kind of a shared role, but the AOR submits it through the system, but more than one person can work on it and prepare it.

So, when is the RPPR due? There are few ways to determine when the progress report is due. Grantees can look in the eRA Commons under the status section, or they can check the pending progress reports area within the eRA Commons. There is also an automatic email that goes through to the PIs, and I believe the AORs, letting you know that your due date is coming up. So, that's also helpful.

But to get into a little more of the details of the timing for the due dates, if your grant is a SNAP grant, a streamline noncompeting, the progress reports are due about 45 days before the next budget period's start date. If your grant is a non-SNAP, which is the more complex grants which require a budget, we're going to ask that they are submitted 60 days before the budget period start date. So, the reason for that is we're going to be looking at the budget, Program and Grant's management are going to be having a more thorough review and a little bit more complexity to those grants that are non-SNAP. So, we need just a little bit more time to look at those.

The exact start date for a specific award may be found in the status section of the eRA Commons. So, keep that in mind. Now I'm going to pass it on to Bob to discuss SNAP versus non-SNAP.

BOB MUNK: So, it used to be in olden days that all grants were non-SNAP grants. And so, the basic understanding with SNAP versus non-SNAP is that with SNAP awards, NIH lessened the burden of reporting. And it's for basically what's considered to be more simple grants. The more complex grants like P Awards, cooperative agreements, training grants, small business grants, clinical trials, and also fellowships or any other award that requires closer monitoring, those are now still non-SNAP grants. SNAP, by the way, stands for streamline noncompeting award processes or procedures.

The NIH streamlined the procedure for your Type 5 reporting. Most awards now are SNAP grants. Most R awards and K awards that don't require close project monitoring. And SNAP grants do make up the large majority of the RPPRs that get submitted. But there is a substantial number that are also non-SNAP. SNAP grants don't have to submit a detailed budget, but non-SNAP grants do.

Julie talked about the submission date for a SNAP grant is 45 days prior to the start date, so that's a month and a half with a non-SNAP grant. It is about two months prior, 60 days. With a SNAP grant, generally speaking, there is automatic carryover authority, with a non-SNAP grant generally not having carryover authority and the FFR, the federal financial report with SNAP grants, those are submitted only at the end of the project. If it's a five-year project, the FFR is submitted at the end of the five years. But with a non-SNAP grant, the FFR gets submitted annually and gets reviewed annually.

How to submit. The recipient will follow the instructions in the NIH RPPR Instruction Guide, and we've provided the link here and at the end of these slides which you'll have access to. The user guide includes instructions for how to submit RPPRs and eRA commons and how to complete the web-based forms and what information is required. As mentioned earlier, this is not like a downloadable form. You have to go into eRA, and you fill out the forms as they come up in the system.

So, the RPPR obviously requests various types of information, including reporting on your accomplishments. So, what were the major goals and the objectives of the project? You're going to restate that. What was accomplished under the goal? What opportunities for training and professional development did the project provide, if any? How were the results disseminated to your communities of interest? And what do you plan to do during the next reporting period to accomplish those goals and objectives?

The RPPR also requests information on products such as publications - that's considered a product -conference papers, presentations, websites, internet sites, technologies or techniques, inventions, patent applications or licenses. Other products such as data or databases, physical collections, audio or video products, software models, educational aids or curricula, instruments or equipment, research material, interventions, clinical or educational, and new business creation.

Now we want to take a pause right here and talk about publications. It's important for you to know that you need to be current on submitting your publications to the PubMed system, because if your publications that are reported in your RPPR are not current, we can't issue the renewal award for your Type 5 until that happens. So, we see fairly frequently that that is a major source of delayed awards for Type 5s.

Also, I want to point out something that we see very frequently with data submissions. For those of you, and it's most, who have to submit to the NIMH data archive, you have to keep on top of this and you also have to report it in RPPR Section C5. Very frequently we see RPPR Section C5, it says nothing to report and that's not acceptable. And so, we go back to the grantee. And so that's a source of delay because it generates traffic back and forth while we get just a simple statement on the status of your data sharing and data submission.

So, it's not a complicated thing, but just be aware that it is a requirement, and you will see that in your notice of award. It's in Section 4. These are the NIMH terms of your notice of award and everyone from the AOR to the principal investigator should be reading the notice of award and paying particular attention to Section 4 that has the NIMH specific terms of award. And in there very often, if you have data sharing, if you signed a data sharing agreement, there is a term there that states that a statement on data sharing is a requirement in Section C5. So, please take note of that right now and please make sure that you're doing that because it'll save everybody some time in back-and-forth emails.

I also want to point out that if there is any kind of a change in protocol, either with human subjects or with animal subjects, that that should be reported in Section F of the RPPR under changes for human subjects or changes for animal subjects. Julie, did you want to add anything there?

JULIE BERGERUD: No, I think I'm ready to go on if you are. Now we're going to dive into the scientific section page of the RPPR and just give you some tips here. So, Section B1, if you answer yes for B1, then you're going to address the major goals and aims change, if that's what's going on with your grant. You would need to provide a revised description of the major goals and specific aims. This will help Grants Management be aware of changes that are coming into play and help us determine the next steps and questions to ask depending on what's going on. So, this may not be applicable if there are no changes, but this is the section you would place that information in if there is updates and changes of the goals and aims.

For B.2, you may need to include information here. Upload a document or a summary here is appropriate for B.3. This is a big one. This is where you are going to report any supplemental funding that you receive. This is important because this is a requirement that you take note and give us a little progress on what's going on with the supplements. If yes, you identify the revision or supplement by grant number or title, describe the aims and accomplishments for each supplement that was funded during the reporting period.

This section is also for diversity or reentry supplements, so you'll also describe the progress for those types of supplements and any other supplements given for any discrete project that you're working on. So, if the grants management specialist is going through your RPPR and we're always looking back at notice of awards and what the grant's all about, if we see that there is a supplement there, but we don't see anything addressed in B.3, we're going to have to pause and reach out to you guys, figure out the information we need so we can collect the progress on the supplement. That's just one of the delays that can happen if we don't have all the information up front in the RPPR. So, just think about that as you're working on your completion of the progress report.

Section B.4 and 5; these may or may not be applicable, but it's good to know. If the research is not intended to provide training or professional development opportunities, or there is nothing significant to report during the reporting period for B.4, it's okay to write nothing to report. The same goes for the next section, B.5. If there's nothing there that's applicable to you, then you can market nothing to report.

A detailed response is only required for awards or award components that are designed to disseminate information to the public or conduct similar outreach activities. So, you can read through those details in the instruction guide and then determine if that applies to you.

So, here Bob touched on this a little bit. Publications, the C.1 section and also, we'll talk about C.5, data sharing. This is a big one. As we review the RPPR, we've gotten a lot better over the years as the public access policy came into play over the last five or six years. The grantees are getting a lot better at knowing what to put there, but please know that if there is non-compliant publications there, that's going to put a stop to the review. We're going to have to wait for that to be resolved.

So, recipients must report publications in Section C.1 if, number one, the publication was accepted for publication or made public during the initial reporting period or the period since the last progress report was submitted. And number two, if the publication directly arises from the award. So, that's important to think about. And again, we cannot issue the Type 5 with an RPPR that has non-compliant publications or citations there.

The other sections here below C.1, they may or may not be applicable to your grant, so you can read through the instructions more carefully for these and see if you need to address any of those. But the other big one here, which we often find a conflict with when reviewing RPPRs, is the fact that C.5 could be left blank when there's data sharing, and we need a report for that. If the initial research plan addressed or the terms of award require a formal plan for sharing final research data, or model organisms, or other specific data, you have to describe the progress in implementing the plan in this section. Or if the sharing plan is fully implemented, provide a final statement on the data sharing in this Section C.5.

Like Bob said in your notice of award, when you look back at year one or the prior notice of award, there is going to be a term there, if this applies to you, and you need to report. There's a nice term that we have. It refers directly to C.5. And if you see that, that's a flag that you need to report the section in your RPPR. A lot of times specialists go back and ask for this, and it could delay the award for like a week if there's a holiday in there or something, or somebody's out the office. So, it's really important to try to include this upfront so program can see it and Grants Management can know if any other questions need to be addressed with it.

This is just a helpful slide. I used this site on the public access website years ago a lot when I was trying to figure out if the grantee had PMC ID numbers ready. So, I thought I would just share this. There's a link at the top that's active, so when you pull up the slides, you should be able to get to it. It's a converter and all you need to put in is the PM ID number and it will tell you right away if the publication has gone through the full review and it has a PMC ID number, which means it would be ready and then in turn be compliant.

This was helpful for our group, so I just wanted to share it. And if you guys want to go in, if you didn't realize that was there and you could quickly maybe pick up a citation and see if it went through the process if you can't easily see it any other way. Because I'm not sure how you can view it on your end exactly, but I thought I would just share that for you.

Going into a different direction here. Bob and I are going to talk about participants and effort. So, this first section is D1 participants and it's a table format which you've probably seen. And we just wanted to note that it's mostly retrospective, so the participants table is for the prior reporting period. It contains all personnel who worked at least one person month on the project in the past year. And you should remember to take note that PIs, postdocs, and students require the Commons ID to be listed. This is a link that talks about it in the slides that you can see. And the Commons ID is also required for individuals supported by a reentry or diversity supplement. So, they would also be included to make sure that Commons ID is there. All right, moving on to Bob to talk about the next slide.

BOB MUNK: Section D.2 A has to do with level of effort. And the question is, will there be in the next budget period, looking forward, either a reduction of 25 percent or more in the level of effort from what was approved by the agency for the PD/PI, or other senior key personnel designated in the notice of award. That's important to note that we track PI, NPIs, and anybody that is named in Section 4 of the terms again, of your notice of award. Or a reduction in the level of effort below the minimum amount of effort required by the notice of award. Again, Section 4 of your terms in your notice of award.

There are some grants that have a minimum required effort level. If the effort is planned to be below that, that would be a prior approval requirement. NIH Grants Management checks the reported effort and what's projected here in Section D.2 A against the level that was last approved. Usually that is from a competing award. And we do check not only the level that was approved for year one, but also say this is the RPPR for the upcoming Year 4.

We would go into the original awarded spreadsheet and look at what was the effort level that was committed for Year 4 of the project. And then we're looking to see, is the planned effort going to be a reduction of 25 percent or more from that approved level. Now, it's important to note here that if there was a request for a reduction in effort or a prior approval request that came in, that was approved by Program and Grants Management after the initial award. Say this happened in Year 3. We also looked to see if there was a subsequent approval of a reduction in effort. And then that is the new level against which we are basing our analysis.

Here's again looking at Section D. We already talked about the participants report in D.1. We talked about D.2 A, level of effort. In section D.2 B, it says new senior key personnel are there. Will there be any new senior or key personnel, if so, these need to be reported here. D.2 C is changes in other support. If there's been changes in other support, that gets noted here and then there will be a file upload that we'll be looking for that has the other support. And we're going to go into that in more detail in a moment. Section D.2 D is new, other significant contributors. Those would be reported here D.2 E, is there a multi-PI, NPI leadership plan. Any changes to that, that would get reported there.

D.2 C, other support. There are new guidelines that everyone should be aware of by now. These were updated January 25, 2022, and the guide notice is NOT-OD-21-110. If you're not familiar with that, you need to be familiar with that. These new reporting guidelines are significant changes from how this was reported in the past. And if you don't abide by these guidelines, we have to reject the other support out of hand, and we will write to you and require that revised other support be submitted. Just backing up a step in general, other support on a Type 5 is requested for all senior key personnel, but not consultants that are in the progress report when there has been a change in their active other support.

Additional updated requirements for recipients in guide notice OD21-O73 again, effective January 25th, 2022, requires the following. That there be supporting documentation which includes copies of contracts and agreements specific to senior key personnel for foreign appointments and or employment with a foreign institution for all foreign activities. And keep in mind that foreign is defined as not United States, and resources that are reported in other support. If the contracts agreements are not in English, recipients must provide a translated copy.

There must be immediate notification of undisclosed other support when that is discovered. When a recipient organization discovers that a PI or senior or key personnel on an active NIH grant failed to disclose other support information outside of just in time or the RPPR as applicable, the recipient must submit updated other support to the grants management specialist named in the notice award as soon as it becomes known. Julie, did you want to follow up on this one?

JULIE BERGERUD: Thanks, Bob. Yeah, I'm just going to add a few more things because the guide notice for this and the policy is still fairly new to our group as well. The past year we were looking at the other support with adding a few different things. And we're still learning and getting used to the policy as well. So, I wanted to share just a few extra things.

We look at the foreign affiliations and appointments and employment for any single investigator. NIH will need additional documentation for foreign affiliation if it's an entity that is not officially part of the grant. In other words, copies of agreements are only required for outside agreements directly between the senior key personnel for foreign institutions, not those managed by the home institution.

What your grant specialist will question will be any outside affiliations that appear to be involved as they're looking through the other support. In addition to the specialist looking at the budgetary overlap as we're going through other support, we're also reviewing what is listed for in-kind resources. This is new. We're still getting adjusted to doing this and knowing when to ask questions. Keep this in mind, and it's all in the guidance under the new other support page. You can see the FAQs and visit there to get more details.

In-kind includes office lab space, equipment, supplies or employees or students supported by an outside source. For example, just to give you an example of what goes through our minds as we're looking at the other supports and looking at your RPPR. If equipment is listed within the in-kind section and we see the same equipment in the budget, that's when we're going to reach out and say, okay, let's talk about this. Should this equipment be truly for in-kind? Is that the way you want to go with it or are we going to support it in the application budget? That's something to think about.

Our goal is to make sure that that equipment isn't paid for twice, supported in both places. As Bob mentioned before, recipients are required to tell us within 30 days if you picked up anything new that needs to be added to the other support. Anything that is an outside appointment that you didn't bring up before. You shouldn't wait until the next RPPR is due because that could be a long time, so make sure you do that promptly.

The next slide is Section E. If you look, E.1 through E.3, you can fill out as it applies to your grant. If you have any impact on technology transfer, development of human resources, this is where you would report it. This is kind of rare for us to see as we go through. We don't send back a lot of questions on this section that I've seen in the past couple of years. But for E.4, as you can see, that is a section you definitely want to fill out. It's going to tell us what dollar amount of the awards budget is being spent in foreign countries.

You should include the prime grantee country as well as any foreign subcontracts involved and show the dollar amounts for each. This is helpful because right away if you have a new site, a new foreign component added right away, we'll be able to see that as we compare what was going on in the prior year. That alone will tell us a lot of information there and then we'll be able to see the dollar amounts as well.

Section F, changes. This is where you can describe any delays to your grant. For F.2, you can describe challenges or delays encountered during the reporting period and actions or plans to resolve them. You'll describe significant challenges that may impede the research. For example, issues gathering patients, hiring personnel, maybe you need more resources. And you can emphasize the issues here and then present a resolution. This is a spot to do that.

Over the past two years, the common theme for delays was obviously COVID and everything going on with the pandemic. You can see an example slide here. There is a summary example, what a grantee wrote for that. For this section, it's really important to include a plan. This will help begin the discussion with programing grants, management, so we'll know what's going on and if further details need to be mapped out between program and the PI for the case that's going on. We'll be able to determine what to do for the next funding period and how to help you continue your research. Lastly, Section F.3 allows you the opportunity to point out any changes in human subject, animal work or biohazards, as Bob discussed as well. You're going to want to point out any changes in human subjects or animal work, and this is the main section to do that in.

This next slide is humans, animals, and performance sites. So, we won't dig in too deep here, but this is a section where we'll be able to tell very quickly, is there a delayed onset, yes, or no. You'll mark that. It'll indicate clinical trial, so it's helpful to see right away. Just looking at the prior award and previous grant correspondence, the specialists will know if there has been a change in either one of these and then if there is, we can address that or maybe there is a discrepancy that we have to address. It's important to mark this G.4 section.

If you have personnel on the project who are now newly involved in human subject research that would be placed in G five. And that's important because the grant specialists we will collect will want to know if human subject certifications are done. We won't necessarily need to collect them, but this is a spot to list X, Y and Z personnel are now involved and upload the documentation that they've done the certifications.

And then there's the stem cell section. If you have those, you can list the stem cell line there. If it's new, that will signal the grants management special specialists to talk to program and just make sure everybody's on the same page with the use of stem cell lines. And if that's the case, we'll know to place the appropriate term on the award, which you will see when you get your NOA. And then, of course, if there is an animal change, you can address it in G.7.

And now we just thought we would dive in a little bit more into what happens when you have changes in human subjects, if there is a protocol change, just a general overview. Any change in research procedures in an act of award that would result in increased risk to human subjects needs to be discussed with your program officer. I would go to the program officer first, keeping these items in mind, which are just a few of them from a recent guide notice that we have linked up that you can look at later.

For example, are you adding or changing a study designer protocol? Maybe there is new inclusion of subject populations. Are there changes in study protocol resulting in an overall increase in the risk of human subjects? Those are the things to think about, and you can review the complete guide notice, which is OD15182. You can take a look at that. The updated notice provides detailed guidance on the types of changes in human subject research that will require prior approval and provides information on the process for submission of these requests.

Rita talked about prior approvals. This takes a little bit one step further, addressing human subject changes and it goes into detail about what you would need to reach out to us for prior approval and how to submit that request. On another note, remember, if there's changes in protocol, you often need to wait for the IRB to give another approval and to review, so keep that in mind.

And again, it's really important to bring up any changes to the program officer early on. This way we can know what questions to ask in advance and to move the process along further because sometimes there's internal checks we have to do when there's changes in protocol or changes in human subjects, and then we get feedback from the grantee. It's just important to keep program in the loop. Think about your I IRB approval and if that needs to be updated and follow that guide notice and that should help a lot.

TAMARA KEES: Can I just interject here quickly? So, the NIMH does our research here. We do mostly human subjects research. It's mental health. So, that is it's really hard to do research on animals. We can do it. But most of the time we have clinical trials and human subjects research. If you're going to make any changes at all and it was not approved, as Julie said, you need to let your program officer know right away and your grant specialist.

We have an office called the Office of Clinical Research. When we have changes in our status of human subjects, we have to feed all that information into a database. And then they have people on the other side to say yes or no. This is appropriate for this study. So, let's just go to where I'm going with this, which is if you start something that was not approved, all of the money that you invest in that process has to be reimbursed to the grant. And if you received any data from that research, it has to be thrown out because you didn't have the proper approvals. And that's just US federal law when it comes to human subjects research.

The takeaway from the human subjects work here is that we really stress if you are going to make any changes that were not approved and could alter the direction. Let's say you have an approval for children 10 to 13 and you decide that, well, hey, I'd be able to get this information if I changed the risk from 10 to 13 to 7 to 13. That's a big step. And because they're high risk, it's even more intense when we send it through our Office of Clinical Research. So, just be very aware of this, and thanks, Julie. You did a great job explaining it all.

JULIE BERGERUD: Okay, great. Thanks, Tamara. Now we're going to go on to the next section. Bob.

BOB MUNK: Thanks Julie. Just continuing on now with Section G. We've got our project and performance sites. All performance sites that are current for the current year should be listed here including the parent and consortia sites. And I should note here that the NIH allows grantees the authority to change consortia sites without prior approval as long as it's domestic United States consortium sites.

If there is a change in consortium site that involves a foreign entity and again, that's foreign to the United States, then that is a prior approval issue. And generally speaking, we have to put those through our foreign award component tracking system. That's facts, which was mentioned earlier and very frequently, those changes need to get approved by the United States State Department. Any change in a foreign component is a big deal and it needs to be noted, and that's a prior approval issue. So, generally speaking, we should not be finding about a change like that in the RPPR. It should be coming to us as a prior approval request beforehand. Section G.9, that's where all foreign components should be listed and identified with a description. And again, if it's a two foreign award, the parent grantee would be a foreign component. And again, foreign component is defined as outside the United States.

Section G. This is a big one for Grants Management. The estimated unobligated balance. Is it anticipated that an estimated unobligated balance, including prior year carryover, will be greater than 25 percent of the current year's total approved budget? And there is a definition for the current year's total approved budget, which on a SNAP award includes prior year carryover. The definition for total year's approved budget is what was authorized as the new award, plus any authorized carryover. On a SNAP award, all carryover is authorized, so that gets factored in.

And if that balance is going to be greater than 25 percent per year, best estimate, then that should be reported here to give Grants Management the best available information that you have, along with a description of the causes that's in Section G.10 B, provide an explanation for the unobligated balance. And in section G.10 C, if authorized to carry over the balance, provide a general description of how it is anticipated that the funds will be spent.

In short, what grants management wants to know is if there is a large unobligated balance, why did it happen? Does it make sense that it happened? Is it within reason? And then if we authorize that carryover greater than 25 percent, what are you going to do with it? We're looking to see, is the plan for the expenditure of that carryover. Is that in line with the original aims of the research? Is it of benefit to the grant or not?

Generally speaking, the more information that you can provide Grants Management here in this action, the easier it is for us to do our analysis and to streamline the awarding procedure. A lot of times, I'll be honest with you, we get a response that says no, and we have a reporting system that we can check. And it's very easy for us to do so. And it's almost to the day now we call it the QVR, which is Query View Report, balance report. And it gives us the reported expenditures on the entire award up until the last drawdown now.

We can see at a quick glance what your actual balance is. And so, we're looking for correspondence between what's being reported here in the RPPR and what we're seeing in our system. Now, of course, there is going to be some variation because the RPPR gets submitted 45 days prior to the start date of the next Type 5, and we're going to be reviewing this within the month prior to that start date. So, there is going to be some minor variation, but we're looking for some consistency.

If we see that you have an unobligated balance that's like 90 percent of the last year's award and you reported, no, that there is not going to be a balance greater than 25 percent, that's going to raise questions. And we're going to be following up on that to say, okay, please tell us what your reported balance is. That's going to generate email traffic back and forth in order to get this resolved to where we feel comfortable, and our program officers feel comfortable with the justified, bona fide need for making the next installment on your award. Again, the bottom-line overview here is that it it's better to be just as completely transparent as you can be. Don't report no if it's not no.

Now, sometimes there is differences in interpretation, some of which are okay and some of which are not. Some grants management specialists don't take into account that prior year carryover is part of the definition of what's authorized. And so that might have to be explained and worked out. Sometimes the grantee will assume that what is authorized is all of the years of the award, which is not the case. You're not comparing your unobligated balance to four years of awarded funding. No, that's not it. It's what was awarded last year, plus any authorized carryover from prior years. Moving on, Section J.11 is on program income. We generally don't have a lot of program income on mental health grants, but if you did, that's where it would be reported. Section G.12 is for changes in performance sites that will affect FNA costs. You would report that there.

This slide has to do with the budget. This is for non-SNAP grants. Remember a SNAP grant streamline noncompeting award procedure, you don't have to provide budget information on your Type 5 RPPR. But if it is a non-SNAP grant, then yes, you do provide your budget information for the current upcoming year. And we compare that budget information to what was initially awarded committed for the current upcoming year.

What Grants Management is looking for here is, are there significant changes. Are there any red flags that would indicate that the grantee is going off in a different direction from what the original plan was? And if we see that, then we're going to be checking in with our program officer and getting their analysis of what they see in these budget figures.

And is this part of an ongoing conversation that the program officer has been having with the principal investigator? Does this make sense to them? And if it does, then probably that's just fine. But if it doesn't, then it's a potential red flag indicator where we're going to be having discussions with the grantee to say, hey, what's going on here, we need to understand this. And we may be not in agreement. We may be putting special terms in the notice of award that restricts funding so that the funds are being spent according to the original plan.

We don't have too much problem with this, at least not in my experience, but this is what we're looking for. In terms of significant re budgeting, the definition of significant re-budgeting is actually 25 percent, not of any individual category, but of the total funds awarded. If you had a grant that was $1 million total cost, significant re-budgeting would be represented by $250,000 of that $1 million total cost. It's very rare that we see that.

Some additional points here. Prior approval is only required for re-budgeting.

Specifically talking about re-budgeting, you only need to ask for prior approval for re-budgeting as it's defined in the grants policy statement. If it is an indicator of change in scope or if the re-budgeting of funds are going towards a new foreign site.

We've already talked about a new foreign component is a prior approval issue. Change of scope is a prior approval issue. For NIH, generally speaking, re-budgeting is not a prior approval issue unless it's an indicator of a change in scope or there is a foreign component involved. That's point number one. The second point, your FFR the federal financial report. If you have a non-SNAP grant, an FFR is required every year and we review it every year. If you have a SNAP grant, like I said previously, it only gets reviewed at the end of every grant segment, which is your five-year project period or your three-year project period, whatever length of time was initially approved. The FFR on a SNAP grant only gets submitted that one time at the end of the project period.

Next item, a no-cost extension. If you have a no-cost extension, a progress report and

RPPR is not required to be submitted during the no-cost extension, only at the end of the grant period when the recipient files the final progress report or the FRPPR. And that's during the closeout process. But I should also note that this is assuming that there is not a special term in the notice of award that does require a progress report. Usually, we call that an interim progress report in such a situation. It's not going in into eRA and filling out the entire RPPR, but it's taking elements of the RPPR and that's an interim progress report. Again, that will be clearly identified in Section 4 of your notice of award, so be sure that you're always reading Section 4 of your notice award every time you get one.

JULIE BERGERUD: Thanks, Bob. Our last slide for the RPPR is just to address the most common problems. Just to summarize, the items we talked about and the things that pop up the most frequently that delay the review of the award, incorrect invention reporting. If there's a discrepancy there, you're writing, yes, and we're pulling, up and we're not finding it. Or you say there is no inventions. And we're clearly seeing in the prior year that there were. That that could hold up, clearing up those inventions.

The second thing we talked about in detail, publications not being compliant. We want to make sure everything's compliant. And you can dig into the public access website for more information. And I'm sure the grantees know far more about the details of how to report in the My NCBI and My Bibliography. We didn't get into all those, but if you need to review those, you can definitely go into those resources.

The missing Section C.5 on data sharing is a big one. Make sure you always include information there if it's included in your grant. And again, as Bob said, Section 4 of the notice of award, that's where you're going to see that term, which is very helpful and a good reminder. Another area we talked about is the effort reductions. Make sure you're reporting if it's 25 percent or more and sharing that with us, so we don't have to go back and forth too much when we correspond about changes.

Other support, again, making sure that you're putting any pending grants that turned active, you're listing that and giving the updates within the RPPR submission, so we're not asking too many questions back and forth that would delay it. And again, we talked about changes in protocols for human subjects or animals and stressed the importance of for any change, please make sure you contact the program officer first because we may have to do some extra approvals. And most of the time there are extra steps that need to be taken before you can start new human subject research or any shifts in the work.

And finally, the large unobligated balances. You want to be as transparent as possible, as Bob mentioned. That's another one that can delay the award as we're discussing details of the RPPR. And that is it for the RPPR. Next slide, there's resources for you. The RPPR instruction guide, of course, provides really great details of answering the questions and gives great definitions and has information there for you. There's FAQs links posted here. The other support link which you should familiarize yourself with if you haven't read the latest other support guidance. That's super important. And of course, the link to the grants policy statement. So, you can have that all there to dive into during your free time. And that is it for the RPPR.

TAMARA KEES: We do have a couple of questions. The first one I answered online, and it says, from Shirley, is there equitable weight in reporting. This is for data sharing. Both quantitative and qualitative information with report submissions. We don't get into the nitty gritty of what's included in the data sharing report. We rely on our program staff to review it and make sure that whatever you propose to do in your application is what you're reporting in the data sharing.

So, I actually don't know, and I asked our data sharing coordinator in Grants Management while Julie and Robert were going over their slides. And she said, it really does depend on the science in your application. And so, I put in the chat the website for the NIH data sharing website. And they have a box that you can submit your questions through. That would be your best bet.

Another option is to go to your program officer. If you're applying to an RFA or a program announcement from the NIMH, you can contact the program officer who is listed as the scientific project officer at the bottom. The very, very last page at the bottom is where you'll find all the contacts for NIMH or any other IC meaning institute within the NIH that is going to be providing funding to your grant. I'm sorry, we can't be more definitive about your question, but it is not something that we normally have to approve on our side. We have to make sure we get the information and that it is recorded in your official file. Another question is, is there an RPPR reporting template. Bob, you want to answer that question?

BOB MUNK: Let me get my mic here. I don't actually submit RPPRs. So, my understanding is, no, there is not a specific template. It's a web-based interface that you go into. And then, like we said, there's the instruction guide, and, in our links, we included the FAQs for the RPPR. There's a section, Chapter 8.4.1 in the NIH GPS that has to do with reporting. There's an RPPR overview link that we included. It's possible that there is something that can be downloaded, that there is not a template, but a sample. I'm not sure that there is or not, but my best advice is go with the information that we provided in those links.

TAMARA KEES: One thing that's important to note is that we share the eRA Commons with you, but we have different portals. So, anything that you do on your side will show up on our side as a link and then we can click on that link and bring up the documents that you share with us there. Same goes for the notice award and any data. I think they can see their FFR in there as well.

BOB MUNK: I was just going to say, it either shows up as a link or it shows up to us as a completed document, a PDF document. To us, it appears like it's a document and maybe that's why it appears that way to you, too. But my understanding is that when you're in the system from the grantee side, it's just a web interface, including the FFR. It's just questions that they're filling out as they as they go through step by step by step.

TAMARA KEES: We can see what you see, which is frustrating because we get a lot of questions about what you're dealing with on that side, and we have to send you to the hell desk and 9 times out of 10, they'll send you back to us. It is a merry go round that I apologize for, but that's just the way it is. We can't see what you see on your side.

BOB MUNK: And sometimes our best advice is just to tell you, look, do the best that you can do. And if we find that it's not sufficient, the consequence is that we're just going to get in contact with you and tell you what we need. It's not like if you make a goof or you make a mistake, that instantly your RPPR gets rejected and your funding gets denied. No, that's not what happens. That's not how it works. Our whole goal at NIH from review, to program, to Grants Management is the successful completion of your research project ideas, because that promotes our mission. We're a granting agency. Our whole focus is to help you be successful. It's not to punish you.

TAMARA KEES: So, we have a comment, and it says, the carryover request to no-cost extensions can be challenging as funding streams, according to project periods, must be kept separately. This is an area that requires repetitive conversations. Yes, definitely. We have a lot of back and forth when we get second no-cost extensions and carryover. I think that's what you mean. If not, maybe you can clarify just so we actually communicate to you what it is if we needed to respond there.

And we have another question. It says there is no template that I have found. It would be helpful to have one. If that's something that can be developed, it would be very helpful. Yes, I agree. We have people within our office who go to the meetings for the eRA Commons, so, we will definitely bring this up. They're making a lot of changes to the system, at least on our side right now. They may have something in the works that we're not aware of.

Generally, what happens is we all give our input and then they make changes to the system. And we're notified they're going to make the changes, and then we have to go in and figure out how to work it. Sometimes we have really great guidance and other times not so much. It's always a work in progress and they're constantly making changes and updates to streamline the whole system. And I think overall they have streamlined the system. It's just that it takes a while to get a handle on it.

I think we can move on to the auditing, if you want to put the slides up. So, I'm going to be closing out the conversation with audit reports, closeout, and record retention, all of which are just so exciting. I can't tell you. The audit requirements. This is something that your institution is required to do if you have U.S. federal funds if you expend more than 750,000 a year. It's 750,000 or more or more in one year, you're required to have an audit.

And on Grants Management side, we don't get involved in the audits. That's something that is taken care of at the grantee organization. The onus is on the institution to comply with this policy. And the only time that we ask for an audit is if we see systemic problems within the system in a sponsored research office. And usually if there's a problem there, it filters down to everybody else and in specific labs, the way that they are keeping track of people, participants that they're looking at in their grants. It's rare that we look at the audit report once it's submitted, but the auditor is chosen by the institution to come into their institution and review their accounting systems and their standard operating procedures to see if anyone is not doing or if they are doing it. And then they compile a report of their opinion of what's going on within your organization.

The audit package, and I'm going to just directly read this because this is exactly what needs to go in there. You need financial statements and schedule of expenditures on your federal awards. An independent auditor report with an opinion on the financial systems and schedules of expenditures. A report on compliance and internal controls over finance reporting. This would include a standard operating procedure where you have a training and all your PIs, and their staff come. And you say, in order for you to submit an invoice, you need to tell us where you went, how much it cost, and why you went there. And every invoice for any travels should include this information. The auditor is going to report on whether or not the PIs and their staff are following the standard practices within the institution.

I report on compliance and requirement applicable to each major program. I'm assuming this means that every program within the institution has another set of standard operating procedures. As with the government, we get our directives from the President or from Congress and all of their decisions filter down to the agencies. And then the DHHS will say, well, this is what we think the policy should be. And then they give it to the NIH, and the NIH says, well, this is how we're going to interpret this as well. So, I'm assuming it's the same kind of flow down as what we have within the federal government.

A schedule of findings and question costs. They have to document any question that comes up and then they have to file a correction plan which involves whoever is in charge in the sponsored research office. They have to come up with a way that they are going to fix whatever problem was found. And as I said, this is only required when you have $750,000 or more in one year. My goal would not be to only spend $600,000 a year. I don't know a PI out there that would say, hey, I don't want to have to do the audit, so I don't want any more grants. But it shouldn't be a scary thing. Most of our research institutions have very well-written standard operating procedures and the vast majority follow them. Occasionally we have an issue where we have to do a site visit because something isn't clear. But overall, the audit requirement is just a policy that all of our grantees have to follow.

Once you have the audit, it is filed here. I went to the website because honestly, I don't do this. The onus is always on the institution to ensure they're in compliance with audit requirements. So, I decided I would just put this up here for any administrators out there who have not filed an audit report but will be doing so in the future. Hopefully, this web page will be helpful.

We have an audit chapter in the grants policy statement, which explains exactly what needs to be done, when it needs to be done and how it needs to be done. And if you have any problems, you can always give us a call. But honestly, I have never been contacted about this, so I'm just letting you know that there is a requirement for audits.

Close out requirements. This one I know because I'm the coordinator in NIMH for closeout, and they have changed the policies over the years. I wouldn't say most recent years. The interim research progress report was implemented in 2017, but prior to that, our closeout process changed, and they introduced unilateral closeout. An interim progress report is needed when your grant is finished.

Let's say you have a grant from year one through five and in year five you're finished. We call it the first segment. An interim progress report is due when you've submitted another application to continue your research. If that application is funded, we would call that the second segment. Years past, if you were funded, we wouldn't ask for a final report until your grant and research, they were just completely over. But at some point, they decided an interim progress report was required, which would help us close out grants where the PI had submitted an application, but it was not funded. At least at that point, we would have the final progress report for the first segment. I hope that makes sense. So, you have to just submit a report on what your findings were and submit that through the eRA Commons on your side.

Closeout. This has been a pain point at the NIH for a long time. When I first started working at the NIH in 2003, I came on as a contractor and my job was to close out grants. And at that point, I think the NIH that ICO was working at had about, I can't even tell you how many. They had done the closeout process for 10 years. My entire job was closing out grants. And it was very difficult because some of those grantees were out of business, or they moved on and we couldn't get the reports. And that was a systemic problem that we had within our system.

So, years ago, Congress said, you're not closing out your grants and you're going to have to start doing that. They gave us regulations and policies and deadlines. What we ended up doing was streamlining the process. We have a closeout center, and then eRA Commons allows you to submit your closeout reports through your portal. And once they're in our system, we can go in and we can review them to make sure we have all of the information needed and close out your grant.

What you need to do when you're closing out your grant is file a final FFR, which is your federal financial report. A final progress report, a fine invention statement, and then final population tracking data. This right here, again, is not something that Grants Management knows what to do with. The program officer will let us know if data is needed, and then we request it. It has to come through the AOR. We take, bring it in, and then we confer with program to make sure that it is tracking the data that you were doing or that you received from your science. So, these are the reports.

Let me just tell you about this. And I have these notes over here just so I get all the numbers right. When your grant ends, 90 or 120 days. It used to be 90. It's 120 days after your grant ends, you are required to submit your final reports. What will happen is after 30 days you'll get a notification saying that you are non-compliant with it and the closeout center will send three letters to you at 30-day increments. At that point, it's turned over to the institute to follow up on and get the reports.

If you do not submit your report after 270 days, we will unilaterally close out your grant. And what that means is we won't ask you for any more documents. We will just go in and close your grant. And there will not be a progress report to publish. There won't be any data from you. And the reason this is problematic usually stems from the financial end. If you haven't used all the money in your grant, let's just say you were waiting for an invoice. It came in and for whatever reason, the funds were not drawn down out of the payment management system, and you have $100,000 sitting in there. If we go in to unilaterally close out your grant, we will remove that money. We will de-obligate it. And when we do that, it goes back to the U.S. Treasury.

What normally happens is as soon as the money's gone, somebody notices, oh, my gosh, where'd the money go, we need to pay this invoice. At that point, it's too late. It is absolutely too late. If we don't have a FFR to look at, and it hasn't been approved by Payment Management or the Office of Financial Management, we have to de-obligate what's in the account. And it really is problematic for the institution because they still need to pay that invoice, and we have no recourse. We cannot go back and say, oh, sorry, we need to get that money back. Once it goes back to the Treasury, it's over.

Other reasons you want to make sure you close out your report is once we know that a grant has not been closed out and we've been asking and not receiving a response, we put notes in all of our, what we call Type 5s. in the RPPR in our system to notify anyone that any grants going to that institution or to that PI will not be paid. We do have some recourse where we will withhold funds until we get the closeout documents. It's part of our procedure. When we go in to review a progress report, we automatically check to see if there are any outstanding closeout documents and then we will pay the grant.

There are many reasons why you want to follow up with this. The institution should file the FFR and the final eviction statement, and the PI should file their progress report. It's very difficult for an administrator to file a progress report on science they didn't do. That is how the responsibilities are split up within the institution.

I just put in here pictures of what the final financial report looks like and the final invention statement. This is what we see when you submit it. I don't know what you see on your side. My understanding is you go into the eRA comments, and you fill out these forms electronically, sign them, and they come in. And that's the extent of what I know about the other side.

I do know the final progress report you have to put into the eRA Commons. They have a section here where you can write up the results of your progress on the grant and electronically submit it. And Bob and Julie, if you know any differently about how the reports are filed, like the FFR, chime in. But I'm pretty sure that everything is electronic, so you have to do it in a system somewhere.

And finally, we are at record retention, which is a very, very short, short subject. In the United States, we all file taxes and we each have to save all of our documents for at least six years so that when we file our tax returns each year or hope to get a return, actually. We're just filing the taxes to see if everything washes and we're good to go. And the same thing is required of all of our grantees, except you have to keep your records for three years.

And basically, you want to keep all of your financial records and programmatic records if you are following participants or pharmacy invoices, MRI scans, all of that stuff needs to be saved for three years. Any documents pertaining to your research need to be saved. And then after that, you don't need to save them anymore. But it's due three years from the date the annual FFR is submitted. So, if you are submitting your closeout documents on time, you had that 120-day mark, you have one year from the date you submit your FFR to the NIH to hold your documents for retention purposes. Pretty easy subject. I believe most of our grantees do this. I've never had to go out on a site visit to ask about it, so I think everybody's pretty good with it.

That is basically the end of our presentation. I hope everything has made some sense. It's a lot of information to take in and settle. And you can always get in touch with us if you have questions that you didn't think to ask. You can send us an email and we're more than happy to answer any questions you may have. Do we have any more questions?

Shirley is replying from her carryover request to no-cost extension comment. She said she wrote up processes and procedures for managing the federal grant she was working on. As many people intra agency had responsibility of management of the grant activities, including fiscal contract management in addition to her oversight. And she wanted everyone that had those responsibilities to understand the processes and flow of management of a federal grant.

Thanks very much for that, Shirley. There is a lot to keep track of when you're the administrator and work in that office. It's crazy. At NIH, we fund our grants, and we know our policy. But research administrators, if you're getting money from other places like the Defense Department or the Gates Foundation, they have their policies that you have to know and implement. It's very tough job out there as administrative researchers. All right, Leo. I think we're done.

LEONARDO CUBILLOS: Thank you so much, Tamara, Julie, and Bob for your presentation. I was letting you guys know offline that I was actually taking notes and learning from you, so thank you much for putting all this together. Thank you all for attending today's webinar. As I said at the beginning, this has been a collective effort these past seven webinars that concluded today. We will be announcing shortly the 2023 webinar series from CGMHR, The Center for Global Mental Health Research. We will have topics along the line of capacity building, but also some more scientific topics which I hope are of your interest and certainly, they are very exciting.

Wherever you are, I wish you a happy morning, happy afternoon, or happy evening. Thanks so much for attending and see you next time.