Dec 13, 2021
Joining Carol Pankow in the VRTAC studio is Felicia Johnson, Commissioner of the South Carolina Vocational Rehabilitation Department, and Lindy Foley, Director of Nebraska VR. Today we are discussing how Felicia and Lindy prepared for the RSA monitoring process, what they experienced, and what they learned. They will share how they see monitoring as a continuous improvement process and an opportunity for growth.
The VRTAC for Quality Management has partnered with CSAVR to co-facilitate a community of practice for the cohort of agencies monitored each year. Over the past three years, we have found that much can be learned by those who are going through monitoring simultaneously sharing their experiences with each other. We provide tools that help you prepare for both the fiscal and programmatic aspects of the review.
You can find out more about VRTAC-QM on the web at:
Please stay up to date by following VRTAC-QM on Facebook and follow us on Twitter @VRTAC_QM
About VRTAC-QM
Partnering with State Vocational Rehabilitation Agencies (SVRAs) to enhance service delivery and maximize outcomes through quality program and resource management.
The purpose of the VRTAC-QM is to provide training and technical assistance that will enable State VR agency personnel to manage available resources, improve effective service delivery, and increase the number and quality of employment outcomes for individuals with disabilities. The VRTAC-QM provides TA and training in VR program and performance quality management, fiscal and resource quality management of the VR program, and general quality management of organizations. You can request technical assistance from the VRTAC-QM by contacting your TA Liaison directly, contacting any member of the Center you wish, or by filling out the information on our main websiteand clicking on submit. While on the main website, join our mailing list to receive updates on training and new activities occurring within the Center.
Manager Minute - Monitoring: Surviving and Thriving Before, During, and After the Process
{Music}
Speaker: Manager Minute brought to you by the VRTAC for quality management. Conversations powered by VR, one manager at a time, one minute at a time. Here is your host Carol Pankow.
Carol: So welcome to the manager minute, I am so excited to have Felicia Johnson, commissioner of the South Carolina Vocational Rehabilitation Department, and Lindy Foley, director of the Nebraska VR General Agency. Thank you both for joining me in the studio today. Well, Felicia, you've been in your position since June of 18. And when I looked you up, actually, you served as the interim commissioner for the whole year prior. So how are things in South Carolina?
Felicia: Things in South Carolina are going very well. You know, if things are continuing to change, we have experienced a lot these last few years, but we continue to move right along. No complaints.
Carol: Glad to hear it. Lindy, you also started as a director just a few months before Felicia back in March of 18, and you had some big shoes to fill following Mark Schultz as he went to be the commissioner of RSA. How are you?
Lindy: I've been very well. Thank you, Carol, and I appreciate the opportunity to spend some time with you, both of you this afternoon.
Carol: Awesome. Well, today we're discussing the 107 monitoring process, and I had the good fortune to work with both of you and your team through this latest 2021 RSA monitoring cycle and with RSA just releasing the list of agencies being monitored this past Monday. You can be a good help to this new group. For those of you who are listening to this podcast, the VRTAC for Quality Management has partnered with CSAVR to co-facilitate a community of practice for the cohort of agencies being monitored each year. We have found over the past three years that much can be learned by others going through monitoring. At the same time, we provide some tools that help you to prepare for both the fiscal and programmatic aspects of the review. And additionally, the VRTAC for Quality Management can help you through the process as you are participating in the visits with the RSA team. This is especially helpful if you're anticipating needing some follow up after the review is complete. So now to you two, both of you had such a great approach to this process, so I wanted to have this discussion. I know monitoring was a topic at the CSAVR Leadership Forum, but I wanted to dig in a little bit more to help those colleagues out there who will be experiencing this very soon. Whether it's this year or next, everyone's going to be reviewed at some point. So let's go. We realize that every agency is structured uniquely, which ultimately may impact the entire monitoring process. So it's important for us to get a feel for how you operate. So tell us a little bit more about your agency and a little bit about you. And on average, you know, like, how many customers do you serve, how many overall staff do you have and how many people are on your leadership team? And Lindy, I'm going to kick that over to you.
Lindy: Ok, great. Thank you. So in Nebraska, we have two VR agencies, so a quick shout out to our partners at the Nebraska Commission for the Blind and Visually Impaired. Nebraska VR is actually located within the Department of Education, and something which might be a little unique is that we also have the Assistive Technology Partnership, or ATP, within the department with us. So this really allows us in this organizational structure to work to our advantage to leverage opportunities and partnerships with our schools to expand the way that we look at special education and even kind of some cool stuff with career and technical education and the opportunities, of course, with assistive technology and the impact that that can make for our clients and also businesses, we currently have about 7200 cases open. We are coming out of an order of selection, so we want to get back to the numbers we once were serving prior to 2017, but we're making some progress there, so that's exciting. We've got 12 field offices across the state and 1 administrative office here in Lincoln, so a total of about 200 staff and you mentioned leadership teams just real quickly. We have a policy and implementation team which consists of staff who we call program directors.
So individuals who help us to write policy, conduct our case reviews, organize training, oversee the financial operations and quality assurances. The other half of that team are our office directors who help to supervise our field offices. I also include in that leadership team, our client assistance program director, and he meets with this leadership group often to help us provide input. And then finally, but certainly not least, the leadership team that I'm referring to also includes our agency's assistant director. I also want to make a special mention to another leadership group within Nebraska VR because they were instrumental in our monitoring process as well. We refer to that group as the Leadership Council. Those are staff from a variety of roles in Nebraska VR from across the state. They're nominated by their teammates as being recognized leaders in our agencies, and I meet with them quarterly. They advised me on a variety of topics, whether that's the culture of the agency. See our mission and values and then, of course, compliance and monitoring activities and ways we could prepare for our monitoring visit. So a couple of different ways that we look at leadership and that infrastructure within our state.
Carol: I love hearing about that Leadership Council. I will have to follow up on that at another time. Very interesting. So Felicia, how about you? How are you all organized?
Felicia: Ok, we're organized a little bit differently. We do like Nebraska. We have two separate agencies, so we have the blind agency and course. I'm at the South Carolina general. We are an independent commission, so we are not under any other departments. We have a board of commissioners, seven individuals that represent each of our districts within the state, and we own and operate all of our CRps. We do not contract out very many services, actually. We just began contracting out services when I started as commissioner around 2018, and those services were specific to pre-employment transition because we were still working our way through that. So being that we own and operate all of our community rehabilitation programs, that makes us unique and we're staffed a little differently. We have 31 offices throughout our state and we have 26 what we call job readiness training centers. Some offices have some offices because the county is so big, but each main office has a job readiness training center partnered with it. And what that is is a community rehab program where individuals with functional limitations and barriers to employment, they come and they do what we call contract work. We partner with businesses throughout our state over four hundred and they bring training opportunities in the form of work tasks to our centers for us to produce and send back to these businesses. And then they sell and all of our consumers make at least minimum wage. At least there's no sub minimum wage in any of our centers.
We do an assessment, thorough assessment of them. When they enter the program, we set goals and objectives for every single consumer, which is individualized based on their functional limitations and employment objectives. They work towards those goals and objectives. They have at least monthly meetings with their counselor and a member of the job readiness training team to see how they're progressing towards those objectives. And we also begin job search and placement opportunities for them while they're in training. We also own and operate an inpatient 28 day comprehensive evaluation program. We have a center actually on this campus where I'm located, where our consumers with severe significant physical, disabling conditions come for a 28 day comprehensive evaluation to help them see what their strengths are with their barriers are and then how we can help them to adjust to those barriers and identify appropriate job opportunities for them. So when they go back to their home area, their local areas, their counselors and the rest of the team can help to find employment opportunities in the fields that we may have recommended. And we have an inpatient rehabilitation facility, which most folks don't have a recovery center is what we call it for our consumers who struggle with alcohol and addictions issues, which also impacts their ability to obtain and maintain employment, but can mask symptoms from maybe pain and other disabling conditions. So we own and operate a 28 day inpatient addictions recovery center. South Carolina General houses our state's disability and determination services department, so we adjudicate and review all claims for citizens who are applying for disability benefits. And as far as our staff goes, our staffing patterns have significantly changed from when I started in this position. Right now, we have just under 1,000 employees. When I first started in this role, we had over 4,900. We've noticed a lot of attrition, especially during the pandemic, with folks who are eligible for retirement and now coming out of the pandemic into the great resignation. We're seeing a lot of folks changing their career objectives, but that's fine. We're in the process of just revamping the way that we provide services. We've learned a lot with using technology, and we're going to continue to use technology to make sure we serve our consumers as far as consumers go. We have about just under 1,300 active consumers that we're working with. That's a little lower than before the pandemic as well. And our executive team, I think that was the last part of the question. I have 14 members on my executive leadership team. All of my executive leaders, our department heads. So our director of consumer services, director of human resources, director of Planning and programming, now director of field operations. All directors are members of executive staff and we meet once a month to strategize and plan for the agency. And that's pretty much South Carolina general.
Carol: That is complex. I remember during the monitoring process when you were explaining to the team, I'm like, Wow, you both have a ton going on. So I know people had not been monitored for some time, and I just wondered when was the last time your agency was monitored prior to that? 2021 monitoring? And if you don't mind sharing, how did that previous monitoring go for the agency and Lindy, I'm going to kick that to you.
Lindy: Ok, great. Thank you. So the last time that Nebraska VR was monitored was in 2012, so it had been a while. That's why we kind of kept expecting each year that we would be on the list. I was with the Department of Education at that time, but I was actually with the Office of Special Education and did a lot of work with transition at that time with the counterpart in VR. So I was kind of an outside looking in at the VR monitoring. Then later I'll describe we had a core group of people within our monitoring process and three out of the four. Me being the only one had been through RSA monitoring before, so it was obviously great to have their experience and expertise through that process. So I asked them, how did this compare to the previous the 2012 visit? One very obvious difference was just within the last few months, of course, are 2021 visit was all virtual, so that had some benefits, but also maybe some fewer opportunities presented. So in 2012 RSA was on site, they could interact more. They did interact more with our staff, visit field offices and maybe had more of an opportunity to even hear from some of our partners. I think one thing that stood out to just a different time, right, and where we were with providing services, I jokingly said, Tell me, you're being monitored without telling me you're being monitored. I mean, the binders and the paper that my staff gave me when I said, you know, let me learn about what the last process looked like and what we were working on. Oh my goodness, it was so overwhelming. So we still have those binders from 2012. But fortunately, this time sent so much of it was streamlined and was electronic. We were able to be, I think, much more efficient in that process. So those were some of the differences that stuck out. And we've had a very good rapport with our RSA team and some of them have been the same since twenty twelve. So very fortunate to have that historical perspective from our RSA partners from then to now.
Carol: That is good to hear. How about you, Felicia? What do you know about the monitoring? I know you weren't there, but I wonder if you'd heard?
Felicia: Yeah, I was with the agency, but I was out in one of our field offices as an area supervisor. Back then it was around 2010, and from what I understand, the process went well. For the most part, I just kept hearing that they had a difficult time understanding our job readiness training centers and how they operated. That was the feedback I recall back then, and that led to some changes being made in the operation of those centers. But that's what I recall.
Carol: I know when I was at Minnesota Blind, we had had a monitoring in eleven, so they came back in ‘19. And when I looked at that report, when I came in ‘13 and I went, Oh my gosh, like my vow was that we are not going to have a report like that again because it wasn't good at all. So that that was my quest, right when I came in the door. So one other fact about my background is I was a CARF Surveyor for several years and CARF, the national accreditation organization. I always think about it like providing the Good Housekeeping seal of approval for various rehab programs and facilities throughout the country. My agency was also accredited, and so all the surveyors came from the field and you would agree to perform at least three surveys a year. And the philosophy I really liked that I learned from CARF training was that you always wanted to be prepared, like, don't just get ready for the CARF review every three years when they're going to come like you want to always be in this kind of constant state of being ready for a visit at any time. So I just want to talk about that philosophy a little bit more. When you both started back in ‘18 was the idea of monitoring on your radar as you dug into your work. And if it was, what did you do to prepare? Felicia, I'm going to kick that to you first.
Felicia: Ok. Absolutely. When I first stepped into the role, we knew that we were overdue for monitoring, and we kept hearing that the monitoring schedule was thrown off because of WIOA. But we knew we were overdue because it had been over 7 years, so we actually started praying that we would not be called at that fall conference. Actually, this is during my interim term fall of 2017 when the conference was here in South Carolina. We were all like, Lord, please don't let them call South Carolina general because we were not prepared. I was brand spanking new. We had just gone through some interesting times in our agency, and we just didn't need that at that time, but we did start preparing in ‘20. We had our planning and program evaluation department. They started to develop reports based on the 2018 and 2019 MTAGs, and they began reviewing some of the published 107 reports for trends so we can get an idea of what was being looked at. We started having monthly meetings in our executive staff teams to review some of the data trends and walk through some of those impact questions we identified who are subject matter experts would be based on some of the things that we saw in the MTAG and in the published reports. We want to make sure that we had supported employment covered because we knew as an agency that was something that we needed to work on our fiscal.
We wanted to identify who our subject matter experts would be for fiscal matters, pre-employment transition services, so forth and so on. So we kind of started in 2019 and into ‘20 is when we really said, OK, guys, we know we're going to get called. So we really started to get intense in 2020 with making sure that we had everything lined up and we were regularly reviewing the MTAG and always looking for other states published reports. Then, of course, the pandemic hit. And so that just pushed everything to the side and we were just trying to survive as an agency. But once we started working our way through that, we had a kick off meeting and we introduce everyone to 107 monitoring and what it was and who our subject matter expert teams would be. We started to have some planning sessions and had each team develop a 107 monitoring to do list for their respective departments. We had meetings planned and scheduled every single month to make sure that everybody was on the same page. This was actually we picked back up during the towards the pandemic. Well, I guess the middle of it because we were sent home, I believe the end of March, March 23rd, but were reopened back. I want to say July the 6th. We continue to work, but we didn't have people teleworking for very long. So once we started migrating back into the offices, we got right back into it in 2021. Then the new 2021 MTAG came out. So we were like, Dang it, but we went through that MTAG and we started working our way through there. We quickly noticed that it had been overhauled to focus on more data and performance than the previous MTAG. No problem. We shifted and we made sure that we went through that step by step. And then we also tried to identify any deficiencies that we knew that we had in our agency. Like, we knew that our policies were outdated. We knew that we needed to update policies. We knew that maybe we need to take a closer look at some fiscal processes. We knew that we need to take a closer look at support, employment and how we were doing that. So we knew where our struggles were and we want to make sure we jumped in front of them and had a plan for asking for technical assistance. So we lined all of that up. So I would say that we spent close to 2 years preparing, thinking every single conference that we were going to be called for monitoring, so we were well prepared or as prepared as I think we could have been under the circumstances.
Carol: That sounds really good. I know I did a similar thing to that with our team. I kept thinking every year they're going to call our name. I just was waiting. They're going to call our name. Then finally, your name gets called. It's like, Oh, finally, but I know what you mean. And having to pivot at that, monitors will change, but that sounds really good. How about you, Lindy, what did you guys do?
Lindy: Well, as I mentioned earlier, we knew we had to be on the short list just because it'd been quite a while since we'd been monitored. So we in 2019, we did start really trying to review and best understand the monitoring technical assistance guide of the MTAG. So what we did and it's been revised, of course, a bit since 2018, but we took that leadership group, our policy staff as well as our office leadership, and we divided up into work groups, which matched then the areas of the MTAG. So we had a group related to VR services and pre-employment transition services. We had a supported employment group and then, oh, by far the most popular, the fiscal and data. That's kind of a joke, but we did get people who volunteered to be a part of that work group, too. And so I thought that was very meaningful and we got good feedback from staff, the teams they met at their own pace. Each work group took the time to go through each of the questions within the MTAG. We had one of our data staff go through and best understand the data tables within the MTAG to make sure that he could actually recreate the data that within those tables. So when it came time, if there were any questions about reporting or consistency that we were well informed about what data went into those tables. So each workgroup got their data tables, they went through their questions. They helped us to identify if there were questions that we struggled to answer. We knew that we needed to flag those questions and then bring them back to the whole group so we could talk about any impact on revision to policy. Or perhaps it was a training and implementation issue that we needed to talk through. I appreciated having that time. Our staff, I think in general, prefer to be proactive, even though we know in our work there's not always that that luxury. But I think that gave us time to really think and reflect on the questions without it feeling rushed, as if maybe what we would have done if we would have done it after we'd been called, after our name had been announced for monitor. So I think that that really did work well, it also created kind of a new level of buy in that I didn't actually expect at the time, of this leadership. They better understood if they were questioning a certain area of our policy, they'd get out their rigs. They'd look in the comments they would alongside of the policy people. They wanted to know why we were being asked those questions and to make sure that we were doing everything within our services to make sure that we could answer that confidently. Again, I think that worked out very well. One other thing I would mention to, in addition to preparing ourselves kind of emotionally there was a component to I was very intentional about calling this process a self-assessment process with the MTAG. I felt like if I was using that type of language, it helped to give the message that this was about continuous improvement. This was a meaningful activity that we can both celebrate our successes, but also identify some challenges and some changes that we would need to make. So again, I think just having that intentional dialogue about how we were referring to those activities was something that we took pretty seriously. So back to the whole notion of this being a continuous improvement process, and I know we'll maybe get into this a little bit later, but we're not done with the MTAG. I think it was a great exercise that we went through in 2018. We revisited it in 2020 one and we'll do it again in a year or two to see how we're doing against those data tables and to see the progress we've made against those questions. So I'm excited that we continue to look at the process and we've found a way to hold ourselves accountable and to take time to celebrate the progress that we've made going through the MTAG.
Carol: Well, Lindy, you teed us up for my next question and both of you are so prepared, and I really think that that topic of preparation helps us transition into the overall sentiment around the monitoring process. And Lindy, you spoke about this, like talking about it in a different way, continuous improvement, you know, words like monitoring, auditing and reviews. They often have a stigma attached to them, and people will usually regard that as some sort of a gotcha moment, and it causes dread in the monitoring process. So I wondered, Lindy, you talked a little bit about your messaging, Felicia, I'm going to kick this one to you. What did you do in communicating to kind of alleviate some of that tension associated with the monitoring process?
Felicia: Well, we did have a kickoff meeting because my executive team either were not with the agency the first time that we were monitored or were not in the roles that we're in now. So for us, we needed to educate a good many folks on 107 monitoring with that was so we had the kickoff meeting introducing 107, but we also were very deliberate and couching it similar to
Lindy as an opportunity for us to assess our program, to identify where we're strong and to also identify where there are opportunities to improve. And I'm the type of person that if I'm doing something wrong, I want to know immediately so I can jump on it and correct it. And if there were things that were happening in the agency that we were doing out of, maybe practice, but we should not be doing it that way. I really wanted to know and my team wanted to know so that we could get it right, so we were actually looking forward to it. Only 3 members of my executive team had been with the agency the first time we monitored, and I think 2 of them had minimal involvement. Very, very minimal. We were ready for it. We had been talking about it for years and when we got the call or saw the email because we saw our name in an email, we were ready, we were excited, we were ready to get it over with.
Carol: I want to put an exclamation point at the end of what you all just said because that is great, especially people listening. And if you're one of the directors and your name was called, definitely think about that whole messaging piece. And I really think that's important to not go into it with that fear factor, like you're just assessing, let's get through it. So going to speak about when your agency's name was announced for monitoring and you all were in the email. I know some people had it happen during a CSA conference. What was the very first thing that you did and why? And Lindy, I'm going to go to you first.
Lindy: Ok, well, I remember it was a Friday afternoon and it was an email that came out as Felicia had mentioned, and I had just gotten done doing a final walkthrough because we had just moved out of our administrative office in which we'd been located for about 45 years. So I'm not sure if it was a relief to move on to something else or what, but that's no joke. But I went home that evening and poured myself a tall, cold glass of milk and really tried to reflect in a weird way. There was some relief there, and I think it was. It's just that anticipation of not really knowing. We knew it was coming, but just not knowing when, and I think seeing our name in that email and knowing that it was coming soon then put a little bit more control into then putting our actions into motion. As far as timelines and we've been practicing, we'd been looking and preparing since 2018. So we'd been through the dress rehearsal and we were ready to make our performance. So I did reach out to a core group of staff who I knew would be going through every step of the monitoring process with me when I called and talked to them that Friday afternoon. Their response was pretty much the same. Ok, we've got this, and that's exactly the type of attitude that I think we all went into our monitoring with just that positivity and the confidence. I think that that was all part of being able to pace things in the way that we did that. When the time did come, we would be prepared. It wouldn't be a panic and there'd be some excitement there too. And I think there was.
Carol: I love that. I love that. How about you, Felicia? When you read that email on a Friday afternoon, I'm sure it all came at the same time ...What was the very first thing you did and why did you do that?
Felicia: Well, the first thing I thought to myself was about time, and I forwarded forwarded it to all of my executive team. And I just said, It's time, guys. And what we had done in all of our prep was I identified a 107 monitoring lead. I have been very fortunate there is a member of my team. She is currently still my director of policy and internal controls. She will be leaving the agency at the end of this month. But fortunately for us, she was here during that time and she organized it all. Once she got the email from me, she started setting up regular meetings, emailing data tables and documents, putting binders together for everyone. So she really led the entire process and did a phenomenal job. So we were all on pins and needles. So when that email came, we were just like, Let's go, we got started right away.
Carol: That is awesome. And I'm giving a shout out to Denise because I know your staff and I never saw anything quite like it, and I sat through several of the different monitorings. But boy Denise was. She was organized to a level I had never seen before. That was something. It was a work of art. Actually, it was pretty amazing. It really was.
Felicia: I call Denise my unicorn, and I told her when she told me she was leaving the agency, I'm going to have to build another unicorn.
Carol: No, honestly, I know when she let me know, I'm like, Oh my gosh, I said, I bet, Felicia said. Yikes, holy smokes. She really did a good job. Yeah, so I know you both talked a lot about how you kind of approach this review with your staff. What did you do about communication? Because obviously you have kind of your core groups that are helping to prepare and get that going. So how did you keep like the whole group and the field sort of involved? So they didn't think this was some big, huge mystery. And Lindy, I'll go to you first on that.
Lindy: Well, I think for us it was I can kind of describe it like a like a target, like 3 circles of communication. So the inside circle, the smaller circle. There were 4 of us who, as I mentioned earlier, involved in really every type of precession, every phone call on every email. And so that group was obviously very involved in any type of communication with RSA. The 2nd circle was really that policy group that I mentioned earlier. They had done so much work to prepare to make sure that our policies and staff were at a place and a comfort level consistent with the tag that group ordinarily meets. Often for a while they were meeting quarterly. But then in response to more of our continuity of operations plan due to COVID, we actually move to monthly. So we were able to really keep that team informed all along the way, and they had good questions. They were curious to see where we were in the process, whether it was policies being submitted, initial questions that we were being asked, keeping them informed of the agenda. Perhaps they had a particular programmatic area or something that their office was really doing well. So they were curious about what RSA, how RSA was framing that two week time with us. And then the outer circle, of course, was with our staff, our field staff. And to me, it was a balance of keeping them informed and engaged while also not overwhelming them with the process. I don't know if this is different from state to state, but RSA did not ask to interview our field staff. We did have some programmatic or some of our program directors that were involved during our actual visit and could respond. I'm thinking one particular area was pre-employment transition services, but really it was that core group of people that had the meetings with RSA. One thing that we did was we uploaded the MTAG into Microsoft Teams. Our agency has been using Teams now for a while and. There's a function within Teams, of course, that you can create channels, so we had a channel dedicated to our monitoring process with the MTAG being uploaded, people that were a part of that channel could go in and add notes. They could go in and encourage one another if they knew that there was a topic coming on in that particular day's agenda. There were times that because we didn't have a big team of people meeting with RSA, a question might come up during conversation. We could put a really quick Teams chat in and we had the answer within just a few minutes because that leadership team was really on call during that visit. So I think the communication came in a lot of different ways. The piece that I would say also is the communication doesn't end, right? So this isn't intended to be an episodic process. So even though we don't have our draft report yet and we're not certain what will be included in our monitoring report, our assistant director and I have gone out to all of our field offices as part of our team tour or something we do twice a year. We talk to them about monitoring. We shared knowing that everything can be put in a report. We shared what RSA said that they were pleased to see some areas that they identified as innovative, and we wanted our staff to be able to hear that good news too. We could pick up on maybe some of the questions during our visit, maybe things that we might see in our draft report. I don't know that any of those were real surprises. But again, back to that messaging, I didn't want staff and I don't want any of the staff at Nebraska VR to think that if something is written in a report that we've done something wrong, it's an opportunity for us to improve and to build. And so I think that communication, as long as it's about continuous improvement and not a gotcha that feels a little bit different, we can relate to it a little bit better with always striving to be at our best for our clients and businesses.
Carol: I think definitely the review is different with it being virtual than when they're on site, when they're on site, they're definitely talking to staff and so many more people are involved. So you both did not get the benefit of having that with more staff kind of engaged throughout that process. And I did add your tip, Lindy, to our tip operation sheet that I've just been revising and sending it to CSAVR on Monday for the new group with all the different tips we have because I love that I wrote that down about the Teams and using that to help with communication and just organization. So Felicia, how about you with the communication?
Felicia: We did things kind of the same kind of different when I say kind of different. We had all of our executive team involved in every meeting for the most part and all of our subject matter experts that were not a part of the executive team, for example, our young lady over supported employment services. So I think altogether on every single call, we had approximately 35 participants. The majority were in listen only mode, but we did have a ton of folks on the call for several reasons. For one, I think everyone on the call, like I said, only two folks on the executive team were here and minimally involved in the last monitoring. So we used it as an opportunity to learn for all executive staff or our subject matter experts. We wanted them to learn about the process. It used to be a big secret in our agency, so we wanted them to know that this isn't something punitive. It's an opportunity, as Lindy said, to learn and for continuous improvement of our program. Part of those 35 participants were members of the VRTAC QM, who I cannot think enough. A team member was on every single call and then even had a follow up session with us after every single call to provide feedback and answer any questions and help us prepare and process for the next call. We did have some field staff interviewed virtually. They wanted to speak with some of our counselors, our general counselors and our pre-employment transition counselors, so they were involved in calls with RSA. We selected several of our field office supervisors to participate on the call. They were listening for the majority of the time. But if RSA had a question that was specific about programmatic operations in the field, we would ask one of the supervisors, You know, can you chime in and explain to RSA how you do this in your area or how this works in your area, which was good for them to have that opportunity to communicate their program with RSA and to get a feel for how the 107 monitoring process works? We also provided updates to our board on a regular basis. Our board chair actually participated in every single call she listened in. So that was good for her to see how the process went. We talked about gave status updates during our field supervisor meetings that the director of field operations has on a monthly basis. So we provide an update because, like I said, my staff is slightly under a thousand folks, so we can have everybody involved, but we let the leadership team in the middleman. Gave updates, and then they would trickle those updates down the line to folks in the field.
Carol: So both of you are underscoring the importance of communication. So again, for those of you listening that are going to be going through this review at some point. Communication, communication, communication. So I know you both were just incredible at figuring out this process and how you worked through it. I just wanted to ask each of you, what are the top 3 things you did throughout the process that you would recommend to your colleagues across the country as they're going to get ready to go for monitoring? And Felicia, I'm going to ask you that first.
Felicia: Well, those are three great ones. So I would underscore everything that Lindy just said. So I guess just to add a little bit more to that, I would say, think about what you want the monitoring visit to mean to your agency and then message that if you want it to be seen as a continuous improvement process, if you want it to be seen as an opportunity for growth, make sure that it's messaged in that way. Number two, seek out opportunities during the onsite visit to celebrate innovation and best practice. I felt like we kind of had to work this in whenever we could. It felt like we used the full time every day to the very last minute. So there were times that I felt like I probably just need to let the questions play out. But for the sake of knowing how hard our staff work, really seeking out times to say, Could we take a moment? And I'd like to just share with you some innovative things we've done surrounding pre-employment transition services? Or can I help celebrate what we do in supported employment and the work with our providers? And so I think being really intentional, even if you're the one that's driving that conversation to celebrate the success and innovation, I think that that's really important. And I know our RSA team was open to that and appreciated it as well. And then the third one is just to identify those champions in your agency and to give responsibility to those champions to be very clear and specific about what everyone's role is during the process. The more specific you can be about those roles, what approvals look like, who needs to be a part and what steps. We just found that the more confident we all were with carrying out the responsibility and making sure that things were done in a timely manner. So I'd say that those were 3 important pointers that we would offer from Nebraska.
Carol: Well said So Lindy, how about you? What are your top three recommendations?
Lindy: I will say, and it's not just because this is your podcast, but I really think that getting the VRTAC-QM involved was very, very helpful, especially if you've got a new leadership team who has not experienced one or seven monitoring. I would definitely recommend that that support and guidance and feedback I felt was very beneficial to our team here. Definitely study the MTAG and review monitoring reports from other states, even if your monitoring doesn't go step by step through the MTAG. Ours did not, but the fact that we studied that MTAG for so long, we were able to identify key questions in which departments should answer that question. We've picked out keywords and figured out if they ask this question or if they make this statement, then the data department needs to answer that question. So just studying that MTAG and looking at other reports to kind of get a sense of what the hot topics are. I think that will be helpful. And then identifying those subject matter experts for your agency who can answer those questions at that time that will alleviate having to respond with a Oh, I'm not sure. Let us get back with you now. There's nothing wrong with that, but I just feel like the more prepared you are, the better. You're able to answer those questions when they come about and those subject matter experts who live and breathe those topics are the best ones to have in the room to answer those questions for you.
Carol: Well, said both of you. Well, I know as we're winding down, I did want to find out at the end of the day, how do you think this process is making your agency better? Did you walk away from the table with anything really good that you believe is going to perfect the recipe of what you're cooking up? And Felicia, I'm going to ask you first.
Felicia: Hey, well, ours is probably very simple. Like, there were times during our visit that our team would say, Is there any type of questions that you have or is there technical assistance that you're seeking? And I think sometimes we get just so caught up into our day to day work that we don't always take the time to reflect like we should about asking RSA questions or thinking about what that technical assistance could look like. So I think just the extra ingredient that this added to our process to is just making sure that we know that we're not in it alone and that we have a lot of partners that we can reach out to lean on. So just something that we were able to reflect on during the process, but then also afterwards.
Carol: Lindy, I'm going to kick it to you for the last words.
Lindy: Yeah, I did feel like we learned some things and we did walk away with ideas to make our agency better. It did help us to provide confirmation on the things that we're doing correctly and things that are going well. And also help to identify issues within our agency that we could improve what we walked away with, actually, RSA gave us a very good recommendation on how we could enhance what we do in our job readiness training centers in a way that could set the stage for it to be an actual model for all other agencies to follow. So that's something that we've been working on, and we also walked away with a better understanding of RSA push for using data to make those data driven decisions as we plan and operate our VR programs. Those were two very specific things that we walked away from monitoring with, and we have started putting things in place to follow through with those recommendations. And also, we kind of got a sense in our exit meeting what we would see in our final report. We don't have it yet. We're expected to have it by the end of the month, but we went ahead and started working on the things that we felt RSA might address in the report based on some of the questions that they had during the 107 monitoring process.
Carol: Well, I know John Connelly and I will both probably be inviting you two to come back to the new cohort just to share some words of wisdom, too, as we get the new group kicked off and rolling. And I sure appreciate your time chatting with me today about the important subject of monitoring, and I wish you the very best now and forever, and hope that today's conversation will spark further conversations about surviving and thriving before, during and after the monitoring process. Thank you both so much.
Lindy: Thank you.
Felicia: Thank you.
{Music}
Speaker: Conversations powered by VR one manager at a time, one minute at a time brought to you by the VRTAC for Quality Management. Catch all of our podcast episodes by subscribing on Apple Podcasts, Google Podcasts or wherever you listen to podcasts. Thanks for listening!