- Who We Are
- What We Do
- Contact Us
- Current Events
Volume 51 Number 2
Edited by Jessica Shaw, Boston College of Social Work
The Criminal Justice Interest Group Column features recent and ongoing work of our members. We encourage readers to reach out to the authors if they are interested in learning more or exploring potential opportunities for collaboration. We also invite readers to join one of our upcoming Learning Community Series presentations in which Criminal Justice Interest Group members share their work virtually to foster a learning community. More information, and recording of prior presentations, can be viewed at http://scra27.org/who-we-are/interest-groups/criminal-justice-interest-group/ .
Fostering Interdisciplinary Collaboration through Process Evaluation
Written by: Sindhia Colburn, Mercedes Pratt, Adam Watkins, and Carolyn J. Tompsett, Bowling Green State University
In 2014, the Lucas County Criminal Justice Coordinating Council (serving the Toledo, OH metropolitan area), with several collaborating agencies, was awarded a Second Chance Act Two-Phase Juvenile Re-entry Demonstration Program Grant by the Office of Juvenile Justice and Delinquency Prevention. The overall purpose of this Second Chance Act (SCA) grant was to support and evaluate family-focused services for high-risk youth re-entering the community from juvenile detention. In Lucas County, the SCA grant brought together several agencies invested in reducing recidivism among juvenile offenders, including the Lucas County Juvenile Court, a local community mental health center, a faith-based family services center, and legal aid providers. Our research team at Bowling Green State University (BGSU) collaborated as the independent evaluator of the grant.
The grant was originally designed to provide services to youth re-entering the community after a 6-month detention at the local juvenile treatment center, which provides mental health, substance use treatment, and educational services in a residential facility. Grant-supported services included: 1) Functional Family Therapy—a short-term structured home-based mental health intervention that targeted family dynamics to decrease negative behavior patterns; 2) alcohol/drug treatment; 3) civil legal services; and 4) family support services. After the grant was received, the court administration recognized early in the planning phase that due to increased efforts to divert youth from detention, there were much fewer youth detained at the treatment center than originally planned for in the grant. As a result, youth released from short-term detention also became eligible for grant services; in addition, some services were changed or dropped by service providers. These changes led to extensions of the initial planning and implementation phases, with all services beginning at least six and sometimes more than twelve months after the proposed implementation date of May 2015. As the grant carried into 2017, several key stakeholders raised concerns about low rates of incoming service referrals and low service utilization. Our evaluation team initiated a process evaluation to learn more about processes contributing to these concerns.
Methods of the Process Evaluation
We conducted semi-structured interviews between February and April 2017 with eleven key informants involved in the grant in various roles, including court administration, probation, mental health providers, and legal aid attorneys. Of the eleven total informants, five informants were responsible for providing services directly to youth and families, whereas the other six informants were primarily responsible for supervising and managing staff and executing administrative tasks. The interviews lasted about 45 minutes and were conducted in a private setting, usually the office of the informant. We audio-recorded and later transcribed each interview. Atlas.ti software was used to facilitate content analysis, identifying themes about attitudes toward the grant, perceived strengths and barriers of grant-funded services, and the effects of the grant on participating youth and families. Themes that emerged across at least two different departments or agencies involved in the grant were summarized in the process report.
Results from Key Informant Interviews
Most informants expressed positive attitudes toward the grant and described it as an avenue for increasing access to mental health and legal services for youth and families involved in the court system. Some strengths of the grant highlighted across multiple interviews included the variety of service options available to families, the inclusion of parent-focused programs and civil legal aid, and the voluntary nature of the grant services which enables families to opt into the programs that best fit their needs. Further, some informants appreciated how the grant facilitated collaborations between previously independent agencies. For example, one informant said, “I think there has been more communication with the community than we have had in the past. And I think that helps collaborating and working together and helps the family and the kids in the system.”
Interviews also revealed knowledge gaps among service providers about the grant and associated services. Informants had different perceptions of the eligibility criteria for grant services, some inaccurately stating that any court-involved youth was eligible for grant services. Informant interviews illuminated a general lack of information about the services included in the grant, as less than a third of the informants accurately identified all grant services. One supervisor described knowledge gaps of staff responsible for service referrals and the impact on family engagement: “Parents ask [about the grant services], ‘What are they supposed to be doing?’ and we have staff say, ‘I don’t know, I don’t know what they’re doing.’ Parents aren’t going to trust [the grant service providers].”
Another important theme that emerged from the interviews is dissatisfaction with the level of communication between service providers and key partners involved in the grant, which likely contributed to the knowledge gaps about the grant and its services. For instance, respondents raised concerns about inter-agency communication during the early stages of conceptualizing and writing the grant, which led to later confusion as services evolved. The grant-writing process involved multiple writers from various agencies, and informants were not given opportunities to read or provide feedback on sections that were written by other agencies. Informants viewed efforts to facilitate regular communication among grant members (e.g., group emails and technical assistance calls) as largely unsuccessful. On a related note, there was little overlap in the grant-related meetings that informants reported attending regularly, and based on their accounts, it was rare for all grant partners to be in the same room together.
Besides these communication barriers, informants noted several other barriers to referrals and family engagement, including the lack of a standard referral protocol and minimal contact between referring agencies and service providers. An overwhelming concern across all interviews pertained to a fear of “overloading” families with services. Informants noted that providers may drop non-mandatory programs from referrals to reduce families’ service load, whether or not those programs fulfill an unmet need. Even when families are referred for grant services, informants noted that families often opt out or disengage from the programs due to their voluntary nature. In particular, families participating in Functional Family Therapy tended to drop out of treatment early if the youth’s probation period ended prior to the end of the treatment process; informants believed that families no longer under court supervision wished to disengage from all services they perceived as associated with the court. Informants also speculated that some staff may be reluctant to refer to other providers because they perceive other services to be a threat to engagement in the services they provide, particularly when staff lack understanding of the unique benefits of those other programs.
Importantly, the interviews revealed several strengths across the grant collaboration. First, providers were genuinely invested in their work with families and concerned about meeting families’ individual needs. Grant partners valued the roles of other agencies in the grant and viewed others to be similarly invested in the work. Informants were also aware of the major communication barriers and offered recommendations for grant-involved agencies to better educate families and other agencies about their services and to streamline communication when coordinating grant services between agencies.
Lessons Learned and Implications for Interdisciplinary Work in Juvenile Justice
After summarizing these results in a process report, we presented our findings at the monthly grant partners meeting and solicited feedback. The partners who attended the meeting stated the report accurately portrayed their perceptions of the grant and the major barriers to its implementation. We found that, as the evaluators, we played a central part in addressing the communication barriers and facilitating new avenues of communication across agencies. For instance, our report served as the basis for discussions among partners about the current referral process. Community agencies that were under-utilized shared tools and strategies for identifying families who would benefit from their services. Grant partners also outlined each agencies’ processes for making service referrals. Our research team compiled a single referral decision tree distributed across agencies, to make each agency’s referral process more transparent and to identify points where more inter-agency contact could facilitate referrals to grant services.
Results from this process evaluation may have overall implications for program evaluation and implementation of collaborative grants in juvenile justice settings. Professionals who work directly with court-involved families within and outside of the system found inter-agency collaboration to be important and helpful, but also impeded by communication barriers. Youth re-entering the community, as well as their families, were seen as benefiting from a diverse array of services even as providers expressed concern about “overloading” families with services. Most importantly, the agencies collaborating in the current grant generally trusted each other’s motives and intent to help families, and this was crucial to the grant’s implementation. Conducting process evaluations such as this one at regular intervals could help develop or build on trust across agencies by enabling more structured communication and facilitating coordination of efforts to serve youth and families.