Increasing Case Management, Expanding Mental Health Services, and Coordinating a City-Wide Service Network for Teen Parents in Hartford, Connecticut: A Community-Based Randomized Control Trial

Patricia Schmidt, MA, Daria Keyes, MSW, Lisa Daley, and Toral Sanghavi, MS
The Village for Families & Children, Inc.
Melanie Martin-Peele, MA, and Cheryl Smith, MA
Ethel Donaghue Center for Translating Research into Practice and Policy
at the University of Connecticut Health Center (UCHC), Farmington, CT

Introduction

The Village for Families and Children’s program delivers a combination of continuous case management (CCM), parenting skills training within the context of support groups (one mixed-gender and one for fathers only), and mental health services for pregnant/parenting teens, and their families. This program is designed to meet the needs, on a case-by-base basis, of pregnant/parenting adolescents of both genders who reside in Hartford, CT; the number and length of sessions per client varies based on the number and type of needs/problems with which a client presents. In this 5-year study, the program is being assessed through both outcome and process evaluation to determine whether this CCM and integrated mental health services (IMHS) approach is associated with better parental and infant outcomes. Using a before-and-after reflexive design and a randomized control trial (RCT) that includes opt-in/opt-out client choice and exclusion criteria for safety, the outcome evaluation will also assess whether program participants receiving IMHS have better outcomes than randomly assigned participants who are referred out for mental health services (i.e. standard of care).

This evaluation addresses five outcome objectives among female clients at 6, 12, and 18 months postpartum:

85% of all clients will avoid a subsequent pregnancy.
85% of all clients will conform with US DHHS infant immunization guidelines.
70% of all clients and 85% of those randomized to IMHS will remain enrolled in or will have completed an educational program.
50% of all clients and 75% of those randomized to IMHS will improve mental health functioning (MHF) by at least one clinical category on the Beck Youth Inventory.
75% of all clients and 90% of those randomized to IMHS will not be considered at risk for abuse per CAPI.

For male clients at one year post-intake, 75% of male clients enrolled in the fatherhood component will not be considered at risk for abuse per CAPI.

Methods

Research Hypothesis and Study Design

Our primary research hypothesis is that among clients who opt-in to a randomized control trial (RCT), those randomized to IMHS (seeing a mental health clinician on staff with the program) will have better outcomes at follow-up compared to those randomized to standard of care (referral out to mental health services with follow-up from the case manager). The outcome evaluation focuses on program effectiveness relative to client pogress in achieving a reduction in repeat pregnancy, conforming to infant immunization schedules, completing educational requirements, improving MHF, and improving parenting skills. For the reflexive design, pre- and post- measurements are taken at intake, and at 6, 12, and 18 month post-partum intervals, and analysis is based on data collected from the primary instruments (see below) to measure the SMART objectives. The process evaluation assesses both the implementation of the program and it effectiveness at recruiting and retaining participants, as well as the feasibility of implementing a RCT in a community-based service setting. This feasibility analysis has important implications given the call for increased scientific rigor in the evaluation of OAPP-funded programs.

Eligibility Requirements and Exclusion Criteria

Female clients considered eligible for evaluation include pregnant/parenting teens age 19 and younger who provide consent, and for minors, parental consent. Clients are ineligible to be offered randomization for 3 reasons: 1) safety: clients determined to be a risk to themselves or others; 2) service duplication: clients already receiving mental health counseling; and 3) early enrollment: clients who contacted the program between July and October 2007. Those who meet these exclusion criteria are offered evaluation only; clients who are eligible for randomization are also offered the opportunity to participate in the RCT; they may choose to opt-in or opt-out of randomization. This results in five groups of clients:

6. Decliners: Declined participation in evaluation; not included in analysis.

7. Evaluation only: Consented to evaluation, excluded from randomization (ER) for reasons listed above.

8. Opted-out Randomization (OOR): Consented to evaluation, eligible, but opted-out of randomization.

9. Opted-in Randomization– Intervention (OIR-I): Consented to evaluation, opted-in, randomized to receive IMHS.

10. Opted-in Randomization– Control (OIR-C): Consented to evaluation, opted-in, randomized to standard of care.

Male clients eligible for the AFL evaluation are partners of female clients receiving parenting services through the Village’s program (regardless of her participation in the AFL evaluation), as well as fathers aged 25 and younger who are accessing services at the Village and do not have a partner involved in the program. This study has human subjects research approval and oversight from the Institutional Review Boards at the Village and UCHC.

Data Collection Timeline and Measures

Data from female clients are collected at four time points: 1) Enrollment (baseline) – all measures; 2) Infant age of 6 months – all measures; 3) Infant age of 12 months – all measures; and 4) Infant age of 18 months – AFL Core evaluation instrument only. Female clients who enroll with an infant older than 3 months of age will receive measures at intake, as well as at the next 6-month interval in infant age that allows for at least 3 months’ exposure to the program, and the following 6-month interval in infant age, then followed by only the Core at the third 6-month interval in infant age. Data are collected from male clients at enrollment (baseline) and at one-year after enrollment. Male clients only complete the CAPI. The Five data collection instruments are used in this evaluation (all are offered in English and Spanish) are: 1) Intake Form: the standard form used by the Village’s program, with some changes made to accommodate AFL-specific requirements; 2) AFL CORE Evaluation Instruments; 3) Beck Youth Inventory (Beck): focuses on children’s emotional and social impairments within five main areas: Depression, Anxiety, Anger, Disruptive Behavior and Self-Concept; 4) Child Abuse Potential Inventory (CAPI): Used to assess parenting skills among female and male clients; and 5) The Family Well-Being Scale: Adapted from the Family Development Matrix and the Family Assessment Form. Intervention dosage is measured by comprehensive tracking of clients’ use of case management and clinical services. These data are collected by the case management and clinical staff, and are tracked by the Village.

Sampling Plan

A post-hoc power calculation based on the educational attainment outcome was used to determine the needed sample size to achieve a medium effect size for clients in the RCT study arm of the evaluation. An a=.05, 2-tailed test, with standard power (1-b)=.80, b=.20, and a medium-small effect size for the difference between two proportions is presumed, resulting in a needed n of 120 per randomized group (i.e. OIR-I and OIR-C). Based on literature and previous program outcomes, we anticipate that among those OIR-C, 70% will report current educational enrollment or graduation at 6 month follow-up, while 85% of those randomized to OIR-I will report current educational enrollment or graduation at 6 month follow-up. We anticipate about 50% of clients will not be randomized (due to safety exclusions or opting-out); thus, we will randomly assign 125 subjects to each of the two groups. This is more than n=120 necessary for the OIR-I and OIR-C groups, meeting all power requirements, including one additional subject total for each additional 1 df covariate analyzed

Data Analysis and Statistical Methods

Analyses used independent samples t-tests, Chi-square tests and ANOVA methods to determine significant differences between groups (RCT vs non-RCT, ER vs OIR, OIR-I vs OIR-C) for continuous and dichotomous variables. Logistic regression will be used to determine influences on the dichotomous outcomes. Hierarchical Linear Modeling will be used to determine significant changes over time for the RCT arm overall and between the OIR-I versus OIR-C groups.

Results

Enrollment began during the second year of the program. By the end of Year 2, 165 (146 females and 19 males) clients were eligible for evaluation. Sixty-seven percent (67%) consented to participate in the evaluation (110/165). Eighty-nine percent (89%) of eligible male clients (17/19) and 64% of eligible female clients (93/146) consented to participate. Of the 93 female clients in the evaluation, 22% (20) were ER, 32% (30) were in the OOR group, 24% (23) were in the OIR-I group, and 22% (20) OIR-C. Ninety percent (90%) (84/93) of female clients and 100% (17/17) of male clients completed baseline measures. No clients were eligible for follow-up during Year 2.

Female clients were Hispanic (59%), African American (80%), unmarried (99%), unemployed (81%), pregnant (56%), and enrolled or graduated from a high school or GED program (72%) with an average age of 18 years (SD=1.43). As measured by the Beck, female clients displayed an average self concept (mean=54.11, SD=8.69), average levels of anxiety (mean=51.10, SD=11.43), average levels of depression (mean=49.04, SD=9.98), average levels of anger (mean=53.01, SD=11.83), and average levels of disruptive behavior (mean=50.21, SD=10.09), and were at low risk of abuse as measured by CAPI (mean=165.55, SD=112.08). Male clients participating in the AFL program were Hispanic (71%), African American (29%), unmarried (100%), and unemployed (65%), with an average age of 20 years (SD=3.11) and at low risk of abuse as measured by CAPI (mean=156.80, SD=115.75).

Among the dichotomous baseline measures, there were no statistically significant differences between those in the RCT arm and the non-RCT arm. For continuous baseline measures, those who were in the RCT arm were significantly older (mean=18.07, SD=1.27) than those excluded from the RCT arm (mean=17.16, SD=1.73) (T=-2.53, p=.013). There were no statistically significant differences between those who OIR (OIR- I and OIR-C groups) and those who OOR on dichotomous measures. Those who OIR had significantly higher scores on the Beck depression subscale (mean=51.97, SD=11.27), the Beck anger subscale (mean=56.86, SD=11.56), the CAPI (mean=212.00, SD=104.28), than those who OOR (mean=46.10, SD=6.96) (T=2.45, p=.017), (mean=49.25, SD=9.67) (T=2.80, p=.007), (mean=132.41, SD=89.99) (T=2.35, p=.025), respectively. Clients randomized to OIR-C were not significantly different than OIR-I on any dichotomous or continuous baseline variables.

ANOVA results suggest that there is a statistically significant difference between the RCT arm and non-RCT arm in scores for the anger subscale on the Beck (F=3.769, p=.027). Sub-group comparisons of the means suggest that clients OIR (OIR-I and OIR-C groups) have significantly higher scores on the anger subscale than those OOR (mean difference= -7.61), while the mean difference between those OIR and those ER was not significant (mean difference= -5.60) and the mean difference between those OOR and those ER was not significant (mean difference = -2.01).

Discussion

The results suggest that clients are willing to participate in this study with approximately two-thirds of eligible clients enrolling. Of those enrolling, approximately one-fifth of female clients are ER. Those ER are significantly younger than those OIR. These results suggest that younger teens may possess greater risks to self and risks to others than older teens. Moreover, enrolled clients either possess safety concerns that prohibit their full participation in randomization or, for those who do not possess such exclusions, are less willing to OIR. The program anticipated that 50% of female clients consenting to participate in the evaluation would also consent to randomization; the preliminary results were slightly lower with 46% (43/93) of enrolled female clients OIR. In addition, clients OIR were statistically significantly more likely to have elevated anger and are at higher risk of abusing their children at baseline. These results suggest that females OIR were different than those ER or OOR. Importantly, the OIR-I and OIR-C are not significantly different on any baseline measures. A self-selection process may be occurring in which females with higher levels of aggression select to participate in mental health services by OIR, since all clients who participate in randomization receive mental health services. Thus, through this self-selection process, clients who may need greater levels of involvement and support in developing parenting skills and interpersonal skills are receiving these services through access to mental health services. Future analysis will determine the impact that IMHS has on client educational attainment, reduction of subsequent pregnancies, conformity with US DHHS infant immunization schedule, improvement in MHF and lowered risk for abuse.

Implications

These findings show that clients will opt-in to RCT, though large sample sizes will be needed as only about half will opt-in. Some self-selection may occur based on the services being offered and randomized. This is an important feasibility finding for RCT given the increasing need for scientifically rigorous evaluations. No follow-up data were collected in Year 2; we are not able to conduct any outcome evaluation.

Contact Information

Patricia Schmidt
Friends of the Family Program Director
The Village for Families & Children, Inc.
860-297-0598, xtn. 764
pschmidt@villageforchildren.org