The Evaluation of REAL: Reinforcing Education through Abstinence and Leadership

Department of Epidemiology and Public Health,  University of Miami Miller School of Medicine at Miami, FL  Korrine C. Bentley, M.A,  and Janet Mapp, B.S. Switchboard of Miami, Miami, FL

Introduction

The REAL (Reinforcing Education through Abstinence and Leadership) project is a curriculum based middle school program aimed at preventing high risk behaviors that could lead to teen pregnancy. The program each year serves the entire seventh and eighth grade at Allapattah Middle School in Miami, Florida. This school is comprised of mostly Hispanic and African American youth. Allapattah Middle School is a zone school which means that the school is being targeted for extra help to boost the school from a failing school to a school with higher achievement on the Florida Comprehensive Assessment Test.  In 2001, Miami’s teens gave birth to four thousand infants and had repeat pregnancies at 44%, twice the national rate of 22%. Another serious concern related to risky sexual behavior is the infection rate of HIV/AIDS within Miami Dade County. Miami is now one of the leading cities in the country for HIV/AIDS among 18-29 year olds. The REAL program consists of three research-based programs approved by OAPP. The Choosing the Best Way and Path curriculums and Baby Think it Over program are being implemented at Allapattah Middle School. Each curriculum offers intervention into the social, psychological and health advantages of choosing an abstinent lifestyle. The goal is to attribute group differences between the intervention and control groups in the fall semester to changes in the program participants due to the intervention or interventions that they received.  There are a total of four outcome objectives: Objective 1: By the end of the project year, 50% or approximately 200 youth will be able to demonstrate an increase in setting goals and have a more positive outlook on the future, Objective 2: By the end of the project year, 50%, or 200 youth, will have a more positive attitude towards abstinence, Objective 3: By the end of the project year, 50%, or 200 youth, will increase communication with a trusted adult, and Objective 4: By the end of the fourth project year, 50%, or 200 youth, will have a higher level of self worth and improve self esteem which may reduce sexual activity.

Methods

A quasi-experimental design is being used at our school rather than a true experimental design because youth are randomized into a particular curriculum by classroom rather than by individual randomization. The seventh and eighth grade classrooms are split into six cohorts each year. Half of the cohorts receive the intervention in the fall and the other half serve as the control group during the fall semester. During the spring, the groups who were the control group in the fall are then served. There are a total of three testing points each year; a baseline in the beginning of the school year, a second testing after the first semester, and a posttest at the end of the year. The purpose of the local evaluation is to test the hypothesis that male and female adolescents who are involved in a comprehensive pregnancy prevention program like Choosing the Best will be more likely to acknowledge their own pregnancy risk and their personal responsibility for avoidance of pregnancy and less likely to endorse casual premarital sexual activity.  We further hypothesize that having the additional Baby Think it Over curriculum will have an incremental effect on attitudes about the burdens of teen parenting and that the addition of a brief but powerful intervention like Baby Think it Over may significantly improve and reinforce overall outcomes.  All seventh and eighth grade program participants are included in the study; thus, there is no sampling.  Students are served for sixty minutes each week. There are a total of twelve lessons. In respect to outcome data collection, all students are surveyed at the beginning of the year, mid-way through the year, and at the end of the year and again, one year later as follow-up.  Identical instruments are used for each time period.  The survey instrument was designed in order to reflect the objectives of the program.  The questions ask about their knowledge of reproduction, sexual behavior, intentions to remain abstinent, communication, negotiation, and refusal skills, consequences of teen pregnancy,  and critical thinking and decision making skills. We will be able to test our hypotheses by the use of half of the student population as a control group during the fall semester for both of the grade levels.  We have a control group only during the fall semester. We expect that those youth involved in services will greatly reduce the initiation of sexual activity compared to other students who are not receiving services.  We expect that the predictors of risk behavior will show significant pre-post change in a positive direction for those youth involved in programming, whereas the comparison group without the program will demonstrate no change. We further expect that when exposed to the intervention during the spring semester, the former control group will exhibit the expected changes. Each year the project is reviewed by the Human Subjects Research Office at the University of Miami.  The IRB process last year was quite difficult because we were getting additional IRB approval for the agency themselves. The University of Miami has gone through the IRB all five years on behalf of the evaluation. The past two years the agency has also gone through the IRB process on behalf of the programming piece.

Results

Project REAL just completed year four. Therefore, there are only three years of data that has been analyzed. For all three project years there have been a total of one thousand students included in the analysis. The preliminary results indicate that at posttest project REAL students, increased their intent to remain abstinent, improved refusal skills, and increased their knowledge regarding the consequences of teen pregnancy. At pretest 40% of youth stated that it was important for them not to have sexual intercourse before being married and at posttest 66% stated it was important for them not to have sexual intercourse before being married. At pretest 57% of youth felt that abstinence from sexual activity will give them more options for their future, help them reach personal goals and experience healthier relationships and at posttest 75% of youth agreed with this statement. At pretest 38% of youth stated that they would not have sexual intercourse before marriage and at posttest 61% of youth stated they would not have sexual intercourse before marriage.  Statistical significance will be determined when all four years of the project are combined.

Discussion

Throughout the four project years, there have been obstacles that the staff had to overcome. The study site school received the poorest rating on a state assessment, during year one and this led to a slight delay in the implementation of the project. The project staff members were not able to follow the evaluation design in year one due to barriers within the school. The staff was only given one semester to work with the students in year one since the emphasis was on academics and improving the school rating. After year one, the staff were trained in the FCAT standards in order to meet both the FCAT goals of the school and the goals of the project.  In year two the school was still a zone school but the timeline was able to be followed. There is a very strong academic focus to improve the school’s academic rating. Although there were many challenges in year one, improvements were seen from pretest to posttest with the youth who received program services. We consider this quite significant because the program was not able to be run throughout the whole school year and a much smaller number of students were serviced during year one than anticipated. However, for year two the school slightly improved its academic rating and the REAL program was implemented in the science classrooms and the initial evaluation design was followed. Year three results were consistent with the other years. However, results have to be reviewed with caution because these are only preliminary outcomes. The final report will look at all years combined following students from seventh to eighth grade. Several methodological limitations also require consideration. Both the control group students and intervention students attend the same school. Therefore, we do suspect control group contamination from the youth who are receiving programming.  In addition, students are not randomly assigned individually but rather are assigned by classroom allowing possible selection bias.

Implications

It appears that having a program for only one semester may not be sufficient. Youth in both the seventh grade and eighth grade may need an entire year of programming in order for the positive changes to remain over time, or at least may need a “booster” session intermittently. Although many changes were seen during the first semester, when no services were received during the second semester the youth often decreased in terms of the knowledge they gained during the first semester when they were receiving programming.  We have youth receiving only one semester of services in order to have a comparison group and in order to test the lasting results of programming

Contact information

Korrine Bentley                                                                           
Research Associate II                                                                 
University of Miami Miller School of Medicine                       
802-779-8122                                                                             
kcanas@med.miami.edu                                                           

Janet Mapp
REAL Program Director
Switchboard of Miami
305-358-11640
jmapp@switchboardmiami.org