DHHS Eagle graphic
ASL Header
Mission Nav Button Division Nav Button Grants Nav Button Testimony Nav Button Other Links Nav Button ASL Home Nav Button
US Capitol Building
HHS Home
Contact Us
dot graphic Testimony bar

This is an archive page. The links are no longer being updated.

Statement on Effects of Changes to the Welfare System by Howard Rolston
Director, Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services

Before the House Committee on Ways and Means, Subcommittee on Human Resources
May 27, 1999

Madam Chairman, members of the Subcommittee, I am pleased to be here today to discuss the critical role of research and evaluation in the implementation of welfare reform. This Subcommittee played a central role in ensuring that these activities would be an integral part of the Temporary Assistance to Needy Families (TANF) program.

A central focus of the Department of Health and Human Services' (HHS') welfare reform research and evaluation effort is to develop reliable, credible information about how different strategies are working in order to inform federal and state policy makers and the public about how welfare reform is progressing and how families are faring. Today I will describe key elements of the Department’s welfare reform research agenda, what we have learned so far and what we will learn in the future. Our efforts have two major parts: working in partnership with states to develop in-depth information; and using national data to understand the overall progress of welfare reform. (There are other important areas, however, for brevity I'll focus on these two parts today.)

State Evaluation Activities

The broad flexibility provided by TANF creates the opportunity and the challenge to evaluate a variety of different approaches to welfare reform to determine which strategies are most effective in moving families to work and self-sufficiency. A state-federal partnership is critical to any successful strategy to evaluate welfare reform for several reasons.

  • Studies of processes, programs and populations that are carried out at a local level can have a depth that is not possible at the national level, especially given the great flexibility in program design that TANF permits.
  • Studies that employ random assignment methods, which are the most rigorous methods for measuring the magnitude of the effects of alternative policies and program designs, must be implemented at the state or local level.
  • States and localities have access to a rich set of administrative data, including linked administrative data sets, that can be used for tracking recipients and former recipients.
  • Involving states as partners assures that the information produced will be relevant to state policy makers and thereby greatly increases the likelihood that states will use the information as they re-design their TANF programs.

We are grateful that Congress has appropriated funds for the Administration for Children and Families (ACF) and the Office of the Assistant Secretary for Planning and Evaluation (ASPE) to allow us to fund over three-quarters of the states for activities related to the assessment of welfare reform. Some of the primary areas for these grants include:

  • Evaluations of Specific Employment Programs and Welfare Reform Policies Using Randomized Field Experiments--As authorized under TANF, we provided funds to nine states to continue evaluations that began under waivers and use random assignment to address the effects of alternative welfare reform programs and policies. In addition, we continued previous multi-site experiments of welfare-to-work strategies and post-employment services. Recently, we also funded two demonstrations to rigorously evaluate: (1) an innovative strategy to identify and provide treatment linked to employment and training for welfare recipients who are substance abusers, and (2) an innovative approach that combines a strong commitment to work (requiring at least 20 hours of work a week) with a strong commitment to post-secondary education.
  • Welfare Leavers--We are providing grants to 11 states and three counties (or consortia of counties) to collect and analyze, using a combination of administrative and survey data, information on families that have left welfare. In the next several months we intend to fund additional studies with an emphasis on studies of families that have been formally or informally diverted from coming on the rolls.
  • Implementation--We are funding numerous studies of the implementation of particular policies and programs in order to identify problems and solutions. For example, we're funding a study of local devolution across the counties of Maryland, a study of implementation of state TANF in rural and tribal areas in Montana, and a study of relocation of families from economically depressed areas in South Carolina.

Two weeks ago the strength of the state-federal partnership was demonstrated in ACF's Second Annual Welfare Reform Evaluation Conference. The two and one-half day meeting was attended by about 300 people from state and federal agencies, universities and policy research and evaluation organizations. The attendees included research and program staff from 49 states plus the District of Columbia who participated in a lively meeting both presenting and hearing the latest findings from evaluations. The gathering also provided the venue for two one-day meetings: an ASPE/ACF-sponsored meeting for states conducting studies of families leaving assistance and a meeting of researchers and states working on a book to improve the design of implementation studies.

Analysis of National Data

While state and local studies provide us with depth, only national studies can provide nationally representative information about how families are faring overall. Another important source of national data is, of course, the national TANF data reported to HHS by the states. These data provide information on the characteristics and conditions of families that continue to receive TANF assistance. There are also a number of surveys that have long been central to the study of welfare issues. All capture information on income from various sources including earnings and transfer payments, and several contain detailed information on employment, childbearing, family structure, and child well-being. While the strength of these surveys is their ability to capture very detailed and rich information, they also generally suffer from some degree of underreporting, and there is some evidence that underreporting of Aid to Families with Dependent Children (AFDC)/TANF benefits is increasing over time. Specific examples of the most important national surveys for assessing welfare reform follow.

  • The Census Bureau's March Income Supplement to the Current Population Survey (CPS) provides an annual snapshot of the economic and employment condition of families.
  • The Census Bureau also conducts the Survey of Income and Program Participation (SIPP) which collects in-depth information three times a year on families' income and program receipt over four-year periods. This Committee was instrumental in providing an additional $70 million to the Census Bureau to extend two panels of the SIPP and create the Survey of Program Dynamics (SPD) which will track some families up to ten years.
  • The Panel Study on Income Dynamics (PSID) has tracked a representative sample of families for nearly 30 years, collecting in depth information on income, family formation, and program participation.
  • The National Longitudinal Survey of Youth (NLSY) has tracked a representative sample of youth and young adults for nearly 20 years collecting detailed information on how these individuals have moved into adulthood.

No one national survey will provide us with all the information needed. It is important to link information across surveys, and also to take advantage of administrative data. For example, we are funding a project to link SPD data to earnings records kept by the Social Security Administration, both to get a better understanding of biases in that SPD sample that may result for attrition, and to get a longer earnings history to complement the survey data.

Except for the CPS and one quarter of TANF administrative data, post-TANF data from these national sources are not yet available, but ultimately they will provide a critical complement to data derived from state and local sources. An important part of what I'll discuss today is based on some early analysis, by staff in the Administration, of CPS data.

What We've Learned So Far

The employment of welfare recipients and former recipients has increased significantly.

A key measure of the success of welfare reform is its effect on employment. Analysis of all available sources of information shows that the employment rate of current and former TANF recipients has increased significantly. Each March the CPS, which is used to calculate unemployment rates, collects information about households' income and program participation in the previous calendar year in addition to employment and earnings data reflecting individuals' March employment status. As a result we know whether adults who received AFDC or TANF in the preceding calendar year were employed the following March. Between 1992 and 1996, the employment rate increased from 20 percent (its approximate level for the previous four years) to 27 percent. However, in the last two years it jumped even more dramatically to 34 percent in 1998. Thus, whereas in 1992 one in five previous year recipients was working the following spring, in 1998, the figure was one in three.

Large employment gains are also evident from rigorous waiver evaluations that measure the effects of reform policies by comparing randomly assigned individuals who were subject to either welfare reform or standard AFDC rules. Unlike the CPS analysis, which does not separate out the effects of state welfare reform policies from those of the economy, other policies which promote employment such as the enhancement of the Earned Income Tax Credit (EITC) or the expansion of child care subsidies, the strength of experimental studies is that they do precisely that. Several studies examined policies which are typical of state TANF programs in that they increase participation in mandatory work activities and/or increase the amount of assistance a family can receive when they go to work. The persistent employment effects of these programs are in the five to 13 percentage point range. I would also note that these are probably quite conservative estimates in that the treatment groups are compared to control groups which received a substantial level of mandatory employment services and also were not isolated from the atmosphere of welfare reform, even though they did not directly experience welfare reform policies.

Preliminary findings from four of the ASPE-funded studies of families leaving welfare indicate that between one-half and three-fifths of former TANF recipients found work in jobs which were covered by their state's Unemployment Insurance program. Employment rates were even higher – 75 to 82 percent – when measured as the percentage of those who were ever employed within the first 12 months. These employment rates are consistent with findings in many other leavers’ studies, although methodological differences cause rates to be slightly higher in some studies (e.g., rates are sometimes higher in studies using survey data, or limiting study population to leavers who do not return to welfare). While these employment rates are not radically different from the patterns of AFDC leavers in earlier studies, they indicate a dramatically large increase in the absolute number of families leaving welfare with earnings, given the significant caseload decline in the past few years.

Finally, there has also been a significant increase in employment of current welfare recipients. Between 1992 and 1997, the percentage of adults on welfare in direct work activities (including employment, work experience and community service) has tripled. All states met the overall work participation rate requirement for 1997. And, the percent of TANF adults who were employed rose from 14 percent in the first three quarters of 1997 to 18 percent in the last quarter of 1997. Thus, each of these sources of information consistently points to higher levels of employment, among current and former welfare recipients.

Earnings have also increased, but not uniformly.

A second important measure of success in welfare reform is whether welfare recipients and former recipients are earning more. Although welfare reform is having a positive effect on the earnings of some categories of recipients, the story here is somewhat more complicated than the employment story. For example, an examination of welfare reform waiver demonstrations suggests that those programs which were strongly oriented toward increasing employment activities and mandatory participation (as measured by an increase in participation and sanction rates) achieved annual earnings gains in the range of $600-$700 for a least one primary target group of applicants or recipients. One employment and training program in Portland which combined a strong employment focus, an emphasis on moving recipients into higher paying jobs with benefits, and the provision of necessary child care produced even larger effects with averaging earnings gains of over $900 per year. Of state welfare reform approaches which relied primarily on incentives without work requirements, earnings gains were not observed.

Along with the employment gains described above, the CPS data suggests average earnings for all female-headed families with children have increased substantially between 1993 and 1997 from $14,668 to $17,646 (both in 1997 dollars). However, the early CPS suggest preliminarily that the gains are not evenly distributed over the period with roughly three-quarters of the gain occurring between 1993 and 1995, and only one-quarter between 1995 and 1997. In addition, while employment gains for the bottom fifth of female-headed families with children were stronger from 1995 to 1997, the average earnings of this group increased from 1993 to 1995 but did not increase from 1995 to 1997. Better understanding of these trends will require both longer term follow up and analysis of other national data sets as they become available.

Finally, TANF administrative data just for welfare recipients who remain on the rolls indicate that average monthly earnings for those who are working increased substantially from $506 in the first three quarters of FY 1997 to $592 in the last quarter.

Income has increased for some families, but there is also some preliminary evidence that some families are experiencing losses.

Income is another central measure of how families are faring under welfare reform. Here our results are even more preliminary than for employment and earnings, although we will have much better data over time. Much of our current information relies on administrative records which typically examine family income defined as the total of TANF, Food Stamps and earnings. However, these analyses do not take into account other sources of income, such as the EITC, child support and Supplemental Security Income (SSI); the income of other household members; in-kind supports such as child care or Medicaid; nor, on the other side of the ledger, the expenses that families incur when they are working. A few current data sources such as the CPS and some early studies of families leaving welfare are based on household surveys, and many more of our studies ultimately will have this information.

Data from the four waiver evaluations in which the reform program succeeded in increasing mandatory work activities, employment and earnings suggest mixed effects on family income, depending on the generosity of benefit levels and earnings disregards. In the two states with both generous benefits and earnings disregards, there were increases in average annual income of $762 for applicants in Iowa, and $1,065 for long term recipients in Minnesota. In Florida, a program that accomplished comparable earnings gains, but had low benefits and generous earnings disregards raised family income by $289, whereas a fourth program in Indiana that accomplished comparable earnings gains but had low benefits and retained the standard AFDC earnings disregards had no effect on income.

Examination of the Florida findings also suggests that these effects are not uniform across recipients and that higher-skilled recipients may gain income, whereas the income of lower-skilled recipients may decline. In Florida, recipients who had both a high school degree and recent work experience averaged $752 higher average annual income for the three years following entry into a welfare reform program, while those with neither experienced losses of about $485. This gain/decline pattern is consistent with patterns in some earlier leaver studies. For example, a study in Iowa of families that lost their entire benefit because they failed to establish a self-sufficiency plan showed that about 40 percent increased their income, about 50 percent suffered a decrease, and about 10 percent had unchanged income.

For the period 1993 to 1997, CPS data indicate that the average annual income of all female-headed families with children increased, as did employment and earnings as described above. This measure of income includes both earnings and a broad range of transfer programs. Again, the income increases were unevenly distributed over the period, with larger gains in the 1993 - 1995 period, and across the income distribution. The bottom quintile did not fare as well as the top four fifths, especially in the 1995-1997 period, suggesting preliminarily that we need to be alert to monitoring more disadvantaged families.

What We Will Learn

As described above all the findings I've noted are preliminary. In almost all the studies I've described above, additional administrative data are being collected and surveys are being fielded or will be in the next several years. In the national surveys, data for 1997 and 1998 have been collected and are being processed. Thus, time and resources are necessary to produce more definitive results. I'll describe some of the more important information that we'll be obtaining.

Longer Term Follow-Up

Our findings are currently based on follow-up periods ranging from 6 months to 3 years. To understand the effects and outcomes of welfare reform will require longer term follow-up on the order of four to six years. Particularly important is that in most states, very few individuals have reached time limits, and in many larger states, none have.

More Comprehensive Information

As indicated above most of our current measures of employment, earnings and income are from administrative records. Over time we will obtain much more comprehensive information through surveys, both in conjunction with waiver demonstrations and through studies of families that have left TANF. Most importantly, we will be able to learn a great deal more about the income families have to support themselves and in particular, what happens to families who leave welfare and do not have earnings.

More Outcomes, Especially Related to Child Well-Being

A critical measure of the success of welfare reform is how it affects children. In 1996 ACF provided grants to 12 states to work with a team of researchers (funded by ASPE) to develop measures of child well-being to examine how different welfare reform programs and policies are affecting children. The partnership proved to be very successful, and subsequently ACF has augmented the funding of five state welfare reform evaluations (Connecticut, Florida, Indiana, Iowa, and Minnesota) to rigorously evaluate the effects of welfare reform on family processes and child well-being. In addition, ASPE has provided funds to 13 states to work with the Chapin Hall Center for Children at the University of Chicago and other experts to develop indicators of child well-being. Some of the 13 states are exploring using similar measures to those used in the five states as well as other administrative and survey data. The importance of measuring child well-being is vividly illustrated by the presentation you are hearing today on the New Hope demonstration. A central finding of the interim effects of this program designed to support working families was that it increased significantly boys' school performance along with increasing their participation in extended day child care and other structured activities, while having other positive family effects.

More Rigorous Information about What Works and What Doesn't

Because of caseload reduction, states now have significant financial resources that are not required for immediate cash assistance, and thus are available for investments in those families still on the caseload, including those with the greatest problems, and to enable families who leave the rolls to keep their jobs and move up instead of returning to welfare. As a result, many states are increasingly focused on strategies to increase job retention and advancement for recipients and former recipients. ACF is working with thirteen states to develop, pilot and ultimately rigorously evaluate the effects of alternative strategies. Through this activity and others, we will be learning about the role of supports for working families such as child care, child support and other services in sustaining and advancing in employment. These kinds of evaluations are critical to using the flexibility provided by TANF to maintain a learning environment in which federal and state tax dollars are used to make investments that really work.

More Information about Sub-Populations, especially the Harder-to-Employ

As more in-depth information is developed it will be possible to understand better how particular sub-populations are faring. We currently have projects underway to examine issues related to: non-custodial parents, rural populations, families with disabled members, Native Americans, victims of domestic violence, child-only cases and families with mental health or substance abuse problems. In addition, we will learn more about the impact of strategies to serve the hardest-to-employ through our Department's recently begun evaluation of the Department of Labor's Welfare-to-Work grants program. The evaluation, which is at an early stage, will include an examination of the impact of Welfare-to-Work on participants' employment and wellbeing. We will share these results with Congress as the project progresses.

More Nationally Representative Data

Over time as more post-TANF data become available from the national surveys, especially longitudinal data, it will be possible to merge findings from in-depth studies in states and localities with nationally representative data and use the strengths of each to develop a comprehensive picture of how the nation's families are faring under welfare reform.


The preliminary results I've described above illustrate the promise of how investments in careful research and evaluation can produce information that can inform policy makers at all levels. Perseverance in these investments can play a critical role in supporting strategies that can realize the goals of welfare reform.

I will be happy to answer your questions.

Privacy Notice (www.hhs.gov/Privacy.html) | FOIA (www.hhs.gov/foia/) | What's New (www.hhs.gov/about/index.html#topiclist) | FAQs (answers.hhs.gov) | Reading Room (www.hhs.gov/read/) | Site Info (www.hhs.gov/SiteMap.html)