Testimony

Statement by
Charles G. Curie, M.A., A.C.S.W.
Administrator
Substance Abuse and Mental Health Services Administration
U.S. Department of Health and Human Services

on
Performance Measurement and Performance Management
before the
Subcommittee on Substance Abuse and Mental Health
Committee on Health, Education, Labor and Pensions
United States Senate

July 20, 2004

Mr. Chairman and Members of the Subcommittee, good morning.  I am Charles G. Curie, Administrator of the Substance Abuse and Mental Health Services Administration (SAMHSA), part of the U.S. Department of Health and Human Services (HHS).

I am pleased to appear before you today to focus on performance and outcome measurement activities being undertaken by SAMHSA.  The issue of performance and outcome measurement is paramount, particularly since our budget for FY 2004 totals nearly $3.4 billion and since the President's FY 2005 budget request for SAMHSA raises that to almost $3.6 billion.  Moreover, they are issues with which we at SAMHSA have been grappling as a priority matter since I came on board as its Administrator.

I am happy to report that we are changing the way SAMHSA does business.  Instead of continuing a history of talking about performance measurement and management, we have taken action to achieve performance measurement and management across all SAMHSA programs.  Through decisive action - grounded in years of deliberation that have preceded it - we are poised to hold our discretionary and block grant recipients - and ourselves - accountable not only for how we spend, but also for how we serve people with or at risk for mental and substance use disorders.

SAMHSA Vision and Mission

We have good reason to believe that, working with our partners at the Federal, State and community levels, we can achieve SAMHSA's mission of building resilience and facilitating recovery.  We have good reason to believe that we can realize the SAMHSA vision of a life in the community for people nationwide with or at risk for substance use or mental disorders.  Both our vision and our mission are consistent with the President's New Freedom Initiative and with the precept that all people deserve the opportunity for a life that includes a job, a home, education, and meaningful relationships with family and friends.

Both research and clinical experience have shown that people with mental and addictive disorders can and do recover when they receive timely and effective care in their communities.  According to SAMHSA's 2002 National Survey on Drug Use and Health, an estimated 22 million persons, age 12 or older, needed treatment for an illicit drug problem or an alcohol problem, or both.  In the same year, an estimated 17.5 million people, age 18 and older, had serious mental illnesses.  An estimated 4 million adults experienced co-occurring serious mental and substance use disorders during the year.  Further, in any given year, about five to nine percent of children and youth have a serious emotional disturbance.

Unfortunately, we also know that for too many people, the need for care is not matched by the availability of evidence-based substance abuse treatment and mental health services to meet those needs.  Some people seek care and cannot get it; others do not seek it at all.  Under either circumstance, their quest for recovery and a life in the community are frustrated; our mission and vision are not being achieved.

The SAMHSA Role

As this Subcommittee is well aware, since I became SAMHSA Administrator, the Agency has been working in partnership with other Federal agencies, with States and with communities to improve how we approach substance abuse treatment and prevention and mental health services delivery.  By restructuring our work around the vision and mission, we have eliminated the functions that were not within our scope as a services agency.

As a result, our work has become more finely honed and our dollars more carefully directed - nurturing a few solid redwoods that can endure over time, instead of cultivating a garden of annuals pleasing for a season but with little lasting impact.

Further, to refine SAMHSA's program development and resources, we developed a Matrix of program priorities and crosscutting principles that pinpoints SAMHSA' s leadership and management responsibilities.  These responsibilities and program directions were developed as a result of discussions with members of Congress, our advisory councils, constituency groups, people working in the field, and people working to obtain and sustain recovery.  The content is dynamic - and will change over time.  We'll be able to know when we've reached a change point through performance measurement and management, both at SAMHSA and in communities and States across the country.

Today's Matrix priorities are aligned with the priorities of both President Bush and HHS Secretary Tommy Thompson whose support and confidence we greatly appreciate.  They have recognized that it is time that program and policy - and America as a whole - recognize that substance use and mental disorders should be treated with the same concern and urgency as diabetes, obesity, heart disease, stroke, and cancer.

To that end, they have supported key elements of SAMHSA's matrix:  transforming the mental health care system; improving services for people with co-occuring disorders; strengthening prevention efforts; expanding substance abuse treatment capacity; and, critically, performance measurement and management.

The ACE Principles

From the perspective of today's hearing, it is also critical that you know that we are building our priority programs around three key principles.  They are principles that, I am sure resonate with your interests and concerns about SAMHSA's programs and policy future.  I am speaking of the principles of Accountability, Capacity, and Effectiveness -ACE.

To promote accountability, SAMHSA tracks national trends, establishes measurement and reporting systems, develops standards to monitor service systems, and works to achieve excellence in management practices in addiction treatment and substance abuse prevention.  We are demanding greater accountability of our grantees in the choice of treatment and prevention interventions they set in place and in the ways in which program outcomes meet the identified needs for services.  Increasingly, we are promoting accountability - through performance measurement and management.

By assessing resources, supporting systems of community-based care, improving service financing and organization, and promoting a strong, well-educated workforce that is grounded in today's best practices and known-effective interventions, SAMHSA is enhancing the Nation's capacity to serve people with or at risk for substance use and mental disorders.

Further, SAMHSA also helps assure service effectiveness by assessing delivery practices, identifying and promoting evidence-based approaches to care, implementing and evaluating innovative services, and providing workforce training.  For example, our National Registry of Effective Programs and Practices - with 60 known effective prevention and early intervention programs in mental health and substance abuse - provides a foundation on which States and communities can build to meet prevention needs and reduce treatment needs.  Our Treatment Improvement Protocols (TIPS) bring the latest knowledge about effective interventions, including treatment for adolescents, co-occurring disorders, and treatment for older adults, to professionals in the field.  And our mental health services best practices toolkits, on topics ranging from medication management to assertive community treatment and from supported employment to illness management and recovery, are being tested in community-based settings across the country.

To measure our effectiveness and to be accountable, SAMHSA must have the capacity to gather and analyze data about our programs.  We are continuing to build on our long history of national surveys, such as the National Survey of Drug Use and Health (which now includes measures of mental health and illness), the Drug Abuse Warning Network and the Drug and Alcohol Services Information System (which includes the Treatment Episode Data Set (TEDS)).  At the same time, we are working with States to build the infrastructure needed to capture and evaluate their own measures and to identify and agree upon specific national outcome measures.

These national outcome measures, to the extent possible, have been drawn from already tested instruments in use by mental health and substance abuse authorities across the Nation.  Many States are already reporting or are substantially ready to begin reporting on these measures, thanks to this work.  Data on specific populations, including women and children, and racial and ethnic minorities, are being and will continue to be captured by these measures.  In this way, the majority of specific components of each measure already are known to and in use by many States, and come from existing data sets, discussed next.

Mental Health Data Sets

Since its inception, SAMHSA's Center for Mental Health Services (CMHS) has worked with the States to develop a mental health services data system, including the identification and specification of performance measures and data.  This resulted in the CMHS Uniform Reporting System (URS) that contains over 20 measures of mental health services, each reported by States in URS "data tables" in their CMHS Block Grant applications.  Today, most States can report on the basic measures contained in the URS.  These measures are indicated as change measures, since annual totals for these measures will be compared year to year.  Work is underway to develop more refined methodologies that can demonstrate system change and transformation.  Currently, under the CMHS Block Grant, States will be expected to report on all 20 URS measures and to establish performance goals and targets for mental health.  In the future, SAMHSA expects that the number of measures the States will report will be refined as specific measures are agreed upon for the Mental Health System Transformation effort.

Substance Abuse Treatment Data Sets

During the past several years SAMHSA's Center for Substance Abuse Treatment (CSAT) convened over 30 SAMHSA/State substance abuse agency meetings on performance measurement and funded two "Treatment Outcome and Performance Pilot Studies" (TOPPS) that resulted in careful identification and delineation of performance measures for substance abuse treatment.  The outcome measures identified through TOPPS included changes in client alcohol and drug use; changes in client illegal activity; changes in employment status; and, changes in homelessness.  Many States have been reporting on these measures voluntarily since 2000.  To add yet another way to help, we have created the Web Infrastructure for Treatment Services or (WITS) which is an interactive technology system designed to aid States in data collection.  I've seen and heard about amazing things done through these efforts - most recently in North Carolina, last fall in Texas, and last summer in Washington State.

In addition, Federal and State substance abuse treatment data also build upon the foundation of the TEDS admission data, generally available for most publicly funded programs throughout the States.  Information produced through a survey conducted by the National Association of State Alcohol and Drug Abuse Directors (NASADAD) indicates that most States exceed the minimum specifications of TEDS and are now collecting many of the relevant variables at discharge and beyond.  To this end, the handful of States that have on-going problems submitting their TEDS reports will be offered an opportunity to participate in a pilot State level operation to help determine which data collection and management system can best generate the most accurate data on a real-time basis.  SAMHSA believes that this will result in States being fully prepared to report on the same performance measures regardless of whether they are reporting on the Block Grants or discretionary grant programs.

Substance Abuse Prevention Data Sets

SAMHSA has also worked carefully over the years with State substance abuse prevention officials to specify and define performance measures for substance abuse prevention activities.  Since 1990, SAMHSA's Center for Substance Abuse Prevention (CSAP) and a group of State prevention officials have met regularly to identify and define the 30+ performance measures currently being addressed by the States as part of the State Incentive Grant program (SIG), many of which are taken from existing data sources, such as CSAP's Minimum Data Set (MDS).  In the future, SAMHSA expects to work with the States also to identify and finalize a smaller group of environmental measures - measures that address the impact of programs on the community or "environmental" level - that will be used in both discretionary programs and the prevention portion of the SAPT Block Grant.

These are all concrete examples of our steadfast commitment to build State data capacity to measure and manage performance.  This foundation has been laid to reorient ourselves to a State-friendly and consumer-friendly performance environment.

Our intention at SAMHSA is to keep moving forward with our partners.  Change comes with challenges.  One of the reasons this hearing is so important is to help ensure that we are moving forward together to meet the needs of people with or at risk for mental and or substance use disorders.

From Talk to Action: Measuring and Managing Performance

To help us present consistent and reliable information we have been developing and implementing a data strategy.  The strategy is simple:  The tighter our measurements become, the more we can prove our effectiveness.  The greater our effectiveness - the greater the number of people served, the greater the chances for a life in the community for everyone.  Developing a data strategy is a task that has been hanging around for years.  Now, we have gotten real about doing it.

Our SAMHSA data strategy is a critical building block to achieve true accountability in a performance environment by transforming the way we do business.  We are looking at what data we are collecting.  We are asking why we are collecting it.  And, we are asking how we are using it to manage and measure performance.  If we don't use it, we need to lose it.

We have learned that a limited number of key outcomes measured in structured ways can help all of us know how well SAMHSA and its grant programs are building resilience and facilitating recovery.  Our emphasis on a limited number of national outcomes and related national outcome measures is built on a history of extensive dialogue with our colleagues in State mental health and substance abuse service agencies and the people we serve.

While the discussions with States focused specifically on SAMHSA's block grant programs - something I will address in a bit more detail later in this testimony - the application of national outcomes and national outcome measures extends across all SAMHSA grant programs.  All of our programs are about achieving our vision of a life in the community for everyone and our mission building resilience and facilitating recovery.  So it only makes sense that we use the same outcomes across our programs.  And it only makes sense that we stop talking about national outcomes and start implementing them.

Naming The National Outcomes

So let me tell you more about the National Outcomes we have identified in our deliberations with the States.  Together we have highlighted specific domains of resilience and recovery as National Outcomes.  These are:

  • Abstinence from alcohol abuse or drug use, or decreased symptoms of mental illness;

  • Increased or retained employment and school enrollment;

  • Decreased involvement with the criminal justice system;

  • Increased stability in family and living conditions;

  • Increased access to services;

  • Increased retention in services (substance abuse) or decreased utilization of psychiatric inpatient beds (mental health); and

  • Increased social connectedness.

These domains are joined by additional outcomes identified by the OMB Program Assessment Rating Tool (PART) process - for example client perception of care, cost effectiveness, and use of evidence-based practices.  Together they constitute the National Outcomes that SAMHSA is applying to its discretionary and block grant portfolio activities.  Already, SAMHSA is implementing these National Outcomes, including them in the grant announcements for its Access To Recovery Program (ATR), and its Strategic Prevention Framework (SPF).  States have voluntarily been collecting and reporting performance information on a variety of measures for SAMHSA's Block Grants and we have required reporting on many of these measures in our discretionary programs, as is evident in our FY 2005 budget submission/GPRA plan and report.

Focusing on this handful of National Outcomes will minimize the reporting burden on the States and other grantees, and will promote more effective monitoring of client outcomes and system improvements.

SAMHSA has also worked carefully with the States to identify and agree upon specific performance measures for each of the National Outcomes.  These measures, to the extent possible, have been drawn from already tested instruments in use by mental health and substance abuse authorities across the Nation.  Now, we need to ensure that we collect the data in the same way across all of our programs, so that we can present aggregated results wherever possible.

However, some of the measures are developmental and require further work by SAMHSA and the States to delineate the best measures to assess progress toward reporting National Outcomes.  For mental health, such developmental measures include ones for decreased symptomatology, criminal justice involvement, school attendance, readmission rates, and number of persons receiving evidence-based practices.  For substance abuse treatment, developmental measures include those for stable living situation, unduplicated counts, length of stay, and services provided within cost bands.  For substance abuse prevention, developmental measures include those for returning to/staying in school, decreased criminal justice involvement, increased stability in family and living conditions, and cost effectiveness (increase services provided within cost bands).Other measures remain to be identified, including those for people with co-occurring disorders, the presence of both mental and substance use disorders.  Collecting data on co-occurring disorders poses unique challenges for States - especially for those with separate mental health and substance abuse treatment systems.  These systems will need to work together to identify measures and methods of measurement that will be reliable, valid, and non-duplicative, and to share data for reporting.  SAMHSA will continue to work with States to further develop and refine these measures.

Implementing New Directions Means Supporting System Change

Critically, the implementation of the National Outcomes is being accompanied by a real-time infusion of SAMHSA support for the improvement of the data infrastructures in place at the Federal, State and local levels to manage this sea change from counting to accounting for success.

As an illustration of SAMHSA's commitment to performance measurement, we will have invested just over $277 million in data infrastructure and related technical assistance to the States over the past five years, up from $49 million in FY 2001 to a requested $66 million in FY 2005, consistent with the President's FY 2005 Budget.

The following table provides greater detail regarding SAMHSA's commitment to States to build the data infrastructure needed to make performance measurement and management realities in how States do business with communities and with SAMHSA, and how SAMHSA does business to achieve its vision and mission for the American people.

SAMHSA Resources for Performance Measurement and
Performance Management (in millions)

SAMHSA Center

FY2001
Actual

FY2002
Actual

FY2003
Actual

FY2004
Estimate

FY2005
Requested

Total

CMHS

$12.2

$12.6

$13.7

$14.9

$15.8

$69.2

CSAP

$10.1

$10.6

$8.5

$12.3

$17.5

$59.0

CSAT

$22.8

$25.4

$26.8

$25.4

$28.4

$128.8

OAS

$3.9

$3.9

$4.0

$4.1

$4.1

$20.1

Total

$49.0

$52.5

$53.0

$56.7

$65.8

$277.1

Performance Partnerships - The Block Grant Programs

All of this leads me to the status of Performance Partnership Grants (PPGs), one of the topics I know is of both interest and concern to this Subcommittee.  After all, Congress, in its 2000 reauthorization of SAMHSA, called for the transformation of the existing substance abuse prevention and treatment block grant and the mental health services block grants into performance partnership grants.

The goal and intent of PPGs were clear - to promote greater flexibility and to infuse greater accountability into the block grant program.  I've already described the years of discussion we have had with State mental health and substance abuse authorities.  I have described the collaboration over that time with them that led to the identification of the National Outcomes on which our performance measurement and management focus.  And I have described the broad range of existing data sets and outcome measures - many of which already are in place.

Yet, what I discovered when I moved from the State of Pennsylvania to the Federal side of the PPG equation, was that clearly, the PPG process had gotten in the way of achieving the PPG purpose.  Talk and debate and discussion had gone on far longer than necessary:  a decade and multitudes of meetings and workshops on block grant performance measurement alone.  SAMHSA had funded data-related grant programs and data collection activities.  SAMHSA had analyzed them and reanalyzed them.  And SAMHSA had made agreements and then remade the same agreements.

As a result, Performance Partnerships still had not happened when I reached SAMHSA.  Process had supplanted progress. The Report we were to submit to Congress on our progress on Performance Partnerships was drafted, but its focus was on the process and not on the action.  A recent GAO report reminds us that we owe Congress that report.

In general, the Report delineates how we are changing the relationship between the Federal and State governments to create more flexibility for States and accountability based on outcome and other performance measures.

By using the National Outcomes, we are changing the questions from "How did you spend the money" and "Did you stay within the spending rules?"  Instead, we are asking questions relevant to building resilience and facilitating recovery, questions like "How did you put the dollars to work?" and "How did your consumers benefit?"

As the change in questions suggests, our focus is squarely on National Outcomes and National Outcome Measures.  The National Outcomes are true measures of recovery.  They assess whether our programs are helping people attain and sustain recovery.  They show that people are achieving a life in the community - a home, a job, and meaningful personal relations.

Clearly, the time for action is long past.  Somehow, we lost sight that block grants are a means to build resilience and facilitate recovery.  Instead, the goal became implementing PPGs solely for the sake of implementing them and not the implementation of performance measurement and performance management.

That is why we are moving forward with our National Outcomes and National Outcome Measures across all of SAMHSA's funding streams.  They will reduce State and community reporting requirements while simultaneously presenting reliable information to you, to other key stakeholders and to SAMHSA about the effectiveness of our services and how they are being applied across the country.

Conclusion

As this testimony suggests, SAMHSA has invested a decade preparing for action, debating about action, and thinking about action.  The time for preparation is over; the time for implementation is now.  We have the knowledge, we have the capacity, and we most certainly have the obligation to be accountable to the American taxpayer - and to you - to show that what we do, what we fund, and what we propose in policy are effective.  Beyond this obligation, we have a responsibility to the millions of Americans who are battling addiction; struggling with a serious mental illness or emotional disturbance; or are fighting a co-occurring serious mental and substance use disorder and their families to put into motion this long-overdue due diligence.

That is why, in our programs, our grant announcements, and our policies, we are taking that long-overdue action.  We have looked to the past and found the delays unacceptable.  And we have looked to the future and found our direction clear.

It is built on the solid ground of customer service - making decisions based on the needs of the people we serve, not on the needs of bureaucracies.  The driving force for our work - as verbalized in our vision and mission - is what people with or at risk for substance use or mental disorders desire - the hope of recovery and a life in the community.  We must open ourselves to accountability for the work that we do for you; for our many partners and for the public health of this nation.

Substance Abuse and Mental Health Services Administration National Outcome Measures

Last Revised: July 21, 2004