Collaborative metrics development
Continuous improvements strategies rely on regularly occurring evaluations to set benchmarks and measure progress. To be effective, consistent metrics must be utilized and compared. The notion of “consistent metrics” can be challenging when multiple stakeholder groups are involved. In several priority areas, the Department has played a lead role in developing common metrics as a tool for measuring our progress towards a stated goal or set of outcomes as well as improving program integrity. Examples of areas in which HHS has taken a lead role in metrics development include:
- Single Audit Metrics Initiative
The Administration for Children and Families (ACF) carries out a large portion of its mission through grants to non-federal entities including states, Indian tribes, local governments, and non-profit organizations. ACF relies on the “Single Audit” process as a key control over its grant programs to provide assurance that grantees properly administer taxpayer funds consistent with applicable laws and regulations and the terms and conditions of each particular grant. The initiative goal is to hold grantees and ACF program offices accountable to ensure deficiencies identified by Single Audits, which cause an unclean audit opinion, are corrected in a timely manner. Metrics provide a means to gage success in improving ACF program integrity. To achieve this goal, careful analysis of Single Audit findings is performed within and across ACF grant programs, as well as across HHS grantees. More importantly, the use of such metrics may lead to the development of strategies for addressing chronic and pervasive internal control and compliance weaknesses in ACF grant programs and the entities that administer them.
ACF is performing a careful analysis of unclean Single Audit findings across ACF grant programs. Supported by a web-based data collection system, OMB MAX, ACF is identifying those Single Audit findings that pose a risk to ACF programs, working with grantees and auditors to identify root causes, obtaining grantee commitment to strategies to correct materially noncompliant findings, and tracking the grantee’s progress in resolving those findings. ACF is developing the Audit Resolution Tracking and Monitoring System, ARTMS, to enhance the Department’s ability to meet OMB’s challenge for consistency and uniformity in audit resolution and debt management. Through the ACF SharePoint site, ACF is also working to establish a repository of best practices when working with grantee’s to resolve findings for ACF staff across its regions. ACF’s goal is to achieve 100% clean audit opinions in target programs by FY 2013.
- STAR METRICS
NIH has taken the lead role in developing STAR METRICS (Science and Technology for America's Reinvestment: Measuring the Effect of Research on Innovation, Competitiveness and Science), a partnership between science agencies and research institutions to explore new ways to document science investments and communicate results to the public. The long-term goal of STAR METRICS is to develop a common empirical infrastructure that will be available to all recipients of federal funding and science agencies. STAR METRICS is led by NIH, the National Science Foundation (NSF) and the White House Office of Science and Technology Policy (OSTP). Effective January 2012, NIH became the host institution for the STAR METRICS consortium which presently consists of NIH, NSF, EPA, USDA, DOE, and OSTP. STAR METRICS has two levels of activity. The Level I goal is to describe the scientific workforce supported by federal funding. Level II is aimed at developing an open data infrastructure and tools to enable the documentation and analysis of the inputs, outputs, and outcomes of federal investments in science. The STAR METRICS initiative began roughly in the second half of 2010, and NIH committed to five years. A formal governance structure was proposed and ratified by agency representatives in January 2012. More information about the STAR METRICS initiative can be found at: https://www.starmetrics.nih.gov