HHS Strategic Plan for Data Center Optimization
The Data Center Optimization Initiative (DCOI) supersedes the Federal Data Center Consolidation Initiative (FDCCI) passed in 2010 to promote the use of green Information Technology (IT) by reducing the energy and real estate foot print of government data centers; reducing the cost of associated data center hardware, software, and operations, increasing the IT security posture of the Federal Government and shifting IT investments to more efficient technologies. M-16-19 promulgates new definitions for data centers and targets for optimization.
While HHS has already been working toward the goals of the FDCCI, the DCOI includes revised definitions of what constitutes a data center and which data centers should be considered tiered versus non-tiered. These new definitions significantly impact the agency’s existing plans for meeting data center optimization goals as well as the metrics used to track progress toward those goals.
1.1. What is a Data Center?
Per M-16-19, any room with at least one server, providing services (whether in production, test, staging, development, or any other environment) is considered a data center. The complete definitions of tiered and non-tiered data centers compared to the FDCCI definition are below.
1.1.1. FDCCI vs. DCOI Data Center Definitions
One of the key distinctions between FDCCI and DCOI is the abandonment of the core vs. non-core data center definition and the consolidation of tiered data centers into one category. DCOI classifies all data centers as either Tiered or Non-Tiered.
Figure 1: FDCCI vs. DCOI Data Center Definitions
|FDCCI Definitions||DCOI Definitions|
Data centers previously classified as tiered in past inventories will automatically be classified as tiered under the DCOI.
|Tier 2 (Includes Tier 1 capabilities)
Capacity Components: Tier 2 facilities include redundant critical power and cooling components to provide select maintenance opportunities and an increased margin of safety against IT process disruptions that would result from site infrastructure equipment failures. The redundant components include power and cooling equipment such as UPS modules, chillers or pumps, and engine generators.
|Tier 3 (Includes Tier 1-2 capabilities)
Concurrently Maintainable: A Tier 3 data center requires no shutdowns for equipment replacement and maintenance. A redundant delivery path for power and cooling is added to the redundant critical components of Tier 2 so that each and every component needed to support the IT processing environment can be shut down and maintained without impact on the IT operation.
|Tier 4 (Includes Tier 1-3 capabilities)
Fault Tolerance: Tier 4 site infrastructure builds on Tier 3, adding the concept of Fault Tolerance to the site infrastructure topology. Fault Tolerance means that when individual equipment failures or distribution path interruptions occur, the effects of the events are stopped short of the IT operations.
All other data centers shall be considered non-tiered data centers.
All other data centers shall be considered non-tiered data centers.
1.2. What is NOT a Data Center?
The following are not considered data centers:
- Telecom closets
- Rooms with only print servers or network equipment
- Any room or closet with equipment dedicated to network, telecommunications, and other non-user facing equipment
1.3. Policy Guidance
In December 2014, the President signed into law the Federal Information Technology Acquisition Reform Act (FITARA), which enacts and builds upon the requirements of the FDCCI. FITARA requires that agencies submit annual reports that are to include: comprehensive data center inventories, multi-year strategies to consolidate and optimize data centers, performance metrics and a timeline for agency activities, and yearly calculations of investment and cost savings.
In addition, FITARA requires the Administrator of the Office of E-Government and Information Technology (henceforth referred to as the Office of the Federal Chief Information Officer (OFCIO)) to establish and publish cost savings and optimization improvements, provide public updates on cumulative cost savings and optimization improvements, and review agencies’ data center inventories and the implementation of data center management strategies.
Per the OFCIO, agencies must align the FITARA Baseline Milestone dates and DCOI Milestone due dates. DCOI milestones are required to be added to files online by September 30, 2016. The next reporting period for FITARA baseline AND DCOI milestones is November 30, 2016. Quarterly updates will continue as part of the Integrated Data Collection (IDC).
The $20 million spending authorization for life cycle projects under FITARA is not applicable to DCOI efforts. Any expenditure on data centers would have to be approved by the OFCIO.
Per the FITARA HHS Implementation Plan:
At the Department-level, the HHS [Chief Information Officer] CIO will work closely with the [Chief Finance Officer] CFO, [Chief Acquisition Officer] CAO, Division leadership, Division CIOs, and governance structures to have visibility throughout the planning, programming, and budgeting processes to assure he/she has insight and understanding of the overall Departmental IT portfolio as well as prospective insight into major new programs and the enabling IT required. The primary responsibility and focus for the HHS CIO, CFO and CAO and governance groups will be the approval of IT budget, acquisitions (through acquisition strategy approval), and investment lifecycle for projects over $20 million annually or $100 million over five years4 4 The CIO and CFO will lead an effort to evaluate the threshold after a cycle of implementation. as well as selected critical investments within a limited number of special Department-wide interest areas (e.g., cybersecurity).
Beginning 180 days after issuance of this memorandum, agencies may not budget any funds or resources toward initiating a new data center or significantly expanding an existing datacenter without approval from OMB OFCIO.
Continued expenditures on new or expanded data centers would undermine the intent of DCOI.
2. The Future
It is the mission of the U.S. Department of Health & Human Services (HHS) to enhance and protect the health and well-being of all Americans. We fulfill that mission by providing for effective health and human services and fostering advances in medicine, public health, and social services.
HHS is committed to a healthier and greener future for the US, and its strategic goals clearly provide a path forward to achieve that by strengthening health care; advancing scientific knowledge and innovation; advancing the health, safety, and well-being of the American people; and enduring efficiency, transparency, accountability, and the effectiveness of HHS programs.
Manage collaborative, secure, cost-effective, and strategic IT solutions and services that enable HHS to help provide the building blocks Americans need to live healthy, successful lives.
HHS OCIO’s IT Strategic Goals provide the technical and architectural support for HHS to achieve its over-arching strategic goals:
- IT Workforce: Acquire, deploy, and sustain a technology-enabled workforce
- Cybersecurity and Privacy: Minimize occurrence and impact of Cybersecurity incidents
- Shared Services: Optimize the ability to accomplish the mission by sharing business systems and services
- Interoperability and Usability: Promote usability, interoperability, data sharing, and integration
- IT Management: Mature IT management and governance to improve stewardship of IT investments and acquisitions
Data is an integral part of not only the IT goals, but all HHS strategic goals as well. It is used to empower the workforce, advance scientific inquiry and development, provide innovative health care solutions, and support management of programs and infrastructure. In his article “How Big Data is Transforming the Healthcare Sector”, Bernard Marr, business and big data expert and CEO of the Advanced Performance Institute, summarizes the impact big data is having on health care including prevention, diagnosis, treatment, and follow up care of patients.
The potential to improve patient outcomes, understand disease — even cure cancer — all seem just around the corner with these advances in the quantity and quality of the data we collect along with the computing power to analyze and understand it. – Bernard Marr
The impact of big data and the management, storage, and protection of that data now extends far beyond the traditional realms associated with IT. The alignment of data center optimization and shared services is clear, but big data also has the potential to not only improve, but save lives. Given the importance of this information, it is more critical than ever that it be managed and maintained in an efficient and accessible manner.
In addition to the direct impacts data and data management have on the health of the American people, providing a more efficient, green, cost-effective data center structure reduces HHS’s environmental footprint and reduces spending on power and superfluous equipment.
According to an article published by Total Site Solutions, green data center initiatives provide improved performance and efficiencies for equipment maintenance as well.
“With the appropriate power management policies and measurement systems, the data center manager will have a greater understanding of, and control over, power consumption. This enhanced control enables more successful execution of equipment refresh, updates and maintenance. It also provides superior visibility over the performance and efficiency of the entire network, facilitating the ability to pinpoint areas of opportunity and demonstrate the effectiveness of greener strategies.”
The article also estimates that for every dollar spent on a server, $.50 is spent to power and cool it. While HHS initiatives cannot impact the raw costs of energy, those initiatives can reduce consumption, which in turn lowers energy costs.
To date, under FDCCI, the Department reached 98% of the 100% goal of electronics with power management features enabled in FY 2015. HHS closed eight data centers in FY2015 as well.
DCOI provides a framework for more energy efficient data centers across HHS, allowing faster access to and analysis of the wealth of data stored by HHS and its members, reduced power consumption, and lower energy costs. By leveraging best practices and lessons learned from other Agencies, HHS can move forward with DCOI compliance, and by extension, a healthier and brighter future for the American people.
2.1. Optimization Requirements
In addition to moving forward with the HHS strategic goals, the Department must comply with the DCOI requirements set forth by OMB.
2.1.1. Cost Savings and Closures
Within 30 days of the August 1, 2016 publication of M-16-19, the Office of Management and Budget (OMB) provided all agencies with targets or requirements for compliance with DCOI.
Figure 2: OMB-Mandated DCOI Requirements
The numbers of data centers are based upon the Fiscal Year (FY) 2016 Quarter 3 IDC.
2.1.2. Performance Metrics
OMB also mandated the following performance metrics in M-16-19.
Figure 3: OMB-Mandated Performance Metrics
FYE 2018 Target
(%) Percent of total gross floor area (GF A) 25 in an agency's tiered data center inventory located in tiered data centers that have power metering.
Total GFA of Energy Metered
Power Usage Effectiveness (PUE)
(Ratio) Proportion of total data center energy used by IT equipment.
Total Energy Used ______________________
≤ 1.5 ( ≤ 1.4 for new data centers)
(Ratio) Ratio of operating systems (OS) to physical servers.
Total Server Count + Total Virtual OS ______________________
Server Utilization & Automated Monitoring
(%) Percent of time busy (measured as 1 -percent of time spent idle), measured directly by continuous, automated monitoring software, discounted by the fraction of data centers fully equipped with automated monitoring.
Average Server Utilization
(%) Portion of total gross floor area in tiered data centers that is actively utilized for racks that contain IT equipment.
Total Active Rack Count X 30 sq. ft ______________________
In both the private and public sector, the nature of data storage and services is changing. Gartner estimates that by 2019, Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) will become a common approach for 80% of enterprises. That is up from less than 10% in 2015; however, Gartner estimates that through 2017, more than 50% of organizations that evaluate moving to IaaS will remain with their existing architecture. They also estimate that 10% of organizations will close their onsite data centers by the end of 2018.
This shift away from large numbers of isolated, physical data centers is the heart of DCOI. M-16-19 recommends progress toward cloud-based and virtualized data centers, and shared services.
DCOI’s strategy recommends, in order of priority, evaluating the following options for consolidation and closure:
- Transitioning to technology such as IaaS, PaaS, and Software as a Service (SaaS), “to the furthest extent practicable” with a focus on Cloud First
- Pursuing internal shared services and co-located data centers
- Using more optimized data centers within the agency
2.2.1. HHS Goals
HHS’s goals for data center consolidation and optimization are in alignment with DCOI mandates:
- Leverage Cloud-based technologies where feasible under budgetary and security constraints
- Reduce expenditures on data center operations by optimizing power usage and real estate, automating monitoring and power metering, and implementing best practices
- Employ and optimize Green IT to reduce power consumption and cooling needs
- Leverage virtualization strategies and technology where applicable and feasible
- Reduce costs on infrastructure through efficient, Green IT purchases and internal shared services where optimal
- Employ the most efficient architecture of datacenters that consolidates where appropriate, allows for geographically dispersed datacenters where appropriate, and leverages cloud and distributing computing where appropriate.
- Eliminate or minimize security risks for all data centers
2.2.2. Leveraging Best Practices
HHS will implement data center best practices in order to meet the OMB-required metrics for data center and server performance. The General Services Administration (GSA) published its “Evidenced-Based Best Practices Around Data Center Management” for Agencies to leverage lessons learned and best practices from the public and private sectors.
The recommendations include:
- Using Data Center Infrastructure Management (DCIM) Software to manage data centers
- Automating power metering at the device level
- Using variable speed drives to match energy usage to workload
- Using alternative cooling methods such as free cooling and direct liquid cooling
- Matching infrastructure power use to IT workload after virtualization
- Eliminating “zombie” servers (unused, but powered servers)
- Benchmarking to track performance over time
- Purchasing Green IT to reduce a facility’s energy footprint
- Developing disaster recovery plans and security awareness curriculums to protect physical and virtual assets
- Increasing automation capabilities to increase uptime
2.3. The Path Forward
In alignment with the HHS IT goals, HHS will pursue a shared services model where feasible that leverages shared data centers and cloud services.
To provide guidance in implementing the DCOI strategic goals, HHS will form a subcommittee of the CIO Council to draft, develop, and propose DCOI programs and expenditures targeted for data center optimization. This subcommittee will coordinate efforts with the DCOI Integrated Project Team (IPT). The IPT consists of representatives of the OpDivs and the Office of the Secretary, which represents the Staff Divisions (StaffDivs).
2.3.1. Internal Shared Services with Enterprise Monitoring
An internal shared services model where several of the major datacenters become centers for consolidation across multiple Operational Divisions (OpDivs) to facilitate closures of smaller and non-tier data centers. HHS has the advantage of this model being used in a limited capacity currently. The National Institutes of Health (NIH) provides data center services for other OpDivs.
HHS would develop a process to identify data centers with the capacity to host services for other OpDivs and allow HHS members to assess these data centers for their needs. Similar to purchasing services from external providers, this would allow OpDivs to identify the best shared services solution for their mission goals while reducing the number of federated data centers, cost, and environmental footprint.
This model could be expanded to leverage key, modernized, and centralized data centers across the country to provide services to all HHS OpDivs and StaffDivs. This would limit the number of smaller, tiered data centers to locations and mission-related services where the shared services model would not be effective due to physical or security-related concerns.
These key data centers could also be located at Trusted Internet Connection (TIC) locations to optimize HHS-Net routing. Locating data centers at TICs reduces the number of network hops information must take between the data center and the requesting users. Aligning network and data center architectures efficiently will also reduce latency and network costs. It will allow HHS to develop maintenance windows and seamless fail over processes.
The shared services model also allows HHS to save money on DCIM utilities such as monitoring software. HHS can leverage enterprise licensing savings to purchase DCIM tools and software, as well as reducing the physical footprint of HHS’s data centers by consolidating under-utilized servers.
This model is also in line with DCOI’s consolidation and Cloud First strategy.
2.3.2. When Consolidation Does Not Make Sense
Some HHS OpDivs have specific scenarios in which isolated data centers serve the best interests of both HHS and its patients. The Indian Health Service (IHS), for example, has medical facilities in extremely remote locations with limited access to the internet and consolidated data centers. It would be virtually impossible for them to effectively store patient data off-site. This scenario, and others like it, will have to be evaluated individually to determine where consolidation does not make sense and proceed in a manner that does not compromise the mission of HHS or its mission to optimize data centers.
These are the tasks HHS has identified to date. Additional tasks will be added to comply with FITARA reporting requirements.
Figure 4: Known DCOI and FITARA Compliance Dates
|Review Strategic Plan Outline and Tasks||September 19, 2016|
|Obtain CIO Plan Approval & Publish||September 30, 2016|
|DCOI/FITARA Reporting Due||November 30, 2016|
|Strategic Plan Update Due||April 14, 2017|
|Strategic Plan Update Due||April 13, 2018|
3. Strategic Considerations
3.1. Federal Shared Services
DCOI offers agency data centers the opportunity to become GSA-certified federal shared service data centers. The DCOI IPT will host a working session to consider this opportunity. Options to consider include promoting one or more of the HHS data centers for federal certification as a multi-agency shared service data center or not pursuing this option at all.
Additional information on the outcome of these working sessions will be incorporated into the DCOI-mandated updates to this plan.
Appendix A - Abbreviations
The following table provides definitions for terms relevant to this document.
|CAO||Chief Acquisition Officer|
|CFO||Chief Financial Officer|
|CIO||Chief Information Officer|
|DCIM||Data Center Infrastructure Management|
|DCOI||Data Center Optimization Officer|
|FCIO||Federal Chief Information Officer|
|FDCCI||Federal Data Center Consolidation Initiative|
|FITARA||Federal Information Technology Acquisition Reform Act|
|GFA||Gross Floor Area|
|GSA||General Services Administration|
|HHS||Health and Human Services|
|IaaS||Infrastructure as a Service|
|IDC||Integrated Data Collection|
|IPT||Integrated Project Team|
|ITIO||Information Technology Infrastructure Operations|
|NIH||National Institutes of Health|
|OFCIO||Office of the Federal Chief Information Officer|
|OMB||Office of Management and Budget|
|PaaS||Platform as a Service|
|SaaS||Software as a Service|
|TIC||Trusted Internet Connection|
|UPS||Uninterruptible Power Supply|
Appendix B - References
|Document Name and Version||Description||Location|
|OMB Memorandum M-16-19||Official document detailing the Data Center Optimization Initiative (DCOI)||https://www.whitehouse.gov/sites/default/files/omb/memoranda/2016/m_16_19_1.pdf|
|OMB Memorandum M-15-14||Official document detailing the Federal Information Technology Acquisition Reform Act (FITARA)||https://www.whitehouse.gov/sites/default/files/omb/memoranda/2015/m-15-14.pdf|
|2016 Strategic Roadmap for Data Center Infrastructure||Gartner Research note prepared for OCIO||https://ocioportal.hhs.gov/itio/projects/Documents/OSPG%20Legacy%20Activities/Data%20Center%20Consolidation%20and%20Optimization/2016_strategic_roadmap_for_d_308122.pdf|
|Evidenced-Based Best Practices Around Data Center Management||Data Centers Shared Solutions Marketplace Best practices document prepared for GSA||https://interact.gsa.gov/tag/max|
|HHS Information Technology Strategic Plan FY 2017-2020||OCIO’s strategic plan for IT management||https://ocioportal.hhs.gov/itio/projects/Documents/OSPG%20Legacy%20Activities/Data%20Center%20Consolidation%20and%20Optimization/HHS%20IT%20Strategic%20Plan_final.pdf|
|FITARA HHS Implementation Plan||HHS’s strategic plan for FITARA compliance||https://ocioportal.hhs.gov/itio/projects/Documents/OSPG%20Legacy%20Activities/Data%20Center%20Consolidation%20and%20Optimization/hhs-fitara-implementation-plan.pdf|
|CAO||Chief Acquisition Officer||Chief Acquisition Officer|
Appendix C - Table of Figures
 OMB Memorandum M-16-19
 FITARA HHS Implementation Plan
OMB Memorandum M-16-19
 HHS Information Technology Strategic Plan FY 2017-2020.
 Bernard Marr, “How Big Data is Transforming the Healthcare Sector”; https://www.linkedin.com/pulse/how-big-data-transforming-healthcare-sector-bernard-marr?trk=mp-reader-card
 Gerard J. Gallagher and Ramzi Namek, “Implementing Green Initiatives in the Data Center Environment”, http://www.totalsitesolutions.com/implementing-green-initiatives-in-the-data-center-environment/
 2016 Strategic Roadmap for Data Center Infrastructure. Gartner Research Note for Leslie Glaser, 2016.
 OMB Memorandum M-16-19.
 Evidenced-Based Best Practices Around Data Center Management. Data Centers Shared Solutions Marketplace for GSA; August 11, 2016.