Table 1 – Definitions of Terms
This Policy provides a standard methodology and establishes roles and responsibilities for conducting an Operational Analysis (OA). An OA is required for all Department of Health and Human Services (HHS) information technology (IT) Investments that are currently in the Operations and Maintenance (O&M) or Steady State life cycle phase. This Policy provides detailed instructions regarding the content of OA documentation as well as the roles and responsibilities for conducting and approving an Investment’s OA. Proper completion of an OA for every applicable IT Investment will help ensure that HHS IT Investments are viable, relevant, and remain pertinent to achieving the agency’s overarching mission.
This Policy is first issuance.
The Clinger-Cohen Act (CCA) of 1996 (Section E of Public Law 104-106, formerly known as the IT Management Reform Act (ITMRA) of 1996), requires federal agencies to use a disciplined CPIC process to acquire, use, maintain, and dispose of IT assets. This CPIC process consists of three phases: Select, Control, and Evaluate. The Evaluate phase of the CPIC process must include methods to manage and report the performance of the Agency’s IT assets using quantifiable measurements of benefits, risks, and progress in terms of cost, capability to meet specified requirements, timeliness, and quality. The OA is the mandated method for evaluating Steady State IT Investments.
The required components of an OA are based on the aforementioned legal mandates of the CCA and on OMB’s Capital Programming Guide, V3.0, the Supplement to OMB Circular A–11, Part 7: Planning, Budgeting, and Acquisition of Capital Assets. This suite of documents provide guidelines for conducting and using the results of the IT OA as a key part of successfully managing the Department’s IT portfolio, ensuring that Steady State IT Investments are continuing to meet their stated goals and intended objectives and yield expected benefits. OMB’s Capital Programming Guide defines operational analysis as “a method of examining the ongoing performance of an operating asset Investment and measuring that performance against an established set of cost, schedule, and performance goals.”
The requirements within this policy will be applied to all Steady State IT Investments and their associated Steady State IT projects, systems, products, and services. Per OMB’s Capital Programming Guide, “regardless of performance of operational indicators, a formal operational analysis is warranted for every Steady State project.” An annual OA is required for all IT Investments that are currently in the O&M or Steady State life cycle phase. These requirements are in effect and are to be implemented regardless of size, scope, complexity, or funding source, whether owned and operated by HHS or operated on behalf of HHS.
This Policy applies to all Department Operating Divisions (OpDivs) and Staff Divisions (StaffDivs), including the Office of the Secretary (OS) and Office of the Inspector General (OIG), and to all organizations conducting business for and on behalf of the Department. Assigned officials within HHS shall apply this policy to contractor personnel, interns, and other non-government employees conducting business for or on behalf of the Department through contractual relationships or memoranda of agreement. The content of and compliance with this Policy shall be incorporated into applicable contract language, as appropriate.
OpDivs and StaffDivs shall use this Policy or may create a more restrictive OpDiv/StaffDiv policy tailored to their specific environment; however, that policy may not be less restrictive, less comprehensive, or less compliant than this Department Policy.
This Policy shall be applied and implemented, as appropriate, in conjunction with the HHS Policy for IT CPIC; the HHS OCIO Policy for IT Enterprise Performance Life Cycle (EPLC); the HHS-OCIO IT Policy for Enterprise Architecture (EA); and any Federal Acquisition Regulation (FAR) and HHS Acquisition Regulation (HHSAR) rules that are promulgated on the subject.
This Policy does not supersede any other applicable law, higher level agency directive, or existing labor management agreement.
Conducting an OA is an essential requirement for any operational Investment because the OA process is used to determine the disposition of an Investment in the Evaluate phase of the CPIC cycle. An OA is required of all IT Investments on an annual basis as a part of CPIC best practices. Additionally, the OA Results Report is a critical EPLC artifact for all IT Investments in the O&M phase of the framework.
Business owners are required to conduct an OA on Major and Non-Major IT Investments on an annual basis as the OA is a critical standard that is used Department-wide to evaluate the performance and continued viability of HHS IT Investments that are wholly or partially in Steady State. As such, the timeliness of an Investment’s OA plays a significant role in the calculation of any Major IT Investment’s Chief Information Officer (CIO) Evaluation, which is publicly available on OMB’s IT Dashboard. Major IT Investments that have not provided an OA Results Report in the preceding 18 month period are at risk of receiving a lower overall CIO Evaluation.
Major IT Investment Managers are also required to provide OMB with the Investment’s latest OA as part of the annual budget cycle. Any new or updated OA for a Major IT Investment must be uploaded to the Datapoint Artifact Sharing site within OMB’s MAX Collect tool.
OA Results Reports must at a minimum cover the areas defined below. The required content implements the recommendations of the OMB Capital Programming Guide, Supplement to OMB Circular A-11, Part 7, which states that beyond the typical developmental performance measures of cost and schedule performance, an OA should seek to answer more subjective questions in the specific areas of: Customer Satisfaction, Strategic and Business Results, Financial Performance, and Innovation. The Government Accountability Office (GAO) Report: Agencies Need to Strengthen Oversight of Multibillion Dollar Investments in Operations and Maintenance derives the 17 key factors listed below from the Capital Programming Guide that should be addressed in the OA Results Report.
For enhanced guidance regarding OA Results Report content, refer to the guidance and templates referenced in section 7 of this policy, “Information and Assistance”.
OA Subsection |
Description |
OA Results Report Key Factor |
Customer and Stakeholder Satisfaction | Assesses feedback from customers and stakeholders to determine the extent to which the Investment supports customer needs as designed. | Identifies whether the Investment supports customer processes as designed and is delivering the goods and services it was designed to deliver |
Strategic and Business Results | Measures and evaluates the Investment’s impact on the performance of the OpDiv and HHS to determine how well the Investment is meeting business needs in its strategic alignment and business impact. |
Includes a structured assessment of performance goals (i.e., measures the performance of the Investment against established goals) Measures the effect the Investment has on the performing organization itself Measure how well the Investment contributes to achieving the organization’s business needs and strategic goals Identifies recommendations to redesign or modify an asset before it becomes a problem Identifies a need to redesign, modify, or terminate the Investment after a problem has been identified Identifies any lessons learned |
Financial Performance | Measures and assesses the Investment’s current and expected performance compared to the pre-established cost baseline through performance baseline metrics, expenditures, and comparative measures. |
Assesses current costs against life-cycle cost Includes a structured schedule assessment (i.e., measures the performance of the Investment against its established schedule) Compares current performance with a pre-established cost baseline and estimates Includes an ongoing review of the status of the risks identified in the Investment’s planning and acquisition phases Identifies if the Investment had a cost or schedule variance |
Innovation | An opportunity to conduct a qualitative analysis of the Investment’s performance within the previously mentioned areas – demonstrating that alternative methods for meeting the same needs have been revisited. |
Identifies any areas for innovation in the areas of customer satisfaction, strategic and business results, and financial performance Identifies if the agency revisited alternative methods for achieving the same mission needs and strategic goals Addresses issues such as greater utilization of technology or consolidation of Investments to better meet organizational goals Includes an analysis on the need for improved methodology (i.e., better ways for the Investment to meet cost and performance goals) Includes information on the overlap of the Investment with other systems |
Investment Recommendation |
|
Provides a recommendation to the OpDiv governance board regarding next steps for the IT Investment. The recommendation might include one or more of the following conclusions:
|
The applicable OpDiv IT governance board will review the OA Results Report taking into account the Investment recommendation in the OA Results Report and will determine the final disposition of the IT Investment. Appeals to the OpDiv IT governance board decision may be made to the HHS CIO.
If an IT Investment consists of multiple operational IT products, services, or systems; a separate OA may be conducted for each or for each related set. In the event that separate OAs are conducted for the individual operational IT products, services, or systems of the IT Investment, the IT Investment’s overarching OA shall comprise summaries of the processes used and results of the annual OAs that were conducted. When an IT Investment includes only one operational IT product, service, or system, the annual OA conducted shall serve as the OA for the IT Investment.
All Department OCIO policies, standards, and procedures will be posted on the HHS Internet (http://www.hhs.gov/ocio/policy/index.html) and the HHS Intranet (http://intranet.hhs.gov/infosec/policies_memos.html).
Further guidance regarding the EPLC process, to include templates, checklists, and practice guides for all artifacts including OAs can be located at: (http://www.hhs.gov/ocio/eplc).
All questions, comments, suggestions, or requests for further information should be directed to the HHS EPLC Program at HHSEPLC@hhs.gov.
The effective date of this Policy is the date the policy is approved.
The undersigned acknowledge they have reviewed the HHS OCIO Policy for Information Technology Capital Planning and Investment Control. Changes to this policy will be coordinated with and approved by the undersigned or their designated representatives.
Beth Killoran
Director, Office of IT Strategy, Policy & Governance
Office of the HHS Chief Information Officer (CIO)
Term |
Definition |
Business Owner | The organizational executive who is the primary IT Investment customer, advocates for the IT Investment, and serves as the primary point of contact to the CIO and the IT Governance Board (or designated governance body). |
Critical Partners | Subject matter experts in enterprise architecture, security and privacy, acquisition management, finance, budget, performance, human resources, and other areas. |
Information Technology (IT) | Any equipment or interconnected system or subsystem of equipment used in the automatic acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information. This includes equipment used by the executive agency directly or used by a contractor under a contract with the executive agency that (i) requires the use of such equipment, or (ii) requires the use, to a significant extent, of such equipment in the performance of a service or the furnishing of a product. The term includes computer, ancillary equipment, software, firmware, and similar procedures, services (including support services), websites, subscriptions to electronic services and products, and related resources. |
Investment Manager | The Investment Manager is responsible for planning and executing the Investment to achieve approved baselines. The Investment Manager may or may not be a subject matter expert in the business area supported by the Investment. |
IT Investment |
The expenditure of resources on selected information technology or IT-related initiatives with the expectation that the benefits from the expenditure will exceed the value of the resources expended (GAO-04-394G, IT Investment Management, A Framework for Assessing and Improving Process Maturity, March 2004, v.1.1). The acquisition of an IT asset and the management of that asset through its life cycle after the initial acquisition. |
IT Operations Manager | The IT Operations Manager reports to the IT Investment Manager and manages the cost, schedule, and performance of one or more of the IT Investment’s operational IT products, services, or systems. The role of the IT Project Manager and Operations Manager might be held by the same or multiple people throughout the life cycle of the Investment. |
IT Project | A temporary planned endeavor funded by an approved information technology Investment, thus achieving a specific goal and creating a unique product, service, or result. A project has a defined start and end point with specific objectives that, when attained, signify completion. |
IT System | A discrete set of information resources organized for the collection, processing, maintenance, transmission, and dissemination of information, in accordance with defined procedures, whether automated or manual to support HHS’s or OpDiv’s (including OS’s) mission. An interconnected set of information resources under the same direct management control, which shares common functionality. A system normally includes hardware, software, information, data, applications, communications, and people. Refers to a set of information resources under the same management control that share common functionality and require the same level of security controls. Includes automated information system (AIS) applications, enclaves, outsourced IT-based processes, and platform IT interconnections. |
Life Cycle | The duration of all activities associated with the Investment from its initiation through disposal of its assets. |
Life Cycle Costs | All initial costs, plus the periodic or continuing costs of operation and maintenance (including staffing costs), and any costs of decommissioning or disposal. |
Major IT Investment |
An IT Investment that meets at least one of the criteria listed below:
|
Operational Analysis | An Operational Analysis (OA) evaluates IT Investment performance, user satisfaction with the IT Investment, the IT Investment’s adaptability to changing business needs, and new technologies that might improve the IT Investment. The OA review is diagnostic in nature and can lead to the initiation of development or maintenance activities. Any major IT Investment modifications needed after the IT Investment has been implemented follow the EPLC framework life cycle process from planning through implementation. The OA ultimately determines whether the IT Investment should continue, be modified or be terminated. |
Operations and Maintenance (O&M) |
The activities related to the performance of routine, preventive, predictive, scheduled, and unscheduled actions aimed at preventing equipment failure or decline with the goal of increasing efficiency, reliability, and safety. The phase within the Capital Planning and Investment Control process which maintains the IT system(s) for ongoing activities within the production environment.
|
Performance Measurement Baseline |
A primary tool to measure IT Investment, IT project, or IT contract performance and identifying risk. The baseline identifies the work that will be accomplished, and defines the cost and schedule to accomplish that work. The Performance Measurement Baseline, which consists of the cost, schedule, and scope baseline, is derived from the scope of work described in a hierarchical Work Breakdown Structure (WBS) – which, in turn, decomposes the entire project into a logical structure of tasks and activities tied to deliverables and to assigned responsibilities – and the associated WBS dictionary. The Performance Measurement Baseline comprises:
The Performance Measurement Baseline is integrated where the time-phased cost baseline is consistent with the schedule baseline, and the costs are related to acquiring, operating, and disposing of the physical and/or logical IT asset represented by the scope baseline. |
Acronym |
Term |
AIMD | GAO’s Accounting and Information Management Division |
AIS | Automated Information System |
AMD | Administration and Management Domain |
ANSI/EIA | American National Standards Institute/Electronic Industries Alliance |
CCA | Clinger-Cohen Act of 1996 |
CIO | Chief Information Officer |
CPIC | Capital Planning and Investment Control |
EA | Enterprise Architecture |
E-GOV | Electronic Government |
EPLC | Enterprise Performance Life Cycle |
FAR | Federal Acquisition Regulation |
FEA | Federal Enterprise Architecture |
FISMA | Federal Information Security Management Act of 2002 |
GAO | Government Accountability Office |
GPRA | Government Performance and Results Act of 1993 |
HHS | Health and Human Services, Department of |
HHSAR | HHS Acquisition Regulation |
HHSD | Health and Human Services Domain |
IRM | Information Resource Management or Information Resources Management |
IT | Information Technology |
ITIRB | Information Technology Investment Review Board |
ITMRA | IT Management Reform Act |
ITSC | Information Technology Steering Committee |
O&M | Operations and Maintenance |
OA | Operational Analysis |
OAMP | Office of Acquisition Management and Policy |
OAP | Operational Analysis Policy |
OCIO | Office of the Chief Information Officer |
OMB | Office of Management and Budget |
OpDiv | Operating Division |
OS | Office of the Secretary |
PBM | Performance Baseline Management |
PIR | Post-Implementation Review |
PMA | President’s Management Agenda |
PMT | Portfolio Management Tool |
SRD | Scientific Research Domain |
StaffDiv | Staff Division |
WBS | Work Breakdown Structure |