OCR evaluated the Pilot Audit program and used this evaluation to verify whether the pilot audit protocol provided the flexible, scalable tools it needs to assess and improve compliance of health care organizations. The evaluation was also intended to improve the audit program design and audited entity selection process. This aided both consumers and health care organizations by ensuring a firm foundation for expansion of the audit program to examine the compliance efforts of more and different types of entities, including business associates. OCR posted lessons learned from the audit program to help organizations address common compliance challenges, learn how to conduct their own self audits, and better understand the process should they be selected for an audit in the future. The General Approach of the Evaluation Study Examination of the pilot audit program’s sampling methodology, workpapers, and supporting documentation to answer a series of research questions designed to understand the pilot program and use the information to drive outcomes Request for audited entity input on the pilot audit program through a non-invasive online survey Selection of a sample of health care organizations audited in the pilot program for input through further inquiries and/or inspection of documents Expectations for Participating Entities The evaluation focused on the pilot audit program’s effectiveness, analyzed the program’s strengths and weaknesses, and gave recommendations for how OCR conducts future audits. The evaluation focused on program design, implementation, and the experience of the covered entities that were audited during the pilot. No direct action was needed of entities that did not participate in the pilot audit program. The evaluation team coordinated with a subset of the 115 entities (approximately 8-10) to arrive at a mutually agreeable time to review their experiences. Details are below. Selected entities: Received advanced notice of at least a week to coordinate personnel and prepare responses to any minor, clearly-defined requests. Did not receive any additional findings or observations as part of this evaluation. Had open lines of communication for any questions and to avoid any surprise requests. Were not expected to provide extensive documents or resources as the evaluation will mostly leverage documents from the pilot audits. Contributed to improving the audit program through their feedback, and making a more efficient and effective audit program. Were not subject to on-site visits. Had the opportunity to convey efforts taken to remediate findings or observations from the pilot audit. Were not provided opportunities to refute findings noted in their audit report. Timeline/Schedule OCR provided information about the evaluation to selected entity officials and staff and could address any questions that entities had about the evaluation and their roles in supporting the evaluation. Between March 2013 and August 2013, the evaluation team examined the audit working papers and audit reports of selected entities. All of these documents were provided by OCR to the evaluation team. In July 2013, an online survey was distributed to the 115 covered entities audited as part of the pilot program. OCR values feedback and encouraged all pilot-audited entities to complete the survey. Responses were, and will remain, confidential. The evaluation team selected approximately 8-10 entities, based on survey results, for further interview in August 2013. The evaluation results and recommendations were provided to OCR in September 2013. OCR greatly appreciates support for its efforts to develop and enforce strong health information privacy and security protections.