Today, Jeff Zients offered an update on our efforts to improve HealthCare.gov. In Jeff’s own words –
Overall, we’ve made measurable progress, as the changes and improvements we’ve made over the last few weeks are having a positive impact on system performance and user experience. The Marketplace system was stable all week, with no unscheduled outages. And for most users, speed and response times were generally good, and error rates were low. In total, we now have crossed more than 200 bug fixes off of our punch list.
Here is an update on where we are at the end of the week:
As we stated last week, two of our key operating metrics are response time and error rate.
Response time is how fast the system responds to user's requests. For the first few weeks after the site launched, we estimate that users were waiting an average of eight seconds for pages across the site to load.
This past week, average response times for most users remained under 1 second.
The other critical metric is the site’s error rate, a measure of the frequency, on a per page basis, of system time outs or failures that prevent the user from advancing to the next page or result in users receiving error messages.
Last Friday, we reported that we had reduced the error rate to 2% down from 6% a few weeks earlier.
This week, by continuing to eliminate glitches and execute software bug fixes, we’ve driven the error rate down to under 1%.
So, on both of these key metrics, response times and error rates, we’ve made measurable progress.
But we still have a lot of work ahead of us.
In the last week, volume on the site increased significantly, both in the number of users and user levels of activity; whether viewing pages or completing stages of the process. During peak periods of volume, some users experienced slower response times.
The increased volumes are a good sign, as they indicate the bug fixes and improvements we’ve made are allowing more consumers to move through the site.
But we clearly need the system to perform reliably with fast response times at higher volumes. This is a key focus of our work now, which we are addressing with additional hardware and infrastructure upgrades, starting this weekend. We will be bringing additional servers online, as well as additional database capacity and data storage. With these upgrades, we will significantly increase the system’s capacity and allow us to maintain good speed and response times at higher volumes.
On the software front, in the last week we have crossed more than 60 improvements and bug fixes off our punchlist, including:
- Adding visual cues to Plan Compare screens to help consumers make the best plan selection.
- Improving Direct Enrollment functionality to help consumers move back and forth to issuers’ websites.
- Fixing a glitch that was preventing some consumers in Pennsylvania from moving through the shopping experience.
In total, we now have crossed more than 200 bug fixes off of our punch list.
For next week, we have a prioritized list of 50 additional fixes and improvements that we’re focused on.
There is still a lot of work to be done to continue to improve the system and user experience. And it’s likely that as we move forward we will find additional glitches and experience intermittent periods of sub-optimal performance.
But we are making measurable progress.
And our bottom line continues to be that by the end of November, we expect to have the site working smoothly for the vast majority of users.
As the President said yesterday, not all consumers going on the website will have a seamless experience. We will continue to improve the site beyond the end of the month.
Most people who use the site will have it operate the way it is supposed to. But it’s important to know that even with a well-functioning website, there are going to be people who need additional help.
Whether it’s due to some complexity in individuals’ particular situations or needs, or simply because buying insurance is a big, complicated decision, some will prefer to get assistance in thinking it through.
So while we are focused on making the site work smoothly for the vast majority of users, the team is also working to make sure we are doing everything we can to give people more assistance in the application process through our call centers and available in-person assistance.