Today, Jeff Zients offered an update on our efforts to improve HealthCare.gov. In Jeff’s own words –
“Though we hit roadblocks that impacted the user experience and slowed us down this week; we’ve made changes and improvements over the last few weeks that have had a positive impact on overall system performance. And we’re making HealthCare.gov better each week.”
Here is an update on where we are at the end of the week:
Two of the metrics by which we are measuring progress are: response time and error rate.
Response time is how fast the system responds to user's requests. For the first few weeks after the site launched, we estimate that users were waiting an average of eight seconds for pages across the site to load. This is an absolutely unacceptable length of time.
This week, response time for many users has been well below one second. But given the variance in the website’s performance, there are periods where the site’s average response time is still too high. We continue to focus on improving overall system performance.
Another critical metric is the site’s error rate - a measure of the frequency, on a per page basis, of system time outs or failures that prevent the user from advancing to the next page. Often times, this results in users receiving an error message.
A few weeks ago, the site’s per page error rate was running over 6 percent and many users were experiencing errors each time they viewed a page. Working through fixes on our punch list, today, the per page error rate is under 2 percent.
We are making progress in reducing errors. However, as users view a lot of pages per visit, a 2 percent error rate is still too high. We need to continue to fix the underlying errors and drive this rate down.
On both our key metrics: response times and error rates, we are headed in the right direction, though we still have a lot of work ahead of us.
While we made progress with the website this week, we again hit roadblocks that impacted the user experience and slowed us down. The system was very slow and sporadic for many users for significant periods of time during the week. As we put new fixes in, volume has been increasing -- exposing new storage, capacity and software application issues. These issues are being added to our punch list, and we are working through them, crossing off fixes and enhancements one by one.
The changes and improvements we’ve made over the last few weeks have had a positive impact on overall system performance and we’re making HealthCare.gov better each week. Though it remains a long way from where it needs to be.
While the system’s speed and response times are generally in an acceptable range, there are intermittent periods of unacceptably slow response times. There is a team dedicated to monitoring and improving the site performance. When there are slow-downs, the team responds as quickly as possible.
This quick response capability will be particularly important as we continue to make fixes and the volume of users increases, as it’s inevitable that new problems will surface that will need to be resolved quickly.
We have a relentless focus on speed and execution, and will continually measure our progress to ensure that the site keeps getting better week by week. And that by the end of November, we’ll have the site working smoothly for the vast majority of users.