U.S. Flag

Making a difference in how people find health information, today and into the future.

HealthCare.gov Metrics

Site Tags: 
Friday, November 8, 2013

Today, Jeff Zients offered an update on our efforts to improve HealthCare.gov.  In Jeff’s own words –

“Though we hit roadblocks that impacted the user experience and slowed us down this week; we’ve made changes and improvements over the last few weeks that have had a positive impact on overall system performance. And we’re making HealthCare.gov better each week.”

Here is an update on where we are at the end of the week:

Two of the metrics by which we are measuring progress are: response time and error rate.

Response time is how fast the system responds to user's requests. For the first few weeks after the site launched, we estimate that users were waiting an average of eight seconds for pages across the site to load. This is an absolutely unacceptable length of time.

This week, response time for many users has been well below one second. But given the variance in the website’s performance, there are periods where the site’s average response time is still too high. We continue to focus on improving overall system performance.

Another critical metric is the site’s error rate - a measure of the frequency, on a per page basis, of system time outs or failures that prevent the user from advancing to the next page. Often times, this results in users receiving an error message.

A few weeks ago, the site’s per page error rate was running over 6 percent and many users were experiencing errors each time they viewed a page. Working through fixes on our punch list, today, the per page error rate is under 2 percent.

We are making progress in reducing errors. However, as users view a lot of pages per visit, a 2 percent error rate is still too high. We need to continue to fix the underlying errors and drive this rate down.

On both our key metrics: response times and error rates, we are headed in the right direction, though we still have a lot of work ahead of us.

While we made progress with the website this week, we again hit roadblocks that impacted the user experience and slowed us down. The system was very slow and sporadic for many users for significant periods of time during the week. As we put new fixes in, volume has been increasing -- exposing new storage, capacity and software application issues. These issues are being added to our punch list, and we are working through them, crossing off fixes and enhancements one by one.

The changes and improvements we’ve made over the last few weeks have had a positive impact on overall system performance and we’re making HealthCare.gov better each week. Though it remains a long way from where it needs to be.

While the system’s speed and response times are generally in an acceptable range, there are intermittent periods of unacceptably slow response times. There is a team dedicated to monitoring and improving the site performance. When there are slow-downs, the team responds as quickly as possible.

This quick response capability will be particularly important as we continue to make fixes and the volume of users increases, as it’s inevitable that new problems will surface that will need to be resolved quickly.

We have a relentless focus on speed and execution, and will continually measure our progress to ensure that the site keeps getting better week by week.  And that by the end of November, we’ll have the site working smoothly for the vast majority of users.

Join the Conversation

By submitting this form, you accept the Mollom privacy policy.

Comments

Submitted by Marsha on
I tried the site to see what im eligible for. I was referred to apply for medicaid due to amt of income . Says i am not eligible for the credits so premium would be as much as my total income. I dont think i qualify for medicaid. Im 60 yrs old, no dependants. No medicaid expansion in my state. I thought this cant be correct so i called the toll free number provided. The lady i spoke with was arrogant and rude. So i said forget it.
Submitted by Mary on
I have been trying to get my application processed since 11/14/13. Every time I look at it through my account, it is incomplete. Every time I call the 800 number, the representative is unable to get it processed because there is an error message at some point in their review or the system goes down. Suggestions from representatives range from wait a few days and try again to submit a paper application. Today, I talked with someone at the Advanced Resolution Center and, during her review, she encountered an error. She submitted the error to the Technical Center hoping that they would be able to fix it and get the application through the process. I was not told if someone would call me or if I just had to keep checking. I was a computer specialist and am ashamed that the United States would put out a system that doesn't work, keeps crashing, and is extremely difficult to navigate and work with.
Submitted by Les on
My Application has been processed. When I press Verify Results, receive error message - HTTP 400 Webpage cannot be found I have tried to verify results of application for several weeks and continue to get same error message. I have called Call Center and talked to representative. They confirmed that the application has been processed and told me that my wife and are eligible for subsidy but cannot tell me how much. If I can get beyond this step to Verify Results, I can then select and purchase health plan. But at this point, I'm stuck. I'm guessing that others have been stopped on the Verify Results as well.
Submitted by Melyssa on
For over a month now my application has been "in-progress" becasue I havent been able to "sign electronically" to get to the next step of looking for coverage. I understand that this website has a lot of kinks it needs to work out, but a month now with the same issue? Can that be on the list of things to fix. I would really like to be able to look at the different coverage options. I have a MBA, but I am unable to find a career that offers benefits. The last thing I need, on top of Student loan payments, a mortage, and providing for a child is to be taxed because I dont have any health care.
Submitted by Dan on
I'd like to set a vision for the page-error rate. Julie, you report progress to a 2% rate. That's 1 out of 50 page displays result in an error. Keep it up. A six-sigma objective would be 1 page error per 294,000 page displays. While that would seem to be an astonishing objective, It is necessary for healthcare.gov to be there soon. Suppose the 2 million or so individuals who had policy cancellations wanted to explore and enroll at healthcare.gov all arrived over 1 - 3 days. You can see how 2% wouldn't work for them. So keep up the good work on getting that error rate down to 1 in 300,000 so you can confidently invite 3 - 5 million people to enroll over a short period of time.
Submitted by Chuck on
My identity verification has been pending for over a month. I upload my drivers license the first week of October and my passport the 2nd week. This despite being confirmed over the phone by Experian. Here's a suggestion for a quality measure to track : how long it takes to confirm uploaded documents. Set a goal, say one week, and then report what percentage of users had that goal met.
Submitted by Linda on
I filled out application on Oct 18. I am waiting to have id verified My application is in progress My application is incomplete Each time I attempt to complete application. It will not save information My application was submitted to four different states I never received any confirmation on my application via email or Us mail. I never received a copy of my application I've been working at this might send day and it is driving me to the nut house. I need help
Submitted by Anonymous on
Why don' t you use Heroku or Amazon AWS to deploy your servers? If you do so you will be able to scale up or down to handle any amount of web traffic.
Submitted by Robert on
From my perspective you are making no progress at all. I have tried over 8 times since October 1, trying different days and times, and I have yet to even be able to create an account, much less the other steps in the process.
Submitted by JEP on
Here's some metrics you could work on: 1) how many people are waiting to have their Identity verified? 2) how many people have applications "in progress"? 3) how many people have their eligibility results waiting but are unable to access them? 4) how many people have called the call center only to be told to try again later? 5) how many people have actually taken the customer survey at the end of the call?
Submitted by Geordie on
Those questions are exactly the ones I would have asked.
Submitted by Gloria on
I filed an application over the phone with a customer service rep who completed the application; however, in researching the results of my "eligibility determination", she had put THIS year's income where it asked for 2014's estimated income. Retirement in December of this year makes a big difference between the 2 years. ARC can't access the application to make the correction. An appeal could take up to 90 days (more likely twice that time if history is an indicator of future experience), and my husband and I will have no insurance January 1, 2014. I'm told that fixes are coming but no indication when. December 15th is coming upon us very quickly, and we are no further ahead in this process than we were September 30th.
Submitted by Carol on
I set up an account in Ohio. I received an initial eligibility determination letter but it is erroneous in denying my access to a subsidy. I cannot get into my application to revise it based upon changing information in my live after a job termination recently. So now, to protect my rights I have to file a paper appeal of that erroneous determination. Also when I get to my enrollment page, I get an error message telling me to review the Application Details but the system will not let me open my application details to correct it. I hope by providing these details you can get my account fixed. Thank you for your efforts and I assume you have some very big data base issues to fix in the next three days. I need to purchase insurance in the exchange with the subsidy by December 15 in order to timely get new health care insurance in place by January 1, 2014.
Submitted by Anonymous on
Since this weekend, I have been unable to log in. It's not that my username and password are not working it's that the log in button is no longer there. Does anyone have any idea what could be causing this? My cookies and cache are emptied automatically every evening.
Submitted by Norman on
Try and fix all glitches before the end of November. The longer you take it hurts Americans who need coverage and view plans in their state, like me in Oklahoma and it affects Obamacare overall.
Submitted by Michael on
Jeff, please institute a process to: 1) track and resolve individual user problems; 2) allow help desk people to view a user's application.
Submitted by Will on
Great that you are making progress! But what are going to do for the people who have already filed an application and received an incorrect "Eligibility Notice"? I am now stuck in a paper chase Appeal process because glitches miscalculated my eligibility. The site won't even let my wife buy insurance at all! So, once the site is fixed how about allowing people to cancel their original application and re-apply online?
Submitted by Roger on
Increasing website speed is no help to people like me that are stuck in the enrollment process and cannot progress any further. I am stuck on the Enroll To-Do List page. I have Set premium tax credit usage (Wife & I) and Answered questions about household. Next is where I am stuck, it wants me to Select a health insurance plan 1 (me), Select a health insurance plan 2 (Wife & I) and Select a health insurance plan 3 (Wife & I). Why???? It is my wife & I applying for a single plan to cover us both. I don’t want 3 plans and it will not let me select 3 plans. How are website errors like this every going to get corrected?
Submitted by wow on
by end of November the site will be working better REALY. I bet your job you are gonna get caught in a nother lie.For the vast majority of the time.
Submitted by JEP on
Hello, is anyone listening to the people trying to register and find new insurance??? Through no fault of our own, we're losing our current HC insurance on January 1st. and no one from your agency is able or willing to help us. Response time? There is NO response in this system!!!!. My application has been "pending" for over a month. I've called your 800 number at least 10 times and although your representatives are very polite, they are unable to assist me. What good is a call center where the representatives are not able to assist you? They just apologize and ask me to be patient and to check back later. I'm just trying to determine my eligibility and to find out what my options are. Why is that so difficult for you to? Does anyone ever monitor this blog? Based on the lack of responses to the many questions on this blog, my guess is that everyone has taken a long Veteran's Day holiday weekend. .
Submitted by Dan on
What about pages that just don't work? How do you detect that? My error rate is 100% when I get to certain pages but there is no error message on the page.
Submitted by abraham on
I am not in a rush to sign up ,I let you guys work on the system to make it better for the nation.
Submitted by Kenneth on
Kenneth Peterson: Thank you for this great progress! I am proud of your efforts and results. We should not forget how important ACA is to this country, and to its most vulnerable.