U.S. Flag

Making a difference in how people find health information, today and into the future.

Progress on Capacity

Site Tags: 
Friday, November 22, 2013

Today, Jeff Zients offered an update on our efforts to improve HealthCare.gov; data on key metrics on site performance, the progress made this week and the view looking forward.  

In late October, we appointed QSSI as the general contractor to deploy their expertise in technology and program management to lead this project forward.

The team from QSSI continues to work with people from CMS and other contractors around the clock to troubleshoot the system, prioritize fixes, and provide real-time management decision making.

Thanks to this team effort, we have made measurable progress.

Before providing some specifics on this week, remember what we said at the end of October: HealthCare.gov will work smoothly for the vast majority of users by the end of November.

We’re on track to do so.

We have two objectives: addressing capacity shortfalls and managing volume, and improving user experience. 

Both are necessary to support the President’s goal that every American has access to affordable quality health coverage.

In terms of capacity, HealthCare.gov will be able to operate at the capacity that was originally intended: a rate of approximately 50,000 users on the site at the same time.

It’s important to keep in mind here, that this is not a simple website. Rather, it’s a complex system doing complicated work.

The system needs to process the millions upon millions of unique circumstances that consumers present. It needs to determine eligibility for hundreds of state and county-level programs and all their permutations. And it needs to factor in subsidy levels based on family size, income, and plan selection.

This is much more than a website for browsing or conducting routine transactions.

With that context, and the capacity we will have by the end of the month, this means the system will be able to accommodate more than 800,000 visits a day from consumers who are seeking information, filling out applications, shopping, and enrolling.

In terms of our second objective, improving the site experience, the vast majority of users will not experience the error messages, slow response times, and system outages that characterized their experience in October.

The system will not work perfectly. But it will operate smoothly for the vast majority of users.

We have said from the beginning that HealthCare.gov is fixable. And we are working around the clock to do just that.

While there will not be a magic moment at the end of the month when the work will be complete, users coming to the site today are already having a greatly improved experience. The site will be better at the end of the month than it is today, and it will continue to improve thereafter.

This past week HealthCare.gov was stable, with the exception of an unscheduled outage on Wednesday and a short period on Tuesday.

On our key operating metrics, response times and error rates, we continue to make progress. For most users, speed and response times were good and error rates were low.

This past week, average response time for most users has been under 1 second. That compares to the first few weeks after the site launched when we estimate that users were waiting an average of eight seconds for pages across the site to load.

Having response time under one second is a significant improvement.

In terms of the site’s error rate, last week we reported it was 1percent. This week, we’ve driven it down to .75 percent. As a reminder, a few weeks ago the error rate was 6 percent.

We are also seeing marked improvements as a result of the more than 300 software improvements, bug fixes, and hardware upgrades we’ve made across the last several weeks.

We have a new prioritized list of 50 additional fixes and improvements that we’re focused on for the upcoming week.

Notably, while we will continue to fix bugs and make improvements throughout the system, the priority fixes we’re focused on now are mostly aimed at improving the user experience further along in the process: in Plan Compare, in Shopping; and in Enrollment.

During peak periods of volume, some users still experience slower response times. Right now, the system is able to maintain good performance with about 25,000 users in the system at the same time. This compares favorably to a few weeks ago when, at lower volumes, the system was often unstable.

This increase in performance results from hardware, software and infrastructure upgrades and fixes that we have implemented to increase the system’s capacity.

These improvements will continue this weekend as the team will be installing additional hardware capacity in the computing, storage, and database environments.

These upgrades, and others planned for next week, will allow us to serve increasingly higher volumes and to double the current capacity to 50,000 users at the same time.

HealthCare.gov was originally intended to handle this load and the improvements we’re making will bring it up to this level so that the site will have the capacity that was intended. We will continue to evaluate ongoing capacity needs.

Now, to be clear, there will be times that volume on HealthCare.gov will exceed this capacity. To specifically prepare for those times when spikes in user volume outstrip the systems’ expanded capacity, we will deploy a customer-friendly queuing system to handle these spikes by serving consumers in an orderly fashion and by allowing consumers to request email notifications when it’s a  better time to come back to the site.

Stepping back, we’ve made a lot of progress working through our punch list, and the consumer experience is much better.

We’re on track to more than double today’s capacity, so that each day, the site will be able to accommodate more than 800,000 visits from consumers who are seeking information, filling out applications, shopping, and enrolling.

We have work to do to further improve the system and user experience. It’s likely as we move forward, we’ll find additional glitches and experience intermittent periods of sub-optimal performance.

Our bottom line is that we’ve made measurable progress in getting the site working smoothly for the vast majority of users.

 

Join the Conversation

By submitting this form, you accept the Mollom privacy policy.

Comments

Submitted by Robert on
Enrolled and entered my data on October 1, 2013, but unable to get past the verification process. Filled out paper application October 27, 2013 and delivered to Healthcare Marketplace October 31,2013. Since that time I have not heard anything by mail or email from the Marketplace as to eligibily. I now cannot log into the Marketplace. Please explain how I am to enroll, when?, and how? Please Help! It is now November 24,2013
Submitted by SP on
My application was processed on 10/25/13 but to date I am unable to view results. I had unchecked the popup blocker but nothing happens. I have logged on twice a day since 10/25/13 and still not able to review results. I was told someone from resolution team will contact me but this never happened. Will this problem be taken care of soon?
Submitted by Steve on
Are you aware the healthcare.gov website has only one, but very negative, comment showing at the bottom of this web page?
Submitted by Harry on
Same problem as JEP "In Progress" "Verification documents submitted" still no answers. I know the CR have to rely on computer also. When my application ask me what I would like to do? I don't think you want my answer……..just saying
Submitted by jdf on
What good are your plans and prices if you don't list Hospitals and Doctors in the plans? Another waste of taxpayer time and money. And what does it will work for vast majority of users mean? Vast majority to car makers means 99.999%
Submitted by Robert again on
Another complaint, one particular insurance company (Geisinger) in my marketplace area on the website (Pennsylvania) does not have my tax subsity taken off any of their policies . You might know it was one of Geisinger's plans that I was interested in! I called the 1800-3182596 number 3 times and they just said that they are still having problems with the site. Funny thing is they say that I'm still able to get the entire subsity back at the end of the year, but I'm not able to use it on Geisinger's plans.I can't imagine they are happy about being the only company without attached subsities!
Submitted by Mike on
I have tried to complete my application 4 times and each has stopped me when I save the summary and gave and error which by the way is a different ID each time. Is there something else I should be doing because each time I go through the hole application through the summaary and this happens.
Submitted by Robin in Indiana on
That's great about the upfront fixes. But what about the back end bugs? I selected a plan on October 14 and am still awaiting word from the folks in the Advanced Resolution Center (ARC) that they can get me enrolled and forwarded to my insurer, Anthem Blue Cross Blue Shield. It is now Nov. 24th and I am losing coverage soon. No one calls back… When I call, they say they are working on. But I called today and the ARC is closed weekends. So much for working around the clock.
Submitted by Glynis on
start over by creating a new account with a new username, new password, and new email address. Delete your cookies and temporary internet files first. Internet explorer, chrome, and safari browsers work better than firefox.
Submitted by Ramon on
As a fellow systems engineer, thanks for explaining that this "system" is more than a website. Most users do not understand or comprehend the complexity of systems. Keep up the good work! Your hard work is appreciated.
Submitted by Anonymous on
I still can not view eligibility results or my marketplace notices. I was told by the call center I am stuck until the website is fixed.
Submitted by Anon on
I would like to also know when this login issue that has been going on for many of us since the beginning of Nov. is going to be fixed. I have not been able to login as of the 1st. I would also be more than willing to bet that the identity verification process has not yet been fixed either. I doubt it ever will.
Submitted by Phyllis on
It is a shames to have all these employees working for healthcare.gov and no one can actually help you.
Submitted by SueB on
Agree with JEP. I have 2 applications out in limbo with no ID Numbers that no one can find. These were created over the phone when I could not get the web site to work. Then I finally created an application online and it has a number. The online application evaluated me as eligible to purchase on the marketplace. BUT the system won't let me shop for coverage because it says I have more than 1 application. No one I talk to on the phone has any idea how or if it will ever be possible to delete the 2 phone applications. The best advice I've gotten is to create yet a 4TH application, but use a different username. Then they say one of them will be able to connect that new application to the one application of mine that has an ID Number. They aren't sure if I can use my same email address in this 4th application or not. Why am I not feeling reassured by this?
Submitted by James Wimberley on
My suggestion is to follow the Moscow Rules for spies - extreme paranoia. New email address, new browser copy in a new PC user account, delete all history, cookies, cache, plugins. Write down all data on a crib sheet before you start and check every datum you enter twice.
Submitted by JH on
Successfully enrolled, thanks for the user posts they provided some good hints. I had to abandon my first application from October. It kept giving me an error when completing the first step of data. I believe the database records and relationships associated with the UserID were corrupt. I created a new Alias for my email account so everything is delivered to my main email account, a new UserID and everything worked and enrollment was successful. I think the key as mentioned in one of the other posts is to start over again with a new UserID and a new e-mail address if you submitted an application in October
Submitted by Kent on
I was glad to see that I could actually start filling out the application and actually got as far as the income screen when the error screen appeared that said the system is temporarily down. I have tried many times since then every day and every time I got to the point of saving the Household Information Summary I get the same error screen that says Error Id: 34930488-0b9f-4ed4-86a1-7203b138666a Application Id: 155887234. I called the help center and she went through the same process and reached the same error screen and, frustrated, said the system is down, and to try again in 30 minutes. What's going on? Is it something on my computer? (I erased my cookies and temp files and still have a problem).
Submitted by Roger on
I'm having this same problem as well. Ironically, I used to be able to get further before the site was "improved".
Submitted by Dan. on
I've been having exactly the same problem, with no apparent fix in sight. So please call the Marketplace, who was able to pull up my 2nd Application ID# (which I had submitted over the phone), and then process it through to completion (ie. my 1st & 3rd applications were submitted on-line, one in mid-Oct. & 1 in mid-Nov., both of which remain stuck, just like yours). In the meantime, I do plan to continue trying, maybe every other day or day, just on the off chance that they ever get this fixed. Hence the only thing that worked for me, was printing off the 7-page paper application, completing it, then submitting it over the phone to personnel at the Marketplace. Even then, it took 2 tries, several weeks apart, before I was successful.
Submitted by Ellen on
I've been going on the website every day since the beginning of Oct., hoping that my identity has been verified. I've sent proof twice already. I went on the Chat and I've called many times. When I go to my application, it keeps telling me that it's incomplete. I answer the same questions over and over. When I called today, the rep started another application for me with a new ID#. She also got stuck halfway through. She told me to call back in early Dec., but assured me that everything would be fixed and I would be able to get my insurance Jan.1. Do you think I should believe her?
Submitted by Lisa on
Agree with JEP. When do you fix it so people who have already started application can continue without hitting the wall, system is temporarily down? 7 days after talking to rep who said to keep trying 7 days of repeatedly answering the same questions that aren't saved in identity, 7 days later and income information that was provided is no longer found, 7 days of starting income info over and site is temporarily down. When does that get fixed? I'm thinking it would have been better to wait until Dec 10 and even then you have to worry if the site will work. I can't imagine people starting this in Oct and still working on it. How frustrating.
Submitted by allen on
Questions raised by JEP are spot on. You should attempt to answer them. Even if your answer is "I don't know".
Submitted by Joe on
1 % error? Why am I always in the 1%. I just want to sign up. This blog is nothing but excuses and does not help me at all.
Submitted by roosevelt louis on
I fill one application by phone 3weeks I am not receive any call
Submitted by Karen on
Stuck in the verify identification section of my application. PLEASE fix this section so that those with apps that are in progress and stuck in the I.d. verify can finally move on and be able to shop for a plan. My high risk insurance pool coverage expires 12/31/2013.

Pages