The Bankwatch

Tracking the consumer evolution of financial services

A quick QA test of healthcare.gov shows some very basic mistakes even in the customer registration system

With all the attention being received by the site performance of healthcare.gov and after hearing that CGI from Canada were the developers, I thought I would take a quick look.

Obviously I cannot comment on things like the reliance of this site to performance of the other sites it has to link to because I am not American and can only go so far.  However at least I can look at the registration process.

I was surprised at basic errors in even in the registration process.  There are some basic issues that simple QA testing would have caught.  I have to keep reminding myself this is a multi year $292 million project.  I see the excuses are already coming about how requirements were changed at the last minute however I cannot imagine a registration process as simple as this one appears to be would have been caught up in requirements changes – more on that in my notes later;

I made notes on each page with screen shots, and here is summary of my conclusions and inferences:

  • sequencing of registration items;  this was the worst design issue.  I was expecting the standard registration email but I received an email confirming subscription to a newsletter and nothing about registration.
  • page delays;  clicking on a page could take several minutes before the next step showed up as indicated in the notes.  The worst was 5 minutes.
  • Blank screen;  the time I got the blue screen for several minutes was far too long and suggested some serious performance issue either from the server or from a connected system.
  • the point regarding requirements changes;  the very fact I was able to register suggests to me there are no checks and balances during this registration process. Things that were not done include nothing about SSN, obviously no IP validation for where I am located, no questions about residence or citizenship.  The fact there are none of those checks suggests this is a very simple registration system that developers build all the time.  Clearly they should have expected larger volumes that the average startup would get on day 1 but scaled registration systems are hardly new and the experience to build and the tools to load test exist.  And site slowness is no excuse for wrongly sequenced pieces of the registration process.
  • lastly – just some common courtesy.  The final page in the attachment has no mention of the issues, no apology, no indication of anything from the last three weeks of this site being the largest technology problem in the world!

Healthcare registration pdf

Summary:

This experience really felt like the site was still running on the developers laptop and this was the first time it had come out for QA testing.  The delays will confuse many people, and I am sure that is causing many to think they are not registered whereas they may well be at least partially registered.

I understand this is a mammoth project, but it is only good if the users can get into it.  Some very easy fixes here.

Written by Colin Henderson

October 21, 2013 at 15:02

Posted in Uncategorized

%d bloggers like this: