Really? So they both were reusing old code when they shouldn’t, and weren’t reusing old code when they should?
The major (only?) problem is the requirement of sign up and verification before one can get a price quote. There is no need to verify my social security number with Experian just to give me a quote. How many millions are just browsing (no different than Amazon or Ebay) and will leave when the prices are shown? Ehealthinsurance.com does it just fine. No need for Manhatten Project II to solve the problem, just do what other large internet services do.
Imagine your local brick and mortar store doing a background check for shoplifting before allowing anyone to walk through the front door to shop. How would that work out for that store?
There is absolutely no way it contains anywhere close to 500,000,000 lines of code. I’m sure it’s a big complex project but that is simply not correct.
Windows Vista had about 50 million lines.
SAP (probably the largest commercial software app) has about 250 million lines of code and there is absolutely no way Healthcare.gov even gets close to approaching the complexity of SAP.
I find it oddly easy.
But more seriously, I also have a degree in software engineering and recognize the inherent problems and delays. Whatever the level of bureaucratic and technical snafusion, though, I don’t see it reaching the metaphorical level of a pelvic exam conducted by a nightmarish version of Uncle Sam.
I visited the site today to see what all the fuss was about, and apparently over the weekend they’ve removed any sign-up requirement to view quotes. You have to enter your state of residence and they do remind you that you need to consider your income level as you may be eligible for subsidies, but that’s pretty much it. I encountered no issues viewing prices for plans from the various coverage classes.
I don’t need one of the exchange-offered plans, so I didn’t see whether there are still problems with the enrollment utility, but there is now an option to enroll by phone.
I’m going to go out on a (short?) limb and predict this will mostly be resolved in 2-3 weeks.
I’m prepared to speculate that technical problems will persist for quite some time, and we’ll get horror stories (well, mildly horrific stories… Scooby-Doo level, tops) about someone trapped in some Kafka-lite bureaucratic cycle like trying to register but having the same name as someone who already registered and is now missing in action in Afghanistan or whatever and “the government” is demanding they fill out a form with affidavits from three witnesses who communicated with the registrant during the last full moon that took place on a Wednesday.
Heh, I have no doubt that certain quarters will be working hard to make sure know about every single such screwup, thereby providing yet more evidence that the ACA is a bad law rammed down our throats by bad people.
I think this is it.
This project was obviously not managed - in technical terms - as a shopping site. Looks like they ran at as standard back-office data warehouse integration project with web interface on top.
Also, if they relied on out of box solution(s) without significant custom low-level coding to resolve throughput and data integration then the solution will be a patch onto a patch.
Well, I don’t think they paid enough attention to the management paradigms implicit in their projected flow models.
Can anyone explain how it is that certain state-run exchanges (eg New York) seem to be running fine, when they presumably have to do the same type of work both on the front and back ends as the federal exchange? Why can’t the administration just adopt the best-functioning state health exchange currently out there?
Daily scrums. I bet there were no daily scrums. Or agreement on Minimum Viable Product.
In the weeks before the start of Obamacare, officials failed to complete exhaustive testing of the program’s website in a push to begin signups by Oct. 1, according to people involved in the rollout.
The federal Healthcare.gov site – which has been plagued by software bugs – went live without attempts to replicate a customer’s complete experience, said a person familiar with the project who asked not to be identified to discuss what happened.
The introduction was so rushed that, as recently as last week, the exchange’s computer code contained placeholder language that programmers typically use in preliminary drafts, said Clay Johnson, a former White House presidential innovation fellow during 2012-2013.
“It was a perfect storm for an IT meltdown,” said John Gorman, a former assistant to the director of the Health Care Financing Administration’s Office of Managed Care, the predecessor to the agency responsible, now known as the Centers for Medicare and Medicaid Services, or CMS.
Tax returns for 2013 are due April 15, 2014. Tax returns for 2014 are due April 15, 2015. ACA mandate takes effect Jan. 1, 2014.
Isn’t the mandate irrelevant to the 2013 return?
There’s something wrong with that number. I suspect bad reporting, and/or inflated LOC reporting by the tech team to make their project sound more impressive. Perhaps they’re including the lines of code that exist in all the open source libraries they are using.
Windows 7 has about 40 million lines of code, much of which is legacy code that Microsoft has developed over a decade. 5ESS, one of the largest software projects ever, has about 100 million lines of code, and it took 5,000 developers 20 years to create that much.
If it’s really true that 5 million lines of code have to be changed to fix this, we’re not talking about a fix measured in weeks - we’re talking about a fix measured in years. There’s no way that much code can be churned in a system this size in any reasonable time frame.
So I suspect something’s wonky with the numbers. Or, this is a mess of a scale that people really haven’t grasped yet.
So the Republicans damaged themselves significantly in order to delay a project that was already delaying itself due to bugs, distracting the public from this fact for 2 weeks, and their only consolation is that polling shows that this has hurt the re-election prospects of a 2nd-term president. Oh well, I guess we find silver linings where we can.
You realize that what they’ve done by doing this is to completely short-circuit the back end, right? Instead, they’re just redirecting you to a bunch of static files. This stop-gap is not a sign that things are ‘improving’ - it’s just a temporary workaround they’re using to claim some sort of ‘progress’.
There is no way this will be resolved in 2-3 weeks. If the administration had to resort to a ‘surge’ of tech people, they know this is a long fix, as it will take weeks just for the new people to get their heads around the project and understand what’s going wrong.
A rule of thumb in software development is that bringing more people into a project late in the game actually slows it down and is diverts other development resources away from what they are doing so they can train the new people.
But who knows what’s actually going wrong? This administration has by lying through its teeth about the exchanges for a long time now. They’re still lying about it.
For example, their claim that the problems were due to ‘incredible demand’ are just nonsense. 5 million people trying to log in may sound like a lot, but they had to be expecting at least that many given the claims they were making. Second, I’ll bet a lot of those 5 million hits were people constantly refreshing / retrying because the site was broken. Third, for a large scale web site 5 million isn’t that big a number.
As of today, Healthcare.Gov has a traffic rank of 3,343 in the world in on Alexa, and 276 in the U.S. If traffic was its problem, it should be running smooth as glass now. It’s not.
I am a software developer and can personally attest to the truthfulness of this. I was working on a bug report not too long ago that related to a module that basically never worked right in the first place and was written by people that were, uhh, no longer on the team due to performance related issues. Every candidate patch “revealed” additional bugs that nobody really had known about before because they were hidden by the stuff that was “obviously” broken and that made the software more or less unusable. Management was upset as to why the software still didn’t work even though we had fixed all the bugs that were estimated as part of the month’s plan.
There is a sweet spot with respect to team size. Per Brooks, if you just throw more heads at the problem, you get serious communications bottlenecks, endless meetings to keep everyone up-to-date on what everyone else was doing to help prevent the same thing from being implemented more than one time and prevent people from introducing conflicting changes. “I changed the DWR export routine to no longer import the PFR Stat Code because nothing else in the system uses it, and taking it out makes the export 5% faster and you know that users have been complaining about how slow it is. (pats own back)” “Wait, you can’t do that! I’m working on Feature Request 45 due in December that is asking for an online dashboard that graphs PFR Stat Code over Number of Pips per Weekly Load Failure against the prior year’s Productivity numbers and I need that field! Also remember that the GGB module relies on the data schema interface and you need to let Bill know before you remove a field or else GGB will start throwing Type G Errors.”
Another problem is managers who think in terms of person-hours (gotta love them timesheets) and figure that they can just lay people off during slow months and then hire new developers later in the year if work picks up.
That sounds like, well, a typically software project (and the big reason for my collection of nervous twitches). “Let’s re-write this throughly tested, well understood date formatter; that’s fun! Also, let’s reuse the original SQL queries, because I have no idea how they work and won’t touch them”.
Hey, if I’m wrong I’m wrong. We’ll see in 2-3 weeks, eh?