More on the crashing federal ACA site — see The real story with Obamacare IT woes is out-of-control private contractors.
Of course, the Obama administration is to blame for the botched rollout, but there are other culprits getting less attention – namely, global tech conglomerate CGI, which was responsible for the bulk of the execution, and in general the ability of big corporations to get massive taxpayer-funded contracts without enough accountability.
Government outsourcing to private contractors has exploded in the past few decades. Taxpayers funnel hundreds of billions of dollars a year into the chosen companies’ pockets, about $80bn of which goes to tech companies. We’ve reached a stage of knee-jerk outsourcing of everything from intelligence and military work to burger flipping in federal building cafeterias, and it’s damaging in multiple levels. …
…To this end the Healthcare.gov experience should serve as a wake-up call to President Obama, who, after all, said early in his first term he wanted to rein in the contractor-industrial complex, and to the state governments doling out multi-million dollar contracts. The revelation here is that an overdependence on outsourcing isn’t just risky in terms of national security, extortionate at wartime, or harmful because it expands the ranks of low-wage workers; it’s also messing with our ability to carry out basic government functions at a reasonable cost.
I still say the Iraq War was mostly a money-laundering scheme. And I don’t think we’ve even begun to look at the money allocated for Hurricane Katrina repairs, and track how many pennies on the taxpayer dollar might have trickled down to the neighborhoods that needed repairing.
And one more time, anyone who believes that the private sector is always efficient and competent never saw corporations from the perspective of someone working in the production or engineering departments. Most of the time, the challenge isn’t doing the best job you can; it’s doing the job at all when upper management belongs in a clown car.
Have you considered adding an “Oh, Hell Yes!” button at the bottom of some entries?
I’ve been railing against privatization since I was a teenager, and it started under Nixon.
And I really railed against it when the cause of privatization was aggressively advanced by Reagan and Bush I.
Private companies aren’t responsible to the voters – they’re responsible to their shareholders.
By privatizing things, not only does government lose responsibility and accountability, but it costs taxpayers money.
Privatization is the greatest scheme Conservatives ever created!!!!!
It takes money from taxpayers, and puts that money in the pockets of political cronies and their companies – and it also provides the politicians a nice feather bed if/when they lose an election, or retire because of “family matters (hookers and blow).”
The result being, instead of accountable government people being held responsible for results, it’s corporate bean-counter’s who answer for the success or failure of a project NOT to the voters, but have to answer for the profitability of the endeavor – successful or not – to the companies shareholders.
And sometimes, like in “The Producers,” there’s more money to be made from a flop, than from a hit!
Jeeez, I’m so tired of singing this same shitty song since I was young.
It’s not a song you choose to sing – it’s cued up for you – and as for the beat?
Who cares?
You have to dance to it anyway!!!
And Republicans will, of course, say, NOT that what’s happened with the PPACA roll-out is the fault of privatization, but of government.
Yet another lie, in and endless stream of lies, lies, and more lies.
If we had privatized our war efforts in WWII, we might now be speaking either German or Japanese. Or, both – with the Western part of America answerable to the Emperor in Tokyo, and the Eastern part “Seig Heiling” whoever the latest Fuhrer is in Berlin.
I’ve maintained for a couple of decades that anyone saying private industry is in any way ‘more efficient’ than government hasn’t actually worked in either…
I second Bill Bush’s suggestion. Oh hell yes!
“anyone who believes that the private sector is always efficient and competent never saw corporations from the perspective of someone working in the production or engineering departments”
My experience is that we are very efficient when we need to be , it all depends on the customer. The federal government has never been one of the customers that demand efficiency, quite the contrary.
Production, engineering, manufacturing people working for consumer products corporations never deal with “the customer,” so that’s irrelevant. Ultimately the customer is the retail consumer who buys the product. The catch is that products have to be developed, manufactured, packaged, and shipped, preferably in a cost-efficient and timely manner. And if you’ve ever worked for such a corporation — whatever the product is; cars, books, soap, whatever — chances are you’ve seen all kinds of waste and inefficiency caused by upper management. Because upper management half the time doesn’t know what the bleep it’s doing.
Of course, their brilliant solution to making things better has been to ship these functions overseas. This doesn’t solve the waste and inefficiency problem; if anything, I understand, it makes it worse. However, this makes upper management look better because they’re getting the work done for slave wages.
I’ve also done work on contract, and usually the only difference is that (per the boss’s directions) we get better at covering up what’s really going on. It’s all about the presentation.
Yes, the whole idea of outsourcing basic government functions to private contractors never has made sense to me. There’s no inherent reason why a private company is more efficient, and if the government is going to pay a contractor X amount of dollars rather than spending the money itself, the contractor of course is going to extract a certain percentage of X in profits.
Of course there are areas where it does make sense, such as when it isn’t really a basic government function. Sometimes the government wants to do something for which they don’t have qualified staff themselves, so you go to a contractor that has that expertise, but you have to be able to discriminate, and there always has to be proper oversight.
And in fact, IT is one of those areas–I work for a company that provides government software myself, mostly on the local level, so I know it can work. Programmers are expensive, so it makes sense to have a group of programmers working on software that can be used by many cities, and save those cities from having to hire their own software development teams.
And if we don’t deliver on our contracts we get sued, and we get very bad reviews. But this probably works differently on the federal level, since there’s only one federal government. When there’s only one customer you don’t really have a market.
A little more technical, but ObamaCare Website Programmers Complained About Unrealistic Deadlines:
Integration testing is almost always done by the engineers who built the product. It’s basically putting all the pieces together (integrating them), and making sure they all play together properly. It is by no means something you pass off to somebody who isn’t intimately familiar with how the product works (or is supposed to work).
When this is complete, the project moves into “acceptance testing”, which is when the customer first gets to kick the tires, and make sure it works the way they want. This is before it even goes public. It sounds like they completely ran out of time for this very critical phase.
The fact that the builders were taking these short-cuts says to me that they were under extremely unrealistic deadlines. I can’t lay the blame on the project’s origins: whether it was public sector or private sector people – whoever did it, they were not able to get a reasonable timeline to do the work. Sometimes this is due to bad estimation, but it’s just as often due to management caving to unrealistic demands.
moonbat,
I can understand some delay until the SCOTUS came out with its decision.
But that decision came down quite a while ago.
But even while waiting on the SCOTUS’s decision, basic frame-working could have, and should have been done.
This all seems like the excuse all kids have at one time or another – “The teacher didn’t give us enough time to do the homework!”
Maybe I’m wrong.
Also, you’d think the Democrats would have learned about how important it was for the system to be tested after the Romney campaign’s super-duper ORCA GOTV program failed spectacularly on Election Day.
I think what most likely happened is that the executives who obtained the contract promised more that could be delivered within the amount of time they had, whatever that was. I’ve seen this happen a lot. Also, the task was complicated by more states opting out of creating their own website than had been anticipated.
I’ve read elsewhere that the federal site is a complex interface designed to screens users by making them jump through a number of hoops before they are allowed to continue to a state-specific site, and that a big part of the problem is that the front end and back ends are not talking to each other. If they had concentrated on state sites and made the federal site just a simple referral center, that might have been do-able.
I’ve often been involved in projects that were conceived and sold by executives who didn’t once bother to ask production if what they were promising was even possible, and sometimes it wasn’t. Of course, the resulting disaster was blamed on production.
Production, engineering, manufacturing people working for consumer products corporations never deal with “the customer,â€
Really? We do engineering service, turnkey design, etc. we deal with the customer directly all the time. Developing a software program for the feds is not a “consumer” product. It is a customized service, it requires intense cooperation between both the contract company and the feds tasked with implementing the program. The point of my post is to point out that my experience is that often times the folks in the federal government do not have the sense of urgency that comes with some more demanding customers. Putting all of the blame for the websites shortfalls on the contractor seems a bit terse. I suspect it was a joint screw up, the way I heard the President’s speech yesterday he pretty much admitted as much?
uncledad — Sure it was a joint screw up; my points were that (a) it probably wasn’t the programmers who did most of the screwing; and (b) big, private corporations tend to be inefficient and wasteful even when the government is not the client, which has been my experience. And this is mostly because in many big corporations there is no communication between the executives who get the contracts or think up what products they want to sell and the people in the cubicle farm who actually do the work. My worst experience with this was when I worked for Simon & Schuster/Paramount (before Paramount was bought by Viacom), but I’ve heard horror stories from people who work for all kinds of big corporations.
I’m not blaming the programmers. Now, it may be that the programmers really were a bunch of screw-ups. Maybe they sabotaged the product deliberately. Lots of things are possible. However, my experience is that most people in professional positions want to do a good job. Programmers love programming; engineers love to engineer; etc. And this was a big, highly visible project, and I can’t believe anyone in the contracted companies didn’t care if the product was any good. My hunch is that nobody talked to the programmers about what was practicable within the time frame.
And what’s the point of outsourcing if the contracted company won’t do a proper job unless their hands are held? That sounds screwy to me.
The history of contracting with private suppliers dates to far before Nixon and it was often bad. In WW II, one of the suppliers had a contract for an airplane that often crashed because its wings were too short. The feds wanted to add two feet to each wing and the supplier balked. Harry Truman, head of the Senate Committe on war waste went public and forced the manufacturer to fix the product. Truman has said that he discovered more than enough waste to fund the Marshall Plan.
In WW i there were problems with gas masks.
More soldiers and sailors were killed by botulism in the Spanish American War than by combat.
The Civil War saw massive corruption and profiteering. Lincoln’s first secretary of War, Simon Cameron, was fired for theft and crooked dealings with and by suppliers. The word shoddy comes from a fabric of reprocessed wool used for Union uniforms. The “wool” promptly disintegrated in actual use. Lincoln had to personally intervene with the Army to get the repeating rifle into production and use. The Army bureaucracy thought it was a waste of bullets. The rifle was known as the Henry during the war but as the Winchester in peacetime (yes, that Winchester).
The nickname Uncle Sam comes from Sam Wilson a food supplier to the Army in the war of 1812. My understanding is that the barrels, stamped with US, were loathed.
Civillian crooks: the San Francisco earthquake of 1906 revealed that the City Hall was not solid masonry but merely had a facade when it cracked wide open. Teapot Dome during the 1920s was a plot during the Harding Administration to rip off US Navy oil reserves for profit. Buchannan’s Secretary of the Interior stole from a trust fund for native Americans. His Secretary of War ripped off the US with private industry in stage coach lines.
In fact a man from Tammany Hall named George Washington Plunkett wrote circa 1900 that there are two kinds of graft: honest graft and dishonest graft. Honest graft included things like bid rigging while dishonest graft was simply stealing and taking bribes.
The bible says that Satan can appear as an angel of light. And with privatization a company like Blackwater can now appear as a company call Academia. It must be nice to shed your history and all negative attributes with something so simple as a name change…Kinda gives credence to the old adage.. Ye must be born again (so as to leave all accountability to the taxpayer behind).
Harry Truman, head of the Senate Committee on war waste
I really, really miss that guy.
If you want to read about a world-class, epic boondoggle, that makes HealthCare.gov look like a minor scratch, check out Will It Fly?, about the F-35 joint strike fighter, in Vanity Fair.
Moreover, this project was “politically engineered” – the various subcomponents come from a multitude of congressional districts strategically scattered across 46 states, so that no matter how bad the cost overruns or how much people want to kill it, it’s almost impossible.
re HealthCare.gov – given the size and complexity of the project, and the political context, it’s very likely that the project team had to deal with shifting requirements, well after the commitment was made to a particular deadline. Depending on how late in the game these changes arrive, this can lead to expensive rework and slipped schedule.
What gets me is that the “private sector” corporations are all about command economies – “socialism” as the tea party calls it.
The imagined failings of our “socialist government” are often said to be a problem because one person is in control (through various departmental bureaucracies). While that’s not true in government it is true to a much greater extent in a corporation. CEO gives orders and workers follow them.
Guess the difference is that the dreaded taxes pay for government. There’s a huge disconnect when the taxes get awarded to corporations then they’re OK. And when taxes help people in need they become tyranny again.
how does that work?
Privatization IS a huge scam.
And a growing one. Ever wonder why Republicans love charter schools? Because you can rob public schools to enrich unregulated, corrupt private companies. At the small expense of our children.
“My hunch is that nobody talked to the programmers about what was practicable within the time frame”
I agree. We have folks in management that promise timeframes and features that are impossible all the time, they just want the order, actually finishing the job is just a formality to many of them. From what I have read in states where the gov has invested in making the ACA work it is going fairly well. The timeframe here was unfortunatly dictated by politics, the baggers are not interested in fixing anything they just want to exploit failure for political gain. And with the media needing a new shiny object to exploit the pressure is even greater to make this thing work quickly.
A fix is in!!!
I’m currently working on a software engineering project where the upper management, for sound business reasons, of course, promised more than we could deliver in the time frame. Luckily for us, because we are careful to never publicize our deadlines, we can slip things out a few weeks late, which, combined with postponing some features and being willing to lower our standards for others, will allow us to ship our product soon, and it will basically fulfill our business needs. This is, in my experience, the way software is done, commercially. You set big goals, and come close, and do a lot of tweaking and cutting near the final release.
Sadly, healthcare.gov had a hard deadline, and a lot of features that were critical and couldn’t be dropped, and not enough time to do integration testing with the various legacy systems it has to connect with. I hear there were some redesigns called for at a bad time in the process, that probably hurt too. It must suck to be one of the engineers involved – they’ve spent hours working on what will be remembered as a historic failure.
Josh Marshall has a good write up about what may have happened, which sounds reasonable:
http://talkingpointsmemo.com/edblog/binders-of-insurance-companies-databases
Contracting out does not work and never has. It normally costs more than hiring employees to work directly for the Federal Government. First, employees working directly for the federal government are loyal to the govt because this is their employer. Contract employees only care about the guy/gal who signs their performance appraisal. It seems to me that most Republicans say govt doesn’t work and that they take govt jobs to prove it. Yet, they never see how they basically are saying that they, themselves, are incompetent. I have more to say; but, I will spare you all.
Just a guess, but with so many states dumping the job on the feds, I would guess that they ended up with more systems to interface with than originally planned. Don’t different state have different rules for Medicare? And different mixes of companies are operating in each state, too. I wonder how many total variations are possible? Also, if it is interfacing with large databases, do those shut down/stall while they are periodically updating themselves? As an English major, I am good at thinking up questions that contain tiny bits of nearly-knowledgable content. Even I can begin to guess at the complexity of all the programs. I know how complex it is just to keep up with foster animals by location, whether adopted or not, spayed or not, vaccinations due, spay and billing for retail spay, grant-supported spay, donated spay, already-spayed, or sent on to another rescue. Oh, and combo-tested or not, and positive or negative. And which vet we owe for which procedures for which animals. And who will take which ones to the pet store and which ones can share a cage so we can show 5 cats in three cages. All it takes is a few big talkers who have some influence to make things worse.
Yah, programmers are rarely consulted when it comes to deciding what can be delivered in what time frame, but that doesn’t excuse it. Well, for the programmers themselves, probably yah, but for the tech leads and the dev team leads and the technical management side, the fact that they were doing integration testing on a project of this type only in the last few weeks, and not by them at that, they should have been throwing not just red flags but red bed-sheets and tablecloths and quilts as well. For this kind of work, building the customer facing site and scaling the physical architecture for the new stuff correctly is the really, really easy part, and should have taken about 10% of the overall effort. Integrating with a bunch of systems that were never designed to be integrated with in the first place is the tough part and the place where the vast majority of the effort should have gone. The fact that the roll-out has gone as well as it has tells me the devs did a good job of integration coding … but without true end to end integration testing, that’s a shot in the dark.
So, either the low level techs were not communicating the impending doom correctly (in which case a bunch of folk should get fired), or the high level account execs chose not to communicate the impending doom to their government counterparts (in which case there should be prosecutions), or the government managers chose not to communicate the doom all the way up the government side of the ladder (in which case, again, mass firings), or the Obama admin has been lying to us about the readiness for a while now and hoping that with time we’ll forget about it (in which case, we’re probably back to prosecutions here).
My guess, tho, is that absolutely nothing will happen. The problems will gradually get better and the administration will not want anybody looking too hard at it, dredging up old business.
Which is too bad. Obama could and should take this opportunity to completely revamp the way federal technology sourcing is done, make it so he could get a team like the one that did his campaign tech doing the government tech. Doubt he will tho, to omuch politics at play.
-Ian
Oh, and just for the record, while I absolutely agree that the privatization of government is objectively bad and harmful, I would consider this as something that absolutely should be sourced to private enterprise, as it requires specialized knowledge and skills that would not be necessary on an ongoing basis for government to do on its own. It’s just the process for outsourcing tech needs to be rebuilt from the ground up.
Bill Bush:
“Don’t different state have different rules for Medicare? And different mixes of companies are operating in each state, too. I wonder how many total variations are possible?”
Yes, dear lord yes.
I am a programmer for one of the Big Name Property/Casualty insurance companies, and the complexity is mind-blowing … insurance is regulated on a state by state basis, so each and every state has completely different rules about, well, absolutely everything. And every time a new insurance commissioner starts in some state, s/he tends to change all the rules. I helped build the software agents use to actually sell the insurance, and it ended up being hundreds of questions long, with each and every question having different visibility and validation rules, up to and including NAME!
Really, REALLY glad I am no longer on that particular project…
-Ian
…anyone who believes that the private sector is always efficient and competent never saw corporations from the perspective of someone working in the production or engineering departments.
This.
I used to work for Ross Perot’s company, which in stark contrast to his cut-the-crap-just-fix-it campaign propaganda was the most suffocating, fear-driven, micromanaged and bureaucratic environment I have ever worked in — and I’ve worked for the UN as well as US local and state gov entities.
Pingback: The Mahablog » More on Technogate