You’re assuming that shops are completely identical, except for how well they follow SENG techniques. That’s definitely not the case.
Because if there’s one thing we’ve learned in the history of software engineering, it’s that throwing bodies at problems solves them better.
It’s no coincidence that one of the first (and still very topical) software engineering books was called “The Man Month”.
What?
I’m not assuming all shops are created equal. I’m just pointing out that there are a disturbingly large number of shops that manage to survive without any SENG techniques whatsoever, therefore, SENG can’t really be essential to the process of building software.
It’s not essential to building software in general, but it sure does make it a lot less painful.
Sometimes I wonder whether engineering is the correct paradigm for writing software; I sometimes think that “writing a novel” (with multiple drafts) better reflects the thought processes that go into it. Oftentimes, people take the “if it’s not broken, don’t fix it” approach, but I’ve seen a lot of code that would benefit from being rewritten from scratch.
Even if you have explicitly detailed requirements, there will inevitably be conflicts between them that won’t be noticed until you code it. Business people who come up with these requirements won’t see the conflicts, because to do so would require a very rigid type of thinking akin to what programmers do. Programmers won’t notice the conflicts at first either, because they are often the side effects of multiple conditions that are seemingly unrelated. That’s why you usually don’t see the best way do something until you’ve done it once.
But once you’ve finished, it’s deliverable, ugly as it may be. Most people don’t want to do it over, just to get it “right”.
You could call “The Mythical Man Month” a “software engineering technique”. But that doesn’t make it any less true that throwing fresh bodies at a deadline is usually a waste of time.
I agree that the user side creates a problem by failing to recognize the complexity of their own business models. (And while there are idiots in the user community, I do not think that it is idiocy so much as simple ignorance that drives that problem.)
However, the programmers described, here, are very much a part of the problem. It is the job of the programmers (or the analysts) to study the situation with sufficient completeness that they are not blindsided by “multiple conditions.” Now, I will agree that part of this problem still goes back to the user community that insists that they should not have to pay for analysis, (“Why are you up here bothering my staff with questions on my dime? Why aren’t you coding my system?”). However, that is, ultimately, the fault of IT management who has never taken the time to demonstrate just how complex software actually is.
I worked at a manufacturing firm that had no problem recognizing that it took so many months to work out a new engineering solution (that was little more than finding out a way to convert so much input power into so much output performance, timed correctly while controlling heat generation) but found it horrendous that IT would ask for more than a week or two to design a system that had to incorporate input from order entry, production, shipping, billing, receivables, credit collection, accounting, and finance.
On the one hand, it was wrong that the business users assumed that translating their complex rules into code was a simple process. On the other hand, they had over 30 years of IT management simply nodding their heads and shuffling off to start one more ill-conceived and poorly executed project with no effort on the part of that management to hold their users accounatble for their own errors.
Given that the culture currently exists to place the burden and blame of all failures on the lowly coder, it is in the coder’s interest to demand sufficient information (or to go find it on his or her own) to avoid being blindsided.
Maybe I got lucky. My very first direct contact user gave me half the information I needed to complete his “simple” project and my error/audit routine flushed out the rest of the information he had failed to recognize, but from that day on, I never assumed that the user knew (or deigned to tell me) everything about his own system and I have rarely been blindsided in the ensuing 25 years.