I’ve certainly had projects given to me where the stakeholders outright lied to me about the requirements, scope or the timescale, or withheld very significant information from me, with the intent of introducing these as changes later - knowing full well (and sometimes actually admitting that they did so because) I would have told them it was underliverable in the full form, and would have asked for more budget, more lead time, more development and delivery resource, etc, and/or would have asked to halt or reschedule other projects already in flight, in order to make room for this one.
These projects all ran further over time and budget, and delivered less efficiently in general than they would have done if the stakeholders had just been honest and open at the start. In trying to pinch a penny, they lost a pound.
I did. I was a Six Sigma Blackbelt. And a good one. I managed to get huge improvements out of my projects - reducing help desk calls, better identification of root cause in help desk calls, reducing time to fix for desktop. I thought it was awesome. But not the proper tool for all things - certainly not the proper tool for projects.
As long as “nice to haves” are allowed to turn into requirements along the way, without changes to the project plan, no custom IT project will be successful.
You have to say “No, we’re not going to do that.” And then stick to it.
In my experience, this is where projects fall down. Not for lack of trying, but because a lot of places don’t necessarily have all the roles filled, or if they do, there’s still confusion about what’s the job of the BA, and what’s for the architect, and what’s for the programmer.
For example, where I’ve worked in the past, we’ve typically had a gap at the system architect position. So what we’d have is a BA who would gather business requirements and the high-level technical requirements (“these data elements need to go from here to here and be used for this other thing” type stuff), and then it would get tossed over to the technical teams. If things were working well, we’d have an entire other sort of requirements gathering exercise, where the devs and the BA would hammer that stuff out, with the BA taking the role of the customer w.r.t. interpretation of the requirements, and go back to the customer if there were things they couldn’t answer.
But as often as not, the devs thought they understood, and didn’t ask questions. Then we’d end up with something critical being misinterpreted by the devs, and all the fun that goes with that once that comes to light further down in the process. Having an architect whose job it would be to actually pair with the BA and translate the requirements into technical designs and directions would be a godsend, but it’s rare in my experience to find architects who actually do that.
The other one that we routinely have run into is that the customer has a very vaguely formed concept of what they want to do- beyond what the BA can realistically clarify. Like they may have an idea that they want something to do some particular function, but they haven’t really thought out their various business process changes, or how it’ll affect their operations, etc… and so as a result, as much as the BA may push, they’re still only getting vague answers, because the business hasn’t thought it out, but they want it on some kind of tight schedule.
In theory, some sort of Agile methodology would be great, in that it would facilitate the sort of prototyping and testing that would clarify a lot of that. But in practice, it seems that if the business/upper IT management aren’t thinking Agile, it turns into a death march of daily scrums and harassment about productivity, without actually doing the sort of iterative prototyping and development that would solve the problems.
In my experience it’s usually scope creep that kills custom IT projects. When you can eliminate it you have a good chance of being successful. This is why custom projects originating from inside the IT department itself usually DO work; there’s little or no scope creep.
Unfortunately, in my experience scope creep often comes from the very top leaving those running the project without a way to say “no.” They’re then stuck with completing a project they know is going to work very poorly (or not at all) and lose all enthusiasm for it.
If the poorly defined spec doesn’t kill you at the outset, mission creep will do so eventually.
My favorite quote from the client on my current project.
Client: “Can you just set it up similar to what you have done for other clients?”
Me: Uhhh…It’s a custom application we are building from scratch specific to your company, technology, and business processes.
I’ve lead teams running implementations of systems with many customizations and agree with most of the above.
Another problem I’ve seen, and tried to fight against up front during the sales process is expectation management. Typically, the sales team and sponsors will have the understanding that the implementation will take X amount of time.
I always point out that X is possible - PROVIDED that there is a lot of set up before X starts: the data center is made ready, the staff that will be involved is freed up from their other work, the approval process has been smoothed in advance, the other stakeholders (eg, the people operating the new system, not the ones developing it) are on board, etc.
I have been in many meetings where we were “behind” schedule on the implementation because all the prep work that needed to be in place wasn’t - so the project started, the clock began ticking, and we waited for some necessary pre-work to be done.
No one remembers that a year later when it launches and they say, I thought we were supposed to be done in 2 months?
We don’t call it scope creep - we call it the scope 100 yard dash.
I’m having serious flashbacks. And it’s terrifying.
What I always see on a lot of custom projects is a “long tail” of bugs, enhancements, fixes for edge cases, and other little nitpicks that make the project seem to drag on for ever. Because the original requirements were often vague or incomplete, and the company is building this completely customized application, it just gets into this endless argument over “is this button in scope?” or “this new report doesn’t include this thing that Jan from Marketing always has to do manually each month.”
Oof - been there; arbitrary, unworkable changes and decisions coming from the top. Members of my team often used to say “How do you tell God he’s wrong?”
With a properly formatted change request document?
Attn: Divine Being
Eternal Plane of Existence
1 Heavenly Estates
Universe, EX, 00001-0001I wish to bring to your attention a matter requiring your urgent response. A number of high level system requirements which are fundamental to this reality have resulted in conflicting derived requirements, functional interfaces, and source control specifications resulting in difficulty with meeting schedule and performance goals as specified in the Part 2 Production Specification, specifically Section 3, Paragraph 4.3.1.5 through Paragraph 4.4.6.2. Please see the enclosed redlined recommendations for modifications to the requirement specification to resolve these conflicts.
Signed:
Ernest P. Fugglebinder, Sr. Requirements Analyst, Navigable Waterways, Coastlines, and Littoral Regions, Oceanographic Design, Terrestrial Planets and Moons
Stranger
Often it’s not ‘scope creep’, it’s scope discovery.
In other words, the scope creeps because the system wasn’t properly defined in the first place.
I worked for 20 years for a large software engineering company. I’m a Six sigma green belt. Over the years we did extreme programming, waterfall, agile, you name it. Managers always chasing the perfect paradigm that will lead to more controllable and repeatable project activities. But the problem is much more fundamenfal.
Here’s how a custom project would work:
-
Management of some factory decides to improve productivity. They call us to come in and help.
-
Business and sales teams visit the customer and gather requirements.
-
Architects design a system from customer requirements.
-
Requirements are then ‘massaged’ by bringing in development leads and negotiating what can or can’t be done on time or on budget. The customer is told how much we can do, or how much extra time and money will be needed to do it all, so the customer re-prioritizes and attempts to break down requirements into must-haves, nice-to-haves, etc. Eventually we come to the final set of requirements.
-
Requirements are divvied up between project teams, who are expected to code up the features described therein. Project people break the requirements down into ‘stories’ for the backlog.
-
Tollgate reviews with the customer invariably lead to changes, because as people spend more time looking at the system, hidden requirements rear their heads, or customers realize they were wrong about what they wanted once they see it, etc.
-
After much slippage and renegotiations and reviews, the project eventually is at a state where the customer can start integrating - which generally reveals more hidden requirements and more development.
-
Once the new system is deployed and in the hands of users, more hidden requirements reveal themselves, or the customer finds out that the plan their managers came up with for a system doesn’t work in the real world, and a major revision is required.
The problem with all this is that it’s a giant game of telephone. It starts with managers at the customer assuming they know their own processes, but often they do not because their process is made workable on the ground by numerous hidden processes the workers develop to get the job done. The environment is complex, and often there are no people in the organization that actually understand it fully.
When business teams gather requirements, much context is lost. A great requirements team will spend a lot of time just observing how the current system works, interview stakeholders across the company, gather multiple perspectives, etc. This almost never happens. Instead, requirements gathering devolves down to a series of meetings with the customer’s IT people, who also don’t do the actual work they are building a system for,
When requirements are translated to stories, even more information is lost, and stories are often guesses as to what the requirements actually mean, while the requirements themselves were interpretations of someone else’s idea of what was needed, which in turn was based on their assumption of what the end users actually do.
The process of negotiating requirements and dropping/delaying some was also fraught with problems. Sometimes you’d keep one requirement, but drop another one that made the one you kept useless, because the people making those decisions had no idea they were related. For example, a UI requires objects to be classified so they can be displayed properly, but another team drops the classification story bexause they are behind and their manager doesn’t think it’s important. So the UI loses a critical feature.
We used to get requirements like this (real example), after all the filtering and breaking-down was done:
- Customer needs a screen for showing how many peoole are logged in.
I pushed back on this, trying to find out why they needed it so we could do it right, What are the goals behind the feature? What are they trying to accomplish? I was told to ‘just get it done’. So I delivered a blank screen with “Logins: —”. Hey it met requirements, When the inevitable complaint came, I explained again fhat the requirement sucked. So they finally allowed me to actually talk to the customer for clarification.
It turned out fhat the real problem is that the customer had X seat licenses, which should have been enough, but users kept complaining of not being able to log in because no licenses were available. This turned out to be a big licensing issue, boiled down into the requirements as, “Customer wants to see how many people are logged in”. Utter fail. We eventually delivered a great UI that helped the user discover the exact problems (shift overlap and people not logging out, mostly), but the original requirement completely obscured the issue that was the actual requiremrnt (“Give us a tool that allows us to manage our licenses effectively and prevent overuse of licenses”).
Also, the original requirement was scheduled for just a few hours of work, but because the problem was transient we had to build a whole audit trail system around licenses. It is scope creep? Nope. The original need was called out - it was described in the original meeting. It’s just that along the way someone decided that it was unnecessary information for lowly software engineers because all that was needed was to show licenses. Just build the screen, code monkeys!
In my experience, large projects die of complexity. We are building tools for the real world, and the real world is complex. The bigger the system, the harder it will be to get it right. And software development itself is complex - complex systems used to build products for complex systems. That will never be easy.
I’m a firm believer in building minimum viable products and development through iteration. It’s the only way to manage complexity. MVP thinking is an admission that you aren’t as smart as you think you are, that you don’t understand either the problem space or the solution space as well as you think you do, that large systems can be impenetrable and impossible to completely understand, and that that large software teams introduce their own complexity to the process.
Build the smallest thing the customer can use, get it working, observe, and iterate. Build continuously. Integrate continuously. Demo often. Prototype as much as is reasonable. Assume you don’t know everything about what you are building. Develop for refactorability. Don’t get married to ideas or code.
Keep the customer in the loop. Get rid of silos of communication to minimize telephone tag and misunderstandings. Make the development team do field trips to customer sites so they can understand the environment: A UI that works for someone sitting at a desk probably won’t work for someone standing at a workstation wearing safety glasses and gloves and looking at a screen through a scratched-up blast-proof window. Understand that software development should be looked at not as an engineering process for building a known thing, but as a form of exploration of customer’s needs, because in any non-trivial system you probably don’t understand them at first.
One last thing: I think software projects are often late and over budget due to a feature of complex systems: They look simpler the farther away from them you are. Normal systems are the opposite.
Let’s take a watch for example. Watches are complicated, but not complex (their movements are literally called ‘complications’). When you look at a watch movement, the first thing you think is, “Holy cow, that’s a lot of stuff. That must be really hard to design and build.” The way you understand the watch is to apply reductionism - take it apart, and look at the smallest parts, and you can suss out the function of the whole thing. The problem gets easier once you dive in and break it down. And once you’ve reduced it to its parts, you can very accurately get a handle on what’s needed to build it.
We try to treat software like this, but complex systems are the opposite: from 10,000 feet they can look simple. Dive in, and they are less simple. Keep diving, and it gets more and more difficult. And if you dive down to the lowest elements, they mean nothing. The system is not the sum of the parts as it is in a watch. And even more daunting for software engineers, complex systems are defined more by interactions and connections than by the atomic elements within them.
All this causes people dealing with complex systems to constantly underestimate their complexity. In short: in complex systems the devil is in the details, and the details are really hard to see. Thus, we constantly underestimate how long it will take to understand them and build them. Building minimum viable products and iterating is a good strategy for managing this.
Stranger:
Loved that!
This is almost always a problem when you are developing a product or system that is on the bleeding edge of technology. It is impossible to define detail requirements when you don’t actually know what the limits of the developing technology is, and is an argument for not doing rigorous top-down system engineering in such cases. What such an effort requires is really someone who is the “technology architect” with good insight into all parts of the project and a rigorous vision of what the result should look like regardless of technical limitations. This kind of talent is rare because it requires the combination of technical knowledge, ability to manage interactions and conflicts between different groups, an ability to dynamically adjust expectations and schedules, and (in the case of a commercial product) provide something that really meets consumer expectations. Steve Jobs arguably fit this role in a way that his successors have not, and I have a hard time coming up with other examples of this; most really innovative “IT projects” have started from a “Wouldn’t it be cool if…” and then result in a usable product after numerous iterations and releases, e.g. the spreadsheet, modern databases, computer games and graphical simulations, engineering and architectural CAD software, et cetera. The problem with this big enterprise-wide IT projects is that they are expected to provide a robust product at the end of a rigorous schedule without the constant tuning and rewriting that most successful software products have gone through.
Stranger
I heard Elon Musk being interviewed, and he was asked if there was one thing he would like to tell engineers who want to build great products.
His answer: “Never blindly accept the constraints you were given.” He gave the example of the engineer who came to him and said, “I know you said to build this out of carbon fiber, but stainless steel woild be better for the rocket.” This at a time when SpaceX had already invested many millions in carbon fiber mandrels, etc. Musk looked at the analysis, agreed, and torched millions of dollars of extant work.
The problem is, it takes an Elon Musk or Steve Jobs to do this. Without a powerful, brilliant person at the top who can override bureaucracy and bean counters, large engineering teams only work by placing constraints on everyone. Steve Jobs was a good product guy, but there are lots of good product guys around. The difference is that he had the clout to enforce his decisions. Everyone else has to live within a large set of compromises. This is why large companies with large bureaucracies have a very hard time actually innovating. Too many constraints.
One of the best books I’ve read on this is Loonshots. Excellent additional examples beyond Jobs.
This might be off topic, but I’ve always got the impression that the celebrity entrepreneurs were more lucky than anything else, and that a lot of companies wind up with unnecessary scope creep because they have a few too many people trying to come up with the next “so crazy it just might work” idea.