This was posted on HN so that we can make fun of it, and I think we're right to do so. You can be as wishy washy as you like about different situations requiring different processes, but have you ever seen somebody successfully isolate software engineering from coding and put them in sequence?
Waterfall is actually considered the very best model for software development, when the requirements are known up front in excruciating detail.
An average project experience around 25% change in requirements. Hence waterfall is highly unsuited for the majority of projects, since it does not handle requirement changes well.
However, in a few areas such as aeroplane control software, space shuttle software and similar, waterfall is indeed the optimal development process.
Different projects DO require different development processes. That's not wishy washy, it's the reality of software engineering.
Mindlessly applying your favourite development process to all projects, be it waterfall, scrum, evolutionary prototyping, spiral development or what have you, will very likely result in failure, if the process does not fit the project.
"Waterfall" was defined in a paper describing how NOT to do software development. The problem was that very few people at the time had any process at all with software so seeing this they gave it a try.
The paper you and "gujk" are referring to, I believe must be Winston Royce's paper.
It's wrong to say that the paper is a description of how not to do software development. Rather, it's suggestions for a set of improvements to the process, while retaining the sequential nature. In the paper, Royce considers waterfall to be a good process, but with flaws which result in high risks. These flaws is what he addresses in his paper.
His suggested approach is derived from pure waterfall, and is very much similar. It falls into the "modified waterfall" category of life cycle models, together with a group of other similarly waterfall-derived models.
He said, and I'm quoting, that the pure waterfall method is a good concept "but is risky and invites failure." He described how it leads to massive cost and schedule overruns. It was definitely a process that he did not think you should use.
His proposed solutions that you call "modified waterfall" are modified quite a bit. (As an aside, he never used the word "waterfall.")
I have read the paper multiple times over the years.
The quote "is risky and invites failure" is correct. However, another quote, even on the very same page:
"However, I believe the illustrated approach to be fundamentally sound. The remainder of this discussion presents five additional features that must be added to this basic approach to eliminate most of the
development risks."
These suggestions are to 1) Add an additional (preliminary) design phase, after requirement analysis. 2) Much greater documentation. 3) Perform the entire sequential process twice (not x amount of iterations, just two times). 4) Additional activities and documentation during testing phase. 5) Involving the customer.
His diagrams require close scrutiny, as on first inspection it seems the process is markedly different, when it is really not. It is still an entirely sequential waterfall process, just with an extra phase (preliminary design).
The term "waterfall" wasn't coined by Royce in his paper. His paper was a set of suggestions to improve what later became generally known as the waterfall process.
And Royce was ultimately advocating an iterative method (rather than a strict waterfall), not unlike most agile methods today, although his cycles were much larger.
If one is interested in software development methodology, I strongly recommend taking 15 minutes to read the actual "waterfall" paper[1].
Poor guy, Winston Royce is one of the most misunderstood folks in the industry.
(By the way, curiously enough, Royce's son William was a major contributor to the Rational Unified Process.)
He wasn't advocating iterative methods in the agile sense, in fact he calls for "complete program design" before any analisys or coding, along with intensive documentation, planning, controlling and monitoring. An iterative approach to development drops nearly all of those.
The graphic where he shows cycles going back stages is used to exemplify failure, which he proposes to fix via the strict practices above.
Or, he could be advocating the most agile process that the technology of the time (1970) would allow. Remember that there were no desktop computers, no internet, the programming language C was just being created.
Agile processes don't depend on technology, just people, paper and communication, in fact technology can be a burden sometimes.
Just in case someone from the future misunderstands this: I don't mean that agile drops analysis, documentation, planning or monitoring (they are essential to it) but it leaves the "intensive" and "before" parts out.
Development processes very similar to todays "agile" family, existed back in 1970. "Evolutionary prototyping" for example (a member of the "prototyping" process-family), is one of the precursors for todays agile practices.
Most of us would probably agree that their method is worse than more modern methods, but spending time making fun of them is like making fun of someone for driving an old car. You could be working on your startup right now.
You have a choice of koolaid. The waterfall brand is implicit in the statement "We don't write a line of code until we know your business as well as the people who keep it running every day." - it is founded on the assumption that you can finish requirements for the entire system before starting to deliver working software, and no feedback or iteration is needed.
My experience is that for the majority of software projects, this assumption is horribly broken. And even in fairly well known problem domains where it might work, for the majority of business users wanting new software, the ability to communicate every detail of what it is that they want is horribly lacking.
Scope creep, architecture problems, unanticipated requirements etc. can and does break waterfall projects; this was well known 10 years ago. The probably of it going wrong rises with the size of the project, and it's a curve that's well above linear.
There are two ways out: You can plan for change or you can make the penalties against change more severe - roughly, do even even more big upfront design and penalise change requests; or do less of upfront design, iterate and embrace change. In many cases the "big upfront design" way is impractical in time and expense.
The most deadly thing in software is the concept, which almost universally seems to be followed, that you are going to specify what you are going to do, and then do it. And that is where most of our troubles come from. The projects that are called successful, have met their specifications. But those specifications were based upon the designers’ ignorance before they started the job.
Very true, and I should have picked an earlier date. But I was erring on the side of caution, and it somewhere between 1990 and 2000 when agile methods, which explicitly reject big upfront design, became mainstream.
I also have the choice to ignore all the methodology snake oil salesmen and actually write software. Every "methodology" is bullshit. The new ones are not any less bullshit than the old ones. The fact that your response assumes I must love the old version of "follow our stupid manager nonsense" is really pretty sad. You honestly can't even imagine a world where I might think all of the bullshit is equally worthless? I have to be an adherent to one of these religions?
That response is even sadder. I haven't chosen the "just code" koolaid, but your religious beliefs are so deeply ingrained that you can only perceive me through that lens. As a result, you miss out on the great big world of "every situation is different and you should do what is appropriate instead of what some dipshit hawking books and seminars says".
Actually, both of my previous comments made mention in passing of situations where I though a method wouldn't work. You say "every situation is different and you should do what is appropriate" but make no mention at all of which method is appropriate for which situation.
Books and seminars, we got these, but good agile is done by adding in some open source software tools, a whiteboard, pens and cards, and people with enthusiasm and an open mind. The last one is most important.
If we're still talking about the original post, then absolutely it's a belief system. Not a very well thought through one, though.
Pure[1] waterfall is doomed from theoretical point of view. You cannot control a system without a feedback loop, unless you have a 100% accurate knowledge of system's state and future evolution. Waterfall lacks feedback loop => you can't really expect much from it.
[1] - in real life there's always some kind of feedback loop in the process; I'd argue that the primary reason why Agile methodologies seem to work better is that they have a much, much tighter feedback loop than waterfall.
Whether or not one drinks the koolaid doesn't matter much.
15 years ago, 1-3 year release cycles were the norm. Now they're uncommon, and becoming less and less so. Some well-regarded shops (e.g., Etsy) now release many times per day. This isn't driven by what process is fashionable; it's happening because of improvements in technology and the way the Internet has shifted everybody's expectations.
Whether you're officially using some Agile(tm) method or not, you'll have a lot in common with those folks if you meet user demand for faster delivery. There are some niches that are lagging on this trend, but my pals in those areas still feel the pressure. They'll still be able to be waterfall-ish for a while longer, but I doubt it will be forever.
I can. I'm seeing automated tests. A whole lot of automated tests. Unit tests, performance tests, integration tests, executable specifications, tests with real hardware.
I can be done; you have to optimize for what's important to the project at hand.
People have been doing it with medical devices, so I don't think it's impossible. I agree it would have to be done differently, but then avionics waterfall has to be done differently than order-entry systems waterfall, so that's nothing new.
I actually once worked for a place that didn't have this exact process, but they had a heavy process I'm sure nobody would describe as agile. And it worked pretty darn well.
They designed educational games for early elementary education. Every game activity was hashed out with a working committee consisting of an educator, a technical writer, an artist, and a programmer. They'd take a few hours and talk about the concept that the kid playing the game was supposed to learn (say, measures of weight), then they'd brainstorm ideas for potentially entertaining scenarios in which the concept could be explored (say, a flea circus), then they'd start to storyboard various activities and interactions.
Then the tech writer would produce notes, people would review them, and 2-3 weeks later, they'd do it again, refining concepts and interactions.
This process would stretch out over months, and by the time it was finished, the tech writer's notes essentially comprised a complete state machine for the activity described in English on paper. And the artists/media people had the assets for you in the right media formats. Usually sensibly named. Programming was generally pretty straightforward.
You know how a lot of the time that non-technical "idea guys" seem to have this terrible attitude? "Hey, I've already done the hard work of coming up with the idea, all you need to do is translate it into code."
This was one of those rare situations where it was often more or less true. But it was because the people coming up with the concepts were really part of a creative and disciplined process where they had to come to grips with the details in a way that usually only programmers have to.
Oh, and after things were done with programming, they had a QA team compare it against the spec and an educator's review.
They produced a pretty good product. Sometimes I think it's a shame I didn't stay longer.
I think this process would work pretty well for educational games as each game was a self contained thing.
As a teacher myself I do often incorporate a game (computer based or actual paper, dice, pieces), an interactive Web site, a video or other objects into a lesson. I also direct students to Web based interactive content as part of homework along with 'traditional' work.
In effect, my lesson planning is the 'interface' between the (no doubt well coded and engagingly designed) objects your team designed.
I suspect, from my own interactions with the business applications used in the largish educational institutions that I work in, that it might be harder to use this model when the interactions between the individual activities get complex...
