My sister works as a project manager at one of the big UK banks. They are anything but agile--projects take months and have to be planned out to the smallest detail for legal and compliance reasons. Makes sense, as the smallest decision affects millions of people, and there are regulations to satisfy.
She complains she now has to do daily standups with other project managers. But she has no idea what they do, they have no idea what she does, and it's a matter of just talking "Yeah, doing the same thing I was doing yesterday".
But since there are like 20 managers in just her group, this takes an hour everyday.
And she's like--we're not in software, why are they forcing this scrum thing on us?
I tried to tell her about the Priesthood the Agile, the Saviours of Our Codes, bless them, the Agile Manifesto Priests, but decided it would be too much history, so I just nodded and said "meh".
Without know any details I can easily say 20 managers in a room for an hour giving updates is for sure not “agile”.
It also contributes to a lot of the other examples in this convo chain. Bad agile is bad. Just like bad waterfall is bad.
A key tenant to agile is the reduction of inefficient processes. So in the next retrospective I would nominate that there are no longer weekly 20 person meetings.
It just sounds like a scrum of scrums on a tricky project.
I'm on a major project now where there are 80-90 people across around a dozen workstreams trying to stay coordinated on a fast moving project in a highly regulated industry. We usually wrap in 20-30 mins but it has definitely taken hour in the past just due to the sheer number of interdependencies and hard gates.
20-30 sounds reasonable. And should more in the weeds discussion be needed it should become the source of specific follow ups.
It’s the standing 1 hour meeting where some people leave thinking it’s a waste of time. The waste is what agile is about, or not about. If it’s productive, needed, or even just healthy then it’s appropriate IMO.
That’s been my world. And no, it doesn’t shine. It’s the worst way to manage a complex project that anyone has conceived of short of anarchic project management with no plans and no deliberate coordination. Waterfall is a great way to fuck up a complex system.
It is unresponsive to change. The smallest ambiguity will result in the undesirable outcome of building the wrong thing. The only good thing about Waterfall is that contractors can make bank on the rework, but their customers will be screwed every time.
Anything that has feedback loops and responds to differences between reality and the plan by favoring reality. I have watched too many Waterfall proponents lead projects to failure because they were too stupid to acknowledge that reality had diverged from expectations.
For software systems, specifically, also anything that encourages early integration. Integrate early, integrate often. A failure to do so means that you've broken your feedback loops or extended them to an arbitrary time in the future so that they have little to no value. You cannot properly verify and validate a system without integrating the major subsystems.
Independent of most of the named processes and the technical discipline, V&V. Verification and validation are critical, ongoing activities that must be part of your work. This largely pushes towards automated testing, formal specs where applicable, and customer collaboration (to get validation of the system at critical junctures).
Waterfall doesn't do any of those things or isolates them (like V&V). It only works on large scale systems if reality matches expectations, which is rare.
My preference is to look towards Lean Software, most of the Agile methods, and the classical methods like Iterative & Incremental and Evolutionary (which were agile before Agile). V-Model is Waterfall-ish but integrates feedback into the system and makes V&V an ongoing activity. I prefer it paired with I&I or Evolutionary for critical systems (my primary domain).
A small team of smart people all working together with a shared goal, a clear vision, flexible roles, open communication with subject-matter experts, and a manager who acts as a shield against external forces (obsessive changes, design by committee) and as a provisioner of resources to the team?
Sometimes, but in my experience anarchy (which adds some business value, in a non specific way) is often better than waterfall (which adds none and actually hinders by preventing small quick schemes that add value now for the promise of value in years to come)
Not much planning. Direct contact with the people who use the software. Good coding practices, preferably TDD. No time pressure. This is pretty much agile how it should be in theory but not in practice. Agile in theory is actually a very good development process. Agile in practice tends to be a bit worse. Waterfall always is the hell. My boyfriend is at the moment doing waterfall development. He hates it. So much useless paperwork.
I disagree. Among the things that I listed was TDD. If one practices that consistently one can create highly reliable software. In fact one might argue that using TDD for the new landing page for the company website might be overkill. Maybe you do need a bit more planning for your 'mission critical system' in order to perhaps attain performance goals. However, I will tell you here and now that if you just do TDD for your mission critical system and you do it well you will already be better than most programming going on for so-called missions critical systems in many, many places.
No, but like any practice that becomes known of outside a narrow group of highly-competent, highly-motivated experts in the relevant area, TDD can be (and often is) cargo culted. It is thus often the case that people's only practical contact with “TDD” is with a cargo cult application of it.
This is true. The most important way in which TDD can turn into a cargo cult practice is to test every class and every method separately. This then turns into the testing of implementation details that are very much subject to change. It can also lead to the testing of trivialities, like 'can the standard library of my language still add elements to a container'. One should in most cases write tests about somewhat higher level properties of a system where the user might actually recognize the tests as being about stuff that they value. In most cases one would write tests about a small set of closely related classes. There should also be tests about somewhat larger clusters but there should be fewer of them because they tend to be more brittle and have worse performance.
Anything taken to extreme is a problem. Too much water will kill you. Not enough water will kill you. System level automatic testing has saved me hundreds of hours of work. And years of enjoying zero bugs in production. However I achieved that with zero unit testing. System level testing only.
And here come the cultists to tell me it's not a cult. TDD is at best training wheels for an incompetent junior developer. All of the TDD cultists I've worked with in the past have needed a lot of extra hand holding in how to properly design architecture, how to write more legible code and have never been allowed to go anywhere near interface design or development.
TDD is an implicit repudiation of Agile. One of the principals of the original Agile manifesto was "working software over comprehensive documentation". After all, a test is a form of documentation, albeit one that is more suited to a programmer's tastes.
Agile places software and documentation at opposition - TDD tries to tie them together.
TDD was developed in the 1990; Agile was articulated in 2001. Were they actually opposed, “Agile is a repudiation of TDD” would be more defensible. But your basic premise is wrong...
> Agile places software and documentation at opposition
No, the “X over Y” statements in the Manifesto aren't about opposition, they are about subordination. ”over” can be substituted with “should shape the use of” rather than “should exist instead of”
You're right about the historical chronology, but TDD adoption IMHO came much later, after Agile exploded in popularity. Maybe it's just my neck of the woods, but I heard of TDD only much later.
As for the "X over Y" statements, well, it's a rhetorical style which definitely creates an implicit opposition. Subordination makes no sense here. Our job and the business value is in working software, of course we want that most of all. Why mention it over something unless that something is in the way and we want to discourage it?
Let's say I told people "Crossing the road over obsessively looking at both sides of the road"*, and kept referring to "obsessive looking" while barely mentioning people should look at all. As guidance this is horrible, for reasons I believe are obvious. If I said later "I only meant obsessive looking" and "you should adjust your crossing process so it suits you" it wouldn't make my advice right, would it now?
* Documentation is of course much less necessary, I'm just taking an extreme example to show the rhetorical style.
> That is, while there is value in the items on the right, we value the items on the left more.
The value of the items on the right (per the manifesto, if you disagree on the relative values that part can be debated) is lower than the value of the items on the left, but it is not zero or negative.
I think I've alrady explained why I don't find that adequate at all.
If we were to declare doing X as contrary to getting the job done, always describe X with demeaning or excessive terms, and give no limiting principal when X is actually important, we shouldn't get shocked when people read what we wrote, and decide X is at best unimportant, because that's what we really wrote whatever we intended.
Sure, we could later write a blog saying 'we never actually told you not to do X' and be technically correct, but in reality we gave bad guidance that drove people away from X even when X was actually important.
The problem is that waterfall is a myth propagated by the Agile evangelists. Development waterfalls were not frozen and rigid as claimed by Agile advocates, but we're in fact fluid allowing certain steps to start before others were finished.
In fact Royce, known for his 1970 paper from which the Waterfall model for software development was mistakenly drawn, demonstrated that while the development of large software systems required a more thorough approach, there was inherent risk in a single-pass sequential approach. He proposed an iterative approach and advocated that projects should pass through this at least twice.
That’s great! The more time managers don’t interrupt developers the better. Suggest that they extend the standups to 2 hours a day. It will make the developers even more productive (I am only half joking).
Good article, but it doesn't really introduce any concepts that I haven't heard (or said) already.
There was the line about "not introducing much process and structure," as basic Agile principles. I completely agree (and have said so, myself[0]), but the issue is that software folks (and engineers, in general), don't like "blurred lines." We need empirical data, measurable artifacts, concrete results, etc., so that "fuzzy logic," (what I consider) "true" Agile really needs, is difficult, in practice.
This is why I think that experienced engineers, that treat the vocation as a "craft," are often better suited to "true" Agile. They understand the ramifications of decisions that need to be made quickly (without having to run a formal risk analysis and methodology review).
Remember that the authors of the Manifesto were all highly experienced and capable engineers. They wrote a document that basically chronicled the way that they, themselves, approached development.
As for bugs and tech debt; personally, I abhor debt, of any kind. That posture has served me very well, in life. Tech debt is just another loan shark.
I like to run on what I call "constant beta." The application is always what I consider to be "bug free" (we all know it isn't, but let me have my happy place). If I encounter a bug, I stop all forward movement, and fix it; even if it is a "small" bug.
As you can imagine, this was not something my managers used to allow, but, as I no longer have a manager, I do it this way.
>If I encounter a bug, I stop all forward movement, and fix it; even if it is a "small" bug.
Reading through the Toyota production system this is one of the things that is recommended for finding issues early before they become big issues. Stop the manufacturing line, find out what's wrong, fix it, continue.
At the beginning you will be stopping a lot, and I still am in awe of the manager that got this technique approved the first time, but after a while you will find that you just produce fewer bugs by virtue of a) recognizing patterns in how you produce bugs and avoiding them, and b) not having to work around as much technical debt, these clever workarounds are what bites you later.
The very idea that software should be produced by the same production techniques we use to manufacture cars is a sickness, and the sooner it can be eradicated from popular culture the better.
It's everywhere from Lean to that insufferable Mary Sue of the Phoenix book. It makes for decent storytelling, but it has nothing to do with how any successful software has ever been made. There is no assembly line and we do not pause it. It is a parable that is actively unhelpful. Lean manufacturing was not even invented to develop or design cars, so it's purported origins doesn't even make sense.
See also "software is like construction work" which used to be popular before. That wasn't useful either, and anyone who's seen the construction business knows it's nothing like management literature of yesteryear would make it out to be.
Sorry for the rant. I don't hate agile. This just happens to be a sore spot, comparing software to "a car" is intellectually lazy. We should be able to do better. It's easy to imagine the day when someone asks why we can't design cars the same way we design software. Not the other way around.
The manufacture Lean and JIT were disseminated by Toyota. But the software development things called of Lean and JIT aren't exactly equivalents of the manufacture versions of them.
I also disagree that the equivalent of stopping the production when you spot a defect is fixing your bugs as soon as you see them. The equivalent would be something more meta, as asking yourself why that bug was created and changing the way you work (like in adding a linter rule).
>The equivalent would be something more meta, as asking yourself why that bug was created and changing the way you work (like in adding a linter rule).
Which is exactly what you do after stopping that production line and finding the issue.
"The application is always what I consider to be "bug free" (we all know it isn't, but let me have my happy place). If I encounter a bug, I stop all forward movement, and fix it; even if it is a "small" bug."
I have the luxory of no manager either, but I like to move fast and not have the cost of context switch, to immediately fix every small UI bug or glitch, like I used to.
I rather keep track of them and usually wait, till I drown in warning messages - and then I take the time to properly clean up and refactor things.
I found out, this way I am more efficient. Because since I suffer from perfectionism, I would get lost in fixing and improving small things. Now I mostly ignore them and focus on the important stuff and only fix things immediately, if they are in the way.
I see, we have to do a coding competition at some point ;)
But sure, routine helps. And probably also what you are building. I am mainly working on a big project with lots of legacy code involved, that is just not right. I cannot just fix anything I come across. I would never get the priority things of my list.
> I completely agree (and have said so, myself[0]), but the issue is that software folks (and engineers, in general), don't like "blurred lines." We need empirical data, measurable artifacts, concrete results, etc., so that "fuzzy logic," (what I consider) "true" Agile really needs, is difficult, in practice.
This is a good observation, but it's not just engineers: the fact is most people don't like ambiguity, and the more it cuts across org structure lines the less they like it. This is no big deal as long as the org structure lines are well drawn and business are covered adequately, however at scale it also limits the type of changes that can be done without massive pain, and creates a risk when new needs are identified that are not well served by the existing structure.
Agile is a way for non-technical managers to exercise arbitrary power over engineers. Your work is turned into a factory-style production line and you're reduced to playing a game of musical chairs over yellow stickys with other engineers at the daily standup. That the rules are "fuzzy" just underlines the fact that the exercise of power is arbitrary.
These fuckers even have a joke about how a bus hits an engineer and no harm is done to the project. How inappropriate is it to make jokes about a bus hitting your colleagues.
To be fair, I'm pretty sure the term "bus factor" (also known as "lottery factor" for the less morbid version) was invented by developers. I use that term all the time, and think a lot about what would happen if I were "hit by a bus". Maybe grim humor is not your style, but I'd recommend you look past that because that complaint is detracting from your otherwise valid points here.
For a senior, a great way to reduce the bus factor is to periodically stop all work in things other people can do. (Catch up on your reading, or write that cool code transformer you can’t stop thinking about or whatever). Hep when asked but very passively “why don’t you share your screen and we can try to debug it” instead of “try again, it should work now.” It might go really well or might show what areas you need to grow the team capabilities, but either way you get to know how great the code transformer idea really was.
>Agile is a way for non-technical managers to exercise arbitrary power over engineers
Agile was explicitly invented by engineers for engineers, just look at the history of the original manifesto.
One big problem is that it was built for self-managing teams composed only of engineers and established businesses are not built like that at all. There are non-technical people which are necessary and have legitimate business needs. Some of these are managers with power, so of course they're going to (ab)use the process to get what they need.
You can either change the structure of the business itself to be more Agile-like (good luck with that in an established company, especially a non-tech one), or adopt a more realistic posture which invariably involve concessions and deviations from the Agile model, but ends up better for all involved.
Process (agile or not) always has three goals: legibility, scalability, and fungibility.
Both management and devs alike can be concerned about the bus factor, which falls under fungibility.
Of course, an organization can have (or lack) humanity beyond those three goals. And hire people who have (or lack) the requisite humanity to bring to their role. If someone treats the bus factor or general fungibility casually, that's a problem.
>Agile is a way for non-technical managers to exercise arbitrary power over engineers.
I mean, that's because managers do have arbitrary power over engineers in most places. No project planning approach will ever make that go away if it's true in your org. Blaming agile for the inherent broken power dynamic of your company just makes it harder to fix the underlying issues.
I’m a developer and I often see the opposite: a manager has a good mental map of everything, and developers develop separate chunks of functionality. Unless you’re a software architect or a very senior dev, like a team lead, you won’t see the big picture.
As a person who has worked on factory-style production line, it is not as bad as you guys make it out! Manufacturing is a unique beast and tons of new problems to solve. To be fair, I worked as a manufacturing engineer and not as a production line worker. Is it that bad to work on software tickets? I suspect there is a good variety of problems to solve?
Honestly the best project management I've seen was a well maintained document listing high level projects, with named owners. Every week or two it got copied up and trimmed by the tech lead, and we would talk about it for 20-30 minutes.
For junior people, we'd pair them with a more senior person to help break down work but other than that there was very little overhead.
I’ve used strict versions of agile, as well as a dozen quasi-versions. They were all horrible. Both for managers and ICs.
It seems what works the best, as least for B2B SaaS, is driven more by building a feature that works, instead of trying to hit a date, or stripping the feature down to nothing in order to hit that date.
Sometimes you have to hit a date, sometimes you don't.
Sometimes having something is better than nothing, other times unless you have all the functionality it's worthless.
Sometimes you can know all the requirements upfront, other times they are ambiguous and require testing and failing.
The only way to be wrong is to pick one methodology and apply it to all circumstances. There are times pure agile will work, there are times waterfall will work, and there are times that neither of these are right and you have to do something else.
Fair enough, but in every instance I’ve encountered agile it’s taken on the methodology of stripping away - “ex: well we can’t build XYZ by date, so we should just aim for X”. Meanwhile, X is essentially useless without Y and Z.
I also think setting dates makes things go slower. We don’t use delivery dates at my company. Instead, we have quarterly cycles. Whatever gets done in that cycle gets done. We only aim to deliver one big feature per quarter, but it ends up being 2-3 because we’re so conservative. This also takes the pressure off of releases.
Agile these days is more like a particular sort of religious cult, where's there's no way to disprove the central mantras and there's this eeeevil mythical[0] enemy the cult is meant to overcome. The solution is always more of the cult's mantras. That said, if some company succeeds with an officially non-Agile process, they are still being Agile since they chose the process suited for them. So Agile is always right.
The good in Agile:
* Think about your processes and your progress.
* Inefficient processes should be culled.
* Overmanagement is bad.
The bad in Agile:
* Complete ignorance of the legitimate needs of everyone in business who is not a professional software engineer (most of the Agile mantras are actively harmful here, and are discreetly walked back as in this entry).
* Iteration is not a value in itself.
* Programmers should be managed, at least so long as the typical corporation has the structure it has, and Agile doesn't provide good tools here (at least none that programmers like...)
> Doing retrospectives once in a while to slow down and look back. Managers and leaders should talk the least in such retros. Encouraging feedback sessions within team.
And act upon them. I have sat through so many retros which turned out to mostly be about writing a report. Now, retros are something I try to get done as quickly as possible.
Every time I join a new team, I get my hopes up for retrospectives. I contribute ideas about how we can improve how we work for the first few months.
Then I realize that my suggestions are ignored and "action items" never have follow up. I revert to contributing nearly nothing. I usually come up with a "what we did better" item so that I don't appear completely disengaged from the process.
Agile, or at least in the organisations I’ve worked at, seems to be detached form any sense of reality and practiced as a sort of religious dogma - according to the Agile coaches, all our problems are caused by the fact we’re just not Agiling hard enough. We just need to expunge our minds of all impure waterfall thinking and reach the next level of Agile holiness and all our troubles will disappear.
There’s also a big push for Lean UX - basically the UX design process squeezed into the dev team’s schedule. Sounds great, but never works well. It’s like trying to complete a jigsaw puzzle a piece at a time without being able to see the picture on the box.
I just feel exhausted with scrum. Everything feels like yet another status report and "Why isn't it done yet?" I think we all may have bit into agile as process that revolutionized process because we have this natural tendency to value process and think that is getting us closer to if not the product.
Agile is about measuring and tracking, full stop. The difference between Waterfall and Agile is that Agile promises short iterations and continuous measuring and tracking throughout. But the important bit is that everything you do as a developer is tracked, recorded, and KPIs extracted from it so the company can see if it's hitting its OKRs and if not, what can be done.
Yeah, I know, that's not Agile according to the Agile Manifesto. Forget the Manifesto. You think the CTO approved an Agile transformation because of a fucking manifesto? NUMBERS. How much money can we save and how much more profitable can we become by delivering exactly the software, and only the software, our users need with ruthless efficiency? This, and ONLY this, is Agile in the enterprise. Shove your manifestos. You have sprint commitments you need to fulfill. Get back to work.
The crazy thing is how much time I see spent talking about how "web developers" or "Electron" is ruining software when the elephant in the room is business, i.e. the market, choosing cost efficiency. Michael O'Church repeatedly yelled this from the rooftops but no one wanted to hear it.
I have a mildly different perspective, that is also shown in the quote about testing infrastructure that I think the author missed - it's not so much about instant gratification as it is identifying how to see the results.
Take the second suggestion, for example, where the author appears to imply that tech debt, test coverage, etc are "2nd rate tasks" yet the speaker in the quote addresses this very point about how Facebook learned the hardway to ignore such things.
Instead of shrugging shoulders and blaming instant gratification, how about taking some measures of the impact that tech debt is making? Are you seeing outages as a result? Are you wasting resources as a result? What kind of cost savings could you expect by fixing it? How many bugs have you released that could have been caught by better test coverage? How much time is wasted on testing that could be automated?
Those are killer KPIs to measure, and extremely satisfying, in my humble opinion.
While its valid to ask devleads to estimate, I want to reach into the codebase and get some metrics. Can you label the functions that are "techdebt"? When did they run ? etc
It takes a lot of time and effort to measure those things. Presenting those measurements also requires some political skill, since bugs/outages/resource waste can also be misattributed to individuals.
> “What about the defect? It is highest priority. And it is needed fast. So let’s fix it for the short term and add a tech debt to fix it permanently”.
> I must admit that I have committed this mistake as well.
Is OP implying that the defect is not the highest priority?
Betteridge's law of headlines. No. I've never seen this be a problem. You set a definition of done for every ticket and stick to it. QED. If you can't do that, then process isn't your problem. It's maturity.
The marshmallow experiment is too simplistic, but for millions of years we usually were better off eating both marshmallows. Now we have supermarkets with products full of sugar. You can consume an unhealthy amount of sugar eating a few fitness granola bars, half a liter of fruit juice, or a single bar of chocolate.
We are especially vulnerable to superstimulus (exaggerated signals), which is what makes McDonalds, TikTok or surgically augmented breasts so popular. More sugar, more salt, more silly faces, more breasts. Even if we don't prefer them our brain is drawn to them. They used to be important signals. Youtube video thumbnails almost always feature people making faces, TikTok is short videos of people making faces. It's a race at providing better superstimulus. A lot of our culture these days is a constant bombardment of stimulus and we are not especially adapted to it.
Couldn't agree more. if this trend continues, the only possible consequence is drugs becoming a form of art.
When external stimuli won't do it anymore people will seek to directly stimulate themselves with chemicals.
I think society would also find out that the risk benefit analysis of club drugs and LSD vs. social media and junk food is tilted heavily in favor of the former
yeah the marshmallow experiment is too simplistic. I was reading somewhere that rather than measuring self-discipline, the researchers just measured which kids listened to the adults / did what they were told.
Yup, as much as I like the idea behind the original analysis of the experiment, another likely insight is that it more effectively tested for socio-economic class.
For someone in a wealthy and stable environment, it is rational to wait 20min for the larger reward, as there is a high expectation that the promised event will occur as planned. However, in a poor and unstable environment, it is often more rational to take the bird-in-the-hand, as there is considerable uncertainty about whether the promised 2nd marshmallow will actually arrive.
The marshmallow experiment was originally done on kids. A kid isn't going to get fat eating two marshmallows instead of one in a trial that's run once, and they're not going to have the forethought to decide to eat less rather than more because obesity epidemic anyway.
Instant gratification is just one pitfall of not understanding Agile before you do it. The article has a lot of useful suggestions at the end.
A thing that gets lost in a lot of discussions about Agile, even the ones like this one that invoke the intent of the Manifesto for Agile Software Development, is that there are material advantages to Agile over the systems analysis style of project planning and management. It isn't just mindfulness over following a recipe.
Agile was needed because replanning with complete functional and implementation specs, and complete task estimates and coder assignments needed for a resource-leveled critical path analysis is way too onerous. That lead to not replanning as tasks, knowledge, and goals change, which is worse.
There are tools, like having an MVP milestone, doing decision tree analysis at key milestones, etc. that are compatible with Agile, and there are techniques, especially metrics, that are voodoo or just adornments on Agile-as-management-fad. Knowing why Agile was needed vs what was used before is key to understanding what is compatible, and what is an impediment to agility.
It is not just easier. Agile avoids assigning people to tasks until tasks are started in an iteration, or continuously as in kanban style Agile.
This means you give up that beautiful "network diagram" CPM schedule estimate with no overloaded resources. But because of the nature of most software development projects, it becomes a burden, or a lie, within weeks.
Yup. Avoiding the overhead of heavy project management practices is also great.
Of course, it becomes less great when you start getting dependencies between teams and 2 week sprint delays mean that it takes months to get something cycled through different teams.
My sister works as a project manager at one of the big UK banks. They are anything but agile--projects take months and have to be planned out to the smallest detail for legal and compliance reasons. Makes sense, as the smallest decision affects millions of people, and there are regulations to satisfy.
She complains she now has to do daily standups with other project managers. But she has no idea what they do, they have no idea what she does, and it's a matter of just talking "Yeah, doing the same thing I was doing yesterday".
But since there are like 20 managers in just her group, this takes an hour everyday.
And she's like--we're not in software, why are they forcing this scrum thing on us?
I tried to tell her about the Priesthood the Agile, the Saviours of Our Codes, bless them, the Agile Manifesto Priests, but decided it would be too much history, so I just nodded and said "meh".