The waterfall which makes Agile pundits go blind

DISCLAIMER: this is a bitter post. If you get offended by this post, I'm sorry, though I had to write this. If you want to leave a comment, please do so, but as it's my blog, I'll remove comments which I think are inappropriate

The last month or so I've been on the 'alt.net' mailinglist which later became 'lists' as the original one, altnetconf, was renamed to cli_dev (for whatever reason) and some group started a new one, altdotnet. I can't say I had a good time. In fact it was more or less a pain in a lot of occasions. I don't mind a sharp debate, and we all can agree on everything, but from the start I've never felt to be accepted as a person who has a valuable opinion. At least not by some more hardcore Agile/XP/TDD people.

As you probably have understood, I left both of the lists and won't come back. Earlier this week I decided to leave the cli_dev list, after a painful thread about the usefullness of comments in code. It's an old debate, so from the start everyone participating would know that it would likely last forever and the basic dumb arguments would be placed on the table. What made it so painful was that it never became a discussion between professionals. There wasn't any serious debate which could all bring us forward. Instead it ended with bickering why some of the posted examples (one of them was mine) were bad, and more importantly: they were used to bash the code and with that the programmer. I've spend almost two decades in newsgroups and mailinglists now so these kind of debates aren't new to me, but as I'm not a teenager anymore (far from that ), I started to wonder why on earth was this even happening.

The point of such a mailinglist falls apart if debates aren't used to get things forward, to express ideas and learn from other people's arguments why they've chosen their point of view. A lot of the discussions weren't about debating arguments, but hammering down opponents, as if there were achievements to gain from doing so. The cli_dev list particulary was a big pain because of the endless debates about what 'Alt.net' meant, why or why not there should be a manifesto, who had the right to call the shots of what would be included in such a manifesto etc. etc. I think the comment debate was the last push I needed to call it a day on that list.

The other list, altdotnet, was OK for a while, however it turned out that the group of people participating in the discussions actually started to look like the same group who did the debates on cli_dev. The end result was that the discussions became more and more painful. The reason was that you had to be very careful with your formulation to avoid awaking the TDD/Agile people who simply want to b*tch about anything that's not TDD/Agile.

This morning I made a mistake: I used the W word in a post. What's the W word? The W word is Waterfall. The W word makes Agile pundits go blind, at least for a couple of minutes. It will make them go blind for any words you might have written after using the W word. What did I do? I used the W word in a short silly overexxagerated-to-get-it-across-example:

Every movement/pressure to go into a given direction in general causes a reaction among people who disagree: they will try to go into a different direction. Waterfall, while being great for systems which must not fail like your MRI scanner or wafersteppermachine or your satellite robot, it sucks when you have to deal with clients who change their minds 10 times a day and who sold their tiny brain at e-bay last year.

A silly example, but I made an error: I stated that Waterfall was great for a situation. I meant it as an example of how one movement can cause the creation of a counter-movement. This is sociology 101. All of a sudden, I was back in the seat meant for the Waterfall lovers and I found myself defending Waterfall, something I don't want to do, because I don't like the methodology that much (read: I in general don't see a lot of cases where it can be successful)

I don't know why some people keep on pushing the 'Bouma likes Waterfall' message, while it's not true. Perhaps it serves their agenda. This latest 'debate' this morning made me realize: "What on earth am I doing here? Why am I writing posts on a mailinglists which has people who like to offend me?" and decided to leave the altdotnet mailinglist as well. It was a sad ending for me, as I had great expectations when I joined the lists a month ago.

I did learn something though. What surprised me to no end was the total lack of any reference/debate about computer science research, papers etc. except perhaps pre/post conditions but only in the form of spec#, not in the form of CS research. Almost all the debates focused on tools and their direct techniques, not the computer science behind them. In general asking 'Why' wasn't answered with: "research has shown that..." but with replies which were pointing at techniques, tools and patterns, not the reasoning behind these tools, techniques and patterns. Answering Why with pointing to techniques, tools and patterns is creating a cyclic debate: the Why question is asked to understand the reasoning why some tools/techniques/patterns/practises are used. Pointing back at tools/techniques/patterns/practises isn't going to make you any wiser, as you then only learn tricks, because you can't put any argument on the table why you use pattern X, use technique Y and practise Z.

There's nothing wrong with applying tool T or pattern P to solve a problem you're facing. However if someone asks you why T or P are used, you should be able to answer that question with solid arguments, and not with "T was shiny, open source and l33t and P is used by everyone else in the group".

For example: if you say "You should use the principle of Separation of Concerns (SoC)", what exactly do you mean? It's not as obvious as it sounds. Please read the paper N Degrees of Separation: Multi-Dimensional Separation of Concerns by Tarr, Ossher, Harrison. It's an example how some 'technique' isn't as general usable as you might think: one has to think it through.

That kind of debates weren't on the alt.net mailinglists, they stalled on the level of 'thou shall use SoC", but what it meant exactly wasn't really worked out, as it was apparently assumed that everyone would know what it means. Referring to the paper didn't help, no-one picked it up. This is just an example of what I noticed.

I do think that it's important. I'm not totally finished with how I would call it, but the process looks like some years ago a separate software engineering world has been created (not only on .net, it's much wider) which isn't connected with the computer science world, but instead is focused inwards, looking for answers in the own world instead of in the computer science world: ideas aren't driven by science and research, ideas are driven by what you can do with a tool, with a technique, a pattern. The consequence is that the result of working on that idea has its root in the tool, technique or pattern, not in fundamental research in computer science. This result causes other ideas, which causes other results etc. etc.

Is this bad? I don't know, but if these two worlds drift apart, and the more I think about it, the more this is going on already, the consequences could be severe: the world in which research is taking place isn't feeding the world which applies techniques/tools/patterns with fundamentals, that world of techniques/tools/patterns is feeding themselves with 'fundamentals', which is actually cyclic reasoning: the fundamentals aren't fundamentals, they were the results of applying fundamentals, which doesn't make them fundamental per se.

With this insight, I also understood all of a sudden why there is even an 'alt.net': it's simply a movement which wants to use a different set of techniques/patterns/tools/practises than Microsoft prescribes to its customers via its product catalog. Leaving the lists behind with that in the back of my head wasn't such a loss after all: it's the science which counts, not the technique/tool/pattern/practise. Always keep asking Why, and search for fundamental answers. Only then you'll gain wisdom, instead of just knowledge.

Published Friday, January 11, 2008 5:13 PM by FransBouma

Comments

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 11:58 AM by Jimmy Bogard

With any mailing list, you're rolling the dice on whether you'll get a meaningful discussion on any given post.  cli_dev and altdotnet are no different.  The conversations are about as valuable as the medium lets them be.

I wouldn't be bitter though, your expectations seem a little too high.  Participate when the conversation is interesting, move on when it transforms in to troll-land.  Even Joe felt bad that he started the conversation, given where it ended up.

I still like the idea of "Pragmatic.NET", choosing the best solution given the context and the problem.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 12:30 PM by Ollie Riches

Totally agree with you Frans.

As you have eluded too alot of the discussions and view points stated aren't based on any logic, science or engineering prinicple.

the dicussions to me are rather 'tabloid' newspaper esque...

Keep up the fight :)

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 12:35 PM by Chris Brandsma

I'm with you on this, I've debated (with myself) leaving the group as well.  Some of the members are far to combative, downright vulgar, and insulting (one in particular for those last two -- and it isn't Scott Belware, who I think has also left the group or is silent all of a sudden).

I guess what I was hoping for would be more of a moderated list.  Far to many topics are thrown in that very few care about, then get side tracked into oblivion.

Unfortunately the group has grown too large to be useful any more.  A smaller group of 10-20 would be vastly more interesting.

BTW: I agree with your Waterfall/Agile statements.  Each has its own place.  

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 12:39 PM by David Hayes

This is why I don't bother much with mailing lists, Slashdot suffers from a similar problem ("Windows" rather than "Waterfall" though). I firmly believe that picking the correct tool for the job is critical in IT. There really is no place for these kind of 'religious' arguements based solely on faith/bigotry and little reasoned argument amoungst experienced professionals.

IT is very fad based, picking the right tools and methodologies for the job is critical to project sucess, it's a shame too few people seem to realise this and jump on the latest bandwagon because it seems cool.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 12:50 PM by Eber Irigoyen

and back to the religious debate about what being "Agile" means, which is the very reason that most developers won't ever even try, because of the people "inside" (a people problem), seems to me they try to hard to become "enterprise", to become what the Java community is, and I don't know that I want to go there

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 12:55 PM by Chris

I'm with you. Glad I didn't join the list like I had planned.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 1:39 PM by Chad Myers

Frans, you talk about science and proof, and then you make claims like "Waterfall, while being great for systems which must not fail like your MRI."

I don't accept that MRI systems were built with Waterfall, or that, if they were STARTED with Waterfall, that Waterfall was successful in carrying through to the end.

I defy you or anyone to show me a Waterfall project post-1970's (i.e. non-Mainframe) that was successful using Waterfall TO THE VERY END.

I have seen many projects that used Waterfall at the beginning and the middle, but which eventually disintegrate at the end into chaos and the only thing that makes the project successful is highly motivated, intelligent people doing what it takes to make the project successful IN SPITE OF the process that failed them.

All Agile is is recognizing that you can't plan everything up front like Waterfall advocates. Agile works from the premise that the scope of the problem is unknowable up front and the only way to know it is to work towards it and adapt as you go.

Waterfall, whatever you make think it is, in practice, typically works from the premise that you can define everything up front and even put it on paper and hand it off to someone else.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 2:02 PM by Brad Mead

Frans, as I said there at one point I really enjoyed your participation. Good debate informs many less-informed spectators and may even spur us to asking a few of the questions we are afraid (MORTified) to ask about CS fundamentals and where we can "fundamentally" improve our approach to software design and implementation.

A lot of the terminology/tools/techniques today are cloaked in mystery and somewhat removed from their origins. To try and approach or discuss these novel subjects without the informing experience of the basics levaves many of us feeling "uncool" and frankly stupid or inept. But that shouldn't be the case; IMO it should be more about taking a couple steps back from it all and "restarting" the learning process at a place where more of the underpinnings of the technology are better understood and internalized.

Sadly, I think a great opportunity is missed when a mentor on a list is unable, for a variety of reasons, to get an attentive audience to follow a rational line of thought through a basic concept and see how more complex structures inherit and build upon that concept.

Alt.Net pursefight, while amusing commentary, begins to shift through it all and get right to the point of many of the better threads. There are several on lists of whom I would want to ask MY dumb questions and expose my ignorance; and my desire to do so is based on a trust that their replies, while potentially harsh, would be well-informed and contain at least a smidge of empathy for my naivete. But to be honest I think what I am really looking for is a CS professor or kindly graduate student.  

I picked up on your CS-fundamentals outlook and it hit right at the heart of where I feel I am really lacking (no CS collegiate study - very little anyway). I'm actually thinking about going back for a CS degree where I can (re)expose, indeed immerse, myself in the basics (computational alogorithims, statistics, discrete math, etc).

Take care and hope to see you on list somewhere, someday again.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 2:27 PM by Thomas Wagner

In the US there is a saying that I am reminded of when reading your post: " Lay down with dogs - get up with fleas"

Tt does seem that our industry has a greater than average share of buffoons. Of course I live in LA which is filled with Hollywood buffoons and has a greater than normal share of huge ego's.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 4:11 PM by Sahil Malik

Frans -

I completely agree with you on this. I have had the displeasure of being in the same situation, and you are absolutely right, you mention a contrarian viewpoint, especially the dirty "W" word, and man they suddenly transform into terrorists who cannot be reasoned with.

Anyway, you made the right decision by dumping that time-drag on your life with possibly no good outcome. I did it a while ago, and have never regretted it.

Sahil

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 6:06 PM by Chad Myers

I don't expect you to defend Waterfall. I know you're not ADVOCATING it, you're merely saying it's not totally dead/worthless and I'm saying it is.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 6:16 PM by Jacques

And in the end everybody with a -slightly- different opinion gets boo'd away. What remains is a group of completely non-critical groupthinkers, that will swallow anything as long as its promoted by someone that has "authority" (big mouth and/or is really good at copying and summarizing information from others). Too sad really, because I've always found comments like yours way more valuable.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 8:48 PM by Darren Neimke

Agree, this is pretty much why I avoid mailing lists like the plague.  It's a sad indictment on our industry of 'professionals' really and probably the single thing that prevents us from being taken seriously by most other pro's.

Do you think that doctors, lawyers, and accountants speak to one another the way that 'tecchies' do?

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 9:07 PM by Walter Lounsbery

I think your post is insightful, not bitter.  I've had training in new manufacturing approaches (Total Quality, the Deming fallout) and various software project management approaches.  Great experience for watching the latest fads.

There is a lot of good stuff in the "radical" new approaches in software development.  But many of them were established like a religion and now must defend themselves like one.  In our world, you get crucified on the lists.

But as you did, we can opt out of the lists.  I'm more upset at the new Agile corporate culture.  At one company there are about 5 managers for each programmer, with various titles.  Basically, Agile handlers to keep projects on track and the customers happy.  That's a lot of staff for a more efficient process!  No matter how good the process, these heads are going to roll in a recession.

# re: The waterfall which makes Agile pundits go blind

Friday, January 11, 2008 11:20 PM by Glenn Wiley

Great post Frans.  What happened to you is a big reason why I've been turned off by a lot of the ALT.Net discussions.  Listening to someone say something along the lines of  "Waterfall sucks, Agile Rules!" reminds me of being in high school and listening to kids bash each other because they didn't like the same rock band - "Poison sucks!  Metallica rules!" Personally, I've used agile practices with success, but by no means do I consider it to be the only way to develop software successfully, and my opinion was formed based on my experience.  As you mentioned, I like the basic premise of exploring other practices and toolsets beyond what Microsoft gives us.  I use the best tools and methodology for the job at hand.  Maybe it's from Microsoft, maybe it's a third party tool, or maybe it's an open-source application.  The reaction you got from mentioning Waterfall isn't unlike the reaction I got when I recently left my web development gig and started doing SharePoint development.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 12:58 AM by Scott

Chad Myers.

"I defy you or anyone to show me a Waterfall project post-1970's (i.e. non-Mainframe) that was successful using Waterfall TO THE VERY END."

Windows NT

If you disagree, define "successful" :)

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 1:28 AM by Scott

You lasted longer on both lists than I did Frans. I have no problem sitting down with anyone on either list and having a cold beer, or warm Guiness, and having a long discussion about software development. But I didn't get a lot of value out of either mailing list. Both of them seemed to de-volve into people posting comments seeking acceptance.

I think a healthy debate is fine. I think a mailing list about alt.net is great. I'm so alt.net that sometimes I don't even USE Windows or .NET. ;) I think that if the members of the mailing list aren't able to disagree with each other, then the mailing list is doomed. At some point you have to accept that someone will have had a different experience than you. I could spend hours spelling out reasons why most Agile techniques won't work at my current position, and why cherry picking out techniques and putting them to work rather than rigorously practicing EVERY technique. But if people on the list can't accept that I have a different environment than they do, then we can't reasonably discuss anything. I've discussed my misgivings about TDD with Phil Haack and Brad Wilson before as well as others. Sometimes we just agree to disagree. I still have a high level of respect for them and their abilities.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 1:29 AM by silky

good decision. i haven't left either list yet, but i certainly have given up trying to talk to anyone anymore. i may stay on it for a week or two, in the hope that maybe an interesting topic may come past, but it's certainly become a useless place for any real logical/well-reasoned discussions.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 1:46 AM by Hernan Garica

Agree 100% with your post. The problem seems to be that people wants to be right all the time and can't even think about being wrong or reading and thinking. Who knows maybe they learn something different? I have been around for a while and I did work in projects way after 1990 actually fairly recent 1999 where we follow the waterfall methodology from beginning to end, big multinational company, headcount and accounting software that needed to work and couldn't fail, a team that spam every time zone.

It was painful and I'm sure that an Agile methodology may have worked as well, but at the end the main issue is a peoples issue.

I have worked in Agile teams that were a disaster as well, because the methodology wasn't follow as it should, because people didn't know what they were doing, whatever.

We should be more tolerant listen to everybody and learn from others, that's the way you can adopt and use something new (ex: Agile), try it  and if is better keep using it, or change it to make it work for you. The same thing with languages, IDE, tools, etc. Pick and choose what is right for you for the task at hand. And most important listen to everybody (specially people that have been around) you never know when you will learn something.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 4:02 AM by FransBouma

@Chad: Then we disagree on that. Fine by me. :)

The irony is, LLBLGen Pro is build with agile principles, and I use it every day. :) That's how weird this is.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 1:54 PM by Bill

@Chad Myers - Unless I'm reading your point wrong about transitioning to RuP, then it's a great note but doesn't dispute what Frans said.  That they did use Waterfall successfully.

Part of the improvement no doubt comes from experience and learning to implement things better. Without control groups and the like it's all speculation. But your challenge was to find anyone anywhere that oould show Waterfall worked from start to finish and unless you have something else, seems Frans made it his quite well.

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 1:58 PM by Bill

@Walter Lounsbery

If you diss Dr Deming again I'm going to get all sorts of Alt.net on you ;-)

# re: The waterfall which makes Agile pundits go blind

Saturday, January 12, 2008 2:10 PM by John Chapman

Sorry to hear you left the list Frans.  Your comments were probably the ones I most looked forward to reading.  You typically have more thought in your posts and raised great points when people seemed to be caught with tunnel vision.

I have pretty much lost interest in the mailing lits, but I still take a skim through them all to see if a gem is lurking in there.  Now there will probably be fewer gems in the list.

# re: The waterfall which makes Agile pundits go blind

Sunday, January 13, 2008 3:22 AM by Steve Wright

Does it really matter what method you use to program if the customer likes the end result?

# re: The waterfall which makes Agile pundits go blind

Sunday, January 13, 2008 10:24 AM by Grant

The core issue isn't really Waterfall or Agile, it's that there's no open-minded discourse. Too much religion disguised as pragmatism. I can find out what I need to about any SDLC approach read-only, I just don't need to see all the bitch-fighting that happens on the margins.

# re: The waterfall which makes Agile pundits go blind

Sunday, January 13, 2008 8:35 PM by John Rayner

> Alt.Net: a movement which wants to use a different set of

> techniques/patterns/tools/practises than Microsoft prescribes to its customers via its product catalog

Yeah. I agree. And I don't think that this is a bad thing. I like to reason things out from first principles and I don't like someone else (i.e. Microsoft) telling me how it should be. Sometimes I'll arrive at the same answer, but at least I'm applying some critical thought.

Sorry to see you leave the mailing list - I thought you often provided a great counterpoint to some of the groupthink. FWIW I thought you got overly flamed for it too.

Cheers,

John

# re: The waterfall which makes Agile pundits go blind

Monday, January 14, 2008 3:02 AM by Colin Jack

Its early in the morning but I agree with a lot of what you are saying but I still think its a pity you are leaving.

I personally just avoid the like of the comment thread, too long and too much bickering. As soon as dogma enters it gets messy too.

Can I also say that you are as guilty as any of us for the misunderstandings. We all read forum entries and interpret them incorrectly from time to time, but thats life.

# re: The waterfall which makes Agile pundits go blind

Tuesday, January 15, 2008 12:23 PM by Jeff Barnes

Frans,

I agree with you.

Initially, I thought ALT.NET was a great idea.  It was finally being publicized to use the tool/technology that is most appropriate for a given problem, which is something that I have always believed.  Unfortunately, the entire "movement" has seemingly regressed into a religious state of mind that is filled with zealots who antagonize anyone with an opposing perspective.  Spirited technical debates are a fundamental part of gaining insight into different ideas, but who wants/has time to try reasoning with close-minded individuals that refuse to carry on a rational conversation?  I sure don’t…that’s why I unsubscribed from the list.

# re: The waterfall which makes Agile pundits go blind

Tuesday, January 22, 2008 9:04 AM by Granville Barnett

I know I am late to this but incidentally a while ago I went onto the ALT.NET list to see what the hype was about and I saw you had contributed to a post on comments and I was shocked by the comments say that code comments were bad.

While I agree to a certain extent that code should be self descriptive in some scenarios this is just not possible and thus a comment from the programmer should be enforced as the code will be read many times.

I think I read another thread (again you were somewhere in there) which confirmed my worries and that was the slandering of CS theory and its practice or rahter how little of what people said had proof in the form of papers etc.

Those threads put me off the list.

The ALT.NET list read like whoever shouted the most won sort of thing - not for me.

# re: The waterfall which makes Agile pundits go blind

Wednesday, February 6, 2008 12:53 PM by robertwharvey

I have noticed of late that when I Google for a solution to a programming problem, I see many examples of "experts" who give a one-line solution to a problem posed by some script-kiddie, and then are miffed when the person does not understand their one-line solution or how to apply it.  Worse, the script-kiddie does not even know how to pose the question correctly in the first place (Here's my code, tell me why it doesn't work).

In the "old days" you could count on programmers doing the work to make themselves knowledgeable enough to ask productive questions.  Today, everyone thinks they're a programmer.