Monday 17 November 2008

Saying you're agile, doesn't make you agile.

So a lot of posts have come up in response to James Shore's article on Agile's decline and fall, and the implications of using "scrum alone, and misapplied." While I agree with James about the bulk of his content, there's a much simpler issue at work, and it's an issue that's been hanging around since the earliest days. It's about branding.

Agile is a brand

I can call myself anything. I can call my self "Christian" (that's my name). I can call myself Canadian. I can call myself a consultant. These are descriptive, verifiable, and helpful designations in understanding what I'm about. I can also call myself "agile." This, depending on how the label is understood, can be helpful or not. Do I mean that I'm physically agile? Do I mean that I'm flexible in my thinking? Do I mean that I observe a particular method of software development? Does that mean that I'm iterating? The term is, at that level, meant to be evocative, and therefore is a very easy to apply, but slightly meaningless brand.

Scrum is a stronger brand, because it has some licensing, and there's a "by the book" approach. Extreme Programming is also a stronger brand. You can measure the organization's application of practices and determine if they're really doing XP or Scrum (to a larger extent). In fact Agile (as a brand) has always suffered from this weakness, in that it can be applied so generically as to be unhelpful as a description. Because of this, Agile has failed a lot in its history, because anyone can do a little piece of agile, call themselves "Agile", and fail. "Agile" is then a nice scapegoat for the failure. Waterfall is in the similar unenviable position, but it was received wisdom in the software industry for so long no one really questioned that it was one of the culprits until alternatives were seriously proposed.

So is "Agile" failing?

At one point I worked with a client that implemented Scrum. Only they didn't. They let teams determine their own processes in many ways, except that they all iterated. But the infrastructure of Scrum™ wasn't there. But they did not deliver working software every iteration. They did not have a clearly defined product owner. They had ambiguity about what "the team" was, in fact. No scrum-master-alike role. Little criteria for acceptance was provided to the teams. Few teams were allowed to prioritize defects to the top above new functionality. Basically I'm listing off a set of Scrum assumptions/directives that were not followed. Yet this organization called what it was doing "Agile." This company did not deliver software for several years, and is now not "doing agile" as an organization. (Some of the teams are actually moving forward with stronger agile processes, but the outer shell of the organization is moving towards stronger "contract-oriented" delivery in large phases.)

So did Agile fail? Did they fail Agile? Most Scrum proponents would probably suggest (and we did) that they weren't even doing agile in the first place. In fact, they were (on some teams) starting to do some of the engineering practices. The teams were learning, but the organization wasn't. They held retrospectives and didn't apply the learning. They weren't fearless in their willingness to change and adapt to what they were discovering about their process. So in effect, they started to do Scrum-but, ended up doing 1/4 XP with iteration, and management did not accept what was obvious to the rank and file. I do not consider this an "agile failure" or a "failure of agile" because, frankly, Agile wasn't even given a fair shake.

I contend, based on my experience there, that had they implemented Scrum "by the book", it may well not have "saved" them. Their engineers, however, are extremely competent (when permitted) and some of their best immediately began to implement agile-supportive engineering practices. And as consultants we helped them with infrastructure issues like moving towards a more coherent infrastructure around continuous integration, separation of types of testing, etc. I think they could have succeeded, because the ingredients were there. Scrum was exposing problems, insofar as they were using its elements. But without applying the learning, Scrum is as useless as any other learning system. As Mishkin Berteig opines frequently in his courses, "Scrum is very hard."

Incidentally, In case you know my clients, don't be too hard on these guys. Management tried very hard, was undermined, interfered, and meant well. For the most part, agile implementations I've observed have been partial implementations largely doomed to produce less value less quickly than was otherwise possible.

Can "Agile" fail if you are using something else.

What James talks about is not "agile" or "scrum" failing, but a wolf in sheep's clothing. I'm not copping out. As I said, most of the implementations I've seen have been half-baked partial agile implementations, which, in retrospective analysis done for my clients, have failed in the obvious faults left where missing practices were ignored or seen as too risky or costly. These methods are systems which work together for reasons. Lean analysis can help, but it still is a learning-system closely aligned to scrum in mental-state (if not in brand), and it will merely uncover things that you have to fix.

If I call myself a Buddhist, but become a materialist freak... did Buddhism fail, or am I truly worthy of the label? If I label myself a Christian, but then fail to love my neighbour, is Christianity failing, or am I failing Christianity? If I call myself a Muslim, then violate the strictures of the Qur'an (by, say, killing innocents), is Islam violent, or am I doing an injustice to the name of the religion? I mean no disservice to any religion when I call them a "brand", but like any other label, you can think of them that way. If a brand is easy to apply, then it's easier to distort and pervert from its original intent.

If I take a bottle of Coca-cola, replace the contents with prune-juice and Perrier, then put it in a blind taste test with Pepsi, which subsequently wins... did Pepsi beat Coca-cola in a blind taste test? No. I subverted the brand, by applying it to something else entirely.

If I've made my point clear, the problem should be obvious. Agile is a weak brand, which can be misapplied. Therefore, in a sense, given that Agile was only a loose community of like-minded methods and system practitioners, maybe it's OK that Agile declines, as a brand. The problem is that such a decline will discourage people from applying the management processes and engineering practices that can encourage organizational learning and product quality.

If that happens, then the sky has fallen, because we weren't making high-quality software before Agile, and I would despair if we just gave up.

Not a fad for me

I'll keep teaching and coaching "agile" teams, encouraging them to implement the full range of agile practices (above and below the engineering line). Not because I'm a believer, but because I find these useful practices which, when applied in concert, powerfully enable teams to deliver high-quality software. And because I enjoy working this way. And because I deliver faster when I do them. That's the bottom line. Agile might be a fad, but I don't really care. I wasn't using these techniques just because they had become popular. Many of them I had used before I had ever heard of the terms Scrum, XP, or Agile. It's a bit like Object-Orientation, which I've been using since the NeXTSTEP days in the early 90's. At that point, it wasn't clear that O-O had "won" over structured programming, and it was not really a "fad" until the advent of Java. That didn't stop me from using a helpful paradigm that improved my quality, integrity, and productivity. And it shouldn't stop anyone else. will announce the new fad soon. Use it if its useful. Use Agile methods if you find them useful. Was O-O hard when I first was trying to do it? Sure! There are books on its pitfalls and how to screw it up. (And in fact I find little of it out there, despite the prevalence of O-O supportive languages.) But I still advocate for its use, and use it myself.

The quality, integrity, and delivery of your software is in your hands. Do what you gotta do.


BVP said...

Good Post :)


Kimota94 aka Matt aka AgileMan said...

Excellent post, Christian! I couldn't agree more with your assessment.

Perhaps what companies need, when they're considering the option of introducing Scrum, XP, or any Agile methodology (including a "roll your own" approach, as we did), is a hard-hitting, gut-check sort of checklist that asks the tough questions. I wouldn't presume to be able to rattle off the complete and polished contents of such a list right here, off the top of my head, but maybe it would contain items along the lines of:

1) Is the notion of having on-site customer representation, readily and capably available to your development teams, something that already happens or could easily be made to happen?

2) Are your customers, or someone empowered to act on their behalf, aligned with the goal of having incremental functionality delivered in short cycles?

3) Has your organization traditionally demonstrated a willingness to prioritize quality above the need to adhere to a previously-published project schedule?

4) Have your current leaders shown that they are comfortable with empowering those below them to make the right decisions?

5) Is it well accepted within your organization that what you've been doing to date (traditional Waterfall) hasn't been working?

6) Are you committed to adapting your processes and expectations based on data that you gather as you go along?

... and so on. Imagine a list of say 15 - 20 such questions, each framed such that a "Yes" answer indicates applicability for change (to something "Agile") and a "No" response speaks volumes about the opposite case being true. At the very least, it would provide a good "shock to the system" for anyone considering such a drastic move, and might make it more clear just what sort of environment is required for an organization to truly "be" Agile.

Certainly if the company I had worked for had taken a cold, hard look at what was going to be asked of them if they adopted Agile, they might - just might! - have realized that the ground wasn't fertile enough in a few key areas. Probably not, but at least it would've been worth a try!