Development is Inherently Wicked

Jeff Atwood

02 Sep 2004 — 2 min read — Comments

Horst Rittel and Melvin Webber defined a “wicked” problem as one that could be clearly defined only by solving it, or by solving part of it. This paradox implies, essentially, that you have to “solve” the problem once in order to clearly define it and then solve it again to create a solution that works. This process has been motherhood and apple pie in software development for decades.** (Steve McConnell)

A computer industry adage is that Microsoft does not make a successful product until version 3. Its Windows operating system was not a big success until the third version was introduced in 1990 and, similarly, its Internet Explorer browsing software was lackluster until the third version. (Seattle Post-Intelligencer)

As far as I’m concerned, all software development can be classified as a Wicked Problem. It’s far too complex and far too annoyingly micro-complicated to allow for a whole lot of rational planning. I know from personal experience that I can never get very far without writing code to better understand the problem I am trying to solve.

I’m not proposing that we all revert to a “code like hell” methodology. But I think it’s incredibly foolish to believe any team of developers, however talented, can plan out an entire project from start to end, foreseeing all the contingencies, emergent problems, and weird-ass edge conditions they’re bound to run into. It’s thinking like this that leads to classic waterfall project catastrophes. Too much up-front planning is counterproductive and potentially disastrous.

Instead, I believe you have to continuously code throughout the lifecycle of a project and constantly integrate that development feedback into your planning. The sooner you’ve attempted to solve the problem, the sooner you will have a handle on the problem. I’d even go this far: if you’re not writing code for more than two days at a time, you are putting your project at risk. But you have to write the right kind of code at the right time:

In the beginning: you should be researching, prototyping, evaluating risky and unfamiliar areas. What alternative approaches can be used? What architectures make sense? What third party tools will work? If there are any software packages that perform a similar task, how do they approach this problem?
In the middle: you will figure out things that obsolete code you’ve already written. Don’t be afraid of this – embrace it. Break stuff. Rebuild it. Refactor it. This is your most productive development phase, as long as you don’t fall into the trap of treating existing code as sacrosanct. You should be first in line to obsolete your own code.
At the end: have the courage to start saying “no”. Even to yourself, which takes discipline. You have to finish, because you’re never really finished – each version is a stepping stone for the next version. Ship something and let users beat on it for a while. The quicker you get a release out, the quicker you can get that critical user feedback to fold into the next version.

There are a lot of different methodologies that cover the same ground. Some people call this Agile Development, XP, or SCRUM. All of these fancy buzzwords have a common ancestor: the classic book Wicked Problems, Righteous Solutions: A Catalog of Modern Engineering Paradigms.

What does Stack Overflow want to be when it grows up?

I sometimes get asked by regular people in the actual real world what it is that I do for a living, and here’s my 15 second answer: We built a sort of Wikipedia website for computer programmers to post questions and answers. It’s called Stack Overflow. As of

Civilized Discourse Construction Kit

Occasionally, startups will ask me for advice. That's a shame, because I am a terrible person to ask for advice. The conversation usually goes something like this: We'd love to get your expert advice on our thing. I probably don't use your thing. Even

How to Stop Sucking and Be Awesome Instead

I've been fortunate to have some measure of success in my life, primarily through this very blog over the last eight years, and in creating Stack Overflow and Stack Exchange over the last four years. With the birth of our twin girls, I've had a few

Books: Bits vs. Atoms

I adore words, but let's face it: books suck. More specifically, so many beautiful ideas have been helplessly trapped in physical made-of-atoms books for the last few centuries. How do books suck? Let me count the ways: * They are heavy. * They take up too much space. * They have