Coding Horror

When I started this blog, way back in the dark ages of 2004, the best of the options I had was Movable Type.

A Perl and MySQL based blogging platform may seem like an odd choice for a Windows-centric developer like me, but I felt it was the best of the available blog solutions at the time, and clearly ahead of the .NET blogging solutions.

Sure, I have areas of expertise that I like to stick to, but my attitude has always been to put religion aside and use what works, regardless of language or platform. That's much more of a reality today than it was five years ago. Today, we have embarrassing amounts of CPU power and memory in our servers, and a plethora of good virtualization solutions. Spinning up a Linux virtual machine to solve some problem is no big deal, and we do it every day on Stack Overflow.

In retrospect, my choice of Movable Type was a fortunate one. Although I also use and appreciate WordPress, it's a bit of a CPU hog. Given the viral highs and lows of my blogging career, there's no way this modest little server could have survived the onslaught of growth with WordPress. It would have been inexorably crushed under the weight of all those pageviews.

What's Movable Type's performance secret? For the longest time -- almost 5 years -- I used the version I started with, 2.66. That version of Movable Type writes each new blog entry out to disk as a single, static HTML file. In fact, every blog entry you see here is a physical HTML file, served up by IIS just like it would serve up any other HTML file sitting in a folder. It's lightning fast, and serving up hundreds of thousands of pageviews is no sweat. The one dynamic feature of the page, comments, are handled via a postback CGI which writes the page back to disk as each new comment is added. (This is also the source of the occasional comment disk write collision, when two commenters happen to leave a comment at the same time.) Yes, it's a little primitive, but it's also very much in the spirit of KISS: why not do the simplest possible thing that could work?

This static publishing mode precludes glitzy dynamic per-page widgets, but I am a minimalist who likes his pages austere. That restriction suits me fine. The other downside is that a site-wide change requires republishing hundreds or thousands of blog entries. Over time, that can get painful. Modern versions of Movable Type offer both static and dynamic publishing modes, which can give you the best of both worlds.

Movable Type was created by Six Apart. Over the last few years, I've had the opportunity to meet Anil Dash, who is not only the chief evangelist for and first employee of Six Apart, but also an old-school blogger from way back in 1999. This is a guy who has been through the intertubes a time or two. That's why I sought out Anil's advice when we were struggling to come up with a decent name for this crazy website concept Joel Spolsky and I were working on -- and it was his excellent advice on naming that eventually guided us to the name Stack Overflow.

Anil isn't just a brilliant blogger and community evangelist, he's quite influential in his own humble way. And despite his well earned status as a lion of the Web 1.0 blogging era, he's also willing to go far, far out of his way to help a fellow blogger. Anil personally helped me drag Coding Horror from the dark ages of 2004-era Movable Type 2.66 to today's modern Movable Type 4.2x. And by that I mean he logged in himself and did the grunt work to make it happen, including following up with me personally and going through at least two rounds of my crazy demands to make everything as primitive and featureless as I need it to be.

In short, Anil's a mensch.

So, if you're considering a blogging platform, I can vouch for not only the Movable Type software, but the Six Apart team, and the community around it. In all honesty, blogging changed my life. I'm not sure that's directly attributable to me choosing Movable Type, exactly, but I can give it the highest praise I give any software I've used:

It Just Works.

While I haven't been unhappy with Windows Vista, it had a lot of rough edges:

This is why the screenshot of the Windows 7 Calculator, although seemingly trivial, is so exciting to me. It's evidence that Microsoft is going to pay attention to the visible parts of the operating system this time around. I'm a fan of Vista, despite all the nerd rage on the topic, but I'll be the first to admit that Vista had all the polish of a particularly dull rock. Let's just say the overall user experience was.. uninspiring. This led many people to shrug, sigh "why bother?", and stick with crusty old XP.

Vista was like a solid B student who shows up at your doorstep reeking of body odor and dressed in shabby clothing from the local thrift shop. There's something decent at the core, but it's a real challenge to get past the obvious surface deficiencies.

Thus, I've been following the development of Windows 7 with cautious optimism. It's important to me not because I am an operating system fanboy, but mostly because I want the world to get the hell off Windows XP. A world where people regularly use 9 year old operating systems is not a healthy computing ecosystem. Nobody is forcing anyone to use Windows, of course, but given the fundamental inertia in most people's computing choices, the lack of a compelling Windows upgrade path is a dangerous thing.

Now that Windows 7 has reached its "release to manufacturing" milestone, I had the opportunity to install it for myself and see.

Within 5 minutes of installation it was immediately obvious to me -- Windows 7 is the best Vista Service Pack ever!

The core of the operating system isn't that different, but the experience is absolutely what Vista should have been on day one. Microsoft took that B student, gave him a bath and a makeover, and even improved his grades ever so slightly.

It sounds like a subtle thing, but it's not. Sit down and use Windows 7 for even a few minutes and you'll find an operating system that is faster, cleaner looking, and filled with lots of little useful, thoughtful touches utterly lacking in Vista. Where Vista was half-implemented and often clunky, Windows 7 is competent bordering on pleasant. I won't bore you with all the details, as Windows 7 has been getting lots of positive press from all corners of the web. There's no need for me to add my voice to the chorus. But suffice it to say that Windows 7 finally offers a compelling upgrade path from Windows XP. So from my perspective, mission accomplished. Three years late, but hey, who's counting.

(Note that this is not an invitation to rekindle the eternal OS flame war, as I'm much more interested in the cool stuff you're creating than what OS you use to create it with. I'm sorry, but screwdrivers just aren't that sexy to me.)

I normally do clean installs for operating system upgrades, but I've been busy recently, and I don't have any new PC hardware builds scheduled. If you're already on Vista, the upgrade path is perhaps more compelling than it otherwise would be. All the breaking fundamental changes were in Vista, so if you've made it over the Vista hump, then an in-place Windows 7 upgrade is relatively painless -- or at least, it has been for me on the two machines I've tried so far.

I think Windows 7 works well as a de-facto Vista service pack. I guess that's not surprising if you compare the version numbers.

C:UsersJeff>ver

Microsoft Windows [Version 6.0.6002]

C:UsersJeff>ver

Microsoft Windows [Version 6.1.7600]

Here's to exactly 0.1.1598 worth of improvement for the Windows ecosystem. Now can we please get the hell off Windows XP already?

A few months ago we bought a new digital camera, all the better to take pictures of our new spawned process. My wife, who was in charge of this purchase, dutifully unboxed the camera, installed the batteries, and began testing it out for the first time. Like so many electronic gadgets, it came bundled with a CD of software. So she innocently ejected the DVD tray, and dropped the CD in.

I happened to notice out of the corner of my eye that this was happening. At which point, I – now, try to imagine this in exaggerated slow motion, for full effect – screamed "noooooooooooo", and frantically launched myself across the room in a desperate attempt to keep that CD from launching and installing its payload of software. It worked, but I nearly took out a cat in the process.

There's nothing wrong with the software that comes bundled with a digital camera. Or is there?

1. It's probably unnecessary. Any modern operating system (and even Windows XP!) can see and automatically download pictures from a new digital camera. No extra software needed. But in a questionable attempt to add "value" and distinguish themselves from their many digital camera competitors, some executive at the camera company came up with a harebrained scheme to include software with a bunch of wacky, unique features that nobody else has.
   
   
2. Hardware companies don't generally do software well. Digital camera companies excel at building digital camera hardware. Software, if it exists at all, is an afterthought, a side effect, a checkbox on some marketing weasel's clipboard.
   
   
3. Software of unknown provenance is likely written by bad programmers. All other things being equal, the odds that new, random bit of software you're about to install will be pleasant, useful, and stress free are ... uh, low.

One of the (many) unfortunate side effects of choosing a career in software development is that, over time, you learn to hate software. I mean really hate it. With a passion. Take the angriest user you've ever met, multiply that by a thousand, and you still haven't come close to how we programmers feel about software. Nobody hates software more than software developers. Even now, writing about the stuff is making me physically angry.

Isn't that an odd attitude coming from people whose job it is to write software? How can we hate what we get paid to create every day?

David Parnas explained in an interview:

Q: What is the most often-overlooked risk in software engineering?

A: Incompetent programmers. There are estimates that the number of programmers needed in the U.S. exceeds 200,000. This is entirely misleading. It is not a quantity problem; we have a quality problem. One bad programmer can easily create two new jobs a year. Hiring more bad programmers will just increase our perceived need for them. If we had more good programmers, and could easily identify them, we would need fewer, not more.

How do I know, incontrovertibly, beyond the shadow of a doubt, that the world is full of incompetent programmers? Because I'm one of them!

We work at the sausage factory, so we know how this stuff is made. And it is not pretty. Most software is created by bad programmers like us (or worse!), which means that by definition, most software sucks. Let's refer to Scott Berkun's Why Software Sucks to nail down the definition:

When people say "this sucks" they mean one or more of the following:

• This doesn't do what I need
• I can't figure out how to do what I need
• This is unnecessarily frustrating and complex
• This breaks all the time
• It's so ugly I want to vomit just so I have something prettier to look at
• It doesn't map to my understanding of the universe
• I'm thinking about the tool, instead of my work

How many of those do you think would be true of the software on that CD bundled with the digital camera? I'm guessing all of them. That's why the best choice of software is often no software – and barring that, as little software as you can possibly get away with, and even then, only from the most reputable and reliable sources.

I don't look forward to installing new software. On the contrary, I dread it.

Let me share a recurring nightmare I have with you. In this dream, I'm sitting down in front of a computer which boots up, running an operating system I've written. I then launch a web browser I've created from scratch, all by myself, and navigate to a website I've constructed. I click on the first link and the whole thing bluescreens. And the bluescreen itself bluescreens and begins to fold in on itself, collapsing into a massive explosion that destroys an entire city block.

That's the good version of the dream. In the other one, there's just … screaming. And darkness.

In short, I hate software – most of all and especially my own – because I know how hard it is to get it right. It may sound strange, but it's a natural and healthy attitude for a software developer. It's a bond, a rite of passage that you'll find all competent programmers share.

In fact, I think you can tell a competent software developer from an incompetent one with a single interview question:

What's the worst code you've seen recently?

If their answer isn't immediately and without any hesitation these two words:

My own.

Then you should end the interview immediately. Sorry, pal. You don't hate software enough yet. Maybe in a few more years, if you keep at it.

I was utterly floored when I read this new IEEE article by Tom DeMarco (pdf). See if you can tell why.

My early metrics book, Controlling Software Projects: Management, Measurement, and Estimates [1986], played a role in the way many budding software engineers quantified work and planned their projects. In my reflective mood, I'm wondering, was its advice correct at the time, is it still relevant, and do I still believe that metrics are a must for any successful software development effort? My answers are no, no, and no.

I'm gradually coming to the conclusion that software engineering is an idea whose time has come and gone.

Software development is and always will be somewhat experimental. The actual software construction isn't necessarily experimental, but its conception is. And this is where our focus ought to be. It's where our focus always ought to have been.

If your head just exploded, don't be alarmed. Mine did too. To somewhat reduce the migraine headache you might now be experiencing from reading the above summary, I highly recommend scanning the entire two page article pdf.

Tom DeMarco is one of the most deeply respected authority figures in the software industry, having coauthored the brilliant and seminal Peopleware as well as many other near-classic software project management books like Waltzing With Bears. For a guy of Tom's caliber, experience, and influence to come out and just flat out say that Software Engineering is Dead …

… well, as Keanu Reeves once said, whoa.

That's kind of a big deal. It's scary.

And yet, it's also a release. It's as if a crushing weight has been lifted from my chest. I can publicly acknowledge what I've slowly, gradually realized over the last 5 to 10 years of my career as a software developer: what we do is craftsmanship, not engineering. And I can say this proudly, unashamedly, with nary a shred of self-doubt.

I think Joel Spolsky, my business partner, recently had a similar epiphany. He wrote about it in How Hard Could It Be?: The Unproven Path:

I have pretty deeply held ideas about how to develop software, but I mostly kept them to myself. That turned out to be a good thing, because as the organization took shape, nearly all these principles were abandoned.

As for what this all means, I'm still trying to figure that out. I abandoned seven long-held principles about business and software engineering, and nothing terrible happened. Have I been too cautious in the past? Perhaps I was willing to be a little reckless because this was just a side project for me and not my main business. The experience is certainly a useful reminder that it's OK to throw caution to the wind when you're building something completely new and have no idea where it's going to take you.

Yes, I could add a lot of defensive software engineering caveats here about the particulars of the software project you're working on: its type (mission critical, of course), its size (Google scale, naturally), the audience (millions of daily users, obviously), and so forth.

But I'm not going to do that.

What DeMarco seems to be saying – and, at least, what I am definitely saying – is that control is ultimately illusory on software development projects. If you want to move your project forward, the only reliable way to do that is to cultivate a deep sense of software craftsmanship and professionalism around it.

The guys and gals who show up every day eager to hone their craft, who are passionate about building stuff that matters to them, and perhaps in some small way, to the rest of the world – those are the people and projects that will ultimately succeed.

Everything else is just noise.

Are you familiar with the term "meta"? It permeates many concepts in programming, from metadata to the <meta> tag. But since we're on a blog, let's use blogging to explain what meta means. If you've read this blog for any length of time you've probably heard me rant about the evil of blogging about blogging, a.k.a. meta-blogging. As I said in Thirteen Blog Cliches:

I find meta-blogging -- blogging about blogging -- incredibly boring. I said as much in a recent interview on a site that's all about blogging (hence the title, Daily Blog Tips). I wasn't trying to offend or shock; I was just being honest. Sites that contain nothing but tips on how to blog more effectively bore me to tears.

If you accept the premise that most of your readers are not bloggers, then it's highly likely they won't be amused, entertained, or informed by a continual stream of blog entries on the art of blogging. Even if they're filled with extra bloggy goodness.

Meta-blogging is like masturbating. Everyone does it, and there's nothing wrong with it. But writers who regularly get out a little to explore other topics will be healthier, happier, and ultimately more interesting to be around-- regardless of audience.

Triple-meta alert! That blog entry was me blogging about blogging about blogging. See? Painful. I told you.

Generally speaking, I am not a fan of the meta. It's seductive in a way that is subtly but deeply dangerous. It's far easier to introspect and write about the process of, say .. blogging .. than it is to think up, research, and write about an interesting new topic on your blog. Meta-work becomes a reflex, a habit, an addiction, and ultimately a replacement for real productive work. It's something I think everyone should watch out for, whatever walk of life or career you happen to have. In fact, I've come up with a zingy little catch phrase to help people remind themselves, and their coworkers, how toxic this stuff can be -- meta is murder.

Yes, you read that right. Murder. I mean it. If enough productive work is replaced by navelgazing meta-work, then people will be killed. Or at least, the community will be.

Joel Spolsky had a great example of how meta-discussion can kill community in our latest podcast.

Let's say that you become a podcaster, so you get really interested in podcasting gear. You're going to buy some mixers, and want to know what kind of headphones to use, what kinds of microphones, when should I do the A/D conversions, all that kind of stuff.

So you find this awesome podcasting gear website. And you go on there, and the first subject of conversation is who's going to be elected to the podcasting gear website board of directors. And the second subject of conversation is whether the election that was done last year was orthodox, or was it slightly ... was there something suspicious about that whole thing. And you find a whole bunch of people arguing about that. And then you find a conversation about whether all the people who came in last year from South America and don't speak very good English should be allowed to hang around or should maybe be read-only users for the first six months.

That's all you find there, and you want to talk about mixers and mics. That's why you came to this site!

But they're bored talking about mixers and mics -- they've already had the full mixers and mics conversation all the way to the end, to its logical extreme. They all have, now, the perfect podcasting setup. Except for there's this one minor little thing about whether you should use Monster Cables that people still argue about.

So all they're talking about on this so-called "podcasting gear" website is the podcasting gear website itself.

If you don't control it, meta-discussion, like weeds run amok in a garden, will choke out a substantial part of the normal, natural growth of a healthy community.

The danger and peril of meta has been known for years. We had Josh Millard, a MetaFilter moderator, as a guest on the podcast last year. He described how quickly MetaFilter realized that meta-discussion, if not controlled, can destroy a community:

Millard: Matt set up MetaTalk sometime like 8 months after he started [MetaFilter], right about the beginning of 2000, because people were talking about MetaFilter on the front page. It's natural enough. People would say, hey what's with this, hey look at the post, hey this guy's a jerk. So he started up MetaTalk and directed stuff that was metacommentary to that part of the site. You could delete something and say, hey take it over there. If people wanted to have an extended argument that was derailing a thread, they could do it there.

A lot of people cite MetaTalk as a reason that MetaFilter works. If you talk to a regular from the site they'll tell you MetaTalk is key to the success of the site because it's a sort of release valve. Talk pages on Wikipedia are a similar thing. I had the same experience as you the first time I checked those out -- it's not necessarily comprehensible to the casual user what is going on there. But for the people who are regulars, the people who develop a certain amount of passionate attachment to the sites, or really, really need to make their voice heard out of day one beyond just normal participation, you have this safe place you can let people ... let their freak flag fly, as it were, without damaging the core function of the site. You don't have big messes on the front page.

So there's a pretty strong culture of regulars who hang out on MetaTalk. Insofar as you have the big contributors and the serious regulars at any given site that make up the core of the community, there's a strong correlation between those people and the people who actually spend time on MetaTalk dealing with policy stuff and talking about user issues.

Atwood: Right. I totally get that. This is one of the things about designing social software -- you don't really understand it until you've lived through it. For the longest time I couldn't understand why people couldn't respect the rule we had to not discuss this meta stuff on the site itself. I totally get this now.

We've dealt with our meta problem on Stack Overflow, finally. OK, I had to be dragged kicking and screaming to finally do what I should have done months ago, but what else is new?

Anyway, my point is that meta isn't just a social software problem. Meta is a social problem, period. It's applicable to everything you do in life.

Software developers are known for their introspection, and a certain amount of meta is healthy. It qualifies as sharpening the saw -- mindfulness of what you're doing, and how it can be improved. But it's amazing how rapidly that can devolve into a crutch, a sort of methadone for Getting Things Done^tm.

So sure, get meta when it makes sense to. But do be aware of what percentage of the time you're spending on meta. And consider: how is progress made in the world? By sitting around and debating the process of how things are done ad nauseam? Or by, y'know … doing stuff?

Allocate your time accordingly.