Coding Horror

Vendors often pitch customization as a feature of their software:

In the end, customizations and enhancements to a software solution are nearly always needed. This allows the software to be tailored to your needs, allowing for greater success, either with users or in business processes. They shouldn't be considered "necessary evils," but rather a "project variable." As long as they are justified, understood, and developed by the right resources, a product's modification can close the gap between a good product, and one that fits your organization perfectly.

In my experience, software customization is as much a danger as a benefit. I am reminded of the Tar-Baby parable:

"How are you feeling this morning?" says Brer Rabbit, says he.

The Tar-Baby didn't say a thing.

"What is the matter with you then? Are you deaf?" says Brer Rabbit, says he. "Cause if you are, I can holler louder," says he.

The Tar-Baby stayed still.

"You're stuck-up, that's what's wrong with you. You think you're too good to talk to me," says Brer Rabbit, says he. "And I'm going to cure you, that's what I'm going to do," says he.

Tar-Baby didn't say a word.

"I'm going to teach you how to talk to respectable folks if it's my last act," says Brer Rabbit, says he. "If you don't take off that hat and say howdy, I'm going to bust you wide open," says he.

Tar-Baby stayed still.

Brer Rabbit kept on asking her why she wouldn't talk and the Tar-Baby kept on saying nothing until Brer Rabbit finally drew back his fist, he did, and blip--he hit the Tar-Baby on the jaw. But his fist stuck and he couldn't pull it loose. The tar held him. But Tar-Baby, she stayed still.

"If you don't let me loose, I'm going to hit you again," says Brer Rabbit, says he, and with that he drew back his other fist and blap--he hit the Tar-Baby with the other hand and that one stuck fast too.

Tar-Baby she stayed still.

"Turn me loose, before I kick the natural stuffing out of you," says Brer Rabbit, says he, but the Tar-Baby just sat there.

She just held on and then Brer Rabbit jumped her with both his feet. Then Brer Rabbit yelled out that if that Tar-Baby didn't turn him loose, he was going to butt her crank-sided. Then he butted her and his head got stuck.

Brer Fox walked out from behind the bushes and strolled over to Brer Rabbit, looking as innocent as a mockingbird. "Well, I expect I got you this time, Brer Rabbit," says he. "Maybe I don't, but I expect I do. You've been around here sassing after me a mighty long time, but now it's the end. And then you're always getting into something that's none of your business," says Brer Fox, says he. "Who asked you to come and strike up a conversation with this Tar-Baby? And who stuck you up the way you are? Nobody in the round world. You just jammed yourself into that Tar-Baby without waiting for an invitation," says Brer Fox, says he.

It's tempting to view customization as the solution to any software limitations you encounter. If the software isn't doing exactly what you want, roll up your sleeves and mold the software into a better solution. This is usually done with the vendor's blessing; it's designed to be "extensible", after all. But like the tar-baby, extensive software customizations can trap you.

Where do you draw the line between customization, extensibility, and full-bore programming environment? I've participated in several projects where extensive customization of a third-party software package precluded us from upgrading to newer versions, or even switching to a competitor. Extracting yourself from a particular software choice is difficult even in the best of circumstances. But once you've performed extensive software customizations, extracting yourself from that software becomes nearly impossible.

And that's why, the next time a vendor sells you on customizations, you should consider the parable of the Tar-Baby before you're stuck in it.

I read Robert X Cringley's book Accidental Empires shortly after it was published in 1992. It's a gripping worm's eye view of Silicon Valley's formative years. It's also Doc Searls' favorite book about the computer industry. Highly recommended.

I didn't realize that the book was later expanded and made into a three-hour PBS documentary in 1995, Triumph of the Nerds. I rented the movie on Netflix and it's a fantastic companion to the book. The time capsule interviews alone make it worth watching: Steve Jobs, Steve Ballmer, Bill Gates, Larry Ellison, and many other key luminaries.

Watching the documentary brought on waves of nostalgia. I've always felt like the home computer and I grew up together. I was born in 1970. In 1971, Bill Gates and Paul Allen formed Traf-O-Data, and Steve Jobs and Steve Wozniak started selling blue boxes in southern California. In 1975, the first "personal" computer, the Altair 8800, was introduced.

It's difficult to get excited about a machine with no display and a row of dip-switches for input. Only two years later, in 1977, the first modern personal computer was introduced: the Apple II.

The Altair is barely recognizable as a personal computer, even though it is technically the first one. And yet the Apple II, a mere two years later, is the archetypal personal computer. Every modern home computer released after 1977 followed the template that Apple established for the industry: molded plastic case, expansion slots, CRT display, integrated keyboard, and floppy disk drive. Apple, along with the first killer app for the personal computer, VisiCalc, dominated the home computer industry until 1981. That's when the IBM PC hit the market-- and the clones followed.

Although I had access to Apple II computers in middle school, we couldn't afford an Apple until 1984, when we brought home the Apple //c. I never quite made the transition to the Macintosh, which was even more expensive. Still, many of my formative programming experiences were in AppleBASIC.

Have personal computers grown up since the early seventies? Sure. They've been around for more than 30 years now. But reading Accidental Empires and watching Triumph of the Nerds, I realized that computers still have a long way to go before they're fully grown up. And so do I.

The first video game to introduce a high score table was Asteroids, and after that they were a key fixture in virtually every arcade game from the 80's and 90's. One of my favorite high score tables was in Gaplus, the little known sequel to the mega-popular Galaga, which was itself the sequel to Galaxian.

I thought the inclusion of blood type in the high score table for Gaplus was a clever commentary on the meaninglessness of high score tables by the game's developers. As it turns out, it's just a Japanese eccentricity:

If this is a Japanese game, this would make some sense. In Japan, blood type is matched with personality, much like horoscopes here in the US. It's not uncommon to see blood type given for different characters in a game or comic, along with sex, age, etc. It's an important vital statistic that gives more insight to a person!

We may not play arcade games any more, but we still have our high score tables.

Technorati maintains a ranking of the top 100 blogs based on their Technorati Rank:

We also have traffic metrics, which is what Alexa's top 100 websites is based on:

And of course, there's the most wizardly of all high scores, Google's PageRank:

What's your high score? And more importantly, what's your blood type?

Here's an excellent bit of halloween advice from Mike Gunderloy: go read some source code you wrote five years ago for a real scare.

It's a good idea to go occasionally back to the well and get a sense of your progress as a so-called professional software developer. My goal is to suck slightly less every year. What were you writing in October, 2001?

I was writing a lot of VBScript code at that time, in the form of Windows Script Host scripts and classic ASP pages. One of the few nice things about WSH was its modern (for the time) regex support, so I first discovered the joy of regular expressionism around this time. I was also beginning to develop a healthy dislike of XML, which has matured into a lifelong ennui. Not that plain text is any better, but angle brackets aren't a silver bullet, either.

Reading through some of my five year old code, it's difficult to tell my personal WTFs apart from the WTFs inherited due to limitations in the WSH languages and Classic ASP coding environments. It's only been five years, true, but I think the clean-room elegance of the .NET framework, and the vastly improved development environment offered by Visual Studio 200x, far outstrip my meager improvements as a developer in the same time frame. Did I make mistakes? Absolutely. But when the only tools you have to choose from are Response.Write and Server.CreateObject, it's hard to imagine what you could have done differently. It's like trying to choose between using an old shoe or a glass bottle to hammer nails. I'm just lucky I still have the use of both of my hands.

Perhaps, in hindsight, this is an argument in support of learning alternative development environments. In 2001, I knew I was wearing blinders-- and I also knew .NET was right around the corner. But what's around the next corner? How will I look back on today's code five years from now?

I recently got into a spirited discussion about Akismet. What is Akismet?

When a new comment, trackback, or pingback comes to your blog it is submitted to the Akismet web service which runs hundreds of tests on the comment and returns a thumbs up or thumbs down.

Akismet is awfully coy about the "tests" they run to distinguish between spam and everything else. I believe Akismet is essentially the same as the old mt-blacklist plugin I use to block trackback spam. But instead of manually entering blacklist terms, Akismet harnesses the collective knowledge of the intarwebs. As soon as one person blacklists something, it's blacklisted for the entire Akismet community. And it definitely works. It's so effective that some people use it as their only protection against spam comments and trackbacks. I think this is very unwise.

First of all, blacklists aren't a panacea. They have their pros and cons. Just ask Matt Mullenweg, the author of Akismet. He recently left this comment on a blog post:

Unfortunately, the DNS realtime blacklists cause an unusually high false positive rate, which is why we don't use them anymore.

Interesting. And if you're going to keep a blacklist, you might as well keep a greylist and whitelist, too:

These three lists have been around as long as spam itself:

   Items on the Blacklist are never allowed through. They are either held in a moderation queue, or deleted.

   Items on the Whitelist are always allowed through.

   Items on the Greylist are held for human moderation.

Akismet also offers a moderation queue, so it has aspects of a greylist as well. Instead of spending time maintaining a blacklist, you spend time staring down a greylist moderation queue. I'm not so sure that's an improvement. If you consider Akismet a success because you ignore the moderation queue entirely, have you really succeeded?

It's also quite possible to use whitelist attacks on blacklists, where spammers use innocent and legitimate URLs in their spam. I've had a few of these myself. Even if you don't have a whitelist, attacks like this greatly reduce the effectiveness of a blacklist-- legitimate domains end up blacklisted through collateral damage.

But let's forget, for a moment, all the problems I just described with blacklists, whitelists, and greylists. The core problem is relying on a single method of defense against spam. Relying only on Akismet means:

1. You've added an external dependency to your website. I hate dependencies, and I always strive to keep the number of dependencies I accept to an absolute minimum.
2. If Akismet goes down, you either get inundated by spam while the floodgates are open, or nobody can comment/trackback. Neither scenario is desirable.
3. I get 75 spam trackbacks per hour on this blog. Multiply that the number of blogs on the internet, and you get an astronomically large number. Why should Akismet have to check every single one of those? Does Akismet have the capacity to scale that large? And is it reasonable to expect them to?

I can understand making the choice to use Akismet exclusively for trackbacks, where our options for combating spam are severely limited. But for comments, abandoning CAPTCHA in favor of Akismet is unforgivable. Engtech explains some of the problems with this approach in a recent comment:

[Akismet] has been pretty effective, but there's been a few interesting cases:

• compliment spam ("great post!" with website field linking to their p-rn/adsense splog site)
• only attacking blogs that appear to still have the default post as the first post -- less likely to monitor spam.
• one p-rn spammer who finds political/pop culture keywords in a post and inserts human crafted messages. Like: "Some people say Matt Damon isn't that good of an actor, I really liked him in Talented Mr. Ripley" whenever it finds a post with "Matt Damon"

The one thing it has absolutely sucked at is spammers-to-be. People who are just testing out spam generation algorithms that have no payload. So you'll get random gibberish from an IP address and it will take a few days for Akismet to learn.

Hearing this pains me greatly. All the of the above could have been completely eliminated by using both methods: CAPTCHA to validate that it's a human, then Akismet to validate that it's not human-entered spam.

Akismet is a fine addition to our anti-spamming toolkit. But that doesn't mean it's a good idea to outsource your entire anti-spam effort to a single website, either. Anti-spam security starts at home. For best results, use defense in depth and combine local anti-spam measures, such as CAPTCHA, with Akismet as a backup.