Coding Horror

In the software industry, the release of newer, better versions is part of the natural order. It's a relentless march towards perfection that started with the first personal computers, and continues today. We expect software to get larger and more sophisticated over time, to track with the hardware improvements that Moore's law has provided us for so many years. Rapid evolution is a good thing, and it's one reason the computer industry is so exciting to work in. If you don't like the way things are today, just wait five years; everything will be different.

Letts' Law: All programs evolve until they can send email.

Zawinski's Law: Every program attempts to expand until it can read mail.

Furrygoat's Law: Every program attempts to expand until it can read RSS feeds.

I love the prospect of upgrading my favorite software. Done right, it's like watching a caterpillar shed its skin and become a beautiful butterfly. Or at least a decent-looking moth.

But for some software packages, something goes terribly, horribly wrong during the process of natural upgrade evolution. Instead of becoming better applications over time, they become worse. They end up more bloated, more slow, more complex, more painful to use.

They spoil.

I know this first hand because I'm a long-time Paint Shop Pro user. As a programmer who doesn't need the kitchen sink of graphics editor features, I found it an ideal match for my modest programmer needs. I didn't upgrade to every new version, but when I did, for every new feature I could actually use and benefit from, there were dozens of other features included that I didn't care about. These new features cluttered up the user interface and often interfered with what I wanted to do. My computers kept getting faster, and yet PSP kept taking longer and longer to start up with each new version.

If this spoilage goes on long enough, eventually you begin to loathe and fear the upgrade process. And that strikes me as profoundly sad, because it rips the heart out of the essential enjoyment of software engineering. We write software. If we inevitably end up making software worse, then why are we bothering? What are we doing wrong?

I'm not against progress by any means. But it sure seems to me that certain software packages have truly lost their way. In their never-ending quest to add feature bullets, they've somehow forgotten their users and their core values. In trying to be everything to everyone, they progressively destroy that tiny core of uniqueness that they started with. I'm singling out Paint Shop Pro here, but this same software spoilage principle applies to many other applications. PC World compiled an annotated list of 13 software applications they liked better before they were "improved":

• AIM
• ICQ
• Windows Live Messenger
• Windows Media Player
• iTunes
• QuickTime
• iMovie
• Paint Shop Pro
• ACDSee
• Adobe Acrobat Reader
• Eudora

They helpfully provide links to oldversion.com, oldapps.com, and old-versions.net, where you can go back in time and obtain those classic, unspoiled versions.

My favorite version is Winamp 2.95. That's before they started bulking up the client and adding completely unnecessary things. I just want something that plays my MP3s. I don't need it to burn CDs for me or download new music or cook my breakfast or massage my feet.

It's depressing to me that there are very few apps I can stick with for more than five years before they become an untenable, unbearable mess. I can think of so many that I've liked and since discarded: Nero Burning ROM, WinAmp, ACDSee, Microsoft Money, WinZip, and many others.

I suppose features sell software. For many companies, putting users on the version upgrade treadmill is their business model; it's how they generate revenue. But if this fiscally rewarding feature creep goes on long enough, spoilage inevitably sets in. So I wonder: Is all software destined to spoil over time? Is it possible for software packages with long histories to avoid the trap of becoming bloated and irrelevant? What are your favorite bits of software that haven't spoiled over the years-- and what is their secret?

I distinctly remember the tribulations my father went through in his career. He worked hard to achieve an MBA from a prestigious business school. The degree opened up many opportunities for him, but I don't think he ever found exactly what he was looking for. We moved throughout my childhood, travelling from job to job, never staying in one place for more than a year or so. I'm not sure he ever found work that satisfied him, even to this day. Copies of What Color is Your Parachute were staples in our household.

It can take a long time to figure out what you want out of your work life.

Like my Dad, I spent many years after college flitting from job to job. I had nothing to complain about. I was making a great living. I was never on the market for particularly long before some new opportunity would come up. I enjoyed my work. But I wasn't choosing a career path. I was letting happenstance determine what I was, and what I was becoming.

At some point in your career, you have to stop floating through life like the symbolic feather in Forrest Gump.

Unfortunately, I don't think my father ever figured out what he loved to do. He never determined the color of his parachute. But I got lucky. A few years ago I realized that what I loved to do, what I really loved to do more than anything else, was write software and fool around with computers. Seems obvious, I know, but you have the advantage of not being me. Self-awareness is a perplexingly difficult thing from here on the inside.

Life is too short to stay at a job where you're not doing the things you want to do, where you're not enjoying yourself. And yet here I was, a guy hopelessly in love with all things computer and software, working at a company where where software was considered a byproduct, a cost center, a necessary evil:

A close friend of mine works for a company that has experienced a mass exodus of developers. The best left first, the mid-range followed. What's left are the people who clock in 9 to 5 for the paycheck and don't take pride in what they're building. The company now has what they asked for: a team of low-level code jockeys. The people with initiative, energy, and passion have left.

Enterprises that consider developers "commodities and low level craftsmen" are doomed to have (at best) average developers working for them.

To be fair, it was post-bubble, and jobs were hard to come by. The work was interesting, but it was abundantly clear that software was not the lifeblood of this organization. Outsourcing was in the air. Although my coworkers were competent, nobody was quite as obsessed with the software as I was. My passion for software, and everything around it, was clearly not shared.

I set out to change that. Companies would no longer be able to select me from a generic lineup of candidates. Instead, I would select companies. Companies that I respected, companies that shared my passion for software. Armed with thirty years of hindsight, I would no longer let random, chance opportunities determine my career path. I will choose where I want to work.

In a recent article, Joel Spolsky describes the guiding philosophy behind his software company:

Frankly, the main reason I had to start this company was to have fun at work. Working at Fog Creek is intentionally designed to be pleasant. We started the business because we wanted a great place to work, to spend our daylight hours. And we have a disturbing tendency to try to do a lot of things ourselves, especially if it's going to be fun or if we think we can do a better job. It takes us a little longer that way, but I figure the journey is the reward.

That's exactly why I chose to work at Vertigo Software. We have the same philosophy. At Vertigo, I'm surrounded by incredibly talented software engineers who are all passionate about software. And dammit, we have fun.

If you love software as much as I do, you deserve to work at a company where people come to work not to punch a clock, but because they love software, too. You deserve to work at a company where software engineering is respected. You deserve to work at a company where peers meet to enjoy building software together.*

We're in the middle of a huge tech boom; some might even call it another bubble. Opportunities abound.

Choose wisely. And remember, this stuff is supposed to be fun.

* did I mention that we're hiring?

uTorrent is my favorite torrent client. It's such a joy to use – a tiny, native application that offers a best-of-breed implementation of the BitTorrent protocol. Everybody loves BitTorrent, and I love it too. I'm not the only one. By some estimates, torrent data may account for as much as 35% of all internet traffic.

I recently rented the television series Boomtown from Netflix, but I belatedly realized that the particular episode I really wanted to see was a part of season two. Unfortunately, Boomtown, like so many other great shows, was cancelled in its prime. In this case, it was cancelled right in the middle of season two – and the incomplete season was never released on DVD. What's a poor, law-abiding citizen of the United States of America to do?

BitTorrent to the rescue. I was able to locate a torrent of all the Boomtown episodes, and I'm downloading it now.

I've talked before about the remarkable parity between the uTorrent web user interface and the windows user interface. However, as good as the web UI is, it pales in comparison to the incredibly deep informatics that uTorrent provides on the state of your BitTorrent download. I love poring over the torrent metrics; they're beautifully presented – an excellent example of how to visually depict a complex set of data in a meaningful way. Let's take a quick visual tour through the main tabs in uTorrent; I'll point out the most interesting parts.

On the General Tab, we can see that the torrent is fairly well seeded. That's important, because the biggest weakness of the torrent system is that it requires a certain level of popularity to work at all. In our case, the availability graphic is a nice, solid blue – there are no red bars indicating missing sections. The darker the bars, the more copies of that particular section are available in the swarm.

It's also indicated numerically; an availability of 1 means the entire file is available. That's the minimum, assuming you want to download everything. An availability of 5.93 indicates there are almost 6 complete copies of the torrent data in the swarm. It's no coincidence that there are also 6 peers with a complete copy of the data – these critically important peers are known as "seeds". The other 19 peers will remain peers until they manage to download 100% of the data, and then they become seeds too. If a torrent loses all its seeds, it is in deep trouble.

On the Files Tab, we can see the state of individual files in the torrent. There's a nice little graphic next to each file that shows how many pieces of the files have been downloaded. The blue sections indicate parts of the file that have been successfully downloaded; green sections indicate parts of the file that are being downloaded right now.

You can also tag files with a priority, so they're retrieved in a particular order. Well, roughly – the torrent protocol retrieves in random order from whatever's available in the peer swarm, so order is never guaranteed. Or, you can set them to "skip" if you don't want to retrieve those files. Since I already rented all the season 1 DVDs, I set all the season 1 epsiodes to skip, and I switched the particular season 2 episode I wanted to "high". This will prevent me from becoming a seed, but it dramatically shortens my download time.

On the Peers Tab, we can get a glimpse into the democratic nature of the BitTorrent protocol. These are our fellow peers and seeds, sharing whatever bits of the data they have with everyone else in the swarm. Given enough peers, downloads are fast for everyone. uTorrent conveniently does a DNS lookup and shows little flags next to each peer, so you can get a sense of how global the BitTorrent protocol really is.

On the Pieces Tab, we can see the real-time state of the current pieces we're downloading from our peers in the swarm. Dark blue means downloaded; light blue means requested but not yet downloaded.

On the Speed Tab, we can view a history of transfer rates over time, including both upload and download speeds. BitTorrent is a shared download, so you're supposed to give as much as you get – although altruism is difficult to enforce.

Another section of the Speed Tab shows an incredibly detailed breakdown of disk activity. BitTorrent is designed to download enormous files. This torrent is over 8 GB in size, for example. With such large amounts of data in play, managing disk caches and optimizing disk activity is unusually important.

Most of this is also explained in the official uTorrent FAQ, and it goes into greater detail, too.

I'll admit that I tend to completely geek out on the way Torrent exposes all the inner workings of the BitTorrent protocol in such a beautiful, highly visual way. It never takes the easy way out and renders as a boring table of numbers when a visualization would work better. I think many programs could learn a lot from the way uTorrent so effortlessly presents the mountain of data it is processing internally.

I think there's a deeper lesson here as well. The commercial market failed me. As far as I can tell, there's no way I could obtain these elusive season 2 Boomtown episodes through legitimate channels. I was only able to obtain them through a torrent graciously seeded and shared by fellow Boomtown enthusiasts. Perhaps that's the real beauty of BitTorrent – it's the world's most efficient and democratic distribution network, because it's driven entirely by us.

Let me be completely honest with you. I have a full-blown mouse fetish. I've owned every single major mouse model from Microsoft and Logitech since the bad old days of the original Microsoft "Dove bar" mouse, and the Logitech MouseMan. I remember quite clearly bringing home my first mouse, an add-on for my Apple //c, and demonstrating this novel method of input to friends. I've been obsessing over these essential input devices since way before the days when USB was just a glint in Intel's collective eye; I have more than my share of mousing experience.

These days, I can't claim experience with every mouse under the sun; there are too many models out there. Mice have long since split into two distinct family trees: premium "performance" mice for gamers and enthusiasts; less expensive vanilla models for everyone else. As an enthusiast and a gamer, I've followed the enthusiast mouse family tree with great gusto. My current mouse of choice is the Microsoft Habu. But that was way back in March. Since then two very interesting new models have emerged.

The Microsoft Sidewinder Mouse:

The Logitech G9 Mouse:

I've now used both models for a few days, long enough to generate some informed opinions. They do have a quite a few things in common, things I'd consider relatively standard for current generation enthusiast mice:

• Five buttons (left, center, right, back, forward)
• Weight cartridges for adjustable "heft"
• Textured aluminum scroll wheels
• Hardware DPI adjustable "on the fly" with visual indicators
• Oversize glide pads on the bottom
• Mouse settings are permanently stored in on-board firmware

Each model also has a few unique features of its own:

I noticed that neither of the mouse wheels allow horizontal (left-right) scrolling. This is a good thing, because horizontal scrolling has always struck me as a dubious sort of feature at best. I think you'd need something other than a wheel to do it justice, more like a mini-trackball, and even then I'm not sure the complexity is worth it. How often do you need to scroll horizontally? I'd rather have a firm, bi-directional mouse wheel that's locked to up-down, anyway.

So which one do I prefer? My old Habu wasn't exactly chopped liver-- nor was the Logitech MX 518 I used before that-- but on the whole, I prefer the Logitech G9. The Sidewider is arguably the more innovative model, but I have a few concerns with it:

1. It is a large mouse. The shape, while unusual, is plenty comfortable-- but bulky. I prefer smaller mice in general.
2. The thumb buttons are in an unusual location. I've trained my thumb to move up, not forward. Every time I hit the "back" thumb button, I have to think and stretch a bit to reach it.
3. It's a bit more awkward in general. Even with equivalent DPI and mouse speed/acceleration settings, I miss small click targets that I had no problem hitting with the G9 or the Habu. I don't think it's a technical limitation; it might be a consequence of the fit.

The G9, on the other hand, is a flawless mouse upgrade. I have no complaints whatsoever-- it's a solid step forward in every respect. Well, there is one minor niggle worth mentioning: the body, because it's designed to be interchangeable, is a tiny bit loose when you pick up the mouse. If you frequently pick up the mouse to adjust the position, you might find that annoying. Also, the frictionless mouse wheel option is fun-- it reminds me of the spinner control in classic arcade games like Tempest-- but useless in practice. It is an option, not a requirement, so I don't deduct anything for that. I'll stick with the Sidewinder at work for a while longer and see if I can adapt. I admire all the innovation at a relatively low price (for this type of enthusiast mouse, at least), but I am sorely tempted to buy another G9.

During all this mouse testing, I spent a lot of time normalizing the pointer speed between the control panel mouse options and the DPI settings in the mouse's hardware. I don't think I realized until now how essential it is to enable mouse pointer acceleration for best pointer "feel" with any mouse. I strongly recommend that you double check to make sure this this feature is enabled. It's available in Control Panel, Mouse, Pointer Options under "Enhance Pointer Precision".

What does Enhance Pointer Precision do? It's a simple concept. When enabled, the pointer moves more precisely when you move the mouse slowly, and more nimbly when you move the mouse quickly. It decouples pointer movement ever so slightly from a basic 1:1 relationship with mouse movement, and introduces something called the mouse acceleration curve.

The translation from physical mouse movement to pointer movement is more sophisticated and more subtle than you might think. It's all documented in an excellent Microsoft article on mouse ballistics. It introduced me to the amusing concept of the mickey: the smallest unit of measurement that the mouse's hardware can produce.

Let's think about this like programmers. If it was our job to translate mouse mickeys into pointer movements, how would we do it? Our first order of business is to figure out how fast the mouse is moving on the table or mousepad-- the mouse velocity.

The accuracy of the mickeys coming from our mouse is strongly influenced by the bus update rate. The math proves it. I talked about this in an earlier post on Mouse DPI and USB Polling Rate. The good news is that fancy enthusiast mice always override the default 125 Hz USB update rate. Both of these mice bump it up to 500 Hz as soon as they're plugged in, which I verified using the Direct Input Mouse Rate tool.

The number of mickeys/inch is similarly influenced by the capabilities of the mouse's sensor hardware, also known as DPI. It should more accurately be called MPI, mickeys per inch. A "dot" isn't a dot at all; it's a completely arbitrary unit, nothing more than the smallest unit of movement that the hardware can measure. The Sidewinder ranges from 200 DPI to 2000 DPI; the G9 from 200 DPI to 3200 DPI. This is dynamically switchable via the buttons on the mouse, and configurable in software as well. Don't get hung up on the fact that the Sidewinder "only" goes up to 2000 DPI; the packets going over the wire only allow a mickey size between 0 and +127, so there's a practical limit on how precise you can be.

Now that we have mouse velocity in the physical world, let's determine how that will map to pointer velocity on the virtual world of our screen.

Screens are bounded by obvious, concrete physical limitations. Refresh rate is typically fixed at 60 Hz for modern LCD displays. Screen resolution varies from 800 x 600 to astronomically huge for those that can afford 30" displays, but the DPI ranges are fairly similar for most monitors.

Let's try plugging in some typical values into our formulas:

V_mouse = 3 mickeys * 500 Hz / 1600 DPI = 0.9375 inches/sec
V_pointer = 3 mickeys * 60 Hz / 80 DPI = 2.25 inches/sec

You can immediately see the disconnect-- 1 inch of physical mouse movement resulted in 2.25 inches of screen pointer movement. There's a physical to virtual gain of 2.4x. Without the mouse acceleration curve, this is as sophisticated as it gets. We might use a simple multiplier based on the pointer speed slider, but that's about it. The relationship is linear. We're doing a basic one to one mapping.

But with "Enhance Pointer Precision", we use a variable curve to determine how far the pointer moves for any given mouse speed. The different colored curves shown here represent different values of the pointer speed slider with acceleration enabled.

It is possible to edit these curves via the SmoothMouseXCurve and SmoothMouseYCurve registry settings, but there's absolutely no documentation I could find on these settings, so be careful. Getting the curve right is crucial. According to the article, the mouse acceleration curves in Windows were determined by a usability study. For example, many people dislike the default mouse acceleration curve in OS X:

So what's wrong with Mac OS X's mouse acceleration curve? Simply put, it's the wrong shape. For mouse motion to feel natural (at least for most people), the curve has to start by moving upward fairly moderately, then gradually flattening out as the value of X increases. Mac OS X's, curve, however, starts off by being too steep, staying too steep for too long, and then flattening out too abruptly. In practical terms this means that, frequently, as a user tries to use the mouse to move the pointer from point A to point B, the pointer motion feels sluggish. The user then tries to compensate for the sluggishness by moving the mouse faster, and the pointer suddenly goes flying across the screen and overshoots point B. A comfortable and useful curve is actually shaped like a curve. Mac OS X's curve, however, is shaped more like a cliff.

Windows may have better curves, as long as that "Enhance Pointer Precision" checkbox is ticked.

Who knew mouse ballistics could be this sophisticated? Heck, who knew that mouse ballistics even existed until now? Experiment with the "Enhance Pointer Precision" setting for yourself, but I believe it should always be enabled-- it results in mouse pointer movement most people will find both easier to control and more accurate.

I thought Ligaya Turmelle's post on SQL joins was a great primer for novice developers. Since SQL joins appear to be set-based, the use of Venn diagrams to explain them seems, at first blush, to be a natural fit. However, like the commenters to her post, I found that the Venn diagrams didn't quite match the SQL join syntax reality in my testing.

I love the concept, though, so let's see if we can make it work. Assume we have the following two tables. Table A is on the left, and Table B is on the right. We'll populate them with four records each.

id name       id  name
-- ----       --  ----
1  Pirate     1   Rutabaga
2  Monkey     2   Pirate
3  Ninja      3   Darth Vader
4  Spaghetti  4   Ninja

Let's join these tables by the name field in a few different ways and see if we can get a conceptual match to those nifty Venn diagrams.

SELECT * FROM TableA INNER JOIN TableB ON TableA.name = TableB.name id name id name 1 Pirate 2 Pirate 3 Ninja 4 Ninja Inner join produces only the set of records that match in both Table A and Table B.
SELECT * FROM TableA FULL OUTER JOIN TableB ON TableA.name = TableB.name id name id name 1 Pirate 2 Pirate 2 Monkey null null 3 Ninja 4 Ninja 4 Spaghetti null null null null 1 Rutabaga null null 3 Darth Vader Full outer join produces the set of all records in Table A and Table B, with matching records from both sides where available. If there is no match, the missing side will contain null.
SELECT * FROM TableA LEFT OUTER JOIN TableB ON TableA.name = TableB.name id name id name -- ---- -- ---- 1 Pirate 2 Pirate 2 Monkey null null 3 Ninja 4 Ninja 4 Spaghetti null null Left outer join produces a complete set of records from Table A, with the matching records (where available) in Table B. If there is no match, the right side will contain null.
SELECT * FROM TableA LEFT OUTER JOIN TableB ON TableA.name = TableB.name WHERE TableB.id IS null id name id name 2 Monkey null null 4 Spaghetti null null To produce the set of records only in Table A, but not in Table B, we perform the same left outer join, then exclude the records we don't want from the right side via a where clause.
SELECT * FROM TableA FULL OUTER JOIN TableB ON TableA.name = TableB.name WHERE TableA.id IS null OR TableB.id IS null id name id name 2 Monkey null null 4 Spaghetti null null null null 1 Rutabaga null null 3 Darth Vader To produce the set of records unique to Table A and Table B, we perform the same full outer join, then exclude the records we don't want from both sides via a where clause.

There's also a cartesian product or cross join, which as far as I can tell, can't be expressed as a Venn diagram:

SELECT * FROM TableA
CROSS JOIN TableB

This joins "everything to everything", resulting in 4 x 4 = 16 rows, far more than we had in the original sets. If you do the math, you can see why this is a very dangerous join to run against large tables.