Coding Horror

Bruce Eckel deftly identifies the root cause of all software development problems:

We are in a young business. Primitive, really -- we don't know much about what works, and we keep thinking we've found the silver bullet that solves all problems. As a result, we go through these multi-year boom and bust cycles as new ideas come in, take off, exceed their grasp, then run out of steam. But some ideas seem to have staying power. For example, a lot of the ideas in agile methodologies seem to be making some real impacts in productivity and quality. This is because they focus more on the issues of people working together and less on technologies.

A man I've learned much from, Gerald Weinberg, wrote his first couple of books on the technology of programming. Then he switched, and wrote or coauthored 50 more on the process of programming, and he is most famous for saying "no matter what they tell you, it's always a people problem."

Usually the things that make or break a project are process and people issues. The way that you work on a day-to-day basis. Who your architects are, who your managers are, and who you are working with on the programming team. How you communicate, and most importantly how you solve process and people problems when they come up. The fastest way to get stuck is to think that it's all about the technology and to believe that you can ram your way through the other things. Those other things are the most likely ones to stop you cold.

Bruce misremembers the actual quote; it's "no matter what the problem is, it's always a people problem." But Bruce's reformulation has a certain ineffable truthiness to it that is certainly in the spirit of Gerald Weinberg's writing.

Let's say I was tasked with determining whether your software project will fail. With the responses to these three questions in hand, I can tell you with almost utter certainty whether your project will fail:

1. How many lines of code will your team write?
2. What kind of software are you building?
3. Do you like your coworkers?

That last question isn't a joke. I'm not kidding. Do you like the company of your teammates on a personal level? Do you respect your teammates professionally? If you were starting at another company, would you invite your coworkers along? Do you have spirited team discussions or knock-down, drag-out, last man standing filibuster team arguments? Are there any people on your team you'd "vote off the island" if you could?

It may sound trivial to focus on the people you work with over more tangible things like, say, the actual work, or the particular technology you're using to do that work. But it isn't. The people you choose to work with are the most accurate predictor of job satisfaction I've ever found. And job satisfaction, based on my work experience to date, correlates perfectly with success. I have never seen a happy, healthy, gelled, socially functional software development team fail. It's a shame such teams are so rare.

As Weinberg said, it's always a people problem. If you aren't working with people you like, people you respect, people that challenge and inspire you-- then why not? What's stopping you?

What is user space? User space is the location in the filesystem where users put their personal files-- their "stuff". Here's the user space folder structure in the Windows XP operating system:

Documents and SettingsUser
Application Data
Cookies
Desktop
Favorites
Local Settings
My Documents
My Music
My Pictures
My Recent Documents
NetHood
PrintHood
SendTo
Start Menu

And here's the user space folder structure in the Windows Vista operating system:

UsersUser
AppData
Local
Roaming
Contacts
Desktop
Documents
Downloads
Favorites
Links
Music
Pictures
Saved Games
Searches
Videos

This new Vista user space folder structure may seem oddly familiar to people using operating systems based on Unix. It gives new life to the famous quote "Those who do not understand Unix are condemned to reinvent it, poorly". Regardless, I'm glad we no longer have to deal with a scarily long default user path -- with aggravating embedded spaces, even-- to our personal stuff. If a user is keeping notes in a text file, we can reasonably expect to find those notes at the following path:

Documents and SettingsUserMy Documentsnotes.txt
UsersUserDocumentsnotes.txt

Now that we've established what and where user space is, I have a message for all the programmers reading this-- including myself. Keep your dirty, filthy paws out of my personal user space!

Take a look in your Documents folder right now. Go ahead. Look. Do you see any files or folders in there that you personally did not create? If so, you've been victimized. Applications should never create or modify anything in your documents folder without your permission. And yet, sadly, it happens all the time. Applications, and more specifically, the programmers who wrote those applications, think it's perfectly A-OK to carpet bomb your personal user space with their junk.

Well, it isn't.

As Omar Shahine originally pointed out almost two years ago, we should be mad as hell, and we shouldn't take it any more. If applications need to store shared files, that's what the AppData and Application Data folders are for. In OS X, which inherits a lot of filesystem conventions from BSD Unix, Apple has a great set of guidance appropriately titled Don't Pollute User Space:

It is important to remember that the user domain (/Users) is intended for files created by the user. With the exception of the ~/Library directory, your application should never install files into the user's home directory. In particular, you should never install files into a user's Documents directory or into the /Users/Shared directory. These directories should only be modified by the user.

Even if your application provides clip art or sample files that the user would normally manipulate, you should place those files in either the local or user's Library/Application Support directory by default. The user can move or copy files from this directory as desired. If you are concerned about the user finding these files, you should include a way for the user to browse or access them directly from your application's user interface.

Discovering this made me realize how much I missed the good old days of strict operating system and GUI conventions. Granted, Apple is cheating quite a bit by inheriting the well-worn conventions of classic Unix operating systems. But so much of Windows seems dangerously ad-hoc in comparison.

At any rate, it is our obligation as programmers to let the user's folders belong to the user. Do the responsible thing and store your application files in an appropriate location, not as virtual litter amongst the user's other files. Please don't pollute user space.

I've often thought that software developers were akin to Magpies, birds notorious for stealing shiny items to decorate their complex nests. Like Magpies, software developers are unusually smart and curious creatures, almost by definition. But we are too easily distracted by shiny new toys and playthings.

I no longer find Scott Hanselman's Ultimate Developer Tool list inspiring. Instead, it's fatiguing. The pace of change in the world of software is relentless. We're so inundated with the Shiny and the New that the very concepts themselves start to disintegrate, the words repeated over and over and over until they devolve into a meaningless stream of vowels and consonants. "Shiny" and "new" become mundane, even commonplace. It's no longer unique for something to be new, no longer interesting when something is shiny. Eventually, you grow weary of the endless procession of shiny new things.

I'm not alone. Jeremy Zawodny also notes the diminishing luster of shiny new things:

Over a year ago I unsubscribed from Steve's blog because he had a habit of writing in breathless fashion about the latest shiny new thing – often several times a day. I see too many people I know getting caught up in the breathless hype and forgetting to think about whether the latest shiny new thing really matters in the grand scheme of things.

Dave Slusher concurs:

[Robert Scoble] says that he gets too much email and that is ineffective for getting PR releases to him. He suggests that what you should do now is leave him a message on his Facebook wall. Dear god and/or Bob. In the time I've followed Scoble, I must have seen something like this a dozen times from him. Don't email, Twitter me. Don't Twitter, Pwnce. Jaiku me. Leave a wall message, send an SMS, just call me, email me, don't email me, don't call me. Enough already! I'm not even trying to get in contact with him, and I find this constant migration from platform to platform to be a load of shit that just wearies me. I felt the same way when I dropped TechCrunch, well over a year ago. I got so tired of hearing about another slightly different way of doing what we were already doing and why that tiny difference was worth dropping everything and moving over. I officially renounce the search for the newer and shinier.

It isn't just the neverending stream of tech news. It's also the tidal push and pull of a thousand software religious wars that continually wears us down, like errant rocks in a rapidly flowing stream. I bet the process David Megginson outlines sounds awfully familiar:

1. Elite (guru) developers notice too many riff-raff using their current programming language, and start looking for something that will distinguish them better from their mediocre colleagues.

2. Elite developers take their shopping list of current annoyances and look for a new, little-known language that apparently has fewer of them.

3. Elite developers start to drive the development of the new language, contributing code, writing libraries, etc., then evangelize the new language. Sub-elite (senior) developers follow the elite developers to the new language, creating a market for books, training, etc., and also accelerating the development and testing of the language.

4. Sub-elite developers, who have huge influence (elite developers tend to work in isolation on research projects rather than on production development teams), begin pushing for the new language in the workplace.

5. The huge mass of regular developers realize that they have to start buying books and taking courses to learn a new language.

6. Elite developers notice too many riff-raff using their current programming language, and start looking for something that will distinguish them better from their mediocre colleagues.

I hope you're sitting down, because I've got some bad news for you. That Ruby on Rails thing you were so interested in? That's so last year. We've moved on.

If you consider that, statistically, the vast majority of programmers have yet to experience a dynamic language of any kind – much less Ruby – the absurdity here is sublime. Some dynamic language features are trickling down to the bastions of Java and .NET, but slowly, and with varying levels of success. These so-called thought leaders have left a virtual ghost town before anyone else had a chance to arrive.

I became a programmer because I love computers, and to love computers, you must love change. And I do. But I think the magpie developer sometimes loves change to the detriment of his own craft. Andy Hunt and Dave Thomas, the Pragmatic Programmers who were a big part of the last sea change in Ruby, said it quite well in a 2004 IEEE column (pdf).

Users don't care whether you use J2EE, Cobol, or a pair of magic rocks. They want their credit card authorization to process correctly and their inventory reports to print. You help them discover what they really need and jointly imagine a system.

Instead of getting carried away with the difficult race up the cutting edge of the latest technology, Pete concentrated on building a system [in COBOL] that works for him and his clients. It's simple, perhaps almost primitive by our lofty standards. But it's easy to use, easy to understand, and fast to deploy. Pete's framework uses a mixture of technologies: some modeling, some code generation, some reusable components, and so on. He applies the fundamental pragmatic principle and uses what works, not what's merely new or fashionable.

We fail (as an industry) when we try to come up with the all-singing, all-dancing applications framework to end all applications frameworks. Maybe that's because there is no grand, unified theory waiting to emerge. One of the hallmarks of postmodernism – which some think is a distinguishing feature of our times – is that there's no "grand narrative," no overarching story to guide us. Instead, there are lots of little stories.

Don't feel inadequate if you aren't lining your nest with the shiniest, newest things possible. Who cares what technology you use, as long as it works, and both you and your users are happy with it?

That's the beauty of new things: there's always a new one coming along. Don't let the pursuit of new, shiny things accidentally become your goal. Avoid becoming a magpie developer. Be selective in your pursuit of the shiny and new, and you may find yourself a better developer for it.

One advantage of being a geek is that our habits-- as such habits go-- are not terribly expensive. I've written before about my interest in auto racing. Instead of spending $100,000 on a sports car, I've built a nifty racing simulation rig that delivers many of the same thrills at a tiny fraction of the price. It's one of my few indulgences, and I'd like to share how I built it with you.

Here are the ingredients:

It's worth noting that the Playseat Evolution is designed to mate with the G25 wheels, pedal, and shifter. The mounting holes match up perfectly. That was a pleasant surprise, as I had to do quite a bit of drilling on the older, original Playseat to get things mounted in the first version of this rig. With the Evolution and G25 combo, it's almost plug and play, although you still have to do some drilling to get the shifter add-on mounted properly.

The premium leather-and-metal (well, mostly) G25 kit includes some fairly esoteric features from a major brand vendor like Logitech, notably a clutch pedal and full shifter kit.

I know $229 may seem like a lot, but it's actually a great deal considering what you'd have to pay for an aftermarket shifter or clutch. You don't have to use these advanced realism features, of course. You can always ignore the clutch pedal, and the shifter can be switched between simple up/down mode and a full 6 speed + reverse layout.

The other item of interest here is the bass shakers. I split the PC audio between the PC and the 100 watt subwoofer amplifier, which is strategically mounted under the seat via bungee cords. I also tuck most of the wires under there.

The amp is dedicated to driving the two 50 watt Aura shakers, which I've drilled and mounted on each side of the seat platform. The bottom of the aura has a cork backing, so there's no metal-to-metal contact.

The wiring is quite basic, but if you'd like more detail there's a great walkthrough on hooking up bass shakers on Dave's site. The net effect of the bass shakers is pretty wonderful-- all the low-end bass is converted to tactile rumbling you feel in the driver's seat. You'll instantly know when you hit a rumble strip, and when revving a powerful engine you can feel the roar. Bass shakers are a clever, if decidedly low-tech, way to extend the sophisticated force feedback effects of the wheel to the rest of your body. It's no force dynamics simulator, but the bang for the buck is off the charts.

The "brains" behind this simulator is a franken-machine of parts left over from various PC upgrades I've made over the last year or so. PCs are so cheap these days, it's hardly worth listing the hardware specs. Any vaguely modern dual core CPU with 2 GB of memory will do fine. There are only two bits worth worrying about:

• Video card. Don't skimp here. I'd recommend the NVIDIA 8800GT or better, as games tend to be heavily video card dependent these days.
• Sound card. Get a discrete sound card with enough outputs to drive a 5.1 surround system. I need three analog outputs to drive the necessary 6 channels on the old Logitech Z-680 surround system I used in this room. A simple stereo plug isn't enough.

For the display, I opted for an inexpensive projection system.

I used a typical 4:3 business class projector, mounted on a shelf at the rear top of the room. The screen is the largest that would fit in the space. I've also mounted the 5.1 speakers on the wall, as you can see. The two rear speakers are on the opposite wall behind us, and the subwoofer sits in a rear corner.

Between the booming sound, the huge eight foot screen, the realistic racing "cockpit", and the force feedback of the leather-wrapped wheel and rumbling bass shakers, it's an impressive driving experience.

I've always loved racing simulations, and now I've assembled a rig that does them justice. Some of my current favorite racing sims are:

• Dirt
• Richard Burns Rally
• Race 07: The official WTCC Game
• GTR 2
• rFactor
• Live for Speed

There's something about the programmer in me that delights in the physics playground afforded by these simulations:

Simulation, by definition, needs to be accurate. Otherwise, well, it's not simulating reality, really, which is of course the idea of simulation. Games like Forza simulate the real physics of racing in a predictable and mathematically precise manner.

The past, present and future of computer simulation of real-time physical events, or simply computer-based simulations that involve highly accurate representations of things moving/changing in space and time that are precisely affected by multiple variables like wind, rain, gravity, mud, oil, planets, waves, etc are fascinating topics for gamers (many may not realize this explicitly, but they sure experience it!), mathematicians, programmers and physicists alike.

I know this racing simulation rig probably barely scratches the surface of what it would actually be like to drive a $100k sports car on a race track. It certainly won't get you as much attention from the opposite sex as a real sports car would. But it's still a heck of a lot of fun, nonetheless-- and it can be built by mere mortals like you and I.

Most operating systems have some method of displaying CPU utilization. In Windows, this is Task Manager.

CPU usage is generally represented as a simple percentage of CPU time spent on non-idle tasks. But this is a bit of a simplification. In any modern operating system, the CPU is actually spending time in two very distinct modes:

1. Kernel Mode

   In Kernel mode, the executing code has complete and unrestricted access to the underlying hardware. It can execute any CPU instruction and reference any memory address. Kernel mode is generally reserved for the lowest-level, most trusted functions of the operating system. Crashes in kernel mode are catastrophic; they will halt the entire PC.

2. User Mode

   In User mode, the executing code has no ability to directly access hardware or reference memory. Code running in user mode must delegate to system APIs to access hardware or memory. Due to the protection afforded by this sort of isolation, crashes in user mode are always recoverable. Most of the code running on your computer will execute in user mode.

It's possible to enable display of Kernel time in Task Manager, as I have in the above screenshot. The green line is total CPU time; the red line is Kernel time. The gap between the two is User time.

These two modes aren't mere labels; they're enforced by the CPU hardware. If code executing in User mode attempts to do something outside its purview-- like, say, accessing a privileged CPU instruction or modifying memory that it has no access to -- a trappable exception is thrown. Instead of your entire system crashing, only that particular application crashes. That's the value of User mode.

x86 CPU hardware actually provides four protection rings: 0, 1, 2, and 3. Only rings 0 (Kernel) and 3 (User) are typically used.

If we're only using two isolation rings, it's a bit unclear where device drivers should go-- the code that allows us to use our video cards, keyboards, mice, printers, and so forth. Do these drivers run in Kernel mode, for maximum performance, or do they run in User mode, for maximum stability? In Windows, at least, the answer is it depends. Device drivers can run in either user or kernel mode. Most drivers are shunted to the User side of the fence these days, with the notable exception of video card drivers, which need bare-knuckle Kernel mode performance. But even that is changing; in Windows Vista, video drivers are segmented into User and Kernel sections. Perhaps that's why gamers complain that Vista performs about 10 percent slower in games.

The exact border between these modes is still somewhat unclear. What code should run in User mode? What code should run in Kernel mode? Or maybe we'll just redefine the floor as the basement-- the rise of virtualization drove the creation of a new ring below all the others, Ring -1, which we now know as x86 hardware virtualization.

User mode is clearly a net public good, but it comes at a cost. Transitioning between User and Kernel mode is expensive. Really expensive. It's why software that throws exceptions is slow, for example. Exceptions imply kernel mode transitions. Granted, we have so much performance now that we rarely have to care about transition performance, but when you need ultimate performance, you definitely start caring about this stuff.

Probably the most public example of redrawing the user / kernel line is in webservers. Microsoft's IIS 6 moved a sizable chunk of its core functionality into Kernel mode, most notably after a particular open-source webserver leveraged Kernel mode to create a huge industry benchmark victory. It was kind of a pointless war, if you ask me, since the kernel optimizations (in both camps) only apply to static HTML content. But such is the way of all wars, benchmark or otherwise.

The CPU's strict segregation of code between User and Kernel mode is completely transparent to most of us, but it is quite literally the difference between a computer that crashes all the time and a computer that crashes catastrophically all the time. This is what we extra-crashy-code-writing programmers like to call "progress". So on behalf of all programmers everywhere, I'd like to say thanks User mode. You rock!