Coding Horror

I consider this the golden rule of source control:

Check in early, check in often.

Developers who work for long periods -- and by long I mean more than a day -- without checking anything into source control are setting themselves up for some serious integration headaches down the line. Damon Poole concurs:

Developers often put off checking in. They put it off because they don't want to affect other people too early and they don't want to get blamed for breaking the build. But this leads to other problems such as losing work or not being able to go back to previous versions.

My rule of thumb is "check-in early and often", but with the caveat that you have access to private versioning. If a check-in is immediately visible to other users, then you run the risk of introducing immature changes and/or breaking the build.

I'd much rather have small fragments checked in periodically than to go long periods with no idea whatsoever what my coworkers are writing. As far as I'm concerned, if the code isn't checked into source control, it doesn't exist. I suppose this is yet another form of Don't Go Dark; the code is invisible until it exists in the repository in some form.

I'm not proposing developers check in broken code -- but I also argue that there's a big difference between broken code and incomplete code. Isn't it possible, perhaps even desirable, to write your code and structure your source control tree in such a way that you can check your code in periodically as you're building it? I'd much rather have empty stubs and basic API skeletons in place than nothing at all. I can integrate my code against stubs. I can do code review on stubs. I can even help you build out the stubs!

But when there's nothing in source control for days or weeks, and then a giant dollop of code is suddenly dropped on the team's doorstep -- none of that is possible.

Developers that wouldn't even consider adopting the old-school waterfall method of software development somehow have no problem adopting essentially the very same model when it comes to their source control habits.

Perhaps what we need is a model of software accretion. Start with a tiny fragment of code that does almost nothing. Look on the bright side -- code that does nothing can't have many bugs! Test it, and check it in. Add one more small feature. Test that feature, and check it in. Add another small feature. Test that, and check it in. Daily. Hourly, even. You always have functional software. It may not do much, but it runs. And with every checkin it becomes infinitesimally more functional.

If you learn to check in early and check in often, you'll have ample time for feedback, integration, and review along the way. And who knows -- you might even manage to accrete that pearl of final code that you were looking for, too.

As a software developer, tell me if you've ever done this:

1. Taken a screenshot of something on the desktop
2. Opened it in a graphics program
3. Gone off to work on something else
4. Upon returning to your computer, attempted to click on the screenshot as if it was an actual program.

And let's not forget the common goating technique where you take a screenshot of someone's desktop, make it the desktop background, then proceed to hide every UI element on the screen. The anguished cries as users desperately double-triple-quadruple click on pixels that look exactly like real user interfaces can typically be heard for miles.

I bring this up to generate some sympathy. I get fooled by my own FUI -- Fake User Interface -- at least once a month. If it can happen to us, it can happen to anyone. Which means FUI can be quite dangerous in the wrong hands. Consider Ryan Meray's story:

Okay, so here's an interesting one. My girlfriend is researching stuff on lilies, so she's trying to find the website for the Michigan Regional Lily Society.

The website address is http://www.mrls.org/

Feel free and browse there directly, there's nothing wrong with it. But if you don't remember the URL, your first response is to Google it. We google and get this:

http://www.google.com/search?q=Michigan+Regional+Lily+Society

Now, if you're in Firefox, everything is fine. You click that first result, and you get to their website, and you learn about lilies.

However, if you are using IE, be aware, you are about to have a Spyware/Virus alert.

Obviously, the poor Michigian Regional Lily Society has fallen prey to website hackers. (Note that it may have been fixed by the time I'm writing this -- but I duplicated everything I'm about to show you.)

The first clever point is that the website appears fine if you navigate there directly. The malicious JavaScript code inserted into the page checks the referer and does something different if you arrive there via a web search engine. This means the people who own the website, and never arrive there through Google, would be scratching their heads, wondering what all the fuss is about. So the hack survives longer.

But if you do arrive at the MRLS site through a search engine, like a huge percentage of the world does, you're redirected to:

http://scanner.antivir64.com/?aff=1050

The very first thing this page does is minimize the browser (Firefox 3, in this case) and present us with this JavaScript alert:

I'm intentionally juxtaposing the browser and the dialog here, but the browser is way off in the very lower right corner of the display and that dialog is smack dab in the middle of the screen. It is not at all clear that the dialog originated from that web page. It's a primitive technique, but it is surprisingly effective.

I didn't have the guts to click OK on that dialog; I clicked the close button. The browser then expanded to show this convincing "real time virus scan".

The static screenshot does not do it justice; the scrollbar moves, the list of files fly by as they are "scanned", and the web page rather successfully simulates an ersatz UI somewhere between Windows XP and Windows Vista. Of course, we know this Fake User Interface is completely invalid, because it is running in the browser, not on our PC. You and I may understand that distinction, but what about your parents? Your wife? Your children? Your less technically savvy friends? Will they understand this scary, authentic looking virus warning coming from an "encrypted secure site" is all a lie?

Honestly, whose PC doesn't "run slower than normal"? Maybe I would want to know if my computer is infected with Viruses, Adware or Spyware. It's all part of the culture of fear that security software companies -- and let's be honest, Windows security software companies -- cultivate so they can rake in millions of dollars per year hawking their software. The difference here, of course, is that it's increasingly difficult to tell the good guys from the bad guys. That's the downside of fear as a selling point: it cuts equally well in both directions.

Woe betide the poor user who is convinced through the trickery of FUI to install this "antivirus" software. The page does its darndest to convince you to run its payload executable. Any click on the page, no matter where, is interpreted as a download request.

The page also attempts a drive-by download, though those have been auto-blocked for years now.

It's tempting to put this down as yet another iteration of phishing, the forever hack. To be fair, this is exactly the sort of thing web browser phishing filters were designed to prevent. This site was already in the Firefox 3 phishing filter -- but it was not caught by the Internet Explorer 7 phishing filter, so I reported it.

I am all for phishing filters as another important line of defense, but like all distributed blacklists, they're only so effective.

What I'm more concerned about here is how well the user interface was spoofed. The browser FUI was convincing enough to even make me -- possibly the world's most jaded and cynical Windows user -- do a bit of a double-take. How do you protect naive users from cleverly designed FUI exploits like this one? Can you imagine your mother doing a web search on flowers -- flowers, for God's sake -- clicking on the search results to a totally legitimate website, and correctly navigating the resulting maze of fake UI, spurious javascript alerts, and download dialogs?

I know I can't. As much as I admire distributed phishing blacklist efforts, there's no way they can possibly keep pace with the rapid setup and teardown of hacked websites. How many compromised websites are out there? How many unsophisticated users surf the internet every day?

As always, we can lay a big part of the blame at Microsoft's doorstep for not adopting the UNIX policy of non-administrator accounts for regular users. But then again, if the spoofing is good enough, the FUI extra-convincing, even a Linux or OS X user could be coerced into entering their admin password for a "system security scan". Or maybe they just wanted to see the dancing bunnies.

And then, like Ryan, you're likely to end up with the same infected computer, and the same distraught spouse. All this for the love of a few lilies.

Short of user education, which is a neverending, continuous uphill battle -- how would you combat a perfectly spoofed FUI presented to a naive user?

One of the early technology decisions we made on Stack Overflow was to go with a fairly JavaScript intensive site. Like many programmers, I've been historically ambivalent about JavaScript:

• The Power of "View Source"
• The Day Performance Didn't Matter Any More
• JavaScript and HTML: Forgiveness by Default
• JavaScript: The Lingua Franca of the Web
• The Great Browser JavaScript Showdown

However, it's difficult to argue with the demonstrated success of JavaScript over the last few years. JavaScript code has gone from being a peculiar website oddity to -- dare I say it -- delivering useful core features on websites I visit on a daily basis. Paul Graham had this to say on the definition of Web 2.0 in 2005:

One ingredient of its meaning is certainly Ajax, which I can still only just bear to use without scare quotes. Basically, what "Ajax" means is "Javascript now works." And that in turn means that web-based applications can now be made to work much more like desktop ones.

Three years on, I can't argue the point: JavaScript now works. Just look around you on the web.

Well, to a point. We can no longer luxuriate in the -- and to be clear, I mean this ironically -- golden age of Internet Explorer 6. We live in a brave new era of increasing browser competition, and that's a good thing. Yes, JavaScript is now mature enough and ubiquitous enough and fast enough to be a viable client programming runtime. But this vibrant browser competition also means there are hundreds of aggravating differences in JavaScript implementations between Opera, Safari, Internet Explorer, and Firefox. And that's just the big four. It is excruciatingly painful to write and test your complex JavaScript code across (n) browsers and (n) operating systems. It'll make you pine for the good old days of HTML 4.0 and CGI.

But now something else is happening, something arguably even more significant than "JavaScript now works". The rise of commonly available JavaScript frameworks means you can write to higher level JavaScript APIs that are guaranteed to work across multiple browsers. These frameworks spackle over the JavaScript implementation differences between browsers, and they've (mostly) done all the ugly grunt work of testing their APIs and validating them against a host of popular browsers and plaforms.

The JavaScript Ninjas have delivered their secret and ultimate weapon: common APIs. They transform working with JavaScript from an unpleasant, write-once-debug-everywhere chore into something that's actually -- dare I say it -- fun.

Frankly, it is foolish to even consider rolling your own JavaScript code to do even the most trivial of things in a browser now. Instead, choose one of these mature, widely tested JavaScript API frameworks. Spend a little time learning it. You'll ultimately write less code that does more -- and (almost) never have to worry a lick about browser compatibility. It's basically browser coding nirvana, as Rick Strahl noted:

I've kind of fallen into a couple of very client heavy projects and jQuery is turning out to be a key part in these particular projects. jQuery is definitely one of those tools that has got me really excited as it has changed my perspective in Web Development considerably from dreading doing client development to actually looking forward to applying richer and more interactive client principles.

There are several popular Javascript API frameworks to choose from:

1. Prototype and Script.aculo.us
2. JQuery
3. Yahoo UI Library
4. ExtJS
5. Dojo
6. MooTools

I don't profess to be an expert in any of these. Far from it. But I will echo what Rick said: using JQuery while writing Stack Overflow is probably the only time in my entire career as a programmer that I have enjoyed writing JavaScript code.

It's sure pleasant to write code against solid, increasingly standardized JavaScript API libraries that spackle over all those infuriating browser differences. I, for one, would like to thank John Resig and all the other JavaScript Ninjas who share their secrets -- and their frameworks -- with the rest of the community.

Occasionally people will ask me what kind of music I like to code by. I'm not sure I am the right person to ask this question of.

Allow me to explain by citing my 2001 Amazon review of a particular album.

It all started so innocently. I purchased this CD on a lark in mid 1998.

Subsequently, I put on this CD at high volume to torture my then-coworkers. It became a running joke. We'd take any opportunity, any pretext at all, to put it on. It had to be played at least once every day for "good luck." We'd force each other to listen to it. We'd have little contests to see who was man enough to listen to it over and over and still silently sit there programming away, not complaining. Sometimes we'd sing along to enhance the effect.

In short: we broke people. It was like a Vietnamese prison camp in stereo.

It was a joke. But then a very strange thing happened -- as I listened to the CD over and over, I began to like it. I mean really like it! I began to listen to it at home on my own time. "There's something about this music", I thought, as I listened to it for the 543rd time. "Maybe it's so bad, it has actually wrapped all the way around and it's.. good again?". I played the album for my wife. At that point I was hooked. I knew all the words to "Having my Baby", and.. I liked it!

For completeness, here's the track list. If you have any kind of musical taste, you may want to look away from the screen momentarily.

1. Tie A Yellow Ribbon Round The Ole Oak Tree - Dawn
2. The Night Chicago Died - Paper Lace
3. Billy, Don't Be A Hero - Bo Donaldson & The Heywoods
4. (You're) Having My Baby - Paul Anka
5. Playground In My Mind - Clint Holmes
6. Feelings - Morris Albert
7. Sometimes When We Touch - Dan Hill
8. The Candy Man - Sammy Davis, Jr.
9. Afternoon Delight - Starland Vocal Band
10. Torn Between Two Lovers - Mary MacGregor
11. Escape (The Pina Colada Song) - Rupert Holmes
12. Muskrat Love - Captain & Tennille

(An anonymous commenter was kind enough to create a Mixwit "mix tape" web page of the above songs, if you're feeling masochistic and want to hear them yourself. Or sadistic, I guess, if you manage to broadcast this music to your coworkers somehow. Not that I would officially endorse said action in any possible way, of course!)

(Update: Fixed with modern YouTube playlist. Music licensing is the hardest problem in computer science!)

In a peculiar twist of fate, one of my then coworkers, Geoff, now works with me on Stack Overflow. He can confirm that what I said above actually happened, although I'm not sure you could make something like that up. Apparently his mind wasn't totally destroyed by exposure to this "music". As far as we know.

While I've mentioned mild forms of coworker griefing -- er, I mean, teambuilding -- before in Don't Forget to Lock Your Computer, I thought this audio form was unique.

What I didn't know then is that this sort of musical griefing had a precedent. It's documented in the 1994 book Show Stopper! The Breakneck Race to Create Windows NT and the Next Generation at Microsoft. I didn't get around to reading this excellent book until 2004, but it's right there in black and white:

[David] Cutler camped in the Build Lab now, scrutinizing the check-ins, so [Kyle] Shannon wanted him to be comfortable. After further musical experiments, he finally hit on a sound that pleased Cutler. It was a raucous album by the rock group Journey. One morning Shannon slapped on Journey, and heavy metal sounds filled the lab. Cutler started bobbing his head, humming to the cacophony. Shannon smiled. Nodding gratefully, Cutler promised to share with the builders a couple of his own favorite albums.

He didn't have any favorite albums, but he saw a chance to relieve tension. That night he asked his companion, Deborah Girdler, to visit a CD store and buy something "really bad." She returned with two discs: Jim Nabors (star of the 1960s TV series Gomer Pyle, U.S.M.C.) singing gospel tunes and the fantasy characters Alvin and the Chipmunks singing children's songs. Perfect, Cutler thought.

The next day Cutler treated his builders to Nabors singing "In the Sweet Bye and Bye," "Onward Christian Soldiders" and other hymns. When Cutler sang along, everyone cringed; it was hard to tell which was more loathesome -- Nabors gone gospel or Cutler gone musical. No one cheered when Cutler asked to hear the Nabors disc over and over again, day after day.

Before long Shannon and the builders regretted ever awakening Cutler's musicality. They finally hid the Nabors disc on the floor under a desk. When Cutler asked for it, Shannon invariably said "It's in my car." Cutler, who caught the lie, laughed and laughed.

So the next time you ask one of your fellow programmers to put on some background coding music for the team, think twice. That's all I'm saying.

Now if you'll excuse me, I'm going to slip on my headphones and get back to coding while listening to one of my favorite albums, The Transformed Man.

Hey, Mr. Tambourine Man, play a song for me ...

Although I love reading programming books, I find software project management books to be some of the most mind-numbingly boring reading I've ever attempted. I suppose this means I probably shouldn't be a project manager. The bad news for the Stack Overflow team is that I effectively am one.

That's not to say that all software project management books are crap. Just most of them. One of the few that I've found compelling enough to finish is Johanna Rothman's Behind Closed Doors: Secrets of Great Management. She co-wrote it with Esther Derby.

After reading it, you'll realize this is the book they should be handing out to every newly minted software project manager. And you'll be deeply depressed because you don't work with any software project managers who apparently have read it.

I originally discovered Johanna when one of her pieces was cited in the original Spolsky Best Software Writing book. Her article on team compensation (pdf) basically blew my mind; it forced me to rethink my entire perspective on being paid to work at a job. You should read it. If you have a manager, you should get him or her to read it, too. (Update: this essay is actually by Mary Poppendieck, who is also great. I'm leaving it in the post because it's fantastic reading, even if it's a little off topic.)

Since then, I've touched on her work briefly in Schedule Games and You Are Not Your Job. But I'd like to focus on a specific aspect of project management that I'm apparently not very good at. A caller in Podcast #16 took me to task for my original Stack Overflow schedule claims way back in late April. What was supposed to be "6 to 8 weeks" became.. well, something more like three months.

My problem is that I'm almost pathologically bad about writing things down. Unless I'm writing a blog entry, I suppose. I prefer to keep track of what I'm doing in my head, only anticipating as far ahead as the next item I plan to work on, while proceeding forward as quickly as I can. I think I fell prey, at least a little bit, to this scenario:

"Look, Mike," Tomas said. "I can hand off my code today and call it 'feature complete', but I've probably got three weeks of cleanup work to do once I hand it off." Mike asked what Tomas meant by "cleanup." "I haven't gotten the company logo to show up on every page, and I haven't gotten the agent's name and phone number to print on the bottom of every page. It's little stuff like that. All of the important stuff works fine. I'm 99-percent done."

Do you see the problem here? I know, there are so many it's difficult to know where to begin listing them all, but what's the deepest, most fundamental problem at work here?

This software developer does not have a detailed list of all the things he needs to do. Which means, despite adamantly claiming that he is 99 percent done -- he has no idea how long development will take! There's simply no factual basis for any of his schedule claims.

It is the job of a good software project manager to recognize the tell-tale symptoms of this classic mistake and address them head on before they derail the project. How? By forcingencouraging developers to create a detailed list of everything they need to do. And then breaking that list down into subitems. And then adding all the subitems they inevitably forgot because they didn't think that far ahead. Once you have all those items on a list, then -- and only then -- you can begin to estimate how long the work will take.

Until you've got at least the beginnings of a task list, any concept of scheduling is utter fantasy. A very pleasant fantasy, to be sure, but the real world can be extremely unforgiving to such dreams.

Johanna Rothman makes the same point in a recent email newsletter, and offers specific actions you can take to avoid being stuck 90% done:

1. List everything you need to do to finish the big chunk of work. I include any infrastructure work such as setting up branches in the source control system.

2. Estimate each item on that list. This initial estimate will help you see how long it might take to complete the entire task.

3. Now, look to see how long each item on that list will take to finish. If you have a task longer than one day, break that task into smaller pieces. Breaking larger tasks into these inch-pebbles is critical for escaping the 90% Done syndrome.

4. Determine a way to show visible status to anyone who's interested. If you're the person doing the work, what would you have to do to show your status to your manager? If you're the manager, what do you need to see? You might need to see lists of test cases or a demo or something else that shows you visible progress.

5. Since you've got one-day or smaller tasks, you can track your progress daily. I like to keep a chart or list of the tasks, my initial estimated end time and the actual end time for each task. This is especially important for you managers, so you can see if the person is being interrupted and therefore is multitasking. (See the article about the Split Focus schedule game.)

I'm not big on scheduling -- or lists -- but without the latter, I cannot have the former. It's like trying to defy the law of gravity. Thus, on our project, we're always 90% done. If you'd like escape the 90% done ghetto on your software project, don't learn this the hard way, like I did. Every time someone asks you what your schedule is, you should be able to point to a list of everything you need to do. And if you can't -- the first item on your task list should be to create that list.