Coding Horror

Jensen Harris recently cited some Microsoft Office usability research which produced a rather counter-intuitive result:

One of the most interesting epiphanies I've had over the last few years seems on the surface like a paradox: "help" in Office is mostly used by experts and enthusiasts.

How can this be? I think my biased assumption was that experts know how to use the software already and eager novices would be poring over the documentation trying to learn how to be more effective using it.

Yet, in usability tests we see it again and again: novices and intermediates click around and experiment, experts try to reason things out and look them up in help.

Dispelling myths based on actual usability data. I love it. Jensen offers a few potential explanations for this phenomenon:

• Only experts know the right "magic words" to search for.
• Users need a broader scope of help: not a recipe, but a community college course in cooking.
• Help requires a context switch-- it's difficult to look at help side-by-side with the software.

While there's definitely merit to these observations, Jensen misses the biggest one. You need only read through a few of the many post comments to see what it is:

It's been my experience that Office help often doesn't include anything useful, so I don't use it. For example, I was using Access and somehow became aware of the DoCmd object. It seemed like it would do what I want, but there is no help for it. How could this object with 100 different features have no online help? Luckily Google came to the rescue, pointing me directly to the microsoft.com page telling me all about it. Why wasn't the included in the help originally?

I once found an Excel function in the help that required I install one of the add-in tools to use it, but the help didn't mention this add-in at all. So I did a google search and found out what I needed to do to use this function.

I have to agree that the online help in Office is not the best. It is a great product, but when I need help I usually give up after examining the first 50 hits and the move directly to Google.

Regarding relevance, I would suggest adopting Google's algorithms for ranking. A Google search for "Excel operators" yields what I was looking for ("About calculation operators") as the second hit from Microsoft, with another relevant document (differences between 1-2-3 and Excel operators) being first. Excel 2002's help gives rankings 4 and 14 respectively, with most of the other hits having nothing whatsoever to do with operators.

I practically never use online help. However, I literally live and die by Google Groups. I am almost never the only person to encounter a problem or misunderstand a feature and Google Groups usually proves a much faster route to success.

I have stopped using help in Office and just rely on Google. The replies are more relevant and better sorted, and it's faster. Even if the answer turns out to reside on the Office Assistance site, it's easier to find it there with Google.

It seems to me that the best "help file" for your software is no help file at all. It's a GoogleMSN Search query.

Local help simply can't compete with internet search. I'll go even further-- if you are building local help files for your application, you're wasting your time. And more importantly, you are wasting your users' time.

Smart developers will stop wasting time on useless local help and build community around their product on the internet. One of my favorite examples of this is http://www.regular-expressions.info. It's the top hit for a Google search on the term "regular expressions". The author, Jan Goyvaerts, is also the author of RegexBuddy, a fantastic regular expression tool. The tool is great, but when I need help with my regular expressions, I don't bother with the local help file. I just type what I'm looking for into MSN Search and bring the full weight of billions of dollars of search optimization to bear on the entire internet. If I happen to be offline, Jan has thoughtfully provided both PDF and CHM versions of the content on regular-expressions.info.

Local help seems rather quaint in comparison.

This kind of online community can easily be linked from the UI, too. One of the few generally helpful "help" features is contextual point-and-help -- eg, clicking on a "what the heck does this do?" button. But even those should spawn generic search engine queries to be truly useful.

If you really want to help your users, set up a wiki for your product. Set up forums for them to participate in. Become an impartial nexus for all information related to your product, whether your company wrote it or not. And most importantly, have the guts to give your users control over these sites. You may have written it, but they have to live with it.

A recent post by Steve Makofsky reminded me that the excellent UI Patterns and Techniques site is now a book from O'Reilly -- Designing Interfaces.

There's technically no reason to buy a book on visual design patterns when you can find the same information online ..

• GUI design patterns
• UI Patterns and Techniques
• The GUI Gallery
• The GUIdebook

Unlike any of the Design Patterns in that famous book, these patterns are visible to your users. Plan accordingly.

Have you ever worked with developers that were charter members of the third-party-control-of-the-month club? You know the kind-- they never met a third party control they didn't like. They spend all day trolling downloads and experimenting with every tool listed on The Daily Grind. Which means deploying your solution is now a tricky balancing act of obtaining and installing the proper license files from a half-dozen different control vendors. Who all do things slightly differently. Oh, and don't forget to make sure the versions of all your controls are all up to date with the latest bugfixes, too.

These are also the kind of developers who are prone to adopt giant, complex frameworks just to get tiny additional bits of functionality. If you're not careful, your entire project is liable to come down with a severe case of frameworkitis.

I, on the other hand, think anything outside the base framework is guilty until proven innocent. I'll adopt third-party code, but as little as I can get away with, and only if it offers significant, proven benefit to the problem I'm working on. I've been burned too many times. My code may suck, but it's a constant level of sucking that I can plan around.

That's why I was heartened to read Joel's account of how the Excel team aggressively avoids dependencies:

When I was the program manager in charge of the first implementation of Visual Basic for Applications, I put together a careful coalition of four, count them, four different teams at Microsoft to get custom dialog boxes in Excel VBA. The idea was complicated and fraught with interdependencies. There was a team called AFX that was working on some kind of dialog editor. Then we would use this brand new code from the OLE group which let you embed one app inside another. And the Visual Basic team would provide the programming language behind it. After a week of negotiation I got the AFX, OLE, and VB teams to agree to this in principle.

I stopped by Andrew Kwatinetz's office. He was my manager at the time and taught me everything I know. "The Excel development team will never accept it," he said. "You know their motto? 'Find the dependencies -- and eliminate them.' They'll never go for something with so many dependencies."

In-ter-est-ing. I hadn't known that. I guess that explained why Excel had its own C compiler.

By now I'm sure many of my readers are rolling on the floor laughing. "Isn't Microsoft stupid," you're thinking, "they refused to use other people's code and they even had their own compiler just for one product."

Not so fast, big boy! The Excel team's ruggedly independent mentality also meant that they always shipped on time, their code was of uniformly high quality, and they had a compiler which, back in the 1980s, generated pcode and could therefore run unmodified on Macintosh's 68000 chip as well as Intel PCs. The pcode also made the executable file about half the size that Intel binaries would have been, which loaded faster from floppy disks and required less RAM.

"Find the dependencies -- and eliminate them." When you're working on a really, really good team with great programmers, everybody else's code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. When you're a cordon bleu chef and you need fresh lavender, you grow it yourself instead of buying it in the farmers' market, because sometimes they don't have fresh lavender or they have old lavender which they pass off as fresh.

The .NET Framework was intended to be the dependency to end all dependencies. It's huge. It's also comprehensive and generally well-written. Any time you're reaching outside the framework for a giant swath of functionality, pause first and think about what you're trying to accomplish. Before you start stacking dependencies on top of the mother of all dependencies itself, make sure what you're getting justifies that risk.

Is it possible to take dependency avoidance too far? Of course. The flip side of reducing dependencies too aggressively is the Lava Flow anti-pattern:

As we delved into it, we interviewed many of the developers concerning certain components of the massive pages of code printed out for us. Over and over again we got the same answer, "I don't know what that class is for, it was written before I got here." We gradually realized that between 30 and 50% of the actual code comprising this complex system was not understood or documented by any one currently working on it. Furthermore, as we analyzed it, we learned that the questionable code really served no purpose in the current system, rather it was mostly there from previous attempts or approached by long-gone developers. The current staff, while very bright, was loath to modify or delete code that they didn't write or didn't know what it did for fear of breaking something and not knowing why or how to fix it.

I've never had a problem with lava flow because I am pathologically addicted to deleting code. I'm not afraid to get all Strunk and White on that codebase. I don't care if it woulda, coulda, shoulda been used-- is it being used right now? No? Then it's gone. Period. If you need it back, well, that's why God invented source control systems.

I'm not against dependencies. Everything software developers do is one giant string of dependencies. I'm a pragmatist. Strive to make your dependency stack as small as you possibly can.

From the "why I don't read Robert Scoble any more" department:

One thing I wish is that Web site developers/designers would look at their site on a small screen with limited bandwidth. So many sites suck really bad. I'm going to call these sites out with increasing frequency in 2006.

If your site makes you scroll for 20 minutes just to see your content, it sucks. It'll get called out.

If your site squeezes a column so that it's only one word wide, it sucks. It'll get called out.

My wish? Please try your site on a cell phone (tonight I was comparing sites on a Treo, on a Blackbery, and on my phone. My phone was best, but there were lots of sites that sucked on all three).

Millions of Web users are out there with cell phones. If you don't get your site to work properly with a cell phone, you're turning away customers and that sucks. It'll get called out.

You know, it would be nice if every single website was designed to scale perfectly across all resolutions from 176x220 to 1920x1440.

It'd also be nice if trees were made of cotton candy, and rain was delicious lemonade.

This is what I refer to as ".. and a Pony!" thinking: the person asking the question doesn't know that what they're asking for is essentially impossible. So you might as well throw a Pony in there while you're at it. Everyone loves Ponies.

I don't mean to single Robert out here. I'm sure he's a nice guy, but he's the absolute poster child for this phenomenon; it comes up over and over again in his blog.

It'd be helpful if he actually offered any thoughts on exactly how one would go about implementing a web site that scales up seamlessly seventy times in resolution. But why explain yourself when you can just use magic?

In Humanizing Peer Reviews, Karl Wiegers starts with a powerful pronouncement:

Peer review – an activity in which people other than the author of a software deliverable examine it for defects and improvement opportunities – is one of the most powerful software quality tools available. Peer review methods include inspections, walkthroughs, peer deskchecks, and other similar activities. After experiencing the benefits of peer reviews for nearly fifteen years, I would never work in a team that did not perform them.

After participating in code reviews for a while here at Vertigo, I believe that peer code reviews are the single biggest thing you can do to improve your code. If you're not doing code reviews right now with another developer, you're missing a lot of bugs in your code and cheating yourself out of some key professional development opportunities. As far as I'm concerned, my code isn't done until I've gone over it with a fellow developer.

But don't take my word for it. McConnell provides plenty of evidence for the efficacy of code reviews in Code Complete:

… software testing alone has limited effectiveness – the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent. Case studies of review results have been impressive:

• In a software-maintenance organization, 55 percent of one-line maintenance changes were in error before code reviews were introduced. After reviews were introduced, only 2 percent of the changes were in error. When all changes were considered, 95 percent were correct the first time after reviews were introduced. Before reviews were introduced, under 20 percent were correct the first time.
• In a group of 11 programs developed by the same group of people, the first 5 were developed without reviews. The remaining 6 were developed with reviews. After all the programs were released to production, the first 5 had an average of 4.5 errors per 100 lines of code. The 6 that had been inspected had an average of only 0.82 errors per 100. Reviews cut the errors by over 80 percent.
• The Aetna Insurance Company found 82 percent of the errors in a program by using inspections and was able to decrease its development resources by 20 percent.
• IBM's 500,000 line Orbit project used 11 levels of inspections. It was delivered early and had only about 1 percent of the errors that would normally be expected.
• A study of an organization at AT&T with more than 200 people reported a 14 percent increase in productivity and a 90 percent decrease in defects after the organization introduced reviews.
• Jet Propulsion Laboratories estimates that it saves about $25,000 per inspection by finding and fixing defects at an early stage.

The only hurdle to a code review is finding a developer you respect to do it, and making the time to perform the review. Once you get started, I think you'll quickly find that every minute you spend in a code review is paid back tenfold.

If your organization is new to code reviews, I highly recommend Karl's book, Peer Reviews in Software: A Practical Guide. The sample chapters Karl provides on his website are a great primer, too.