Coding Horror

programming and human factors

Programmers and Chefs

From an audio interview with Ron Jeffries:

The reason the kitchen is a mess is not because the kitchen is poorly designed, it's because we didn't do the dishes after every meal.

Michael Feathers recently wrote an eerily similar entry about the professional chef's concept of working clean:

One other thing that I liked about the Pastry Chef's competition was the way that the chefs were judged. There was more to it than just the final judging. All during the preparation, judges walked from station to station with clipboards, making little notes. One of the criteria that they used was working clean. Imagine that.. working clean.. The judges were watching to make sure that the chefs rubbed clean every bowl and utensil immediately after use. If they didn't, well, they were marked off.

Micah Martin left a great comment that illustrates how integral "working clean" is to professionals in the restaurant industry:

In my college years I was a chef/line cook at a few restaurants. Indeed working clean is a common theme in the kitchen. The term I heard over and over was "Clean as you go!". "Clean as you go!" wasn't so much a suggestion, but rather a law. Those cooks who didn't constantly clean would wind up in trouble. Their workspace would become so messy within a matter of an hour or two that the quality of food rapidly diminished. This problem would progress until the other cooks were forced to step in and clean up. This had a negative impact on the entire kitchen and Nobody was happy it happened. Interestingly, line cooks, even without college degrees, were extremely efficient as self-management. Those cook who didn't work clean, were taunted, teased, and pushed around until they cleaned up, or quit.

For software developers, working clean means constant refactoring. If you don't stop occasionally-- frequently, actually-- to revisit and clean up the code you've already written, you're bound to end up with a big, sloppy ball of code. If you forget to regularly clean up behind yourself, things get smelly. Working clean means following your nose and addressing those nagging issues before they become catastrophic.

In addition to working clean, cooks also spend a lot of time thinking about mise en place, how their cooking stations are arranged for optimal work. Michael Feathers explains:

There's an section in [the book Kitchen Confidential] where he talks about what cooks do late at night after the customers have gone home. They generally do what many people after work, they go out for beers and sit around talking about work, but what do they talk about really? Tony says that the subject that always comes up is something called mise en place. Mise en place is a blanket term for how you set up your station.

Mise en place

Is your tub of butter at the eleven o'clock position or at the one o'clock? Do you have two paring knives, and do you keep them next to your cutting board or next to your garnish bin? When you spend the night churning out meals, these decisions make a difference. Everyone has their favorite theory about the proper miz. Tony says that many cooks get downright mystical about it. According Tony, you'd better run if you are caught messing with another cook's miz. Sharp knives have multiple uses, apparently.

The concept of mise en place should be familiar to software developers. It's why every member of the team has their development system set up identically. It's why we use a common set of development tools. It's why we take advantage of existing frameworks like nUnit and Log4Net instead of writing our own.

Good programmers should be borderline obsessive about their "miz". Our craft changes too rapidly for us to ever be completely satisifed with the way we're working today. There's always something better on the horizon.

Discussion

Code Smells

I'm often asked why the book Refactoring isn't included in my recommended developer reading list. Although I own the book, and I've read it twice, I felt it was too prescriptive – if you see (x), then you must do (y). Any programmer worth his or her salt should already be refactoring aggressively. It's so essential to the craft that if you have to read a book to understand how it works, you probably shouldn't be a programmer in the first place.

There's nothing wrong with codifying refactoring guidelines in a book. But the most important guideline is to watch for warning signs in your own code – so called "code smells".

A collection of smelly cheeses

Developing your "code nose" is something that happens early in your programming career, if it's going to happen at all. I combined all the documented code smells I could find into this reference; most of these smells should be familiar to you.

Code Smells Within Classes

Comments There's a fine line between comments that illuminate and comments that obscure. Are the comments necessary? Do they explain "why" and not "what"? Can you refactor the code so the comments aren't required? And remember, you're writing comments for people, not machines.
Long Method All other things being equal, a shorter method is easier to read, easier to understand, and easier to troubleshoot. Refactor long methods into smaller methods if you can.
Long Parameter List The more parameters a method has, the more complex it is. Limit the number of parameters you need in a given method, or use an object to combine the parameters.
Duplicated code Duplicated code is the bane of software development. Stamp out duplication whenever possible. You should always be on the lookout for more subtle cases of near-duplication, too. Don't Repeat Yourself!
Conditional Complexity Watch out for large conditional logic blocks, particularly blocks that tend to grow larger or change significantly over time. Consider alternative object-oriented approaches such as decorator, strategy, or state.
Combinatorial Explosion You have lots of code that does almost the same thing.. but with tiny variations in data or behavior. This can be difficult to refactor-- perhaps using generics or an interpreter?
Large Class Large classes, like long methods, are difficult to read, understand, and troubleshoot. Does the class contain too many responsibilities? Can the large class be restructured or broken into smaller classes?
Type Embedded in Name Avoid placing types in method names; it's not only redundant, but it forces you to change the name if the type changes.
Uncommunicative Name Does the name of the method succinctly describe what that method does? Could you read the method's name to another developer and have them explain to you what it does? If not, rename it or rewrite it.
Inconsistent Names Pick a set of standard terminology and stick to it throughout your methods. For example, if you have Open(), you should probably have Close().
Dead Code Ruthlessly delete code that isn't being used. That's why we have source control systems!
Speculative Generality Write code to solve today's problems, and worry about tomorrow's problems when they actually materialize. Everyone loses in the "what if.." school of design. You (Probably) Aren't Gonna Need It.
Oddball Solution There should only be one way of solving the same problem in your code. If you find an oddball solution, it could be a case of poorly duplicated code-- or it could be an argument for the adapter model, if you really need multiple solutions to the same problem.
Temporary Field Watch out for objects that contain a lot of optional or unnecessary fields. If you're passing an object as a parameter to a method, make sure that you're using all of it and not cherry-picking single fields.

Code Smells Between Classes

Alternative Classes with Different Interfaces If two classes are similar on the inside, but different on the outside, perhaps they can be modified to share a common interface.
Primitive Obsession Don't use a gaggle of primitive data type variables as a poor man's substitute for a class. If your data type is sufficiently complex, write a class to represent it. 
Data Class Avoid classes that passively store data. Classes should contain data and methods to operate on that data, too.
Data Clumps If you always see the same data hanging around together, maybe it belongs together. Consider rolling the related data up into a larger class.
Refused Bequest If you inherit from a class, but never use any of the inherited functionality, should you really be using inheritance?
Inappropriate Intimacy Watch out for classes that spend too much time together, or classes that interface in inappropriate ways. Classes should know as little as possible about each other.
Indecent Exposure Beware of classes that unnecessarily expose their internals. Aggressively refactor classes to minimize their public surface. You should have a compelling reason for every item you make public. If you don't, hide it.
Feature Envy Methods that make extensive use of another class may belong in another class. Consider moving this method to the class it is so envious of.
Lazy Class Classes should pull their weight. Every additional class increases the complexity of a project. If you have a class that isn't doing enough to pay for itself, can it be collapsed or combined into another class?
Message Chains Watch out for long sequences of method calls or temporary variables to get routine data. Intermediaries are dependencies in disguise. 
Middle Man If a class is delegating all its work, why does it exist? Cut out the middleman. Beware classes that are merely wrappers over other classes or existing functionality in the framework.
Divergent Change If, over time, you make changes to a class that touch completely different parts of the class, it may contain too much unrelated functionality. Consider isolating the parts that changed in another class.
Shotgun Surgery If a change in one class requires cascading changes in several related classes, consider refactoring so that the changes are limited to a single class.
Parallel Inheritance Hierarchies Every time you make a subclass of one class, you must also make a subclass of another. Consider folding the hierarchy into a single class.
Incomplete Library Class We need a method that's missing from the library, but we're unwilling or unable to change the library to include the method. The method ends up tacked on to some other class. If you can't modify the library, consider isolating the method.
Solution Sprawl If it takes five classes to do anything useful, you might have solution sprawl. Consider simplifying and consolidating your design.

This list was derived from the Smells to Refactorings PDF, and the Smells to Refactorings Wiki, which also provide additional guidance on the specific refactorings that might be helpful in each instance. The important thing, from my perspective, isn't the refactoring – it's learning to recognize the scent of your own code.

And if you want examples of the stinkiest code imaginable, How to Write Unmaintainable Code is a good place to start.

Discussion

Snappy Answers to Stupid Programming Questions

Here's a not-so-gentle reminder from David Pickett that some programming interview questions – in this case, "how would you write a routine to copy a file?" – are, well, stupid*:

Q. What about the attributes?
A. Make the attributes the same.

Q. Should I modify the attributes of the source file? If this file copy is part of a backup or archive operation, it'd probably be a mistake to leave the 'Archive' attribute on.
A. No, leave them as-is.

Q. What if the source file has the Archive attribute off? If I make it off on the new file as well, it could screw up the user's backup software.
A. Just make it the same. I don't care about the user's backup software.

Q. Well, I'm not sure that's the best approach to take when thinking about designing software FOR users, but if you say so.
A. ...

Q. What about compression? It's a file attribute, but the copy destination may not support compression.
A. Don't compress the copy.

Q. Even if the source is compressed, and the destination supports compression?
A. YES.

Q. What about encryption? What if the source file is encrypted, but the destination does not support encryption?
A. Don't encrypt the copy if the destination doesn't support it.

Q. Mmmmm, sorry, don't mean to digress, but … that could be a serious security hole. Especially if wherever this file copy function ends up supports arbitrary parameters (directly or indirectly).
A. Look, just copy the damn file.

"How would you write a routine to copy a file" is just another interview riddle; it doesn't deserve a face value response. David's answers are even better-- they implode the question under its own weight.

Al Jaffee's Snappy Answers to Stupid Questions book cover

The only rational answer to "how would you write a routine to copy a file?" is another question: why would any competent programmer ever write a file copy routine?

The last time I checked, moving mount Fuji wasn't a part of our business plan, either.

* With apologies to Al Jaffee.

Discussion

Localhost HTTP debugging with Fiddler

I've had great success using ethernet sniffers (such as Etherdetect, or Ethereal) to troubleshoot communication problems. Installing a sniffer, even after installing the required WinPcap packet capture library, doesn't require a reboot. I frequently use sniffers to troubleshoot servers and desktops alike. Ethernet sniffers should be a standard tool in your development troubleshooting toolkit, too.

However, Windows ethernet sniffers do have one significant limitation: they can't sniff localhost traffic. Localhost packets don't pass through the regular network stack, so they're invisible to an ethernet sniffer.

What's a poor developer to do? The only recourse is a local HTTP proxy, such as Fiddler:

A screenshot of Fiddler

Fiddler has some special integration with IE that makes it particularly easy to use, but it can be used with Firefox as well.

I had some erratic results under IE7, but Fiddler basically works as advertised. There's tons of supporting documentation on how to use it, including two MSDN articles. I'm using Fiddler as a localhost sniffer that's limited to the HTTP protocol, but it does have some capabilities beyond what you'd see in a sniffer. For example, with Fiddler you can set breakpoints and tamper with the HTTP data before it is sent or received.

On the whole, I'd prefer to stick with a sniffer for localhost debugging. But a HTTP proxy like Fiddler is a reasonable workaround.

Discussion

The Long, Dismal History of Software Project Failure

From the IEEE article Why Software Fails:

Last October, for instance, the giant British food retailer J Sainsbury had to write off its US $526 million investment in an automated supply-chain management system. Merchandise was stuck in the company's depots and warehouses and was not getting through to many of its stores. Sainsbury was forced to hire about 3000 additional clerks to stock its shelves manually.

This is only one of the latest in a long, dismal history of [software] projects gone awry. Most IT experts agree that such failures occur far more often than they should. What's more, the failures are universally unprejudiced: they happen in every country; to large companies and small; in commercial, nonprofit, and governmental organizations; and without regard to status or reputation. The business and societal costs of these failures -- in terms of wasted taxpayer and shareholder dollars as well as investments that can't be made -- are now well into the billions of dollars a year.

The problem only gets worse as IT grows ubiquitous. This year, organizations and governments will spend an estimated $1 trillion on IT hardware, software, and services worldwide. Of the IT projects that are initiated, from 5 to 15 percent will be abandoned before or shortly after delivery as hopelessly inadequate. Many others will arrive late and over budget or require massive reworking. Few IT projects, in other words, truly succeed.

From Rapid Development:

If Las Vegas sounds too tame for you, software might just be the right gamble. Software projects include a glut of risks that would give Vegas oddsmakers nightmares. The odds of a large project finishing on time are close to zero. The odds of a large project being canceled are an even-money bet (Jones 1991).

In 1998, Peat Marwick found that about 35 percent of 600 firms surveyed had at least one runaway software project (Rothfeder 1988). The damage done by runaway software projects makes the Las Vegas prize fights look as tame as having high tea with the queen. Allstate set out in 1982 to automate all of its office operations. They set a 5-year timetable and an $8 million budget. Six years and $15 million later, Allstate set a new deadline and readjusted its sights on a new budget of $100 million. In 1988, Westpac Banking Corporation decided to redefine its information systems. It set out on a 5-year, $85 million project. Three years later, after spending $150 million with little to show for it, Westpac cut its losses, canceled the project, and eliminated 500 development jobs (Glass 1992). Even Vegas prize fights don't get this bloody.

The history of software development is a tremendous success. Just look around you for evidence of that. But that success has a long, dark shadow that we don't talk about very much: it's littered with colossal failures. What's particularly disturbing is that the colossal failures keep recurring year after year. The names and dollar amounts may change, but the story is otherwise the same. Two recent examples are the Canadian gun registry and the FBI's Virtual Case File system.

If you're looking for more examples of colossal software project failure, you don't have to look very far:

You'd think that the software development industry would have matured over the last ten years. And it has:

The 10th edition of the annual CHAOS report from The Standish Group, which researches the reasons for IT project failure in the United States, indicates that project success rates have increased to 34 percent of all projects. That's more than a 100-percent improvement from the success rate found in the first study in 1994.

Asked for the chief reasons project success rates have improved, Standish Chairman Jim Johnson says, "The primary reason is the projects have gotten a lot smaller. Doing projects with iterative processing as opposed to the waterfall method, which called for all project requirements to be defined up front, is a major step forward."

The Standish Group has studied over 40,000 projects in 10 years to reach the findings.

Project failures have declined to 15 percent of all projects, a vast improvement over the 31-percent failure rate reported in 1994. Projects meeting the "challenged" description -- meaning that they are over time, over budget and/or lacking critical features and requirements -- total 51 percent of all projects in the current survey.

Failing is OK. Failing can even be desirable. But you must learn from your failures, and that requires concerted postmortem introspection and analysis. I'd like to think that a large part of the statistical improvement cited above is attributable to sharp project managers and savvy developers who studied the first CHAOS report. Once you know what the common pitfalls are, it's easier to avoid them.

Discussion