Does Having The Best Programmers Really Matter?

Jeff Atwood

03 Aug 2005 — 2 min read — Comments

Joel has a lengthy entry in which he asks, does having the "best programmers" really matter?

This is something I've talked about before: extreme skill disparity is unique to the profession of software development. The odds of working with a genius or a jackass on any given job are about fifty-fifty.

What's worse is that there's no correlation between experience and skill, either. I've worked with amazingly talented interns and 20 year vets who produced terrible code. Joel provides additional data to support this hypothesis:

The data I rely upon comes from Professor Stanley Eisenstat at Yale. Each year he teaches a programming-intensive course, CS 323, where a large proportion of the work consists of about ten programming assignments. The assignments are very serious for a college class: implement a Unix command-line shell, implement a ZLW file compressor, etc.
There was so much griping among the students about how much work was required for this class that Professor Eisenstat started asking the students to report back on how much time they spent on each assignment. He has collected this data carefully for several years.
I spent some time crunching the data; it's the only data sets I know of where we have dozens of students working on identical assignments using the same technology at the same time. It's pretty darn controlled, as experiments go.

Here's a representative sample for a single CS 323 assignment, showing time spent versus grade/score:

hours vs. score scatter plot for a Yale CS 323 assignment

There is absolutely no correlation whatsoever between time spent on the problem and the resulting score for these CS students. It's not much of a stretch to extend this to time spent in the profession of software development.

The rest of Joel's post veers dangerously close to being self-serving-- look at my awesome company and the smart employees I hired! but at least he's finally acknowledging that he is really talking about a very narrow niche:

It's not just a matter of "10 times more productive." It's that the "average productive" developer never hits the high notes that make great software.
Sadly, this doesn't really apply in non-product software development. Internal, in-house software is rarely important enough to justify hiring rock stars. Nobody hires Dolly Parton to sing at weddings. That's why the most satisfying careers, if you're a software developer, are at actual software companies, not doing IT for some bank.

Since Joel's commercial shrink-wrap software development is a tiny percentage of the overall IT market, does this mean that having the best programmers doesn't matter most of the time?

Sadly, I believe the answer is yes.

Complaint-Driven Development

If I haven’t blogged much in the last year, it’s because we’ve been busy building that civilized discourse construction kit thing I talked about. (Yes, that’s actually the name of the company. This is what happens when you put me in charge of naming things. Pinball

The Rule of Three

Every programmer ever born thinks whatever idea just popped out of their head into their editor is the most generalized, most flexible, most one-size-fits all solution that has ever been conceived. We think we've built software that is a general purpose solution to some set of problems, but

Today is Goof Off at Work Day

When you're hired at Google, you only have to do the job you were hired for 80% of the time. The other 20% of the time, you can work on whatever you like – provided it advances Google in some way. At least, that's the theory. Google&

Coding Horror: The Book

If I had to make a list of the top 10 things I’ve done in my life that I regret, “writing a book” would definitely be on it. I took on the book project mostly because it was an opportunity to work with a few friends whose company I