A Visit from the Metrics Maid

For the last few days, I've been surveying a software project. Landing on a planet populated entirely by an alien ecosystem of source code can be overwhelming. That's why the first first thing I do is bust out my software tricorder -- static code analysis tools.

The two most essential static code analysis tools, for .NET projects, are nDepend and FxCop. Like real software tricorders, they produce reams and reams of data -- lots of raw metrics on the source code.

Even basic metrics can identify potential trouble spots and/or areas of interest in the code, such as..

  • Methods that are too large or too small.
  • Classes that are too large or too small.
  • Methods that are too complex (as measured by cyclomatic complexity).
  • Methods with too many parameters (more than 7 plus or minus 2).
  • Methods with too many local variables.
  • Classes with an excessively deep inheritance structure.
  • Types that are excessively large.

These simple metrics are already quite valuable. You can imagine how valuable more advanced software metrics could be, such as code coverage. Or how quickly you're finding and fixing bugs. And more advanced static analysis tools can offer literally hundreds of recommendations, ranging from mundane to mission-critical.

Having more data about your software development project can never be bad. The real trick, of course, lies in interpreting all that data, and deciding how to act on it. There's a huge temptation to become a metermaid-- to use the metrics as a reward or punishment system.

A metermaid

If Joe wrote a method with a cyclomatic complexity of 52, then he better get slapped with a complexity ticket, right? No excess complexity in the simplicity zone, you idiot!

Not necessarily. Responsible use of the metrics is just as important as collecting them in the first place. Gregor Hohpe elaborates:

Some of the most hated people in San Francisco must be the meter maids, the DPT people who drive around in golf carts and hand out tickets to anyone who overslept street cleaning or did not have enough quarters for the meter. On some projects, the most hated people are the metric maids, the people who go around and try to sum up a developer's hard work and intellectual genius in a number between 1 and 10.

Many managers love metrics: "You can't manage it if you can't measure it". I am actually a big proponent of extracting and visualizing information from large code bases or running systems (see Visualizing Dependencies). But when one tries to boil the spectrum between good and evil down to a single number we have to be careful as to what this number actually expresses.

Martin Woodward calls this the measurement dilemma.

The reporting aspects of Team Foundation Server are a new, more accurate instrument to take measurements inside your software development process. But you need to be wary about the things you measure. The metrics need to mean something useful rather than just be interesting. The effect of taking the metric should be carefully considered before taking it. This is not a new problem. But because Team Foundation Server makes it so easy to get data out of the system, the temptations are greater.

Martin also references the Heisenberg Uncertainty Principle, which states that you can't measure something without changing it. I believe this is true for software development metrics only if you are using that metric to reward or punish.

Recording metrics on your project can be beneficial even if you don't explicitly act on them. Having a public "wall of metrics" might be a better idea. It can be a focal point for discussion about what the metrics mean to the team. This gives everyone on the project an opportunity to discuss and reflect, and act on the metrics as they deem appropriate. Maybe the team will even remove a few metrics that are of no value.

What metrics do you find helpful on your software projects? What metrics do you find not so helpful? And if you have no project metrics to talk about, well, what are you waiting for?

Related posts

What does Stack Overflow want to be when it grows up?

What does Stack Overflow want to be when it grows up?

I sometimes get asked by regular people in the actual real world what it is that I do for a living, and here’s my 15 second answer: We built a sort of Wikipedia website for computer programmers to post questions and answers. It’s called Stack Overflow. As of

By Jeff Atwood ·
Comments

Civilized Discourse Construction Kit

Occasionally, startups will ask me for advice. That's a shame, because I am a terrible person to ask for advice. The conversation usually goes something like this: We'd love to get your expert advice on our thing. I probably don't use your thing. Even

By Jeff Atwood ·
Comments

How to Stop Sucking and Be Awesome Instead

I've been fortunate to have some measure of success in my life, primarily through this very blog over the last eight years, and in creating Stack Overflow and Stack Exchange over the last four years. With the birth of our twin girls, I've had a few

By Jeff Atwood ·
Comments

Books: Bits vs. Atoms

I adore words, but let's face it: books suck. More specifically, so many beautiful ideas have been helplessly trapped in physical made-of-atoms books for the last few centuries. How do books suck? Let me count the ways: * They are heavy. * They take up too much space. * They have

By Jeff Atwood ·
Comments

Recent Posts

Stay Gold, America

Stay Gold, America

We are at an unprecedented point in American history, and I'm concerned we may lose sight of the American Dream.

By Jeff Atwood ·
Comments
The Great Filter Comes For Us All

The Great Filter Comes For Us All

With a 13 billion year head start on evolution, why haven’t any other forms of life in the universe contacted us by now? (Arrival is a fantastic movie. Watch it, but don’t stop there – read the Story of Your Life novella it was based on for so much

By Jeff Atwood ·
Comments
I Fight For The Users

I Fight For The Users

If you haven’t been able to keep up with my blistering pace of one blog post per year, I don’t blame you. There’s a lot going on right now. It’s a busy time. But let’s pause and take a moment to celebrate that Elon Musk

By Jeff Atwood ·
Comments
The 2030 Self-Driving Car Bet

The 2030 Self-Driving Car Bet

It’s my honor to announce that John Carmack and I have initiated a friendly bet of $10,000* to the 501(c)(3) charity of the winner’s choice: By January 1st, 2030, completely autonomous self-driving cars meeting SAE J3016 level 5 will be commercially available for passenger use

By Jeff Atwood ·
Comments