Coding Horror

Scott Stanfield forwarded me a link to Roger Sessions' A Better Path to Enterprise Architecture yesterday. Even though it's got the snake-oil word "Enterprise" in the title, the article is surprisingly good.

I particularly liked the unusual analogy Roger chose to illustrate the difference between iterative and recursive approaches to software development. It starts with Air Force Colonel John Boyd researching a peculiar anomaly in the performance of 1950's era jet fighters:

Colonel John Boyd was interested not just in any dogfights, but specifically in dogfights between MiG-15s and F-86s. As an ex-pilot and accomplished aircraft designer, Boyd knew both planes very well. He knew the MiG-15 was a better aircraft than the F-86. The MiG-15 could climb faster than the F-86. The MiG-15 could turn faster than the F-86. The MiG-15 had better distance visibility.

The F-86 had two points in its favor. First, it had better side visibility. While the MiG-15 pilot could see further in front, the F-86 pilot could see slightly more on the sides. Second, the F-86 had a hydraulic flight control. The MiG-15 had a manual flight control.

The standing assumption on the part of airline designers was that maneuverability was the key component of winning dogfights. Clearly, the MiG-15, with its faster turning and climbing ability, could outmaneuver the F-86.

There was just one problem with all this. Even though the MiG-15 was considered a superior aircraft by aircraft designers, the F-86 was favored by pilots. The reason it was favored was simple: in one-on-one dogfights with MiG-15s, the F-86 won nine times out of ten.

How can an inferior aircraft consistently win over a superior aircraft? Boyd, who was himself one of the best dogfighters in history, had a theory:

Boyd decided that the primary determinant to winning dogfights was not observing, orienting, planning, or acting better. The primary determinant to winning dogfights was observing, orienting, planning, and acting faster. In other words, how quickly one could iterate. Speed of iteration, Boyd suggested, beats quality of iteration.

The next question Boyd asked is this: why would the F-86 iterate faster? The reason, he concluded, was something that nobody had thought was particularly important. It was the fact that the F-86 had a hydraulic flight stick whereas the MiG-15 had a manual flight stick.

Without hydraulics, it took slightly more physical energy to move the MiG-15 flight stick than it did the F-85 flight stick. Even though the MiG-15 would turn faster (or climb higher) once the stick was moved, the amount of energy it took to move the stick was greater for the MiG-15 pilot.

With each iteration, the MiG-15 pilot grew a little more fatigued than the F-86 pilot. And as he gets more fatigued, it took just a little bit longer to complete his OOPA loop. The MiG-15 pilot didn't lose because he got outfought. He lost because he got out-OOPAed.

This leads to Boyd's Law of Iteration: speed of iteration beats quality of iteration.

You'll find this same theme echoed throughout every discipline of modern software engineering:

• Unit tests should be small and fast, so you can run them with every build.
• Usability tests work best if you make small changes every two weeks and quickly discard what isn't working.
• Most agile approaches recommend iterations no longer than 4 weeks.
• Software testing is about failing early and often.
• Functional specifications are best when they're concise and evolving.

When in doubt, iterate faster.

Raymond Chen notes that, in his personal experience, users don't read dialogs:

How do I make this error message go away? It appears every time I start the computer.

RC: What does this error message say?
User: It says, 'Updates are ready to install.' I've just been clicking the X to make it go away, but it's really annoying.

Every time I start my computer, I get this message that says that updates are ready to install. What does it mean?

RC: It means that Microsoft has found a problem that may allow a computer virus to get into your machine, and it's asking for your permission to fix the problem. You should click on it so the problem can be fixed.
User: Oh, that's what it is? I thought it was a virus, so I just kept clicking No.

When I start the computer I get this big dialog that talks about Automatic Updates. I've just been hitting Cancel. How do I make it stop popping up?

RC: Did you read what the dialog said?
User: No. I just want it to go away.

Sometimes I get the message saying that my program has crashed and would I like to send an error report to Microsoft. Should I do it?

RC: Yes, we study these error reports so we can see how we can fix the problem that caused the crash.
User: Oh, I've just been hitting Cancel because that's what I always do when I see an error message.
RC: Did you read the error message?
User: Why should I? It's just an error message. All it's going to say is 'Operation could not be performed because blah blah blah blah blah.'

He wonders if software should have a Check Engine light:

Automobile manufacturers have learned to consolidate all their error messages into one message called "Check engine". People are conditioned to take the car in to a mechanic when the "Check engine" light goes on, and let the mechanic figure out what is wrong. Can we have a "Check engine" light for computers? Would it be feasible?

It's an interesting concept, insofar as it relieves the users from having to look at dialogs they won't understand anyway. But it seems highly unlikely to me that these users would pay any more attention to a subtle software Check Engine light than they do to the giant, screaming dialogs it's replacing.

And there's another problem with the automobile analogy, too. Unlike a car, computers-- at least the ones connected to the internet-- are perfectly capable of diagnosing and fixing themselves. The examples Raymond provides shouldn't have asked the user anything; they should have quietly gone about their business.

If you need to update, do so. if you need to download and apply security patches in the background, do so. If you need to send crash data, do so. Silently. And do it in the background, when the PC is idle-- without bothering the user.

If you're an advanced user who want to change and control this behavior, or view the status of these activities, you can certainly do so through control panels, options dialogs, and event logs. But the rest of the world doesn't care; they're relying on your software to do the right thing on their behalf without subjecting them to a barrage of questions they'll neither read nor understand.

A software check engine light is a mildly less invasive form of stopping the proceedings with idiocy. Your software should be more considerate than that.

I'm no fan of the classic login/password scheme. I can barely remember any of the zillion logins and passwords I have. More often than not, I end up using the "forgot password" link. Which means, in effect, that my email account is my global password. And if you're like most people, your email password isn't very secure. As Bruce Schneier recently observed:

We used to quip that "password" is the most common password. Now it's "password1." Who said users haven't learned anything about security?

It's a depressing state of affairs. Switching to passphrases helps, but is a band-aid at best.

The relentless increase in phishing attacks may soon force some changes on this front. I saw in the news that PayPal is switching to two-factor authentication. Specifically, they're providing users with a keyfob that produces a new six-digit code every 30 seconds. Users will now have to type in their name, password, and a valid code from the keyfob.

The PayPal system isn't SecurID, but I'm sure the implementation is very similar. There's a matching seed value stored on the server for each keyfob, so the server can calculate what the correct code should be. If the user enters the correct password and the correct code (within 30 seconds), they're allowed in.

So what's the value in doing this? It's more hassle and more expense. Well, consider that all security is based on three things:

What you have

What you know

What you are

We all use logins and passwords. That's something we know. When we enter the code from the keyfob, we've added something we have to the mix. That's two factor authentication, and it increases security dramatically.

But even with the keyfob, we haven't quite removed the risk of phishing entirely. All we've done is make the window of opportunity smaller. If a phishing site can relay the user-provided data to the server in real time (or close enough), they will still be authenticated.

A common form of local two-factor authentication is the Smart Card.

Smart cards have an embedded microprocessor that uniquely identifies each card, a private key of sorts. Some even have the ability to store data. The secrets on each card stay secret because it's impossible to extract the data without destroying the chip in the process. Since smart cards are read by hardware on your PC, they're of no use online. But they can dramatically enhance security locally. For example, Windows has embedded support for smart cards; it's possible to log into the operating system using nothing but a smart card and a short PIN code. The PIN code is still a password of sorts, but it's much shorter and easier to remember.

Once you switch over to smart cards, it's no longer possible to log in using a traditional username and password. Your underlying password becomes a randomly generated 64-character string. As you can imagine, this is a huge boon for local security compared to user-selected passwords. I don't personally care for smart cards, but I can certainly understand why organizations choose to use them.

But two-factor authentication, although more secure, isn't a panacea. Bruce Schneier is quick to remind us that two-factor authentication is vulnerable to two primary forms of attack:

Man-in-the-Middle attack. An attacker puts up a fake bank website and entices user to that website. User types in his password, and the attacker in turn uses it to access the bank's real website. Done right, the user will never realize that he isn't at the bank's website. Then the attacker either disconnects the user and makes any fraudulent transactions he wants, or passes along the user's banking transactions while making his own transactions at the same time.

Trojan attack. Attacker gets Trojan installed on user's computer. When user logs into his bank's website, the attacker piggybacks on that session via the Trojan to make any fraudulent transaction he wants.

We already knew about the man-in-the-middle attack; we refer to it as real-time phishing. As for trojans, it might be a little unfair to blame two-factor authentication for not protecting the user from a compromised system. I'm not sure any security measures can work on a compromised system with trojan keyloggers and screenloggers installed.

Despite Schneier's skepticism, I think two-factor authentication is worthwhile. Anything that moves the security bar beyond the hopelessly insecure and ineffective username/password combos we're currently stuck with is a welcome change.

As far as I'm concerned, Windows Media Center is one of the best-- if not the best-- applications Microsoft has ever created. And it was written in .NET to boot.

I've been a huge MCE enthusiast since the original version was released in 2003, so I was greatly looking forward to the Vista edition of Media Center. I've slowly been upgrading my Home Theater PC over the last two years in anticipation of the shift to Vista:

• 2.1 GHz Pentium-M "Dothan"
• AOpen i855GMEm mobo
• GeForce 7600GS, 256 MB, AGP 4x
• 2 GB RAM
• Hauppauge PVR-500 MCE dual tuner (highly recommended)
• 1 TB storage (dual 500 GB low-noise SATA drives)

Eventually I want to plop an internal HD-DVD drive in this machine once prices and configurations stabilize. But that's probably another 8-12 months out.

This weekend I took the plunge and upgraded my HTPC from Windows XP Media Center Edition 2005 to Windows Vista Home Premium. I wasn't disappointed. Vista's Media Center is a vast improvement over XP's Media Center. It's faster, it's prettier, and it's thoroughly improved in every way.

The default UI makes better use of the horizontal, widescreen arrangements most home theater setups will have. Recorded shows are now displayed as a linear timeline with a graphic still, rather than plain text in a list.

Under Vista's Media Center, my 60+ GB music library is now a pleasure to navigate. Like videos, much better use of horizontal screen real estate; I can see dozens of albums at once. And the music library is dramatically faster. Displaying, searching, scrolling-- it's all nearly instantaneous now. I love the new "play all" shuffle mode, too.

The program guide-- which is completely free, no monthly charges whatsoever-- now overlays the live video as a transparency. There's also a new popup Mini-Guide (not pictured) which lets you browse nearby channels without obscuring playback.

The main menu no longer stops whatever I'm doing and zaps me back to a flat menu screen. It's more of a pop-up style menu, which can be accessed at any time through the big green MCE button. I can now continue watching my program in the background while navigating the main menu, too.

Another big quality of life improvement in Vista's Media Center is that a DVD codec is included right out of the box. So Vista's Media Center, unlike the one in Windows XP, is fully usable after a clean install. It even works with my SPDIF out for Dolby Digital sound playback. There's no longer any need to rely on questionable, expensive third-party DVD playback apps.

Did I mention burning TV shows to DVD is now included out of the box, too? As far as I'm concerned, Media Center is the killer app for Vista. And at $120 for the OEM Home Premium edition, it's a flat-out bargain for a better-than-Tivo experience-- without all those onerous monthly fees.

If you're interested in a home theater PC, all you need is the following:

1. Vista Home Premium (or Ultimate)
2. relatively modern PC
3. MCE compatible PVR card
4. MCE remote

One caveat: I've stuck exclusively and intentionally with analog cable. All my digital video needs are satisified at the moment through DVD rentals and downloads. However, it is possible to record and play back over the air HDTV signals with Media Center, assuming you have a MCE compatible HDTV tuner installed (such as the AverMedia MCE A180). The only unresolved issue at this point is CableCard, for digital cable.

One of the sadder recent news stories is the disappearance of Turing award-winning researcher Jim Gray. I've written about Jim's research before; he has a knack for explaining fundamental truths of computer architecture in uniquely clear ways.

For example, in this ACM interview, Jim illustrates how the unusual economics of bandwidth can make a sneakernet worthwhile – if you're sending a terabyte of data.

JG We built more than 20 of these boxes we call TeraScale SneakerNet boxes. Three of them are in circulation. We have a dozen doing TeraServer work; we have about eight in our lab for video archives, backups, and so on. It's real convenient to have 40 TB of storage to work with if you are a database guy. Remember the old days and the original eight-inch floppy disks? These are just much bigger.

DP "Sneaker net" was when you used your sneakers to transport data?

JG In the old days, sneaker net was the notion that you would pull out floppy disks, run across the room in your sneakers, and plug the floppy into another machine. This is just TeraScale SneakerNet. You write your terabytes onto this thing and ship it out to your pals. Some of our pals are extremely well connected – they are part of Internet 2, Virtual Business Networks (VBNs), and the Next Generation Internet (NGI). Even so, it takes them a long time to copy a gigabyte. Copy a terabyte? It takes them a very, very long time across the networks they have.

DP When they get a whole computer, don't they still have to copy?

JG Yes, but it runs around their fast LAN at gigabit speeds as opposed to the slower Internet. The Internet plans to be running at gigabit speeds, but if you experiment with your desktop now, I think you'll find that it runs at a megabyte a second or less.

DP Megabyte a second? We get almost 10 megabytes sustained here.

JG That translates to 40 gigabytes per hour and a terabyte per day. I tend to write a terabyte in about 8 to 10 hours locally. I can send it via UPS anywhere in the U.S. That turns out to be about seven megabytes per second.

DP How do you get to the 7-megabytes-per-second figure?

JG UPS takes 24 hours, and 9 hours at each end to do the copy.

DP Wouldn't it be a lot less hassle to use the Internet?

JG It's cheaper to send the machine. The phone bill, at the rate Microsoft pays, is about $1 per gigabyte sent and about $1 per gigabyte received – about $2,000 per terabyte. It's the same hassle for me whether I send it via the Internet or an overnight package with a computer. I have to copy the files to a server in any case. The extra step is putting the SneakerNet in a cardboard box and slapping a UPS label on it. I have gotten fairly good at that. Tape media is about $3,000 a terabyte. This media, in packaged SneakerNet form, is about $1,500 a terabyte.

Does transferring a terabyte of data via sneakernet make sense?

First, consider the bandwidth capabilities and monthly cost of a few common Internet connections.

Of course, costs may vary; I chose costs that jibed with my personal experience and lined up with a few cursory searches for pricing around the web. Please let me know you think these costs are way out of line. Assuming for the sake of argument that these are representative costs and throughput rates, how much would it cost to transfer, let's say, a 20 gigabyte high definition video file?

And how much could we upload or download in total, assuming we had these connections going full-bore, around the clock?

Let's say we wanted to send a terabyte of data via sneakernet:

• Two 500 GB hard drives weigh about five pounds; we can wrap the drives in bubble wrap and fit them in a standard FedEx box.
• It costs about $60 to send five pounds in a standard FedEx box coast-to-coast in 24 hours.
• The total transit time is 32 hours: 24 hours, plus 8 hours to copy the data on and off the drives.

We just transferred data at the rate of 9.1 megabytes per second. The only internet connection that's capable of our sneakernet throughput level is the OC-3. None of the others are even close, particularly if you consider the highly asymmetric nature of consumer connections, where upload rate is a fraction of the download rate.

And what about the cost? Not including the $300 expense of the two hard drives (which I think is fair, beause they're reusable), the total cost per gigabyte breaks down like so:

It wasn't obvious to me, but the sneakernet math clearly works. This is exactly the kind of insight Jim Gray was famous for.

Jim also says the cost of internet bandwidth was roughly a dollar a gigabyte for Microsoft in 2003. Is that still how much internet bandwidth costs today? According to the figures I found, the only connection that expensive today is a modem. And who uses modems any more? It seems implausible that consumer internet bandwidth would be sold cheaper than large blocks of commercial internet bandwidth. Let's take a look.

• Amazon's S3 service charges 20 cents per gigabye to transfer data.
• Robert X Cringely regularly gets charged 20 cents per gigabyte for NerdTV, and guesses that large bandwidth consumers like YouTube can negotiate rates as low as 10 cents per gigabyte.
• Mitch Ratcliffe did a survey of internet providers and found that most charge 85 cents per gigabyte, and he proposes YouTube could negotiate a rate of 45 cents per gigabyte.

I'm not sure who to believe. It's a good sign that most estimates are under the $1.00 per gigabyte rate that Jim quoted in 2003. I'd like to think that the cost of internet bandwidth is getting less expensive over time. High bandwidth costs lead to a de-facto "popularity tax", and that's like a giant wet blanket over content creators. Cheaper bandwidth is a net public good: it leads directly to more content, and higher quality content, for everyone.