Coding Horror

programming and human factors

The Greatest Invention in Computer Science

What do you think the single greatest invention in computer science is? Besides the computer itself, I mean.

Seriously, before reading any further, pause here for a moment and consider the question.

I've talked before about how young so-called modern computer programming languages really are, and it bears repeating for context.

C is roughly as old as I am; FORTRAN is as old as my parents. But what about the new kids on the block? The TIOBE software TCPI metrics page provides some data on language popularity going back to the year 2001. Consider the tender age of many of the newest, hippest programming languages:

Ruby is barely a teenager. JavaScript and PHP haven't even hit their teens yet.

For all our talk about fancy new programming language features, I sometimes think we forget the one fundamental building block underlying all of them: the humble routine. Take it from Steve McConnell, who urges us to Use Routines, Routinely:

Aside from the invention of the computer, the routine is arguably the single greatest invention in computer science. It makes programs easier to read and understand. It makes them smaller (imagine how much larger your code would be if you had to repeat the code for every call to a routine instead of invoking the routine). And it makes them faster (imagine how hard it would be to make performance improvements in similar code used in a dozen places rather than making all the performance improvements in one routine). In large part, routines are what make modern programming possible.

If you're not old enough to remember life before routines, I thought James Shore had a great example of the stark difference in his excellent article Quality With a Name:

Before structured programming:

1000 NS% = (80 - LEN(T$)) / 2
1010 S$ = ""
1020 IF NS% = 0 GOTO 1060
1030 S$ = S$ + " "
1040 NS% = NS% - 1
1050 GOTO 1020
1060 PRINT S$ + T$
1070 RETURN

After structured programming:

public void PrintCenteredString(string text) {
  int center = (80 - text.Length) / 2;
  string spaces = "";
  for (int i = 0; i < center; i++) {
    spaces += " ";
  }
  Print(spaces + text);
}

The humble routine is the backbone of all programming in any modern language. I'm sure you're the very model of a modern programmer, so I won't bore you with a long explanation of why routines are a good idea. The original 1998 IEEE McConnell article covers the rationales behind routines quite well. There's also a greatly expanded version of that material in Chapter 7 of Code Complete 2.

Routines are so fundamental to today's programming that they are essentially invisible. That's the problem with routines: they only take a minute to learn, but a lifetime to master. If bad unstructured programming was possible, so is bad structured programming. You can write FORTRAN in any language. Wrestling with the ineffable essence of a routine is, almost to a first approximation, what programming now is:

  • How long should this routine be? How long is too long? How short is too short? When is code "too simple" to be in a routine?
  • What parameters should be passed to this routine? What data structures or data types? In what order? How will they be used? Which will be modified as a result of the routine?
  • What's a good name for this routine? Naming is hard. Really hard.
  • How is this routine related to other nearby routines? Do they happen at the same time, or in the same order? Do they share common data? Do they really belong together? What order should they be in?
  • How will I know if the code in this routine succeeded? Should it return a success or error code? How will exceptions, problems, and error conditions be handled?
  • Should this routine even exist at all?

Good programmers – regardless of whatever language they happen to be working in – understand the importance of crafting each routine with the utmost care. The routines in your code should be treated like tiny, highly polished diamonds, each one more exquisitely polished and finely cut than the last.

I'll grant you this isn't a particularly deep insight. It's not even original advice. But if you believe, as I do, in constantly practicing the fundamentals, you'll never stop mastering the art of writing the perfect routine.

It is, after all, the single greatest invention in computer science.

Discussion

Please Give Us Your Email Password

A number of people whose opinions I greatly respect have turned me on to Yelp over the last six months or so. Yelp is a community review site, and a great way to discover cool new places in whatever neighborhood you happen to be in.

I've enjoyed using Yelp, and I wanted to participate by submitting my first review, so I created a new account there. As part of the account creation process, I was presented with this.

Yelp friends check

The idea is that I tell Yelp what email service I use, then provide my login and password information so Yelp can determine if any of my email contacts are Yelp members. How convenient!

Here's how I see that page.

Yelp friends check: EPIC FAIL

I'm willing to give Yelp the benefit of the doubt here, but let's think about what it means to give out your email account and password to anyone, no matter how ostensibly trustworthy they may be:

  1. Number one with a bullet: your email account is a de-facto master password for your online identity. Most -- if not all -- of your online accounts are secured through your email. Remember all those "forgot password" and "forgot account" links? Guess where they ultimately resolve to? If someone controls your email account, they have nearly unlimited access to every online identity you own across every website you visit.

  2. If you're anything like me, your email is a treasure trove of highly sensitive financial and personal information. Consider all the email notifications you get in today's highly interconnected web world. It's like a one-stop-shop for comprehensive and systematic identity theft. How do I know Yelp isn't going to dip into other areas of my email?

  3. Even if I trust Yelp absolutely, how do I know they're not going to store my email password, perhaps insecurely, in a place some disgruntled programmer or hacker can eventually get to it? Giving out your password puts the recipient in the highly unfortunate position of having to secure your password. Give that email password out enough, and you're now vulnerable in dozens of places spread across the face of the web. The odds start to look pretty dire.

I'm sure Yelp means well. They just want to help me find my friends, doggone it! But the very nature of the request is incredibly offensive; they have effectively asked for the keys to my house in order to riffle through my address book.

I don't think so.

Frankly, it's irresponsible to even ask this question. Naive internet users may not understand why it is such a profoundly bad idea to give out their email credentials to random websites. Worse, they might eventually get the idea that giving out their email credentials is typical or normal.

It's not. This is outlined quite literally in most privacy policies:

The security of your account also depends on keeping your account password confidential, and you should not share your account name or password with anyone. If you do share your account information with a third party, they will have access to your account and your personal information. -- Google Checkout

If a password is used to help protect your accounts and personal information, it is your responsibility to keep your password confidential. Do not share this information with anyone. If you are sharing a computer with anyone you should always choose to log out before leaving a site or service to protect access to your information from subsequent users. -- Microsoft Passport

Your Yahoo! ID and password are confidential information. A Yahoo! employee will never ask you for your password in an unsolicited phone call or email. Do not respond to any message that asks for your password. -- Yahoo

How did we end up in a world where it's even remotely acceptable to ask for someone's email credentials? What happened to all those years we spent establishing privacy policies to protect our users? What happened to the fundamental tenet of security common sense that says giving out your password, under any circumstances, is a bad idea?

I can understand the cutthroat desire to build monetizable "friend" networks by any means necessary. Even if it means encouraging your users to cough up their login credentials to competing websites. But how can I take your privacy policies seriously if you aren't willing to treat your competitors' login credentials with the very same respect that you treat your own? That's just lip service.

Email is the de-facto master password for a huge swath of your online identity. Tread carefully:

  • As a software developer, you should never ask a user for their email credentials. It's unethical. It's irresponsible. It is wrong. If someone is asking you to code this, why? For what purpose?
  • As a user, you should never provide your email credentials to anyone except your email service. Sites that ask you for this information are to be regarded with extreme suspicion if not outright distrust.

Beyond those ethical guidelines, I do wonder why the technological solution to this problem has barely been addressed. If all Yelp wants is my address book, why can't I grant them temporary access to my public email address book without giving out the keys to my email kingdom?

If even a fraction of the coding effort that regularly goes into convincing people to cough up their email or website login credentials went into finding other, more reasonable solutions to this problem -- perhaps we could have arrived at a saner solution by now. And we can start by taking obnoxious, utterly inappropriate credential requests completely off the table.

UPDATE: Several commenters brought to light some efforts underway to address this pernicious problem:

A more general solution may be OAuth, billed as an open standard for API access delegation. In other words, a valet key for websites:

Many luxury cars today come with a valet key. It is a special key you give the parking attendant and unlike your regular key, will not allow the car to drive more than a mile or two. Some valet keys will not open the trunk, while others will block access to your onboard cell phone address book. Regardless of what restrictions the valet key imposes, the idea is very clever. You give someone limited access to your car with a special key, while using your regular key to unlock everything.

Chris Messina of the OAuth project was kind enough to provide a number of related links in the comments and a followup post on the OAuth blog as well.

I was encouraged to learn about some of the recent progress we've made on this front. If you were looking for a way to be part of the solution, instead of the problem, read up on these solutions and participate!

Discussion

Large USB Flash Drive Performance

In the last three years, I've gone from carrying a 512 MB USB memory stick to a 16 GB USB memory stick. That's pretty amazing.

According to the storagereview.com archives, hard drives with 16 GB of storage were introduced sometime around the beginning of 1999. Barely 10 years later, we carry around that much on our keychains. Heck, my laptop only has a 32 GB solid state drive, and I manage to scrape by with that. These things are essentially miniature hard drives. I'm starting to wonder why we don't just take our entire computing environment, operating system and all, along with us and boot it up on whatever computer we happen to encounter in the wild.

There is one big problem with this approach, however. USB flash drive performance, even for the best models, is a small fraction of typical hard drive performance.

Modern 2.5" hard drive performance looks something like this:

HDD Sequential Read55 MB/sec
HDD Sequential Write55 MB/sec

Mind you, those aren't particularly fantastic numbers, just typical ones. You can get significantly faster hard drives, such as the Western Digital Velociraptor I just bought. Storage Review described the Velociraptor thusly:

single-user scores .. blow away those of every other [hard drive]

I was immediately sold once I read that. It's not cheap, but you get what you pay for, and I'm firmly in the "hard drive performance matters" camp. To quote Ferris Bueller, it is so choice. If you have the means, I highly recommend picking one up.

But what about large USB flash drives? How do they compare to typical hard drive speeds, much less the awe-inspiring Velociraptor? X-Bit Labs recently reviewed three 32 GB USB flash drives:

Sequential ReadSeqential Write
32 GB Corsair Flash Voyager22 MB/sec10 MB/sec
32 GB OCZ Rally 230 MB/sec22 MB/sec
32 GB Patriot Xporter XT31 MB/sec17 MB/sec

Not bad by any means, but punishing next to typical hard drive throughput. If you really were booting and running your operating system entirely from a USB flash drive, you'd feel like you had stepped back in time five full years.

Even if you're only planning to use your USB flash drive for the mundane task of storing files, you should care deeply about read and write speeds. Throughput wasn't much of an issue when USB drives were "only" a gigabyte or two. But when we're talking about 8 GB, 16 GB or 32 GB of data, being limited to 10 MB/sec write speeds means 13, 26, and 52 minutes respectively to fill that flash drive up with data. Do you have that kind of time?

The OCZ Rally 2 flash drive appears to be the winner in the Xbit labs roundup, but does the 32 GB size really offer the best bang for your buck? I did a quick spot check of OCZ Rally 2 flash drive prices on Amazon:

2 GB$17$8.50/GB
4 GB$26$6.50/GB
8 GB$37$4.62/GB
16 GB$78$4.88/GB
32 GB$136$4.25/GB

Surprisingly, yes. The 32 GB OCZ Rally2 flash drive offers the best price per gigabyte of storage. I actually didn't do the math before I purchased, thinking that the 16 GB one would be a better deal. Now I wish I had! I just noticed there's a $20 rebate on the 32 GB model, too, which makes it an even more outstanding deal.

I wasn't sure which 16 GB flash drive would be faster -- the Corsair Flash Voyager, or the OCZ Rally 2. So I ended up purchasing both.

Corsair Flash Voyager USB Flash drive   OCZ Rally 2 USB Flash drive

I formatted both of the drives with the default NTFS filesystem and did a bit of ad-hoc testing in Vista. You can find the raw ReadyBoost benchmark results if you know where to look in the event viewer.

16 GB Corsair
Flash Voyager
16 GB OCZ
Rally 2
3GB ISO Copy, Read26 MB/sec26 MB/sec
3GB ISO Copy, Write9 MB/sec10 MB/sec
Readyboost Random Read6,426 KB/sec6,434 KB/sec
Readyboost Random Write3,292 KB/sec4,695 KB/sec

In practice, the Rally is noticeably faster at writing, and a smidge faster at reading (not exposed here, but the chddspeed results confirm this). Not the results I expected based on reading the Xbit Labs review, but apparently there's quite a bit of variance for USB flash drives depending on the vagaries of manufacturing and what particular flash memory chips the manufacturer happened to be using that month.

The results are close enough that you may want to pick on ergonomics rather than performance. The Corsair drive has a chunky rubberized coating, which works well on a keychain (less jangly, more durable) but becomes annoying in a pocket or next to a narrow USB slot. I think I prefer the Rally's narrower metallic casing. Since it is a slightly better performer overall, the OCZ Rally 2 series gets my recommendation, if you're in the market.

One caution: if you do plan to use your USB flash drive to run applications or even operating systems, pay close attention to the random read and write speeds. Sequential throughput is a good overall baseline, but it's not the entire performance story. While typical portable storage usage does correlate well with sequential throughput, applications running on a USB flash drive are largely bounded by random access throughput.

You can never have enough storage on your keychain. Now if I can just figure out what else to put on mine...

Discussion

Whatever Happened to UI Consistency?

Although I rather like Windows Vista -- I think the amount of Vista nerd rage out there is completely unwarranted -- there are areas of Vista I find hugely disappointing. And for my money, nothing is more disapponting than the overall fit and finish of Vista, which is truly abysmal. It's arguably the worst of any operating system Microsoft has ever released.

What do I mean by fit and finish? Well, take a look at Long Zheng's Windows UI Taskforce examples. Vista is absolutely filled with bits of user interface that are inconsistent with the new Vista design.

Vista Add Fonts dialog

You're never more than two clicks away from some discontinuity or visual gaffe that zaps you right back into the seven year old Windows XP "experience". Or worse. Consider Chris Pirillo's observations on his Windows Vista beta 2 install:

Windows Calendar font and icon alignment are all wonky.

The Windows Media toolbar pop-up preview window is using Arial.

Safely Remove Hardware dialog is in Microsoft Sans Serif.

This goes on for about, oh, eleven pages. Granted, these comments refer to the beta, but the shipping version of Vista is every bit as schizophrenic in design. There's very little consistency.

It also seems every individual team at Microsoft has a profoundly different idea of what the user interface should look like, as Paul Thurrott notes:

And what's up with the glaringly inconsistent UI across Windows Vista and all of its applications? Some windows have menus, some don't, and some have hidden menus. Some have these new black toolbars, some don't. And so on. Why isn't there a team of people just working on consistency issues?

Aren't these trivial, nitpicky complaints? Yes. They are. And that's entirely the point. This little stuff matters.

If all those individual teams at Microsoft can't be bothered to follow the design conventions of their own operating system -- how can they possibly be building applications that I would actually want to use? In software, attention to detail is everything; all these glaring little oversights in Vista's user experience collectively add up to a huge vote of "no confidence" in the whole shebang. A mismatched font here, an ugly pixelated icon there, soon enough you feel that you're living in a neighborhood with an awful lot of broken windows.

If Microsoft's developers can't muster the basic level of craftsmanship necessary to make Vista's bundled applications consistently look and work the same as the rest of the operating system, how can users or third party developers be expected to give a damn about the user experience? Honestly, it's embarrassing.

John Gruber has been critical of Apple's minor UI inconsistencies in the past.

Consistency in and of itself has been a fundamental pillar of the Mac user experience from 1984 onward. But with Apple no longer leading the way, it's fading. "At least it's still more consistent than Windows" is not high praise.

That comment was made well before Vista was released. Nobody's perfect, but from what I've seen of OS X and Vista, I'd say Apple cares a lot more about consistency of user interface today. Microsoft has all but abdicated their responsibilities with Vista.

But the saddest part of this whole situation is that it doesn't have to be this way. Every major operating system is released alongside a set of design guidelines, guidebooks for developing applications that are consistent with the conventions and standard applications provided by the OS.

As a young developer, I remember eagerly paging through the early design guidelines for Windows 95 and the Mac OS. You can always do worse than following the well-worn paths the OS designers have conveniently laid out in front of you. Much, much, worse. And many have.

Of course, all design guidelines begin at home. These guides are only as good as the underlying operating systems they're based on, which are de facto reference implementations. It's hard to take Vista's design guidelines seriously, since Microsoft's own development teams clearly didn't.

If you're a software developer, please don't make this mistake. Understand the design guidelines for your platform -- and for God's sake, follow them!

For a great platform agnostic primer on the importance of UI consistency and design guidelines, look no further than GUI Bloopers 2.0.

GUI Bloopers 2.0

The original version of GUI Bloopers has been on my recommended reading list for years, and this greatly updated version was long overdue. There's a sample chapter (pdf) on the official book website if you'd like to get a sense of what the book is about. Jeff Johnson, the author, also provides an excellent companion reading list on his website, which also includes the Web Blooper of the Month archive.

Like everyone else, I'd prefer to use only the most beautifully designed applications. If you can't be beautiful, at least be consistent. Start with the basic level of consistency afforded by following the design guidelines of your chosen platform, and you might just avoid the "homely" and "ugly" end of the spectrum.

Discussion

Revisiting the Black Sunday Hack

One of the most impressive hacks I've ever read about has to be the Black Sunday kill. Since the original 2001 Slashdot article I read on this is 99.9% quote, I'm going to do the same. I can see why they quoted so extensively; it'd be difficult to improve on the unusually succinct, well written summary provided by Pat from Belch:

One of the original smart cards, entitled 'H' cards for Hughes, had design flaws which were discovered by the hacking community. These flaws enabled the extremely bright hacking community to reverse engineer their design, and to create smart card writers. The writers enabled the hackers to read and write to the smart card, and allowed them to change their subscription model to receive all the channels. Since the technology of satellite television is broadcast only, meaning you cannot send information TO the satellite, the system requires a phone line to communicate with DirecTV. The hackers could re-write their smart cards and receive all the channels, and unplug their phone lines leaving no way for DirecTV to track the abuse. DirecTV had built a mechanism into their system that allowed the updating of these smart cards through the satellite stream. Every receiver was designed to 'apply' these updates when it received them to the cards. DirecTV applied updates that looked for hacked cards, and then attempted to destroy the cards by writing updates that disabled them. The hacking community replied with yet another piece of hardware, an 'unlooper,' that repaired the damage. The hacker community then designed software that trojanized the card, and removed the capability of the receivers to update the card. DirecTV could only send updates to the cards, and then require the updates be present in order to receive video. Each month or so, DirecTV would send an update. 10 or 15 minutes later, the hacking community would update the software to work around the latest fixes. This was the status quo for almost two years. 'H' cards regularly sold on eBay for over $400.00. It was apparent that DirecTV had lost this battle, relegating DirecTV to hunting down Web sites that discussed their product and using their legal team to sue and intimidate them into submission.

Four months ago, however, DirecTV began sending several updates at a time, breaking their pattern. While the hacking community was able to bypass these batches, they did not understand the reasoning behind them. Never before had DirecTV sent 4 and 5 updates at a time, yet alone send these batches every week. Many postulated they were simply trying to annoy the community into submission. The updates contained useless pieces of computer code that were then required to be present on the card in order to receive the transmission. The hacking community accommodated this in their software, applying these updates in their hacking software. Not until the final batch of updates were sent through the stream did the hacking community understand DirecTV. Like a final piece of a puzzle allowing the entire picture, the final updates made all the useless bits of computer code join into a dynamic program, existing on the card itself. This dynamic program changed the entire way the older technology worked. In a masterful, planned, and orchestrated manner, DirecTV had updated the old and ailing technology. The hacking community responded, but cautiously, understanding that this new ability for DirecTV to apply more advanced logic in the receiver was a dangerous new weapon. It was still possible to bypass the protections and receive the programming, but DirecTV had not pulled the trigger of this new weapon.

Last Sunday night, at 8:30 pm est, DirecTV fired their new gun. One week before the Super Bowl, DirecTV launched a series of attacks against the hackers of their product. DirecTV sent programmatic code in the stream, using their new dynamic code ally, that hunted down hacked smart cards and destroyed them. The IRC DirecTV channels overflowed with thousands of people who had lost the ability to watch their stolen TV. The hacking community by and large lost not only their ability to watch TV, but the cards themselves were likely permanently destroyed. Some estimate that in one evening, 100,000 smart cards were destroyed, removing 98% of the hacking communities' ability to steal their signal. To add a little pizzazz to the operation, DirecTV personally "signed" the anti-hacker attack. The first 8 computer bytes of all hacked cards were rewritten to read "GAME OVER".

Nobody knew how the satellite companies had suddenly developed such smarts. Until now. A recent Wired article exposes Christopher Tarnovsky as the mind behind the epic Black Sunday Hack.

Among the countermeasures he says he created was one known among pirates as the "Black Sunday" kill -- an elaborate scheme that destroyed tens of thousands of pirate DirecTV cards a week before Super Bowl Sunday in 2001.

Instead of being delivered all at once like other measures, the Black Sunday attack code was sent to pirate cards in about five dozen parts over the course of two months, like a tank transported piece by piece to a battlefield to be assembled in the field. "They never expected us to do this," Tarnovsky says.

The kill didn't last long before pirates found a way to jump-start the cards. But it holds an enduring position in pirate lore; for the first time, they could see a cunning mind at work on the other side.

It's fascinating to finally hear the Black Sunday kill described so intimately from the inside. It's a gripping tale of high stakes programming, a life of electronic warfare with millions of dollars at risk on both sides. I've never been a satellite television subscriber, but apparently the war rages on even today -- at least according to the Wikipedia entry on pirate decryption.

Discussion