Coding Horror

programming and human factors

Rainbow Hash Cracking

The multi-platform password cracker Ophcrack is incredibly fast. How fast? It can crack the password "Fgpyyih804423" in 160 seconds. Most people would consider that password fairly secure. The Microsoft password strength checker rates it "strong". The Geekwisdom password strength meter rates it "mediocre".

Why is Ophcrack so fast? Because it uses Rainbow Tables. No, not the kind of rainbows I have as my desktop background.

A screenshot of my windows desktop. Oh Hello Kitty, I've fallen in love with you all over again.

Although those are beautiful, too.

To understand how rainbow tables work, you first have to understand how passwords are stored on computers, whether on your own desktop, or on a remote web server somewhere.

Passwords are never stored in plaintext. At least they shouldn't be, unless you're building the world's most insecure system using the world's most naive programmers. Instead, passwords are stored as the output of a hash function. Hashes are one-way operations. Even if an attacker gained access to the hashed version of your password, it's not possible to reconstitute the password from the hash value alone.

But it is possible to attack the hashed value of your password using rainbow tables: enormous, pre-computed hash values for every possible combination of characters. An attacking PC could certainly calculate all these hashes on the fly, but taking advantage of a massive table of pre-computed hash values enables the attack to proceed several orders of magnitude faster-- assuming the attacking machine has enough RAM to store the entire table (or at least most of it) in memory. It's a classic time-memory tradeoff, exactly the sort of cheating shortcut you'd expect a black hat attacker to take.

How enormous are rainbow tables? The installation dialog for Ophcrack should give you an idea:

rainbow-hash-table-sizes

It takes a long time to generate these massive rainbow tables, but once they're out there, every attacking computer can leverage those tables to make their attacks on hashed passwords that much more potent.

The smallest rainbow table available is the basic alphanumeric one, and even it is 388 megabytes. That's the default table you get with the Ophcrack bootable ISO. Even that small-ish table is remarkably effective. I used it to attack some passwords I set up in a Windows XP virtual machine with the following results:

found? seconds
Password1! 700
Fgpyyih804423 yes 159
Fgpyyih80442% 700
saMejus9 yes 140
thequickbrownfoxjumpsoverthelazydog 700

You wouldn't expect this rainbow table to work on the passwords with non-alphanumeric characters (%&^$# and the like) because the table doesn't contain those characters. You'll also note that that passphrases, which I am a big fan of, are immune to this technique due to their length. But then again, this attack covered 99.9% of all possible 14 character alphanumeric passwords in 11 minutes, and that was with the smallest of the available rainbow tables. We could do better by using larger, more complete rainbow tables. The Ophcrack documentation describes the differences between the available rainbow tables it uses:

Alphanumeric 10k 388 MB Contains the LanManager hashes of 99.9% of all alphanumerical passwords. These are passwords made of mixed case letters and numbers (about 80 billion hashes). Because the LanManager hash cuts passwords into two pieces of 7 characters, passwords of length 1 to 14 can be cracked with this table set. Since the LanManager hash is also not case sensitive, the 80 billion hashes in this table set corresponds to 12 septillion (or 283) passwords.
Alphanumeric 5k 720 MB Contains the LanManager hashes of 99.9% of all alphanumerical passwords. However, because the tables are twice as large, cracking is about four times faster if you have at least 1 GB of RAM.
Extended 7.5 GB Contains the LanManager hashes of 96% of all passwords made of up to 14 mixed case letters, numbers and the following 33 special characters: !"#$%&'()*+,-./:;<=>?@[]^_`{|} ~. There are about 7 trillion hashes in this table set covering 5 octillion (or 292) passwords.
NT 8.5 GB You can use this table set to crack the NT hashes on machines where the LanManager hash has been disabled. The set contains 99.0% of the hashes of the passwords made of the following characters:
  • up to 6 mixed case letters, numbers and 33 special characters (same as above)
  • 7 mixed-case letters and numbers
  • 8 lower-case letters and numbers

There are 7 trillion hashes in this table, corresponding to 7 trillion passwords (the NT hash does not suffer from the weaknesses of the LanManager hash).

Note that all rainbow tables have specific lengths and character sets they work in. Passwords that are too long, or contain a character not in the table's character set, are completely immune to attack from that rainbow table.

Unfortunately, Windows servers are particularly vulnerable to rainbow table attack, due to unforgivably weak legacy Lan Manager hashes. I'm stunned that the legacy Lan Manager support "feature" is still enabled by default in Windows Server 2003. It's highly advisable that you disable Lan Manager hashes, particularly on Windows servers which happen to store domain credentials for every single user. It'd be an awful shame to inconvenience all your Windows 98 users, but I think the increase in security is worth it.

I read that Windows Server 2008 will finally kill off LM hashes when it's released next year. Windows Vista already removed support for these obsolete hashes on the desktop. Running OphCrack on my Vista box results in this dialog:

All LM hashes are empty. Please use NT hash tables to crack the remaining hashes.

I'd love to, but I can't find a reliable source for the 8.5 GB rainbow table of NT hashes that I need to proceed.

The Ophcrack tool isn't very flexible. It doesn't allow you to generate your own rainbow tables. For that, you'll need to use the Project Rainbow Crack tools, which can be used to attack almost any character set and any hashing algorithm. But beware. There's a reason rainbow table attacks have only emerged recently, as the price of 2 to 4 gigabytes of memory in a desktop machine have approached realistic levels. When I said massive, I meant it. Here are some generated rainbow table sizes for the more secure NT hash:

Character Set Length Table Size
ABCDEFGHIJKLMNOPQRSTUVWXYZ 14 0.6 GB
ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 14 3 GB
ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-_+= 14 24 GB
ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-_+=~`[]{}|:;"'<>,.?/ 14 64 GB

A rainbow table attack is usually overkill for a desktop machine. If hackers have physical access to the machine, security is irrelevant. That's rule number 3 in the 10 Immutable Laws of Computer Security. There are any number of tools that can reset passwords given physical access to the machine.

But when a remote hacker obtains a large list of hashed passwords from a server or database, we're in trouble. There's significant risk from a rainbow table attack. That's why you should never rely on hashes alone-- always add some salt to your hash so the resulting hash values are unique. Salting a hash sounds complicated (and vaguely delicious), but it's quite simple. You prefix a unique value to the password before hashing it:

hash = md5('deliciously-salty-' + password)

If you've salted your password hashes, an attacker can't use a rainbow table attack against you-- the hash results from "password" and "deliciously-salty-password" won't match. Unless your hacker somehow knows that all your hashes are "delicously-salty-" ones. Even then, he or she would have to generate a custom rainbow table specifically for you.

UPDATE: Please read Thomas Ptacek's excellent and informative response to this post. It goes into much more detal about the nuts and bolts of password hashing. Unlike me, Thomas is a real security expert.

Discussion

The Problem With Tabbed Interfaces

Cyrus Najmabadi* hates tabs in web browsers:

Ok, I seriously don't get tabs on Windows. Hell, I don't get tabs on OSX either. In the latter there's a great system called Expos, and in the former the taskbar does the job. Once I start using tabs, things go all to hell. On OSX, I can't tell which FireFox/Safari window has the tab I want (since it's too small). In Windows I find myself scanning the taskbar for a site I was looking at, but I can't find it because the taskbar only lists the currently active tab. This makes it so difficult to actually find the site I want and it ends up being far slower than just having a window available for each site.

Initially I disagreed with Cyrus. However broken tabbed browsing may be, it's still a better solution than any of the existing alternatives. For example, Microsoft's own flagship Office suite, even today, suffers from some highly inconsistent, bizarre pseudo-MDI behavior for multiple documents. I'll take simple, reliable tabs over oddball MDI any day. No contest.

Lately I'm starting to come around to Cyrus' way of thinking. I don't hate tabbed interfaces-- yet-- but I definitely see what Cyrus was talking about. Tabs are increasingly the source of two aggravating mistakes for me:

  1. I inadvertently open multiple copies of a web site, because I can't see that I already had that web site open in an obscured tab of an existing browser window.
  2. I accidentally close a browser window containing information that I needed, because the information was in an obscured tab of that particular browser window.

In a tabbed interface, it's difficult to see anything except the active tab at any given time. Tabbed interfaces obscure as much as they organize. Tabs are great in moderation, but once they become a keystone navigational technique of your core applications, something peculiar happens. As Cyrus so aptly said, "once I start using tabs [extensively], things go all to hell."

Allow me to illustrate with an example. Let's say I want to check my GMail account, which I do frequently throughout the day. It's likely I already have GMail running, somewhere, so job #1 is to find it.

First, I scan the text in the taskbars. I use UltraMon, so each of my three monitors has its own distinct taskbar, summarizing every window on that monitor.

taskbar

But I don't see the word "GMail" in any of the three taskbars.

Next, I press ALT+TAB-- the poor man's Expos-- and scan through thumbnails of all the windows I have open. Do I see anything that looks like GMail?

desktop-alt-tab-small

No. I don't see the distinctive look of the GMail user interface in any of those thumbnails. I've actually enlarged the Alt+Tab thumbnail size via registry tweaks, so this is as good as it gets for visibility. Squinting doesn't help.

The Alt+Tab dialog only uses the primary monitor, even though I have a three-monitor configuration. But we can harness the entire screen area of all the monitors if we install the amazing Switcher. I have Switcher mapped to Windows+Tab, replacing the incredibly lame Flip3D. This is functionally identical to the OSX Expose that Cyrus mentioned. Now can I find GMail?

desktop-switcher-small

No. Even with the additional resolution of Switcher across all three monitors, I don't see GMail in any of the windows. At this point I would usually launch the URL using the keyboard entry area of the Vista Start Menu. I could use the fancy Start++ add-in to make this easier, but the vanilla Vista menu works well enough.

But wait! I actually had GMail open already. I've made a mistake.. again. I have two copies of GMail running now. Did you see it? Here, let me show you:

desktop-switcher-closeup.png

That muddy, tiny little morass of pixels is the only visual indication that I already had GMail running as a tab in an existing browser instance. It's not in the taskbar, it's hardly visible at all in the small Alt+Tab screenshot, and you'd need the Six Million Dollar Man's bionic eye to see that barely visible tab in Expose, I mean, Switcher.

The depressing thing is that it's usually faster to mindlessly launch a new browser than it is to go through this tedious routine of playing Where's Waldo with (n) browsers and (n) tabs. And that's what I often do. But it bothers me.

Let me be very clear-- I like tabs. I think they're a far better option than the terrible MDI-alike alternatives. But I also think tabbed interfaces present some pretty severe navigational problems of their own. In the above example, if GMail had been in its own browser window, I could have found it instantly by looking in the taskbar, or at worst, by visually selecting it from even a smallish thumbnail image. Because GMail was in a tab, I wasted my time trying to find it, and I wasted even more time needlessly launching another browser. And this isn't an isolated incident. This happens to me every day. More times than I'd care to admit.

So how can we fix this? How can we integrate tabs with the existing navigational features of the operating system, such as the taskbar, and Expose? I keep coming back to search as the dominant computing metaphor. The only thing I can think of is a plain-text search facility where I type "Gmail", and the OS would automatically highlight that tab (or window) and bring it to the front. That presupposes a very high level of integration between the application tabs and the operating system, however.

Despite the undeniable convenience of tabs for grouping and organizing related topics together in a single browser instance, I feel like tabs create as many problems for me as they solve. I wish I could "tear off" tabs into standalone windows on demand, too. That might be a reasonable workaround in the meantime.

* whatever happened to Cyrus? The last entry on his blog is two years old, and I can't find hide nor hair of him via internet searches. It's a shame, because I thoroughly enjoyed his blog.

Discussion

The Peanut Butter Theory of User Interface Design

Task-Centered User Interface Design is a 1993 book delivered in digital shareware form, and also available as a PDF. Although it's almost fifteen years old, it's still highly relevant-- a testament to the timelessness of studying human interface design principles. It was written by Clayton Lewis and John Rieman, who were at the University of Colorado at the time. It looks like Rieman works on usability for Nokia now, but Lewis is still teaching HCI in the UC Boulder computer science department.

Task Centered User-Interface Design

The book presages so many things we now accept as standard in the usability community: paper drafts, user personas, effort metrics, prototyping, testing, and iteration. It's solid advice written well. And it opens with this crucial warning:

We've designed this book to be most useful for people who are actually developing user interfaces. That's in contrast to the full-time interface professionals who do research and evaluation in large corporations. We strongly believe that effective interactive systems require a commitment and an understanding throughout the entire development process. It just won't work to build a complete system and then, in the final stages of development, spread the interface over it like peanut butter.

Usability is something everybody on your team-- not just the designers-- should be thinking about throughout the lifecyle of your project. If your development team suffers from the delusional peanut butter theory of user interface design, perhaps they should find time to read through Task-Centered User Interface Design, too.

Discussion

Online Newspapers, Offline

One of the premium features of the New York Times website is the Windows Reader. It's free if you subscribe to home delivery of the paper, otherwise it's $14.95 per month.

New York Times Reader screenshot

One of the key attractions of the Times Reader is that it lets you read the newspaper offline. The application runs silently in the background, caching up 7 days worth of news, which you can browse through at your leisure-- with or without an internet connection. And for that, it's great. But given the increasing prevalence of internet connectivity everywhere, through cellular networks as well as WiFi, does offline mode still matter?

In many ways, the Times Reader is a superior newspaper reading experience. It supports dynamic, scalable full page layouts, unlike the narrow column of content we're stuck with in the web browser. And the typography features (through Windows Presentation Foundation) are desktop publishing quality. The "pages" are artfully presented, free of webjunk, and transitions between the articles are all beautifully animated, iPhone style. All of this is nearly impossible to do in a standard web browser.

I encourage people to try out the Times Reader. If nothing else, it's a window into some of the missed opportunities of viewing the online world exclusively through the lens of a traditional web browser.

But after using the Times Reader for a few days, I can't shake the feeling that the newspaper reading experience it delivers is still inferior to the web browser. Despite the many technical merits (and the offline mode), dividing the browsing experience between two different mediums is not a good idea.

New York Times website screenshot

The Times Reader version of the paper lacks much of the dynamic content offered on the web page:

  • opinion column links
  • market summary numbers
  • articles with embedded flash audio
  • articles with embedded flash video
  • search function
  • personalization
  • top (n) most popular / most emailed stories

I miss this stuff. I always jumped directly to the most popular stories, and I can't do that in the Times Reader. I also find the the constrained view of the reader unnatural. I prefer the large scrolling content area of the web browser. I can use my mouse wheel to scroll to different sections in the reader, but it's alien.

The web browser is clearly the design focus for the New York Times. Perhaps that's the way it should be, as that's how most of the world will experience the paper online. In comparison, the Times Reader feels like a B-movie version of the genericized RSS content with better special effects. It just doesn't get the same attention as the home page.

Maybe this is a classic case of Don't Repeat Yourself. The Times Reader is a great effort. It's certainly pretty to look at. But I think the Times would be better served by focusing all their effort on delivering an outstanding web experience-- and, if necessary, offering programs that cache sections of the website for offline browsing.

Discussion

Keeping The Menu Simple

In-N-Out Burger is a fast food institution here in California. Part of their appeal, I think, is their radically simplified menu.

In-n-Out Burger menu

Instead of forcing customers to process a complex menu with a hundred choices, In-N-Out got real and pared it down to what really matters: a burger, fries, and a drink. It's a limited experience, but it's also a tightly focused one. The menu may be small, but you can still customize it -- for advanced In-N-Out customers, there's the secret menu.

Most software lacks the discipline to present the tightly controlled user experience of the In-N-Out menu. Instead, we befuddle users with the software equivalent of a fourteen-course restaurant menu. Software developers lapse into complexity by default, because it's the path of least resistance.

notetab options dialog

Why can't we build software the In-N-Out way, and keep the menu simple? Stop trying to do everything. Don't make users think. Focus on doing a few things exceptionally well, and leave the giant, confusing menu of options for your competitors.

Discussion