Coding Horror

It's been almost a year since I covered The State of Solid State Hard Drives. Not a heck of a lot has changed, but the topic is still worth revisiting, because if you care at all about how your computer performs, solid state hard drives remain a life changing experience. Here's why:

1. A solid state hard drive is easily the best and most obvious performance upgrade you can make on any computer for a given amount of money. Unless your computer is absolute crap to start with.
2. The practical minimum solid state hard drive size, 128 GB, has modestly declined in price -- from about $350 to about $250.

(yes, you can get by with 64 GB, but at least with my Windows installs I find that I have to think about disk space with 64 GB, whereas with 128 GB I don't have to worry -- ever. Don't make me think, man!)

The rest of the components inside your PC are downright boring in comparison. CPUs? All stupid fast at any price, with more cores and gigahertz than you'll ever need unless you're one of those freaks who does nothing but raytracing and video transcoding all day long. Memory? Dirt cheap, and average users won't need more than 2 gigabytes of the stuff in practical use, which at the current going rate for DDR3 is less than 50 bucks.

Thanks to the neverending march of Moore's Law, PCs are becoming speedy at any price these days. It's difficult for me to muster any enthusiasm for the latest Intel CPU updates when I spend almost zero real world time waiting for the CPU to do anything on my computer. I guess it's true: absolute power corrupts absolutely.

But hard drives, now, there's where you can pay a bit more and see a groundbreaking, generational leap in performance worthy of that investment -- as long as you skip over the old and busted spinning rust hard drives, and choose a newfangled solid state hard drive.

The current king of the hill seems to be the Crucial RealSSD C300.

Pretty sexy, right? Oh, who am I kidding, it's a boring slab of aluminum and silicon. But like all truly sexy things, what turns me on is the part I can't see -- the sexy, sexy, sexy performance inside this baby.

See those bars dragging down the bottom of this graph? All spinning rust. Heck, even the SSD I recommended last year is only middle of the pack here. AnandTech concurs -- the Crucial RealSSD C300 is top dog, at least for now.

(Be careful, though, that your operating system supports the SSD TRIM command, otherwise you'll suffer severe performance degradation over time with almost any SSD. Operating systems earlier than Windows 7 and the latest, greatest Linux kernel should beware -- and, shockingly, OSX still doesn't support TRIM!)

Where it gets trickier, though, is when you need more than 128 GB of storage, or when you are limited to one 2.5" hard drive -- like in a laptop. In that case, ideally you'd like something that has the speed of a solid state hard drive, but the capacity (and economical price per gigabyte) of a traditional magnetic platter hard drive. You might say … a hybrid hard drive, the kind I was dreaming about back in 2006 …

After all this analysis, it's clear to me that traditional hard drives and flash memory are quite complimentary; they're strong in different areas. But flash drives are the future. They will definitely replace hard drives in almost all low end and low power devices-- and future high performance hard drives will need to have a substantial chunk of flash memory on board to stay competitive.

I had always been disappointed that hybrid hard drives, drives that combine both flash memory and traditional magnetic platters, never came to fruition. It was either traditional or SSD and nothing in between. It seemed like such an obvious "best of both worlds" scenario to me. But I recently discovered that decent hybrid drives do finally exist -- though in a small and mostly unheralded way.

Seagate's Momentus XT takes a totally respectable 2.5", 7200 RPM drive with a 32 megabyte buffer and combines it with 4 gigabytes of flash memory. The result is the exactly what I had always hoped:

Seagate's Momentus XT should become the standard hard drive in any notebook shipped. The biggest problem I have with using any brand new machine, regardless of how fast it is, is that it never feels fast because it usually has a HDD and not an SSD. While the Momentus XT isn't quite as fast as an SSD, it's a significant improvement over the mechanical drives found in notebooks today.

In many cases the Momentus XT performs like a VelociRaptor, but in a lower power, quieter package. The impact of adding just a small amount of SLC NAND is tremendous. The potential for hybrid drives continues to be huge; what Seagate has shown here is that with a minimal amount of NAND you can achieve some tremendous performance gains.

And the best part? 500 gigabytes of near-SSD performance for $130! Or, if that's too spendy, how about 320 gigabytes of near-SSD performance for $99?

I've ordered a few of these drives to upgrade my laptops and home theater PC. Sure, I'll invest in a SSD for my beastly desktop, but I can't justify $300 to put a SSD in a laptop I spent all of $800 on, or a home theater PC that set me back a mere $500. But a hundred bucks for near-SSD performance and decent capacity? Sign me up. And hard drive vendors: although I love SSDs to death, please keep these improved hybrid drives coming, too, please!

Discussion

When it comes to running Stack Overflow, the company, I take all my business advice from one person, and one person alone: Curtis Armstrong.

More specifically, Curtis Armstrong as Charles De Mar from the 1985 absurdist teen comedy classic, Better Off Dead. When asked for advice on how to ski down a particularly treacherous mountain, he replied:

Go that way, really fast. If something gets in your way … turn.

(I recommend watching the entire movie. It's brilliant in ways I can't possibly explain here.)

In the five months since we announced our funding, we have …

• Built an international team
• Created an entirely new open, democratic process for creating Q&A sites at Area 51
• Launched ~24 new community-driven Stack Exchange network sites
• Implemented per-site meta discussion and per-site real time chat
• Rolled out new versions of Careers and Jobs
• Built and open-sourced a tool for exploring and sharing all our creative commons data in the Stack Exchange Data Explorer
• Finalized V1 of the Stack Exchange API, for building your own apps against our Q&A platform

… and honestly, I'm a little worried we're still not going fast enough.

There are any number of Stack Overflow engine clones out there already, and I say more power to 'em. I'm proud to have something worth copying. If we do nothing else except help lead the world away from the ancient, creaky, horribly broken bulletin board model of phpBB and vBulletin – attempting to get information out of those things is like panning for gold in a neverending river of sewage – then that is more than I could have ever hoped for.

It is our stated goal as a company to live in harmony with the web, by only doing things that we believe make the internet better, at least in some small way. No, seriously. It's in writing and everything, I swear! We're not here to subvert or own anyone or anything. We just love community, and we love getting great answers to our questions. So if something gets in our way while doing that, well, we're not gonna fight you. We'll just turn. And keep going forward, really fast. Which is why those clones better move quick if they want to keep up with us.

While I like to think that having Charles De Mar as a business advisor is unique to our company, the idea that speed is important is hardly original to us. For example, certain Google projects also appear to understand Boyd's Law of Iteration.

Boyd decided that the primary determinant to winning dogfights was not observing, orienting, planning, or acting better. The primary determinant to winning dogfights was observing, orienting, planning, and acting faster. In other words, how quickly one could iterate. Speed of iteration, Boyd suggested, beats quality of iteration.

Speed of iteration – the Google Chrome project has it.

Chrome was a completely respectable browser in V1 and V2. The entire project has moved forward so fast that it now is, at least in my humble opinion, the best browser on the planet. Google went from nothing, no web browser at all, to best-of-breed in under two years. Meanwhile, Internet Explorer took longer than the entire development period of Chrome to go from version 7 to version 8. And by the time Internet Explorer 9 ships – even though it's actually looking like Microsoft's best, most competent technical upgrade of the browser yet – it will be completely outclassed at launch by both Firefox and Chrome.

The Google Android project is another example. Android doesn't have to be better than the iPhone (and it most definitely isn't; it's been mediocre at best until recent versions). They just need to be faster at improving. Google is pushing out Froyos and Gingerbreads and Honeycombs with incredible, breakneck speed. Yes, Apple has indisputably better taste – and an impeccably controlled experience. But at their current rate of progress, they'll be playing second or third fiddle to Google in the mobile space inside a few years. It's inevitable.

So, until further notice, we'll be following the same strategy as the Android and Chrome teams. We're going to go that way, really fast. And if something gets in our way, we'll turn.

Kyle Brandt, a system administrator, asks Should Developers have Access to Production?

A question that comes up again and again in web development companies is:

"Should the developers have access to the production environment, and if they do, to what extent?"

My view on this is that as a whole they should have limited access to production. A little disclaimer before I attempt to justify this view is that this standpoint is in no way based on the perceived quality or attitude of the developers -- so please don't take it this way.

This is a tricky one for me to answer, because, well, I'm a developer. More specifically, I'm one of the developers Kyle is referring to. How do I know that? Because Kyle works for our company, Stack Overflow Internet Services Incorporated©®™. And Kyle is a great system administrator. How do I know that? Two reasons:

1. He's one of the top Server Fault users.
2. He had the audacity to write about this issue on the Server Fault blog.

From my perspective, the whole point of the company is to talk about what we're doing. Getting things done is important, of course, but we have to stop occasionally to write up what we're doing, how we're doing it, and why we're even doing it in the first place -- including all our doubts and misgivings and concerns. If we don't, we're cheating ourselves, and you guys, out of something much deeper. Yes, writing about what we're doing and explaining it to the community helps us focus. It lets our peers give us feedback. But most importantly of all, it lets anyone have the opportunity to learn from our many, many mistakes … and who knows, perhaps even the occasional success.

That's basically the entire philosophy behind our Stack Exchange Q&A network, too. Let's all talk about this stuff in public, so that we can teach each other how to get better at whatever the heck it is we love to do.

(Sometimes I get the feeling this idea makes my co-founder nervous, which I continually struggle to understand. If we don't walk the walk, why are we even doing this? But I digress.)

The saga of System Administrators versus Programmers is not a new one; I don't think I've ever worked at any company where these two factions weren't continually battling with each other in some form. It's truly an epic struggle, but to understand it, you have to appreciate that both System Administrators and Programmers have different, and perhaps complementary, supernatural powers.

Programmers are like vampires. They're frequently up all night, paler than death itself, and generally afraid of being exposed to daylight. Oh yes, and they tend think of themselves (or at least their code) as immortal.

System Administrators are like werewolves. They may look outwardly ordinary, but are incredibly strong, mostly invulnerable to stuff that would kill regular people -- and prone to strange transformations during a moon "outage".

Let me be very clear that just as Kyle respects programmers, I have a deep respect for system administrators:

Although there is certainly some crossover, we believe that the programming community and the IT/sysadmin community are different beasts. Just because you're a hotshot programmer doesn't mean you have mastered networking and server configuration. And I've met a few sysadmins who could script circles around my code. That's why Server Fault gets its own domain, user profiles, and reputation system.

Different "beasts" indeed.

Anyway, if you're looking for a one size fits all answer to the question of how much access programmers should have to production environments, I'm sorry, I can't give you one. Every company is different, every team is different. I know, it's a sucky answer, but it depends.

However, as anyone who has watched the latest season of True Blood (or, God help us all, the Twilight Eclipse movie) can attest, there are ways for vampires and werewolves to work together. In a healthy team, everyone feels their abilities are being used and not squandered.

On our team, we're all fair-to-middling sysadmins. But there are a million things to do, and having a professional sysadmin means we can focus on the programming while the networking, hardware, and operational stuff gets a whole lot more TLC and far better (read: non-hacky) processes put in place. We're happy to refocus our efforts on what we're expert at, and let Kyle put his skills to work in areas that he's expert at. Now, that said, we don't want to cede full access to the production servers -- but there's a happy middle ground where our access becomes infrequent and minor over time, except in the hopefully rare event of an all hands on deck emergency.

The art of managing vampires and werewolves, I think, is to ensure that they spend their time not fighting amongst themselves, but instead, using those supernatural powers together to achieve a common goal they could not otherwise. In my experience, when programmers and system administrators fight, it's because they're bored. You haven't given them a sufficiently daunting task, one that requires the full combined use of their unique skills to achieve.

Remember, it's not vampires versus werewolves. It's vampires and werewolves.

Discussion

Like any self-respecting geek, I'm mostly an indoor enthusiast.

But on those unfortunate occasions when I am compelled -- for reasons entirely beyond my control -- to leave the house, I do so fully armed with my crucial utility belt items. Yes, you heard me, I transform from the geeky Bruce Wayne to the gosh-darned Batman!

At least, that's how I like to think of it.

I've been talking about this every-day carry stuff for quite a while now. The 2010 edition of my personal utility belt is mostly subtle tweaks, but I daresay it's the best one yet.

The art of every-day carry must go on. What you see here is the contents of my pocket:

1. Patriot 32 GB USB flash drive
   
   Now you can have a whole freakin' hard drive worth of files in your pocket. Just in case, you know, you have an emergency need to upload a virus to an alien mothership, or something. Beware the many cheap, slow USB flash drives out there; this one is a real gem. It's inexpensive, and per my measurements, about as fast as they get. This is important because the larger the flash drive, the more important speed becomes. Hard to believe I've gone from carrying a 512 megabyte flash drive in 2005 to a 32 gigabyte flashdrive in 2010.
   
   
2. Leatherman Squirt P4
   
   Ounce for ounce, nothing beats the utility of the Leatherman Squirt. This time I opted for the plier (P) version instead of the scissors (S), and after seeing how much more generally useful the pliers are, I am now a little ashamed to admit I ever carried the wussy scissors version. Pliers all the way, baby. And yes, that is a Pulp Fiction joke you see on it.
   
   
3. Fenix mini AAA LED flashlight model LD01R2
   
   Since I've been carrying them in 2005, the average LED flashlight has gone from bright, to very bright, to amazingly bright, to ridiculously blinding laser-like bright. It's scary how bright these fancy milled aluminum AAA LED flashlights get now. What I like about this one is that it lets you trade off stupid-brightness for something practical, like greater runtime: you can twist the top to switch levels: 9 Lumens for 11 hours, 28 Lumens for 3.5 hours, or 85 Lumens for 1 hour.
   
   
4. Small Nite Ize s-biner
   
   These little nite-ize carabiners are awesome for quick attachment and detachment of your EDC items, but I'll warn you: resist the urge to put everything you carry on a carabiner, because if you do, the weight and "jangliness" goes up a lot -- and this way lies madness. Consider how many items you actually remove from your keychain regularly. For me, the only item I frequently removed to work with was the Squirt, so that's the only one I put on a carabiner.

Rest assured, everything here is carefully selected with the appropriate levels of monomaniacal attention to detail. For this weight and size, I don't think you can do better. (And don't think I've forgotten about optimizing my wallet, either. Oh no. Quite the contrary.)

However, I have to add a special category this year for the other must-have EDC utility belt item: the smartphone. What self-respecting superhero would leave the house these days without their smartphone? I'm not religious about it, but I use and rather like the iPhone 4, and I'm continually amazed how many things it does that I used to carry separate items for:

• cell phone (obviously)
• "Nintendo DSwhatever" for portable gaming
• GPS
• point and shoot digital camera
• near-desktop quality mobile web browser and email client
• mp3 player with speakers
• audio and hi-def video recorder
• DVD player
• ebook reader
• watch, alarm
• emergency flashlight (via front facing LED flash control)
• scanner
• level and ruler

Smartphones really are the ultimate gadget. The list of functions is already enormous, and I'm sure I'm leaving out a few other things that you can do with a modern smartphone.

In a pinch, I could conceivably drop the AAA LED flashlight and the USB flash drive from my EDC kit and substitute the smartphone. Not exactly, mind you, but it's getting closer every year. At this rate, Apple could introduce a flip-out blade on the iPhone 7 and reduce my entire EDC kit to one item.

Anyway, that's what's on my utility belt in 2010. What's on yours?

Discussion

On a recent airplane flight, I happened to catch the movie Groundhog Day. Again.

If you aren't familiar with this classic film, the premise is simple: Bill Murray, somehow, gets stuck reliving the same day over and over.

It's been at least 5 years since I've seen Groundhog Day. I don't know if it's my advanced age, or what, but it really struck me on this particular viewing: this is no comedy. There's a veneer of broad comedy, yes, but lurking just under that veneer is a deep, dark existential conundrum.

It might be amusing to relive the same day a few times, maybe even a few dozen times. But an entire year of the same day – an entire decade of the same day – everything happening in precisely, exactly the same way? My back of the envelope calculation easily ran to a decade. But I was wrong. The director, Harold Ramis thinks it was actually 30 or 40 years.

I think the 10-year estimate is too short. It takes at least 10 years to get good at anything, and alloting for the down time and misguided years [Phil] spent, it had to be more like 30 or 40 years [spent reliving the same day].

We only see bits and pieces of the full experience in the movie, but this time my mind began filling in the gaps. Repeating the same day for decades plays to our secret collective fear that our lives are irrelevant and ultimately pointless. None of our actions – even suicide, in endless grisly permutations – ever change anything. What's the point? Why bother? How many of us are trapped in here, and how can we escape?

This is some dark, scary stuff when you really think about it.

You want a prediction about the weather, you're asking the wrong Phil.

I'll give you a winter prediction.
It's gonna be cold,
it's gonna be gray,
and it's gonna last you for the rest of your life.

Comedy, my ass. I wanted to cry.

But there is a way out: redemption through repetition. If you have to watch Groundhog Day a few times to appreciate it, you're not alone. Indeed, that seems to be the whole point. Just ask Roger Ebert:

"Groundhog Day" is a film that finds its note and purpose so precisely that its genius may not be immediately noticeable. It unfolds so inevitably, is so entertaining, so apparently effortless, that you have to stand back and slap yourself before you see how good it really is.

Certainly I underrated it in my original review; I enjoyed it so easily that I was seduced into cheerful moderation. But there are a few films, and this is one of them, that burrow into our memories and become reference points. When you find yourself needing the phrase This is like "Groundhog Day" to explain how you feel, a movie has accomplished something.

There's something delightfully Ouroboros about the epiphanies and layered revelations in repeated viewings of a movie that is itself about (nearly) endless repetition.

Which, naturally, brings me to A/B testing. That's what Phil spends most of those thirty years doing. He spends it pursuing a woman, technically, but it's how he does it that is interesting:

Rita: This whole day has just been one long setup.

Phil: No it hasn't.

Rita: And I hate fudge! Yuck!

Phil: [making a mental list] No white chocolate. No fudge.

Rita: What are you doing? Are you making some kind of list or something? Did you call up my friends and ask what I like and what I don't like? Is this what love is for you?

Phil: No, this is real. This is love.

Rita: Stop saying that! You must be crazy.

Phil doesn't just go on one date with Rita, he goes on thousands of dates. During each date, he makes note of what she likes and responds to, and drops everything she doesn't. At the end he arrives at – quite literally – the perfect date. Everything that happens is the most ideal, most desirable version of all possible outcomes on that date on that particular day. Such are the luxuries afforded to a man repeating the same day forever.

This is the purest form of A/B testing imaginable. Given two choices, pick the one that "wins", and keep repeating this ad infinitum until you arrive at the ultimate, most scientifically desirable choice. Your marketing weasels would probably collapse in an ecstatic, religious fervor if they could achieve anything even remotely close to the level of perfect A/B testing depicted in Groundhog Day.

But at the end of this perfect date, something impossible happens: Rita rejects Phil.

Phil wasn't making these choices because he honestly believed in them. He was making these choices because he wanted a specific outcome – winning over Rita – and the experimental data told him which path he should take. Although the date was technically perfect, it didn't ring true to Rita, and that made all the difference.

That's the problem with A/B testing. It's empty. It has no feeling, no empathy, and at worst, it's dishonest. As my friend Nathan Bowers said:

A/B testing is like sandpaper. You can use it to smooth out details, but you can't actually create anything with it.

The next time you reach for A/B testing tools, remember what happened to Phil. You can achieve a shallow local maximum with A/B testing – but you'll never win hearts and minds. If you, or anyone on your team, is still having trouble figuring that out, well, the solution is simple: just watch Groundhog Day again.