Coding Horror

programming and human factors

App-pocalypse Now

I'm getting pretty sick of being nagged to install your damn apps.

This-website-has-an-ipad-app

XKCD helpfully translates:

Xkcd-download-our-app

Yeah, there are smart app banners, which are marginally less annoying, but it's amazing how quickly we went from "Cool! Phone apps that finally don't suck!" to this sad, eye rolling, oh-great-of-course-you-have-an-app-too state of affairs.

Four years, give or take a few months, if you were counting. So what happened?

Millions of pointless apps

Your platform now has a million apps? Amazing! Wonderful! What they don't tell you is that 99% of them are awful junk that nobody would ever want.

Let's start with the basics. How do you know which apps you need? How do you get them installed? How do you keep them updated? How many apps can you reasonably keep track of on a phone? On a tablet? Just the home screen? A few screens? A dozen screens? When you have millions of apps out there, this rapidly becomes less of a "slap a few icons on the page" problem and more of a search problem like the greater web. My son's iPad has more than 10 pages of apps now, we don't even bother with the pretense of scrolling through pages of icons, we just go straight to search every time.

Walledgarden-cover

The more apps out there, the more the app stores are clogged with mediocre junk, the more the overall noise level keeps going up, which leads directly to this profligate nagging. Companies keep asking how can we get people to find and install our amazing app instead of the one question they really should have asked.

Why the hell are we building an app in the first place?

I want to know who exactly is going to all the trouble of installing the McDonalds app on their device instead of simply visiting the McDonalds website in the browser as needed. What problem does that app solve for french fry enthusiasts that it needs to be permanently installed on your device? Why are they giving away free Big Macs just to get people to install this thing?

Fragmentation into parallel and incompatible app worlds

It was so much easier when iOS was totally dominant and the iPhone was the only player. Before the iPad and tablets. Before Android got decent in 4.0 and Google standardized the Play store. Now there are, at minimum, four radically different mobile platforms that every serious app player has to support:

  1. Android phone
  2. iOS phone
  3. iOS tablet
  4. Android tablet

(For extra credit: how many of these are actually "mobile"?)

Unless you're careful to build equivalent apps in all those places, it's like having multiple parallel Internets. "No, sorry, it's not available on that Internet, only the iOS phone Internet." Or even worse, only on the United States iOS phone Internet.

If you're feeling generous, we should technically include Windows 8 and Windows Phone in here too. All with different screen dimensions, development stacks, UI guidelines, and usage patterns. Oh and by the way, that's assuming no other players emerge as serious contenders in the computing device market. Ever.

At the point where you find yourself praying for a duopoly as one of the better possible outcomes, that's … not a good sign.

Paying for apps became a race to the bottom

Buying an app is the modern Support Your Favorite Small Software Vendor Day. I was always fine with dropping ten or twenty bucks on software I loved. I'm a software engineer by profession; apps are cheaper so I can buy even more of them.

Have you ever noticed that the people complaining about apps that cost $3.99 are the same people dropping five bucks on a cup of fancy coffee without batting an eyelash? Me too, and I'm with the coffee people. $3.99 for your app? Outraaageous!

Now, contrast this with your app, Mr. Developer. I don’t know you from Adam. You’re pitching digital Instant Refresher Juice 1.0 to me in the form of a new app. The return I’m going to get is questionable at best. I already have 30 apps on my phone, some of them very good. Do I need another one? I don’t use the 30 I have. The experience I’m going to get from adding one more app is not trustable. I’m assured of nothing. Last week I bought an app for 99 cents and it was terrible. I used it once, for 15 seconds. I could be shoving $1 straight down the toilet again for all I know. Your app, good sir, is a total gamble. Sure, it’s only a $1 gamble… but it’s a gamble and that fact matters more than any price you might place on it.

For some reason I don't completely understand, mobile app review systems are frequently of questionable value, so all you really have to go on are the screenshots and a bit of text provided by the developer.

Imagine you bought your coffee, only to open the lid and find it was only half full, or that it wasn't coffee at all but lemonade. If only 1 in 5 cups of coffee you bought actually contained coffee, a $3.99 price for that coffee starts to seem unreasonably high. When you buy an app, you don't really know what you're going to get.

Turns out, the precious resource here isn't the money after all. It's your time. In a world of millions of apps, free is the correct and only price for most apps except those rare few of extreme, easily demonstrable value – probably from well known brands of websites you already use daily. So hey, everything is free! Awesome! Right? Well…

When apps are free, you're the product

I know, I know, I'm sick of this trite phrase too. But if the market is emphatically proving that free is the only sustainable model for apps, then this is the new reality we have to acknowledge.

Geek-and-poke-pigs-free

Nothing terrifies me more than an app with no moral conscience in the desperate pursuit of revenue that has full access to everything on my phone: contacts, address book, pictures, email, auth tokens, you name it. I'm not excited by the prospect of installing an app on my phone these days. It's more like a vague sense of impending dread, with my finger shakily hovering over the uninstall button the whole time. All I can think is what shitty thing is this "free" app going to do to me so they can satisfy their investors?

For the sake of argument, let's say the app is free, and the developers are ethical, so you trust that they won't do anything sketchy with the personal information on your device to make ends meet. Great! But they still have to make a living, don't they? Which means doing anything useful in the app requires buying three "optional" add-ons that cost $2.99 each. Or there are special fees for performing certain actions. Isn't this stuff you would want to know before installing the app? You betcha. Maybe the app is properly tagged as "offering in-app purchases" but the entire burden of discovering exactly what "in-app purchases" means, and how much the app will ultimately cost you, is placed completely on your shoulders. You, the poor, bedraggled user.

The app user experience is wildly inconsistent

Have you ever tried actually using the Amazon app on iOS, Android, and Windows? iOS does the best, mostly because it's been an app platform for longer than the others, but even there, the Amazon app is a frustrating morass of missing and incomplete functions from the website. Sure, maybe you don't need the full breadth of Amazon functions on your phone, though that's debatable on a tablet. But natural web conveniences like opening links in new tabs, sharing links, the back button, searching within the page, and zooming in and out are available inconsistently, if at all.

The minute you begin switching between platforms – say you use an iOS tablet and an Android phone and a Windows 8 touch laptop, like I do – you'll find there are massive differences between the Amazon apps (and the eBay apps, and the Netflix apps, and the..) on these different platforms. At some point, you just get fed up with all the inconsistencies and oddities and quirks and say to hell with these apps, can I please just use the website instead?

Now, if your website is an awful calcified throwback to 2003, like eBay, then the mobile apps can be a valuable opportunity to reinvent your user interface without alienating all your existing users. If there's one thing I love about tablet and phone design it's that their small screens and touch interfaces force people to think simpler. This is a good thing. But if you don't eventually take those improvements home to the mothership, you're creating two totally different and incompatible UIs for doing the same things.

It seems like a fool's errand to dump millions of dollars of development time into these radically different, siloed app platforms when Amazon could have spent it improving their website and making that experience scale a bit better to every device out there.

The World Wide App

But that's not an option, because apparently the web is dead, and mobile apps are the future. I'm doing my best to resist a sudden uncontrollable urge to use my Ledge Finder app to find the nearest ledge to jump from right now.

The tablet and phone app ecosystem is slowly, painstakingly reinventing everything I hated about the computer software industry before the web blew it all up. Even fans are concerned:

I’m waiting for something that will unify the world of apps and make manually going to an App Store to find a new app as weird as typing in a URL to find a new website. My bet is that this won’t be Facebook. Instead, I would not bet against some young upstart, perhaps one inspired upon reading about a $19 billion deal, to go heads-down and come up with something crazy.

I'll have more to say about this soon, but I expect there to be an explosion of new computing devices all over the world in the next few decades, not a contraction. Sometimes the craziest solution is the one that's been right there in front of you the whole time.

[advertisement] Hiring developers? Post your open positions with Stack Overflow Careers and reach over 20MM awesome devs already on Stack Overflow. Create your satisfaction-guaranteed job listing today!
Discussion

Complaint-Driven Development

If I haven't blogged much in the last year, it's because we've been busy building that civilized discourse construction kit thing I talked about.

Civilized-discourse-construction-kit-inc

(Yes, that's actually the name of the company. This is what happens when you put me in charge of naming things. Pinball machines, people, what's the difference? I've apologized to Bill Budge already.)

So if you, like my investors, are wondering why this process took a whole entire year, I should explain how I build things, or at least, how we built Stack Overflow and Stack Exchange and now Discourse:

  1. Do a ton of detailed research on everything out there in your space. The successes: what are they getting wrong? The failures: what did they get right? Nobody should know more about the history of your area than you do. Have a story that makes sense, something you believe in, and more importantly, a story you can get others to believe in.

  2. Based on this research, assemble a team and build the minimum viable product that does something useful. If you need seed funding, this is the time to get it, so I hope you're pretty good at all that stuff in step 1, and maybe famous too, and ideally already successful as well, otherwise you are screwed.

  3. Have your team and yourself start using that minimum viable product, every day, all day long. This is way more than mere software development: it's your whole life. If you aren't living in the software you're building, each day, every day, all day … things are inevitably going to end in tears for everyone involved. And honestly, if I have to explain this to you, guess what? You're screwed.

  4. Launch a brief closed beta and get feedback from your Special Internet Friends™ on what you've built so far. I know what you're thinking: Friends! Damn it! I knew those things would be useful to me at some point! Listen to all their feedback with an open mind, no matter how dumb it probably is. Identify and fix everything major that comes up. Your product will still be terrible, but it'll be slightly marginally less terrible, and you'll now be slightly marginally less screwed than you otherwise would. (This is what we business experts call a "competitive advantange". Look it up.)

  5. Rapidly get to a public launch. It will suck, but you will ship it anyway. Don't screw up the basic logistics of the launch. You know what I'm talking about because you've seen those sad launches. Don't be those companies. Don't be those teams. Don't worry, you'll have ample time to screw everything up royally in the next step.

  6. Hey, remember all those brilliant ideas you had based on all that painstaking, detailed research you did in step 1? Turns out once you put them in front of actual honest-to-god real world users they were all … completely … wrong. Now spend the next year doing nothing but fixing all your idiotic screwups and stupid mistakes.

  7. ???

  8. Profit!

I never said it was a good plan for building software, but hey. Y'know. It's a plan.

I-love-it-when-a-plan-comes-wait

Each one of those steps is worthy of a blog entry in its own right, but it's step six that I want to focus on today because in my opinion that's the most critical part of this whole so-called "plan". I like to refer to this phase as complaint driven development:

  • Get your software in front of as many real users as you can.
  • Listen to all the things they complain about. It will be… a lot.
  • Identify and fix the top 3 things people keep repeatedly complaining about.
  • Do it again.

Now, we have a bit of an unfair advantage here because Discourse is discussion software. We host the discussions about all the things that are wrong with Discourse … on Discourse itself. But that's also why we built an open source discussion platform in the first place – my deeply held belief that actually listening to your customers should matter to your business.

Provided you're equipped to listen to your customers, complaint driven development isn't that difficult. Until you get deep into a multi-year design, you're dealing with fairly obvious, easy to fix complaints from users. You just have to be out there listening. As Steve Krug says in Don't Make Me Think:

You don't need to find all the problems. In fact, you'll never find all of the problems in anything you test. And it wouldn't help if you did, because of this fact:
You can find more problems in half a day than you can fix in a month.
You'll always find more problems than you have the resources to fix, so it's very important that you focus on fixing the most serious ones first. And three users are very likely to encounter many of the most significant problems related to the tasks that you're testing.

For example, we launched Discourse with a requirement that all topic titles and bodies be above a certain minimum character length, because we believe that extremely short posts and particularly titles aren't conducive to actual conversation. Philosophically, this is an important default for us, because it strongly relates to our core mission of building software that helps cultivate meaningful conversation on the Internet.

Unfortunately, users hated it:

I think it's especially annoying that there's no indicator of how many characters that you have to type. You only have whether or not the "Reply" button is greyed out or not, and not all users will realize that it's greyed out at first. Even then, if you click on the reply button it can bounce back on you if your post was mostly white-space. It's annoying as hell.

This was one of the consistently strongest bits of early feedback we got. So in the first 7 days after launch we quickly added a real time character count to the bottom right of the editor.

Discourse-character-count-1

I thought that'd help. It didn't. The complaints about our terrible, awful, onerous default title and body length restrictions kept pouring in. So we experimented with ways to make these requirements clearer, by using a red border, or a red background on the fields.

Discourse-character-count-2

Discourse-character-count-3

We deployed all of the above and more. Complaints did not abate one bit. Now this is a configuration setting, if you want the minimum title and body length to be 1 character in your community, it's trivially settable via a web browser in about 15 seconds. Frankly I started getting really sick of hearing all the complaints about the setting.

So we finally deployed the nuclear option: bouncy error dialogs right next to the field as soon as they lose focus.

Discourse-character-count-4

Since that change, I haven't heard word one about our terrible, onerous, awful default body and title character limit policies. Not one. Single. Complaint.

So that's the sort of thing we've been doing post launch, each day, every week, for the last year. It took us a full year of complaint driven development to get to software worth using. And even though we are now cautiously accepting customers, we're still practicing complaint driven development every day, just perhaps weighted a bit more heavily towards the people actually paying us money.

It's true that gathering feedback from your community can be hard work. And 90% of the feedback you'll get will be terrible for a whole host of reasons. It's a lot easier to imagine some heroic expert swooping in and magically blessing you with the correct answer. Well, good luck with that fantasy. The only thing I've ever seen work is getting down deep and dirty in the trenches with your users, communicating with them and cultivating relationships. That's how you suss out the rare 10% of community feedback that is amazing and transformative. That's how you build a community that gives a damn about what you're doing – by caring enough to truly listen to them and making changes they care about.

[advertisement] Stack Overflow Careers matches the best developers (you!) with the best employers. You can search our job listings or create a profile and even let employers find you.
Discussion

The Road to VR

A month after I wrote about John Carmack, he left id Software to become the CTO of Oculus. This was big news for two reasons:

  1. Carmack founded id in the early 90s. An id Software without Carmack is like an Apple without Woz and Jobs. You wouldn't leave the prestigious company you founded unless you had some pretty compelling new dreams to pursue.

  2. Oculus is the company many are betting will break VR headsets into the mainstream. And even if they don't manage to pull that off, they are now the most credible contender to make serious headway towards consumer VR the industry has ever seen.

Virtual reality is the stuff of programmer legend. Every software engineer that's ever read Snow Crash (or more recently, the excellent Ready Player One) has dreamed of jacking into the metaverse. But why now? Well, if you think of it in very coarse terms as strapping two smartphones on your face and writing clever glue software, modern consumer VR is a natural outcome of what Chris Anderson calls the "peace dividend of the smartphone wars":

It's hard to argue that we're not in an exponential period of technological innovation. The personal drone is basically the peace dividend of the smartphone wars, which is to say that the components in a smartphone – the sensors, the GPS, the camera, the ARM core processors, the wireless, the memory, the battery – all that stuff, which is being driven by the incredible economies of scale and innovation machines at Apple, Google, and others, is available for a few dollars. They were essentially "unobtainium" 10 years ago. This is stuff that used to be military industrial technology; you can buy it at RadioShack now. I've never seen technology move faster than it's moving right now, and that's because of the supercomputer in your pocket.

It's no coincidence that another programming legend, Michael Abrash, is also head over heels in love with VR. He worked with Carmack on Quake, and joined Valve software in 2011. His recent treatises on VR are practically religious tomes – "excited" doesn't even begin to cover it:

I apologize that these are both PDFs, but like everything else Abrash writes, they are amazing. You should read them. Closely. I don't call him one of the best technical writers I've ever encountered for nothing. If you find these interesting – and if you don't, I will personally drive to your house and pull your damn geek card myself – you should also dip into his blog, which drills into the specific challenges VR presents.

I thought VR would be at best a novelty in my lifetime. I remember playing Dactyl Nightmare at a storefront in Boulder, Colorado in the mid 90s.

If nothing else, it is abundantly clear that even after all these years, VR presents deep, hairy technical challenges even on today's insanely fast, crazily powerful hardware. That's exactly the sort of problem suited to the off-the-charts skill level of legendary programmers like Abrash and Carmack. Having both of these guys working on the newest Oculus Rift prototype with an enthusiasm I haven't felt since the early 90's means we could be on the verge of a Doom or Quake style killer app breakthrough in VR.

Oculus-rift-crystal-cove

There's no shortage of breathless previews, such as this one at Gizmodo which ends with

But if the original Oculus was a proof of concept, this model is proof that the concept is genius. There's zero doubt in my mind that when the final version of this device comes out it is going to change the world. For me, today, already has.

I'm optimistic about the next generation of Oculus Rift. But cautiously so.

Thanks to a friend, I had an opportunity to borrow the older Oculus Rift developer kit. And to be honest … I wasn't that impressed.

  • It's a big commitment to strap a giant, heavy device on your face with 3+ cables to your PC. You don't just casually fire up a VR experience. It takes substantial setup and configuration to get it ready. And even after configuring it, entering and exiting that VR experience is a far cry from quickly sitting down in front of a TV and grabbing that extra controller, or turning on a tablet.

  • Demos are great, but there aren't many games in the Steam Store that support VR today, and the ones that do support VR can feel like artificially tacked on novelty experiences. I did try Surgeon Simulator 2013 which was satisfyingly hilarious.

  • Having your eyes so close to the screens means the display is effectively very low resolution. And I mean extremely low resolution; I'm talking literally 320x200 type stuff. Everyone talks about the "screen door effect" which is the actual matrix of pixels. I personally found it very distracting, probably the number one thing that bothered me about the experience. Any kind of text was basically unreadable. The prototype is only 720p though, whereas the newer models will be 1080p. That will help, but the resolution problem was so severe to me that I'm not sure it'll be enough.

  • VR is a surprisingly anti-social hobby, even by gamer standards, which are, uh … low. Let me tell you, nothing is quite as boring as watching another person sit down, strap on a headset, and have an extended VR "experience". I'm stifling a yawn just thinking about it. I suppose games could present a friendlier set of data on the screen for others to spectate while sending a different set of data to the VR headset, but most of the games we played showed the actual VR screen, which is extreme distort-o-vision to the naked eye. Not really something you can watch or enjoy.

  • Wearing a good VR headset makes you suddenly realize how many other systems you need to add to the mix to get a truly great VR experience: headphones and awesome positional audio, some way of tracking your hand positions, perhaps an omnidirectional treadmill, and as we see with the Crystal Cove prototype, an external Kinect style camera to track your head position at absolute minimum. Eventually maybe even wear a suit to track your whole body. Notice how quickly we get into geez-this-is-a-lot-of-equipment territory.

The Oculus Rift prototype was an excellent and interesting and worthwhile experience, don't get me wrong, but it was more of a tech demo than anything else. It felt a long way from something that I'd be comfortable donning on a regular basis.

I'll leave you with Michael Abrash's summary:

  • Compelling consumer-priced VR hardware is coming, probably within two years
  • It’s for real this time – we’ve built prototypes, and it’s pretty incredible
  • Our technology should work for consumer products
  • VR will be best and will evolve most rapidly on the PC
  • Steam will support it well
  • And we think it’s possible that it could transform the entire entertainment industry

But that hardly does it justice; read the entire presentation (pdf).

If you want some of the hardest practical problems in computer science to work on, bringing VR to the world is as ambitious (and fun!) a goal in software and hardware engineering I can think of. So like any proper card-carrying geek, I'll certainly be ordering the new Crystal Cove model of Oculus Rift as soon as it's available.

It's a start. Maybe a big one.

[advertisement] Hiring developers? Post your open positions with Stack Overflow Careers and reach over 20MM awesome devs already on Stack Overflow. Create your satisfaction-guaranteed job listing today!
Discussion

Why Does Windows Have Terrible Battery Life?

I've become a huge fan of touch computing. I believe that most things we think of as "computers" will be de-facto tablets, either in our pocket, in our hands, possibly even mounted on our wrists or forearms.

I can't wait for the iPad 5 this week (I'll be ordering three), and my Surface Pro 2 should arrive this week too. Because it is a blazingly fast, modern Intel machine, I like to use the Surface Pro to predict where tablet performance ought to be for everyone in 2 to 3 years. I think of it as an iPad 7.

My main complaint with the Surface Pro is the incredibly lackluster battery life. Granted, this is a classic Intel x86 box we're talking about, not some efficient ARM system-on-a-chip designed to run on a tiny battery. Still, I was hopeful that the first Surface Pro with Haswell inside would produce giant gains in battery life as Intel promised. Then I saw this graph:

Web browsing battery life, Surface Pro vs Surface Pro 2

So WiFi web browsing battery life, arguably the most common user activity there is on a computer these days, goes from 4.7 hours on the Surface Pro to 6.7 hours on the Surface Pro 2, a 42% increase. That's a decent increase, I suppose, but I was hoping for something more like 8 hours, something closer to doubling of battery life – to bring the Surface Pro in line with other tablets.

Nearly 7 whole hours of WiFi web browsing for a real computer in tablet form factor … that's not bad, right? Let's see how the 2013 MacBook Air does, which spec-wise is about as close as we can get to the Surface Pro 2. The screen is somewhat lower resolution and not touch capable, of course, but under the hood, the i5-4200u CPU and LPDDR3 RAM are nearly the same. It's a real computer, too, using the latest Intel technology.

Macbook-air-2013-web-browsing

The Surface Pro 2 has a 42 Wh battery, which puts it closer to the 11 inch Air in capacity. Still, over 11 hours of battery life browsing the web on WiFi? That means the Air is somehow producing nearly two times the battery efficiency of the best hardware and software combination Microsoft can muster, for what I consider to be the most common usage pattern on a computer today. That's shocking. Scandalous, even.

UPDATE: Turns out the Surface 2 Pro was shipped with bad firmware. Once updated, the WiFi adapter enters lower idle power states and this helps a lot, going from 6.6 hours of browsing time to 8.3 hours, a 25% improvement! That puts it much more in line with the rest of the field, at least, even if it doesn't achieve Mac like runtime.

It's not exactly news that Windows historically doesn't do as well as OS X on battery life. Way back in 2009, AnandTech tested a MacBook Pro with multiple operating systems:

2009 15-inch MacBook Pro (73WHr battery) OS X 10.5.7 Windows Vista x64 SP1 Windows 7 RC1
Wireless Web Browsing (No Flash) Battery Life 8.13 hours 6.02 hours 5.48 hours

That's fine, I knew about this discrepancy, but here's what really bothers me:

  1. The Windows light usage battery life situation has not improved at all since 2009. If anything the disparity between OS X and Windows light usage battery life has gotten worse.

  2. Microsoft positions Windows 8 as an operating system that's great for tablets, which are designed for casual web browsing and light app use – but how can that possibly be true when Windows idle power management is so much worse than the competition's desktop operating system in OS X – much less their tablet and phone operating system, iOS?

(It's true that Bay Trail, Intel's new lower power CPU from the Atom family, achieves 8.6 hours of WiFi web browsing. That's solidly in the middle of the tablet pack for battery life. But all the evidence tells me that the very same hardware would do a lot better in OS X, or even iOS. At least Intel has finally produced something that's reasonably competitive with the latest ARM chips.)

Perhaps most damning of all, if you take the latest and greatest 13" MacBook Air, and install Windows 8 on it, guess what happens to battery life?

One of the best things about the standard 2013 MacBook Air 13" is that it has record-breaking battery life of 14 hrs 25 min (with the screen brightness at 100 cd/m², headphones plugged in and the Wi-Fi, Bluetooth and keyboard backlighting turned off). Under Windows 8 the results are more mixed [..] in the same conditions it lasts only 7 hrs 40 min. That's still very high—it's better than the Asus Zenbook Prime UX31A's 6 hours and the Samsung Series 7 Ultra's 5 hours—but it's only half the astronomical 14 hours + that the 13" MacBook Air is capable of.

Instead of the 26% less battery life in Windows that Anand measured in 2009, we're now seeing 50% less battery life. This is an enormous gap between Windows and OS X in what is arguably the most common form of computer usage today, basic WiFi web browsing. That's shameful. Embarrassing, even.

I had a brief Twitter conversation with Anand Shimpi of Anandtech about this, and he was as perplexed as I was. Nobody could explain the technical basis for this vast difference in idle power management on the same hardware. None of the PC vendors he spoke to could justify it, or produce a Windows box that managed similar battery life to OS X. And that battery life gap is worse today – even when using Microsoft's own hardware, designed in Microsoft's labs, running Microsoft's latest operating system released this week. Microsoft can no longer hand wave this vast difference away based on vague references to "poorly optimized third party drivers".

Apple is clearly doing a great job here. Kudos. If you want a device that delivers maximum battery life for light web browsing, there's no question that you should get something with an Apple logo on it. I just wish somebody could explain to me and Anand why Windows is so awful at managing idle power. We're at a loss to understand why Windows' terrible – and worsening! – idle battery life performance isn't the source of far more industry outrage.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!
Discussion

You Don't Need Millions of Dollars

Masters of Doom is the story of John Carmack and John Romero creating the seminal games Wolfenstein 3D, Doom, and Quake.

Masters-of-doom-book-cover

It's an amazing work on so many levels – but primarily because of the exhaustive research the author undertook to tell this story.

To re-create the story of the Two Johns, I conducted hundreds of interviews over six years, often with each person on multiple occasions. After moving to Dallas in the fall of 2000 for research, I became known in offices, barbecue joints, and bars around town as “the guy writing the Book.” John Romero and John Carmack each spent dozens of hours in person answering my most picayune questions: how they were feeling, what they were thinking, what they were saying, hearing, seeing, playing. What they and others couldn’t recall, I unearthed from websites, newsgroups, e-mails, chat transcripts, and magazines (though I drew from some of these articles, I made a point of getting the gamers’ own versions of what happened as well). I also played a delirious amount of games: at home, online, and at a couple tournaments (yeah, I lost).

I spent six months transcribing all my taped interviews. From this material, I assembled a narrative of dialogue and description that re-creates the events as faithfully and accurately as possible. As often as appropriate, I told the story from each person’s point of view to give readers the different perspectives.

It's unusual to find a book about a contentious, complex friendship and business relationship that both parties sign off on – and even a decade later, regularly recommend to people interested in their personal back stories. But it is a testament to just how right Kushner got this story that both Romero and Carmack do. This is exactly the sort of meticulously researched, multiple viewpoint biography that you'd want to read about important people in your industry. In that sense, it's kind of the opposite of the Jobs biography, which I liked well enough, but it presented one viewpoint, and often in a very incomplete, sloppily researched way. I would kill to read a book this good about Jobs.

In a way, I grew up with these guys. I am almost exactly the same age they are. I missed the Wolfenstein 3D release because I was still in college, but come December 1993, there I was, bursting with anticipation waiting for the release of Doom along with every other early PC gamer. And who gave Doom its name? Oddly enough, Tom Cruise did.

I've had a lifelong love affair with first person shooters since encountering Wolf3D and Doom. I played about every Doom engine game there was to death. I even had a brief encounter with Romero himself on the modem based multiplayer hub DWANGO where I proverbially "sucked it down". And after the Internet hit around '95, I continued to follow Quake development obsessively online, poring over every .plan file update, and living the drama of the inevitable breakup, the emergence of GLQuake and 3D accelerators, and the road to Quake 3.

It is also an incredibly inspiring story. Here's a stereotypical group of geeky programmers from sketchy home backgrounds who went on to … basically create an entire industry from scratch on their own terms.

Shareware. Romero was familiar with the concept. It dated back to a guy named Andrew Fluegelman, founding editor of PC World magazine. In 1980, Fluegelman wrote a program called PC-Talk and released it online with a note saying that anyone who liked the wares should feel free to send him some “appreciation” money. Soon enough he had to hire a staff to count all the checks. Fluegelman called the practice “shareware,” “an experiment in economics.” Over the eighties other hackers picked up the ball, making their programs for Apples, PCs, and other computers available in the same honor code: Try it, if you like it, pay me. The payment would entitle the customer to receive technical support and updates.

The Association of Shareware Professionals put the business, largely domestic, between $10 and $20 million annually—even with only an estimated 10 percent of customers paying to register a shareware title. Forbes magazine marveled at the trend, writing in 1988 that “if this doesn’t sound like a very sound way to build a business, think again.” Shareware, it argued, relied not on expensive advertising but on word of mouth or, as one practitioner put it, “word of disk.” Robert Wallace, a top programmer at Microsoft, turned a shareware program of his called PC-Write into a multimillion-dollar empire. Most authors, however, were happy to break six figures and often made little more than $25,000 per year. Selling a thousand copies of a title in one year was a great success. Shareware was still a radical conceit, one that, furthermore, had been used only for utility programs, like check-balancing programs and word-processing wares. [Shareware] had never been exploited for games.

Does anyone even remember what shareware is? What is the equivalent to shareware today? Distributing software yourself on the Internet? Sort of. I'd say it's more analogous to the various app stores: Google Play, Apple App Store, Windows Store. Going directly to the users. But they found shareware games didn't work, at least initially:

When it came time to distribute the games, Scott took a long, hard look at the shareware market. He liked what he saw: the fact that he could run everything himself without having to deal with retailers or publishers. So he followed suit, putting out two text-based games in their entirety and waiting for the cash to roll in. But the cash didn’t roll; it didn’t even trickle. Gamers, he realized, might be a different breed from those consumers who actually paid for utility shareware. They were more apt simply to take what they could get for free. Scott did some research and realized he wasn’t alone; other programmers who had released games in their entirety as shareware were broke too. People may be honest, he thought, but they’re also generally lazy. They need an incentive.

Then he got an idea. Instead of giving away the entire game, why not give out only the first portion, then make the player buy the rest of the game directly from him? No one had tried it before, but there was no reason it couldn’t work. The games Scott was making were perfectly suited to such a plan because they were broken up into short episodes or “levels” of play. He could simply put out, say, fifteen levels of a game, then tell players that if they sent him a check he would send them the remaining thirty.

You know how game companies spent the last 5 years figuring out that free games with 100% in-app purchases are the optimum (and maybe, only) business model for games today? The guys at id had figured that all out twenty seven years ago. Those sounds you hear in the distance are a little bit of history repeating.

Id Software was more than a unique business model that gave almost all the power to the programmers. It was the explosive combination of shareware delivery with a particular genius programmer inventing new techniques for PC games that nobody had seen before: John Carmack. It may sound prosaic and banal now, but smooth scrolling platforming, texture mapped walls, lighting models, and high speed software 3D rendering on a PC were all virtually unheard of at the time Carmack created the engines that made them commonplace.

Carmack_Headshot_PR_660

Carmack, like Abrash, is a legend in programming circles, and for good reason. The stories in this book about him are, frankly, a little scary. His devotion to the machine borders on fanatical; he regularly worked 80 hour weeks and he'd take "vacations" where it was just him and a computer alone in a hotel room for a whole week – just for fun, to relax. His output is herculean. But he also realizes that all his hard work is made possible by a long line of other programmers who came before him.

Al had never seen a side scrolling like this for the PC. “Wow,” he told Carmack, “you should patent this technology.

Carmack turned red. “If you ever ask me to patent anything,” he snapped, “I’ll quit.” Al assumed Carmack was trying to protect his own financial interests, but in reality he had struck what was growing into an increasingly raw nerve for the young, idealistic programmer. It was one of the few things that could truly make him angry. It was ingrained in his bones since his first reading of the Hacker Ethic. All of science and technology and culture and learning and academics is built upon using the work that others have done before, Carmack thought. But to take a patenting approach and say it’s like, well, this idea is my idea, you cannot extend this idea in any way, because I own this idea—it just seems so fundamentally wrong. Patents were jeopardizing the very thing that was central to his life: writing code to solve problems. If the world became a place in which he couldn’t solve a problem without infringing on someone’s patents, he would be very unhappy living there.

In that spirit, Carmack regularly releases his old engines under GPL for other programmers to learn from. Don't miss Fabien Sanglard's epic deconstruction of the Doom 3 codebase, for example. That's only one iteration behind the current id engine which was used for Rage and (apparently) will be used for the upcoming Doom 4.

One of my very favorite quotes of all time comes at the end of the book.

Carmack disdained talk of highfalutin things like legacies but when pressed would allow at least one thought on his own. “In the information age, the barriers just aren’t there,” he said. “The barriers are self-imposed. If you want to set off and go develop some grand new thing, you don’t need millions of dollars of capitalization. You need enough pizza and Diet Coke to stick in your refrigerator, a cheap PC to work on, and the dedication to go through with it. We slept on floors. We waded across rivers.”

And indeed they did, as the book will attest. Both @ID_AA_Carmack and @romero are still lifelong, influential, inspiring members of the game and programming communities. They are here for the long haul because they love this stuff and always have.

The ultimate point of Masters of Doom is that today you no longer need to be as brilliant as John Carmack to achieve success, and John Carmack himself will be the first to tell you that. Where John was sitting in a cubicle by himself in Mesquite, Texas for 80 hours a week painstakingly inventing all this stuff from first principles, on hardware that was barely capable, you have a supercomputer in your pocket, another supercomputer on your desk, and two dozen open source frameworks and libraries that can do 90% of the work for you. You have GitHub, Wikipedia, Stack Overflow, and the whole of the Internet.

All you have to do is get off your butt and use them.

[advertisement] Hiring developers? Post your open positions with Stack Overflow Careers and reach over 20MM awesome devs already on Stack Overflow. Create your satisfaction-guaranteed job listing today!
Discussion