After working with VB6 and "classic" ASP for so long, I got spoiled with effectively nonexistent compile times. Part of that, of course, is due to how old the environments are-- or were. I remember using VB5 shortly after its release on Pentium 1 class hardware, and it wasn't what I would call 'fast' either.
If you're curious, as I was, about which of the current CPU generation is fastest at VS.NET compilation, check out this page at Xbit Labs. Very interesting. It's quite rare for a CPU roundup to perform any benchmarks related to development.
It's technically a C++ eMule client compiled using the latest VS.NET -- so probably not managed code, but I assume the results would be similar to a large, managed C# or VB.NET project.
The Athlon 64 is absolutely mopping the floor with everything else. I think L2 cache is clearly a factor-- look at the results of the P4 extreme edition with 2mb L2 cache -- but the on-die memory controller must be what gives it the runaway victory. It certainly can't be clock speed, since the Athlon 64 runs at ~ 2ghz.
The other interesting thing to consider, is that the fastest CPU compiles debug releases almost twice as fast as the the "slowest" CPU. And in this case "slowest" is a Pentium 4 3.0ghz, which isn't exactly chopped liver... depending on how much you compile, and how much your time is worth, an Athlon 64 upgrade could pay for itself in no time. Something to consider.