The archetypal Hello World program has always had a calming effect on developers. It's been a programming staple for decades:

[Hello World] is typically one of the simplest programs possible in a computer language. Some, however, are surprisingly complex, especially in some GUI contexts; but most are very simple, especially those which rely heavily on a particular command line interpreter ("shell") to perform the actual output. In many embedded systems, the text may be sent to a one or two-line LCD. In yet other systems, a single LED may be turned on.

A "hello world" program can be a useful sanity test to make sure that a language's compiler, development environment, and run-time environment are correctly installed. Configuring a complete programming toolchain from scratch to the point where even trivial programs can be compiled and run may involve substantial amounts of work.

While small test programs existed since the development of programmable computers, the tradition of using the phrase "Hello world!" as the test message was influenced by the 1978 book The C Programming Language. The example program from that book prints "hello, world" (no capital letters, no exclamation sign; those entered the tradition later). The book inherited the program from a 1974 Bell Laboratories internal memorandum by Kernighan -- Programming in C: A Tutorial -- which shows the first known version of the program.

Wikipedia also has a ridiculously comprehensive list of Hello World programs in hundreds of programming languages, some of them extremely esoteric (read: joke) languages.

On the Great Win32 Computer Language Shootout page, amongst all the various benchmark tests, there's even a Hello World performance test :

Of course, it's a completely useless performance test. But that's very much in the spirit of Hello World.