The comparison between TeX and Word is an apple and orange comparison. (BTW, LaTeX is the markup language, TeX is the system which applies the language to format data in a document.) As others have already stated, TeX is designed from the ground up as a digital typesetting/structured document system for desktop publishing, similar to Adobe FrameMaker or QuarkXPress. That means that it controls the format as the primary feature and treats text and other objects as data to be inserted into the defined template. Word (and other word processors) are really WYSIWYG (“What You See Is What You Get”) text editors with formatting tools. The formatting is mingled with the text data, and your primary control is via the graphical display, which may differ from how it is displayed when printed or transported to another media. Microsoft has added some low grade document control features that allow it to function as a poor man’s desktop publisher (e.g. simple tables, handling picture/data objects with crude format control and captioning, et cetera) but these are features that are glommed on, not inherent functionality. It is noteworthy that Microsoft also provides Microsoft Publisher with the Office Suite, which is an actual (if simplistic) desktop publishing application, so they don’t even consider Word to be a structured document system.
Functionally, the difference is this: with a structured document system, you have to start out by defining a template for your document/data, e.g. where titles go, how pages are marked, borders, frames for text and images, bibliographic format (if you have that information) et cetera. Before you ever start entering text you have a screen full of boxes that define how the content will be displayed in the intended media. If you’ve ever seen a news or magazine copyroom, you’ll see screens with big blank boxes with "X"s across them where the text and images will go once the entire document is compiled. This frees the format editor from having to deal with the actual text, and any possibility that formatting will be applied arbitrarily or contrary to intent.
With a word processor, you don’t have to define any format in the document. (Word starts by defining a default font and borders, but again, what you see on the screen may be different than what is on the printed page.) You don’t generally apply any pre-formatting to the document; you just start entering text and pasting in objects. Everything that isn’t an object is by default text–in other words, the document is a big formatted text box with other objects in it. This formatting within formatting is considered very bad form in desktop publishing, and most packages don’t even allow it.
An analogy is the difference between ASCII text files and HTML “pages”; your browser can display both, and you can put links and some limited formatting (tabs, page breaks) in your ASCII text file, but you have no real control as to how it will be displayed in the browser, i.e. the font will display at the size and type defined as the default by the browser for unformatted text, text may run off the edge of the screen, et cetera. An HTML file controls the display of the page data by default; everything has a tag that gives it format and location properties, and (at least in a well-designed page) the content will be displayed in a controlled fashion without data going off the side of the frame, fonts and colors are as defined in the HTML code, et cetera.
Arguing which one is best is missing the point; you wouldn’t use TeX to write a short essay, or notes, or an e-mail, as you would spend more time messing with the formatting of a document for which display control is relatively unimportant. All you care is that the text is clear, and that line spacing, borders, and tabs are okay. However, if you are writing a long paper with a lot of citations that is to be submitted to a publisher, or generating a repeated report with a lot of data, and so forth, spending the time to set up a digital typesetting system may be worthwhile or even required. This allows the data to be conveyed as intended without any misinterpretation due to formatting. For instance, when Word creates a caption for an image object, it makes it plain text that is actually part of the body of the text. This means that if you add text above, and it shoves the image object to the bottom of the page or if printer margins are adjusted to move the object from where it is displayed, the caption may actually appear on the next page, separated from the image. This would never happen in TeX because the typesetting rules don’t permit it (unless you force it to place the image across the page break). The caption is attached to the object, not the document body.
I’ll provide a practical example: we have a system that does semi-automatic data reduction and then runs a comparison between an analysis model and the data. It dumps out formatted text files and images, which can then be directly referenced by a LaTeX template, such that it would then be possible to write a remarks/conclusions section into a text file and print the entire document to a PDF file, without ever touching the formatting. Of course, our customer wants reports in Word format, so instead, we have to laboriously cut and paste pictures and data into a Word document, and then adjust all of the formatting that gets farked up no matter how careful we are. (We did have a set of VBA scripts to try to import the data automagically, but they never seemed to work flawlessly no matter how much tweaking we did, and then Microsoft came along and broke the functionality they used with a new release of Office, so we’re back to dumb monkey formatting.) Worse, the customer then tries to take the content and plug it into their Word documents, virtually assuring that the formatting is going to go crazy. I guess we’re lucky they don’t ask for the report on punchcards, but it turns what should be a fifteen minute process of turning the crank, popping out a professionally formatted report in a structured document, and copyediting the results to make certain that the entire data set was reduced is instead several person-days of effort to generate this kludgy report in Word that never seems to print the same way twice. It isn’t really the fault of Word; it is just using the wrong tool for the job.
Regarding tables and bibliographic references, the advantage of LaTeX is that you define the format and provide the data in raw form. If you have to change references or the order in which they appear, the formatting is done automatically by the TeX interpreter, and it won’t let you make an error (or at least, it will highlight that you didn’t cite the page number, or the volume, or whatever is missing from the format). Word just treats this all as dumb text and lets you enter anything you want. If you are just citing a few articles it isn’t a big deal to control the formatting yourself, but if you have three pages of citations–not unheard of in long technical papers that make take months or years to write–having a system to update all of this automatically may save you from making errors that will delay submission. As for tables…Word just doesn’t do them well, to the point that I’ll create tabular data in Excel and import it as an HTML object rather than muck about with Word “tables”.
Back when I was involved in screenwriting, I saw numerous fledgling writers either trying to set up their Word templates to follow script format (film, television, and theatre all have very specific and controlled formats for submission, and they all differ slightly.) While it is doable, there is usually some finagling to get the formatting just right. I wrote a simple LaTeX template that lets you write out the dialogue and instructions in plain format and plunks it into the correct script format automagically. It was a huge timesaver, and also makes it a lot easier to actually read and write straight dialogue.
Word processors are fine if you’re going to bash something out in a couple of drafts without any complicated formatting beyond tabs and a few images interspaced in the text. A structured document package is necessary, however, if you are generating content that has to be in a precise, controlled format.