This discussion could go on for pages, but I’ll try to compress my understanding of it into a paragraph or three.
Once upon a time, everything was fantasy. Poe was a fantasist. Even Welles and Verne were fantasists. (They were also romantics, in an older use of the term that isn’t heard much today.)
The engineering-obsessed Hugo Gernsback started a whole series of magazines dealing with radio and other hot technologies. He finally started Amazing Stories in 1926 and used the term “scientifiction” to describe its contents. This eventually became science fiction and was intended to apply to the melding of radio and other hot technologies of the day (as well as just around the corner technologies like space travel) to fiction.
But pulp magazines were themselves a marketing offshoot, not related to “real” literature except in the most nebulous manner. Science fiction was just another slice of specialized fiction, along with romance, westerns, nurse stories, railroad stories and a million others. They were handled by the same publishers, usually by the same editors, and often filled with stories from the same authors, although many became specialists in their narrow fields.
John W. Campbell is famed as the editor of the sf classic magazine, Astounding Stories, but he simultaneously edited Unknown to cater to fantasy tastes. Again, the authors overlapped to a great extent.
There were few books in these categories until specialty presses began in the late 30s and 40s. Arkham House was created to reprint H. P. Lovecraft’s work and has remained almost entirely fantasy-oriented since, but others like Shasta, Gnome Press and even Fantasy Press (whose first book was by A. E. van Vogt) published both fantasy and science fiction. It was a marketing category first and foremost, but the readership and authorship were largely the same, even if there were vocal minorities who were devoted to one side or the other.
Even The Magazine of Fantasy changed its name to The Magazine of Fantasy and Science Fiction with its second issue in 1950 and immediately became a better magazine and a better seller.
Lin Carter changed this delicate balance in 1969. You’d think that Tolkien did, but you’d be wrong. The mantra among publishers, as it was and as it ever will be, was not that x sold, but that brand-name author sold (x being fantasy and brand-name author being Tolkien in this case). But the Ballantines brought in Carter, who had been writing Conan books, to edit a line of adult classic fantasies. Oddly enough, these did sell, so in 1975 Ballantine made Lester del Rey the editor of a new fantasy line. He started with Terry Brook’s The Sword of Shannara, and it too sold and spawned millions more imitation Tolkiens and they too sold and this is the reason that Brooks made the top 50 significant sf and fantasy books list. http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=167408
For the first time, there appeared to be a market, a separate market, for high fantasy. An increasingly separate readership developed, and increasing numbers of female writers and readers entered the field.
Internally, however, fantasy publishing lines and science fiction publishing lines relied on the same set of editors, the same marketing channels, the same advertising channels, the same distribution channels. The readership still overlapped to a great extent. Many authors continued to write both. The Science Fiction Writers of America changed its name to the Science Fiction and Fantasy Writers of America, but nothing else about them changed at all.
And although some bookstores really do separate out the sf and fantasy books, most don’t for very good reasons. Doing so places the books of some authors in two different places; the line between fantasy and science fiction is extremely hard to draw; and their idiot staffs have to alphabetize twice and try to find things in two different places when it all could more easily be smooshed together. And it all comes from the same places to begin with.
OK, that was more than three paragraphs but less than pages. There is not fantasy and science fiction in the world but f&sf. And lo shall it ever be.
