I agree with you 100%.
I think there are several parts to the explanation.
First, there’s simply a question of money. Special effects cost $$$.
Next, “hard” science fiction traditionally doesn’t do well at the box office. (I’m sorry but I don’t consider “Star Wars” or “E.T.” to be “hard” science fiction. Science fiction is about mind-bending ideas as well as nuts-and-bolts gadgetry. I didn’t see any mind-bending ideas in either movie.) Hollywood is very reluctant to risk money on Sci-Fi. I think the Star Trek movies basically “lucked out” that way, in that there was a big enough market for the first few movies that Hollywood saw, hey, this might fly, so continued to invest money in it.
Also, a lot of Heinlein’s stuff consists not just of rocket-boy-to-the-moon type stories, but of stories that are essentially unfilmable. How would you film Time Enough for Love? I don’t mean “unfilmable” as “not worthy of being filmed,” I mean look at it from Hollywood’s perspective, money-men who need a 25 words or less, high-concept sound bite to describe it. Same thing for Methusaleh’s Children, “well, it’s about this guy who’s immortal…” Whoops, the money-men are beating a hasty retreat to the screening room to look at rushes of the Silence of the Lambs sequel.
I think translating some of Heinlein’s less video-ready concepts to the big screen would be rather difficult. I think if you did manage to do it, and did justice to the original story, you’d end up with more of a Merchant/Ivory art film kind of thing, and we all know how well those do at the box office, and how eager Hollywood money-men are to back them, when they’re still just a screenplay and a director’s dream.
Also, there’s this: I long ago stopped trying to keep track of movies that I loved as books, but which Hollywood totally screwed up when translated to the Silver Screen, by changing things, by adding and subtracting characters, by leaving out (or adding) subplots. A book always undergoes a strange sea-change when it’s translated by a team of screenwriters, directors, producers, and money-men into a movie.
I actually did not realize for some time that the movie “Puppet Masters” was supposed to be based on Heinlein’s book.
So I have to say that I would actually prefer that Hollywood kept its grubby fingers off some of my favorites. I look forward to the release of “Lord of the Rings” with a sick feeling of mounting dread. “Dear Lord, please let it be, if not good, at least not totally lame. Amen.”
Because if it’s totally lame, there’s 20 million people who will be turned off Tolkien, forever.
“Why, sometimes I’ve believed as many as six impossible things before breakfast!” - the White Queen