Read Justman and Solow’s book and you’ll find that Roddenberry had been rubbing the network the wrong way practically from the beginning. That was one reason why NBC’s support for the series waned over the years; by the third season, they didn’t care who was running things behind the scenes, or if the series was renewed. Hence the increasingly worse time slots it was given.
Roddenberry could see the writing on the wall, and his participation in the third season was minimal (I believe Nimoy has said “virtually nonexistent”). He hired Freiberger as third season producer on the basis of his track record, without so much as having him write a sample episode, and Freiberger brought in Singer as his script consultant. Experienced Trek staff like Fontana and Justman were left out in the cold and departed in disgust halfway through the season.
After Paramount acquired Desilu, Solow left because of their creative interference and MBA-style budget cutting. So a whole constellation of forces was arrayed against the series by the end of 1968.
Focusing on serious science fiction wouldn’t necessarily have saved the series. The audience for it has never been that great, which is why TOS was sold as an action–adventure series. The audience for the show was always relatively small but intensely loyal and with desirable demographics; unfortunately, the latter wasn’t a consideration in the '60s. Raw numbers was what TV executives were interested in.
The decline in the quality of TOS episodes over the last season and a half was due more to production pressures and reduced budgets than anything else. Scripts that had been junked in preproduction were dusted off and used in an attempt to meet deadlines. In addition, many of the writers who had contributed to the series earlier had no desire to work with Freiberger and Singer, and Roddenberry was no longer available for rewrites.
TNG, with all of its feel-good touchy-feeley psychobullshit, psocial engineering, emphasis on “relationships,” and incessant moralizing was so much a product of the late '80s (“after MASH***”), I can’t imagine it being made at any other time. I watch it now, and the characters and most of the stories both seem incredibly lame.
It’s a pity the projected Phase II series never got off the ground in the mid-'70s. There would have been much better continuity with TOS (and the animated episodes as well), the original cast would have been given a new chance to shine, and there would have been far less damage caused by foreign influences (among which I count the flood of horrible fan fiction churned out after 1975).