Lucas made the second trilogy first because they are more conventional. Ep 4, the original Star Wars is a straight forward space opera, straight out of Campbell’s Hero with a Thousand Faces. But Lucas (maker of the bizarre THX 1138) always had something far more perverse in mind, something that would, at its denoument, possibly at the end of Episode 2 or in 3, repulse the audience. In Ep. 5, the Empire Strikes Back, we found out what it was, one of the good guys became a bad guy. In fact, the Star Wars stories were never about Luke as the protagonist, they were about Anakin. The question then becomes, why did Anakin become evil.
In Ep 1, we learn that Anakin is “the chosen one”, something of a messiah. Obviously his choice about becoming evil is even more puzzling. After all, Jesus refuses all of the devil’s temptations. (Remember "The Last Temptation of Christ?) Well, for some reason, Anakin chooses to join the devil/Palpatine. Why? We can only guess at this point. Does Palpatine hold Anakin’s mother hostage? Does Anakin just become a bad teenager doing drugs? Does his love affair with Natalie Portman turn him evil?
We’ve seen him offer Luke an opportunity to “end this destructive conflict”, so I’ll jump out on a limb and go with something bad happens to Anakin’s mother, the Jedi are unsatisfactorily philosophical about it (or the blame is pinned on them, Palpatine is good at that), and he starts out on the road to hell with the best intention of making the world a better place. He sells out bit by bit and is frustrated at every step along the way. He sees Palpatine as evil and intends to betray him, but Palpatine is always one step ahead of him. After he is severly injured in his showdown with Obi-wan (undoubtedly the balance of the battle is tipped by some bone head clumsy thing done by Jar-Jar Binks, explaining why this annoying character was foisted on us) he realizes that he will never again be strong enough to defeat the emperor without assistance. So when he discovers Luke he realizes that this is his opportunity to train Luke in the more powerful Dark Side, and then turning on the Emperor. The Emperor and Vader are fully aware of their power struggle with each other at all times, each hoping that Luke will wind up killing the other and taking the apprentice position to the new master. They don’t just go at each other without a new apprentice (and here I’m really going out on a limb) because without an apprentice/master relationship their powers are very weak.
Palpatine wants Luke because Vader spends most of his own power just surviving his wounds.
You will also notice that so far, the most evil deed Vader has actually personally performed is choking to death a captured officer in Ep 4., bad, but not the devil himself. (He tortures Leia and Solo too.) Vader merely stands by for other evil acts. Is it possible that he is a double agent?
Other interesting Star Wars observations/tangents. Ever notice that the Jedi/Sith fight a lot of useless battles against each other? Ever notice that R2D2 always saves the day for everyone? My personal theory is that R2D2 is really the chosen one.