Generally speaking, we use different “fitness” criteria for different things. “Most” of something is always the winner, but what that something is often comes into dispute. For voting in particular, no system will satisfy all criteria that we would ideally think constitutes a “good” method of counting votes.
For a non-voting example, consider baseball. The “winner” of a season is the team that wins the world series, i.e., roughly a pairwise winner, not the team with the lowest ERA, or the most number of runs, or other possibilities (which are numerous, when you consider the kind of statistics sports fans keep). This is not to say that the team that wins the world series (or the Super Bowl, or the World Cup) is unambiguously the best team, only that the selection criteria we use selects this team as the winner. I’m quite sure good arguments exist that might suggest other methods of ranking teams.
Same with voting, except voting is a slightly easier phenomenon since there is only one statistic to count: the number of affirmations.
In America, the Electoral College is simply a method of assigning weights to geopolitical groups (in this case, states). Unarguably, larger populations get more votes, as we might expect. There is nothing about the EC which demands that an entire state’s EC votes must go to the most popular candidate (plurality). This is not an artifact of the EC, this is the result of the way states apportion their EC votes. Thus, if the concern of the EC is simply that an entire state’s vote goes to a barely winning candidate, one’s concern should be with the way the state handles its EC votes, not with the EC itself (see Colorado). The most salient example would be states that give their EC votes to candidates in proportion to the popular vote in a state. It would take a rather extreme case whereby, if all states adopted this method of apportioning their EC votes, the candidate that one failed to represent the simple plurality vote.
The other point to consider is that first-past-the-post voting also produces some effects which are, to some, undesirable, such that the winner of an election is not necessarily representative of those preferenced by a plurality of voters even in a particular state (i.e., even before the “hated” Electoral College comes into play). Consider a case, for example, where the ballot in Florida was set up under a transferable voting system, such that if your preferred candidate lost the initial counting, your vote could instead go to a second candidate. Clearly this changes the dynamic of elections. (It also brings other effects into play, but nevermind that now.) Many argue that this is, in fact, able to elect a winner that is preferred by most voters. Under our current system the same assertion cannot be made. We don’t know, for instance, that Buchannan voters strictly preferred Bush to Gore, or that Nader voters strictly preferred Gore to Bush in the 2000 election, so we cannot suggest that Bush is the most preferred candidate because we reduced the way voters are able to express preferences. Broadening their ability to express preferences can indeed select a more preferrable candidate… and still has nothing to do with “popular vote”.
For the federal government, I personally feel that the popular vote is a remarkably poor way to select a winner in a national, single-seat election such as the presidency. But if you like it, the Electoral College isn’t really what’s in the way.