Taking us back to the OP’s idea for a moment, let’s review how political parties got going in the United States, to understand why we have them and why they aren’t going away.
When the founders created the Constitution, one of their primary concerns was the formation of political parties. They had had ample experience with the difficulties such groupings were creating for England’s kings (the whole damn Revolutionary War can be laid in large part to the dominence during the critical phase of the “King’s Friends”, ministers who had primarily Tory sympathies, though the actual term ‘party’ hadn’t yet been applied. Various parts of the Constitution of 1787 were intended to stop factionalism, including the Electoral College, the method of selecting Senators, etc. They failed because you can’t take the politics out of government.
During the administration of Washington (himself not a member of any particular faction), those who advocated a strong federal government and those who advocated a much more state’s rights government began squaring off over policy decisions. By the 1796 election, these groups actually reached decisions on who from among them should run for the office of President; John Adams was selected as the primary opponent of Thomas Jefferson. However, because electors could vote for anyone they wanted, several unusual things happened, including at least one Federalist elector voting for TJ (at the time each elector still voted for two people; one elector voted for both TJ AND Adams). In the 1800 election, the anti-Federalists, known as Democrat-Republicans, won the first election contested solely between candidates of each party. Thus, by 1800, not 12 years after the formation of the new government, we had two parties, representing ‘opposing’ viewpoints.
This system was further entrenched by two occurrences; the initiation of the popular vote for election of electors, and the initiation of party conventions for the selection of candidates. On rare occaision, it has been upset; 1860 was one such instance, where the new Republican party overcame split opposition among the Democrats to win the Presidency contested by four candidates of three parties. But since then, only once has a third-party candidate for President out polled a major party candidate for President (1912, Teddy Roosevelt over Taft, causing Wilson to win).
So, the parties exist because the grew in the making, so to speak.
Can we eliminate parties? Not without violating the First Amendment, which has been interpreted as including a right to associate as part of the right of free speech; political partyism is one of the specific things this protects.
As for the idea of three or four party politics, let’s look at how they occur and what effect they have.
The American system encourages the limitation of parties to two, because we have a strong executive voted for seperately from the legislature. Thus, all politics in America ends up boiling down to the haves vs. the have-nots. Anyone who thinks the Southern Democrats were ideologically similar to the northern liberals of Kennedy stripe in the 50’s and 60’s is nuts; they banded together because both groups didn’t like the Republicans, who had been in office almost consistently from 1861 to 1933, then again from 1952 through 1992. In the typical parliamentary government of the European continent, however, the executive, even when seperated from the legislature, is usually weak; it is the control of the legislature that counts. THAT creates the potential for multiple factions, which indeed we see in France, Germany, Italy, Israel, etc. No offense, but with the possible exception of Germany, I can’t imagine why anyone would want to suffer the sort of silliness that goes on in countries where more than two parties exist. Why? Coalitions.
When you have more than two parties, you make inevitable the possibility of plurality governance. Thus, if the center doesn’t win 50%+1 of the seats in the legislature, they have to accept either the left or the right into government with them to make sure their policies are passed in votes. This creates strange results, with small parties holding up important legislation while they get their piece of the pie. Imagine, for instance, a flood-relief bill held up by the Green party until the Republicans agree to a highly environmentally friendly rule for administering the national monuments, or a health care bill proposed by Democrats stalled until they agree to the demand of the Anti-Abortionists to refuse federal funding of abortion procedures. Often, the result is unstable governments; Italy has had so many different official governments since WWII you can’t keep track.
I don’t like two party systems; I tend to be one of those unserviced people in the middle who get faintly disgusted by both sides (people on the flanks feel even worse; they don’t get serviced by anyone except in primaries). But I can’t stand one-partyism (Mexico hasn’t been helped by it), and I shudder at multiple partyism. So, until we stop being creatures of politics, I’ll take what we have. 
