I think there’s an argument to be made that the primary election process, over time, has fallen more into the hands of ideologues who do not believe in compromise. This has lead to fewer politicians on both sides who are elected with any willingness at all to work with the other side.
Eric Cantor was a pretty loyal, party-line Republican, but he wasn’t intransigent enough and participated in a few (very few) of the deals Speaker Boehner negotiated with the White House, and as thanks for it he was defeated in a primary by a candidate who will never negotiate with anyone.
Paul Ryan has even faced a challenge from a similarly recalcitrant type (Ryan is likely to crush him) but the fact that politicians who would likely compromise in a different political climate are drifting towards complete intransigence toward the other party is a serious problem, and only getting worse. For the last few cycles we’ve mostly focused on the Republicans, because the Tea Party movement has become emblematic of this trend. But I think we’re going to slowly start seeing it with the Democrats as well as their progressive wing gets more assertive.
What I find amusing is these ideologues punish “establishment” politicians largely for not being able to do the impossible. Boehner got dinged for “caving” to Obama and not, you know, defaulting on our national debt, and for not “forcing” Obama to repeal Obamacare. Boehner would’ve probably been happy to repeal Obamacare if he had a 2/3rds majority in both houses that would support it, but he didn’t. Instead, Boehner recognized what was possible versus impossible, and took the best possible path he could as a Speaker dealing with a President of the alternate party. The far right who blasted him seemed almost ignorant of the fact that there’s no magical path around the Presidential veto.
On the left, the progressives have largely chosen to ignore the fact that for 6 out of Obama’s 8 years in office he’s faced either a divided Congress or a Republican Congress. The Presidency is limited in many ways if it lacks legislative support. But fuck that, Obama and Boehner represent “corrupt politics” because they aren’t willing to blow the country up in a fit of ideological purity. Luckily we’ll be able to make sure, through the primary process, fewer and fewer traitors like them can get nominated to offices at the various level of government, am I right?!
If we may segue across the Atlantic Ocean quickly, look at how Britain’s parties have traditional operated.
Right now, to stand for election as the leader of Labour, you need the support of 20% of the Labour MPs (or if you’re the incumbent, apparently you can stand even without said support), and then you have an election pitting all the candidates who have more than 20% MP support. This process has tended over time to exclude people that aren’t willing to compromise or work within the normal Parliamentary process. Something notable about Britain’s party systems is very, very few people are party members. Being a party member in Britain comes with it responsibilities and a requirement of paying dues.
In fact, if you note, Labour recently democratized this process somewhat. Instead of requiring full membership to vote, it now allows people to register as “supporters” by paying only £3; at least in part, support of these £3 “supporters” allowed Britain’s recalcitrant left wing ideologue, Jeremy Corbyn, to win Labour leadership. This has likely doomed the party to electoral irrelevance for a generation (particularly since he appears unwilling to leave his post even though over 80% of Labour MPs want him out.)
The party leadership in Britain actually select who will stand for election in various districts–there is no primary process. In fact, if you keep looking across the Atlantic, at various different countries, we are the only one that lets voters decide who will represent a party at an election. This intermeshing between the party apparatus and actual elections have caused significant problems here in the United States, and arguably while it’s “more democratic in theory” it produces worse results for our democracy.