Well, you have to consider that while the libertarian ideology is generally compatible with right-wing economics, it clashes on several points with left-wing social policies which go against the principle of noncoercion. Gun control and affirmative action are the most obvious ones.