The air will try to establish an equilibrium. The balloon starts out with high pressure, so equilibrium will strongly favor helium leaving the balloon. You don’t have to lose very much before it affects buoyancy. As the pressure inside comes closer to the pressure outside, the situation looks more like an even exchange, with helium coming out and regular air going in.
Just because the net flow of gas is outward, it doesn’t mean that there is no flow of gas into the balloon. Rivers have a detectable salt content well before they empty into a salty body (e.g. ocean). I keep my glovebox under a positive pressure of nitrogen, but it still gets oxygen in it. Helium out + air in eventually leads to a buoyancy that is equal to and then less than the weight of the balloon and string.
Sort of. The balloon starts out with a high partial pressure of helium compared to the rest of the atmosphere. The absolute pressure in the balloon is nearly identical to that outside of the balloon; the additional pressure created by the elastic balloon walls is fairly small. Otherwise, yes, the high helium partial pressure forces helium out of the balloon, while the high oxygen and nitrogen partial pressures force those gasses into the balloon. The balloon material is, I believe, more permeable to helium than the other gasses, so helium wins the race.