I think isamu was referring to the many states that run closed primaries. In those states choosing to not declare a party affiliation as a matter of public record when you register to vote is choosing to forfeit your right to have any input into who those parties choose as candidates for the election you eventually do get to vote in.
You may never have lived in such a state. But many Americans do.
I do live in a state with closed primaries. I have no “right” to input into the candidates of a party I don’t belong to. That’s no more a “penalty” than the fact that I can’t workout at Planet Fitness if I don’t have a membership.
If I want to have the privilege of input into a party’s candidates, I’ll have to join that party.
Most democracies don’t have open primaries: parties make their own rules, but the norm is that if you want to choose a party’s nominee you have to join it and subscribe. I don’t think most of us would see that as a penalty (but we would find it odd for there to be a public register of one’s political preferences, even if optional).