I think tolerance is one of those things where prudence and proportion are a very important part of whether or not it is a good thing. A person cannot, and should not contend with every matter to which he feels opposition. No one point of view or personal choice can be sufficient for every person. Certainly we have some considerable limits to what we can refuse to tolerate in public behavior.
My desire does not compel the world. It demonstrably does not enjoin the world in any legal sense. Civil association requires those broad ranges of behavior and custom, which must be tolerated by all. Freedom has little meaning in a social matrix so narrow that all things not required are forbidden. So, tolerance seems at some level to be a prerequisite of civilization.
At the other end of the scale, tolerance becomes complicity, in even the legal sense, and certainly in an ethical frame of reference. I might have little valid right to object to your private acts, when those acts are of no consequence to me, or any other person. That situation changes when your behavior begins to have consequences involving other people. This is especially so in the case where those people have little likelihood of successfully disassociating themselves from your behavior.
I cannot think of a simple statement of when tolerance changes from a respect for the right of difference among people, to a cavalier indifference for the rights of others, or even myself. At some level, I have to accept that my own assessment of whether your acts are right or wrong is insufficient. I certainly do not wish to have my every act reviewed, or impeded by every person who feels I am out of accord with his or her principals. Neither do I wish to see casual bigotry, cruelty and greed accepted without notice, out of a desire for tolerance.
The unlimited extension of the concept of tolerance invites savagery and the total dominance of the strong. That type of tolerance is no different than abject subjugation by tyranny. So, I reject the concept that I must tolerate anything from anyone, at the same time that I reject the opposing premise that I must be subject to the intolerance of every person, on every judgement of my being.
Prudence and proportion are required from each of us. That is, unfortunately rather optimistic. In our social agreements, therefore, we must give great care before we allow the extremes of either tolerance, or intolerance to be codified. Once a matter of law, those things become the limits of our freedom, and the boundaries of security.
He that would make his own liberty secure must guard even his enemy from oppression; for if he violates this duty he establishes a precedent that will reach to himself.