First of all, because you have to establish that there is a statistically significant problem of slavery that exists. A few anecdotes that some people in some countries have been forced to work for X years is simply not enough to base public policy on.
Secondly, you have to establish a statistically significant causal link between free trade and slavery. I don’t find it hard to believe that Burmese workers desperate to find better living and working conditions wouldn’t end up in similar conditions anyway, free trade or no.
Your question is too imprecise to answer. I am of course not OK with slavery at any point. But that doesn’t mean I would oppose free trade among countries if a small number of slaves were being used to supply labour in some of them.
Let’s take the example of Bangladesh that you brought up earlier. It is true that many work in dangerous conditions, which occasionally results in tragic loss of life, but if someone was to argue on that basis for Bangladesh to institute regulations which result in the apparel industry in Bangladesh becoming uncompetitive, the utilitarian in me certainly thinks that would be a greater tragedy. Does that make me OK with loss of life? I don’t think so, but you’re welcome to your own interpretation.