The term to be applied is “adverse inference”. Even if the refusal to answer can’t be the basis to charge a person with a crime, the other party in the civil case may ask the court to make an “adverse inference” that the answer would have tended to establish the fact in question.