I think ad hominem is a statement about an argument, or assertion. It’s a term for debating. If you interpret something differently based on who is saying it, that’s not ad hominem, because you are not putting forth an argument.
There is a dictum that ‘facts are facts, regardless of who says them’ but communication and discussion have more to them than just stating facts.