You really, really should not take rhetorical advice from your MRA or GamerGate buddies. Calling someone a social justice warrior means pretty much nothing to most people (well, other than telling us who you hang around with).
Those six words you said don’t matter because the rest of what you say completely contradicts them. When you say that people should do X, and they didn’t do X, you are inherently blaming them for not doing X. That’s just how language works.
What you did was the equivalent to saying “I’m not racist, but <racist statement here>.” You said “I don’t blame him, but <statement blaming him>.”