My first thought is that Christ is not judging someone who divorces & then later on meets someone & marries, but that He is against divorcing a spouse to then take another.
Also, Christ notes that Moses & the Torah allowed divorce because of hard-heartedness, a part of human sinfulness, but is telling His disciples that they should be better than that, & so He only allows divorce/remarriage in cases of adultery (and I think that is not only sexual infidelity but any serious betrayal of trust, and would include abandonment & abuse.) Paul backs up that interpretation in I Corinthians 7.
Finally, for Christ to totally disallow divorce/remarriage makes Him harsher than Torah rules, and if an interpretation of Scripture makes Christ less Gracious than The Law, then that interpretation should be highly suspect.
Btw, Eastern Orthodoxy also allows for divorce & remarriage. And it can be argued for all practical purposes that Roman Catholicism does also but just calls it ‘annulment’.
One last note- Christ is answering if divorce is allowed “for any reason”. We think that Christ is saying “No, only for adultery. Other than that, no other reason.” However, the Rabbinical school of Hillel allowed “divorce for any reason” (that day’s equivalent of no-fault-divorce) whereas the school of Shammai only allowed divorce for serious reasons, such as adultery. It’s VERY possible that Christ was addressing the Hillel vs. Shammai debate. And if so, it’s significant because Jesus is usually regarded as closer to the Hillel school due to its lenience & graciousness as opposed to Shammaic emphasis on Law.