If you’re just talking about fat then you just don’t like the prized portion of the fish. Swordfish are often quite lean today because the smaller ones are more easily caught. You could easily be surprised to have a choice fatty piece of fish when you expected typical loin meat. If was all fat, not just fatty meat, then you bought an unusual piece of sword.