Swordfish, like other fatty fish, is quite fatty and oily towards the belly area. If you got a piece from only this area (and it’s quite possible; a swordfish is a big animal), then you likely got all belly fat.
I’ve had an issue with mushy fish in the past, but that was because I got ahold of a mackerel or a snapper that had been squashed in the net or in packing. I could tell when I got it home and felt the raw fish - it was squishy like it had been stomped on. Ick.