It was a base before Castro took over, under Batista. The Cuban army wasn’t ever strong enough to attempt to retake it and we certainly saw no reason to abandon it.
This question was just asked last week (in this thread. Just a quick archive search brought that up.
Guantanamo (aka Gitmo) was around long before Batista. Short version: the U.S. signed an indefinite lease on the base after liberating Cuba from Spain at the start of the 20th century, the nominal payments for which Castro refuses to cash.
Retaking Gitmo would likely be a prelude to a very short war. While we did pledge not to invade cuba, I would think that we’d have just cause for disregarding it if Cuba struck first. And Castro isn’t nearly that stupid…