There are many islands in the Bahamas. I was stationed for a year at a LORAN station on San Salvador. We had water. We had a well and a catchment to collect rainwater. The main Hospital Corpsman type of thing our corpsman did was a daily check on the quality of our water and I suppose add stuff as needed to make it safe for drinking. Locals would send kids with buckets or coolers to us and we’d give them some ice. We had electricity. There were 3 diesel powered generators in our Signal & Power building. One was online, one was on standby, and one was the spare. They gave us power for the LORAN equipment, the galley, lighting, well, everything. There wasn’t any other electricity going to our corner of the island. Nearby locals used candles, oil, and Coleman type lanterns for lighting. There was running water and electricity in Cockburn, the town on the island. There was also power and water at the retired US Navy station from the Mercury/Gemini days. It had been bought by a group of colleges in the Finger Lakes area of NY as a remote campus for their Marine Biology students. I imagine they got their power and water the same way we made ours.
The thing about the Bahamas is even if some land is cheap, not much is available there. You want to build there, you’re going to have to arrange to have lumber, nails, stuff like that brought in. Hope you like sea showers too.