A sattelite photo of Wards Island, now joined by landfill with Randalls Island to its north, doesn’t show a whole lot of single-family residential buildings. In fact, I don’t see any when I zoom in.
The Marble Hill section of Manhattan has many detached single family houses.
And I seem to recall that there are a few detached houses in some of the little, out-of-the-way streets in the West Village, like Macdougal Alley.
Eh, scratch that. It was between 1st and 2nd on 82nd but it’s been torn down.