Well, you’re not entirely making up the Bay of Fundy part, but there’s no land border.
Nova Scotia is mostly an oddly shaped peninsula, with an island offshore to the north of it. The Bay of Fundy is an expanse of water between Maine, (solidly on the continental mainland,) and the southern extent of the Nova Scotian peninsula. To the north of the Bay is New Brunswick, which is the only other region that Nova Scotia has a land border with - that’s where it connects to the rest of North America.