Was there freedom of migration/movement in the old British Empire?

What was the British Empire has been fragmented into many little and not-so-little independent nations. When the Empire still existed, was there freedom of movement between the realms or other areas that eventually became independent? For example, could someone born in Kingston, Jamaica whose parents were born in Jamaica just up and move to Boston, Massachusetts in 1720 if they could afford it, or did they need some sort of migration visa or pass? Could someone from Nova Scotia just up and move to Bermuda, New Zealand, Rhode Island, Maryland, the British Virgin Islands, or Newfoundland?

Did this vary over time, or did restrictions on migration only come about when regions became independent (e.g. moving from Ireland to Pennsylvania or from New York to New Brunswick became immigration matters only after 1776).

If you were white, sure. I don’t have my books here at home (they are all in my classroom) but IIRC migration from one part of the Empire to another was pretty easy if you had the cash.

My understanding is it used to be super easy, and it slowly became more and more difficult throughout the 20th century as the UK wanted to avoid various persons who were technically born as citizens of the Empire from being able to just mass immigrate to the UK (most of these people would be third worlder types and most first world countries want tight controls on how many people from undeveloped countries immigrate.)

My great-grandfather moved from England to Canada and, so the family story goes, no paperwork was involved. This was in the late 19th/early 20th Century. He was white, but was also Jewish - so competing lines of prejudice there.

It used to be super easy to move from anywhere to anywhere. As far as government officials were concerned, that is. If you wanted to move to France, you just up and moved, and nobody cared. The reason this didn’t happen very often is that it was extremely costly to move to another country or continent, and how are you going to make a living once you get there?

So it wasn’t just that there was freedom of movement within the British Empire, but there was freedom of movement everywhere. Unless you were a slave.

What about people transported from England to the penal colonies in North America and Australia? I thought they were not allowed to return to England, ever (I’m sure people *did *return, but we’re talking legalities, not practicalities). What about traveling from a penal colony to some other part of the Empire - were there penalties for that?