I’ve lived in both NZ and Australia - would generally say ‘going into town’, although I’m familiar with both ‘high street’ and ‘main street’ from time in the UK and USA.
High Street is a very British thing. Here in Ireland if one was being specific it would be “the main street” but usually it’s just “town” or you’d name the street.
ETA: In a lot of Irish towns/city the main street is called “O’Connell St.”
The street itself is the “main street” in Australia, even in cities like Penrith or Maitland where it is the High Street – though perhaps Henry Street is the real main street of Penrith.
That part of the city is the CBD, or central business district – a term which seems to confuse people in the United States.
The terms I know of (in a town of 8000 or so)
CBD
the main street
the main drag (I live on the main drag - the bit with the shops is the shopping drag)
the shops: as in heading down to the shops or going to the shops or heading for the shops
head down [insert street name here] street
town centre
“Down the village” too. 
I agree with “The main street” in New Zealand and Australia except that my little town in NZ had streets of equal “mainness” so we really had to use the name.
I’m not sure why it reminds me but anyone old enough from that part of the world who remembers a stupid Australian song will know that the main street of Newcastle NSW is called Hunter Street.
http://lyricsplayground.com/alpha/songs/t/thenewcastlesong.shtml
That takes me back. I lived in Newcastle a long time, and knew Bob Hudson around the time that he published the Newcastle Song – though I’ve lost contact with him. My other connection with him is that his then wife went to the same high school as my wife. (Newcastle is a small town).