Why do Americans have such trouble differentiating UK and Australian/NZ accents?

In Colombia I shared a house with a Brit, a Canadian, a Kiwi, and a South Carolinian. Sometimes during dinner it was easier to just speak Spanish.