It sounds better in Latin.
The explanation I once heard was that capital cities are usually chosen because they’ve in the middle of their state. Look at this map.
But the center of a state is a bad place to locate a city. Cities need a strong economy to build a large population and a strong economy requires good transportation. And for most of our history, good transportation meant being on a major river or the coast. And major rivers and coasts tend to be at the boundaries of states not in the middle of them.
Places like Austin, Sacramento, Albany, Columbus, Harrisburg, Springfield, Lansing, Jefferson City, and Columbia have central locations. Places like Houston, San Francisco, Los Angeles, New York City, Buffalo, Cleveland, Cincinnati, Philadelphia, Pittsburgh, Chicago, Detroit, St. Louis and Charleston have access to shipping.
I am not saying Harvard’s education model is shoddy, it is just that the major idea isn’t to teach but to bring the best and brightest together so that things spontaneously happen. That model certainly works for entrepreneurship but it doesn’t work for fields like engineering or hard science. Schools like MIT, Stanford and CalTech specialize in that. To put it in other terms, if we needed to do another moon landing in 5 years, Harvard grads may make fine managers but I wouldn’t want them in charge of the engineering details.
And no, not every good school is going to have people that attended like Bill Gates and Mark Zuckerberg. They are among the richest and most influential people in the world. It would be rare for any school to have even one person like that let alone multiple ones. It is all about exclusivity and selection criteria especially for the ones that never even finished or bothered with their classes much at all.
Mostly.
The ‘STATES’ are NSW, Vic, Tas, WA, and Queensland and South Australia. Our mainland territories are the Northern Territory (as you mentioned) and the Australian Capital Territory which houses Canberra, the official national capital. It’s where the politicians hang out, so we like to just forget it exists sometimes. 
I don’t think that New York should belong in that list. Most people will know the city of New York, of course. But many won’t have a clue that the state is also called New York. My guess for the top five would be:
California
Texas
Florida
Alaska
Hawaii
I’m across the pond!
Here’s what I think are US States (and why I remember them)
- Nevada (because I visited Las Vegas)
- Utah (Shakespeare Festival)
- California (because I visited Los Angeles)
- Arizona (borders Nevada)
- Florida (CSI Miami)
- Louisiana ('Bond film ‘Live and let die’)
- Delaware (Perry Como song)
- New Jersey (Perry Como song)
- Hawaii (Hawaii 5-0 “book him, Danno!”)
- North Carolina (err … North?)
- North Dakota (err … North?)
- South Carolina
- South Dakota
- Pennsylvania (Sherlock Holmes “Valley of Fear”)
- Michigan (dunno)
- Wisconsin (went to a roleplaying convention in Milwaukee)
- Puerto Rica (“West Side Story”)
- New Hampshire (“West Wing”)
- Vermont (“West Wing”)
- Washington State (dunno)
- Illinois (Chicago)
- Texas (“The Alamo”)
- New Mexico (“The Alamo”)
I’m pretty confident about that lot - here are some more that might be States:
- Baltimore (“Life on the Street”)
- New York (famous!)
- Georgia (“Georgia on my mind”)
- Colorado (Denver)
- Massachusetts (dunno)
Puerto Rico is a commonwealth of the U.S. It isn’t a state, although Puerto Ricans are U.S. citizens. Baltimore is a city in the state of Maryland. There is both a city and a state of New York. People often refer to New York City and New York State to avoid confusion. New York City is a part of the state of New York, although the state is huge, with other cities such as Albany (the state capital), Schenectady, and more.
Columbus has the Ohio State University, which is immense.
And poor Cleveland has been bleeding residents (like me) for decades. When I was a kid it had the 8th largest urban population in the country, and was full of artists, beatniks, urban sophisticates, intellectuals, and Ghoulardi. It’s still home to the best U.S. symphony orchestra, IMO.
Cincinnati, I know nothing about. It’s at the other end of the state, abutting (ugh) Kentucky.
I’m in the UK, and I would say:
The four you’ve mentioned - California, New York, Texas and Florida.
I think most people would know Rhode Island because of the trivia of it being the smallest.
Tennessee, thanks to Jack Daniel’s, Memphis and Nashville.
Nevada, because of Vegas.
The two “extra” states of Hawaii and Alaska. I have had many an argument with people in the UK who actually think there are 52 states.
Anyone who has actually spent any time trying to learn them will get the North and Souths, the News, and the Ms next.
I remember my Mother told me that when I was a kid. She was around when it happened and mistakenly believed it was 50+2, rather than 50 after the 2 were added. I think 52 being a commonly heard number (cards in a deck, weeks of the year) was partly to blame.
I don’t remember when I finally reconciled that there were 50, but I was still in my early teens I think, maybe younger. For the record, America never came up in my years of High School history or geography classes. We had plenty of local knowledge to learn instead. Everything I know about America is assembled pretty much only from pop culture, or places like this very board. I’ve learned a bunch of things, but I’m still burdened with a lot of ignorance.
As well as Salt Lake City, Utah.
Trivia time: Fillmore, Utah was created in 1851 to become the capital of the newly formed Utah Territory, and named after President Milford Fillmore. (Fillmore is located in Milford county.) Great Salt Lake City (“Great” was dropped at some point) was the unofficial but real capital as Brigham Young, the territorial governor maintained his residency in Great Salt Lake City. After a few years, they did move the capital.
The official city limits of Columbus make it the largest in Ohio, but that’s only because it absorbed most of its suburbs. In terms of the entire metropolitan area, it is in fact still behind Cleveland and Cincinnati.
Puerto Rico, mentioned by glee, is another special case in the US, in that no other place has quite the same status as it. It’s sort of vaguely kind of like a state, but even less so than DC. Whether to become an actual state (or at least, try to: They’d need approval from Congress) or not is the perennial hottest political question there.
Based on the actual responses of foreigners in this thread so far, here’s the current count:
Alabama
Alaska 2
Arizona 2
Arkansas 2
California 6
Colorado 2
Connecticut 2
Delaware 4
Florida 6
Georgia 3
Hawaii 4
Idaho 2
Illinois 4
Indiana
Iowa 1
Kansas 4
Kentucky 1
Louisiana 2
Maine
Maryland 3
Massachusetts 3
Michigan 3
Minnesota 1
Mississippi 1
Missouri 1
Montana 1
Nebraska 1
Nevada 3
New Hampshire 1
New Jersey 2
New Mexico 4
New York 6
North Carolina 3
North Dakota 4
Ohio 2
Oklahoma
Oregon 1
Pennsylvania 2
Rhode Island 4
South Carolina 3
South Dakota 4
Tennessee 1
Texas 6
Utah 2
Vermont 2
Virginia 3
Washington 4
West Virginia 2
Wisconsin 2
Wyoming 2
Non-states guessed as states:
New England 2
Winnipeg 1
Cleveland 1
Washington, DC 1
Puerto Rico 1
Baltimore 1
So far, no foreigner has gotten
Alabama, Indiana, Maine, or Oklahoma.
States have been spoilered so as not to influence any other foreigners who might still want to try to guess.
But if someone says “Kansas City” they mean the one in Missouri. Kansas and Missouri share a border, and the Kansas City metro area spans that border (resulting in two "Kansas City"s, technically), but the part in Missouri is larger and more well known.
It’s only because of the arbitrary nature of political boundaries.
The populations of the Cleveland and Cincinnati metropolitan areas taken as a whole are much bigger than the Columbus metropolitan area’s.
It’s just that the way the municipal boundaries are drawn that makes the City of Columbus larger on population than the City of Cleveland and the City of Cincinnati.
That’s interesting because:
[spoiler]Alabama, Indiana and Oklahoma: Famous songs
Maine: Lobster, Stephen King.
I hope people don’t think of “Maine lobster” as “main lobster”. 
[/spoiler]
ETA: I had to delete Chronos’ spoiler box because the new software left it unspoiled in my reply.
What’s the famous Indiana song? All I could think of is the Indiana Beach’s song “There’s More than Corn in Indiana” that we were pummelled with over the TV airwaves here in Chicago back in the 80s/90s.
New South Wales is the one I hear the most. Is there also a Victoria?
“Commonwealth” has no legal meaning. Massachusetts, Virginia, Kentucky, and Pennsylvania are all commonwealths. The official distinction is “territory.”
Millard, not Milford. Which proves how well that particularly crappy president is remembered.
Not “much bigger.” The Cleveland MSA is only slightly bigger than Columbus. Cincinnati is bigger than both by about 100,000 people. It also includes parts of KY and IN though.
As I understand it, Ohioans have described it as Cleveland has all the “stuff,” Cincinnati has some too but is a little more run down, and Columbus is kind of boring. Your mileage may very much vary.
This discussion of New England (by the way, could your friend be thinking of the fact that Connecticut claimed (and settled) some land in what is now Pennsylvania?) reminds me of the “Tri-State Area” question. New York and New Jersey are definitely part of the Tri-State area, but there are some who would call Pennsylvania the third state in that area (making a New York and Philadelphia axis) while most would say that Connecticut is the third state in the “Tri-State”.