Essentially if someone has the legal right to sell something, you have the legal right to buy it, unless specifically prohibited by law.
So the big question is whether the company does or does not have the right to sell music. If they do, then it’s just too bad that American artists aren’t getting paid enough. They or their representatives made the legal arrangements for the rights overseas. Caveat vendor (eh, something like that).
But there’s a big difference between having the right to distribute the music within the country in question, which is what the sale of overseas rights would cover, and distributing it worldwide, which is what’s being done here.
Manufacturing a CD is making a copy, though. I don’t work in the record industry, but it seems to me that the (legally made) CDs you find in countries where CDs are cheap are probably manufactured in countries where they’re cheap to make, not imported from the U.S.
We can divide this into two questions: (1) Is it legal for a U.S. citizen to buy a copy of a song that was legally made in another country? (2) Is AllOfMP3 legally making the copies they sell?
The problem with viewing the issue that way is that AllOfMP3 is not actually making the copies at all. What they are doing is offering the general public access to their collection, and the copy is being made by the buyer. If AllOfMP3 is doing anything illegal/immoral–and I believe they are–it would be in the realm of distribution/publication, not reproduction.
The buyer doesn’t have access to the files until AllOfMP3 begins sending them from their server. At that point in time, the entire song is still on AllOfMP3’s drive while a copy of the song is being sent over the internet. It seems clear that only the sender is actually making a copy - the song that the buyer saves to disk is the same copy that was sent over the internet.