I have a theory as to why soccer hasn’t become popular in the U.S. in the way you’re asking about, but first let’s note that in a certain sense soccer is popular in the U.S. Every middle- to upper-middle-class suburban neighborhood has lots of soccer teams. It’s played there on a highly competitive basis in teams that run from 5-year-olds to high school level. It’s played by both boys and girls. It’s played in most colleges. It’s played to a smaller extent by adults in amateur leagues.
So why hasn’t this popularity extended to professional soccer, let alone to world soccer? Here’s my theory: People’s attitude towards sports can be classified as either treating them as neighborhood games or as gladitorial contests. (Perhaps it can be argued that this distinction varies on a spectrum rather than being a rigid yes/no thing.) Increasingly, professional sports in the U.S. are seen as gladitorial contests, not as neighborhood games. There just hasn’t been any way to sell professional soccer as a gladitorial game.
This wasn’t always true. Baseball in the U.S. from the late nineteenth century to the middle twentieth century was the epitome of what I am calling a neighborhood game. In a neighborhood game, everybody (or at least all males) dreams at one point of being a great player of the game. There are teams of every age group in the sport. The quality of play rises gradually in ability from neighborhood pickup teams to organized local teams to semi-professional leagues to professional leagues. Every father dreams that his son could someday be a great player. Nearly everybody has somebody in a professional league who came from (sort of) their neighborhood. Ideally, a neighborhood game is played by people of varying income levels and ethnic groups. There’s at least a myth that the sport equalizes such differences. The great players of the game are considered role models, both intelligent and moral, and there will be a collusion between the players, the team-owners, and the sports writers to suppress any news story that indicates differently.
Until the '50’s, baseball was the American sport, and it was treated very much on the neighborhood game model. As ethnic groups entered the American mainstream, they each produced great baseball players. Blacks never entered the American mainstream till about 1950, and one indication of this was that they couldn’t play professional baseball in the major leagues until the late '40’s. There were separate, poor-paying Negro leagues until then. Even baseball though lost some of its neighborhood game aura and turned more towards a gladitorial game status in the late twentieth century.
A gladitorial game is one in which the professional players, even though they may have immense numbers of fans, are never seen as “one of us” by most of these fans. The players often come from completely different social and ethnic backgrounds from their fans. There seems to be little connection between the professional players and the good neighborhood players. Nobody has any illusions that these players are intellectual or moral giants, and news stories about them often go out of their way to make the players seem like louts. Most fathers don’t dream of their sons growing up to be a great player. While people will try to find any sort of vaguely scientific support for the intellectual superiority of players of neighborhood games, they will go to absurd lengths to show that being good at playing a gladitorial game has no connection to intelligence and indeed is an indication of the subhuman abilities of the players. Basketball, which has been the most popular sport in the U.S. since the late '80’s, is the epitome of the gladitorial game. Despite its popularity and the immense salaries of its players, many of its biggest fans look on the players as if they were performing animals. Football, which was the most popular sport in the U.S. from the mid-'50’s to the late '80’s, was halfway between a neighborhood game and a gladitorial game.
Professional soccer hasn’t become popular because most American sports fans don’t want it to become popular. They could never get their heads around the idea that a sport which draws its players from middle- to upper-middle-class suburbs could be a glaitorial game.