I heard a recent interview with someone (an Englishman) who was asked why he thought this was the case. His response was that the most popular sports in America have a strong managerial/coaching aspect. ‘Our’ sports have breaks, more substitutions or adjustments, and the coach is a big presence during the game. Soccer doesn’t have that - the sport is played by the team, and while coaching is certainly important, it doesn’t direct the flow of the game directly.
The interview was too brief to go into why he thought it was the case, but it at least points up something in common with popular American sports that soccer lacks.
The game most similar to soccer that is very popular (though not nationally so) is Hockey[sup]*[/sup]. Comparatively, it is a faster moving between offense & defense, more definitive in outcome, and involves more coach interaction. Nonetheless, it is still joked about as being only worth it for the fighting. More seriously, penalty situations involve a shift in the way the team has to play (requiring adjustments). It would be an interesting twist if, say, a yellow card meant playing a man down for 10 minutes.
[sup]*[/sup]To those outside North America - this is ice hockey.