In the comic book world, all of these characters exist in a single shared universe. So yes, the Fantastic Four, Spider-man, and the Avengers could all get together and play cards if they wanted. Almost every year the company publishes at least one series that involves all of these characters coming together in a giant battle.
The problem is translating this same dynamic to the screen. The common industry practice has been to sell the “rights” to make a movie based on a certain character or intellectual property. As you have observed, Sony owns the rights to Spider-man, Fox owns the rights to Fantastic Four, etc. These contracts inevitably have an exclusivity clause. Since Fox paid Marvel for the right to make an FF movie, Sony should not be able to make an FF movie (or include FF characters) and piggyback off Fox’s success.
As I’m sure you know, the reason the “Avengers” is such a big deal is that it bucked industry convention. Marvel retained the rights to these characters and, as such, can allow them to have cross-overs or cameos in each other’s movies.
This also leads to some interesting speculation… For example, Marvel sold the film rights to the Punisher to Lion’s Gate. However, they recently bought those rights back (essentially, they paid a fine in order to break their contract with Lion’s Gate). This implies (A) Marvel wants to make more Punisher movies and (B) the Punisher could conceivably appear in the ongoing “Avengers” series.
Now, that’s all well and good as an explanation of how intellectual property rights work, but I still haven’t answered your main question.
As to why the FF films sucked, all I can say is nobody knows. There are a thousand reasons a film could tank, but most of them revolve around getting a committee of people who want to publish something aimed at the lowest common denominator. If anyone could identify a single reason why films fail, then it could be repaired and every movie would be a hit.
Look at the recent “John Carter” movie. They had a property that was considered an enduring classic, a director who bats a thousand, and one of the most experienced and wealthy film companies to give it an unlimited budget. And yet it was one of the most disastrous movies of all time. You can point to various flaws, but nobody can nail down one single reason the movie failed.
Seen through this lens, maybe Marvel’s decision to buy back the “Punisher” rights was in an of itself punishment for the studio for making awful movies.