I respectfully disagree. I think many of the more unique and interesting rappers have done their best work post-2000. Guys like Drake, Kanye, Kendrick Lamar, Lil Wayne, are really popular, and they make great music.
Well, I think it’s important to remember that music of previous generations is always curated to exclude all the shitty stuff that didn’t age well. That said, I think today’s music is MUCH broader in terms of range and personality. Now, we are seeing people make music who grew up listening to rap in its fully evolved form. Now, you can be a respected, popular rapper without having to sell your back story about how you sold drugs, got shot, and grew up poor.
Regardless of how you feel about their music, people like Iggy Azalea, Childish Gambino, and even Kanye could not have done the music they do 15 or 20 years ago. Artists like Estelle, Nicki Minaj, Janelle Monae, and MIA would have have to dramatically change their style to fit into a narrow genre. They certainly would not be working as much across genres with musicians and bands like Fun. or Diplo or J.R. Rotem. You almost NEVER saw popular rappers who grew up rich or middle class, or those from Australia or Canada. Now such things are rarely questioned.
But that is a function of fewer people making and listening to Jazz music, not because the music suddenly became shitty. I think if you have a critical mass of people making a certain type of music (as you do with Rap), you are likely to get pretty similar quality over a given time period.
I think what you hear in the top 40 is safer, but I don’t think it’s necessarily limited creatively speaking. It’s probably more a little repetitive because popular music (which includes Rap) relies a bit more on specialization than less popular music. By that I mean that rather than a group like Wu Tang, or Gang Starr doing essentially everything in house, most rappers pick and choose between top producers and collaborators in order to create a safer, popular sound. It’s just more of a business now. The same is true of movies. The top 40 is the equivalent of the summer blockbuster movie market. It’s mostly known quantities and sequels. Some of it is still worth seeing or listening to, but it’s purpose is to provide people with a diversion they can use to pass the time and sing along to; not (primarily) to challenge the audience. The same applies to other genres as well. It’s not a coincidence that Pharell produced, wrote, or performed on many of the top songs this past summer, or that bands like Fun. use producers known for rap music.
That said, given the number of talented musicians working and the democratization of the tools of production and distribution, I think FAR more good music is being produced today even if it’s harder to find.