The interesting question for me that arises out of the "the very best acts (Grandmaster Flash, Run DMC, Public Enemy, NWA, Biggie, etc) are both excellent within the terms of the genre and extremely popular" is what that says about the hip-hop audience. Is it more knowledgeable? More apt to recognize talent and quality?
My theory has always been that because a rapper doesn't so much write music as lyrics that are then arranged to other people's music his talent as a lyricist is front, center, and undiluted."
When I was writing the post, I was trying to think of how the analogues work in other areas of aesthetics. I came to the conclusion that it's the American indie aesthetic that's backwards, which makes us perceive hip-hop as being bizarre in this regard. After all, think of the underlying premise of hip-hop: those acts which are significantly better than average will become very popular. The relationship between the two is clear, and, moreover, exactly the relationship we might expect.
The same is true in literature, so far as I can tell, at least through the early 20th century: Dickens was unbelievably popular, as was Balzac, or Jane Austen, etc. The first case I can think of in which a retrospective judgment of quality does not match initial popularity is The Great Gatsby, but Fitzgerald was at least also a great short story writer and wildly popular at that (it makes sense that the audience for "A Diamond as Big as the Ritz" isn't quite the same as that for Gatsby).
It's not until, perhaps, the 50s and the rise of 'cool' that one gets credit for liking things other people actively dislike (it was a matter of some caché, though not much, amongst my high school friends, to own, or at least have listened to, some portion of Lou Reed's Metal Machine Music, which is a horrific assault on the ears); but it seems like this actively overturns most of the history of aesthetics. But maybe I'm wrong about the history--this is something of a blind spot.