Yeah, unfortunately there aren't more studies of and dissertations based on pop music. Academia is usually a bit slow on figuring out that "oh wait there's a new genre of music" (jazz didn't start entering into academia until the late 60s/early 70s, iirc).
Clercq and Temperley's "A Corpus Analysis of Rock Harmony" is another good one; it focuses on using statistical analysis to determine what's actually going on in pop and rock harmony. It's a hugely influential essay for me as a music theorist, and it helped me appreciate what's going on in a lot of pop music much more once I understood that the rules were different from what I was used to lol. As far as edutainment lighter stuff,
Todd in the Shadows is a good critic to watch and
Holistic Songwriting does a fantastic job on using some theory and industry knowledge for breaking down pop and rock artists. Understanding breeds familiarity, familiarity breeds enjoyment, and enjoyment breeds more understanding!
Back to your main point, tho, I think it's important to remember that there was overall
less music being produced and released then. The last 20 years or so have democratized music and boosted the population to such an extreme degree that there's an absurd amount of music being produced nowadays. It's not that there's
less good experimental and left-field music pop music being made (there's a lot more of it now), it's just that there's a lot more stuff out there that's competing for your attention. And, as usual, most of it is mediocre.
Even arguing for popularity, a lot of these more experimental outings are incredibly popular. Radiohead's last album was massive on social media when its first single was released, FO's Blonde hit #1 and has spent 116 weeks on the chart, Kendrick Lamar/Kanye/Tool are pretty much household names, etc. etc. Heck, Snarky Puppy and Joey Alexander have gotten a lot of attention and Dream Theater hit the Top 10 with The freaking Astonishing. Even major artists like Katy Perry are having trouble putting butts in seats because it's 2019 and everyone is still poor and honestly pop is in a temporary slump outside of Taylor Swift (because of sad rap becoming the "musical embodiment of the zeitgeist" of the last 5 years).