If I connect to an artist in a big way, of course I strive to get at least a few of their best albums - their entire discography if I really like them. But I'm not going to go and collect every album by a band I think are just "pretty cool."
That said, though, I only have a couple of songs by... Amorphis, Anthony, The Attery Squash, The Automatic, Between the Buried and Me, Big Big Train, The Butterfly Effect, Calvin Harris, Deep Purple, Department of Eagles, Dizzee Rascal (!), Dream Evil, Elbow, Eldritch, Electric Six, Engineers, Finntroll, Fozzy, The Gossip, Great Northern, Heed, The Human Experimente (although, admittedly, they've only released one song), The Jam, Jeff Buckley, John Petrucci, Jonsi, Katatonia, Katy Perry, Kevin Moore, the Killers, La Roux, the Libertines, Lily Allen, Lost Horizon, Massive Attack, Men Without Hats (!!), The Ocean, Pain, Pendulum, Phideaux, Prodigy, Pulp, Scissorfight, Screaming Trees, Shadow Gallery, Shaman, Sia...
[deep breath!]
...The Stranglers, The Strokes, Suede, Tarja, Theatre of Tragedy, Turtlehead, U2, Vampire Weekend, The White Stripes, The Who, Will Smith, The Wombats and Zero 7.
Jesus, I only thought I had a handful. Didn't realise how many bands I'd picked and chosen from. Yes, a song is definitely the smallest unit in my book.
Do I feel like I want to hear more music by some of these artists? Definitely. Do I feel bad about not having full albums? Not particularly. Heck, not at all, actually. I've got a few songs that I like, and I just see getting an album as a possible next step, should I decide I like the songs I have enough.