I'm probably somewhere in between "I can take them or leave them" and "I'm not a fan but I still listen to some bands who use them," with a splash of something else. I voted "not a fan" because that's probably a bit closer to my opinion, but it doesn't quite fit.
I'm not against them entirely, but I think they should be used judiciously. If an album is primarily harsh vocals, I'm unlikely to enjoy it; I think that's too much.
On the other hand, at their best, I think they can enhance certain moments, but usually do so when they are unexpected. I think the way they're used in The Human Equation and 01001101 from Ayreon, for example, is pretty strong. And the couple of small sections of growling on Kamelot's The Black Halo are perfectly placed.
In the middle, when bands use growls as something like 20-50% of the vocals, I can get accustomed to that and enjoy their albums, but I wish they would be a little more selective with when they use those elements.