if you can listen to them and hear other parts of the music that you love, than they don't ruin the music. Granted, I can attest to finding some bands whose vocalists I don't care for, ruin music I might enjoy. So I can relate to the issue with them, it's just in a lot of cases, the growling or shrieks don't distract me, and eventually seem to complement the music.
Some exceptions tho. Some of the hardcore punk or posthardcore bands, the vocals suck, and are at times too high in the mix to not distract, I think of bands like Deftones or Thrice even as examples of that.
But with a lot of Death and tech/death, the vocalists often can complement the music. I suppose it can depend on how samey they may sound, on a song or album. There are examples when every vocal line sounds very much the same, and ends up hurting the music. I guess it can depend on how much they are used, and if they mix them with clean vocals.