Yeah, in the past, it's been a group of like 8-9 people who just go in and vote for either the popular favourite, or a particular song of their choice.
At one point during last year's Opeth survivor, I had mentioned that The Drapery Falls was my favourite Opeth song. Because I'm vocally opposed to trolling, the group of trolls decided to spend several rounds attempting to vote it out. No one who actually posted their votes were owning up to voting for it, so it was pretty clear that it was a trolling problem. I put the survivor on hiatus for about 2 weeks until the trolls were called out and stopped. During that time, several other survivors got trolled.
For the most part, it's pretty obvious who is trolling. When someone vehemently defends the idea of trolling with the classic "srs bsns" argument, they are usually put on my radar. Not saying that everyone who doesn't mind trolling is a troll, but many are.