I did enjoy the episode and I'm not saying he's terrible, but I'm still just not really threatened by him. It's like, yeah I get what he's capable of and I know I should find him scary, but I don't for some reason. But no, that didn't ruin the episode for me or anything. Sherlock's ridiculousness and his relationship with Watson are really the reasons I love the show, with the antagonists just there to sort of... shake things up.
Now that I think of it, I haven't really been super-impressed with any of the others. The taxi driver guy was kinda lame (I really liked the story leading up to his identity being revealed, though I was a bit surprised Sherlock didn't figure it out sooner- it was kind of a DUH) and the Chinese smugglers, meh. I really liked the whole HOUND concept but there was nothing special about that dude who was the brains behind it. Not sure how to classify her, but I do like Irene.