I think it's probably both, in both cases. Now that I don't smoke cigarettes anymore, I find the smell of cigarette smoke pretty disgusting, but I'm also quite aware of the second-hand smoke issue.
I haven't completely given up weed, but I do know that not everyone enjoys the smell. And I wouldn't be surprised if some people are paranoid about the second-hand effects, both the health issue and the possibility of it showing up in a drug test. Justified or not, if these are actual concerns of people, they will affect behavior, and voting.