I usually feel one or two a year, of the ten or so that hit where I'm at each year. Most of the time I don't know of them till my FB feed blows up with locals posting stuff like "did you feel that?".
Of the hundreds I've been through, only had two in my life that truly scared me though. Loma Prieta in '89 obviously, was on the public transportation and got stuck in Oakland for about 4 hours till they got the bus bridges up and running. The other was a 4.5 that struck a half mile from my house, and the hundreds of aftershocks in the following day. By the end my brother and I were sitting out front, drinking beer and watching the houses shake as each aftershock rolled through the neighborhood. One earthquake gets your attention, a few hundred in a few hours breaks your trust of the earth you stand on. That shit was so unnerving.