Romantic? Go to the Griswold Inn in Essex. Been in operation since 1776 (yes, you read that correctly) and their tavern is often mentioned in "best bars for each state" things. It's got a nice fireplace and is very warm and cozy. You will miss it, but in February, the restaurant does a "game menu" and it's pretty special (if eclectic).
The Black Seal - just up the road - used to be very good. When I was married the first time and spending a lot of time down by the shore, that was a go-to place. Sorry, but Norwich itself is kind of a dump, and everything around it is pretty nice. If you have some names and want to run them by me, PM me. You also have the two casinos there (Foxwoods and Mohegan). Not very romantic or New England-y, but they are always packed for a reason.