Maybe it's because I've done this so much (in process now of buying my sixth home) or because I a lawyer (no jokes please) but I left the "feel bad" thing behind long ago, for several reasons. One, the "investment" is purely subjective. When it comes to homes everyone has their "personal issues", meaning some of those "investments" aren't really. Just because you spend "$x" dollars on a house doesn't mean that the value goes up the same amount. In fact, more often than not, it has little if any effect. Two, can't speak for your case, but in mine (also buying from an elderly couple, been in the house 45 years!!) there are agents and lawyers on both sides, and in fact I played HARDER ball because of the presence of the agent. I don't know if she felt some obligation to the couple to be a pitbull, but she was totally obnoxious and on several occasions was abusive to our agent. So for me, gloves were off. Three, she set the price, and she - as a presumably able and competent adult - had the right at any time to say "no deal".
Seriously, if you feel that bad and you want to make right, do it on the side out of the transaction, but within the terms of the home purchase you owe it to yourself and your family to cut the best deal you can. You only have one shot at it (outside of re-financing).