Fans definitely have a right to say "we". In fact, I would say fans are more a part of the team than even the players themselves. This gets down to the fundamental question of what really makes an organization.
Think of it this way: Do you stop being a fan because players change? Coaches? Front office? Do you stop being a fan if the current owner sells the team? Go look at your favorite team now, and then go and wikipedia the page of the team three seasons ago. Most likely, there are more new faces on the team than players and coaches you rooted for just a few years ago.
I would argue that FANS are the only real constant in sports. Everything else is a revolving door. That's why, if you're a true fan, you don't quit being a fan a find a new team to root for just because you don't like the current QB, or GM, or owner. That's why a true fan will stick with a team through good and bad, as players, coaches, front office personnel, and owners come and go. So, yes, fans ARE entitled to say "WE" and, in fact, fans may be the ONLY people entitled to say "WE" for the longhaul. Every other facet of a team is essentially a hired gun, supposed to be working toward bringing fans a successful franchise.