For me, Moonraker certainly isn't one of the better films in the series, but it's not at the bottom of the list either. It gets a bit stupid in places, but is at least still fun.
I'd put Die Another Day at second worst for the series. Second, because while it's really bad and almost ruined the series, it's at least a spectacular trainwreck.
My pick for worst Bond film is easily Diamonds Are Forever. Some of the reasons have already been mentioned here; Most unforgivably, it completely drops the ball as a followup to OHMSS.
The whole thing just feels so cheap and tacky as well. Connery obviously doesn't want to be there, and feels like a completely different character than from his previous films. As far as tone, the whole thing feels more farcical than almost anything from the Moore films (and at times, almost more than the Austin Powers films), and in a way that feels like a big 'fuck you' to the audience.
For me, the best Connery film is From Russia With Love. It's just such a fantastic cold war spy thriller. Connery is at his best, the supporting cast are all fantastic, and the story just clicks.
For OHMSS, I definitely like it more in spite of Lazenby than because of, though he does an okay job for the most part. I will say, if he was going to pick one scene to just get spot on, he picked the right one (the ending). A great all around cast, very unique story and pacing for the series, and phenomenally directed and put together.
I will say, it's difficult to see either Connery or Moore in that film. Interestingly, Dalton was actually considered, but he turned it down because he thought he was too young (around 21/22 at the time). Now that could have been something.
Moore is probably my least favorite take on the character, though I've started to appreciate his films for what they are in the last few years. His films are probably all around the middle of the pack for me; nothing amazing, but nothing terrible. I hear a lot of people say that The Spy Who Loved Me is one of the best in the series, but it never really clicked with me.
While not his best, I always found A View To A Kill to be his most interesting in some ways. Obviously, Christopher Walken is a fantastic villain. There's just an overarching weirdness to the film that's pretty intriguing. Obviously one of the main issues with the film is Moore's age by that point. Honestly, I think they really could have had something if they'd addressed it a bit. Have Bond deal a bit with getting older. They basically reset the series to a degree with Dalton anyway, so it could have worked.
I've always heard that Moore kept trying to leave after his third film or so. That the producers kept convincing him to come back, and he would more or less say "Okay, one last film, but then you have to find someone else". The main reason being, he felt that he was getting too old for the role.
Dalton is fantastic as Bond, and I think it's often overlooked just how important he was to the series. He basically brought the films back to actually being spy thrillers, and reeled things in a bit after Moore's run. The Living Daylights is still one of my favorite Bond films. License To Kill is good too, and Dalton brings it, but I find that film suffers from feeling a bit cheap at times.