I have no idea when you're leaving, but all of these fit into your rough time frame. Pollstar is the best source for figuring stuff like this out.
Rush at the O2 on 5/25
Terry Jones and Michael Palin are doing something together 6/13 in London. Don't know what, but how could that possibly even matter?
Roger Waters is all over the UK and France throughout May and June. You could actually see the show that Gilmour plays at.
System of a Down in Paris 6/6 and 6/8. I think they suck ass, but to each their own.
Judas Priest 6/20 in Paris
Primus 6/22 in Paris
Maiden 6/27, 6/28, Paris
If you're going to check out Versailles, be sure to check the schedule. They only run the fountains on certain days of the month (and even then, quite sparingly). I wouldn't bother, myself, but you've got 3 weeks, so why not.
My favorite thing to do in France was to head to a point 3 random metro stops away and just wander. Go someplace completely random and seek out something to eat.
Start walking now. Make sure you can cover 10 miles without feeling sore of tired, because you'll do that pretty easily once your in country.
Edit: as a matter of fact, if I were in Paris on 6/13, I'd probably ride the Chunnel train back to London just to see what the hell Palin and Jonesy are up to.