Interesting thing about boxers is that it's about the last sport where the drug-testing regime is laughable. Pacquio and Mayweather have fallen out over the testing regime. So 'the greatest' there might not be as fair a comparison as in other sports.
Adding some 'minority sports' onto the list, you've got to consider Ben Ainslie and Steve Redgrave.
John Claude Killy and Hermann Maier in downhill skiing.
A statistician thinks Bradman is the best ever, followed by Pele and Ty Cobb:
http://en.wikipedia.org/wiki/Donald_Bradman#World_sport_context