If you add an even number of fish, they will form "teams" and beat up on each other. Since the two teams are equally matched, they will fight and fight and fight, possibly causing death to the fish on other teams. They see each other and are like, "Hey, that fish team has two guys! We have two guys too! We can take them!"
If you add an odd number, say 5, they form "teams" of 3 and 2. The 2 fish team looks at the 3 fish team and automatically thinks, "Oh man, we'd better not fight with those guys, they've already got one more fish than we do." So adding odd numbers of fish usually decreases the chance of fighting among them.
That is, naturally, the scientific explanation. :D
Of course, a lot of fish do well in pairs. So maybe you should stick to those species :).