Meglaodon, draw straws?? No way, the fat guys out, plain and simple.
Seriously though, you bring up some good points, but keep in mind that the raft is a lifesaving device to only be used, God forbid, in an emergency. It's not a comfortable cruise type thing. It's designed to help save your life and if they can fit 20+ guys in a VW Bug, I think you could squeeze 4 men into a 4 man raft. Yes, bigger is better, but there are also other things to consider, i.e., weight, there is a BIG difference between the 4 man and a 6 man and for me and my boat, I don't need that additional weight on the hardtop, which is the only place I could put it, unless of course you get the valise type which I wouldn't recommend. Often times when an event happens that you need these things, it happens very fast and having your raft stowed down below might come back to bite you real good.
Just my $.02