If you are cutting in existing the 6" gives you a bigger hole to drill & fish with. The 5" will fit in tighter places like between joists & pipes but some are 1/2" taller. Have not found a 4" remodel IC yet. So it's new construction 4" or clear away if any insulation.
As for looks I don't know if most customers could spot the difference between a 5" & 6" can. Smaller cans look better in smaller places. Depending on the can & trim but some 5" will take the same bulb as a 6".
The 5" & 4" cost me more so it is passed on to the customer.
Tom