I think you confused 'grounded' with 'grounding' conductor.

While there are circumstances were we can reduce the size of the neutral, a service isn't usually one of them. You would have to be able to show that the bulk of the load was from 240v circuits, and size the neutral to serve ALL of the 120v. loads.