Where's the 480/208 transformer? Are you keeping the neutral for the 277 isolated from the neutral for the 120 isolated from each other in the transformer?
No matter what you do, you need a ground wire between the buildings, and a grounding electrode at the detached building as well.
IMO, you should keep the 277 neutral apart from the 120 neutral at the transformer. Indeed, I see no reason to bring the 'gray' neutral in with the transformer feed at all. This makes the transformer a 'separately derived service'.
[red] ECN regulars double-check me on this: [/red]
With a separately derived service, the first breaker after the transformer becomes the point where you separate your grounds from your neutral, just as with your service from the PoCo. You'll want to use bonding bushings on your connectionc between the transformer and that breaker.