I would consider the following:
1. If you size a feeder, run it from the existing service to the walk in building, set a panel, or gutter, and run your branch circuits from the panel, or tap the gutter, you will (as long as the panel has a main, and you set OCP at the gutter) effectively eliminate the situation that Don has pointed out above. I wouldn't take what Don says lightly, he, from what I've seen, "knows his stuff".
2. Doing it this way will allow the building to be moved in the future by disconnecting the feeder, and "hitting" it with a new feed at it's new location. It makes the building a "self contained" unit.
3. Unless you run the branch circuits from the existing service in separate conduits, you will have to derate the wire.
4. Remember your "within sight of" rules for disconnects.
"Walk in" through the "out" door,
The Watt Doctor