Since the neutral path back to the main panel is broken, you aren't drawing current from a hot wire and neutral. You are drawing current from the hot wire and all of the appliances, lights, stereos, etc on the other hot wire.
When the load is balanced perfectly between the two legs, the voltage will be balanced. Any imbalance in load will cause an imbalance in voltage.
