I assume these voltage readings are taken with the normal load on in the barn.
Did you measure the current in each hot and in the neutral?
It looks like the wire is too small and the load is unbalanced.
For that distance with a 40 amp load you need #4 AL or #6 CU to limit voltage drop to 3%
Ed
