The only absolutely correct way to do this is to sum the instantaneous current of all three ungrounded phases to get the instantaneous current in the neutral. However this is usually much more work than is necessary.
If the current flowing in each phase happens to be a perfect sine wave, then it can be completely _described_ by amplitude, frequency, and phase angle. Because frequency is a constant, all we are left with is amplitude (level of current flow) and phase angle. These two parameters can be represented as a vector, and vector math can be used to answer your question.
You simply represent the current flow in each phase by a vector, and add these vectors up. This will tell you the _net_ current flow from all the phases combined, and thus the current that the neutral must balance.
Remember that the vector representation is a vast simplification; you are using a single pair of numbers (magnitude and direction) to represent a continuously changing value (voltage over time), and this simplification requires a large number of assumptions. For example you are assuming that the current flow remains constant and is perfectly sinusoidal. This means that the equations themselves are mathematically exact, but that some of the input data has been thrown away, so your results may not agree with reality if reality doesn't agree with your input assumptions.
If the current flow is not sinusoidal, but includes _third harmonic_, then the answers that these equations give will be well off the mark.
As an even rougher (but faster) approximation, you assume that the phase angles are 0, 120, 240 degrees; in reality the phase angles will differ from this because of power factor issues.
Here is a reference for actually doing the vector math:
http://www.electrician.com/electa1/electa3htm.htm -Jon
