The math is simply not applicable in this case.
The equations for calculating neutral current given the phase currents make the assumption of pure sinusoidal current flow. If the current flow is _not_ a sinusoid, then the equations cannot be expected to work.
In a perfectly balanced system, the _fundamental_ current flow is exactly 120 degrees out of phase between each phase. There is no neutral current flow. The phase difference means that current flowing out of one phase goes directly into the others. In a real system, of course, the current flow probably isn't exactly 120 degrees out of phase, and the individual phase currents are not balanced, so you expect _some_ neutral current to flow.
In a perfectly balanced system with third harmonic current flow, the third harmonic component of the current flow is exactly 3*120 = 360 degrees out of phase between each phase...but 360 degrees == 0 degrees. In other words, in a perfectly balanced system with third harmonic, the currents are _in phase_ for all of the phases. Current flowing out of one phase will be matched by current flowing out of each of the other phases, and the only place for this current to go is down the neutral. In the perfectly balanced system, the third harmonic current down the neutral will be 3x the third harmonic current in any given phase. Of course, again, in the real world the currents will not be balanced, but the third harmonic current on the neutral will be pretty darn close to the _sum_ of the third harmonic currents on each of the phases.
If you can arrange it, put an oscilloscope with a current probe on the neutral in question. You'll probably see lots of 180 cycle and 360 cycle current flow.
Non-linear loads in general introduce harmonic current flows into electrical systems. It could be a switching power supply or a linear power supply; somewhere there will be a diode rectifier on the input. This diode rectifier is a classic non-linear load, only conducting in peaks for part of the AC cycle, providing a rich supply of harmonic current to the power system.