The first thing that I would eliminate is the possibility that you’re not reading the neutral current.
If all of the phase conductors pass through a zero sequence CT (probably not because of the 3000a it would take a big CT) but not the neutral the neutral current is being seen as GF current.
I would do every thing I could to eliminate the possibility that the neutral current is not included in the GF sensing.
In other words the unbalanced current is being seen a GF current.
What else could cause a current great enough to trip a 3000a breaker if the GF pickup were set at the highest setting?