After rereading I am not sure I understand your situation. I have two questions.

1) Are you sure that the rigid conduit is continuous from one panel to the other?

2) Is the ground and neutral bonded in the first panel (because it should be)?