Let this Yank take a wild guess or two...

Assuming that your rcd's trip at a low level, say 30ma, it mould resemble a problem I had with a UPS system once; in that case, there was a connection between the utility neutral and the back-up system neutral that shouldn't have been there.

Paul, you may also be on to something...if the generator is diesel, and has a heater in the fuel tank, and that continues to be powered by the utility while the generator is operating, this sort of problem might come up.

Failing these ideas, are we sure that the transfer switch is operating properly- or is the system getting a little crossover power for an instant during the transfer?

Just some thoughts.