You are correct, it doesn't matter what size conduit you use, the derating requirements start with 4 current carrying conductors

Roger