Sounds like another home inspector that has his head up his a**. I don't think we have wired any homes in the past 20 years that the drywall contractor didn't get mud on the wires in every box. Its best to clean as much of it off as you can but a little mud is not going to hurt anything as long as it is not on the panel bus bars or in any of the connections. There is no reason to rewire you house because of this.
