If this is a pancake box for hanging the paddle fan he might be right. I seem to remember that the screws for mounting the paddle fan pancake box are special (sometimes). If your just using a standard pancake box to junction the wires, the only concern I'd have would be the fill in the box. And since he knows you are an inspector, he's probably being a little bit anal and telling you "Hey, those screws are for drywall"
