Yes, you are correct. I was speaking with our local ESA inspector and we were talking about this as well. Todd Brown (ESA) said that a homeowner must sign a form that says they have done thier own work, he also mentioned that one can take paermits out for work for: family, service groups or church, basically he said that if you are doing the work for free there isn't a problem it's when someone is taking work away from contractors that causes the issue.
