The real problem is the work that doesn't get inspected. If you get rid of the home owner permit, the work that didn't get inspected still won't get inspected. In addition, the work that was inspected under the home owner permit, won't get inspected either.