I'm the same way. In many cases, an offset nipple will offer enough leeway to accommodate inconsistent knockouts. I also agree that the cost difference between a tiny panel and a full-sized one is negligible in the grand scheme of things.
Unless the original panel is a Zinsco or FPE, I always try to use the same brand/style.
