Yes, you should be upset. I've run into similar things in new restaurant construction (vendors coming in before the project is completed and "modifying" our electrical system in horrible ways, then having the gall to send a bill).
In this day of newly heightened security, the hospital really needs to tighten up their control over keys and who's working in their facility. You just can't be too careful.
(This guy's even got me mad!)
