That's exactly what it is! Part of it is a bridge, and part of it is a tunnel. The tunnel part being there so ships can pass into the bay.
I have been over it myself, and there's a $10 toll to go over and through it. (it may be even more now.) I took when I travelled to Hampton, Virginia a few years ago via the Delmarva peninsula. Very scenic ride, and quite a bit of poultry along the way too.