That "clay stuff" is duct seal (also known as monkey-poo).
I use silicon caulking to fill the weatherhead.

Earl