Asbestos fibers have to be JUST the right size, then be inhaled, to cause problems. Various procedures have been developed to reduce our risk when working with asbestos.
Perhaps more invidious are the risks posed by paints. OK, so everyone has heard about "lead paint" ... but how many know what that actually means?
Lead is but one of the heavy metals whose compounds have been used to color paint. In the case of lead, it was/is used to produce white, yellow, and greenish hues. I say "is," because it is STILL IN USE in automotive, industrial, and specialty applications.
What about other 'heavy metals?' Another commonly used - this one for reds and oranges - is cadmium. Cadmium can be just as nasty. So are mercury compounds, used for the same colors. "Hexavalent Chrome," made famous by Erin Brockovitch, is another colorant used.
The means of ingestion differs from asbestos, as does the sort of damage caused. That's not really relevant here. What matters is that we daily cut, sand, rub, tear, and sweep up these other materials as well.
There's some sense to making a greater effort overall to contain our messes.
I was installing a switch for my aunt just last summer, flipped it on, and saw a nice big (paint can) full of "Black Asbestos Furnace Caulk." Obvious what it was used for, just funny that it's still around and still full. Any takers?
Also, being in the theatre industry (live theatre), most of the older lighting used asbestos cable. It was the only material capable of withstanding the heat until synthetics were developed. We got used to wrapping the whips in e-tape when we worked with them, to keep the fibers contained.