As long as nobody is serious about sniffing your connection you should be OK but there are tricks using coffee cans or DSS dishes that let them reach out to pretty much anything they can see.
The thing you have to worry about is someone getting into your system if you have file sharing turned on.
