You can, that's if of course you're able to stop unwanted callers from getting any way of leaving messages. No kind of answer eventually gets on their nerves enough to where they'll stop calling. Without any answer, threatening or hearing anything but a ringing phone is impossible.
I got this call from a man with a thick accent, who started to leave his message before the beep so I didn't catch the beginning. Nonetheless, it was difficult to understand him, and it was of course urgent that I call him back - NOT!
Got the same call today. Called back and harassed them right back. I have them on auto dial on my phone. I have called back 150 times today and told them I will continue to call and blow up their phone until I have an apology for attempting to steal from me and for thinking I am ignorant. So far I have the supposed IRS cursing at me and asking me to stop calling him. I plan on calling non stop till I get an apology. Kind of fun handing it back to them...