Combat that by letting your phone ring if you can, or with a call blocker. Kind of hard to leave a message if there's no kind of answer from the numbers they call.
Scam! If it is a legitimate IRS issue, the IRS will mail you documents regarding the issue (they know where you live, and how to find you). They will not call you like that.