actually that is ALL they need to steal your identity. I work with collections in the medical field. We ask to verify the patient with the last four digits because the last four digits are distinct. They identify the person.
Actually, if they have access to a service like Lexis-Nexis they can find the other numbers and piece your SSN together if you give them the last four. It is not really that hard for them to get the whole thing.