Just FYI, even if you had said yes nothing would have happened. The "Say Yes" scam is an unsubstantiated rumor with no documented proof of anyone actually being harmed by simply saying the word "yes".