DONT SAY YES, IT'S A SCAM!
The Federal Communications Commission back on March 27, 2017 issued a consumer alert against just such scammers. When a caller says, "Yes," that they can hear the robocall, their reply is recorded and used to authorize fraudulent charges via telephone on the victim's utility or credit card account, the FCC says.
The scam is prevalent, based on complaints the agency has received and from news reports across the U.S. The fraudulent callers may impersonate familiar organizations to get you to answer and talk.