"I think we do need to delete entire agencies, as opposed to leave part of them behind. ... It's kind of like leaving a weed," Musk said.