Reopened
Seems to me that How to Distinguish... needs a referee. On hold for lacking context, seems to me like a perfectly reasonable question.
In particular it seems clear to me that at least one of the on-hold voters simply didn't understand the question (and I assert that's not the OP's fault). See the Answer. That's simply not a proof "in first-order logic" as requested. See the comments on the answer: When I pointed out that it wasn't "in first-order logic" the guy seemed to think I was complaining that it was insufficiently formal!
Yes, asking for a proof "in first-order logic" was not the best way to put the question, since of course ZF is a first-order theory. BUT it seems to me that anyone familiar with this sort of thing is going to realize that what the OP wanted was a proof that the first-order theories of the two structures are distinct. (And it also seems to me that, optimal or not, people do phrase that question the way the OP did.)
Hence the question about moderators who know some logic - seems to me that it matters whether I'm right in various places above where I say "seems to me"...