Do moderators know who downvotes?
This is a related question.
From this meta.SO thread:
Volunteer moderators have very limited access to voting information. The only voting information we can really see is related to anomalous voting patterns. This makes it easier for us to spot sock puppet accounts. We can't see specific votes cast on each question and answer by individual users.
No, such voting information is kept secret.
See also: List of Moderator Privileges.