I have noticed that a lot of times when an answer gets downvoted, there is rarely any feedback (unless the answerer asks for one or depends on the popularity of the question) for the downvote. With questions, there is a flagging option for "bad quality", but with answers people seem to retract their answers more than moderators deleting them due to the silent discouragement from the downvote(s).
This statistic is based on experience, so it may not be reliable. Also, I occasionally see questions in the Meta asking "Why are all my answers suddenly getting random downvotes (perhaps by a single person)?"
But if we can implement an incentive to downvoting, then perhaps we can eliminate these problems? We could also let a user know who has been downvoting their question/answers when there is an unusual pattern of downvoting. This I assume usually occurs when someone else makes a troll account to downvote, but the relationship between the troll account and the actual account should only be known to the mods to avoid conflict between the user and the downvoter.
I hate leaving an answer with downvote(s) and staring at the screen for $N$ minutes and realizing either I misread the question or it was just completely wrong or sometimes just don't know why it was downvoted.
I think my initial post may have sounded like a rant, but I am really trying to offer suggestions to reduce these frustrations, though the first implementation seem too extreme for anyone to even consider it.
So I suggest giving reputation points or some other type of incentive to encourage a downvoter to comment.