If I downvote someone's answer with detail in the comment as to why, then why is my reputation deducted ( by -1). I am helping the community. Reputation is hard to earn, and if it's deducted every time I downvote, then I'd just stop downvoting altogether, I suppose. Downvoting helps the answerer to know they lack something in their answer, or have answered wrong. It helps OP (who doesn't know if it's right or wrong) to see which is right and which is wrong, if it's the case.
It helps keep people from getting carried away with downvoting. If you didn't lose any points from it, you could downvote a lot of stuff (and you can kinda see that here on Meta).
That $-1$ point is enough for most people to step back and ask themselves: do I really want to take a hit for this?
These points are not tied to money or anything of material value, but people do take them seriously, and there's even strategizing that goes along with it.
Suppose for example that you answer a question with two other answers. Alice's answer has 7 upvotes, your answer 6 and my answer has 0 (based on a true story). Then you get an upvote but Alice's answer is still first
because she answered a few minutes before you for some reason or other.
If you downvote Alice's answer, it might not be because her answer is technically wrong, it could be that you want to get ahead of Alice, and presumably catch more upvotes than you would if you stayed in the middle of the pack.
I doubt you would actually do that, but I believe it does happen in general. Why else would there be a Sportsmanship badge?
The $-1$ reputation is to deter serial or competitive down voting.
For a scenario, say you have an answer with 4 up votes, and someone else has an answer with 3. You're afraid of theirs coming ahead of yours, so you down vote, even though it's a valid answer. This is not what down voting is for.
You can see the effects of not having the $-1$ penalty on meta, many things are down voted much more, and the average score of an answer is much lower than on the normal site.
The $-1$ is also for people who are down voting for the write reason to think before they answer and see if they really want to down vote an answer, or flag it instead, or even just leave it alone.
IMHO, it's there for a good reason, and is a critical piece of the voting system.
There are some points I would like to make:
1 unit of rep is nothing. What could you possibly buy with a penny? And if you dropped the penny and knew, would you bother to pick it up? Most Americans probably wouldn't.
Not everyone who votes will help the community. We have things like serial voting, or as the other answers have described, using votes to change the position of answers. It's a psychological game trying to earn the most upvotes.
The price is meant to help deter less worthy downvotes. It pressures you to think twice about whether or not something is less than worthy.
To emphasize the first point: 1 rep is quite literally nothing in the grand scheme of things. Take, for example, the fact that within about 1 year, I have willingly given up over 1.3k rep for bounties, many of which are not even on my own questions, but on other people's questions!
Not to mention, such rep is easily replaceable (well, at least for me). If you are helpful to the community via answers and questions, you will easily find that making up 1 rep, or even +200 rep in a single day is never impossible. (of course, don't forget to dedicate time to other activities)
I posted a duplicate of this question, and then a version in which I propose an answer. It was suggested (in comments to one of the answers here) that I post an answer here. I just copy some of the points I made in my question Progressive deduction in rep for downvoting.
Proposal: A downvoter would not lose rep if they cast only one downvote per day.
Any subsequent downvotes (by the same downvoter, the same day) would result in the loss of one reputation point.
(Added details, measuring time 24 hours starts when that first downvote is cast. Subsequent votes that were cast would remain penalized, if the first downvote is retracted later, but the user could use this retracted first downvote on another answer without penalty, within the original 24 hour limit. If the answer with the downvote is deleted, then the downvote is lost and could not be reused without penalty on another question.)
Variations: No penalty for first downvote, but 2 point penalty for the second downvote within 24 hours (and one point per each subsequent downvote). Yet another variaton: One point reward for first downvote, two point penalty for second, and one point for subsequent downvotes.
Finding the right balance between several (at least three) things:
(i) The amount of reputation that a user gains when their answer is upvoted,
(ii) the amount of reputation that a user loses when their answer is downvoted,
(iii) the amount of reputation that a user loses when they downvote someone else's answer.
(Similar balance could be sought for up or downvotes on questions, but I prefer to focus on answers here.)
In my opinion 10 points for upvoted answer is good, it is significant, as it should be, and I have never had any issues with that. (At present, 10 points upvote for a question, I was more comfortable with 5 points, but again this is another topic, I would prefer to stay away from that for the moment.)
The amount of 2 points loss (for the answerer) for a downvoted answer is inadequate and insufficient, in my opinion. It promotes quick and wrong answers, as the gain from an upvote far outweighs the loss from a downvote. There are plenty of users, at least that is my impression, that would post inadequate answers, and plenty of users that would vote up such answers without reading them carefully and making sure that (1) they are correct, and (2) they are relevant and do answer the question asked. (Such rushed answering and upvoting may happen soon after a question is posted, and before it goes down the stack and may then be forgotten.) Such an imbalance between 10 points for an upvote, 2 points for a downvote promotes, in my opinion, a proliferation of poor answers. (Let me also add that a user who posted an answer with three upvotes and one downvote may simply choose, consciously and comfortably, to do nothing. The most likely result of this is that they would keep their net gain of 28 points, the question and answer will be forgotten (and never deleted), and they would have no incentive whatsoever to either correct or delete their wrong answer.)
I find that there is an imbalance between the one point reputation deduction for a user who downvotes an answer, and two points reputation deduction for a user whose answer is downvoted. (Assuming I cite these numbers correctly.) One may argue all this is insignificant (why bother discussing one or two points), but on the other hand to me one point is quite significant in comparison to two points. If I downvote someone else's answer, they would shrug it off (if they already got a few upvotes, deserved or not), and I would lose one point, which is quite comparable to two points, and I would not have an easy way to make up for this point (unlike the user with the wrong answer). So, I would be punishing myself, losing a point, and the user with the wrong answer will be laughing at me, even if they lost two points, because their net gain was (at least) 8 points.
My answer: I would not bother to downvote. Presumably "others" (more responsible or braver than me) would downvote anyway, and as a result that answer would eventually sink. Looks like "others" may not be as stingy as I am on casting downvotes, so my concerns may safely be ignored. But, I am not convinced. I do find that there is a gross imbalance, as indicated above! I wonder what would happen if: (*) there was no punishment for the user who casts a downvote, or at least no punishment for just one downvote in 24 hours, and (**) a downvote would bring a more substantial loss of points to (the user with) the downvoted answer.