If I downvote someone's answer with detail in the comment as to why, then why is my reputation deducted ( by -1). I am helping the community. Reputation is hard to earn, and if it's deducted every time I downvote, then I'd just stop downvoting altogether, I suppose. Downvoting helps the answerer to know they lack something in their answer, or have answered wrong. It helps OP (who doesn't know if it's right or wrong) to see which is right and which is wrong, if it's the case.
It helps keep people from getting carried away with downvoting. If you didn't lose any points from it, you could downvote a lot of stuff (and you can kinda see that here on Meta).
That $-1$ point is enough for most people to step back and ask themselves: do I really want to take a hit for this?
These points are not tied to money or anything of material value, but people do take them seriously, and there's even strategizing that goes along with it.
Suppose for example that you answer a question with two other answers. Alice's answer has 7 upvotes, your answer 6 and my answer has 0 (based on a true story). Then you get an upvote but Alice's answer is still first
because she answered a few minutes before you for some reason or other.
If you downvote Alice's answer, it might not be because her answer is technically wrong, it could be that you want to get ahead of Alice, and presumably catch more upvotes than you would if you stayed in the middle of the pack.
I doubt you would actually do that, but I believe it does happen in general. Why else would there be a Sportsmanship badge?
The $-1$ reputation is to deter serial or competitive down voting.
For a scenario, say you have an answer with 4 up votes, and someone else has an answer with 3. You're afraid of theirs coming ahead of yours, so you down vote, even though it's a valid answer. This is not what down voting is for.
You can see the effects of not having the $-1$ penalty on meta, many things are down voted much more, and the average score of an answer is much lower than on the normal site.
The $-1$ is also for people who are down voting for the write reason to think before they answer and see if they really want to down vote an answer, or flag it instead, or even just leave it alone.
IMHO, it's there for a good reason, and is a critical piece of the voting system.
There are some points I would like to make:
1 unit of rep is nothing. What could you possibly buy with a penny? And if you dropped the penny and knew, would you bother to pick it up? Most Americans probably wouldn't.
Not everyone who votes will help the community. We have things like serial voting, or as the other answers have described, using votes to change the position of answers. It's a psychological game trying to earn the most upvotes.
The price is meant to help deter less worthy downvotes. It pressures you to think twice about whether or not something is less than worthy.
To emphasize the first point: 1 rep is quite literally nothing in the grand scheme of things. Take, for example, the fact that within about 1 year, I have willingly given up over 1.3k rep for bounties, many of which are not even on my own questions, but on other people's questions!
Not to mention, such rep is easily replaceable (well, at least for me). If you are helpful to the community via answers and questions, you will easily find that making up 1 rep, or even +200 rep in a single day is never impossible. (of course, don't forget to dedicate time to other activities)