First, downvotes on meta indicate disagreement with the premise of the question. I downvoted here because of the concerns Joel Reyes Noche indicates. If downvoters were listed by name, there is a real concern that disgruntled posters would harass them or try to keep arguing the same points over and over. In some situations, the best option is to just downvote and move on.
Nevertheless, it is also important for downvoters to try to help posters know the reason for downvotes, especially when the posters are new users. This gives the posters a chance to improve their posts.
So it is a good practice to leave a comment when downvoting or voting to close. (Unfortunately, the review system does not facilitate this.) Sometimes, someone else has already left a comment, though, or the poster has enough experience to know the likely reason for the downvote. In such cases, leaving a comment could be a worse option than just downvoting.