Not so long ago I could see in my reputation tab which questions/answers have been downvoted, but it seems now this feature is gone. Is it me or is this a new policy which I'm not aware of?
If I understand well from the comments it seems this happens only for the deleted by the system questions.
There are two ways to lose 2 points: receiving a downvote, or losing a +2 for suggested edit. When you receive a downvote, it's recorded in the reputation history as such, and links to corresponding posts are provided. Here's an example from mine:
Selecting the checkbox "Show removed posts" will also show the votes you received on the posts that were subsequently deleted. (Note that votes on deleted posts do not affect reputation.)
On the other hand, the loss of the +2 bonus for a suggested edit is more difficult to track down, but this does not happen often.