today I saw that I had 180 points taken from me. I see that one user who I had answered a question to had up voted many of my posts, so I guess that is fair for some to be taken. BUT, I had answered other questions and spent time and got real accepted answers and up votes and still lost that rep! that doesn't seem fair! why does this happen, and what can be done?
why does this happen,
This is explained on the Help Center page which is linked from the entry of your reputation history showing the reversal.
and what can be done?