If I accepted an answer, but later choose to un-accept it for whatever reason, will that person lose reputation points?
If you unaccept an answer, the author of the answer will lose 15 reputation points. Additionally, you yourself will lose 2 reputation points. However, if you select a new answer, then the new author gains the 15 reputation and you gain back your 2 reputation.
Note that if you accept an answer which you yourself have written, there is no reputation gain.
This amount of reputation is not very important. If you feel you want to unaccept an answer, you should do so regardless of reputation.