There is no need to ask this on meta.stackexchange, this is a topic which can be found in our help center. I really recommend that you take a look and read the important topics there. This gives a much better understanding of the community.
Regarding your reputation loss, when you downvote an answer, this person who wrote the answer looses two reputation points, the person which downvotes looses one point. This is done to show that this is an important event which shouldn't be done without reason. As @mattmdo pointed out this is not true if you downvote questions, here only the person who asked it will loose two points. See here for more information.
BTW: This is different on meta.bio, here up- and downvotes have no effect on your reputation and are either seen as agreement or disagreement with an argument.