You keep mentioning voting. Some people reflexively downvote any post that talks about votes. If you didn't mention voting you may get fewer downvotes.
I don't care about getting the votes. I do care about pointing out irrational behavior.
In this case, I am pretty sure it was the person I responded to because I saw the karma score drop right along with the response to my previous comment.