If the comment is over 1 day old, you can't downvote it anymore. See, you can still downvote this comment on that thread because it's only 18 hours old. https://news.ycombinator.com/item?id=8081863
There are downvotes and flags. People with at least 750(?) karma can downvote comments, so long as they are less than 24 hours old. No one can down vote submissions.
If a submission does not belong on HN you can flag it. Be cautious with flagging because you can (apparently) lose flagging privlidges. (I think I flag too many submissions and I have cut back on that. I think I flag comments carefully).
You can flag individual comments by clicking the [link] link which should then provide an option to flag.
There have been some discussions and guidance about the difference between downvoting and flagging but I can't find them at the moment. (But if you do get downvoted it's nice to remember that some people use it to express disagreement).
There is no story downvoting. At a higher karma level you can downvote comments. It seems like it's not published anywhere what that number is, but you can figure it out easily enough by googling around.
I said submissions because it seemed to apply to the whole post. Another comment has indicated it is because of the posts age, which makes sense for my example.