"attacking my argument for being libertarian is argumentum ad hominem"

No. Saying that the framework you're basing an argument on is flawed isn't dismissing the argument because of a personal flaw of the person asserting something. It's saying that the core axioms/assumptions are flawed.

Maybe you over-identify with that ideology and are taking criticisms of the ideology personally since it feels personal, but it's not at all about you, it's about the ideas you're espousing.