I think you're straw-manning. Decreasing degrees of concern for those farther removed isn't a problem until you reach zero concern or empathy for people you don't know/don't look like or don't share a language or god with. That is how you justify taking their freedom, raping or killing their defenders and stealing their land.

Letting it get to that point, so as to put your tribe first is selfishness and IMO a deep character flaw. A flaw in so many of us that the idea of star-manning is, in so many cases, naive.