Whether or not something is good has nothing to do with what it's labeled. Sure by definition it might not be racist but certainly that's how it would play out. A better equation might be:

exclusion != racism ≡ you're a minority