The problem is that there's no effective way with (currently constituted) zoning law to 'trade' the externalities of density. You can't compensate your neighbours for shadows or traffic or other impacts in exchange for being able to build larger. So the choice faced by most residents is:
1) Acquiesce to high-density development, and receive zero compensation, or
2) Fight high-density development, and probably get zoning laws imposed at minimal personal cost.
You can see why this setup incentivizes people to push hard for "static neighbourhood" zoning law.
3) Agree to high-density rezoning, endure the construction noises and shadows and whatnot for a couple of years, then sell your house to a condo developer for a massive profit and never have to work again.
This isn't a hypothetical, I saw it happen on the street I lived on in college. The detached houses fell like dominos and were all replaced by high-rises until there were none left.
Sure, if you want to sell. But if only some of the people want to sell, then the problem of non-compensated externalities arises again. It's a classic collective action problem arising mostly from the fact that things like air rights, street space, and so on, are barely tradable.
1) Acquiesce to high-density development, and receive zero compensation, or
2) Fight high-density development, and probably get zoning laws imposed at minimal personal cost.
You can see why this setup incentivizes people to push hard for "static neighbourhood" zoning law.