Self selecting and perpetually reinforcing population of young people living a certain life style you have to adopt to fit in... Or I could stop beating about the bush and flat out say it's just too cut-throat, crime-ridden and debauched for my liking. There's also pollution, traffic, over-population, poor living conditions (even adjusted for the increased salary) and crime (worth mentioning twice). At that point, it matters little how vibrant, exciting and full of opportunities it might be.
Isn't this the same for most big cities though? I'm not a massive fan of London, it's just where I have to work, but I think you'd find the same amount of crime and over-priced flats in places like NY.