I agree that you shouldn't use the pi as a server. The biggest drawback is that arm is still not as well supported as x86 and isn't as heavily optimized.