My understanding is that it's important to keep SSH (and other services) on a privileged port. (I think the default is <1024.) Otherwise, unprivileged malware that could cause the SSH server to crash and take over the (unprivileged) port. No idea if this actually happens in practice though.