One of my hats is 'system administrator'. I like X a lot more than VNC/RDP. I don't run linux on my desktop and don't really care about performance there.
I'm not looking forward to the day when X is gone. Being able to fire up system-config-* on a remote system is awesome.
One the 2nd page near the end they discuss the feasibility of adding network-transparency to Wayland; according to the author Wayland is both hugely asynchronous (so network lag isn't a blocking issue) as well as completely event-driven, so receiving those events from a local process or a remote connection are the same thing to it.
I don't think we'll all have what we want soon, it sounds like mid-2013 or end of 2013 before all the dust is settled and we know what our Wayland-future has in store for us.
Until then, there looks to be just too much low-level work to get done to try and guess where it will all fallout.
My impression is that Wayland planned to support X as a compatibility option. The difference is that X isn't the low level mediator of all input/graphics by default. As I understand.
I can imagine this being a real issue if it happens at the toolkit level; for example, GTK dropping X11 support. But until Wayland proves itself in the field as a real and superior competitor to X, it's absurd to suggest that even a significant proportion of distros would abandon X altogether. And as long as any distro of note (Linux or otherwise) still ships with X, it seems really unlikely that the major toolkits would indulge in such madness.
I wouldn't worry about it for another decade or so.
I'm not looking forward to the day when X is gone. Being able to fire up system-config-* on a remote system is awesome.