That's subjective. I know org-mode and prefer it. The people espousing the intrinsic comparative readability of Markdown all appear to be deeply steeped in that world.

For me the org-mode syntax is more logical than Markdown—and I can have cross references.

You know org mode and prefer it, fair enough. To someone unfamiliar with both syntaxes, Markdown is much easier to grasp; because it's simpler it's just more readable.

To be honest, Markdown feels simpler because it's quite less featured than Org.

If you put a Markdown document and an Org document implementing only what Markdown can, there's no readability difference between the two as I know both markup style fairly well.

Is that true because you're privy to the results of that experiment? Or is it true because you're most comfortable with Markdown and are starting from the unshakable belief that Markdown is intrinsically easier?

Org Mode has a lot more functionality than Markdown. A document that only uses features present in Markdown seems to me just as readable in either format.