But why would you ever want as a user to go from an AMP article to the same article but non-AMP? I could see the user going to the homepage of that site to see more similar articles (or clicking on other articles on the side), but I don't see reason why they would want to see the exact same content but slowly loaded.

And again, definitely not by pressing X. Pressing X means you're done with the article. And if you're interested, you'd press the websites logo at the top to go home, not X.