This is incorrect. Every time I click the Unity button, the first icon is Amazon. No way to remove it that I can see. Just Launch. Please step me through clicking the top right Unity button to see all apps, then removing Amazon via the UI only.
It's a .desktop file just like any other icon in Unity. It's at /usr/share/applications/ubuntu-amazon-default.desktop; right click, delete, just like you would any other .desktop file.
The default in Ubuntu is to send crash reports: https://wiki.ubuntu.com/Apport