Can't speak to AWS, but I know that the e-commerce side of Amazon.com is A/B tested to within an inch of its life. They test absolutely everything, including colors, size, layout, spacing, etc to check for the best conversion rate. They have a huge team of developers and designers working just on this site optimization and they're constantly running experiments.
It may not be good UX as measured by some aesthetic standard or set of best practices, but the site is accomplishing its goal for Amazon and arguably the consumers who use it.
that seems like the google way of building product. you test all these micro changes but miss out on the macro changes the site actually needs, which doesn't really work out. no one can argue that amazon has good ux. from product to product and day to day the website layout changes. each product site has just a ton of data thrown up on it. it has horrible tracking of product purchases. for example, if looking at a paperback book, it gives no indication that you may have purchased the hardcover. it has terrible management of books that may have multiple editions. buying older books takes some time because they'll often have multiple pages for the exact same book. pre-ordering is a mess. i have been sent two video games before because i pre-ordered a game super early which then apparently got a new product page. i thought i had never pre-ordered it some time later and amazon didn't mention it, so i ordered it again since it gave no indication i had ordered that exact product. the search is not great either. comparing products is non-existent. just cycling through various options like color can be a chore just to see a different price.
just because they have a process doesn't mean it works. but yea, like someone said, i guess the general user doesn't care. i know it has lessened my use of the site.
And there might have well reason to change the small things only, but nothing fundamentel. People hate change. If they start to turn the website upside down, their customers would start to whine, and cry, and complain. A lot of wasted time they could spend with buying stuff.
So, it ugly and bad as hell, but everybody is trained and accustomed to it. Why risking confusing customers without need?
1) You can A/B test yourself into a corner. They are definitely doing it right, whether it is the right thing to do is a different question.
2) Having some process insight into the operations / logistics side I can confirm that everything thing is connected to the webside one way or the other. That is what gives Amazon its huge operational advantage. Downside is, with everything developed in-house, this approach can end up as a motely collection of legacy systems. And that makes changing things difficult and had the potential to become a major pain in the ass long term.
Ex amazonian here, all of the internal systems have well defined API interfaces. So long as you don't blow up the API's you can replace any legacy system you want because it doesn't have tendrils that reach into 50 other systems.
Pretty doesn’t necessarily mean effective at converting. I think Amazon’s team is perfectly competent at testing whether one design makes more money than another — and I think they’re competent enough to go with the one that makes more money.
I have a suspicion they A/B test the little things, like is it better to show the price in dollars or the local currency, and is it better to have a 'add to cart' or 'buy it now' button.
They miss the big changes like 'shall we show all the products in a 3d immersive VR gallery' or 'make a site which downloads all data and works entirely offline'.
Those big changes might usually fail, but if you never try any of them, you'll end up with a site that looks and behaves like it's from the 1990's, while the rest of the world has moved on.
It may not be good UX as measured by some aesthetic standard or set of best practices, but the site is accomplishing its goal for Amazon and arguably the consumers who use it.