Except that the majority of PC users never upgraded or changed out their hardware.
Its building a device for a market that barely, if ever existed. The replaceable model benefited manufacturers (not customers) by making customizable PCs to sell with price discrimination easier.
You make a good point supporting why Ara is an excellent idea, and why it will be good for consumers if Ara succeeds.
Consumers don't need to upgrade their modules to benefit from the economic dynamics that standardization/modularity brings. Modularity brings better components, a greater diversity of systems, and lower prices - even when the consumer doesn't upgrade and change out their hardware themselves.