I'm working on reducing a $50K per month bill for Inter-AZ traffic at the moment.
> but multi-az is prohibitively expensive if you need to run anything data heavy distributed.
If you communicate between your AZs via ALBs, multi-az is effectively free. Our bill is so high because within our Kubernetes cluster, our mesh isn't locality aware; it randomly routes to any available pod. 2/3rds of our traffic crosses AZs.
Yes. You've got to be aware of where those boundaries are when adopting the cloud and the cost information around these cases are inadequate at best. Too many people get surprise bills.