If you have 100k you can just buy a few b200s.

One does not simply "buy a few b200". They come in 8 packs minimally, at >5x that budget.

Out of curiosity, what do you do to distribute the weights and inference across the multiple machines?

Thunderbolt 5, and software

at a 100k$ you're not buying apple products to be limited by the 120gb/s thunderbolt port.