Would you mind sharing your infra budget needed to spin these VMs ?

Surely it is reasonable, but also way more than our budget. Id like to compare.

Sure. It's a bit hard to quantify because we need to run these on bare-metal machines and the unit cost is pretty high.

We run our test workload as well as a few other startups' that we have onboarded on one AWS ARM bare-metal machine at $1.7k a month. We don't saturate that machine fully either so I'm not really sure what the amortized cost would be. Certainly more expensive than Github Actions but not by a crazy amount, and the value we get out of it is way higher than GA.