Benchmarking on VMs (especially on EC2) is a no-no. It introduces a lot of other factors that can skew results drastically. There are entire companies built around the idea of variable performance in the cloud. If you want to do a decent benchmark, do it on a real piece of hardware.
Aren't most application hosted in the cloud anyway? I understand how VMs are not ideal, but aren't they more "real-life" than "real hardware" nowadays (not a rhetorical question, genuinely curious).
The problem with benchmarking on a VM is that VM effects may totally dominate code effects. It's sort of like using laptop speakers to mix a dubstep album.
