I also need to test on IE (since our biggest client uses it) so I have to spin up a Windows 10 VM and test it that way. Absolutely worth the time, time spent increases exponentially if a bug is found later in the process.
How unpleasant. Have you found that the W10 VM takes way more disk space then seems reasonable? I had to fire one up the other day, and between the VM image and the VM it took 50ish gigs of space iirc.