The best solution remains in my opinion browserslist-targeted compiling. I don't think it makes sense to pretend you can/should write non-trivial JS today without a build step, especially if you're worried optimizing delivery. Are you seriously going to fret about delivering unnecessary polyfills to modern browsers while simultaneously delivering unoptimized code and styles because you felt like going old school?

You can write tomorrow's JS and CSS today while explicitly supporting whichever of yesterday's browsers you know you need to, and have the delivered code improve over time as older browsers are phased out without you having to rewrite it.