I just signed up for a Heroku free account, so I may well give this a go.... I'd like to do some experimentation. I want to pick the most fun/difficult way to do this if I can.
I don't think so. Heroku is a PaaS (Platform as a Service). You can connect with SSH but everything is mostly set-up for you already but I don't have Heroku so I'm not sure. A VPS (Virtual Private Server) is just a remote computer typically with a bare Linux installation, and you build up from there and install what you need, Ruby, Node, PHP... Check https://www.digitalocean.com/, they're pretty popular.
