We've been experimenting with remote workers for the last two years or so. Our team was used to work in the same place (in Paris), so it took a little bit of time to get used to, as we tried different ideas. Now it works really well with phone/video conferences, trello boards, github, remote pairing, and a (few) permanent chat rooms (thank you Freenode <3), etc.

I think the biggest complaint is that the offices sometimes feel empty to those who chose to work "on site". My take on this is "let's get smaller office space". Plus it costs less. :)