Perhaps a better way to put it is that it is difficult to retrofit an office to allow a few telecommuters.

In my experience, I was working remotely in the company's early days. The in-office employes really started to grow after me. While I remain the only one that really works outside of the office, the culture of remote workers was instilled from nearly the beginning and don't really find any of the problems you suggest to be true in my case.