I'm interacting with my co-workers while doing my job. I don't have any desire to also interact with most of them during my breaks, which includes lunch. Part of my lunch is to not just take a break from my work, but to take a break from the people I work with.

If your work is an island and you don't collaborate with anyone, that is probably true.