In theory garbage collectors could do a good job maintaining locality of data structures initially allocated in a local fashion. In practice, they don't, but compacting GC may be better than fragmentation for the average non-cache-aware program. In the case of Java, you have the additional problem that everything on the heap is boxed (except primitive arrays and primitive members of objects), which definitely strains memory (both capacity and bandwidth).