The Stanford Prison Experiment calls that premise into question, though I understand that there are questions about the underlying science. Or did you have another means of isolation in mind?

That aside, my personal experience leads me to believe that prison is as horrible as it is because of the people locked inside. Evidence for this can be inferred from the differing degree to which low security prisons are less harsh than higher security prisons.