I don't see how this Victorian ethic applies. He's not talking about the poor, but rather those who have "chosen to spend money on a lot of other things", i.e. the middle-class.
The argument suggests that your net worth is mostly your own choice and has little to with circumstances. My point is that this flawed notion is responsible for other, more noticeably flawed notions.
