> Speaking logically; there can't be anything is making it physically harder to build infrastructure over time. And nothing physical is making the economics of railroads worse.

Building could be harder over time when lands to build is used for other purpose. Railloads economics could be worse relatively over time when competitor(car, airplane) getting cheaper and improves quality.