It's actually the complete opposite - space is not "infinitely cold" it's more like "infinitely isolating" - like the perfect giant thermos. Space suits for example have a whole layer of water tubes just to cool down the astronaught, otherwise they will pass out from their own body's temperature.