> As for "key right next to house" how else do you propose letting agents show your property to prospective buyers or tenants?

Give the letting agent a key to hold on to? Use an electronic keypad lock? Perhaps in your specific situation those are not feasible, but it’s not like there aren’t alternatives.