As long as your code can be unit tested, it's okay to test it later.

But what if the "exploratory" code becomes hard or almost impossible to test (due to dependency et al.)? Refactor it? but you have no test that acts as a safety net during the refactoring period.