This might be tough to answer due to survivorship bias, but can anyone name a startup, payment focused or otherwise, that failed because of a security vulnerability/vulnerabilities?
Would you consider DigiNotar a startup? It's certainly a famous example. Also, I happen to know of "RockYou", the facebook app people who were pretty much demolished not long after their massive account breach. Co-incidence or cause? They may be still alive but under new management.
Your average web startup that's providing jazzy ringtones and a new poke app, sure, should leave security for last.