I've written this before on HN, so will re-share it to buy more time before completing a big write-up on Starfighter:
All three of us are software developers. We like developers. We spend a lot of time talking to developers, in particular about their careers.
We had no trouble finding clients, but a lot of trouble convincing clients to take the CTF model seriously; virtually all of them wanted us to function like a traditional recruiter, offering 10 candidate profiles for every open req. That's a job that is 90% saying "no" to people.
It was dispiriting watching developers we knew to be strong get rejected for positions because they didn't have the right resume, or didn't solve some pen-and-paper programming puzzle quickly enough. It was a drag for all of us. It made us unhappy.
Now I understand why neither you or Patrick ever replied to my email asking about some career advice -- I managed to ask at the worst possible time, a bit before closing Starfighter so probably when you were really unhappy.
Don't ever apologize for reaching out! As it happens, we're all in a very happy place right now (Patrick is having a blast with Stripe, and Erin and I started a company with two friends that is booked solid through 2017), so if you're still looking for advice, we're all happy to help.
It's perfectly possible to make money recruiting by matching exactly what your customers want, but at that point, you are exactly like every other recruitment firm out there. What separates you? Asking people to do awesome CTFs doesn't really help of success at CTF doesn't line up with success at passing your typical Silicon Valley interview.
Anyone that has spent time working outside of firms that hire this way (which includes stockfighter's founders), realizes that the technical interview today is very expensive, throws away a lot of candidates, stifles all axis of diversity, and ultimately it's not really all that predictive of on the job performance. A CTF is not an ideal predictor either, but I'd argue it's better.
Stockfighter found that their best candidates by the CTF were receiving terrible ratings by the companies they send them to, because they lacked the shibboleths required to pass the interviews. They'd only have success if the company filtering after being presented a candidate was minimal, and then they saw that yes, the job performance of the candidates that were good at the CTFs were at least as good as those that came through the traditional recruiting process. Apparently there wasn't enough appetite for that kind of process, and thus, Stockfighter made no sense as a company.
That's a pity, because I share their view that technical interviewing is broken. I just don't see how we can systematically improve it without the kind of data that you can only get at huge companies that hire thousands of engineers a year.
No, that's not what we were solving. If we wanted to be conventional dial-for-dollars recruiters, we would have done that.
I understand and appreciate the mentality that your comment comes from. But the premise of our startup wasn't Steve Blank-style customer development, where we started with a vague idea of the space we wanted to be in and then worked with customer prospects to figure out exactly what product we would end up building.
Rather: the company we started was an experiment, a very opinionated one, to see if we could port a hiring model that had worked extremely well for us at one company to a larger set of companies.
I think there's a way to make that model work, but I've become convinced that "contingency recruiter" isn't the way to deliver it.
Innovation in the recruiting space is a really hard task. Most recruiters would rather throw money at yet-another-job-board-and-funnel-source than try something unique which requires putting in extra time and effort.
The last "innovation" I've witnessed in the space was the Hired model, and everyone is trying different versions of that now. It's a hurtful model that abuses the inefficiencies and lack-of-transparency in the two sided market. (For anyone wondering what I mean: When you sign up on Hired they ask you what your preferred salary range is, then, as of last time I checked, they show _that exact value_ to the companies looking through prospective candidates. So guess what the first offer you get is? Not what a "fair" market rate is, but exactly what you typed into the sign-up form.)
I do a lot of talking to recruiters as part of helping them use our Stack Overflow Talent product, and honestly it feels like taking crazy pills half the time. Having to answer "why?" to "You need to at very least explain what your company actually does, or maybe the tech stack you use" makes me feel insane.
I loved Starfighter, but thinking back what I liked most about it was the content you and Erin and Patrick wrote about it. Your point about "life is too short" is very very poignant.
I guess the whole point of this comment is for me to say I completely respect that decision, you all did the right thing. Trying to convince people who would rather just throw more money at a wall than try something unique to use your unique product is an awful feeling.
I'm very glad to hear your new project is easier to make money off :)
I'd better do a stupid coding test online that have to schedule a phone call during working hours with a guy who just want to get away from the phone call and could barely present his company and what he's recruiting for.
My view is that Hacker Rank just perpetuates the "textbook academia" and "trivia question" model of interviewing, except it automates out part of that step. If that's innovative, my bar is too high, I guess.
Standard interviews: multiple choice trivia, red black tree programming puzzles.
Innovative interviews: Custom code exercises with custom unit tests written by the company, ssh exercises where a server is created automatically per candidate and he has to connect to it and configure/fix/change some stuff.
After many interviews (both given and received). I have definitely settled that:
1) I'd rather write a stupid program on HackerRank than deal with an annoyingly scheduled phone interview (where I might be expected to spell that code over the phone WTF). Automation is nicer than the average interviewer. A proper code editor for coding is better than the average shared editor.
2) No matter what we try to do: 20% of candidates won't show up + 20% won't be able to code a program to print numbers from 1 to 10. There's nothing we can do about that (the sad state of interviews is probably all because of that).
3) For the first initial screening, very simple exercises are all that's needed (see 2). Hacker Rank is perfectly appropriate for that.
There's probably a way to ask this politely, but I'll just ask bluntly to save time and confusion...
Did you think you could change the way people hired overnight?
There probably is a way to do what you set out to do, but it seemed obvious to me from the moment you announced it would take years of slogging and building a track record to change such ingrained processes at companies.
Revolutions on that scale are never going to happen in a few months. The reason I thought you guys had a shot was that you'd be stubborn enouh to stick to it in the face of resistance.
Yes, I do think that it's possible. To get Google to do it? No. To get a bunch of smaller companies to do it? NCC is larger than most YC startups, and this is how we did recruiting there.
I have thoughts on how I'd do contingency recruiting differently if I had to do it over again, and I have thoughts on what I think a better model for this would be than trying to do contingency recruiting from day one.
What I don't want to do is write them up in HN comments that will further sap me of any will to do a real writeup.
> I think there's a way to make that model work, but I've become convinced that "contingency recruiter" isn't the way to deliver it.
I think you need to find the few rare companies that are interested in this recruitment/selection model.
9x% of places are doing standard resume review and interviews. They don't have the will nor the expertise to evaluate CTF (or any similar) contests and candidates.
>Rather: the company we started was an experiment, a very opinionated one, to see if we could port a hiring model that had worked extremely well for us at one company to a larger set of companies.
I'm not him, but there are already like a hundred companies doing that. All it has accomplished is one more hoop to jump through in front of all the hoops that already existed. It has only made a ridiculous process more ridiculous. I think they were trying to make the process less ridiculous, not just be yet another hoop. But it seems like people don't really want that.
When you do your write-up, I'd be interested to hear more about the mismatch between what you thought your clients wanted and what their eventual actions demonstrated they wanted. Could this mismatch have been detected in the early planning of the company?
Oh, you were asking a simpler question: "why is the site down?" Sorry!
We'll probably put it back on the air at some point in 2017. It's down for a lot of little reasons (inefficient AWS configuration, occasional need to babysit, the need to adjust a lot of little messaging things around the site so as not to make people think we're still recruiting off it, &c) that add up to enough hassle that it's easier to have it be down for now.
