Agreed completely. But how do you get more information cheaply?
Remember, all you know is P(qualified| resume says autodidactic)= P(qualified | autodidactic) * P(autodidactic | resume says autodidactic) . The second term can easily kill you. Do you trust everything people put on their resume?
On the other hand, P(degree | resume says degree) is probably close to 1, since a degree is so easy for an HR person to check.
Look, I've got nothing against hiring good people who have no degree, and I certainly don't think a degree proves much (my current students prove that conclusively). If you know you have such a good person, hire them, ignore the degree.
I'm just pointing out that the game is to find the good people. And that's a bit tricky to do.
[edit: by this way, in the interest of clarifying, what you want to optimize is not candidate quality, but (candidate quality - search costs). Don't ignore the search costs term.]
In the programming industry, it's not really that hard to filter out bad candidates. You shouldn't waste time talking to any programmer not willing to submit to some simplistic programming test in lieu of a resume. Resumes are crap and have no place in hiring programmers. Code is all that matters, either they can do it or they can't and if they can, they'll accept your challenge.
Nothing makes for a better interview than doing a code review and critique of the candidates own code. Something simple but telling like write a little slot machine that lets you bet money, spin the wheel, and win or lose with the game ending when you run out of money.
Anyone who refuses such a test isn't a programmer you want anyway, and you can tell an enormous amount about a potential candidate by simply looking at the quality of the code submitted. Don't bother interviewing anyone who's code you can't stand. You can see everything you need to know about their skills in that code.
You wouldn't hire a graphic designer who refused to submit samples of his designs, nor should you hire a programmer who refuses to submit samples of his code. Merely by having such a test, you'll weed out all the fakers because they won't bother with it, they'll submit their fake resumes elsewhere.
Many good candidates could be put off by such test. More importantly, even if they agree to your test and send you some code, you need an engineer's time to evaluate the code sample, and if you're going to invest engineer's time you might as well do a full phone screen. The recruiters are by and large non-technical people. They would love to be able to ask some technical questions which could help them predict whether a candidate would pass the interviews or not. I've been actually asked by a recruiter friend of mine how to do it. But, if you think about it, this is not so easy if you're not a programmer yourself.
You are absolutely right that it would be madness to hire a programmer without having seen him code. The companies are well aware of it and they will surely ask you to write a lot of code during the interviews. But that comes at a much later stage, as it costs them much more money than checking on your resume whether you have a degree or not.
I'm not saying that I like the way things are, but that's just life. If that's of any consolation, in most other industries the requirement of having a degree is much more strict than in case of Software Engineers.
I disagree, any candidate worth having would enjoy such a test. Any programmer who is put off by being asked to program needs to find another profession, period. You should not ask someone to program during an interview, you should review the code they've submitted and make them explain it, all if it, their design choices, their idioms, naming conventions, etc. If they can't discuss code they just wrote prior to the interview, you don't want them, but many good people don't perform well under the pressure of an interview so you won't get an accurate feel for them if they program at the interview.
No one but a programmer is qualified to evaluate another programmer, HR and recruiters have no place here and they'll just waste time and effort pretending to be useful, they aren't.
By doing the test you've already filtered out the wanna be's so you don't need phone interviews, the submitted samples should go strait to a qualified programmer, it'll take him only a few minutes to toss out any bad submissions and he'll quickly know who's worth actually interviewing and who's not.
One issue to be aware of is that it's not just the corporation interviewing multiple candidates. It's also the candidate interviewing multiple companies. Putting a large burden on each candidate may not scale if every company does it. I agree that it's important to assess actual coding ability but I don't know if a programming test is the best way to go about it.
Any programmer who considers such a trivial test a large burden, you don't want. There's no other way to assess programming ability than programming, it's really that simple. If you hire an unknown programmer without making him submit a code sample, then you deserve what you get because you're gambling and likely to lose. The whole point is to find good programmers, and good programmers enjoy programming, they'll happily write a trivial program to prove it, happily. Anyone who balks at such a test is not someone you want to hire, period.
Good code is good code, I don't agree that it takes a vastly different skill set to work on different sized programs. During the interview you can simply ask questions such as "How would you provide the ability to configure various strategies for different users interfaces for this slot machine? Say a web UI, a command line UI, and a native app UI?". "How might you make the randomizer for spinning the slots configurable?". The discussions these open up will tell you a lot about their knowledge of making flexible software.
Again, the test is just to weed out the bad candidates and allow you to only interview worthy people, everything else you'll find out in the actual interview. Interviews will be rare, few people will submit code you'll find acceptable at all. The vast majority of people applying for programming jobs, can't program, it's sad, but true.
There is a set of skills and body of experience that applies to working on large bodies of code. In fact, there are different skill sets for working on bodies of bad code and good code.
The language syntax may be the same, but programming approaches might be very different. I've seen parts of a Smalltalk program where it was coded like spaghetti copy-paste Fortran, and it's actually hopeless to try and fully understand the semantics and still meet your deadline. But I was able to use a few tricks to prove that certain modifications wouldn't alter other functionality so I could get my work done. In the same program, there was well factored code with a nice object model, where it would behoove you to understand the part you're working on in a more conventional way.
On reflection, I think you may be right that good code is good code. Size of the system matters most when the code is bad.
There are lots of other cheap filters that companies do not use. For example, "Have you read Ayn Rand?" This question may well have better predictive power than a degree. And it's not hard to verify if the person is lying about whether they've read some at all.
Or, "how many blog posts do you have about programming?" Again this is easy to verify.
Who is Ayn Rand? An utterly dreadful writer and philosopher who believed that the axiom of reflexivity of identity had substantive ontological, social, moral and political consequences. It's no accident that Objectivists sought to justify their belief in the "virtue of selfishness" in the law of reflexivity; they have no use for altruism. Rand's philosophy glorifies the sociopath homo economicus, whose sole objective in life is to maximize his expected utility.
However, results in evolutionary game theory show that a society of self-seeking, self-regarding agents will generally face conditions that ultimately lead to its collapse.
Gary Cooper's goofy speech in The Fountainhead ( see http://www.youtube.com/watch?v=Zc7oZ9yWqO4 ) typifies Rand's attitudes. Among other preposterous propositions, Cooper is made to utter the nonsense that great inventions are uniformly the work of sole inventors, selfishly and reflexively seeking their own interests. This is ahistorical; see Against Intellectual Monopoly by Michele Boldrin and David K. Levine ( http://www.dklevine.com/general/intellectual/againstnew.htm ) for the history of inventions such as the steam engine, radio, telephone, and so on. In each case, ideas were in the air, and there were a number of people who came up with similar inventions more or less at the same time.
Cooper argues the basic notions of intellectual monopoly, which are that intellectual property is essentially indistinguishable from tangible property, and that all copies of ideas "belong" to their creator. These arguments come straight from the RIAA legal playbook. I'm surprised that any culture of hackers would want to subscribe to notions more commonly associated with corporate monopolists.
I suspect the point is that having heard of and bothered to read one of her books, whether you liked or agreed with it or not, probably implies various things about you.
A) You socialize(d) with people who read things that aren't sold in the grocery store.
B) You not only know how to read, but most likely voluntarily read a 700+ page book in your spare time in order to learn/see what it was about/etc...
C) If you can speak about what was in the book, and what you thought about it, you can follow the plot of a 700+ page book, you can understand the points the author was making, perhaps you can intuit the not-very-subtle philosophical and societal messages she was delivering, and you can discuss how you agree or disagree with those messages.
It's no IQ test, but frankly it's probably a much better question than "Do you have a degree?".
At least I'd rather work with people who have read a book like that, and have an opinion on it's content and the author's points (even if they hated the book/points/etc...), than the average CS degree graduate.
I'd rather work with a smart, well read person who likes to think than someone who hasn't read much.
Sure asking about Ayn Rand isn't really an intelligence test, but it's not a bad start. Smart people can learn about data structures and algorithms. Slow people who've managed to get a CS degree from some random college may have learned enough to pass, but there's no demonstration of smarts there.
I'm hiring based on people being smart, self-motivated, willing to learn, willing to think, and people I can hold a conversation with. Teaching someone like that how data structures work is a lot easier than teaching a degree holder how to be someone I want to work with and someone I can trust to be on the ball as new technologies come out.
Ayn Rand seems to be (significantly) more popular here than in the general population, judging from a dozen or so times I've seen her come up in comment threads. Isn't that more relevant than your personal, negative opinion of her philosophy?
So, out of curiosity, do these "results in evolutionary game theory" have a source?
"Isn't that more relevant than your personal, negative opinion of her philosophy?"
No, because it's not only a personal opinion, but a statement that Rand's philosophy of rational self interest is logically invalid ('rational' does not imply 'self interest'; the basis on the reflexivity of identity could fairly be called desperate) and scientifically incorrect. Rand's philosophy is incompatible with findings of reciprocal altruism in evolutionary biology and experimental game theory.
"So, out of curiosity, do these "results in evolutionary game theory" have a source?"
As you must be aware, many smart people disagree with you about Rand, and can back it up with something better than a youtube video. And anyway, you said her philosophy contains certain flaws. That doesn't imply it's not valuable and useful overall, so even if I concede your points, it's not very important. The reasons I like Rand have nothing to do with "axiom of reflexivity of identity" or that other stuff you said.
edit: no online sources? I don't normally pay $35+ because a hostile, anonymous internet commenter said something would refute someone I respect but didn't want to explain the ideas himself.
As for $35, it pains me to mention libraries...it was a source, with a link so that you could see something about the book.
I can't say I know of a single public intellectual or professional philosopher who takes Rand seriously. I do know of a well-regarded mathematical logician who does, but this is an aberration.
The degree question is also silly, but has non-zero predictive capability. So do these. Certainly they rule out plenty of good programmers, but the issue is: do they leave plenty of good programmers, and a higher quality pool of remaining applicants?
If good programmers read Rand or write blogs at a higher rate than bad programmers, then it works, even if it's frequently wrong about individual people.
The difference is, if companies started using the filters you suggest, then the candidates would soon catch up and everyone would have a programming blog and have read Cliff's notes for "Atlas Shrugged." So it's not, as they say, an evolutionarily stable strategy. That's why no one does it on a larger scale I guess.
For a demonstration of this principle, compare the quality of a person with a degree in Computer Science. Has the average gotten better in the last 30 years, stayed about the same, or gotten worse?
My feeling is it has gotten much worse as people have sought degrees solely for the purpose of using them to get jobs in the field, precisely as you suggest would happen to blogs and reading cliff's notes.
However, may I point out that blogging is very different from reading the cliff's notes of a book? A hiring manager can read your blog. If they simply check that you have a blog, well, whoop-de-doo. I have a blog, so that clearly proves nothing.
However, if the hiring manager reads your blog, they can deduce a great deal about what you pretend to think and how you communicate it. So it is a small example of your work, much as posting source code is a small example of your work.
they are generic and poor filters. for example do you read person x. If person X isn't a household name (e.g. Bill Gates) Then it says nothing about them other than none ever introduced them to that person. I'm willing to bet there are a significant number of good C++ programmers that don't/can't care or know who Bjarne Stroustrup is. I had to look up the spelling of his name (although I don't consider myself a good c++ programmer).
The ability to write blogs is irrelevant here too as what we most likely care about is the ability to program. Hence, the programming test.
Take the group of programmers who have written more than 30 blog posts about programming. X% of them are good hires, and 100-X% are bad hires.
Now take the group of programmers who have written less than 30 blog posts about programming. Y% of them are good hires, and 100-Y% are bad hires.
Is X > Y, or X == Y, or X < Y?
That is the issue.
And that is the way degrees are used (when used rationally). The claim with degrees is that X is more than Y, not that most good programmers have degrees or anything else. That may be so. Then someone defended using degrees by saying there is a lack of alternative tests that are sufficiently cheap. That's not true. There are lots of cheap tests, and I have suggested 2 for which I believe X>Y is likely, and which, if studied, might turn out to have a higher X than the degree test.
to a degree I did. Maybe not to the degree you desired.
My point was is that even though it filters out many of the worst programmers it would also filter out all of the really great programmers. I'd rather have my search take longer and cost more to have those great programmers on my team than end up with just good programmers.
Provided your filters (1) increase the probability that the remaining applicants are qualified and (2) do not reduce the applicant pool too much, then it is probably a good idea (even if unconventional).
Note however that criteria (2) is important. If you use the filter "Is named Linus Torvalds", you would certainly increase the probability that any given applicant is good. Then again, your applicant pool will drop to either 0.
Remember, all you know is P(qualified| resume says autodidactic)= P(qualified | autodidactic) * P(autodidactic | resume says autodidactic) . The second term can easily kill you. Do you trust everything people put on their resume?
On the other hand, P(degree | resume says degree) is probably close to 1, since a degree is so easy for an HR person to check.
Look, I've got nothing against hiring good people who have no degree, and I certainly don't think a degree proves much (my current students prove that conclusively). If you know you have such a good person, hire them, ignore the degree.
I'm just pointing out that the game is to find the good people. And that's a bit tricky to do.
[edit: by this way, in the interest of clarifying, what you want to optimize is not candidate quality, but (candidate quality - search costs). Don't ignore the search costs term.]