So, maybe this will come as a shock, but "computing olympiad" success doesn't correlate with success as a professional programmer. There's so much more than raw intellectual horsepower to being a good team engineer that it can't be captured with any one test.

Which is to say, the parent is right. Identifying good programmers is a hard problem. Harder than identifying people who do well at coding contests.

IOI and similar programming competitions do not select for good programmers. They select for people who can solve small algorithmic problems efficiently in a short time period with throwaway code. There is some overlap, but also a lot of perverse incentives. In professional programming, the tortoise usually beats the hare.

(I have competed at a national level and won a national college-level programming competition myself, and don't really take it all that seriously.)

I'm not just trying to find talented programmers. I'm trying to address the more general problem of figuring out whether somebody is good at X. You're right that the easiest way to judge on a case-by-case basis is to ask known experts. But since this is/was a cognitive bias of mine, I'm trying to overcome it myself.

I have spent a lot of time, done plenty of reading, and spent my own money on this problem. Like you said, I actually have paid experts to give me advice on how to identify talented people their field. More frequently though, I'll network and ask in an informal setting. And as a result, I have managed to severely correct several biases I had, such as thinking that anyone who couldn't do basic Math must be not be very bright.

One thing that's relatively easy (i.e., only took me a few months of effort) is learning how to identify generally smart people who learn fast. (I would actually divide this into learning social systems quickly vs. learning technical systems quickly.) If two people are entering a field or starting a job at the same time, I can generally predict which one of them will learn faster after a 30 minute conversation. The biggest surprise was that very smart people can be catastrophically bad at things "normal people" would consider basic-like eating spaghetti or not offending their interviewer (or even realizing they'd offended someone.) Before I tended to assume that fast learners would be good at most things and exceptional at a few things.

A more challenging problem: say I meet a lawyer, or doctor, or anyone with a lot of experience in an area I don't know much about. How can I tell if they're genuinely good at what they do? So far, general intelligence + percentile-based accomplishments ("I boosted revenues by X which is better than 95% of marketers...") + asking them to walk through a real life situation has been my best predictor. I'll validate this by asking known experts afterwards or looking up the person's career track.

Even so, it's not a particularly good predictor, and asking a real expert works much better. But by looking at actual data, even if it's relatively weak and anecdotal, my ability to accurately judge competence has become much stronger over the last few years. I think this is something that most businesses could benefit from, but the more common response seems to be throwing up their hands and complaining about the lack of qualified people.

Edit: when I said "how we judge talent is actually one of the biggest problems", I should have specified "we as a society." It clearly came across as "we as a company", sorry about that.