I worked in AppSec in the past, made sense to me. Maybe you aren't the target au... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

rohansood15 6 months ago | parent | context | favorite | on: N-Day-Bench – Can LLMs find real vulnerabilities i...

I worked in AppSec in the past, made sense to me. Maybe you aren't the target audience?

You don't really need manual verification for these, the CVEs (vulnerabilities) are public and can be programmatically validated.

muldvarp 6 months ago [–]

Manual verification that the "judge" judges correctly.

Also, how exactly do you programmatically validate CVEs?

rohansood15 6 months ago | [–]

Most open-source CVEs will have a patch linked in their disclosure. You can get vulnerable code via the git diff, then just verify if it is part of the LLM's finding.

Consider applying for YC's Winter 2027 batch! Applications are open till November 2.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact