> We also tested Astra on SRE-Bench [15], a benchmark that measures whether models can reverse engineer software binaries to understand its core logic without access to raw source code. Astra solved 88.0% of tasks in a single attempt and 99.2% within four attempts, compared with 55.9% and 68.7% for GPT‑5.6 Sol, respectively.
So the closed source application should open its source in near future?
Surprisingly, I've had really good luck with reverse engineering on frontier models (without being part of CVP or similar). It's more the exploit development/PoC that it locks up on, which, as I use `pi`, I just switch the model to Kimi K3 to finish up making the PoC.
Ironically, due to the stringent guardrails on American models that exist to avoid giving adversaries a leg-up in cybersecurity, I end up feeding dozens of 0days straight to the CCP lol
I do RE mostly, and they do lock up for me. But what works for me is: warm up context with non-RE things with American models > switch to GLM 5.3 with actual request, let it fill context with some RE work > switch to American > switch back to GLM 5.3 and keep switching back and forth if anything gets flagged.
Solves the guardrail problem - check
Feeds China data to US - check
Feeds US data to China - check
But I think OpenAI has caught up with this, and it's why they're pushing so many things server-side and encrypt it (subagent messages, compaction, new context history thing).
I was listening to the Lex Fridman / DHH podcast last night [0], and DHH was saying that this is a new era for open source software. I'd agree, and also extend it open hardware.
Recently I've seen quite a few posts from people using AI to reverse engineer the Bluetooth protocol or such on devices that need a proprietary app. The same thing for firmware is surely coming, which is great as you can get lots of fun hardware from China, but it often has shitty firmware. Once that becomes the norm there's no reason not to make it open in the first place.
Not really? Nobody is talking about source-available code with a restrictive license. People are talking about having LLMs write up protocols and libraries and firmwares from scratch and making those freely available, as in open source.
So the closed source application should open its source in near future?
[15] https://arxiv.org/abs/2608.11469v1