Feel like this could be the solution to “LLMs will write in binary” Elon was talking about. The problem with it is, assembly is tightly coupled to the hardware, so LLMs might have an easier time generating/understanding context of a “typed” assembly language but now need to understand the hardware and its constraints as part of the context or have it in its training data. I personally just think having that knowledge codified deterministically in the compiler is simpler, but I guess time will tell.
What would be the advantage over generating human-readable code like, say, c? Why not just tell it what hardware you're targeting as part of the prompt?
In theory, you could create a hardware block that decodes text based assembly generated by an LLM, which would allow you to load the direct output the LLM produced into memory. In effect the LLM could write self modifying code on the fly without a compiler / runtime / operating system. Say if the LLM was a hardware chip itself instead of software, that’s incredibly powerful.
I suppose. But you could just as easily do this with C as the target language and then invoke a C compiler. Or just generate machine code directly. It's not clear what assembly specifically provides.
Assembly decreases the complexity of the hardware decode block significantly, lower complexity, better speed. C was created to reduce the cognitive load on the developer, with LLMs we don’t technically need to reduce cognitive load if we say compute isn’t a finite resource (it typically is but for sake of argument here).
Assembly is just more aligned to creating purely hardware primitives of LLMs and reasoning than C or upper languages
