AI needs a great deal of handholding which is different than just offloading tasks. You spend a lot of cognitive energy playing a slot machine hoping the RNG works out this time.
There’s definitely a skill to using AI but it just doesn’t generalize very well.
I still believe the slot machine analogy holds to some extent, but I can honestly say my winning percentage is at least 90% for one shot generated code now.
I think if you know it's limitations (inlcluding your own), I don't think about hoping anymore.
I should note that when I say AI, I mean the collective models from all the major providers. The most important lesson is, you need to ask around.
> There’s definitely a skill to using AI but it just doesn’t generalize very well.
This I agree with. The only way working with AI can really be benefical outside of dealing with AI is, we are visited by extremely intelligent beings that will fuck up in the weirdest ways.
When it works the first time you’re done, when it fails you spend more time on it. This biases the cognitive investment around failure.
> I don't think about hoping anymore.
Running the exact same prompt again that already failed doesn’t have a that high success rate, but it’s also very low effort. So IMO it’s often worth attempting.
> There isn’t a magic prompt I could have written before knowing the solution. A retrospective prompt can reproduce the 200 lines only because it embeds the invariants you discovered. The useful AI task would have been systematically finding counterexamples and iterating toward those invariants, not generating the final file in one shot.
For me the value with AI isn't the one shot, it is the fact that it can help you iterate faster. My biggest take awway so far with AI is it can help you fail fast. The faster you can fail, the faster you can iterate.
mine def has to be the initial impl of my python to my personal fave compiler ir. i got claude chat to write it in two sentences. 4 turns helping it rmeemver where it put shit becsuse of transcript issues
The code in http://github.com/gitsense/gsc-cli contains over 300 go files and I would say 90% were one shots. I can even prove that they are AI generated since I attribute the AI in each file, as shown in the example below:
Since I don't know go, I can't say which is the best. I am also sure they would not live up to a go experts expectations. I just know that since I don't know go, I would not have been able to create the tool. I can also prove that you can easily maintain AI generated code since I have been expanding and evolving it for over 6 months with pretty much no issues.
There’s definitely a skill to using AI but it just doesn’t generalize very well.