Doesn't this just suggest that the next frontier for powerful AI models is to ask challenging questions, not simply solve them?
Terry even says this: "In fact, it is now the identification of a promising problem which is the scarce and precious resource."
The creativity and insight needed to ask a question that Terry gets excited about is the next step. Perhaps OpenAI should create a set of challenging questions and offer a prize to solve them.
The incentives are massively skewed towards the AI labs investing their massive amounts of compute into being the first to solve an outstanding problem.
It's a marketing game for them, any societal benefits are secondary. Winning a prize is going to get headlines and feed into the "AGI soon, machine replaces another career" narrative they crave unlike coming up with some (possibly) interesting problems.
I think the problem with AI asking questions is that it will ask questions that are interesting to it but not necessarily us. AI, as a model, will never be a perfect copy of a human. It will always be a simulation, and thus to some extent, will ask questions that humans find irrelevant and solve problems that humans find irrelevant.
For anyone facing an existential crisis on AI, your ace in the hole is your humanity. Only you have it, and only you will be the best judge of what is good and interesting (to a human at least).
As far as I can tell, it's still not possible for an agent to reliably determine if a question is a good question. That means the test part of the loop cant be fulfilled.
That should be the easiest part of the loop. Frontier models are good judges of human preference and taste and should be excellent at prioritizing proposed questions. The generation part is much harder.
The water levels don't change seasonally..So I'm guessing it's a very high rainfall area? In which case erosion and nutrient leaching are bound to be issues for the agentic farmers. I wonder if they've thought to build soil with perennial ground cover species..
But as I say this, I realise the tree species seem to change very dynamically from season to season. So I'll have to assume either the temporal rate of change is actually a lot slower than one would expect in a model like this (allowing large tree species to regenerate annually), or there are new coniferous flowering tree varieties in this universe.
"What comes after vibe coding" is a different question to the one you're answering.
Feels like your answer is about better vibe coding: I'm using a tiered document system. It's just docs all the way down. Seems to be working for me, ask in another ten years.
What comes after vibe coding? At some point the AI will be asking humans if its new software saves us time and whether we'd pay money for it. Until then, there'll be some level of humans-asking-AI to solve a problem, probably with coarser and less detailed spec at each step.
Yes I think this is the current "standard" - keeping notes in tools like Obsidian and having agents query it, or peppering README.md files throughout the codebase. I call it "breadcrumbs" because I just leave things in the path for the agent to pick up on. But it doesn't answer questions like what are all the callers of this function? where is this data going? where is it coming from? So I'm trying to solve THIS part which is much more time consuming and prone to drifting in time.
Better vibe coding - maybe. I wanted to distance myself from vibe coding - trying to get agents to output more deterministic code as opposed to vibing. (if I understand vibe coding correctly it's stating your intention and letting yourself go wherever the agents are taking you)
I also think agentic coding is not going anywhere so I'm looking for ways to make it more appropriate for my job.
Is 'contextualised pretraining' a solution to baking in human alignment?
You can only post-train so much...
Try telling a child that martial arts isn't the solution to everything right after they've watched karate kid.
A weak analogy, but it seems very clear that the healthy psychological development of frontier models is something necessary to solve.
Some good insights could come from those working at the coalface of child-psychology.
(Uninsightful comment but I’m gonna put it here anyway)
The US spelling of haemoglobin is all kinds of wrong.
Love the site. Would love some algae in there though. Perhaps a desmid or diatom?
Quite a tangent, but for the purpose of avoiding anaerobic decomposition (and byproducts, CH4, H2S etc) of the dog poo and associated compostable bag (if you’re in one of those neighbourhoods), I do the same as your mum. If possible, flick it off the path. Else use a bag. Nature is full of the faeces of plenty of other things which we don’t bother picking up.
Depending on where you live, the patches of "nature" may be too small to absorb the feces, especially in modern cities where there are almost as many dogs as inhabitants.
It's a similar problem to why we don't urinate against trees - while in a countryside forest it may be ok, if 5 men do it every night after leaving the pub, the designated pissing tree will start to have problems due to soil change.
It’s a great process where I live. But you’re right. Doesn’t scale to populated areas.
Wonder what the potential microbial turnover of lawn is? Multiply that by the average walk length and I bet that could handle one or two nuggets per day, even in a city.
That’s a side hustle idea for any disengaged strava engineers. Leave me an acknowledgement on the ‘about’ page.
It's ok in wild bushes (as long as children don't usually play there), but what's the justification for dumping it in other people's bushes and gardens?
They probably would say "no" if you asked them, so you probably shouldn't. The OP's mom, I mean.
Dear Paul,
I'm sure it has been said elsewhere in the comments, but ironically I struggle to agree with this essay...which happens to be nicely and succinctly written.
I'm arguing that it's your own bias generated from the synthesis of your own idea that selects for sentences that effectively express the idea, and nothing to do with the writing itself.
The anecdote about the puddle who suddenly gains consciousness and remarks that the world is so perfectly formed around it, that it's proof of divine creation, seems to apply here.
The author generates an idea and is trying to articulate it. A well written sentence or paragraph that flows, pleases the author. This is because the idea they are trying to express is done in a satisfying way.
Thus the more pleasing the writing to the author, the more efficiently it articulates the original idea.
It's the author's bias, based on their own idea, that defines the level of 'pleasingness'.
Lastly, Paul, do you think the LLMs are any less satisfied with their confident and irrational hallucinations, than they are with their more well supported claims? Further, if you weren't aware that the output was ridiculous, would you be able to tell a accurate statement from a false one?
Terry even says this: "In fact, it is now the identification of a promising problem which is the scarce and precious resource."
The creativity and insight needed to ask a question that Terry gets excited about is the next step. Perhaps OpenAI should create a set of challenging questions and offer a prize to solve them.