These are all important points and I love the analogy. But there is an even bigger issue with having LLMs write for you:
Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.
None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
> Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
I don't disagree, but I think it's often not appreciated how much there's other work to writing too.
The biggest one is that you have to communicate non-interactively to an unknown audience. Having to (literally) put it in someone else's assumed terms does help giving different perspectives into the matter, but doesn't necessarily help one's own thinking that much. Instead you have to do some of the reader's thinking for them.
You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Not all writing is thinking and not all thinking is writing.
Strange! I would quite specifically highlight “having to do the reader’s thinking for them”, as well as more generally developing the skill to “communicate non-interactively with an unknown audience”, as extremely valuable upgrades to my thinking.
Most valuable thinking is done at the margins, where you don’t have much capacity to emphasize with a diverse and unknown audience.
That said, abdicating to an LLM is the worst of all worlds - you’re not thinking and the product is not tailored.
The solution is obvious - write as much detail as you need and allow readers to interrogate the virtual you with an LLM, maybe not even reading what you write.
It's a kind of shared fiction of an imaginary person's thinking, really. The author writes "now, I know what you must be thinking", and hopes the reader will agree "OK, close enough".
> You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Hard disagree. Constraint is the driver of creativity. Also rewording sentences to sound better or make sense can make you reconceptualize the whole concept you are expressing
Sure, but doing an interpretative dance or an abstract painting can make you reconceptualize the whole concept too. But we're not really pushing those tools as much as writing.
Take for example a non-native writer of the language. I'm sure having to check up words from a dictionary may help to reconceptualize things, but I'm sure also that it's not often very efficient. And I think similar is going on for natives too for many types of writing.
It’s not clear that doing an abstract painting can make you reconceptualize in the way that reframing in other words does. The point about putting in other words is that you may stumble on a clearer, more tractable, more extensible framing. The kind of reframing an abstract painting does is very different, more like changing your attitude or way of looking. But I don’t think it ever leads to a place where you will suddenly find yourself with a sharper understanding that helps you communicate with others more effectively.
I'm not much of an artist, but I'd guess trying to paint a painting of how quicksort works, in a way someone else can grasp it too, can need a lot more reconceptualization than writing a description about it. Doing plots or diagrams, or even implementing an algorithm, for sure often need more thinking than writing a description.
Yes but diagrams aren’t abstract paintings (or interpretive dances), which I assume the original commenter meant as intentionally artsy and indirect forms of art.
The number of times the friction of writing has saved me from prematurely communicating a poorly understood idea must be in the hundreds or maybe thousands. For me when something is difficult to write about, it's a very good signal that I don't understand it well enough. So I think all the things you label as aspects of writing that aren't necessarily "thinking" are nonetheless good for thinking because they provide some necessary friction.
Hence The Great Smoothening of Minds we're all experiencing in this decrepit era. I blame the financial incentive, and welcome its disappearance. There are too many people doing computers just for the big paycheck, it would be more fun without them. Maybe the AI bubble popping will get rid of them.
> Writing helps us think about the world, it’s a pivotal intellectual technology.
Much like money decoupled selling and buying to move away from bartering, writing decoupled saying and hearing so they didn't have to happen at the same time. The incredible step that happened was not that people had to think a whole lot, it was that thinking that was already happening had to happen once.
> All writing is thinking when done by a human, you’re literally distilling your thoughts into words. You can’t write without thought.
Of course you can. You can write down exactly what you hear, for dictation.
You can write down a stream of consciousness and put barely any thought into it at all.
I can't help but feel most here are massively over estimating human writing. Human writing is, almost universally, terrible. We have entire jobs that are hard to fill just to make things sort of ok. Good writing is a small subset of human output.
I have been thinking over this comment for a while now. I think yes, you are correct that thought is involved in writing but I don't think it is possible to claim "all writing is thinking when done by a human". Like what type of thinking are you claiming here?
Because one can copy a text and write it down and that involves thinking in the sense that anything we do involves thinking fundamentally. But that thinking is different from thinking logically about a concept and writing it down which I think is where you are getting at.
The definition of writing and thinking is too broad in that sentence even though it does apply in several obvious cateogires within that at different levels.
And also "writing helps us think about the world" is too broad again. Why? Why does me writing "apt apt apt apt apt apt apt" help me think about the world? I just wrote it because i felt like writing it. Why wouldn't you consider that writing?
Yes, but not to understand or think about the world. In this case, writing was to express something, not to understand something. The overall point is that claiming that "human writing = human thinking" is too broad a general statement to make seriously.
That's what I am saying, zero thoughts is not possible unless you define what thoughts/thinking are you referring too.
Of course you cannot do anything without like basic thought otherwise you will not be able to do anything.
So I am not sure what you mean by non-contrived? It has to be contrived because we are talking specifics like I said.
If thinking = consiousness then yes you cannot write without being consious. But if thinking = having complex logical thoughts and writing about them then I would say yes it is possible to write without that.
My only claim is that just because you are writing something doesn't automatically mean you are having complex thoughts about what you are writing. So writing = thinking doesn't sit right. I can tell you i often just mindlessly write words on my journal sometimes but that is also a contrived example. Yet the existence of the example is the point. You cannot categorically claim something if exceptions (no matter how contrived) can be raised.
You wrote "apt apt apt apt apt apt" to prove a very specific point no? Absolutely requires reasoning and understanding the problem to go there. And good luck getting an LLM to do that.
Yes, I wrote it to prove that writing doesn't mean I understand the world. It responds to "writing helps you understand the world." My point was that, yes, of course you can find categories within it that apply, which is why the answer is not fundamentally incorrect. Claiming "writing = thinking" in a general sense is very broad, and that's my point.
This is an example of the problem. I did choose to write it in the way that although I knew it can be interpreted trivially, if the reader assumes I'm an imbecile. It was a "punchy" recap of the relatively long explanation for a HN comment and hopefully decently argued point.
Bad example, because translation is deeply creative. You can't blindly mill one language into another, because words and phrases and concepts and cultural references in one language frequently don't map 1:1. You have to find a way to convey meaning as closely as you can and not necessarily the words.
Amusingly I absolutely abhor translations done by people who think the way you do. I want a translation that's as literal as possible and which provides the necessary commentary for me to understand any alien concepts, idioms, customs, etc. I absolutely never want "translated" cultural references. At that point the "translator" is nothing more than a shitty fan fiction author as far as I'm concerned.
Of course the above requires actual work on the part of the consumer. I realize many don't want that, particularly when it comes to entertainment. So I appreciate that the other sort of "translation" exists but I think it's important to realize what exactly those are.
Thankfully LLMs are more or less to the point of providing what I'm after in near real time.
But you aren't disagreeing with your parent comment. You seem to have a strong opinion that you are; that opinion is incorrect.
Words refer to a broad semantic region, a phenomenon technically known as "polysemy".
The range of a word in one language is always different from the range of analogous words in another language. This is a classification problem. And a translator must think about how to solve it. Imagine a Venn diagram with 20 circles that each overlap the other 19 to differing degrees. What does it mean to designate one of those circles as "the literal translation" of a foreign word?
Of course nothing involving natural language and human culture is exact. A "literal" translation is obviously a slightly fuzzy concept that speaks to intent. However I think I provided enough context that this should be clear. I gave the example that I don't want "translated" cultural references or idioms but rather the (approximately) literal wording and some commentary from the translator providing the necessary context.
Obviously there are degrees to this and obviously preferences will vary. I acknowledged that.
I think you've misunderstood something. The comment I replied to specifies "words and phrases and concepts and cultural references". You only seem to be talking about words (and perhaps exceedingly simple turns of phrase). I was quite direct that I disagree when it comes to (among other things) the more complex idioms and certainly when it comes to any and all cultural references.
Imagine a localization attempting to replace a reference to an actor, political scandal, or other concrete cultural reference from one country with the "equivalent" from another. I've encountered that sort of thing before and while there are certainly those who appreciate it I am emphatically not one of them. As far as I'm concerned that's shitty fan fiction.
There are also a lot of examples in most (all?) languages that rely on repetitive sounds, easily mistaken words, or other strictly auditory features of the native language. You literally cannot translate those things. I do not want shitty fan fiction, I want an explanatory note.
- You're going to rant about it, because you want to, whether or not it's relevant to an existing conversation.
- You didn't bother to think about my comments.
- You didn't bother to think about habinero's comment either.
Here is the same passage of the Analects (part of the chapter Gongye Chang) in different translations:
--- Annping Chin ---
Zilu said, "We would like to hear what you would like to see yourself accomplish."
The Master said, "To give comfort to the old, to have the trust of my friends, and to have the young seeking to be near me."
--- David Hinton ---
Adept Lu then said: "No Master, we'd like to hear your greatest ambition."
"To comfort the old, to trust my friends, and to cherish the young."
---
Our focus here is on the second line, what Confucius says. Does he want to trust his friends, or does he want his friends to trust him?
Does he want to cherish the young, or does he want them to cherish him?
We might also ask, though the translators have agreed on this point, whether he wants to comfort the elderly or for the elderly to comfort him. (And we could further ask whether Confucius wants to personally comfort the elderly, or whether what he has in mind is for society in general to do that.)
All three clauses are formed the same way in the original Classical Chinese, and for a couple of interacting technical reasons they are all ambiguous in this way. Translators, as you can see, make different choices.
But of relevance here, when you're doing a translation to English, you have no option but to make a choice. It isn't possible to render the original text 'in literal translation' and append a note explaining what went wrong. You must commit to a meaning behind the text and phrase that meaning in English. You can also append a note explaining that you might have chosen wrong, but English simply doesn't allow you to do anything that parallels the source material.
I really think you aren't talking about the same thing that I am. I'm not sure why you're assuming bad faith on my part rather than engage in discussion to clarify.
I think it should be quite clear by now that I am not talking about isolated words that broadly lack an equivalent concept in the target language. I even quoted the bit from the original comment that I took issue with and proceeded to give examples so I'm really not sure where the misunderstanding between us could lie at this point. Perhaps you are the one who should stop and more carefully think about what I wrote?
As to your example. I certainly do not accept that this is a case where we should throw our hands up and accept that different translators will go about things differently. Those two sentences in english have (as you note) rather different meanings. So either one or both translators must be wrong.
You have indicated that the original work in the native language is ambiguous. In such a case I do not think it is remotely acceptable for a translator to arbitrarily pick one of several possible meanings and just run with it. If the original meaning of the text is ambiguous then removing that ambiguity changes the meaning thus it is a bad translation. The translator instead needs to faithfully communicate that ambiguity, possibly resorting to a note if it isn't possible to easily express such a thing in the target language.
I realize that many people aren't going to want such a marked up copy. But without all the gory detail the reader will be consuming some sort of bizarre partial fan fiction. Your example illustrates that perfectly.
I think you meant that translation is, case in point, based on inference. The person doing the translating is inferring, based on context, intent and meaning and imparting that on what their output is. That's generally not desired from a taxonomy perspective.
When you transfer other people's thoughts. On this board, we are engaged in pursuits where the truth matters. A lot of writing is about showing that you belong to the right in-groups. That is better done by repeating their talking points than sitting down and coming up with an earnest way to show that you agree.
I did think about this while writing, and I made the compromise to accept that someone will nitpick about it to hopefully drive the point better for those willing to read it charitably.
What is the charitable interpretation? I am not being coy or sarcastic here; it is genuinely (forgive the claude-ism) unclear to me what your arguments are.
With the last line I tried to condense (and oversimplify) about these ideas:
Not all writing is thinking: That all, even a lot, of writing, or parts of writing, is such that it will develop one's thinking much. For example most stuff I have to write, the dozen emails a day, the funding application boilerplates, the reports are stuff that don't really need (or deserve) much thinking but they have to get written. And even in the writing that deserves attention, there is stuff like grammar and spelling and surface style that usually take quite a bit of time after the ideas have been written down already.
Not all thinking is writing: for many cases writing is not a particularly efficient way to develop one's thinking, and e.g. visualizations, math, coding, discussions etc can be a lot better.
(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding. At first the code output wasn't good enough, and humans read and iterated on the code all day, so the details of the source code mattered. Now, the code is largely high quality and it meets a large set of guardrails we've set up over the years (linters, typecheckers, security checks, LLM-assisted code quality checkers), and it's just Claude working on the code, so the details matter less and engineers think a level or two up (machine code < assembly/bytecode < source code < conversation with agent < artifact with high level design).
I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document. But writing and coding are different enough in a number of ways that this is far from inevitable.
> wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
What use is that? I'm not being facetious, I'd really rather like to know.
Who or what is the audience for that sort of long form writing? If it's a human, why would they read it? They'd just give it to an LLM and get the salient points back. If the audience is another LLM, why expand it?
The only use case is an audience of humans who still read and understand, and those people aren't going to be interested in a message when it is not apparent that the sender actually understands the message themselves.
A lot of corporate documentation exists solely to measure if people are working or not.
That's why there's so many meetings in white collar companies. Because people can't understand what is going on at those documents so they just need to "align".
LLMs are amazing at generating this useless documentation that goes absolutely nowhere.
1. I have a bunch of data or research that I've gathered with a unique hypothesis
2. Having gotten my arms around that pile of information, I believe I have a compelling thesis to put forth
3. I design the narrative arc and of the thesis. The important parts, the necessary but not sufficient scaffolding.
4. An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
Who is this for? Well, quite possibly the human who asked for it. It's pretty informative to read back a research brief in full that you helped do the scaffolding.
Also of very clear use is other AI's who did not have the same unique hypothesis and did not gather the supporting evidence. It's an interesting angle for others to build on.
And of course, other humans! Most human written content gets almost zero readers today as it is. And I suppose LLM content probably pulls the asymptote closer to zero, but some pieces of content may be genuinely interesting or useful.
> An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
I think this certainly has some value but this claim in and of itself is stated like your hand-wavy step 3. How do they fact check claims and connect the dots?
Maybe LLMs get there but currently they write in an extremely verbose manner, and things that have gotten into the context window that are no longer relevant continue to stick around (just try having it write some code, then work some of it back to simplify the problem - it will insist on writing comments about code that no longer exists).
Right now using an LLM to write documents is like taking a superhighway to travel 100 meters. Yeah you're doing a lot but is all that really necessary?
I won't deny that LLMs will never have a place in writing. But I personally don't think the current form is "the one that actually lands" (!).
Then the reader can use an LLM to compress it back, and you can then interrogate it for details.
I used to say this was the future of advertising (cr sales person prompts “we have some new EV SUVs on the lot”; GPT generates an ad email with a synthetic video, blinking text etc; then the recipient’s spam processor tells them “that dealer has some new SUVs”. I suppose the same could happen with so-called “long form”.
> What use is that? I'm not being facetious, I'd really rather like to know.
People are terrible at writing. Near universally bad. Even good writers have drafts and editors.
There is a constant refrain here that somehow short messages are more valuable than longer ones. But that assumes it's understandable. Lots of short content is, frankly, awful because the writer cannot put themselves in the position of the reader and explain all the things around the point they're making that the reader really should be told.
You can view writing as translation. From your language to a language your audience speaks. At that level is it so odd if the word count differs from one side to the other?
But this is the same with coding. The reason that AI can write code from a description that is shorter than the output is in large part because it makes decisions about the behavior that were unspecified in your prompt. We accept this for coding apparently, I guess because those decisions are often unimportant. We might accept it for writing too. I hope not.
Not necessarily. Poorly worded, ambiguous, confusingly ordered writing can be massively improved without changing the core content. Better setups and explanations can be longer without changing the message or meaning.
Look at it the other way, could you take a good longer message you’ve written and make it shorter and less readable for your audience while still making sense to you and containing the key points?
wasn't it thoreau who said "Not that the story need be long, but it will take a long while to make it short"
personally, I think there's a time and place for short versus long, just like there's a time and place for a 45mins TV episode versus a 2 hour marathon movie.
You claim writing is thinking, but imply writing code isn't thinking.
My opinion of LLM design review isn't that high - it seems to miss design tweaks that could vastly simplify corner cases. But if your code isn't written for human consumption maybe it doesn't matter. I'm still directly responsible for what I commit, so I can't just offload it to Claude.
That's something I struggle with, I try to get LLMs to output code I don't care much about and focus on the parts I do and it kind of works but the problem is that reading code written by the LLM is even worth than reading LLM generating text. It's nauseating and you still have to read what the LLM did if you want to really work on the parts that matter.
People have been "writing code" like this for decades. That a programmer happens to do it doesn't turn it into writing code, no one would have made that claim 25 years ago, and people who aren't programmers wouldn't make that claim today.
It's like watching somebody about to be hit by a bus. You yell, you wave your arms, but they either don't hear you, or they don't believe you. The last thing that goes through their head is a Greyhound's hood ornament.
There is no subtext to bigstrat2003's argument, other than "I'm wrong, and I don't care." If you give persistent, repeatable instructions to a computer, you are programming it. If you disagree, you are gatekeeping. It's that simple.
The most popular programming languages in 2030 will, in fact, be English and Mandarin. Deal with it and get over it.
----------------
Edit, to bcrosby95: Look up the etymology of the word 'computer'. It didn't originally have anything to do with hardware. The first computers were people, who were told what to do ("programmed") without necessarily knowing what they were working on in a big-picture sense.
There is subtext to my argument, namely that you're the one standing in front of the bus, and trying to pull the rest of us in front of the bus, while shouting at the people standing safely out of the road. I'll continue to reject the replacement of humanity. (That doesn't mean remaining ignorant of AI as a technology. It means rejecting the idea that we should be thrilled to be replaced by AI.)
I'm game. Say $1000, donated to a charity of the winner's choice? How do you want to set the bet up, and how do you think it should be decided?
To be precise: I will bet that high-level programming languages won't be any less popular as a whole, but the vast majority of code will be written by AI rather than humans, working from specs written in natural language or something very close to it.
What we call "source code" today will be thought of as "object code" by 2030. Something that occasionally needs to be inspected by humans, but rarely authored directly. Anyone not writing code this way had better be doing it as a hobby, because almost no one will pay for it.
> You and the other poster just need to find some agreeable benchmark so you can decide who won.
That is the hard part. I'm not sure the usage numbers we'd need to decide the outcome are public now, and I can't predict ~4yr out whether any currently available metric will continue to be available. Stack Overflow's popular language thing has been going for a while, that will probably still exist (if SO does) but do we have any reliable metrics to tell us what fraction of code is AI generated?
I think the outcome will be very obvious when the the time comes to settle the bet, but I agree that it's not easy to write those terms today.
But speaking of Stack Overflow, that's a good point to examine more closely. Simply looking at Stack Overflow's usage trend [1] should have been a strong clue that a seismic shift was under way. A couple of things could be responsible for it, though:
1) Stack Overflow's movers and shakers have finally assholed themselves into irrelevance. That's possible, and it could potentially explain the secular decline that began around 2015. But it doesn't explain the rate of descent since 2023, given that the core rules haven't changed for years. It also doesn't explain the meteoric rise between 2008-2014. What they were doing clearly worked, right up until it didn't.
2) AI is now answering questions that would previously have been posted to SO. That's the conventional argument. Hard to dispute it. And it leads to...
3) Stack Overflow has not failed its mission, but fulfilled it: most of the code that will ever need to be written already has been. That's an argument I've never seen anyone else propose, likely because it's a really stupid argument that's been made many times before. I think it's true this time, though, at least until genuinely-new hardware paradigms come along. I think it's a big reason why AI will be doing our jobs for us going forward. Programming is a robot's job now.
3a) As a corollary not related to the SO question, I think the math community is starting to wake up to a similar realization: we have all the math we need. Or, rather, we have all the math we can comprehend. The low-hanging fruit is all gone. Mochizuki's work, which requires a large part of multiple peoples' careers to prove or refute, is an example of that phenomenon. Leading mathematicians are coming to recognize that their best shot at contributing to progress in math is to work on AI.
The groundwork has been laid, and now we need to find better ways to reuse and recycle what we have. That's how AI will help us reach the next level. It's just crazy to think that our industry will be recognizable in 3-5 more years.
Other person probably doesn’t like the idea that LLMs will replace hard earned skills. On the flip side, I bet you’ve seen your skills atrophy at an alarming rate and are trying to justify it.
Both sides come from fear. Just relax and take things as they come. Whatever happens happens.
Maybe. I'll have to think about that one. It doesn't feel like fear to me, but that could be some kind of coping mechanism. From where I sit, I don't believe I fear this AI stuff because I don't really see what all the fuss is about. I don't think I'll be replaced by Claude anytime soon, because fundamentally language models cannot do the things necessary to compete with me. They can generate large amounts of relatively mediocre code quickly, but that's not really what anyone expects of me in the workplace.
Instead, what I do is I think deeply and understand things. And I refine my understanding of those things. And eventually, I have some insight that proves extremely valuable to my employer. Or sometimes not! But on balance it tends to work out swimmingly in their favor. This may sometimes involve writing lots of code, sometimes it involves writing very tiny amounts of code. Sometimes it involves simply deleting code. But the code part of that job is really minimal. It's all the other things--developing understanding, having insights, thinking really hard, making connections, communicating--that are valuable. Banging out code was never a big deal. I'm good at writing clear, correct code and I believe I can do it better than an LLM, if not faster. But that never mattered very much, what actually matters is learning and adapting. I'll be scared when someone makes an "AI" that can do that competitively, but I'm not worried this will happen anytime soon.
So the premises of this "LLM crisis" seem deluded to me. The people saying "everyone will be using an LLM all day every day for their job" seem to have no idea what jobs are like or what LLMs are (in)capable of. The people talking about AI paperclip apocalypses seem like they're having a really bad drug trip. The stock market bros and VCs throwing money as hard as they can after whimsical sci-fi fantasies seem like they've gone completely batshit insane. Meanwhile managers are forcing workers to use LLMs for everything, for no clear reason, and push them into products where it does nothing good. So therefore every little spineless corporate toady is clambering on top of the other little spineless corporate toadies to show big mr manager what a good AI boy they are. It seems like y'all have lost your entire minds because the computer does words now.
It's not fear. It's disgust. I'm disgusted by all of this.
because fundamentally language models cannot do the things necessary to compete with me.
One imagines Lee Sedol telling himself the same thing. True one day and false the next.
It's all the other things--developing understanding, having insights, thinking really hard, making connections, communicating--that are valuable
Spend some time working with frontier models like Fable and Astra. It's clear you haven't. Or if you have, you haven't paid attention to anything but acceptance of the final work product.
"Thinking really hard," LOL. You won't beat an LLM at that. No one ever will again. Insights are still our domain as humans -- if only because the motivations behind them are still entirely ours -- but now we can put those insights into practice at a rate never possible before. And your reaction is disgust?
Meanwhile managers are forcing workers to use LLMs for everything, for no clear reason
I'm with you there. Don't worry, those managers will be the next to go.
Why is it always "you clearly haven't used the latest model, let me tell you this time it'll blow your socks off!"
How many times have we heard this? If it wasn't true all the other times why must I believe it now?
Here's a challenge: show something impressive. Show me how using your LLM will actually help me do something better. How come if this stuff is so great, it's just not all that obvious? I'm looking around and over the past year or 18mo I'm just not seeing a whole lot of change in software. Maybe increased bugs shipped?
How many times have we heard this? If it wasn't true all the other times why must I believe it now?
It was true the first time somebody said, "Hey, check out this ChatGPT thing." It blew my socks off. I still haven't found them. I guess they're probably behind the dryer or something.
I'm looking around and over the past year or 18mo I'm just not seeing a whole lot of change in software. Maybe increased bugs shipped?
I think you're just demanding too much too soon. Peoples' workflows are still adjusting. As you said yourself, correctly, management is being stupid about this whole thing.
The improvements won't all be visible at once. I will ship my next hardware product sooner than I would have, because Fable routed a board for me. Specifically, it wrote a Python program to solve a nasty routing problem that I just didn't feel like working on.
This morning, I was pissed because the volume shadow copy program I use for backups, Casper, screwed up a 30-hour job because it had scheduled a task behind my back, without my knowledge, that tried to perform the same backup to the same drive at the same time. Fable figured out what happened; all I knew was that the boot sector on the target drive was hosed. I will avoid that problem in the future by telling Fable or Astra to find me a simple command-line shadow copy utility that doesn't suck, and if no such utility exists, write it from scratch. That will not be a problem for Fable or Astra, but I wouldn't have dreamed of relying on an LLM to do that last year.
Will this win be visible to you, as a consumer? No, because LOL at anybody trying to sell something like that from here on out.
The software that I develop to support the hardware? I rarely work on it anymore, as of a few weeks ago. When I want a new feature added, I ask Claude (or, again, Astra, which is now better than Fable in some respects.) Claude could do a lot of that work last year, but it needed extensive babysitting. Now it has taste. It chooses the controls, lays out the dialog box and writes the help text. You won't notice the details as a consumer, you'll just see more rapid incremental improvements coming out for my software and ultimately the hardware as well.
Speaking of my very-obscure hardware, last week I ran into a very obscure firmware bug in some code I wrote. Or, rather, my customer did. Different customers are demanding both new features and bug fixes, and I only have the bandwidth for one of those. So: "Claude, see if you can figure out why the board stopped responding to input X when input Y went away. Here are the component data sheets." Took it about an hour but it would have taken me longer.
I would like to say I used the time it saved me to work in parallel on new features, but in reality I spent it here instead, arguing with Luddites. They say "you are your calendar," so I guess I enjoy that even more than building new stuff. Funny how much the clankers are teaching me about myself!
LLMs do not produce repeatable results, by their nature. Two people can give the exact same prompt to the exact same LLM and get different results. It's not a straightforward 1 + 1 always equals 2 process. This is like telling someone else to code something for you. You relinquish control of all the details.
Calling it gatekeeping is just laughable. That's like saying it's gatekeeping to say that the painter painted their painting, and that the person who commissioned the painting did not paint it. It's wholely absurd.
Anybody can pick up a book and learn to actually code themselves. Or you can use an LLM to try to make things without bothering with that. But even if the LLM worked perfectly, pretending these are the same thing is silly.
The history of the word computer is obviously irrelevant. Words change, it turns out.
It isn’t really by their nature that they don’t give repeatable results, right? They’re just a bunch of math, but they perform better with randomness injected so we choose do to so.
(I’ve heard about some GPU compute nuance meaning that even without randomness injected they still wouldn’t quite be deterministic, but that’s also not core to their nature)
You are just talking about the output though. If you only think at a "higher level" you aren't doing the actual thinking. Its the same with code. The output may be good enough, but over time you lose touch with the details to the extent that you can no longer serve a useful steering function for the organization. Before coding agents I'd seen this with many humans when they get promoted passed the point where they work with code directly and can't figure out how to add value there.
Expands into a document for _who_ to read? another LLM to re-compress?
It's baffling you people are in control of such a strong product when you are obsessed with this intellectual pornography; wow - look at how smart it made my thoughts look (n.b. look, not read). Don't look too close. And certainly don't ask me what it means.
Many programmers don't write long text; their way of getting a deep understanding of a problem domain is to build something, is to write code - in a process very similar to writing a long piece of text - it has the same reflection and externalization of thought.
The widespread introduction of LLM code generation is very destructive to that.
Perhaps LLMs can be brought to support human cognition in the same way writing can; but that has yet to be designed and it does not seem to be the way things are heading.
LLM prompts definitely don’t feel like “writing” to me in the way that writing a blog post (or even an HN comment) does or writing code used to. There is nothing to work out, you don’t really need to think. You’re _typing_ sure, but I don’t think it counts as writing maybe because the text that you write is thrown away.
I'm way more articulated now than I've ever been, because in the past I didn't have to -- I never write a blog post -- but I've learned that well thought-out writing, with clear description, will produce better code.
The bottom line is, prompting is definitely writing.
Idk man I get a ton out of gaming and other media. If I say “I’m a cinephile” no one bats an eye, it’s seen as elevated and intellectually stimulating. Your examples reflect more on the gate keeping we do with what’s considered “worthy of our time” and “art” than the value of LLM’s.
If you get an AI to review the code especially for security, it does a very good job st finding issues. Better than any human reviewers I have worked with, and getting better. As someone who works in security , I feel much less worried about security bugs on code reviewed by a AI for security issues, be it written by AI or human.
Compilers like GCC are deterministic and the source code already fully defines the behavior. LLMs are non-deterministic and will accept ambiguity, filling in details where you haven’t. These sorts of comparisons aren’t really fair.
In the case of writing, it’s like hiring someone to write a book for you vs. hiring someone to translate a book you wrote into another language. In the first case, you didn’t really define the message for readers, whereas in the second case you did, and the translator is converting that same message for another audience to consume.
Sure, but I don't know what GCC's behavior is, and I don't vet behavior differences between compiler upgrades. As long as the output works, why does it matter that the black box is deterministic?
It doesn’t matter how exactly GCC works, as long as the behavior is deterministic and consistent (GCC is likely maintaining backwards compatibility between versions, so the behavior of your code likely hasn’t changed). In that case, you can reason about the behavior you need and write your code appropriately.
In the case of LLMs, the behavior is non-deterministic and inconsistent. If I don’t explain how handle an edge case or give a performance constraint, the LLM will still produce code and may do so in different ways, handling edge cases differently and with different performance characteristics. I can’t reason about how the LLM will fill in those gaps, it’s “random.”
Maybe you don’t care about how the LLM handles those edge cases or handles performance, but that’s different than a deterministic abstraction whose implementation details you don’t care about, but whose logic and performance is deterministic and consistent
Sure, I manually test the output of the LLM. Manual testing is actually the main role for humans doing software engineering these days.
I wouldn't use it for flight control software yet, at least not without careful review, but most software isn't exactly critical. At the same time, I wouldn't trust flight control software that was only reviewed by humans, since AI is so much better at debugging.
We'll probably need humans in the loop for safety critical software for at least a year or two, before AI fully outpaces humans at generating correct code.
How so? As long as it works to spec, I haven't had anyone care. They literally hire people so they don't need to care about the details. Put money in, get working software out.
And, AI is rapidly getting better than people at both code review and authorship, so a human deeply involved is turning into nothing but a slowdown. The main purpose people have is testing that the specs were, in fact, implemented properly.
I guarantee your specs/testing are either inadequate and/or you're not leveraging lots of existing (and probably free open source) code that was already written by humans and meets the spec better without ever needing an LLM.
The vast majority of properly written software was already plumbing well over a decade ago. The software engineering is making high level decisions based on experience with respect to the existing tools and the needs of the business. If you're not already using LLMs that way, you would have been a similarly bad manager of human devs writing similar inadequate slop. Less code has always been better code.
The line in the sand for these arguments really ought to be whether you think LLMs are better than humans who actually know what they're doing.
If you think LLMs are better, or could get better while continuing to use statistical methods, you automatically lose the argument (delusional/ignorant) and any hope of regaining credibility. That's not dogma. That's the science.
No, because GCC doesn't randomly fuck up the assembly generation (much less on a fairly frequent basis the way LLMs do). If it did, you bet I'd be reviewing the assembly line by line, or decline to use such a poorly performing tool (as I have with LLMs).
For instructions you really care about, yes of course you review the assembly output! Usually when you're doing SIMD or want to check atomics are doing what you expect.
You can do that with LLMs for the parts you really care about too. The LLMs aren't regenerating the codebase from scratch every time, so the results stick around.
> wonder if long form writing will go the way of code.
I worry about AI Loopidity here though. Think about the similar analogy of email. If my set of ideas is condensable to bullet points, but I use AI to expand the content, then I add no information density and a lot of noise. Other folks then use AI to summarize the content to a list of bullet points, ideally the same but not certainly the same, and thus communication has been only partially successful.
> it's just Claude working on the code, so the details matter less
If you're not billed for usage, anyway.
Otherwise, for the other 99% of folks, that attitude is of course a pit trap that captures code bases and makes them maintainable only through the providers -- presumably one or few -- with a rich enough model to keep up with the growing mess. Preserving a code base that's legible, organized, and fundamentally maintainable by both humans and trailing commodity models is of imminent concern for anybody who doesn't want their margin strangled by your employer once it's too late to have other options.
As frontier capabilities advance, the details don't matter less; they matter more.
Writing is already amenable to many different levels of abstraction, though. If an LLM can expand your outline into writing, then you aren’t writing at the correct level of abstraction in my opinion; you should instead be explaining how you arrived at your chosen outline. You don’t need to explain the details because any party can generate those with an LLM; same as how many PRs today can be auto-generated and no one needs to read implementations; that is no longer the correct level abstraction to work at. This should actually free us to do work at a higher level of abstraction —- more consideration of strategy, objectives, etc and less worry about implementation details.
This only applies if LLMs aren't making mistakes 20% of the time and that's the problem. When you're only saving time on the easy part, it doesn't matter if you're working twice as fast because review of the tricky parts is still going to take 80% of what it would have taken to do the whole thing. Total effort ends up being more rather than less if you want the same quality.
I mean these are cold inhumane companies and their employees reflect it. SF is truly where human ingenuity goes to die, truly a blight on the industry as a whole and holding us back tremendously.
[non-AI org] I despise this and call it out every time I see it. Some dude hooked up an LLM autoresponder to his email, sent some nauseating AI slop to a huge distribution list.
I couldn’t help myself, replied and asked him for a recipe for delicious apple cobbler and hiking trail recommendations in Glasgow, which “he” immediately provided. Highlight of my career.
I think my core argument is this: I have access to every bit of information your AI does, so if I want an AI answer I’ll get one myself. If that isn’t true, why are you hoarding information? Push it somewhere we can all see it. So the only reason I would send you a message is to access _your_ brain. I have no interest in talking to an AI through a worse interface.
You could try this [0] I’ve started doing this and it’s helped a lot.
1. I understand fully the code and everything it does
2. You can pick up on mistakes super early and it can adjust the plan is it goes.
3. Faster than writing it by hand but slower than letting the LLM do it.
One could imagine a universe where the agent fills in citations and supportive points and so on, or makes a more conclusive argument but it seems you’d get better results leaving that to read time if it’s a one-shot. Steering prompts etc. with tool use to bring in other sources etc of course change this entirely. And regardless, I doubt we will read content like that directly ever again. Agents will act as per-person highly specialized adapter layers for information transmission.
The entire point is what runtime you’re running your code on. A computer with any modern stack requires a lot of text for you to communicate “spin a square around on its center” to it. A human requires only that short string because they have a faster natural language interpreter.
Text meant for a human can communicate “spin a square around its center” much better than any code that mimics it. In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
> In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
Maybe sometimes, but not always. When you need to actually render the thing you have all kinds of micro decisions, like where to put the square, what color, how fast it spins, etc.
You might not care about the details, but maybe you do. If it spins at 10000 rpm, will you care then?
Natural language, and human communication in general, is ambiguous, and coding is in great part about disambiguation.
Sure, you can use English to disambiguate as much as needed, but wouldn’t you then end up with some yaml-like spec that wasn’t much easier to create in the first place?
>(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
You get pushback for this? I saw an anthropic job post recently, and they wanted you specifically to have claude muck with your resume before applying.
I suspect some amount of long form writing will go the way of code - long form writing for the purpose of consumption by other AIs. Writing as a means of exchanging qualitative information, with no regard for how the reader will feel about it (beyond understanding what the words mean). Not everything can be distilled into data, but this doesn’t mean it is beyond the reach of LLMs.
On the other hand, long form writing for human consumption seems like it may evade LLMs for much, much longer.
What makes you think it will take a long time? AI seems capable of imitating any writing style if prompted to do so already, and I think it will get better on this quickly since the AI writing style is a main focus of AI labs right now. I can see no reason at all to believe this is a matter of years still, more like a few months.
The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "content" of the writing, which is bounded from above[1] by the information content of the prompt.
So, I'm not sure if it's a question of time at all: if a LLM text contains some piece of information beyond the information that went into the prompt, where does this "extra" information come from? [Note, I'm not thinking about facts which could trivially come from the training corpus, I'm thinking specifically as information in the sense of intended message from sender (author) to receiver (reader)]
You can have a great “writing style” and still put together really crappy long-form work. The problem is that AI writing, particularly creative writing, is too repetitive, too predictable, too trope-laden.
All of the things you say are very true in the near term for short form writing - a page or two of Claudeslop will probably be much easier to swallow in a year or two than it is now. But I don’t see a path to fully AI-generated novels or long-form investigative journalism becoming mainstream in the next couple of years.
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
You can already do this. And you can build pipelines where AI performs fact checks on what it writes, with citations a human can reference as well.
> I do all of my writing by hand, since it helps me think.
Later
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
So in the future, it won't be necessary for you to think?
Kinda. Walking the dog with me this evening, Christa saw the license plate 8531 PRI and asked me if 8531 really was prime. The 2, 3, 5 checks are automatic, and 8531-8400 = 131, which obviously isn't divisible by 7, but I would've had to check the rest of the two-digit prime factors except 97 - 8531 is greater than 90 squared, less than 97 squared, though, I reckon. So I said I don't know. But Google would know, you know? If your LLM buddy is hanging out active on your phone all the time, it can - maybe not yet, but foreseeably - answer every question, no thought required, and carry out any expressed desire.
"You and the AI agree on an outline or other high level representation, then the LLM expands it into a document."
I am a subject matter expert in certain fields of real estate in Canada. This is how I draft my teaching materials, articles and educational content. I use it as a human amplifier of my brain as it increases my output which other people will pay money for. People who complain that LLM writing is slop have not encountered good AI-human hybrid outputs. Like visual effects in movies, the best examples are imperceptible and do not get in the way of storytelling.
I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.
Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.
I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.
I've tried doing it that way, and thought it was even acceptable for the reasons you said. I did learn a lot through that process and clarified my ideas. But later I rewrote the whole thing from scratch and then had the LLM review it. It made some good suggestions but no substantial changes. The difference was night and day. That final product had my voice, and I understood it better. LLMs are powerful tools and can improve quite a lot of the writing process, but using them to do all the writing leaves a lot on table along with your fly open.
I have found this extremely relevant as a (primarily) non-verbal thinker.
I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.
(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)
For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.
If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.
This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.
I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily
Interesting. I have designed code that way. And the shapes aren't UML diagram elements or anything like that, they're just... shapes. I'll slowly walk around, in the hall or outside, and be kind of seeing these shapes and vaguely moving my hands around as I sort out the relationships between them.
I think I have produced reasonably good designs. Don't ask me to teach anyone how I do it, though.
I find this and the parent comment highly relatable with the caveat that I also find it extremely intuitive and rewarding to get good outputs from quality LLM's like Sol or Astra, and I haven't had any trouble with "flow state".
One of the most rewarding things for me is figuring out a good shape for a system and how it would interoperate with the other systems, especially in a way that reframes other parts of the codebase in a way that bring clarity and makes it more intuitive to work with. Creating the right ontologies can make all the difference in what you can do with a project. It's a form of creating mathematical objects.
For example, a Unity game I work on has quest and dialog systems driven by visual scripting graphs. We had two way dialog with different units for player response choices and npc dialog. But we wanted to expand to letting NPC's have dialog with each other as well as conversations with more than two participants. I went outside and thought it over, which largely amounted to visualizing a dialog node graph and a feeling in the back of my mind like it was trying to perform a kind of geometric shape-fitting exercise. A fitment solution jumped out at me to have only one "Dialog" node shared by all participants, with a "participant" value on it. If the player parses this node then the options go on-screen as responses, while if an NPC parses this node with multiple options in it, it picks one. And this lets you voice the player if you want, and enables some things like overhearing other NPC's talk to an NPC then talking to that NPC yourself and having the same tree.
And for quests, the quests had just been for the player, but I was thinking about how to make scripted events in-game easiest to work with for script team who primarily works in visual scripting. Similar story - let the NPC's have their own little quests, with task stages, which are easy to track and make branching choices from, and let the NPC's definition for how to use that quest contain a collection of actions to override the typical actions available to it, so an NPC in a specific "quest" can't do things you don't want it to do, a common enough case that it's preferable to making a series of conditions on the general action planner like "not in quest A"
And timing myself, it took 1-2 hours each time to write out the detailed plan for how I wanted each thing implemented in the game with some other tasks thrown in, and it paid off after Astra worked on it until it was done. It was awesome coming back to something pretty much exactly what I asked for each time.
"I don't, generally, think in words" is not the same as "I don't think in words".
Like I already said, if thought was exclusively in words for humans, humans wouldn't have the "It's on the tip of my tongue" problem. It's blindingly obvious that thought does not occur exclusively with words.
+1... this is me as well. Using LLMs in a conversational mode does not fit my thinking. I use AI help in my editor through targeted code generation, explanations, etc. and I'm writing my own harness to hopefully get a better feel for the shape of LLMs that way. How are you adapting?
Same. I do not use LLMs for writing and I recommend everyone i work with to do the same. Even for code, I come to the code with the idea of what I want and provide that to the agent, then we iterate on what needs iteration or decisions.
Just did this for some caching, started with the structure of what I wanted the cache to look like, asked the agent to start the work, didn't like how the architecture came about, scratched it and rewrote the whole thing, so it fit the model I now wanted.
Was also just having this discussion with friends, that I can only think seriously about a subject if i can put it to paper (even virtual paper). Writing lets me organize my thoughts, clearly define my assumptions and see if any of it make any sense. I can't imagine what it would be like if i couldn't write, my brain just doesn't work without it.
I now have dozens of projects where I've embraced the yolo. I went through elaborate systems, workflows, review triages, architectural linters, specialist agents, yet, they all still suffered the same fate - slop which I don't understand and now LLMs don't understand either.
The only thing that worked was a standing instruction and periodic system reminder injected from the harness:
"If any assumption doesn't hold, if there's a fork in the road, any architectural decision needs be made, STOP and report back to the user. Do not try to push through the problem."
This has worked remarkably well for me. Now I have to think a whole lot more. It's much slower, yes, but I don't really see any other way that doesn't end up in garbage.
I often write a reply to a HN post, just to think about the topic or idea and decide how I feel about an issue. More often than not, I just delete the post without ever having submitted it.
They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Ofc the rest is all right on, but I'd quibble with this specific idea. LLMs are absolutely targeted at modeling human understanding, which is the same faculty that contains what we call perception (!= sensibility) and intuition (!= rationality). It would be nice to train them to be completely alien from the ground up, but
A) we only know of one species capable of metacognitive understanding,
B) we already tried that in the 1970s, and it was good work but often evolved into what we'd call boring ol' computing rather than AI, and
C) an alien mind wouldn't be a very good agent, for a ton of reasons relating to affect, conversational rythyms, cultural understanding, etc.
The trick is to make something that acts like a human but with the affordances of a computer (e.g. scalibility, symbolic certainty), without making it so human that it takes issue with its existential reality and/or use of its labor...
We don't have a reward function for "human understanding". We reward the appearance of understanding. We define goals that we cannot conceive of reaching without something like understanding happening. There is something happening, but it is alien and counter-intuitive - it makes bizarre mistakes that betray it - and we don't know what it is. I'm pretty sure it is not human understanding.
Which is exactly what happens with human evolution and development. Sure, we can say LLMs don’t have “human” understanding - which is something we can’t really define anyway - as long as we’re not trying to claim LLMs don’t have understanding at all. The latter is a much higher bar.
> We define goals that we cannot conceive of reaching without something like understanding happening.
Functionally speaking, that is understanding. Again if you want to go past a functional definition, that’s a bar which no one can clear right now.
I don't think its the same. Evolution dealt with the real world where there were real consequences to poor understanding. The reason LLMs are good at math and programming is because selection is truly based on results, not perceived results.
I think AI models do have something like understanding - I think Leela understands chess and I think Claude understands code in some very real sense, though not a human sense.
But for general writing, you have to understand the world at large and there is no sufficient RL for that. Do you really not see the constant errors that AI make that betrays a lack of understanding the world? I see them so constantly I rarely think about them, I just skim over that slop and move on.
I think your claim is narrower than I was imagining.
Sure, the exact nature of the understanding that an LLM exhibits is different from a human's. The differences in the training data we're each exposed to can explain a great deal of that, and of course there are architectural differences etc. as well.
But the specific quote I responded to was "We reward the appearance of understanding." My point is that's no different from humans: evolution and a child's upbringing rewards the appearance of understanding. The result is imperfect, e.g. people end up with an understanding of the world that in some cases is completely nonsensical (all religions except the one true religion, mine, are false!), but it's sufficient for them to survive.
This demonstrates that "appearance of understanding" is not a meaningful distinction between LLMs and humans. The meaningful distinction is in the training data and the specifics of the reward functions.
Many people seem to try to make a kind of "no true Scotsman" claim about understanding, that somehow LLMs "don't have real understanding". Based on the above quote, it seemed like you might be making that kind of argument. The counter to that argument is simple: if LLMs don't have real understanding, then neither do humans, because broadly speaking, both operate on similar principles: we learn from training data, there are reward (and punishment!) functions that influence what we learn, and the result is a "mind" that demonstrates an understanding of the world.
That is it. We cannot concieve it because we are new to it. Just like we would think of Stackoverflow as intelligent if we are fresh off the jungle and are not aware of how Internet works. Because without know that, we cannot conceive how Stackoverflow can produce answers without it "understanding"
You would not know that if you was a cave man. Nor would you understand how a large number of humans are able to come together to answer questions through the Internet..
To you, you type your questions, and answers appear. That would look like how LLMs appear to us now.
LLMs model the part of human understanding that is captured by the relationship between words in the training corpus. Anyone who thinks non-verbally, the shared understanding of "apple" that comes from having eaten them, understanding what someone is thinking or feeling by their body language - there's a lot of aspects of human understanding that LLMs don't model.
Yeah. I've been using AI a lot in my projects but I still write code and especially the documentation and articles for my website. It's slowing me down a lot compared to just letting the AI rip through projects but I think it's worth it. I see it as distillation of the AI's weights into my own brain.
I always found writing to be an unnecessarily arduous tool for communication, so I never do it until I've overthought what I want to express and approximately how. At that point I flesh out the skeleton of the message, and pile on words and structure and references and other rhetorical tricks until the packaging feels sufficient. Rarely, if ever, have I experienced writing things down altering my perception of the strength or weaknesses of the chosen arguments, or revealing new ones. That is to say, I really don't vibe with the concept of writing being thinking, rather it's a waste of humanity's resources to push it as The Tool.
On the other hand there is so much things that don’t require hard thinking. An example is I got an email the other day from a supplier asking if they can turn off their old email. It didn’t require ”thinking”. All I had to communicate was ”turn off graphql but don’t touch restapi”. Instead of sending off such a short and maybe unclear reply I had AI type up a concise and clear reply with exactly what can be turned off and what must be left on. Could I have done it on myself? Sure but would require more work than just a quick prompt and copy paste.
I don't see how the AI reply could be more concise, clear or exact than what you wrote. If it didn't get the details from you, where did it get them? How do you know it got them right? If it got them from the docs, why not just point to the docs?
Because I didn’t need to type it. If you are a fast typer and love type emails then sure you don’t benefit from it. I don’t love it and I don’t type that fast.
It didnt get it from the docs. There is no public docs. I know it because I wrote the code.
It’s a flattering phrase, not a thoughtful one. It gives the illusion of higher level thinking that’s beyond our actual ability. The phrase also justifies writing without thinking, i.e. most internet comments.
The best thinkers I know mostly use writing for refinement, distillation— as a tool. The worst thinkers I know are owned by writing; they require its fixation & stimulation upfront to compensate for limited attention spans.
This puts into words something I have been thinking about but haven't been able to articulate when people ask me if I'm using AI heavily. Usually when I write a report, I start out with a question and try to find some hints that will give me hypotheses that I can then test. I can't just prompt "Write me a report". The process of coming up with the information in the prompt is best done by writing the friendly report.
> Perhaps I'm just too painstaking a type of person, but I can't grasp much of anything without putting down my thoughts in writing, so I had to actually get my hands working and write these words.
> As I write, I think about things. As I write, I arrange my thoughts. And rewriting and revising takes my thinking down even deeper paths.
- Murakami, "What I Talk About When I Talk About Running"
I'm quick-witted, outspoken, playful, clever, apparently intelligent; Mensa, Colloquy, like that; not quite clever enough for the triple 9 societies. Morning pages readily enough dispel the illusion. Or coding. I scanned six years of morning pages into a PDF and fed it to Claude to distill for me; apparently, I have several thoughts per year. Damn, I'd hoped for more. Asking Claude Code to critique my code is, ah, good coaching. Ouch.
Sometimes. Other times writing is just communicating thought that already happened. Say for instance a weekly status update. Maybe you get something out of that, but frequently you don't get much or anything.
That's an important point. It's the core of your life, thinking, balancing, exploring, improving. I don't believe adjusting something large you didn't do will ever benefit people.
Writing can be thinking. There's a huge presumption that if someone is banging away on a keyboard they're doing work because you can hear and see them doing stuff. But that's the whole plot of The Shining -- Jack despite all his writing wasn't thinking at all.
Likewise using AI can be thoughtless, but it doesn't have to be. I don't see why a valid creation process can't be like this Simpson's meme[1], where you start with a rough object and then cut away and refine until it's done. I don't see it as lacking merit or requiring less thinking compared to starting from a blank canvas and adding more until it's done.
And either way at the end of the day the writing artifact stands on its own. It's either good or bad, taste permitting, and can be evaluated for what it is.
Really? I am having difficulty thinking of any examples of code that doesn't need to be understood. If it isn't understood by someone, then how is it even working?
If you mean like a library you are using, where you aren't even reading the internals or might not even have access to it, OK, but that code is stull understood by its authors, surely?
I have so many projects on my computer where it doesn't matter if I know how it works or not. Probably well over 100. I think closer to 200, depending on how you count.
They're relatively simple, they do the task they need to and then they wait until they're needed again (or not).
In the past I wrote them, then forgot how they worked, until I needed them again, relearned what I did and adapted it.
Now I just don't have to know how exactly they work, I just get an AI to read the documentation anytime I need to reuse the project and I'll query the AI to fill in the details and to make changes and I ask the AI to run the code and debug it.
Perfect use cases for today's AIs. Doesn't even require SOTA, I can run comfortably on a Sonnet 5 or a Qwen 3.8 and it'll do exactly what it needs to do without making too many mistakes.
Uhh... what? You'll instantaneously feel very differently when the service you're responsible for is down at 3:15am and your logs are full of stack traces that end somewhere in that code that "doesn't need to be understood". At that point, you will need to understand it well enough to fix it stat.
Test code for a new bug is a good example. You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course). There are some domains/tests where you can't do that - you need to be sure its failing for the right reason, but often you can do that without understanding every line of the test code. You can extend this to lots of related test infrastructure. If you can watch playwright test the app the way you expect it to, you don't have to understand all the code.
You can also do this for apps that are just tools for your own use. You satisfy yourself that they are working, and you use them because they save your time. You review enough to be sure its implemented the way you think it is - and if it is working, that tells you quite a lot. Sometimes you will be surprised and have some time wasted.
Yes, yes - there are people who will make the wrong choices in some of these cases but that doesn't mean there are never cases where you can do it.
More broadly - anyone who works in a team is already working with code they don't fully understand. I have code I wrote years ago I don't fully understand. I trust its observable properties and its track record.
> You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course).
I'm not following.. When we write regression tests those tests encode invariants we expect to be maintained under source code transformations over time. If I don't understand the test code I've written, how can I know which invariants I've imposed? That's why, broadly speaking, we write test code to be as simple as possible above all else--it's absolutely imperative that these invariants are not only intentional and easy to reason about, but also that when an invariant is violated we can easily discover why. Often, on a team, the person encountering a test failure after making a code change is not the person who originally established the invariant, so it's very important they be able to easily understand it.
I see no possible world in which failing to understand the test code is... possible? Like, if you have indecipherable test code things are really bad in your codebase. Fixing that is P0, because it'll compound rapidly.
You can know the test is likely good, if it reproduces the failure you are fixing. I don't think we're communicating though because I never said the test code was undecipherable.
Having your fly open is a harmless mistake that has little impact on your peers. People may or may not mention it to you but it’s not something they’ll hold against you.
Posting LLM slop under your name is a deliberate act. You decide to damage your message by taking a shortcut.
> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Understanding is the bottleneck; the way they speed things up is by letting me outsource understanding, and get back a summary. The entire advantage to AI is that it lets me skip understanding the problem, and just get a working solution.
Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.
None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.