Everyone's asking tech details and "how", but I wonder about the "why". Do we want LLMs to always write for us, or whisper in our ear what to say? By design LLMs tend toward the most commonplace, mainstream ideas and ways of saying things. They're not much for originality or human idiosyncracy. Are we engineering a bland world full of pablum?
Because at work I’m typing the same bland things all the time in documents and communications. I appreciate stuff like the predictive word stuff in Google Docs. It’s helpful because business language is expected to be normalized and boring.
On the other side of that token, the average language abilities of the average American office worker are pretty low so I’m assuming they view this as an enhanced AutoCorrect and they appreciate it because it makes them look less dumb.
I agree with your point though. And to answer your last question, unfortunately I think the answer is “yes”.
> Because at work I’m typing the same bland things all the time in documents and communications.
This problem can be solved without text prediction by building a personal knowledge base with hyperlinking[0] and backlinking[1] for discovery, and transclusion[2] for automating writing the same bland things. How I org in 2023 by Nick Anderson[3] goes over a great workflow for this. The advantage of this is that all of the words that you share are actually words that you wrote, instead of sharing words that Google suggested you share.
I use macros for frequently-reused messaging but I meant business communication itself. It’s essentially a different dialect of English meant to be “inclusive”, unambiguous, and not prone to misinterpretation.
Anyway, I like and use the feature daily at work. So do lots and lots of other people.
On the other side of that token, the average language abilities of the average American office worker are pretty low so I’m assuming they view this as an enhanced AutoCorrect and they appreciate it because it makes them look less dumb.
I would also assume this to be largely true, mainly because of how language in media has gone from being formal and informative to casual and less expressive.
I agree there has been a style shift, I don't think writing has gotten any less informative. I personally don't like formal writing, but either way it has no bearing on whether or not ideas can be gotten across effectively. Substance over style!
I might say it a bit differently. The amazon writing style is to avoid flowery prose, weasel words (maybe, perhaps, etc), give precise dates, use data, etc. I'd love a model that I could hand to engineers to help them write in this style.
It's not LLM's writing for us, it's just autocomplete.
If the suggestion doesn't match what you were already planning on saying, you just ignore it.
The human desire to be original and authentic is always going to be stronger.
(It's much less effort to ignore it and keep typing your original thought, than it is to think about it, compare with what you were going to say, decide its version is better, and then accept it.)
1) there is a sort order to them. And we don't know how the 'recommendations' work. What is a recommendation? Why is Google recommending something over something else?
2) it bogs down creativity. You end up not thinking and accepting the suggestion as 'good' enough.
My job requires me to correspond with dozens of people over email, Teams, and Slack every day. We're all trying to get work done and need our communications to be as succinct as possible. Sure, occasionally I might dress it up to add some humor, but that's ~1% of cases. An AI, with access to my entire corpus of work-related communication, could likely very easily predict most of my communications, since they fall into a small set of categories.
"When can I expect to get $workproduct?"
"Here's when I can commit to getting you $workproduct"
"What's the estimated date for $milestone?"
"Here's the project plan for $initiative"
"I can't make this meeting, can you be sure to record it?"
I welcome any tool that can predict what I want to type and does it for me. I'm not sure if it's my imagination or not but Outlook and Teams seem to have gotten better in this regard. I'll take more of that.
Most writing does not need creativity. It is wrote communication. If you want to put some creativity into something or write something where you want to write with some personality, then turn off the predictions or use it just for spell check.
> The human desire to be original and authentic is always going to be stronger.
Authenticity didn't start to become a thing people cared about until about the early 1990s, and it didn't blow up and take over the mainstream culture until the 2000s and 2010s.
Prior to the 1990s, there was a lot of interest in professionalism. People spoke and wrote in overly formal, jargony ways that they perceived as being a marker for competence in some specialized professional domain. Being too honest/authentic would have been seen as unsophisticated and lower class.
Just pointing out that what people strive to emulate can change over time. Once LLM writing becomes commonplace, it will probably become trendy to write in the exact opposite way that an LLM does!
> Authenticity didn't start to become a thing people cared about until about the early 1990s…
In your lifetime, but…
Another common theme in the philosophy of both Dostoevsky and Descartes is the idea of authenticity. Dostoevsky believed that people must live authentically, in accordance with their true selves, and that this is essential for their happiness and well-being. Descartes, on the other hand, argued that the path to knowledge and certainty begins with the rejection of all previous beliefs and the adoption of a completely new and authentic perspective.¹
And of course, the U.S. counterculture movement of the 1960s was deeply preoccupied with authenticity:
You note three themes underlying the American hippie experience: authenticity, individualism and community. Why did these concepts stand out, and what did they mean in the context of the hippie/counterculture movement? / W.R.: Although hippies often disagreed about beliefs and practices, they shared a desire to be authentic. Members of the counterculture condemned mainstream society for being conformist, rule-driven and uptight. Authenticity meant “doing your own thing.” Because freaks distrusted both society and government, individual decisions were applauded as the most authentic.²
Well it's not so much about deliberately / affectedly being original and weird (which is annoying), but just leaving some space for natural idiosyncratic ways of writing.
Yes, but sometimes that's what's called for. I think I may have found my answer to where LLMs might be useful for short communications: softening something that might be interpreted as "curt" or even "rude" while not really changing the message.
As someone with Dyslexia, who struggles a lot with written communication, and frequently finds myself fighting with spellcheckers, trying to get them to provide the correct correction. I’m very excited for these types of autocomplete systems. They’re like spellcheckers, except they also use context to produce better results, and frequently remove the need to play “guess the right misspelling” to get a normal spellcheck to provide good suggestions.
I’ll accept the risk of blandness, if it means that written communication finally becomes “easy” for me to participate in.
Maybe not always (I assume an annoyed person could turn this feature off), but I think the general trend is, yes, especially for boring rote writing we have to get done for work or school that doesn’t require a tremendous amount of creativity.
I really look forward to using GPT to help me throw together RFCs, documentation, announcement letters, daily standup write ups and other artifacts like that that prevent me from getting actual work done.
One approach that I've been using with a local LLM, mostly brainstorming for my friends and my dungeons and dragons sessions, is to set up a prompt to be completed with some certain detail of storyline, character background, etc., Then running that prompt maybe 50 or 100 times and keeping informal statistics on the different results that come out. In such an application, it can be used as much for inspiration of how not to sound bland, commonplace, or derivative. You can pick the one in 100 oddity that strikes you as interesting, or simply make sure not to use any of the outcomes that the LLM came up with.
I find they're useful as a tool for achieving a particular structure or tone that isn't "my voice," including populating boilerplate from bullet points. Sometimes I'll go back and revisit everything they've produced, but they let me put something tolerable in place early. This lets me focus on the meaty parts of the text sooner than I'd otherwise find myself capable of.
I personally don't find them useful for quick / short / informal communication like email, or at least not yet.
"Bing Chat's" implementation already allows you to select more creative generation of text. It's just a radio button option. There are also different technical solutions for the LLM to select which word to generate that allow either for more interesting, or more predictable, words.
This isn't to say a human element doesn't have a ton to offer! Just to say that we aren't necessarily engineering a bland world of pablum, either.
That's just temperature, which evens out the random probability a little of the N most probable next words. It's still vastly favoring the N most common ones based on the training corpus, and will have a hard time producing uncommon ones.
E.g. try asking an LLM to name a real, non-famous person. The internet and it's training corpus is full of regular people, but you won't have much luck - they're statistically too uncommon to remember.
I was assuming those Bing Chat settings are temperature-based, those types of "creative/precise" controls usually are - but perhaps there's more to it.
Early versions of GPT would tell me about myself, but July of this year it was saying that it could not comment on a private individual. I'm not at all famous, but there's plenty of writing by and about me in the common training datasets.
I would be skeptical of the results of that experiment, just because I assume the minders of the big LLMs have attempted to make it deeply uncomfortable with discussing anything that might be "personal." For a fun time, ask ChatGPT who was executed (as in capital punishment) in the US in a certain long-past year. It responds with a bunch of rubbish about privacy and about how an LLM can't be completely sure about stuff so it wouldn't be ok to speculate for fear of tarnishing someone's reputation, as though the person executed 15 years ago is going to sue OpenAI for sharing their name and what crime they were publicly convicted of and killed for.
> Are we engineering a bland world full of pablum?
Like any other technological advancement, we're freeing ourselves from the burden of wasting mental energy manually doing those commonplace unoriginal tasks which can be easily automated, so there's more bandwidth left for human beings to focus specifically on the things which can't be
I disabled Gmail Smart Compose for this reason; I felt like it was putting words into my mouth suggesting entire sentences for the email.
I'm much more open to using transformers as a better auto-correct, where it's one word at a time and uses the first letter as a filter. Especially on a tiny phone keyboard on the go.
If I’m writing an email to my coworkers, what I really care about is the information I’m getting across and not really the presentation of it. If LLM autocomplete can speed up writing a totally functional email then how bland it is really isn’t at the top of my mind.
