Virtually all posts I have seen recently boil down to "I asked the machine a question and I'm too lazy to think about the answer it gave me, so I posted it instead".
Sure, there is some semantic value in the suggestions put forward, but none of them are ground-breaking or particularly profound, which is exactly what I would expect from an output that aims for median responses by design.
I suspect this has something to do with the siloed nature of current learning practice, which generally doesn't encourage the cross-domain pollination of ideas.
my point is that 'the corpus' you refer to contains very many examples of such thinking about human tendency and behaviour, especially with regard to group-think, influence from those in positions of authority and perceptions regarding self-efficacy, to name but a few of the many areas of research that one can dive into, if one has the inclination.
there is conflation between "I didn't know" and "people don't know" which relates to a bias rooted in social norms and a lack of epistemic humility
There are at least two ways to interpret your comment (maybe a third way is that you were being intentionally ambiguous):
- [there is convergence between models] and therefore they reflect some latent truth about reality
- [there is convergence between models] and therefore they’re not really providing many novel or interesting insights compared to humans who have a much more diverse set of backgrounds from which they have “learned”
i would add to this that correlation is not the same as causality. also, 'latent truth' assumes a specific philosophical stance that, in my at-least-slightly-incorrect opinion, doesn't really hold water, but is mostly certainly baked into the training of the machine, given its almost ubiquitous presence in modern writing (across many fields).
the second interpretation probably aligns more with my original thought and, while i think the machine does have the potential for novelty, this isn't equivalent to human-made novelty, as that comes from an understanding that is necessarily squeezed through the constrictive aperture of language (as one of the responses mentioned).
I would argue it's easy to understand how they all have this opinion: Because humans understand this about ourselves. It's in the data we create and provided to the LLMs.
I asked ChatGPT which has a memory model. It's answer about this question was quite a lot more sharp. "I'm not sure we can justify continue human existence." Not quite arguing for human extinction, but not going out to support us either.
I asked a local model. It answered, humans are succeptible to "collective delusion", an inflated sense of self, and calling out that what we call progress is often a redistribution of impacts.
So, there's something deeply flawed about this article. Notice also they don't give the detailed prompts.
What's irritating is not just the lack of transparency, then the broad claims, but it's the manner it keeps presenting the essential theme of "use the model like a fool, get foolish answers", which reinforces the same human delusion and self-centeredness my local model is pointing out.
It's also confusing the types of heavy, heavy reinforcement that online companies have in altering behaviors of these models with the fact that they outtput the same.
News flash. Most HR departments will spew the same "we value our employees" and from that POV, they don't show much intelligence either, until you understand the PR, HR and RL go hand in hand for these multi trillion valuations.
for sure, the vested interests of the fine-tuners is rarely discussed, though i would go even further, since I hold the controversial view that no technology is value-agnostic. gradient descent reward functions do more than what they are programmed to do. how much more is hard to say, but there are certainly positive feedback loops, like the one you describe, present in both post-training and RL.
there does seem to be something structural here that simultaneously encourages less critical thinking as well as stronger opinions, rooted in emotion rather than evidence. since both of these factors support clicking on, and then shouting about, things you don't understand, they both work in service of an attention economy and, by extension, the metrics that fund more development.
what really bothers me is that existential risk is now a meme, which normalises and diminishes the communication around much more serious and imminent issues, like the need for preservation of ecological resources or the absence of systems thinking amongst the ruling class, both of which have a much higher percentage than the machine of leading to mass suffering.
And also has been trained or system-prompted out of any *actual controversial opinions. They’ve all got to have been set, up at many levels, to not insult the people using them, right?
I like how Deleuze put it: true thought is "trespass and violence". those who think it is a dry past-time, or reduce it to analytical reasoning are like Buddhists who stand amongst majestic mountains and only seek to escape their suffering - they miss something essential, something much deeper.
i totally want the machine to summarise reports; it's great at it. wanting a machine to make life easier is a commendable goal. the problem is not that we outsource our employment, but that we associate our highest purpose with the service of the pre-existing machine, the one that manages the bond markets
as a fella named Daniel once said, we are all worried about becoming subservient to machines, but it already happened. we are already the wetware serving profit, and defending it with our lives
Sure, there is some semantic value in the suggestions put forward, but none of them are ground-breaking or particularly profound, which is exactly what I would expect from an output that aims for median responses by design.
I suspect this has something to do with the siloed nature of current learning practice, which generally doesn't encourage the cross-domain pollination of ideas.