We also don't know how this approach will pan out and the extent of its final functionality. Some people think it'll just get better, but nothing in the world scales linearly forever, why would this?

We will likely construct a general solution machine one day and help us create resource abundance, and with that lead to a world where we can work on a completely new foundation of a human philosophy and social structure to live in, but there's no guarantee we're on that path with this garbage, and it's a moral failure to just accept that we are and deem ourselves replaced prematurely.

But LLMs dont prevent you from doing this? If it does it probably is an organisational issue.

It’s not hard to swap “LLM” speak and make it sound like arguments made about ever higher-level language.

We are all operating at a couple of layers up the stack and 99% of us would be lost at N-1 let alone N-2/N-3 but we don’t bemoan that. I really don’t see how LLMs are different.

Coding with an LLM could be at most compared with some kind of non-deterministic and non-documented compiler, where you could not predict in any way how it will generate assembly code, so every time when you compile a program you have to also list the generated assembly code and then you must read carefully the assembly code and think hard to determine if it really does what you want.

Of course that does not mean that an LLM is useless for coding or that there are no tasks that can be vibe-coded without worries.

But for anything of long-term importance, like a product delivered to a customer, reviewing carefully a lot of LLM-generated code is not guaranteed to be a simpler task than writing that code yourself. In some cases it may be more productive to partition the new code into code generated by an LLM according to your instructions and code that you write yourself.

Output from compilers is already verified - after all they have an extensive suite of tests.

With LLMs you're getting into the role of someone who keeps validating the compiler over and over, because its output is not deterministic.

They had jobs and reputations and self-images about being "good" at what they did and psychological needs. And further, they had the same incentives as me to not ship bugs when they could avoid it. And I could see, in general, the conversations humans were having, first on usenet and listservs and then on all the little situations on the larger internet.

I didn't trust them blindly, but if a bunch of us all seemed to be okay with the software for the mysql database or the apache webserver. And while I certainly have no idea who those specific folks were I do know how human communities work. When folks found malicious things in, say, a ruby yaml parser or a node package, it was loud.

And like all human scale things, the pacing was much slower so that things like a person's situation in a community and the human trust associated with that trust were things we could evaluate.

I can't evaluate the output of an LLM in the same way. It does not have the same incentives or needs. And it's not deterministic- if you recall comparing checksums between what you downloaded and what the folks putting the software published, then you understand what I mean here when I note that we could trust-but-verify a lot of things.

With the output of an LLM, it actually is important to check over, much of the human-level surface of how we review software is stripped out.

Few people are thrown into a codebase that they understand because they wrote it. Not every developer is issued their own greenfield project and said go have fun.

"on that path with this garbage"

You mean with this the negative things you attribute LLMs? Limited understanding, function and risk, etc.?

Because what LLMs did is freeing the genie from the bottle. It showed what scaling and ML and Compute can do. It doesn't matter much if LLMs will be Frontier for one year or the next 10 years because VC to alternatives like World Models are available now thanks to LLMs.

The compute required and data required for any further progress through ML is also unblocked by LLM.

The economy is solving the memory wall issue today just because of AI.

I had so many issues with our build system because not a lot of people cared about scaling stateful workloads which need a lot of data shared between, now tx to AI y problems got solved.

In the end, my immediate productivity with LLM is a lot lower than before. But I assume that quality is a bit higher, at least in the long run, because issues fixed early saves a lot of time later.

I guess I don't use LLMs like a lot of other people. In fact I continue to write most code myself and I just use LLM to aid me or to generate something that's very similar to existing entity and just requires careful copy&paste&replace. My only fear is that this approach to work is not very popular and I might be asked one day to switch to wipe code, which I don't enjoy at all.

You may caveat what type of software you are talking about, like complex or out-of-the-ordinary, and say that llms are still bad at that. Fine. But LLMs are actively being used for developing thousands of websites all the time - you can see several examples of them on hackernews every week.

Amount of lines is irrelevant when talking about a landing page or a huge number of internal tools that people are building at the moment. At least until a certain point.

Any programmer should be able to hack one up. LLMs are good at _programming_, but I haven't seen evidence they are good at engineering yet. But then again, many people with the title "software engineer" are not great at it either.

Engineering normally requires thinking long term and if you want to use LLMs it needs to be way more structured. For this use-case LLMs are on way more shaky ground.

Or like the internal tool I whipped up in a few days to try out how we can structure some specific data we have internally in our company.

When pouring concrete, I'm not going to use expensive hardwood to frame the concrete base, I'm gonna use stuff that's just strong enough. It's going down the bin anyway.

A lot of internal tools are fine if they are just good enough, they don't really need to be that maintainable if 2 people are gonna use it every once in a while

LLM's are still kinda shit at porting large in-house game projects. They can quickly and easily mangle codebases.

Some regulations would have to be normalized and agreed upon between the countries for trade open up. Can't have different quality standards and ways of evaluating and declaring them.

Are you defending PPWR? Not in spirit but actual implementation?

It will never get so-far as the judge because the EU has already admitted the law cant be followed and arent going to be taking people to court over it.

Regardless of the intent, it's an abject failure.

I also think GDPR is too strict for small businesses and non-profits and not strict enough to deal with big businesses that trade data.

Its would have been trivial for a big business, but it was not for small businesses.

Maybe just calibrated poorly. Small enough and IMO the only important rule is no sharing with others. That should always be strictly enforced. But why worry about how some small timer handles my personal data or if I can make various requests of him? The stakes are too small to matter.

Canadian provinces have a harder time trading with each other than countries in the EU, although thanks to the sales tax and package transit changes the EU is becoming competitive in this area.

Mostly just wouldn't really work because Ottawa would have to somehow represent the provinces to the EU and the provinces and Ottawa don't work together in a way that would be compatible, I don't think anyone in Canada would want to change how Ottawa works with them. We may also end up in the same position again as we have with the Americans, nobody but us likes our supply management system[1]. GDPR is also something I think Canadians would struggle to get behind, our PIPEDA framework is quite good and I think most folks approve of it (except Quebec, who have their own framework). Taking the Euro would be a problem, etc etc, I think I could probably build quite a big list. btw, I respect the EU regulation, and that Europeans quite like it, I was just saying I don't think it would be practical for us to re-work the whole country to accommodate them, that's all.

[1]https://en.wikipedia.org/wiki/Dairy_and_poultry_supply_manag...

You'll get 100% of the way there, because, lacking any other training, it will only ever be able to reproduce this text verbatim.

Hence, 100%, hence mission accomplished.

Training tries to produce something that scores highly in training evaluations. With one data point, the evaluation is solely how closely the model output resembles the single input text.

Let's say you do that, and the training text is 58,100 tokens long. Let's say you ask the model to produce 58,101 tokens. Will it "reproduce [the] text verbatim"? No, it can't, because of the dissimilar requested length. Something "new" will come out.

It's also entirely possible that no matter how long you train, the model never converges on generating exactly the same output as its training data - you could end up with an average loss value of 0.001 instead of 0.0. It's not a perfectly deterministic process.

You're correct in principle, but in reality even with limited training data real-world models produce something that isn't exactly their training, especially when sampled stochastically. They're biased toward their training data, not forced to it.

... It's never been tried. Look, I was actually joking when I initially replied, but now that I think about it, it's never been tried!

So now I am left wondering exactly what will happen if I train a new model from scratch, on only a single short corpus of 50k words.

Not fine-tuning an existing model, but literally creating a new model by feeding a blank (or random) matrix the same poem 100k times.

Computers didn't "figure out the maths". People did the math, the computers did the calculations.

Human knowledge is nowhere complete, it's silly to think so, and by extension it's silly to think the LLM's have anything near a complete knowledge set.

They don't even have access to all of current human knowledge, much less so future knowledge. A lot of text and information is locked away from the public internet and the internet altogether.

Not all types of useful ideas for applications have been invented, that's ridiculous, there are plenty of ideas that people haven't had yet. There are however a massive amount of shitty copies of the same shitty ideas.