Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It doesn’t matter to me how good the LLM is at writing Go if the compiler can’t stop it from accidentally leaving another part of the software with invalid state as a result of a change the LLM is making.

What am I talking about? Nil and partially constructed structs are impossible to prevent the creation of in Go.

Sure, if you’ve got a small program with limited scope, that’s probably fine if you look through squinted eyes. But the teams I work with are working on sprawling, evolving software where the compiler saying “hey, that’s not a valid Widget” would be extremely useful and save much heartache.

An LLM does a good job of “checking” for other uses and “checking” if everything is going to work correctly, but - supposedly we’ve committed the concept to code so that the compiler can actually verify it - and Go intentionally permits invalid states of structs. This makes Go a fundamentally problematic language choice for the kind of software I work with teams on, LLM or not.



Fair criticism, it's my main gripe with Go as well - it's not strict enough when it comes to e.g. nil, enums, and type safety. Annotations are supported but they're just strings. Projects require additional tooling / linters to check for things like unchecked errors and many more "gotchas" that I think could (should?) be part of the compiler or standard tools. Trivial example, Go's compiler will error when you have an unused variable, but won't if you reuse and overwrite an error variable a dozen times and only handle it once.

But on the other hand, I suppose it makes it a bit more pragmatic - less checks makes for a faster compiler, and fast compilation was/is very high up in the language's requirements and motivation. If you want / need more strictness in your language, there's Rust, Java, C#, etc.


i never run into these issues in >100k loc of productionized llm generated elixir (nil safety, type issues). i wonder, is there something architecturally in go that makes this a particular problem?


Up until quite recently, golang lacked generics, so `interface{}` was passed everywhere. It does not have typesafe enums (you can pass integers and it will compile). It has no way of declaring immutable structures. It does not have pattern matching.


> It does not have typesafe enums (you can pass integers and it will compile).

sigh. Sure, if you literally pass an untyped, hard-coded integer, like so:

    foo(100)
then, and only then will it compile. However, this won't:

    a := 100
    foo(a)
How horrific.


How about Swift and Erlang? Why are these always left out of comparisons?


Haven’t you heard? Swift is the first-ever corporate programming language, and we on HN can’t take its community efforts seriously to the extent of considering its potential value in general-purpose software.

/s


I see it as a great language being driven (fw atm) by same buy who did Rust, no? And my take really is that the fact they were no good corpo created languages (GO is not one or what?), then it is totally fine to some big corpo like the much-disloved-yet-selling-shitloads-of-phones Apple to actually do something which benefits the world and does not incur costs ojn it.


Good point.

It's sometimes challenging to get a Rust program to compile... but if you do, it's probably going to work.


The authors make some good points: compile time and test speed matter, platforms matter. But they really dodge the whole “guardrails matter” thing. And guardrails are going to win long term.

As for readability, the fact that AI-written Go closely resembles human-written Go is not necessarily a point in Go’s favour.


> "guardrails matter" This thing has been solved in the 70s. Basically, have a powerful type system, and a compiler that beats you into submission when you try to stray from the straight and narrow. Languages like (S/OCa)ML, Haskell and Rust will bring this to you. As a consequence, you fight the compiler, and once it submits, you have a good chance it's going to work. The compiler is also the ultimate refactoring tool here. Change the concept? Just change the type in the code and follow through by fixing the error messages the compiler spits out.


Of those, the weakest points for Haskell and the MLs is the platform and documentation. However, I don’t know if we can call this solved just yet. We may need to invent whole new types of guardrail.


> As for readability, the fact that AI-written Go closely resembles human-written Go is not necessarily a point in Go’s favour.

There's just not many ways of writing Go. It's a very dull language. It was designed to be dull and easily understandable.


but there are a whole lot of ways you can mess up a dull language, like shitty variable names, bad code organization, etc.

if an llm has learned dumb things from dumb users it could disproportionately cause provlems versus other languages, just by being "in a sloppy mood" when writing go.


Not panicking doesn’t mean working as intended, the biggest issue with LLM generated code is that it will do something that’s subtly wrong not that it will crash. It anything LLMs are too careful with Golang code and litter useless nil checks everywhere e.g. for function calls with pointer receivers. That whole fear of panics is totally overblown.


Sure, rust has better memory safety than most languages, but it also has a strong enough type system that many other kinds of programming errors won't pass compilation.


So the language knows what you want to program and if it’s functionally correct? I don’t think so.


And by that point a program written in go has been deployed and making money for months. These are two wildly different languages, people should stop comparing them as if they're targetting the same niche. Checks and protections that rust has aren't necessary in most cases, but increase development and maintenance time.


Do you have some evidence of that? I use both daily and my experience has been the opposite, if anything. Once I was as proficient at Rust as I was at Go, the "increased development and maintenance time" disappeared completely.


> Once I was as proficient at Rust as I was at Go

That's a big time investment for many people though. The one they—or rather their employers—cannot really afford.


Rust's disadvantage in adding time is mostly a one-time initial investment, in my experience.

After you get comfortable and learn the important idioms, you go through the development loop quite quickly.

Compilation/linking speed can still suck though.


I've found that keeping very healthy test coverage solves issues like this. LLMs are very good at validating their own implementations with TDD.


With unit tests you gotta be careful though, oftentimes LLMs skip implementations with mockups that just say "not implemented yet" or similar and then the unit tests become pointless because they start to only test internal structures for being set / not default values.

For me it helped a lot to try to make containerized end-to-end tests and a custom TestMain for this, where I am using podman to run the integration tests. This way the end-to-end tests are forced to be on network level, and you can test protocol and API quirks much easier with LLMs.

Also, never forget to write a bootstrapping docs/ folder so that you don't have to re-explain these things all the time.


+1, I like to test in a similar way. For example if I'm making a CLI, the test will spawn the CLI for each test-case, instead of invoking the code directly. This provides a more realistic flow, and makes it clear which user journeys one is supporting.

Faking responses I often do with environment variables, like:

  ts := httptest.NewServer(...)
  cmd := exec.Command(...)
  cmd.Env = append(os.Environ(), fmt.Sprintf("MYCLI_PROD_ADDR=%s", ts.URL))
Of course, it's better still to go down this turtle stack (e.g. by spawning a local instance of your backend server instead of some faked handlers), but that adds more cost. I find the trade-off OK here.


I actually had to use a similar hack there due to the limitation that go test compilates cannot spawn themselves where I needed to have an environment variable with the actual binary prebuilt before the tests run. Took me a while to understand that TestMain doesn't cover that use case...


On Linux, "self" is /proc/self/exe. In general, I've seen people use `os.Args[0]`. But on checking again, I see there's even `os.Executable()` (https://pkg.go.dev/os#Executable) for this purpose.

I'm quite sure go test compilates can spawn themselves, I'm doing it on many platforms.

But, the test setup I was referring to was explicitly not that: the tests are spawning the main binary, not themselves. Using bazel+runfiles this is pretty easy to do. With the pure Go build tool, I'm not sure what approach I'd use to "guarantee" that I get a binary build for the same environment as the test.


i think they validate less with TDD; if you tell them a bare bones spec to validate they tend to write just that. if you write the test after, then their testing is causally conditioned on what was written. if you are going to write tests, it seems like teat first is way better than test last.


>> Nil and partially constructed structs

C# has "nullable reference types" and "required" keyword for that


So your problem is the default/zero values of properties?

In Go the convention is kind of to have a constructor pattern with a NewStruct(...) *Struct method that initializes all properties.

Also can't you build your own validator for that with the reflect package in the Add() method of your UI graph to prevent this sorta thing?


> the convention is kind of to have a constructor pattern with a NewStruct(...) *Struct method that initializes all properties

But that doesn't stop you from declaring a var s Struct, and never initializing it, or making a NewStruct {}.

> can't you build your own validator for that with the reflect package in the Add() method of your UI graph

Besides the fact that that would almost certainly significantly hurt performance, how would you be able to differentiate between unitialized data and data that was intentionally set to the zero value?


> But that doesn't stop you from declaring a var s Struct, and never initializing it, or making a NewStruct {}.

Static analysis tools can catch this, no?


Static analysis does not help if the type comes from a library and is _meant_ to be initialized using a literal.

And then upstream adds new fields where the zero value is different from the previous behavior, causing users to silently drift away from the intended behavior. I had this happen to me with a type from std, and had to add a specific test to guard against it with future std upgrades: https://github.com/sapcc/go-bits/pull/309/changes#diff-f5721...


That's a dependency making a breaking change, right?


The Go std library, as well as practically all go library code, is full of things that don't fully initialize all properties and things that nil-pointer-panic if you hold them wrong, so no, no matter what you do you have to deal with this wart of Go.

The go type-system is simply incapable of enforcing nil-safety without being no longer able to compile the go stdlib nor most code in the wild, so it's a quite valid criticism of the go type-system and language, and your comment doesn't hit on a valid solution.


You're painting a picture where people writing Go are constantly drowning in nil pointer panics. This is not reality. You hit them occasionally and they're trivial to understand and fix.


nil pointer panics aren't nearly as bad as values getting zero initialized, then used in places that assume they were initialized, and getting subtle bugs because the state is inconsistent.


I always wonder how those types of mistakes make it through your test suite. You'd need some kind of non-deterministic path to reaching the unintended zero value — but it would need to be a non-deterministic path that you wouldn't make deterministic during testing. I think we'd be curious to see what that code looks like.


Again, it happens, but the impact is wildly overstated. I get so confused with people saying "once it compiles, it probably works" (regardless of language). The problems I struggle with need invariants that can't be expressed by any modern language features.


Very good point, thanks for confirming that! What is the reasoning behind the Go design choices you described?


With test, which LLM are good at writing, you can reduce this tremendously. Even Rust won't protect you in this case, remember the cloudflare outage. At the end it is not the language, it is you intrinsic ability to architecture well your software from there any LLM can do the work.


i am using nilaway from fb it's great for those


It's from uber, I believe.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: