I think lots of people misunderstand that the "non-deterministic" nature of LLMs come from sampling the token distribution, not from the model itself.

It's also the way the model runs. Setting temperature to zero and picking a fixed seed would ideally result in deterministic output from the sampler, but in parallel execution of matrix arithmetic (eg using a GPU) the order of floating point operations starts to matter, so timing differences can produce different results.

Good point. Though sampling generally happens on the CPU in a linear way. What you describe might influence the raw output logits from a single LLM step, but since the differences are only tiny, a well designed sampler could still make the output deterministic (so same seed = same text output). With a very high temperature these small differences might influence the output though, since the ranking of two tokens might be swapped.

I think the usual misconception is to think that LLM outputs are random "by default". IMHO this apparent randomness is more of a feature rather than a bug, but that may be a different conversation.

I think the point was that Python syntax is simpler than e.g. borrow checking.

Although Python has some seriously PERLesque YOLO moments, like "#"*3 == "###". This is admittedly useful, but funny nonetheless.

I suppose if you accept the innocent-looking "#"+"#"=="##" then your example kind of algebraically follows. Next it's time to define what exp("#") is :)

* does different things depending on the types of the operands, which is Python's strong typing at work, not Perlesque weak typing. Repeating a string is a useful thing to be able to do, and this is a natural choice of syntax for it. The same thing works for lists: [1]*3 == [1, 1, 1].

It does unfortunately mean that sometimes `*` will work (and produce an incorrect result) rather than immediately failing loudly with a clear error message in the context in which it's actually intended to be numerical.

More broadly this is the same argument as whether overloading `+` for strings is a bad idea or not, and the associated points, e.g. the fact that this makes it non-commutative - the same all applies to `*` as well, and to lists as much as strings. At least Python is consistent here.

Although there is one particular aspect that is IMO just bad design: the way `x += y` and `x = y` work. To remind, for lists these are not equivalent to `x = x + y` and `x = x y` - instead of creating a new list, they mutate the existing one in place, so all the references observe the change. This is very surprising and inconsistent with the same operators for numbers, or indeed for strings and tuples.

I was referring to the "creative syntax" and it wasn't meant to be an attack on Python.

We cannot deny that Python has some interesting solutions, such as the std lib namedtuple implementation. It's basically a code template & exec().

I don't think these are necessarily bad, but they're definitely funny.