however, you can test the model, by asking it things. What are you concerned is hiding inside of that big block of numbers? Order 66?

Yes. The fact is we have no idea what's in these models. People are afraid of mkultra-ed humans that turn into killing machines with a given trigger word, but here we've built machines that are exactly that, not even intentionally! Yes, guard rails have gotten better and reports of LLMs talking people into killing themselves have gone down, but sorry if my trust has been broken!

this also applies to proprietary black box software. You can still test it, interact with it

the difference is, that a model is literally just numbers.

proprietary software is data and code, and while it's true that code is data, the difference is that some data is intended to be for control flow, while in a model it is all the same stuff.

You can point to any single part of an LLM's model and say "this here is a weight" but if you point to any single part of a binary file you will have no idea what you're looking at.

And yes, you could decompile the binary, but that still doesn't give you the entire picture, and all proprietary software does the shady shit on servers these days anyway so you're not even going to find anything interesting.

it isn't really just numbers though.

what if i want to change out the middle part of the training, and then still run the rest as before? or if i want to cull a bunch of the initial training set?

a binary is also just numbers, but we know there's other parts to it

a low stress tolerance and a highly sensitized stress reaction can often look the same. If someone is hurt repeatedly in the same way, they will react when a stimuli that reminds them of it comes near. Similarly, people who are constantly stressed out often develop things like "learned helplessness", where they are incapable of solving their problems because they've failed, been rejected, or told no so many times that they stop trying.

Wow! The future is parallel.

Increasingly, the "what do we have the most of?" questions will return "RAM" less and less often.

Nuclear-powered hyperscale data centers in your neighborhood disagree. Not saying it's a good thing, but like it or not, if you ask "what generally drives how we do things", that's probably it.

make it multiplayer, I dare ya >:)

(if you need this kind of motivation)

Yup totally support that.

This is the most brilliant thing I've read all year. That note at the bottom... Amazing.

if it's the lowest level available, then it's as close to "native" as we can get, so therefore it has to qualify, if we want to consider anything at all to be running "natively"

Isn’t that moving the goalposts? If an API isn’t exposed for native code then maybe we should just accept that we can’t write native code anymore instead of stretching the definition.

Um... Are you referring to WSL? Wouldn't that be the linux kernel running under windows?

WSL 1.0 was doing something like that. Doing syscall translation in real time. Eventually edge cases forced them to abandon that architecture and now it's just a VM.

Was it edge cases? I thought the main driver for WSL2 was better filesystem performance.

> There is the Von Neumann method of generating an unbiased coin from a biased coin. Of throwing it twice, and checking if you got HT or TH. And completely discarding all HH or TT results. It doesn't matter if the coin you are using is 20% or 80%, the result will be a true 50/50.

This blew my mind. Thank you!

I had to think about it a bit, so for anyone scratching their head right now trying to figure it out, consider it this way:

what matters is the ordering, of heads-then-tails, or tails-then-heads.

It doesn't matter that it's biased one way or the other, if you keep flipping pairs until you get a result with two different values, it's a 50/50 chance whether the less-likely result comes first, or second.

You might only have a 20% chance of any particular pair having a tails (for example), but in the cases where you do have a tails, it's a 50/50 chance that it comes first or second.

And for people who like equations, here is my attempt at explaining it.

Assume each flip is independent and the bias remains same in each flip.

Let

  P(H) = p,
  P(T) = 1 - p.

Then

  P(HH) = p^2,
  P(HT) = p(1 - p),
  P(TH) = (1 - p)p,
  P(TT) = (1 - p)^2.

Therefore

  P(HT or TH) = 2p(1 - p).

Now calculate

  P(HT | HT or TH) = p(1 - p) / (2p(1 - p)) = 1/2,
  P(TH | HT or TH) = (1 - p)p / (2p(1 - p)) = 1/2.

You don't need conditional probability here, as the flips are independent.

It's just p(H)p(T).

And p(H)p(T) = p(T)p(H), thus 2*p(H)p(T) = 2p(1-p).

Independence tells us how to compute the probability of a sequence like HT or TH:

  P(HT) = P(H)P(T) = p(1 - p)

But the question I am addressing is not just "what is the probability of HT?" It is "given that the two flips are different, what is the probability that the order was HT rather than TH?"

That is a conditional probability:

  P(HT | HT or TH)

That wasn’t what he was trying to prove, but the proof could be done without conditionals like this:

If: p(H)p(T) = p(T)p(H)

And: p(H)p(T) + p(T)p(H) = 1

Then: p(H)p(T) = p(T)p(H) = 0.5

Thats how i noodled thru it internally

Afaics it's just basic commutativity – p(H)p(T) = p(T)p(H) – since instances are independent.

Same, of course, holds for flipping it multiple times. But there you get more than Head or Tail (binomnk(n, k)).

Thanks for your explanation. I did not get it in the first read, and was too lazy to think, until saw your comment.

Just want to point out, that one is actually doing the experiment with a biased coin, then one must ignore all pairs.

e.g in case a coin which is heavily biased, say .9 H and .1 T. One should start with ignoring all the HH pairs, and start only at odd index. Lest, one picks a value like HHHHT (in the case the 2nd HH pair was not skipped, instead they greedily picked up the first HT, which will make the experiment HT biased).

I was doubting this for a minute as I wondered with a significantly biased coin towards the head side would you be more likely to get HT. With probability problems like Monty Hall I like to think about extreme cases like say it's 99 heads to every 1 tails. You'd expect HT 0.99% of the time. Ditto TH.

You can’t flip coins until you get the first different outcome… You have to flip twice each time, until you get a pair with different outcomes.

Oh wow that’s really amazing. What’s the source - I love Von Neumann.

Not very random if it's only TH or HT. Trivial to brute force with no more than two tries!