Particularly this subthread which was my moment of enlightenment https://news.ycombinator.com/item?id=44507991. A truly restful API isn't for a program to consume, it's for _you_, the user to consume through your user agent. Clicking on links is a hypermedia driven, restful api.
I think I saw a comment somewhere by the htmx dev, that Fielding was just giving a name to how the web already was at the time, hyperlinks rendered by a browser that you can navigate in a stateless way to get things done without the user agent being coupled to the sites themselves. It makes sense when you view it as backward-looking academic analysis rather than practical prescriptive advice.
But once you have that defined you can talk about whether things are more or less restful. That's separate from whether something being "more restful" is even good or desired though.
Yeah, it's Fielding's Ph.D. thesis where he's expounding on his work on HTTP 1.0 and 1.1. He's literally just explaining how the web works. It's pretty crazy how much people have misinterpreted it.
I guess the connection is that if you assume neural networks are in fact filled with meaningful features instead of superposition blackbox stuff, then what the LLM is doing here isn't quite magic but rather methodical interpretability. And if you can isolate the neuron that is e.g. responsible for steering the paddle then you can probably tweak it to steer more/less aggressively.
It is still very surprising though, considering it apparently displays an "intuitive" understanding of the weights, that make whatever Goodfire did look like primitive monkeys poking at it by comparison.
> It is still very surprising though, considering it apparently displays an "intuitive" understanding of the weights, that make whatever Goodfire did look like primitive monkeys poking at it by comparison.
FYI, Nick/Goodfire used the models extensively in their work, so it's very possible that there were some similarities in their work. I'm really curious to see his response to this.
I think this work does vindicate their hypothesis that alignment/superalignment might actually turn out to be "easy", in that you can just train LLMs do to the interpretability work. Maybe this is in fact what OpenAI did, or if it's purely "emergent" then it seems they have great priors that can be made more powerful with some RL
Oh I remember this from the classic SO post, but it was just presented as a funny trick. I never realized that it solves the issue mentioned in the article and there is a legitimate use-case for post-decrement in the comparison itself.
That's an odd way of phrasing it, since the $1 is still worth $1, and a naive reading would make you think it's "worth more" when the buying power has actually reduced.
Wouldn't it be clearer to say $1 today only buys you 64% of what you could have bought in 2008.
The single-neuron steering stuff has been going around twitter lately, do you happen to know who was the first one to do it?
I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
There was a paper that popped up a month or two ago (I think published earlier though) about being able to inject "thoughts" into the neurons. E.g. Ask it what I should eat, and steer it towards "bread".
> I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?
By my reading of the TL;DR of "NNs are already clean" (given the full thing is a 65-minute read, I've not been through it yet), the result can be rephrased as "it made an ontology different to ours, so we were confused". (Example: we might expect a language model to identify nouns and there to be a noun-neuron, but the neurons actually trigger at punctuation based on the emotional cue from the preceding clause).
Regardless, a one-neuron evil switch is quite clean, possibly (I'm not sure) clean enough for the older processes to find it.
The conundrum: (1) this was only discoverable because of open weight models, and research labs (and governments, if they regulated faster than the glaciers are melting) could be informed by this; (2) yet because it is known, it now means that it's easy for mid-skilled people to download open models, flip the switch to make the model evil, and then setting them lose to do evil things.
Security by obscurity vs. fully automated insecurity. Clichés need not apply.
This was not a neuron. This was from a trained sparse auto encoder.
In a few words, they found a way to map residual stream activations (so, a vector) into a few interpretable sparse concepts. They also had the converse maps, that translate concepts into activations.
Golden Gate claude was obtained by artificially increasing the activations in the direction of the 'Golden Gate' concept. It was not neuron surgery.
You don't even need SAE for residual stream based steering, you can do so much more naively by just taking the average difference in activations of prompted vs. unprompted versions, then PCA to get a "control vector" that you can use to elicit the prompted behavior
>used to use the same certificate store as the rest of the OS
It was technically curl that dropped SecureTransport but I guess because no one at Apple was willing to maintain it and it apparently is a frozen API that doesn't support newer than TLS 1.2 (!). For a while their inbuilt curl could use both but preferred LibreSSL anyway.
Particularly this subthread which was my moment of enlightenment https://news.ycombinator.com/item?id=44507991. A truly restful API isn't for a program to consume, it's for _you_, the user to consume through your user agent. Clicking on links is a hypermedia driven, restful api.
I think I saw a comment somewhere by the htmx dev, that Fielding was just giving a name to how the web already was at the time, hyperlinks rendered by a browser that you can navigate in a stateless way to get things done without the user agent being coupled to the sites themselves. It makes sense when you view it as backward-looking academic analysis rather than practical prescriptive advice.
But once you have that defined you can talk about whether things are more or less restful. That's separate from whether something being "more restful" is even good or desired though.
reply