This is the first time in my HN membership where I was excited to read about the dialectic, only to be disappointed upon finding out the article is about Rust.
PBT is for sure the future - which is apparently now? 10 years ago when I was talking about QuickCheck [0] all the JS and Ruby programmers in my city just looked at me like I had two heads.
TBF PBT has been the present in Python for a while now.
10 years ago might have been a little early (Hypothesis 1.0 came out 11 years ago this coming Thursday), but we had pretty wide adoption by year two and it's only been growing. It's just that the other languages have all lagged behind.
It's by no means universally adopted, but it's not a weird rare thing that nobody has heard of.
It will only take one agent-led compromise to get some Claude-authored underhanded C into llvm or linux or something and then we will all finally need to reflect on trusting trust at last and forevermore.
Reflect in what way? The primary focus of that talk is that it’s possible to infect the binary of a compiler in a way that source analysis won’t reveal and the binary self replicates the vulnerability into other binaries it generates. Thankfully that particular problem was “solved” a while back [1] even if not yet implemented widely.
However, the broader idea of supply chain attacks remains challenging and AI doesn’t really matter in terms of how you should treat it. For example, the xz-utils back door in the build system to attack OpenSSH on many popular distros that patched it to depend on systemd predates AI and that’s just the attack we know about because it was caught. Maybe AI helps with scale of such attacks but I haven’t heard anyone propose any kind of solution that would actually improve reliability and robustness of everything.
I believe the issue is if an exploit is somehow injected into AI training data such that the AI unwittingly produces it and the human who requested the code doesn't even know.
That’s a separate issue and specifically not what OP was describing. Also highly unlikely in practice unless you use a random LLM - the major LLM providers already have to deal with such things and they have decent techniques to deal with this problem afaik.
You're right though. There's been talks of a big global hack attack for a while now.
Nothing is safe anymore. Keep everything private airgapped is the only way forward. But most of our private and personal data is in the cloud, and we have no control over it or the backups that these companies keep.
While LLMs unlock the opportunity to self-host and self-create your infrastructure, it also unleashes the world of pain that is coming our way.
To slightly rephrase a citation from Demobbed (2000) [1]:
The kernel is not just open source, it's a very fast-moving codebase. That's how we win all wars against AI-authored exploits. While the LLM trains on our internal APIs, we change the APIs — by hand. When the agent finally submits its pull request, it gets lost in unfamiliar header files and falls into a state of complete non-compilability. That is the point. That is our strategy.
If that would happen, The worry I would have is of all the sensitive Government servers from all over the world which might be then exploited and the amount of damage which can be caused silently by such a threat actor or something like AWS/GCP/these massive hyperscalers which are also used by the governments around the globe at times.
The possibilities within a good threat could be catastrophic if we assume so, and if we assume nation-states to be interested in sponsoring hacking attacks (which many nations already do) to attack enemy nations/gain leverage. We are looking at damage within Trillions at that point.
But I would assume that Linux might be safe for now, it might be the most looked at code and its definitely something safe.
LLVM might be a bit more interesting as it might go a little unnoticed but hopefully people who are working at LLVM are well funded/have enough funding to take a look at everything carefully to not have such a slip up.
Yeah, and they can write code with vulnerabilities by accident. But this is a new class of problem, where a known trusted contributor can accidentally allow a vulnerability that was added on purpose by the tooling.
But now you have compromise _at scale_. Before poor plebs like us had to artisinally craft every back door. Now we have a technology to automate that mundane exploitation process! Win!
> Don't kid yourself. If you use this junk, it's making you dumber and damaging your critical thinking skills, full-stop. This is delegation of core competency.
This is a good way to frame the problem. Consider the offshoring (delegation) of American manufacturing to China, followed by the realization decades later that the US has forgotten how to actually make things and the subsequent frenzied attempt to remember.
I expect the timelines and second-order (third-order...) effects to play out on a similar decadal scale - long after everybody has realized their profits and the western brain has atrophied into slop.
I hope someone takes those Meta glasses or an Oculus or Apple Vision or something and hooks it up to clearview or some other facial recognition service and agentically scrapes OSINT sources to doxx people on the street, in real time.
One glance and I have your full name, home address, SSN, all online handles and aliases, employment history, email, and phone number, instantaneously on a HUD. It doesn't even need to be marketed as "doxxing as a service;" it can just be marketed as "professional networking" or "social media." That way people will voluntarily submit their information and all rights over it to the platform.
Until people feel their privacy being viscerally raped on a minute to minute basis nothing will change.
My black-mirror prediction for how augmented reality and AI will interact: In order of horribleness.
1> Auto-nude. Today we can "nudify" photos and videos. Soon, augemented reality glasses will be able to nudify eveyone in real time. (This is totally possible today.)
2> Auto-tranlation. Cool. Everyone can talk to everyone, but users will have censorship options. I don't much like hearing australians so I will just have the glasses make them all sound like proper Texans. And the sound of people with alternative views to my own are replaced with calming country music.
3> Lie detection. Glasses will look for facial/voice body ticks suggestive of deception. Good luck talking your way out of a ticket, or explaining to you boss how you were "sick", when they have a lie detector online 24/7.
4> Censorship of "bad" objects. Signs with ads or news that I do not agree with will be blocked and replaced with more appropriate text. Mosques will appear as churches. Garbage and pollution will become happy birds and clear blue skies. Homeless people will be replaced with attractive young people (see #1 above).
5> Race replacement. I don't like certain races. So my glasses now make everyone Chinese. So long as I don't turn off the glasses, I can live my custom racist utopia.
It isnt the translation. Translation if good. But if you have a machine handling the voices of other people the option to censor/edit/replace those voices can lead to bad things.
This is great. I finally feel for the first time in my life that science has in fact gone too far. At this point living in the so-called "third world" to avoid digital-rape-as-a-service and the ever increasing pace of technology sounds eminently reasonable.
Lie-detection is not going to happen (for a long time). There are no known 'ticks' that can reliably detect lies. Even if there were, there is so much variability in individuals that there is basically no way to find a generalized way of telling if somebody is lying.
Yes, and then people wonder why you are a gaping hole in the social media surveillance dragnet with your absence. It took 12 years for me to make my first comment on this account.
If you think the heat has started, you're mistaken. We're not even in the fire yet. It can and will get waaaay worse.
We've been able to push back against these efforts time and time again. Don't stop. Call your legislators. Talk with your friends and get them to do the same. Vote against politicians that support it.
The problem is that, as a constituency, we are and have always been a tiny minority. Call and vote all you want, it won't change a thing because most people just don't care - or at least don't care enough. And there aren't any good (as far as they are concerned) arguments to convince them otherwise.
My understanding is that Abraham Lincoln literally had all the nation's telegraph lines routed through DC during the civil war, and AT&T has been an honorary branch of the US government ever since.
This is great. Journalists are impeding the preservation of the historical record by blocking archivist traffic while simultaneously manhunting those archivists who find ways around their authwalls.
Soon the news and the historical facts will be unnecessary. You can simply receive your wisdom from the AIs, which, as nondeterministic systems, are free to change the facts at will.
>This is great. Journalists are impeding the preservation of the historical record by blocking archivist traffic while simultaneously manhunting those archivists who find ways around their authwalls.
You are deliberately misrepresenting the situation. The journalists who block archivist traffic are not in any way connected to the blogger who was attempting to investigate the creator of archive.is. You have portrayed them as related in an attempt to garner sympathy for the creator of archive.is.
Indeed. I am highly supportive of archive.is, but let's remember that he hijacked his own users to become a bot net. That should make all us hackers furious. Is a complete violation of trust. Our residential IPs were used to attack someone, meaning he put us all at risk for his own personal goals. It's disgusting behavior and he should be called out for it. But we should also realize he's offering an important and free service to us all. I support him, but this is not something we should just ignore. Trust is very important.
I didn't think I was going to side with the DDoS-er, but considering what happened with Aaron Schwartz, that blogger was trying to get them killed or put in a box forever.
Thanks for this. I didn’t know about the details, and there are probably mor... but this gyrovague person is clearly being a privileged trouble. Their “boringly straightforward curiosity” is an admittance of their shallow thinking. When you are pointed out that you’re hurting someone in some respect that you weren’t intentional about, you should stop, sit down, and reconsider everything in that respect.
You may end up deciding to continue inflicting harm, intentionally so this time---that is a perfectly valid course to take. But you cannot anymore remain unintentional about it.
> When you are pointed out that you’re hurting someone in some respect that you weren’t intentional about, you should stop, sit down, and reconsider everything in that respect.
> You may end up deciding to continue inflicting harm, intentionally so this time---that is a perfectly valid course to take. But you cannot anymore remain unintentional about it.
To be clear, are you talking about the harm of commanding a botnet (which includes you and me) to attack an investigative journalist for investigatively journaling?
It seems like a non-question, but I’ll bite: No. I’m talking about the harm the investigative journalist is doing to the anonymous operator of archive.today by compromising their anonymity and promoting this. You can’t “investigatively journal” to someone’s detriment and say “I was just doing my job ;)”. You can say “I was just curious” (which is “I was unaware” in disguise), but now you are pointed out and are aware, so you must just decide.
And the decision seems to be intentionally do the harm and be insincere about it. Personally, my primary annoyance is with the latter, that they are being insincere about it.
> You can’t “investigatively journal” to someone’s detriment and say “I was just doing my job ;)”.
That description seems to encompass most useful investigative journalism, so I'm not sure it is a useful distinction that an investigation is unpalatable to someone (usually the investigatee).
Suppose we ignore that for a moment, though: it does not justify attacking the investigative journalist, nor does it justify surreptitiously using my computer as part of a botnet to do so.
> it does not justify attacking the investigative journalist
1. Person A hits Person B.
2. Person B hits Person A in return.
Is it ok that Person B hit Person A? I don’t know. I don’t think so. People would unanimously agree, however, that Person A making the first hit makes Person B’s hit more understandable, and that Person A is relatively more to blame here.
So, yeah, I agree: the attack from archivist isn’t justified by the attack from the journalist. It is, however, made more understandable by it.
As for what counts as attack: I think it’s a bit of a stretch to call DDoS to a blog an “attack”. It’s more like a protest. And I think the users of the service would in general not mind taking part in that effortless protest against the actor that is being hostile against the service’s continued operation.
Sadly, it backlashed quite a bit, it appears. People took the words “DDoS” and “botnet” as something much more serious than what they actually entail in this situation, probably because they sound very obscure and vile.
I dunno. Leave? The economics increasingly don't make sense here relative to other places in the world.
It is a bizarre spin on the situation here but the executive summary from the World Happiness Report does indeed put "social media" front and center. I guess.. No interest in digging into the minutiae right now.
I agree the economics here don’t make sense, but leave where? The rest of the world has increasingly strange, or at least unattractive, economics too.
The US is a difficult and long process to get a green card. Other English-speaking countries aren’t necessarily better: Australia seems similar in terms of being a natural resource extraction economy with insanely high real estate prices. Same productivity and salary concerns with the UK.
It's trivial to move to the USA on a TN1 from Canada compared to any other visa category.
If you have a job letter, you show up to the airport and CBP can issue it immediately.
This includes software development which is responsible for GDP growth. Which is why 80-90% of CS students at the University of Waterloo immediately move to the USA after graduation.
There is no tech talent in Canada due to the pay/tax difference and TN1 is good for SWEs.
e.g. Nobody at my office has heard of Gas Town yet. I had to get an invite to a predominantly American Teams chat to discuss it. It's a very draining environment.
Also, senior devs make US$110k and a detached home costs US$800k. I would pay less in taxes and a home would be cheaper relative to income in California.
The "there is no tech talent in Canada" folks have their heads buried in the sand. Markham is a tech hub, and so are a bunch of other places. Obviously it doesn't have the scale of the Bay Area, and the salaries are indeed lower than the Bay Area.
Many of us choose to live in Canada for a variety of reasons, and it is not because we couldn't get a job down south. Some of us even had to turn down moving to the US multiple times in our careers, but that idea is uncomfortable for some folks. It is almost as if some people value other things in addition to money.
If I could, I wouldn't. The U.S is fine usually to visit, but I wouldn't prefer to live there. Thankfully there are theoretically other alternatives that are much more appealing regardless of absolute earnings. The vibes could be better north of the border, but the U.S gives the ick
PBT is for sure the future - which is apparently now? 10 years ago when I was talking about QuickCheck [0] all the JS and Ruby programmers in my city just looked at me like I had two heads.
[0] https://github.com/ryandv/chesskell/blob/master/test/Test/Ch...
reply