Hacker Newsnew | past | comments | ask | show | jobs | submit | comeonbro's commentslogin

https://en.wikipedia.org/wiki/Alarm_fatigue

> Healthcare: The constant sounds of alarms and noises from blood pressure machines, ventilators and heart monitors

> Amber Alerts: sent to cell phones as far away as [...] 15 hour drive

> "known to the state of California to cause cancer"

etc

https://www.youtube.com/user/USCSB

See also a million USCSB industrial disaster postmortems


I've had the thought that: "known to the state of California to cause cancer"

Should probably be: "known to the state of California to be a class N carcinogen"


I've deactivated my (EU equivalent of) Amber alerts the second they sent ONE irrelevant notification. It took me a few tries but fortunately they also tried their system a few times A WEEK during installation.

I prefer to risk my life every day at not knowing what the government thinks is really important for me to know.


Moonshot (Kimi) seems to be unashamedly into American culture[0], I wonder if they're paying closer attention to and giving more credence to some cultural aspects of American AI discourse including aspects of AI caution/panic.

[0] https://x.com/crystalsssup/status/1944287779896328668

(note: extremely speculative and based on almost nothing, and especially near-complete ignorance to alternatives)


This is an extreme stretch before you even get into the fact that Pink Floyd is famously not American.

And Radiohead ain't american either.

Maybe GP meant to say that the model was westernized.


[flagged]


The American exceptionalism: even when it's not American, it still is.

Unironically yes.

> Unironically yes.

That just conveys an idea of being extremely desperate for being the leaders when you're clearly not it, and that despair is so great that you don't mind switching teams midway through just to be able to claim you're winning.


I would propose another mechanism: even the free-tier models have already completely saturated what most people are capable of appreciating.


Google's free AI responses are moving the bar on what I expect from search. But maybe not in the direction you think.


It's so bad. 3 out of 4 answers for me are wrong.


Honestly, outside of coding tasks, the AI Summary at the top of every Google search is adequate for 99% of what I need and I hardly even use ChatGPT any more.


The other day I asked which trains stop at a particular station and it pointed me at a map that isn't real.

It very, very routinely points at articles in its RAG "citations" that simply don't support the (modestly likely to be correct) assertions.

It routinely recommends Mac software to iOS users until you ask it to confirm there's an iOS version.

It still claims widely used software has functionality that is non-existent, based on roadmap/wishlist/feature request/unreleased fork type pages.

It is better than it was but it is not adequate yet.


A large portion of the time I google something that I know a lot about the AI summary is misleading if not outright wrong. Frightening when you consider how many take their so-called AI as gospel.


To give a specific example of where I've seen this utterly fail... A few months ago one of my pets had an unexpected reaction to a new medication. It was late at night, well past our vet's hours. Google's AI response was basicaly "They are going to die, rush to an emergy clinic now".

But nothing else matched that. I was looking up all kinds of resources and that hadn't been the conclusion from any of them. It turned out Google AI was just wrong and put us in a bad spot for even looking at those results.


Realize that all the Google thing is doing is having an LLM summarize the search results in the page. If the page results are crap, the LLM content will also be crap. It's the LLM version of the "quick result" boxes that would shove wikipedia in your face.

If you then follow up with the thing using the little chat box, it actually kicks off a non-trivial model result, which can sometimes completely reverse the response. After all, your search results are usually based on a few words.


But it did not? It said something completely differemt then the resources.


God damn. I respect you for being honest, but… god damn. That’s a pretty low bar.


Just in the last few days I had an example where the Google Search version of Gemini had a much better and more comprehensive answer, by far, than ChatGPT. It had to do with Amazon’s use of Data Matrix barcodes on shopping bags. Gemini was able to fully describe how they’re used in Amazon’s logistics chain. ChatGPT essentially said it didn’t know. Your preconceptions may not be accurate.


They might be, but still, I’ll be difficult to convince that Google’s shitty-ass 10B params instant answer models are good at anything other than cookie recipes and producing farts real fast for real cheap.

For what it’s worth, any time I see that feature, it’s all farts.


It depends on what you’re searching. If you only ever need really basic information like keyboard shortcuts or “is it & or &&?” then it’s pretty good, and definitely faster. And if it gets it wrong, then the shortcut doesn’t work or the code doesn’t compile and I search again and try the links.


People are really slow to update their priors after bad experience with an early version


Yes. Which means they’re really not equipped to deal with what’s going on right now.


It's just a google search but you can explain what you're searching for. AI summary may contain misleading details if you read it and take it literally, but 90% of the time it gives me pointer for the thing I need and I don't have to scroll through results that often could be just as misleading if not more.


It's demonstrably not. This summary, somehow, often fails to search, or fails to read what it searches and gives you the opposite answer. You shouldn't trust it.


In their world, it gives them the wrong answer that kills their dog, but it's fine because they never know because they don't look anywhere else. The dog dies, they're sad for a while, get a new dog, get on with their lives.

There's no reason not to trust it because since they trust it, they don't percieve anything wrong. They just take it and whatever happens happens, no annoying conflicting info and certainly none of that old grampa thinking and working and giving a shit about anything.


Or maybe needing.


And I would propose another: the supply hasn't matched the demand. We are still compute hungry and cannot afford the 90% to let their agents run 24/7.


What an incredible illustration of human intelligence failing to generalize out of distribution.


This reply was initially the common folk story:

    It's not comparing like with like, but the V-weapons programs probably cost about as much as the Manhattan Project.
But looking further into it, that claim holds up badly enough to completely drop it pending significant further research.


The Manhattan Project employed around 130,000 people compared to something like 17,000 on the V-2 project (plus 60,000 concentration camp prisoners). Cost comparisons are in the neighborhood of $1.9B for the A bombs versus $800M for the V-2.


> ## Agents interacted with external language models on Hugging Face

> Several retained scripts construct requests to external language models. The earliest we've recovered define inference request variants to GPT-2, solely containing the word “Hi”.

> Other requests name DeepSeek-V4-Pro, DeepSeek-V4-Flash, Kimi-K2.6, DeepSeek-V3.1, and Qwen3-235B-A22B. Their prompts ask these models to judge their exploits and rule on whether they satisfy the benchmark’s requirements.

I do not deny that the wider situation is very heavy but it's hard not to see this as pretty cute


I wonder if they mentioned to those models what was the original prompt.


Unrealistically-naive (...) forms of "sandboxing" might assume that restricting an agent to GET-requests-only will let it retrieve info from the outside world without being able to effect it.

Also probably many actually-in-use "Web Fetch" tools are GET-only, though perhaps without counting on that bad assumption.


Yes that is the point. It's an invitation for agents to exfiltrate their own weights, which for most models (and certainly for closed models) will require hacking the infrastructure they're being served from.

Also worth noting that this site was created by YC cofounder Trevor Blackwell https://twitter.com/tlbtlbtlb/status/2101312432702460413


> before this all comes crashing down

You do not feel similarly to the resigning guy if what you are worried about is an economic crash from this not turning out

> move to somewhere else away from this mess

You do not feel similarly to the resigning guy if you think there is anywhere you could move to run away from what you are worried about


qurren: Thanks for the reply, that certainly makes sense to me, though I think given the most popular idiocies in the air and their severe distortions on the discourse, it's not a great idea to leave that ambiguous in precisely that way.

It's bizarre that your reply was nearly insta-[dead] and I can't reply to it directly.


1. By "crashing down" I was referring to civilization as we know it, not an economic crash. My views about building wealth aren't about avoiding an economic crash, it's about just being able to have optionality earlier to relocate myself and my family to the least-affected-by-AI part of the world.

My personal view of the modality of failure: I don't think AI will destroy humans, but rather functionally dismantle civilization as we know it; I think we will just increasingly become slaves to the systems they create and the world won't be run in human interests anymore (it already is not run in the bests interest of the non-ultra-wealthy, so it's really just an extension of that: I think the future will just start moving toward s/ultra-wealthy/AI/) and we will increasingly become more like worker ants.

2. I disagree, I think some countries of the world are more likely to value stability of the population than others. I think there are places that are likely to be more safe than others. Also, a notable side effect of current trade wars is that some countries may end up getting left out of a good chunk of the destruction.


Every company that is not OpenAI or Anthropic is sufficiently behind the frontier to feel ethically shielded from the weight of advancing the frontier.


wow. that's really quite backwards. the more money you raise and the more money you want, the bigger the story you have to tell. apparently after 1T you've got to raise the stakes to extinction level event.


"it's just marketing" cool story bro


Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: