"The study, authored by Capraro with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, deliberately used questions where AI models typically fail: visual details from films, such as the colour of a team’s uniform in Bend It Like Beckham."
I get why they used questions where AI models fail, but it also really reduces the value of this study. Nobody is really asking AI the color of a team's uniform in a movie, and if they do and confidently get it wrong, it just doesn't matter at all.
Asking trivial questions also feels like it would affect the rate at which people are willing to confidently say things that are wrong. If you ask me some question of pointless trivia and I ask ChatGPT, I'll probably just repeat the answer because who cares. If you ask me something even mildly important and I ask ChatGPT, I'll either verify the information before I repeat it to you, or I'll qualify that I looked it up with ChatGPT and didn't verify. But some things are just so unimportant that they don't even warrant the disclaimer.
Yeah good god OP please start writing a Substack about this! Engineer who buys a run-down bowling alley and cleverly fixes problems is exactly what the internet needs more of.
I don’t even care that much about the email thing, and most newsletter sites make it easy to have your articles available on the web, but Substack specifically is a horrible company that needs to die. I refuse to read anything hosted on Substack on principle.
I heard an episode of the Odd Lots podcast about HayWire (haywireag.com), a site that pulls public data from government PDFs + APIs, uses LLMs to parse it and turns it into an easily readable website that has all of the latest info on hay prices.
The host made an offhand mention that there's probably a bunch of other similar sites that could be created with all the of useful but difficult-to-access government data out there. That sounded interesting, so I thought I'd give it a whirl!
All pretty fascinating topics to learn about, plus it's been interesting to see how much of the website setup I can fully delegate to Claude. With Cloudflare to buy domains and put the sites up, a Google Service Account with access to Google Search Console and GA4 to create those properties and a Buttondown API key for weekly email sending, it's almost all hands off for me. Though it refuses to take control of the browser and create a new Buttondown account, which I was surprised is a red line.
Thanks! I am definitely not an expert in either, and I've run the content by both Fable and GPT-5.6 with instructions to make sure it's written in such a way that it would read normally to people in the industry. They assure me the wording makes sense in that context, but we'll see if the sites actually get traction or not.
Not an email newsletter service but a contact form. I recently added the ability for LLMs to sign up to my service https://www.simplecontactform.org autonomously via API. Curious to hear your experience if you can ever make use of something like it.
The Limitations section at the bottom certainly has a lot of limitations:
> This paper is a review, meaning it synthesizes and interprets existing research rather than presenting new experimental data. The authors themselves note that current visual tests for susceptibility to discomfort are subjective and poorly standardized. They also acknowledge that the proposed mechanism (that discomfort is the brain’s response to overwork) has not been fully tested, particularly the hypothesis that colored tints reduce discomfort by steering visual stimulation away from overactive brain areas. The relationship between the brain’s excitatory and inhibitory chemical signals and visual discomfort also remains, in their words, “unsettled.” Several key research questions are flagged as unresolved, including how to best quantify the real-world impact of visual stress on people’s lives and how to objectively measure susceptibility.
Flickering lights are about the only thing I saw in here that seem like they'd be a problem in the long term. Everything else your brain just adjusts to over time and stops noticing. Maybe the first few days in an office with bright colors would be slightly distracting, but after that you just stop seeing them. I would guess that a lot of the studies they reviewed probably tested people's reactions to these things when they saw them one time, not the hundredth time.
The article does explicitly state that the brain doesn't adapt to this.
From the article:
"And when the brain encounters something it can’t process efficiently, it doesn’t simply adapt. Brain imaging studies cited in the review show it generates stronger neural responses in visual areas, consumes more oxygen, and in some people produces pain, distortion, or worse."
> And when the brain encounters something it can’t process efficiently, it doesn’t simply adapt. Brain imaging studies cited in the review show it generates stronger neural responses in visual areas, consumes more oxygen, and in some people produces pain, distortion, or worse.
If the studies are of a person's initial exposure to these sorts of conditions, then that doesn't tell us anything about whether people adapt over time (and to be clear I have not read all the studies, but given the limitations listed I'm comfortable assuming they're not incredibly robust until someone tells me otherwise). I suspect the article's use of the word "adapt" is not the same as mine; from the context when they say the brain doesn't adapt they just mean that it shows a response at the time of the particular exposure they're measuring.
I think there were studies on this, leading to, among other things, painting control rooms seafoam green to reduce visual fatigue. This implies that people don't simply adjust (or that the studies were too limited).
I own a dozen Amazon brands that are probably largely of the kind that OP would want this to get rid of (sourced from China, not name brands by any stretch, only sell on Amazon). For the most part I would say this extension is not a great idea (obviously very biased!), since I purchase brands that have high quality products that typically have pretty poor branding/online presence that I can improve. My stuff is very frequently of the same quality (and sometimes from the same factories) as much pricier stuff but at a lower cost. To some of those suggesting you can get this stuff on Aliexpress, in some cases that is true, though of course the big benefit of buying from Amazon is that there's no risk to buying no-name stuff because if it's junk you can return it.
In any case, I gave this a try to see which of my brands it would filter out. It's weirdly inconsistent.
One of my brands was filtered out because there's no brand name at the beginning of the listing. That's just an outright bad rule, because Amazon generally decides whether or not the brand name appears first. This brand is trademarked and has Brand Registry, so it qualifies for that treatment, just not getting it right now. Also, a number of other brands without brand names did not get the same treatment (and these are very much the type of products this is designed to filter out).
On another one, it misunderstood the product model, which is at the beginning of the product name, as the brand and hid it based on that. That one was a bad one because the model is only three characters, which is extremely unlikely for a brand name.
One product I sell is a hunting accessory, so I did some searches there. It hid everything by the brand KUIU, which is a well-known and very high end hunting brand. Definitely wrong there.
So yeah, sort of an interesting idea, but the execution is pretty sloppy and the creator clearly doesn't have a full understanding of how Amazon listings work.
Another PL seller here with a few brands and a couple hundred products:
"No brand name" flag is tricky because the Amazon catalog team actively does A/B tests to hide brand names as part of their goal of commodifying all the sellers to increase price competition, when they see you're selling a commodity item.
Same goes to wellknown brands that get caught in the crossfire because they're using their brand name from another language but don't make sense in English.
Agreed that it's an interesting idea, but execution has a LOT of false positives.
> It might seem obvious to coders, but the difference between Claude Code and Claude.ai's chat is enormous, even if those two run the same model.
In my experience, Claude Code is vastly better for doing tasks, writing code, etc., but Claude.ai is better for analysis and high-level planning. When I'm working on a new project, I've started using the latter to do the initial planning, get feedback and draw up a spec, which then goes to Claude Code.
For this project, I probably would've done something similar - use CC to get whatever you need out of the image files, but have Claude.ai do the actual review/diagnosing.
Either way, I often think about how far behind most of the world is in really understanding AI. The overwhelming majority of people would never guess that you get vastly different outcomes from the exact same model in a different harness (tbf most people don't know what a harness is). I spend hours every day using AI for a broad range of tasks and still feel like I know a fraction of what there is to know. I haven't even tried the new GLM model (or really any of the open source Chinese ones of the most recent generation). With so many people thinking that the free version of ChatGPT is SOTA AI, a lot of folks are in for a very rude awakening at some point soon.
"Agent, research where there are gaps in SaSS products for ____. Then create it. Then go online and spread the word. Then do your best to convince people why it makes sense."
Whether or not AI "kills SaSS" it sure as hell changes the landscape. I write off most of these posts that used to excite me. Not because I needed the product but it was always fun seeing someone put in the work to solve problems they know exist.
Not worthwhile feels a bit strong (a good FAQ is definitely worthwhile!), but I definitely agree that there is a big difference between any kind of art (writing, playing music, creation of images/videos/etc.) for its own sake and for commercial purposes. AI is terrible for the former but perfectly fine for the latter.
There will always be value in a human writing fiction or a memoir or even a Substack. The human perspective is inherently valuable there. Much less so with ad copy that's just going to get A/B tested ad infinitum until a winner is picked out based entirely on data.
Same with visual art. Art painters aren't going to lose their jobs to AI, but once you've got a robot that can paint a house reliably, house painters are done for.
> To investigate whether skills are being lost in the field of computer science, researchers at the AI firm Anthropic in San Francisco, California, designed a randomized controlled trial in which 52 software engineers were asked to perform a basic coding task3. During the exercise, all 52 participants could search the web and access instructions on how to do the task. Half of the participants were prompted to use an AI assistant as well.
> Afterwards, all of the software engineers were asked to complete a quiz about what they had learnt from the task. The participants who had used an AI assistant did significantly worse on the quiz than those who hadn’t: the average score was 50% in the AI group versus 67% in the non-AI group.
This doesn't strike me as a great test? Most engineers aren't going to learn anything from a basic coding task anyway, so I do wonder exactly what they were testing there. If it was just recall about what the issue was, then it doesn't really strike me as a problem - using AI to handle simple problems that it's clearly capable of dealing with is the right way to use it, and of course you're not going to spend time poring over the details because then you haven't saved any time by using AI.
There are other examples that don't strike me as particularly problematic, like GPS eroding people's sense of direction. It's totally reasonable to let a skill atrophy that you no longer really need because you have an ever-present tool to handle it. I'm a lot worse at doing long division than I was when I was <whatever grade one learns long division in>.
The whole skill atrophy thing seems like much less of a problem than it's made out to be. We've been letting skills atrophy for good reason long before the advent of AI. If you start at McDonald's as a fry cook and work your way up to regional manager, if you suddenly have to work a shift on the fry station you're going to be worse than you were when you were doing it all the time. MDs at investment banks almost certainly can't put together a pitch deck as well as the junior bankers who are doing that task regularly. These things are fine - part of moving up in the world and having a broader impact is being able to successfully delegate tasks, and when you delegate tasks your skill at those tasks will atrophy. No real difference whether you're delegating them to AI or not.
To be clear, there are of course cases where skill atrophy is bad. iLoveOncall posted about senior engineers in their org who have lost all of those skills and their judgment along with them. That's definitely bad! If you delegate so much that you lose the ability to even judge good work, now you can't even delegate effectively any more.
I think the real lesson with AI is that you need to be self-aware about what skills you should practice and retain vs. what skills you can let atrophy, since it's easier than ever to hand things off. I've lost most of my ability to write a SQL query, but that's fine because it was only a skill I used intermittently and AI can always do the job fine at the level of complexity I need. I have not let my skill of writing product specs atrophy (I am a PM, in case you haven't read my username), because that's critical to using AI correctly in the first place.
So many! I manage a fund that buys small e-comm brands, and at this point the whole thing runs on a combination of AI and tools created with AI. My favorite is one that scrapes my Alibaba/WeChat/WhatsApp/email supplier convos daily and uses that to build a dashboard tracking the status of my orders.
I get why they used questions where AI models fail, but it also really reduces the value of this study. Nobody is really asking AI the color of a team's uniform in a movie, and if they do and confidently get it wrong, it just doesn't matter at all.
Asking trivial questions also feels like it would affect the rate at which people are willing to confidently say things that are wrong. If you ask me some question of pointless trivia and I ask ChatGPT, I'll probably just repeat the answer because who cares. If you ask me something even mildly important and I ask ChatGPT, I'll either verify the information before I repeat it to you, or I'll qualify that I looked it up with ChatGPT and didn't verify. But some things are just so unimportant that they don't even warrant the disclaimer.
reply