> ... it chains several bypass methods per request
I'm curious what are the techniques? I'm familiar with spoofing yourself as a Googlebot, or extracting the content directly when paywall enforcement is done in client-side JS only. But surely you need more tricks than that to get around WSJ or NYTimes. Also, ironically, I assume archive.is's CAPTCHA would need bypassing to fall back to that. (but volume might be low enough you can do it with a few cookies you've established trust on)
Good guesses. Crawler spoofing and pulling content that’s only hidden client-side are both in the mix, but you’re right that they aren’t enough for the bigger sites. For what it’s worth, NYT works reliably right now and WSJ is hit or miss, so WSJ is still the hard one.
I’m going to stay a little vague on the rest, because anything I describe in detail here is something a publisher can patch by Monday. The general idea is to treat each site as its own problem: several methods per request, a check on whether the article actually came back (not just a teaser), and on to the next method if it didn’t.
On archive.is: yes, the CAPTCHA is the annoying part. Volume is low enough that it’s manageable, and it’s a last resort rather than the main path.
Is it fair to say that Crawler spoofing only works on sites that don't care?
Google recommends [1] checking the reverse DNS hostname of the source IP. I don't see any room for tricks there. So if you spoof a whitelisted Crawler simply by choosing the right values for some request headers, it means the publisher's web team is either clueless or doesn't care enough, no?
News site probably want to be accessible to all search engine crawlers, not just Google, and I bet some of them don't have the reverse DNS verification method.
But also, tons of news sites also only implement the paywall in client-side JS! This is so obviously easy to bypass it seems intentional, and they just don't care about the 'hackers'. Or perhaps they've know that people that are going to use tricks to access an article are very unlikely to pay for a subscription anyway.
(I wish I could just pay $1 on demand and read a paywalled article. I'd do it all the time, but I'm quite reluctant to sign up for any forever recurring subscriptions)
Agreed that makes it quite nice! Though I wonder how long that will last, since I imagine archive.today is dealing with some problems that necessitated it.
The whole John Donne meditation the title is referencing:
No man is an island,
Entire of itself;
Every man is a piece of the continent,
A part of the main.
If a clod be washed away by the sea,
Europe is the less,
As well as if a promontory were:
As well as if a manor of thy friend's
Or of thine own were.
Any man's death diminishes me,
Because I am involved in mankind.
And therefore never send to know for whom the bell tolls;
It tolls for thee
(had to memorize it in High School, and its one of my favorites. It was written over 400 years ago and has aged quite well.)
Not totally related but The Tuft of Flowers by Robert Frost always reminds me Donne's poem.
The Tuft of Flowers
I went to turn the grass once after one
Who mowed it in the dew before the sun.
The dew was gone that made his blade so keen
Before I came to view the levelled scene.
I looked for him behind an isle of trees;
I listened for his whetstone on the breeze.
But he had gone his way, the grass all mown,
And I must be, as he had been,—alone,
‘As all must be,’ I said within my heart,
‘Whether they work together or apart.’
But as I said it, swift there passed me by
On noiseless wing a ‘wildered butterfly,
Seeking with memories grown dim o’er night
Some resting flower of yesterday’s delight.
And once I marked his flight go round and round,
As where some flower lay withering on the ground.
And then he flew as far as eye could see,
And then on tremulous wing came back to me.
I thought of questions that have no reply,
And would have turned to toss the grass to dry;
But he turned first, and led my eye to look
At a tall tuft of flowers beside a brook,
A leaping tongue of bloom the scythe had spared
Beside a reedy brook the scythe had bared.
I left my place to know them by their name,
Finding them butterfly weed when I came.
The mower in the dew had loved them thus,
By leaving them to flourish, not for us,
Nor yet to draw one thought of ours to him.
But from sheer morning gladness at the brim.
The butterfly and I had lit upon,
Nevertheless, a message from the dawn,
That made me hear the wakening birds around,
And hear his long scythe whispering to the ground,
And feel a spirit kindred to my own;
So that henceforth I worked no more alone;
But glad with him, I worked as with his aid,
And weary, sought at noon with him the shade;
And dreaming, as it were, held brotherly speech
With one whose thought I had not hoped to reach.
‘Men work together,’ I told him from the heart,
‘Whether they work together or apart.’
I'm also reminded if "I am a rock" by Simon and Garfunkel. Last two stanzas:
I have my books
And my poetry to protect me
I am shielded in my armor
Hiding in my room safe within my womb
I touch no one and no one touches me
I am a rock I am an island
And a rock feels no pain
And an island never cries
For fun I looked up how Donne originally wrote it in 1624:
> No man is an Iland, intire of it ſelf; euery man is a piece of the Continent, a part of the maine; if a clod be waſhed away by the Sea, Europe is the leſſe, as wel as if a Promõtory were, as well as if a Mannor of thy friends, or of thine owne were; Any mans death diminiſhes mee, becauſe I am inuolued in mankind; and therefore neuer ſend to know for whom the bell tols; It tols for thee.
Italics are original. Weird chars are best attempts by LLM transcription. Still quite readable, but interesting how much spelling has changed/standardized. Because its prose, I think we feel more free to modernize it than if it was poetry.
It doesn't end on "It tolls for thee" or start on "No man is..". It keeps going in both direction, and no indication he every meant for this section to be pulled out into its own small piece the way it has. Here's some of what comes after:
> ... It tolls for thee. Neither can we call this a begging of Miserie or a borrowing of Miserie, as though we were not miserable enough of our selves, but must fetch in more from the next house, in taking upon us the Miserie of our Neighbours. Truly it were an excusable covetousnesse if wee did; for affliction is a treasure, and scarce any man hath enough of it....
Hemingway also borrowed the penultimate line in For Whom The Bell Tolls. I also had to memorize this back in the day but I still stop to read it whenever I see it somewhere.
This poem is beautifully referenced in Mark Campbell's libretto for _As One_, an opera about the life and self acceptance of a trans woman, Hannah. Her class is also assigned the poem to study. This part destroys me every time:
*Hannah before, Hannah after, Conductor, Quartet:*
“…Any man’s death diminishes me,
Because I am involved in mankind…”
*Hannah before:*
I am the lone,
Dissenting voice
In the classroom.
I rise and declare
To a sea of non-islanders:
“It isn’t true,
I am an island.”
With all the experience
Of my fourteen years,
I see no other life
Than one apart,
Alone.
I need no one.
No one needs me.
I consigned myself
To my own island long ago.
Long ago.
I argue, and,
Satisfied by
My brilliant discourse on independence,
Sit back down.
The classroom is silent.
The teacher seems impressed.
(Or was she concerned?)
Generally it's fair use if it's a legal copy and you have a legitimate reason such as backup/recovery or working around a bug and the models don't care.
If you stumble into vulnerability/cryptography territory sometimes they'll whine.
Surprisingly, no. I've been using Fable and Astra both to orchestrate decompilation of a relatively modern game (delivered via Steam) and they have no qualms about it.
Sadly not updating the OS is very bad from a security perspective. Though it's frustrating that Apple rarely provides any sort of "security fix only" updates, and to fix security issues you just have to install all the new things at once.
My experience has been that they do provide those updates, but they intentionally make it hard to notice compared to getting the major version updates that are available for the OS. I distinctly remember a new junior engineer out of college joining my team back in maybe 2022, seeing there were security updates, and then not noticing the button saying something like "more updates" to be able to get them without updating to the latest major version of MacOS.
More recently, at the end of August, by the dates on my screenshots on my work laptop (the only Apple device I use) I took screenshots because I was incredulous of the sheer audacity of the UI shown for updating from MacOS 15.7.7 to 15.7.9 along with, in the words of the update UI, "1 more" update. When I clicked the button to view all of the available updates, it showed three available updates, which were, in order, MacOS 26.6.2 (which was checked), MacOS 15.7.9 (which was unchecked), and Safari 26.6.1 (which was checked). They literally previewed the only update that was not checked despite not even sorting it at the top of the list of updates available to install. I have absolutely no clue what would have happened if I clicked the "Update now" button next to the prompt for 15.7.9 rather than the identically-styled-to-non-link-element text for viewing the full set of updates, but at absolute best it would have just done something entirely different than what I'd get from the default options when viewing the expanded list, and it seems far more likely that it would have just also done the default options of the expanded view and updated to a completely different version that it put next to the update button.
on MacOS they provide security fixes for two years for unsupported OSes (after that the only remaining security protection is Gatekeeper/Notarization). On iOS you generally get no security fixes unless the vulnerability is really bad
AFAIK Apple has no published policy. Their last several macOS releases have received 3 years of security fixes.
iOS now seems to get 5 years of security updates, but only for devices that can't upgrade to newer iOS versions - so iOS versions that don't strand any devices get cut off early. e.g. iOS 15 received 3 updates in 2026, but iOS 17 hasn't been updated since 2025.
I recently sold a nine year old iPad running iOS 17 that is still getting security updates (17.7.11 a few months ago). Even iOS 15 (released 2021) got a security update in May.
I believe if you have a much older iDevice that no longer supports the latest OS, they will keep releasing security updates for that device for a long time, which is good on them. But if you have a newer device, after awhile the only way to get security updates is a major version upgrade. You luck out if your device happens to be old enough not support to newer iOS versions.
I love horseback riding and do it all the time. I can't walk past an open field without wishing my horse Fawkes and I were cantering across it.
So when I heard about Jev, I was pretty excited. But as I learned more, I realized Jev can't ride horses, can't muck a stall, can't teach a flying lead change, and can't even reset a thrown shoe.
Maybe I'm doing it wrong, but I can't figure out why this exciting new thing doesn't work for my personal interest. Its lack of horseback riding support is a dealbreaker for me.
For Home Assistant, the MCP can enable the LLM to make changes without using up excessive tokens.
For example, the only way to make a change to an HA automation via the API is to POST a whole new copy of the YAML, even if changing one things. The skill and HA MCP I use allows the LLM to use tools which make more precise changes without excessive context usage.
Certainly still works either way though. And of course your LLM instance could just roll its own tools to do the exact same thing.
For browsers that implement strict cookie partitioning, the privacy concern is moot. With this, the cookie jar that's used when chatgpt.com is a 3rd party site is completely separate for the jar used on the 1st party site. So your chatgpt.com account can't be linked to your ad views.
Since this has been standard ad tech for awhile, browsers have reacted to this to implement cookie partitioning for exactly these privacy reasons.
Modern adtech fingerprints your browser regardless of cookies, so they can still know who you are. The fingerprint consists of a combination of facts about your browser that together uniquely identify it (screen resolution, timezone, installed fonts and browser extensions, OS version, User agent, even GPU with some forms of canvas fingerprinting, etc.) ChatGPT uses fingerprinting too. Cookie isolation isn't relevant, only anti fingerprinting techniques are.
> Google simply regrets android being open source.
Could Google backtrack on this if they wanted? If they suddenly decided to stop releasing public code updates, what would they be violating? I can guess: Anti-trust maybe. probable at least contractual commitments Or is there some stronger forcing function?
Relatedly: I suspect LLMs are influencing baby names. If you ask Claude or ChatGPT for its favorite baby names, you'll get baby names that right now are skyrocketing in terms of popularity.
I'm curious what are the techniques? I'm familiar with spoofing yourself as a Googlebot, or extracting the content directly when paywall enforcement is done in client-side JS only. But surely you need more tricks than that to get around WSJ or NYTimes. Also, ironically, I assume archive.is's CAPTCHA would need bypassing to fall back to that. (but volume might be low enough you can do it with a few cookies you've established trust on)
reply