Residential Proxies are the most emblematic technology of our era- a group of people looked at something that used to be considered a crime (botnets) and realized that if they just did it openly, no one would ever punish them.
And it's made necessary because another group of people thought that selling IP blocking services would be a good idea. One party sells walls, another party sells ladders.
Well, one party gives away free walls if you agree to fill your castle with surveillance cameras you don't control.
This is a super dishonest characterization. Running software on a bunch of machines, even machines in other peoples' homes has never been a crime. Folding@home isn't a crime (obviously). It's controlling those machines without consent via malware that is criminal. And if it is open and consensual in exchange for something a person wants, it is unreasonable to compare it to botnets.
No one's firing up a residential proxy to read your blog, and the corporate AI scrapers have all the resources in the world even without residential proxies
They're most useful for getting information from the cloud hosted sites that hoarde most of humanity's output today like Youtube and Reddit.
I actually read a really interesting article from a relatively small blog explaining that they're receiving a massive amount of scraper traffic from residential proxies.
The article was called "The one we're commenting on"
> The LWN content-management system contains over 750,000 items (articles, comments, security alerts, etc) dating back to the adoption of the "new" site code in 2002. We still have, in our archives, everything we did in the over four years we operated prior to the change as well. In addition, the mailing-list archives contain many hundreds of thousands of emails.
Does that sound like your typical self-hosted blog?
The Bright Data mentioned in the article, as well as other similarly malicious but even harder to identify parties, most certainly do fire up residential proxies, no matter what the site, no matter how useless or duplicate the data which they're trying to get. Not at first - they start by trying to get your content from cheap data center connections - but as soon as some kind of bot-mitigation appears, they move to residential proxies to try and evade that, with a first tier coming from "global south" residential proxies, and then scaling up to (presumably more expensive / less widely available) proxies from the USA (I've seen some from Europe, but very few in relative terms). Each tier also appears to have the option to run JavaScript.
You're one with the lower case shibboleth so I have no doubt you surround yourself with dishonest founders, but faking users is pretty damn low on the usecases for residential proxies.
I said they're unethical because they tend to be hidden in innocuous seeming apps or sprung on unwitting individuals via clickwraps on their smart devices.
ive seen unscrupulous founders fake traction during diligence, which is my day job
but ive never seen one raise $4.5m for an ai agent startup built around pulling fresh web data, then openly cheer the unethical proxy infrastructure used to evade consent and blocks
then inventing a fantasy about who i associate with instead of answering that conflict is an unusually loud form of projection
> Many providers build their proxy pools by partnering with device owners who agree to share their bandwidth, while others use embedded SDKs in free apps or VPNs.
Mmm... your quote (IDK where it's from) mentions them having consent from device owners, but your FBI link cautions on how to avoid getting infected by malware.
If they have consent, they're not really botnets. Botnets involve infecting devices without the owners knowing.
With consent, it wouldn't be much different from e.g. open WiFis at restaurants and hotels, companies using a single ISP and single public IPv4 address for all their employees, and most VPN services.
And even if you spent a minute explaining the proposition to a user off the street, it still wouldn't be fair unless you laid out the drawbacks. Which leads me to a question.
There must be countless individuals all over the world who suddenly can't log into their Gmail or create any new accounts because a fraudster sent spam from their IP. I wonder: has anyone has tried to quantify that problem?
> There must be countless individuals all over the world who suddenly can't log into their Gmail or create any new accounts because a fraudster sent spam from their IP.
Places with open WiFi like hotels and restaurants would be having the same problem. People on CGNATs would be having the same problem. An IP doesn't correspond with a single user.
Thank you. Gmail must not be like our fellow HN users we see here, quoting a couple:
“I'm tiny and only run little personal stuff. I just block vast IP address blocks.”
“Apologies. :( Since you say you've never visited the website before, then that means you're either in one of the countries or in one of the residential IP ranges that I've had to block.”
Although Google isn’t afraid to completely block iCloud private relay from Google scholar. Other sites may reject the first iCloud private relay visit, then reopen site in new tab and often automatically assigns new unblocked IP. Anyway, I would’ve thought it’s an acceptable cost from e.g. Google’s perspective to block ranges that did something bad once in spite of collateral damage.
> Anyway, I would’ve thought it’s an acceptable cost from e.g. Google’s perspective to block ranges that did something bad once in spite of collateral damage.
Almost nobody is on a static IP. ISPs have an interest to keep their customers connected and not affected by other customers, so they'd probably use as large address pools as possible for their dynamic IP rotations. "Did something bad once" is short-lived unless address ranges are large.
Tech companies like Google are interested in being global monopolies to dominate markets and also having as good and large-sampled statistical data as possible from everyone. Blocking thousands of users for each bad user that can likely easily switch to another block if they really want to doesn't seem effective. It would also affect their reputation. Who wants to rely on an email service that may block you because of other users on your IP block? Having a reputation for being reliable is important for businesses.
With small personal sites/blogs like the ones you mentioned, they can just block most of the planet without a problem. There's not as much reason to be as reachable as possible (some may be satisfied with an audience of a handful of people), and they're also more budget conscious.
In the case of Google blocking an Apple service, they not blocking users; they're blocking Apple. The users can just not use Apple's service.
