The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness. It's usually wrong to assume that everything in a /24 is controlled by one botnet, and it's not that hard to check whether it was just one or two particular addresses that were compromised. But if you're an ISP and you want to take the nuclear option to every spam threat, at least be willing to listen to your own customers when they complain that they're expecting mail, and there's absolutely NO reason to assign "group punishment" to everyone using the same service provider. I think the thought was that that would make service providers more accountable, but it's totally unfair to use everyday customers as pawns in a war between ISPs.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
Yes, in that situation it's laziness, but on the part of your ISP. ISPs already spend huge amounts of time and money dealing with spam, hacking attempts, phishing attacks, etc. If your ISP is irresponsible and isn't doing their job keeping those things from leaving their network then your ISP is gong to find their entire IP space blocked and that's 100% reasonable. Why should we ever accept traffic from an ISP that refuses to keep their corner of the network clean when it's just going to cause problems for us and our users?
That's the situation for every ISP on the internet. Keep your users in line, keep trash off your network or else no one is going to accept traffic from you. You could call it laziness, but it simply isn't worth it. We'd rather spend our time cleaning up abuse on our own network and working with ISPs who are doing their job than dealing with the problems we get from bad actors.
If you own an IP surrounded by a bunch of spammers and it's giving you trouble step 1 should be to contact your ISP and tell them to get their shit in order or you'll take your business to an ISP who does their job. Step two is to switch to a new ISP if they don't. No one has the right to force us to accept traffic from anyone else. It's every ISPs responsibility to make sure the traffic leaving their network isn't more trouble than it's worth. Good ISPs are rewarded because users will give them their business and stay and bad ISPs are punished because users will drop their service when they see they are blocked.
The goal isn't to punish the poor sucker who signed on with an irresponsible host, but to cut down on the number of bad ISPs on the internet and the amount of work we have to deal with coming from them.
That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
There is a reason that collective punishment is considered immoral by civilised cultures. It hurts the innocent and often fails to achieve its original goal anyway.
> There is a reason that collective punishment is considered immoral by civilised cultures
Rejecting email submissions isn't punishment, collective or otherwise. It's something you have to do if you run a mailserver. In the same way, I'm not punishing trespassers if I secure my front-door with a deadlock.
> That seems a lot of rationalisation for a situation where a genuine sender on another system sends legitimate mail to a genuine recipient on your system, that mail is not properly delivered, and it's your fault.
It's how the internet stays functional. If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture. Bad actors have been ostracized from communities for as long communities have existed. If you want reliable service, choose a responsible ISP. Blacklists have enabled the internet and email to remain useful for most users most of the time. Without them, email wouldn't be usable. If you have a better solution for spam, the whole world would love to hear it.
If 0.001% of legitimate mail has to go undelivered in order to prevent overwhelming amounts of spam/attacks from an irresponsible network that's an acceptable loss to most people in our civilized culture.
1. It's way more than 0.001%. Like, several orders of magnitude more.
2. What overwhelming amounts of spam/attacks? Those of us using traditional mail systems with traditional spam filtering seem to be doing OK not getting overwhelmed by incoming spam without the kind of "help" you advocate.
3. You don't get to decide what's acceptable to everyone else. Except that apparently you've decided you do, which is why regulation is needed to remove that ability from service providers who can't do their jobs properly and hurt others as a result.
If you want reliable service, choose a responsible ISP.
This is a poor argument. Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them. The decent ones will identify the problem and block it reasonably quickly, but there is plenty of evidence that they can still be blacklisted and it can still be difficult or impossible to get removed from those blacklists again after the problem is fixed.
The kind of policy you advocate punishes small ISPs just for being ISPs. I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
Or maybe the rest of us should apply the same policy to organisations that do what you advocate. If they won't deliver mail reliably, we won't forward any mail to them at all, so their mail service becomes useless. Except of course it's some of the biggest mail services that do this, so just like no-one's going to block incoming mail from AWS or MailChimp, no-one's going to block outgoing mail to Google or Microsoft.
If you have a better solution for spam, the whole world would love to hear it.
Block actual spam sources and provide a reasonable method for removing blocks that are no longer necessary. Don't carpet bomb whole chunks of the Internet just because there are a few bad actors around. Don't fire and forget. It's really not difficult and plenty of small organisations operate just fine on this basis every day.
The majority of all email has been spam, for more than a decade.
> You don't get to decide what's acceptable to everyone else
If you operate a mailserver, you do actually get to decide what kind of stuff you are willing to accept, and from who. "Everybody else" does not get an automatic right to inject data into my computer.
> The kind of policy you advocate punishes small ISPs
Not at all (perhaps you meant MSPs?) Blocklisting Google was a reasonable policy, at one time. Blocklisting MailChimp is perfectly reasonable now.
> Block actual spam sources
Of course, good plan. Unless the sender's ISP is in the habit of moving spammers from one address to another, so they can evade blocks. Then you have to block the ISP, or eat their spam.
At times that number has been even higher with over 90% of all messages sent over the internet being spam. If 90% of all the messages in your inbox were spam how long would you continue to use it? Email systems can't bear the costs that spam forces on them. Even with tools like blacklisting which you think shouldn't exist that cost is measured in tens of billions annually. source: https://www.aeaweb.org/articles?id=10.1257/jep.26.3.87
If not for the ability to filter common sources of spam, email would never have survived as a viable means of communication.
> It's way more than 0.001%. Like, several orders of magnitude more.
Whatever the actual number, it's clearly acceptable to us because blocking irresponsible networks is standard practice. We depend on it.
> What overwhelming amounts of spa
Again, 80-90% of all mail is spam, costing billions. If you're able to run a mail system without blacklists that's great for you, but it clearly doesn't work for everyone.
> You don't get to decide what's acceptable to everyone else.
That's the beauty of the internet. I don't have the power to force an abusive network to do their job and prevent spam from leaving their network and that abusive network can't force me to accept mail from them. No one can force anyone to do anything. All we have is a loose set of standards and expectations and it's up to each network to decide what to accept or not based on how well those standards and expectations are followed.
> Any ISP that accepts significant numbers of customers will occasionally have a customer who is either malicious or operating with imperfect security allowing someone else who is malicious to exploit them.
A responsible ISP identifies those users and prevents them continuing to cause problems. If they refuse to do that their reputation suffers and they will get blocked. If they do their job too slowly or too poorly they will be blocked.
Is it possible for a responsible ISP to end up on blacklists? Yes, it is, and there are blacklists that don't maintain their lists well. That's fine too because no one is forced to use them. It's still the case that every network has the choice of what blacklists they will or won't use and how they use them. They can whitelist blacklisted IPs they decide to trust and they can use blacklists to greylist instead of block.
> The kind of policy you advocate punishes small ISPs just for being ISPs
Nope. Even small very ISPs can staff their internet abuse departments adequately and implement anti-spam technologies to prevent their IP space from becoming a safe haven for hackers and spammers. If they choose not to do that they will and should be blocked.
> I invite you to apply the same policy fairly and neutrally to larger organisations such as the major mail forwarding services and cloud hosts as well and see how long you survive in this industry.
I'll agree that there are problems when certain services (either cloud providers or mail providers like Gmail) become "too big to blacklist". We've had that problem with AOL and we have it now with Google. Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
> Block actual spam sources
If your ISP is a safe haven for spammers and hackers their IP space is the spam source.
> provide a reasonable method for removing blocks that are no longer necessary.
So your alternative to blacklists is just more blacklists that are run better? I think everyone who depends on blacklists would like those blacklists to be better at detecting spam sources and better at clearing unnecessary listings. The good news is that badly run blacklists don't tend to get widely adopted because they cause more trouble for ISPs than they are worth.
If some network won't accept your mail and you're convinced that your ISP is acting responsibility and that it's the blacklist that's wrong, you can have the person you're trying to reach contact their ISP to get your mail server whitelisted. If an ISP sees that a blacklist they use is catching too many messages that it shouldn't they'll adjust their thresholds or stop using that list.
It's not a perfect system, but it's the best one we have.
Did you actually read that, and the sources it cites, before posting it? If you had you might have noticed that it's full of the worst kind of junk stats. Several of the sources cited, the ones that supposedly support your arguments here, don't even say what the piece you linked claims. They literally have completely different numbers. Not that it matters since there is no indication of methodology used and the exact figures are clearly impossible for anyone to measure accurately. Some of the other "sources" are just links to organisation home pages without identifying any specific research or analysis at all.
If 90% of all the messages in your inbox were spam how long would you continue to use it?
As someone old enough to remember the time when that was actually the case, obviously we managed. But this is distorting the argument again because you are implying a false dichotomy where the alternative to overly aggressive blacklisting policies such as you advocate is all of the spam reaching our inboxes. Clearly that is not realistic as less aggressive defences are still highly effective and have consistently been so for a long time.
No one can force anyone to do anything.
Really? Then where can I sign up for a mail service that will reliably deliver both my incoming and outgoing legitimate messages without undue monitoring or interference with my own business? I contend that possibly no such service currently exists.
Personally, I'd prefer to hold them to the same standards as everyone else, but the problem of the largest players throwing their weight around giving them unfair advantages exists in every industry and until someone comes up with a solution for it, we're all just stuck playing along.
Which is exactly why some of us are in favour of statutory regulation to compel anyone participating in such an important technological ecosystem to be a good citizen.
So your alternative to blacklists is just more blacklists that are run better?
I don't believe I have ever suggested anywhere in this discussion that using blacklists to block traffic from proven spam sources was unfair or inappropriate. My objection, which seems to be in line with the submitted article, is to big mail services that think spraying fire into a crowd of 250 indefinitely because there was once one bad person there is a reasonable response to the problem. There is huge collateral damage being caused and the defenders of this policy are trying to sweep it under the carpet and use highly debatable arguments of necessity to justify their damaging policies.
This is not the best system we have. That's the point being made here.
> As someone old enough to remember the time when that was actually the case, obviously we managed.
I'm also old enough to remember that and we managed by blocking huge amounts of IP space. Even massively popular services like AOL have blocked the IP space of entire ISPs or entire countries from being able to send them email. Eventually spam filtering improved, things like SMTP auth, DKIM etc caught on and wide range blocking could be scaled back somewhat, but I doubt it will ever go away entirely.
> Really? Then where can I sign up for a mail service that will reliably deliver both my incoming and outgoing legitimate messages without undue monitoring or interference with my own business?
Use your own servers and you can do whatever you want. Again, you can't force others to accept email from your mail servers, but you can choose to accept or reject whatever you want from others. No one can stop you from sending mail from one mail server you own to another mail server you own.
> Which is exactly why some of us are in favour of statutory regulation to compel anyone participating in such an important technological ecosystem to be a good citizen.
You can't really regulate the internet. If you could enforce regulations on a global network made up of discrete but interconnected networks we could just make spam, phishing, and hacking illegal on the internet, enforce that law/regulation and there would be zero need for blacklists. Because laws and regulations don't work on the internet we instead have to come up with blacklists, filtering technology, and other tricks to keep the internet even semi-functional.
> My objection, which seems to be in line with the submitted article, is to big mail services that think spraying fire into a crowd of 250 indefinitely because there was once one bad person there is a reasonable response to the problem
It's the only one that works. I've seen with my own eyes ISPs who didn't care enough to invest at all in abuse handling, but were forced to because of being blacklisted and in order to keep their customers they had to clean up their network, pay attention to abuse notices, participate in feedback loops, and slowly rebuild and maintain their reputation as responsible network operators.
If you limit blocks to individual IP addresses than spammers just cycle IP addresses. ISPs that ignore anything sent to their abuse@ address (if they even have one) never have any pressure to invest in preventing spam and can just keep accepting money from spammers and hackers and give them new IPs whenever they need to.
IPv6 makes the problem much much worse since a single spammer would get a huge amount of IPs to burn through before they have to bother their ISP about it. Blacklists themselves could become so massive and cumbersome that restricting larger and larger ranges may be the only option.
Can you imagine what would happen if we applied your argument to other important communications channels like postal mail or telephone calls? Sorry, someone in your old friend's city was using a robodialler so now none of the local phone service providers available to you will accept calls from anyone in that area code.
We absolutely can regulate the Internet on this kind of issue. We don't have to regulate everywhere in the world to make a big improvement, just businesses above a certain size that operate a commercial email service. If our governments can effectively lean on social networks enough that they add warnings to potentially misleading comments about science, they can lean on email services to do better with this problem. They only difference is that there is an obvious and unambiguous way the mail services could do a better job.
And again, just to be crystal clear, I am not arguing for giving real spammers a free pass. I am only arguing for credible, realistic measures to try to avoid the huge numbers of false positives we get from mail filtering today.
> Can you imagine what would happen if we applied your argument to other important communications channels like postal mail or telephone calls?
The only reason we don't is because unlike email, it's the sender who pays not the receiver. Telecoms do monitor and block outbound international calls if the connection times are excessive, if they occur at unusual hours, or if they going to certain "blacklisted" countries where phone fraud is common. They do it because hackers will break into a business's PBX and use it to place a bunch of international calls and the business suddenly gets a massive phone bill. They call their phone company about the changes, the phone company waves the changes (once) but that leaves the phone company on the hook for them. When false positives happen, the business has to call into the phone company and explain the calls were legit and they will be whitelisted and similar outbound calls will be allowed going forward.
I wouldn't oppose using regulation in the US against US based mail services if it meant forcing them to do a better job preventing spam from leaving their networks, but I'd be hesitant to support legislation forcing them to accept more spam. Maybe the largest ones could be pressured to invest more money in handling the influx of spam after they accept it, but I'm guessing there would be costs to consumers such as long delays in delivery, or "free" services like Gmail suddenly requiring payment or closing their services for good. At the ISP I work for now we stopped hosting our own mail servers and outsourced email services to a third party because spam filtering was too expensive and time consuming, and now we're looking at possibly no longer offering an email product at all and telling all of our customers to migrate to services like gmail and yahoo. Killing our email service today would eliminate a lot of problems in terms of help desk calls, phishing attacks, and spam problems. Make it too much harder for people to provide email service and there may only be giant providers left.
Other guy sounds like a giant dick-wad - we should not be wholesale blocking IP ranges without recourse to "unblock".
Whatever the other guy thinks about it being "necessary" or whatever, there is not commonly a way for a user to whitelist a service. And services providing email dont normally take that sort of signal into account, either.
Once you are operating a large system that is used by many people, you become a public utility - furthermore, at that scale we can generally find where you live and come lock you up. This kind of thing is 100% regulatable.
Either let users choose what mail they receive, or implement regulation forcing compliance. If that doesnt happen, and you snub my lawyer like the irresponsible mega corp you probably are, guess thats one more reason for me to polish off my shotgun and takeout the dickwads running the megadoom corp.
> The thing is, if you own a good IP in a mixed block with some bad ones, that's no reason for you to be blacklisted. It's pure laziness.
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive. It's to incentivize the sending MSP to remove their spammer (or move them to address-space where they can be blocked without causing collateral damage).
It's not laziness; the intention of collateral blocklisting, as with UCEPROTECT L2 and L3, is punitive.
Unfortunately the people it punishes are the legitimate users of both systems who only wanted the system to do its job and let them communicate. The bad actors will just move on and abuse another system instead.
> who only wanted the system to do its job and let them communicate
"The system" you are referring to consists of a bunch of private networks. Your opinion about what the job of those networks is may not coincide with the opinions of the operators of those networks.
Email is not a public service, and there is no entitlement to send whatever "vital business communications" you like to anyone you want. It's not even reasonable to require a postmaster to state what their rejection policy is; that would just tell spammers what they have to do to evade your blocks.
If email doesn't work for your business, then switch to another channel, such as huge billboards or whatever. Ranting about blocklists isn't going to help, people have been doing that for two decades.
