CVE classify a lot of things that have nothing to do with security.
Not having a Message-ID can cause problems for loop-detection (especially on busy netnews and mailing lists), and with reliable delivery status notification.
Dealing with these things for clients who can't read the RFC wastes memory and time which can potentially deny legitimate users access to services
> It seems that Gmail is being pedantic for no reason
Well, gmail does not manage usenet groups and mailing lists. Delivery status notifications are considered best effort so it wouldn't make sense to block messages for that case.
Additionally, Gmail adds its own message identifier on every message (g-msgid) because it knows that message ids can not be trusted to be unique.
Finally just calling me ignorant is the cherry on top – please try to keep things civil on here.
So add a message id at the first stop, or hard ban the sender server version until they confirm. A midway point that involves a doom switch is not a good option.
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.
If the client SHOULD do something and doesn't, and your server does not know why, you SHOULD disconnect and move on.
If the server has considered fully the implications of not having a Message-ID header, then it MAY continue processing.
In general, you will find most of the Internet specifications are labelled MUST if they are required for the protocol's own state-processing (i.e. as documented), while specifications are labelled SHOULD if they are required for application state-processing in some circumstances (i.e. other users of the protocol).
> If the client SHOULD do something and doesn't, and your server does not know why, you SHOULD disconnect and move on.
That is not a rule.
In this situation the server can reject any message if it wants to, and not doing a SHOULD tests the server's patience, but it's still ultimately in the "server wanted to" category, not the "RFC was violated" category.
The RFC is a request for comments. The specific one in question is about Internet Mail.
If server implementers want their mail to be delivered these are things they SHOULD do.
That's it.
It isn't something you can give to your lawyer, and nobody cares about your opinion about what you think "should" means you can make someone else do. This is how it is.
How does Google know whether or not the sender has a valid reason? They cannot know that so for them to reject an email for it means they would reject emails that have valid reasons as well.
How would the sender know the consequences of sending without the header? You shouldn’t assume anything here. As a sender, you should include it unless you’ve already worked out what the recipient is expecting or how it will be handled. Doing this with email is silly because the client is sennding to so many different servers they know nothing about so it’s basically a requirement to include it.
It is not required for the protocol of one SMTP client sending one message to one SMTP server, but it is required for many Internet Mail applications to function properly.
This one for example, is where if you want to send an email to some sites, you are going to need a Message-ID, so you SHOULD add one if you're the originating mail site.
> How does Google know whether or not the sender has a valid reason?
If the Sender has a valid reason, they would have responded to the RFC (Request For Comments) telling implementers what they SHOULD do, rather than do their own thing and hope for the best!
Google knows the meaning of the word SHOULD.
> it means they would reject emails that have valid reasons as well.
No shit! They reject spam for example. And there's more than a few RFC's about that. Here's one about spam that specifically talks about using Message-ID:
> If the server has considered fully the implications
The server "considers" nothing. The considerations are for the human implementers to make when building their software. And they can never presume to know why the software on the other side is working a certain way. Only that the RFC didn't make something mandatory.
The rejection isn't to be compliant with the RFC, it's a choice made by the server implementers.
Either the server must explicitly confirm to servers or the clients must accept everything. Otherwise message delivery is not guaranteed. In the context of an email protocol, this often is a silent failure which causes real-world problems.
I don’t care what the protocol rfc says, the client arbitrarily rejecting an email from the server for some missing unimportant header (for deduction detection?) is silly.
These changes MUST NOT be applied by an SMTP server that
provides an intermediate relay function.
That's Google in this situation.
> Stop hiding behind policy and think for yourself.
Sometimes you should think for yourself, but sometimes, and friend let me tell you this is one of those times, you should take some time to read all of the things that other people have thought about a subject, especially when that subject is as big and old as email.
There is no good reason viva couldn't make a Message-ID, but there's a good reason to believe they can't handle delivery status notifications, and if they can't do that, they are causing bigger problems than just this.
I made an attempt to enumerate them[1], and whilst I catch this issue with feImage over a decade ago by simply observing that xlink:href attributes can appear anywhere, Roundcube also misses srcset="" and probably other ways, so if the server "prefetched every image" it knew about using the Roundcube algorithm the one in srcset would still act as a beacon.
I feel like the bigger issue is the W3 (nee Google). The new HTML Sanitizer[2] interface does nothing, but some VP is somewhere patting themselves on the back for this. We don't need an object-oriented way to edit HTML, we need the database of changes we want to make.
What I would like to see is the ability to put a <pre-cache href="url"><![CDATA[...]]></pre-cache> that would allow the document to replace requests for url with the embedded data, support what we can, then just turn off networking for things we can't. If networking is enabled, just ignore the pre-cache tags. No mixing means no XSS. Networking disabled means "failures" in the sanitizer is that the page just doesn't "look" right, instead of a leak.
Until then, the HTML4-era solution was a whitelist (instead of trying to blacklist/block things) is best. That's also easier in a lot of ways, but harder to maintain since gmail, outlook, etc are a moving target in _their_ whitelists...
Note that the API is split into XSS-safe and XSS-unsafe calls. The XSS-safe calls [0] have this noted for each of them (emphasis mine):
> Then drop any elements and attributes that are not allowed by the sanitizer configuration, and any that are considered XSS-unsafe (even if allowed by the configuration)
The XSS-unsafe functions are all named "unsafe". Although considering web programmers, maybe they should have been named "UnsafeDoNotUseOrYouWillBeFired".
> What I would like to see is the ability to put a <pre-cache href="url"><![CDATA[...]]></pre-cache> that would allow the document to replace requests for url with the embedded data
As to why I prefer one thing that doesn’t exist over another thing that doesn’t exist depends on my priors. You might as well be asking my opinion and making fun of it before you know the answer.
What do you think the impact would be if Content-Location: would be if it suddenly gained the interpretation I suggest?
What do you think a script in the package can do to reference a part of the URL is constructed by code?
Netflix might be spending as much as $120m (but probably a little less), and I thought they were probably Amazon's biggest customer. Does someone (single-buyer) spend more than that with AWS?
Hertzner's revenue is somewhere around $400m, so probably a little scary taking on an additional 30% revenue from a single customer, and Netflix's shareholders would probably be worried about risk relying on a vendor that is much smaller than them.
Sometimes if the companies are friendly to the idea, they could form a joint venture or maybe Netflix could just acquire Hertzner (and compete with Amazon?), but I think it unlikely Hertzner could take on Netflix-sized for nontechnical reasons.
However increasing pop capacity by 30% within 6mo is pretty realistic, so I think they'd probably be able to physically service Netflix without changing too much if management could get comfortable with the idea
A $120M spend on AWS is equivalent to around a $12M spend on Hetzner Dedicated (likely even less, the factor is 10-20x in my experience), so that would be 3% of their revenue from a single customer.
> A $120M spend on AWS is equivalent to around a $12M spend on Hetzner Dedicated (likely even less, the factor is 10-20x in my experience), so that would be 3% of their revenue from a single customer.
I'm not convinced.
I assume someone at Netflix has thought about this, because if that were true and as simple as you say, Netflix would simply just buy Hetzner.
I think there lots of reasons you could have this experience, and it still wouldn't be Netflix's experience.
For one, big applications tend to get discounts. A decade ago when I (the company I was working for) was paying Amazon a mere $0,2M a month and getting much better prices from my account manager than were posted on the website.
There are other reasons (mostly from my own experiences pricing/costing big applications, but also due to some exotic/unusual Amazon features I'm sure Netflix depends on) but this is probably big enough: Volume gets discounts, and at Netflix-size I would expect spectacular discounts.
I do not think we can estimate the factor better than 1.5-2x without a really good example/case-study of a company someplace in-between: How big are the companies you're thinking about? If they're not spending at least $5m a month I doubt the figures would be indicative of the kind of savings Netflix could expect.
We run our own infrastructure, sometimes with our own fincing (4), sometimes external (3). The cost is in tens of millions per year.
When I used to compare to aws, only egress at list price costs as much as my whole infra hosting. All of it.
I would be very interested to understand why netflix does not go 3/4 route. I would speculate that they get more return from putting money in optimising costs for creating original content, rather than cloud bill.
> I would be very interested to understand why netflix does not go 3/4 route. I would speculate that they get more return from putting money in optimising costs for creating original content, rather than cloud bill.
I invest in Netflix, which means I'm giving them some fast cash to grow that business.
I'm not giving them cash so that they can have cash.
If they share a business plan that involves them having cash to do X, I wonder why they aren't just taking my cash to do X.
They know this. That's why on the investors calls they don't talk about "optimising costs" unless they're in trouble.
I understand self-hosting and self-building saves money in the long-long term, and so I do this in my own business, but I'm also not a public company constantly raising money.
> When I used to compare to aws, only egress at list price costs as much as my whole infra hosting. All of it.
I'm a mere 0,1% of your spend, and I get discounts.
Of course netflix is optimising costs, otherwise it would not be a business, I just think they put much more effort elsewhere. They could be using other words, like "financial discipline" :)
My point is that even if I get 20 times discount on egress its still nowhere close, since i have to buy everything else - compute, storage are more expensive, and even with 5-10x discounts from list price its not worth it.
(Our cloud bills are in the millions as well, I am familiar with what discounts we can get)
How is this reasonable? At what point do they pull a Dropbox and de-AWS? I can’t think of why they would gain with AWS over in house hosting at that point.
I’m not surprised, but you’d think there would be some point where they would decide to build a data center of their own. It’s a mature enough company.
I'm largely just thinking $HUGE when throwing out that number, but there are plenty of companies that have cloud costs in that range. A quick search brings up Walmart, Meta, Netflix, Spotify, Snap, JP Morgan.
For some reason nobody seems to like this sorcery, probably because it combines the worst of all worlds.
But there's almost[1] nothing special about what the dynamic linker is doing to get those .so files into memory that it can't arrange them in one big file ahead of time!
> What if the library you use calls dlopen later? That’ll fail.
Nonsense. xemacs could absolutely call dlopen.
> There is no universal, working way to do it. Only some hacks which work in some special cases.
So you say, but I remember not too long ago you weren't even aware it was possible, and you clearly didn't check one of the most prominent users of this technique, so maybe you should also explain why I or anyone else should give a fuck about what you think is a "hack"?
Once upon a time I worked on a project where we SSH'd into a satellite for debugging and updates via your standard electronics hobbiest-tier 915mhz radio. Performance was not great but it worked and was cheap.
I haven't heard much about the ArduCopter (and ArduPilot) projects for a decade, are those projects still at it? I used to run a quadroter I made myself a while back until I crashed it in a tree and decided to find cheaper hobbies...
Well at least crashing drones into trees has never been cheaper hahaha. So it's super easy to get into nowadays, especially if it's just to play around with flight systems instead of going for pure performance.
Because TCP interprets packet loss as congestion and slows down. If you're already on a slow, lossy wireless link, bandwidth can rapidly fall below the usability threshold. After decades of DARPA attending IETF meetings to find solutions for this exact problem [turns out there were a lot of V4 connections over microwave links in Iraq] there are somewhat standard ways of setting options on sockets to tell the OS to consider packet loss as packet loss and to avoid slowing down as quickly. But you have to know what these options are, and I'm pretty sure the OP's requirement of having `ssh foo.com` just work be complicated by TCP implementations defaulting to the "packet loss means congestion" behavior. Hmm... now that I think about it, I'm not even sure if the control plane options were integrated into the Linux kernel (or Mac or Wintel)
> But it is also ebay's right to decide whether or not its computer will allow requests from your computer.
That is dangerous thinking right there: Ebay does not have rights.
Of course ebay may do it anyway, and it may take time for justice to correct things, but it is not Right, nor their right, to violate law even to protect themselves.
No, that is the actual dangerous thinking. Ebay enjoys the same freedom of association that you do. Their right to not do business with you is exactly the same as your right to not do business with them. It's the very same right you exercise every time you use an add blocker.
> Their right to not do business with you is exactly the same as your right to not do business with them.
You are incorrect about that. They are subject to the ADA. I am not.
As a publicly listed company they have a tremendous number of other laws that apply to them and not to me.
> It's the very same right you exercise every time you use an add blocker.
Exactly: As an accessibility tool, it is illegal for a company to deny service in the US (and Ebay is a US corporation, despite their Canadian roots) for the use of the tool.
Yes they do, as they should. Ebay is in an extremely competitive market and you have lots of other options, if you're abusing their service they need to be allowed to ban you. Imagine if Amazon wasn't allowed to ban scammers, or if they couldn't refuse a login portal to a user, allowing infinite attempts. It's important they get to decide whether to deliver a page to you, let alone keep you as a user.
If we were talking about some government-run water utility then sure, it would different, but a private online store can ban users without ruining their life, and if you're opposed to this new rule you should stop using them in protest.
> > Ebay does not have the right ... to violate law even to protect themselves.
> Yes they do, as they should.
No they should not, and I cannot believe you could say any such thing in good faith.
> if you're abusing their service they need to be allowed to ban you
Who said anything about "abusing their service"?
> Imagine if Amazon wasn't allowed to ban scammers
Nobody is talking about banning scammers.
Don't do this: Don't argue in bad faith. You can still disagree and think companies have the right to commit crimes, but you don't have to act like I'm saying something that I'm clearly not!
> but a private online store can ban users
Actually they can't, because we're now talking about users instead of scammers and abusers: There's something called the Americans for Disability Act, and it protects access to storefronts and no a private online store CANNOT ban users who need an accessibility tool.
CVE classify a lot of things that have nothing to do with security.
Not having a Message-ID can cause problems for loop-detection (especially on busy netnews and mailing lists), and with reliable delivery status notification.
Dealing with these things for clients who can't read the RFC wastes memory and time which can potentially deny legitimate users access to services
> It seems that Gmail is being pedantic for no reason
Now you know that feeling is just ignorance.
reply