But if they fetch a lot of URLs from cloudflare and it takes 30 seconds to answer with a timeout instead of a http/200 under 20ms, then some architectural decisions that were sound under the latter case may make the whole system slow in the former one.
People have started testing network failures but so often fail to test slow network failures. All of a sudden your code has 500x more pending open sockets, both in and out, and the memory spikes and it all goes to hell. Even if the fast fail code path is indeed best effort.
I would love to see graphs showing "length of reply that eventually succeeded" - I suspect that most networks today, if you don't get a response in 5 seconds, you ain't never gonna get anything useful.
In other words, I wonder if going to fail fast would help the health of the Internet more than wait forever timeouts. Might reduce DDoS effects, too.
> When we plotted the data geographically and compared it to our total numbers broken out by region, there was a disproportionate increase in traffic from places like Southeast Asia, South America, Africa, and even remote regions of Siberia. Further investigation revealed that, in these places, the average page load time under Feather was over TWO MINUTES! This meant that a regular video page, at over a megabyte, was taking more than TWENTY MINUTES to load! This was the penalty incurred before the video stream even had a chance to show the first frame. Correspondingly, entire populations of people simply could not use YouTube because it took too long to see anything. Under Feather, despite it taking over two minutes to get to the first frame of video, watching a video actually became a real possibility. Over the week, word of Feather had spread in these areas and our numbers were completely skewed as a result. Large numbers of people who were previously unable to use YouTube before were suddenly able to.
If you're building complicated systems, you should probably reduce traffic to sites that are failing to respond, rather than continuing to send traffic, but timeout faster. Depending on what stage in the process the request is failing, it might not make a big difference (in a typical HTTPS exchange, the costs ramp the farther you go, processing a syn < processing a client hello < processing a complex request. (If it's a simple request, processing the client hello is more expensive than the request, of course).
If you send all the same traffic, and probably more because of retries with shorter and shorter timeouts, chances are you are going to keep the system overloaded, never detect success and never return to default timeouts. Dropping most of the traffic, and then turning it back on when the system recovers can lead to oscillation where the system works enough to drive more traffic that overloads the system etc, but at least you're getting some processing done.
Well, you do need, jitter, exponential backoff, caching, black and white listing, a stat base decision tree, etc. That's why it's a complicated and costly problem.
But if you are consuming a lot of API content, if you have crawlers or if you provide features like "get article title/summary/image/thumbs", at some scale it's an important decision to make.
I guess that comes with the type of end point in that network. A typical website, absolutely, yes, I'd agree. An API end point allowing requests of large data pools that might take a few seconds to generate but yet not a total time out would be acceptable.
There is no conclusion to the article, the weird syntax is just a mean to an end:
- This makes people curious, so they read.
- This gives an opportunity to show a lot of cool features of the language in a short format. As the post mentions, Python's learning curve is smooth but long, so it's nice to be able to introduce some tricks to coders that would otherwise not be exposed to them.
- This demonstrates those things indeed, as you said, work together. Even if Python is a language on which a lot of things have been bolted on after the fact, it's still decently integrated.
- This allow me to insert a warning about not abusing them because even if Python is a readable language, you can always go too far. It's good maintain this message in the community.
- It's just fun, really. It's an article I would enjoy reading.
Must admit I wasn't quite sure what the point of the first one was. 0xBEAF is literally just 48815, as is 0x_B_E_A_F, as the numeric literals now ignore interleaved "_", and you just format it as a float after-the-fact as normal in an f-string. So why? There's nothing particularly special in the use of syntax here, so hardly a curious or weird artifact, let alone abuse.
> * has a completely different meaning
I'd personally rephrase this, as the idea around splatting is that it symmetrically can "gather" and "spread" multiple values, albeit with the syntactic limtations of starred expressions (likewise with *).
You do seem to confuse syntax for semantics: [][:] = [...] is a semantic trick to avoid binding a sequence, if you did a = [][:] = [1,2] you'd find a == [1,2] because it just passes along the reference to the right hand side, so without another layer it's just skipping binding it, and that's the major trick with the second one, being that it "discards" the first value when destructuring this way (so long as that first value is a sequence). This can be done "better" with _, *_ destructuring, which probably weirds more people out to discover that shadowing in a statement is legal (and the unused/shadowed variable is "better" as it works even when the first value isn't a sequence, making [][:] more fragile). People with knowledge of other languages might get confused by _ being a variable, but that's a known stumbling block.
Naturally, everyone has their own journey through discovering Python (or whatever language) and may or may not touch upon many parts of it syntactically and semantically. However, as far as "weird" syntax that has valid semantics goes, I'd say this is far from esoteric or unusual (where's the walrus?), and some parts are just the natural expression (such as with the splatting). Really, the only code smell I find here is the inlining of everything and redundancy (i.e. splatting a generator into a list)!
I'm a proponent of investing in solar and wind energy, but it will be only cheap because we build solar panels and wind turbines using fossil and nuclear fuel.
Oil and urnanium have a miraculous energy density, and can be easily transported.
It's good we build alternative capabilities while we have cheap energy, but we should not forget why we can.
Well, fossil fuels are indeed very convenient. However I disagree that they are/were cheap. The costs are just hidden. I mean, that's the whole point we are having to replace them.
This is pretty far afield from the current conversation, but I think that there would be far far more support for Ukraine if Russia didn't have massive amounts of fossil fuels. Russia's ability to withhold them is a source of economic and political power.
It is absolutely related to the current conversation, though - fossil fuels as diplomatic leverage. Russia's economy couldn't support the war without trade in fossil fuels either.
I don't really see the point of remembering this. More salient perhaps is the behaviour of the oil companies: their cover ups of climate science and pollution, their lobbying, their massive, massive subsidies and their (frankly cruel) profiteering.
It'll be good that they're gone, when they're gone, and may we never have to rely on such morally unsound people or technology ever again.
Well, it's important because if we want to transition, we need a strategy and we need people to understand it.
If you can't understand energy density, you may be under the impression we can go 100% renewable right now.
But we can't sustain our current life-style with them today.
Also if you think oil companies are going to disappear after we migrated, you are going to be disappointed if you don't understand the role they play in systems.
> only cheap because we build solar panels and wind turbines using fossil and nuclear fuel.
This is not logical. We are building solar panels and wind turbines with fossil fuels becuase that's the energy source we have now, now becuase they are cheap.
Once we have enough energy from solar and wind, that energy will be cheaper than fossil fuels, and that input to their production will be cheaper, reducing the cost of wind turbines and solar panels further.
Fossil fuels have been made obsolete for most use cases, and we are building the industrial capacity to replace them as fast as we possibly can.
If you don't believe me, check out what the world is deploying for new electricity production. Especially in profit driven markets like Texas' ERCOT. Outside of ERCOT, most of the grid decisions are smoky-room deals with lots of corruption, and there fossil fuels tend to have a huge leg up, since more expensive electricity means a higher profit level for the utility.
Fossil fuel are dense, easy to transport, and extracting energy from it is super simple.
It's not the case for wind and solar.
Right now, we are calculating the cost of wind and solar energy by evaluating the cost of transport and manufacturing of very complex system powered by fossil fuel. We say they are profitable in this context, and only in this context.
If we were to transport and manufacture them solely self-bootstrapped by solar and wind, it would not be cost efficient.
I think most people underestimate how much energy transport and metal work uses. Every single electronic circuit, every single screw, tire and wire are made using fossil fuel. And the food, home, roads and heat systems of the people working on those are all powered mostly with fossil fuel.
Not just because it's a habit.
It turns out you can escape thermodynamics .
Energy density matters A LOT. If you take an entire cinema of air and pass it through a wind turbine, you get the equivalent of a drop of oil. That's how magical oil is.
Now it's quite possible in the future we improve the tech so much, and we have such an abundance of it that it will be a net win. I'm rooting for that. Oil has terrible side effects, no matter how awesome it is, we need to get rid of it.
But for now, we are funding the green revolution with fossil fuel.
Don't get me wrong, this is a smart strategy. There is no way around it.
However, putting our head in the sand is not going to help.
But if they fetch a lot of URLs from cloudflare and it takes 30 seconds to answer with a timeout instead of a http/200 under 20ms, then some architectural decisions that were sound under the latter case may make the whole system slow in the former one.