I have the opposition opinion. I think generators should have had an explicit keyword for defining them, like:
gen def stuff()
Because it doesn't, people are very confused about generators, and think they are functions, which they are not. They are completely different objects.
Plus there is no way to tell if a definition is a generator without reading all of it.
async def foo() is not a function and it doesn't have the same signature: the async is very explicit.
It's a different primitive, and it's not used AT ALL for the same thing at a function.
It's like writting:
def foo():
yield "bar"
and saying, 'how do you cal that function? How do you know how to call that function? It's called the same thing. It has the same signature.. It returns the same thing. One has one exit point, the other has several. This is "colored functions".
You have to call the 2nd one like:
next(iter(foo()))
It's absurd'
Well, no, it's not. It's a different tool. It's not a function, it's a generator.
You don't have to care about async/await. In fact, you can live your entire life without using them.
But it's a handy tool that allows you do:
- delegate asynchronous behavior, like I/O, outside of your callable
- make you match the syntaxic order with execution order despite said delegation
- make it clear and explicit where you chose to do so, indentifying the context switching of your program
Does it seems complicated ? It's because it is, and you probably don't need it.
But if you need it, it's easier than the alternative of using callbacks, and less error prones that implicit asyncronicity.
async / await are just primitives, they are, in no way, incompatible with the rest of Python code.
At worst, if you mix blocking and non blocking, you loose the performance benefit, that's all. It all works. You are just back to non async execution.
You can use async functions in sync code and vice-versa.
In fact, you can use async / await for asyncio, or another lib, or no lib at all, it's a completly lib agnostic concept. It doesn't even need any io. async / await is just a "yield from" which understands __await__.
It's an interface. A contract. Like generators.
What's more, the python community is now following the best pratice of creating Sans I/O libs (https://sans-io.readthedocs.io/), which means you can plug in whatever I/O lib you want.
There is a lot of confusion around asyncio, partly because of the communication and doc around it, partly because it sucked up until python 3.7, and partly because a lot of python devs have absolutely no need for it and see it from the point of view of their day to day task.
But it's definitly not fiasco, it's quite a handy tool, and the generic interface of async / await is nifty and allows cool designs.
Asyncio rocks. It’s so easy to create performant little apps that can get put up without a web server in front of them.
Type hinting rocks. Use it properly everywhere and you can a lot of confidence and speed when refactoring.
I use attrs (I know there are alternatives) to get legit type checking on object fields. Yes it is runtime type checking but if you write tests, those errors will pop up before you ever run the app.
Honestly I think the priority with Python should just be around digging deep into the implementations for performance gains. I’m very happy with how the ergonomics of the language are now.
Before 3.7, you had to be able to fetch the loop, handle its reference, pass it around, manage its life cycle, etc.
Starting from 3.7, only low level code (E.G: framework authors) would need to, the API is organized so that by default, you always perform something in the current running loop.
It's a big deal, and remove a ton of complexity. Maintenance is easier, learning curve is flatter.
The best example, is that in 3.6 and lower, you had to do "loop = asyncio.get_event_loop()", then call "loop.run_until_complete()". In 3.7, you use asyncio.run(). It seems trivial, it's really not:
This has consequences on the whole design, which is now full of things like that (run_in_executor, create_task, run_forever, etc), changing completly the way to write correct code.
Another thing is that in 3.7, the asyncio uses async / await and not yield from. Because of this dog fooding, the whole lib improved.
Speed (Future has been rewritten in C, and many other tweaks), fixes, _way_ better doc...
Obvious shortcomming were resolved, like asyncio.wait_for() that now waits until the cancellation is complete, or asyncio.all_tasks() returning only pending tasks. A lot of little things that add up.
The whole ergonomics are better, up to the command line experience with "python -X dev" which automatically add warnings and set asyncio in debug mode, and frankly you can't cleanly code async without.
The final nail in the coffin are the context vars (https://docs.python.org/3/library/contextvars.html). It's the only sane way to deal with a global registery that is dependant of the current running task.
It's almost always easier to upgrade to 3.7 than trying to use asyncio before it.
The idea is to (very slowly) remove access to direct CPython internals and give a handle to important low-level behavior through a standardized API hidding implementation details.
It's a lot of work, and going to affect a huge part of the ecosystem, but I'm hopeful this is something the extensions authors will actually want to be part of because it promises to solve a lot of problems:
- on the long run, this will make maintaining a c-extension much easier: more stability no matter the evolution of upstream, more certainty.
- there will be a clear way to do things, which means better documentation and conventions, potentially libs to do common things. Right now creating extensions is more a set of receipes people found out to work.
- because of this, scrutiny will be given to common patterns, which will be abstracted and optimized, and given the security of an API contract
- extensions using this API will work out of the box, or faster if they already did, on alternative python implementations like pypy
Of course, the end goal is to free the python core devs to experiment with breaking changes at the c level, such as changing the memory management model, and eventually maybe remove the GIL or allow for a JIT.
Oh yes, there are tons of giants creating software for health applications you never heard of, defense purposes you cannot hear about, logistic you would never consider (managing competitions, port services, private jet traffic, truck fleets...) and so on.
And those don't use $LATEST_TECH, $FANCY_FRAMEWORK or $EXOTIC_LANGUAGE.
They use $BORING_TECH, $OLD_LANGUAGE, $DEPRECATED_FRAMEWORK.
They are not in the cloud, don't update often, have to deal with legacy stacks, stacked on a stack of legacy stacks, and administrative burden from left to right.
Those are the ones that pay for an Oracle licence, and sync it to dozens of excel sheets from an FTP and a lone Access DB, where all auth is manage by a Tomcat service actually proxying to Active Directory and their custom CRM.
And they certainly don't have a team of 10x programmers. In fact, most are windows dev that never used the terminals and don't have admin rights on their machine.
I should know, I make half of my income going from company to company to train their team.
I'm pretty sure the virtual word here refers to the geometry being in a virtualized form, meaning that the renderer refers to it but the GPU does not have all of it in RAM but instead is paged in on demand.
This paper about geometry images[0] was mentioned on Twitter from an old post by the programmer who implemented the functionality in UE5. Geometry images encode geometry in rectangular 2D textures that can be reconstructed later (the paper is kinda old for this, but i guess it can be done on a computer shader nowadays). If those images are stored as virtual textures, which are supported by the GPUs nowadays, they could be essentially using sparse textures/resources to store geometry.
You have to balance "one obvious way to do it", "saying relevant" and "backward compatibility".
We broke compat once in 25 years to stay relevant with I/O, and gave 13 years to migrate.
And still, it was an outrage.
Let's see what Go looks like in 2034.