I agree wholeheartedly. Almost every time I hear from someone who is upset about the GIL, I find that they would be much better suited to using multiprocessing instead of multithreading.
With 80% of the developers out there, they are basically assured of producing better, more stable code this way.
Python's "multiprocessing" means launching another Python interpreter in a subprocess. Each process has a full copy of the Python environment. They may share the base interpreter, but there's a separate copy of every package loaded and all data. Memory consumption is bloated and the CPU caches thrash. Launching a subprocess is expensive; it means a full interpreter launch and a recompile/reload.
"Multiprocessing" is useful when you have a lot of work to do concurrently and not too much data to pass between processes. I've used Python subprocesses that way. Parallelizing your number crunching is probably not going to work very well.
> Parallelizing your number crunching is probably not going to work very well. [...]
The question is, what exactly does "number crunching" mean? We do aerial imagery analysis, so image processing in essence, which I would classify as a "number crunching" problem. A common thing e.g. is to do a time-series analysis and you can simply start multiple (2, 4, ..., N with clusters, etc.) processes for each problem. Obviously this works because most methods are computation and/or memory heavy - the additional memory requirements and "overhead" of Python itself (IMHO people overestimate the weight of starting new processes instead of threads) is completely dwarfed by the requirements (memory and CPU) of the method itself.
...which is true, but doesn't mean you can just ignore it.
Interpreter state is among the most frequently accessed memory in many applications, meaning it's ideal to have it in cache. The difference between two interpreter states and one might not be big compared to the data being processed, but it's big enough to bump a lot of interpreter state out of cache, which for many programs can have drastic performance implications.
If you don't think cache locality is important, look at radix sort versus quicksort. Radix sort has a much lower O, but performs worse in most cases because of its poor cache locality.
Look, I get that there are fairly easy ways to work around these problems, but let's not just blithely pretend they aren't problems.
Agreed, but there is a lot of misinformation about the topic. I met developers that thought the GIL prevents you from running your program in multiple instances at the same time on one machine - which is obviously not the case.
Sure, it's a problem for specific workloads, and Python will get there eventually - I just don't think it is a deal breaker.
The ``forkserver`` method eliminates most of the problems you mention: child processes are only started once, and they fork() from a totally separate process so they don't inherit all of the resources of the main process (in particular, they don't copy the whole heap). I've found this eliminates 90% of the performance-related issues I used to experience with multiprocessing.
If you're CPU bound (only reason to care about the GIL anyway), then you want one process per core. So at least the L1 memory cache isn't shared. The separate memory consumption is minimal (3-5MB*N cores).
You don't need to do setup/destroy more then once.
It seems like people never make this assessment, or use the GIL argument to put interpreted languages down. I personally run into I/O bound problems way more often than CPU bound ones. That said, I'm mainly doing things in the realm of a Python web developer. Scientists probably hit CPU bound problems more often with Python, but seem to drop down to C/C++ extensions without needing to complain about the problems.
No, no, if you're manipulating Python objects from C code, you have to hold the lock. You can release it only when not doing anything with objects in Python's memory space. Otherwise you get race conditions and intermittent crashes.
By using numpy, etc. you're basically _not_ using the interpreter because you're using C/C++/fortran code that's been compiled with python bindings.
To combine both your points, the best approach (if you like python) is to stick with python due to is ease of development and use libraries such as numpy as far as possible. However, if your use case is CPU bound but not served by those libraries, then you'll either need to develop your own extensions or throw away the interpreter altogether (and go with a different language).
>Memory consumption is bloated and the CPU caches thrash. Launching a subprocess is expensive
Statically and dynamically loaded binaries are resident in the kernel's page cache. Which while each process will have different locations within its process address space for each process (b/c ALSR), they _should_ be de-duplicated in RAM, ultimately all these seperate in process images will be pointing at the same physical RAM page(s).
So from a hardware cache standpoint you're mostly okay.
That's just the interpreter's executable. All the stuff that's generated from the Python code you load, and any data it generates, is unique to the process.
With Python that's a lot of stuff; I suggest running strace python some_small_script.py to see just how much data Python loads on every single startup.
The other issue with multiprocessing is that it requires the enclosing code to be pickleable, and many Python objects are not pickleable. For example, if I have a thread-safe RPC client and want to send thousands of RPCs using the client, I can't do that with multiprocessing (subprocess pool; threading pools work). RPC clients manage a TCP connection, if you use multiprocess you end up having to make many TCP connections.
Absolutely agree. Almost all tasks will perform very well when using multiprocessing. It also has a nice side-effect of steering you towards explicitly coding data flows without fine-grained sharing.
If you need to close that gap between the performance of multiprocessing, and multithreading, then you probably shouldn't be using Python, or any language of the same shape, in the first place.
There is one other option I'd like to see: multiprocessing style, but with multiple Python interpreter instances in the same process — one per thread. There would still be the hard delineation of data boundaries between instances, but less overhead for pushing data between them.
> If you need to close that gap between the performance of multiprocessing, and multithreading, then you probably shouldn't be using Python, or any language of the same shape, in the first place.
Unfortunately, these performance concerns often manifest well after the "rewrite it in a different language" date has expired. There are a lot of people in that boat, and they need better options.
> There is one other option I'd like to see: multiprocessing style, but with multiple Python interpreter instances in the same process — one per thread. There would still be the hard delineation of data boundaries between instances, but less overhead for pushing data between them.
If I understand correctly, the article discusses this ("subinterpreters"), but claims that there is no advantage to this approach vs multiprocessing. Presumably any overhead savings are eaten by GIL contention or some such?
I took it to mean that it is feasible. Instead of saying "well we used the wrong language, I guess we're screwed," you rewrite one component at a time, piece by piece, until the whole has been replaced.
This is the approach I try to use myself. It's nearly impossible to replace an entire system all at once. But replacing one part at a time is doable and you can see the improvements much sooner.
Lately I've found out that multiprocessing will not help you if your program is multithreaded. There is no sane way of forking a multithreaded program. For one, the child process will inherit a copy of all locks in the state they where at forking time, possibly causing random crashes and deadlocks.
It is possible to do more after a fork (cf. async-signal-safe), but it's hairy enough to just say — don't, always exec (similar to how doing actual work in a signal handler is generally a very bad idea).
If the child program is multithreaded then it's almost certainly not pure Python in the first place. So, wrap it up in `with nogil:` Cython statements and use the threading module (or concurrent.futures.ThreadPoolExecutor).
It doesn't if you need to manage atomic data across the processes, as there's no way to lock and block the other cache consumers (think the data you need to handle cache evictions, etc.)
Also, you're describing multiple python processes + an extra server (redis) process - as a "simpler" solution for the limitation that Python doesn't do multi-threads well.
Of course there are a ton of use cases out there where you can scale in other ways, but threads and shared memory exist for a reason - there's no reason not to call a spade a spade and say the GIL is still a limitation.
Blocking workers in a Redis queue is not hard... You can simply put them all on a pubsub control channel and then orchestrate them that way you need to do shit. Or literally just take down the processes, or the network, so they disconnect and stop BLPOPing the queue.
Cache evictions can be handled by Redis natively with TTL.
For retries and failure mitigation, you can still lean on Redis via BRPOPLPUSH/RPOPLPUSH.
If you want to scale beyond one machine, you can't rely on threading to help you. So why not just do it right to begin with, and use a parallel worker queue?
It's not a matter of the GIL being a limitation, a single machine is a limitation too. Don't blame your tools because you're misusing them.
As for threading in Python... on a single machine, for one reason or another... I would still rather use multiple processes, or at the very least, would just simply use eventlet and greenthreads.
Not saying it covers all use cases, it's not a silver bullet, and it doesn't replace threading natively, but damnit, it scales better, and it's the right way to do the task at hand.
Wasn't reading data from or using a redis queue, this wasn't a blocking queue issue.
Second, my use case wasn't a simple cache, I was omitting details. So redis having a TTL eviction policy for the values it stores is a moot point. The resources I was dealing with ranged from around 0.5GB to several gigabytes. That was the important working data - but whether or not these objects were available was what had to be coordinated (as well as some other bookkeeping data.)
Also, in this case - of course scaling beyond one machine was important. We were. The issue is that for each machine you allocate, you want to maximize usage of its resources. So each machine gets its own data cache, but nonetheless, we still wanted to max CPU usage per machine. So again, it's multi-process, vs. multi-thread, and in this case - multi-threaded with shared memory was a much easier paradigm than handling co-ordination among separate python processes.
I was just giving an example of reasons one would want true multi-threading in python. I wasn't trying to go into explicit details of an entire project. Please consider this when you reply to people and tell them they're "misusing their tools."
Of course it is - and that's what people do. The reason for the parent article is that there ARE people that would like to continue to use Python the language, and their existing source code/libraries, but would like not to deal with the GIL. Just because it is not a priority for you doesn't mean it isn't for others.
> Except when your use case requires a massive shared data cache that needs to be atomically updated
You can delete the last 6 words. Anything where multiple processes would have to read in/acquire a massive dataset to do some independent work qualifies. For instance, running some number (e.g., hundreds to hundreds of thousands) of analytical or statistical tests over a set to pick parameters, etc.
Your parent comment gives good advice, because the GIL is probably here to stay and so there's no use complaining about it. But the idea that multiprocessing gives better results than multithreading is ridiculous.
In languages which don't have a GIL, threads are almost as capable as processes, but lighter weight. Threads are almost always preferable to processes in most languages.
I understand why the GIL is still around, and don't necessarily support removing it, but it's definitely not there because it produces "better, more stable [Python] code".
> In languages which don't have a GIL, threads are almost as capable as processes, but lighter weight.
But also plagued with shared state concurrency bugs, something multi-processing completely avoids so...
> Threads are almost always preferable to processes in most languages.
No, they aren't. It's too easy to write buggy code with threads, it's a flawed model. Now it's certainly true that more people choose threads than processes but that's because they vastly overestimate their ability to write bug free lock based code. Processes are better.
1. In Python, Python provides mechanisms for communicating between processes. Literally the exact same mechanisms can be used to communicate between threads. So I'm not sure why you think processes are inherently safer than threads.
2. If we're taking about all languages, I'm really just not sure why you would assume threads imply locks. There are a ton of threading models out there which don't rely on explicit locking, and there are even some that don't use locking, period.
> So I'm not sure why you think processes are inherently safer than threads.
Because they remove the unsafe way of sharing state from the programmer. The issue isn't that state can be shared correct in threads, it's that it doesn't have to be done correctly and programmers are simply terrible at doing it right.
> There are a ton of threading models out there which don't rely on explicit locking, and there are even some that don't use locking, period.
It's not about locks, it's about shared mutable state. Programmers are bad at dealing with shared mutable state, regardless of how access is synchronized.
> Because they remove the unsafe way of sharing state from the programmer. The issue isn't that state can be shared correct in threads, it's that it doesn't have to be done correctly and programmers are simply terrible at doing it right.
Please read what I said before the part you quoted. In fact, maybe read the rest of the chain of comments--the topic of conversation is threads versus processes in Python, and threads in Python do not require you to use shared mutable state, locks, or any of the assumptions you've made. If you can write multiprocess code in Python, you can write multithread code using the same mechanisms for memory-safe interthread communication as you would for interprocess communication.
> It's not about locks, it's about shared mutable state. Programmers are bad at dealing with shared mutable state, regardless of how access is synchronized.
It's not about shared mutable state, because that's not what anyone was talking about before you brought it up, and there are plenty of threading models that don't have shared mutable state, too.
You're preaching to the choir here about locks and shared mutable state being bad, but it has nothing to do with anything that was being discussed before your showed up with a bunch of assumptions.
No one claimed it did. Threaded code is plagued with bugs, it was not claimed that implies all threaded code uses shared state. Your frustration is unwarranted.
The question of shared state vs. message passing is orthogonal to processes vs. threads. Both techniques can be and are commonly used in both situations.
No it isn't. Clearly both techniques "can" be used, but that one "allows" shared state trivially and one doesn't matters greatly; it is not orthogonal, you just don't grasp the point being made about the nature of the choice of abstractions and the problems that come with them.
When you're jumping between C/C++ code and Python code you don't care much about the GIL... until you have a GUI which needs to be kept responsive and needs the GIL to do so.
Ive done a fair bit of GUI development in Python, mainly using Qt and not hit any significant responsiveness issues. The multi-threading support in Python is perfectly fine for providing responsive switching between activities and event loops, as long as you don't have anything that locks hard for too long. But in that case you can always split that off into a separate process e.g. The way browsers nowadays run a process per tab.
I can see how using multiprocessing trumps threads for smaller programs. However it can become memory inefficient to have larger programs running in multiple processes, especially on servers with less resources.
If I run N versions of program that occupies 8mb of memory the memory footprint of the code is much less than N*8mb due to shared libraries/memory pages.
It's a factor, sure. But, one you should weigh with other factors to determine what is best.
If you have long running, computationally intensive code, with simple interactions, sure. Then multiple processes is the right thing.
But sometimes you are writing a GUI app, or some "real time" code [1]. You put blocking calls onto a different thread to keep the UI responsive. But then you find that still the blocking calls freeze the UI across threads due to the GIL.
Pure Python code is not the problem in this case - the GIL gets released between statements often enough. It is long running C code. You could release the GIL manually in there, but it is not done everywhere. Also, there are often calls that are supposed to be instant (like opening a file, or starting an async operation), that take seconds under bad conditions (when the network is down).
----
[1] well, with Python probably not in the strict definition of real time, but say you are controlling some external device
Those reasons aren't what they used to be, resources aren't nearly as limited these days and we now have the hindsight to see that threads lead to very buggy code due to shared state. Processes are better.
With 80% of the developers out there, they are basically assured of producing better, more stable code this way.