Does anyone know what the 2nd "processes" is referring to, in the 2nd paragraph here?
---------------------------
In a sane world, specification 30 words long, like the one above, would be implementable in 100 lines of C code. If it requires several thousand lines of code, there's clearly something broken with our programming model.
As I've argued in part I of this essay, to get a simple programming model you have to spawn a thread per TCP connection. But given that threads are expensive (and processes even more so) and that sharing state between threads is insane, bordering on suicidal, what you need are lightweight green threads and a simple communication mechanism to send messages between them. In UNIX of old they were called processes and pipes. In Go they are called goroutines and channels. In Erlang they are known as processes and mailboxes.
-----------------------------
I understand this:
"But given that threads are expensive (and processes even more so)"
but I am confused by this 2nd use of "processes":
"In UNIX of old they were called processes"
Was there an old version of UNIX that had lightweight threads? What were they? How were they implemented? How were they different from the processes we have now?
When I think of processes on Unix I think "threads are expensive (and processes even more so)". Was this different at some point in the past?
As to implementing new communication protocols, none of us should underestimate the complexities. Jon Postel was not stupid, but even he made mistakes. Recall that by 1973 they felt they had TCP 3 in a mostly final form, but then in August of 1977 Postel said they had made a terrible mistake by lumping too much together in TCP. He then spent a year pulling TCP and IP apart into the 2 separate protocols that we have now.
That example is well known, but if you read through the old RFCs, you realize there are some ideas that were explored and then abandoned. Among the really wild ideas:
"we are contemplating allowing sender and receiver to specify different byte sizes and consider bits as the basic unit "
The 2nd type of process is just an ordinary Unix-style process. It refers to the way that on older hardware process switches were cheaper on account of the smaller amount of state that needed to be shuffled about and cheaper system calls. So no need to (as you might today) prefer green threads over OS threads, and/or OS threads over processes, on account of the overheads.
This is mentioned in part 1 near the bottom:
When UNIX was still young ... you were supposed to fork a
new instance of the process for each TCP connection
I guess it made sense from performance point of view back
then. All the stuff that makes process switching slow today
haven't yet existed. ... In such an environment process
switching was closer to what we call green threads today.
I guess it made sense from performance point of view back then. All the stuff that makes process switching slow today haven't yet existed. Machines had single processor, there was no virtual addressing or memory caches. In such an environment process switching was closer to what we call green threads today.
So he is referring to hardware changes that made processes relatively more expensive than they used to be. I would think that the old machines still incurred some hit from a state change in the MMU. But I am far from an expert in that area.
Also, this statement seems suspect, because multiple cores properly used can make processes CHEAPER. Because there will be ZERO process-switching cost -- you have parallel processes. This is why I advocate an architecture based on heterogeneous threads or processes and message passing -- in contrast to process-per-connection, which implies homogeneous processes and little message passing.
The Go designers also advocate heterogeneous goroutines, toward the goal of program modularity rather than performance.
If you have heterogeneous threads/processes pinned to cores, that's pretty much the most efficient way of using your hardware. But writing this type of code is not so obvious without some planning, foresight, and working around language quirks.
He is referring to traditional chaining of programs through STDOUT and STDIN. For example, a "program" that counts number of occurrences of foo in a file:
cat file.txt | grep "foo" | wc -w
---------------------------
In a sane world, specification 30 words long, like the one above, would be implementable in 100 lines of C code. If it requires several thousand lines of code, there's clearly something broken with our programming model.
As I've argued in part I of this essay, to get a simple programming model you have to spawn a thread per TCP connection. But given that threads are expensive (and processes even more so) and that sharing state between threads is insane, bordering on suicidal, what you need are lightweight green threads and a simple communication mechanism to send messages between them. In UNIX of old they were called processes and pipes. In Go they are called goroutines and channels. In Erlang they are known as processes and mailboxes.
-----------------------------
I understand this:
"But given that threads are expensive (and processes even more so)"
but I am confused by this 2nd use of "processes":
"In UNIX of old they were called processes"
Was there an old version of UNIX that had lightweight threads? What were they? How were they implemented? How were they different from the processes we have now?
When I think of processes on Unix I think "threads are expensive (and processes even more so)". Was this different at some point in the past?
As to implementing new communication protocols, none of us should underestimate the complexities. Jon Postel was not stupid, but even he made mistakes. Recall that by 1973 they felt they had TCP 3 in a mostly final form, but then in August of 1977 Postel said they had made a terrible mistake by lumping too much together in TCP. He then spent a year pulling TCP and IP apart into the 2 separate protocols that we have now.
That example is well known, but if you read through the old RFCs, you realize there are some ideas that were explored and then abandoned. Among the really wild ideas:
"we are contemplating allowing sender and receiver to specify different byte sizes and consider bits as the basic unit "
This is from:
https://tools.ietf.org/html/rfc128
And if you try to create your own communication protocol, you will also go through a similar period of trial and error.