Location: USA - San Antonio
Remote: Yes | Hybrid
Willing to relocate: No
Technologies: Azure, AWS, Linux, TypeScript, NodeJS, Python, Ruby, C#, MySQL, Java, Groovy, Postgres, React, VueJS, Twilio, LLM
Résumé/CV: https://www.linkedin.com/in/gregzapp/
Email: [email protected]
Product engineer with extensive experience in cloud engineering, SRE, and full-stack product development including at Rackspace, Rundeck, and PagerDuty. Seeking high impact and ownership roles.
My most recent role was Principal Engineer at HungerRush where I led a number of major initiatives, including designing and implementing their next-generation conversational voice AI ordering platform (OrderAI).
That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking..
One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.
It worked out pretty well for Go. Nothing else besides Erlang cracked the parallelism UX/runtime-performance code as well as Go did, and it didn't have the drawbacks of Erlang.
> rule broken by almost every language in common use
I don't think so. Go/Erlang/Java do break it (Java very recently). C/C++/Rust/C#/JavaScript/Ruby/Python/Perl/TCL don't break it. PHP ... sort of breaks it, (IMO they just hide cooperative nonparallel concurrency in a nice API, but I'm not well-informed on php)?
I think you're confusing userland threading (also known as green threads) with a regular thread pool implementation (that uses regular OS threads).
Because most web servers use OS threads. Nginx, Apache, litespeed, they all spin up `core count` threads but they can all work single threaded just fine as well.
No, I mean that Tokio, Go, etc., were built with general-purpose web servers in mind, and they decided that lightweight threads were necessary for that purpose because OS threads had too much overhead to be used per-request.
NGINX uses a state machine per task, which as far as I know is kind of what Rust's async functions end up desugaring to.
Tokio's not a lightweight thread/green thread system. It's an event loop, just like Javascript's, optionally with a fixed number of threads running work underneath.
For Go: you're likely right that OS thread overhead was the main reason for Goroutines, but I don't think it was the only one. There is also a ton of ergonomic efficiency to be had via green threads; like async/await, they're kind of a syntactic unlock that allows you to very liberally use goroutines/channels as a means of semantically modelling actors/state/data flow in your program, where doing that with regular OS threads is both dangerous and (given the start/stop overhead of threads on lots of other platforms) feels like kind of using a howitzer for a flyswatter. Golang saying "yeah, create a million of these, we don't care, it's 4 machine instructions per to start one" was pretty liberating for a lot of people.
The goroutine architecture also unlocked some nice performance at the OS level: GC latency improvements became easier, and some IO multiplexing efficiency a la epoll became almost free to implement due to the ability to cheaply move and suspend goroutines.
> Tokio's not a lightweight thread/green thread system
It's not green threads (and maybe not lightweight threads; there are so many names for things that may or may not be considered the same...) in the sense of stackful coroutines, but either way, it's a userspace implementation of multitasking on top of a smaller number of OS threads, which is the part that's relevant to my question.
Reasonable clarification. My response to your original "never do this" is a is a bit more specific then. There are two broad userspace concurrency models: cooperative and preemptive. Cooperative is a "sometimes the right call, sometimes more trouble than it's worth" situation; I can see what you're getting at there. But you're flat wrong about preemptive.
Cooperative concurrency: it depends. It's sometimes the right move, but has drawbacks. It's usually implemented to enable efficiency or partial parallelism of kernel-space code (IO). There are plenty of platforms that absolutely need it for that purpose: c10k, nodejs's rise to fame, all that. It behaves poorly when mixed with non-cooperative (blocking) code, and is poorly understood by many folks who expect userspace code to be parallelized as well. A common claim is that cooperative concurrency is harder to use than pre-emptive, but that's not true: some common things are harder, but some less common, but more dangerous things like race conditions tend to be much easier. Tokio and other cooperative systems that schedule onto fixed kernel threads can help with the hard parts a bit, but it's a partial mitigation at best.
Preemptive userspace concurrency: firmly disagree with your original point. Green threads are usually combined with userspace-code parallelism (e.g. goroutines scheduled onto multiple threads) and efficient kernelspace interactions (IO multiplexing). In the popular implementations of green threads I know well (Erlang and Go), this is an unalloyed blessing, in that it both removes many of the dangerous edges of threaded programming, facilitates increased efficiency, and lets users semantically model concurrency/parallelism with much less fear of e.g. creating too many threads, torn reads, etc. That's a big benefit: now we can model things like actors or CSP as subprograms written in procedural, synchronous, ordinary-looking code without getting Ph.Ds in operating system internals or having to carefully reason about complex interlocking layers of callbacks and futures to minimize risk of blocking or deadlock. In many ways, this brings the good parts of single-responsibility/what folks call "the UNIX philosophy" into a single process with minimal pain.
Rare cases where the userspace-concurrent runtime fights with your ability to interact directly with the OS, as with Docker's fork/thread/process-global-state challenges, do emerge. But I don't think those are primarily because of userspace concurrency; rather, they're because it's hard to do low-level things in a high-level, managed runtime. You can fork in the JVM, too, and you'll have problems. Different ones, because the runtime's different (e.g. endless RSS burn at idle due to the child GC moving parent objects), but problems all the same. Same for Python: forking in the presence of regular OS threads and/or a running event loop is equivalent to arming a minefield; doable but troublesome, just like it was for the Docker folks with Go.
To be fair, some of the same pains as the cooperative model appear with pre-emptive, too, when userspace concurrency systems are retrofitted onto runtimes that do a lot of other kinds of concurrency as well--but that's because of the hybrid/retrofit nature of the product, and not because of green threading as a concept. Java's rocky adoption of virtual threads is an example of that.
This advice comes from threads on Linux being second-class citizens. As TFA itself states, there are some syscalls that really should only be made in a single-threaded process which is frankly insane... but that's the legacy of starting with the processes-only model and patching it years later to have threads as well.
Shameless plug of my Typescript C# port. It's a case study with some interesting (perhaps) notes from me in the README about approach, performance, and cost(~$75).
I'm going to do a taboo and comment on my own comment.
I didn't push performance in a way that would cause big deviations from the Golang project's algorithms and data structures; similar to how the TS team didn't want to deviate too far from the JavaScript source till now. Aggressive unmanaged arenas and object pooling was the direction I would have pursued..
The GC in DotNet is a source of major frustration for a tool like this. Without tuning lots of gen 0/1 work occurs to determine if objects should be promoted. The tuning is still not an ideal solution IMHO. Fully unmanaged arenas and aggressive object pooling could help here, but an alternative GC may be part of an ideal, comprehensive solution.. I have have a play with ZeroGC.
There is a warning on the repo that it's not maintained, but I've been curious about the process of updating it to sync "upstream" changes. And it sounds like the TypeScript team are going to start incorporating some of the performance techniques identified in the Rust port in the 7.1 release so... I'll probably at least sync it to up to 7.1 and keep it as an active experiment and case study on LLM porting and DotNet AOT performance.
The benefit of those enhancements being wrapped into the reference implementation is it's "easier" for humans and LLMs to reconcile and keep things in sync.
Analogue hole is how I did a more efficient smart garage door hack than what people were talking about here at the time. Soldered zigbee switch into the physical button; worked great :D
The Rust port struck a cord with me. I had always been curious how a C# implementation would stack up against the Golang project in performance. And I was curious how cheaply the port could be done..
I know C# is a more direct port than Rust, but I'm still impressed how cleanly and cheaply it was possible to execute on.. And I learned some stuff :)
I'm doing this right now with Codex. It's currently working on porting the Binder.
7-11 phases to go and so far it looks like it might only cost $100. I'm focusing on efficient execution strategy minimizing token in/out, turns, and rework.
Will prob post up something interesting when it's done.
Location: USA - San Antonio
Remote: Yes | Hybrid
Willing to relocate: No
Technologies: Azure, AWS, Linux, TypeScript, NodeJS, Python, Ruby, C#, MySQL, Java, Groovy, Postgres, React, VueJS, Twilio, LLM
Résumé/CV: https://www.linkedin.com/in/gregzapp/
Email: [email protected]
Product engineer with 15 years of experience including Rackspace, Rundeck, and PagerDuty. As a full-stack, kitchen-sink product engineer I'm as comfortable running with unsafe pointers as working on React front ends. I'm seekign IC leadership roles with high impact and ownership.
My most recent role was Principal Engineer at HungerRush where I led a number of major initiatives, including designing and implementing their next-generation conversational voice AI ordering platform (OrderAI).
Location: USA - San Antonio
Remote: Yes | Hybrid
Willing to relocate: No
Technologies: Azure, AWS, Linux, TypeScript, NodeJS, Python, Ruby, C#, MySQL, Java, Groovy, Postgres, React, VueJS, etc.
Résumé/CV: https://www.linkedin.com/in/gregzapp/
Email: [email protected]
Seasoned software engineer with heavy startup experience and a focus on lean process, velocity, and business outcomes. 15yoe including Rackspace, Rundeck, and PagerDuty. As a full-stack, kitchen-sink product engineer I'm as comfortable running with unsafe pointers as working on React front ends. I look for high impact, ownership, and autonomy roles.
My most recent role was Principal Engineer at HungerRush where I delivered a number of high impact projects, including most recently their next-gen conversational voice AI ordering platform (OrderAI).
Location: USA - San Antonio
Remote: Yes | Hybrid
Willing to relocate: No
Technologies: Azure, AWS, Linux, TypeScript, NodeJS, Python, Ruby, C#, MySQL, Java, Groovy, Postgres, React, VueJS, etc.
Résumé/CV: https://www.linkedin.com/in/gregzapp/
Email: greg (dot) zapp (at) gmail
I'm a serial startup engineer with 15yoe including Rackspace, Rundeck, and PagerDuty. As a full-stack, kitchen-sink product engineer I'm as comfortable running with unsafe pointers as working on React front ends. I look for high impact, ownership, and autonomy roles.
My most recent role was Principal Engineer at HungerRush where I delivered a number of high impact projects, including most recently their next-gen conversational voice AI ordering platform (OrderAI).
My most recent role was Principal Engineer at HungerRush where I led a number of major initiatives, including designing and implementing their next-generation conversational voice AI ordering platform (OrderAI).
reply