I love the simplicity of Node.js that each process or child process can have its own CPU core with essentially no context switching (assuming you have enough CPU cores).
Most other ways are just hiding the context switching costs and complicating monitoring IMO.
I like Node.js' simple and fully isolated concurrency model. You shouldn't be blocking the main event loop for 30 seconds! The main event loop is not intended to be used for heavy processing.
You can just set up a separate child process for that. The main event loop which handles connections should just co-ordinate and delegate work to other programs and processes. It can await for them to complete asynchronously; that way the event loop is not blocked.
I recall people have been able to get up to around a million (idle) WebSocket connections handled by a single process.
I was able to comfortably get 20k concurrent sockets per process each churning out 1 outbound message every 3 to 5 seconds (randomized to spread out the load).
It is a good thing that Node.js forces developers to think about this because most other engines which try to hide this complexity tend to impose a significant hidden cost on the server in the form of context switching... With Node.js, there is no such cost, your process can basically have a whole CPU core for itself and it can orchestrate other processes in a maximally efficient way if you write your code correctly... Which Node.js makes very easy to do. Spawning child processes and communicating with them in Node.js is a breeze.
> You shouldn't be blocking the main event loop for 30 seconds! The main event loop is not intended to be used for heavy processing.
This article is talking about an SDK that runs in users' apps. Users can run whatever code they want, so the SDK has to find a way to keep sending the outgoing heartbeats
My current project is the culmination 15 years of software development.
I started out building a full stack framework like Meteor framework (though I started before Meteor framework was created in 2012 and long before Next.js).
Then I ported it to to Node.js because I saw an advantage to having the same language on the frontend and backend.
Then I noticed that developers like to mix and match different libraries/modules and this was a necessity. The whole idea of a cohesive full stack framework didn't make sense for most software. So I extracted the most essential part of it that people liked and this became SocketCluster. It got a fair amount of traction in the early days.
At the time, some people might have thought SocketCluster was trying to be a more scalable copycat of Socket.io but actually I had been working on it for several years by that point. I just made the API similar when I extracted it for better compatibility with Socket.io but it had some additional features.
A few years ago, I ended up building a serverless low-code/no-code CRUD platform which removes the need for a custom backend and it can be used with LLMs directly (you can give them the API key to access the control panel). It can define the whole data schema for you. I've built some complex apps with it to fully prove the concept with advanced search functionality (including indexing with a million records).
I've made some technical decisions which will look insane to most developers but are crucial and based on 15 years of experience, carefully evaluating tradeoffs and actual testing with complex applications. For example my platform only has 3 data types. String, Number and Boolean. The string type supports some additional constraints to allow it to be used to store any kind of data like lists, binary files (as base64)... Having just 3 types greatly simplifies spam prevention and schema validation. Makes it much easier for the user (or LLM) to reason about and produce a working, stable, bug-free solution.
That said I've been struggling to sell it because there are some popular well funded solutions on the market which look superficially similar or better. Of course they can't handle all the scenarios, they're more complex, less secure, don't scale, require far more LLM tokens, lead to constant regressions when used with AI. It's just impossible to communicate those benefits to people because they will value a one-shotted pretty UI over all these other aspects.
> I've been struggling to sell it ... they will value a one-shotted pretty UI over all these other aspects.
I do not think that it's your competitors' pretty one-shotted UIs that are losing you your leads. FWIW your website:
* Does not explain the benefits nor the feature-set of what your platform offers
* Has a Docs page which seems to be a link to 6 sets of reference APIs (or similar) in disparate Github pages - that's worse than most open source products.
* Uses cute animated characters which does not give off the "we are a serious enterprise and you can rely on us" vibe
* Has a novel pricing scheme ("implements a techno-feudalist pricing model") that requires a 4 paragraph explanation and somehow tries to frame introducing additional uncertainty for potential buyers as a feature (it's a good thing that _my_ pricing tiers could be adjusted by other customers' votes!?)
* Additionally prominently features cryptocurrency in the topnav, which is for many people a yellow or red flag (regardless of what seems to be your good intentions behind it)
Doesn't have any demo apps I can click around in to smoke test the platform's functionality, let alone fiddle with the backend
* Has almost zero information on basic business-y compliance-y things - no info on security, availability, SSO support, etc let alone more hardcore things like compliance standards your platform meets.
Saasufy itself isn't open source. I'm planning to sell licenses of the code (a limited number of them to make it scarce). SocketCluster is a core component of Saasufy. The goal did evolve slightly; originally, it was to make it easier to build full stack applications. Now it actually lets you build entire full stack apps without code. That bigger goal has been achieved. I have some videos linked from the Docs page showing how it works.
But yes, I'm a bit paranoid about my situation. I do feel like my work is suppressed by algorithms. Things feel very different for me now than they did before in terms of finding users. It's really hard to find people to try my work. Difficult even to convince them to watch a 10 minute video. Though I guess many people are in the same boat right?
That makes more sense but I don't see what pub/sub has to do with a no-code full-stack framework. Other than that some of them might want a chat widget?
It's real-time by default. Done in a cheap (and efficient) way. The views update automatically when relevant data changes, no possibility of overriding conflicts when editing concurrently. Changes are only distributed to users/clients who are looking at the affected data.
It's not just for chat. Anytime people update data collaboratively, for any data-driven software, you run into issues of people overwriting each other's changes... Sadly users got really used to poor experiences like having to 'refresh the page', or 'locking down resources' during edits to avoid concurrent editing altogether.
It's been kind of disturbing for me to realize how few people care about data integrity in collaborative scenarios. It's a kind of hidden issue which can cause major problems but users tend to tolerate. I've had this experience first-hand with colleagues using various other platforms where someone else had the same resource open for a while in the browser, we updated different fields and my field was overwritten with its old value but we didn't realise for some time until it caused a bug with one of the workflows. Then it was like "I'm sure I updated that value."
SC's pub/sub channels are part of the secret sauce to make this work. SocketCluster makes some unique tradeoffs with its pub/sub mechanism; the channels are extremely cheap to create and are cleaned up automatically (and cheaply)... And at any scale with sharding. This was an intentional design decision which is different from message queues like Kafka and RabbitMQ which prioritise throughput and are typically used in the backend as opposed to being end-to-end.
I find that skills work very well. The main SKILL file has an overview of all the capabilities of my platform at a high level and each section links to a more specific file which contains the full information with all possible parameters for that particular capability.
Then I have a troubleshooting file (also linked from the main SKILL file) which basically lists out all the 'gotchas' that are unique to my platform and thus the LLM may struggle with in complex scenarios.
After a lot of testing, I identified just 5 gotchas and wrote a short section for each one. The title of each section describes the issue and lists out possible causes with a brief explanation of the underlying mechanism and an example solution.
Adding the troubleshooting file was a game changer.
If it runs into a tricky issue, it checks that troubleshooting file. It's highly effective. It made the whole experience seamless and foolproof.
My platform was designed to reduce applications down to HTML tags which stream data to each other so the goal is low token count and no-debugging.
I basically replaced debugging with troubleshooting; the 5 cases I mentioned are literally all that was left. It seems to be able to quickly assemble any app without bugs now.
The 'gotchas' are not exactly bugs but more like "Why doesn't this value update in realtime?" kind of issues. They involve performance/scalability optimizations that the LLM needs to be aware of.
Google buried my popular open source project deep in its search results. It's a very niche technical field with niche keywords but it shows all the scams and paid services on the front page and my project is not even in the first 5 pages when I type exact keywords that are present in the page. It only shows my project when I type its exact name.
It feels like Google is actively discriminating against my project in its algorithms. At this point, I wouldn't be surprised if there is some code in the google algorithm which is something like if (untrustedDomain(domain)) score -= x and they probably have some highly paid 'engineer' maintaining this blacklist. It definitely feels like this.
The key is to stay hands-on. I go for the tech lead role.
Actually at my current company I'm co-lead of a team of 10 people. I've been CTO and also the sole team lead in the past but I was always hands-on coding.
I'm glad I have a co-lead at my current company; in fact, I'm the one who recommended to the big boss that he promote both of us. Normally, I'm the sole leader but in this company, we deal with corporate clients and so there is a fair amount of compliance work, team coordination and stakeholder management and also the project itself is very complex on the tech side. AI adds a lot of complexity. My co-lead is really good with coordination, planning, meetings, stakeholder management and alignment but at the same time he has less experience on the tech side so I have the last say on technical matters like architecture and choice of technologies and I'm in charge of general-purpose modules, configurations, migrations, data management...
I think staying hands-on is very important, especially now with LLMs. Managing complexity is itself a separate concern from managing people. There is a human/psychological component to managing complexity but it's different than pure time management and coordinating work based on priorities. On a project with 10+ people, with AI, the complexity can grow rapidly and so it needs to be managed.
Initially, I was in the weeds quite often with my engineers. Over the years I’ve learned to maintain side projects that directionally follow what my teams are doing. It gives me some dispassionate separation and a unique look at things that my engineers appreciate.
And BEAM was the reimplementation of the Erlang runtime, the actual model is part of the language semantics which was pretty stable by the late 80s, just with a Prolog runtime way too slow for production use.
Yep, I had the same reaction. It was like. "Huh? What? Actual acknowledgement of contributions? Cannot compute." They even made the requirements just low enough for me to qualify. We'll see if I actually get the deal though but this could be the most generous thing that ever happened to me in the open source sphere. I have a tendency to fall through every possible crack so this is an actual shock to me.
Don't get me wrong, I definitely see the cynical side that Claude may potentially benefit from learning my high quality coding practices as a result of this... This is clearly also a way to source high quality training data. Maintainers of open source projects with 5K+ stars are among the most competent engineers you can find and they're not biased towards unnecessary complexity as most corporate folks are. The reason is simple; if you code for free, there is no incentive to maximize billable hours; it's the opposite. This is a real gold-mine of quality coding data. AI companies should be fighting over us.
But still, I think this is nice in either case. These days, I appreciate people using even cold calculated logic as a motivation for doing the right thing. I'm tired of people being irrational and doing the wrong thing because the wrong thing sounds more marketable to investors.
I don't know of any good backbone service or library with 5K+ stars, only npm or python hyped bozos. There is certainly not any competence in github stars. Aren't any GNU libs maintained on GitHub? No, they are mostly not. Just very few.
Yeah well it's like money these days; having a $10 million net worth doesn't actually say much about a person; maybe it fell on their lap because they knew the right people or maybe they had to struggle to earn every cent. It's the same with GitHub stars.
Unfortunately, those who had it easy tend to get much more attention and are much more visible; attention is how they got there in the first place so of course there is not much merit behind their work. A lot of software tooling is a Potemkin village. It's over-hyped and developers/users are forced into it by their boss who happens to be an investor in the project founder's company.
It often seems like nobody from Millennial or Gen Z generations built any good popular software tool or library... It's like nothing innovative came out since the time of Linux, GNU and GIT... No competent software developers since John Carmack? It's not true of course, it's just that we are a heavily suppressed and manipulated generation. Firstly, we are demoralized, so there are less of us actually putting in the effort to build quality stuff, but even those who do, our work is often marginalized and covered up by algorithms. Often nipped in the bud.
We have to consider that Linus Torvalds didn't build Linux all by himself. Had the community not come together and made all these distros, today, nobody would even know what Linux is and Linus Torvalds would be a failed developer living in the shadow of Bill Gates and Steve Jobs.
Most other ways are just hiding the context switching costs and complicating monitoring IMO.
reply