When I was younger I would have agreed wholeheartedly with this article because he seems more knowledgeable. After a few years experience I would have disagreed with him. Now I'm experienced enough to realize I have no idea if he's right or wrong but he seems to make reasonable points.
I've only worked in I/O bound, memory bound, and CPU bound code before (but never at the same time.) My hats off to anyone or group that has to work in these types of situations. Guess that's why I'm not a kernel level developer.
I've had to sort out a lot of performance problems. Almost always they were algorithm problems, architecture problems, or some simple bottleneck. Only once have I encountered a performance problem which was best solved by writing in a lower level language. More than that, my experience says that people who brag about how they've designed for scalability have generally made stupid design mistakes that cost them huge amounts of performance.
In fact here is a performance tip. If you want performance, make sure you have verbose logging options written in. Because when you hit performance problems, it is incredibly valuable to flip logging on, take the logs, study them, and identify your performance problems that way. Try it. For most applications that will matter a lot more than what language you write it in.
In all of the performance sensitive systems I've worked on, "flipping logging on" in production is simply not an option for, well, performance reasons (i.e. the reason that it wasn't turned on in the first place). If you're then going to test in a non-production environment, why not just run it through a profiler rather than trying to divine performance problems from log statements?
Depending on your architecture, logging can be turned on selectively. Just log one process/thread/database handle/whatever. Sure that one will be slowed, but the rest of the system still runs at full speed so you're OK. This can be an invaluable tool.
For instance a tool I incorporated into one system would see a parameter to a web request and would issue a command to Oracle telling it to log everything that happened on the database for that connection, and then would turn it off afterwards. So, for instance, we could take a slow web page, add a parameter, and a minute later be studying a log generated by Oracle telling us exactly what that web request did, and where it spent its time.
Having the ability to selectively do this on the live production system against live data with a problematic request while it was being problematic was huge. We were tracking down problems that only showed up in production, under production load, so no amount of profiling in development would have helped. Using the same idea, every day we would just take one random database handle, turn logging on for half an hour, and use it as a canary to look for potential problems. We found a lot of things that way.
Addendum (added later) It is also worth noting that in many horizontally scaled systems you can trivially have a fair amount of logging, even in production, if you're willing to accept a constant factor overhead in inefficiency. This can be utterly invaluable in tracking down latency, bottlenecks, and other larger scalability problems. Every large system that I've seen that was well-run did this to some extent.
A data point: the first version of memcached was written in perl. It was pretty good perl, with an async epoll-based event loop and everything. It was hopeless, orders of magnitude away from the desired performance.
I'm not saying that there are not good examples where performance matters. In fact I encountered one where I had to move something from stored procedures in a database into C++. Furthermore I can easily believe that memcached is one of them.
But that doesn't change the fact that most of the places that I have seen worry about it have not been among them, even if they thought they were. (I admittedly mostly work in the web world.)
I felt that article had a point, but overstated it.
There are lots of domains -- simulation, computer vision, robotic control, machine learning -- which are CPU bound and will remain so for the next decade. In fact, one of the cutting edges of these domains is always CPU bound, practically by definition (sensor densities also increase quickly with time).
I wonder if there are any startups out there who hire folks with "kernel developer" mentality/experience. I love dynamism and excitement of startup life but unfortunately it often comes with Ruby/JavaScript, which is fine but I prefer lower-level hacking: hardware interrupts, malloc-free environment, etc. I do believe startups who need such skills exist, they just seem to be quieter for some reason. :-(
Come to Austin Texas! When I lived there I met quite a few folks who were hacking on some cool hardware in their garages: embedded video analytics, signal processing, etc. Maybe its the "extreme proximity" to giant campuses of IBM/Motorola/Dell that favors a certain flavor of startup ecosystem, not sure...
Maybe not start-ups but I hear and I've seen postings at many hedge funds and their ilk for those that can do low latency algorithms. They're specifically looking for kernel level developers.
> When I was younger I would have agreed wholeheartedly with this article because he seems more knowledgeable. After a few years experience I would have disagreed with him. Now I'm experienced enough to realize I have no idea if he's right or wrong but he seems to make reasonable points.
Slightly offtopic, but one of the things I love about software is that it's trained me to recognize patterns in everything. Abstract thinking is really useful.
And I've recognized the same tendency you mention in myself, and across a few different topics.
It makes me wonder when I wholeheartedly agree or disagree with something... maybe I'm just still on that journey.
I think along these lines every time someone mentions the Dunning-Kruger effect, which is frequent these days on Hacker News. It reminds me of a funny aphorism I once heard about higher education:
First you get your Bachelor's Degree, and you think you know everything. Then you get your Master's Degree, and you realize you don't know anything. Finally, you get your Doctorate, and you realize that nobody knows anything.
Finding those patterns between disciplines is always a delight--there is a surprising amount of crossover between many fields of study and human behavior. Programming was where I first found that humility is very positively correlated with competence, and the same principle shows up in a lot of other places.
Reading the article, he discusses some places where language choice doesn't matter and some where it would (the unpredictable latency you get with Garbage collector even if average speed is the same). So it's not refreshingly undogmatic.
One thing the article doesn't mention is that Java once had a slow interpreter and now it has a potentially faster interpreter. When Java had a slow interpreter, then it would be inherently slow for a larger spectrum of problem - but still not all of them.
The Java problem is not that JVM is itself slow, but that Java is so inconvenient to use, that people use tons of stacks of libraries to accomplish even simplest things (think dependency injection) that JVM becomes bloated by loading and calling all these libraries, and in effect is slow.
I've seen most ridiculous stacktraces only in Java.
I've only worked in I/O bound, memory bound, and CPU bound code before (but never at the same time.) My hats off to anyone or group that has to work in these types of situations. Guess that's why I'm not a kernel level developer.