Yeah lets see how it plays out. I might have a look at rust at some point.
On the benchmark though, its doesn't seem quite fair. At least that first one is comparing a single threaded c++ program, to a multi threaded rust program. And for the regex example the author doesn't use C++ std::regex and std::futures (std::async) like he does for Rust. It makes me think a proper C++ implementation would do much better. (Personally I use the Parallel Patterns Library. I like it a lot. We will get something similar in C++ in a few years)
I don't know much about rust. It worries me though the examples are still using threads and mutexes. I think we need much higher level abstraction (like ppl::task, coroutines, etc) to get better scaling. Also lock free data structures dont scale that well either. As they still require synchronization and that hurts scaling). I think we need some kind of revolution in how we code to scale our programs to hundreds of cores.
> And for the regex example the author doesn't use C++
> std::regex and std::futures (std::async) like he does
> for Rust.
Rust's standard library implements neither regexes nor futures, so the code you're seeing must be coming from third-party libraries. If anything, this comparison would favor C++, since its own libraries are likely to be more mature and optimized than Rust's.
> It worries me though the examples are still using
> threads and mutexes.
You shouldn't be worried. :) C++ may require higher-level abstraction to make concurrency tenable, but Rust was designed as a concurrent language from the outset. Rust's type system prevents data races at compile-time, so programming with raw threads isn't nearly as fraught as it is in every other language and refactoring concurrent code can be performed with compiler-assisted confidence. I recommend Aaron Turon's blog post "Fearless Concurrency with Rust": http://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.ht...
> Also lock free data structures dont scale that well
> either. As they still require synchronization and that
> hurts scaling
This is another assumption that I suspect that Rust obviates. Let me recommend another of Aaron Turon's blog posts, "Lock-freedom without garbage collection": http://aturon.github.io/blog/2015/08/27/epoch/
That was the dream of the mid-2000s. But now hundreds of cores are never going to happen. The CPU vendors have decided that the industry has run out of time to parallelize their programs and are now refusing to scale up. Our hope for speedups now lies in using SIMD and GPUs effectively ("heterogeneous computing").
For what it's worth I actually think that Rust (and a smattering of other languages and tools) could push us to revisit the "hundreds of cores" thing again if these parallel-first/friendly tools get popular enough that CPU vendors see a market of highly parallel consumer-grade applications.
It is worth noting that the implementations for those benchmarks are contributed by a community, not all written by the same person (IIRC), so if someone is interested in using a language to the best of its hypothetical performance ceiling, they can definitely submit a solution.
I'm not sure about the details for the "fasta" benchmark, but I know it's at least partly I/O bound, and many implementations for the benchmark are single-threaded:
Notably, Rust also edges out a multi-threaded C implementation for that one. I'm sure they've done some very hairy optimizations to get to the top on that board, but it's cool to see that it's possible.
On the subject of regexes, I am not familiar with the C++ std::regex implementation, but it does look like (some rather slow) C++ implementations are using a boost regex library. Those are generally fairly "standard" in C++, right?
> Notably, Rust also edges out a multi-threaded C implementation for that one. I'm sure they've done some very hairy optimizations to get to the top on that board, but it's cool to see that it's possible.
The most important optimization I did was to avoid regex machinery as much as possible. In particular, Rust's regex library has very good support for prefix literal optimizations. In particular, regexes like `abc[x-z]foo|bar` will compile down to an aho-corasick[1] automaton over the strings `abcxfoo`, `abcyfoo`, `abczfoo` and `bar` with the failure transitions completely evaluated. (The end result is an automaton represented by a matrix. This is memory intensive, so only prefix literals of a certain size can be accommodated. But you don't need a lot to realize huge gains!)
Hint: most of the benchmark game regexes are relatively simple and compile down to either simple `memchr` calls (most regex engines will do that, nothing special) or a aho-corasick DFA that completely avoids the traditional regex evaluation machinery.
In general, regex implementations differ dramatically in the optimizations they perform. It's hard to draw conclusions about them without some explicit knowledge of how it is implemented.
The boost regex library is fast and widely used. It doesnt look like the example is using it though, but a library called "re2". Never heard of it before. Googling turned up this https://github.com/google/re2
There are 4 C++ benchmark submissions for regex-dna. One of them does indeed use boost, but it is significantly slower than RE2: https://benchmarksgame.alioth.debian.org/u64q/program.php?te... --- I don't know much about boost's regex support, but after a quick search, it does support backreferences, so it's likely a backtracking implementation. It's no surprise to me that it is beaten handedly by an implementation that uses DFAs (RE2, and in this case, Rust's regex library as well).
I have no idea whether it's the regex implementations that are causing the comparative slowness, to be honest. Just seemed interesting that some C++ implementations using what I would think are common techniques fall behind many other languages' implementations.
Of course, benchmarks are always arbitrary, and some of them will just disadvantage approaches that would be perfectly realistic ways to solve "real-world" problems, so it's always a grain-of-salt situation. However I'm not sure it's necessarily fair to say that "C++ should be at least as fast as C for any fair benchmark." Doesn't that expose you to the danger of redefining fair benchmarks as "those benchmarks which reinforce my preconceived notions of which tools have what performance characteristics"?
Probably bad wording by me. Cpp vs c has been benchmarked so extensively that it would be big news if C was quicker than Cpp at anything. Cpp is often faster though because of inlining of templates. Finally c is mostly just a subset of cpp. Its difficult to see how compiling the same code would be slower with a cpp compiler.
On the benchmark though, its doesn't seem quite fair. At least that first one is comparing a single threaded c++ program, to a multi threaded rust program. And for the regex example the author doesn't use C++ std::regex and std::futures (std::async) like he does for Rust. It makes me think a proper C++ implementation would do much better. (Personally I use the Parallel Patterns Library. I like it a lot. We will get something similar in C++ in a few years)
I don't know much about rust. It worries me though the examples are still using threads and mutexes. I think we need much higher level abstraction (like ppl::task, coroutines, etc) to get better scaling. Also lock free data structures dont scale that well either. As they still require synchronization and that hurts scaling). I think we need some kind of revolution in how we code to scale our programs to hundreds of cores.