That is very exciting. I like the direction the project is going.
But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see how it roughly compares.
Or is it too ealry too talk about it? Am I pointing to the elephant in the room? I am ok with saying "we are not there to talk about it, stop stirring the pot". I understand that.
But looking at the future, if Rust is to compete with C++, it needs to perform close to it. Not necessarily in the "safer"/default mode, but it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode.
C++11/C++14 improved C++ many enjoy and like using it. Many that don't are probably using it for performance reasons. That is a niche Rust will have to compete in.
> Not necessarily in the "safer"/default mode, but it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode.
By design, Rust allows you to opt-in to this mode locally via `unsafe { ... }`; this allows you to be 100% safe through the vast majority of the code base, but then eek out the very last drop of performance in core routines where there's no safe alternative. Then only those regions need to be checked for memory safety vulnerabilities rather than the whole codebase.
Although, many things can be written in safe code and still be performant: iterators provide safe and highly efficient sequential access to vectors and arrays (right now they optimise to the same code as the equivalent C/C++, other a bug in LLVM where it's not recognising that some pointers are guaranteed to be nonnull).
> By design, Rust allows you to opt-in to this mode locally via `unsafe { ... }`;
It should be noted that contrary to e.g. Swift's -Ofast, Rust's unsafe {} does not change language semantics. It allows operations marked as unsafe (non-bounds-checked array accesses, raw pointer and pointer munging, etc…) but operations usable outside unsafe {} have the exact same semantics inside.
Rust's performance is competitive with clang and improving, even in the "safer" mode. We have benchmarks in many of the libraries (search for #[bench]) and some in src/test/bench. We're on the benchmarking game, but that is really just a game, not indicative of anything besides how much time someone has spent microoptimizing a specific program. Someone is working on a custom LLVM pass that will remove essentially every null check that Rust adds. On top of that, there are still a few more changes on the way that aren't easy, but should give a nice perf improvement.
Here is a link to the source code of the initial version of the custom LLVM pass to eliminate null checks. Hopefully this will be upstreamed to LLVM in the future.
Very much so, agreed. It is a PR game most of all. It is unfortunate. But it is probably a game that needs to be played. I know microbenchmarks are bogus, but I still click and look at them.
The games are really awful. Look at NBody for example: http://benchmarksgame.alioth.debian.org/u64/performance.php?.... The fastest C program does it in 10 seconds. Rust is at 21. What's the difference? The C program uses vector intrinsics, while the Rust program is a straightforward implementation. The next four fastest C programs, which don't use vector intrinsics, are only a hair faster than the Rust program (20.x seconds).
The script kiddie's takeaway is: "well, C is still twice as fast as Rust!" The real professional's takeaway is: "wow, Rust manages to get within 4% of C while using iterators and higher order functions in the inner loop! Also: "Rust can do inline assembly too if that's what I need." https://gist.github.com/luqmana/5150557
Hate to break it to you, but most professional programmers have no deep knowledge of:
* runtime systems
* detailed memory management
* algorithmic complexity
Yet, quite a lot of those build parts of huge systems that work and meet business requirements all the time, which makes them very desirable professionals.
Being able to competently benchmark is a fringe skill.
Sorry, downvoted you by mistake. Wanted to upvote, because yes, that's the definition of "professional", contrary to an "amateur" which is somebody engaged in a certain activity without monetary compensation (for the love of it only).
Sorry, but "by definition" there are several meanings in the dictionary. Merely employed is the lower level. And I think I made the way I mean it clear.
E.g From dictionary.reference.com:
noun:
- a person who is expert at his or her work: You can tell by her comments that this editor is a real professional.
- a person who engages in an activity with great competence
adj:
- extremely competent in a job, etc
- (of a piece of work or anything performed) produced with competence or skill
Yes, and I think I also made clear that I don't accept you using solely that definition to discredit and downplay achievements of people not concentrated on a fringe of things. Your comment is the elitist as hell.
For example, someone writing a huge distributed video processing system rarely needs to consider any of these topics. Because everything needed in that range is built. Making sure that the data is at the right place at the right time and predicting that needs none of the skill given by you. Is that person not a professional?
Expecting professional programmers to be familiar with "runtime systems, detailed memory management and algorithmic complexity" is "elitist as hell"?
That's one of the reasons the industry is lead by charlatans pushing the latest fad, doesn't know it's own history, and has built pile upon pile of cruft.
>For example, someone writing a huge distributed video processing system rarely needs to consider any of these topics. Because everything needed in that range is built.
Blind reliance upon things build by others, the characteristics of which one building the "video processing system" you describe is not able to assess, and is not able to modify to suit the project's specific needs. That we accept that is a lowering of the profession. It's like doctors deciding it's no need to know pathology and anatomy anymore, because there are expert systems they can use.
> it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode
This will never happen, as it is contrary to the goals of the project. As mentioned by others, locally, you can ignore some checks, but there won't be a compiler flag to turn them off.
In some ways, it's too early to talk about it, and in some ways, it's not. It's not too early to talk about decisions which would place a certain upper bound on performance, but it's kinda too early to do some kinds of direct comparisons. The team has not really been working on performance generally, as the focus is to get the language semantics correctly. There's lots of places, for example, where rustc generates subpar LLVM IR, and it's a matter of someone writing a patch.
There technically is an "unsafe" mode, but not in the way you're thinking. You can drop down to C level by writing code in unsafe blocks and using methods prefixed with "unsafe." This will stop the compiler from doing things like inserting bounds checking and should perform fairly close to Clang.
Generally though, rust strives for the same ideal as C++: you only get what you pay for. You can carve out the subsection of the language you need through things like feature gating without too much trouble right now (and the standard library is still improving in this regard).
>But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see how it roughly compares.
Or is it too ealry too talk about it? Am I pointing to the elephant in the room?
I'd say it's too early.
First, Rust is a compiled, statically typed languaged using LLVM, including stuff such as non aliasing guarantess over C, and designed to be secure AND fast. It's not like it's some scripting language.
Second, you should benchmark your program first. Worrying about a language's "performance" in abstract is more premature than even Knuth anticipated.
A fast language runtime helps all programs. If it's slow, there's precious little you can do about it. And the design of the language may end up such that certain aspects are just always going to be slow. So, I very much disagree with the notion that this a premature optimization (and I'm starting to think that phrase needs to just go away). You seem to allude to as much by delineating between compiled and scripting languages.
You can run Rust completely without a runtime. Part of the reason for that is that there are much bigger performance hawks than you or me monitoring Rust who have significant input on its implementation. That is one of the reasons that Rust moved GC into a library, for example. The last thing I am worried about with Rust is its performance.
That doesn't say much about performance. Languages that do need a runtime, like Java, aren't slower than C++ because of the virtual machine per se (which otherwise does a good job at doing awesome optimizations that are only possible with runtime profiling), but rather the reliance on a garbage collector and the lack of options for fine tuning the memory layout / access patterns and unfortunately the runtime is not smart enough yet to do that for you. Every language that depends on a GC, but that wants to be close to C/C++ in performance, starts with this handicap, even though for most purposes an efficient garbage collector will behave much better than inefficient usage of malloc or a half-assed object pool.
Of course I wouldn't worry about Rust's performance either.
Awesome. Rust's performance is something that impresses me, so I'm certainly not suggesting that it's slow. The point I was making is it's a valid question to ask and a valid thing to measure. Having fought with slow language features (e.g., blocks in Ruby), slow stdlib (e.g., older versions of Scala's collections), and slow techniques adopted by library authors (e.g., ask for forgiveness with exceptions instead of permission) there's just a lot that impacts many if not all apps and if it's outside your own application code, it can be very hard to fix. One of the things I really like about go releases is they indicate exactly what's changed on the performance front, so I have a good idea of what to expect. And since they measure it, they can improve it.
Sure. I think that until they hit 1.0, the Rust devs want to concentrate on getting the language semantics right. The most important thing is that Rust CAN be made fast even if it is not fast today (but I'm not suggesting that it's not fast but it probably has room for improvement). You can't make e.g. Ruby fast while maintaining backwards compatibility.
Piggy-backing on premature optimization: Yes, 100% agreed. Totally needs to go away. I encounter it most used as a crutch for otherwise skilled programmers to ignore any analysis what-so-ever in any area of software development.
IME they produce late, over-budget, fragile systems because they're averse to thoughtful consideration of the problem at hand and end up rewriting the same thing three times over.
I'm generally not a fan of the term "Software Engineer", but I think attempting to earn the label is worthwhile at least. The "burn someone else's money" mind-set is at the opposite end of the spectrum though. I can't imagine any other field where the "customer" (the person cutting the cheques) would find that acceptable.
> Oh sure, I told you up-front I had two kids and my mother-in-law visits frequently so I want a 4 bedroom house, but you built a 3 bedroom one instead. But that's fine, just go ahead and tear that down and start over. I don't mind paying for both. Maybe by the fourth iteration you'll include a kitchen and bathrooms!
> Then on iteration 5 maybe the bathrooms will even have toilets in them. And don't worry about building codes. We'll just rebuild the house after the under-gauged wiring burns it down. Also, after the sewer gases fill the house we can always break up the foundation and redo the plumbing to add P-traps later.
> It's not like any of this is stuff that you could know up-front. That's why I pay you. The professional. With the "get-to-it" attitude to dive in blindly and show me just how fraught with mistakes this whole thing can be. And you're right, how do I really know I'll want a kitchen up-front? Maybe we'll just eat out every night after moving in. Previous evidence to the contrary not withstanding.
Er... went a little off-tangent. Just needed to get that off my chest I guess. ;-)
Agreed -- I've found these pearls of received wisdom often crumble on non-trivial projects. "Premature optimization" can lead some to defer thinking about performance at all. "Profile and fix the hotspots" without simultaneously thinking about architecture can yield a mass of uniformly slow branchy code with no path forward. It's hard to appreciate until you've been there. However, I do have faith that mozilla folks have most certainly been there and that those lessons inform the rust design.
As I understand it, the goal is for Rust to perform as well or better than C++ without the need for an unsafe mode. From the long discussions I've seen on the Rust mailing list, I think the likelihood of a "disable all checks" mode is nil.
Yes, idiomatic C++ which uses the safety features that C++ has to offer, as far as I understand. Not the raw performance that C++ can give you. Just wanted to point that out.
It's the job of the compiler to optimize out things it knows are secure. If an array lookup is known to be secure at compile time, then LLVM already optimizes out bounds checking without the need of a fast mode [1].
If there is not enough evidence available to the compiler that
a particular array lookup is not safe, then that evidence is also not available to the programmer. Therefore, one would have to write a manual bounds check anyway. Double bounds checking, one by the compiler and one by the compiler, can also be optimized out.
Maybe Rust is not capable of completely covering all optimization cases right now, but I don't think there's a need for a turn-off-all-checks mode.
Does anybody have any material about how a turn-off-all-checks-mode improved performance for another language? I'm curious about what the difference is.
> If there is not enough evidence available to the compiler that a particular array lookup is not safe, then that evidence is also not available to the programmer.
That's quite wrong. Humans are capable of deductions that compilers, which must necessarily use conservative, constructive methods, can't dream of. They can also have information that isn't available to a compiler even in principle, although this is less likely to apply to eliminating bounds checking.
It's worded quite strongly, but the programmer can only be certain that a certain bounds check is guaranteed to succeed if the compiler agrees. In other situations something about the current context may allow a bounds check to pass, but the "current context" is always bound to change in any sufficiently complicated program.
Currently Rust's perfmance is about that of Java's. Which is and isn't in contention with C++. Something's yes, some no. Rust beats C++ in a few areas on average (tree look ups for example) but that's about it.
It's not to early for it to be a concern. It should be a concern in system languages aiming to compete with lanagues like C/C++/Fortran. But I feel rust isn't in a terrible competition for being at least 15 years late to the party.
Do you have specific benchmarks to back up this claim? I think it's completely unfounded. Every test we've done against C and C++, we're competitive. Exception is the performance of the IO/task runtime, which still needs work, but is unrelated to the language.
There's also the factor of immaturity: the language is young and changing, so it hasn't really been possible/sane to spend the effort writing high-performance micro-optimised libraries for various tasks.
That isn't quite true. Swift will be using LLVM as a code generator, but because it will use ARC (a form of garbage collection), it will still end up slower than C++ in many benchmarks. That's not the only example, of course--while LLVM helps immensely, it's not a miracle worker and the original language semantics do make a difference.
I just meant you don't need necessarily need pointers at all, since you put things directly onto the stack (or into vectors/your favourite data structure), no need for additional indirection.
(Maybe Swift offers this too, but it's certainly something C++ excels at.)
The stack doesn't remove the need for heap allocation, since the stack is only useful for things that don't escape their context. The JVM can even do escape analysis and allocated short-lived objects straight on the stack - too bad the currently implemented algorithm is fairly limited.
But anyway, for shared reads or writes between CPU cores, you need the heap. Speaking of which, usage of smart pointers is most of the times less efficient than a concurrent garbage collector if that means reference counting.
C++ excels at fine-tuning the memory access patterns, depending on use-case, which can provide a big boost in performance if you know what you're doing - whether that means stack allocation, smart reference counted pointers, object pools, or whatnot. I don't yet see anything preventing people from doing that in Rust, while getting some extra safety for the common use-cases.
I wasn't meaning to suggest that heap allocation is not useful (e.g. the vectors I mentioned are heap allocations, as I'm sure you know), just, as you say, that C++ offers fine control over memory access patterns.
However, for shared read/writes you don't need the heap: global variables work fine, as does passing around pointers into some thread's stack. (They may not be the best practice, but it's still possible.)
Smart pointers are not just reference counting, e.g. the single owner pointer types Box (in Rust) and unique_ptr (in C++). Furthermore, the majority of the cost with many reference counting implementations is the atomic instructions required for safety, if you can get away with non-atomic instructions then the reference counting probably won't look so bad. (Rust can express a reference counted pointer that is statically restricted to a single thread `Rc`, while also providing the more expensive threadsafe one `Arc`.)
Rust is designed to offer the similar power in terms of controlling memory access patterns.
Swift does offer this, though in a slightly weird way; structs (and I think arrays) are stack-allocated, classes are heap-allocated. Swift structs are more featureful than, say, C++; they mostly behave like classes, but live on the stack and are passed by value.
Please, be direct with your criticisms of me rather than just implying them.
What other commonly used languages offer pointerlessness to the same degree as C++? Rust, for sure. Not C as there are no generics, making allocations + void* and simplistic data structures like linked lists more common than necessary. Maybe D but AIUI there is some struct/class difference that forces GC allocation for classes. What else?
(Also, please read my original comment properly, I was not just talking about "stack allocation".)
I am just answering about placing data into the stack, and that is offered by all descendents of Algol/Pascal/CLU/Mesa/Modula/Ada/... family of languages.
You can place data in the stack in all of them.
Quite a few of them allow for generics, starting with CLU.
Ada also offers RAII, although a bit more verbose than C++ way of doing it.
All of them allow reference arguments, hence no need for pointers for output parameters.
RAII is totally irrelevant to this discussion, and I clearly said I was not just talking about placing data on the stack.
> All of them allow reference arguments, hence no need for pointers for output parameters.
I wasn't very clear, the use of a C-style pointer is irrelevant: if it is being used as a reference (i.e. not owning, no need for clean-up/deallocation), then it counts as a reference. I was using the Rust terminology where 'pointer' mostly refers to 'smart pointer'.
> but because it will use ARC (a form of garbage collection), it will still end up slower than C++ in many benchmarks
Well, depends on the C++. ARC is reference counting behind the scenes; in Objective C and Swift, the compiler inserts retains and releases as appropriate. A lot of C++ code also uses reference counting a fair bit; that's what shared_ptr is for, for instance.
But, one thing worries me -- performance. Anyone know a list or tracker of benchmarks to see how it roughly compares.
Or is it too ealry too talk about it? Am I pointing to the elephant in the room? I am ok with saying "we are not there to talk about it, stop stirring the pot". I understand that.
But looking at the future, if Rust is to compete with C++, it needs to perform close to it. Not necessarily in the "safer"/default mode, but it needs to have an unsafe, turn-off-all-checks-run as fast as you can mode.
C++11/C++14 improved C++ many enjoy and like using it. Many that don't are probably using it for performance reasons. That is a niche Rust will have to compete in.