>Versions are only importable from tags (semver style)
Tags are mutable in git, which sometimes causes problems. Not that I think they had much choice here - git and tags are the de-facto community standard, they're just using it like everyone else.
>There is no NPM or PyPi like host, sources originate from code hosts
>Modules are prefixed with a domain name (avoid dep confusion)
Agreed, I like this quite a bit... but it comes with the significant risk of domain name or hosting changes. Github probably won't add the metadata to redirect your old module name to your new non-github name, for instance. I've watched several modules fight through this sequence, and the end result is always "can't do anything, contact your users yourself / hope they read the readme".
You win some, you lose some. In large part I think this part of Go modules is better than others'.
>Hashes are calculated on the module, it will fail if it is not correct
Nearly all lock-file-based dependency systems do this, Cargo included.
>GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes
Publishing a log now doesn't make it any safer tomorrow, it's purely about convincing you to trust them. This is precisely the same attack target as an NPM or PyPi. The same party hosts the tar'd source files + documentation + checksums: if it ever gets changed there, it's changed for everyone. Lock and go.sum files will only protect you from changes to your previously-known dependencies.
They've done a pretty good job making goproxy/gosum easy to run yourself, and you can skip past the goproxy and download directly from the module's owner (which is a domain name, which works great here), which mitigates this specific risk at least. But no, they're absolutely running a pypi, they just don't let repo owners make changes directly (only by committing changes, e.g. module revocation).
RE: git, if the package has been fetched through Go Proxy once before the tag change, the new tag will never pass validation
> if it ever gets changed there, it's changed for everyone
sum.golang.org has a hash chain, so you have to change everything that comes after a malicious hash change. So it does seem safer the more time that passes.
Go uses zip files, as a fyi.
I've had more trouble with custom domain based Go mods than the major code hosts. I've come to avoid those on custom domains as much as possible.
>sum.golang.org has a hash chain, so you have to change everything that comes after a malicious hash change. So it does seem safer the more time that passes.
Is there a work-factor attached to these? I kinda don't think so, the only reference I can see is "sha256" (I have not checked the code.... nor is the public proxy the same thing they run, though it should be good enough for this purpose). Unless it's "costly", the time needed to recompute the whole history of every module is probably well below one second. The chain just makes it so you don't need to do that on every single update.
It's a good structure for making attacks more noticeable (you can't change any piece of history without everyone who already has that history noticing), and it enables efficient and trustworthy partial updates (backfills? meh), so I think it's a very solid addition and I'm glad it exists. But it really doesn't protect against much in the way of practical attacks - it's a canary, not a preventative measure, mostly serving to say "see? we've been trustworthy, you can trust us going forward".
(edit: I'll completely believe there's a small niche of attacks that it prevents... but I can't think of any that aren't massively overshadowed by other also-possible attacks in the same situation)
>I've had more trouble with custom domain based Go mods than the major code hosts. I've come to avoid those on custom domains as much as possible.
Yeah, custom domain ones have definitely had more problems - the proxy helps a fair bit there :) I don't mean to imply I'm unhappy they're running a PyPi-alike; I'm absolutely thrilled that they finally admitted they need it, and it's a decent implementation too. But that's pretty much all it is.
You can reproduce it if there is no private key or cert chain involved. I'm not clear on the details, but they where basing it on The Update Framework (TUF) so there may be some cert chains involved
When you use `go get` to retrieve a version of a dependency, it adds a line to a `go.sum` while with a hash of the code at the version specified. You can distribute your code without a copy of the dependency. When someone else runs go get on your module, it will attempt to retrieve the same version of the source, checking the hash. If the hashes are different, an error is thrown.
That's what a Cargo.lock has. But it's recommended that you don't ship Cargo.lock that with libraries. I think the reason is because you can't have multiple versions of the same package. But I could be wrong. (If I'm wrong then you end up with horrific bloat with multiple trees of the same dep).
What does go do when you have different deps depending on different versions of the same common dep?
> If a library ends up being used transitively by several dependencies, it’s likely that just a single copy of the library is desired (based on semver compatibility). If Cargo used all of the dependencies' Cargo.lock files, then multiple copies of the library could be used, and perhaps even a version conflict.
> In other words, libraries specify SemVer requirements for their dependencies but cannot see the full picture. Only end products like binaries have a full picture to decide what versions of dependencies should be used.
We can determine that one of the following things is true:
1. mywebsite.com has a 'meta name="go-import"' html tag in it when accessed by the go tool that points to a git repo/svn repo/etc, and the contained package has the folder structure "foo/bar/baz/v2/fizz"
2. mywebsite.com/foo has a 'meta name="go-import"' html tag...
(several obvious cases omitted)
3. mywebsite.com/foo/bar/baz is a git repo that has no version tag, but has a "v2" folder in it
4. mywebsite.com/foo/bar/baz is a git repo that has a version tag with v2.x.y
5. the go.mod has a replace or other directive for this import path
The 4th case is the relevant one here. github.com/foo/bar/v2 means that the bar repo has a v2 tag usually.
So, what version of that repo is used if there's v2.0.0 and v2.1.0? Well, the version is the smallest version that you or any module you depend on uses, so you simply have to recursively parse all go.mod files to understand the version used. Much simpler than Cargo.lock.
How do you enable or disable features in a sub-go-module? Let's say I have a module that has a "+ui" or "-ui" build tag because you can optionally build an openGL ui or such.
In rust, this would be a 'features = [ "ui" ]' in your Cargo.toml. In go, you would change your own compilation step to be "go build -tags ui"... And if you have two dependencies with those flags, and you wish to make them opposite values, that is simply impossible in go even though it's trivial in rust.
There are other differences, but I feel like I've already made my point. My main point is that go modules make import paths way more opaque. It used to be that every path element was either part of the import path or a folder on the filesystem. Now, it's either part of the import path, or a git tag, or a folder, and none of those things are obvious.
My secondary point is that go has a much simpler set of requirements than rust, mostly due to missing important features rust has.
Yeah. My problem is that my "really bad cold that lingered" was in February 2020.
That is late enough in the timeline, and the testing at that time was so insufficient, that it's really hard to say.
I don't recall losing my sense of smell, but I have been struggling more with concentration issues and depression since then. Of course, that could also be due to all the stress over losing a friend to COVID, the lockdown, the political situation in the US, etc.
The biggest disappointment for me in the whole COVID response has been the complete failure of ramping up COVID testing and doing random testing or testing of those who hadn't traveled to China.
>The biggest disappointment for me in the whole COVID response has been the complete failure of ramping up COVID testing and doing random testing or testing of those who hadn't traveled to China.
I had a viral illness back in July with symptoms that couldn't rule out Covid-19. I went to get tested the very next day as soon as I possibly could have and quarantined while I waited. At the time I got tested I was told the backlog was on the order of 3-5 days for Quest. I ended quarantine 2 weeks later, not because I got test results back, but because they waited so long that even if it came back positive I already met the CDC guidelines to end quarantine regardless. Rather than processing the tests they could manage in a timely fashion they were wasting tests on samples so old it didn't even matter what the result was.
Rather than a priority queue they should have been handling tests like a priority stack. I'm sure it would suck to be told "sorry we're not going to check your sample, assume it's positive" but at least then you could manage quarantine and contact tracing in a timely manner instead of waiting so long it's basically useless. Not to mention the government no doubt paid Quest for all of those useless tests vs. Quest only getting paid for the amount they could do in a timely manner. Doing it a reasonable way would just be throwing money away for them.
Heh. I guess I have an interview question to update.
One of my quick warm-up questions is asking about what data structure to choose if inserting an element after an element you already have a reference to is a common operation that you want to optimize for.
Of course, I don't consider there to be a right or wrong answer, but just make sure that people can intelligently discuss it. Some people start off with vec or the like, and then we discuss how fast that would be compared to a linked list. I generally stop after we get to the O(n) vs. O(1) comparison.
But maybe if we have some extra time I should also talk about the tradeoff with actually traversing the list.
After all, I suppose this is why things like gap buffers are popular for text editors, rather than linked lists.
Hey, nice! I have a notes file experimenting with a similar tree-style minimalist data notation; it's definitely an area that needs more exploration.
I like what you've done with it, I'll need to take a closer look.
While I was thinking about it, I also decided to take a look into prior art, which took me all the way back to Landin's "The Next 700 Programming Languages" from 1966 which introduced the hypothetical language ISWIM, which was the first to introduce indentation based syntax and inspired the ML and Haskell family of languages (https://www.cs.cmu.edu/~crary/819-f09/Landin66.pdf)
It wasn't exactly what I was looking for, but it's fascinating to see how long we've been thinking about these kinds of issues.
Yes, and surprisingly there are not too many that use any indentation based syntax (about 1 in 50). However, of the ones that do (abc, aldor, boo, buddyscript, cobra, coffeescript, csl, curry, elixir, f-sharp, genie, haml, haskell, inform, iswim, literate-coffeescript, livescript, madcap-vi, madcap, makefile, markdown, miranda, nemerle, net-format, nim, occam, org, promal, python, restructuredtext, sass, scss, spin, stylus, xl-programming-language, yaml) there are some real gems.
Very nice! The nice thing about these kinds of notations is that they last. Simplicity is timeless. I can immediately grok exactly what you are doing here. :)
What is Blender's funding model? I recall that its initial open source release was essentially crowdfunded before that was a trendy thing, but I haven't been keeping up with how they've been funding continuing development. They seem to be doing pretty well at that, so I'm curious about where the majority of revenue is coming from.
RISC vs CISC isn't really about instructions doing "one simple thing period."
It's about increased orthogonality between ALU and memory operations, making it simpler and more predictable in an out-of-order superscalar design to decode instructions, properly track data dependencies, issue them to independent execution units, and to stitch the results back into something that complies with the memory model before committing to memory.
Having a few crazy-ass instructions which either offload to a specialized co-processor or get implemented as specialized microcode for compatibility once you realize that the co-processor is more trouble than it's worth doesn't affect this very much.
What ARM lacks are the huge variety of different instruction formats and addressing mode that Intel has; which substantially affect the size and complexity of the instruction decoder, and I'm willing to bet that creates a significant bottleneck on how large of a dispatch and reorder system they can have.
For a long time, Intel was able to make up this difference with process dominance, clever speculative execution tricks, and throwing a lot of silicon and energy at it which you can do on the server side where power and space are abundant.
But Intel is clearly losing the process dominance edge. Intel ceded the mobile race a long time ago. Power is becoming more important in the data center, which are struggling to keep up with providing reliable power and cooling to increasingly power-hungry machines. And Intel's speculative execution smarts came back to bite them in the big market they were winning in, the cloud, when it turned out that they could cause information leaks between multiple tenants, leading to them needing to disable a lot of them and lose some of their architectural performance edge.
And meanwhile, software has been catching up with the newer multi-threaded world. 10-15 years ago, dominance on single threaded workloads still paid off considerably, because workloads that could take advantage of multiple cores with fine-grained parallelism were fairly rare. But systems and applications have been catching up; the C11/C++11 memory model make it significantly more feasible to write portable lock-free concurrent code. Go, Rust, and Swift bring safer and easier parallelism for application authors, and I'm sure the .net and Java runtimes have seen improvements as well.
These increasingly parallel workloads are likely another reason that the more complex front-ends needed for Intel's instruction set, as well as their stricter memory ordering, are becoming increasingly problematic; it's becoming increasingly hard to fit more cores and threads into the same area, thermal, and power envelopes. Sure, they can do it on big power hungry server processors, but they've been missing out on all of the growth in mobile and embedded processors, which are now starting to scale up into laptops, desktops, and server workloads.
I should also say that I don't think this is the end of the road for Intel and x86. They have clearly had a number of setbacks of the last few years, but they've managed to survive and thrive through a number of issues before, and they have a lot of capital and market share. They have squeezed more life out of the x86 instruction set than I thought possible, and I wouldn't be shocked if they managed to keep doing that; they realized that their Itanium investment was a bust and were able to pivot to x86-64 and dominate there. They are facing a lot of challenges right now, and there's more opportunity than ever for other entrants to upset them, but they also have enough resources and talent that if they focus, they can probably come back and dominate for another few decades. It may be rough for a few years as they try to turn a very large boat, but I think it's possible.
> I'm willing to bet that creates a significant bottleneck on how large of a dispatch and reorder system they can have
My understanding is the reorder buffer of the m1 is particularly large:
"A +-630 deep ROB is an immensely huge out-of-order window for Apple’s new core, as it vastly outclasses any other design in the industry. Intel’s Sunny Cove and Willow Cove cores are the second-most “deep” OOO designs out there with a 352 ROB structure, while AMD’s newest Zen3 core makes due with 256 entries, and recent Arm designs such as the Cortex-X1 feature a 224 structure."
> These increasingly parallel workloads are likely another reason that the more complex front-ends needed for Intel's instruction set, as well as their stricter memory ordering, are becoming increasingly problematic; it's becoming increasingly hard to fit more cores and threads into the same area, thermal, and power envelopes. Sure, they can do it on big power hungry server processors, but they've been missing out on all of the growth in mobile and embedded processors, which are now starting to scale up into laptops, desktops, and server workloads.
Except ARM CPUs aren't any more parallel in comparable power envelopes than x86 CPUs are, and x86 doesn't seem to have any issue hitting large CPU core counts, either. Most consumer software doesn't scale worth a damn, though. Particularly ~every web app which can't scale past 2 cores if it can even scale past 1.
Parallelism isn't a good idea when scaling down, nor is concurrency often. Going faster is still a good idea on phones (running the CPU at higher speed uses less battery because it can turn off faster) but counting background services there is typically less than one core free, there is overhead to threading and asyncing, and your program will go faster if you take most of it out.
Yes but if we are being pedantic no. Of course the language does still greatly lean into the null terminated string concept.
You get a null terminated char array but the length is actually available since arrays (as long as they haven't decayed into pointers) can still share their length at compile time.
Here is an example which, while trivial, demonstrates some of the optimization issues you can run into in C and C++ but not in Rust, even with noalias currently only applying to &T.
In C and C++, compilers are allowed to infer "noalias" based on pointer types; two pointers of different type are not allowed to alias. This is known as type-based alias analysis. But char * is given special treatment, because of its use as a generic pointer type; so if one of your arguments is char * or even signed char * , that disables any optimizations which were relying on type-based alias analysis.
This provides both for performance and undefined-behavior footguns. If you ever try to use some pointer type other than char * or signed char * to refer to data of another type, you may inadvertently cause type-based alias analysis to kick in, causing invalid optimizations and miscompilation. On the other hand, if you have a function which takes both a char * and another pointer, the compiler may not apply optimizations that it otherwise could because char * is allowed to alias anything.
In Rust, there is no such undefined behavior footgun. Because of the LLVM bug, noalias isn't applied to &mut pointers so there are still some cases which could be better optimized, though it sounds like there is some progress being made on the LLVM front so it should be fixed at some point, and there are already places where the compiler can do better optimizations with better safety due to the stronger semantics of &T.
And yet, rather than ignoring it, you are commenting on it, with a pithy retort which dismisses the entire benchmark without actually providing any additional insight.
Programming languages, compilers, library ecosystems, the groups of people who decide to sit down and try to produce a better result for a given language, and the benchmark maintainers who decide what submissions count for a given language (does a C solution that just uses entirely inline ASM count?) are incredibly complex systems. Any single metric is never going to capture the full richness of the language, is never going to be representative of the experience you will have for every single program, etc.
But does that make metrics useless? No, it just means that you should be informed about their limitations. You shouldn't just look at a single number, but instead make sure you understand well enough what is being measured to know how well that number represents anything useful.
So rather than just dismissing this benchmark, it would be useful to ask "why are the C++ results better than the C results on this benchmark?"
Some benchmark challenges like this allow pretty much any program that accepts the right input and produces the right output; which means you get results in which no computation is actually done, the output is simply hard-coded and you are basically just measuring the startup time or request time of the language or library.
This particular set of benchmarks imposes some constraints to avoid that kind of behavior. Programs have to follow the same basic algorithm, so you can't figure out some clever algorithmic optimization which applies only for the particular input used in this benchmark. For things like the regex challenge, you are expected to use either the built-in regex in your languages standard library, or a common general-purpose regex implementation, not a specialized regex implementation optimized just for this particular benchmark.
The goal of this set of benchmarks is to provide a reasonable set of reasonably realistic small problems, implemented using the same algorithm, and using the normal language and library features. It uses small simple problems in order to make it easy to read the programs and learn about the performance characteristics of the language.
So rather than dismissing, why don't we take a look at the fastest C and C++ implementations of some of the problems?
Here's the fastest implementation of the k-nucleotide problem in C and C++:
I haven't sat down to do detailed profiling and performance comparison of each; but just off the top of my head, here are a few things I see which could be relevant.
The C++ implementation takes advantage of a number of C++ features; it uses a hash table from GNU pb_ds, a C++ container library which allows for a good amount of customization based on template parameters. The C implementation uses khash, a very fast hash table implementation which uses macros for similar customization.
The C++ implementation makes note of using move semantics in a number of places, which potentially allows for certain optimizations that wouldn't be possible if the compiler had to copy data.
The C++ implementation uses insertion into a templated ordered map to sort the results, while the C implementation uses the standard library qsort. This allows the comparison function to be inlined into the C++ sort, while it's called through a function pointer in the C implementation.
Without actually doing some experimentation and profiling, it's hard to say which of these makes a difference, or if it's something else. But this does show that C++ provides facilities for generic container and algorithm types that C does not. In the C implementation, macros are used to work around this for the hash table case, while function pointers are used for sorting.
Anyhow, rather than simply dismissing these results, why not dig into where the difference really lies, and provide a better implementation in C if you think that you can?
No one set of benchmark results should be taken as gospel. But I think this particular benchmark game is fairly useful for getting a rough sense of "if I write all of my code in this particular language, using either the standard library or commonly available off the shelf libraries, how much of a performance penalty am I likely to pay?"
I also find that the grouping of languages that he does, based on the minima of the kernel density estimation of their geometric mean scores, to be a bit more informative than absolute ranking within those groups. That gives a sense of the general class of languages. There's one group for C, C++, and Rust; languages which allow for performance without compromise, at the expense of lack of safety, higher complexity or learning curve, or both.
There's a next big group with a lot of languages; most of them have been around for a while, or been designed with an eye towards performance, but still have some amount of overhead due to GC or pointer chasing or greater thread synchronizaiton overhead or any number of other reasons; this group includes Fortran, Ada, C#, Java, Go, Haskell, etc.
Then there are a few groups of fairly high-level, dynamic languages, designed for scripting or rapid development, and which require you to trade off a fairly significant amount of performance for this. Dart, PHP, Python, Erlang, Ruby.
And finally, there's Matz's Ruby, all alone in a group at the end, slower than pretty much everything else. I'm not quite sure why it's separated out from Ruby, which seems to refer to yarv, but maybe it's so people who come here wondering what they can do about their slow Ruby programs can see that they can at least get a big boost by switching to yarv.
Anyhow, this benchmark and this grouping helps if you're considering what to do about some performance bottleneck you have in some code base; or if you're starting a project for something which will potentially be performance critical. Moving between languages in one group isn't all that likely to make a substantial difference; but moving to a language in another group would. For example, it lets you know that there's a pretty good chance that just rewriting a Python program that's a performance bottleneck in Go would improve that performance; but rewriting a Go program in Java, or vice versa, is less likely to be a performance win.
Ah, thanks. It's been a while since I've used Ruby seriously. Still not quite sure why "Matz's ruby" is separate on the Benchmark Game rankings, while all other languages are grouped based on the best implementation of the language. Anyhow, that's pretty much irrelevant to my overall point.
Thanks for the kind words, not sure why it's getting downvoted other than that I called someone out on their lazy dismissal. But I wouldn't worry about it too much, I've been on HN and Reddit for long enough to not care too much about downvotes.
Your tone at certain points was overly incendiary, imo (the comment you were replying to was a little bit too). But there was also lots of good substance around those bits. Just my two cents.
Yeah, my tone was a bit incendiary at first, because I get a bit frustrated at the shallow dismissal you see so often on Hacker News (and any number of other tech discussion sites). This kind of snarky dismissive comment is easy to make, and has a tendency to get upvoted because it matches misconceptions that many people have.
These kinds of comments, and downvotes in place of discussion, are corrosive to honest, informative discussion, and I've seen way too many tech discussion communities get taken over by people who make low-effort dismissive comments and avoid engaging in actual substantive discussion.
