One of the reasons Python is so popular as a scripting language in science and ML is that it has a very good story for installing Frankenstein code bases made of assembly, C and Pascal sprinkled with SIMD.
I was here before Anaconda popularized the idea of binary packages for Python and inspired wheels to replace eggs, and I don't want to go back to having to compile that nightmare on my machine.
People that have that kind of idea are likely capable of running kubs containers, understand vectorization and can code a monad on the top of their head.
Half of the Python coders are struggling to use their terminal. You have Windows dev that lives in Visual Studio, teachers of high school that barely show a few functions, mathematicians that are replacing R/Matlab, biologists forced to script something to write a paper, frontend dev that just learned JS is not the only language, geographers that are supplicating their GIS system to do something it's not made for, kids messing with their dad laptop, and probably a dog somewhere.
Compiling Python extensions is a nightmare because we allow Autoconf, CMake, Visual Studio, Bazel etc. to make it complicated and nonportable; when someone sets out to wrap some library for Python the quality of the result is limited by the quality of the tools and by low expectations.
A serious engineering effort along the lines of the Zig compiler would allow Python to build almost everything from source out of the box; exotic compilers and binary dependencies, not "Frankenstein code bases" per se, are the actual obstacles.
What you’re proposing here is essentially “if we fix the C/C++ build systems environment, this would be easy!”. You’re absolutely right, but fixing that mess has been a multi-decade goal that’s gone nowhere.
One of the great victories of new systems languages like Rust and Zig is that they standardized build systems. But untangling each individual dependency’s pile of terrible CMake (or autoconf, or vcxproj) hacks is a project in itself, and it’s often a deeply political one tied up with the unique history of each project.
> What you’re proposing here is essentially “if we fix the C/C++ build systems environment, this would be easy!”. You’re absolutely right, but fixing that mess has been a multi-decade goal that’s gone nowhere.
Not sure I would call it easy, as it would still take a lot of effort to update how PyPI works to account for these new capabilities, but that's exactly what the Zig compiler & build system solved.
Rust is completely hands off when it comes to C/C++ dependencies, Zig can package and build them. That's why I created https://github.com/allyourcodebase/.
Haivision/srt, the upstream C++ project
mbedtls
googletest
When you run zig build all these 3 dependencies are downloaded and their build.zig is run if present (the first one doesn’t have a build.zig since it’s just the vanilla C/C++ upstream project that we are providing a build script for).
The work to package everything must still happen, but once it’s done correctly you get the ability to build from any host for any target, which you can literally see happening in the CI runs of that repo: https://github.com/allyourcodebase/srt/actions/runs/10982569...
This kind of skepticism is exactly why I wrote this post. The details of actually coming up with a realistic upgrade path for PyPI are certainly much more nuanced than what I wrote in the post, but the core insight is that the C/C++ build intractability problem has been solved... and that you shouldn't depend on free big tech money if you can.
In my experience, `zig cc` is great at cross-compiling (which is a hard problem that it solves!) but doesn't actually help with the rest of the problem.
You still have to run cmake/autoconf/meson/whatever else, which is the part that's project-specific and often quite fiddly.
This has been my experience as well. I previously used cargo-zigbuild for packaging at work (and contributed both to cargo-zigbuild and Zig as a byproduct), and I still had several troubles that I had to analyze and tackle in spite of them.
Well if you're limiting yourself to zig cc, then all you get is a C compiler.
If you take the time to kick out of your project that soup of build systems and replace them with a build.zig, then you got yourself a complete solution. Takes some effort but it's perfectly doable, see https://github.com/allyourcodebase
The "upgrade path" is for individual packages (providing portable build scripts) and for client-side Python package management (actually providing and running next-generation tools), not for PyPI which already supports package metadata stating what platforms a package is compatible for.
You're the second person to post a link to a comment that's prominently highlighted on the issue I already linked. Yes, I've read that; nowhere do I see a statement like "Zig will always ship Clang with it" and instead I see a number of statements that imply it won't. I'm not even saying that getting Clang out of Zig is a bad thing. It's not like CMake or Rust come bundled with a C compiler.
I suggest you read it again then, because it heavily implies that.
> These use cases can still be satisfied by, again, an independently maintained project that combines Clang main() and Zig main() together. For users of these CLI tools, I don't expect there to be any difference in user experience.
Means when someone installs ziglang from their package manager, it will be able to build c.
Yeah, I've read it multiple times. Every time, it says to me "somebody somewhere else can package these things together, it just won't be Zig". There must be something about Andrew Kelley's communication style that clicks with other people but not me. But I just can't read into it what you say is there. Since this all boils down to interpreting one somewhat arrogant man's words, it ceases to be a technical discussion and just becomes an argument in semantic parsing. I'm just not going to comment much on Zig anymore as clearly other people know what's going on and I keep getting it wrong.
Zig is and will be shipping clang for a great long while. Not only will it be shipping clang so you can build other code with it, they aren't removing LLVM support.
It is not entirely clear what any of this means for the future, and in any case, it keeps changing all the time.
The promise to maintain LLVM support just applies to Zig code. That doesn't require Clang. LLVM alone is not a C compiler, and neither is Zig without Clang.
What very well may happen is that the "standard" distribution of Zig will continue to bundle Clang, while a "minimal" distribution also gets created which omits Clang, and thus cannot compile C code. However, none of this has been spelled out clearly yet, and so I think it's reasonable to say don't depend on Zig to give you Clang.
Zig is migrating away from LLVM as the mandatory backend, it will still ship clang and llvm so that it can build everything it needs which is critical for its mission for all your (code) base.
The typical build scripts and tools and terrible hacks that are adequate for a standalone C/C++/Fortran project are lacking for building the same project as a Python extension, which requires portability and supported, not too custom, build steps.
The ambition and usefulness of aiming for the latter higher standard is quite new: it has gone (relatively) nowhere because it has been a multi-decade non-goal, which only a small minority of users cares about.
Hard-to-build python extensions are basically .so files with some special symbols exposed.
As others said, a general task to "build libfoo.so" is very complex, and currently requires a wide variety of build systems. I don't see why this will get any easier if we require this .so file to export some Python-specific symbols.
It wasn't even intentional for Zig at the beginning, though. And any such solution should be usable across all platforms, while Nix on Windows is still not remotely usable (the latest attempt seems to be [1]).
Autoconf is easy, it's the highly bespoke build systems that someone thought would be a good idea that require the right phase of the moon that are the challenge.
Wheels do predate conda (though manylinux did base itself on the experience of Anaconda and Enthought's base set of libraries), and there were distributions like Enthought (or even more field specific distributions like Ureka and individuals like Christoph Gohlke) that provided binaries for the common packages.
What the conda ecosystem did was provide a not-horrible package manager that included the full stack (in a pinch to help fix up a student's git repository on a locked-down Windows system I used conda to get git, and you can get R and do R<->python connections easily), and by providing a standard repository interface (as opposed to a locked down and limited version that the other providers appeared to do), conda pushed out anyone doing something bespoke and centralised efforts so spins like conda-forge, bioconda, astroconda could focus on their niche and do it well.
Python shouldn't need special lib to design by contract, `assert cond, "reason"` and `if __debug__` work very well for that.
Unfortunately, people don't know they are made to be stripped in prod with `-OO` so that the perf hit is zero. This means you can never be sure a 3rd party lib you depend on is not using an assert somewhere for a very important check you are going to disable.
Respectfully, i am not sure that i agree with that. I think if it were allowed to slip into, say, public domain, then, yes, i agree.
HOWEVER, on a slight tanget from that point...If this company and its software have ever received any sort of taxpayer funds, then my opinion is that from the beginning, such software should have been open sourced. Publicly-paid software should be publicly available. Of course, said business has every right to earn profits from providing the service such as hosting said software for convenience for customers. But, every citizen who paid taxes has a right to see (and access!) all the code...well, that's my belief anyway. ;-)
There should be a very high standard for "the government can force me to do [something]" and it shouldn't be thrown around as casually as, one of a thousand enterprise CRUD apps went away.
Also it breaks the languagr barreer, you can now read the Chinese internet if you want, or chat transparently in Arabic. That's going to be interesting.
At the moment though (and ever since decent online translation services were a thing), it feels one-way, that is, people from that side of the internet coming to the anglosphere internet moreso than anglosphere people going internet-abroad. I may be wrong.
As a Frenchman, I learned very quickly that my language sphere market and resource pool is so much smaller than the English one that it's 10 times less effective to do anything in it.
I understand the position.
The only exception would be China, but the GFW is probably not helping.
LLV might lower the cost of that so much that it will become more interesting to do so.
Did you think the same thing when photoshop came out?
It's relatively trivial to photoshop misinformation in a really powerful and undetectable way- but I don't see (legitimate) instances of groundbreaking news over a fake photo of the president or a CEO etc doing something nefarious. Why is AI different just because it's audio/video?
And it's not the grounbreaking the problem, it's the little constant lies.
Last week a photoshopped Musk tweet was going around, people getting all up in arms against it despite the fact it was very easy to spot as a fabricated one.
People didn't care, they hate the guy, they just wanted to fuel their hate more.
The whole planet run on fake content, magazin covers, food packaging, instagram pics of places that never looks that way...
And now, with AI, you can automate it and scale it up.
People are not ready. And in fact, they don't want to be.
The people disliking the language are very vocal about it, but there is a huge amount of silent people that loves it and an even bigger amount that just like it as much as alternatives. It's mainstream now, not trending like 10 years ago, so there is no hype about it anymore. We just use it to do stuff.
Add to that the existing excellent ecosystem, the strong culture of scientific stacks and a very good story for providing c-extentions (actually the best one in all scripting languages because of things like cibuildwheel).
It's only in small tech bubbles like HN that devs find it surprising.
Python has many issues that are quite clear when you operate at some kind of scale and need proper multiprocessing/multithreading support. And its not just the GIL, you get very unexpected behaviors when dealing with exit handlers and signal handlers in edge cases. Having seen what other languages look like it just doesn’t feel like a language that was designed well for running at scale.
The tooling has markedly improved though. Things like typing and compile time checks, great. But its also funny to me that some of the fastest tools for python are being built in rust (eg uv).
I’ve always found Python to be sort of loved on HN. Not by everyone or course but I guess it depends on each of our experiences on here. I’m usually rather surprised when I meet people who genuinely dislike Python, because that seems like such an odd occurrence. Even if people don’t “love” the language most people seem to have had rather fond experiences or memories of it. Usually criticism comes down to its inefficiencies, but those aren’t exactly unreasonable critiques.
As I said it’s anecdotal, but in my experience Python gets a lot of love compared to something like Java or C#. Both of which are often met with real harshness. Hell I’ve ranted unseemly about C# myself.
I mean, even on HN, I'd say if there's derision, it's mostly one uttered with a yawn rather than genuine hate. And that's almost justified; while I spend a lot of my time with lots of different languages (I can't think of a single one I outright hate btw), Python is the one that pays for my things and... Well, there's not much drama there is there (now that we're lost 2->3 anyway)? It's a glue language that's easy to learn, but offers tons of depth should you want it. My primary annoyance at Python used to be the typing, but type annotations have made this less of an issue. It's a nice language and you can do almost everything with it. It's a bit boring, but I guess that's a good thing.
My only big critism is the CPython folks resistance to any kind of performance improvements, and the way PyPy efforts have been largely ignored, making Python the last major dynamic language to finally start caring about performance and having a JIT in the box.
Finally thanks to data science, and people getting fed up with always writing bindings, this is changing, and Python can join Common Lisp, Scheme, Smalltalk, SELF, JavaScript, Ruby, Lua, Dylan, Julia, BASIC club.
To be fair, Ruby YJIT can still somehow be slower than Python 3.11+ at times. Lua JIT is a bit of a separate effort and the ones that truly try the hardest here are Julia (which is not necessarily interpreted, it's really difficult to call it a pure scripting language at this point) and JavaScript engine implementations. I can see pretty good numbers for SBCL on BenchmarksGame too.
I was here before Anaconda popularized the idea of binary packages for Python and inspired wheels to replace eggs, and I don't want to go back to having to compile that nightmare on my machine.
People that have that kind of idea are likely capable of running kubs containers, understand vectorization and can code a monad on the top of their head.
Half of the Python coders are struggling to use their terminal. You have Windows dev that lives in Visual Studio, teachers of high school that barely show a few functions, mathematicians that are replacing R/Matlab, biologists forced to script something to write a paper, frontend dev that just learned JS is not the only language, geographers that are supplicating their GIS system to do something it's not made for, kids messing with their dad laptop, and probably a dog somewhere.
Binary wheels are a gift from the Gods.