Hacker Newsnew | past | comments | ask | show | jobs | submit | more BiteCode_dev's commentslogin

Honestly, pip freeze includes the whole content of a venv site-packages, and the exact versions. For most projects, that's equivalent to all the dependancies recursively pins with peotry, although you don't have the clean pyproject-dev-prod/lock file separation.

So a huge number of cases can be handled with just that. It will be "reproducible" enought for a lot of people.


> although you don't have the clean pyproject-dev-prod/lock file separation

That's why I use "poetry export -f requirements.txt > requirements.txt" instead of pip freeze. It only exports prod requirements from the poettry lock file.


Nice trick, and very useful to make sure people never have to know about poetry if your team can't deal with it.


Most of the time, you don't need all that, since Python has zipapps. You defined deps, you zip it, you ship to any same os with the same python version. It embeds everything, and just run.

We even how have a nice tool to automate the bundling for you:

https://pypi.org/project/shiv/

Of course you still have to figure out how to get a Python installed on the final machine, that's the price to pay to be an interpreted language.

We don't have yet a story to ship a beautiful exe/dmg/deb/rpm that embeds the zipapp and libpython in an easy way.


> you ship to any same os with the same python version

This is a non-trivial thing to handwave away.


I agree, but it's still way easier than the original story, which is the one you also have with PHP, Ruby, JS, etc.

Using an interpretted language always leads to this.

I know no popular interpretted language with a seamless experience to ship a standalone exe.

In fact, Python is probably the one with the best story here, since it has nuitka (https://nuitka.net/), which allows to compile Python code into a fully standalone exe.

But then you need to install a compiler, headers, etc. And no cross compilation of course. Not to mention on Linux, you have to ensure you target the lowest version of libc you can.

You are still very far from Go or Rust, and I'm hoping one day that RustPython will succeed because that would mean an amazing deployment story.

Meanwhile, you trade the ease of deployment of compiled languages for the ease of development of interpretted ones.

I think it's a fare trade for most people: you dev the program much more often that you deploy it.

That doesn't mean we shouldn't work, as a community, to improve the deployment story. It's a serious hindrance.

That's the raison d'être of the Briefcase project (https://beeware.org/project/projects/tools/briefcase/). It's still in progress, but the last prez I saw on it was quite impressive already.


> I'm hoping one day that RustPython will succeed because that would mean an amazing deployment story.

Isn't RustPython just an alternative interpreter to CPython, implemented in Rust instead of C?

How would RustPython offer better deployment than CPython?


Rust has a fantastic deployment story: compiling a rust program is super easy, and you can cross compile. Using cargo and rustc is a breath of fresh air compared to any similar experience with C compiling.

So if one day RustPython gets compatible enought with CPython that you can use it as a drop in replacement, you can start creating a tool that compiles any Python VM for any target, and bring along your program with it. Making a standalone version of it would become much easier.

Right now, doing so either requires you to bring in a pre-compile version of cpython for your target (which is what briefcase does) or compile the thing yourself with gcc + headers + deps(which is what nuitka does).

It's not easy.


> So if one day RustPython gets compatible enought with CPython that you can use it as a drop in replacement

I don't think this will ever happen unless the community converges on a standard C-extension interface. Presently Python leans so hard on C-extensions, but there is no standard interface--if you're writing a C-extension library, you just depend on whatever obscure corner of CPython that suits your purpose. If you're writing an alternative Python interpreter, you have to implement the entire surface area of CPython, which generally means you must implement CPython exactly and you are severely restricted on the improvements you can make. At that point, why even bother?

Fortunately, I think there are emerging candidate interfaces, but the community needs to either update C-extension packages to use those interfaces or support packages (and maintainers) who already do. https://github.com/pyhandle/hpy.


There are probably only a dozen of c popular extensions that needs to support HPY reach the tipping point of mass adoption: numpy, scipy, pycuda, tensorflow, matplotlib, uvloop, etc. and some db drivers.

The rest is not popular enought to be a blocker. You will hear them scream a lot, but they will be like 0.00001% of the user base, and we can just tell them to stay on CPython with its limitations. They don't lose anything, just not gain anything either.

Those C extensions authors are directly in communication with Python core devs, when they are not core devs themselves, so if HPY is adopted, we can expect a total adoption under 5 years.

Numpy authors already said it would take 1 year to adopt it.

Give the huge number of benefits of HPY, I deeply hope it will be a success.


I'm not sure. I would certainly add psycopg2 to that list, since it's really the only well-supported way to speak to a Postgres database via Python. I imagine other database dialects will have similar issues. And there's probably a whole host of other prominent libraries that we're just not thinking about because we only run into them when we're trying to use something like Pypy, and even then we only run into one or two at a time before giving up and going back to CPython.


> a breath of fresh hair

Indeed.


:) Fixed


youtube-dl for example is distributed as a zipapp and it seems to be distributed just fine. It only requires you to have Python installed on your system, which isn't too burdensome of a requirement on macOS/Linux. On Windows they do actually distribute a Python interpreter.


Youtube-dl is a great tool, but the audience (primarily technical people) of any CLI app won't be representative of shipping an app for general use.


GP's point was not about "general use" apps, but apps for other developers / system maintainers


What's the story if my zipapp depends on tensorflow and/or cuda?

I don't know exactly how it goes but I'm pretty sure it's a horror story.


As usual with extensions, you are not using Python anymore, but a compiled language. To get 100% certainty, you'd need to compile the whole thing.

That being said, a lot of extensions are pre-compiled and provided as wheel, which is the case for tensorflow (I don't know for CUDA, I can't test on a laptop without a GPU).

Let's see what this means:

    $ py -m venv test
    $ test\Scripts\activate
    $ pip install tensorflow
    $ code hello_tensor.py
    # import tensorflow as tf

    # def main():

    #     with tf.compat.v1.Session() as sess:
    #         a = tf.constant(3.0)
    #         b = tf.constant(4.0)
    #         c = a+b
    #         print(sess.run(c))

Now with shiv:

    $ copy hello_tensor.py test\Lib\site-packages
    $ shiv -e hello_tensor.main --site-packages test\Lib\site-packages\ -o hello_tensor.pyz
    $ python hello_tensor.pyz
    ...
    2020-05-28 17:31:46.580704: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    7.0
So it works fine, but remember:

- it will only run on the system this particular wheel has been designed to run on. In my case cp38-win_amd64.

- it will come bundled with tensorflow, which is a behemot, meaning your hello world pyz will around 500 Mo.

- it needs to unzip, so the first run will be REALLY slow

For something like this, I would advice a more generic deployment tool, like fabric 2 if it's remote, or a make-like tool such as doit if it's local only.

Make your deployment script, and zipapp that.


Zipapps are an order of magnitude improvement in the Python world, but there are still lots of other major pain points like dependency management and performance which still leaves Python several orders of magnitude behind its competition. Hopefully these things change going forward.


It looks like zipapps built with “shiv” need to extract the contents of the zip file to disk before they can run? Does it delete the extracted files on exit?

If so, the extraction is going to make startup very slow. If not, that’s just messy. Either way, it’s not ideal.


It's doing the work only once.

Yes, it's not ideal.

But it beats shipping your entire dev env to the server.

I find it a good compromise. The extraction is done is $HOME/.shiv/{zipappname}_{zipapphash} so it's not a horrible mess. But if your project is big, you do have to clean up the old install because it can eat a significant amount of space.

For you regular cmd tool though, it's a blip.


> you still have to figure out how to get a Python installed on the final machine, that's the price to pay to be an interpreted language

Also for Java. The installs usually include their own favorite version of Java alongside the actual application.


Installing Java is a bit of a difficult task these days. Even worse if you want to get the JDK which is behind a register-wall.


Not if you are installing OpenJDK or Amazon Corretto. Add the PPA and install with apt-get. Like every other Ubuntu/Debian package.


jlink[1] solves this for JDK 9 and later.

[1] https://docs.oracle.com/javase/9/tools/jlink.htm


I don't see what's modern about it.

Those tools have been existing for years, and as always with Python, they serve only a subset of the Python users.

Not that it's a bad article. If you don't know about them, give it a read, it's good to know it exists and the post is well written.

But don't think it's the ultimate stack or whatever.

And if you learn Python, you should always strive to master the basics first:

- installing python.org for windows and mac, or apt/yum on Linux (those ones are tricky, it's more than just the "python" package to get pip and venv)

- don't try to get the latest and greatest version (which is 3.8, or 3.9 alpha today). 3.6 is great already and is wildly available. I personnally strive for 3.7 now, but I'm happy with 3.6 if I don't have to use asyncio. So get the most modern version that is easy to install for you.

- be comfortable with "-m", the "py" command on windows, "pip" (-m, --user, install, install -r, freeze), and "-m venv".

- make sure you understand what the PYTHON PATH is and how it works regarding the import system

- know how to configure your favorite editor to work with those

Once you have those basics, you can go and try whatever have you: peotry, pyenv, pew, virtualenvwrapper, pipenv and so on. It will be easy because you will have a solid understanding of where it comes from and how it plugs in into the ecosystem. You will be able to chose if they are worth it or not for you. And more importantly, when you will have to move away from them (because of work, because they don't suit you, because they are abandonned...), you will always be able to fall back to basics.

But franckly, there are so many things that you should be learning before those extras: pdb, black, pylint, zipapps... Or even concepts like generators, decorators, unpacking, etc. They all have a better value for the time you spend on them than learning a new "modern" stack.

Not that it's not useful to have a modern stack. I update my stack all the time.

I'm currently using pip + pew + doit + nox + pytest, and I'm experimenting with dephell.

But it's my full time job. I'm an expert.

Most Python devs have a much limited amount of resource to spent in learning stuff. They have deadlines, and other stuff to care about than Python.

I know because I go from company to administrations to students to train them, and it's always the same: they chose Python because it's an efficient use of their time.


Context is important. Developing applications vs libraries vs scripts is different, developing internal company software vs external software is different.

If you're shipping a package on pypi, you will want to develop and automatically test using multiple Python versions and maybe a range of versions of your dependencies. Having good tools to manage multiple sets of Python versions and dependencies and parallel automated testing is a godsend.

If you're writing an internal or web application that will run only in a single well-known environment, pip + venv + pytest might well be good enough.


Sure, but again, there is nothing modern about this. And certainly not, hyper modern.

Besides, if you want to develop and automatically test using multiple Python versions and a range of versions of your dependencies, none of the tools from the article will do it.

At best they are one of the ways to get dependancies before you use a tool to do it. You can do that with raw pip and venv as well.

What you need though, is something like tox, which has existed for years.

As mentioned in my comment, I personnally use nox for this, as it is, ironically, more modern to my taste.

Nox doesn't care if you use poetry, pew, virtualenv wrapper or venv manually.

I would certainly advice to learn something like nox or doit before learning poetry, for example.

Now I don't advice against learning poetry, mind you. It's a good tool, well written.

Pyenv is another matter entirely. The day one really need what pyenv gives you to the point it's worth investing in it, they can discard all my advices entirely, as they would have the skill and experience to not need my explanations.


Not sure if you've read just the first article, it's a whole series, they cover nox later on.

I don't see how this stack is "hypermodern" either, mind you.

My point was just that these articles and other like it tend to omit context completely. What is this stack good for? Who should use it? Who shouldn't? Without answering these questions the articles are not all that useful, or even could be harmful, if a dev who doesn't know better apes a complicated stack for no good reason.


I missed that it was serie, indeed.


Spam filters often white list any emails comming from an address in your contact list.

When you send an email, said address is often automatically added to your contact list.

So if you ever a test message the other way around, it will be white listed.


This is super cool, and you probably have a business model here: create a proxy, allow people to pay you for it, and use it transparently with htmx.

This way they can create entire products with zero backend code by simply calling 3rd party API.

This is probably also some very nice static blog engine concept to imagine around this.

And some mix with VueJS as well.


I think all cmd should have a "tutorial", "doc", and "quickref" subcommands.

Eg:

git quickref

This will list the most commonly used git invocations and their affect

Git quickref chekcout

This will list the most common checkout invocations.

Git doc open your browser to an offline doc html doc, etc


If you see "quantum" in a general public health product, you know it's going to be a scam, but probably a successful one.

My mother loves those kind of things and no amount of discussion with her will create any doubt in her mind.

I can't blame people falling for this though. They have been disrespected by the traditional health institutions for so long any escape seems good to them.

You can see the same effect in politics. To an extend, the current authorities have a lot of responsability in people listening to crooks.


It can also even be literally radioactive https://youtu.be/mEm9ooXyKss


Yes. More value, more effect.



Related but not exactly the same. Veblen goods derive their "worth" from showing them to others, while this is something you can enjoy on your own.


I love this one: simple, light, good Python bindings, decent cross plateform support and not hard to install.

But show an app made with it to anybody below 30 and they will call you an ancient fossil.

If something doesn't have transition and gesture nowaday, you better market it to accountants because the general public won't want it.


If the app is free, I would just politely ask such users to eff off. On the other hand I am probably as far away from iPad-damaged demographic that you can come: Emacs isn't enough of an OS, I have to leave it a couple of times a day and I hate every single minute of it.

Everything I do on a computer, apart from the odd movie I watch, is centered around information in text form, yet literally every program except decent text environments try their best to ruin working with text. The only place where I think a non-text-based ui actually adds something is in calendar applications, yet most of the suck as well.

So I sit here alone with the marvel that is 1980s technology, thinking most things regarded as "modern" in computing are a farce.

The only new general purpose tool I can stand is VSCode, where my o my critique is that it isn't emacs (and that multiple cursors are a decent but not a great replacement for macros)


Git is not for kids, it should be learned only by people that will need to code regularly in the future.

But, if do need to learn git, it's not hard at all.

The problem is that most people teaching git suck at it.

Git has been poorly explained since the begining of time, and because it has a terrible UI, plenty of gotchas and untold prerequisite, you can't really fiddle with it and learn it by yourself without putting a lot of efforts.

It's a shame, because it really is a simple system to use... provided you understand it, which a lot of teachers don't.


What do you think is a great way to teach git? Let's say for first-year CS students.


I use a toy for 5 years old, that let you construct a 3D structure with magnets, to represent the history, and I stick colored post its on it to represent references.

Important points:

- git log and checkout must be explained in details, as they are a huge source of confusion.

- don't use the famous graphs with revert arrows, they confuse the heck of students

- explain push/pull/clone as late as you can. Most of the course should be on a local repo.

- help them setup diff tools for their particular env. It's harder than you think given the diversity of them.

- you will often have to explain ssh and key management as well. It's not git, but git is close to useless without it. For windows users, you may have to spend an hour of terminal tutorial, putty or not. Use cmder or windows terminal, not cmd.exe.

- do give them a cheat sheet, but only incrementally once they understand the concepts behind them, otherwise they will copy/paste blindly.

- teach them to have a clean working copy before some key operations, and show them why.

- make a big final exercice where they all work on the same code base and push/pull/merge from each others. Twice. Once where you help them. Once to check they can do it now without your help.

- rebase, squash and cherry pick are optional. Do them only if you have plenty of time at the end, and that the rest is well understood.


This deserves its own blog post


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: