Hacker Newsnew | past | comments | ask | show | jobs | submit | more BiteCode_dev's commentslogin

Packaging for debian anything non trivial is damn too hard.

It took me 5 days to figure out how to package a complete web app with params, upgrades, post install transpiling, db init, etc.

And I haven't put that on a private repo yet, it's yet another annoying thing to do.

Nothing is well documented, doc is old and confusing, the tooling is archaic and wants to inflict pain (debconf anyone ?), the life cycle of a Deb package is atrocious to get right.

And you have to do all that in raw bash scripts. Not there are no alternatives, any scripting language is potentially usable, but the support is poor enough to deter you from them.

It's not I don't want to contribute to the ecosystem, but I won't invest the colossal effort and exercice in frustration to overcome the barrier to entry. My packages don't even need to go to the official repo, just let me do my things in peace.

Make a python lib that let you describe a package, hook on life cycle events to run code, with clear documented recipes et where to put what types of files, and let me run that to generate the Deb. Event web pack is easier to use for God sake.

I'm not even touching the process of packaging something to be included in debian repositories here, which is another beat entirely.

Quit the smug act, debian packagers. You don't know better.

You do know better on how to design a distro and protect the official repository. Great. I praised you for that for decades.

But you know nothing about making your users life easy. You just don't. So ask them, and fix that, or don't complain about no contrib. This is not news, we raised our voices for years.

I contribute all the time to Foss in code and doc, I donate in mass. We ARE willing to help. And we do.

It's not us. It's you.


As for building packages for your own use, the tooling is not that difficult. The documentation is just oriented to contribute packages for Debian itself. This implies some unwanted complexity and navigating outdated documentation. Packaging for Debian becomes simpler, but it's heavily rooted into having multiple files describing the package, use of a tarball as the source package and having each language comes with its own helper and documentation.

Here is a "pragmatic" way to write Debian packages without relying on non-Debian tools: https://vincent.bernat.ch/en/blog/2019-pragmatic-debian-pack... (I am the author).

But other than that, I quite agree with your stance: we (Debian) makes life of our contributors harder than it should be. Unfortunately, there is no awareness in the project for that. People trying to change that just silent themselves, like https://michael.stapelberg.ch/posts/2019-03-10-debian-windin....


I also made myself a deb packager script in 100 lines: https://github.com/qznc/simpledeb

It doesn't cover anything fancy like post-install actions though.


Thanks for this tutorial, it covers a lot.


> And you have to do all that in raw bash scripts.

If only that were true. The entrypoint in Debian packaging is a damn Makefile.

I've been a Debian Developer for 16 years, and creating packages for more than that. Saying that I like it or that there aren't problems would be the archetype of Stockholm Syndrome.


On the other hand, Debian is a large community of volunteers providing a distro on democratic principles for a long time successfully. Comparable distros are either companies (Ubuntu, RedHat, SuSE) or heroic deeds of one or few people (Slackware). I find that an amazing even if it is heavily burdened by technical debt these days.


Oh, sure.

But that's beyond the point that if what they really desire is contribution, which they have been claiming for years, they should make it less painful to package software for the plateform, which we have been claiming for years.


You have some points here, but the Bash part is _not_ the problem. Bash is everywhere and the true fix to that bit is to learn Bash: It's not hard and I benefit from my Bash skills everyday, it's one of the best things to invest even a bit in. Just don't write "actual software" in Bash.


Oh, but bash is definitively part of the problem.

I'm pretty proficient in bash myself, and error handling, debugging, decoupling and refactoring in bash are all horrible. It's just not design for being good at a script with more than a couple of lines. The fact we do use it for such purpose does not make it good at it.

The fact it has no namespace, the lack of decent data structures and the very limited functions also all leads to making it a poor foundation to build a lib on. Hence deb packages don't have any kind of framework for basic things: you are on your own.


I think many package managers take longer than a week to figure out. RPM is more difficult that deb, setuptools takes months to understand it fully and is changing constantly.

Meson, Conan, winget+msi, and the hundreds of other mixtures of build system + package manager also take long to understand. And you have to understand each of them.

Homebrew does not even want package authors to create packages themselves.

It may be a matter of perception: If you count all web searches, interactions with CI, conference talks that help you understand a more recent package manager, it will also add up to 5 days.

Creating a deb is more boring, it's just the machine and you.


I'm not counting the research I did prior the 5 days to actually understand deb packaging, only the 5 days trying to package something.


Well, just me finding it a joy to work with? All the dh helpers that basically does the hard work, I just need to update one or two things in control file and that is it.

I have packages closure, Java, python, php, c and very complex piece of software with weird version formats and so far Debian packages cover all those cases for me.

I find that is pretty easy to complain about things we don't know, or rant when frustrated, but after investing a bit o time to learn and understand how it works pays off.


Back in the day, I created my packages manually (creating the TAR archives, the preinstall/postinstall scripts, and putting all of that in an AR archive, renaming it to .deb).

I never even read the documentation, I just did:

  $ file some.deb
  $ ar --output somedir some.deb
  $ ls somedir
Definitely not the recommended way, but I never spent more than 10min on a Debian package (installed with dpkg and not through repositories).


I'll put it like this: Debian bureaucracy makes your usual governmental bureaucracy seems sane and sensible in comparison

No accountability, tightly knit cabal with high barrier to entry, refractory to external opinions, etc


Yes. It’s a living breathing example of how Anarchy operates.

Which was the intention I believe - hence why https://packages.debian.org/stable/doc/anarchism is a package.


Creating arch or nix packages is a breeze in comparison. I'd argue nix is what everyone wants whether they know it or not. All this container junk is a recipe for disaster.


I had to create a custom package of some C++ header files. I failed. I was shocked at how hard it was.


For my (local) C++ projects, I use CMake to make .deb packages. It is easy enough, uninstalls cleanly, so does the job.


If some scientist comes to you and state "we proved that people can get hurt ridding bikes", it's fair to point out that there is little surprise about the formal proof of something so many people empirically experiment in their daily life.


I concur, I had 140 btc 10 years ago. Sold it all when it reached $800, which I though was peak.

I doubt the guy who sold the 5000 btc pizzas still have them.


I always though it would be a nice alternative to an ORM: having a tool that take a marshmallow/pydantic/whatever model, optionally passing it additional db specific options, then it generates a bunch of sql files you can call with aiosql. The whole things would then let you optionally get the result wrapped in a model if you need to, with ORM like helpers for common CRUD things.

That would have the benefit of the standardized api of an ORM and the flexibility of SQL, without the coupling.


Absolutely! I think it should be doable, FastAPI and other libraries do heavy Pydantic model inspection, and some do automatic code generation. I should explore this a bit more.


It also has some nice properties:

- because you have all your queries written in the same place, when you create a db migration, you know where to look for code to update, and can even perform some automatically.

- sql queries could be turned into a stored procedure with a simple marking, without changing any code.

Of course the downside is that dynamically exploring data is not as easy as with an ORM.


> list(map(lambda n...

List comprehensions are much better for this. Functional doesn't mean you have to use a function call. If you can use the paradigm with literal syntax, just do so.


I never said functional means you necessarily have to use a function call, and I also understand list comprehensions are semantically functional, but sometimes you do need to use a function call.


They don't really compose nicely. (At least from my point of view, as I prefer fluent interfaces, eg. those you usually find in Rust/Scala/Java.)


Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.


The ecosystem.

You got loads of libraries built around flask, and a lot of doc and tutorials, not to mention how much battle tested it is.

Those can matter much more than hypothetical perfs in a synthetic benchmark.


There is absolutely a lack of a decent SQLA tutorial for IRL code.

The doc is very formal, and if you want to make a website, a quick script or a data migration, you are left on your own to find the proper setup.


> Beware and use Django until you’re sure you AND your team know what you’re doing.

That's one of my pet peeve in the Python world: people thinks Flask is for beginners and Django for advanced users. The Flask API is simple, the hello world is 5 lines. By the time you render your first Django ViewModel, you have read and tried out the full Flask doc.

So it's easy to say, "I'll start with Flask, and check out this Django thing later".

But in reality, it's exactly the opposite. Flask requires you to know a lot of things and make a lot of decisions, not to mention select/write additional libraries for features Django packs.

So if you use Flask for anything non trivial, you should show your trade or you are going to mess up, big time.

If you are a beginner, you should always start with Django. Yes, it's more work up front, but it will give you an idea of how important concepts work, plug in to each others, and can be organized in a code base.

Django is not perfect, and its ORM is really far from SQLAlachemy, but it's good enough for a loads of thing. And it will be mostly secure and extensible by default.


100%

The upfront cost of Django makes people thing it's much more complicated than a micro-framework like Flask.

I like Flask, but not because it's a simple easy to use framework, especially once you chuck SQLAlchemy into the mix. Shit, even now if I don't touch it for a while I end up trying to remember how things work


If you don’t use Django’s ORM or SQLAlchemy’s ORM, then do you really need to use Django?

Like an adjacent commenter said, ORM is ok for simple blog websites. But I can’t imagine using an ORM for complex data tables and relationships with normalized data.


I used the Django ORM for surprisingly complex applications

I created a video streaming website, with 600k visitor days, that features tagging, categories, related vidéos, user upload, encoding queue, search with auto-completion and so on. It relies heavily on the ORM.

Django migration system and db model generation also make it a very nice tool for data export/migration/exploration.

The ORM provides a nice API, but above all, a standardized one, which lead the Django ecosystem to become so reach because of the assumptions the libraries could do, making you very productive.

Now, one has to remember:

- you can, and should by pass the ORM if you need it, writing raw SQL. Django let you do that.

- plenty of things can end up be in a cache, using varnish, redis, etc. Plus a lot of things should be done in task queues anyway, not in the django views. So the cost of the ORM is rarely an issue.

- complex data tables are sometimes better presented using SQL views, making the ORM model simple.

As usual with coding, it's a matter of gain and cost, and the right answer is "it depends". But you can go very far, and very comfortably with this tools, as long as you don't fight against its nature.

So yes, definitely use Django with the ORM. I am not advocating to use Django + SQLA, but rather contrast Django with Flask + SQLA.


>But I can’t imagine using an ORM for complex data tables and relationships with normalized data.

Whenever I read a statement like this from someone, I always feel that they actually haven't thoroughly read the docs for Django's ORM. Can you give me one concrete complex example that you can't imagine doing with the ORM? Because 95% of the time, these "complex" models are trivially easy to pull off with the ORM (from my experience).


Django orm doesn't support composite primary keys.


The beauty of the Django ecosystem, is that you have packages for everything: djangopackages.org

E.G: https://github.com/Arisophy/django-compositepk-model


Fair enough. There is `unique_together` though, which you may look into.


In the context of database transactions, any APIs serve only as backend endpoints. For this purpose I question whether flask is the best choice. For rest API, I always go for Falcon.


While I do keep my eyes on FastAPI, even in this case I would recommend Django to a beginner, solely because of the amazing Django Rest Framework.

Not only it makes creating an API from ORM models quick and easy (although the documentation doesn't let you believe so), but it also let you do custom views seaminglessly, including goodies such as standardized permissions, authentication and pagination with several levels of granularity. Having that being able to be integrated with all the rest of the Django ecosystem transparently (your website, a wagtail CRM, etc.) is really sweet.

But the biggest advantage for a beginner is again that it will prevent them from having to make decisions they are likely to get wrong. Especially on the serialization, format, identifier policy and homogeneity of the API: people crafting that by hand often make a big mess and don't even know about it.


It's more of a design issue here. You could do away from classes while keeping types by using 2 lines:

    Expr = Callable[[str]], Callable[..., Iterable[str]]] 
    AstNode = Expr | Callable[... ], Callable[..., Iterable[str]]]
You then don't event need to call representation() , as each node is callable. No classes needed.

But my instinct tells me a node will eventually have more attributes and methods. Stuff to get parents, children, siblings, filtering and so on.

So it's probably a good call to use a class for later. Also you want to keep AstNode as a node could be an expression, a statement or a comment.


> But my instinct tells me a node will eventually have more attributes and methods.

I couldn't agree more with Jack Diederich's thoughts on that sentiment, at 5:40 in the video linked above:

https://youtu.be/o9pEzgHorH0?t=340

> Lots of times people think they might need something later. You don't. Or, you can just do it later.


https://docs.python.org/3/library/ast.html

Experience helps with this.

It's a middle path as usual: naive approaches waste as much time as overenginering.


YAGNI is a double-edged sword. It's almost no additional overhead to write a class now if you think you might need to add methods and internal state later. But it's quite a bit of refactoring work if you decide not to write a class and then realize you need one after the fact.

Maybe this is an indictment of Python as a language, but I think it's a case of not taking anything too literally. YAGNI fetishism is like DRY fetishism all over again.


I guess? At the same time, working in some languages (Racket and C come to mind), it's surprising how long I can go without anything ever becoming a class, ever. Seemingly indefinitely.

My sense is that many of us are taught to think in terms of classes and methods instead of data structures and functions, so it's very easy to slip into thinking you need objects in order to manage state, even when that's not the simplest way to do it.

Strictly speaking, though, I think that the only time in Python that you really have no workable alternative to classes is when you also need dynamic dispatch.


Personally: it's much easier to YAGNI in languages with relatively-strong compile-time checks and relatively-sophisticated tooling.

If you lose either of those, a little bit of preventative work goes a loooooong way. The friction (largely due to risk) of correcting things later is so large that it just starts snowballing unless your team is extremely attentive.

In that sense, Python is... middle of the road, with an extremely broad spread. Tooling is an odd blend of surprisingly good vs incapable of doing simple inference / detecting all calls, and type safety depends heavily on how you write things (some tactics are fairly strong, many are so weak as to be useless or make things much worse).


Personally:

Python has a great type checker. Several, actually. They're not compile time, because Python doesn't exactly have a compile time, but still. 11/10, use everywhere, spend less time on unit tests, be happy.

Second, to me, these sorts of concerns about refactoring are. . . maybe not a smell, but worrisome, all the same. They suggest poorly-modularized designs with no internal component boundaries that might have limited the scope of impact of such a change. That said, even if you do find yourself needing to do a far-reaching refactor like that, refactoring in a dynamic codebase isn't necessarily harder, it's just different. Not being able to lean on the compiler to help you boil the ocean, for example, is fine, because boiling the ocean wasn't necessarily the best approach in the first place. The strangler pattern will help here, not just with type errors, but also with other problems the compiler might not have been able to help with, and will also be friendlier to the people who do your code reviews.


> Python has a great type checker. Several, actually. They're not compile time, because Python doesn't exactly have a compile time.

Python does exactly have a compiler time, since it actually compiles things.

The typecheckers are run before that (but I think that's usually true of “compile time” checkers, too, they just don't break between checking and compilation; the analysis I would imagine almost invariably comes before actually generating compiled code; especially for languages where, unlike Python, typing effects what code gets generated rather than only whether the typechecker passes or fails.)


To echo the sibling - I believe the most important skill a developer can have is reworking* code, and doing it readily and continuously. If you can do that, then you don't ever have to worry about what code MIGHT do in the future, you just design it for what it DOES do and then rework it later if necessary. It's quicker and the result is cleaner. Also, even if you tried your hardest to predict what was going to be needed, there are always things you can't predict and at that point you either rework your code or have bad code.

* I could have said refactoring, but some people have very specific definitions attached to that word so I use something more general here


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: