I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints.
On first load, not only the page is ready, but the video has started to auto play (including buffering), under 3 seconds. Less than 500ms on a reload.
The whole project has been running for 10 years on 7 servers. Hosting costs under $1K/mo.
And even better, the code base is a mere 11K LoC, so a single dev can manage it. And make a living out of it.
That's because:
- Python is almost never the bottleneck in a website
- Python wraps compiled languages for slow stuff. That's the whole point.
- your archi matters much more than the language you chose. Because good db providers, proper caching and optimized data fetching will be the pareto moves, unless you are google size.
- you will not code most things anyway. You are not going to code the video encoding, you will use ffmpeg. You will not recode a database, you will use an existing one. You will not code a server, you will use nginx or something else. What you will spend time on is creating logic for your clients.
So I think your are extrapolating this rare use case where your POV makes sense, and ignore the 99% of the case were, indeed, a slow language doesn't matter.
In fact, if electron success tells anything, it's that while I do value speed a lot, the market has voted.
> I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints.
Sounds much more likely that you wrote in python the code that bundle together the libraries that peform the actual network streaming, machine learning, encoding, search, etc, and that those libraries were not written in python.
Where in the stack do you see that Python performances matter?
Only one software out of a 1000 will actually need it. The others will mostly do plumbing or automation.
And scripting languages excel at both.
If you don't fall in that category, then yes, do use a fast language. If you are coding a chat server, use Erlang. If you want to do a rendering engine, use Rust. But if you most likely want to code your company CRUD software and glue it to some external services and providers, use Python.
> Where in the stack do you see that Python performances matter?
As we just saw, most of the code forming your stack is not written in python because its performance would not allow it.
> Only one software out of a 1000 will actually need it.
Not what I see from your example.
It seems to me that you are overinflating the volume of the python code that you wrote and are blind to the huge volume of performance critical code you just take for granted.
You probably mean: most of the code that most of the programmer write is disposable code assembling reusable libraries. This code performance does not matter therefore python is fine.
I agree on this. The problem I see with this reasoning though is that since python is so slow and error prone compared to compiled alternatives that it is unfit when scaled up from small disposable programs to largish permanent software, in a world where all programs trend to grow and stick in production for longer than expected.
3 seconds to load and $1000/mo hosting costs is way over the unacceptability threshold for me.
I don't know what is the website you're running or how many people use it or how much money you make. I know you can make a lot of money doing stuff in python or electron. Does that mean the market has voted that performance doesn't matter? I don't think so. Jira makes a lot of money but almost everyone hates it. I know a lot of other software that is very slow but still makes tons of money.
> you will not code most things anyway
You say that like it's a good thing! It's a bad thing.
You may say it's because you don't need to. But the flip side is that you can't, even if you wanted to.
You can't program a database or a video encoding or a web server in this language you are using to write your program.
So what is the point of this language?
You can't even use it to cache things in memory. That's why everyone uses redis or something similar.
Briar is really neat, and I hope it will land on Linux, Mac and Windows. Being able to send messages and files directly to anybody on a local network without any account just rocks.
It's very slow for big files though, and requires several clicks on the 2 devices. If you are on the local network, kde-connect is still the golden standard provided you use linux (even with gnome) and android:
- click on the top bar menu to select your device
- select the file
- done
It uploads automatically in the download folder of the other device. It uses the local network speed. Doesn't require additional clicks.
But as soon as I'm on 4G, telegram saved messages is what I use too. In fact, I use it to takes vocal notes, pictures of stuff to remind me later, schedule reminders (yes, it has delayed messages!) and so on. It's my inbox basket for GTD at this point.
Any valid json can be parsed by any 1.2 yaml parser. That's how it gained popularity in the first place: you didn't have to migrate. Like ascii and utf8.
I've tried dhall, cue and jsonet, and cue is so far my fav. It's very well designed, expressive, but restrictive enough so that config files don't look like scripts.
The way it blend types and values makes learning it super easy, yet you can do complex things with few lines.
But the main implementation exports to yaml without quoting the strings, which kinda defeat the purpose :(
I’m kind of unsure about the way CUE achieves reuse: if I understand correctly, you have files in a directory tree and the (result of processing the) bottommost files are the things you’re supposed to point your consuming tools at. So there’s no way to share structure among a collection of items if that collection is nested inside your config, the only operation available is essentially the generation of a set of similar but separate configs. Or am I wrong here? I’d very much like to be.
(Also, the type system is absolutely delicious, but it badly needs a paper with a complete description. I’m extremely interested in how it works, but fragmentary “notes on the formalism underlying” CUE are not enough.)
CUE is based on Typed Feature Structures, which predate Deep NLP, and for which there is limited literature. We do need a good writeup on the theory. I've written a bit here: https://cuetorials.com/cueology/theory/
Think of a graph with lots of attributes within which paths are searched for.
You do have imports and functions so you can reuse what you want.
The doc is also quite clear and rich, but the way it's organized means I have to read it entirely before writting my first CUE file. It also lacks IRL examples so trials and errors were my best friends.
I actually use CUE for large configuration files, used YAML before and had many issues once configurations became larger and more complex. It validates and exports JSON, which is easily readable in C++ and Python :-) have been happy doing the switch
On first load, not only the page is ready, but the video has started to auto play (including buffering), under 3 seconds. Less than 500ms on a reload.
The whole project has been running for 10 years on 7 servers. Hosting costs under $1K/mo.
And even better, the code base is a mere 11K LoC, so a single dev can manage it. And make a living out of it.
That's because:
- Python is almost never the bottleneck in a website
- Python wraps compiled languages for slow stuff. That's the whole point.
- your archi matters much more than the language you chose. Because good db providers, proper caching and optimized data fetching will be the pareto moves, unless you are google size.
- you will not code most things anyway. You are not going to code the video encoding, you will use ffmpeg. You will not recode a database, you will use an existing one. You will not code a server, you will use nginx or something else. What you will spend time on is creating logic for your clients.
So I think your are extrapolating this rare use case where your POV makes sense, and ignore the 99% of the case were, indeed, a slow language doesn't matter.
In fact, if electron success tells anything, it's that while I do value speed a lot, the market has voted.