"Don't use a self hosted monolithe, it's not reliable! You need a cloud FS with a load balancer under observability and your data in a db that scales horizontally, all orchestrated by kubs."
Meanwhile, I currently have a gig to work on a video service which features a never updated centos 6, an unsupported python 2 blob website, and a push to prod deployment procedure, running a single postgres db serving streaming for 4 millions users a month.
And it's got years of up time, cost 1/100th of AWS, and can be maintained by one dev.
Not saying "cloud is bad", but we got to stop screaming old techs are no good either.
Purely out of interest, I'd like to know more about your streaming architecture. I assume postgres just holds the meta data, and the actual video content is stored elsewhere? What strategies have you employed to scale the streaming part of your service? I imagine 4 million users a month is quite a significant amount of traffic!
1 - For the last 10 years, servers have been beasts. You have a lot of cores, plenty of HD and RAM. Servers are less expensives than devs. Scaling vertically can go VERY far.
2 - Caching is life. We have 3 layers of caching: cloudflare, varnish, and redis. Most things don't need to be real time. A lot of things can be a month old and the user doesn't care. User need immediate feedback to be happy, but not necessary fresh data.
3 - if you compile nginx manually, you get to use a lot of plugins that can do stuff super fast, including serving videos. You can script stuff in lua that will just skip the backend completly.
4 - mind your encoding. We carefully chose how we encode videos. The ffmpeg parameters are pretty insane, but the space / quality ratio is amazing, espacially on mobile. It takes a lot of time to experiment with those, nobody share them :)
5 - we offload everything we can to cron tasks or task queues. Including, obviously, encoding, screenshooting, etc.
6 - don't hold data you can't lose. E.G: billing. This way you can have a relaxed attitute toward data. If we ever loose a day of business, users will be in a bad mood for a week, but that won't be the end of the world. We don't need a bullet proof system if bullets can't kill us.
7 - give money to ffmpeg and opencv, because damn those things are fast. And good.
8 - servers are hosted accross 2 providers. This way, if one goes down, or decide to stop doing business with us Google style, we have a second one. Happened recently with leaseweb: they shutdown a whole room without offering an alternative.
E.G: votes.
They don't hit the backend on write. We pile them from nginx to redis, then once a day, we aggregate and store on postgres, which the backends will consumme. We just store each vote on localstorage as well so that the user feels like it's real time when they vote, but in reality it's updated once a day. But votes don't affect the money side of our business, so if we lose them one day, it does not mean death.
P.S: yes, posgres/redis/elasticsearch only hold metadata. Videos are stored on disk. There is no docker images, no mircoservices, FS is ext4. Which means with a lot of RAM, the OS FS cache will have most popular videos already loaded and ready to be streamed. Everything is raid 0, so if we get one disk corrupted, you lose the server. But we upload each videos on severeal servers, so when a disk get corrupted, we just replace the whole server. In fact, anything goes wrong on a server, we replace it. It's not worth it to find the root cause, unless 2 servers die in the same way successively.
The main One Weird Trick that I remember from messing with ffmpeg is that for in-browser viewing, I had to generate a lot more mp4 reference frames than the default. This was because of deficiency in the HTML5 view extensions in browers a few years ago. I don't know if that is still an issue. The extra frames increased bandwidth consumption, but surprisingly, only slightly rather than big bloat.
- Ca m'en touche une sans m'en faire bouger l'autre: "It touches one of my balls without moving the other one". Actually popularize by French President Chirac :)
- Je m'en branle : "I wank about it"
Seems like involving sexual organs is pretty much international.
And if you want you and your team mates to hate you when they need to work on your code later, and you’ve got random, mystery functions all over the place.
I've been using nim for a few years. I wouldn't have made either design choice (i always do qualified imports, it's just a few extra characters of typing). That said, there is a reason these superficial things always come up in HN threads. If you haven't used a language before, syntax is the only thing you can talk about. It's the equivalent of low effort political banter and the dopamine hit you get from junk food or watching a tiktok. These things melted away after my first 30 minutes with the language, and that's about when the real work began.
This just feels like gatekeeping. I like nim, I've used it, talked to Dom and the author a few times in chat, filed some small bug reports. But even if I hadn't, I'd still hate the case insensitivity; I'm an experienced developer and I don't need to have used a language to know I will like or dislike some aspect of it.
To me it feels more like gate opening: if you feel discouraged to try a language by this specific feature that you do not like, do not fear, come in and try it since as far as we know no people actually using the language suffered from this feature
I don't think Nim has anything that I'd call a "global namespace". By default importing a module will include its exported symbols into the local module, but that doesn't impact anything globally.
In my opinion, this is the correct default, at least for Nim. Operator overloading and UFCS (https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax) don't really work if I have to prefix everything with a module name, and the static nature of Nim means that I'll get a compiler error if there's any ambiguity. There are downsides as well, but language design is all about tradeoffs, and I think Nim got this one right.
Regarding the case insensitivity, I was initially put off by this as well, but in 2 years of using Nim as my primary language, I have never, ever, encountered a real-life issue with it. I've never seen a Nim code base that uses mixed casing, and never encountered or heard of a bug caused by this behaviour. However, it has allowed my own code bases to stay 100% consistent, regardless of the code style of my dependencies, even when those dependencies written in other languages. Contrast this with Python where the `logging` module uses different casing than everything else, so you're forced to use an inconsistent style if you want to consume it. This type of thing is a non-issue in Nim. I think a case sensitive Nim would still be a fine language, but in my experience the pros of being mostly insensitive outweigh the cons.
In the end you may still disagree with both of these decisions, which is fine. Just understand that it's a nuanced discussion, and there's some solid reasoning behind their choices.
Implicit imports on the global namespaces? What do you mean?
Case insensitivity is somewhat complex topic in Nim, as there are some places where the case actually matters, plus there are some more rules for equivalence of identifiers (related to `_` IIRC). In any case, it was never a problem for me in practice; nimsuggest works quite well.
> Implicit imports on the global namespaces? What do you mean?
Not OP, but they are referring to `import std/strformat` (from homepage). That just imported things into the global namespace. Maybe that isn't required and its just for short examples, I don't know.
Go and Python can both import into the global namespace too (`from thing import *` and `import . somepackage`), but they are pretty frowned upon in most projects.
> Not OP, but they are referring to `import std/strformat` (from homepage).
> That just imported things into the global namespace.
That's what tripped me up: there's definitely no such thing as importing into GLOBAL namespace. That's an import into a top-level (true) namespace of a MODULE. In Python, if you put something into the dict returned from `globals()`, you are indeed importing into global namespace, but I've never seen it actually done. Anyway, that might seem like nitpicking, but it's an important distinction. [EDIT: C `#include`s are an example of actually importing into global namespace]
With that out of the way: the wildcard imports are frowned upon for a reason. What reason? Basically, it's hard to tell a) what identifiers were imported, b) if any of them are still used in the module (ie. if it's safe to remove the import). With multiple wildcard imports in one file you also get c) hard to tell where a given identifier comes from.
Now, none of these problems apply to Nim. Nim is statically typed. The compiler and the tooling knows exactly which identifier comes from where. If you delete all the places you used identifiers from a wildcard import, the compiler and the tooling will tell you that it's safe to delete the import. Another problem with wildcard imports is that the imported modules can change, leading to disasters like silently shadowing an identifier imported from somewhere else. Again, this cannot happen in Nim - it simply won't compile.
Every single linguistic feature has its pros and cons, but you have to remember to only weigh those in the particular context they're used in. What I mean is that wildcard imports in Python and Nim are very, very different beasts, and shouldn't be directly compared, if at all. It would make more sense to compare Nim to Elixir or Scala in this regard.
Note: obviously, Nim also has selective imports and whole module imports, too.
Nim having selective imports solves any annoyances I would have.
But you mentioned all the reasons it was ok in Nim. But the answer is always that the compiler knows. That isn’t why I personally dislike this type of import. I dislike it as a user. I find it super annoying not knowing where an identifier is coming from.
I love having ‘fmt.Printf()’ instead of ‘Printf()’. Trivial example I know. I’m sure everything becomes fine once you are familiar with the ecosystem though.
or, if you wanted to keep uniform call syntax, you'd have to invent some new syntax:
"a string".split<strutils>(" ").join<sequtils>("\n")
Which also doesn't look very appealing.
So, there are trade-offs here, and you obviously might not like the choices made by Nim, which is fine! But, there are reasons for the design decisions made by the language implementers, and they are almost never predicated on a single feature: all the features of a language interact with each other, sometimes in very strange and unpredictable ways. So it makes sense to consider the features your interested in the proper context :)
[1] `import modname` and `from modname import fun1, fun2`
I don't mind the comparison but do wish the comparison wouldn't so often be "it's a better Python."
I've been programming Python 14+ years and I've dabbled in Nim and I agree it's a very different language. Yes, there are similarities, but my expectations about it being a lot like Python from the community and docs actually led to some significant frustration and discouragement.
"Inspired by"..."borrows concepts from" are better descriptors IMO. But it would be better if an unqualified "it's like Python" disappeared.
As much as I hate it for correctness, 2 is better than 3 indeed. No sane developer encodes meaning in the case of the name of the variables, in the overwhelming majority of cases it's a typo.
In the C world ALL_CAPS very often signify preprocessor macro identifiers (or some other "class" of identifier), but, yeah..they may all be "insane". ;-)
OMG, what if C did significant case because of that? I mean, it's fine to write "#import", why didn't K&R use a special syntax for macros, like $MACRO or something and left the uppercase as a convention?
I am no C historian, but I believe part of the idea was to be able to have macro wrappers around proper language symbols like "#define sin(x) real_sin(x, 1e-7)" or some such, for example. The "warning character" (what you used `$` for) was thus only for introduction (#define) not for reference. Whether the initial intent or not, this idea certainly came to become an application, also invoked via compiler command line "-Dsin=real_sin" style definition. With this you want the case conventions/sensitivity to be the same in the preproc/macro language as the base language.
FWIW, I do think that the late 1960s saw a broad "new prog lang" evolution away from case insensitive (often ALL CAPS due to low resolution dot matrix printers/display devices) stylistic tendencies in, e.g. LISP and FORTRAN, towards case sensitivity and lowercase. So, I suspect this was the salient driver - "We have lowercase now/text is more legible! Let's use it!". (Yes, mechanical typewriters/typesetting had it for decades to centuries, but programming was not done on those, but with printers and punch cards and just starting to be on CRTs.) There are analogies with Unicode in PLs these days (though there are obviously also other motivations these days/decades with the compute world i18n).
Those are just the two languages I spend the most time in, but, but I'm sure it's trivial to find numerous examples of cases carying meaning in case-sensitive languages. Especially ones which encourage camel casing.
The options are nim c --styleCheck:hint --styleCheck:usages (you need both). Though not on by default you could put it in your config.nims or nim.cfg's to make it on by default for you or for your code.
The global namespacing is what allows UFCS to work. You can rewrite parseInt("6") to "6".parseInt, but you can't rewrite strutils.parseInt("6") to anything.
Yes, you have listed the two things that have stopped me from giving Nim a chance. I especially dislike the global namespaces, it makes it very hard to understand where a function is defined.
It doesn't matter where you are on the scale if the scale of "helping dictators with malicious intents".
Don't get me wrong, I don't think facebook or any big tech companies has any kind of moral upper hand, given they all spy on people, lie, engage in corruption, monopoly abuses, and so on. And they all have weapon dealers in their portfolio, which sell arms to the same people using pegasus.
imho it does. In the "IG Farben" example the managers were held accountable (correctly) because they should have known better, but not the ppl producing/packing/shipping some chemicals.
They had to explicitly pass through the DoD certifications and contracts requirements, and probably had to implement some custom features on Windows for it.
So hes, they are.
Now, the question is: would you rather have lived the cold war without a nuclear deterrent in your country?
You can't eat a smartphone, you can't sleep in an internet connection, you can't do your nursing job through a computer.
Formal education is more expensive. A balanced diet is harder to get, and more expensive. Housing is more expensive near your work. But gas as well if you live far from it.
Today's people debt is higher, and you can't easily work one job, raise a family, and afford education plus buy a home.
Oh come on. People were literally starving in the US in the 60s-70s. Not really something that is easy to happen nowadays, even if it is difficult to get a balanced diet.
One in 9 people in the US suffers from food insecurity (not having regular access to adequate meals). During the height of the disruptions from the pandemic last year, one in 4 households experienced food insecurity.
Why are you bringing up "food insecurity" rather than deaths by starvation which is an objective measure of food shortness, and what the person you replied to is talking about?
How many people starve to death in the United States?
Let's not forget securities. Today most people live in some state of constant existential dread, because they know, if they somehow lose the flat, they can't afford a new contract; they can't plan long term, as their job isn't safe (although, that point is in interesting turmoil right now, /r/antiwork and all); they can't work towards a pension plan; ...
The promise of salvation past generations had with their working lifestyle doesn't hold truth anymore. I think that's why we see more "new religions" popping up right now, like conspiracy narratives, new nationalism, "white man identities" (the group without a distinct identity in the past; their self-efficacy experiences seem to be in decline, too), the MLM-fantasy of self-employment/all-in hacks/dire fight for "untapped markets" (podcasts, porn, stock market ...).
- google reads all your files
- google judges your files
- google has a set of opaque, arbitrary, ever changing list of things that can turn your files into a reason for locking you out
So, basically: s/your files/their files