> It's a big glob of homegrown thoughts and ideas. Some of them are really slick, like build deduplication. Others are clever and hard to reason about, or in the worst case, terrifying to touch.

This is true of packaging and build systems in general. They are often the passion projects of one or a handful of people in an organization - by the time they have active outside development, those idiosyncratic concepts are already ossified.

It's really rare to see these sorts of projects decomposed into building blocks even just having code organization that helps a newcomer understand. Despite all the code being out in public, all the important reasoning about why certain things are the way they are is trapped inside a few dev's heads.

As someone who has worked in the space for a while and been heavily exposed to nix, bazel, cmake, bake, and other systems, and also been in that "passion project" role, I think what I've found is that these kinds of systems are just plain hard to talk about. Even the common elements like DAGs cause most people's eyes to immediately glaze over.

Managers and executives are happy to hear that you made the builds faster or more reliable, so the infra people who care about this kind of thing don't waste time on design docs and instead focus on getting to a minimum prototype that demonstrates those improved metrics. Once you have that, then there's buy-in and the project is made official... but by then the bones have already been set in place, so design documentation ends up focused on the more visible stuff like user interface, storage formats, etc.

OTOH, bazel (as blaze) was a very intentionally designed second system at Google, and buildx/buildkit is similarly a rewrite of the container builder for Docker, so both of them should have been pretty free of accidental engineering in their early phases.

I don't think you can ever get away from accidental engineering in build systems because as soon as they find their niche something new comes along to disrupt it. Even with something homegrown out of shell scripts and directory trees the boss will eventually ask you to do something that doesn't fit well with your existing concepts.

A build system is meant to yield artifacts, run tools, parallelize things, calculate dependencies, download packages, and more. And these are all things that have some algorithmic similarity which is a kind of superficial similarity in that the failure modes and the exact systems involved are often dramatically different. I don't know that you can build something that is that all-encompassing without compromising somewhere.

Blaze and bazel may have been intentionally designed, but it was designed for Google's needs, and it shows (at least from my observations of bazel, I don't have any experience with blaze). It is better now than it was, but it obviously was designed for a system where most dependencies are vendored, and worked better for languages that google used like c++, java, and python.

> This is true of packaging and build systems in general. They are often the passion projects of one or a handful of people in an organization

This is a very insightful comment

Tax revenue per area is generally a good metric in terms of the sustainability of a municipal budget. Very few suburbs in the world have enough people density to clear that bar, no matter how much they tax them. More people density lets you clear the bar so much easier even when the incomes are drastically lower.

Take a look at LA’s budget then, it’s literally all police and police liability payouts which are already hundreds of millions of dollars over the budget for them.

The budget is public, it's all going to police and police liability payouts

We actually finally have a great city controller in LA, Kenneth Mejia, who has been working his but off (and literally getting sued by the city) for trying to un-screw the complete lack of accounting that has taken place in LA's budget.

He's active on socials and would definitely be interested in a concept like this to correctly attribute and predict costs if you reach out.

Good things there are entire fields of medical experts working to understand the exact mechanisms and harm and we're not leaving it up to you.

Not to mention how often we keep catching these companies with explicit policies to make people never want to leave the app.

> Good things there are entire fields of medical experts working to understand the exact mechanisms and harm and we're not leaving it up to you.

No, that doesn't work. Harm is a normative concept, not an empirical one, so there's no role for "expertise" to play in defining it. Medical experts can describe mechanisms of causality, and their associated effects, but deciding whether those effects constitute harm is something that actually is up to each individual to decide, since it is an inherently subjective evaluation.

> Not to mention how often we keep catching these companies with explicit policies to make people never want to leave the app.

Yes, and attesting one thing while doing another is certainly something they can be held accountable for -- perhaps even legally, in some cases. But this attempt at treating social media as equivalent to physically addictive chemicals is pure equivocation, and making claims like this actually undercuts the credibility of otherwise valid critiques of social media.

At the end of the day, this is a cultural issue, not a medical one, and needs to be solved via cultural norms, not via political intervention based on contrived pretenses.

Just to make sure I wasn't misunderstanding you, I double checked the meaning of "normative." This is the first result from google:

"establishing, relating to, or deriving from a standard or norm, especially of behavior."

And other sources have something similar. I'm interpreting your comment as saying "(psychological) harm is subjective, and because it can not be measured empirically, it's not possible to have expertise on this topic."

Fortunately, there are real world consequences that can be measured. If I take an action that makes many people say "ow!" and we acknowledge that expression as an indicator of pain, even though I can't measure the exact level of pain each person is experiencing, I can measure how many people are saying "ow!" I can measure the relationship between the intensity of my action, and the number of people that respond negatively. There's plenty of room for empiricism here, and a whole field of mathematics (statistics) that supports handling "normative" experiences. They even have a distribution for it!

The foundation of law is not scientific exactness or scientific empiricism. It is the mechanism by which a state establishes norms. A law against murder does not stop murder, but it does tell you that society does not appreciate it.

They are saying that judgements of what qualifies as harm is something like a judgement of what is good, or what is right or wrong. That’s not the same thing as evaluating whether something causes pain. You can measure whether something caused pain, sure. (Well, the sort of limitations you mentioned in measuring pain exist, but as you said, they are not a major issue.)

“Harm” isn’t the same thing as “pain”.

I would say that when I bite my finger to make a point, I experience pain, but this doesn’t cause me any suffering nor any harm. If something broke my arm, I claim that this is harm to me. While this (“if my arm were broken, that would be harm to me”) might seem like an obvious statement, and I do claim that it is a fact, not just an opinion, I think I agree that it is a normative claim. It is a claim about what counts as good or bad for me.

I don’t think normative claims (such as “It is immoral to murder someone.”) are empirical claims? (Though I do claim that they at least often have truth values.)

I'd go beyond that and even say that one might consider something harmful, but be willing to endure a certain level of harm in pursuit of something of higher value.

For example, I once asked a smoker why she smoked, and the response was "because I love it" -- when I asked if the enjoyment was worth the health risks, she said "yes; I never planned to live forever". She was making a conscious decision to seek short-term pleasure at the cost of potential longer-term damage to her health. At that point, there wasn't really anything remaining to debate about.

I didn’t mean to imply that the harmful effects of something can’t be worth it for the beneficial effects of that thing. Yeah, if someone is trapped, doing something that frees them and also breaks their arm, may well be an appropriate action for them to take.

> The foundation of law is not scientific exactness or scientific empiricism. It is the mechanism by which a state establishes norms.

Exactly. So it sounds like you're agreeing with me that qualification of a particular effect as "harm" is not a matter of "medical expertise", but is rather a question of subjective norms that is in fact on the opposite side of the is-ought gap from the side at which expertise is applicable.

> A law against murder does not stop murder, but it does tell you that society does not appreciate it.

Well, not exactly. This presumes that "society" in the abstract (a) actually has a general consensus on the question, and that (b) the rules imposed by the legal system reflect that broad consensus, rather than reflecting the values or intentions of the people administering the legal system, without necessarily aligning with those of the general public.

There are a lot of questions that do have broad consensus across society, but also a lot of subjective questions that different people answer very differently. And I think that the level of consensus that actually exists in terms of considering things causing physical injury or pain as "harm" is far, far greater than the level of consensus on treating anything that causes emotional stress as "harm".

I don't think that the "negative response" criteria that you're articulating is sufficient to reveal an underlying normative consensus: I would not presume that most people would equate harm with any kind of negative reaction. For example, I would personally not consider something harmful merely on account of being annoying, insulting, or even morally questionable (though there's often overlap in the last case).

I have to point out that your original post is technically correct because you specified "medical expertise" as the focus of your argument and psychologists aren't MDs. The field has some questionable aspects (and outcomes) to be sure, but I don't think it's completely without merit, and as a consequence, I feel the spirit of your argument is still wrong. You said:

> At the end of the day, this is a cultural issue, not a medical one, and needs to be solved via cultural norms, not via political intervention based on contrived pretenses

It is possible to consider people's subjective experiences in tandem with the consequences of those experiences and make an empirical judgement. The consequences can be quantified, even though the subjective experience itself can't.

If we found that people began committing suicide after using social media, would you suggest this can't be studied, and that a government wouldn't have good reason to want to legislate against social media in these circumstances?

This is really all I'm trying to get at. Replace suicide with depression, reduced quality of life, addiction. Whatever you like. If it holds in the suicide case, it holds in all of them.

> I have to point out that your original post is technically correct because you specified "medical expertise" as the focus of your argument and psychologists aren't MDs.

It's also correct because "harm" is a normative concept, which expertise per se doesn't apply to.

> It is possible to consider people's subjective experiences in tandem with the consequences of those experiences and make an empirical judgement.

Well, no, not really. First, you have to be aware of their subjective experiences, and not just speculating or projecting your own assumptions on to them, then you have to know what criteria to apply to the evaluation of the consequences of those experiences, which can only come from the particular values that they subscribe to, irrespective of your own. And "empirical judgment" is a dubious concept, since, again, judgment is inherently normative.

> If we found that people began committing suicide after using social media, would you suggest this can't be studied,

Anything can be studied, but the extent to which the conclusions of study can be validated for something like this is quite limited. First, you'd be studying something that is a drastic outlier -- only a tiny proportion of the population even attempts suicide for any reason at all.

Second, you're dealing with something with complex causality, much of which can't be directly observed or measured except by the subject themselves, so there's no way to eliminate confounding factors or construct control groups.

Finally, with so many ideological and pecuniary interests attached to a topic like this, it would be difficult to conduct such a study in an institutional setting without it being potentially skewed by bias, and the aforementioned difficulty in setting up controlled experiments would make it difficult for replication to factor out bias.

So I don't think I'd rely on formal studies for this sort of thing, especially when the motivation is to rationalize normative conclusions rather than understand the world as it is.

> and that a government wouldn't have good reason to want to legislate against social media in these circumstances?

No, I don't think that would be a sufficient reason. Even if it were happening, not everything is the government's responsibility, and not every social problem has a political solution.

> Replace suicide with depression, reduced quality of life, addiction. Whatever you like. If it holds in the suicide case, it holds in all of them.

I don't think it holds in any of them.

> The magnitude of the investment also probably makes it impractical for anyone but the very wealthy to retire before that starts paying out...

But they can pull out for housing right? That's an enormous portion of most people's expenses. If I didn't have to worry about housing, I could be living large on less than half of my salary, I would certainly semi-retire at least.

Sort of. So far as I can tell, you can withdraw to buy housing but I don’t think you can pay rent out of it.

The loans are also 75% max loan-to-value so I think until you can get 25% of the purchase price in your account you have to pay CPF and rent (or live with family).

Also, not an economist, but I suspect the forced savings has a wildly inflationary effect on housing prices. You can’t do much else with the money until you retire, so I would guess the price of housing rises up to match the forced savings rate.

> the forced savings has a wildly inflationary effect on housing prices

Housing prices are inflationary independent of CPF, because flats in Singapore are powerful investment vehicles. For HDB flats, however, there is means-testing and rebates to the amount of ~50%, sufficient for anyone on the 30th percentile and above to afford.

Since the government controls the supplies of HDBs, it controls the price inflation.

So it would be more accurate to say “housing prices are inflationary because the government wants them to be”.

Yet this introduces a ton of new problems as well. In order to keep them “good investments” it becomes ever increasing prices with ever increasing rebates to help lower income afford them.

But eventually prices will stop going up.

All housing stock is controlled by governments everywhere through zoning.

American cities could solve their housing shortages in short order but it'd piss off too many people who are "invested' in housing so we accept dead bodies in our streets and social instability instead.

> American cities could solve their housing shortages in short order but it'd piss off too many people who are "invested' in housing so we accept dead bodies in our streets and social instability instead.

I agree with you about fixing the housing market, but I think you underestimate the instability caused by changing housing prices rapidly.

If housing prices drop by something like 25%, a lot of people are going to be upside down on their loans (outstanding principal exceeds the value of the asset). The banks now have mortgages that aren’t fully secured anymore, and borrowers are heavily incentivized to allow a foreclosure unless they’ve paid down the principal by a lot.

We’re talking the 08 recession all over again.

Very few locals pay rent here. Most people buy houses. Its kindof forced thanks to the system, but its designed in a way that unless you are a decimillionaire housing is expensive, but attainable. This is done by splitting the housing market into private and public housing. Is this perfect? No.

And yes it does drive inflation of house prices.

Crazy how it was clearly orchestrated by his wife whose family has had dreams of forcing war with North Korea for some time, but he's the fall guy.

If you play quarterback, you take the blame when things go south even if the coach is the one scheming.

Silly nitpick but I think a better analogy would be the coach takes the blame for bad ownership decisions.

In my years watching sports coaches are almost always the first one to be made the fall guy and I've witnessed plenty of situations where I can't really say they're the one at fault. There are two simple reasons in my opinion. Teams invest WAY more money in players so they have to try to commit to them even if the player is potentially not good enough and owners are never going to go "wow I made some bad decisions I should sell the team". All of this is to say coaches are the cheapest and easiest ones to pin the problems on.

Oh yeah, I mean by all means he should receive consequences.

But he's not the chaebol, he's just a tool for people walking away unscathed to try again at a more opportune time.

Ceph is a non-starter because you need a team of people managing it constantly

I'm not posting to convince people they should use it, just that it's a really cool piece of open source infrastructure that I think is less well known, and I resepect it. It is very configurable and tunable, has a lot of features, command lines, and things to learn, and that does need people with skills and time.

That said, it doesn't need constant management; it's excellent at staying up even while damaged. As long as the cluster has enough free space it will rebuild around any hardware failure without human intervention, it doesn't need hot spares; if you plan it carefully then it has no single point of failure. (The original creator introduces the design choice of 'placement groups' and tradeoffs in this video[1]).

Most of the management time I've spent has been ageing hardware flaking out without actually failing - old disks erroring on read, controllers failing and dropping all the disks temporarily causing tens of seconds of read latency which had knock-on effects, or when we filled it too full and it went read-only. Other management work has been learning my way around it, upgrades, changing the way we use it for different projects, onboarding and offboarding services that use it, all of which will vary with what you actually do with it.

I've spent less time with VMware VSAN, but VSAN does a lot less, it takes your disks and gives you a VMFS datastore and maybe an iSCSI target. There can't be many alternatives which do what Ceph does, and require less skill and effort, and don't involve paying a vendor to manage it for you and give you a web interface?

[1] https://www.youtube.com/watch?v=PmLPbrf-x9g

That's was not my experience. Deploying and configuring ceph was a nightmare due to the mountain of options and considerations, but once it was deployed, ceph is extremely hands-off and resilient.

Yeah sure. I manage a ceph cluster (4PB) and have a few other responsibilities at the same time.

I can tell you that ceph is something I don't need to touch every month. Other things I have to baby more regularly

Cloudflare actually has this as a free tier feature so even if you don't want to use it for your site you can just setup a throwaway domain on Cloudflare and periodically copy the robots.txt they generate from your scraper allow/block preferences, since they'll be keeping up to date with all the latest.