I really wish there was an 80% kubernetes. I think you could get there with some changes:
1. No overlay networks. 1 IP per machine. pods use dynamically allocated ports, and the kubelet enforces pods listen only on their assigned ports using seccomp.
2. No kube-proxy or equivalent Layer-4 "load-balancer". It's not good, but it's often used. You should use some kind of Layer-7 load balancing instead. Also you need to look up the port number from (1). This also greatly lessens the need for DNS.
3. A better config language. YAML and helm templates are terrible. kustomize is built into kubectl, but it's frustratingly limiting and also still very complicated. Something like nix would have been great. This can make it easier to upgrade third party configs since you can have more logic to validate and merge your settings with upstream defaults or templates.
4. Maybe an EBF-like for the api server? If the built-in k8s objects don't have a setting for something, then you need to write an operator or control loop yourself and then run that too, which is a big lift. Over time, k8s just keeps adding more and more built-in things and then revising them, which creates a ton of churn. If you could easily script simple operations, then they wouldn't have to build in every permutation ahead of time. E.g. the HorizontalPodAutoscaler has 24 config object types with several fields each, but all it does is set replicas based on data read from the api-server, so it could be replaced by some kind of flexible script that runs in the control plane.
Unless you hate HCL, 1, 2, and 3 pretty much describe Nomad exactly. We run over 100k production applications on Nomad. But migration to AWS from private data centers, our HashiCorp bill, and the severe lack of Nomad talent, have finally pushed us to k8s (EKS).
1. You can't force third-party software to do that. There are programs with hardcored ports. There are programs which require XML modifications and container rebuilds to change port number. If your platform does not support launching of unmodified containers, it is severely restricted and not suitable for general use. All my programs always use port 8080 for HTTP, I don't make it configurable because I have no reason to.
2. Does not work for all protocols. Again your solution restricts the number of protocols to HTTP protocols. Might work for many uses, but still this restriction doesn't sound very good. Universal load balancer is much simpler conceptually.
3. YAML is not terrible. YAML is awesome. Kubernetes manifests are terrible, that's I agree with. Docker compose is nice, for example. Kubernetes manifests felt like they were designed to be generated from something, but everyone ended up writing them directly or with templates. Though I think that XML generally is superior format so I'd vote for XML in the end.
Overall your suggestions look like you want to shift complexity from cluster operator to software developer. I'm not sure industry supports that, recently it seems to move in the opposite direction, but that's interesting perspective. I guess with some wrappers for some containers it could be made usable.
But honestly you just want to throw away years of progress in containers and network namespaces. I understand that kubernetes mechanisms are somewhat complicated, but the core idea is to make pods look like virtual machines and I think this is very worthy idea.
Even with all its complexity, k8s doesn’t solve every problem — good luck running an FTP server or anything that needs to dynamically allocate a large range ports on k8s.
I would absolutely trade flexibility for complexity. Particularly for edge cases like hard coded ports.
If your working set is 20 TB, then it's pretty big. Each database has its own mix of hot/cold data, so it's impossible to compare without more information. A better measure might be IOPS. RDS has fairly low maximum IOPS unless you spend a lot more for provisioned IOPS or use Aurora.
I have mysteriously lost comments/descriptions I wrote on issues. I figured it was related to a failed and lost opportunistic update like this, although I suppose it could have been caused by a fixable bug.
It's circular since Google owns part of SpaceX. According to [1] they own 7% of SpaceX, so a $1.75T IPO would value their stake at $120B. The target IPO price is >90x revenue, so if Google increases SpaceX's revenue by $11B, SpaceX's valuation could increase by $990B to maintain the same multiple, which would increase the value of Google's stake by $69B.
Not what circular financing means. Buying from a company you own stock in can be a conflict of interest, but it's only circular if you invest in the company and then they use those proceeds to buy your stuff. A past investor buying services from a company they are affiliated with is pretty par for the course in business.
I'm not sure this is true for Google. Ignoring their equity in Anthropic, AI is generally a threat to Google, since it's the closest thing to upsetting their search monopoly. The best case for Google is if OpenAI and Anthropic are way out over their skis, and Google is the only major player left with their sturdier financial position. The worst case is if ChatGPT/Claude completely displace search and nobody wants to pay for gemini. I find it unlikely that all three go down for the same reason.
I haven't worked there in several years, but assuming memegen hasn't wildly changed: Management likes having a pulse on employees, and they tolerate memegen since it's mostly fun, it builds shared culture in a massive company, lets workers (mostly) harmlessly blow off steam, and it would be massively unpopular to shut it down. Management does not like that memegen is often a nexus of cynicism and employee activism. Also in my experience, most employees were nearly completely agnostic or ignorant about whatever trend was on memegen, so it wasn't necessarily representative.
> Management likes having a pulse on employees, and they tolerate memegen since it's mostly fun ...
A long long time ago I used to work at Yahoo. There was an internal mailing list called "[email protected]", which was basically a forum for engineers to let off steam. I used to enjoy the occasional emacs-vs-vim threads, or the ribbing it frequently gave to Jan Koum (founder of Whatsapp).
When Marissa Mayer became CEO in 2012, one of the first things she did was to join this forum, to get a pulse on the developers.
I know this, because my VP comes running to me one day: how do I join this group "devel-random"?
I asked him: are you sure you want to join it? It's a huge time suck if you're not careful.
No, no, he replied; Marissa wants us to join it so we can get a feel for the company (turned out she said no such thing, but you know how senior management is: aping everything that a CEO does).
A couple of weeks later he quietly quit the list. :-D
After 5 years, you'll have an extra $1M in savings, and you can safely pay yourself 4% or $40k each year in perpetuity without doing any work.
This is also a really extreme version of the prisoners dilemma. In the standard formula, there are 2 prisoners, so it's somewhat practical to not defect, but there are hundreds of thousands of qualified candidates for working at Meta in these roles, so your personal decision to defect or not has likely no effect on the ultimate outcome. I.e. for the second option to work, you actually need to organize a unified labor movement with no defectors, which is probably impossible.
These guard types are great and I've heavily used them in the past. But why codegen them?
E.g. the jwt auth example has some major problems since the verification rules aren't fully specified in the spec. The jwt-token verified rule only checks that the string isn't empty, but it doesn't actually verify that it is correctly parsed, non-expired, and signed by a trusted key. The authenticated-user rule doesn't check that the user-id actually came from the jwt. If you hand-wrote your constructor, you would ensure these things. Similarly, all the other constructors allow passing in whatever values you like instead of checking the connections of the real objects.
By calling the constructor for these types, you are making an assertion about the relationship of the parameter values. If AI is calling the constructor, then it's able to make it's own assertions and derive whatever result it wants. That seems backwards. AI should use the result of tenant-access to deduce that a user is a member of tenant, but if they can directly call `(tenant-access user-id tenant-id true)`, then they can "prove" tenant-access for anything. In the past, we have named the constructors for these types `TenantAccess.newUnverified`, and then heavily scrutinized all callers (typically just jwt-parsers and specific database lookups). You can then use `TenantAccess.{userId,tenantId}` without scrutiny elsewhere.
I think you're right on the substance. A production-grade spec (or guard type) needs stronger assertions than the toy example in the post — predicates for signature verification, claim-binding, and expiry-from-token, at minimum. The example is only illustrating the proof-chain shape, and isn't a good example of a full-fledged JWT validator.
Your underlying point, that calling the constructor is the assertion so AI passing `true` can "prove" whatever — is true of any smart-constructor pattern, including your own `newUnverified` approach. The trust still has to live somewhere. In your pattern it lives in the small set of audited callers; in shengen's case it lives in the same place — the wrappers (like `CheckTenantAccess`) that actually establish the premise via a DB query or a JWT parse. Structurally the two approaches are doing the same thing. To harden it, you'd keep the raw constructors package-private and export only the wrappers, so the handler code the LLM is writing physically cannot call NewTenantAccess(..., true) — only CheckTenantAccess.
On the deeper question about "why codegen": the short answer is "obviously, you don't have to." But if we assume that we're using AI to write at least some of the code, now you have to either (1) describe the constructor in very precise English and have the LLM generate it, (2) inject yourself into the loop closely with the LLM, or (3) not use an LLM for this part. My proposition is that writing the core invariants as proofs that can be deterministically checked for internal consistency and written declaratively is (1) more efficient, (2) less lossy, and (3) easier for the developer to read and reason about than writing the constructor from scratch. This puts a lot of trust in the codegen, as you point out; but as a practical matter, having a formal representation of what you want plus an English prompt is stronger context to the LLM anyway.
The other reason I started down this path, which I didn't get into in the post because I haven't figured out yet if it's truly practical, comes from a property specific to Shen: it has a very small kernel that has been ported into a lot of runtimes — Lisp, C, JS, Go, Python, Erlang, Scheme, Java, etc (https://shen-language.github.io/#downloads). That opens up the possibility of writing specs whose predicates run as runtime gates from the same Shen expression, no translation step — and even mixing compile-time and runtime assertions into the same spec. I find this very interesting conceptually, but I'm not sure yet whether it's practically useful for anything.
Specifically it decrements the TTL of routed packets, so hotspot traffic will tend to have a TTL of 63 instead of 64. You could theoretically disable this at the risk of creating infinite routing loops, although android probably makes it inaccessible if the kernel has a setting for it at all, so you might have to rewrite packets in user space.
It has been a long time since I've done this, but:
If your Android is rooted, it's pretty easy to get tethering working. There's magisk modules that can fix the TTL problem and/or disable the hidden carrier-installed software that Android will ask for permission before enabling tethering.
Yes. The original Dapper used 64 bit trace ids and collisions were rarely a problem.
If you don't drop any spans from a trace, you can completely disambiguate a collision since the trace will have two distinct root spans. If you are missing spans, you might have a break in the parent-child links.
Even with infinite retention, your analysis will bucket by time somehow, so a collision might have no effect if the collision doesn't happen at a proximate time. If you are manually looking at traces, it will be very obvious there is a collision unless they happen at the same time.
Also, birthday paradox only expresses probability that there is a collision somewhere, but if you are filtering or looking at single spans, then the probabiliy that you actually see a collision is greatly reduced.
I think for basically all systems, an additional 64-bits has insignificant additional cost, so you may as well prevent collisions, but I think it could be a reasonable tradeoff if it mattered.
I thought it was clear and am also surprised by the reaction (en-US speaker). "Is/are expected" is generally used as a passive-voiced form of "we/they predict" (obviously without having to specify a specific pronoun). E.g. "It's expected to rain tomorrow" means a weather forecast says it will rain tomorrow and usually not that people want it to rain tomorrow.
I wonder if this phrase has different connotations among other English readers? A lot of these comments are fairly early for US timezones.
I don't think US vs. non-US has anything to do with it. It's an ambiguous phrase, whose meaning is usually resolved by context.
"It's expected to rain tomorrow" is a prediction, whereas "students are expected to behave themselves" is an expectation (with consequences, presumably).
In the former case we clearly aren't saying we want it to rain, just that we believe it's likely, whereas in the latter example we are clearly expressing that we do want students to behave.
It's ambiguous because "expect" has two different meanings:
1. No overlay networks. 1 IP per machine. pods use dynamically allocated ports, and the kubelet enforces pods listen only on their assigned ports using seccomp.
2. No kube-proxy or equivalent Layer-4 "load-balancer". It's not good, but it's often used. You should use some kind of Layer-7 load balancing instead. Also you need to look up the port number from (1). This also greatly lessens the need for DNS.
3. A better config language. YAML and helm templates are terrible. kustomize is built into kubectl, but it's frustratingly limiting and also still very complicated. Something like nix would have been great. This can make it easier to upgrade third party configs since you can have more logic to validate and merge your settings with upstream defaults or templates.
4. Maybe an EBF-like for the api server? If the built-in k8s objects don't have a setting for something, then you need to write an operator or control loop yourself and then run that too, which is a big lift. Over time, k8s just keeps adding more and more built-in things and then revising them, which creates a ton of churn. If you could easily script simple operations, then they wouldn't have to build in every permutation ahead of time. E.g. the HorizontalPodAutoscaler has 24 config object types with several fields each, but all it does is set replicas based on data read from the api-server, so it could be replaced by some kind of flexible script that runs in the control plane.
reply