It doesn't have to be the statement, see the recent incident where someone used an LLM to generate a lean refutation of the collatz conjecture, the lean proof exploited bugs in the lean kernel.
That proof was artificially constructed specifically to show the exploit. It wasn't a real attempt at a proof that was later shown to be using an exploit.
I'm unaware of any serious proofs that have been shown to have a kernel exploit in them.
That's a different argument than ijustlovemath is making
I agree with Jtarii that it's very unlikely a Lean bug is critical to most of these proofs. But we're in strange times, so I agree wtih the sentiment that we should wait for further analysis before declaring complete confidence in the proofs.
No one is disputing that the Lean formaization of Navier-Stokes is correct, so we should have high confidence that the generated Lean proof is valid.
The authors are claiming that the Lean proof is not the same proof as the NL one. Therefore, we shouldn't yet have confidence that the NL proof is valid.
This is an important claim which the math community will need to work through. However, the Lean proof alone is sufficient for OpenAI to (reasonably confidently, leaving aside questions of academic manners) claim to have proven NS.
So, the AI wrote a NL proof of Navier-Stokes, then incorrectly auto-formalised it to Lean, but still ended up with a verifiable proof of Navier-Stokes? That seems... strange?
This same thing happened back in the 10 advances in math and CS release a month or two ago. The non sofic group construction relied on false prior literature. They realized this and fixed it in the lean program but didn't modify it in the writeup, so the written proof was both incorrect as stated and did not correspond to the lean proof. I was surprised how little press it got at the time, it seems like a huge risk factor.
Or the AI wrote a NL proof of Navier-Stokes, began rewrite in Lean, then discovered a false / handwavy / easier to write in Lean / etc approach of some parts of the proof, and modified it accordingly. Since there wasn't any backpass from Lean to NL to include any changes it did due to any of the above reasons, the proofs aren't identical. That's what I think is most likely.
If the reason for the differences was done intentionally in Lean (as opposed to hallucinate e.g. m+4 vs m+5 as mentioned in remark 3.2), then a simple recording of differences, and then afterwards pass back any changes to the original NL would fix the issue. If it was hallucinated, then there is no guarantee it wouldn't keep hallucinating, and thus you might never end up with the same proof no matter how many passes you do back and forth (see remark 3.4).
I don't know if we can really be clear about the order of things, but I think even without AI maths is filled with "someone provides a proof of X, and the proof itself is wrong/incomplete but X itself is true".
"Incomplete" proofs might be a way of viewing this. You have a NL argument to prove X. It turns out the NL proof has holes you can drive a truck through. So... you go around and patch the holes.
The resulting proof is different! You can start off with a bad proof and find a correct proof. Sometimes.
EDIT: for French speakers (maybe autodub gets you there) I saw a very nice simple case of this recently. A commonly stated proof for a relatively simple theory. The proof has a giant hole in it, and completing it requires some work [0])
e.g. one of the lemmas in the NL description is false, but a weaker version of the lemma (that does hold) is sufficient for the proof, so the (incorrect) lemma is never formalized.
It's not actually the same model that solved the problem that did the translation. Astra did the translation after the intenral model produced the NL Proof. As for the discrepancies, It's not necessarily right to think of this as 'incorrect formalisation'. Maybe it was essentially a 'proof refactoring'. Maybe Astra thought some parts could be easier expressed in a certain way, or maybe aspects of the NL proof were kind of handwavey etc.
Why do you assume this order? I would assume the AI starts with lean (that's what it was RL'd on at least) then tried to translate the proof for us humans, which is quite hard.
I'm not sure about it but it seems plausible given the potential error.
Edit: they explicite say that your order is correct in the post
I would expect it's the other way around. The AI wrote a Proof of NS in lean and write up a NL proof based on the lean one. The authors claim that openAI did NL -> Lean, but that is unsubstantiated.
> The agents arrived at their resolution on Saturday, September 5, about 88 hours after the first agents were launched. Lean formalization and verification took an additional 17 hours via GPT‑6 Astra.
This seems pretty clear that they found the proof first and then mechanized it afterwards.
Trying to map this observation to code and appreciate them starting with very simple examples (that Astra screw-fixed into lean.)
But in a simplified way this is close to promting the model to create a set with the members 1,2,1,5 and it correctly creates {1,2,5} in code. So The initial "proof" / instruction was wrong and it silently fixed that. ( Which is one of the failure modes they're describing)
Proof steps can become different in Lean but still work to prove the theorem. Incorrect proof steps in the NL paper could also be converted to correct ones in Lean. The paper shows some examples of this. So the NL paper can't be trusted even if Lean verification passes, but it doesn't immediately mean that the NL proof is wrong or that Lean is checking the wrong thing.
Or maybe the AI didn’t write Lean proof at all, or rather, not the LLM at least. But instead OpenAI has an internal traditional reinforcement model that is able to stumble on the Lean proof by the share amount of compute power available to them thousand monkeys on a thousand typewriter style. And then pretend LLM did it because that is what they are selling.
I don’t. But I do know how the scientific method works, and OpenAI’s display is anything but. Until what they have demonstrated is reproduced I take their claims to be nothing but marketing. A for profit company will lie in order to maximize their profits. Above I presented an alternative hypothesis, which is probably wrong, but until OpenAI’s results are replicated I will believe my alternative hypothesis just as much as I believes the claims of the for profit company making them.
In the absence of evidence, a guess is the best you got.
And, no until OpenAI’s results have been replicated, I don‘t believe companies own words about their own products, and therefor a marketing statement is not evidence.
It's the method they used to create the proof that's in question here. They claim their amazing product did it, and therefore you should buy their product because will do amazing things for you too!
>No one is disputing that the Lean formaization of Navier-Stokes is correct, so we should have high confidence that the generated Lean proof is valid.
These authors don't seem to be disputing that this Lean formalization of Navier-Stokes is correct. I don't think that gives us any new information about whether the generated Lean proof is or isn't a valid proof of this N-S blowup thing.
> No one is disputing that the Lean formalization of Navier-Stokes is correct, so we should have high confidence that the generated Lean proof is valid.
Why? It is a correct proof. Is it a correct (dis-)proof of NS? These statements do not seem particularly related. People, including (especially?) mathematicians, find NL proofs far easier to read than Lean proofs. I would bet the ratio of people who read (part of) the NL proof vs the Lean proof is at least 100 to 1. It seems incredibly unlikely that anyone has read and reviewed the whole Lean proof (including the authors of the OP).
Is it particularly easy to see that the Lean proof does actually disprove NS, rather than proving something similar but not the same?
Well the theorum statement (what it proves) is short and easy to verify. I would not be surprised if hundreds of mathematicians have looked at it at this point.
The thing is that Navier-Stokes has a definition split off separate from the formalization, and that is what has been completed. People have looked at the definition of the final statement. This paper only mentions the proof and intermediate statement, not the final statement. The most likely case to me is that intermediate statements do not match, but the end result still holds.
Seems kinda odd then that it didn't occur to OpenAI to iterate until they reached a fixedpoint for both the informal & formal development b/c it's obvious that correspondence should have been part of their training pipeline.
Jesus Christ, so many people here who have no clue what they are talking about.
A proof of a theorem is different from the statement of the theorem. OpenAI has a Lean proof of the statement. That is all they need. There may be many different proofs of this statement, including NL proofs. It does not matter that these NL proofs may or may not be different from the Lean proof, at least for the correctness of the Lean proof. But of course the NL proof may be wrong. But who cares?
It reminds me of how provably secure software was all the rage for awhile. Until people found that the idealized system/lemmas were so far from reality that the proved security was worse than meaningless because it gave a false sense of security.
In order to prove security, you must first simulate the universe.
Why would they do that, knowing that the one known to be correct is the Lean one? Just to claim that the (correct) Lean proof did not translate well to English? That would be weak, and a colossal waste of energy and time.
Why do people program in Python instead of writing machine code?
Why are review papers published? Executive summaries? "Introduction to X" books?
People's time and computational resources are finite. Summarising information -- ideally in structured ways that preserve important properties, but even in informal, unstructured ways -- is critical for making any kind of progress in this world.
The statement written in Lean, is not actually a correct description of the Navier-Stokes problem. It's some other easier statement. That's why the Lean code may not be a proof.
Here's a quote from the paper.
"3.1. When the NL paper declares stronger statements than what Lean proves
We commence with an example that complements those in §2. In the following example the AI autoformalisation results in a different Lean proof of a weaker statement."
"Tracing the proof of (3.3) we find that the series arises from applying the Lean theorem coefficient_seminorm_bound, just as Figure 3 mentions. Consequently, the Lean results discussed in this section are weaker than (3.1) in the NL proof."
So the question is, was the Navier-Stokes problem framed properly in the lean code, or is some easier problem represented in the Lean code?
Here is their remark.
"Remark 3.4 (Further potential mistranslations of the Navier-Stokes proof). The above examples require careful manual checking of both the NL proof as well as the Lean proof, which is delicate and highly time consuming. Moreover, the fact that we display only two examples does not mean that these are the only cases of mistranslations."
The "statements" here are statements of lemmas, not the statement of the main theorem. The main theorem statement was written by humans (prior to the OpenAI work), so there shouldn't be any concern that an AI mistranslated it.
That could be that these examples are lemmas, but in the introduction the authors conclude,
"In all cases the correctness of the formal proof does not say anything about the correctness of the NL proof."
The Clay Mathematics Institute has not yet accepted the proof as it takes a while to review.
"The Clay Mathematics Institute is dedicated to furthering the beauty, power and universality of mathematical thought. Curating the Millennium Prize Problems is one of the most important ways in which it pursues this goal. The rules governing the prizes describe the process for evaluating what has been achieved and for assigning credit. The process is deliberately unhurried, but we will provide updates."
We can see therefore that we don't know if the OpenAI program wrote the NL statement or proof correctly. Or if maybe it needs work. Or if maybe it has significant errors.
We also don't know if Astra properly wrote the initial statement of Navier-Stokes correctly in Lean.
My point is that Astra did not write the statement of Navier-Stokes in Lean, there was already a human-written statement, so there should be no concern that it has errors.
This article shows that we can't conclude from the working Lean proof that the NL proof is also correct, because they are different in some places. On the other hand I think it's unlikely that it's "very" broken. OpenAI first spent a lot of computer time to find the NL proof, and then less time to translate it into Lean. So if there were errors in the NL proof it seems they were not fatal, the agent translating it into Lean was able to patch them up as it went along.
Everyone. I don't think many people working in fluid dynamics were surprised you can find a blow-up in Navier-Stokes. What would advance human knowledge is understanding the situations in which a blow-up might occur. In that context, the lean proof is necessary, but the non-lean proof is more important.
> Jesus Christ, so many people here who have no clue what they are talking about.
Indeed. If only some of those people would see the irony.
What matters most of all, as any first year student of mathematics would know, is whether the formal problem statement corresponds to the NL statement. TFA specifically states that at least some of the allegedly proven formal statements DO NOT.
No. What the paper says is that in principle, translating NL statements to Lean statements is hard. Nobody doubts that, translating informal to formal text cannot be formally proven correct, so...
Does the paper give a single example of one of the OpenAI solved theorems with a Lean certificate where the Lean statement does not correspond to the actual statement from the mathematical literature? I don't think so, but in case I am wrong, feel free to provide that example.
> To demonstrate the effect of this result we provide
several examples of AI mistranslations of NL statements and proofs into Lean in practice, resulting in mismatches between NL proofs and their Lean ‘verifications’. These include OpenAI’s announced Navier-Stokes proof.
Could /I/ be mistranslating the paper’s formal statement to NL? I don’t think so, but in case I am wrong, feel free to cite the correct formal statement that they claim as divergent between Lean and NL formulations by OAI.
Yes, you are misunderstanding what the paper claims. Navier Stokes for example is not such an example, only the proof is different. For their other examples, none of them they claim that they concern actually the open ai solved theorems. You are welcome.
And yet I cited a specific statement made concerning N-S specifically, whereas all you’ve done is make patronizing remarks, and strawman arguments.
Have you actually read it? They give a handful of examples of mistranslations of both claims and proofs thereof, in relation to NS and Euler, though I do concede that they do not go as far as claiming outright the NS statement itself is mistranslated.
Unfortunately, lean proof alone is not enough. sidestepping the raging discussion regarding the meaning of mathematics, just because the lean compiles is not proof in itself that it is correct in the sense that mathematicians mean. Unless of course, you can prove that lean itself is correct, which you can’t.
we have seen “proofs” earlier this year that essentially abused some bugs in the kernel. it would be very convenient if every program written in rust was automatically correct if compiles - something i strongly suspect you believe - unfortunately, this is not the case for either.
In case you are not aware, the actual theorem statement of N-S was never translated by an LLM, but was written independently by formal conjectures, as they say in the README [1].
A Lean proof has a much higher probability of being correct (in my opinion) than any published (either preprint or peer-reviewed) paper, yet no one before LLMs were walking around claiming every result published can't be trusted yet (without an actual reason).
We have seen one such instance of Lean bugs, which was found adversely against Lean (as in find bug then use this bug to prove Collatz, not just found when being asked to prove it).
It's also worth to note that the way N-S (and all the other proofs by OpenAI etc) have been found is first prove it in NL then translate to Lean. I.e. it would have to first believe it found a correct proof in NL, and then afterwards either accidentally or on purpose use a Lean kernel bug.
edit:
Probably also worth to mention that the proof have been checked both by the Lean kernel and the independent nanoda kernel, so it would need to exploit bug(s) from both.
2. No exploit of a kernel soundness bug in Lean (and the independent kernel nanodo, which OpenAI also checks against).
In particular, if you trust this, then you don't need to care about anything else the Lean program does, no matter how many lines, lemmas etc. it makes along the way.
As I said above, the theorem statement have been written independently by formal conjectures, and you are free to read it yourself (or trust other people have done it).
So, assuming you don't disagree the theorem statement have been written correctly, you pretty much need to believe 2 is false, if you don't trust the proof [1]. And that's what I find has a rather low probability personally.
[1] As the book says, you also need to trust the hardware, firmware etc.
You may be overconfident here. The NL paper may be properly stating the Navier Stokes problem, and the Lean code may not be.
Here's from the paper--
3.1. When the NL paper declares stronger statements than what Lean proves
We commence with an example that complements those in §2. In the following example the AI autoformalisation results in a different Lean proof of a weaker statement.
None of the examples are of the NS lean statement. The validity of the latter is not under question ( in this paper ). Please note the distinction between “statement of NS” and “statement of lemmas used in the NL proof of NS”.
3) The comments promise functionality the code does not implement.
The first post was "It doesn't meet the business requirements the comments indicated, so the software is broken." The replying post was "Actually, the business requirements were faithfully turned into tests, so the software is fine. The comments need to be rewritten"
They published both a natural language proof and a lean proof. This paper says the NL proof does not match does not match the lean proof, using different arguments and some stronger claims. It does not claim the lean formulation of NS or the rest of the lean proof is invalid.
"The Lean code merely tells us that the theorem is correct, yet the NL proof with intermediate arguments, lemmas and results may be incorrect or have incorrect arguments."
You didn't answer the question. Quoting from upthread:
A: "In particular, we show that the formalised Lean proof does not correspond to the NL proof of blow-up of solutions to the Navier-Stokes equations."
B: "No, they're not claiming that [...] The authors are claiming that the Lean proof is not the same proof as the NL one"
The question was: what's the difference between A and B?
Your answer to that question was "This paper says the NL proof does not match does not match the lean proof".
It seems to me like A is the exact same thing as B, and you are here claiming that they're different things while merely repeating the same claim that was already claimed in both A and B.
Statement A is upthread quoting the paper, and is not the statement B is arguing against. B is arguing against the incorrect statement "So these authors seem to be claiming that OpenAI has not really proven Navier-Stokes at all."
This doesn't answer the question. The question was: what was the difference in A and B. Your answer looks like this:
A: The English language description doesn't do the same thing as the code the AI wrote.
B: The English language description doesn't do the same thing as the code the AI wrote.
A and B are the same; there is no difference. Is this what you meant? Or did you mean the opposite of this? If you meant the opposite of this, what exactly is the difference?
Which is exactly why you want a Lean proof in the first place!
NL proofs rely on not-entirely-correct prior art, and contain countless holes both big and small. When formalizing human NL proofs it is probably very normal that the Lean proofs doesn't quite follow the NL proof.
Lean is a theorem prover. If it "compiles", and the proposition as coded in Lean is equivalent to what the theorem proposes in natural language, the theorem is correct. https://en.wikipedia.org/wiki/Lean_(proof_assistant)
I don't understand then, because it doesn't sound like you understand how Lean works.
The actual proposition of the theorem is very short in Lean, and at this point has been looked at by probably hundreds of mathematicians (including the authors of the article). All that needs to be confirmed manually is that the proposition statement matches what people say the theorem actually proves. That has been done.
At that point, the rest of the actual proof steps are automatically verified by Lean - that's literally the entire point of Lean. There is no additional verification step - "What has shown the Lean proof to be correct" is the fact that the Lean program automatically validated it and generated a certificate.
But what mathematicians really care about is the actual steps. What the authors of the article have shown is just because you have a Lean program it doesn't mean the steps of the natural language theorem have been "autoformalized", because there is no guarantee the steps from the natural language description match the Lean code.
A Lean program that compiles proves what it states. The question is whether the program states what we think it does. If I recall, this program is 12 million lines long. Mathlib is not perfect, and it's not hard at all to have a Lean statement that isn't what you think it is.
So my question is whether people have actually checked that the generated program, all 12 million lines of it, has the statements that are expected.
But that's where you're wrong. All 12 million lines do not need to be checked (again, that's literally the entire purpose of Lean). All that needs to be checked is that the proposition of the theorem is what we think it is, and that has been done. What Lean guarantees is that if you start with a set of axioms -> whole bunch of steps that nobody really ever needs to read -> final theorem proposition, that those steps do successfully prove the theorem.
You say "it's not hard to have a Lean statement that isn't what you think it is", and that's true - indeed, what the entire paper is about is that the intermediate steps in Lean don't match the natural language description. But the statement of the proof in Lean (which again is quite short) is what we think it is, and hence the Lean proof is valid.
That's extremely hard to believe. First, one question arises: what are the 12 million lines doing if they're not needed? Secondly, don't those 12 million lines build up the theory that is eventually proved? Mistakes could be there that makes the outer statement seem like something it isn't, could there not be?
There's maybe an even simpler way of illustrating this.
Fermat's Last Theorem is very easy to state. There are no positive integers a, b, and c such that a^n + b^n = c^n. This likewise easy to write in Lean. The statement of the theorem is exactly one line of code.
theorem fermat_last_theorem (n : ℕ) (n_ge_3 : 3 ≤ n) (a b c : ℕ) (a_is_positive : 0 < a) (b_is_positive : 0 < b) (c_is_positive : 0 < c) : a ^ n + b ^ n ≠ c ^ n
Kevin Buzzard's team of human mathematicians attempted to prove this in Lean. They got to about 60000 lines of Lean code and were not anywhere close to finishing the job.
Anthropic came around with a Claude model that proved it in 13 million lines of code. (Buzzard's team might have ultimately used within the same order of magnitude if they finished. It's very unclear. They were at minimum years away.)
Whether 60000 or 13 million, only one line of code was necessary to manually verify whether they had successfully proved Fermat's Last Theorem, just the statement of `fermat_last_theorem`. None of the rest of the code built up the theory at all.
Indeed if those lines of code built up the theory, that would suggest that the statement of Fermat's Last Theorem is incomprehensible, if professional mathematicians cannot define Fermat's Last Theorem in even 60000 lines of code! But that's pretty absurd. You can explain the statement of Fermat's Last Theorem, completely rigorously, to a middle schooler.
> First, one question arises: what are the 12 million lines doing if they're not needed? Secondly, don't those 12 million lines build up the theory that is eventually proved? Mistakes could be there that makes the outer statement seem like something it isn't, could there not be?
This seems to reflect an incorrect understanding of formal mathematics.
Formal mathematics can be thought of as a game with rules where a statement to be proved is some game end state. If you play 12 million valid moves and arrive at the desired end state then you have proved that the statement is valid. If your logical system is sound (e.g. first order logic), then you have proved your statement is true.
The entire point of the formalization of logic and mathematics through the 19th and 20th centuries was to unlock this capability. Such that, if you only care about whether your statement is true or not and not why it's true, the intermediate steps are entirely unnecessary.
To answer each of your questions:
> First, one question arises: what are the 12 million lines doing if they're not needed?
They are the intermediate computations necessary to show that the final statement is valid. This is much like intermediate arithmetic operations in a large complicated arithmetic expression. As long as you believe that the computer can calculate each individual arithmetic expression correctly, you don't need to manually review the billions of intermediate calculations that result in the final number.
> Secondly, don't those 12 million lines build up the theory that is eventually proved?
No. Not in the way you're thinking of. It seems you are using "theory" to mean the concepts necessary to express the statement itself. This is a very different set of lines than the proof itself (and generally much much more compact). Very difficult to prove statements can be stated in very simple language. Often the theory is <0.01% of the total lines of code.
> Mistakes could be there that makes the outer statement seem like something it isn't, could there not be?
No. Only the language used in the final statement can cause this issue, i.e. the theory. The intermediate 12 million lines cannot affect this, assuming your computer can carry out arithmetic correctly.
Another way of thinking about it is that the big breakthrough of 19th and 20th century mathematical logic is to conclusively show that formal mathematics is fundamentally just a set of arithmetic operations.
Hence in a very real sense, proving a mathematical statement is exactly identical to finding x in an arithmetic expression f with some constant k such that f(x) = k, where k is determined by the statement you wish to prove (not the 12 million lines).
Showing that x causes f(x) to equal k is exactly equivalent to proving the mathematical statement associated with k.
The 12 million lines are an alternative notation, no different in spirit to decimal notation or binary notation, of writing down x. It is some gigantic natural number.
You must understand k. If you use the wrong k, then you have proved another statement than what you intended. This is usually about a dozen lines of code.
You must be confident in the correctness of f. This corresponds to the believing that your type checker (since Lean uses dependent typing as its method of proof checking) has no fundamental soundness bugs. This can be controversial and indeed Lean's type checker has had soundness bugs in the past! But crucially f does not depend on k or x. Every time you fix a bug in f you can rerun it for any k or x you care about.
But you don't have to actually read all the digits of x, in whatever notation you care about. You can copy paste it in from some gigantic file handed to you by a stranger. As long as f(x) = k, you know that you have proved the mathematical statement associated with k.
Overall, this is roughly like asking why you trust a computer to solve a long division problem that has 12 million intermediate steps if you don't actually read all 12 million intermediate steps yourself. Well I trust the computer to do one step of division correctly and to be able to repeat that faithfully 12 million times. Hence I trust the computer can solve a long division problem involving millions of digits, even if I don't actually look at all 12 million intermediate computations myself.
My understanding is that the statements in the Lean proof might not actually correspond to the Navier-Stokes problem. And that requires mathematical insight to be able to verify, which could take quite a while.
There's no "prompt injection" like the OP claims. You can just ask Muse for all the info for its dedicated VM, and Muse will tell you. Meta has confirmed this it intended behavior.
And idk what's even being implied by that quote. (Yes, it tries to help you shop. And? Or is the critique just that OP doesn't like the prose in that markdown file?)
What's being implied is that your nefarious company is building another product that exists solely to suck up people's attention and time in order to build a profile of them to sell them garbage that they don't need and use their personal data to manipulate them.
I don't know how anyone can use Muse. It's like taking everything that Facebook has had to guess and infer about you over the years using various analytics and instead just dumping it directly into a chat bot and giving the company everything it wants. Zuckerberg and his soulless C-suite must be licking their lips.
> What's being implied is that your nefarious company is building another product that exists solely to suck up people's attention and time inorder to build a profile to sell them garbage that they don't need and use their personal data to manipulate them.
You're free to speculate that Meta is lying about what it will or won't use the data for. I won't argue over that. That's an important topic and I'm not gonna persuade anyone via an HN comment. Skepticism is fine
But the example linked to in the above comment is not evidence of that at all. A skill explaining how to format purchase history in markdown format is not evidence of hidden motivations.
In general the entire thread is nothing-burger, and that's all I'm getting at.
I understand what is being implied. I'm saying that the provided example is poor evidence of what's being implied.
We all know Meta can already track things like certain purchases. A skill file telling AI how to format purchase history in markdown format is not going to help any of Meta's ads or RecSys systems.
You can believe Meta does bad things without thinking everything Meta does is an example of said bad things
So we should judge each piece of evidence in a vacuum?
Facebook has broken its promises about user data many times before. I don't buy anything they say about Muse, and nobody in their right mind should. Unless they’re getting paid to, of course.
Aside, if “a skill file organizing purchase history in markdown format” is the only thing you took away from that thread, you might want to take a closer look.
> So we should judge each piece of evidence in a vacuum?
You're free to believe whatever you want about Meta. But no, you shouldn't point to non-evidence, interpret it through the lens of your prior beliefs, and then claim that it's actually evidence supporting your beliefs.
> Aside, if “a skill file organizing purchase history in markdown format” is the only thing you took away from that thread, you might want to take a closer look.
That's what that specific quote I was referencing was referring to. I already responded to the supposed prompt injection/security concerns above.
<< you shouldn't point to non-evidence, interpret it through the lens of your prior beliefs, and then claim that it's actually evidence supporting your beliefs
Not parent, but it is worth responding to as it applies. Not prior beliefs. From prior actions. Unless you are only now becoming aware of the existing ecosystem, there is almost no way in which you can buy into that ecosystem without expecting hard rug pull. And the beauty is, everyone gets to make a choice, but they best do it is with eyes wide open and the understanding of the deal for it will be changing soon.
I think you're saying that prior actions justify the beliefs you hold. Fair enough, I'll concede that for the sake of argument.
My above comment still stands. Even if your beliefs are justified, that doesn't mean you should treat non-evidence as evidence supporting your beliefs. (Tbh my point is kinda trivial/pedantic. But I feel like folks just always want to double down that every criticism of meta is correct).
You do have a point, and I sometimes clamp down on random meta complaints myself, but mostly because there are enough of real complaints to worry about. In other words, your comment in useful in providing appropriate balance. I don't like meta. That much is evident, but there is no reason to make argument against them less potent by not considering how it lands.
> So we should judge each piece of evidence in a vacuum?
I mean… little drops of water make the ocean, so if the complaint is with the drop then yes. Doesn’t mean the ocean is poisoned by some of these droplets though.
Two things can be true at the same time - Facebook as a whole is untrustable but you can use Muse Glimmer from them without giving them this data.
The comment you're responding to has two sentences separated by a period. You can use that as an indication that the two issues being pointed out are, in fact, separate things. Do you understand now?
Totally fair if that's just a public stratch pad (which happens to make several specific, unsubstantiated claims). Not hating on that guy.
But if someone posts the thread with no other context than "check this out", then it seems appropriate explain this missing context.
And for the record, I did scroll back read through what he's saying as you suggested. The broader context is exactly what makes it clear he's just a bit confused about what he's "uncovered". (Obvious caveat: I haven't read through everything this guy has written, I could be missing something)
> I did scroll back read through what he's saying as you suggested. ... Obvious caveat: I haven't read through everything this guy has written...
Did you also scroll forward to read what he wrote after the stuff in the thread that was linked to? This tool is pretty obviously designed -whether intentionally or not- to enable the exfiltration of a ton of sensitive data to untrusted third parties while leaving its intended user none the wiser.
a) More being surprised the muse will just provide internal details (which again, as intended)
b) the bash scripts are ugly (okay? I don't think that's what we're arguing about)
c) surprise that muse runs as root (then realizing this is explained by the VM-in-VM model and linking to the blog which explains this)
d) surprise that muse can run as a seed box (idk is this supposed to be the security concern?)
Could you please be more specific about which part you're concerned about? I think that'd be more productive than linking to a long thread, me responding to what I see, then being told that it's just a scratch pad and I'm silly for taking it seriously but also if I look closer I'll see the deeper truth.
> ...then being told that it's just a scratch pad and I'm silly for taking it seriously...
Yeah, given how this has been going, I expected that sort of misrepresentation. A careful reader remembers that this is what I said:
You should pop up to that -er- blog's top level, scroll back until you find the start of the investigation, and then start reading. Yeah, it's a huge pain in the ass, but this is -AFAICT- the guy's personal notepad, rather than some place where formal reports usually go. If someone walked over, picked up my personal notepad at $DAYJOB, and complained about how they couldn't make sense of it, I'd tell them to fuck right off, yanno?
Anyway. I'm certain that your answer to this is going to be some flavor of "no, of course it isn't", but are the combination of these things simply no problem?
I remembered what you said and I genuinely don't understand what I mischaracterized.
TBH I saw none of those comments by scrolling forward or backwards from the original link. Now it turns out I supposed to be checking quote tweets, not just scrolling like you said above? (tbf I'm not familiar with the neuromatch UI, could be user error on my part).
Again, I think this would've gone smoother if you actually articulated the problem you're concerned about, instead of linking to a "personal notepad" which you acknowledge has mistakes and isn't easily intelligible, and then blaming me for missing the actual point.
I'll look through these new comments later.
Edit: I see from your original comment what you meant by "top-level". I originally assumed you just meant top of the thread. I'll take the L here.
I've read through those comments and I see more interesting concerns being raised ("interesting" as in "not completely confused for trivial reasons", I'm not conceding anything).
Like I mentioned I don't work on Muse, and I suppose it'd wouldn't be appropriate for me (self-identified Meta employee) to publicly speculate on what defenses do and don't exist or what is intended. (sorry kinda a cop-out, I know).
Thanks for providing specific links to the specific concerns being raised.
> I originally assumed you just meant top of the thread. I'll take the L here.
I'd very much rather you commit to
* Slowing the hell down when you're reading things that other people write. If you don't have time to slow down and actually read, stay the hell out of the conversation.
* Upon receiving a notification that you've massively misunderstood something that someone has said in writing, immediately stop, carefully inspect what you know and what you assume, and then go back and carefully re-read the thing you've been told you massively misunderstood while keeping at the front of your mind the knowledge that your task is to find out what parts you totally misunderstood during your prior reads
This second point is a huge part of that whole "principle of charity" thing that's the fundamental part of good "netiquette". This principle should be abandoned when you have personal knowledge that your conversation partner is a fucking bozo or is otherwise intentionally fucking with you... but -as I'm sure you know- one cannot possibly possess that knowledge at the start of one's first conversation with a stranger.
> I suppose it'd wouldn't be appropriate for me ... to publicly speculate on what defenses do and don't exist or what is intended.
With all due respect, that's not what I asked for. What I asked was
...are the combination of these things simply no problem?
If it helps to focus your thinking, remember that "prompt injection" is a fancy phrase for "the tool is now blindly executing code handed to it by some random stranger on The Internet in the same way it would execute operator-issued or tool-manufacturer-embedded instructions".
No? I'm rather excited that perhaps you'll take this as a "teachable moment" to adjust your behavior in a much more prosocial direction.
So many people wade into a conversation without actually having the time and energy required to actually participate in that conversation. Even if we ignore the ragebait bots and people paid to act like them, I'd expect that this describes the behavior of the vast majority of folks -when measuring by comment volume- who choose to comment on the internet.
Public discourse would be far better if more people were aware of the liberation delivered to each of us long ago by Mr. Marcus Aurelius. [0]
And -well- if you jumped in here because of a vague sense of duty towards your employer... remember that not only are they a -more or less- multi-trillion-dollar company, but they have survived -among so many other things- renaming the company to commemorate its new "this will wreck the company if we don't win it!" project and then burning tens of billions of dollars on said project over the course of just a few years with precious little to show for it. They'll survive some negative sentiment in some random HN thread just fine... they don't need your backup.
Don't forget to be less of a shithead in the future, and stay out of conversations where you can't actually contribute anything useful -like this one- yeah? We're each personally responsible to promote healthy discourse... noone else can!
1. I responded to the quote and thread tokepi posted. I stand by that response—the supposed "prompt injection" was a trivial misunderstanding which Meta had already publicly clarified.
2. You sarcastically replied by linking me to another comment that said "pop up to that -er- blog's top level, scroll back until you find the start of the investigation, and then start reading." I read that as the top of the linked thread. You meant his profile page, which would have meant reading everything he'd posted since the investigation began to find the posts you had in mind, which came days later in separate threads. I noted I didn't read everything he wrote, clarified what I did read, and asked you to state your specific concern.
3. You didn't state it. You linked five more posts on issues unrelated to the thread I'd responded to. I acknowledged I hadn't seen those, said they were at least not trivially confused the way the first thread was, and bowed out because I shouldn’t discuss specifics of my employer’s security setup beyond what's been publicly disclosed (for obvious reasons).
4. You responded with more sarcasm and a lecture on "netiquette". Then you finished by explaining "prompt injection", which is strange: even @jonny acknowledged the supposed prompt injection from the initial thread was intended behavior, and the posts you linked provide no evidence of prompt injection. So you bringing it up calls into question whether you were actually parsing our conversation and jonny's investigation.
5. I tried twice to end this amicably. Both times you came back with insults and more lectures.
Do you think you upheld your own standards of "netiquette" and "healthy discourse" here? How is being simultaneously antagonistic while refusing to articulate your own perspective helpful to anyone?
Doesn't look any different than doing the same in C# or Java to me, it is kind of pointless in Python, the one thing the pattern gives you is building a class in such a way that you the developer know exactly what's what, so its really a developer ergonomics thing is how it looks like to me.
Referring to "25 centimeters per year" as "so fast it can be seen from space" is so funny to me.
We have very good satellites! Lots of things can be seen from space. "can be seen from space" is an awful way to provide readers intuition for the scale of something.
In which case doing this in the dark is clearly bad for the community -- if that location is what's scarce then they should be demanding a better deal.
The only way this makes sense for communities is as a kind of "finder's fee", i.e. you might argue that if BigTechOne™ knew that they'd have to bid against BigTechTwo™ they'd never even bother to scope out the location.
Still, if the prospecting is the bottleneck there could be 3rd parties (or even the tech companies themselves) entering into agreements with towns which allow both a finders fee and open bidding for the lot.
The described pattern is standard in Meta. This, along with the infrastructure and tooling to support it, was the single largest "devx quality of life improvement" in my experience moving to big tech.
Microservices should have clear owners reflected in the org chart, but the topology of dependencies should definitely not be isomorphic to your org chart.
