Why was the comment of the author of the translation-site that explains the methodology flagged / dead?
The mere fact that the translation was AI-assisted shouldn't be a reason to flag a comment. AI translations, especially supervised translations, can actually achieve a reasonable level of quality.
Disagreement on AI philosophy shouldn't be a reason to flag a contribution.
> AI translations, especially supervised translations, can actually achieve a reasonable level of quality.
That may be true, but also not really what i'm worried about. With AI on an ancient book written from a world view very different from the contemporary one, i'd worry more about misleading translations.
Yes and know. The biggest problem I've experienced vis a vis hallucinations is tabular data. In prose, where context is anchored by surrounding words. Even the subtle points are conveyed appropriately.
I'm not a book editor or translator, just a guy with a reasonable familiarity with rabbinic literature in its original language. I spent some time reading the translation of Tzemach David, and I can say the following: it's legible to me but the AI is clearly lacking a style guide. It's inconsistent about what it chooses to translate and what it chooses to transliterate, and when it transliterates, it's inconsistent in how it does so. Eg "Rosh mesivta" vs "metivta" vs "exilarch". The AI is drawing on a hodge-podge of training materials with diverse audiences and no clear audience of its own.
Can you email me with more info? It is likely the distinctions you noted are from the source. There is a glossary maintained that keeps specific words or phrases consistent. But slightly different phrases can resolve in inconsistent ways.
The problem is not the AI, but rather a weakness in the design. In other words, a bug. Or a regression and we call them today. It is fixable.
I think that I might not have been clear enough. Of course Rosh mesivta and exilarch differ in the source, but I don't think a human translator would choose to translate one title and not the other. In circles where people use the word exilarch I am almost 100% positive no one would say Rosh mesivta, and the kind of folks who use Rosh mesivta in conversation would say Reish Galusa, not exilarch.
I can't imagine that metivta and mesivta are not both transliterations of the word מתיבתא.
You need to identify the target audience and stick to their language. Are you translating for a general lay audience? An academic audience? Contemporary religious Jews? If the latter, is it a yeshivish audience or a wider group? How much familiarity with Hebrew is expected from the reader?
> Would critics rather have NO translations of these pivotal works?
I sent a Classicist the link because it's adjacent to his area of interest, and by chance he addressed this question. I don't think he'll mind me quoting him:
"There's a lot of this shite popping up at the moment. [...] There seems to be some idea that publishing a shit translation of an untranslated work is better than no translation, but that's obviously wrong."
Speaking for myself: Some of the anomaly rules are obviously patches to deal with a specific situation, and might be better off in code. But relying on the chatbot to flag wider anomalies is odd. How does it know what it doesn't know?
At a minimum, I'd use a spread of models to gain binocular vision, and I wouldn't publish until a human was prepared to sign their reputation to it.
In other words, I think what you've got there is a first draft of a translation, not a translation. Given that, if you are going to publish, I think the NoDerivatives restriction is a mistake. But that's a minor issue.
So, he's commenting on something he didn't read and compares it to shit? Good to know. I'll ignore him. I understand his point. Still, obviously my rhetorical question had in mind that what I do is good enough quality to make the question worthwhile. Better than a "first draft". As mentioned in another comment, some works were translated up to four times as I refined the process.
The thing with LLM-generated code is you can test it on a real CPU - that's a tight feedback loop with a cast-iron verification mechanism.
Translations of human language have no analogous mechanism. So yes, the translation might be perfect, but until a human puts their reputation on the line and says "I certify this translation is accurate", it's still shit. It's shit because of the way it was created, not its absolute accuracy (or otherwise). He doesn't need to read it.
I'm sorry, I'm not trying to upset you, and I know I won't change your mind. We just have different philosophical positions on this, I think.
I know what I know from experience. You know what you know from nothing at all.
Numerous people have commented to me about Hebrew and Latin works, who know what they are talking about, and none has criticized the fidelity of the translation to the source. The criticisms proffered are minor, while the praise extensive.
A couple years ago, I shared your opinion. Now I don't. You are the one who won't change your mind. Like you said, it's "philosophical" for you, while for me, it isn't. If you insist on calling it shit regardless, you are retarded.
I've done it myself. Transcription of 19th century newspaper articles to markdown, mostly. Some earlier wills (which were an absolute pig - secretary hand). Oh, and categorisation of postcards. That's why I was poking around your pipeline - to see if I could learn anything. You're right, tabular data is hard. Also columns, and proper nouns.
Feeding the LLM a context-aware cheat sheet helped with the nouns. BTW, what I said about using multiple models for parallax was good advice.
> Numerous people have commented to me about Hebrew and Latin works, who know what they are talking about, and none has criticized the fidelity of the translation to the source.
Have these people gone on the record saying that? What are their credentials?
I got to be honest, claims that some unnamed person privately told you they thought your product was good is pretty meaningless. I feel like that makes me trust you less not more.
But you can't read the original! That's why you built the pipeline in the first place! Only people who can read both versions can have a grounded opinion on whether or not the translation is accurate.
C'mon man, I know criticisms of your baby feel like attacks on you, but take a step back. You must see that "the name of the author of the pipeline is public knowledge" is not a sensible response to "nobody who can read both versions has attested to its accuracy in public".
Right now, everybody believes that LLMs can't self-correct (see https://arxiv.org/abs/2310.01798) and that they hallucinate when asked to do OCR tasks. It doesn't matter if that's true or not, that's the prevailing belief you're working against. If you want to convince people your pipeline can self-correct, you're going to need to supply evidence. I see two possibilities:
(1) Pay someone to audit one of the translations publicly.
Crucially they need to answer the question: does the pipeline introduce inaccuracy? If it does then it really is worse than useless, because it tells lies. Clunky, inconsistent, partial translations would still be better than no translation - it's the risk of hallucination that's the killer.
(2) Are you doing test runs on similar documents that have also been translated by humans? (Preferably very recent translations). Publishing those test runs would allow poor uneducated slobs like me to line up a human translation and your machine translation and see for ourselves that your pipeline works.
This is meant as helpful advice - I'm trying suggest paths that respond to valid critique with something other than bluster. I wish you and your project well.
By way of example, there are several comments by someone else noting a problem with terminology drift. This was genuinely constructive, and I fixed and repaired the problem, which I thought I had resolved months ago.
These are the kind of comments I also typically get regarding regressions. Problems with mostly mechanically repairable aspects of the translation, but not with the quality of the translation itself. From people who know of what they speak. That I do not put them on the record is not relevant, except to people like you. They are real, and their attestations are real.
Professional translators don't throw around the word "impressive" like that.
You don't, yet I still stand by my work.
I translated some books up to four times as I refined the process. The default behavior is repeatedly to not attempt to translate text it can't resolve for whatever reason. AIs do have a tendency to hallucinate, and I expended a lot of effort on minimizing this problem to the point it can be considered mitigated.
Professional publishers collect blurbs which are attached to real names of real people, and bona fide reviewers write in full sentences with context, without
"The solar system's distant reaches exhibit a wealth of anomalous dynamical structure, hinting at the presence of a yet-undetected, massive trans-Neptunian body - Planet 9. Previous analyses have shown how orbital evolution induced by this object can explain the origins of a broad assortment of exotic orbits, ranging from those characterized by high perihelia to those with extreme inclinations. In this work, we shift the focus toward a more conventional class of TNOs, and consider the observed census of long-period, nearly planar, Neptune-crossing objects as a hitherto-unexplored probe of the Planet 9 hypothesis. To this end, we carry out comprehensive N−body simulations that self-consistently model gravitational perturbations from all giant planets, the Galactic tide, as well as passing stars, stemming from initial conditions that account for the primordial giant planet migration and sun's early evolution within a star cluster. Accounting for observational biases, our results reveal that the orbital architecture of this group of objects aligns closely with the predictions of the P9-inclusive model. In stark contrast, the P9-free scenario is statistically rejected at a ∼5σ confidence-level. Accordingly, this work introduces a new line of evidence supporting the existence of Planet 9 and further delineates a series of observational predictions poised for near-term resolution."
This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed.
There are few limitations left when compared with other backends. Instead of using 'cuda' device, one simply uses 'MPS' as device.
What remains is: the optimizations Pytorch provides (especially compile() with 2.1) focus on cuda and it's historic restrictions that result from CUDA being _not_ unified memory, and lots of energy goes into developing architectural work-arounds in order to limit the copying between graphics HW and CPU memory, resulting in proprietary compilers (like triton) that move parts of the python code into proprietary hardware.
Apple's unified memory would make all of those super complicated architectural workarounds mostly unnecessary (which they demonstrate with their project).
Getting current D/L platforms to support both paradigms (unified/non unified) will be a lot of work. One possible avenue is the MLIR project currently leveraged by Mojo.
> This is (partly) outdated. MPS (metal performance shaders) are now (since torch 2.x) fully integrated in standard Pytorch releases, no external backends or special torch versions are needed.
Not sure what you're referring to, the link I provided shows how to use the "mps" backend / device from the official PyTorch release.
> lots of energy goes into developing architectural work-arounds in order to limit the copying between graphics HW and CPU memory
Does this remark apply to PyTorch running on NVidia's platforms with unified memory like the Jetsons?
The project probably at least partially serves as documentation for other platforms to integrate Silicon acceleration. It basically demonstrates how to use macOS Accelerate and Metal MPS (metal performance shaders) using C++ for Machine Learning and training optimization.
Thus other platforms can simply take this backend-code and integrate it. (Pytorch basically did that already with Apple's help).
Nailed it.
I think more than partially. What happens in this repo will spread to the other major frameworks and over time, clever ideas that spawn on other projects will be reimplemented with Apple's adjustments back into the repo. It's a brilliant and efficient way to interact with the community, that can likely be measured in more sales of their hardware over time.
Neural engine is not helpful for training, its inference hardware, whereas this targets training and research. They use Accelerate and Metal (with seemingly similar/identical performance shaders that their Pytorch adaption uses) which allows for high performance training.
This project additionally serves as documentation for other platforms to integrate Silicon, which is good.
Still, being to run LLaMa2 on the NPU would be awesome due to the unified memory. Apple's restricting its use to only Apple-approved models is frankly irksome.
The main thing about this framework is, that it uses unified memory with GPU. This gives maximum performance. Neural engine one the other hand is optimized for low-energy inference (which is mostly an advantage on mobile devices), and imposes limitations and restrictions since it's hardware supports only very specific neural network operations. Thus supporting neural engine within a universal machine learning platform doesn't make much sense, it would just be a bottleneck.
The way to use neural engine is to convert existing models that strictly adhere to the limitations of the neural engine hardware (excluding many operations used in non-restricted NN models) for use in energy-restricted inference applications only. It's a different application scenario.
The mere fact that the translation was AI-assisted shouldn't be a reason to flag a comment. AI translations, especially supervised translations, can actually achieve a reasonable level of quality.
Disagreement on AI philosophy shouldn't be a reason to flag a contribution.