Pierer invests many millions in Sepp Hochreiter's LLM alternative architecture. Shortly before Christmas, the company NXAI was founded in Linz with Albert Ortig as Managing Director. Its task: to market Hochreiter's XLSTM.
---
Can xLSTM really compete with Transformers, Mamba etc.?
Is there a professional analysis about XLSTM? It seems they are secret/stealth [1]? With a quick analysis I cannot differentiate if it is a Theranos or another OpenAI.
I don't have that hope with past "due diligence" track records. Performing a quick peek for the author at Google Patents I only see mentions of his involvement with neural networks in past work [1]. I am not an expert in patents but I assume that they will apply there and/or publish papers before the company is launched. Except if the breakthrough is not there or it is really revolutionary but statistics are not in their favor.
I listened to the latest about it from Prof Sepp himself on the Industrial AI podcast channel. See the link below.
He consistently called Transformers... "nonsense" and said that xLSTM has beaten Transformer-based models across several small dataset benchmarks. They are busy scaling the training to large datasets and planning to keep the technology in Europe. I doubt they will make it public once it is available. The coming months will be interesting. More about the interview here.
People say lots of stupid shit. If there is no code or even paper, there is no reason to believe. Beating benchmarks requires something more than a blind faith.
[Press Release]: Faraday Technology Corporation (TWSE: 3035), a leader in ASIC design services and IP solutions, announces its collaboration with Arm and Intel in spearheading the development of a 64-core System-on-Chip (SoC) utilizing Intel 18A technology. This innovative SoC seamlessly integrates Arm® Neoverse™ Compute Subsystems (CSS), delivering unparalleled performance and power efficiency tailored for the construction of scalable hyperscale data centers, infrastructure edge, and advanced 5G networks.
Hah, touché my friend. It's just that every time I think about touching that page it scope creeps into making it autogenerated from the kernel sources via CI etc. etc. :)
Yesterday AVM released a new update for the Fritz!OS, which includes "stability and security updates", however it seems that it importantly fixes a security vulnerability that is maybe already actively getting exploited. So make sure to update if you happen to have a FritzBox :)
For some time now I've been getting videos recommended with barely any views (often <100), which are basically completely random.
I wrote this simple extension[0] that removes those videos from the main page, as well as on the sidebar of video pages. The limit is set to below 100 by default, however this can be adjusted. The source code[1] is available as well.
I'm not planning on releasing the Extension for Chrome, since you can't do that for free[2].
---
Can xLSTM really compete with Transformers, Mamba etc.?