You can do more if you have a lot more RAM. Otherwise you really are that restricted.
In the country I live in, there is no comparable Chromebook spec-wise on par with the Neo at a similar price point. You're basically stuck with 4GB RAM.
Justifying having 8gb as a good amount, while downplaying 4gb as not enough is pretty hilarious. Chromebooks run fine with 4gb of ram, especially if you install linux and use zram+swap.
You can get a regular laptop and have even more ram with Linux. Not sure why you are stuck on the Chromebook.
Aside from gaming, I can do basically everything on Mac that I can on Linux or Windows. That's a hell of a lot more than a Chromebook. Take it from someone who has owned both a Chromebook and a Macbook; suggesting that they are in the same league is silly.
Also, used != new. I'm surprised people need to be reminded of this.
It appears most people - even on Hacker News(!) - are unaware that Chromebooks have a one-click Linux VM (currently Debian Trixie is the default). It is well-integrated into the Chrome desktop/launcher, and any Linux app can even be pinned onto the taskbar, next to your browser. Any Linux package you can `apt get` or `curl | sh` can run on Chromebooks made in the last 5ish years.
Yep, I've been using ChromeOS/ built-in Debian VM for light VS Code, web dev and terminal stuff on a 150 dollar Lenovo ARM Chromebook with 4GB RAM for the last 2 years as my couch PC. I just disabled Android apps because that pushed it over the line.
Gets about 10 hours battery life, touchpad is way better than my $799 Lenovo Ideapad (ChromeOS is weirdly good with even cheap touchpad hardware) and does an incredible job of suspending idle tabs without being noticeable. No rooting, jailbreaking, etc required and unlike my M1 Macbook I can actually install apps without the ridiculous click app->can't open unverified app->settings->security->open anyway->click app second time-> open anyway song and dance.
Would I recommend it as your primary development device? Certainly not, and Neo would be a much better experience for sure but it also costs 4x as much so shrug.
I bought it entirely because I wanted the cheapest modern ARM Chromebook I could find with good battery life since my m1 Macbook is pretty much always tied to a dock and but pleasantly surprised by how much it could actually do beyond just web browsing.
Yes because normal people want to run Linux just like normal people would rather “build such a system quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.”
Normal people won't even know there's a VM in the background, Linux apps launch and behave like any other ChromeOS app. The integration is very well done, and its evident you've never used it, or even seen how it works in practice and youre hallucinating non-existing complexity. All one has to enable a setting, and they can double-click a Linux app flatpak or AppImage to launch it.
My personal laptop is my phone which is a Samsung S25 ultra with Dex that I use with a lapdock.
When I travel and need to do work (i.e coding), I don't even bring my mac because I can do everything on my phone with a VPN. VSCode runs as a local web app, python works. The only thing that doesn't work is pytorch with pip install, but I don't need it for work and I could get it to work easily if I compiled it myself.
The UI is fast, I have twice the ram of the Neo, all my apps in one place, my phone lasts longer because lapdock charges it, and I can easily multitask between work and personal all on one device.
And thats with the "limitation" of android. Before I got that setup, I had a $300 ebay refurbished Thinkpad (don't even remember the model, just one where I could get a ram stick to get it to 32gb), and I ran with #!++ linx and i3wm. It booted up faster than my work macbook, was way more responsive, and I didn't have to jump through MacOS bullshit like permissions and all the other crap when trying to do stuff.
The simple truth is that Macs never were, are not, and never will be worth it for anything. Anytime you try to argue this, you out yourself as an obvious fanboy thats wants his shiny new metal laptop to feel like he as some sort of better tool.
As someone who has done this very thing, and is a lifelong Linux fanboy (I run Linux on literally everything else), I would strongly suggest you don't do this if you're using a Macbook. The losses on battery life are far too high to accept, and if you have lower specs on the Mac laptop, you will really feel them on most Linux flavours.
> The losses on battery life are far too high to accept,
Why do people keep saying this? I have been on M1 Air on Asahi for the last 4 weeks, getting 8-10 hours daily. I see my wattage consumption on screen at all times, it varies between 2.5-3W when scrolling web and around 5W when actively working with apps. I see no difference between macOS and Linux! The only difference is the s2idle consumption but personally I don't care, besides all other modern Linux laptops have same exact issue, often worse.
On my Intel T14s 4th Gen I was getting maybe 5 hours, and that's already with heavily optimized setup!
Impressive, that must be a recent fix then, and it's good to hear. I tried Asahi some time ago and it was about 3-4 hours on average. I am still running Linux Mint on an old 2015 Macbook Pro and had to make some major power management tweaks (preventing it from _ever_ boosting up from base CPU and GPU frequency) to get close to the battery life I had before.
Definitely not 5h, not anymore. I just got off the train after working on my laptop for 3.5h, connected over wifi to the internet, browsing, searching files, etc., and ended with my battery down to 65%. I have no complaints, this is as good as it gets for Linux users. I think it's worth noting that Linux and its stack is probably most efficient OS nowadays, performance wise, so while not totally optimized for hardware, the software gets extra 10% or so over macOS and it might be showing.
I mean no offense here, but I really don't like this attitude of "I thought for a bit and came up with something that debunks all of the experts!". It's the same stuff you see with climate denialism, but it seems to be considered okay when it comes to AI. As if the people that spend all day every day for decades have not thought of this.
Dataset limitations have been well understood since the dawn of statistics-based AI, which is why these models are trained on data and RL tasks that are as wide as possible, and are assessed by generalization performance. Most of the experts in ML, even the mathematically trained ones, within the last few years acknowledge that superintelligence (under a more rigorous definition than the one here) is quite possible, even with only the current architectures. This is true even though no senior researcher in the field really wants superintelligence to be possible, hence the dozens of efforts to disprove its potential existence.
I don't see how this is an architectural problem though. The problem is that music datasets are highly multimodal, and the training process is relying almost entirely on this dataset instead of incorporating basic musical knowledge to allow it to explore a bit further. That's what happens when computer scientists aim to "upset" a field without consulting with experts in said field.
I agree with your counterpoint to OS integration, but Microsoft Teams is infamous for not being "good enough" otherwise. Laggy, buggy, unresponsive, a massive resource hog especially if it runs at startup. It's gotten a bit better, but not enough. These are not complaints on HN, they're in my workplace.
Not everyone is running the latest and greatest hardware, very few actually have the money for that. If you're running hardware from before this decade, or especially the early 2010s, the difference between an Electron app and a native app is unbelievably stark. Electron will often bring the device to its knees.
This is the point where most of the public would probably acknowledge that digital privacy is worth seeking. If you're in a fascist or communist state, announcing your political opinions online without anonymity is generally not advisable.
The interesting thing is that the time to oppose it these encroachments was somewhere between 2001 and say.. 2015 ( some events, but nothing in particular other than general acceptance by general populace ). And now the masses are crying foul? Now is absolutely not the time to try to get online invisibility cloak.
Some couldn't vote then broskie. In particular because of things like age, and school, and parents being spoonfed propaganda and having the desire for vengeance stoked for dat Middle East invasion and things. So since that couldn't happen, it seems the next best time to address the issue is now.
Not with the assholes we have running the tech industry right now trying to groom every nation state into letting them help them morph into a technocratic hellhole, no.
> Until all of these things are addressed, I certainly won't support the freedom of speech for people that won't support mine.
That means you don’t support freedom of speech. But we already knew that because you already explained your authoritarian views.
What I don’t understand is why authoritarians such as yourself (as well as many of the what I call “blue MAGA” authoritarian counterparts on the left) still pay lip service to concepts such as free speech and the rule of law. These concepts fundamentally encapsulate that they are applied equally. If you don’t support them for your enemies (and criminals, and immigrants, and trans people, etc), then you simply don’t support them, period. “Free speech for my side” simply isn’t. “The rule of law (but only for citizens)” isn’t support for the rule of law.
I find all forms of government censorship to be abhorrent, regardless of which party is in power. I support free speech and freedom from government interference for the MAGA crowd as I do for everyone else, despite their active and continued efforts to curtail my legal rights to same (as Trump has repeatedly said out loud).
Republican propagandists were quite successful at spinning the emergent corporate infringement of natural rights as a bona fide illegal government action led by "the left", to fool enough useful idiots into supporting their "alternative" of a wannabe-dictator planning a full scale governmental assault on our rights.
>How about when Amazon engineers colluded with the federal government to shutdown Parler? It would be like Trump working with hosting servers for Blue sky and getting it shutdown.
Private companies can't choose their customers anymore? Interesting.
>Twitter and Facebook were caught colluding with the Biden administration to censor Americans. There weren't 10 posts a day on HN about it, and it was pretty quickly ignored and forgotten.
The laptop? They didn't. FBI warned facebook etc about possible russian fake stories and they decided to suppress it own their own until it was fact checked. Biden did ask Twitter to take down nudes of his son as it was against twitters revenge porn rules.
You are deluded if you think this is anywhere close to what trump is doing.
I disagree that the majority of it is anti-LLM ranting, there are several subtle points here that are grounded in realism. You should read on past the first bit if you're judging mainly from the initial (admittedly naive) first few paragraphs.
Not GP, but... the author said explicitly "if you believe X you should stop reading". So I did.
The X here is "that the human mind can be reduced to token regurgitation". I don't believe that exactly, and I don't believe that LLMs are conscious, but I do believe that what the human mind does when it "generates text" (i.e. writes essays, programs, etc) may not be all that different from what an LLM does. And that means that most of humans's creations are also the "plagiarism" in the same sense the author uses here, which makes his argument meaningless. You can't escape the philosophical discussion he says that he's not interested in if you want to talk about ethics.
Edit: I'd like to add that I believe that this also ties in to the heart of the philosophy of Open Source and Open Science... if we acknowledge that our creative output is 1% creative spark and 99% standing on the shoulders of Giants, then "openness" is a fundamental good, and "intellectual property" is at best a somewhat distasteful necessity that should be as limited as possible and at worst is outright theft, the real plagiarism.
It's more intellectually lazy to think boolean logic at a sufficient scale crosses some event horizon wherein its execution on mechanical gadgets called computers somehow adds up to intelligence beyond human understanding.
It is intellectually lazy to proclaim something to be impossible in the absence of evidence or proof. In the case of the statement made here, it is provably true that Boolean logic at sufficient scale can replicate "intelligence" of any arbitrary degree. It is also easy to show that this can be perceived as an "event horizon" since the measurements of model quality that humans typically like to use are so nonlinear that they are virtually step function-like.
Doesn't seem like you have proof of anything but it does appear that you have something that is very much like religious faith in an unforeseeable inevitability. Which is fine as far as religion is concerned but it's better to not pretend it's anything other than blind faith.
But if you really do have concrete proof of something then you'll have to spell it out better & explain how exactly it adds up to intelligence of such magnitude & scope that no one can make sense of it.
> "religious faith in an unforeseeable inevitability"
For reference, I work in academia, and my job is to find theoretical limitations of neural nets. If there was so much of a modicum of evidence to support the argument that "intelligence" cannot arise from sufficiently large systems, my colleagues and I would be utterly delighted and would be all over it.
Here are a couple of standard elements without getting into details:
1. Any "intelligent" agent can be modelled as a random map from environmental input to actions.
2. Any random map can be suitably well-approximated by a generative transformer. This is the universal approximation theorem. Universal approximation does not mean that models of a given class can be trained using data to achieve an arbitrary level of accuracy, however...
3. The neural scaling laws (first empirical, now more theoretically established under NTK-type assumptions), as a refinement of the double descent curve, assert that a neural network class can get arbitrarily close to an "entropy level" given sufficient scale. This theoretical level is so much smaller than any performance metric that humans can reach. Whether "sufficiently large" is outside of the range that is physically possible is a much longer discussion, but bets are that human levels are not out of reach (I don't like this, to be clear).
4. The nonlinearity of accuracy metrics comes from the fact that they are constructed from the intersection of a large number of weakly independent events. Think the CDF of a Beta random variable with parameters tending to infinity.
Look, I understand the scepticism, but from where I am, reality isn't leaning that way at the moment. I can't afford to think it isn't possible. I don't think you should either.
As I said previously, you are welcome to believe whatever you find most profitable for your circumstances but I don't find your heuristics convincing. If you do come up or stumble upon a concrete constructive proof that 100 trillion transistors in some suitable configuration will be sufficiently complex to be past the aforementioned event horizon then I'll admit your faith was not misplaced & I will reevaluate my reasons for remaining skeptical of Boolean arithmetic adding up to an incomprehensible kind of intelligence beyond anyone's understanding.
Which part was heuristic? This format doesn't lend itself to providing proofs, it isn't exactly a LaTeX environment. Also why does the proof need to be constructive? That seems like an arbitrarily high bar to me. It suggests that you are not even remotely open to the possibility of evidence either.
I also don't think you understand my point of view, and you mistake me for a grifter. Keeping the possibility open is not profitable for me, and it would be much more beneficial to believe what you do.
I didn't think you were a grifter but you only presented heuristics so if you have formal references then you can share them & people can decide on their own what to believe based on the evidence presented.
Fine, that's fair. I believe the statement that you made is countered by my claim, which is:
Theorem. For any tolerance epsilon > 0, there exists a transformer neural network of sufficient size that follows, up to the factor epsilon, the policy that most optimally achieves arbitrary goals in arbitrary stochastic environments.
Proof (sketch). For any stochastic environment with a given goal, there exists a model that maximizes expected return under this goal (not necessarily unique, but it exists). From Solomonoff's convergence theorem (Theorem 3.19 in [1]), Bayes-optimal predictors under the universal Kolmogorov prior converge with increasing context to this model. Consequently, there exists an agent (called the AIXI agent) that is Pareto-optimal for arbitrary goals (Theorem 5.23 in [1]). This agent is a sequence-to-sequence map with some mild regularity, and satisfies the conditions of Theorem 3 in [2]. From this universal approximation theorem (itself proven in Appendices B and C in [2]), there exists a transformer neural network of a sufficient size that replicates the AIXI agent up to the factor epsilon.
This is effectively the argument made in [3], although I'm not fond of their presentation. Now, practitioners still cry foul because existence doesn't guarantee a procedure to find this particular architecture (this is the constructive bit). This is where the neural scaling law comes in. The trick is to work with a linearization of the network, called the neural tangent kernel; it's existence is guaranteed from Theorem 7.2 of [4]. The NTK predictors are also universal and are a subset of the random feature models treated in [5], which derives the neural scaling laws for these models. Extrapolating these laws out as per [6] for specific tasks shows that the "floor" is always below human error rates, but this is still empirical because it works with the ill-defined definition of superintelligence that is "better than humans in all contexts".
[1] Hutter, M. (2005). Universal artificial intelligence: Sequential decisions based on algorithmic probability. Springer Science & Business Media.
Good question. It's because we don't need to be completely optimal in practice, only epsilon close to it. Optimality is undecidable, but epsilon close is not, and that's what the claim says that NNs can provide.
That doesn't address what I asked. The paper I linked proves undecidability for a much larger class of problems* which includes the case you're talking about of asymptotic optimality. In any case, I am certain you are unfamiliar w/ what I linked b/c I was also unaware of it until recently & was convinced by the standard arguments people use to convince themselves they can solve any & all problems w/ the proper policy optimization algorithm. Moreover, there is also the problem of catastrophic state avoidance even for asymptotically optimal agents: https://arxiv.org/abs/2006.03357v2.
* - Corollary 3.4. For any fixed ε, 0 < ε < 1, the following problem is undecidable: Given is a PFA M for which one of the two cases hold:
(1) the PFA accepts some string with probability greater than 1 − ε, or
(2) the PFA accepts no string with probability greater than ε.
Oh yes, that's one of the more recent papers from Hutter's group!
I don't believe there is a contradiction. AIXI is not computable and optimality is undecidable, this is true. "Asymptotic optimality" refers to behaviour for infinite time horizons. It does not refer to closeness to an optimal agent on a fixed time horizon. Naturally the claim that I made will break down in the infinite regime because the approximation rates do not scale with time well enough to guarantee closeness for all time under any suitable metric. Personally, I'm not interested in infinite time horizons and do not think it is an important criterion for "superintelligence" (we don't live in an infinite time horizon world after all) but that's a matter of philosophy, so feel free to disagree. I was admittedly sloppy with not explicitly stating that time horizons are considered finite, but that just comes from the choice of metric in the universal approximation which I have continued to be vague about. That also covers the Corollary 3.4, which is technically infinite time horizon (if I'm not mistaken) since the length of the string can be arbitrary.
> "...it would sometimes regurgitate training data verbatim. That’s been patched in the years since..."
> "They are robots. Programs. Fancy robots and big complicated programs, to be sure — but computer programs, nonetheless."
This is totally misleading to anyone with less familiarity with how LLMs work. They are only programs in as much as they perform inference from a fixed, stored, statistical model. It turns out that treating them theoretically in the same way as other computer programs gives a poor representation of their behaviour.
This distinction is important, because no, "regurgitating data" is not something that was "patched out", like a bug in a computer program. The internal representations became more differentially private as newer (subtly different) training techniques were discovered. There is an objective metric by which one can measure this "plagiarism" in the theory, and it isn't nearly as simple as "copying" vs "not copying".
It's also still an ongoing issue and an active area of research, see [1] for example. It is impossible for the models to never "plagiarize" in the sense we think of while remaining useful. But humans repeat things verbatim too in little snippets, all the time. So there is some threshold where no-one seems to care anymore; think of it like the % threshold in something like Turnitin. That's the point that researchers would like to target.
Of course, this is separate from all of the ethical issues around training on data collected without explicit consent, and I would argue that's where the real issues lie.
At the frontier of science we have speculations, which until proper measurements become possible, are unknown to be true or false (or even unknown to be equivalent with other speculations etc. regardless of their being true or false, or truer or falser). Once settled we may call earlier but wrong speculations as "reasonable wrong guesses". In science it is important that these guesses or suspicions are communicated as it drives the design of future experiments.
I argue that more important that "eliminating hallucinations" is tracing the reason it is or was believed by some.
With source-aware training we can ask an LLM to give answers to a question (which may contradict each other), but to provide the training-source(s) justifying emission of each answer, instead of bluff it could emit multiple interpretations and go like:
> answer A: according to school of thought A the answer is that ... examples of authors and places in my training set are: author+title a1, a2, a3, ...
> answer B: according to author B: the answer to this question is ... which can be seen in articles b1, b2
> answer ...: ...
> answer F: although I can't find a single document explaining this, when I collate the observation x in x1, x2, x3; observation y in y1,y2, ... , observation z in z1, z2, ... then I conclude the following: ...
so it is clear which statements are sourced where, and which deductions are proper to the LLM.
Obviously few to none of the high profile LLM providers will do this any time soon, because when jurisdictions learn this is possible they will demand all models to be trained source-aware, so that they can remunerate the authors in their jurisdiction (and levy taxes on their income). What fraction of the income will then go to authors and what fraction to the LLM providers? If any jurisdiction would be first to enforce this, it would probably be the EU, but they don't do it yet. If models are trained in a different jurisdiction than the one levying taxes the academic in-group citation game will be extended to LLMs: a US LLM will have incentive to only cite US sources when multiple are available, and a EU trained LLM will prefer to selectively cite european sources, etc.
In addition to providing training sources, it's important to identify overlaps among the fragments used in the answer. For me, overlap doesn't mean simply identical expression, but conceptually identical.
We are much more likely to find conceptual overlap in code than in language and prose because Many of the problems we solve, as mathematicians say, reduce to previously solved problems, which IMO means substantially identical code.
A related question is how much change is necessary to a work of art, image, prose, or code for it to escape copyright? If we can characterize it and the LLM generates something that escapes copyright, I suggest the output should be excluded from future copyright or patent claims.
I wasn't aware of source-aware training, so thank you for the reference! It does seem a bit too good to be true; I believe in a system of tradeoffs so I feel like this must have an issue with reducing creativity. That's at first glance though, so I could be wrong.
> But humans repeat things verbatim too in little snippets, all the time
Also, it's possible, although statistically improbable, for a human to generate the exact same thing another human generated (and copyrighted) without even knowing it.
> This is totally misleading to anyone with less familiarity with how LLMs work. They are only programs in as much as they perform inference from a fixed, stored, statistical model. It turns out that treating them theoretically in the same way as other computer programs gives a poor representation of their behaviour.
Maybe it is to a child or average citizen, but I don't believe that "not understanding the consequences" is the case here on HN. This is just a difference in philosophy, the old "freedom vs. security" tradeoff that everyone falls down on a little differently. Giving up your data to a company (and therefore the government) in exchange for services is a trust exercise, and there are ways to avoid making it, but they have significant unavoidable costs. It's an easier decision when you don't fear your own government, but where you fall on the spectrum rapidly changes when your government makes you the target. Of course you can say "the government is always going to turn on you, so you should never trust them!", but you'll sound like a loon to many native citizens of a Western nation that have had little to fear for decades.
The US is just experiencing a little more of what the citizens of communist and fascist nations have experienced. Over time, that might lead to rapid societal change, or maybe it's too late.
In the country I live in, there is no comparable Chromebook spec-wise on par with the Neo at a similar price point. You're basically stuck with 4GB RAM.
reply