> If intelligence is fundamentally linked to specific sensorimotor modalities, a specific environment, a specific upbringing, and a specific problem to solve, then you cannot hope to arbitrarily increase the intelligence of an agent merely by tuning its brain — no more than you can increase the throughput of a factory line by speeding up the conveyor belt.
The latter part makes no sense at all. Of course you can increase the throughput of a factory by speeding up "the conveyor belt"--a standin for the complex processes going into manufacturing.
The whole statement is also wrong. Of course you can increase intelligence by optimizing the process of learning. Fewer trials, quicker reactions, faster construction of models, more complex understanding of fundamentals of a given problem.
The author makes broad assertions like this with glaring holes with zero evidence.
Also, intelligence includes the ability to identify other bottlenecks and remove them. Of course you can't expect that that a faster conveyor belt, by itself, can yield ever-greater output, but you can expect further improvements if the factory is feeding engineers who are working on ways to further re-design the factory, and they are currently constrained by the factory's output.
The whole reason that some people predict an intelligence explosion is because intelligence is the resource that can find arbitrary ways to self-improve, other than tuning one specific parameter.
I would go so far as to say that intelligence is the same thing as the ability to make improvements that climb out of a local domain of attraction.
>> The whole reason that some people predict an intelligence explosion is because intelligence is the resource that can find arbitrary ways to self-improve, other than tuning one specific parameter.
But not even a super-intelligence can find ways to self-improve that don't exist. We don't know what we don't know and we don't know if it's possible for intelligence to improve indefinitely. If there is a hard limit to the amount of improvement an intelligence can acquire, then superintelligence is not going to happen.
As an analogy, think of the speed of light. No amount of technology will get you past that. You might find a way around it for the purposes of interestellar travel but nothing will ever move faster than light.
How can we know that there don't exist similar impassable barriers for the development of intelligence? Maybe it is an -yet undiscoered- law of intelligence that an intelligent species cannot create an artificial intelligence more intelligent than itself. Who knows?
The problem is that we can sit around thinking of possibilities and impossibilities for ever, but the fact of the matter is that we don't, currently, have any evidence to the point that super-intelligence is possible. We don't have any evidence to the contrary either- but the discussions of superintelligence start with people assuming it is possible and that is an assumption that must first be substantiated (but rarely is).
Those are all good points, and good reasons to be skeptical of the Intelligence Explosion narrative, but the author wasn't basing his objection on those reasons.
> The whole reason that some people predict an intelligence explosion is because intelligence is the resource that can find arbitrary ways to self-improve, other than tuning one specific parameter.
Yes, exactly. General intelligence is "thinking about thinking", and that inevitably entails removing bottlenecks in creative ways.
"The whole statement is also wrong. Of course you can increase intelligence by optimizing the process of learning."
One of the author's main points is that "intelligence" in the way you mentioned, learning and optimization, is simply one aspect of the human mind. So we could optimize our minds (or an AI program) to beat anyone at the game of Go and play perfectly, but that's all it is optimized to do. It can understand "the given problem" but there is FAR MORE to our minds than optimizing and learning how to solve a task.
Proponents of "build a general AI system that will surpass humans at everything" don't seem to understand this.
"A further reason why it is senseless to speak of machines that are larger than people is that humans already possess the property of universal largeness.
By this, we mean that humans are capable of augmenting their bodies or coming together to become indefinitely large, no matter the metric chosen. If a human would like to be taller, they can stand on a chair or climb onto another human’s shoulders. If they would like to be wider, they can begin consuming a high-calorie diet or simply put on a thick sweater (Hensrud, 2004; Figure1)."
Isn't there a loophole in this argument - if we could only include "building specialized sub-AIs" in the set of problems our AI can solve? Is there any fundamental reason why this can't be done? I agree that there needs to be more subsystem in AI for 'learning' and 'improving' to even work.
Not really sure where I stand on the AI debate, surpassing humans at everything seems misguided to me, like why would you include all this human-like cruft in the AI when you can put in stuff humans could never have?
> It can understand "the given problem" but there is FAR MORE to our minds than optimizing and learning how to solve a task.
Humans are bound by the size of their brains. Machines can "duplicate themselves", in a way, it can freeze its Go-playing part and learn to do something else to perfection. Humans can't do that. If pro Go players stop playing Go and decide to master sword-making, they will forget about Go. A machine could be designed not to.
"it can freeze its Go-playing part and learn to do something else to perfection"
People like Elon Musk will write sentences like this but hidden away in it is a subtle reference to an AI program understanding what it's doing. An AI program cannot jump from mastery of Go (with its neural networks and weighted matrices) to all of a sudden decide to figure out what the best course of action is to stop global warming or find a cure for cancer.
Humans write AI programs to learn how to perform a specific task via optimization, neural networks, deep learning, and all sorts of other algorithms. One of the authors main arguments is that "learning and optimization" that AI programs do today does not mean they are close to achieving "general AI" which the author also thinks is not possible:
"Decades later, the concept of an “intelligence explosion” — leading to the sudden rise of “superintelligence” and the accidental end of the human race — has taken hold in the AI community. Famous business leaders are casting it as a major risk, greater than nuclear war or climate change."
I believe that we are not close to "general AI", at all.
Yet, that is a very different position than believing that "general AI" is not possible at all, which I think is a foolish position considering highly adaptative beings exist "biomechanically" in the form of humans.
> Of course you can increase the throughput of a factory by speeding up "the conveyor belt"--a standin for the complex processes going into manufacturing.
You are treating it as a "standin" for something else, when it's unequivocally not. It's one part of a larger process, which is the point he is obviously making. It doesn't make sense when deliberately misinterpreted. Most importantly, intelligence isn't a ladder that can be "sped up" in every dimension.
The latter part makes no sense at all. Of course you can increase the throughput of a factory by speeding up "the conveyor belt"--a standin for the complex processes going into manufacturing.
The whole statement is also wrong. Of course you can increase intelligence by optimizing the process of learning. Fewer trials, quicker reactions, faster construction of models, more complex understanding of fundamentals of a given problem.
The author makes broad assertions like this with glaring holes with zero evidence.