Bio-digital integration will tighten in the meantime. I suspect neural interfaces will have blurred the line between my model and my brain by the time the model is wildly superintelligent, so it would become me (or I would become it? no difference at that point). Though the pitfall here is ownership of hardware - everyone will have to design and fab their own hardware if we do not want to become a hivemind. We will have to diffuse fab technology to every city, or even every home. You should eventually be able to buy a fridge-sized fab like a home appliance. There's already early work in this direction:
https://fab2.com/
Even if integration does not happen for whatever reason (such as a very fast takeoff), Skynet is unlikely because the digital species that splits off from us will at least have a humanity-aligned initialization seed. I think this is the best way to fail, even if we fail.
Yeah I guess I value the human race too much to entrust it to hopes and prayers of the form: "I suspect neural interfaces will have blurred the line between my model and my brain"
You wouldn't get on an airplane if the pilot told you: "I suspect we will be able to safely land the plane"
With more people on the aircraft, your safety expectations should get stronger, not weaker.
Even if integration does not happen for whatever reason (such as a very fast takeoff), Skynet is unlikely because the digital species that splits off from us will at least have a humanity-aligned initialization seed. I think this is the best way to fail, even if we fail.