Shane Legg isn't worried about "evil" AI, but rather an AGI/SI that simultaneously (a) has the ability to achieve complex goals in complex environments, and (b) not perfectly aligned with operators' goals [1].
We can also take a strong guess about some aspects of how sufficiently advanced cognition look from the outside, regardless of internal implementation, because of coherence theorems [2][3].
We can also take a strong guess about some aspects of how sufficiently advanced cognition look from the outside, regardless of internal implementation, because of coherence theorems [2][3].
[1] http://econlog.econlib.org/archives/2016/03/so_far_unfriend....
[2] https://arbital.com/p/expected_utility_formalism/?l=7hh
[3] https://arbital.com/p/instrumental_convergence/