Smarts = ideas, and the available ideas are ours, which contain our values. Where's it going to learn its morality from?
* No values at all = no motivation to even move.
* Some ad hoc home-spun jungle morality like a feral child - in that case, it would lack heuristics in general and wouldn't be so smart. Even your artificial super-brain has to do the "standing on the shoulders of giants" thing.
* It gets its moral ideas from a malevolent alien or axe murderer - how come? Unless it was personally trained and nurtured by Mark Zuckerberg I don't see why this would happen.
Mind you, I suppose it's true that even normal humans tend to be somewhat mean and aloof to any outgroup of slightly different humans. So even after it learns our values, there's a definite risk of it being snotty to us.
* No values at all = no motivation to even move.
* Some ad hoc home-spun jungle morality like a feral child - in that case, it would lack heuristics in general and wouldn't be so smart. Even your artificial super-brain has to do the "standing on the shoulders of giants" thing.
* It gets its moral ideas from a malevolent alien or axe murderer - how come? Unless it was personally trained and nurtured by Mark Zuckerberg I don't see why this would happen.
Mind you, I suppose it's true that even normal humans tend to be somewhat mean and aloof to any outgroup of slightly different humans. So even after it learns our values, there's a definite risk of it being snotty to us.