Very similar to the Nathan For You episode

(http://www.autoadmit.com/thread.php?thread_id=5858192&forum_id=2.#49825962)

“The definition I most like is that an AI system is aligned if it’s trying to help you do what you want to do,” said Paul Christiano.

There’s also the question of how paternalistic an AI might be. Imagine that you’re out one night with an old friend who unexpectedly came into town. You’re having a great time, and “one glass of wine” turns into four. The AI assistant on your phone knows that you have an important meeting at eight a.m. the next day. It politely nudges you to go home, then becomes increasingly insistent.

By one a.m., it’s threatened to go nuclear: I’ve called you an Uber, and if you don’t get in the car right now I’m going to send a series of sexually harassing drunk texts to your subordinate. The next morning, you’re sharp enough at the meeting to secure a round of Series A funding for your startup and deeply appreciative for the AI’s intervention.

Is this a well-aligned AI or poorly aligned one? Are we willing to hand over agency to machines if they can make higher EV choices for us than we’d make for ourselves?

(http://www.autoadmit.com/thread.php?thread_id=5858192&forum_id=2.#49825983)