In general, I don’t think AI safety people have updated much from LLMs. Their ethical behavior is robust already even just using reinforcement learning from human feedback on a pre-trained model. The central argument of AI risk seems to be that human values are a narrow target, so pointing an AI agent precisely at them is extremely difficult and unlikely to happen. But they aren’t hand coding this, they are pointing a pre-trained model that was already optimized hard on an objective that encourages the model to learn human preferences. Moving a model like that down a gradient to good behavior is far less difficult than creating a program from scratch that can do it. Their argument would seem to make false predictions too about the ability of stochastic gradient descent to produce generalizing solutions. LLMs could theoretically just memorize stuff they have seen and not generalize, but in practice they don’t

(http://www.autoadmit.com/thread.php?thread_id=5716142&forum_id=2#48880894)

Yud is actually super smart and he has come up with a lot of good ideas about decision theory but he's just gonna keep doubling down on his previous claims forever and refuse to admit he was wrong (which there wouldn't even be any shame in doing, LLMs were a total paradigm shift) because, well, he's ridiculously Jewish, in every way

(http://www.autoadmit.com/thread.php?thread_id=5716142&forum_id=2#48880916)