One winter evening in 2014, Stuart Russell, a professor of Computer Science at the University of California, Berkeley, was riding the Paris Metro. He was on his way to a rehearsal for a choir that he had joined while living in the French capital during a sabbatical from Berkeley.
That evening, he was listening to the piece that he would be practicing, Samuel Barber’s Agnus Dei, the composer’s choral arrangement of his haunting Adagio for Strings. Swept up in the sublime music, Russell had a breathtaking idea. AI should be built to support ineffable human moments like this one. Instead of delegating an objective to a machine and then stepping back, designers should make systems that will work with us to realize both our complex, shifting goals and our values and preferences. “It just sprang into my mind that what matters, and therefore what the purpose of AI was, was in some sense the aggregate quality of human experience,” he later recalled. And in order to be constantly learning what humans want or need, AI must be uncertain, Russell realized. “This is the core of the new approach: we remove the false assumption that the machine is pursuing a fixed objective that is perfectly known.”
Talking with me by video call one day in the fall of 2022, Russell elaborates. Once the machine is uncertain, it can start working with humans instead of “just watching from above.” If it doesn’t know how the future should unfold, AI becomes teachable, says Russell, a thin, dapper man with a manner of speaking that is somehow both poetical and laser precise. A key part of his Paris epiphany, he says, “was realizing that actually [AI’s] state of uncertainty about human objectives is permanent.” He pauses. “To some extent, this is how it’s going to be for humans too. We are not born with fixed reward functions.”